mirror of
https://github.com/codician-team/growmos.git
synced 2026-08-18 06:47:17 +02:00
growmos 0.1.0 — living knowledge graph for any repo, agent-native and zero-dependency
Implements the knowledge-graph playbook (extraction → resolution → assembly → querying + evaluation loop) as a plug & play CLI + .growmos/ JSONL store: task packets for Claude Code / Codex / Grok / Cursor / Gemini, MCP stdio server, git hooks, CI, optional headless provider mode, Apollo example corpus, 18 tests, docs and methodology. Repo dogfoods its own graph.
This commit is contained in:
codician-dev
@@ -0,0 +1,96 @@
|
||||
# Using growmos with your agent CLI
|
||||
|
||||
growmos is **agent-native**: your CLI agent is the model. The CLI does the deterministic
|
||||
work and hands judgment work over as *task packets*. No API key is required.
|
||||
|
||||
## The protocol every agent follows
|
||||
|
||||
1. **Session start** → `growmos context` (Claude Code does this automatically via a `SessionStart` hook).
|
||||
2. **Cross-cutting question** → `growmos query "<question>"` and answer only from the triples, citing edge ids.
|
||||
3. **Learned/decided something durable** → `growmos remember` / `growmos link` / `growmos journal`.
|
||||
4. **Grow the graph** → `growmos next` → produce the JSON → run the printed `growmos apply …` → repeat.
|
||||
5. **Before asserting facts** → `growmos check "<claims>"`.
|
||||
6. **Session end** → `growmos journal "<summary>"`.
|
||||
|
||||
The block that teaches this to agents is written by `growmos integrate <target>` (or `growmos init --agent …`).
|
||||
|
||||
## Claude Code
|
||||
|
||||
```bash
|
||||
growmos init --agent claude # or: growmos integrate claude
|
||||
```
|
||||
Writes:
|
||||
- `CLAUDE.md` — protocol block (marker-delimited, idempotent)
|
||||
- `.claude/skills/growmos/SKILL.md` — a skill that triggers on graph-related asks and explains the packet rules
|
||||
- `.claude/settings.json` — hooks: `SessionStart` runs `growmos context --brief` (injected into context), `Stop` runs `growmos scan --quiet` (queues changed docs)
|
||||
- `.mcp.json` — registers `growmos mcp` as an MCP server (tools: `growmos_query`, `growmos_remember`, …)
|
||||
|
||||
Try: *"grow the knowledge graph until it's up to date"* · *"what does the graph say about the Store?"* · *"remember that the Scheduler now depends on Kafka"*.
|
||||
|
||||
## Codex CLI
|
||||
|
||||
```bash
|
||||
growmos init --agent codex # writes the AGENTS.md block
|
||||
```
|
||||
Codex reads `AGENTS.md`. To use MCP tools instead of shelling out, add to `~/.codex/config.toml`:
|
||||
```toml
|
||||
[mcp_servers.growmos]
|
||||
command = "growmos"
|
||||
args = ["mcp"]
|
||||
```
|
||||
|
||||
## Grok CLI and other CLIs
|
||||
|
||||
```bash
|
||||
growmos init --agent grok # AGENTS.md block + .mcp.json
|
||||
```
|
||||
Most CLIs honour `AGENTS.md`. If yours uses a different instructions file, append the block:
|
||||
```bash
|
||||
growmos integrate file --file .grok/GROK.md
|
||||
```
|
||||
If it supports MCP, point it at `growmos mcp` (stdio).
|
||||
|
||||
## Cursor
|
||||
|
||||
```bash
|
||||
growmos init --agent cursor # .cursor/rules/growmos.mdc, alwaysApply: true
|
||||
```
|
||||
|
||||
## Gemini CLI
|
||||
|
||||
```bash
|
||||
growmos init --agent gemini # GEMINI.md block
|
||||
```
|
||||
|
||||
## Everything at once
|
||||
|
||||
```bash
|
||||
growmos init --agent all # claude + codex + gemini + cursor + git hooks + CI workflow
|
||||
```
|
||||
|
||||
## Git hooks & CI
|
||||
|
||||
- `growmos integrate hooks` → `post-commit`, `post-merge`, `post-checkout` run `growmos scan --quiet` so edited docs are queued for the next session (safe no-op if growmos is absent; respects `core.hooksPath`).
|
||||
- `growmos integrate ci` → `.github/workflows/growmos.yml` runs `status`, `doctor`, `eval` on PRs.
|
||||
|
||||
## Multi-agent teams (orchestrator–workers)
|
||||
|
||||
The graph is the blackboard. Give each worker its slice of sources (`growmos add …` per worker,
|
||||
or separate include globs), let workers run `next`/`apply` in their own context windows, and let
|
||||
the resolver step (`growmos resolve` → apply) merge surface forms across workers ("Acme Corp",
|
||||
"ACME Corporation", "acme"). The synthesizer never re-reads the raw documents: it runs
|
||||
`growmos query` and cites edges. Everything writes to the same JSONL files; commit and merge
|
||||
like code.
|
||||
|
||||
## Packet anatomy
|
||||
|
||||
```
|
||||
=== growmos task packet: extraction · docs/adr-001.md · chunk 1/1 ===
|
||||
Respond with JSON of this shape (strict: no extra keys):
|
||||
{"entities": [...], "relations": [...]}
|
||||
Write it to `.growmos/cache/extract_src_….json` (or pipe it on stdin), then run:
|
||||
growmos apply extraction .growmos/cache/extract_src_….json --source src_… --chunk 0
|
||||
--- prompt ---
|
||||
<the prompt from .growmos/prompts/extract.md, rendered>
|
||||
```
|
||||
`growmos next --json` gives the same as structured data (`text`, `meta.apply`, `meta.out_file`).
|
||||
@@ -0,0 +1,66 @@
|
||||
# Evaluation: the loop that turns a demo into a production system
|
||||
|
||||
> Change the extraction prompt → rerun the scorer → watch F1 move. A team that ships without
|
||||
> this loop cannot tell whether a prompt change improved or degraded quality.
|
||||
|
||||
## 1. Build a gold set
|
||||
|
||||
Pick at least two representative documents. Pre-fill from the current extraction, then
|
||||
**hand-correct** (remove wrong entities, add missed ones — the whole point is human judgment):
|
||||
|
||||
```bash
|
||||
growmos gold-template docs/apollo-11.md
|
||||
$EDITOR .growmos/eval/gold/apollo-11.json
|
||||
```
|
||||
|
||||
Gold format:
|
||||
```json
|
||||
{"source": "docs/apollo-11.md",
|
||||
"entities": [{"name": "Apollo 11", "type": "EVENT"}, {"name": "Neil Armstrong", "type": "PERSON"}],
|
||||
"relations": [{"source": "Neil Armstrong", "target": "Apollo 11"}]}
|
||||
```
|
||||
Relations are scored on (source, target) pairs, direction-agnostic, ignoring predicate wording —
|
||||
an upper bound on relation recall that catches structural errors (missing/wrong connections),
|
||||
which matter more than wording.
|
||||
|
||||
## 2. Keep the scorer alias map current
|
||||
|
||||
If the resolver picks a canonical form the gold set doesn't use ("Neil Alden Armstrong" vs
|
||||
"Neil Armstrong"), resolved recall drops — a scoring artifact, not a resolver bug.
|
||||
`growmos eval` lists such canonicals; add them to `.growmos/eval/aliases.json`:
|
||||
```json
|
||||
{"Neil Alden Armstrong": "Neil Armstrong", "John F. Kennedy Space Center": "Kennedy Space Center"}
|
||||
```
|
||||
|
||||
## 3. Run it
|
||||
|
||||
```bash
|
||||
growmos eval
|
||||
```
|
||||
```
|
||||
document raw F1 P R | resolved R | rel F1
|
||||
docs/apollo-11.md 0.75 1.0 0.6 | 0.6 | 0.833
|
||||
missed: columbia, eagle, president kennedy, saturn v
|
||||
```
|
||||
Precision 1.0 with recall < 1 is the *intended* posture: everything extracted was correct; the
|
||||
misses are peripheral mentions ("Purdue University") or scope mismatches ("Saturn V" in the
|
||||
Apollo 11 gold but extracted from the Saturn V article). Loosening "extract only central
|
||||
entities" trades noise for recall — make that trade deliberately, and record it in the journal.
|
||||
|
||||
## 4. Tune
|
||||
|
||||
Edit `.growmos/prompts/extract.md` (or `resolve.md`, `summarize.md`), re-extract the gold
|
||||
documents (`growmos scan` after touching them, or `growmos extract <ref>` → apply), rerun
|
||||
`growmos eval`. Bump the schema (`growmos schema bump --note "loosened centrality rule"`) when
|
||||
the change alters what the graph means, so rows produced under different prompts can be told apart.
|
||||
|
||||
## 5. Health signals worth graphing over time
|
||||
|
||||
`growmos status --json` exposes: `nodes`, `edges`, `density`, `components`, `compression_ratio`,
|
||||
`pending_sources`, `provisional`, `stale_profiles`. Extraction rate per document is in each
|
||||
source's `stats`. Wire them into CI (`growmos integrate ci`).
|
||||
|
||||
## 6. The human sample
|
||||
|
||||
`growmos sample` prints a random (degree-weighted) node with its profile, edges, provenance and
|
||||
sources. Read it against the sources. `growmos doctor` turns red if nobody has done this in 7 days.
|
||||
@@ -0,0 +1,73 @@
|
||||
# `.growmos/` file format
|
||||
|
||||
All files are UTF-8, deterministic (sorted keys, sorted rows) and safe to commit. JSONL rows are
|
||||
one JSON object per line.
|
||||
|
||||
## sources.jsonl
|
||||
```json
|
||||
{"id":"src_b33563055168","ref":"README.md","kind":"file","sha256":"…","title":"README.md",
|
||||
"status":"extracted","added":"2026-08-17T10:00:00Z","updated":"…","extracted_at":"…",
|
||||
"schema_version":1,"stats":{"entities":8,"relations":6}}
|
||||
```
|
||||
`kind` ∈ `file` (repo-relative path) · `url` · `text` (stdin/foreign file; body cached under `cache/`) · `note` (provenance label such as `session:2026-08-17`).
|
||||
`status` ∈ `pending` · `extracted` · `note` · `missing`.
|
||||
|
||||
## mentions.jsonl (append-only)
|
||||
```json
|
||||
{"source":"src_…","chunk":0,"name":"A. Chen","type":"PERSON","description":"Approved ADR-001.","ts":"…","schema_version":1}
|
||||
```
|
||||
|
||||
## entities.jsonl
|
||||
```json
|
||||
{"id":"person/alice-chen","name":"Alice Chen","type":"PERSON","description":"Owner of the Scheduler component.",
|
||||
"sources":["src_a","src_b"],"mentions":2,"provisional":false,"created":"…","updated":"…","schema_version":1}
|
||||
```
|
||||
Ids are stable (`<type>/<slug-of-first-seen-name>`); `name` is the canonical display form and may change on resolution.
|
||||
|
||||
## aliases.jsonl
|
||||
```json
|
||||
{"alias":"A. Chen","type":"PERSON","entity":"person/alice-chen"}
|
||||
```
|
||||
|
||||
## relations.jsonl
|
||||
```json
|
||||
{"id":"r_4e1920f801f6","source":"dependency/postgres","predicate":"replaces","target":"dependency/redis",
|
||||
"sources":["src_a","src_b"],"confidence":2,"created":"…","updated":"…","schema_version":1,
|
||||
"when":{"start":"2026-08","end":"ongoing"}}
|
||||
```
|
||||
`id` = hash of (source, normalized predicate, target). `confidence` = number of distinct sources (cross-document corroboration). `when` is optional.
|
||||
|
||||
## profiles/<id>.json
|
||||
```json
|
||||
{"entity":"component/scheduler","summary":"…","key_facts":["…"],"time_range":{"start":"unknown","end":"ongoing"},
|
||||
"sources_hash":"ab12cd34ef56","ts":"…"}
|
||||
```
|
||||
`sources_hash` is the hash of the entity's source set at write time; a mismatch marks the profile stale.
|
||||
|
||||
## schema.json
|
||||
```json
|
||||
{"version":1,"entity_types":["PERSON","ORGANIZATION","LOCATION","EVENT","ARTIFACT","COMPONENT",…],
|
||||
"predicate_hints":["depends on","calls",…],"history":[{"version":1,"ts":"…","note":"init with preset 'software'"}]}
|
||||
```
|
||||
|
||||
## config.json
|
||||
```json
|
||||
{"preset":"software","include":["README.md","docs/**/*.md",…],"exclude":["node_modules/**",…],
|
||||
"max_docs_per_run":25,"max_entities_per_doc":40,"chunk_chars":6000,"profile_min_degree":3,
|
||||
"resolve_batch_size":80,"provider":{"name":"","extract_model":"","reason_model":""}}
|
||||
```
|
||||
|
||||
## state.json
|
||||
Runs (last 200), `pending_resummarize`, `open_packets` (names handed out in the last resolution packet per type), `last_sample`, `last_eval`, counters.
|
||||
|
||||
## eval/
|
||||
`gold/<doc>.json` — `{"source": "<ref>", "entities": [{"name","type"}], "relations": [{"source","target"}]}` ·
|
||||
`aliases.json` — `{"<canonical form>": "<gold name>"}` · `last_report.json`.
|
||||
|
||||
## Packet payloads (what agents return)
|
||||
|
||||
Extraction: `{"entities":[{"name","type","description"}],"relations":[{"source","predicate","target"}]}`
|
||||
Resolution: `{"clusters":[{"canonical","aliases":[…]}]}` (optionally `"input_names":[…]`)
|
||||
Profile: `{"summary","key_facts":[…],"time_range":{"start","end"}}`
|
||||
|
||||
The strict JSON schemas (for structured-output APIs) are in `growmos.schema`.
|
||||
@@ -0,0 +1,50 @@
|
||||
# Headless mode: cron, CI, overnight loops
|
||||
|
||||
Agent-native mode needs no API key. When no agent is at the keyboard — a nightly job, a CI step,
|
||||
a batch backfill — growmos can call an LLM API itself. It is stdlib-only and provider-neutral.
|
||||
|
||||
## Configure
|
||||
|
||||
```bash
|
||||
export ANTHROPIC_API_KEY=sk-ant-… # provider auto-detected from whichever key is set
|
||||
# or
|
||||
export OPENAI_API_KEY=… # OpenAI
|
||||
export XAI_API_KEY=… # xAI Grok
|
||||
# or any OpenAI-compatible server:
|
||||
export GROWMOS_PROVIDER=openai GROWMOS_BASE_URL=http://localhost:11434/v1 GROWMOS_API_KEY=x
|
||||
# optional per-stage overrides:
|
||||
export GROWMOS_EXTRACT_MODEL=… GROWMOS_REASON_MODEL=…
|
||||
```
|
||||
Or set `provider.name / base_url / extract_model / reason_model` in `.growmos/config.json`
|
||||
(never commit an API key there — use env vars).
|
||||
|
||||
Model split follows the playbook's Table IV: a fast, cheap model for high-volume extraction
|
||||
(`claude-haiku-4-5`, `gpt-4o-mini`, `grok-3-mini` by default) and a stronger reasoning model for
|
||||
resolution, summarization and answering (`claude-sonnet-5`, `gpt-4o`, `grok-3`).
|
||||
Structured outputs are requested (`output_config.format` json_schema on Anthropic,
|
||||
`response_format json_schema` on OpenAI-compatible APIs) so payloads validate by construction.
|
||||
|
||||
## Run
|
||||
|
||||
```bash
|
||||
growmos ingest --scan --limit 25 # extraction → resolution → hub profiles, respecting caps
|
||||
growmos query "…" --auto # grounded answer via the reasoning model
|
||||
```
|
||||
|
||||
## Cron / GitHub Actions
|
||||
|
||||
```yaml
|
||||
- run: pip install growmos
|
||||
- run: growmos ingest --scan
|
||||
env: { ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} }
|
||||
- run: growmos doctor && growmos eval
|
||||
- run: git add .growmos && git commit -m "growmos: overnight growth" || true
|
||||
```
|
||||
|
||||
## Cost notes (from the playbook's scaling guidance)
|
||||
|
||||
- Extraction dominates for large corpora — cache the fixed prompt prefix and batch when your
|
||||
provider offers it; growmos already caps documents per run.
|
||||
- Resolution is one call per entity type per block (blocks of ≤ `resolve_batch_size`), not per document.
|
||||
- Summarization is per hub node and only when its source set changed.
|
||||
- Querying cost is proportional to subgraph size — tune `--hops` and `--max-triples`.
|
||||
Reference in New Issue
Block a user