roboco/docs/map/vault.md

Purpose

The Obsidian vault (V1+V2): a rebuildable, human-readable DB projection of the org's memory (tasks, journal entries, A2A thread digests) as wikilinked markdown, plus a default-off inbox watcher that turns #roboco-tagged vault notes into board-review intake drafts. V2 adds three things on top of the V1 projection: materialize-on-create (a task's note exists from the moment it's created, not just at curation/rebuild), a drift janitor (hourly-ticked, daily/weekly-gated: re-projects changed tasks, verifies a random sample, archives old terminal tasks, writes the weekly org-report), and KB ingest (the CEO's own RoboCo/Notes/ notes become one more RAG corpus the fleet can retrieve). Default-off (ROBOCO_OBSIDIAN_VAULT_ENABLED; both compose files arm it true). Still structurally different from the other default-off engines: the projection never originates delivery work itself — the ONE writer-side effect that reaches delivery (the intake watcher) rides the existing board-review path, not a held-artifact queue.

Files

Path	Role	approx LOC
roboco/services/vault_writer.py	VaultWriter — pure, DB-free markdown materializer. write_task / write_journal_entry / append_a2a_message / write_agent / touch_task_frontmatter / write_org_report. Every note carries aliases: [<id8>] so a rename never breaks a [[id8|title]] wikilink; existing_narrative reads back an Auditor-authored ## Narrative section so a rebuild never clobbers it. V2: write_task is archive-aware (TaskNoteData.archive_year routes it to RoboCo/Archive/<year>/Tasks/<project>/ instead of Tasks/<project>/, removing the stale copy on a move); find_task_note/task_note_status locate/inspect a note wherever it lives (recursive id8 lookup across both trees) for the janitor's drift check; write_org_report renders RoboCo/Reports/<ISO-week>.md. (Uncommitted, feature/findings-ledger) FindingRow + _FINDINGS_CAP=20 + _findings_section render a ## Findings section (one [F-id8] (severity, round N, status) file:line — expected → actual → fix line per open/resolved finding, an overflow line past the cap) into every task note — see docs/map/review-findings.md.	534
roboco/services/vault_assembly.py	assemble_task_note_data — resolves a task's project slug, parent, subtasks, dependencies, and (V2) archive eligibility (_archive_year, gated on vault_archive_days) via the live TaskService/ProjectService into a TaskNoteData. reproject_task (V2) bundles assemble + narrative-preservation + write_task into the one code path shared by rebuild, the janitor's changed/sample/archival passes, and the create-on-task seam — none of them can drift on how a note gets refreshed. (Uncommitted, feature/findings-ledger) _resolve_findings fetches the task's ledger via ReviewFindingsRepository.list_for_task, fails open (empty tuple) on a missing session or any exception — a findings-fetch failure drops the section, never blocks the note write.	~125
roboco/services/vault_intake_engine.py	VaultIntakeEngine.run_cycle — scans the vault's inbox dir for #roboco-tagged notes, dedupes via vault_seen_notes (path + content-hash), screens the body through injection_guard.screen_external_text, extracts a title/description/action-items via a local-model chat call (deterministic fallback: first heading / raw body / checkbox lines), and opens ONE PENDING board-review draft (source=vault_note, Product-Owner-assigned, team=board) per note. Appends a feedback callout back into the note (best-effort). V2: the frontmatter split + content-hash helpers moved to the shared foundation/policy/vault_notes.py (this module now just imports them).	~350
roboco/services/vault_janitor.py (new, V2)	VaultJanitor.run_cycle — one state-gated sweep: re-project tasks changed since the last sweep (TaskService.list_updated_since, capped/paged, per-item isolated), verify a random stale sample (sample_stale_tasks), archive old terminal tasks (list_archive_candidates), and (weekly) render the org-report. Dueness is tracked in a JSON state file (RoboCo/_meta/.janitor_state.json: last_sweep, last_report_week, archive_watermark), not the loop's own cadence — restart-proof.	~343
roboco/services/vault_kb_engine.py (new, V2)	VaultKBEngine.run_cycle — scans the allowlisted vault_kb_dirs (default RoboCo/Notes), dedups by content hash, screens every note body through the injection guard as a hard GATE (flagged → quarantined, never indexed), and ingests/deindexes into IndexType.VAULT_NOTES via OptimalService.index_vault_note/unindex_vault_note. Defense-in-depth containment: symlinks and any resolved-path escape from the vault root are skipped, independent of the config-load validator.	~315
roboco/foundation/policy/vault_notes.py (new, V2)	Shared pure helpers: content_hash (sha256 with every > [!kind] RoboCo: ... feedback callout stripped first, so appending one doesn't change what the next scan considers "changed") and split_frontmatter (YAML frontmatter + body). Used by both the intake watcher's "drafted" callout and the KB engine's "quarantined" callout — one shared convention instead of two copies drifting.	~46
roboco/services/optimal_brain/indexes/vault_notes.py (new, V2)	VaultNotesIndexPlugin — IndexType.VAULT_NOTES plugin, mirrors PlaybooksIndexPlugin's shape (index_note/delete_note/search_notes, source URI vault://<relpath>). Scope enforced by the KB engine's dir allowlist, not this plugin.	~70
roboco/vault.py	python -m roboco.vault {rebuild,relocate} CLI. rebuild re-projects every agent/task/journal-entry/A2A-thread from the DB (now archive-aware via vault_assembly.reproject_task — an old terminal task projects straight into Archive/<year>/) and materializes .obsidian/ + RoboCo/_meta/ from roboco/vault_assets/ (never overwrites an existing file). relocate <path> moves the tree; grafts RoboCo/ into an existing destination vault without touching its own config.	~237
roboco/vault_assets/	Packaged templates copied by ensure_vault_assets: .obsidian/ (Dataview, Kanban, graph-group config — V2 adds Archive/Reports graph color groups) + meta/ (dashboard + kanban-board + README, V2 adds Task Board.base + Reports.base for Obsidian's core Bases plugin, and Sync to your Mac.md, the Syncthing/SMB/Obsidian-Sync runbook). Dataview dashboard queries now exclude Archive/.	—
roboco/foundation/policy/injection_guard.py	screen_external_text / detect_injection — the shared prompt-injection screen-and-neutralize (data path) and hard-deny (interactive-input path) pattern set. V2 reuses it a third time: the KB engine's ingest-time hard gate (quarantine on a hit, vs. the intake watcher's screen-and-still-process posture).	125
roboco/services/gateway/content_actions.py curate_vault	Server-side do-action: Auditor-only, re-materializes a task's note with the Auditor's narrative filling ## Narrative. Inert (invalid_state) when the flag is off.	—
roboco/mcp/do_server.py curate_vault	Do-tool the Auditor calls exactly once per completed root, POSTing to /api/v1/do/curate_vault.	—
roboco/db/tables.py VaultSeenNoteTable	Dedup ledger for the intake watcher: (note_path, content_hash) — an unchanged note is never reprocessed; an edited one is eligible again.	—
roboco/services/task.py _materialize_vault_note / list_updated_since / list_archive_candidates / sample_stale_tasks	V2: the create-time seam + the janitor's three query methods. See docs/map/task-service.md.	—

Data Flow

PROJECTION (always-on when the flag is armed). TaskService.create (V2) calls _materialize_vault_note — best-effort, same swallow-and-log posture as every other seam — so a task's note exists from the moment it's created, not just at curation/rebuild. Three more best-effort event seams fire from existing services: TaskService._emit_status_transition_audit → _touch_vault_frontmatter patches an EXISTING note's status/team/pr fields in place (now effectively always finds one for any task created post-V2, since materialize-on-create ran; a pre-V2 task without a note is still a no-op here — the janitor's changed/sample passes are what backfill it); JournalService's entry-write path → _materialize_vault_note writes one immutable file per non-private entry; A2AService.send → _materialize_vault_note appends to a per-thread digest file, deduped per message id via an in-body marker comment. All import get_vault_writer() lazily and catch every exception.

CURATION (root-completion hook, orchestrator-driven). AgentOrchestrator._dispatch_vault_curation_work (one of the 18 tick dispatchers, gated on obsidian_vault_enabled) reads TaskService.list_completed_roots_pending_vault_curation and calls _maybe_spawn_vault_curation per candidate: an in-memory one-shot guard (_board_dispatched) plus a durable vault_curation_dispatched marker (survives a restart) precede a bindingless Auditor spawn. The Auditor writes one narrative paragraph and calls curate_vault(task_id, narrative) exactly once; the verb re-resolves the task's parent/subtasks/dependencies fresh via assemble_task_note_data and fully re-materializes the note, filling the ## Narrative section a deterministic write otherwise leaves as _Pending Auditor curation._.

INTAKE (independently-gated inbox watcher). AgentOrchestrator._vault_intake_loop (both obsidian_vault_enabled AND vault_intake_enabled required) ticks VaultIntakeEngine.run_cycle every vault_intake_interval_seconds. Per note under the inbox dir: skip if no #roboco tag; skip if already seen (path + content-hash in vault_seen_notes); screen the body via screen_external_text; extract title/description/action-items via a local-model chat call against the SCREENED text, falling back to deterministic extraction; open ONE PENDING task (source=vault_note, Product-Owner-assigned, team=board) capped by vault_intake_max_open_drafts/vault_intake_max_per_cycle; append a feedback callout (best-effort). Never starts delivery directly — the board-review path is the only door.

JANITOR (V2, hourly-ticked, day/week-gated). AgentOrchestrator._vault_janitor_loop (gated on obsidian_vault_enabled alone) ticks every JANITOR_LOOP_INTERVAL_SECONDS (3600, no config knob) and calls VaultJanitor.run_cycle. Actual work only happens when the restart-proof state file (RoboCo/_meta/.janitor_state.json) says it's due:

• Sweep (due when last_sweep is >= 24h stale): _reproject_changed re-projects every task touched since the last sweep (TaskService.list_updated_since, ascending, paged 100 at a time, capped at _MAX_REPROJECT_PER_CYCLE=200 per tick, one bad item logged-and-skipped rather than wedging the pass) via the shared reproject_task; _verify_sample pulls a random 20-task sample of tasks last touched before the sweep window (sample_stale_tasks) and repairs any whose note is missing or whose frontmatter status disagrees with the DB (via touch_task_frontmatter, not a full re-projection); _archive_pass moves terminal tasks past vault_archive_days into Archive/<year>/ (see below). A capped tick advances last_sweep/archive_watermark only to the last-processed item's stamp (not "now"), so the very next hourly tick — already due again — picks up the tail with no gap. Logs one vault_drift_repaired line: count (repaired) / archived / failed.
• Weekly report (due when last_report_week != the current ISO week, and vault_report_enabled is on): _run_weekly_report pulls MetricsService.get_velocity/get_cycle_time_by_stage/get_bottleneck_distribution/get_rework_metrics (days=7) + UsageService.get_summary("7d"), renders VaultWriter.write_org_report, and best-effort notifies the CEO (NotificationService.send_weekly_report_notification) — a notification failure never invalidates the already-written note.

ARCHIVAL (V2, folded into the janitor sweep). Policy: a terminal (completed/cancelled) task whose terminal timestamp (completed_at else updated_at else created_at) is older than vault_archive_days (default 30; 0 disables archival outright) moves from RoboCo/Tasks/<project>/ to RoboCo/Archive/<year>/Tasks/<project>/. TaskService.list_archive_candidates(after, before, ...) returns terminal tasks whose terminal timestamp falls in [watermark, cutoff), paged/capped identically to the changed-task pass (_MAX_ARCHIVE_PER_CYCLE=200). The move itself is free: VaultWriter.write_task is archive-aware (TaskNoteData.archive_year set by vault_assembly._archive_year) — it looks up an existing note across BOTH Tasks/ and Archive/ by id8, writes the new copy at the archive-aware target directory, and deletes the stale copy if it moved. Alias-based wikilinks ([[id8|title]]) mean nothing pointing at an archived task ever breaks. rebuild is archive-aware for free (routes through the same reproject_task), and the shipped Dataview dashboard + graph color groups exclude Archive/.

KB INGEST (V2, independently double-gated). AgentOrchestrator._vault_kb_loop (BOTH obsidian_vault_enabled AND vault_kb_enabled required) ticks VaultKBEngine.run_cycle every vault_kb_interval_seconds (default 900). Per allowlisted dir in vault_kb_dirs (default RoboCo/Notes; config-load validation in Settings._validate_vault_kb_dirs rejects an absolute/..-carrying entry or one overlapping vault_intake_dir or a reserved projection dir): recursively scan *.md, skip a note that's a symlink, escapes the resolved vault root, or exceeds 64KB; content-hash-dedup against the currently-tracked IndexType.VAULT_NOTES docs (an unchanged note is skipped); screen the frontmatter-stripped body through screen_external_text as a hard GATE — a flagged note is quarantined (skipped, warn-logged, a one-line feedback callout appended, and any PRIOR indexed chunks removed if it was previously clean) rather than indexed; a clean note ingests via OptimalService.index_vault_note (bounded to _MAX_INGEST_PER_CYCLE=50 per tick — the tail waits for the next cycle). A deletion pass deindexes any previously-tracked path no longer seen on disk. Consumers: roboco_kb_search picks up VAULT_NOTES for free once the enum exists; MentorService's default/general domain search list includes it (labeled "Vault Notes"); EvidenceRepo.similar_memory includes it in claim-time briefings (kind vault_note), same relevance floor as learnings/playbooks; the panel's KB browser has a full type entry (nav/filter/badge/stats).

REBUILD/RELOCATE (operator/CLI, not agent-facing). python -m roboco.vault rebuild walks every agent, then every task (via the shared reproject_task — archive-aware, narrative-preserving), then every non-private journal entry, then every A2A thread, and materializes the shipped .obsidian//_meta/ assets if absent. relocate <path> moves RoboCo/ into a destination, refusing if the destination already has a RoboCo/ subtree.

Config Flags

• ROBOCO_OBSIDIAN_VAULT_ENABLED — master switch; off = VaultWriter is never invoked from any seam, curate_vault returns invalid_state, the janitor/KB loops return immediately, and python -m roboco.vault refuses. Config default false; both compose files set it true.
• ROBOCO_VAULT_PATH (default /data/vault) — root directory the vault materializes into; bind-mounted in both compose files.
• ROBOCO_VAULT_INTAKE_ENABLED — independent switch for _vault_intake_loop; inert unless the master switch is ALSO on. Config default false; both compose files set it true.
• ROBOCO_VAULT_INTAKE_INTERVAL_SECONDS / ROBOCO_VAULT_INTAKE_DIR / ROBOCO_VAULT_INTAKE_MAX_PER_CYCLE / ROBOCO_VAULT_INTAKE_MAX_OPEN_DRAFTS — cadence, inbox subfolder, per-cycle origination cap, rolling open-draft cap.
• ROBOCO_VAULT_ARCHIVE_DAYS (default 30, 0 disables) — age past which a terminal task's note archives during the janitor sweep. Checked only under the master switch — no separate enable flag.
• ROBOCO_VAULT_REPORT_ENABLED (default true) — the janitor's weekly org-report + CEO notification. Config default true in both compose files (deterministic, no LLM, cheap to leave on).
• ROBOCO_VAULT_KB_ENABLED (default false) — master switch for KB ingest; off = _vault_kb_loop returns immediately and IndexType.VAULT_NOTES stays empty. NAS compose (docker-compose.yml) sets it true; the public registry compose (docker-compose.registry.yml) leaves it false (optional engines ship off).
• ROBOCO_VAULT_KB_DIRS (default RoboCo/Notes, CSV) — vault-relative folders the KB engine scans. Rejected at config load if absolute, ..-carrying, or overlapping vault_intake_dir/Tasks/Journals/A2A/Agents/Archive/Reports/_meta/.obsidian.
• ROBOCO_VAULT_KB_INTERVAL_SECONDS (default 900, min 60) — KB-engine scan cadence.

Health

The projection side is zero-risk by construction: every seam is best-effort and DB-free from the writer's perspective, so a filesystem or permission failure degrades to a stale/missing note, never a blocked verb. Materialize-on-create closes the V1 gap where the Dataview board only ever showed curated/rebuilt tasks — a fresh task is visible immediately. The janitor is the freshness backstop for everything best-effort seams can miss: it's restart-proof (dueness lives in a state file, not loop cadence — an orchestrator that restarts more often than daily still sweeps exactly once per elapsed day), self-healing against a corrupt/hand-edited state file (any unparseable value degrades to "no state," never a wedged loop), and every per-item drain (changed-task, sample-verify, archive) isolates failures — one bad row is logged and skipped, never aborts the pass, and re-qualifies on its next change or the next sample draw. Per-cycle caps (200 reprojects, 200 archives) mean a first-enable or long-downtime backlog drains in bounded hourly slices via the resume-marker convention (a capped tick advances the marker only to the last item it actually processed) rather than one unbounded burst.

The KB-ingest side is the one path with real security stakes — once a vault note is agent-retrievable, unscreened note text is injection into the fleet's retrieval context, not just a drafting risk. It layers defense-in-depth: the config-load validator rejects a dangerous vault_kb_dirs entry outright (can't even start with an escaping/overlapping dir); the engine independently re-checks every allowlisted dir resolves under the vault root before scanning it; every individual note is re-checked for symlink-ness and resolved-path escape before it's read (belt-and-suspenders against a dir-level check being bypassed by a per-file symlink); and the injection guard runs as a hard GATE (not the intake watcher's screen-and-still-process posture) — a flagged note is never embedded, only quarantined with a visible callout so the CEO knows why. Content-hash dedup (shared with the intake watcher's ledger convention) makes both re-scans and the quarantine callout's own append idempotent — appending the callout never itself re-triggers reprocessing.

Rebuild/relocate remain idempotent and additive-safe (ensure_vault_assets never overwrites an existing file), so re-running against a CEO-customized vault cannot clobber .obsidian//_meta/ edits.

Related

• docs/rag/architecture/obsidian-vault.md — the agent-facing doc (what the Auditor and vault-intake-originated tasks actually see, plus what changed for KB retrieval)
• docs/rag/roles/auditor.md — the curate_vault verb
• docs/map/orchestrator.md — _dispatch_vault_curation_work / _maybe_spawn_vault_curation / _vault_intake_loop / _vault_janitor_loop / _vault_kb_loop
• docs/map/task-service.md — _materialize_vault_note / list_updated_since / list_archive_candidates / sample_stale_tasks
• docs/map/product-strategy-research-pitch.md — XEngine, the sibling engine sharing injection_guard.screen_external_text
• docs/internal/specs/2026-07-09-obsidian-vault.md — the original V1 design spec (vault layout, link-stability rationale)
• docs/internal/specs/2026-07-11-obsidian-vault-v2.md — the V2 spec (materialize-on-create, janitor, archival, KB ingest, weekly report, Bases, sync doc)