# RoboCo — The Complete Map (2026-06-29) > Scope: full codebase, baseline `fd10cc862`..HEAD. The CEO suspects recent changes may have broken the system. This is the exhaustive record + drift/regression audit, built from 35 per-slice maps. Branch: `feature/metrics-granularity` (work merged into `master` via PRs #283 and #285), **NOT deployed**. > > Verified range figures: `git log --oneline fd10cc862c2020b3f639cdb686d427b0198a2441..HEAD` returns **2 commits** on `master` (`15effce0` "Chore: 141 Gaps fill-in (#283)" and `3aff6e04` "Chore: Close gaps (#285)"), but the bundled diff is enormous — **577 files changed, +36,653/-4,214 lines** (verified via `git diff --stat`). The two PRs squash months of per-fix work (F0xx audit gaps, F123 worktrees, sequencing, pr_fail loop-closers, model rename, etc.) into two merge commits, so the effective change surface is far larger than the commit count implies. Earlier per-fix commits (e202ce39, 250be5c2, a957e4fa, 82541077, cf7603f3, e52fd05d, 919aa7e2, 12621a36, 9927d248, c03e76c4, 2f322286, c34e978f, 3a4a3fe5, 53d60da3) are ancestors of `15effce0` and are the substance of the gap-fill. ## Table of Contents **Chapters (this front matter):** - System at a Glance - Data Model ERD - Task Lifecycle State Machine - Cross-Cutting Flows - What's Wrong — Drift & Regression Risks - Verified Regression Risks *(appended by a later step)* - Appendix: Git Log fd10cc862..HEAD **Per-slice sections (concatenated after this front matter, in this order):** 1. foundation-lifecycle 2. foundation-batch-sequencing 3. foundation-conventions-identity 4. foundation-policy-misc 5. models 6. db-migrations 7. api-core-websocket 8. api-routes-schemas 9. gateway-support 10. mcp-servers 11. choreographer 12. task-service 13. worksession-git 14. workspace 15. support-services 16. orchestrator 17. runtime-providers 18. prompts-roles-taxonomy 19. notification 20. a2a-audit-journal-permissions 21. pr-gate-review 22. review-findings 23. metrics-observability 24. conventions-service-validator 25. intake-secretary 26. product-strategy-research-pitch 27. engines-heal-ciwatch-depupdate 28. release-manager 29. engine-docs-sync 30. video-engine 31. org-memory-playbooks 32. vault 33. panel 34. deployment-tooling 35. tests --- ## System at a Glance ```mermaid graph TB CEO["CEO (Human)"] Intake["Intake / Prompter
(on-demand, human-only)"] Sec["Secretary
(on-demand, human-only)"] PRRev["PR Reviewer
(read-only reviewer)"] Board["Board"] PO["Product Owner"] HM["Head of Marketing"] Aud["Auditor (silent observer)"] MainPM["Main PM
(coordinates all cells)"] BE["Backend Cell"] FE["Frontend Cell"] UX["UX/UI Cell"] BEDev["2 Devs + QA + PM + Documenter + PR Reviewer"] FEDev["2 Devs + QA + PM + Documenter + PR Reviewer"] UXDev["2 Devs + QA + PM + Documenter + PR Reviewer"] CEO --> Intake CEO --> Sec CEO --> PRRev CEO --> Board Board --> PO Board --> HM Board --> Aud Board --> MainPM MainPM --> BE MainPM --> FE MainPM --> UX BE --> BEDev FE --> FEDev UX --> UXDev Agents["Agent containers
(Claude Code / Grok CLI)"] BEDev --> Agents FEDev --> Agents UXDev --> Agents GW["Gateway
Choreographer + MCP servers
(roboco-flow / roboco-do / roboco-git-readonly / roboco-optimal / roboco-docs)"] Agents -->|intent verbs + content tools| GW Svcs["Services
TaskService · GitService · WorkSessionService
WorkspaceService · Messaging · Notification
Optimal/RAG · Journal · Audit · Permissions"] GW --> Svcs PG[("PostgreSQL + pgvector")] Redis[("Redis")] Svcs --> PG Svcs --> Redis Orch["Orchestrator
(spawn / reap / park-probe / respawn-tracker / engines)"] Orch -->|spawn/reap| Agents Orch --> Svcs Ollama["Ollama
(qwen3-embedding:0.6b + glm-5.2:cloud)"] Svcs -->|embeddings + local LLM| Ollama Panel["Next.js Panel"] Nginx["nginx :3000"] Panel --> Nginx Nginx -->|/api/* /ws/*| Orch Nginx -->|everything else| Panel ``` ```mermaid flowchart TD RT["RoboCo"] RT --> LC["Lifecycle"] LC --> LC1["15 statuses"] LC --> LC2["about 30 intent verbs"] LC --> LC3["22 atomic actions"] LC --> LC4["spec gates and import validators"] RT --> GW["Gateway"] GW --> GW1["Choreographer"] GW --> GW2["VerbRunner SAVEPOINT"] GW --> GW3["MCP servers flow do git optimal docs"] GW --> GW4["role manifests"] RT --> OR["Orchestrator"] OR --> OR1["spawn and reap"] OR --> OR2["park-and-probe"] OR --> OR3["respawn-tracker durable"] OR --> OR4["gateway-health recovery"] RT --> SV["Services"] SV --> SV1["TaskService"] SV --> SV2["GitService"] SV --> SV3["WorkSessionService"] SV --> SV4["WorkspaceService"] SV --> SV6["Notification"] SV --> SV7["Optimal RAG"] SV --> SV8["Journal Audit Permissions"] RT --> EN["Engines"] EN --> EN1["self_heal"] EN --> EN2["ci_watch"] EN --> EN3["dep_update"] EN --> EN4["release_manager"] RT --> OM["OrgMemory"] OM --> OM1["distill learnings"] OM --> OM2["briefings injection"] OM --> OM3["playbooks"] RT --> CO["Comms"] CO --> CO2["A2A"] CO --> CO3["notifications dedup"] RT --> PA["Panel"] PA --> PA1["Next.js 16"] PA --> PA2["nginx"] PA --> PA3["WebSocket streams"] RT --> MG["Migrations"] MG --> MG1["001 to 054"] MG --> MG2["pgvector"] RT --> PR["Prompts"] PR --> PR1["roles"] PR --> PR2["taxonomy"] PR --> PR3["guard"] ``` --- ## Data Model ERD Synthesized from `models` + `db-migrations` (38 ORM tables in `roboco/db/tables.py`, migrations 001→054). Key fields shown on `Task`; relationships traced through the migration chain (single-active work_session via 047, batch/collision cols via 046, cell-project map via 052, respawn counter via 051, playbooks via 050, conventions cache via 043, observability `revision_count` + audit index via 045). ```mermaid erDiagram PROJECT ||--o{ TASK : "project_id" PROJECT ||--o{ TASK_CELL_PROJECT : "per-cell map (052)" PROJECT ||--|| PROJECT_CONVENTIONS_CACHE : "per HEAD sha (043)" AGENT ||--o{ WORK_SESSION : "assignee" TASK ||--o{ WORK_SESSION : "one ACTIVE (047)" TASK ||--o{ COMMIT_REF : "commits" TASK ||--o{ AUDIT_LOG : "transition journey" TASK ||--o{ TASK : "parent_task_id / subtasks" TASK ||--o{ TASK_CELL_PROJECT : "root-subtask map" TASK ||--o{ PLAYBOOK : "draft→approved→indexed (050)" AGENT ||--o{ MESSAGE : "sender" AGENT ||--o{ NOTIFICATION : "to_agents / from_agent" AGENT ||--o{ JOURNAL : "owner" AGENT ||--o{ AGENT_SPAWN_SESSION : "per-spawn token totals (026b)" AGENT ||--o{ AUDIT_LOG : "actor" RESPAWN_TRACKER }o--|| TASK : "durable PM-respawn counter (051)" WAITING_RECORD }o--o{ TASK : "persisted dispatcher waiting (restore at start)" PROJECT ||--o{ PROVIDER_CONFIG : "model routing (004)" PROVIDER_CONFIG ||--o{ MODEL_ASSIGNMENT : "scope=SLUG/ROLE/GLOBAL (004)" PROJECT ||--o{ PRODUCT : "product_projects (016)" PRODUCT ||--o{ TASK : "product_id" PROMPTER_SESSION ||--o{ PROMPTER_MESSAGE : "intake live chat (024)" PROMPTER_SESSION ||--o{ TASK_DRAFT : "drafts (024)" SECRETARY_DIRECTIVE ||--o{ SECRETARY_DIRECTIVE : "gate queue (035)" INDEXED_DOCUMENT ||--o{ CHUNKS : "RAG corpus (pgvector, 017/030/031)" PROJECT ||--o{ PROJECT_CONVENTION_FINDINGS : "violations (044)" AUDIT_LOG ||--o{ AUDIT_LOG : "details JSONB (010)" TASK { uuid id PK str title str status "15-value StrEnum" str task_type "code/documentation/research/planning/design/administrative" uuid project_id FK "nullable (018)" uuid product_id FK "nullable (016)" uuid parent_task_id FK str branch_name "auto on claim" int pr_number str pr_url bool docs_complete bool pr_created bool self_verified uuid batch_id "MegaTask (046)" str_arr intends_to_touch "collision globs (046)" bool adds_migration "046" bool touches_shared "046" int revision_count "rework counter (045)" str source "prompter/release_manager (023)" bool confirmed_by_human "CEO-held (023)" uuid claimed_by uuid work_session_id str_arr completed_dependency_ids "026a" bool board_review_complete "021" jsonb markers "original_developer / head_sha (041)" } WORK_SESSION { uuid id PK uuid task_id FK uuid agent_id FK str branch_name str status "ACTIVE superseded by 047 partial-unique" int pr_number str pr_url bool merged uuid merged_by } AGENT { uuid id PK str slug str role "agentrole enum" str team "team enum" str model_provider "ANTHROPIC/GROK/LOCAL/OLLAMA_CLOUD/OPENAI" } PROJECT { uuid id PK str slug str git_url str default_branch "master (022)" bytes git_token_encrypted "Fernet" str quality_command "029" bool ci_watch_enabled "048" str ci_watch_workflow "048" str dep_update_command "049" str_arr dep_update_paths "049" } AUDIT_LOG { uuid id PK uuid task_id FK str event_type "task. / task.qa_fail / task.pr_fail" uuid agent_id FK jsonb details "transition journey (010)" } PLAYBOOK { uuid id PK str status "draft/approved/archived/rejected (050)" uuid approved_by str source_uri "roboco://playbooks/..." } RESPAWN_TRACKER { str agent_slug PK uuid task_id PK int strike_count "durable PM-respawn (051)" } ``` --- ## Task Lifecycle State Machine Synthesized from `foundation-lifecycle` (`_STATUS_TRANSITIONS` / `STATUS_GRAPH`). Role labels mark the allowed actor for each edge; the in-path PR-review gate (`awaiting_pr_review`) sits between PM `submit_up`/`submit_root` and `awaiting_pm_review`. ```mermaid stateDiagram-v2 [*] --> backlog backlog --> pending: activate (PM) pending --> claimed: claim (role matches task type) claimed --> in_progress: start claimed --> pending: unclaim (assignee/PM) in_progress --> blocked: block in_progress --> paused: pause blocked --> in_progress: unblock (PM) blocked --> pending: unblock (never-claimed) blocked --> awaiting_ceo_approval: escalate_to_ceo paused --> in_progress: resume in_progress --> verifying: submit_verification verifying --> awaiting_qa: submit_qa (requires self_verified + commits + pr_number + progress) in_progress --> awaiting_pr_review: submit_up / submit_root (PM, opens assembled PR) awaiting_pr_review --> awaiting_pm_review: pr_pass (PR reviewer) awaiting_pr_review --> needs_revision: pr_fail (PR reviewer) awaiting_qa --> awaiting_documentation: qa_pass (QA) awaiting_qa --> needs_revision: qa_fail (QA) awaiting_documentation --> awaiting_pm_review: docs_complete (documenter/dev) in_progress --> completed: pr_review_done (PR reviewer, external PR) awaiting_pm_review --> completed: complete (PM) awaiting_pm_review --> awaiting_ceo_approval: escalate_to_ceo (PM) awaiting_pm_review --> needs_revision: PM reject awaiting_ceo_approval --> completed: ceo_approve (CEO — merges) awaiting_ceo_approval --> needs_revision: ceo_reject (CEO) awaiting_ceo_approval --> cancelled: ceo_reject (CEO) needs_revision --> claimed: claim (dev/PM) needs_revision --> in_progress: re-entry in_progress --> pending: reaper (stale-claim) [*] --> cancelled: cancel (PM/CEO, any non-terminal) completed --> [*] cancelled --> [*] ``` --- ## Cross-Cutting Flows ### (a) Agent → MCP → Choreographer → Service → Envelope ```mermaid sequenceDiagram participant Agent participant MCP as MCP flow_server participant Ch as Choreographer participant Spec as lifecycle spec participant VR as VerbRunner participant Svc as TaskService / GitService Agent->>MCP: verb(agent_id, task_id, notes) MCP->>Ch: intent Ch->>Svc: task.get Ch->>Ch: _briefing_for (institutional memory) Ch->>Spec: can_invoke_intent(role, verb, task, ctx) alt rejected Ch-->>Agent: Envelope error (remediate, missing, next) else allowed Ch->>Ch: preflight guards (claim / conventions / behind_base / unchanged_pr) Ch->>VR: run_intent(verb) VR->>Svc: pre_side_effects (create_root_pr) [outside SAVEPOINT] VR->>Svc: session.begin_nested() SAVEPOINT loop composes (claim/set_plan/start/submit_qa/...) VR->>Svc: atomic action Svc-->>VR: updated task (None on concurrent mismatch) alt intermediate None VR-->>Ch: raise INVALID_STATE end end VR->>Svc: commit savepoint VR->>Svc: side_effects (push_branch / create_pr / pr_merge) VR-->>Ch: final task Ch-->>Agent: Envelope ok(status, next, context_briefing, evidence) end ``` ### (b) Orchestrator dispatch / reap / park-probe / respawn durability ```mermaid graph LR Tick["dispatch tick"] --> Spawn["spawn_agent"] Spawn --> Running["running instance"] Running --> Reaper["stalled-claim reaper"] Reaper --> Live{"live container?"} Live -- yes --> GWHealth{"gateway-health probe
(ROBOCO_GATEWAY_HEALTH_ENABLED)"} GWHealth -- broken past grace --> Kill["kill + evict + release claim"] GWHealth -- healthy/inconclusive --> Spare["spare"] Live -- no --> Release["release claim + respawn"] RL["provider 429 / 529 / session-limit"] --> Park["park provider
(ROBOCO_OVERLOAD_BREAK_ENABLED)"] Park --> ProbeLoop["background probe-and-resume loop"] ProbeLoop --> Revive["revive parked work"] Strike["respawn strike"] --> RT["respawn_tracker table (051, durable)"] RT --> Restore["restore_respawn_tracker at start()"] ``` ### (c) MegaTask sequencing ```mermaid graph TB Umb["MegaTask umbrella
(branchless, Main PM)"] --> Root1["root-subtask #1
(Main PM, branch+PR)"] Umb --> RootN["root-subtask #N
(Main PM, branch+PR)"] Root1 --> Seq["SequencingService.analyze
(surfaces, cell_of, cell_capacity)"] RootN --> Seq Seq --> Waves["Kahn-layered waves"] Waves --> CellTasks["cell tasks (cell PMs)"] CellTasks --> DevSub["dev subtasks"] Seq -- "file-overlap" --> Serial["serialize (priority, idx)"] Seq -- "migration-adders" --> Chain["chain serially"] Seq -- "shared-surface" --> SharedAfter["run after each non-shared overlap"] Seq -- "independent" --> Parallel["parallel"] ``` ### (d) In-path PR-review gate ```mermaid stateDiagram-v2 in_progress --> awaiting_pr_review: submit_up (cell PM) / submit_root (Main PM) awaiting_pr_review --> awaiting_pm_review: pr_pass (PR reviewer) awaiting_pr_review --> needs_revision: pr_fail (PR reviewer) needs_revision --> in_progress: dev re-entry awaiting_pm_review --> awaiting_ceo_approval: escalate_to_ceo (Main PM) awaiting_ceo_approval --> completed: ceo_approve (CEO merges) ``` ### (e) Default-off engines — each originates one task into the normal delivery flow (+ PR-review gate), never auto-merges, bounded by caps, held for CEO where applicable ```mermaid graph LR SH["self_heal_engine
(ROBOCO_SELF_HEAL_ENABLED)"] -->|HELD task awaiting_ceo_approval| Flow["normal delivery flow"] CIW["ci_watch_engine
(ROBOCO_CI_WATCH_ENABLED)"] -->|READY task| Flow DEP["dep_update_engine
(ROBOCO_DEP_UPDATE_ENABLED)"] -->|READY task| Flow RM["release_manager_engine
(ROBOCO_RELEASE_MANAGER_ENABLED)"] -->|release proposal HELD for CEO| Flow Flow --> Gate["PR-review gate"] Gate --> CEOGate["CEO approval (never auto-merge / never auto-deploy)"] OM["org_memory loop
(ROBOCO_ORG_MEMORY_ENABLED)"] -.->|distill on complete + inject on claim| Flow ``` --- ## What's Wrong — Drift & Regression Risks **Method.** Thirty-one per-slice maps were produced from the `fd10cc862..HEAD` tree. Each slice's `## Regression Risks` and `## Drift from CLAUDE.md` sections were extracted with `grep` (480 + 781 lines of raw evidence), then deduplicated and ranked across slices. The table below consolidates the top ~25 risks, critical/high first; the full per-slice risk tables follow in each slice section. A separate "Verified Regression Risks" chapter (appended by a later step) will adversarially verify the top 12 against the live code. ### Top Regression Risks (consolidated across 31 slices) | # | Risk | Slice | File:Line | Claim | Severity | |---|------|-------|-----------|-------|----------| | 1 | `ceo_approve` skips work-session close | task-service | `roboco/services/task.py:5146` | `ceo_approve` calls `_remove_task_worktree_on_terminal` but NOT `_close_work_session_for_task` (only `complete()` does). CEO-approved root tasks leave the WorkSession row not marked closed → reporting/session-resolution drift. **FIXED post-snapshot: `536bbb64` added `_close_work_session_for_task(task, reason="ceo approved")` before worktree removal (task.py:5379).** | High | | 2 | `fail_qa` route depends on unreliable `original_developer` marker | task-service | `roboco/services/task.py:4228` | Fast path reads the marker; if absent falls to `_resolve_revision_dev`. If both miss (no dev work session, e.g. parent-only edit) task is unassigned to pool → a PM may grab a dev task (the original 2026-06-27 loop). | High | | 3 | `do`/`a2a` any-role token gate | api-routes-schemas | `roboco/api/routes/v1/do.py:43`, `a2a.py:114` | `require_any_authenticated_agent` only verifies HMAC + agent exists; does NOT assert role matches the verb's role family. A QA-signed token could call `do/commit`; a dev could call `a2a` admin paths. Service-layer scope is the sole guard → a missed service check = privilege escape. | High | | 4 | 422 response echoes secrets | api-routes-schemas / api-core-websocket | `roboco/api/middleware.py:407` (resp body), `:434` | `_scrub_secrets` redacts only the **log** body; the JSON response still contains `body` with the caller's original `git_token`/`api_key`. A 422 returns the secret to the client (and any MITM/log of the response). | High | | 5 | `pr_merge` project_id scoping assumes non-None | choreographer | `roboco/services/gateway/choreographer/_verb_runner.py:263` | `project_id=task.project_id` — if a coordination/umbrella task reaches `pr_merge` with `project_id=None`, the cross-repo collision guard silently matches nothing or None-keys the scoping; could merge the wrong PR or no-op. | High | | 6 | `_submit_*_unchanged_pr_guard` fails open on resolver regression | choreographer | `roboco/services/gateway/choreographer/_impl.py:6175,6240` | Any future break in `_current_pr_head_sha` / `_project_slug_for` / `git.get_pr_head_sha` makes the pr_fail loop-stopper a no-op, re-opening the 2026-06-27 pr_fail re-submit loop silently. | High | | 7 | Intermediate-None trailing-None contract | choreographer | `_verb_runner.py:89` + `_impl.py:1277,6358` | A verb that forgets the trailing-None guard None-derefs `t.status`; any NEW verb using `run_intent` with a possibly-None last action inherits the trap. **FIXED post-snapshot: `0e7674af` added `if task is not None:` guard before the side_effects loop (_verb_runner.py:107); trailing None now flows out without touching side effects.** | High | | 8 | `has_cell_projects` threaded incorrectly breaks branchless exemption | foundation-batch-sequencing | `roboco/foundation/policy/batch.py:66` | `is_branchless_coordination` now requires callers to pass `has_cell_projects`. A call site that omits it (defaults False) for a cell-map root will NOT recognize it as branchless and will demand a branch/PR the root cannot supply — wedging that root in the git gate. Any new call site is a landmine. | High | | 9 | Kanban admin-override drag skips lifecycle | panel | `panel/src/components/kanban/core/kanban-board.tsx:160` | A confirmed override routes through `useUpdateTask` (admin status-override), bypassing the in-band validator. `skippedPreconditions` is precision-over-recall — a careless confirm can complete a task with no PR / QA-bypass / docs-incomplete. | High | | 10 | `create_all` schema drift — NOT NULL ORM-mapped columns break all DB tests | tests | `tests/conftest.py:182` | Schema built via `Base.metadata.create_all`, not alembic. A post-052 migration adding a NOT NULL ORM-mapped column without `server_default` breaks every `db_session` test. Conftest only backfills migration-006 cols. | High | | 11 | Cycle-time SQL depends on audit-log event naming | metrics-observability | `roboco/services/metrics.py:554` | A future named audit event whose `to_status` resolves under `event_type='task.'\|\|to_status` could inject zero-length stages or skew dwell averages across every cycle-time/bottleneck panel. | High | | 12 | Rework cost join on `agent_spawn_sessions.task_id` | metrics-observability | `roboco/services/metrics.py:742` | If spawn sessions stop populating `task_id` (orchestrator regression), rework cost silently drops to $0 — underreported CEO spend. | High | | 13 | Release Redis mutex TTL shorter than worst-case execute | release-manager | `roboco/services/release_proposal.py:39` | `_RELEASE_LOCK_TTL_SECONDS=3000` (50min) but execute can run clone+gate+CI+publish ≈ 85min. TTL expires mid-execute → a second approve acquires and `rm -rf`s the in-flight clone, corrupting the release. **FIXED post-snapshot: `05616607`+`2759edf7` added a background `_heartbeat_loop` (`_RELEASE_LOCK_HEARTBEAT_SECONDS=60`) that refreshes the TTL while the lock is held; TTL is now a crash-backstop only and cannot expire under a live execute. Fencing token (compare-and-del Lua script) also prevents a stale first-finally from stealing a usurper's lock.** | High | | 14 | Stream-bus handler failure leaves message pending → duplicate side effects | support-services | `roboco/events/stream_bus.py:338` | ACK only when all handlers succeed; `recover_pending` re-runs idle≥60s messages. Non-idempotent notification handlers can double-fire after a crash/restart. **FIXED post-snapshot: `e4ed970f` added per-`(event.id, handler)` SET-NX idempotency guard (`_run_handler_guarded`): successful handlers set a Redis key that blocks replay; failed handlers clear the key so replay re-runs them. Also added a dead-letter stream for undecodable poison pills and a periodic `_reclaim_loop` (every 60s).** | High | | 15 | `resolve_for_agent` silently downgrades to Anthropic | support-services | `roboco/services/llm.py:124,193,205` | Decrypt failure / unreachable LOCAL / missing assignment all return the legacy Anthropic route instead of raising — a misconfigured Grok/Ollama fleet spawns against Anthropic with only a log warning. | High | | 16 | LLM model rename breaks cached ollama deployments | deployment-tooling | `docker-compose.yaml:86` | `ollama-init` verify now greps for `glm-5.2` exactly. A NAS volume with only the old `glm-5:cloud` cached (no network) hits FATAL exit and blocks boot until the new model is pulled. | High | | 17 | Gateway-health over-reap of live containers | orchestrator | `roboco/runtime/orchestrator.py:8689` | `_maybe_recover_broken_gateway` kills a live container past `gateway_health_grace_seconds`; a flaky false-broken probe streak could kill a healthy agent mid-long-edit. | Medium-High | | 18 | Readopt liveness false-positive | orchestrator | `roboco/runtime/orchestrator.py:8547` | `_readopt_running_agents` registers ACTIVE for any running `roboco-agent-{slug}` container at startup, including a zombie from a prior orchestrator that already released the claim — blocks re-spawn until the stale container is noticed. | Medium | | 19 | Stalled-claim reaper live-skip blind spot | orchestrator | `roboco/runtime/orchestrator.py:8750` | `_should_skip_live_reap` spares any live container that is neither grok-wedged nor gateway-broken; a Claude agent alive but stuck in a non-verb loop keeps its claim forever. | Medium | | 20 | DB purpose-dedup gated to ack-required types only; `_persist_and_deliver` skips it entirely | messaging-notification | `roboco/services/notification.py:521`, `notification_delivery.py:875` | Task-handoff notifications (blocker/escalation/ceo-rejection) are not DB-deduped past the 60s Redis window — a retried `i_am_blocked`/`escalate` beyond 60s re-creates an unacked duplicate (the inbox-inflation + i_am_idle soft-block the DB dedup was added to prevent). | Medium | | 21 | `acknowledge` publishes `NOTIFICATION_ACKED` directly, not via the transactional outbox | messaging-notification | `roboco/services/notification_delivery.py:451` | Same phantom-event class F107 fixed for `deliver`, left unfixed for the ACK path — a rollback after a successful ACK publish emits a phantom ACK. | Medium | | 22 | A2A legacy notification suppressed by new loop-prone re-fire guard | a2a-audit-journal-permissions | `roboco/services/a2a.py:640` | Since `3aff6e04`, `send_a2a_notification` runs the 60s Redis guard for loop-prone types before creating the notification. A legitimate A2A peer notification re-sent within 60s (real state change, not a respawn loop) can be silently dropped. | Medium | | 23 | `sync_branch` has no source-status gate — callable on terminal/paused/blocked tasks | foundation-lifecycle | `roboco/foundation/policy/lifecycle.py:1085` | `sync_branch` composes=() and is not in the special-case list, so `can_invoke_intent` only checks role + OWNERSHIP. A dev who owns a COMPLETED/CANCELLED task passes the spec gate; the rebase runs against a finished task's branch. | Medium | | 24 | `_curate_playbook` explicit `session.commit` before index | gateway-support | `roboco/services/gateway/content_actions.py:847` | If the caller's session is in `PendingRollbackError` (prior mid-verb failure poisoned it), this commit raises and the whole curation verb 500s instead of a clean envelope. | Medium | | 25 | Coverage omit list hides orchestrator/git/workspace regressions from the 80% gate | tests | `pyproject.toml:259` | `[tool.coverage.run].omit` excludes `orchestrator.py`, `git.py`, `workspace.py`, `mcp/*`, `agents/*`. A regression in the respawn-tracker upsert, pr_merge cross-repo scoping, or worktree routing will NOT fail `make quality`'s `--cov-fail-under=80`. | Medium | > Additional notable Medium risks not deduplicated into the table above (see per-slice sections): `archive_playbook` behavior change (gateway-support); notify to prompter/secretary now refused (gateway-support); blocked task now blocks new claims (gateway-support); `apply_escalation` bypasses validator (task-service); branchless `ceo_reject` uses `admin_set_status` (task-service); `revision_count` bump is in audit helper only (task-service); `cancel` cascade swallows role violations (task-service); `_merge_with_retry` 405 → `MergeConflictError` (worksession-git); `close_pull_request` deletes branch on close by default (worksession-git); F123 worktree merge-sync runs in clone root not worktree (worksession-git); 1-cell map silently drops `product_id` (intake-secretary); malformed `project_id` in multi-cell map silently collapses shape (intake-secretary); collision-surface declaration is prompt-only not gate-enforced (prompts-roles-taxonomy); `submit_root` branch-keyed-vs-task_type-keyed prompt assertion (prompts-roles-taxonomy); `resolve_task_project_slug` cell_projects branch `AttributeError` (pr-gate-review); breaker substitution masks fixable rejection (mcp-servers); 404 synthesis assumes every route returns 200 (mcp-servers); intake composer SSE stuck (panel); panel token on live-chat bridges (panel); `/ws/system` ungated while siblings require panel token (api-core-websocket); cross-repo PR collision via `/api/work-sessions/{id}/pr/merge` (api-routes-schemas); orchestrator CEO gate vs release CEO gate divergence (api-routes-schemas); Grok directory mount widens RO exposure (runtime-providers); 6h `expires_at` default can burn the single-use refresh_token (runtime-providers); worktree `.venv` symlink self-heal depends on a later ensure (workspace); `ensure_worktree` reuses existing branch ref without validating base (workspace); `commit_and_push` RuntimeError unhandled by execute (release-manager); Redis outage fully blocks release approval (release-manager); `TranscriptionService` sync callbacks stall flush (support-services); `get_ready_buffers` unbounded growth (support-services); pitch partial-failure orphans GitHub repos (product-strategy-research-pitch); self-heal CEO notification spam (engines); ci_watch multi-workflow monorepo under-count (engines). ### Drift from CLAUDE.md (consolidated) | Slice | Drift | |-------|-------| | foundation-lifecycle | `BLOCKED -> AWAITING_CEO_APPROVAL` via `escalate_to_ceo` is in the spec but missing from the doc's Role-Based Transitions table. Per-role verb table omits `i_am_idle` (stated only in prose). Doc undersells the enforcement shim (it owns `GitContext`/`validate_git_requirements`/SLA tables, not a pure view). | | foundation-batch-sequencing | Doc omits the undeclared-surface same-assignee lane fallback (`a957e4fa`), the cell-map branchless shape, `is_valid_batch_shape`, `main_pm_cannot_own_code`, and edge kinds 2–4. | | foundation-policy-misc | Doc does not mention `VERB_RETRY_LIMITS` / per-verb circuit breaker / `pm_respawn_max_tracing_resets`. Agent learnings role-exclusion lives in `notification_delivery`, not `journaling.py`. | | foundation-conventions-identity | None material (additive `role_for_slug_or_none` helper). | | models | `Role`/`Team` are aliases to `foundation.identity` (base.py:21–24), not defined in `base.py` — CLAUDE.md's "Role/Team in agent.py+base.py" is slightly stale. `Task` carries `cell_projects`/`batch_id`/`intends_to_touch`/`adds_migration`/`touches_shared` (task.py:171,217–228) that the "Data Models" prose omits (but the MegaTask section covers). No `AuditEvent` class (it's `AuditEventType`) and no `A2AEnvelope` in models (gateway `Envelope` lives in `services/gateway/`). | | db-migrations | Doc says "52 migrations 001..052" — correct, but does not mention the two chained 026 files. No factual drift. | | task-service | None material. | | worksession-git | Doc undersell: commit header format and gateway merge-path description are documentation-undersell, not behavioral mismatch. | | workspace | Doc's "fresh claim `git reset --hard`" narrative diverges from the post-F123 worktree model (by design) — doc drift to reconcile. | | choreographer | Verb table omits `sync_branch` from the developer list (added since baseline). Otherwise matches. | | pr-gate-review | None material. | | gateway-support | Auditor surface doc under-states `notify_list`/`notify_get` (additive, consistent with footnote). PM coordinator-skip lives in Choreographer not `claim_guards.py`. | | orchestrator | None material (well-instrumented). | | runtime-providers | `ClaudeCodeProvider` is dead reference code; its "default" label in CLAUDE.md is misleading. | | engines-heal-ciwatch-depupdate | Minor framing: engines consume telemetry via `MultiProjectCITelemetrySource`, not `GitService` directly. Engine does not enforce `awaiting_ceo_approval` itself. | | release-manager | None material. | | org-memory-playbooks | None material. | | notification | None material. | | a2a-audit-journal-permissions | Doc lists `PermissionsService` (plural); actual class is `PermissionService` (singular). Legacy A2A-protocol path (`create_a2a_notification` / `TASK_ASSIGNED` re-spawn) undocumented. `AuditService.has_recent_tracing_gap` undocumented. | | conventions-service-validator | None material (all doc claims match code). | | intake-secretary | None material. | | product-strategy-research-pitch | None material (slice unchanged). | | metrics-observability | None material (slice unchanged). | | support-services | None material (slice unchanged). | | mcp-servers | Doc's server table is stale: lists 3 servers + omits many tools; intake/secretary/search are agent-facing MCP servers not listed. | | api-core-websocket | No direct CLAUDE.md contradiction; the stale security docstring lives in `websocket.py` itself (describes old query-param model vs actual HMAC). | | api-routes-schemas | None material; `post_pr_review` is additive, not contradictory. | | panel | None material. | | deployment-tooling | Doc omits panel/nginx from the compose services table; reverses panel/orchestrator build order in prose; `roboco-bootstrap = roboco.bootstrap:cli` console script points at a non-existent symbol; **Configuration section still documents `ROBOCO_LOCAL_LLM_MODEL=glm-5:cloud` while code now defaults to `glm-5.2:cloud`**; documented image set incomplete (grok-prompter/secretary/pr-reviewer images unlisted). | | tests | None material. | | prompts-roles-taxonomy | Stale agent count in `base.md` (22 vs CLAUDE.md's 25). | ### Assessment The baseline→HEAD diff is unusual: only two commits appear on `master`'s first-parent line (`15effce0` and `3aff6e04`), but they bundle a **+36,653/-4,214 line, 577-file** change that squashes months of per-fix work — F0xx audit gaps, F123 per-task worktrees, sequencing S1/S2/S3, the pr_fail loop-closer, the model rename, the enum-gate fix, and the bash-guard `/app` venv protection. Read against the 2026-06-28 logic-gap audit (140 confirmed gaps, all resolved), the picture is not "the system is broken"; it is "a hardened system that absorbed a massive consolidation pass and, in doing so, opened several new seams." **What clearly hardened.** The cross-repo PR-number collision that crashed `cell_pm_complete` is fixed with `project_id` scoping. The pr_fail re-submit loop is closed at three layers (head-sha capture, `submit_root`/`submit_up` unchanged-PR guards, a2a to owning PM). The single-active work-session defect is enforced both at the service layer and by migration 047's partial-unique index. The PM-respawn counter is now DB-durable (051) with an upsert race fix. The 60s Redis loop-prone re-fire guard and `VERB_RETRY_LIMITS` circuit breaker tame the notification/respawn storms. F123 per-task worktrees eliminated the coordinator-PM clone clobber and routed commit/conventions/rebase into the worktree. Sequencing S1/S2/S3 + the per-dev lane barrier (`82541077`) close the out-of-order-start wedge. The WS fan-out no longer back-pressures on a slow client, 422 logs no longer leak credentials, and the WS + HTTP panel-token gates close the operator-only invariant. **What is genuinely new and wrong.** Five gaps appear that did not exist (or were not load-bearing) at the baseline. (1) `ceo_approve` is asymmetric with `complete()`: it removes the worktree but skips work-session close and the full completion hooks, so CEO-approved root tasks leave unclosed sessions and never get code-changes/decision RAG indexing — a reporting and corpus drift. (2) `fail_qa` routing still depends on the unreliable `original_developer` marker with a work-session fallback that has no guarantee a dev session exists; the 2026-06-27 dev-loop it was meant to close can still recur on a parent-only edit. (3) The `do`/`a2a` any-role token gate means the HMAC check never asserts the role matches the verb's role family — privilege escape is one missed service-scope check away. (4) The 422 response body still echoes `git_token`/`api_key` back to the client (`_scrub_secrets` only scrubs the log) — a real secret-leak surface. (5) Gateway-health recovery, while closing a real blind spot, can over-reap a live healthy container on a flaky false-broken probe streak — killing an agent mid-long-edit. None of these are crash bugs on the happy path; all are correctness/privilege/integrity drift that the happy path never exercises. **Standing landmines the diff did not touch but the diff's blast radius now amplifies.** The enum-parity gate can false-green on an empty/mismatched `roboco` DB; `sa.Enum(create_type=False)` in 001 is a latent no-op on clean re-apply; missing pgvector aborts `init_db`; `has_cell_projects` is a sharp footgun for any new `is_branchless_coordination` call site; the release Redis mutex TTL (50min) is shorter than worst-case execute (~85min), re-opening the `rm -rf`-clone race it was added to prevent; the coverage omit list excludes the very modules that changed most (orchestrator/git/workspace), so a green `make quality` does not mean those hot paths are covered. The single-commit bundling of nearly every panel logic fix means a partial revert can drop several independent fixes at once. **Verdict.** The system is **not at its prime, but it is not broken either — it is hardened-but-drifting.** The 159-commit-equivalent gap-fill closed more real race conditions and cross-repo collisions than any prior wave, and the core delivery flow (claim → plan → start → submit → QA → PR-gate → PM/CEO review → complete) is structurally sound and well-instrumented. But the consolidation pass introduced a small set of new integrity seams — the `ceo_approve` completion asymmetry, the `fail_qa` routing fragility, the any-role token gate, the 422 secret echo, and the gateway-health over-reap — that are worth fixing before the next deploy, and the standing landmines (enum-parity, pgvector, mutex TTL, coverage omissions) are worth arming against. The CEO's suspicion that recent changes "may have broken the system" is, on the evidence, **partially warranted at the edges and not warranted at the core**: no meltdown-class regression is present, but five correctness/privilege gaps and several standing landmines mean a deploy without addressing them carries real (if non-fatal) risk. --- ## Appendix: Git Log fd10cc862..HEAD `git -C /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco log --oneline fd10cc862c2020b3f639cdb686d427b0198a2441..HEAD`: ``` 3aff6e04 Chore: Close gaps (#285) 15effce0 Chore: 141 Gaps fill-in (#283) ``` Two commits on `master`'s first-parent line, bundling a **577-file, +36,653/-4,214** diff (`git diff --stat fd10cc862..HEAD`). The substance of the gap-fill is the per-fix commits squashed into `15effce0` (ancestors: `e202ce39`, `250be5c2`, `a957e4fa`, `82541077`, `cf7603f3`, `e52fd05d`, `919aa7e2`, `12621a36`, `9927d248`, `c03e76c4`, `2f322286`, `c34e978f`, `3a4a3fe5`, `53d60da3`, and the F123/F-fix wave). The `feature/metrics-granularity` branch is **NOT deployed**. > Post-snapshot updates (since 2026-06-29, branch `chore/logical-gaps-element-sweep-fixes` + merged via PR #286 `536bbb64`): logical-gap sweep fixed ceo_approve work-session closure (Risk #1), verb_runner trailing-None side-effect guard (Risk #7), release-mutex heartbeat (Risk #13), and stream-bus idempotency guard + dead-letter (Risk #14). Two new migrations landed: `053_playbook_archived_attr` and `054_a2a_message_skill` (total now 001→054; ORM table count increased from 37 to 38). Chat-subsystem commits (`76ce53e3` MESSAGE_SENT wired end-to-end in websocket_bridge, `0065ecbb` session task_links, `2da72f3f` closed-session/reply_to guard, `77958c1e` read IDOR fixes, `5cb4e85f` secretary SSE hardening, `a1127daf` session-task endpoint fix) landed after the snapshot. A2A-routes hardening: `5bec3ec5` stamped authenticated caller slug as responder (spoof fix) and added PM-only gate on cancel_task. Hotfixes: `cfe725da` worktree clone-root recovery, `00513399` push_branch named-branch, `9faf2763`/`7be10057` VIRTUAL_ENV agent-image strip. Key sweep commits: `ec2e49af` pr_review_claim active_claimant_id, `d8a5bb48` a2a hierarchy gate + skill persist, `e4ed970f` stream-bus (see Risk #14), `0e7674af` verb_runner (see Risk #7), `05616607`+`2759edf7` release executor (see Risk #13), `ef33d56c` lifecycle-enforcement validators, `16b71be8` lifecycle 6-gap fix, `f90565ea` pr_gate MegaTask-root classification, `b49337e7` route-layer force/privileged-field gates, `115061f3` notification_delivery over-fetch fix. --- ## Delta 2026-07-02 — hotfix batch (PR #293 `0f1ed3cc` + local `8e5f84c4`) Nine live-run fixes, all merged to `master` the same day (commits `81f448bb`, `011158db`, `9d10217c`, `7dff3237`, `1cf24ff1`, `569a6157`, `c1acbd5b`, `298751e6`, `53bb0420`, `4de81d92`, `a260f903`, `caecb816`, `fe9e5589`, `8e5f84c4`): 1. **Delegate MCP tool carries the collision surface** (`81f448bb`, `7dff3237`) — `roboco/mcp/flow_server.py` `delegate` gains `intends_to_touch` / `adds_migration` / `touches_shared` / `depends_on` and forwards them. The `TASK_AT_DELEGATE` gate (`roboco/foundation/policy/task_completeness.py`) required `intends_to_touch` on code delegations but the tool could not send it — fleet-wide `incomplete_input` delegation wall. Parity test locks plan-gate fields ⊆ tool params (`tests/unit/mcp/test_flow_server_delegate_surface_parity.py`). 2. **Assembly-integrity guard accepts squash-merged children** (`011158db`) — `roboco/services/git.py` (~L4035): when `git cherry` reports a child unmerged, a parent-branch commit carrying the child's `[taskid8]` prefix now proves it landed (squash = N patches → one commit, new patch-id). Markerless children stay flagged. 3. **Diff head prefers origin when local is behind** (`9d10217c`) — `roboco/services/git.py` `_resolve_head_ref` (~L4359): both refs exist + local strictly behind (`merge-base --is-ancestor`) → resolve `origin/`; local-ahead/diverged keep priority. Kills the stale-evidence false `pr_fail` on assembled PRs that advanced on origin. 4. **`GET /api/tasks/summary` + bounded list routes** (`1cf24ff1`, `c1acbd5b`, `298751e6`) — `roboco/api/routes/tasks.py` wires the previously-dead `TaskSummaryResponse` (`roboco/api/schemas/tasks.py`, now + `completed_at` / `board_review_complete`) into a trimmed route (default limit 500, cap 1000); eleven unbounded task list routes gain `limit` params; the `/tasks` status-only branch honors its limit. 5. **Panel request-flood/fat-payload fixes** (`569a6157`, `298751e6`) — `prefetch={false}` on all Links (sidebar, cards, queues), task list + CEO queue consume `/api/tasks/summary` (`panel/src/lib/api/tasks.ts` `TaskSummary`), logo/icon PNGs slimmed (446KB → 7KB), ReactQueryDevtools dev-only (`panel/src/components/providers.tsx`). 6. **Spawn manifest `workspace_path` follows the task's project** (`53bb0420`) — `roboco/runtime/orchestrator.py` `_build_manifest_for_agent(workspace_path=)` + new `_resolve_workspace_cwd`: manifest and container `-w` share one resolver (was hardcoded to the roboco-project workspace for every spawn). 7. **Respawn breaker catches status ping-pong** (`4de81d92`) — `roboco/foundation/policy/agent_loop.py` `pm_respawn_max_revisit_resets: int = 2` + orchestrator `_respawn_status_change_resets`: a never-seen status fully resets strikes; a REVISITED status gets a bounded reset budget (mirrors tracing_resets), after which strikes accrue. `seen_statuses` is in-memory only (rebuilds post-restart, can only under-gate). 8. **Unassigned-QA dispatch no longer pre-claims** (`a260f903`, `caecb816`) — orchestrator (~L10968): the transitioning pre-claim moved `awaiting_qa` → `claimed` before the agent existed, stranding `claim_review`/`pass_review` (both demand `awaiting_qa`). Now matches `_spawn_assigned_qa` / external-PR dispatch: spawn unclaimed, agent self-claims via `claim_review`. 9. **Team-match enforcement armed** (`fe9e5589`, `8e5f84c4`) — `roboco/foundation/policy/lifecycle.py`: `resume`/`unblock`/`activate` flip `needs_team_match=True`; new `_ORG_WIDE_ROLES` exemption (main_pm, CEO, PO, head_marketing, auditor, pr_reviewer) in `_check_team_match(…, role)`. `8e5f84c4` threads `agent_team` through all 27 gateway `Context` construction sites (`choreographer/_impl.py`, `doc.py`, `pr_gate.py`, `pr_review.py`, `qa.py`) so the gate actually receives the team — it had sat in its permissive fallback since shipping. Slices touched: worksession-git (2, 3), choreographer/gateway-support (9), orchestrator (6, 7, 8), foundation-lifecycle (7, 9), mcp-servers (1), api-routes-schemas (4), panel (5). `docs/map/_complete_map.md` is the pre-delta concatenation — not regenerated. --- ## Delta 2026-07-02 (evening) — leak-fix batch (branch `fix/leak-fix-batch`) The remaining live-run leak fixes from the S6/fb836f80 postmortems, TDD'd per item: 1. **Spawner attribution** — `roboco/runtime/orchestrator.py` `spawn_agent(spawned_by=)` → `_launch_spawn(spawned_by=)` stamps the dispatching loop's name into `agent.spawned` / `agent.spawn_failed` audit details (`"unspecified"` when absent). All ~29 call sites pass their loop name (orchestrator dispatchers by method name; `bootstrap`, `api.orchestrator.spawn`, `event.auditor_spawn`); `OrchestratorAccessProtocol.spawn_agent` (roboco/models/events.py) gains the kwarg. A whole-package AST sweep test (`tests/unit/runtime/test_spawn_attribution.py`) fails any future caller omitting it. `pyproject.toml` adds PLR0913 to the orchestrator's per-file ignores (spawn contract > 5 params, bundle refused). NOTE found in passing: `_safe_spawn` + `gateway_pre_spawn_check` (orchestrator ~L741/L1932, the trigger_filter dispatch-time cooldown) have NEVER had a caller — flagged to CEO, untouched under the freeze. 2. **Admin-complete merge-or-refuse** — `roboco/api/routes/tasks.py` `_apply_forced_status_override`: `status=completed` without `force` on a task whose work session records `pr_status == "open"` now refuses naming PR number/URL + the stranding consequence (checked before the generic hatch text). New `TaskService.open_pr_ref` (`roboco/services/task.py`, beside `_assert_pr_merged_for_complete`) is the lookup. 3. **Panel CEO-approve** — `panel/src/components/tasks/task-detail/task-header.tsx` `AWAITING_CEO_APPROVAL`: primary action now `ceo-approve` (CeoApproveDialog → `POST /ceo-approve`, notes ≥ 20); `approve-and-merge` only when `task.pr_number` (umbrellas 400'd `NO_PR`). 4. **Live-Redis test leakage killed** — `tests/unit/services/test_self_heal_originate_db.py` wrote `self_heal:notified:*` (2h TTL) and `tests/unit/gateway/test_i_am_blocked_rate_limited.py` wrote a NO-TTL `roboco:rate_limit:anthropic:state` "rate_limited" blob into live localhost Redis on every run (verified live, keys scrubbed). Both now patch `cfg.redis_host/redis_port` (redis_url is computed) to an unreachable port; engines' fail-open paths keep assertions intact. The one-off `test_self_heal_engine` full-run failure was NOT reproduced (adversarial orders, 5× dir loops, green full gate) — this leakage class + the 2026-07-02 corrupted-venv day remain the suspects. 5. **Phase1 smoke mock team** — `tests/integration/test_foundation_phase1_smoke.py` parent mock gains `team="backend"`; the armed team-match gate (delta above) rejected the auto-generated MagicMock team before the asserted `incomplete_input` (the sweep fixed 13 files; this integration file was outside the gateway/foundation/runtime subsets it ran). 6. **Gate green again: ten xenon C-ranks + global test-Redis isolation** — master CI red at the smoke test meant no gate (CI or local) had reached xenon since the team-match sweep; its inline `agent_team=…` kwarg had pushed 9 verb bodies to C(11-12) unseen (`_impl.py` i_am_done/resume/unclaim/submit_up/submit_root/complete/escalate_up/escalate_to_ceo, `qa.py` fail_review). New shared `actor_context_fields()` (`choreographer/_protocol.py`) computes `(actor_slug, agent_team)` once per verb; the admin-complete open-PR check is extracted to `_refuse_unforced_complete_with_open_pr` (routes/tasks.py). `tests/conftest.py` gains the autouse `_no_live_redis` fixture (root fix for the live-Redis leak class; notif_dedup purpose-dedupe keys were a third, self-expiring family). 7. **Dead spawn-cooldown path deleted (CEO-ratified)** — `_safe_spawn` + `gateway_pre_spawn_check` (orchestrator) + `roboco/services/gateway/trigger_filter.py` + both test files + the dead `spawn_cooldown_seconds`/`role_spawn_rate_per_minute` settings. Never called in repo history; superseded by provider parking (in `spawn_agent`), claim guards/reaper, the respawn circuit breaker, and the notification-spawn cooldown. Rule 4 (per-task cooldown) would have queue-stalled every stage handoff if wired. `GatewayTriggerTable` kept inert (drop = future migration decision). Slices touched: orchestrator (1, 7), api-routes-schemas + taskservice (2), panel (3), choreographer/gateway-support (6, 7), tests (4, 5, 6, 7). `docs/map/_complete_map.md` still not regenerated. --- ## Delta 2026-07-02 (late) — CEO-ratified follow-ups (branch `fix/leak-fix-batch`) 1. **Dead spawn-cooldown path deleted** (see item 7 above; commit `78fa0f6f`). 2. **uv serialization in the Makefile** (`cf5043fe`) — `export UV_NO_SYNC := 1` + `sync` prerequisite on quality/quality-fast/gate: gate recipes never implicitly re-sync the venv (the recurring rich/pip/bandit corruption came from two uv writers racing one `.venv`). 3. **`_api_base()` in git.py** (`13b9c5d4`) — 15 hardcoded `https://api.github.com` PR/merge/branch sites now honor `settings.github_api_base_url` like the CI/open-PR sites already did (GHE/test override fix; enables the smoke harness's fake GitHub). 4. **CI split** (`db6b3088`) — ci.yml (backend `quality` only; FILE name kept — self-heal/ci-watch/release default to it), `panel-ci.yml` (panel job, `panel/**` paths), `e2e-smoke.yml` (new job). Panel-only reds now land on Panel CI, unwatched by the ci.yml-pinned engines. 5. **e2e smoke harness** — `tests/e2e_smoke/{conftest,harness,test_dev_lifecycle}.py` + `make e2e-smoke` (env-gated `ROBOCO_E2E_SMOKE=1`, skipped in the default suite). Harness: real routers/middleware on uvicorn over the ephemeral test DB (`settings.database_*` patched + `_DbHolder` reset), bare origin at `/github.com/e2e-smoke/proj.git` (satisfies `_parse_git_url`, clones tokenless), fake GitHub REST router doing real squash-merges via an admin clone, `ScriptedAgent` reloading the real MCP modules per role. Scenario 1 (leaf dev arc → awaiting_pm_review) GREEN in ~5s. Learned seams scripted: post-claim tracing gap keeps the claim; note scopes are decision/learning/note/reflect/struggle (no 'progress'); i_am_done demands during-work+handoff+reflect+per-AC artifacts; pass_review demands learning note + ac_verdicts; A2A resolves roles from the STATIC agents_config registry (seed canonical slugs: be-dev-1/be-qa/be-doc/be-pm/main-pm). Slices touched: worksession-git (3), orchestrator (1), deployment-tooling (2, 4), tests (5). --- ## Delta 2026-07-02 (night) — wave 1 begins (branch `feat/wave-1`, post-v0.16.0) 1. **PR-gate turn cut** — `orchestrator._try_auto_submit` (+ `_AUTO_SUBMIT_VERB_BY_ROLE`), hooked in `_maybe_spawn_pm_closure` after the all-descendants-terminal check: POSTs the real `submit_up`/`submit_root` through the internal API as the owning PM (X-Agent-ID = task.assigned_to, fallback static `AGENT_UUIDS`); any refusal falls back to the classic PM closure spawn. Flag `ROBOCO_PR_GATE_AUTO_SUBMIT_ENABLED` (config, default on); audit `task.auto_submitted`. Tests: `tests/unit/runtime/test_pr_gate_auto_submit.py` (7) + e2e scenario 2b. 2. **e2e scenarios 2/2b** — `tests/e2e_smoke/arcs.py` (canonical-company seeding — A2A resolves roles from static agents_config so slugs must match; `seed_hierarchy` with task-short-id branch chains; dev/qa/doc/reviewer arcs; `dispatcher_assign` mirroring `_dispatch_pm_review_work`'s claim-for-PM lane since `pr_pass` clears ownership BY DESIGN) + `test_pm_merge_chain.py`. Seams scripted: commit-subject validator ≥20 chars; reviewer learning-note gate before pr_pass; child PR bases on the parent's branch via ancestor resolution. 3. **Trace timestamps** — `content_notes.apply_structured_note` stamps `written_at` into each stored section; panel `tab-notes.tsx` renders it (`FIELD_TO_SECTION` mirror map). 4. **Task search** — `TaskService.search_tasks` (ILIKE title/description + id-prefix) behind `GET /tasks/summary?q=`; panel debounces into the fetch, client title-filter removed. PLR0913 added to the routes per-file ignore (route signatures ARE the HTTP contract). 5. **Secretary task writes** — `control_task` action `edit` (allowlist title/description/acceptance_criteria/priority, `_EDITABLE_TASK_FIELDS`) + `GET /secretary/tasks?q=` name→id resolver (Secretary/CEO). PM-side expansion deliberately deferred (CEO: "PMs not that much"). Slices touched: orchestrator (1), tests (1, 2), taskservice + api-routes-schemas (3, 4, 5), panel (3, 4), secretary (5). --- ## Delta 2026-07-02 (late night) — e2e scenario 3 (branch `feat/wave-1`) `tests/e2e_smoke/test_root_ceo_chain.py`: 3a pr_fail→needs_revision→`i_will_plan` re-entry (route demands approach≥150 + sub_tasks even on re-claim — pydantic fires before the gateway short-circuit)→real fix commit (`origin_commit` helper)→resubmit→pass→merge; 3b submit_root→gate→complete-escalates→REAL `approve-and-merge` (tasks router now mounted in the harness app; CEO row seeded)→hello.txt on origin master. Seed corrections that ARE the documentation: delivery roots are team=main_pm + planning-typed (backend-team roots get closure-routed to the cell PM; code-typed roots hit the main_pm+code impossibility guard). Fake GitHub `get_pr` now recomputes head.sha live (the unchanged-PR gate reads it via the REST API, not local refs). Latent fix en route: dep-update probe env scrub (VIRTUAL_ENV). uv-rot root cause: shared ~/.cache/uv with long-lived uvx MCP servers — per-repo UV_CACHE_DIR pinned. --- ## Delta 2026-07-03 — wave 2 begins (branch `feat/wave-2`) 1. **Five dead comms panel components deleted** (−422 lines; audit-verified zero consumers; MessageComposer/MessageTypeBadge stay). 2. **e2e scenario 4 (MegaTask umbrella)** — `tests/e2e_smoke/test_megatask_umbrella.py` + arcs helpers (`wire_dependency` via the real sequencing edge, `seed_cell_and_dev`, `set_branch_name`). Proves: sequencing hold (`unmet_dependency` on RS2's i_will_plan), serial root merges to master, umbrella branchless close via ceo-approve (never approve-and-merge). 3. **PRODUCT FIX: batch root-subtask completion wall** — `_main_pm_complete_guard` (_impl.py ~6708) + `escalate_to_ceo` (task.py ~5321) refused ALL parented tasks; both now consult `is_batch_root_subtask`. Live root-subtasks previously closed only via CEO god-mode. Regression tests in test_choreographer_pm.py + test_task_service_transitions.py. First product bug found BY the harness (subagent-built, Sonnet 5, reviewed). --- ## Delta 2026-07-03 (2) — A2A live view (branch `feat/wave-2b`, SDD/Sonnet 5, reviewed) Backend: `EventType.A2A_MESSAGE_SENT` published from `A2AService.send` (excerpt-capped payload), `websocket_bridge` forwarder → `/ws/system` `a2a.message` frames; admin REST (`/a2a/chat/admin/conversations{,/{id}/messages}`, CEO-only) + CEO reply route via the publish-bearing `send` path. `agents_config.can_a2a_direct`: `from ceo` → allowed to anyone (the one asymmetry); `to ceo` stays blocked at creation. **Reply budget** (`_enforce_ceo_reply_budget` in `send_chat_message`): agent msgs ≥ CEO msgs in the conversation ⇒ reject; three-layer enforcement (matrix block → reply-channel resolve → budget). Panel: `/a2a` page + components, `use-a2a-live` hook over `useWebSocket("/system")`, composer gated on task-linked conversations. v1 seams flagged: legacy raw REST chat sends don't publish the live event; agent→CEO reply lookup matches topic-less conversations only; deduped re-send re-publishes an identical frame (idempotent consumer). --- ## Delta 2026-07-03 (3) — prompter memory (branch `feat/wave-2b`, SDD/Sonnet 5, reviewed) `TaskService.list_recent_for_project` (recency = coalesce(completed,updated,created)); pure digest builders in `prompter.py` (15 lines/project, 70-char titles, 4000-char total cap); orchestrator `_resolve_history_digest_ambient` (+`_resolve_intake_ambient` merge, best-effort/non-blocking) injected at `_spawn_intake_container` → `_generate_composed_prompt(ambient=…)`; `GET /prompter/live/{id}/search-tasks` (session-liveness = trust boundary, q 2–200, limit ≤10); `query_past_tasks`/`format_search_results` shared by the grok MCP tool AND the Claude SDK in-process tool (full parity, one implementation). FLAGGED pre-existing gap (untouched): `_resolve_conventions_ambient` doesn't cover the MegaTask `project_ids` scope — conventions ambient absent on MegaTask intakes. --- ## Delta 2026-07-03 (4) — switchboard + task access (branch `feat/wave-2c`, SDD/Sonnet 5, reviewed) 1. **Switchboard**: `A2A_ALLOWED_PAIRS` (agents_config, import-time; 70 pairs: 15×3 cells, 6 pm-chain, 3 board, 16 cross), `A2AService.list_admin_pairs` (bulk tuple_ IN join, latest conversation per pair), `GET /a2a/chat/admin/pairs` (CEO-gated); panel `a2a-switchboard{,-utils,-pair-card}` (pure lighting utils, rAF/CSS 45s fade, no timers), /a2a defaults to switchboard w/ v1-list toggle. FLAGGED pre-existing: `agent-utils.ts` static maps miss the per-cell pr-reviewer slugs. 2. **Task access**: Secretary `_EDITABLE_TASK_FIELDS` full content surface + enum coercion + claim-aware reassign (`reassign_active_claim` when claimed/in_progress); `read_task` full detail (progress bounded 50). PM lighter: `_pm_editor_scope`/`_enforce_pm_lighter_fields` in routes/tasks.py — closed the pre-existing PM-unrestricted-admin hole (cross-team cell PM hard-403; content allowlist; zero status via PATCH). submit_directive tool docs never mentioned `edit` (fixed — undiscoverable). --- ## Delta 2026-07-03 (5) — wave 3 → v0.17.0 (branch `feat/wave-3`, SDD/Sonnet 5, reviewed) Six subsystems, all default-off + additive: 1. **Sandboxed dev DB/Redis/Mongo** (`ROBOCO_SANDBOX_DB_ENABLED`, migration 057 `projects.sandbox_services`): `SandboxProvisioner` (`roboco/runtime/sandbox.py`) `docker run`s throwaway `postgres:16-alpine`/`redis:8-alpine`/`mongo:8` sibling containers per spawn (random creds, tmpfs, labeled), injecting `ROBOCO_TEST_DB_*`/`ROBOCO_TEST_REDIS_*`/`ROBOCO_TEST_MONGO_*` in place of the prod-creds gate-env. The service set is a pluggable engine registry (`roboco/models/sandbox.py`: `SandboxEngine` ABC + `_PostgresEngine`/`_RedisEngine`/`_MongoEngine`, `SANDBOX_ENGINES` / `VALID_SANDBOX_SERVICES` derived from it); adding an engine is one class + one registry line — no provisioner or env-emitter branch. Container-tracked lifetime + orphan janitor (grace-windowed). REVIEW FIX: pre-spawn stale-clear must not tear down the just-provisioned sandbox (`teardown_sandbox=False`) + provision pre-clears stale + janitor grace; `_ensure_image` inspects+pulls (300s) before `docker run` so a NAS cold pull isn't killed at the 20s run deadline. 2. **DB network isolation** (`ROBOCO_DB_NETWORK_ISOLATED`): second `roboco_data` compose bridge = postgres+redis only, orchestrator multi-homed; agents can't reach prod DB. Suppresses `_append_gate_env` prod-creds injection. In BOTH build+registry composes (topology-coupled). 3. **Mobile UI** (panel): `useIsMobile` (useSyncExternalStore, hydration-safe), `ResponsiveTable` table→card, snap `TabsList`, bottom tab bar, Comms/A2A single-pane drill-down below lg, vh→dvh. REVIEW FIX: `justify-center-safe` (overflow clip), memoized matchMedia subscribe. 4. **Cloud auth** (`ROBOCO_CLOUD_AUTH_ENABLED`, migration 058 `users`): FastAPI Users, single seeded user, cookie sliding 30-day session (pwd-fingerprint JWT), `get_agent_context` dual-path (deps.py). Off = byte-identical. `proxy.ts` (Next 16). REVIEW FIX: on-mode rejects EVERY non-CEO role without a token (not just ceo — closed the PM/board :8000 spoof). 5. **X engine** (`ROBOCO_X_ENGINE_ENABLED`, migration 059 Fernet `x_credentials` + `x_seen_mentions`): release-post + mention-reply drafts held for per-post CEO approval; local-model draft, hand-rolled OAuth1 signer, `XPostService.approve` sole `post_tweet` caller. REVIEW FIX: closed a double-post race (in-lock re-read + commit-before-release). 6. **Board roadmap engine** (`ROBOCO_ROADMAP_ENGINE_ENABLED`, marker-backed, no migration): weekly one-shot PO-solo explore → `propose_roadmap` verb (PO-only, 3-7 items) → CEO per-item approve→BACKLOG. `_dispatch_roadmap_exploration` bypasses the two-reviewer board path. REVIEW: builder found+fixed a SQLAlchemy dirty-check bug (deepcopy marker before mutate); I whitelisted `create_task_from_draft` source ({prompter, roadmap}). Compose: every optional feature armed `:-true` in the NAS composes, OFF in registry; two opt-in exceptions (`CLOUD_AUTH`, `ROUTING_STRICT`) default off (change auth/failure behavior, not capability). --- ## Delta 2026-07-04 — v0.18.0 Three workstreams plus a migration-chain catch-up (all reviewed; all default-off except the doc-only design bar): 1. **Fable mode** (`ROBOCO_FABLE_MODE_ENABLED`, default off) — opus-fable-playbook adoption on the existing model tiers (no new tier; Fable 5 is not a model option). Two levers: **doctrine** — `fable_doctrine_layer()` (`roboco/agents/factories/_base.py:203`) injects the vendored behavioral doctrine (`agents/prompts/doctrine/fable.md`) into `compose_prompt`'s layer tuple right after `base.md`. **Hooks** — `AgentOrchestrator._fable_hook_groups()` (`orchestrator.py:1414`) appends 5 vendored scripts (`docker/scripts/fable-{stop-gate,bash-discipline,honesty-nudge,prompt-nudge,precompact}-hook.sh`) AFTER RoboCo's own per-event hooks in the Claude settings.json; the grok path installs only the honesty-nudge (`write_grok_fable_hooks`/`fable_honesty_nudge_hook_config`, `roboco/llm/providers/grok_cli_config.py:308-345`) since a grok `PreToolUse`/`Stop` hook deny cancels the whole run (verified live) while `PostToolUse` never denies. Off = byte-for-byte unchanged spawn path. 2. **X feature-spotlight** (`ROBOCO_X_FEATURE_SPOTLIGHT_ENABLED`, default off, sub-switch of `x_engine_enabled`) — a third `XEngine` origination path alongside release posts and mention replies. `Orchestrator._x_feature_spotlight_loop` opens one held exploration task for the Head of Marketing every `x_feature_spotlight_interval_seconds` (default 3d); the HoM investigates and calls the new `propose_feature_spotlight` do-verb (HoM-only, `ContentActions.propose_feature_spotlight`, `content_actions.py:1238`, role gate `_FEATURE_SPOTLIGHT_ROLES`), which materializes a held draft (`XEngine.materialize_feature_spotlight`) into the same X Post Queue as release/reply drafts. Migration 061 adds `x_seen_features` (dedup ledger) + `company_goals.brand_voice` (feeds `_voice_guide` for all three draft kinds). **Gap found (static read, not live-verified):** `propose_feature_spotlight` has no wrapper function or `_TOOLS` entry in `roboco/mcp/do_server.py` (unlike `propose_roadmap`, which has both) — `_register_tools()` only registers the intersection of the manifest's granted verbs and `_TOOLS`, so a spawned Head-of-Marketing agent cannot see this tool via MCP today despite the role-config grant. 3. **FE/UX-UI design bar** (doc-only, no flag) — a `## Design bar` section added to `agents/prompts/teams/frontend.md` and `agents/prompts/teams/ux_ui.md` (distilled from `Leonxlnx/taste-skill`, MIT: three tuning dials — DESIGN_VARIANCE/MOTION_INTENSITY/VISUAL_DENSITY — plus typography/spacing/motion rules and an "AI tells to avoid" list), plus a pointer section in `agents/prompts/roles/developer.md` scoping it to frontend/ux_ui tasks only (backend unaffected). 4. **Migration-chain catch-up** — `060_drop_messaging` (the comms-teardown migration: drops `messages`/`session_tasks`/`sessions`/`groups`/`channels` + 4 enum types + `journal_entries.session_id`; A2A is now the sole directed-message channel) had already landed on master but was never appended to this doc. Chain head is now **061** (this doc previously said 059); `roboco/db/tables.py` now carries **38** ORM table classes (verified via direct count), up from the 37 this doc's Purpose section stated (that figure predates 055-061 and was never recomputed). Slices touched: orchestrator (1, 2), runtime-providers (1), prompts-roles-taxonomy (1, 3), deployment-tooling (1), gateway-support + mcp-servers (2), product-strategy-research-pitch (2, already current from an earlier pass), db-migrations (2, 4).