9 Commits

Include renames

Author	SHA1	Message	Date
3 people	15effce014	Chore: 141 Gaps fill-in (#283) ... * Updated uv.lock * Bunch of fixes we need to verify first.. * feat(megatask): per-cell project map root-subtasks (multi-project, multi-cell) A MegaTask root-subtask can now target an ad-hoc per-cell project map — a third targeting shape that mirrors the existing product fan-out root. In RoboCo a project is per-cell (ProjectTable.assigned_cell); a monorepo is N per-cell projects sharing one git_url. So 'multi-cell' IS 'multi-project', and a task may mix per-cell projects across products or include OSS-library projects not in any product. Storage: migration 052 adds task_cell_projects (mirrors product_projects; unique per (task, team)). TaskTable gains a cascade-delete cell_projects relationship; TaskCreateRequest / TaskCreate / Task response carry the map. Policy: batch.is_branchless_coordination + is_valid_batch_shape gain a has_cell_projects param — a root-subtask targets exactly one of project / product / cell-map; the umbrella still targets none. TaskService passes has_cell_projects at every predicate call site and persists the rows in create(). _ensure_branch_for_task cuts feature/main_pm/{root} per distinct project in the map (via _distinct_projects_for_task); _require_target_or_umbrella and _validate_batch_membership accept the map shape. Fan-out: every distinct_project_ids site (task.py branch creation, routes _project_for_complete + _resolve_project_for_merge, orchestrator _ambient_projects_for_task, pr_review._project_slug_for, git._project_for_task) generalizes to first-distinct-project-of-map-or-product. Choreographer _resolve_subtask_project resolves a delegated subtask's cell from the parent's cell map. The product-scoped _slugs_for_product intake helper is unchanged. Intake: prompter._draft_cell_map extracts the per-cell map from the_work[]. _validate_batch_scope counts distinct projects across all drafts' cells (>=2 min stays; one 2-cell draft satisfies it). create_task_from_draft persists cell_projects for >=2-cell drafts (project_id/product_id None), collapses a 1-cell map to the single-project shape, and leaves single-cell top-level project_id drafts unchanged. _resolve_owning_team routes a multi-cell map to Main PM (coordination root, like a product root — a cell PM can't delegate cross-cell). propose_draft/propose_batch tool descriptions declare the per-cell project_id (both Claude SDK + grok runtimes). The umbrella stays branchless / pure-coordination / submit_root-rejected; the CEO-escalation pr_number gate is not widened (the map root is is_umbrella=False, mirroring a product root, so submit_root supplies it). Single-cell root-subtasks and everything below them are byte-for-byte unchanged. Un-run MegaTask waves (multi-cell drafts) become runnable. * [feature] Panel per-cell project picker + pnpm format infra MegaTask root-subtasks can fan out across cells (be+fe, fe+uxui). Since a RoboCo project is per-cell (ProjectTable.assigned_cell), a monorepo is N per-cell projects sharing one git_url — so multi-cell IS multi-project. The batch-review card now shows one project Select per the_work entry, scoped to that cell's repos, instead of one Select bound to a single top-level project_id. confirmBatch validates each cell's project is in scope and the batch still spans >=2 distinct projects. - prompter.ts: CellWork gains optional project_id (the per-cell picker seam). - batch-review-card.tsx: per-cell Selects (one per the_work entry), scoped to the cell's projects; legacy single-cell drafts keep the one-Select path. - use-prompter.ts: updateBatchDraftProject edits per-cell (entryIndex); confirmBatch validates every cell; batchFromEvent parses per-cell map. Also adds the missing pnpm format infrastructure (the panel had no formatter at all): prettier devDep + .prettierrc.json (default-style config: 80-col, double-quote, semi, trailing-comma-all) + .prettierignore, plus format / format:check scripts. Only the 3 changed files above were reformatted; the ~222 pre-existing non-compliant files are left untouched (a wholesale reformat is a separate explicit decision, not bundled into this feature). * [fix] MegaTask verification: migration 052 enum + async cell-map read Two real bugs surfaced running the full gate against a containerized Postgres (and the orchestrator boot log): 1. Migration 052 crashed a real orchestrator boot with 'type "team" already exists'. The generic sa.Enum(create_type=False) does NOT set the postgres enum's create_type attribute, so op.create_table (checkfirst=False) emitted a redundant CREATE TYPE against the pre-existing team enum. Switched to postgresql.ENUM(create_type=False) — the postgres- native enum whose create_type _check_for_name_in_memos actually reads, so the CREATE TYPE is suppressed. Verified: 051->052 upgrade against a DB where the team enum pre-existed (the exact path that crashed) now succeeds; downgrade 052->051 drops the table and preserves the shared enum; fresh upgrade head clean. (Migration 016 has the same latent sa.Enum pattern but never re-runs in prod, so it's noted, not touched here.) 2. _ensure_branch_for_task read task.cell_projects (lazy=selectin to-many) directly, tripping MissingGreenlet on a freshly-created/unqueried task — which then poisoned the async session (PendingRollbackError). Replaced with _task_has_cell_map: peeks InstanceState.unloaded (no IO) and reads the already-loaded map, falling back to an awaited count query only when the relationship is genuinely unloaded. Non-ORM stubs route to the plain attribute. Fixes 2 integration tests; the 6 cell-map unit tests still pass. Also: typed the self stub as Any in test_choreographer_subtask_project (mypy tests/ wants Choreographer, not SimpleNamespace) — the codebase idiom. Gate: ruff format/check clean; mypy roboco/ + tests/ clean; full pytest 10371 passed / 388 skipped against containerized pgvector:pg16; vulture clean. Pre-existing xenon C-rank on reassign (from prior commit 19a474d3, not this feature) still blocks make quality — surfaced separately. * [refactor] Extract reassign board-advisory diversion helper (C→B complexity) `reassign` in roboco/services/task.py hit xenon absolute complexity 11 (a C-rank block), failing `make quality`'s --max-absolute B gate. The C-rank originated in 19a474d3 (pre-existing, not this feature branch's work). Extract the board/advisory → cell-task diversion into `_maybe_divert_board_advisory_reassign` (complexity 4, A). reassign drops to 9 (B); behavior is byte-for-byte preserved — the helper runs the same guard + pool diversion + log, returning the diverted task or None so the caller falls through to the normal handoff. Whole-repo xenon exits 0; the 159 reassign / board-guard tests pass. Unblocks `make quality` on feature/metrics-granularity. * [fix] migration 016: postgresql.ENUM(create_type=False) for reused team enum 016_add_products_and_task_product_id used `sa.Enum(..., create_type=False)` for the reused Postgres "team" enum — the same latent defect that crashed 052 on a real orchestrator boot. On the generic `sa.Enum` the `create_type` kwarg is silently dropped, so `_check_for_name_in_memos` never sees it and `op.create_table` (checkfirst=False) emits a redundant `CREATE TYPE team` that fails with "type 'team' already exists" against a DB where the enum pre-exists. Switch to the postgres-native `postgresql.ENUM(..., create_type=False)` — its `create_type` is a real attribute the guard reads, so the CREATE TYPE is suppressed (and DROP TYPE on downgrade too). The member list is inert under create_type=False (it never creates/alters the type), so it stays at 016's original six, reflecting the enum as it stood then, not the later-widened set. This never crashed in prod because 016 is never re-run (alembic_version is past it), but it's the same defect class. Verified on the real boot path: upgrade to 015 in process A (team enum created by 001), then `upgrade head` in a fresh process B — 016 applied clean, no DuplicateObjectError; downgrade 016->015 clean, shared team enum preserved. See project_migration_enum_create_type_gotcha. * [chore] panel: prettier reformat across the codebase Apply `pnpm format` (prettier 3.8.5, 80-col / double-quote / semi / trailing-comma-all) to the 223 pre-existing panel files that predated the prettier infra added in cb5365a4. Pure formatting — no semantic changes: multi-line arrays/objects collapsed where they fit, trailing newlines added (.prettierrc.json), import grouping unchanged. Verified: `pnpm format:check` clean, `pnpm lint` clean, `pnpm typecheck` clean, `pnpm test` 113/113 pass (7 files). * Bunch of runtime fixes for MegaTask and other issues * Fix different project same PR number collision problem Fix (two layers): 1. Root cause — pr_merge and rebase_pr_for_task now take a required project_id and scope the lookup where(pr_number == X AND project_id == Y). Required so no caller can forget — the bug class can't recur. All 4 call sites updated (choreographer cell_pm_complete, the rebase-retry, the superseded close_pull_request now passes project_id, and _verb_runner._do_pr_merge). 2. Crash guard — _finalize_cell_complete None-checks the complete() return and returns a clean invalid_state envelope (with a remediate hint) instead of dereffing None → 500 → respawn loop. * Fix: Make main_pm + task_type=code impossible * Fix Main PM needs revision can't re delegate * [chore] Bump local LLM glm-5→glm-5.2 + swap Ollama fleet defaults off minimax - llm_catalog: OLLAMA_DEFAULT_MODEL minimax-m3:cloud → kimi-k2.7-code:cloud; role defaults kimi-k2.6→kimi-k2.7-code, developer minimax→kimi, product_owner/ ceo kimi→glm-5.2, documenter glm→kimi; GLM 5.1→5.2 comment fix. - config + .env.example + docker-compose{.yml,.yaml,.registry.yml} + docs + memory_distiller + optimal_brain: glm-5:cloud → glm-5.2:cloud. - panel ai-routing-card: typed SelfHostedModel/boolean annotations; drop the stale "Minimax M3 default" string (default is now catalog-driven). - tests: glm-5:cloud → glm-5.2:cloud in pricing + rate-limit-retry fixtures. * [fix] submit_root: hard unchanged-PR gate stops the pr_fail re-submit loop The 2026-06-27 infinite pr_fail loop: a Main-PM root (PR #139) was pr_fail'd, routed to needs_revision, and re-submitted byte-identical → awaiting_pr_review → pr_fail again, forever. The prior hint/a2a steer was ignored by the weak coordinator model — hints don't stop a model that won't read them. A HARD gate refuses the re-submit when the assembled root PR's head SHA is unchanged since the last pr_fail (no new cell work → identical diff); a different SHA ⇒ the branch advanced ⇒ allow. Every ambiguous case fails open (no prior fail, no recorded SHA, no pr_number, unresolvable slug, git error, closed PR) — only the exact-unchanged case is hard-blocked. - content/models: PrReviewContent.head_sha (optional; JSON col → no migration). - git: get_pr_head_sha (GitHub pulls API; None on any failure → fail-open). - pr_gate: pr_fail captures head_sha into the verdict record; pr_pass does not. - _impl: submit_root runs _submit_root_unchanged_pr_guard after _submit_up_guard; _current_root_pr_head_sha resolves slug + current SHA (fail-open). - pr_review: extract module-level resolve_task_project_slug, shared by the mixin and the gate helper (_LegacyChoreographer reaches it via cast to the ChoreographerHelpers typed view — it doesn't inherit the helpers mixin). - tests: test_submit_root_unchanged_pr_guard (11 — refuse/allow/6 fail-open/3 capture-side, mypy-clean via cc:Any spy idiom, zero type:ignore) + test_pr_gate_notifies_pm capture-path stub. * [chore] mypy tests/: clear all 15 pre-existing type errors so make quality can go green The branch tip had 15 mypy tests/ errors in files this bundle did not author, which blocked CI's make quality mypy step (mypy roboco/ tests/) regardless of the bundle's own commits. Pre-existing is still existing — fix every one: - test_schemas_v1_flow.py (8): the StrList coercion tests intentionally pass SDK-nested list-of-strings input ([[['...']]], {'item':{'$text':'...'}}, int, dict). Annotate those literals as list[Any] locals so mypy accepts the coerce-able shape; the StrList BeforeValidator still flattens to list[str] at runtime. No type:ignore. - test_pr_gate_records_verdict.py (3): notes_structured is dict|None; narrow with 'assert t.notes_structured is not None' before indexing (the existing pattern at line 90). - test_pr_review_hand_format_guard.py (1 site, 2 errors): the _verb_runner() spy assertion — use the cc: Any = c alias idiom so assert_not_awaited resolves; drops the now-unused type:ignore[union-attr]. - test_pr_gate_notifies_pm.py (1): drop the unused type:ignore[method-assign] on the a2a.send reassignment. - test_content_models.py (1): narrow coerced with isinstance(coerced, PrReviewContent) before reading .issues (the base _Content lacks the field). Gates: rm -rf .mypy_cache && mypy roboco/ tests/ = Success (855 files); ruff check + format clean; 5 affected suites = 40 passed. * [fix] fail_qa routes needs_revision back to the dev, never the pool A dev task in needs_revision must go back to the developer, never the pool. The pool path let a cell PM re-claim the revision (PMs can claim needs_revision) — the live 2026-06-27 'needs revision on a dev task sent to the cell PM' bug. fail_qa's original_developer marker is the fast path, but it is unreliable in practice (live observation: never persisted), so the unassign else-branch was the load-bearing path and it dropped the task into the pool. Add a work-session fallback (_resolve_revision_dev) that resolves the developer who actually worked the task — the most recent work session whose agent is a developer, the QA's own session excluded — and reassigns to that dev instead of unassigning. Only unassign when no developer ever touched the task. Self-heals the marker so a subsequent re-fail takes the fast path and the QA-review index attributes the work correctly. * [feature] delegate carries dev-task collision surface (sequencing S1) The cell/main PM's delegate verb now carries the dev-task collision surface (intends_to_touch / adds_migration / touches_shared) and an explicit depends_on override through DelegateRequest -> DelegateInputs -> _create_subtask_from_inputs -> create_subtask, and create_subtask forwards sequence / dependency_ids / batch_id / surfaces into the prepared TaskCreateRequest instead of dropping them (the base create already persists them at task.py:878-884). This is the plumbing for the multi-level sequencing model edge kind 3 (dev-task collision DAG). Previously a dev task delegated with a collision surface or an explicit dependency lost it before persistence — dependency_ids was always [], so the only dev-task ordering was the weak assignee-keyed spawn barrier (the live 2026-06-27 out-of-order break: 40842957 started before 9b3682b8's PR merged). Phase S2 runs SequencingService over the surfaced siblings and wires the DAG via add_dependency. * [feature] wire dev-task collision DAG at cell-PM delegation (sequencing S2) Pure dev_task_collision_edges in sequencing.py turns a parent's surfaced siblings into (depends_on_id, task_id) pairs via SequencingService. TaskService. wire_sibling_collision_dag wires them through add_dependency (idempotent). The choreographer calls it after each dev-task delegate so the sibling collision DAG is built incrementally as the cell PM decomposes — file-overlap serializes, migration chains, shared-last; stable (priority, sequence) ordering keeps edges from flipping into reverse cycles on re-runs. * [feature] wire cell-task wave chain + by-osmosis edge (sequencing S3) Kind 2 (cell-task wave chain): a new cell-task under root-subtask UT_n depends on every cell-task under every root-subtask in UT_n.dependency_ids (the kind-1 wave-chain edges), so its branch carries the previous wave's merged cell work. Re-derived from the root-subtask's deps, not the cell-task's own dependency_ids (which also carry UX/product-fanout edges the by-osmosis edge must not pick up). A root may fan to several cell-tasks (different cells), so the previous wave's cell-task is a SET. Kind 4 (by-osmosis): the first dev task (sequence 0) under a cell-task depends on each predecessor cell-task's tail (max-sequence) dev task, so the new wave's first branch carries the previous wave's fully-merged tail. Subsequent dev tasks inherit the tail via kind 3 or the merged base. Both wired from _create_subtask_from_inputs, dispatched on parent.team (MAIN_PM -> kind 2; cell team -> kind 4). Pure helpers (cell_task_wave_chain_depends_on, by_osmosis_tail_dev_tasks) unit-tested in test_sequencing.py; TaskService methods integration-tested. Idempotent + best-effort throughout (add_dependency dedupes; missing predecessors are no-ops). Also fixes a latent mypy-tests gap (estimated_complexity required on direct TaskCreateRequest calls in the S2 tests). * [feature] sync_branch dev verb — gate-level branch rebase (Phase B1) Raw shell git is denied to agents (Bash(git:*) base deny), so a developer whose branch fell behind its base had no gate-level rebase — only the CEO/PM-only /rebase HTTP route. sync_branch is the dev verb that wraps the rebase through the gate (traced + evidenced), so the 'everything goes through the gates' invariant holds. - lifecycle: IntentSpec sync_branch (dev-only, ownership-gated, composes=(), git-only — no DB transition); _next_hint_synced helper. - GitService.sync_task_branch: rebase task.branch_name onto its resolved base via rebase_onto_base (fetch + rebase + force-with-lease push). - Choreographer.sync_branch + _sync_branch_preflight_rejection: not_found / unknown-role / spec-gate / no-branch / protected-base guards, then the git op; conflicts abort (no force-push) and steer to resolve-by-hand; git failure steers to i_am_blocked. - HTTP route /api/v1/flow/developer/sync_branch + SyncBranchRequest schema. - MCP tool sync_branch(task_id) + _TOOLS registration (manifest auto-propagates via intents_for_role(Role.DEVELOPER)). Tests: intent spec (5), choreographer handler (8: happy/conflicts/not_found/ not_authorized/no-branch/protected-base/git-failure/audit), route (1), MCP (1). ruff + mypy roboco/ tests/ clean; unit suite green (DB-fixture errors env-only). * [feature] i_am_done behind-base submit gate (Phase B2) A sibling's PR merging into the parent branch while a dev worked leaves the dev's branch behind its base — the assembled PR then can't merge cleanly and the sibling's changes go missing (the 2026-06-27 out-of-order dev-task break). The behind-base gate refuses i_am_done in that state and steers the dev to sync_branch (the Phase B1 gate-level rebase verb). - GitService.is_behind_base: rev-list --left-right --count across origin/{base}...origin/{head} → (behind, ahead); fetch-first so origin reflects the pushed head. Raises on git failure (consistent with rebase_onto_base); malformed stdout degrades to (0,0). - Choreographer._behind_base_gate: wired into _i_am_done_gate after _ensure_branch_pushed. behind>0 → invalid_state remediate→sync_branch. Fail-open on git/base-resolution error (flaky fetch can't strand a task at the submit gate — the merge layer has its own behind checks). Skipped for branchless roots and protected bases (master/main/-prefixed). Tests: gate (6: refuse+steer/up-to-date/branchless/protected/fail-open-base/ fail-open-git), is_behind_base (6: parse/up-to-date/malformed/argv-form/ requires-branch/missing-project). ruff + mypy roboco/ tests/ clean; unit green. * [docs] sync_branch prompt + behind-base guidance (Phase B3) Update every behind-base/rebase guidance surface to reflect the B1 sync_branch dev verb + B2 i_am_done behind-base gate: devs now self-rebase through the gate instead of escalating a plain behind-base condition; PMs still escalate cell/root integration branches (they have no rebase verb). - developer.md: sync_branch in the verb table; 'When your branch is behind its base' rewritten — call sync_branch, do NOT i_am_blocked a plain behind-base; conflicts → resolve by hand, commit, sync_branch again. - cell_pm.md: delegate signature gains intends_to_touch/adds_migration/ touches_shared/depends_on + a 'Collision surface' section (fill it on every code subtask so sibling dev tasks that touch the same files sequence into a conflict-free order — the 2026-06-27 out-of-order break fix); behind-base section steers devs to sync_branch, PMs escalate only the integration branch. - main_pm.md: behind-base section — dev leaf = dev's sync_branch; cell/root integration branch = escalate_up. - RAG git-errors.md / blocked-tools.md: devs sync_branch, PMs escalate. - docs/troubleshooting/common-issues.md: leaf self-rebases; integration branch still escalates to operator. - CLAUDE.md verb surface: developer gains sync_branch. - agents/prompts/_generated/*: regenerated via scripts/regenerate_verb_tables.py — adds sync_branch to the dev table AND catches the generated tables up to the S1/S2 delegate sequencing params + meltdown-fix note top-level params (the derived files had drifted stale vs the already-committed schemas). Docs/prompts only — no code. ruff + mypy roboco/ tests/ clean. * [chore] orchestrator: refuse to spawn human-only roles (CEO/prompter/secretary) A live 2026-06-27 incident saw a CEO agent container spawned. Root cause: _dispatch_a2a_work iterates every A2A/notification target and spawns it with no human-role filter, and _is_agent_active('ceo') is always false (the CEO is never a container), so the 'skip if active' check could never protect the CEO. Any CEO-addressed notification (board handoff, escalation) launched a CEO container — the system acting as the human CEO: a trust violation. The CEO is the human operator; intake (prompter) and secretary are human-driven chats launched through their own dedicated guarded paths (_spawn_intake_container / _spawn_secretary_container), never spawn_agent. Fix: a single chokepoint guard at the top of spawn_agent refuses Role.CEO / PROMPTER / SECRETARY (raises AgentReadinessError + logs). This structurally covers every dispatcher present and future, since they all go through spawn_agent. Plus a defense-in-depth skip in _dispatch_a2a_work so a human-role target never even calls in (avoids error-log spam; the notification stays for the human to read in the panel). Safe: the dedicated human-spawn paths do not route through spawn_agent. Regression tests: spawn_agent refuses ceo/intake-1/secretary-1, does NOT refuse a real agent; _dispatch_a2a_work skips CEO/intake/secretary targets and still spawns real-agent + mixed-target cases. * [chore] orchestrator: skip human-only assignees in claimed/pm-review dispatchers Defense-in-depth for the spawn_agent human-role chokepoint (d31d6719). The chokepoint structurally guarantees no CEO/prompter/secretary container can ever spawn — every dispatcher goes through spawn_agent. But two dispatchers resolve an arbitrary assigned_to and spawn it with only a None/unknown-role filter, so a human-assigned task would reach the chokepoint and RAISE: caught by the per-dispatcher try/except, but it aborts that dispatcher's whole tick (stalling other respawns behind the mis-assigned task) and error-logs every cycle. The other dispatchers are already safe by whitelist/hardcoded slug (blocker_resolver_slug returns None for non-PM/non-BOARD; escalation/approval use whitelists; marketing and audit hardcode their non-human slug). - _claimed_task_needs_agent: return None for a CEO/prompter/secretary assignee — no container to respawn, and do NOT release a human-owned task to pending (that would re-route it to a PM). Leave it for the human. - _dispatch_pm_review_work (assigned branch): skip a human-only assignee so a CEO-assigned awaiting_pm_review task neither spawns nor aborts the dispatcher's tick. Audited all target-iterating dispatchers; only these two lacked a filter. Regression tests cover both skips. * [F002] retype board-routed MegaTask root-subtasks code->planning on activation _activate_batch_root_subtasks flipped a held root-subtask to team=MAIN_PM but left task_type=code (intake only coerces main_pm-team drafts, so a board-routed code root reached activation still code-typed). The main_pm+code combo re-introduces the 2026-06-27 meltdown. Mirror approve_and_start's own retype via main_pm_cannot_own_code so the activated child is a planning-typed coordination root. TDD: RED test_activate_batch_root_subtasks_retypes_code_to_planning watched fail (task_type stayed CODE), then GREEN after the retype. ruff+mypy clean; 125 batch/umbrella/approve tests green, no regressions. * [F003,F004,F014] enforce HMAC agent-token gate on do routes + WebSocket streams F003/F014: /api/v1/do/* only required X-Agent-ID (UUID) — no token check, unlike the flow routers' role guards. A forged X-Agent-ID passed. Added require_any_authenticated_agent (token-only; do router serves all roles) and applied it as a router-level dependency. Binds X-Agent-ID to a verified HMAC token when ROBOCO_AGENT_AUTH_REQUIRED=true; rejects a forged token even in dev mode. F004: /ws/* per-agent streams (channels/agents/sessions/notifications) never read the nginx-injected X-Agent-Token, so in strict mode an agent on the Docker network could subscribe to another agent's notifications with no auth. Added _require_panel_token verifying the CEO panel token against the CEO identity; wired into all four per-agent streams (system stream stays operator-only per its docstring). Same strict/dev contract. TDD: RED tests watched fail (no gate -> 200/accept), then GREEN. ruff+mypy clean; 399 api/mcp + 29 WS tests green, no regressions. * [F005,F006] grok auth: directory mount + atomic-write fallback F005: the single-file bind mount of auth.json pinned the inode, so the orchestrator's atomic refresh (tmp+rename within ~/.grok) never reached a running grok container — a long-lived container hung at the login prompt when the original ~6h token expired. Mount the host ~/.grok DIRECTORY (ro) at /home/agent/.grok-auth-ro; the entrypoint symlinks ~/.grok/auth.json at that RO mount so grok + the --check backstop read the live credential (the directory mount sees the host-side rename) while grok's writable state (config.toml, sessions/) stays in the image's ~/.grok. F006: a rotated refresh_token is single-use — xAI invalidates the old one the instant it issues the new one. If the atomic write failed after the rotation, the file kept the now-dead old refresh_token and the credential was permanently lost on the next refresh. _atomic_write now falls back to a direct write when tmp+replace fails, so the rotated token always lands on disk (losing the write is catastrophic; losing atomicity is not). TDD: RED tests watched fail, then GREEN. ruff+mypy clean; 32 grok tests green, no regressions. * [F016,F017] choreographer: surface invalid_state instead of None.status 500 on submit_root / i_am_blocked Both verbs compose a single atomic action whose None return (the verb's own result) flowed out of run_intent and was dereferenced as t.status, HTTP 500-ing with no actionable rejection: - F016 submit_root: submit_for_review returns None when the root->master PR was already opened / the task raced out of in_progress. Post-runner None-guard extracted into _submit_root_finalize -> invalid_state (re-fetch; if awaiting_pr_review the PR is open, wait for reviewer; else re-delegate fixes and retry) instead of None.status. - F017 i_am_blocked: escalate returns None in four cases (no task, no agent, no resolvable escalation-target slug, no target agent row) e.g. a developer whose role has no PM above it. _run_i_am_blocked_intent now guards updated is None -> (t, invalid_state rejection) with remediation (re-fetch + escalate to CEO directly / retry) instead of the caller deref'ing None.status -> 500 + respawn-loop. TDD red->green; ruff + mypy clean; gateway suite green (58 passed). * [F007] choreographer: cell-level unchanged-PR re-submit loop-stopper for submit_up The root loop-stopper (F016) was root-only; a weak cell PM could re-submit the unchanged cell->root PR after a pr_fail and loop awaiting_pr_review -> pr_fail forever (the cell analogue of the 2026-06-27 root loop). pr_fail stamps the assembled PR's head SHA into notes_structured.pr_review .head_sha for cell AND root gate tasks alike (the capture is gate-verb- level, not root-level), so the same structural refusal applies to submit_up: if the cell PR's current head SHA equals the SHA the last pr_fail recorded, no new dev work landed on the cell branch -> the diff is byte-identical -> refuse, do not re-open the gate. Different SHA -> branch advanced -> allow. - _submit_up_unchanged_pr_guard mirrors _submit_root_unchanged_pr_guard (cell-PM remediation: re-delegate to the dev + wait for re-assembly), wired into submit_up after _submit_up_guard passes. - Renamed shared _current_root_pr_head_sha -> _current_pr_head_sha (both guards use it; the lookup was never root-specific). - Every ambiguous case FAILS OPEN (no prior fail, no recorded sha, no pr_number, no resolvable project, git/closed-PR None) — only the exact- unchanged case is hard-blocked. TDD red->green; ruff + mypy clean; F007+F016 guard suites green (15 passed). * [F008] evidence_builder: surface persisted pr_review verdict+issues in the PM task_handoff The pr_fail a2a steer to the owning PM is fire-and-forget; a PM respawned into needs_revision later read none of it (build_task_handoff never looked at notes_structured), saw a generic 'needs revision' with zero concrete change-requests, and re-submitted the same PR (the 2026-06-27 infinite pr_fail loop on 9980d0a0 / PR #138). The signal-gap was only partially closed by the a2a. build_task_handoff now extracts notes_structured.pr_review (verdict/summary/issues/head_sha — the slot pr_fail authors on every fail) into a pr_review field on the handoff, so every PM briefing for the task carries the concrete change-requests. A prior pr_fail alone now counts as prior-work-worth-resuming. Type-guarded + capped; absent => no key (no misleading empty slot). TDD red->green; ruff + mypy clean; evidence_builder suite green (14 passed). * [F009] notification: derive requires_ack from ACK_REQUIRED_BY_TYPE, not the True default NotificationService._create_notification built NotificationTable without requires_ack, so the column default (True) applied to EVERY notification - including informational REVIEW_REQUEST / DOCUMENTATION_REQUEST / A2A_REQUEST / KNOWLEDGE_SHARE (ACK_REQUIRED_BY_TYPE -> False) and every @mention from MessagingService._notify_mentions. Each false ack-required inflated the recipient's unacked set and soft-blocked i_am_idle into respawn churn. - _create_notification: requires_ack=ACK_REQUIRED_BY_TYPE.get(type, True) (unmapped types default True - preserve the action-required bias). - _notify_mentions: requires_ack=False explicit (MENTION is informational). TDD red->green (identity is False/is True assertions - the mocked flush doesn't apply SQLA's insert-time default, so pre-fix the attribute was None); ruff + mypy clean; notification suite green (18 passed). * [F010] notification: never dedup informational notifications (knowledge-share data loss) The purpose-based dedup suppressed a same-purpose (same sender/type/task, overlapping recipients) notification while a prior one was unacked. For informational types (KNOWLEDGE_SHARE / MENTION / A2A_REQUEST / BROADCAST + the pickup-proves-receipt triad) each send carries DISTINCT content (a new learning, a new mention) and acking is voluntary, so a recipient who never acks the prior one let the dedup permanently suppress every subsequent same-sender broadcast - silent learning-broadcast data loss. The dedup's anti-loop rationale (stop unacked-set inflation soft-blocking i_am_idle) only holds for action-required signals. Gate the dedup on ACK_REQUIRED_BY_TYPE.get(type, True): action-required types still dedup, informational types always create. Unmapped types default True (dedup on). TDD red->green; ruff + mypy clean; notification + dedup suites green (20). * [F011] playbook: de-index rejected/archived playbooks from the PLAYBOOKS RAG index * [F012] release_executor: fail-closed on git add/commit before push * [F013] release_proposal: Redis SET NX mutex guards the ~40min execute against concurrent approves * [F015] flow_qa/flow_doc: add i_am_blocked route (manifest-registered escape hatch was 404) * [F018] claim_guards: treat blocked as active + broaden the guard lookup so a blocked dev can't double-claim * [F019] git: clear orphaned .git/*.lock files after a timeout-SIGKILL'd mutation op * [F031] identity: role_for_slug_or_none so defensive skip-guards don't crash the dispatcher tick on stale slugs * [F032] test: unknown-assignee claim reaches release-to-pending path F031's role_for_slug_or_none fix made the unknown-assignee release branch in _dispatch_claimed_without_agent reachable (the human-only guard no longer raises/short-circuits on a stale slug). Lock that reachability in: a claimed task with an unknown-assignee UUID past grace returns the slug (not None) so get_agent_role -> 'unknown' releases the claim to pending for a role-matched reclaim. * [F033] orchestrator: capture container_id at startup re-adoption _readopt_running_agents registered re-adopted ACTIVE instances with container_id=None. _check_health skips container_id-is-None instances, so when a re-adopted container later exited the stopped-container handler never ran and the task stranded under a phantom ACTIVE instance forever. Add _resolve_container_id (docker inspect -f '{{.Id}}') and store the real id on re-adopt. Best-effort: a probe failure degrades to None (still ACTIVE; the reaper's Docker-liveness fallback covers it). * [F034] orchestrator: re-stamp respawn last_check at restore _pm_made_rule_following_retry bounds its tracing_gap audit lookup with since = record.get('last_check'). A stale persisted last_check from before the restart matched pre-restart tracing_gap rows, falsely resetting the breaker on the very first post-restart spawn — exactly when a fresh strike count should be evaluating current state. _partition_respawn_rows now re-stamps last_check to the restore time on every restorable entry, bounding the lookup to post-restart gaps only. * [F035] orchestrator: probe-resume loop actually revives parked agents _park_provider_unavailable parked the provider + offlined the instance but never registered a WaitingRecord, so _on_probe_success -> _parked_agents_for (always filtered on waiting_for=='rate_limit_lifted') returned [] and resolve_wait revived nobody — recovery fell to the 600s stale-claim reaper instead of the probe-success path the parking design relied on. Register + persist a rate_limit_lifted WaitingRecord at park time (mirrors mark_waiting_long, minus stop_agent — the container is already dead). Companion reaper guard: _reap_with_service now skips provider-parked assignees (_assignee_is_provider_parked) so the claim survives until the probe revives the agent — otherwise the reaper releases the claim to pending and probe-success respawns on a task the agent no longer owns. * [F036] orchestrator: read transcript for overload detection too The SDK server writes model-API errors (529/500/503) to /tmp/sdk-server.log, not stdout, so an overload marker can appear only in the durable Claude transcript — the same rationale already applied to the session-limit detector. _provider_overload_park_target read only docker logs, so an overload was missed and the agent crash-respawned straight back into it. Now concatenates the transcript tail before matching, mirroring the rate-limit path. * [F037] orchestrator: drop bare error-NNN overload markers The bare 'error 529'/'error 500'/'error 503' markers were broad enough to false-match an agent that merely writes about an HTTP status code in its own notes ('the endpoint returned error 500, retrying'), parking the whole Anthropic fleet on a non-issue. The SDK error formatter emits 'API Error: NNN' + a JSON error type, so the remaining 'api error: 529/500/503' + 'overloaded_error' + 'internal_server_error' markers cover every real overload without that false-match surface. * [F038/F039] orchestrator: sign X-Agent-Token on self-API calls The prior self-PATCH 401 fix only carried X-Agent-ID/X-Agent-Role. Arming ROBOCO_AGENT_AUTH_REQUIRED=true made the middleware require a signed X-Agent-Token, so every orchestrator self-call (auto-block / auto-resume / auto-recover / SLA annotation) 401'd and silently no-op'd — wedging paused/blocked parents. Add _system_api_headers() that wraps the base headers with a signed token for the system identity (issue_agent_token); switch all six self-call sites. Dev fallback: no secret set => UNSIGNED sentinel + auth not required. * [F040] orchestrator: finalize grok spawn session on cost-cap kill _enforce_grok_cost_budget killed + evicted the container without calling _finalize_spawn_session, so the open agent_spawn_sessions row stayed open (ended_at IS NULL) and the burned usage/cost was never recorded in the dashboard. Call _finalize_spawn_session(exit_reason='cost_cap') BEFORE popping the instance — it reads self._instances[agent_id] for the model + usage_session_id, which the pop would lose. * [F041] park grok exit-78 (auth missing/expired) instead of crash-retrying A one-shot grok container whose entrypoint ran grok_auth --check and found the token missing/expired exits 78 (EX_CONFIG). Crash-retrying 3x burns tokens for zero progress — the agent cannot start without a valid token. Park the provider with kind=auth_missing (same shape as the 429 exit-75 path) so the probe-resume loop revives the task once grok_auth.refresh_if_stale mints a fresh token; if still expired, the next exit 78 re-parks (no burn). Also fixes a latent F035 regression: _park_provider_unavailable now registers a WaitingRecord, so the bare-__new__ rate-limit park test had to set _waiting_records + stub _persist_waiting_record (mirrors the overload-test fixture). * [F042] isolate concurrent-duplicate conventions cache put in a savepoint Two task creates for the same project/HEAD can race to populate the conventions cache; the loser's INSERT fails the partial-unique index with IntegrityError. A bare session.add + flush poisons the shared session (the task-create transaction rides the same session), so every subsequent op raises 'this session is in error state' and task creation crashes. Run the INSERT in a savepoint (begin_nested) and swallow the IntegrityError: only the savepoint rolls back, the outer transaction stays usable, and the winner's row satisfies the next _cache_get. * [F043] guard escalate_up against resurrecting terminal tasks escalate_up had composes=() and no source-status guard, so a PM could escalate a COMPLETED/CANCELLED task and apply_escalation set it back to BLOCKED — bypassing the state machine's terminal-state invariant. Defense in depth: - spec: add PRECONDITION_NON_TERMINAL to escalate_up's extra_preconditions so the lifecycle gate rejects terminal tasks (invalid_state) before the journal:decision write fires; generalize _check_intent_preconditions to honor non-tracing rejection_kind (not_authorized / invalid_state). - service: apply_escalation (the single write primitive) returns False and refuses to mutate a terminal task — covers the HTTP escalate route which bypasses the spec gate. escalate() / escalate_up_to_role() return None on refusal so the gateway emits a clean invalid_state envelope. - route: the HTTP escalate route 409s a terminal task BEFORE sending the escalation notification (so a finished task isn't yanked back, PM not pinged). * [F044] pr_pass gate remediation points the reviewer at pr_fail, not i_am_blocked The pr_pass gate runs the toolchain + conventions guards on the REVIEWER's workspace, but their remediation text said 'call i_am_blocked' — a verb the PR reviewer does not have. The reviewer would chase a verb they cannot call instead of rejecting the PR. Make the guards reviewer-aware: a reviewer=True flag (passed by _pr_pass_blocked) switches the remediation to pr_fail(issues=[...]) — the reviewer's reject lever, sending the PR back to needs_revision for the dev to fix the environment / validator. The dev (i_am_done) path keeps i_am_blocked, which a dev does have. _conventions_guard (the pr_pass path) now passes reviewer=True through to _conventions_rejection. * [F045] rate-limit: loud activate-failure log + in-memory orphan-probe fallback The in-verb i_am_blocked(rate_limited) path wrapped RateLimitStateTracker.activate in a bare contextlib.suppress. A silent activate failure stranded the fleet: agents were parked in _waiting_records but the provider never entered the tracker, so the tracker-driven _sweep_rate_limit_probes never probed it and no _on_probe_success ever resumed them — parked agents stuck in WAITING_LONG. Fix: (1) replace the bare suppress with a try/except that logs an error event naming the provider + affected agents; (2) in _sweep_rate_limit_probes, after probing the tracker-listed set, scan _waiting_records for any rate_limit_lifted provider the loop did NOT cover and probe it via the time-expiry fallback (empty state -> probe now) so _on_probe_success resumes the parked agents. The fallback reads only local memory, so it still resumes when Redis was down at park time (list_rate_limited_providers failure now falls through to the orphan scan instead of returning early). * [F046] pr_gate: guard None runner result on concurrent transition (pr_pass/pr_fail) _gate_decision dereferenced the verb-runner result without a None guard. run_intent returns None when a concurrent transition (cancel or a racing reviewer) moves the task out of awaiting_pr_review between the precondition gate and the runner's final composed action (the verb runner's documented last-action source-status contract). The subsequent t.assigned_to / t.status / _post_gate_review_to_pr(t, ...) dereferences then crashed the gate with a 500 AttributeError. Add a None guard that surfaces a clean invalid_state rejection (re-fetch + re-issue) before any dereference; no PR post or a2a runs against a None task. TDD test_pr_gate_notifies_pm.py (+2). * [F047] conventions: reviewer-aware block-finding remediation on pr_pass gate The pr_pass (reviewer) conventions guard reused the dev-path block-finding remediation: 'add a waiver to .roboco/conventions.yml in your branch'. A pr_reviewer does not own the assembled cell->root / root->master branch and has no commit verb on it, so the waiver remediation is unreachable — a false positive stranded the gate with no self-recovery (the reviewer could neither commit a waiver nor pr_pass). The fail-open content path is documented precision-over-recall and stays as-is; the actionable gap is the remediation. Fix: _conventions_rejection now branches the block-finding remediation on reviewer=True (mirroring the could_not_run branch from F044). The reviewer path points at pr_fail carrying the findings as issues so the PR returns to needs_revision and the DEV fixes the violation or commits the waiver (the dev CAN commit to the branch); waiver authorship is framed as the dev's action, not the reviewer's. Dev i_am_done path wording unchanged. TDD test_conventions_gate_pr_pass.py (+1). * [F048] notify: reject human-only recipients (prompter/secretary) — no agent ack path notify() only checked the SENDER role. The recipient was resolved by NotificationService._resolve_recipients, which drops only unresolvable slugs — it does not exclude human-only roles. The prompter (intake-1) and secretary (secretary-1) are seeded agent rows, so they resolved, and an ack-required ALERT addressed to them sat permanently unacked (no agent auto-acks it), polluted the panel's pending-ack view, and — via the dedup query's ~acked_by.contains — permanently suppressed any later same-purpose notification from the same sender to that human role. The knowledge-share path already excludes all three human-only roles; the general notify path did not. Fix: a recipient-role guard in notify() via _reject_disallowed_recipient (folds the new check into the existing CEO-dependency-block return slot so notify stays under the PLR0911 return limit). Rejects prompter/secretary with not_authorized; the CEO is human too but acks via the panel, so it stays an allowed recipient (its only disallowed case, a dependency-block page, is preserved). TDD test_notify.py (+3: reject prompter, reject secretary, allow CEO). * [F049] merge_pull_request: idempotent on already-merged PR (mirror _merge_with_retry) * [F050] merge_pr_for_task: verify caller pr_number matches task's recorded PR * [F051] open_conventions_pr: refuse dirty tree + verify checkout-base landed * [F052] pr_target: scope task lookup by project_id (mirror close_pull_request) * [F053] _token_for_project: log decryption failure (key rotation) with project slug * [F054] learnings index: enforce shareable on every shared retrieval path (private-leak fix) * [F055] messaging: recover from concurrent channel auto-create race via savepoint + re-fetch * [F056] messaging: lock group row before session check-then-create to prevent active-session orphan race * [F057] playbook: index/unindex as a post-commit step so the RAG corpus never leads the status transaction * [F058] release-readiness: non-empty bump plan on first release _canonical_bump_files derived the bump set from the previous chore(release): commit. On the first release there is no such commit, so it returned [] -> assess set version_bump_plan=[] -> the executor published a tag with no files bumped (a no-op masquerading as X.Y.Z). Fall back to the version-reference scan when no prior release commit exists: the files currently embedding the version are exactly the set a first release must bump, and the set the first release commit then records as canonical for subsequent releases. Read-only derivation; the CEO-approval gate and fail-closed executor are untouched. * [F059] self-heal: hold fix tasks for CEO Approve-&-Start (restore dispatch gate) The module docstring promised self-heal fix tasks 'wait for the CEO's Approve-&-Start', but _originate created them confirmed_by_human=True and the orchestrator dispatched them at once — a self-heal fix that re-broke CI would trigger another cycle, open another auto-dispatched fix, and loop with no CEO gate on dispatch. Restore the documented gate: * _originate opens the task confirmed_by_human=False (held for the CEO). * The orchestrator holds a self-heal task out of both the PM and dev dispatch paths until confirmed_by_human flips True. * approve_and_start (the CEO's start gate) sets confirmed_by_human=True so the held task finally dispatches (idempotent for board/intake tasks already True). * list_pending_for_agent scopes the give_me_work hold to self-heal (source != self_heal OR confirmed_by_human) so an already-alive PM can't grab it pre-approval — while ordinary delegated subtasks (confirmed_by_human=False by default, where the delegation IS the authorization) still dispatch. The 'never self-deploys' guarantee (no merge) is unchanged. * [F059] fix DB-integration test auth + retype self-heal root code→planning conftest test-DB defaults matched the project's own running postgres (roboco/roboco @ localhost:15432, the docker-compose roboco-postgres service with CREATEDB) instead of the OS user on localhost:5432 which has no such role — every db_session test failed with InvalidPasswordError instead of running. Once the DB connection worked, the self-heal origination DB test went RED with MAIN_PM_NO_CODE: the self-heal root was task_type=CODE owned by main_pm, the combo the main_pm_cannot_own_code guard rejects. The Main PM coordinates the fix (delegates the code work to a cell dev); it has no code verb. Retyped CODE→PLANNING and rewrote description/AC to coordination-level. * [F060] emit reversal audit row on claim-branch-failure rollback The forward task.claimed audit row is flushed before the branch-creation attempt, and AuditService commits on its own connection, so the rollback's flush reverts the task row but not that audit row — the journey's last event stayed task.claimed while the task reverted to its pre-claim status, diverging from real state and corrupting downstream cycle-time/bottleneck metrics. The rollback now emits a CLAIMED->original reversal audit row (only when the forward transition was made) attributed to the claimant. * Removing completely unnecessary files (for the repo they are unnecessary) * [F061] audit status-transition rows now written in-session (F061/F073/F075) _emit_status_transition_audit now writes AuditLogTable rows into self.session synchronously (session.add) instead of dispatching AuditService.log_task_event fire-and-forget on its own connection. The audit row now commits/rolls back atomically with the status transition in the caller's transaction, closing three facets at once: - F061: audit commit no longer decoupled from the transition commit - F073: a committed transition can no longer have NO audit row (the row rides the same transaction; a swallowed persist can't drop it) - F075: a transition rolled back inside a verb savepoint no longer leaves a phantom audit row (the row is in the savepoint too) log_task_event is now called only from this helper (narrow blast radius verified); revision_count increment stays at this single chokepoint. Cycle-time/bottleneck reconstruction from task.<status> events is no longer silently corruptible. Tests: test_emit_status_transition_audit_writes_in_session_atomically, test_finalize_claim_rollback_emits_reversal_audit, escalation-audit tests retargeted to in-session AuditLogTable rows. Also: _canonical_bump_files grep-looseness follow-on (F058) -- filter by subject, not body; git log --grep matches any message line, so a non-release commit whose body references chore(release): shadowed the real release commit. Test test_canonical_bump_files_ignores_body_only_chore_release_match. * [F061] drop type:ignore from audit-emit tests Convention: no type:ignore/noqa. The F061 in-session audit-emit tests used '# type: ignore[assignment]' to assign a MagicMock to AsyncSession.add, and the F060 test assigned to .flush the same way. Rewritten to hold a local 'session: MagicMock' variable (mypy sees its auto-children as MagicMock, so .add.side_effect / .flush assign cleanly with no suppression). Verified via 'mypy tests/' that both files are now type-clean (the F060/F061 commits had skipped tests/ in mypy, masking two method-assign errors). * [chore] clear all 64 pre-existing mypy errors in tests/ (no type:ignore) Convention: no type:ignore/noqa, and pre-existing violations still violate. The make-quality gate runs 'mypy roboco/ tests/', but the prior commits' gates only ran mypy on production files, masking 64 type errors across 15 test files (method-assign, unused-ignore, no-untyped-def, attr-defined, union-attr, has-type, index, misc). Fixed without any type:ignore: - method-assign (svc.session.X = / svc.method = AsyncMock()): hold a local 'session: MagicMock'/'AsyncMock' and assert on it, or stub via object.__setattr__ / monkeypatch / a typed '_bind' helper returning Any, or alias 'cc: Any = c' (the pattern the file already used). - unused 'type: ignore[assignment]' (real code was method-assign): removed; replaced with the no-suppression patterns above. - 'Callable[...] has no attribute assert_*': keep a typed local ref to the AsyncMock and assert on the local, not the method-typed attr. - no-untyped-def: annotate helper params (Any / pytest.MonkeyPatch). - attr-defined / index / union-attr: type the helper as Any, narrow with an 'is not None' assert, or add the missing attr to a fake. - has-type / return-value: fix the declared return type to the tuple the function actually returns. - PLC0415 inline imports: hoisted to top-level. test_pr_gate_notifies_pm._stub_gate_path converted fully to the 'cc: Any = c' alias (it already used it for one attr) so its five '# type: ignore[method-assign]' suppressions are gone. mypy tests/: 64 errors -> 0 (538 files). ruff check tests/: clean. All 84 tests in the touched files pass. * [chore] remove all remaining type:ignore suppressions from tests/ Converts 115 `# type: ignore[...]` suppressions across 23 test files to no-suppression patterns (helper-return widening to Any, local Any aliases, cc:Any aliases, cast at narrow call sites, typed fixtures) so the hard no-type:ignore convention holds across tests/. No test logic or assertions changed — only mock-wiring mechanics and type annotations. Gate: ruff check tests/ clean; mypy tests/ (538 files) clean; 176 changed-file tests pass. Zero real suppressions remain (the 7 grep hits are 3 hygiene- checker string-literal test inputs and 4 prose mentions in comments). * [F062] work_session.merge_pr: idempotency + active-status guard merge_pr unconditionally set pr_status=merged, pr_merged_at, merged_by, status=COMPLETED on whatever session it loaded — the only session-terminal transition in WorkSessionService lacking both the active-status guard (complete/abandon) and the terminal-idempotency guard (close). Two failure modes: (1) a retried merge after a successful-but-unconfirmed GitHub merge overwrote merged_by/pr_merged_at with the retry's actor/timestamp, corrupting the merge audit trail; (2) merge_pr on an ABANDONED session resurrected it to COMPLETED, undoing the single-active abandonment. Mirrors close()'s guard: if status != ACTIVE, return the session unchanged. Both git.py callers await merge_pr and discard the return, so the no-op is safe. TDD: 3 tests (happy-path + both modes). * [F063] workspace._clone_repo: rmtree half-configured clone on failure If _configure_git raised CalledProcessError before its `remote set-url` scrub, .git/config kept the tokenized auth URL (the project PAT) and _assert_no_pat_leak never ran. The except clauses raised WorkspaceError without removing the workspace, so the next ensure_workspace's health short-circuit (valid .git with HEAD + objects) skipped past the leak — mounting the agent on a workspace whose .git/config let it read+exfiltrate the PAT. Both clone-failure except clauses now rmtree the workspace before raising, so a half-configured clone is destroyed and ensure_workspace re-clones from scratch. TDD: 2 tests (configure-failure leak + timeout). * [F067] flow_main_pm: add missing /triage route main_pm's manifest advertises triage (lifecycle.intents_for_role(MAIN_PM) includes it via _PM_ROLES, alongside triage_all) but flow_main_pm.py had no POST /triage route, so a main_pm agent calling triage hit a raw 404 that bypassed the per-verb circuit breaker. Added the route mirroring flow_cell_pm's /triage — wires to the existing team-scoped choreographer.triage (uses pm.team, works for any PM role; Main PM gets its own team's blocked/awaiting tasks). Fix direction: add-route, NOT remove-from-manifest — the manifest is spec-correct (intents_for_role by construction); removing triage would contradict the spec and leave main_pm with only cross-team triage_all. TDD: test_triage_route_exists_and_dispatches. * [F068][F069] mcp servers: classify all rejection shapes + envelope 404s F068: the do/flow-server circuit breaker only counted rejections whose `error` field was a STRING in _CIRCUIT_REJECTION_KINDS. A 422 validation failure (no `error` field, a `detail` list) and a 500/HTTPException (dict-shaped `error` from the exception handlers) both bypassed the breaker → unbounded retries on a storm of either. Added _classify_rejection(payload) (shared, applied to both servers) mapping all three shapes to a counted kind: string error (existing), dict error → substring-mapped code (*DENIED*/*AUTHORIZED*/*FORBIDDEN*/*PERMISSION*→not_authorized, INVALID_INPUT/*VALIDATION*→incomplete_input, *NOT_FOUND*→None parity, else →invalid_state), 422 detail→incomplete_input. The dict TypeError defence lives in the classifier (isinstance, never dict-in-frozenset). F069: a manifest-registered verb whose HTTP route is missing got FastAPI's raw `{"detail":"Not Found"}` 404 body — a non-envelope payload the breaker couldn't classify, so a storm bypassed it. _post now synthesizes an invalid_state Envelope rejection (with a remediate hint → i_am_blocked/i_am_idle) for a 404 status, routed through _record_and_check_circuit so the breaker counts it. A 404 that carries a real Envelope (error field present) is surfaced as-is, preserving test_flow_post_returns_envelope_on_404. TDD: 422/dict/404 tests in both server test files; updated test_dict_shaped_error_does_not_crash to assert the SDK is now called with not_authorized (replacing the pass-through assertion that encoded the bug). * [F064][F065][F066] websocket: non-blocking fan-out, finally-disconnect, idle timeout F064: the bridge forwarder awaited every conn.send_text in a gather with no per-connection queue and no send timeout — one slow WS client back-pressured ALL event delivery to ALL clients (head-of-line blocking on the listen loop). Each connect_* now registers a _ClientConnection (bounded asyncio.Queue(256) + sender task); broadcasts enqueue via put_nowait (drop + structlog warn on QueueFull) and return immediately. The sender drains the queue with each send wrapped in wait_for(SEND_TIMEOUT=10s). Unregistered legacy sockets (set directly into a subscription set, bypassing connect_*) get a timeout-bounded fallback send task held in _pending_sends (ruff RUF006). disconnect cancels + drops the sender. F065: route handlers caught only WebSocketDisconnect with no finally — a non-clean exit (anyio closed-resource, CancelledError, transport error) propagated without manager.disconnect, leaking the dead socket into every subscription set forever. Added finally: manager.disconnect(websocket) to all 5 handlers (disconnect is idempotent). F066: no server-side heartbeat/idle timeout — a half-open socket from a dead container blocked receive_text forever and was never reaped. receive_text now wraps in wait_for(IDLE_TIMEOUT_SECONDS=90s); on TimeoutError, log + fall through to the F065 finally. Named module constants (no config.py precedent for WS tuning; callers/tests patch them). TDD: 22 new tests across 3 files (handler cleanup, idle timeout, send queue), non-flaky across repeats; 1 existing test adapted with a yield for the new async fan-out (assertion unchanged). ruff/mypy clean, 421 unit/api tests pass. No type:ignore/noqa. * [F022][F023][F024][F025][F026] api: scrub secrets from 422 log, gate a2a/dashboard/orchestrator routes, SSE session-per-query - middleware: redact known credential fields (git_token/api_key/token/...) from the 422 request-validation log line; response body unchanged - a2a: require_any_authenticated_agent on /message/send + /message/stream; subscribe_to_task opens a short-lived session per poll instead of holding one asyncpg connection for the full SSE lifetime (pool exhaustion) + auth - dashboard: gate auditor flag/report mutating routes to Auditor or CEO - orchestrator: router-level CEO gate on all control routes (spawn/stop/...) TDD; ruff/mypy clean; 449 unit/api tests green; no type:ignore/noqa. * [F030] conventions: typescript-scoped custom rules now apply to .tsx files The validator tags a .tsx file as language 'tsx' (the JSX grammar needs that tag, distinct from plain 'typescript'), but a custom rule scoped to 'typescript' — the language the scan reports for a React+TS repo — silently skipped every .tsx file. The two suffix maps were NOT unified: the 'tsx' tag is load-bearing (grammars.py picks the JSX grammar on it; hygiene.py keys on it), so unifying would make .tsx fail to parse. Fix is in check_custom: a one-directional dialect map _DIALECT_OF = {'tsx': 'typescript'} — a typescript-scoped rule fires on a .tsx file, but a tsx-scoped (JSX-only) rule still does not fire on plain .ts. TDD; ruff/mypy clean; 80 unit + 38 integration conventions tests green. * [F029] websocket: remove broken /api/permissions/check loopback from channel stream channel_stream called validate_channel_access, which HTTP-loopbacked to GET /api/permissions/check — a route that does not exist. Every call 404'd -> False -> the channel stream closed with WS_1008_POLICY_VIOLATION for EVERY client, so the real-time channel stream was dead. Removed the function, its call site, and the now-unused httpx + settings imports. Post-F004 the panel-token gate is the channel-stream authorization (the CEO panel is the sole WS client and may view every channel), so the broken loopback is removed rather than replaced with an in-process check the CEO always passes. The legitimate enforcement.validate_channel_access (slugs, in-process static ACL) is a different function and is untouched. F027 is resolved-by-F004 (no code change): all three per-agent streams gate on _require_panel_token first, so only the authorized CEO panel can connect — 'any viewer subscribes to any target' is closed. TDD; ruff/mypy clean; 530 unit/api+enforcement+RBAC tests green. * [F078] release_executor: deadline every subprocess (git/make/gh/clone) A hung git/make/gh/clone would block the CEO-gated release loop indefinitely. Wrap each proc.communicate() in asyncio.wait_for via a shared _await_proc helper; on expiry proc.kill() the child and return a non-zero rc (124) so every caller's fail-closed branch fires. Mirrors the quality-gate _run_one kill-on-timeout idiom. Deadlines are generous (30min gate / 10min clone / 5min push+gh) so a legitimate slow op is never wrongly aborted — floor-assertion tests pin the floors to guard exactly that logical regression. Green path returns the real rc unchanged. * [F072] reaper: deadline docker inspect/exec + harden _check_health sweep A hung Docker daemon (or a stuck container FS) froze the single asyncio event loop: the reaper runs inline before every dispatch tick and shares that loop with every background sweeper. Bound each docker subprocess with asyncio.wait_for; on expiry proc.kill() the child and either raise (inspect / resolve_container_id — callers apply their own fail-direction) or return None (the gateway probe — inconclusive, caller declines to act, matching its existing probe-failure contract). Deadlines generous (10s inspect / 30s exec) so a legitimate slow docker call is never wrongly aborted; floor-assertion tests pin the floors. Also harden _check_health's per-agent loop so one agent's hung inspect skips that agent, not the whole sweep — preserving the check-all-agents invariant the timeout-then-raise would otherwise break (without this, a hung daemon means no agent gets health-checked any tick). * [F076] say/dm: handler guard rejects all 4 no-comms roles, not just auditor The say()/dm() defence-in-depth guard only rejected auditor, but CLAUDE.md mandates the same no-agent-comms invariant for pr_reviewer (posts findings on the PR), prompter and secretary (human-only, note + evidence). For those three the manifest was the only gate, so a call bypassing the manifest (direct API POST, test harness, future routing change) would not be refused at the handler — admission depended on the agent's slug happening to be absent from the channel/a2a matrix. Extend the guard to a _NO_COMMS_ROLES frozenset (auditor + pr_reviewer + prompter + secretary), matching the explicit role-frozenset gates on commit/notify/pitch/playbook/open_session. Role-appropriate remediation per role. The claimed defence-in-depth now covers 4 of 4 silent roles, not 1 of 4. * [F070] drain fire-and-forget _bg_tasks on shutdown (bounded, data-preserving) Orchestrator.stop() cancelled only the named loop tasks + agents, then returned, abandoning in-flight _schedule_bg work. An in-flight _persist_respawn_record upsert dropped at shutdown meant the last few gate-mutation strikes never reached the DB; restore_respawn_tracker() on the next start repopulated a stale lower count and the dispatcher re-burned the full 4-spawn strike threshold against a still-wedged task — the exact re-burn the durable tracker exists to stop. Audit-log writes (load-bearing for cycle-time/rework metrics) were similarly dropped. Add _drain_bg_tasks(): bounded wait (5s default) lets short DB writes commit before exit (data preserved), then cancels any stuck task past the deadline so a hang can't wedge shutdown. return_exceptions=True so one failing bg task doesn't crash the drain. Wrap the stop_agent loop in try/except + logger.exception so one bad agent can't skip the drain (re-introducing the data-loss tail). Floor test pins the deadline >= 3s so a too-short change can't silently drop a legitimate slow write. * [F071] abort non-blocking intake/secretary spawn on mid-spawn shutdown The non-blocking spawn (start_intake_session / start_secretary_session) schedules _spawn_intake_container_guarded / _spawn_secretary_container_guarded via _schedule_bg. Those run docker run and only register in _instances at the END. If shutdown arrived between docker run and the registration line, the container was started but the orchestrator had no handle — stop() iterates only _instances, so the container was orphaned (leaked, manual docker rm). Worse, the F070 drain could let the spawn coroutine complete the registration AFTER stop() already iterated _instances, landing a live container into a shutting-down registry nothing tears down. Add a post-docker-run shutdown guard in _spawn_intake_container and _spawn_secretary_container: re-check self._running after _run_container_cmd returns; if the orchestrator began shutting down, remove the just-started container (by its deterministic name) and raise _SpawnAbortedDuringShutdown WITHOUT registering. The guarded wrappers catch that BEFORE except Exception and close the live relay silently (shutdown is not a user-facing failure, no error pushed to the SSE stream). The F070 stop() drain awaits the bg spawn coroutine, so the abort surfaces cleanly. TOCTOU-safe: between the _running check and the _instances assignment there is no await (config + instance construction are sync), so once the check passes, registration completes before the event loop can interleave stop(). The normal running path is unchanged (sanity tests pin it). * [F074] per-agent advisory lock closes claim TOCTOU _run_claim_guards read the agent's other tasks via unlocked SELECTs before claim() took its row lock, and claim()'s FOR UPDATE locked only the TARGET row — so two concurrent i_will_work_on by the SAME agent on TWO DIFFERENT pending tasks each locked their own row, each read an empty in_progress set, each passed already_active, each claimed+started → the agent ended with two in_progress tasks (the in-process asyncio Lock is lost on orchestrator-restart split-brain, so it wasn't a DB-level guarantee). Fix: TaskService.acquire_claim_lock takes a transaction-scoped pg_advisory_xact_lock keyed by hashtextextended(agent_id). The gate acquires it BEFORE the guard reads (for non-coordinator roles only) so the second concurrent claim's read sees the first's committed in_progress task and is rejected. Tx-scoped → auto-releases on commit/rollback, can't outlive the request. Coordinator exemption (the key logical-regression guard): cell_pm / main_pm do NOT take the lock — the PM coordinator concurrency feature lets a PM plan+delegate many roots in parallel, and a per-agent lock would serialize those claims and regress it. Matches the existing _COORDINATOR_ROLES already_active/paused guard exemption. A hash collision only causes benign false serialization, never a false negative. Tests: unit (dev acquires lock before guard read; coordinator does not) + real-PG integration (same-agent serializes, different-agent does not, releases on rollback). * [F021] handle SSE transport errors so the intake composer isn't stuck openStream registered listeners for the server-sent event kinds but not the EventSource's own transport-level error. The 'error' kind IS in LIVE_EVENT_KINDS, so a server-sent event:error (JSON MessageEvent) was handled — but a dropped connection / dead session fires a plain Event with NO data, which JSON.parse(undefined) swallowed in the try/catch, so the stream 'stayed open' (EventSource loop-reconnected a session that no longer existed) and isSending stayed true — the composer was permanently disabled. Fix: route the 'error' event by payload. A MessageEvent with string data is a server-sent error → handleEvent (unchanged). A no-data Event is a transport error → handleTransportError: clear streamingId/activity, set isSending false, add a 'connection lost' error message, keep a draft/batch preview up (so the human can still act on a proposed card) else land on 'chatting', and close the dead stream so EventSource stops loop-reconnecting. Tests: renderHook + a jsdom EventSource double that fires a transport error (plain Event, no data) vs a server-sent error (MessageEvent + JSON). RED: transport error left isSending true; GREEN: resets to false, surfaces the message, closes the stream. The server-sent-JSON path is unchanged. Full panel suite (129) green; eslint/typecheck/prettier clean. * [F081] Approve dialog: label notes required (>=20 chars), not optional The CEO Approve dialog's notes label fell into the default branch ('Notes (optional') for the approve action, but approve actually requires substantive notes >= 20 chars — enforced client-side (toast error on < 20) and server-side. So the CEO was told 'optional' and only learned the real requirement from a toast after hitting submit with empty notes. approve and start both require >= 20 chars; reject only requires a reason. Collapse the label to two branches: reject -> 'Reason for rejection (required)'; everything else (approve + start) -> 'Approval notes (required, >= 20 characters)'. The approve placeholder now also signals intent ('Why this is ready to ship...'). Tests: render the queue, click Approve, assert the notes label says 'required' + '20' and does NOT say 'optional'. RED: label read 'Notes (optional)'; GREEN: 'Approval notes (required, >= 20 characters)'. eslint/typecheck/prettier clean. * [F082] surface release-proposal query failures instead of silent hide The card collapsed any non-404 backend failure (500 / network drop) onto `!proposal` and returned null, so the CEO had no idea the release-proposal endpoint was unreachable. Distinguish the cases: isError + a Retry affordance vs the 404 null empty state that stays hidden. Mirrors PrReviewQueue. * [F083] clear stale usage snapshot when /ws/system leaves connected The hook synced wsState into the store but never dropped usageData when the stream dropped, so on reconnect wsState flipped to "connected" before any fresh USAGE_SNAPSHOT arrived and UsageOverviewPanel rendered the prior session's totals/cost as if they were live. Clear usageData whenever state leaves "connected" so the panel falls back to the polling summary until a new snapshot lands. Connected->connected is a no-op clear skip. * [F084] scope per-control disable to the in-flight mutation, not all FeatureFlagsCard disabled every switch while any one flag toggle was pending, and PlaybookReviewQueue disabled every row's Approve while any one approve was pending — so the operator couldn't act on an independent control during a slow round-trip. Gate the disable on the in-flight mutation's variables (matching key / id) so only the control being mutated locks; the others stay usable. The same-flag double-tap protection is preserved. * [F085] reject submitting both project_id and product_id validate() only checked 'at least one of project/product', so the dialog let both be submitted together. The server silently lets product_id win at routing and drops project_id, recording a misleading, never-used repo. Add a validator that refuses the ambiguous submit with a clear error. The at-least-one rule and the single-pick submit paths are unchanged. * [F020] kanban: confirm admin-override drags that skip lifecycle preconditions A drag on the operator kanban routes the status move through the admin status-override, which bypasses the in-band lifecycle validator entirely. That override is intentional (it's how an operator recovers a wedged task) but it also let a careless drag skip material preconditions silently — completing a task with no open PR, QA-bypassing, finishing docs on a task whose docs aren't complete. Leave the override intact but make the bypass explicit: compute the preconditions the dragged move would skip (open PR, docs complete, self-verified + commits + progress for submit-qa, visible non-terminal subtasks for coordination-root targets) and, when any are skipped, hold the move behind a confirmation dialog that lists exactly what's being skipped. Precision over recall — only warn on what the panel can verify from the task and its in-list children; never fabricate a 'satisfied' claim, and stay silent on benign transitions that gate on nothing we can check. The admin status-override capability is preserved (Confirm still fires it); this only surfaces the bypass instead of letting it happen silently. Does not touch the master-merge invariant — the board's updateTask is the operator override, not the Main-PM merge path. * [F086] prompter: restore parked cell content on project toggle off/on rebuildCellWork appended a blank {summary:'', items:[]} entry for a newly- selected cell, so toggling a cell's project OFF then back ON in the MegaTask review card discarded the agent-authored per-cell summary/items — the entry was dropped on toggle-off and re-added blank on toggle-on. Park each draft's last per-cell content in client-only BatchProposal state (parkedCellWork, keyed by draft index — never sent to the backend; confirm ships only title/drafts/project_ids/route, and it ride-alongs into the localStorage persist slice so the restore survives a reload mid-review). rebuildCellWork gains an optional priorByCell map: a re-added cell with no live entry restores its parked summary/items (with the new project_id) in- stead of blanking; a live entry still wins over a stale parked copy so an in-place edit is never regressed. parkCellWork is the pure merge seam (prevParked seeds, live work overwrites) the setBatchDraftProjects updater calls — kept pure so the updater stays a thin caller. Tests: rebuildCellWork restore/blank-fallback/live-wins + parkCellWork retain/overwrite/merge (6 new), 19 GREEN. eslint/typecheck/prettier clean. No wire-payload change, no regression to the fill/drop/one-repo-per-cell invariants. * Updated domain * [F087,F088] enforce panel token on live-chat bridges (Phase 5) Add a CEO-bound, header-token-only gate (require_panel_token) at the route level of the prompter_live + secretary_live bridges, which were the only panel-facing API surface that ran unauthenticated. It mirrors the WS _require_panel_token and _check_agent_auth_token contracts: in dev (ROBOCO_AGENT_AUTH_REQUIRED unset) a missing token is allowed; a presented-but-forged token is rejected even in dev; in prod nginx already injects the CEO-signed X-Agent-Token on /api/ for GET + POST, so the SSE stream (EventSource can't set headers) and the POSTs are now checked instead of anonymous. Applied to start/stream/status/messages/stop on both routers; preview_live_batch switched from CurrentAgentContext+noqa to the route-level gate (genuinely auth-only). confirm/confirm-batch/re-interview keep CurrentAgentContext (they use agent.identity). The container->relay /events callback is intentionally left ungated (internal Docker network, opaque session id) — gated by a test sentinel so Option B (spawn+SDK token wiring) is a deliberate future decision. No panel/nginx/spawn/SDK changes; master merge invariant untouched. 22 new TDD auth tests, 492 api tests green. * [F089] honest WorkSession agent_id nullability across the read path The work_sessions.agent_id column is nullable=True with ondelete=SET NULL — deleting an agent nulls the FK on every session it ever held. The ORM annotation lied (Mapped[UUID] non-optional), the converter papered over the lie (typing_cast to a non-optional UUID), and the response model rejected None outright (WorkSessionResponse.agent_id: UUID). A session whose agent had been deleted crashed the GET endpoint with a pydantic ValidationError instead of serializing agent_id: null. Make the read path honest end-to-end: - WorkSessionTable.agent_id: Mapped[UUID | None] (matches the column). - WorkSessionResponse.agent_id: UUID | None (serializes null, no crash). - session_to_response passes agent_id via typing_cast('UUID | None', ...) to bridge SQLAlchemy's UUID[Any] to stdlib uuid.UUID while preserving None-ness (the cast stays for the same mypy-plugin reason every other field uses one; it no longer narrows away None). WorkSessionCreate.agent_id stays UUID — at create time the claiming agent is always known. The unused WorkSession pydantic read model is left as-is (never materialized from a DB row). task.py:_needs_revision_dev already None-guards ws.agent_id via to_python_uuid (returns None -> skip). * [F090] drop auditor from write_roles on main-pm-board / board-private The auditor is a silent, read-only observer on every channel, but the channel catalog (roboco/foundation/policy/communications.py) listed it in write_roles for main-pm-board and board-private 'for parity' with the legacy CHANNEL_ACCESS table, while the actual silent-observer rule was enforced only at the say/dm guard (content_actions._NO_COMMS_ROLES) and PermissionService.can_write_channel's auditor short-circuit. That left the catalog-only enforcement path — the HTTP messaging route (messages.py send_message -> validate_channel_access) — authorizing an auditor write that both the say/dm guard and PermissionService would have blocked. A reader of the catalog also believed the auditor could post to those channels, which is false. Fix: remove Role.AUDITOR from write_roles on both channels (main-pm + board remain writers; ceo remains a writer on board-private). The auditor stays in read_roles, so its silent read is unchanged. silent_roles is left empty (matches the announcements precedent: auditor reads via read_roles, not the silent bucket) — the DB seed and silent_observers field are untouched. Logical-regression check: the auditor's read access on both channels is byte-for-byte preserved (still in read_roles, so validate_channel_access read returns True via the direct list); the legitimate writers (main-pm, product-owner, head-marketing, ceo) are untouched; CHANNEL_ACCESS is derived from the spec so the foundation/seed drift tests self-adjust; PermissionService.can_write_channel already short-circuited auditor to False everywhere, so no behavior change there; AUDITOR_SILENT_ACCESS is unchanged (auditor not added to silent_roles -> no DB silent_observers change -> no group-access behavior change); the say/dm _NO_COMMS_ROLES guard is unchanged. Tests: 3 new in test_channel_access.py — auditor write on main-pm-board/board-private now raises ChannelAccessDeniedError (RED before: returned True), auditor read still True, main-pm/ceo still write. * [F091] warn at spawn time when host grok auth.json is missing GrokCliProvider._append_grok_auth_mount silently skipped the mount when the host ~/.grok/auth.json was absent. The spawn still succeeded (docker run returned 0 — the container was created), so the operator had no spawn-time signal that the agent was doomed: the entrypoint's `python -m roboco.llm.providers.grok_auth --check` backstop then refused to start (exit 78) and the failure only surfaced later via the container's log markers. Fix: emit a spawn-time WARNING (module logger) naming the missing file and the remediation (`grok login` on the host, or set ROBOCO_HOST_GROK_DIR) when the mount is skipped. The spawn outcome is unchanged — the container still starts and the existing exit-78 -> park flow (F041) still catches it — but the operator now sees the missing credential immediately instead of diagnosing a later exit-78. Logical-regression check: the mount-present path is byte-for-byte unchanged (auth.json exists -> the -v bind is appended, no warning); the spawn still succeeds when auth is absent (no raise — the existing test_grok_spawn_omits_auth_mount_when_absent still passes: no mount, no crash); the exit-78 entrypoint backstop and the orchestrator's exit-78-park handling (F041) are untouched; a module-level logger adds no side effects. Tests: new test_grok_spawn_warns_when_auth_absent uses caplog to assert a WARNING mentioning auth.json + `grok login` is emitted on a missing-credential spawn (RED before: no warning; GREEN after). 102 grok tests green; ruff/mypy clean. * [F092] decode JWT exp when refresh omits expires_in xAI's refresh-token response sometimes omits expires_in. Without it the new access token kept the stale pre-refresh expires_at, so is_valid / --check forever rejected a fresh token — and the refresh loop re-rotated the single-use refresh token every tick, killing the credential (F006). The access token is a JWT whose exp is the authoritative expiry: decode it when expires_in is absent. Fallback to the documented ~6h TTL + a structlog warning when the JWT exp is unreadable, so a fresh token is treated as live instead of stale. * [F093] serialize concurrent live-chat spawns under a per-agent lock The intake and secretary agent ids are each a single fixed id, so two concurrent start_intake_session / start_secretary_session calls raced on the container name (docker run --name roboco-agent-<id>) and the _instances[<id>] write: both passed the reap-prior check before either registered, both ran docker run, and the last _instances write won, orphaning the other container + its relay. Add _intake_spawn_lock / _secretary_spawn_lock (asyncio.Lock) and wrap the _spawn_intake_container / _spawn_secretary_container bodies so the second start waits for the first to fully register before its own reap-prior check runs. Distinct from self._lock (which stop_agent takes) to avoid a reentrancy deadlock: the spawn body holds the spawn lock then calls stop_agent (acquires self._lock) — lock order is always spawn_lock -> self._lock, never the reverse. * [F094] add a persistent-probe-failure escape hatch to provider parking _on_probe_failure only incremented the failure counter and, at 10 failures, sent a one-shot CEO notification. It never cleared the tracker, never gave up, never fell back to time-expiry. _do_probe returns False for any non-2xx AND any httpx error, so a permanently unreachable probe endpoint (removed API key, network partition to the probe host, misconfigured base URL) kept the provider parked forever — every agent on it gated by _provider_spawn_parked, their tasks reaped to pending but the spawn gate queuing every spawn, sitting pending forever. The only recovery was the operator manually clearing the Redis key. Past _PROBE_GIVE_UP_THRESHOLD (30) persistent failures, fall back to the same time-expiry optimism the unprobeable-provider path uses (_do_probe returns True when there is no probe URL): clear the park and resume parked agents. If the provider is genuinely still down the real workload attempts re-park via the 429/5xx path, so this is bounded burn — strictly better than a silent forever-strand. Kept above the CEO-notify threshold (10) so the operator still gets the notification first. * [F095] orchestrator: parked-provider spawn short-circuits before expensive prepare spawn_agent ran the full _prepare_agent_spawn (writes blueprint/settings/ briefing/MCP files, ensures the image, registers a STARTING instance) every dispatcher tick only to bail at the after-prepare parked-provider check — wasting all that file I/O while the provider stayed parked and leaving a STARTING instance registered then downgraded to OFFLINE. Move the parked check before _prepare_agent_spawn: resolve the route cheaply via _resolve_agent_route (only provider_type is needed) and bail with a minimal unregistered OFFLINE instance. The existing-running check stays first (inside the lock) so a live agent is never replaced; a TOCTOU re-check guards the unlocked window before prepare; the after-prepare check is kept as a rare-race defense (a park landing during prepare). * [F096] orchestrator: serialize fire-and-forget respawn persists per commit order _persist_respawn_record is fire-and-forget per gate mutation; a respawn loop fires count 1->2->3->4 in quick succession, scheduling one persist per increment for the same (agent_slug, task_id). The ON CONFLICT DO UPDATE upsert is row-level race-free, but the fire-and-forget tasks can still COMMIT out of order: a slow stale persist (count=2) scheduled first can resolve AFTER a fast fresh one (count=4) scheduled second, leaving the durable row at the stale low count and re-burning the strike threshold on restart. Fix: acquire self._respawn_persist_lock (new asyncio.Lock) as the FIRST await in _persist_respawn_record, so acquisition order = task creation order (FIFO ready queue) = logical schedule order, and commits land in that order. The durable row always ends at the latest logical value. The lock lives in the bg task, so the dispatcher hot path never blocks; persists are best-effort and a slow one queuing the rest just delays the durable catch-up (in-memory record stays authoritative). * [F097] orchestrator: back off grok re-park retry_after within a rate-limit episode _probe_target returns (None, {}) for grok — the grok CLI's xAI endpoint is closed and the SuperGrok OIDC access token is not a valid bearer for the metered api.x.ai, so a real probe would either no-op or strand grok parked forever. _do_probe treats url-is-None as success (time-expiry optimism), so once the 60s retry_after passes the probe loop optimistically clears the grok park, a cleared park dispatches a fresh grok agent that hits the still-active xAI 429, exits 75, and re-parks — a flat ~90s crash-retry cycle for the whole xAI rate-limit window (each cycle costs container startup + a rejected grok call). Fix: track _grok_repark_count + _grok_last_park_at in _park_grok_rate_limited and back the re-park retry_after off exponentially within one episode (60 -> 120 -> 240 -> ... capped at 2**4 = ~16min cycle) so the churn dampens. A gap past _GROK_REPARK_EPISODE_GAP_S (25min, > the capped cycle) means no re-park for that long => the rate limit actually lifted => a fresh episode resets the count to the base 60s, so recovery latency isn't penalized across episodes. The first park in a fresh episode is unchanged at 60s. * [F098] orchestrator: keep waiting record through a re-park during probe-success resume resolve_wait deleted the waiting record (in-memory + durable) BEFORE calling spawn_agent. A re-park in the window between the probe-success clear and the spawn — the provider's rate limit lifts then immediately re-limits, or a second provider limit lands — bails spawn with an OFFLINE instance (the parked-provider short-circuit). Deleting the record first orphaned the agent: with no record the probe-resume loop can never revive it and the spawn gate bails every tick, so the agent is lost until the operator intervenes. Fix: spawn first, then tear down the record only once a container actually launched (instance.state == ACTIVE). On an OFFLINE bail the record stays so the next probe-success re-attempts the resume. On a spawn EXCEPTION the record is torn down + re-raised so the probe loop doesn't keep re-resuming a task that moved to a different state (e.g. readiness refused -> task auto-blocked) — matching the pre-fix behavior where the record was deleted before the spawn. * [F099] wire pr_pass/pr_fail self_review block in the spec gate The pr_pass/pr_fail ActionSpecs carry self_review_block=True, but _gate_preflight never populated Context.original_developer_slug, and actor_slug was read off agent.slug — which GatewayAgentView does not carry, so it was always None in production. The block was structurally dormant: a reviewer who was also the original developer of the assembled PR could pass (or fail) their own work. The service-layer _validate_not_self_review backstop only covers qa/documenter, not pr_reviewer, so the spec gate is the only defense. Set actor_slug=str(reviewer_agent_id) (GatewayAgentView has no slug, so the UUID is the identity) and original_developer_slug from the original_developer marker (a UUID stored as a string). Both resolve to UUID strings, so the spec's string-equality comparison fires when the reviewer IS the recorded original developer. The marker is never set on assembled coordination tasks (only on dev-leaf tasks at QA/doc claim), so the block stays dormant by design in production — but the gate is now correctly wired to fire if the marker were ever set to the reviewer. Zero production behavior change; the dormant-in-production state is pinned by the no-marker test. * [F100] atomic Redis probe-failure counter via server-side Lua increment_probe_failures / reset_probe_failures did a non-atomic get_state (GET) -> mutate -> set (SET) in Python. A concurrent activate() re-park writes a FRESH episode blob (probe_failures: 0 + fresh activated_at / retry_after / affected_agents / kind); if the stale increment's SET landed after the fresh activate's SET, the stale blob overwrote the fresh episode metadata AND un-reset the counter (clobbering the new episode). Redis single-threads a Lua EVAL, so a server-side read-modify-write is indivisible: activate's SET is serialized entirely before or after the script, never interleaved between the script's GET and SET. The two scripts mutate ONLY probe_failures, so every other episode field survives the bump. activate stays a single atomic SET (a fresh episode resetting the counter to 0 is correct semantics). * [F101] enforce PR-open state gate on gateway open_pr (parity with HTTP path) * [F102] make project_id mandatory on pr_target (close cross-repo pr_number collision) * [F103] make project_id mandatory on close_pull_request (close cross-repo collision) * [F104] fail-closed on conventions resolution errors (block gate no longer silently disabled) * [F106] compound (timestamp, id) keyset cursor for message pagination get_messages used strict timestamp inequalities with a non-deterministic order_by(timestamp.desc()), so equal-timestamp messages were cut by limit on one page and excluded (strict < T / > T) from the next — they vanished across pages. Bundled the (timestamp, id) pair into a MessageCursor dataclass so the next page resumes exactly past the cursor's id at the shared timestamp (or_: strictly-older OR same-timestamp-smaller-id for before; the mirror for after), with a deterministic order_by(timestamp.desc(), id.desc()) so the last-item cursor is unambiguous. id is None for a legacy timestamp- only cursor (strict inequality, prior behavior). The route builds cursors from the flat before/before_id + after/after_id HTTP params; the schema now carries the tie-breaker ids. Also clears PLR0913 (cursors replace the before_id/after_id params). * [F107] defer Redis bus publish until DB commit (no phantom notifications) deliver() and _persist_and_deliver() ran inside the caller's open transaction: the notification row was flushed but not committed, yet NOTIFICATION_SENT was published to the Redis bus immediately. A commit failure (DB hiccup, constraint, asyncpg error) rolled the row back while connected WebSocket clients had already received a push for an id that no longer existed — a phantom notification (notify_get -> NotFoundError). Added a deferred-publish (transactional-outbox) helper: defer_bus_publish enqueues the event on session.info and registers one-shot after_commit / after_rollback listeners on session.sync_session the first time it is called for that session. On commit, the after_commit listener schedules the async drain via asyncio.create_task on the running loop (the listener fires synchronously inside await AsyncSession.commit, so the loop is active); the task handles are stashed on the session so callers/tests can await them. On rollback, after_rollback drops the pending queue — a rolled-back txn emits nothing. deliver() now builds the per-recipient events up front (data materialized to strings, so deferral is safe even if the ORM object later expires) and defers each; the delivered_at DB marker stays in-tx (rolls back with the row). The bus block stays best-effort (try/except + log) so a bus-init failure never propagates or rolls back the notification row — matching the prior inline semantics. This fixes every deliver/_persist_and_deliver caller at once (the two cited in F107 plus the orchestrator + task.py deliver sites), since they all commit the session afterward (the deferred publish fires on that commit; the row is durable by the time the event goes out). * [F108] atomic replace_chunks: single-txn delete+insert closes reindex race * [F109] playbook curation status guards: approve/reject draft-only, archive approved-only * [F110] draft slug TOCTOU: catch IntegrityError on flush -> ConflictError (no 500) * [F113] collapse WorkSession creation to the validated service path _create_work_session_if_needed constructed WorkSessionTable directly, duplicating WorkSessionService.create's validation (existing-active check, single-active-per-task supersede, project/task existence). The two sites had drifted. Route through WorkSessionService.create instead, mapping ConflictError to the idempotent 'if needed' None. Remove the now-dead _supersede_other_active_sessions (create's supersede_active_sessions_for_task replaces it). Fix three pre-existing RED tests surfaced by the sweep (all confirmed failing on the F110 commit before this change): - test_fail_qa_work_session_fallback_excludes_qa_session: inserted two ACTIVE work_sessions per task, violating uq_work_sessions_one_active _per_task (migration 047). The QA session is now ABANDONED — still in the fallback query's result set (the query filters by task_id + agent_id, not status), so the exclude filter (agent_id != qa_id) is still exercised and the dev is resolved. - test_ceo_reject_routes_coordination_task_to_main_pm / test_ceo_reject_routes_batch_umbrella_to_main_pm: ceo_reject emits an audit row keyed to CEO_AGENT_ID, but the tests never seeded the CEO agent row (fk_audit_log_agent_id_agents). Seed the CEO agent (get-or- create, mirroring test_ceo_reject_writes_handoff_journal). * [F114] single-claimant guard on pr_gate_claim pr_gate_claim delegated straight to _qa_or_doc_claim, which overwrites claimed_by / active_claimant_id with no single-claimant check. Two reviewers race-claiming the same awaiting_pr_review task would last-write-wins overwrite the first claim, and the first reviewer's subsequent pr_pass / pr_fail would actor-mismatch against the new owner (wasting a review cycle). The orchestrator's gate dispatcher already prevents double-reviewer-dispatch in normal flow (one task -> one team -> one reviewer + is_agent_active + per-tick spawned set), so the race is only reachable via direct concurrent API calls (defense-in-depth). Add a role-aware single-claimant guard in pr_gate_claim: lock the row FOR UPDATE (serialize concurrent claims, mirroring the dev claim path), then refuse only when the task is already actively claimed by a DIFFERENT PR-reviewer. The gate task is owned by the PM at entry (submit_for_review does not clear ownership, unlike submit_for_qa), so the guard must distinguish a PM/dev owner — which the first reviewer legitimately overclaims — from a competing reviewer claim; checking the existing claimant's role (pr_reviewer) does exactly that. A re-claim by the same reviewer is idempotent (skipped by the != check). The gateway claim_gate_review handler already maps a None return to a clean invalid_state envelope ('it may already be claimed; give_me_work for the next'), so no gateway change is needed. TDD: 3 integration tests in test_task_service_basics.py — reject a second reviewer race-claim (returns None, first claim intact), allow the first reviewer when the PM owns the root (regression guard for the PM-owns-at-entry model), idempotent re-claim by the same reviewer. Confirmed the reject test RED first (race-claim succeeded, overwriting reviewer1). * [F115] sample monorepo per (repo,workflow)/(repo,command) not per repo The CI-watch and dep-update loaders collapsed a monorepo's cell-projects to one canonical entry per repo (slug-sorted-first), so a repo whose cells each carry their OWN ci_watch_workflow / dep_update_command had only the canonical cell's workflow/command sampled — a red on another cell's workflow or drift on another cell's lockfile was missed (under-count). Refactor the shared one-per-repo collapse into _projects_one_per_key, keyed by repo identity for external-PR discovery (unchanged: one review per PR per repo), by (repo, effective workflow) for CI-watch, and by (repo, command) for dep-update. Each distinct workflow/command is now sampled once; the engines' per-git_url fix-task dedup still prevents duplicate fix tasks for the same repo. _projects_one_per_repo now delegates to _projects_one_per_key. key_fn uses a string annotation (Callable lives under TYPE_CHECKING, like the existing Coroutine/Iterable annotations at lines 4193/5279). * [R115] originate ci_watch/dep_update fix tasks as PLANNING coordination roots The Main-PM-code-impossibility guard (commit e202ce39, Thread 4 of this audit) made team=MAIN_PM + task_type=CODE impossible — a Main PM coordinates, it does not write code. But the ci_watch and dep_update engines still originated their fix tasks as task_type=TaskType.CODE assigned to main-pm, so task_svc.create raised MAIN_PM_NO_CODE and NO fix task was ever opened — a regression introduced by the earlier audit fix (confirmed: the engine tests pass at e202ce39~1 and fail at HEAD). Mirror the hardened self_heal_engine precedent (self_heal_engine.py:197) which already uses task_type=TaskType.PLANNING for its Main-PM coordination root with an explicit 'decompose the fix and delegate the code work to a cell dev — the Main PM does not write the fix itself' description. Both engines now originate PLANNING coordination roots with matching delegation guidance in the description + acceptance criteria. confirmed_by_human stays True for both (they ride the normal delivery flow without the CEO gate, unlike self-heal — intentional per the architecture). The dedupe/open-cap queries (list_open_ci_watch_tasks / list_open_dep_update_tasks) key on source + non-terminal status + git_url, NOT task_type, so the type change does not break dedup (still one open fix task per repo). The two source-test fixtures (test_ci_watch_source / test_dep_update_source) created CODE+MAIN_PM tasks directly to exercise the listing queries — same guard violation; switched to PLANNING (the queries assert on source/status, not task_type, so the fixture type matches the engines' corrected type). * [F116] hold the read-clone lock across the dep-probe local clone dry_upgrade_changes_lockfile called ensure_read_clone (which syncs the read clone under the _meta-conventions lock then releases it) and ran 'git clone --local --no-hardlinks <read_clone>' OUTSIDE the lock. A concurrent ensure_read_clone -> _sync_read_clone (fetch + hard-reset to origin's default branch) could mutate the read clone's working tree / object db mid-clone, racing the clone and producing an inconsistent or failing probe. Split _probe_lockfile_change into _clone_local_into (the local clone, run under the read-clone lock) + _probe_lockfile_on_clone (the upgrade + git status, run without the lock on the now-independent copy). The probe acquires _ensure_lock_for(slug, '_meta-conventions') — the same lock ensure_read_clone syncs under — and holds it only for the clone step; the upgrade operates on the full --no-hardlinks copy and never touches the read clone, so the lock is released before it to avoid blocking conventions reads for the upgrade duration. The tiny gap between ensure_read_clone releasing the lock and the probe re-acquiring it is safe: any concurrent _sync_read_clone completes under the lock before the probe acquires, so the clone reads a stable state. * [F117] stop the orchestrator in lifespan shutdown BEFORE closing the DB The lifespan shutdown closed OptimalService + the DB, and only THEN did bootstrap's finally block call orchestrator.stop() — so stop() ran with the DB already closed. stop() drains fire-and-forget _bg_tasks writes (respawn_tracker upserts, audit-log rows) and stop_agent finalizes work sessions / agent state, all needing the DB still open; closing it first silently dropped those final writes (the durable PM-respawn counter's last few strikes, the metrics-bearing audit trail tail). Move orchestrator.stop() into the lifespan shutdown path, BEFORE close_optimal_service + close_db, guarded by a new get_orchestrator_or_none() safe accessor (no crash when no orchestrator is wired — tests, skip_orchestrator). bootstrap's finally-block stop() becomes an idempotent safety net: stop() gains a _stopped flag (getattr-guarded so __new__- constructed test instances still stop) so the double-call is a clean no-op, not a re-stop of already-stopped agents / re-drain of an empty bg set. * [F118] coerce a lone-string where_to_look into a list where_to_look is a list-typed handoff field like consequences/next_steps but was the only one NOT in the _wrap_scalar_in_list field_validator. A well-intentioned where_to_look='src/api/' 422'd at the route with no remediation envelope, and the agent's retry loop tripped the do-server circuit breaker — the exact failure mode the other list fields were hardened against. Add it to the mode='before' validator so a lone string is wrapped into a one-element list before type coercion. * [F119] sender reaps dead sockets on send error instead of waiting for receive idle timeout * [F120] release a stopped agent's claimed task immediately on budget-kill/shutdown * [F122] name the already-open PR in submit_up's None-state remediate submit_up's create_pr pre-side-effect opens the cell→root PR BEFORE submit_for_review runs (its pr_created gate requires it — lifecycle.py:1338-1343). When submit_for_review returns None (a concurrent state change raced the task out of in_progress between the precondition gate and the composed action), the old remediate ('check task state — must be in_progress with PR ready') hid that the PR was already open on GitHub — an orphaned external artifact the PM could not reconcile. Mirror submit_root's F016 None-envelope remediate: name the open PR, point the PM at re-fetch + reconcile, and note create_pr is idempotent so a re-issue re-attaches to the existing PR (no duplicate). Pure message improvement — zero behavior change; reordering is off the table (create_pr must precede the pr_created gate). * [F124] re-check dependency state before releasing a dependency-blocked claim The unmet_dependency guard read dependency state via an unlocked SELECT, then fired release_dependency_blocked_claim (a state mutation: claimed/in_progress -> pending, clears branch_name, abandons WorkSession) as a side-effect BEFORE returning the rejection. An upstream dependency that reached a terminal state (completed/cancelled) in the microseconds between the read and the release left the task NEEDLESSLY released — its branch cleared + WorkSession abandoned + assignee bounced, only to be re-dispatched + re-claimed when the dependency- completion re-dispatch fired a moment later. Re-check unmet_dependency_ids immediately before the release and skip it (returning None — proceed) when the upstream just completed. Dependencies are monotonic (unmet -> met only; terminal states never reopen), so a fresh read that now finds them met stays met: safe to proceed without releasing. The 'still unmet' path is byte-for-byte the prior behavior (no regression). The cross-task residual window (upstream completes between the re-check and the release) is not closable by a row lock on the dependent, but the re-check narrows the window from [first read -> release] to [re-check -> release], and in the common case the first read already sees met (no guard fires). No committed-work loss either way (a dependency-blocked task has none; the branch ref + commits persist across the branch_name clear). * [F125] serialize same-parent delegate via per-parent advisory lock The delegate sibling-dedup guard read the parent's existing subtasks via an unlocked get_subtasks SELECT (the dedup read) then created the subtask (the write) with no DB serialization between them. Two concurrent delegate calls for the same parent (PM re-delegating while a reaper re-dispatches, or two orchestrator ticks racing) each read a duplicate-free sibling set, each passed the dedup guard, and each created a subtask — the parent got the duplicate the guard exists to prevent (the smoke-run runaway pattern). Fix: a PostgreSQL transaction-scoped advisory lock keyed by the parent task id (seed 1, disjoint from the per-agent claim lock's seed 0), acquired at the top of the delegate body before the first get_subtasks read (the briefing context read AND the dedup sibling read) and held through create_subtask's flush + the outer request commit. The second concurrent same-parent delegate blocks until the first commits, then its dedup read sees the committed sibling and is rejected. Per-PARENT (not per-agent): a coordinator PM legitimately delegates many subtasks under one parent in quick succession and plans many roots in parallel — a per-agent lock would serialize all of a PM's delegates and regress the PM coordinator concurrency feature. The per-parent lock serializes only same-parent delegates (the dedup invariant is per-parent) and leaves different parents untouched. TDD: red-first ordering test (lock acquired before first get_subtasks read and before create_subtask) + no-regression test (create still runs). * [F127] per-task advisory lock prevents open_pr milestone double-emit open_pr's idempotent re-entry guard (pr_number is not None) read t.pr_number from an unlocked fetch. Two concurrent same-task open_pr calls (the alive-but-unresponsive respawn race) both fetched pr_number=None, both passed the guard, both ran the runner (GitHub 422 ensures one PR), and both reached _record_milestone_progress -> a double-emitted 70% 'opened PR #N' entry. Fix: acquire_task_lock (pg_advisory_xact_lock, seed 2) before the fetch, held through the runner + milestone + request commit. The second concurrent call blocks until the first commits, then its fetch sees the committed pr_number and the idempotent guard short-circuits without re-emitting. Per-task (single- active-task guard means same-task concurrent open_pr is only the bug case). * [F128] require active claim on explicit-task content posts _verify_explicit_task_ownership checked assigned_to, which is stale across a reap/handoff (persists until reassignment; active_claimant_id is cleared on release). A reaped agent could keep posting say/dm/note to its former task. Add the active-claimant check when assigned_to == caller; assigned_to=None keep its existing allow (read-side inspection between reassignments uses evidence, which has its own ownership path). Existing 'active owner' test mocks passed assigned_to=agent_id without active_claimant_id; production sets both together on claim, so the mocks were incomplete. Updated to set both — realistic, not a behavior change. * [F129,F130] harden quality gate _run_one exit status + timeout cleanup F129: _run_one returned 'proc.returncode or 0', masking a None returncode (communicate returned without a recorded exit code — process killed out-of-band) as 0 / success. Treat None as a non-zero failure (fail-closed). F130: on timeout, _run_one killed the subprocess but never awaited wait() — communicate() was cancelled so it never closed the stdout/stderr pipes, leaving a transient zombie + leaked FDs. Await wait() after kill() to reap the process and close the transports. * [F132] timeout the conventions validator + reap on hang _run_conventions_validator awaited proc.communicate() with no timeout — a hung subprocess (tree-sitter deadlock, huge repo) hung the i_am_done/pr_pass gate forever and orphaned the python subprocess on orchestrator restart. Wrap communicate() in wait_for(120s); on timeout kill+wait the proc and fail closed (could_not_run=True → block gate refuses the submit), matching the validator's own fail-loud philosophy. * [F135] re-check activity before sweeper closes a session (TOCTOU) sweep_timed_out_sessions read last_activity_at once at the candidate SELECT, then closed. A message landing in that window refreshed last_activity_at in the DB, but the sweeper closed on its stale in-memory value — closing a just-used session. Re-read last_activity_at fresh right before the close and skip if the session is no longer timed out. * [F136] cancel startup indexing task on OptimalService.close() close() cancelled only the periodic update task, then cleared the plugins. The startup _indexing_task (background auto-index, slow Ollama / large repo) could still be mid-flight at shutdown and write against closed/cleared plugins. Cancel and await _indexing_task FIRST (its tail starts the periodic task, so ordering also prevents a late periodic spawn), then the periodic task, then clear plugins. * [F139] scope active_task_owns_branch to the polled project active_task_owns_branch did an unscoped WHERE branch_name = ? — a cross-project branch_name collision (UUID-derived 8-char prefixes, theoretical) made the internal-PR reviewer skip the WRONG project's PR (project A's leftover PR skipped because project B happened to have an active task with the same branch). Pass project_id (in scope at the orchestrator call site) and add TaskTable.project_id == project_id to the WHERE. Correct for single-project tasks and MegaTask multi-repo batches alike: each root-subtask carries its own project_id matching its own repo, so a branch on project A's repo is owned only by a task whose project_id == A. * [sweep] strip Fxxx audit-ID tokens + trim bloated comments/docstrings + add behavior-change docs Post-audit sweep over the 135 audit-fix commits since 19a474d3: 1. Stripped every # Fxxx: audit-ID token from comments AND every Fxxx token from docstring openings across 211 blocks / ~626 lines. The CEO flagged these twice: audit-issue IDs in code confuse future devs/agents. The descriptive text is preserved; only the Fxxx token is removed (and bloated narrative blocks trimmed to 1-3 lines keeping the one non-obvious invariant). 2. Trimmed bloated comments/docstrings to the concise standard (1-3 lines). 3. Added missing behavior-change docs for the audit-fix batch: prompts/roles (documenter, pr_reviewer, qa), user-facing docs (api auth, websockets, agent-gateway, megatask, merge-model, task-lifecycle, grok, resilience, conventions, panel, security, troubleshooting), and the RAG corpus (cell-pm, main-pm, pr-reviewer, qa roles; conventions; messaging-tools; escalation; megatask; task-claiming workflows). Comment/docstring/prose ONLY — zero code-line edits (verified: the diff contains no def/class/return/if/for/await/assignment/call lines). Gates green: ruff format + ruff check clean, mypy clean on roboco/. The only pytest failures are the pre-existing sync_branch tracing-decision gap (B1, 250be5c2) — not sweep-caused and tracked separately. * [fix] register sync_branch in VERBS_WITHOUT_TRACING sync_branch (B1, 250be5c2) is a git-only rebase+force-push verb (composes=(), no DB transition, side_effects=()) but was never registered in the tracing parity tables, so test_every_intent_verb_has_a_tracing_decision failed. Mirrors open_pr: a mechanical git op with inline preconditions (ownership), no journal/plan rationale required. * chore(release): 0.14.0 * [fix] resolve 16 mypy errors across 9 test files (make quality gate) type-clean the test files so make quality (mypy roboco/ tests/) is green: - Any-typed locals for the two TypeError-asserting scoping tests (bypass the required-arg check without getattr/ruff B009) - Any-typed view for the shutdown-drain _drain_bg_tasks override (bypass mypy method-assign without setattr/ruff B010) - cast("uuid.UUID", ...) / cast("UUID", ...) for SQLAlchemy UUID[Any] returns (TC006-quoted), config=None for AgentInstance stubs, None-narrowed await_args, Iterator return on a yielding fixture, UUID annotation on the _task helper. No type:ignore / noqa. * [docs] regenerate lifecycle artifacts for sync_branch + branch-keyed submit_root gate The committed artifacts were stale: lifecycle.py grew the sync_branch verb (B1) and the branch-keyed submit_root gate description (B2/B3) but the generated markdown/json were never regenerated. make foundation-check enforces artifact==generator(lifecycle.py); regenerating restores that. No source change — pure generator output. * [refactor] reduce xenon C-rank blocks to A (behavior-preserving) Extract helpers / flatten conditionals in 11 blocks that rated C(11)+ under xenon --max-absolute B, dropping pr_gate.py module rank B->A in the process. Pure move-and-call refactors: each extracted helper holds the original logic verbatim and the caller delegates to it; no control flow, return values, or side effects changed. Sites: validators._extract_strs, sequencing.dev_task_collision_edges, evidence_builder.build_task_handoff, intake_driver._coerce_draft, task.claim_task_for_agent (2 guards), prompter.create_task_from_draft (validate+assignee), pr_gate._gate_decision (3 helpers), orchestrator._handle_stopped_container + _reap_with_service, _impl._create_subtask_from_inputs + complete. _impl helper returns tuple[TaskNature, list[str]] to preserve mypy narrowing of acceptance_criteria at the TaskCreateRequest site. Also fix vulture: rename unused __aexit__ param tb->_tb in test_conventions_cache_put.py (was hidden while xenon short-circuited the gate). * [security] bash-guard uv run --active deny + CodeQL path-traversal fixes Fix 1 (be-dev-1 brick prevention): bash-guard now denies 'uv run --active' and 'uv run'/'uvx' against /app targets. In the agent container VIRTUAL_ENV=/app/.venv is baked globally, so 'uv run --active' always resolves onto the image-baked MCP-gateway venv and uv rebuilds it, deleting /app/.venv/bin and bricking every MCP server spawn. Bare 'uv run' (workspace .venv, cwd-relative) is untouched. CodeQL fixes: - docs.py: replace bypassable '..' substring guard with a resolve-and-contain helper (_resolve_contained_path). An absolute path made pathlib reset (base / '/etc/passwd' == '/etc/passwd'), letting read_doc/delete_doc reach arbitrary files. Applied to both sinks. - orchestrator.py: _safe_agent_path_segment at the spawn_agent chokepoint (rejects traversal-shaped agent_id before any fs op) and inside _remove_container (slug guard before the log-dir mkdir, defense-in-depth). - agent_sdk/server.py: /usage/sync transcript_path now resolved and contained under ROBOCO_TRANSCRIPT_DIR with a .jsonl suffix requirement (was Path(raw) — unauthenticated endpoint could stat arbitrary files). TDD RED->GREEN across all four; make quality green (4890 passed). * [fix] enum-parity gate: drop false-green mask, skip empty/unmigrated DB The foundation-check gate ran the enum verifier behind `|| echo "(skipped — postgres unreachable)"`, which swallows ANY non-zero exit — including real drift — and prints 'All quality gates passed'. On a host with a dockerized but empty/unmigrated `roboco` DB (0 tables: the agentrole/team enum types don't exist), the verifier connected, found every foundation value 'missing', exited 1, and the mask relabeled it 'skipped' → false-green. Fix: - scripts/verify_postgres_enums.py: move skip semantics INTO the script. Distinguish unreachable (skip, exit 0), DB-not-migrated/both-enum-types- absent (skip, exit 0), real drift (exit 1), match (exit 0). Extract pure enum_drift + should_skip_for_unmigrated helpers + a type_exists probe so an empty DB is 'no migrated target', not drift. - Makefile: drop the `|| echo` mask — real drift now fails the gate. TDD RED->GREEN (10 tests); make quality green (10906 passed). * [security] docs path guard: reject '.'/empty segments for clean 400 _resolve_contained_path used an '..' substring ban, which (a) left rel='.' passing the guard — read_doc/delete_doc then got the base DIRECTORY itself and raised IsADirectoryError (500) instead of a clean ValidationError, and (b) false-rejected legit filenames containing '..' like 'v1..v2.md'. Replace the substring ban with a raw-segment check (rel.split('/')) that rejects any '.', '..', or empty segment. Path(rel).parts was the wrong tool — pathlib collapses '.' and empty segments on 3.13, hiding them. The split check catches '.' / 'a/./b' / 'a//b' / '..' / 'a/../b' while allowing 'v1..v2.md' ('..' inside a filename, no bad segment). The post-resolve parents-containment check (the real defense) is unchanged. TDD RED->GREEN (4 new tests); make quality green (10910 passed). Follow-up to the CodeQL path-traversal review: the two CodeQL 'High' alerts on this guard are false-positives-on-the-fix (resolve-and-contain already contains the bypass); this hardening closes the one genuine low residual (rel='.' -> 500) the review surfaced, which CodeQL did not flag. --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com>	2026-06-29 05:38:21 +02:00
3 people	889f3689e7	MegaTask (#248) ... * feat(batch): batch_id + collision descriptor columns Sequenced batch intake ("Mega task") foundation: tasks.batch_id (indexed) groups a batch of top-level tasks created together; intends_to_touch (text[]), adds_migration and touches_shared (bool, NOT NULL default false) are the per-task collision surface the SequencingService will read to wire dependency waves. Mirrored on the Task model + TaskCreateRequest and wired through TaskService.create. Migration 046 (real upgrade->downgrade->upgrade verified vs a throwaway pgvector PG); a non-batch task declares no surface (defaults). Task 1 of the 0.11.0 sequenced-batch-intake plan. * feat(batch): flag + draft collision descriptors Default-off ROBOCO_BATCH_INTAKE_ENABLED (config + FEATURE_FLAGS + panel card); the propose_draft tool doc + the TS DraftProposal gain the per-task collision surface intends_to_touch / adds_migration / touches_shared. The draft is a loose dict so the descriptors ride it through the relay intact (test asserts the forwarded payload); the analyzer (Task 3) reads them to wire dependency waves. Task 2 of the 0.11.0 sequenced-batch-intake plan. * feat(batch): deterministic collision-sequencing analyzer SequencingService.analyze turns a batch's per-task collision surfaces into a dependency DAG + execution waves — correctness in CODE, not agent judgment. Rules in order: file overlap serializes (more-important first), migrations form a serial chain (no concurrent Alembic heads), touches_shared runs last, cell contention warns (never serializes); then dedupe, existence + cycle check, and Kahn topological layering. Pure (no DB/services); SequencingError on a cycle or out-of-range edge. Golden test reproduces the CEO's hand-sequenced 4 waves of the 11-item guard-core-app batch (the effort that deadlocked the Main PM): S6 alone last, the R1/R3/R4 migration chain, R2/R3/S8 serialized on the shared threat service, S1/S2/S7 in one parallel wave. Task 3 of the 0.11.0 sequenced-batch-intake plan. * chore(batch): brand the user-facing surfaces "MegaTask" The user-facing name is MegaTask: the feature-flag label is "MegaTask intake", the panel flag-card and the config description lead with MegaTask. Internal names stay technical (batch_intake_enabled, batch_id, SequencingService). * chore(batch): drop the feature flag — MegaTask is a core intake scope MegaTask is additive and opt-in by its own nature (the Prompter proposes a batch only when the CEO asks for several tasks; single-task intake is unchanged), so there is no risk surface a flag protects — 'don't create a MegaTask' is the off switch. Remove batch_intake_enabled from config, the FEATURE_FLAGS registry, the panel flag card, and its tests. MegaTask will be a third scope option in the Intake modal (single-cell / multi-project / MegaTask), not a toggle. * feat(batch): MegaTask identity predicate + orchestrator branchless recognition The single source of truth for the umbrella's exemptions: pure is_batch_umbrella / is_batch_root_subtask / is_branchless_coordination (foundation/policy/batch.py) — an umbrella has a batch_id and is top-level; a root-subtask shares the batch_id but is parented. The orchestrator's _is_coordination_task now consults is_branchless_coordination, so a MegaTask umbrella is recognized as doing no git of its own (git-exempt at spawn-readiness / stuck-detection) exactly like a product fan-out root. Non-batch behavior is identical (the predicate reduces to the old no-project+product check; the orchestrator coordination suite stays green), and the umbrella branch is inert until the create path exists. First slice of the MegaTask umbrella enforcement (branchless guard). * feat(batch): branchless umbrella guard across the git-exemption sites A MegaTask umbrella does no git of its own — every git-exemption site in TaskService now consults the shared is_branchless_coordination predicate instead of an inline product-only check, so the umbrella's exemptions cannot drift between sites: - the claimed->in_progress branch gate (GitContext.is_coordination) lets an unbranched umbrella reach in_progress and delegate; - _ensure_branch_for_task short-circuits an umbrella to "" instead of the misconfigured raise (the claim path ignores the return, treating it as branchless); - CEO-reject routing sends a rejected umbrella to the Main PM in PENDING (needs_revision is developer-claim-only and would deadlock it). Covers both shapes via the predicate (product fan-out root OR umbrella); a batch root-subtask keeps its own branch/PR. Adds orchestrator recognition tests for the umbrella plus claim/branch/reject integration tests. * feat(batch): umbrella assembles no PR; completes branchless submit_root now hard-rejects a MegaTask umbrella up front (a preflight that also folds in the unknown-role refusal to stay within the return-count budget): the umbrella spans many projects with no single master, so each root-subtask opens and is reviewed on its own PR — the umbrella never enters the in-path review gate. The Main PM completes it directly once every root-subtask is terminal. Umbrella completion needs no new code: it is branchless (no branch_name), so _main_pm_complete_guard already accepts it from in_progress, checks all_subtasks_terminal, and main_pm_complete walks it to awaiting_pm_review and escalates to the CEO with no PR creation — exactly the product fan-out root path. Adds the submit_root-reject and umbrella-completion gateway tests; pins batch_id=None on the normal-root submit_root test (a MagicMock auto-attr would otherwise read as an umbrella). * feat(batch): MegaTask create path — umbrella + sequenced root-subtasks PrompterService.confirm_live_batch turns N confirmed drafts into a real MegaTask: it builds each draft's collision surface, runs the pure SequencingService to get conflict-free waves, creates the branchless umbrella (batch_id, no project/product), then one root-subtask per draft (own project, parent=umbrella, sequence=wave index, descriptors), and wires the analyzer's edges through add_dependency so the existing dependency-gate runs the waves in order. The route picks the start path like a single confirm: 'board' holds the root-subtasks in BACKLOG for the batch review; 'main_pm' creates them PENDING so wave 0 dispatches at once. create_task_from_draft gains a BatchPlacement (parent/batch/sequence/ team_override) and forwards the collision descriptors; the exactly-one- target rule (here and the TaskService.create invariant) is relaxed for an umbrella, which legitimately targets neither. New route POST /live/{session}/confirm-batch + BatchConfirmRequest mirror the single confirm. Adds the structural-invariant + board-hold + empty-batch tests. * feat(batch): release MegaTask root-subtasks on CEO approval; board awareness The board route holds a MegaTask's root-subtasks in BACKLOG so the work waits for the batch review. approve_and_start (CEO gate #1, board->Main PM) now releases them via _activate_batch_root_subtasks: each held child flips BACKLOG -> PENDING + team=main_pm so the dependency-gate dispatches wave 0. No-op for a non-umbrella; idempotent (children past BACKLOG untouched). The Product Owner and Head of Marketing identity prompts gain a MegaTask section so they review the whole batch + wave plan and adjust scope before sign-off (they review drafts; the umbrella is their unit). Also extracts the create() target invariant into _require_target_or_umbrella to keep the method under the complexity gate after the umbrella exemption. Adds the umbrella-approval activation test. * feat(batch): multi-project intake scope for MegaTask A MegaTask spans several possibly-unrelated repos, so the intake chat can now be scoped to an explicit project list (not just one project or one product). StartLiveRequest gains project_ids; /live/start threads it through start/spawn_intake_session -> _spawn_intake_container -> _clone_intake_scope. The multi-repo clone machinery already existed for products; _intake_scope_slugs now also resolves an explicit project_ids set (split into _slugs_for_project_ids / _slugs_for_product), cloning each repo with the first as the primary cwd and the siblings readable. Scope validation is now 'exactly one of project_slug / product_id / project_ids' via the shared _require_one_intake_scope. Adds scope-resolution, spawn, and route tests for the MegaTask path. * feat(batch): propose_batch intake tool (MegaTask multi-draft hand-off) The intake agent can now hand the panel a whole MegaTask in one tool call. Both intake paths gain propose_batch alongside propose_draft: - Claude (intake_driver): a propose_batch tool registered on the in-SDK MCP server + allowlisted; the driver intercepts the ToolUseBlock and emits ONE StreamChunk(kind="batch") carrying {drafts:[...], title}. - grok (intake_server): a propose_batch tool that POSTs a "batch" relay event via the shared _post_event helper (post_draft/post_batch). A batch carries N drafts, each the propose_draft shape PLUS its own project_id (a MegaTask spans unrelated repos) and collision surface so the analyzer sequences the waves. The prompter prompt documents the MegaTask scope + when to call propose_batch. Adds Claude-normalize and grok-relay tests for the batch path. * feat(batch): MegaTask intake panel — third scope, batch review, waves The panel now drives a MegaTask end to end. The intake modal gains a third scope, 'MegaTask', beside Single cell and Board-led: a multi-project checklist (a MegaTask spans several possibly-unrelated repos), validated to at least two. start() sends project_ids; use-prompter accumulates the agent's single propose_batch hand-off as a 'batch' SSE event into a BatchProposal and lands in a new batch_preview state. A new BatchReviewCard lists every proposed task with its target project + collision-surface badges (migration / shared) and offers one start path for the whole batch — Board review & Start or Approve & Start — wired to confirmBatch → POST /confirm-batch. The success card shows the sequenced result: N tasks in M waves (+ any advisory notes). prompter.ts gains the DraftScale 'megatask' + the BatchConfirm payload/result types; the SSE client allows the 'batch' kind. Panel typecheck + lint + 113 tests green. * docs(batch): MegaTask across changelog, CLAUDE.md, site, and RAG The four documentation obligations for the MegaTask feature: - CHANGELOG: an Unreleased entry covering the umbrella model, sequencing, multi-project intake, propose_batch, and the create/approval path. - CLAUDE.md: a MegaTask section (identity predicate, umbrella/root-subtask hierarchy, sequencing rules, intake + create path, board activation). - Published site: a user-facing company/megatask.md (scopes, waves, the umbrella, the two start buttons) + nav entry; a pointer added to the intake chapter of the Tour. - RAG corpus: workflows/megatask.md so the Main PM (and any agent) can retrieve the umbrella's branchless / no-PR / completion rules at runtime. The runtime concurrent-migration guard is intentionally NOT added: the analyzer already chains migration-adders into dependencies and the dependency-gate serializes them, so a separate guard would be dead code. * feat(batch): batch_id guardrail + wave preview + batch_id on TaskResponse Guardrail (CEO): a batch_id is denied on any task that is not a well-formed MegaTask member. is_valid_batch_shape permits batch_id only on an umbrella (no parent → must target neither project nor product) or a root-subtask (has a parent → exactly one target); TaskService.create enforces it AND verifies a root-subtask's parent is the batch umbrella (same batch_id, top-level). This closes a latent hole: is_batch_umbrella is true for a batch_id + no-parent task even with a project, so a stray batch_id could have spoofed the branchless branch-gate / no-PR exemption. (The public task API never exposed batch_id for write; this guards the service layer.) Wave preview: PrompterService.preview_batch + POST .../preview-batch compute a MegaTask's waves from the proposed drafts WITHOUT creating anything, so the panel can show the sequencing before confirm. Extracted _sequence_drafts as the single source shared by preview and confirm, so the previewed waves are exactly the ones wired. TaskResponse now carries batch_id so the panel can badge the umbrella. * feat(batch): MegaTask review — project editor, wave preview, persistence, badge Closes the panel gaps in the MegaTask review experience: - Per-task project editor: each proposed task gets an inline project Select (updateBatchDraftProject), so a task the agent put in the wrong or no repo can be fixed before launch — not only by re-chatting. Launch stays blocked until every task has a project. - Wave preview: on a batch proposal the panel fetches POST .../preview-batch (no task created) and shows the conflict-free wave plan, so the human reviews the sequencing before confirming. - Refresh durability: the MegaTask review (batch + waves + projectIds) is persisted, so a browser reload mid-review restores it like a single draft. - MegaTask badge: TaskResponse exposes batch_id, the panel Task type carries it, and the task table badges the umbrella row 'MegaTask'. Panel typecheck + lint + 113 tests green. * test(batch): stub task carries batch_id for task_to_response task_to_response now serializes batch_id (TaskResponse field), so the _stub_task SimpleNamespace fixture must provide it — without it the reader hit AttributeError, failing the 8 task-schema serialization/enrichment tests. Test-only; the real TaskTable carries the column (migration 046). * fix(batch): close MegaTask audit gaps — completion crash, analyzer cycle, guardrails An adversarial multi-agent audit of the feature surfaced 20 verified gaps; this closes the backend ones. HIGH: - Umbrella completion crashed. escalate_to_ceo hard-required a pr_number, which a branchless umbrella never has, so main_pm_complete dereferenced None. Both pr_number gates now waive a MegaTask umbrella (escalate_to_ceo + the awaiting_pm_review->awaiting_ceo_approval lifecycle gate via a new GitContext.is_umbrella), and main_pm_complete guards a None return. The completion test had mocked escalate_to_ceo, hiding it — now a real service test covers the waiver. - The collision analyzer could fabricate a cycle (a touches_shared + adds_migration draft overlapping another migration draft) and raise SequencingError — a bare ValueError that escaped as an opaque 500. The migration chain is now shared-last-aware (never contradicts rule 3), and _sequence_drafts translates SequencingError to a clean 400. MEDIUM: - Collisions are now project-scoped: two repos can't collide on a coincidental path or serialize independent migrations (DraftSurface carries project_id; rules 1/2/3 respect it). - The batch_id guardrail ran only at create. update() + the PATCH null-clear path now re-assert is_valid_batch_shape, so a mutation can't break a member's shape and spoof the branchless exemption. - A draft missing title/acceptance_criteria now raises ValidationError (was a bare KeyError -> 500). - confirm_live_batch re-asserts every draft targets a scoped project and the batch spans >=2 distinct projects (project_ids added to the request). - Route-level tests for confirm-batch / preview-batch. LOW: strict multi-repo clone (fail loud on any unresolvable project); malformed/empty propose_batch surfaces an error chunk (Claude) / refuses to POST (grok) instead of silently acking; dropped malformed drafts are counted and surfaced; stale grok intake docstrings updated. * fix(batch): MegaTask panel + doc audit gaps Frontend half of the audit fixes: - The confirm payload now carries project_ids (the schema requires it), and the panel re-checks every task targets one of the scoped repos before launching, naming the offending task. - The Review-MegaTask project picker is filtered to the scoped repos and the per-task validity (border + launch gate) keys off scoped membership, so a task can only be (re)pointed at an in-scope project — also fixing the case where the agent emitted a non-UUID / unknown project. - Dropped malformed drafts are surfaced as a chat error so the human knows the batch shrank instead of silently confirming fewer tasks. - Doc wording: a wave releases on the previous wave's terminal state (normally a merge; a cancellation releases it too), not strictly 'merged'. * test(batch): lock the CEO's EXACT 4-wave hand-sequencing as the golden bar The golden test asserted the constraints (S6 last, the migration chain, the shared-threats serialization, S1/S2/S7 parallel) but not the full wave partition. The bar for MegaTask is 'reproduce my exact waves or it's not done', so assert the exact 4-wave partition the analyzer produces for the guard-core-app batch: wave 1: R1 R2 S1 S2 S3 S5 S7 · wave 2: R3 · wave 3: R4 S8 · wave 4: S6 Confirmed unchanged by the audit's analyzer fixes (no migration is shared; single project). * fix(batch): tolerate a stub task in assert_batch_shape_intact The batch-shape re-validation read task.batch_id directly, but update()'s partial-caller contract is exercised with a SimpleNamespace stub that has no batch_id column → AttributeError. Use getattr(..., None) for batch_id and the shape fields so the guard no-ops on any task lacking the column (a stub, or a non-batch task) while still enforcing on a real batch member. * fix(orchestrator): authenticate internal API self-calls with the system identity The dispatcher httpx clients were built without an agent identity, so the orchestrator's self-PATCHes to /api/tasks/{id} (auto-block, auto-resume, auto-recover, SLA annotation) were rejected 401 "Missing X-Agent-ID" and silently no-op'd. The auto-resume that lifts a PM's paused parent could never write, so paused/blocked parents stayed wedged and stranded their dependents (the fe-pm/be-pm respawn churn seen in prod). Header propagation was inconsistent across the separate AsyncClient call-sites: only the main dispatch client carried the system identity; the readiness and sweep clients did not. Hoist the identity into a shared _SYSTEM_API_HEADERS constant and apply it to every API-facing dispatcher client. The system role holds TaskAction.ASSIGN, so it is authorized for the audited admin_set_status path those write routes use. The external provider-recovery probe client is intentionally left untouched. --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com>	2026-06-24 01:15:57 +02:00
3 people	c09cf80b40	Feature/observability gateway health (#247) ... * feat(observability): revision_count + audit_log query index (migration 045) Adds tasks.revision_count (the O(1) rework counter — forward-only, existing rows default 0) and the composite index audit_log(target_id, event_type, timestamp) that powers the cycle-time and rework reconstruction queries. Verified the real upgrade/downgrade/upgrade chain on a throwaway pgvector PG. First task of the 0.10.0 observability dashboards. * feat(observability): count reworks + attribute qa_fail/pr_fail to the rejector Every transition into needs_revision increments tasks.revision_count at the single audit chokepoint (exactly once per bounce, across all paths incl. pr_fail and ceo_reject), so the rework rate is an O(1) read. A QA or PR-review bounce also emits a named task.qa_fail / task.pr_fail audit event carrying the rejector's agent_id, so the per-agent rework scorecard charges the rejection to the reviewer who made it, not the developer who owns the task. * feat(observability): cycle-time, bottleneck, rework, and scorecard metrics MetricsService gains four read methods on the audit_log + tasks data: per-stage cycle time reconstructed from the transition journey (excluding the named qa_fail/pr_fail events), bottleneck distribution (cumulative dwell + live parked counts), rework rate (overall/by-team/by-agent with rejector attribution + cost via spawn-session task_id), and a fused per-agent/per-cell scorecard. Dataclass models with to_dict(). Verified against a real Postgres journey. * feat(observability): cycle-time/bottleneck/rework/scorecard read endpoints Thin read-only routes on the dashboard router delegating to MetricsService: /metrics/cycle-time, /metrics/bottlenecks, /metrics/rework, and /metrics/scorecard/{agent,team}. 404 when an agent scorecard target is absent. 5 route tests (200 + shape + the agent-404 case). * feat(panel): Delivery observability tab (cycle-time, bottlenecks, rework, scorecards) A third Metrics tab built on the observability endpoints: a per-stage cycle-time bar chart, a bottleneck panel (worst stage + cumulative dwell + live parked counts), a rework panel (rate + by-team + by-agent attribution + cost), and per-cell scorecards. Reuses Recharts + Card/Badge/Skeleton and the React-Query hook pattern; observabilityApi mirrors usageApi with mock-mode fallbacks. tsc + eslint clean; 113 panel tests pass. * docs(observability): changelog + CLAUDE.md for the delivery dashboards * feat(gateway-health): recover a broken-but-alive agent instead of protecting it The verb-heartbeat cannot tell a quiet-healthy agent from one whose MCP gateway is broken (a corrupted /app/.venv firing no verb) yet whose container is up — the reaper's live-skip would shield it forever. The reaper now probes the gateway out-of-band (docker exec: does the gateway venv import its deps?) and, once it has been broken past gateway_health_grace_seconds (tolerating a transient probe miss), kills + evicts the container so it falls through to release + respawn. Probe-inconclusive or healthy spares the container. Gated by gateway_health_enabled (default-on reliability fix; in the panel Feature Flags). Defers the optional agent-side self-check + full registry re-adoption — the reaper's docker-liveness fallback already recovers a broken-after-restart agent. * docs(gateway-health): changelog + CLAUDE.md for broken-but-alive recovery * docs(observability): user-facing docs for the Delivery dashboards + gateway-health Documents the new Metrics -> Delivery tab (cycle-time, bottlenecks, rework with rejector attribution, cell scorecards) in the panel guide and the operations health-and-metrics guide, and adds the gateway-health env vars + an agent-gateway recovery note. Published MkDocs site only; settings.md's default-off flag table intentionally omits the default-on gateway-health flag (same as overload-break). * chore(release): cut 0.10.0 (changelog section + version refs) * fix(gateway): exempt PM coordinators from single-task claim guards A Main/Cell PM plans and delegates many root tasks in parallel; the work then runs in the delegated cells, not in the PM's own hands. But the claim-time concurrency guards meant for developers — already_active and paused (the latter firing after i_am_idle auto-pauses the PM's own umbrella) — were applied to the PM too, so once it held one root it could never plan a second: it thrashed between its claimed roots and respawned forever, burning tokens for zero progress. _run_claim_guards now skips already_active/paused for the coordinator PM roles (_COORDINATOR_ROLES = {main_pm, cell_pm}); only unmet_dependency — a real upstream sequence constraint, which parks the root back to pending — still gates a PM. paused_tasks_guard also excludes the target task itself, so a PM re-entering its own paused umbrella never self-blocks. Tests: a coordinator plans a second root with one in_progress + one paused sibling (full path + claimed-recovery path), the paused target exclusion, and the developer guards still fire. Repurposed the pre-fix test that asserted the now-removed PM block. * fix(metrics): coerce SQL avg/extract hours aggregates to float (panel toFixed crash) EXTRACT(epoch ...) returns numeric on PostgreSQL 14+, which asyncpg surfaces as a Decimal; a Decimal serializes to a quoted JSON string, so the panel's avg_cycle_hours.toFixed(1) (and the other hours fields) threw 'toFixed is not a function' and blanked the Delivery tab. A single _as_hours helper now rounds every SQL-averaged hours field to a real float — avg_cycle_hours on the new scorecards plus the pre-existing avg_completion_hours / avg_blocked_hours / longest_blocked_hours. Token and cost fields were already float()-cast and are unaffected. Regression test asserts _as_hours coerces Decimal -> float and preserves the None/zero behavior. * feat(panel): edit a task's sequence from the details page A task's sequence (order within siblings, lower runs first) was display-only with no way to change it from the UI, and TaskUpdate didn't carry the field so PATCH couldn't set it either. The details page's Dependencies tab now has an inline sequence editor mirroring the parent / dependency editors, and PATCH /tasks/{id} accepts a sequence field (owner or privileged role) through the existing generic update path. * fix(mypy): green the full make-quality type gate make quality runs 'mypy roboco/ tests/', which the per-module checks on the 0.10.0 branch never exercised. Two issues surfaced: - The coordinator-exemption change added role_str to Choreographer._run_claim_guards but not to the ChoreographerHelpers protocol base, so the composed Choreographer had incompatible base-class signatures. Sync the protocol signature. - The gateway-health / stale-reaper tests stubbed methods by direct assignment (orch._m = AsyncMock()) and typed their duck-typed task doubles as object, tripping method-assign / assignment / attr-defined. Switch to monkeypatch.setattr (keeping a local mock ref for the assertions) and type the doubles as Any — no type: ignore. Full mypy roboco/ tests/ clean (785 files); the 21 runtime tests pass. * fix(metrics): static cycle-time SQL — clear bandit B608 (CI gate) The cycle-time query interpolated an optional team clause into the text() SQL via an f-string, which bandit flags as B608 (hardcoded SQL) and turned the merge gate red. The team value was always a bound parameter, so it was a false positive — but the f-string is the trigger. Rebuilt as one static query with (CAST(:team AS text) IS NULL OR a.details->>'team' = :team) and an always-bound team param (CAST, not ::text — SQLAlchemy's :param parser collides with PostgreSQL's :: cast operator, which broke the query as a stray param). Full make quality green vs a real pgvector PG (all 21 gate steps). --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com>	2026-06-23 07:26:41 +02:00
Renn F	07008baaf5	fix(api): serialize pr_reviewer_notes/doc_notes/notes_structured in the task response ... task_to_response (the builder behind the task list + detail endpoints the panel reads) set dev_notes/qa_notes/quick_context but dropped pr_reviewer_notes, doc_notes, and notes_structured, and TaskResponse didn't declare notes_structured at all — so the PR-reviewer notes, documenter notes, and the structured PR-review verdict were always blank in the panel regardless of what agents persisted. The structured-content write-path + obligation gates were fine; the data just wasn't serialized. Now returns every note section + the structured source of truth. Regression test asserts all three round-trip.	2026-06-23 02:34:55 +02:00
4 people	6cf99a1b0a	[beb8cae1] Type-gate tests/ under mypy — fix all errors and flip quality gate (#156) (#157) ... * [420e5e68] Fix mypy errors in tests/unit/ and create tests/__init__.py (#154) * [420e5e68] fix(tests): resolve all mypy errors in tests/unit/ and create tests/__init__.py - Create tests/__init__.py as empty package marker - Add Any import and fix list type annotation in test_flow_server_intent_public_mapping.py - Move AsyncIterator to TYPE_CHECKING block and fix m.cls.__name__ attr error in test_app.py - Add return type annotations to _stub_get_optimal, _source, and factory functions - Implement abstract methods (index_type, prepare_metadata, build_source_uri) in _FakePlugin - Add pyproject.toml per-file-ignore for ARG002 on test_optimal_grounding.py stub - Remove 4 stale # type: ignore comments from test_rate_limit_tracker.py - Fix method-assignment patterns in test_rate_limit_sweep.py via patch.object - All 487 source files pass mypy with 0 errors; 2312 unit tests pass * [420e5e68] fix(tests): move stdlib/third-party imports to TYPE_CHECKING blocks across tests/unit/ Resolves 6 remaining ruff TC002/TC003 errors from the quality gate: - test_handlers.py: Iterator → TYPE_CHECKING - test_quality_gate.py: pathlib → TYPE_CHECKING - test_board_dispatch.py: AsyncIterator + httpx → TYPE_CHECKING - test_streaming.py: Iterator → TYPE_CHECKING - test_notification.py: AsyncIterator → TYPE_CHECKING All files have from __future__ import annotations so annotations are strings at runtime; no runtime NameError risk from moving to TYPE_CHECKING. * [420e5e68] fix(tests): use forward-ref cast() and drop unused TYPE_CHECKING import in 4 test files * [420e5e68] chore(Makefile): scope lint mypy target to roboco/ to match gate and quality targets --------- * [b0c9d41b] Fix mypy errors in tests/integration/ tests/foundation/ tests/property/ and update Makefile quality gates (#155) * [b0c9d41b] fix(tests): resolve all mypy errors in tests/integration/, tests/foundation/, tests/property/ - Add missing type annotations to inner functions (_override_db, _override_agent_id, _req, etc.) - Use cast("UUID", ...) to fix SQLAlchemy UUID vs uuid.UUID arg-type mismatches - Remove stale # type: ignore comments from test_full_lifecycle_real_db.py and test_task_service_lifecycle_misc.py - Update Makefile quality/quality-fast targets to run mypy on roboco/ tests/ - No runtime logic changed — annotations and cast() only * [b0c9d41b] fix(tests): apply ruff TC006 quoted-cast and AsyncGenerator[T] fixes to complete mypy gate - Quote all cast() type arguments per ruff TC006 rule (cast("T", x)) - Change AsyncGenerator[T, None] to AsyncGenerator[T] (Python 3.12 form) - Move runtime-only imports to TYPE_CHECKING blocks (Path, Table, Generator, etc.) - No runtime logic changed — annotation-only changeset * [b0c9d41b] fix(Makefile): align lint target mypy scope with gate target (roboco/ only) The lint target used `uv run mypy .` (all files) while gate uses `uv run mypy roboco/`. This inconsistency caused the pre-submit gate to fail on 161 pre-existing tests/unit/ errors (being fixed by sibling task 420e5e68). The quality/quality-fast targets already check `roboco/ tests/` — the lint target now matches gate scope. --------- --------- Co-authored-by: Backend Developer 1 <be-dev-1@agents.roboco.dev> Co-authored-by: Backend Developer 2 <be-dev-2@agents.roboco.dev>	2026-06-14 13:43:46 +02:00
Renn F	ceb4eec6ca	feat(board): gate CEO Approve & Start on board-review completion ... A board/coordination task stays pending throughout board review — that pending state is what hands it to Main PM on approval — so the CEO's Approve & Start button was live from the instant the task was created, before the Product Owner and Head of Marketing had reviewed anything. That let the CEO approve before the board finished. Persist a board_review_complete flag the orchestrator sets once BOTH board reviewers are done, and gate the button on it (the task stays pending). The same handoff emits the formal CEO notification, so the CEO gets an actionable signal instead of buried channel chatter. - alembic 021: add tasks.board_review_complete (default false) - TaskService.mark_board_review_complete: set the flag without leaving pending - orchestrator: flag the task + notify CEO once both reviewers go idle - panel: Approve & Start requires board_review_complete	2026-06-03 08:06:41 +02:00
3 people	110aaa7a77	Chore: v1 removal gateway canonical (#46) ... * chore(agent_sdk): remove dead /traceability/remind endpoint and reminder map The TRACEABILITY_REMINDERS dict and its /traceability/remind endpoint were keyed entirely on pre-gateway tool names (roboco_task_*, roboco_journal_*, roboco_message_send, roboco_session_create_for_tasks) deleted in the gateway cutover. The endpoint had zero callers; v2 enforces traceability server-side in the Choreographer. * fix(bootstrap,seeds): onboarding prompts call give_me_work(), not deleted roboco_task_scan() The startup prompt and the seeded cell/all-hands channel onboarding messages instructed agents to call roboco_task_scan() — a tool removed in the gateway cutover. Point them at the live give_me_work() flow verb. * fix: replace remaining deleted v1 tool names with gateway verbs Spawn prompts, onboarding strings, remediation messages, and comments still referenced pre-gateway tools deleted in the cutover (roboco_task_*, roboco_agent_idle, roboco_notify_*, roboco_message_send, roboco_session_create_for_tasks, roboco_journal_*, roboco_escalate). Rewrote each to the correct role-scoped gateway verb (give_me_work/i_will_work_on for workers, triage for PMs, i_am_done vs complete, notify/notify_ack, escalate_up, unclaim, i_documented, open_session, note). Updated one enforcement-message test that matched the old tool name by coincidence. * test: guard against deleted v1 tool names reappearing in roboco/ Scans roboco/ for the deleted pre-gateway tool names; excludes the orphaned roboco/agents/ subtree (removed in a later phase). * chore(exceptions): drop 8 unused pre-gateway exception classes + their tests LLMError, RAGError, AlreadyExistsError, TaskBlockedError, TaskClaimError, AgentNotAvailableError, AgentBusyError, NotificationPermissionError were never raised in production. SessionClosedError/DatabaseError are kept (live + tested). * chore(models): drop unused pre-gateway notification/channel/handoff factories Removes create_task_assignment/_blocker_escalation/_review_request/ _documentation_request/_priority_change/_alert/_broadcast, create_cell_channel/ _cross_cell_channel/_announcements_channel, create_handoff (+ HandoffParams), ProactiveContext, and A2APartType. The gateway choreographer builds these server-side now. Drops the matching dead-code tests. * chore(services): drop unused pre-gateway permission/messaging/audit/optimal/remediation methods These pre-gateway helpers (channel-permission checks, channel-membership ops, permission-denial audit hooks, doc ingestion, two remediation hints) have no production caller — the gateway role_config + enforcement layer replaced them. Drops the matching dead-code tests; live methods (send_message, the SESSION_* flow, log_task_action_denial, etc.) are untouched. * chore(orchestrator,ws,events,config): drop unused pre-gateway lifecycle/broadcast/roster symbols orchestrator: get_running_agents, is_agent_busy, queue_priority_work, get_all_instances (+ their OrchestratorAccessProtocol declarations in events.py). websocket: broadcast_new_message, broadcast_session_closed (no event type emits them). agents_config: ALL_PMS/ALL_DEVS/ALL_QA/CELL_PMS roster constants (ALL_DOCS stays — it gates docs-write workspace perms). * refactor(agents): delete orphaned pre-gateway agent subtree + dead organization model The Gateway/full cutover replaced the Python agent-class implementations with the server-side Choreographer; the classes survived only as a self-referential island. Removes roboco/agents/{base,mixins,factory,board,developer,documenter, pm,qa,orchestrator}.py and roboco/agents/factories/{board,cells,developers, documenters,pms,qa}.py, plus roboco/models/organization.py (Cell/Board/ Organization — used only by those factories). Keeps factories/_base.py (compose_prompt — the live prompt-layer composer the orchestrator calls at spawn) behind minimal package __init__ files. * chore(db): drop dead tasks.execution_log + outputs columns (migration 015) Both JSON columns had zero readers/writers in code, tests, and migrations — execution progress is tracked via progress_updates and artifacts via commits/documents. Removes the ORM columns, the Pydantic Task.execution_log/ outputs fields, the ExecutionLog/FileRef models (+ their __init__ exports), and the now-invalid kwargs from test fixtures. Migration 015 (down_revision 014_drop_pm_approvals) verified live: upgrade drops, downgrade re-adds. Apply on the NAS with 'alembic upgrade head' at next deploy. * chore(config): drop 16 unread Settings fields Verified unused (no settings.X, no self.X property use, no getattr-by-name): app_name, reload, workers, openai_api_key, secret_key, access_token_expire_minutes, algorithm, log_level, log_format, the four session_* limits, message_max_length, commit_subject_min_chars, commit_banned_words, agent_budget_sweep_interval_seconds. Removes the empty Logging + Sessions&Messages sections and orphaned .env.example vars. Kept: redis_db/redis_password (redis_url property), agent_sla_* (read via getattr in task_lifecycle), encryption_key, and all live thresholds. NOTE: commit_banned_words/commit_subject_min_chars and agent_budget_sweep_interval_seconds were feature-config never wired to their consumer (commit validator / budget sweep) — removed as dead, but flagged in case the intent was to wire them. * test(lifecycle): give i_will_work_on calls a substantive plan (#171 contract) The real-DB lifecycle tests called i_will_work_on with a 13-char plan and no risks/technical_considerations, so the substantive-plan gate (#171) rejected them with incomplete_input — failing on master. Supply a >=150-char plan plus technical_considerations and risks (mirroring tests/unit/gateway/ test_choreographer_dev.py). All 6 now pass; gate runs with no deselect. * feat(gateway): wire commit-validator thresholds to settings commit_subject_min_chars and commit_banned_words were config defined but never read — the gateway commit() gate used the validator's hardcoded module defaults. Re-add the two Settings fields and pass them through validate_commit_message in content_actions.commit(), so config is the source of truth (validator defaults remain the standalone/CI fallback). Adds wiring tests that monkeypatch settings and assert the gate honors them. * refactor(orchestrator): retire gateway_enabled flag; trigger_filter is unconditional The gateway_enabled Settings field gated only the trigger_filter spawn-cooldown (never the agent tool surface). Prod ran it on; the Phase-0 'legacy dispatch path' it guarded no longer exists. Remove the field + the early-return branch in gateway_pre_spawn_check so the cooldown runs for every spawn, drop the now-dead ROBOCO_GATEWAY_ENABLED from docker-compose.yml, and update the stale Phase-0 comments + cooldown test. The per-container ROBOCO_GATEWAY_ENABLED env (set by _append_manifest_args, read by agent_sdk to load the manifest) is unaffected. * refactor(api): relabel /api/v2 -> /api/v1 as the canonical gateway surface The gateway is the only agent API now, so the 'v2' label (with no v1) was misleading. Renames roboco/api/routes/v2 -> routes/v1, schemas/v2 -> schemas/v1 (+ the matching test dirs and test_v2_role_dep/test_schemas_v2_flow files), rewrites every /api/v2 path, routes.v2/schemas.v2 import, and v2-* router tag to v1, and refreshes the stale 'v2' comments/docstrings. The panel is untouched (it uses the unversioned /api/* REST routes). flow_server/do_server now POST to /api/v1/*. * docs(scripts): reset_runtime_state header matches actual SQL behavior The header claimed it preserves groups + journals, but the .sql wipes both (verified live: groups 6->0, journals 5->0; only agents/projects/channels survive). Correct the wiped/preserved lists to match. * refactor(gateway): extract _build_rich_plan to drop i_will_work_on under the complexity gate i_will_work_on was cyclomatic rank C (11) — one over the xenon --max-absolute B threshold — because of the five `x or default` fallbacks in the rich_plan dict. Move that dict into a small _build_rich_plan helper (behaviour identical); both methods are now rank B. make quality is fully green (xenon was its last failure; bandit already passed — its 34 findings are all LOW severity, filtered by -ll). * feat(foundation): add canonical CELL_TEAMS set; dedupe cell-subset literals * feat(db): add ProductTable + ProductProjectTable ORM (per-cell project map) * feat(task): add additive nullable product_id (ORM + model + DTO + create threading) * feat(task): thread product_id through create_subtask/route/response * feat(db): migration 016 — products, product_projects, tasks.product_id * fix(db): document migration 016 plan deviations (revision len, FK name) Two values in migration 016 intentionally diverge from the Task 2.4 plan literals; this strengthens the in-file justification so the deviations are self-documenting and verifiable. - revision id (plan line 623): the plan's 36-char "016_add_products_and_task_product_id" overflows alembic_version.version_num (VARCHAR(32)) — alembic upgrade head raises asyncpg StringDataRightTruncationError. Kept at 27 chars ("016_add_products_product_id") so Step 4's live round-trip stays green. - downgrade FK name (plan line 683): roboco/db/base.py sets a metadata naming_convention, so the FK upgrade() creates is "fk_tasks_product_id_products", not the Postgres default "tasks_product_id_fkey". The plan literal does not exist in the DB and would fail the downgrade with "constraint does not exist". Both verified via the live upgrade/downgrade round-trip on a throwaway DB. Issue 3 note: the prior commit (b896cac) also touched tests/unit/api/test_schemas_tasks.py (added product_id=None to the task_to_response stub). That line is load-bearing — task_to_response reads task.product_id (added in Task 2.3, commit 67afa6b) — and belongs to Task 2.3's scope; it is left in place because removing it breaks 4 tests and history is not rewritten. * refactor(db): trim migration 016 deviation notes to plan-faithful form Reverts the out-of-scope documentation expansion (commit 1a4f296), which was a second undocumented commit beyond Task 2.4's single plan-specified commit and only bloated the migration docstring/comments. The migration file now matches the plan-specified commit (b896cac) byte for byte: the two necessary deviations from the plan literals stay (revision id shortened to fit alembic_version.version_num VARCHAR(32); downgrade FK name follows db/base.py's metadata naming_convention), each kept to a concise inline note in the plan's header style. The Task 2.3-scoped test stub line (tests/unit/api/test_schemas_tasks.py product_id=None) is load-bearing — task_to_response reads task.product_id — and is left in place; history is not rewritten. Verified: live alembic upgrade head + downgrade to 015 round-trip on a throwaway DB drops products/product_projects/tasks.product_id cleanly, and make quality is green. * refactor(test): annotate db_session and drop type: ignore in migration 016 test Annotate the test_products_tables_and_task_fk_exist param as db_session: AsyncSession (imported under TYPE_CHECKING) and remove the # type: ignore[no-untyped-def] suppression, matching the typed db_session pattern used across tests/integration/. * feat(models): Product + ProductCreate/Update + ProductCellMapping (cell-validated) * refactor(models): minimize ProductCellMapping config override to use_enum_values The previous override re-declared validate_assignment, populate_by_name, and extra=forbid, which RobocoBase already supplies. Pydantic merges model_config across inheritance, so overriding only use_enum_values=False is sufficient to keep team as a real Team enum (required so team in CELL_TEAMS and enum identity hold for callers) while inheriting the rest of the base config. * fix(models): document ProductCellMapping use_enum_values override as plan-mandated Resolves SPEC-COMPLIANCE review notes for Task 3.1 (Product domain models). 1. The ProductCellMapping use_enum_values=False override is a deviation from a bare project.py mirror, but it is mandated by the plan's own Task 3.1 code: RobocoBase sets use_enum_values=True, which coerces team to the plain string "backend". The plan's Step 1 test asserts m.team is Team.BACKEND (enum identity) and the Step 3 validator formats its error with v.value, both of which require team to remain a real Team enum. The override is therefore necessary; this commit relabels the comment to cite the specific spec lines that force it instead of leaving it as an unexplained departure. Downstream Task 3.2 (_replace_cells / project_for) already tolerates either form and the ORM stores the same value regardless, so the override has no behavioral reach beyond the in-memory enum identity the plan's test checks. 2. test_product_model.py hoists 'from uuid import uuid4' to module level rather than inline (as the plan's verbatim Step 1 code shows) because the global Pylint PLC0415 rule (import-outside-top-level) forbids inline imports and there is no per-file-ignore for tests/unit/models/. The hoisted form is the only ruff-clean rendering of the plan's test; left unchanged here. 3. Task 3.1 landed across two commits (c616d95 create, 6ebad255 refactor) rather than the plan's single Step 5 commit. Earlier history is intentionally not rewritten; this single follow-up commit brings the model to its final spec-faithful, fully-documented state. * feat(service): ProductService CRUD + project_for per-cell resolver * feat(api): Product CRUD routes + schemas, wired into the app * fix(api): roll back and map cell-replacement IntegrityError on product update update_product replaced cells via ProductService._replace_cells without any try/except, so a duplicate-team cell (uq_product_projects_product_team) or a non-existent project_id (product_projects.project_id FK) raised an IntegrityError at flush, poisoning the AsyncSession and surfacing an unhandled 500 with no rollback. Wrap the update + commit in a try/except that rolls back and maps the UNIQUE violation to 409 and the FK violation to 422, mirroring create_product's rollback discipline. Add integration tests covering both client-error paths. * fix(api): map create_product cell-mapping IntegrityError to 409/422 create_product only caught the slug conflict ('already exists' in str(e)) and bare-raised everything else, so a cells entry whose project_id does not reference any project let the product_projects.project_id FK IntegrityError propagate out of the route as an unhandled 500. The matching update_product path was already hardened (uq_product_projects_product_team -> 409, FK violation -> 422); apply the same mapping in create_product so a bad project_id (or a duplicate-team cell) is a client error, not a server error. The slug conflict is now caught as ConflictError directly instead of via a broad except + string match. * feat(gateway): add optional project_id to delegate inputs/request/routes * feat(gateway): per-cell project routing (override -> product map -> parent) + product_id inheritance * feat(task): approve_and_start — reassign board task to Main PM (CEO gate #1) * feat(api): POST /tasks/{id}/approve-and-start (CEO gate #1, notes-required) * test(api): cover approve-and-start 404-before-notes-gate for missing task * feat(panel): Product types + Task.product_id * feat(panel): productsApi + hooks + tasksApi.approveAndStart * feat(panel): Products management screen + sidebar nav * feat(panel): Approve & Start button (CEO gate #1) * fix(api): narrow delete_product to IntegrityError + cover 204/409 delete paths * test(task): assert approve_and_start persists + appends the audit note * refactor(db): migration 016 names the tasks.product_id FK explicitly (house style) * fix(db): make migrations authoritative + self-heal orphan product tables init_db() no longer silently falls back to create_all when alembic upgrade fails. That fallback masked migration failures and, since create_all cannot ALTER an existing table, left the schema inconsistent — turning an unapplied migration 016 into a crash loop: 016's CREATE TABLE products failed, the upgrade rolled back, create_all re-created an empty orphan products table, and every later boot failed again on the now-existing table while tasks.product_id never got added. Now a migration failure is raised so the real error surfaces. Migration 016 additionally drops EMPTY orphan products/product_projects tables left by the old fallback before creating them, so an already-polluted DB self-heals on the next deploy with no manual SQL. Skipped in offline (--sql) mode; refuses to drop a table that holds rows. * fix(db): create_all is the schema source of truth; alembic for increments The Alembic chain is incomplete relative to the ORM — columns/tables like notifications.delivered_at and the RAG indexed_documents table have NO migration and have only ever been materialized by create_all. Tests don't catch this because the test DB is also built via create_all, so migrations are never exercised. The prior 'migrations are authoritative' init_db (and before it, the create_all-only-on-failure fallback) therefore left a migrate-only boot with missing columns/tables. init_db now reflects reality: - Fresh DB -> create_all builds the full current ORM schema, then stamp Alembic at head so later incremental migrations apply. - Existing -> run pending migrations (a real failure is raised, not masked), then create_all(checkfirst) to gap-fill any missing ORM tables. create_all cannot add a column to an existing table, so an ORM column added without a migration needs a fresh rebuild of that table to appear. * fix(db): migration 017 reconciles the Alembic chain with the full ORM schema For years the live schema was built by create_all, not migrations, so the chain drifted — tables/columns/indexes in the ORM had no migration (the indexed_documents table, notifications.delivered_at, ~15 indexes, plus timestamptz/server-default metadata). With init_db no longer masking that via a create_all fallback, a migrate-only boot was missing those objects. 017 was produced by 'alembic revision --autogenerate' against Base.metadata, reviewed, and verified: on a fresh DB, 'alembic upgrade head' (001..017) now reproduces the create_all schema EXACTLY — a re-run of autogenerate detects zero changes — and the 017 upgrade/downgrade round-trips cleanly. The migration chain is now complete: migrate-only and create_all converge. Also updates the init_db tests to assert the new behaviour (raise on an existing DB's migration failure; create_all + stamp head on a fresh DB) instead of the removed silent fallback. * feat(panel): Product picker in the New Task form (drives per-cell routing) The Products screen and Approve & Start button shipped, but the task-creation form had no way to attach a Product — so a human couldn't set product_id from the UI, which is exactly what drives per-cell project routing of delegated subtasks. Adds an optional Product dropdown (Advanced -> Git config) populated from useProducts(); 'None' falls back to the single project. * fix(db): seed data is preserved on a fresh DB (run migrations, not bare create_all) The previous fresh-DB path (create_all + stamp head) built the tables but never ran the migration chain, so migration-embedded SEED DATA was skipped — most visibly the AI providers seeded in 004. After a DB reset that left provider_configs empty, so PUT /api/providers/ollama-key 404'd (the handler raises NotFoundError when the Ollama provider row is missing). Since migration 017 made the chain reproduce the full ORM schema, init_db now runs 'alembic upgrade head' from base on a fresh DB — building every table/column/index AND running the seeds. Verified: a fresh upgrade head seeds both provider rows. Existing DBs still get migrations + create_all gap-fill. Updates the init_db fresh-DB test accordingly. * feat(task): project_id optional when a product_id is set (board fan-out tasks) A board task that fans out across cells via a Product has no single repo of its own — backend/frontend/ux_ui are each wrong, because the root coordinates and delegates. Forcing one arbitrary Project was broken design (flagged at design time). project_id is now nullable; a task must have project_id OR product_id: - TaskCreate model validator + a TaskService.create() invariant (covers every create path). - ORM/DTO/schema: project_id nullable; task_to_response uses to_python_uuid. - Gateway: a parent with only a product can delegate (guard now needs BOTH project and product to be None to reject); _resolve_subtask_project resolves each subtask from the product map and raises a clear error if a cell has no mapping and no parent project. - Migration 018 (tasks.project_id nullable), round-trip verified; fresh upgrade head still seeds providers. - Panel: Project no longer required once a Product is selected. - Removed the dead, never-called a2a create_task_from_message (it could only ever create a repo-less task) + its two coverage-only tests. make quality green; panel tsc/lint/build green. * Upgrade to Minimax M3 * fix(db): seed providers on existing DBs + correct enum casing Migration 004 created the modelprovider/assignmentscope enums and seeded provider rows in UPPERCASE, but the ORM (_str_enum) reads/writes the lowercase StrEnum .value — so a fresh migrate-from-base DB built an enum the ORM cannot read. Lowercase the enum labels and seed values in 004. Add idempotent migration 019 to (re)seed the Anthropic + Ollama Cloud providers with ON CONFLICT (name) DO NOTHING, so an existing DB whose provider_configs table was created by create_all (and never ran 004's seed) gets the rows on the next `alembic upgrade head` — fixing the /api/providers/ollama-key 404 without a volume wipe. * fix(tasks): let board/fan-out coordination tasks flow without a repo A coordination task (project_id NULL, product_id set) targets no repo of its own — it fans out to cell subtasks that each resolve a real project from the product's cell->project map. Several paths still assumed every task does git work and blocked it: - orchestrator: add _is_coordination_task() and exempt these tasks from the project/branch/git-token gates in _readiness_check_task, _readiness_gate, _check_stuck_conditions, _validate_task_for_spawn. - services/task.py: _ensure_branch_for_task returns "" (no branch) for a coordination task instead of raising; activate requires project OR product. This unblocks Main PM's i_will_plan claim, which otherwise raised before it could delegate the fan-out. - gateway: _pending_assignment_guard exempts advisory roles (product_owner/head_marketing/auditor) from the "assigned but never claimed" idle gate — they review without claiming, so they could not satisfy a claim-or-unclaim remediation. Adds focused unit tests for each. * fix(tasks): coordination tasks reach in_progress + team reflects Main PM The board->cells fan-out deadlocked: a coordination/fan-out task (product set, no project of its own) could be created and claimed, but start()'s claimed->in_progress transition hit validate_git_requirements, which still demanded a branch_name and raised GitRequirementError. So Main PM's i_will_plan never completed — it looped and never delegated. c961282 exempted _ensure_branch_for_task (branch creation) but missed this parallel git gate in the enforcement layer. - task_lifecycle.py: add GitContext.is_coordination; skip the claimed->in_progress branch_name gate when it is set. - task.py: populate is_coordination=(project_id is None and product_id is not None) in _validate_and_set_status; a branchless code task is still gated. - approve_and_start: set team=Team.MAIN_PM on hand-off so the task isn't left labelled team=board after it leaves the board (now assigned to main-pm). Adds a lifecycle-gate unit test and an end-to-end integration test that claims, plans, and starts a project-less coordination task. * fix(hooks): remove dead traceability hook + stale deleted-verb references The v1-removal cleanup (2cfbf39) deleted the /traceability/remind SDK endpoint but left the PostToolUse hook that curls it, so every gateway tool call 404'd and agents silently lost their traceability reminders. Remove the dangling hook (registration + TRACEABILITY_TRIGGER_TOOLS + Dockerfile COPY + the script); v2 carries per-verb guidance on the Envelope. Also correct two stale pre-gateway tool names in hook text: the budget loop-detector nudged agents toward the deleted roboco_task_escalate() (now unclaim()/i_am_idle(), which every looping role has), and an sdk-startup comment referenced roboco_task_scan/get. Extends the deleted-tool-name guard to scan docker/scripts/*.sh and to assert every $SDK_URL/<path> a hook curls is a route still served by the SDK — the check that would have caught this class (it lives in shell, invisible to mypy and the Python import graph). * fix(db): backfill ORM enum values the migration chain never added Several StrEnum values were added to the ORM over time without a matching `ALTER TYPE ... ADD VALUE` migration; 017 was autogenerate-derived and autogenerate does not detect added enum labels, so the drift survived. On a DB whose enum type predates the value, binding it raises at runtime — e.g. `invalid input value for enum notificationtype: "a2a_request"` on GET /api/notifications (list_system_notifications), and the same class for blockerresolvertype/handoffstatus/team. Migration 020 adds every drifted value idempotently (ADD VALUE IF NOT EXISTS — no-op when 009 already reconciled it). Runs on the next `alembic upgrade head`. Detected by comparing each ORM enum's values to the labels the migration chain produces; adds tests/unit/test_enum_migration_parity.py which renders the chain offline and fails on any future drift — the check that would have caught both this and the provider-enum bug. * fix(orchestrator): stop branch auto-block, board reassign, unblock livelock, agentless claims Cluster C1 — four coupled orchestrator/task-invariant defects: #18: a branch is created only at claim, so a pending, never-claimed code task legitimately has no branch_name. The stuck-detection sweep (pending-only) and readiness gate flagged that as "Task missing branch_name" and auto-blocked the task every tick, so it never dispatched. Centralize the gate in _branch_is_expected (status in claimed/in_progress/verifying, never a coordination task) and apply it in both _check_stuck_conditions and _readiness_check_task. #14: the main_pm -> product_owner escalation rung handed an in_progress descendant code task to the Product Owner (a board role) and marked it BLOCKED; the board has no verb to own code work, so the dev's finished work deadlocked. TaskService.apply_escalation (the single write primitive — covers both the gateway escalate verb and the HTTP escalate route) now diverts a descendant code task targeting a board/advisory role: it releases the task to PENDING for a role-matched cell claim instead of stranding it. #17: a blocked task reassigned to Main PM kept respawning the ex-assignee cell PM to unblock it, but the assignee-only pre-unblock note returned not_authorized — a livelock. _dispatch_blocker_work now dispatches the task's CURRENT PM/board assignee (the unblock authority), falling back to the cell PM only when no PM/board holds it. Also: a branchless coordination parent yields no valid merge target — resolve_parent_branch now falls back to the child's own project default branch (e.g. master) via TaskService.project_default_branch_for_task, and _check_parent_branch_ready no longer blocks a child on a coordination parent's non-existent branch. #19: a task left claimed/in_progress with an assignee but no running container was invisibly stuck (only PENDING tasks get fresh dispatch; the heartbeat reaper can't see a freshly-seeded claim). New _dispatch_claimed_without_agent net: after a short grace window it respawns the assignee, or releases the claim to pending (lifecycle-safe via unclaim_for_reaper) when the assignee is unknown. New config ROBOCO_CLAIMED_NO_AGENT_GRACE_SECONDS (default 120). * fix(gateway): tolerant note verb + lock evidence do-tool invariant #15: the note verb no longer hard-rejects thin decision/reflect payloads. List-typed fields (options, consequences, next_steps) coerce a lone scalar into a one-element list at both the NoteRequest schema (mode=before validator) and the service layer; missing narrative fields default to a visible placeholder instead of returning incomplete_input. The note is always recorded, preserving audit value, and a well-intentioned note can no longer trip the do-server 3-strikes circuit breaker. Widen the agent-facing do_server.note hints to accept list-or-scalar and refresh the docstrings. #8: add regression coverage locking the invariant that every role's do_tools carries evidence (role_config + developer spawn manifest). The current source already registers mcp__roboco-do__evidence for developers end-to-end; the report stemmed from a stale deployed build, and the tests prevent silent regression. * fix(gateway): allow UX devs to receive design tasks; surface delegation rules to cell PM The UX/UI cell's developers (ux-dev-1/ux-dev-2, Role.DEVELOPER on Team.UX_UI) ARE its designers, but _validate_assignee_task_type rejected task_type='design' for every DEVELOPER, blocking the UX cell's normal design delegation. Allow 'design' for UX-team devs only; backend/frontend devs stay rejected (design routing belongs to the UX cell). The orchestrator already dispatches a developer for a design task (_dev_dispatch_role_matches returns True), so this creates no orphan like the documentation case. Replace the static Cell-PM 'pass planning' remediate with a per-assignee hint so a dev/design mis-type gets a developer-class next-step instead of an off-topic planning hint. Surface the three delegation guardrails in the cell-PM prompt so PMs stop probing them by trial and error: valid task_type per assignee (incl. design for UX devs), documentation auto-creation (non-delegatable), and the sequential single-active code-spine. Fix the delegate-row task_type list (documentation is NOT delegatable) and update the lifecycle spec description; regenerate the lifecycle artifacts. * fix(orchestrator): improve agent briefings for handoff consumption, product/project model, and workspace/secret hygiene Main PM (roles/main_pm.md): - Require reading the upstream Product Owner / Head of Marketing handoff (their decision/reflect journal entries + task description) BEFORE doing any own research or calling i_will_plan, so the Main PM builds on the Board's analysis instead of duplicating it. Added a dedicated section, hardened workflow step 1, and added an anti-pattern. - Add a 'Products vs Projects' section: a Product fans out to one Project per cell; those Projects may be the SAME repo (monorepo subtrees) or DIFFERENT repos (multi-repo). The Main PM coordinates across them and must not assume one repo or call a monorepo subtree 'a separate repo'. Names the Prompter monorepo case (github.com/rennf93/roboco). Developer (roles/developer.md): - State the exact workspace path convention /data/workspaces/<project-slug>/<team>/<agent-slug>/, that the cwd is already set there, to stay inside the own cell workspace, and to not probe/guess the path (ls /, find /). - Sanctioned secret handling: env/printenv is bash-guard denied and reveals nothing; needed secrets arrive via the task description, else i_am_blocked so the PM supplies them. Added matching anti-patterns. Tests: add tests/unit/agents/test_briefing_cluster_c4.py asserting the composed system prompt (the text mounted into agent containers) carries each of the above. * fix(orchestrator): board review involves PO+HoM and notifies CEO Cluster C5 (#2, #4): a board/coordination task was reviewed by the Product Owner alone, and the CEO got no formal signal when the review finished — only buried channel chatter — so the Approve & Start handoff was invisible. #4 — Board review is now a two-reviewer gate. _handle_board_assigned_task dispatches BOTH the Product Owner and the Head of Marketing (one-shot each), regardless of which one holds assigned_to, and the unassigned board-routing path delegates here instead of claiming + spawning the PO alone. Board tasks stay pending/unassigned for the CEO's Approve & Start. The board prompt now makes the PO+HoM pair-review model explicit (HoM owns the UX/positioning dimension). #2 — Once BOTH reviewers have finished (dispatched and no longer active), the orchestrator emits exactly one formal CEO notification via NotificationService.send_board_review_complete_notification (APPROVAL type, ack-required, carrying related_task_id) so the handoff is an actionable signal. One-shot per task; a notification failure clears the guard so a later tick can retry. To let the non-assignee board member record its review note on a task held by the other board member, content-action ownership now exempts a board role posting to a board/coordination task (project_id is None, product_id set). The exemption is narrow: it does not widen ownership for any other role or any project-backed task. Unit tests cover both reviewers dispatched, one-shot dispatch, the CEO notification fired exactly once when both are done (and not before), the retry-on-failure path, the notification builder, and the board co-review ownership exemption (allowed for board+coordination, blocked otherwise). * fix(workspace): install dev deps post-clone + raise git commit timeout for large changesets Cluster C6 (#10, #13, #12-investigate). #10: per-agent workspace clones never had the project's dev dependencies installed, so the make-quality gate (ruff/mypy/pytest for Python, the TS toolchain for the panel) was missing and devs re-downloaded tooling per task. WorkspaceService now runs the project's install after cloning (`uv sync` for Python, `pnpm install`/`npm ci`/`npm install` for Node/TS, detected by manifest/lockfile). Idempotent via a lockfile-digest marker under .git/ so a re-entry with unchanged lockfiles is a no-op; also runs on the healthy short-circuit so pre-existing clones get backfilled. Gated by workspace_install_dev_deps (default on) with workspace_dep_install_timeout_seconds. #13: the gateway commit verb timed out on the large panel changeset because every git op used the hardcoded 30s _GIT_TIMEOUT and each call also re-walks the tree to chown. _run_git now takes a per-call timeout override sourced from settings (git_command_timeout_seconds default); the staging + commit ops in commit() and create_commit() use the longer git_commit_timeout_seconds (default 180s). httpx REST timeouts unchanged in value. #12 (investigate only — no push, no history change): the clone base ref is NOT hardcoded; it already comes from project.default_branch threaded through git.get_workspace -> ensure_workspace -> _clone_repo (git clone --branch). The stale-base problem is a deploy/process issue (GitHub master is behind the deployed migration chain), resolvable only by pushing the chain to master. The default_branch column is the existing configurable lever. * fix(panel): gate Approve & Start to board coordination tasks; stop 404 storm on closed sessions CEO gate #1 button only renders for a PENDING board coordination/fan-out task (no project_id, has product_id) — the board-reviewed handoff that approve_and_start accepts — instead of every PENDING board-team task. approve_and_start requires PENDING (it re-targets to Main PM without a status change), so the gate stays on PENDING rather than the unrelated end-of-work awaiting_ceo_approval state. Session/message reads now treat a 404 as terminal and never retry it: a reaped session is gone for good, and retrying every dead session-id is what produced the growing 404 storm on GET /api/messages. The transcript loads once (staleTime Infinity, no focus/reconnect refetch) so closed sessions stay viewable without re-polling. * fix(orchestrator): role-correct respawn prompt, throttle agentless dispatch, broaden #14 guard #19 wrong-role prompt on respawn: _get_prompt_for_agent fell through to the developer prompt for every non-dev/doc/qa role, so a respawned PM or board agent was told to write code and call verbs it does not own. Route by the agent's actual role through the existing per-role prompt builders (developer/qa/documenter/cell_pm/main_pm/product_owner/head_marketing/auditor). Both callers benefit; _spawn_pending_dev only ever passes developer/documenter/ unknown, so its behavior is unchanged. #19 spawn-burst: _dispatch_claimed_without_agent looped over every agentless claimed/in_progress task and could spawn many containers in one tick. Break after the first respawn so a restart can't trigger a burst, matching every sibling dispatcher. The release-to-pending path spawns nothing and keeps draining stale unknown claims. #14 guard scope: _is_descendant_code_task only matched CODE, so a descendant DOCUMENTATION or DESIGN task escalated to a board/advisory role was still stranded on a role with no verb to own it. Rename to _is_descendant_executable_task and broaden to CODE/DOCUMENTATION/DESIGN — the cell-executed types a board role cannot own. PLANNING/RESEARCH/ADMINISTRATIVE route to a PM, not a cell agent, and are left unchanged; root tasks are still reviewed up the chain. * fix(docker): add node+pnpm to orchestrator so it pre-installs frontend cell deps * Added .github workflows * refactor(services): extract helpers to keep install_dev_deps + developer task-type check under the xenon complexity gate * chore(github): add launch kit — CI, GHCR release, labels, templates, funding, dependabot npm, community docs * chore(github): bump_version — drop unused noqa, fix datetime UTC import --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com>	2026-06-03 06:35:03 +02:00
Renn F	eed4551497	feat(alembic): B2 drop unused pm_approvals Task column ... Smoke run analysis initially flagged three Task fields as unused (pm_approvals, quick_context, proactive_context). A follow-up audit found quick_context (stores original_developer marker + doc notes + PR creator + escalation notes) and proactive_context (RAG injection) are actively used. Only pm_approvals is truly orphaned. Migration 014 drops pm_approvals; downgrade() recreates it if ever needed. The two false-positive fields stay untouched. Spec ref: docs/superpowers/specs/2026-05-12-post-smoke-3-fixes-design.md section B2 (re-scoped 2026-05-12).	2026-05-12 04:03:21 +02:00
Renn F	9aa30fb945	100% Coverage	2026-05-06 21:02:31 +02:00