* fix(prompts): point agents at Makefile, drop raw uv run instructions
backend.md:23-26 literally instructed raw uv run ruff/mypy/pytest (copied from
the human-facing CLAUDE.md), so agents bypassed the Makefile's UV_NO_SYNC=1 +
private UV_CACHE_DIR venv-corruption guard. Replace with make targets across
backend/developer/qa/cell_pm + a universal rule in base.md. Regenerate verbs.md
from the updated regen script (baked instruction now make foundation-check) and
align the Makefile drift message. Ships with the bash-guard deny in the next
commit so agents don't loop fighting the guard.
* feat(bash-guard): deny raw uv/pip/conda/poetry, point at Makefile
When a Makefile is present, deny raw uv run/uv pip/uv lock/add/remove, pip/pip3
install/uninstall, conda install/create/run, poetry run/install/add and remediate
to make quality/gate/lint/test. Skipped when no Makefile (Makefile-less projects
not blocked). ROBOCO_GUARD_SKIP_PM=1 (grok path) nudges exit 0 instead of the
run-canceling exit 2. Overrides the prior bare-uv-run-allowed stance by CEO
direction; the /app-targeted blocks above keep priority.
* feat(grok): deny raw uv/pip/conda/poetry via native --deny + PM-skip nudge
Add _RAW_PM_DENY (uv run/pip install/lock/add/remove, pip/pip3 install, conda
install/create/run, poetry run/install/add) to _deny_rules so grok's graceful
native --deny blocks raw package-manager commands (model adapts to make, run
continues — unlike a hook deny which cancels the run). The bash-guard hook
keeps the compound-command fallback (cd x && uv run) and nudges exit 0 there via
ROBOCO_GUARD_SKIP_PM=1 in the grok hook env, never canceling.
* test(bash-guard): align existing tests with W1 Makefile-gate policy
Raw uv run / pip install are now Makefile-gated (W1, CEO item #15), so two
existing bash-guard invariants reverse:
- test_allows_pytest_even_if_suite_uses_requests keeps its HTTP-injection
allow-path intent but uses bare `python -m pytest` (raw `uv run` is now
denied); the deny case is covered by test_bash_guard_makefile_guardrail.
- test_allows_pip_install_in_workspace -> test_denies_pip_install_when_makefile_
present: a workspace clone carries a Makefile, so bare pip install is now
denied -> agents use `make` / `uv sync --extra dev`. Makefile-less skips
stay covered.
Gate: 12994 passed, 439 skipped, 94.81% cov (DB env :55432 user renzof);
the lone flaky integration error passes in isolation (DB-state race, not W1).
---------
Co-authored-by: Renn F <rennf93@users.noreply.github.com>
* fix(security): lock down shared Claude Code credential mount + curl|sh RCE
Audit of Claude Code capabilities reachable inside a spawned agent
container turned up two live gaps against the shared harness state:
- Every agent container bind-mounts the host's ~/.claude (OAuth store) and
~/.claude.json read-write (_build_mount_args) — the shared subscription
auth used by the whole fleet. Nothing denied the native Read tool or the
bash-guard hook from reading .credentials.json / .claude.json, so any
role could exfiltrate the harness's own Claude Code auth. Deny both at
the settings.json layer (absolute // form, per the #167 gotcha) and in
the bash-guard hook's credential-exfil checks (cat/grep/source/base64/
interpreter one-liners), mirroring the existing .netrc/.git-credentials
treatment.
- The bash-guard hook only blocked curl/wget to github.com or internal
hosts; `curl <any other host>/install.sh | bash` (or `bash <(curl ...)`,
`eval "$(curl ...)"`) executed untrusted remote code unchecked. New
checks deny piping a fetch into an actual shell (sh/bash/zsh/dash/ksh)
while leaving non-executing consumers (tar, jq, -o file) untouched.
Also add --disable-slash-commands to every container agent spawn: skills
resolve independently of the --tools allowlist, so a contaminated shared
~/.claude could otherwise leak host skills/plugins into an agent session.
No RoboCo role's workflow uses a Claude Code skill.
64 -> 78 shell bash-guard cases, 54 -> 71 pytest bash-guard cases, plus a
new 5-case settings/CLI test module. ruff/mypy/xenon B clean.
* docs: changelog for the CC capability lockdown
---------
Co-authored-by: Renn F <rennf93@users.noreply.github.com>
The agent-base image baked VIRTUAL_ENV=/app/.venv globally (since 8e201901,
conventional venv-baking, no targeted rationale), so every bare uv run in a
workspace clone warned 'VIRTUAL_ENV=/app/.venv does not match project .venv
and will be ignored' on every gate run. uv used the workspace .venv correctly
(the warning was noise), but it flooded every agent transcript.
The load-bearing MCP/SDK pin is UV_PROJECT_ENVIRONMENT=/app/.venv + --no-sync
(added in #179, the actual startup-stall fix), NOT VIRTUAL_ENV — so removing
VIRTUAL_ENV does not regress #179. The gateway tools stay on PATH.
Verified by building the image (roboco-agent-base:venv-verify) and smoke-
testing in a real container: (1) VIRTUAL_ENV is empty, python resolves to
/app/.venv/bin/python via PATH; (2) bare uv run from /app imports a roboco
module with no warning; (3) the MCP-launch path — cwd with its OWN .venv —
UV_PROJECT_ENVIRONMENT=/app/.venv uv run --no-sync resolves to /app/.venv/
bin/python, overriding the cwd .venv (the exact MCP-server invariant), no
warning; (4) grep for 'does not match|will be ignored' across all output is
empty. bash-guard-tests.sh 64/0, ruff clean, test_workspace_uv_resolves_clone_venv 3/3.
Ships on next agent-image rebuild + redeploy.
* Updated uv.lock
* Bunch of fixes we need to verify first..
* feat(megatask): per-cell project map root-subtasks (multi-project, multi-cell)
A MegaTask root-subtask can now target an ad-hoc per-cell project map — a
third targeting shape that mirrors the existing product fan-out root. In
RoboCo a project is per-cell (ProjectTable.assigned_cell); a monorepo is N
per-cell projects sharing one git_url. So 'multi-cell' IS 'multi-project',
and a task may mix per-cell projects across products or include OSS-library
projects not in any product.
Storage: migration 052 adds task_cell_projects (mirrors product_projects;
unique per (task, team)). TaskTable gains a cascade-delete cell_projects
relationship; TaskCreateRequest / TaskCreate / Task response carry the map.
Policy: batch.is_branchless_coordination + is_valid_batch_shape gain a
has_cell_projects param — a root-subtask targets exactly one of project /
product / cell-map; the umbrella still targets none. TaskService passes
has_cell_projects at every predicate call site and persists the rows in
create(). _ensure_branch_for_task cuts feature/main_pm/{root} per distinct
project in the map (via _distinct_projects_for_task); _require_target_or_umbrella
and _validate_batch_membership accept the map shape.
Fan-out: every distinct_project_ids site (task.py branch creation, routes
_project_for_complete + _resolve_project_for_merge, orchestrator
_ambient_projects_for_task, pr_review._project_slug_for, git._project_for_task)
generalizes to first-distinct-project-of-map-or-product. Choreographer
_resolve_subtask_project resolves a delegated subtask's cell from the parent's
cell map. The product-scoped _slugs_for_product intake helper is unchanged.
Intake: prompter._draft_cell_map extracts the per-cell map from the_work[].
_validate_batch_scope counts distinct projects across all drafts' cells
(>=2 min stays; one 2-cell draft satisfies it). create_task_from_draft
persists cell_projects for >=2-cell drafts (project_id/product_id None),
collapses a 1-cell map to the single-project shape, and leaves single-cell
top-level project_id drafts unchanged. _resolve_owning_team routes a
multi-cell map to Main PM (coordination root, like a product root — a cell
PM can't delegate cross-cell). propose_draft/propose_batch tool descriptions
declare the per-cell project_id (both Claude SDK + grok runtimes).
The umbrella stays branchless / pure-coordination / submit_root-rejected;
the CEO-escalation pr_number gate is not widened (the map root is
is_umbrella=False, mirroring a product root, so submit_root supplies it).
Single-cell root-subtasks and everything below them are byte-for-byte
unchanged. Un-run MegaTask waves (multi-cell drafts) become runnable.
* [feature] Panel per-cell project picker + pnpm format infra
MegaTask root-subtasks can fan out across cells (be+fe, fe+uxui). Since a
RoboCo project is per-cell (ProjectTable.assigned_cell), a monorepo is N
per-cell projects sharing one git_url — so multi-cell IS multi-project. The
batch-review card now shows one project Select per the_work entry, scoped to
that cell's repos, instead of one Select bound to a single top-level
project_id. confirmBatch validates each cell's project is in scope and the
batch still spans >=2 distinct projects.
- prompter.ts: CellWork gains optional project_id (the per-cell picker seam).
- batch-review-card.tsx: per-cell Selects (one per the_work entry), scoped to
the cell's projects; legacy single-cell drafts keep the one-Select path.
- use-prompter.ts: updateBatchDraftProject edits per-cell (entryIndex); confirmBatch validates every cell; batchFromEvent parses per-cell map.
Also adds the missing pnpm format infrastructure (the panel had no formatter
at all): prettier devDep + .prettierrc.json (default-style config: 80-col,
double-quote, semi, trailing-comma-all) + .prettierignore, plus format /
format:check scripts. Only the 3 changed files above were reformatted; the
~222 pre-existing non-compliant files are left untouched (a wholesale reformat
is a separate explicit decision, not bundled into this feature).
* [fix] MegaTask verification: migration 052 enum + async cell-map read
Two real bugs surfaced running the full gate against a containerized
Postgres (and the orchestrator boot log):
1. Migration 052 crashed a real orchestrator boot with
'type "team" already exists'. The generic sa.Enum(create_type=False)
does NOT set the postgres enum's create_type attribute, so op.create_table
(checkfirst=False) emitted a redundant CREATE TYPE against the pre-existing
team enum. Switched to postgresql.ENUM(create_type=False) — the postgres-
native enum whose create_type _check_for_name_in_memos actually reads, so
the CREATE TYPE is suppressed. Verified: 051->052 upgrade against a DB where
the team enum pre-existed (the exact path that crashed) now succeeds;
downgrade 052->051 drops the table and preserves the shared enum; fresh
upgrade head clean. (Migration 016 has the same latent sa.Enum pattern but
never re-runs in prod, so it's noted, not touched here.)
2. _ensure_branch_for_task read task.cell_projects (lazy=selectin to-many)
directly, tripping MissingGreenlet on a freshly-created/unqueried task —
which then poisoned the async session (PendingRollbackError). Replaced with
_task_has_cell_map: peeks InstanceState.unloaded (no IO) and reads the
already-loaded map, falling back to an awaited count query only when the
relationship is genuinely unloaded. Non-ORM stubs route to the plain
attribute. Fixes 2 integration tests; the 6 cell-map unit tests still pass.
Also: typed the self stub as Any in test_choreographer_subtask_project
(mypy tests/ wants Choreographer, not SimpleNamespace) — the codebase idiom.
Gate: ruff format/check clean; mypy roboco/ + tests/ clean; full pytest
10371 passed / 388 skipped against containerized pgvector:pg16; vulture clean.
Pre-existing xenon C-rank on reassign (from prior commit 19a474d3, not this
feature) still blocks make quality — surfaced separately.
* [refactor] Extract reassign board-advisory diversion helper (C→B complexity)
`reassign` in roboco/services/task.py hit xenon absolute complexity 11 (a
C-rank block), failing `make quality`'s --max-absolute B gate. The C-rank
originated in 19a474d3 (pre-existing, not this feature branch's work).
Extract the board/advisory → cell-task diversion into
`_maybe_divert_board_advisory_reassign` (complexity 4, A). reassign drops to
9 (B); behavior is byte-for-byte preserved — the helper runs the same
guard + pool diversion + log, returning the diverted task or None so the
caller falls through to the normal handoff. Whole-repo xenon exits 0; the 159
reassign / board-guard tests pass.
Unblocks `make quality` on feature/metrics-granularity.
* [fix] migration 016: postgresql.ENUM(create_type=False) for reused team enum
016_add_products_and_task_product_id used `sa.Enum(..., create_type=False)`
for the reused Postgres "team" enum — the same latent defect that crashed
052 on a real orchestrator boot. On the generic `sa.Enum` the
`create_type` kwarg is silently dropped, so `_check_for_name_in_memos`
never sees it and `op.create_table` (checkfirst=False) emits a redundant
`CREATE TYPE team` that fails with "type 'team' already exists" against a
DB where the enum pre-exists.
Switch to the postgres-native `postgresql.ENUM(..., create_type=False)` —
its `create_type` is a real attribute the guard reads, so the CREATE TYPE
is suppressed (and DROP TYPE on downgrade too). The member list is inert
under create_type=False (it never creates/alters the type), so it stays at
016's original six, reflecting the enum as it stood then, not the
later-widened set.
This never crashed in prod because 016 is never re-run (alembic_version is
past it), but it's the same defect class. Verified on the real boot path:
upgrade to 015 in process A (team enum created by 001), then `upgrade head`
in a fresh process B — 016 applied clean, no DuplicateObjectError; downgrade
016->015 clean, shared team enum preserved.
See project_migration_enum_create_type_gotcha.
* [chore] panel: prettier reformat across the codebase
Apply `pnpm format` (prettier 3.8.5, 80-col / double-quote / semi /
trailing-comma-all) to the 223 pre-existing panel files that predated the
prettier infra added in cb5365a4. Pure formatting — no semantic changes:
multi-line arrays/objects collapsed where they fit, trailing newlines added
(.prettierrc.json), import grouping unchanged.
Verified: `pnpm format:check` clean, `pnpm lint` clean, `pnpm typecheck`
clean, `pnpm test` 113/113 pass (7 files).
* Bunch of runtime fixes for MegaTask and other issues
* Fix different project same PR number collision problem
Fix (two layers):
1. Root cause — pr_merge and rebase_pr_for_task now take a required project_id and scope the lookup where(pr_number == X AND project_id == Y). Required so no caller can forget — the bug class can't recur. All 4 call sites updated (choreographer cell_pm_complete, the rebase-retry, the superseded close_pull_request now passes project_id, and _verb_runner._do_pr_merge).
2. Crash guard — _finalize_cell_complete None-checks the complete() return and returns a clean invalid_state envelope (with a remediate hint) instead of dereffing None → 500 → respawn loop.
* Fix: Make main_pm + task_type=code impossible
* Fix Main PM needs revision can't re delegate
* [chore] Bump local LLM glm-5→glm-5.2 + swap Ollama fleet defaults off minimax
- llm_catalog: OLLAMA_DEFAULT_MODEL minimax-m3:cloud → kimi-k2.7-code:cloud;
role defaults kimi-k2.6→kimi-k2.7-code, developer minimax→kimi, product_owner/
ceo kimi→glm-5.2, documenter glm→kimi; GLM 5.1→5.2 comment fix.
- config + .env.example + docker-compose{.yml,.yaml,.registry.yml} + docs +
memory_distiller + optimal_brain: glm-5:cloud → glm-5.2:cloud.
- panel ai-routing-card: typed SelfHostedModel/boolean annotations; drop the
stale "Minimax M3 default" string (default is now catalog-driven).
- tests: glm-5:cloud → glm-5.2:cloud in pricing + rate-limit-retry fixtures.
* [fix] submit_root: hard unchanged-PR gate stops the pr_fail re-submit loop
The 2026-06-27 infinite pr_fail loop: a Main-PM root (PR #139) was pr_fail'd,
routed to needs_revision, and re-submitted byte-identical → awaiting_pr_review
→ pr_fail again, forever. The prior hint/a2a steer was ignored by the weak
coordinator model — hints don't stop a model that won't read them. A HARD gate
refuses the re-submit when the assembled root PR's head SHA is unchanged since
the last pr_fail (no new cell work → identical diff); a different SHA ⇒ the
branch advanced ⇒ allow. Every ambiguous case fails open (no prior fail, no
recorded SHA, no pr_number, unresolvable slug, git error, closed PR) — only the
exact-unchanged case is hard-blocked.
- content/models: PrReviewContent.head_sha (optional; JSON col → no migration).
- git: get_pr_head_sha (GitHub pulls API; None on any failure → fail-open).
- pr_gate: pr_fail captures head_sha into the verdict record; pr_pass does not.
- _impl: submit_root runs _submit_root_unchanged_pr_guard after _submit_up_guard;
_current_root_pr_head_sha resolves slug + current SHA (fail-open).
- pr_review: extract module-level resolve_task_project_slug, shared by the mixin
and the gate helper (_LegacyChoreographer reaches it via cast to the
ChoreographerHelpers typed view — it doesn't inherit the helpers mixin).
- tests: test_submit_root_unchanged_pr_guard (11 — refuse/allow/6 fail-open/3
capture-side, mypy-clean via cc:Any spy idiom, zero type:ignore) +
test_pr_gate_notifies_pm capture-path stub.
* [chore] mypy tests/: clear all 15 pre-existing type errors so make quality can go green
The branch tip had 15 mypy tests/ errors in files this bundle did not author,
which blocked CI's make quality mypy step (mypy roboco/ tests/) regardless of
the bundle's own commits. Pre-existing is still existing — fix every one:
- test_schemas_v1_flow.py (8): the StrList coercion tests intentionally pass
SDK-nested list-of-strings input ([[['...']]], {'item':{'$text':'...'}}, int,
dict). Annotate those literals as list[Any] locals so mypy accepts the
coerce-able shape; the StrList BeforeValidator still flattens to list[str] at
runtime. No type:ignore.
- test_pr_gate_records_verdict.py (3): notes_structured is dict|None; narrow
with 'assert t.notes_structured is not None' before indexing (the existing
pattern at line 90).
- test_pr_review_hand_format_guard.py (1 site, 2 errors): the _verb_runner()
spy assertion — use the cc: Any = c alias idiom so assert_not_awaited
resolves; drops the now-unused type:ignore[union-attr].
- test_pr_gate_notifies_pm.py (1): drop the unused type:ignore[method-assign]
on the a2a.send reassignment.
- test_content_models.py (1): narrow coerced with isinstance(coerced,
PrReviewContent) before reading .issues (the base _Content lacks the field).
Gates: rm -rf .mypy_cache && mypy roboco/ tests/ = Success (855 files);
ruff check + format clean; 5 affected suites = 40 passed.
* [fix] fail_qa routes needs_revision back to the dev, never the pool
A dev task in needs_revision must go back to the developer, never the
pool. The pool path let a cell PM re-claim the revision (PMs can claim
needs_revision) — the live 2026-06-27 'needs revision on a dev task sent to
the cell PM' bug.
fail_qa's original_developer marker is the fast path, but it is
unreliable in practice (live observation: never persisted), so the
unassign else-branch was the load-bearing path and it dropped the task
into the pool. Add a work-session fallback (_resolve_revision_dev) that
resolves the developer who actually worked the task — the most recent
work session whose agent is a developer, the QA's own session excluded
— and reassigns to that dev instead of unassigning. Only unassign when
no developer ever touched the task. Self-heals the marker so a
subsequent re-fail takes the fast path and the QA-review index
attributes the work correctly.
* [feature] delegate carries dev-task collision surface (sequencing S1)
The cell/main PM's delegate verb now carries the dev-task collision
surface (intends_to_touch / adds_migration / touches_shared) and an
explicit depends_on override through DelegateRequest -> DelegateInputs
-> _create_subtask_from_inputs -> create_subtask, and create_subtask
forwards sequence / dependency_ids / batch_id / surfaces into the
prepared TaskCreateRequest instead of dropping them (the base create
already persists them at task.py:878-884).
This is the plumbing for the multi-level sequencing model edge kind 3
(dev-task collision DAG). Previously a dev task delegated with a
collision surface or an explicit dependency lost it before persistence
— dependency_ids was always [], so the only dev-task ordering was the
weak assignee-keyed spawn barrier (the live 2026-06-27 out-of-order
break: 40842957 started before 9b3682b8's PR merged). Phase S2 runs
SequencingService over the surfaced siblings and wires the DAG via
add_dependency.
* [feature] wire dev-task collision DAG at cell-PM delegation (sequencing S2)
Pure dev_task_collision_edges in sequencing.py turns a parent's surfaced
siblings into (depends_on_id, task_id) pairs via SequencingService. TaskService.
wire_sibling_collision_dag wires them through add_dependency (idempotent). The
choreographer calls it after each dev-task delegate so the sibling collision DAG
is built incrementally as the cell PM decomposes — file-overlap serializes,
migration chains, shared-last; stable (priority, sequence) ordering keeps edges
from flipping into reverse cycles on re-runs.
* [feature] wire cell-task wave chain + by-osmosis edge (sequencing S3)
Kind 2 (cell-task wave chain): a new cell-task under root-subtask UT_n
depends on every cell-task under every root-subtask in UT_n.dependency_ids
(the kind-1 wave-chain edges), so its branch carries the previous wave's
merged cell work. Re-derived from the root-subtask's deps, not the cell-task's
own dependency_ids (which also carry UX/product-fanout edges the by-osmosis
edge must not pick up). A root may fan to several cell-tasks (different cells),
so the previous wave's cell-task is a SET.
Kind 4 (by-osmosis): the first dev task (sequence 0) under a cell-task depends
on each predecessor cell-task's tail (max-sequence) dev task, so the new wave's
first branch carries the previous wave's fully-merged tail. Subsequent dev
tasks inherit the tail via kind 3 or the merged base.
Both wired from _create_subtask_from_inputs, dispatched on parent.team
(MAIN_PM -> kind 2; cell team -> kind 4). Pure helpers
(cell_task_wave_chain_depends_on, by_osmosis_tail_dev_tasks) unit-tested in
test_sequencing.py; TaskService methods integration-tested. Idempotent +
best-effort throughout (add_dependency dedupes; missing predecessors are
no-ops). Also fixes a latent mypy-tests gap (estimated_complexity required on
direct TaskCreateRequest calls in the S2 tests).
* [feature] sync_branch dev verb — gate-level branch rebase (Phase B1)
Raw shell git is denied to agents (Bash(git:*) base deny), so a developer
whose branch fell behind its base had no gate-level rebase — only the
CEO/PM-only /rebase HTTP route. sync_branch is the dev verb that wraps the
rebase through the gate (traced + evidenced), so the 'everything goes through
the gates' invariant holds.
- lifecycle: IntentSpec sync_branch (dev-only, ownership-gated, composes=(),
git-only — no DB transition); _next_hint_synced helper.
- GitService.sync_task_branch: rebase task.branch_name onto its resolved base
via rebase_onto_base (fetch + rebase + force-with-lease push).
- Choreographer.sync_branch + _sync_branch_preflight_rejection: not_found /
unknown-role / spec-gate / no-branch / protected-base guards, then the git
op; conflicts abort (no force-push) and steer to resolve-by-hand; git failure
steers to i_am_blocked.
- HTTP route /api/v1/flow/developer/sync_branch + SyncBranchRequest schema.
- MCP tool sync_branch(task_id) + _TOOLS registration (manifest auto-propagates
via intents_for_role(Role.DEVELOPER)).
Tests: intent spec (5), choreographer handler (8: happy/conflicts/not_found/
not_authorized/no-branch/protected-base/git-failure/audit), route (1), MCP (1).
ruff + mypy roboco/ tests/ clean; unit suite green (DB-fixture errors env-only).
* [feature] i_am_done behind-base submit gate (Phase B2)
A sibling's PR merging into the parent branch while a dev worked leaves the
dev's branch behind its base — the assembled PR then can't merge cleanly and
the sibling's changes go missing (the 2026-06-27 out-of-order dev-task break).
The behind-base gate refuses i_am_done in that state and steers the dev to
sync_branch (the Phase B1 gate-level rebase verb).
- GitService.is_behind_base: rev-list --left-right --count across
origin/{base}...origin/{head} → (behind, ahead); fetch-first so origin
reflects the pushed head. Raises on git failure (consistent with
rebase_onto_base); malformed stdout degrades to (0,0).
- Choreographer._behind_base_gate: wired into _i_am_done_gate after
_ensure_branch_pushed. behind>0 → invalid_state remediate→sync_branch.
Fail-open on git/base-resolution error (flaky fetch can't strand a task at
the submit gate — the merge layer has its own behind checks). Skipped for
branchless roots and protected bases (master/main/-prefixed).
Tests: gate (6: refuse+steer/up-to-date/branchless/protected/fail-open-base/
fail-open-git), is_behind_base (6: parse/up-to-date/malformed/argv-form/
requires-branch/missing-project). ruff + mypy roboco/ tests/ clean; unit green.
* [docs] sync_branch prompt + behind-base guidance (Phase B3)
Update every behind-base/rebase guidance surface to reflect the B1
sync_branch dev verb + B2 i_am_done behind-base gate: devs now self-rebase
through the gate instead of escalating a plain behind-base condition; PMs
still escalate cell/root integration branches (they have no rebase verb).
- developer.md: sync_branch in the verb table; 'When your branch is behind
its base' rewritten — call sync_branch, do NOT i_am_blocked a plain
behind-base; conflicts → resolve by hand, commit, sync_branch again.
- cell_pm.md: delegate signature gains intends_to_touch/adds_migration/
touches_shared/depends_on + a 'Collision surface' section (fill it on every
code subtask so sibling dev tasks that touch the same files sequence into a
conflict-free order — the 2026-06-27 out-of-order break fix); behind-base
section steers devs to sync_branch, PMs escalate only the integration branch.
- main_pm.md: behind-base section — dev leaf = dev's sync_branch; cell/root
integration branch = escalate_up.
- RAG git-errors.md / blocked-tools.md: devs sync_branch, PMs escalate.
- docs/troubleshooting/common-issues.md: leaf self-rebases; integration branch
still escalates to operator.
- CLAUDE.md verb surface: developer gains sync_branch.
- agents/prompts/_generated/*: regenerated via scripts/regenerate_verb_tables.py
— adds sync_branch to the dev table AND catches the generated tables up to
the S1/S2 delegate sequencing params + meltdown-fix note top-level params
(the derived files had drifted stale vs the already-committed schemas).
Docs/prompts only — no code. ruff + mypy roboco/ tests/ clean.
* [chore] orchestrator: refuse to spawn human-only roles (CEO/prompter/secretary)
A live 2026-06-27 incident saw a CEO agent container spawned. Root cause:
_dispatch_a2a_work iterates every A2A/notification target and spawns it
with no human-role filter, and _is_agent_active('ceo') is always false
(the CEO is never a container), so the 'skip if active' check could never
protect the CEO. Any CEO-addressed notification (board handoff, escalation)
launched a CEO container — the system acting as the human CEO: a trust
violation. The CEO is the human operator; intake (prompter) and secretary
are human-driven chats launched through their own dedicated guarded paths
(_spawn_intake_container / _spawn_secretary_container), never spawn_agent.
Fix: a single chokepoint guard at the top of spawn_agent refuses
Role.CEO / PROMPTER / SECRETARY (raises AgentReadinessError + logs). This
structurally covers every dispatcher present and future, since they all go
through spawn_agent. Plus a defense-in-depth skip in _dispatch_a2a_work so
a human-role target never even calls in (avoids error-log spam; the
notification stays for the human to read in the panel).
Safe: the dedicated human-spawn paths do not route through spawn_agent.
Regression tests: spawn_agent refuses ceo/intake-1/secretary-1, does NOT
refuse a real agent; _dispatch_a2a_work skips CEO/intake/secretary targets
and still spawns real-agent + mixed-target cases.
* [chore] orchestrator: skip human-only assignees in claimed/pm-review dispatchers
Defense-in-depth for the spawn_agent human-role chokepoint (d31d6719).
The chokepoint structurally guarantees no CEO/prompter/secretary container
can ever spawn — every dispatcher goes through spawn_agent. But two
dispatchers resolve an arbitrary assigned_to and spawn it with only a
None/unknown-role filter, so a human-assigned task would reach the
chokepoint and RAISE: caught by the per-dispatcher try/except, but it
aborts that dispatcher's whole tick (stalling other respawns behind the
mis-assigned task) and error-logs every cycle. The other dispatchers are
already safe by whitelist/hardcoded slug (blocker_resolver_slug returns
None for non-PM/non-BOARD; escalation/approval use whitelists; marketing
and audit hardcode their non-human slug).
- _claimed_task_needs_agent: return None for a CEO/prompter/secretary
assignee — no container to respawn, and do NOT release a human-owned
task to pending (that would re-route it to a PM). Leave it for the human.
- _dispatch_pm_review_work (assigned branch): skip a human-only assignee
so a CEO-assigned awaiting_pm_review task neither spawns nor aborts the
dispatcher's tick.
Audited all target-iterating dispatchers; only these two lacked a filter.
Regression tests cover both skips.
* [F002] retype board-routed MegaTask root-subtasks code->planning on activation
_activate_batch_root_subtasks flipped a held root-subtask to team=MAIN_PM
but left task_type=code (intake only coerces main_pm-team drafts, so a
board-routed code root reached activation still code-typed). The
main_pm+code combo re-introduces the 2026-06-27 meltdown. Mirror
approve_and_start's own retype via main_pm_cannot_own_code so the
activated child is a planning-typed coordination root.
TDD: RED test_activate_batch_root_subtasks_retypes_code_to_planning
watched fail (task_type stayed CODE), then GREEN after the retype.
ruff+mypy clean; 125 batch/umbrella/approve tests green, no regressions.
* [F003,F004,F014] enforce HMAC agent-token gate on do routes + WebSocket streams
F003/F014: /api/v1/do/* only required X-Agent-ID (UUID) — no token check,
unlike the flow routers' role guards. A forged X-Agent-ID passed. Added
require_any_authenticated_agent (token-only; do router serves all roles)
and applied it as a router-level dependency. Binds X-Agent-ID to a verified
HMAC token when ROBOCO_AGENT_AUTH_REQUIRED=true; rejects a forged token
even in dev mode.
F004: /ws/* per-agent streams (channels/agents/sessions/notifications)
never read the nginx-injected X-Agent-Token, so in strict mode an agent on
the Docker network could subscribe to another agent's notifications with
no auth. Added _require_panel_token verifying the CEO panel token against
the CEO identity; wired into all four per-agent streams (system stream
stays operator-only per its docstring). Same strict/dev contract.
TDD: RED tests watched fail (no gate -> 200/accept), then GREEN. ruff+mypy
clean; 399 api/mcp + 29 WS tests green, no regressions.
* [F005,F006] grok auth: directory mount + atomic-write fallback
F005: the single-file bind mount of auth.json pinned the inode, so the
orchestrator's atomic refresh (tmp+rename within ~/.grok) never reached a
running grok container — a long-lived container hung at the login prompt
when the original ~6h token expired. Mount the host ~/.grok DIRECTORY (ro)
at /home/agent/.grok-auth-ro; the entrypoint symlinks ~/.grok/auth.json at
that RO mount so grok + the --check backstop read the live credential (the
directory mount sees the host-side rename) while grok's writable state
(config.toml, sessions/) stays in the image's ~/.grok.
F006: a rotated refresh_token is single-use — xAI invalidates the old one
the instant it issues the new one. If the atomic write failed after the
rotation, the file kept the now-dead old refresh_token and the credential
was permanently lost on the next refresh. _atomic_write now falls back to a
direct write when tmp+replace fails, so the rotated token always lands on
disk (losing the write is catastrophic; losing atomicity is not).
TDD: RED tests watched fail, then GREEN. ruff+mypy clean; 32 grok tests
green, no regressions.
* [F016,F017] choreographer: surface invalid_state instead of None.status 500 on submit_root / i_am_blocked
Both verbs compose a single atomic action whose None return (the verb's
own result) flowed out of run_intent and was dereferenced as t.status,
HTTP 500-ing with no actionable rejection:
- F016 submit_root: submit_for_review returns None when the root->master
PR was already opened / the task raced out of in_progress. Post-runner
None-guard extracted into _submit_root_finalize -> invalid_state
(re-fetch; if awaiting_pr_review the PR is open, wait for reviewer;
else re-delegate fixes and retry) instead of None.status.
- F017 i_am_blocked: escalate returns None in four cases (no task, no
agent, no resolvable escalation-target slug, no target agent row) e.g.
a developer whose role has no PM above it. _run_i_am_blocked_intent
now guards updated is None -> (t, invalid_state rejection) with
remediation (re-fetch + escalate to CEO directly / retry) instead of
the caller deref'ing None.status -> 500 + respawn-loop.
TDD red->green; ruff + mypy clean; gateway suite green (58 passed).
* [F007] choreographer: cell-level unchanged-PR re-submit loop-stopper for submit_up
The root loop-stopper (F016) was root-only; a weak cell PM could re-submit
the unchanged cell->root PR after a pr_fail and loop awaiting_pr_review ->
pr_fail forever (the cell analogue of the 2026-06-27 root loop).
pr_fail stamps the assembled PR's head SHA into notes_structured.pr_review
.head_sha for cell AND root gate tasks alike (the capture is gate-verb-
level, not root-level), so the same structural refusal applies to submit_up:
if the cell PR's current head SHA equals the SHA the last pr_fail recorded,
no new dev work landed on the cell branch -> the diff is byte-identical ->
refuse, do not re-open the gate. Different SHA -> branch advanced -> allow.
- _submit_up_unchanged_pr_guard mirrors _submit_root_unchanged_pr_guard
(cell-PM remediation: re-delegate to the dev + wait for re-assembly),
wired into submit_up after _submit_up_guard passes.
- Renamed shared _current_root_pr_head_sha -> _current_pr_head_sha (both
guards use it; the lookup was never root-specific).
- Every ambiguous case FAILS OPEN (no prior fail, no recorded sha, no
pr_number, no resolvable project, git/closed-PR None) — only the exact-
unchanged case is hard-blocked.
TDD red->green; ruff + mypy clean; F007+F016 guard suites green (15 passed).
* [F008] evidence_builder: surface persisted pr_review verdict+issues in the PM task_handoff
The pr_fail a2a steer to the owning PM is fire-and-forget; a PM respawned
into needs_revision later read none of it (build_task_handoff never looked
at notes_structured), saw a generic 'needs revision' with zero concrete
change-requests, and re-submitted the same PR (the 2026-06-27 infinite
pr_fail loop on 9980d0a0 / PR #138). The signal-gap was only partially
closed by the a2a.
build_task_handoff now extracts notes_structured.pr_review
(verdict/summary/issues/head_sha — the slot pr_fail authors on every fail)
into a pr_review field on the handoff, so every PM briefing for the task
carries the concrete change-requests. A prior pr_fail alone now counts as
prior-work-worth-resuming. Type-guarded + capped; absent => no key (no
misleading empty slot).
TDD red->green; ruff + mypy clean; evidence_builder suite green (14 passed).
* [F009] notification: derive requires_ack from ACK_REQUIRED_BY_TYPE, not the True default
NotificationService._create_notification built NotificationTable without
requires_ack, so the column default (True) applied to EVERY notification -
including informational REVIEW_REQUEST / DOCUMENTATION_REQUEST /
A2A_REQUEST / KNOWLEDGE_SHARE (ACK_REQUIRED_BY_TYPE -> False) and every
@mention from MessagingService._notify_mentions. Each false ack-required
inflated the recipient's unacked set and soft-blocked i_am_idle into
respawn churn.
- _create_notification: requires_ack=ACK_REQUIRED_BY_TYPE.get(type, True)
(unmapped types default True - preserve the action-required bias).
- _notify_mentions: requires_ack=False explicit (MENTION is informational).
TDD red->green (identity is False/is True assertions - the mocked
flush doesn't apply SQLA's insert-time default, so pre-fix the attribute
was None); ruff + mypy clean; notification suite green (18 passed).
* [F010] notification: never dedup informational notifications (knowledge-share data loss)
The purpose-based dedup suppressed a same-purpose (same sender/type/task,
overlapping recipients) notification while a prior one was unacked. For
informational types (KNOWLEDGE_SHARE / MENTION / A2A_REQUEST / BROADCAST +
the pickup-proves-receipt triad) each send carries DISTINCT content (a new
learning, a new mention) and acking is voluntary, so a recipient who never
acks the prior one let the dedup permanently suppress every subsequent
same-sender broadcast - silent learning-broadcast data loss.
The dedup's anti-loop rationale (stop unacked-set inflation soft-blocking
i_am_idle) only holds for action-required signals. Gate the dedup on
ACK_REQUIRED_BY_TYPE.get(type, True): action-required types still dedup,
informational types always create. Unmapped types default True (dedup on).
TDD red->green; ruff + mypy clean; notification + dedup suites green (20).
* [F011] playbook: de-index rejected/archived playbooks from the PLAYBOOKS RAG index
* [F012] release_executor: fail-closed on git add/commit before push
* [F013] release_proposal: Redis SET NX mutex guards the ~40min execute against concurrent approves
* [F015] flow_qa/flow_doc: add i_am_blocked route (manifest-registered escape hatch was 404)
* [F018] claim_guards: treat blocked as active + broaden the guard lookup so a blocked dev can't double-claim
* [F019] git: clear orphaned .git/*.lock files after a timeout-SIGKILL'd mutation op
* [F031] identity: role_for_slug_or_none so defensive skip-guards don't crash the dispatcher tick on stale slugs
* [F032] test: unknown-assignee claim reaches release-to-pending path
F031's role_for_slug_or_none fix made the unknown-assignee release branch
in _dispatch_claimed_without_agent reachable (the human-only guard no
longer raises/short-circuits on a stale slug). Lock that reachability in:
a claimed task with an unknown-assignee UUID past grace returns the slug
(not None) so get_agent_role -> 'unknown' releases the claim to pending
for a role-matched reclaim.
* [F033] orchestrator: capture container_id at startup re-adoption
_readopt_running_agents registered re-adopted ACTIVE instances with
container_id=None. _check_health skips container_id-is-None instances, so
when a re-adopted container later exited the stopped-container handler
never ran and the task stranded under a phantom ACTIVE instance forever.
Add _resolve_container_id (docker inspect -f '{{.Id}}') and store the real
id on re-adopt. Best-effort: a probe failure degrades to None (still
ACTIVE; the reaper's Docker-liveness fallback covers it).
* [F034] orchestrator: re-stamp respawn last_check at restore
_pm_made_rule_following_retry bounds its tracing_gap audit lookup with
since = record.get('last_check'). A stale persisted last_check from before
the restart matched pre-restart tracing_gap rows, falsely resetting the
breaker on the very first post-restart spawn — exactly when a fresh strike
count should be evaluating current state.
_partition_respawn_rows now re-stamps last_check to the restore time on
every restorable entry, bounding the lookup to post-restart gaps only.
* [F035] orchestrator: probe-resume loop actually revives parked agents
_park_provider_unavailable parked the provider + offlined the instance but
never registered a WaitingRecord, so _on_probe_success -> _parked_agents_for
(always filtered on waiting_for=='rate_limit_lifted') returned [] and
resolve_wait revived nobody — recovery fell to the 600s stale-claim reaper
instead of the probe-success path the parking design relied on.
Register + persist a rate_limit_lifted WaitingRecord at park time (mirrors
mark_waiting_long, minus stop_agent — the container is already dead).
Companion reaper guard: _reap_with_service now skips provider-parked
assignees (_assignee_is_provider_parked) so the claim survives until the
probe revives the agent — otherwise the reaper releases the claim to pending
and probe-success respawns on a task the agent no longer owns.
* [F036] orchestrator: read transcript for overload detection too
The SDK server writes model-API errors (529/500/503) to /tmp/sdk-server.log,
not stdout, so an overload marker can appear only in the durable Claude
transcript — the same rationale already applied to the session-limit
detector. _provider_overload_park_target read only docker logs, so an
overload was missed and the agent crash-respawned straight back into it.
Now concatenates the transcript tail before matching, mirroring the
rate-limit path.
* [F037] orchestrator: drop bare error-NNN overload markers
The bare 'error 529'/'error 500'/'error 503' markers were broad enough to
false-match an agent that merely writes about an HTTP status code in its own
notes ('the endpoint returned error 500, retrying'), parking the whole
Anthropic fleet on a non-issue.
The SDK error formatter emits 'API Error: NNN' + a JSON error type, so the
remaining 'api error: 529/500/503' + 'overloaded_error' +
'internal_server_error' markers cover every real overload without that
false-match surface.
* [F038/F039] orchestrator: sign X-Agent-Token on self-API calls
The prior self-PATCH 401 fix only carried X-Agent-ID/X-Agent-Role. Arming
ROBOCO_AGENT_AUTH_REQUIRED=true made the middleware require a signed
X-Agent-Token, so every orchestrator self-call (auto-block / auto-resume /
auto-recover / SLA annotation) 401'd and silently no-op'd — wedging
paused/blocked parents.
Add _system_api_headers() that wraps the base headers with a signed token
for the system identity (issue_agent_token); switch all six self-call sites.
Dev fallback: no secret set => UNSIGNED sentinel + auth not required.
* [F040] orchestrator: finalize grok spawn session on cost-cap kill
_enforce_grok_cost_budget killed + evicted the container without calling
_finalize_spawn_session, so the open agent_spawn_sessions row stayed open
(ended_at IS NULL) and the burned usage/cost was never recorded in the
dashboard.
Call _finalize_spawn_session(exit_reason='cost_cap') BEFORE popping the
instance — it reads self._instances[agent_id] for the model +
usage_session_id, which the pop would lose.
* [F041] park grok exit-78 (auth missing/expired) instead of crash-retrying
A one-shot grok container whose entrypoint ran grok_auth --check and found
the token missing/expired exits 78 (EX_CONFIG). Crash-retrying 3x burns
tokens for zero progress — the agent cannot start without a valid token.
Park the provider with kind=auth_missing (same shape as the 429 exit-75
path) so the probe-resume loop revives the task once grok_auth.refresh_if_stale
mints a fresh token; if still expired, the next exit 78 re-parks (no burn).
Also fixes a latent F035 regression: _park_provider_unavailable now registers
a WaitingRecord, so the bare-__new__ rate-limit park test had to set
_waiting_records + stub _persist_waiting_record (mirrors the overload-test
fixture).
* [F042] isolate concurrent-duplicate conventions cache put in a savepoint
Two task creates for the same project/HEAD can race to populate the
conventions cache; the loser's INSERT fails the partial-unique index with
IntegrityError. A bare session.add + flush poisons the shared session (the
task-create transaction rides the same session), so every subsequent op
raises 'this session is in error state' and task creation crashes.
Run the INSERT in a savepoint (begin_nested) and swallow the IntegrityError:
only the savepoint rolls back, the outer transaction stays usable, and the
winner's row satisfies the next _cache_get.
* [F043] guard escalate_up against resurrecting terminal tasks
escalate_up had composes=() and no source-status guard, so a PM could
escalate a COMPLETED/CANCELLED task and apply_escalation set it back to
BLOCKED — bypassing the state machine's terminal-state invariant.
Defense in depth:
- spec: add PRECONDITION_NON_TERMINAL to escalate_up's extra_preconditions so
the lifecycle gate rejects terminal tasks (invalid_state) before the
journal:decision write fires; generalize _check_intent_preconditions to
honor non-tracing rejection_kind (not_authorized / invalid_state).
- service: apply_escalation (the single write primitive) returns False and
refuses to mutate a terminal task — covers the HTTP escalate route which
bypasses the spec gate. escalate() / escalate_up_to_role() return None on
refusal so the gateway emits a clean invalid_state envelope.
- route: the HTTP escalate route 409s a terminal task BEFORE sending the
escalation notification (so a finished task isn't yanked back, PM not pinged).
* [F044] pr_pass gate remediation points the reviewer at pr_fail, not i_am_blocked
The pr_pass gate runs the toolchain + conventions guards on the REVIEWER's
workspace, but their remediation text said 'call i_am_blocked' — a verb the
PR reviewer does not have. The reviewer would chase a verb they cannot call
instead of rejecting the PR.
Make the guards reviewer-aware: a reviewer=True flag (passed by _pr_pass_blocked)
switches the remediation to pr_fail(issues=[...]) — the reviewer's reject
lever, sending the PR back to needs_revision for the dev to fix the
environment / validator. The dev (i_am_done) path keeps i_am_blocked, which a
dev does have. _conventions_guard (the pr_pass path) now passes reviewer=True
through to _conventions_rejection.
* [F045] rate-limit: loud activate-failure log + in-memory orphan-probe fallback
The in-verb i_am_blocked(rate_limited) path wrapped RateLimitStateTracker.activate
in a bare contextlib.suppress. A silent activate failure stranded the fleet:
agents were parked in _waiting_records but the provider never entered the tracker,
so the tracker-driven _sweep_rate_limit_probes never probed it and no
_on_probe_success ever resumed them — parked agents stuck in WAITING_LONG.
Fix: (1) replace the bare suppress with a try/except that logs an error event
naming the provider + affected agents; (2) in _sweep_rate_limit_probes, after
probing the tracker-listed set, scan _waiting_records for any rate_limit_lifted
provider the loop did NOT cover and probe it via the time-expiry fallback (empty
state -> probe now) so _on_probe_success resumes the parked agents. The fallback
reads only local memory, so it still resumes when Redis was down at park time
(list_rate_limited_providers failure now falls through to the orphan scan instead
of returning early).
* [F046] pr_gate: guard None runner result on concurrent transition (pr_pass/pr_fail)
_gate_decision dereferenced the verb-runner result without a None guard.
run_intent returns None when a concurrent transition (cancel or a racing
reviewer) moves the task out of awaiting_pr_review between the precondition
gate and the runner's final composed action (the verb runner's documented
last-action source-status contract). The subsequent t.assigned_to /
t.status / _post_gate_review_to_pr(t, ...) dereferences then crashed the
gate with a 500 AttributeError. Add a None guard that surfaces a clean
invalid_state rejection (re-fetch + re-issue) before any dereference; no
PR post or a2a runs against a None task. TDD test_pr_gate_notifies_pm.py (+2).
* [F047] conventions: reviewer-aware block-finding remediation on pr_pass gate
The pr_pass (reviewer) conventions guard reused the dev-path block-finding
remediation: 'add a waiver to .roboco/conventions.yml in your branch'. A
pr_reviewer does not own the assembled cell->root / root->master branch and
has no commit verb on it, so the waiver remediation is unreachable — a false
positive stranded the gate with no self-recovery (the reviewer could neither
commit a waiver nor pr_pass). The fail-open content path is documented
precision-over-recall and stays as-is; the actionable gap is the remediation.
Fix: _conventions_rejection now branches the block-finding remediation on
reviewer=True (mirroring the could_not_run branch from F044). The reviewer
path points at pr_fail carrying the findings as issues so the PR returns to
needs_revision and the DEV fixes the violation or commits the waiver (the dev
CAN commit to the branch); waiver authorship is framed as the dev's action,
not the reviewer's. Dev i_am_done path wording unchanged. TDD
test_conventions_gate_pr_pass.py (+1).
* [F048] notify: reject human-only recipients (prompter/secretary) — no agent ack path
notify() only checked the SENDER role. The recipient was resolved by
NotificationService._resolve_recipients, which drops only unresolvable slugs
— it does not exclude human-only roles. The prompter (intake-1) and secretary
(secretary-1) are seeded agent rows, so they resolved, and an ack-required
ALERT addressed to them sat permanently unacked (no agent auto-acks it),
polluted the panel's pending-ack view, and — via the dedup query's
~acked_by.contains — permanently suppressed any later same-purpose
notification from the same sender to that human role. The knowledge-share
path already excludes all three human-only roles; the general notify path
did not.
Fix: a recipient-role guard in notify() via _reject_disallowed_recipient
(folds the new check into the existing CEO-dependency-block return slot so
notify stays under the PLR0911 return limit). Rejects prompter/secretary
with not_authorized; the CEO is human too but acks via the panel, so it stays
an allowed recipient (its only disallowed case, a dependency-block page, is
preserved). TDD test_notify.py (+3: reject prompter, reject secretary, allow
CEO).
* [F049] merge_pull_request: idempotent on already-merged PR (mirror _merge_with_retry)
* [F050] merge_pr_for_task: verify caller pr_number matches task's recorded PR
* [F051] open_conventions_pr: refuse dirty tree + verify checkout-base landed
* [F052] pr_target: scope task lookup by project_id (mirror close_pull_request)
* [F053] _token_for_project: log decryption failure (key rotation) with project slug
* [F054] learnings index: enforce shareable on every shared retrieval path (private-leak fix)
* [F055] messaging: recover from concurrent channel auto-create race via savepoint + re-fetch
* [F056] messaging: lock group row before session check-then-create to prevent active-session orphan race
* [F057] playbook: index/unindex as a post-commit step so the RAG corpus never leads the status transaction
* [F058] release-readiness: non-empty bump plan on first release
_canonical_bump_files derived the bump set from the previous
chore(release): commit. On the first release there is no such commit,
so it returned [] -> assess set version_bump_plan=[] -> the executor
published a tag with no files bumped (a no-op masquerading as X.Y.Z).
Fall back to the version-reference scan when no prior release commit
exists: the files currently embedding the version are exactly the set a
first release must bump, and the set the first release commit then
records as canonical for subsequent releases. Read-only derivation; the
CEO-approval gate and fail-closed executor are untouched.
* [F059] self-heal: hold fix tasks for CEO Approve-&-Start (restore dispatch gate)
The module docstring promised self-heal fix tasks 'wait for the CEO's
Approve-&-Start', but _originate created them confirmed_by_human=True and the
orchestrator dispatched them at once — a self-heal fix that re-broke CI would
trigger another cycle, open another auto-dispatched fix, and loop with no CEO
gate on dispatch.
Restore the documented gate:
* _originate opens the task confirmed_by_human=False (held for the CEO).
* The orchestrator holds a self-heal task out of both the PM and dev dispatch
paths until confirmed_by_human flips True.
* approve_and_start (the CEO's start gate) sets confirmed_by_human=True so the
held task finally dispatches (idempotent for board/intake tasks already True).
* list_pending_for_agent scopes the give_me_work hold to self-heal
(source != self_heal OR confirmed_by_human) so an already-alive PM can't grab
it pre-approval — while ordinary delegated subtasks (confirmed_by_human=False
by default, where the delegation IS the authorization) still dispatch.
The 'never self-deploys' guarantee (no merge) is unchanged.
* [F059] fix DB-integration test auth + retype self-heal root code→planning
conftest test-DB defaults matched the project's own running postgres
(roboco/roboco @ localhost:15432, the docker-compose roboco-postgres
service with CREATEDB) instead of the OS user on localhost:5432 which has
no such role — every db_session test failed with InvalidPasswordError
instead of running.
Once the DB connection worked, the self-heal origination DB test went RED
with MAIN_PM_NO_CODE: the self-heal root was task_type=CODE owned by
main_pm, the combo the main_pm_cannot_own_code guard rejects. The Main PM
coordinates the fix (delegates the code work to a cell dev); it has no
code verb. Retyped CODE→PLANNING and rewrote description/AC to
coordination-level.
* [F060] emit reversal audit row on claim-branch-failure rollback
The forward task.claimed audit row is flushed before the branch-creation
attempt, and AuditService commits on its own connection, so the rollback's
flush reverts the task row but not that audit row — the journey's last
event stayed task.claimed while the task reverted to its pre-claim status,
diverging from real state and corrupting downstream cycle-time/bottleneck
metrics. The rollback now emits a CLAIMED->original reversal audit row
(only when the forward transition was made) attributed to the claimant.
* Removing completely unnecessary files (for the repo they are unnecessary)
* [F061] audit status-transition rows now written in-session (F061/F073/F075)
_emit_status_transition_audit now writes AuditLogTable rows into
self.session synchronously (session.add) instead of dispatching
AuditService.log_task_event fire-and-forget on its own connection.
The audit row now commits/rolls back atomically with the status
transition in the caller's transaction, closing three facets at once:
- F061: audit commit no longer decoupled from the transition commit
- F073: a committed transition can no longer have NO audit row
(the row rides the same transaction; a swallowed persist can't drop it)
- F075: a transition rolled back inside a verb savepoint no longer
leaves a phantom audit row (the row is in the savepoint too)
log_task_event is now called only from this helper (narrow blast
radius verified); revision_count increment stays at this single
chokepoint. Cycle-time/bottleneck reconstruction from task.<status>
events is no longer silently corruptible.
Tests: test_emit_status_transition_audit_writes_in_session_atomically,
test_finalize_claim_rollback_emits_reversal_audit, escalation-audit
tests retargeted to in-session AuditLogTable rows.
Also: _canonical_bump_files grep-looseness follow-on (F058) -- filter
by subject, not body; git log --grep matches any message line, so a
non-release commit whose body references chore(release): shadowed the
real release commit. Test
test_canonical_bump_files_ignores_body_only_chore_release_match.
* [F061] drop type:ignore from audit-emit tests
Convention: no type:ignore/noqa. The F061 in-session audit-emit
tests used '# type: ignore[assignment]' to assign a MagicMock to
AsyncSession.add, and the F060 test assigned to .flush the same way.
Rewritten to hold a local 'session: MagicMock' variable (mypy sees
its auto-children as MagicMock, so .add.side_effect / .flush assign
cleanly with no suppression). Verified via 'mypy tests/' that both
files are now type-clean (the F060/F061 commits had skipped tests/
in mypy, masking two method-assign errors).
* [chore] clear all 64 pre-existing mypy errors in tests/ (no type:ignore)
Convention: no type:ignore/noqa, and pre-existing violations still
violate. The make-quality gate runs 'mypy roboco/ tests/', but the
prior commits' gates only ran mypy on production files, masking 64
type errors across 15 test files (method-assign, unused-ignore,
no-untyped-def, attr-defined, union-attr, has-type, index, misc).
Fixed without any type:ignore:
- method-assign (svc.session.X = / svc.method = AsyncMock()): hold a
local 'session: MagicMock'/'AsyncMock' and assert on it, or stub via
object.__setattr__ / monkeypatch / a typed '_bind' helper returning
Any, or alias 'cc: Any = c' (the pattern the file already used).
- unused 'type: ignore[assignment]' (real code was method-assign):
removed; replaced with the no-suppression patterns above.
- 'Callable[...] has no attribute assert_*': keep a typed local ref to
the AsyncMock and assert on the local, not the method-typed attr.
- no-untyped-def: annotate helper params (Any / pytest.MonkeyPatch).
- attr-defined / index / union-attr: type the helper as Any, narrow
with an 'is not None' assert, or add the missing attr to a fake.
- has-type / return-value: fix the declared return type to the tuple
the function actually returns.
- PLC0415 inline imports: hoisted to top-level.
test_pr_gate_notifies_pm._stub_gate_path converted fully to the
'cc: Any = c' alias (it already used it for one attr) so its five
'# type: ignore[method-assign]' suppressions are gone.
mypy tests/: 64 errors -> 0 (538 files). ruff check tests/: clean.
All 84 tests in the touched files pass.
* [chore] remove all remaining type:ignore suppressions from tests/
Converts 115 `# type: ignore[...]` suppressions across 23 test files to
no-suppression patterns (helper-return widening to Any, local Any aliases,
cc:Any aliases, cast at narrow call sites, typed fixtures) so the hard
no-type:ignore convention holds across tests/. No test logic or assertions
changed — only mock-wiring mechanics and type annotations.
Gate: ruff check tests/ clean; mypy tests/ (538 files) clean; 176 changed-file
tests pass. Zero real suppressions remain (the 7 grep hits are 3 hygiene-
checker string-literal test inputs and 4 prose mentions in comments).
* [F062] work_session.merge_pr: idempotency + active-status guard
merge_pr unconditionally set pr_status=merged, pr_merged_at, merged_by,
status=COMPLETED on whatever session it loaded — the only session-terminal
transition in WorkSessionService lacking both the active-status guard
(complete/abandon) and the terminal-idempotency guard (close). Two failure
modes: (1) a retried merge after a successful-but-unconfirmed GitHub merge
overwrote merged_by/pr_merged_at with the retry's actor/timestamp, corrupting
the merge audit trail; (2) merge_pr on an ABANDONED session resurrected it to
COMPLETED, undoing the single-active abandonment. Mirrors close()'s guard:
if status != ACTIVE, return the session unchanged. Both git.py callers await
merge_pr and discard the return, so the no-op is safe. TDD: 3 tests
(happy-path + both modes).
* [F063] workspace._clone_repo: rmtree half-configured clone on failure
If _configure_git raised CalledProcessError before its `remote set-url`
scrub, .git/config kept the tokenized auth URL (the project PAT) and
_assert_no_pat_leak never ran. The except clauses raised WorkspaceError
without removing the workspace, so the next ensure_workspace's health
short-circuit (valid .git with HEAD + objects) skipped past the leak —
mounting the agent on a workspace whose .git/config let it read+exfiltrate
the PAT. Both clone-failure except clauses now rmtree the workspace before
raising, so a half-configured clone is destroyed and ensure_workspace
re-clones from scratch. TDD: 2 tests (configure-failure leak + timeout).
* [F067] flow_main_pm: add missing /triage route
main_pm's manifest advertises triage (lifecycle.intents_for_role(MAIN_PM)
includes it via _PM_ROLES, alongside triage_all) but flow_main_pm.py had no
POST /triage route, so a main_pm agent calling triage hit a raw 404 that
bypassed the per-verb circuit breaker. Added the route mirroring flow_cell_pm's
/triage — wires to the existing team-scoped choreographer.triage (uses pm.team,
works for any PM role; Main PM gets its own team's blocked/awaiting tasks).
Fix direction: add-route, NOT remove-from-manifest — the manifest is spec-correct
(intents_for_role by construction); removing triage would contradict the spec
and leave main_pm with only cross-team triage_all. TDD: test_triage_route_exists_and_dispatches.
* [F068][F069] mcp servers: classify all rejection shapes + envelope 404s
F068: the do/flow-server circuit breaker only counted rejections whose
`error` field was a STRING in _CIRCUIT_REJECTION_KINDS. A 422 validation
failure (no `error` field, a `detail` list) and a 500/HTTPException
(dict-shaped `error` from the exception handlers) both bypassed the breaker
→ unbounded retries on a storm of either. Added _classify_rejection(payload)
(shared, applied to both servers) mapping all three shapes to a counted kind:
string error (existing), dict error → substring-mapped code
(*DENIED*/*AUTHORIZED*/*FORBIDDEN*/*PERMISSION*→not_authorized,
INVALID_INPUT/*VALIDATION*→incomplete_input, *NOT_FOUND*→None parity, else
→invalid_state), 422 detail→incomplete_input. The dict TypeError defence lives
in the classifier (isinstance, never dict-in-frozenset).
F069: a manifest-registered verb whose HTTP route is missing got FastAPI's raw
`{"detail":"Not Found"}` 404 body — a non-envelope payload the breaker
couldn't classify, so a storm bypassed it. _post now synthesizes an
invalid_state Envelope rejection (with a remediate hint → i_am_blocked/i_am_idle)
for a 404 status, routed through _record_and_check_circuit so the breaker counts
it. A 404 that carries a real Envelope (error field present) is surfaced as-is,
preserving test_flow_post_returns_envelope_on_404. TDD: 422/dict/404 tests in
both server test files; updated test_dict_shaped_error_does_not_crash to assert
the SDK is now called with not_authorized (replacing the pass-through assertion
that encoded the bug).
* [F064][F065][F066] websocket: non-blocking fan-out, finally-disconnect, idle timeout
F064: the bridge forwarder awaited every conn.send_text in a gather with no
per-connection queue and no send timeout — one slow WS client back-pressured
ALL event delivery to ALL clients (head-of-line blocking on the listen loop).
Each connect_* now registers a _ClientConnection (bounded asyncio.Queue(256) +
sender task); broadcasts enqueue via put_nowait (drop + structlog warn on
QueueFull) and return immediately. The sender drains the queue with each send
wrapped in wait_for(SEND_TIMEOUT=10s). Unregistered legacy sockets (set
directly into a subscription set, bypassing connect_*) get a timeout-bounded
fallback send task held in _pending_sends (ruff RUF006). disconnect cancels +
drops the sender.
F065: route handlers caught only WebSocketDisconnect with no finally — a
non-clean exit (anyio closed-resource, CancelledError, transport error)
propagated without manager.disconnect, leaking the dead socket into every
subscription set forever. Added finally: manager.disconnect(websocket) to all
5 handlers (disconnect is idempotent).
F066: no server-side heartbeat/idle timeout — a half-open socket from a dead
container blocked receive_text forever and was never reaped. receive_text now
wraps in wait_for(IDLE_TIMEOUT_SECONDS=90s); on TimeoutError, log + fall
through to the F065 finally. Named module constants (no config.py precedent for
WS tuning; callers/tests patch them).
TDD: 22 new tests across 3 files (handler cleanup, idle timeout, send queue),
non-flaky across repeats; 1 existing test adapted with a yield for the new
async fan-out (assertion unchanged). ruff/mypy clean, 421 unit/api tests pass.
No type:ignore/noqa.
* [F022][F023][F024][F025][F026] api: scrub secrets from 422 log, gate a2a/dashboard/orchestrator routes, SSE session-per-query
- middleware: redact known credential fields (git_token/api_key/token/...)
from the 422 request-validation log line; response body unchanged
- a2a: require_any_authenticated_agent on /message/send + /message/stream;
subscribe_to_task opens a short-lived session per poll instead of holding
one asyncpg connection for the full SSE lifetime (pool exhaustion) + auth
- dashboard: gate auditor flag/report mutating routes to Auditor or CEO
- orchestrator: router-level CEO gate on all control routes (spawn/stop/...)
TDD; ruff/mypy clean; 449 unit/api tests green; no type:ignore/noqa.
* [F030] conventions: typescript-scoped custom rules now apply to .tsx files
The validator tags a .tsx file as language 'tsx' (the JSX grammar needs
that tag, distinct from plain 'typescript'), but a custom rule scoped to
'typescript' — the language the scan reports for a React+TS repo — silently
skipped every .tsx file. The two suffix maps were NOT unified: the 'tsx'
tag is load-bearing (grammars.py picks the JSX grammar on it; hygiene.py
keys on it), so unifying would make .tsx fail to parse.
Fix is in check_custom: a one-directional dialect map _DIALECT_OF =
{'tsx': 'typescript'} — a typescript-scoped rule fires on a .tsx file,
but a tsx-scoped (JSX-only) rule still does not fire on plain .ts.
TDD; ruff/mypy clean; 80 unit + 38 integration conventions tests green.
* [F029] websocket: remove broken /api/permissions/check loopback from channel stream
channel_stream called validate_channel_access, which HTTP-loopbacked to
GET /api/permissions/check — a route that does not exist. Every call 404'd
-> False -> the channel stream closed with WS_1008_POLICY_VIOLATION for
EVERY client, so the real-time channel stream was dead. Removed the
function, its call site, and the now-unused httpx + settings imports.
Post-F004 the panel-token gate is the channel-stream authorization (the
CEO panel is the sole WS client and may view every channel), so the
broken loopback is removed rather than replaced with an in-process check
the CEO always passes. The legitimate enforcement.validate_channel_access
(slugs, in-process static ACL) is a different function and is untouched.
F027 is resolved-by-F004 (no code change): all three per-agent streams
gate on _require_panel_token first, so only the authorized CEO panel can
connect — 'any viewer subscribes to any target' is closed.
TDD; ruff/mypy clean; 530 unit/api+enforcement+RBAC tests green.
* [F078] release_executor: deadline every subprocess (git/make/gh/clone)
A hung git/make/gh/clone would block the CEO-gated release loop
indefinitely. Wrap each proc.communicate() in asyncio.wait_for via a
shared _await_proc helper; on expiry proc.kill() the child and return a
non-zero rc (124) so every caller's fail-closed branch fires. Mirrors the
quality-gate _run_one kill-on-timeout idiom.
Deadlines are generous (30min gate / 10min clone / 5min push+gh) so a
legitimate slow op is never wrongly aborted — floor-assertion tests pin
the floors to guard exactly that logical regression. Green path returns
the real rc unchanged.
* [F072] reaper: deadline docker inspect/exec + harden _check_health sweep
A hung Docker daemon (or a stuck container FS) froze the single asyncio
event loop: the reaper runs inline before every dispatch tick and shares
that loop with every background sweeper. Bound each docker subprocess
with asyncio.wait_for; on expiry proc.kill() the child and either raise
(inspect / resolve_container_id — callers apply their own fail-direction)
or return None (the gateway probe — inconclusive, caller declines to act,
matching its existing probe-failure contract). Deadlines generous
(10s inspect / 30s exec) so a legitimate slow docker call is never
wrongly aborted; floor-assertion tests pin the floors.
Also harden _check_health's per-agent loop so one agent's hung inspect
skips that agent, not the whole sweep — preserving the check-all-agents
invariant the timeout-then-raise would otherwise break (without this, a
hung daemon means no agent gets health-checked any tick).
* [F076] say/dm: handler guard rejects all 4 no-comms roles, not just auditor
The say()/dm() defence-in-depth guard only rejected auditor, but CLAUDE.md
mandates the same no-agent-comms invariant for pr_reviewer (posts findings
on the PR), prompter and secretary (human-only, note + evidence). For those
three the manifest was the only gate, so a call bypassing the manifest
(direct API POST, test harness, future routing change) would not be refused
at the handler — admission depended on the agent's slug happening to be
absent from the channel/a2a matrix. Extend the guard to a _NO_COMMS_ROLES
frozenset (auditor + pr_reviewer + prompter + secretary), matching the
explicit role-frozenset gates on commit/notify/pitch/playbook/open_session.
Role-appropriate remediation per role. The claimed defence-in-depth now
covers 4 of 4 silent roles, not 1 of 4.
* [F070] drain fire-and-forget _bg_tasks on shutdown (bounded, data-preserving)
Orchestrator.stop() cancelled only the named loop tasks + agents, then
returned, abandoning in-flight _schedule_bg work. An in-flight
_persist_respawn_record upsert dropped at shutdown meant the last few
gate-mutation strikes never reached the DB; restore_respawn_tracker() on
the next start repopulated a stale lower count and the dispatcher re-burned
the full 4-spawn strike threshold against a still-wedged task — the exact
re-burn the durable tracker exists to stop. Audit-log writes (load-bearing
for cycle-time/rework metrics) were similarly dropped.
Add _drain_bg_tasks(): bounded wait (5s default) lets short DB writes
commit before exit (data preserved), then cancels any stuck task past the
deadline so a hang can't wedge shutdown. return_exceptions=True so one
failing bg task doesn't crash the drain. Wrap the stop_agent loop in
try/except + logger.exception so one bad agent can't skip the drain
(re-introducing the data-loss tail). Floor test pins the deadline >= 3s
so a too-short change can't silently drop a legitimate slow write.
* [F071] abort non-blocking intake/secretary spawn on mid-spawn shutdown
The non-blocking spawn (start_intake_session / start_secretary_session)
schedules _spawn_intake_container_guarded / _spawn_secretary_container_guarded
via _schedule_bg. Those run docker run and only register in _instances at the
END. If shutdown arrived between docker run and the registration line, the
container was started but the orchestrator had no handle — stop() iterates
only _instances, so the container was orphaned (leaked, manual docker rm).
Worse, the F070 drain could let the spawn coroutine complete the
registration AFTER stop() already iterated _instances, landing a live
container into a shutting-down registry nothing tears down.
Add a post-docker-run shutdown guard in _spawn_intake_container and
_spawn_secretary_container: re-check self._running after _run_container_cmd
returns; if the orchestrator began shutting down, remove the just-started
container (by its deterministic name) and raise _SpawnAbortedDuringShutdown
WITHOUT registering. The guarded wrappers catch that BEFORE except Exception
and close the live relay silently (shutdown is not a user-facing failure,
no error pushed to the SSE stream). The F070 stop() drain awaits the bg
spawn coroutine, so the abort surfaces cleanly.
TOCTOU-safe: between the _running check and the _instances assignment there
is no await (config + instance construction are sync), so once the check
passes, registration completes before the event loop can interleave stop().
The normal running path is unchanged (sanity tests pin it).
* [F074] per-agent advisory lock closes claim TOCTOU
_run_claim_guards read the agent's other tasks via unlocked SELECTs
before claim() took its row lock, and claim()'s FOR UPDATE locked only
the TARGET row — so two concurrent i_will_work_on by the SAME agent on
TWO DIFFERENT pending tasks each locked their own row, each read an
empty in_progress set, each passed already_active, each claimed+started
→ the agent ended with two in_progress tasks (the in-process asyncio
Lock is lost on orchestrator-restart split-brain, so it wasn't a
DB-level guarantee).
Fix: TaskService.acquire_claim_lock takes a transaction-scoped
pg_advisory_xact_lock keyed by hashtextextended(agent_id). The gate
acquires it BEFORE the guard reads (for non-coordinator roles only) so
the second concurrent claim's read sees the first's committed
in_progress task and is rejected. Tx-scoped → auto-releases on
commit/rollback, can't outlive the request.
Coordinator exemption (the key logical-regression guard): cell_pm /
main_pm do NOT take the lock — the PM coordinator concurrency feature
lets a PM plan+delegate many roots in parallel, and a per-agent lock
would serialize those claims and regress it. Matches the existing
_COORDINATOR_ROLES already_active/paused guard exemption. A hash
collision only causes benign false serialization, never a false
negative.
Tests: unit (dev acquires lock before guard read; coordinator does
not) + real-PG integration (same-agent serializes, different-agent
does not, releases on rollback).
* [F021] handle SSE transport errors so the intake composer isn't stuck
openStream registered listeners for the server-sent event kinds but not
the EventSource's own transport-level error. The 'error' kind IS in
LIVE_EVENT_KINDS, so a server-sent event:error (JSON MessageEvent) was
handled — but a dropped connection / dead session fires a plain Event
with NO data, which JSON.parse(undefined) swallowed in the try/catch,
so the stream 'stayed open' (EventSource loop-reconnected a session that
no longer existed) and isSending stayed true — the composer was
permanently disabled.
Fix: route the 'error' event by payload. A MessageEvent with string
data is a server-sent error → handleEvent (unchanged). A no-data Event
is a transport error → handleTransportError: clear streamingId/activity,
set isSending false, add a 'connection lost' error message, keep a
draft/batch preview up (so the human can still act on a proposed card)
else land on 'chatting', and close the dead stream so EventSource stops
loop-reconnecting.
Tests: renderHook + a jsdom EventSource double that fires a transport
error (plain Event, no data) vs a server-sent error (MessageEvent +
JSON). RED: transport error left isSending true; GREEN: resets to
false, surfaces the message, closes the stream. The server-sent-JSON
path is unchanged. Full panel suite (129) green; eslint/typecheck/prettier clean.
* [F081] Approve dialog: label notes required (>=20 chars), not optional
The CEO Approve dialog's notes label fell into the default branch
('Notes (optional') for the approve action, but approve actually
requires substantive notes >= 20 chars — enforced client-side
(toast error on < 20) and server-side. So the CEO was told 'optional'
and only learned the real requirement from a toast after hitting
submit with empty notes.
approve and start both require >= 20 chars; reject only requires a
reason. Collapse the label to two branches: reject -> 'Reason for
rejection (required)'; everything else (approve + start) ->
'Approval notes (required, >= 20 characters)'. The approve
placeholder now also signals intent ('Why this is ready to ship...').
Tests: render the queue, click Approve, assert the notes label says
'required' + '20' and does NOT say 'optional'. RED: label read
'Notes (optional)'; GREEN: 'Approval notes (required, >= 20
characters)'. eslint/typecheck/prettier clean.
* [F082] surface release-proposal query failures instead of silent hide
The card collapsed any non-404 backend failure (500 / network drop) onto
`!proposal` and returned null, so the CEO had no idea the release-proposal
endpoint was unreachable. Distinguish the cases: isError + a Retry affordance
vs the 404 null empty state that stays hidden. Mirrors PrReviewQueue.
* [F083] clear stale usage snapshot when /ws/system leaves connected
The hook synced wsState into the store but never dropped usageData when the
stream dropped, so on reconnect wsState flipped to "connected" before any
fresh USAGE_SNAPSHOT arrived and UsageOverviewPanel rendered the prior
session's totals/cost as if they were live. Clear usageData whenever state
leaves "connected" so the panel falls back to the polling summary until a
new snapshot lands. Connected->connected is a no-op clear skip.
* [F084] scope per-control disable to the in-flight mutation, not all
FeatureFlagsCard disabled every switch while any one flag toggle was pending,
and PlaybookReviewQueue disabled every row's Approve while any one approve was
pending — so the operator couldn't act on an independent control during a
slow round-trip. Gate the disable on the in-flight mutation's variables
(matching key / id) so only the control being mutated locks; the others stay
usable. The same-flag double-tap protection is preserved.
* [F085] reject submitting both project_id and product_id
validate() only checked 'at least one of project/product', so the dialog let
both be submitted together. The server silently lets product_id win at routing
and drops project_id, recording a misleading, never-used repo. Add a validator
that refuses the ambiguous submit with a clear error. The at-least-one rule and
the single-pick submit paths are unchanged.
* [F020] kanban: confirm admin-override drags that skip lifecycle preconditions
A drag on the operator kanban routes the status move through the admin
status-override, which bypasses the in-band lifecycle validator entirely.
That override is intentional (it's how an operator recovers a wedged task)
but it also let a careless drag skip material preconditions silently —
completing a task with no open PR, QA-bypassing, finishing docs on a task
whose docs aren't complete.
Leave the override intact but make the bypass explicit: compute the
preconditions the dragged move would skip (open PR, docs complete,
self-verified + commits + progress for submit-qa, visible non-terminal
subtasks for coordination-root targets) and, when any are skipped, hold the
move behind a confirmation dialog that lists exactly what's being skipped.
Precision over recall — only warn on what the panel can verify from the
task and its in-list children; never fabricate a 'satisfied' claim, and
stay silent on benign transitions that gate on nothing we can check.
The admin status-override capability is preserved (Confirm still fires it);
this only surfaces the bypass instead of letting it happen silently. Does
not touch the master-merge invariant — the board's updateTask is the
operator override, not the Main-PM merge path.
* [F086] prompter: restore parked cell content on project toggle off/on
rebuildCellWork appended a blank {summary:'', items:[]} entry for a newly-
selected cell, so toggling a cell's project OFF then back ON in the MegaTask
review card discarded the agent-authored per-cell summary/items — the entry
was dropped on toggle-off and re-added blank on toggle-on.
Park each draft's last per-cell content in client-only BatchProposal state
(parkedCellWork, keyed by draft index — never sent to the backend; confirm
ships only title/drafts/project_ids/route, and it ride-alongs into the
localStorage persist slice so the restore survives a reload mid-review).
rebuildCellWork gains an optional priorByCell map: a re-added cell with no
live entry restores its parked summary/items (with the new project_id) in-
stead of blanking; a live entry still wins over a stale parked copy so an
in-place edit is never regressed. parkCellWork is the pure merge seam
(prevParked seeds, live work overwrites) the setBatchDraftProjects updater
calls — kept pure so the updater stays a thin caller.
Tests: rebuildCellWork restore/blank-fallback/live-wins + parkCellWork
retain/overwrite/merge (6 new), 19 GREEN. eslint/typecheck/prettier clean.
No wire-payload change, no regression to the fill/drop/one-repo-per-cell
invariants.
* Updated domain
* [F087,F088] enforce panel token on live-chat bridges (Phase 5)
Add a CEO-bound, header-token-only gate (require_panel_token) at the route
level of the prompter_live + secretary_live bridges, which were the only
panel-facing API surface that ran unauthenticated. It mirrors the WS
_require_panel_token and _check_agent_auth_token contracts: in dev
(ROBOCO_AGENT_AUTH_REQUIRED unset) a missing token is allowed; a
presented-but-forged token is rejected even in dev; in prod nginx already
injects the CEO-signed X-Agent-Token on /api/ for GET + POST, so the SSE
stream (EventSource can't set headers) and the POSTs are now checked instead
of anonymous. Applied to start/stream/status/messages/stop on both routers;
preview_live_batch switched from CurrentAgentContext+noqa to the route-level
gate (genuinely auth-only). confirm/confirm-batch/re-interview keep
CurrentAgentContext (they use agent.identity). The container->relay /events
callback is intentionally left ungated (internal Docker network, opaque
session id) — gated by a test sentinel so Option B (spawn+SDK token wiring)
is a deliberate future decision. No panel/nginx/spawn/SDK changes; master
merge invariant untouched. 22 new TDD auth tests, 492 api tests green.
* [F089] honest WorkSession agent_id nullability across the read path
The work_sessions.agent_id column is nullable=True with ondelete=SET
NULL — deleting an agent nulls the FK on every session it ever held. The
ORM annotation lied (Mapped[UUID] non-optional), the converter papered
over the lie (typing_cast to a non-optional UUID), and the response
model rejected None outright (WorkSessionResponse.agent_id: UUID). A
session whose agent had been deleted crashed the GET endpoint with a
pydantic ValidationError instead of serializing agent_id: null.
Make the read path honest end-to-end:
- WorkSessionTable.agent_id: Mapped[UUID | None] (matches the column).
- WorkSessionResponse.agent_id: UUID | None (serializes null, no crash).
- session_to_response passes agent_id via typing_cast('UUID | None', ...)
to bridge SQLAlchemy's UUID[Any] to stdlib uuid.UUID while preserving
None-ness (the cast stays for the same mypy-plugin reason every other
field uses one; it no longer narrows away None).
WorkSessionCreate.agent_id stays UUID — at create time the claiming
agent is always known. The unused WorkSession pydantic read model is
left as-is (never materialized from a DB row). task.py:_needs_revision_dev
already None-guards ws.agent_id via to_python_uuid (returns None -> skip).
* [F090] drop auditor from write_roles on main-pm-board / board-private
The auditor is a silent, read-only observer on every channel, but the
channel catalog (roboco/foundation/policy/communications.py) listed it
in write_roles for main-pm-board and board-private 'for parity' with the
legacy CHANNEL_ACCESS table, while the actual silent-observer rule was
enforced only at the say/dm guard (content_actions._NO_COMMS_ROLES) and
PermissionService.can_write_channel's auditor short-circuit.
That left the catalog-only enforcement path — the HTTP messaging route
(messages.py send_message -> validate_channel_access) — authorizing an
auditor write that both the say/dm guard and PermissionService would
have blocked. A reader of the catalog also believed the auditor could
post to those channels, which is false.
Fix: remove Role.AUDITOR from write_roles on both channels (main-pm
+ board remain writers; ceo remains a writer on board-private). The
auditor stays in read_roles, so its silent read is unchanged. silent_roles
is left empty (matches the announcements precedent: auditor reads via
read_roles, not the silent bucket) — the DB seed and silent_observers
field are untouched.
Logical-regression check: the auditor's read access on both channels
is byte-for-byte preserved (still in read_roles, so validate_channel_access
read returns True via the direct list); the legitimate writers (main-pm,
product-owner, head-marketing, ceo) are untouched; CHANNEL_ACCESS is
derived from the spec so the foundation/seed drift tests self-adjust;
PermissionService.can_write_channel already short-circuited auditor to
False everywhere, so no behavior change there; AUDITOR_SILENT_ACCESS is
unchanged (auditor not added to silent_roles -> no DB silent_observers
change -> no group-access behavior change); the say/dm _NO_COMMS_ROLES
guard is unchanged. Tests: 3 new in test_channel_access.py — auditor
write on main-pm-board/board-private now raises ChannelAccessDeniedError
(RED before: returned True), auditor read still True, main-pm/ceo still
write.
* [F091] warn at spawn time when host grok auth.json is missing
GrokCliProvider._append_grok_auth_mount silently skipped the mount when
the host ~/.grok/auth.json was absent. The spawn still succeeded (docker
run returned 0 — the container was created), so the operator had no
spawn-time signal that the agent was doomed: the entrypoint's
`python -m roboco.llm.providers.grok_auth --check` backstop then
refused to start (exit 78) and the failure only surfaced later via the
container's log markers.
Fix: emit a spawn-time WARNING (module logger) naming the missing file
and the remediation (`grok login` on the host, or set
ROBOCO_HOST_GROK_DIR) when the mount is skipped. The spawn outcome is
unchanged — the container still starts and the existing exit-78 -> park
flow (F041) still catches it — but the operator now sees the missing
credential immediately instead of diagnosing a later exit-78.
Logical-regression check: the mount-present path is byte-for-byte
unchanged (auth.json exists -> the -v bind is appended, no warning); the
spawn still succeeds when auth is absent (no raise — the existing
test_grok_spawn_omits_auth_mount_when_absent still passes: no mount, no
crash); the exit-78 entrypoint backstop and the orchestrator's
exit-78-park handling (F041) are untouched; a module-level logger adds no
side effects. Tests: new test_grok_spawn_warns_when_auth_absent uses
caplog to assert a WARNING mentioning auth.json + `grok login` is
emitted on a missing-credential spawn (RED before: no warning; GREEN
after). 102 grok tests green; ruff/mypy clean.
* [F092] decode JWT exp when refresh omits expires_in
xAI's refresh-token response sometimes omits expires_in. Without it the
new access token kept the stale pre-refresh expires_at, so is_valid /
--check forever rejected a fresh token — and the refresh loop re-rotated
the single-use refresh token every tick, killing the credential (F006).
The access token is a JWT whose exp is the authoritative expiry: decode it
when expires_in is absent. Fallback to the documented ~6h TTL + a structlog
warning when the JWT exp is unreadable, so a fresh token is treated as live
instead of stale.
* [F093] serialize concurrent live-chat spawns under a per-agent lock
The intake and secretary agent ids are each a single fixed id, so two
concurrent start_intake_session / start_secretary_session calls raced on
the container name (docker run --name roboco-agent-<id>) and the
_instances[<id>] write: both passed the reap-prior check before either
registered, both ran docker run, and the last _instances write won,
orphaning the other container + its relay.
Add _intake_spawn_lock / _secretary_spawn_lock (asyncio.Lock) and wrap the
_spawn_intake_container / _spawn_secretary_container bodies so the second
start waits for the first to fully register before its own reap-prior check
runs. Distinct from self._lock (which stop_agent takes) to avoid a
reentrancy deadlock: the spawn body holds the spawn lock then calls
stop_agent (acquires self._lock) — lock order is always spawn_lock ->
self._lock, never the reverse.
* [F094] add a persistent-probe-failure escape hatch to provider parking
_on_probe_failure only incremented the failure counter and, at 10 failures,
sent a one-shot CEO notification. It never cleared the tracker, never gave
up, never fell back to time-expiry. _do_probe returns False for any non-2xx
AND any httpx error, so a permanently unreachable probe endpoint (removed
API key, network partition to the probe host, misconfigured base URL) kept
the provider parked forever — every agent on it gated by
_provider_spawn_parked, their tasks reaped to pending but the spawn gate
queuing every spawn, sitting pending forever. The only recovery was the
operator manually clearing the Redis key.
Past _PROBE_GIVE_UP_THRESHOLD (30) persistent failures, fall back to the
same time-expiry optimism the unprobeable-provider path uses (_do_probe
returns True when there is no probe URL): clear the park and resume parked
agents. If the provider is genuinely still down the real workload attempts
re-park via the 429/5xx path, so this is bounded burn — strictly better
than a silent forever-strand. Kept above the CEO-notify threshold (10) so
the operator still gets the notification first.
* [F095] orchestrator: parked-provider spawn short-circuits before expensive prepare
spawn_agent ran the full _prepare_agent_spawn (writes blueprint/settings/
briefing/MCP files, ensures the image, registers a STARTING instance) every
dispatcher tick only to bail at the after-prepare parked-provider check —
wasting all that file I/O while the provider stayed parked and leaving a
STARTING instance registered then downgraded to OFFLINE.
Move the parked check before _prepare_agent_spawn: resolve the route cheaply
via _resolve_agent_route (only provider_type is needed) and bail with a
minimal unregistered OFFLINE instance. The existing-running check stays
first (inside the lock) so a live agent is never replaced; a TOCTOU
re-check guards the unlocked window before prepare; the after-prepare
check is kept as a rare-race defense (a park landing during prepare).
* [F096] orchestrator: serialize fire-and-forget respawn persists per commit order
_persist_respawn_record is fire-and-forget per gate mutation; a respawn loop
fires count 1->2->3->4 in quick succession, scheduling one persist per
increment for the same (agent_slug, task_id). The ON CONFLICT DO UPDATE upsert
is row-level race-free, but the fire-and-forget tasks can still COMMIT out of
order: a slow stale persist (count=2) scheduled first can resolve AFTER a fast
fresh one (count=4) scheduled second, leaving the durable row at the stale low
count and re-burning the strike threshold on restart.
Fix: acquire self._respawn_persist_lock (new asyncio.Lock) as the FIRST await
in _persist_respawn_record, so acquisition order = task creation order (FIFO
ready queue) = logical schedule order, and commits land in that order. The
durable row always ends at the latest logical value. The lock lives in the bg
task, so the dispatcher hot path never blocks; persists are best-effort and
a slow one queuing the rest just delays the durable catch-up (in-memory record
stays authoritative).
* [F097] orchestrator: back off grok re-park retry_after within a rate-limit episode
_probe_target returns (None, {}) for grok — the grok CLI's xAI endpoint is
closed and the SuperGrok OIDC access token is not a valid bearer for the metered
api.x.ai, so a real probe would either no-op or strand grok parked forever.
_do_probe treats url-is-None as success (time-expiry optimism), so once the
60s retry_after passes the probe loop optimistically clears the grok park, a
cleared park dispatches a fresh grok agent that hits the still-active xAI 429,
exits 75, and re-parks — a flat ~90s crash-retry cycle for the whole xAI
rate-limit window (each cycle costs container startup + a rejected grok call).
Fix: track _grok_repark_count + _grok_last_park_at in _park_grok_rate_limited
and back the re-park retry_after off exponentially within one episode
(60 -> 120 -> 240 -> ... capped at 2**4 = ~16min cycle) so the churn dampens. A
gap past _GROK_REPARK_EPISODE_GAP_S (25min, > the capped cycle) means no re-park
for that long => the rate limit actually lifted => a fresh episode resets the
count to the base 60s, so recovery latency isn't penalized across episodes.
The first park in a fresh episode is unchanged at 60s.
* [F098] orchestrator: keep waiting record through a re-park during probe-success resume
resolve_wait deleted the waiting record (in-memory + durable) BEFORE calling
spawn_agent. A re-park in the window between the probe-success clear and the
spawn — the provider's rate limit lifts then immediately re-limits, or a second
provider limit lands — bails spawn with an OFFLINE instance (the parked-provider
short-circuit). Deleting the record first orphaned the agent: with no record
the probe-resume loop can never revive it and the spawn gate bails every tick,
so the agent is lost until the operator intervenes.
Fix: spawn first, then tear down the record only once a container actually
launched (instance.state == ACTIVE). On an OFFLINE bail the record stays so the
next probe-success re-attempts the resume. On a spawn EXCEPTION the record is
torn down + re-raised so the probe loop doesn't keep re-resuming a task that
moved to a different state (e.g. readiness refused -> task auto-blocked) —
matching the pre-fix behavior where the record was deleted before the spawn.
* [F099] wire pr_pass/pr_fail self_review block in the spec gate
The pr_pass/pr_fail ActionSpecs carry self_review_block=True, but
_gate_preflight never populated Context.original_developer_slug, and
actor_slug was read off agent.slug — which GatewayAgentView does not
carry, so it was always None in production. The block was structurally
dormant: a reviewer who was also the original developer of the
assembled PR could pass (or fail) their own work. The service-layer
_validate_not_self_review backstop only covers qa/documenter, not
pr_reviewer, so the spec gate is the only defense.
Set actor_slug=str(reviewer_agent_id) (GatewayAgentView has no slug,
so the UUID is the identity) and original_developer_slug from the
original_developer marker (a UUID stored as a string). Both resolve to
UUID strings, so the spec's string-equality comparison fires when the
reviewer IS the recorded original developer.
The marker is never set on assembled coordination tasks (only on
dev-leaf tasks at QA/doc claim), so the block stays dormant by design
in production — but the gate is now correctly wired to fire if the
marker were ever set to the reviewer. Zero production behavior change;
the dormant-in-production state is pinned by the no-marker test.
* [F100] atomic Redis probe-failure counter via server-side Lua
increment_probe_failures / reset_probe_failures did a non-atomic
get_state (GET) -> mutate -> set (SET) in Python. A concurrent
activate() re-park writes a FRESH episode blob (probe_failures: 0 +
fresh activated_at / retry_after / affected_agents / kind); if the
stale increment's SET landed after the fresh activate's SET, the stale
blob overwrote the fresh episode metadata AND un-reset the counter
(clobbering the new episode).
Redis single-threads a Lua EVAL, so a server-side read-modify-write
is indivisible: activate's SET is serialized entirely before or after
the script, never interleaved between the script's GET and SET. The
two scripts mutate ONLY probe_failures, so every other episode field
survives the bump. activate stays a single atomic SET (a fresh episode
resetting the counter to 0 is correct semantics).
* [F101] enforce PR-open state gate on gateway open_pr (parity with HTTP path)
* [F102] make project_id mandatory on pr_target (close cross-repo pr_number collision)
* [F103] make project_id mandatory on close_pull_request (close cross-repo collision)
* [F104] fail-closed on conventions resolution errors (block gate no longer silently disabled)
* [F106] compound (timestamp, id) keyset cursor for message pagination
get_messages used strict timestamp inequalities with a non-deterministic
order_by(timestamp.desc()), so equal-timestamp messages were cut by limit
on one page and excluded (strict < T / > T) from the next — they vanished
across pages. Bundled the (timestamp, id) pair into a MessageCursor dataclass
so the next page resumes exactly past the cursor's id at the shared
timestamp (or_: strictly-older OR same-timestamp-smaller-id for before; the
mirror for after), with a deterministic order_by(timestamp.desc(), id.desc())
so the last-item cursor is unambiguous. id is None for a legacy timestamp-
only cursor (strict inequality, prior behavior). The route builds cursors
from the flat before/before_id + after/after_id HTTP params; the schema now
carries the tie-breaker ids. Also clears PLR0913 (cursors replace the
before_id/after_id params).
* [F107] defer Redis bus publish until DB commit (no phantom notifications)
deliver() and _persist_and_deliver() ran inside the caller's open
transaction: the notification row was flushed but not committed, yet
NOTIFICATION_SENT was published to the Redis bus immediately. A commit
failure (DB hiccup, constraint, asyncpg error) rolled the row back while
connected WebSocket clients had already received a push for an id that
no longer existed — a phantom notification (notify_get -> NotFoundError).
Added a deferred-publish (transactional-outbox) helper: defer_bus_publish
enqueues the event on session.info and registers one-shot after_commit /
after_rollback listeners on session.sync_session the first time it is
called for that session. On commit, the after_commit listener schedules
the async drain via asyncio.create_task on the running loop (the listener
fires synchronously inside await AsyncSession.commit, so the loop is
active); the task handles are stashed on the session so callers/tests can
await them. On rollback, after_rollback drops the pending queue — a
rolled-back txn emits nothing. deliver() now builds the per-recipient
events up front (data materialized to strings, so deferral is safe even
if the ORM object later expires) and defers each; the delivered_at DB
marker stays in-tx (rolls back with the row). The bus block stays
best-effort (try/except + log) so a bus-init failure never propagates or
rolls back the notification row — matching the prior inline semantics.
This fixes every deliver/_persist_and_deliver caller at once (the two
cited in F107 plus the orchestrator + task.py deliver sites), since they
all commit the session afterward (the deferred publish fires on that
commit; the row is durable by the time the event goes out).
* [F108] atomic replace_chunks: single-txn delete+insert closes reindex race
* [F109] playbook curation status guards: approve/reject draft-only, archive approved-only
* [F110] draft slug TOCTOU: catch IntegrityError on flush -> ConflictError (no 500)
* [F113] collapse WorkSession creation to the validated service path
_create_work_session_if_needed constructed WorkSessionTable directly,
duplicating WorkSessionService.create's validation (existing-active
check, single-active-per-task supersede, project/task existence). The
two sites had drifted. Route through WorkSessionService.create instead,
mapping ConflictError to the idempotent 'if needed' None. Remove the
now-dead _supersede_other_active_sessions (create's
supersede_active_sessions_for_task replaces it).
Fix three pre-existing RED tests surfaced by the sweep (all confirmed
failing on the F110 commit before this change):
- test_fail_qa_work_session_fallback_excludes_qa_session: inserted two
ACTIVE work_sessions per task, violating uq_work_sessions_one_active
_per_task (migration 047). The QA session is now ABANDONED — still in
the fallback query's result set (the query filters by task_id +
agent_id, not status), so the exclude filter (agent_id != qa_id) is
still exercised and the dev is resolved.
- test_ceo_reject_routes_coordination_task_to_main_pm /
test_ceo_reject_routes_batch_umbrella_to_main_pm: ceo_reject emits an
audit row keyed to CEO_AGENT_ID, but the tests never seeded the CEO
agent row (fk_audit_log_agent_id_agents). Seed the CEO agent (get-or-
create, mirroring test_ceo_reject_writes_handoff_journal).
* [F114] single-claimant guard on pr_gate_claim
pr_gate_claim delegated straight to _qa_or_doc_claim, which overwrites
claimed_by / active_claimant_id with no single-claimant check. Two
reviewers race-claiming the same awaiting_pr_review task would
last-write-wins overwrite the first claim, and the first reviewer's
subsequent pr_pass / pr_fail would actor-mismatch against the new owner
(wasting a review cycle). The orchestrator's gate dispatcher already
prevents double-reviewer-dispatch in normal flow (one task -> one team
-> one reviewer + is_agent_active + per-tick spawned set), so the race
is only reachable via direct concurrent API calls (defense-in-depth).
Add a role-aware single-claimant guard in pr_gate_claim: lock the row
FOR UPDATE (serialize concurrent claims, mirroring the dev claim path),
then refuse only when the task is already actively claimed by a
DIFFERENT PR-reviewer. The gate task is owned by the PM at entry
(submit_for_review does not clear ownership, unlike submit_for_qa), so
the guard must distinguish a PM/dev owner — which the first reviewer
legitimately overclaims — from a competing reviewer claim; checking the
existing claimant's role (pr_reviewer) does exactly that. A re-claim by
the same reviewer is idempotent (skipped by the != check). The gateway
claim_gate_review handler already maps a None return to a clean
invalid_state envelope ('it may already be claimed; give_me_work for
the next'), so no gateway change is needed.
TDD: 3 integration tests in test_task_service_basics.py — reject a second
reviewer race-claim (returns None, first claim intact), allow the first
reviewer when the PM owns the root (regression guard for the
PM-owns-at-entry model), idempotent re-claim by the same reviewer.
Confirmed the reject test RED first (race-claim succeeded, overwriting
reviewer1).
* [F115] sample monorepo per (repo,workflow)/(repo,command) not per repo
The CI-watch and dep-update loaders collapsed a monorepo's cell-projects
to one canonical entry per repo (slug-sorted-first), so a repo whose cells
each carry their OWN ci_watch_workflow / dep_update_command had only the
canonical cell's workflow/command sampled — a red on another cell's
workflow or drift on another cell's lockfile was missed (under-count).
Refactor the shared one-per-repo collapse into _projects_one_per_key, keyed
by repo identity for external-PR discovery (unchanged: one review per PR per
repo), by (repo, effective workflow) for CI-watch, and by (repo, command)
for dep-update. Each distinct workflow/command is now sampled once; the
engines' per-git_url fix-task dedup still prevents duplicate fix tasks for
the same repo. _projects_one_per_repo now delegates to _projects_one_per_key.
key_fn uses a string annotation (Callable lives under TYPE_CHECKING, like
the existing Coroutine/Iterable annotations at lines 4193/5279).
* [R115] originate ci_watch/dep_update fix tasks as PLANNING coordination roots
The Main-PM-code-impossibility guard (commit e202ce39, Thread 4 of this
audit) made team=MAIN_PM + task_type=CODE impossible — a Main PM coordinates,
it does not write code. But the ci_watch and dep_update engines still
originated their fix tasks as task_type=TaskType.CODE assigned to main-pm,
so task_svc.create raised MAIN_PM_NO_CODE and NO fix task was ever opened
— a regression introduced by the earlier audit fix (confirmed: the engine
tests pass at e202ce39~1 and fail at HEAD).
Mirror the hardened self_heal_engine precedent (self_heal_engine.py:197)
which already uses task_type=TaskType.PLANNING for its Main-PM coordination
root with an explicit 'decompose the fix and delegate the code work to a
cell dev — the Main PM does not write the fix itself' description. Both
engines now originate PLANNING coordination roots with matching delegation
guidance in the description + acceptance criteria. confirmed_by_human
stays True for both (they ride the normal delivery flow without the CEO
gate, unlike self-heal — intentional per the architecture).
The dedupe/open-cap queries (list_open_ci_watch_tasks /
list_open_dep_update_tasks) key on source + non-terminal status + git_url,
NOT task_type, so the type change does not break dedup (still one open fix
task per repo).
The two source-test fixtures (test_ci_watch_source / test_dep_update_source)
created CODE+MAIN_PM tasks directly to exercise the listing queries — same
guard violation; switched to PLANNING (the queries assert on source/status,
not task_type, so the fixture type matches the engines' corrected type).
* [F116] hold the read-clone lock across the dep-probe local clone
dry_upgrade_changes_lockfile called ensure_read_clone (which syncs the
read clone under the _meta-conventions lock then releases it) and ran
'git clone --local --no-hardlinks <read_clone>' OUTSIDE the lock. A
concurrent ensure_read_clone -> _sync_read_clone (fetch + hard-reset to
origin's default branch) could mutate the read clone's working tree /
object db mid-clone, racing the clone and producing an inconsistent or
failing probe.
Split _probe_lockfile_change into _clone_local_into (the local clone,
run under the read-clone lock) + _probe_lockfile_on_clone (the upgrade +
git status, run without the lock on the now-independent copy). The probe
acquires _ensure_lock_for(slug, '_meta-conventions') — the same lock
ensure_read_clone syncs under — and holds it only for the clone step; the
upgrade operates on the full --no-hardlinks copy and never touches the
read clone, so the lock is released before it to avoid blocking
conventions reads for the upgrade duration.
The tiny gap between ensure_read_clone releasing the lock and the probe
re-acquiring it is safe: any concurrent _sync_read_clone completes under
the lock before the probe acquires, so the clone reads a stable state.
* [F117] stop the orchestrator in lifespan shutdown BEFORE closing the DB
The lifespan shutdown closed OptimalService + the DB, and only THEN did
bootstrap's finally block call orchestrator.stop() — so stop() ran with
the DB already closed. stop() drains fire-and-forget _bg_tasks writes
(respawn_tracker upserts, audit-log rows) and stop_agent finalizes work
sessions / agent state, all needing the DB still open; closing it first
silently dropped those final writes (the durable PM-respawn counter's
last few strikes, the metrics-bearing audit trail tail).
Move orchestrator.stop() into the lifespan shutdown path, BEFORE
close_optimal_service + close_db, guarded by a new get_orchestrator_or_none()
safe accessor (no crash when no orchestrator is wired — tests,
skip_orchestrator). bootstrap's finally-block stop() becomes an idempotent
safety net: stop() gains a _stopped flag (getattr-guarded so __new__-
constructed test instances still stop) so the double-call is a clean no-op,
not a re-stop of already-stopped agents / re-drain of an empty bg set.
* [F118] coerce a lone-string where_to_look into a list
where_to_look is a list-typed handoff field like consequences/next_steps
but was the only one NOT in the _wrap_scalar_in_list field_validator. A
well-intentioned where_to_look='src/api/' 422'd at the route with no
remediation envelope, and the agent's retry loop tripped the do-server
circuit breaker — the exact failure mode the other list fields were
hardened against. Add it to the mode='before' validator so a lone string
is wrapped into a one-element list before type coercion.
* [F119] sender reaps dead sockets on send error instead of waiting for receive idle timeout
* [F120] release a stopped agent's claimed task immediately on budget-kill/shutdown
* [F122] name the already-open PR in submit_up's None-state remediate
submit_up's create_pr pre-side-effect opens the cell→root PR BEFORE
submit_for_review runs (its pr_created gate requires it — lifecycle.py:1338-1343).
When submit_for_review returns None (a concurrent state change raced the task
out of in_progress between the precondition gate and the composed action), the
old remediate ('check task state — must be in_progress with PR ready') hid
that the PR was already open on GitHub — an orphaned external artifact the PM
could not reconcile. Mirror submit_root's F016 None-envelope remediate: name
the open PR, point the PM at re-fetch + reconcile, and note create_pr is
idempotent so a re-issue re-attaches to the existing PR (no duplicate). Pure
message improvement — zero behavior change; reordering is off the table
(create_pr must precede the pr_created gate).
* [F124] re-check dependency state before releasing a dependency-blocked claim
The unmet_dependency guard read dependency state via an unlocked SELECT, then
fired release_dependency_blocked_claim (a state mutation: claimed/in_progress
-> pending, clears branch_name, abandons WorkSession) as a side-effect BEFORE
returning the rejection. An upstream dependency that reached a terminal state
(completed/cancelled) in the microseconds between the read and the release left
the task NEEDLESSLY released — its branch cleared + WorkSession abandoned +
assignee bounced, only to be re-dispatched + re-claimed when the dependency-
completion re-dispatch fired a moment later.
Re-check unmet_dependency_ids immediately before the release and skip it
(returning None — proceed) when the upstream just completed. Dependencies are
monotonic (unmet -> met only; terminal states never reopen), so a fresh read
that now finds them met stays met: safe to proceed without releasing. The
'still unmet' path is byte-for-byte the prior behavior (no regression). The
cross-task residual window (upstream completes between the re-check and the
release) is not closable by a row lock on the dependent, but the re-check
narrows the window from [first read -> release] to [re-check -> release], and
in the common case the first read already sees met (no guard fires). No
committed-work loss either way (a dependency-blocked task has none; the branch
ref + commits persist across the branch_name clear).
* [F125] serialize same-parent delegate via per-parent advisory lock
The delegate sibling-dedup guard read the parent's existing subtasks via an
unlocked get_subtasks SELECT (the dedup read) then created the subtask (the
write) with no DB serialization between them. Two concurrent delegate calls
for the same parent (PM re-delegating while a reaper re-dispatches, or two
orchestrator ticks racing) each read a duplicate-free sibling set, each passed
the dedup guard, and each created a subtask — the parent got the duplicate the
guard exists to prevent (the smoke-run runaway pattern).
Fix: a PostgreSQL transaction-scoped advisory lock keyed by the parent task
id (seed 1, disjoint from the per-agent claim lock's seed 0), acquired at the
top of the delegate body before the first get_subtasks read (the briefing
context read AND the dedup sibling read) and held through create_subtask's
flush + the outer request commit. The second concurrent same-parent delegate
blocks until the first commits, then its dedup read sees the committed
sibling and is rejected.
Per-PARENT (not per-agent): a coordinator PM legitimately delegates many
subtasks under one parent in quick succession and plans many roots in
parallel — a per-agent lock would serialize all of a PM's delegates and
regress the PM coordinator concurrency feature. The per-parent lock
serializes only same-parent delegates (the dedup invariant is per-parent)
and leaves different parents untouched.
TDD: red-first ordering test (lock acquired before first get_subtasks read
and before create_subtask) + no-regression test (create still runs).
* [F127] per-task advisory lock prevents open_pr milestone double-emit
open_pr's idempotent re-entry guard (pr_number is not None) read t.pr_number
from an unlocked fetch. Two concurrent same-task open_pr calls (the
alive-but-unresponsive respawn race) both fetched pr_number=None, both passed
the guard, both ran the runner (GitHub 422 ensures one PR), and both reached
_record_milestone_progress -> a double-emitted 70% 'opened PR #N' entry.
Fix: acquire_task_lock (pg_advisory_xact_lock, seed 2) before the fetch, held
through the runner + milestone + request commit. The second concurrent call
blocks until the first commits, then its fetch sees the committed pr_number
and the idempotent guard short-circuits without re-emitting. Per-task (single-
active-task guard means same-task concurrent open_pr is only the bug case).
* [F128] require active claim on explicit-task content posts
_verify_explicit_task_ownership checked assigned_to, which is stale
across a reap/handoff (persists until reassignment; active_claimant_id is
cleared on release). A reaped agent could keep posting say/dm/note to its
former task. Add the active-claimant check when assigned_to == caller;
assigned_to=None keep its existing allow (read-side inspection between
reassignments uses evidence, which has its own ownership path).
Existing 'active owner' test mocks passed assigned_to=agent_id without
active_claimant_id; production sets both together on claim, so the mocks
were incomplete. Updated to set both — realistic, not a behavior change.
* [F129,F130] harden quality gate _run_one exit status + timeout cleanup
F129: _run_one returned 'proc.returncode or 0', masking a None returncode
(communicate returned without a recorded exit code — process killed
out-of-band) as 0 / success. Treat None as a non-zero failure (fail-closed).
F130: on timeout, _run_one killed the subprocess but never awaited wait()
— communicate() was cancelled so it never closed the stdout/stderr pipes,
leaving a transient zombie + leaked FDs. Await wait() after kill() to reap
the process and close the transports.
* [F132] timeout the conventions validator + reap on hang
_run_conventions_validator awaited proc.communicate() with no timeout —
a hung subprocess (tree-sitter deadlock, huge repo) hung the
i_am_done/pr_pass gate forever and orphaned the python subprocess on
orchestrator restart. Wrap communicate() in wait_for(120s); on timeout
kill+wait the proc and fail closed (could_not_run=True → block gate
refuses the submit), matching the validator's own fail-loud philosophy.
* [F135] re-check activity before sweeper closes a session (TOCTOU)
sweep_timed_out_sessions read last_activity_at once at the candidate
SELECT, then closed. A message landing in that window refreshed
last_activity_at in the DB, but the sweeper closed on its stale in-memory
value — closing a just-used session. Re-read last_activity_at fresh right
before the close and skip if the session is no longer timed out.
* [F136] cancel startup indexing task on OptimalService.close()
close() cancelled only the periodic update task, then cleared the plugins.
The startup _indexing_task (background auto-index, slow Ollama / large repo)
could still be mid-flight at shutdown and write against closed/cleared
plugins. Cancel and await _indexing_task FIRST (its tail starts the periodic
task, so ordering also prevents a late periodic spawn), then the periodic
task, then clear plugins.
* [F139] scope active_task_owns_branch to the polled project
active_task_owns_branch did an unscoped WHERE branch_name = ? — a cross-project
branch_name collision (UUID-derived 8-char prefixes, theoretical) made the
internal-PR reviewer skip the WRONG project's PR (project A's leftover PR
skipped because project B happened to have an active task with the same
branch). Pass project_id (in scope at the orchestrator call site) and add
TaskTable.project_id == project_id to the WHERE. Correct for single-project
tasks and MegaTask multi-repo batches alike: each root-subtask carries its own
project_id matching its own repo, so a branch on project A's repo is owned
only by a task whose project_id == A.
* [sweep] strip Fxxx audit-ID tokens + trim bloated comments/docstrings + add behavior-change docs
Post-audit sweep over the 135 audit-fix commits since 19a474d3:
1. Stripped every # Fxxx: audit-ID token from comments AND every Fxxx token
from docstring openings across 211 blocks / ~626 lines. The CEO flagged
these twice: audit-issue IDs in code confuse future devs/agents. The
descriptive text is preserved; only the Fxxx token is removed (and bloated
narrative blocks trimmed to 1-3 lines keeping the one non-obvious invariant).
2. Trimmed bloated comments/docstrings to the concise standard (1-3 lines).
3. Added missing behavior-change docs for the audit-fix batch: prompts/roles
(documenter, pr_reviewer, qa), user-facing docs (api auth, websockets,
agent-gateway, megatask, merge-model, task-lifecycle, grok, resilience,
conventions, panel, security, troubleshooting), and the RAG corpus (cell-pm,
main-pm, pr-reviewer, qa roles; conventions; messaging-tools; escalation;
megatask; task-claiming workflows).
Comment/docstring/prose ONLY — zero code-line edits (verified: the diff
contains no def/class/return/if/for/await/assignment/call lines). Gates green:
ruff format + ruff check clean, mypy clean on roboco/. The only pytest failures
are the pre-existing sync_branch tracing-decision gap (B1, 250be5c2) — not
sweep-caused and tracked separately.
* [fix] register sync_branch in VERBS_WITHOUT_TRACING
sync_branch (B1, 250be5c2) is a git-only rebase+force-push verb (composes=(),
no DB transition, side_effects=()) but was never registered in the tracing
parity tables, so test_every_intent_verb_has_a_tracing_decision failed.
Mirrors open_pr: a mechanical git op with inline preconditions (ownership),
no journal/plan rationale required.
* chore(release): 0.14.0
* [fix] resolve 16 mypy errors across 9 test files (make quality gate)
type-clean the test files so make quality (mypy roboco/ tests/) is green:
- Any-typed locals for the two TypeError-asserting scoping tests (bypass
the required-arg check without getattr/ruff B009)
- Any-typed view for the shutdown-drain _drain_bg_tasks override (bypass
mypy method-assign without setattr/ruff B010)
- cast("uuid.UUID", ...) / cast("UUID", ...) for SQLAlchemy UUID[Any]
returns (TC006-quoted), config=None for AgentInstance stubs, None-narrowed
await_args, Iterator return on a yielding fixture, UUID annotation on the
_task helper. No type:ignore / noqa.
* [docs] regenerate lifecycle artifacts for sync_branch + branch-keyed submit_root gate
The committed artifacts were stale: lifecycle.py grew the sync_branch verb
(B1) and the branch-keyed submit_root gate description (B2/B3) but the
generated markdown/json were never regenerated. make foundation-check
enforces artifact==generator(lifecycle.py); regenerating restores that.
No source change — pure generator output.
* [refactor] reduce xenon C-rank blocks to A (behavior-preserving)
Extract helpers / flatten conditionals in 11 blocks that rated C(11)+
under xenon --max-absolute B, dropping pr_gate.py module rank B->A in
the process. Pure move-and-call refactors: each extracted helper holds
the original logic verbatim and the caller delegates to it; no control
flow, return values, or side effects changed.
Sites: validators._extract_strs, sequencing.dev_task_collision_edges,
evidence_builder.build_task_handoff, intake_driver._coerce_draft,
task.claim_task_for_agent (2 guards), prompter.create_task_from_draft
(validate+assignee), pr_gate._gate_decision (3 helpers),
orchestrator._handle_stopped_container + _reap_with_service,
_impl._create_subtask_from_inputs + complete.
_impl helper returns tuple[TaskNature, list[str]] to preserve mypy
narrowing of acceptance_criteria at the TaskCreateRequest site.
Also fix vulture: rename unused __aexit__ param tb->_tb in
test_conventions_cache_put.py (was hidden while xenon short-circuited
the gate).
* [security] bash-guard uv run --active deny + CodeQL path-traversal fixes
Fix 1 (be-dev-1 brick prevention): bash-guard now denies 'uv run --active'
and 'uv run'/'uvx' against /app targets. In the agent container
VIRTUAL_ENV=/app/.venv is baked globally, so 'uv run --active' always
resolves onto the image-baked MCP-gateway venv and uv rebuilds it,
deleting /app/.venv/bin and bricking every MCP server spawn. Bare
'uv run' (workspace .venv, cwd-relative) is untouched.
CodeQL fixes:
- docs.py: replace bypassable '..' substring guard with a
resolve-and-contain helper (_resolve_contained_path). An absolute
path made pathlib reset (base / '/etc/passwd' == '/etc/passwd'),
letting read_doc/delete_doc reach arbitrary files. Applied to both
sinks.
- orchestrator.py: _safe_agent_path_segment at the spawn_agent
chokepoint (rejects traversal-shaped agent_id before any fs op) and
inside _remove_container (slug guard before the log-dir mkdir,
defense-in-depth).
- agent_sdk/server.py: /usage/sync transcript_path now resolved and
contained under ROBOCO_TRANSCRIPT_DIR with a .jsonl suffix requirement
(was Path(raw) — unauthenticated endpoint could stat arbitrary files).
TDD RED->GREEN across all four; make quality green (4890 passed).
* [fix] enum-parity gate: drop false-green mask, skip empty/unmigrated DB
The foundation-check gate ran the enum verifier behind
`|| echo "(skipped — postgres unreachable)"`, which swallows ANY
non-zero exit — including real drift — and prints 'All quality gates
passed'. On a host with a dockerized but empty/unmigrated `roboco` DB
(0 tables: the agentrole/team enum types don't exist), the verifier
connected, found every foundation value 'missing', exited 1, and the
mask relabeled it 'skipped' → false-green.
Fix:
- scripts/verify_postgres_enums.py: move skip semantics INTO the script.
Distinguish unreachable (skip, exit 0), DB-not-migrated/both-enum-types-
absent (skip, exit 0), real drift (exit 1), match (exit 0). Extract
pure enum_drift + should_skip_for_unmigrated helpers + a type_exists
probe so an empty DB is 'no migrated target', not drift.
- Makefile: drop the `|| echo` mask — real drift now fails the gate.
TDD RED->GREEN (10 tests); make quality green (10906 passed).
* [security] docs path guard: reject '.'/empty segments for clean 400
_resolve_contained_path used an '..' substring ban, which (a) left rel='.'
passing the guard — read_doc/delete_doc then got the base DIRECTORY itself
and raised IsADirectoryError (500) instead of a clean ValidationError, and
(b) false-rejected legit filenames containing '..' like 'v1..v2.md'.
Replace the substring ban with a raw-segment check (rel.split('/')) that
rejects any '.', '..', or empty segment. Path(rel).parts was the wrong tool
— pathlib collapses '.' and empty segments on 3.13, hiding them. The split
check catches '.' / 'a/./b' / 'a//b' / '..' / 'a/../b' while allowing
'v1..v2.md' ('..' inside a filename, no bad segment). The post-resolve
parents-containment check (the real defense) is unchanged.
TDD RED->GREEN (4 new tests); make quality green (10910 passed).
Follow-up to the CodeQL path-traversal review: the two CodeQL 'High' alerts
on this guard are false-positives-on-the-fix (resolve-and-contain already
contains the bypass); this hardening closes the one genuine low residual
(rel='.' -> 500) the review surfaced, which CodeQL did not flag.
---------
Co-authored-by: Renn F <rennf93@users.noreply.github.com>
An agent that ran `uv sync` / `pip install` against /app rebuilt the
orchestrator's MCP-gateway venv (/app/.venv) — breaking its own gateway tools
(every roboco-flow/-do/-git verb), stranding it, and getting its task reaped.
Deny package-mutation verbs (uv sync/lock/add/remove, uv pip, pip install)
whose target resolves to /app (cd /app, --project/--directory /app, /app/.venv,
UV_PROJECT_ENVIRONMENT=/app). Placed outside the ROBOCO_GUARD_SKIP_GIT gate so
it fires for every provider — the Claude PreToolUse hook and the grok exfil
hook alike. Reads of /app and workspace installs are untouched.
Closes the real bypass antfleet flagged in PR #223 (credit to them for the
finding): a denied git verb inside $(...) or backticks is expanded by the
shell before the wrapping echo/printf runs, so the skeletonizer's strip hid it
from the git check. This reworks it correctly where #223's fix could not land:
- targets the live source (docker/scripts/bash-guard-hook.sh, COPY'd to
/app/scripts/), not a path that doesn't exist;
- runs INSIDE the ROBOCO_GUARD_SKIP_GIT guard, so on grok it stays the native
--deny's job and never hard-cancels the run (#223 ran it unconditionally);
- excludes single-quoted strings and heredoc bodies (literal / data, matching
the skeletonizer), so a README documenting git verbs isn't a false positive;
- fails closed (a non-sentinel / python failure denies).
44 bash-guard tests pass (5 new: dollar/backtick/double-quoted substitution
deny, single-quoted literal allow, grok-skip allow).
Co-authored-by: Renn F <rennf93@users.noreply.github.com>
* feat(providers): pluggable agent providers + Grok (xAI) backend
Add a roboco/llm/providers/ seam — an AgentProvider lifecycle ABC and a
ProviderRegistry keyed by ModelProvider — so the orchestrator can drive
agent backends other than Claude Code.
The first non-Claude backend is GrokProvider for xAI's grok-build-0.1.
xAI is OpenAI-compatible only (no Anthropic-Messages endpoint), so a Grok
agent runs an OpenAI-protocol runtime pointed at https://api.x.ai/v1
rather than the ANTHROPIC_BASE_URL injection the other providers use. It
reuses the orchestrator's existing mount/auth assembly, so it inherits the
same MCP gateway + tool-manifest wiring as every other agent by
construction, and passes its prompt via env (never an argv positional).
The change is purely additive: only GROK routes through the registry;
Anthropic / Ollama Cloud / self-hosted spawns run the existing
_spawn_container path unchanged.
Includes:
- ModelProvider.GROK (migration 038) + a seeded Grok provider row
(migration 039) + a grok-build-0.1 catalog entry
- GET/PUT /api/providers/grok-key to store the xAI key (Fernet-encrypted,
reusing the existing provider-key machinery)
- ClaudeCodeProvider reference adapter over the current spawn
- unit tests for the registry, GrokProvider (gateway wiring, no
ANTHROPIC_* leak, prompt-injection safety, failure paths) and routing
The dedicated roboco-agent-grok image and the exact OpenAI-protocol CLI
invocation are the remaining piece to finalise with xAI.
* feat(providers): native Grok runtime — opencode image, config gen, panel key
Complete the native Grok (xAI) path so grok-build-0.1 runs as a real
RoboCo agent, not just the provider seam.
- roboco-agent-grok image (docker/agent-grok.Dockerfile): FROM agent-base
+ opencode (the OpenAI-protocol runtime). One image serves every role;
role behaviour comes from the mounted manifest / mcp-config / system
prompt, exactly as on the Claude path.
- Entrypoint renders opencode.json at spawn from the GrokProvider env
contract + the mounted Claude Code mcp-config.json
(roboco.llm.providers.opencode_config): translates RoboCo's gateway
servers (roboco-flow / roboco-do / ...) into opencode's mcp block,
declares the xAI OpenAI-compatible provider + model, and wires
permissions + instructions. Pure, unit-tested translation.
- Orchestrator registers GrokProvider with the registry-qualified image
(_qualify_agent_image) so it resolves in local and registry deploys.
- Compose (both files + the registry compose) gain an agent-grok-image
builder service.
- Panel: a Grok (xAI) API key card on the AI Providers page, plus the
grok ModelProvider value.
KNOWN PARITY GAP (opencode runtime): the bash-guard PAT-scrub and the
transcript-based usage/cost capture are Claude Code hooks and do not
transfer to opencode. bash permission is operator-tunable
(ROBOCO_GROK_BASH_PERMISSION) so a deployment can fail closed until a
security/usage-parity opencode plugin lands. That plugin and live E2E
validation are the remaining work to finalize with xAI.
* ci(release): build + publish the roboco-agent-grok image
Add roboco-agent-grok to the release workflow's image build/publish map so
registry deploys carry the Grok runtime image (parity with every other
agent image). Split from the feature commit because pushing a workflow
change requires a workflow-scoped token.
* fix(migration): commit the grok enum value before seeding (autocommit_block)
CI's "Apply database migrations" failed with asyncpg
UnsafeNewEnumValueUsageError: alembic runs the whole upgrade in a single
transaction, so migration 039's INSERT used 'grok' in the same transaction
that 038 added it — which Postgres forbids. Splitting into two migration
files did not help (one transaction spans both). Wrap the ALTER TYPE ADD
VALUE in op.get_context().autocommit_block() so the value commits before 039
(and any later migration) uses it. Still renders in offline --sql, so the
enum-migration-parity test is unaffected.
* feat(grok): price grok-build-0.1 + secret-scrub opencode plugin
- pricing.py: add grok-build-0.1 rates ($1/1M input, $0.20 cached, $2/1M
output), verified against xAI's published pricing. Grok is a priced
non-Anthropic model, so cost computes the moment usage is captured.
- secret-scrub.js: an opencode tool.execute.before plugin porting the
security-critical bash-guard deny rules (git network ops, credential-file
reads, /proc env, internal-host HTTP, roboco.* imports, ROBOCO_AGENT_ID
forgery, env dumps, destructive rm) to the opencode runtime — restoring the
guard the Claude Code hook can't provide there. Throwing denies the call
(confirmed by opencode's env-protection example). Wired into the generated
opencode.json plugin array + baked into the grok image.
Deny logic verified via node (9 deny + 5 allow cases). UNVALIDATED against a
live opencode runtime: confirm it fires in the live E2E spawn before a Grok
dev-agent touches a real repo; the bash permission is operator-tunable as a
second gate.
Cost CAPTURE (distinct from pricing) is intentionally NOT built yet: opencode's
plugin hooks expose model info but no token/usage object, so the capture path
is unconfirmed and needs the live spawn to settle.
* feat(grok): read opencode session usage for cost capture
Confirmed by inspecting a local opencode run: opencode persists per-session
usage in SQLite at ~/.local/share/opencode/opencode.db — the `session` table
carries cost + tokens_input/output/reasoning/cache_read/cache_write. xAI's
response usage object (prompt_tokens, completion_tokens,
prompt_tokens_details.cached_tokens, completion_tokens_details.reasoning_tokens)
maps directly onto those columns.
Add opencode_usage.read_session_usage / cost_for_session: read the opencode DB
and price the tokens via roboco.billing.pricing (our cost stays authoritative;
opencode's own `cost` column is kept for reference). Tested against a fixture DB
mirroring the real schema (single session, summed sessions, missing/empty DB).
Remaining wiring (for the live spawn): mount the opencode data dir on grok
spawn + call cost_for_session at reap to record the usage rollup.
* fix(grok): correct opencode provider (Responses API), stdin, reasoning cost
A live opencode run against api.x.ai/v1 surfaced three real bugs:
1. Provider package — grok-build-0.1 is driven via the OpenAI Responses API
(opencode calls model.responses()). @ai-sdk/openai-compatible is
chat/completions only and errors "responses is not a function". Switch the
generated opencode.json provider + the grok image to @ai-sdk/openai.
2. Headless hang — `opencode run` blocks after init without a TTY; close stdin
(`< /dev/null`) in the entrypoint so it proceeds to the model call.
3. Reasoning-token cost — grok-build-0.1 is a reasoning model; reasoning tokens
bill as output but opencode stores them in a separate column. cost_for_session
folds tokens_reasoning into output (else ~22x undercount).
Verified end-to-end against a real session row (input=6120, output=1,
reasoning=226, cache_read=1856): our pricing reproduces opencode's stored USD
cost ($0.0069452) exactly. Tests anchored to that real row.
* feat(grok): first-class xAI/Grok routing mode (UI + backend)
The Routing-mode toggle had Anthropic / Ollama / Self-Hosted / Mix but no way
to route the whole org to Grok. Add it end to end:
- backend: apply_mode("grok") + _apply_grok (GLOBAL default -> grok-build-0.1) +
derive_mode "grok" detection; ApplyModeRequest/ModeResponse accept "grok".
- panel: a "Grok" routing-mode card (between Anthropic and Ollama, gated on the
xAI key) + flipToGrok; a Grok group in the per-agent mix dropdown +
catalogGrokOnly + a grok ProviderBadge variant; the mix-save key check and
the AI-routing description now cover Grok.
- tests: integration derive_mode/apply_mode "grok" cases (+ grok provider row
in the fixture).
Gated: ruff + mypy clean; panel typecheck + lint clean.
* feat(grok): reasoning-effort by role (cut grok-build cost on cheap roles)
grok-build-0.1 reasons heavily by default and reasoning bills at the output
rate (a live "say ok" call emitted ~300 reasoning tokens, ~85% of its cost).
Confirmed live that opencode's `--variant minimal` cuts reasoning ~54%
(298 -> 136 tokens, same prompt).
GrokProvider now picks reasoning effort by role: code-quality roles (developer,
qa, pr_reviewer) keep full reasoning; coordination / docs / board roles
(cell_pm, main_pm, documenter, product_owner, head_marketing, auditor, prompter,
secretary) run "minimal". It's passed to opencode via the entrypoint's
`--variant`. Operators can force one effort for ALL grok agents with the
ROBOCO_GROK_REASONING_EFFORT env (minimal | high | max, or default/full).
Tests cover the role map, the env override, and the spawn env wiring.
* style(panel): show the Grok (xAI) key card above the Ollama card
* fix(grok): stop opencode subagent-stream hang at the config layer
The Grok pr_reviewer wedged in_progress forever: opencode's default agent ran
with the subagent `task` tool enabled, spawned an Explore subagent on
grok-build-0.1 whose model call opened an SSE stream that went idle, and the
run hung with no timeout.
- Hard-disable opencode's subagent `task` tool in the generated opencode.json.
No RoboCo role uses opencode-internal subagents — work flows through the
gateway verbs — so removing the tool kills the hang trigger outright.
- Set provider.xai.options.timeout + chunkTimeout (operator-tunable via
ROBOCO_GROK_REQUEST_TIMEOUT_MS / ROBOCO_GROK_CHUNK_TIMEOUT_MS) as the
defence-in-depth backstop; chunkTimeout aborts an idle stream.
- Bundle the permission + timeout + subagent knobs into an OpencodeGuards
dataclass (keeps the builder under the arg-count gate).
- Drop the dead ROBOCO_AGENT_TOOLS spawn env (it had no consumer); opencode
tool restriction lives in the rendered config now.
* feat(grok): reaper watchdog kills wedged opencode containers
The heartbeat reaper deliberately skips a task whose assignee holds a live
ACTIVE container, so a Claude agent deep in a long edit/test cycle isn't
churned out from under live work. A wedged opencode container breaks that
assumption: it stays ACTIVE while firing no gateway verb, so its heartbeat
never advances and the live-instance skip would shield its task forever — the
exact way the Grok pr_reviewer parked in_progress.
Add a longer grok-idle kill threshold (ROBOCO_GROK_IDLE_KILL_SECONDS, default
900s, well past the stream chunk timeout). A GROK instance idle past it is
force-removed (its logs dumped to disk first) and evicted from the instance
registry, so the same reaper pass then releases the task. Only GROK runtimes
are eligible — a quiet Claude agent keeps the heartbeat-skip protection.
* feat(grok): guard interactive roles from GROK routes (interim)
intake (prompter) and secretary run a held-open chat session driven by the
Claude Agent SDK. GROK has no interactive runtime yet, and a GROK route for
those slugs would be spawned with the route creds injected as ANTHROPIC_*
against api.x.ai/v1 — the wrong protocol — producing a silent, empty reply
(the blank intake we observed).
Downgrade a GROK route for intake-1/secretary-1 to the Anthropic default with
a logged warning. The one-shot delivery roles route to GROK unchanged. This
guard is replaced by the real interactive fork once the opencode interactive
driver lands.
* feat(grok): capture one-shot Grok usage/cost from the opencode store
A GROK agent runs opencode, not Claude Code: it has no SDK /usage/status
server and writes no Claude transcript, so _resolve_final_token_usage found
nothing and every Grok agent finalized at 0 tokens / $0 — the opencode_usage
reader existed but had no caller.
- Mount a per-agent opencode data dir ($DATA/opencode/<agent_id> →
/home/agent/.local/share/opencode) so opencode.db is captured, and mount the
same host dir into the orchestrator (/data/opencode) in all three compose
files so the finalizer can read it back — the opencode analogue of the
mounted Claude transcript.
- _resolve_final_token_usage branches on provider_type: GROK reads opencode.db
via opencode_usage (reasoning folded into output, billed at the output rate)
and skips the SDK/transcript path. A 0-token read logs a WARNING so a silent
mount failure isn't mistaken for a real zero-cost run.
- ROBOCO_OPENCODE_DATA_DIR overrides the in-orchestrator path for local runs.
* feat(grok): make interactive spawns first-class on AgentProvider (additive)
The AgentProvider ABC modelled only the one-shot lifecycle (spawn/stop/
health_check/remove), so the interactive intake/secretary roles could never
route through a provider. Add an opt-in interactive surface:
- supports_interactive class flag (default False).
- InteractiveSpawnSpec: the resolved AgentConfig + session id + role-specific
image + optional HMAC token — everything a provider needs without importing
orchestrator internals.
- spawn_interactive(spec): a non-abstract default that declines via
ProviderError, so every existing one-shot provider is unchanged.
Pure scaffolding — no provider opts in yet (GrokProvider flips the flag when
its interactive driver lands). Zero behavioural change.
* feat(grok): Grok-native interactive runtime (opencode serve) — container side
Builds the Grok analogue of the Claude intake/secretary live-session runtime,
satisfying the same IntakeSession seam so the existing IntakeDriver loop,
message source, relay, and StreamChunk panel contract are reused unchanged:
- OpencodeServeSession: a held-open `opencode serve` session (context persists
across turns) where each human turn is one synchronous POST /session/:id/
message; normalize_opencode_message maps the reply parts to text/thinking/
tool_use/draft/turn_end chunks (draft via a propose_draft tool part or the
fenced roboco-draft fallback). Doc-verified against opencode's server API.
- grok_intake_main / grok_secretary_main: container entrypoints mirroring the
Claude mains but yielding an OpencodeServeSession; they render opencode.json
(xAI provider + MCP + system prompt) first, then run the receiver + driver.
- roboco-agent-grok-prompter / -secretary images (FROM roboco-agent-grok) +
their builder services in all three compose files.
UNVERIFIED-LIVE: the opencode serve flow + exact Part schema + draft path need
a live run against grok-build-0.1 (the part mapping is defensive). The
orchestrator wiring that routes a GROK intake/secretary route to these images
is the next step (a design decision is open — see the handoff notes).
* feat(grok): route interactive intake/secretary to opencode-serve images
Wire the GROK interactive path the in-place way (matching how the interactive
roles already choose ANTHROPIC_* per route), so a GROK route launches the
Grok-native opencode-serve image instead of the Claude SDK-driver image:
- _spawn_intake_container / _spawn_secretary_container pick the
grok-prompter / grok-secretary image (ensuring the base→grok→interactive
build chain) when the route is GROK, and stamp provider_type on the spec +
AgentConfig so finalize routes usage to the opencode store.
- _build_intake_run_cmd / _build_secretary_run_cmd inject OPENAI_* + the
opencode store mount + system-prompt env for GROK via a shared
_append_interactive_provider_env, keeping ANTHROPIC_* for every other
provider. The intake's minimal mounts (no gateway MCP) are preserved, so
Grok intake matches the Claude intake's tool surface (the spec).
- Add a per-agent opencode store mount to the interactive host paths so
interactive Grok usage/cost is captured like the one-shot path.
Removes the interim Phase-0 routing guard (the real path supersedes it) and
retires the unused AgentProvider.spawn_interactive/InteractiveSpawnSpec seam —
the interactive roles have a bespoke assembly that the one-shot provider
surface doesn't fit, so the fork lives in their own builders.
UNVERIFIED-LIVE: end-to-end intake/secretary chat on Grok needs the stack up +
opencode serve confirmed against grok-build-0.1.
* feat(grok): surface intake/secretary in the mix-mode picker; doc guardrail parity
- Panel: add intake-1 (prompter) and secretary-1 to the mix-mode per-agent
routing list so an operator can assign Grok (or Claude) to the interactive
roles from the UI; assigning a Grok model routes them to the opencode-serve
image. tsc + eslint clean.
- opencode_config: correct the now-stale parity note — bash-guard is ported
(secret-scrub.js) and usage/cost is captured (opencode store); the remaining
gap is the budget/loop/stop/prompt-injection hooks, which need a sidecar
plugin (open decision), with ROBOCO_GROK_BASH_PERMISSION as the interim gate.
* test(grok): mypy-clean the reaper watchdog + interactive spawn tests
The CI mypy scope (roboco/ tests/) flagged test-only typing issues my per-file
runs missed: direct method assignment (orch._remove_container = AsyncMock())
trips [method-assign], and a module-level dict[str,str] is invariant against
the dict[str, str|None] the run-spec expects.
- Use monkeypatch.setattr for _remove_container in the watchdog tests.
- Annotate the shared _HOSTS as dict[str, str | None].
Production code unchanged; mypy roboco/ tests/ is green.
* feat(grok): cost-ceiling kill-switch (budget-guardrail parity)
Claude Code's per-agent token-budget hook fires against the SDK :9000 server;
opencode exposes NO usage/budget hook to a plugin (confirmed against its plugin
docs), so the budget kill-switch can't be a plugin/sidecar — the orchestrator
enforces it instead.
_enforce_grok_cost_budget runs each dispatch tick: for every ACTIVE GROK
container it reads cumulative cost from the opencode store (the Phase-2 reader)
and kills + evicts it past ROBOCO_GROK_MAX_COST_USD (0 = off), after which the
reaper releases the freed task. This also catches a runaway loop that keeps
firing verbs (so it evades the idle watchdog) but still burns cost.
Covers the budget/runaway-burn slice of guardrail parity. The remaining Claude
hooks (prompt-injection PRE-gate, stop-guard terminal-verb) have no blocking
opencode equivalent — opencode's message/stop hooks are observe-only — and the
interactive reasoning-variant has no opencode.json/serve knob (CLI-flag only);
both are pinned for a live probe rather than shipped as a guess.
* docs(grok): document Grok's reduced guardrail posture (honest, not blocking)
Grok agents run on opencode, not Claude Code, so they do NOT have full
guardrail parity — claiming otherwise would be false. Document it truthfully
and keep them usable rather than blocking them.
- Panel routing card: an amber caveat shown in Grok/Mix mode — command/
secret-exfil guard + cost cap apply to Grok, but the prompt-injection guard
does NOT (opencode cannot block a turn); Anthropic/Ollama/Self-Hosted run
through Claude Code with the full guard set; prefer those for agents that
ingest untrusted or cross-agent content; Grok is safe for trusted work.
- docs/self/architecture/llm-provider-security.md: the reference — the two
runtimes, which provider uses which, the per-guardrail parity matrix, why
the injection/stop gaps exist (opencode hooks are observe-only), and the
routing recommendation (delivery roles handling untrusted content → a
Claude-Code-runtime provider).
Panel tsc + eslint clean.
* fix(grok): make the live interactive path work — store perms, error surfacing, variant
Found by actually running opencode serve locally (the path was doc-verified but
never executed). Three fixes:
1. EACCES on the opencode store mount (the live intake crash): on Linux docker
auto-creates a missing bind source as root:root, so the non-root agent user
could not mkdir/write in /home/agent/.local/share/opencode and opencode died
at boot. _ensure_opencode_data_dir pre-creates the per-agent dir 0777 before
the mount (one-shot via the _GrokHost seam, interactive in both spawns).
2. Silent blank reply on a model error: opencode reports a turn failure in
info.error with parts=[], NOT as a part — verified live (a bad xAI key
returns info.error APIError). send() / normalize_opencode_message now surface
it as an "error" StreamChunk so a failed turn is never blank (the original
intake bug class). Confirmed live: the error now renders.
3. Reasoning variant on the serve path: the live OpenAPI shows the message body
accepts a "variant" field (it is NOT CLI-only, as the docs implied), so the
pin is unblocked. send() passes ROBOCO_GROK_VARIANT as the per-turn variant;
the orchestrator sets it per-role (_reasoning_effort_for) for interactive
Grok, the same lever as the one-shot --variant.
opencode serve startup, POST /session, session-id extraction, the part-type
mapping (text/reasoning/tool), and the error path are all validated against a
live opencode 1.17.8. A real successful grok reply still needs a funded key.
* fix(grok): pre-create agent-owned ~/.local in the grok image (opencode state EACCES)
Running the built grok-prompter container surfaced a second EACCES the
mechanism analysis missed: bind-mounting the opencode store at
~/.local/share/opencode makes docker create the intermediate ~/.local AS ROOT,
so the non-root agent user then cannot create its sibling ~/.local/state and
opencode dies at boot. Pre-create the ~/.local tree agent-owned in the image so
the mount leaves the parents writable. Complements the orchestrator 0777
host-source pre-create (which covers the bind source on Linux).
Verified live: with this fix the container starts clean, opencode serve opens
the session, a POST /turn produces a real grok reply, and all chunks
(thinking/text/turn_end) reach the relay endpoint.
* feat(grok): prompt-injection guard for Grok (parity with the Claude hook)
The injection guard is RoboCo's own hook (user-prompt-hook.sh), not a runtime
built-in, so it can be recreated at our input boundary regardless of runtime —
opencode's lack of a blocking pre-prompt hook is irrelevant.
- prompt_guard.detect_injection: the deny patterns ported to reusable Python.
- IntakeDriver._run_turn scans every interactive turn before sending it to the
model and denies a match as an error chunk. Covers BOTH Grok (opencode) and
the Claude SDK intake (which runs with setting_sources=[] and so never loaded
the bash hook — it was unguarded too).
- The one-shot grok entrypoint scans ROBOCO_INITIAL_PROMPT and refuses a
poisoned task prompt (parity with the Claude UserPromptSubmit deny).
- Broadened the pattern (Python + the bash hook, kept in sync) to catch the
multi-qualifier canonical phrasing "ignore all previous instructions", which
the single-qualifier original missed — without false-positiving on
"ignore the linting rules" (an intermediate non-qualifier word breaks it).
So Grok now has the command/secret-exfil guard (secret-scrub), the cost cap,
AND the injection guard. Verified: 94 agent_sdk tests pass; bash + Python agree
on detect/miss cases.
* docs(grok): drop the security disclaimers — injection guard closes the gap
With the prompt-injection guard now recreated for Grok (prior commit), the
"Grok lacks the injection guard / prefer Claude for delivery roles" warning is
no longer true, so remove it:
- Panel routing card: replace the amber "prefer Claude / not safe" caveat with
a neutral one-liner — Grok agents run on opencode; the command/secret-exfil
guard, the prompt-injection guard, and the cost cap all apply.
- docs/self/architecture/llm-provider-security.md: prompt-injection row flips to
"yes" for Grok; intro + routing recommendation updated to "effective security
parity, any agent (incl. delivery roles) can run on Grok"; the only remaining
unported hook is the non-security stop-guard.
- opencode_config docstring: the remaining gap is now just the stop-guard
(budget + injection are covered).
Panel tsc + eslint clean.
* fix(grok): allow external-directory reads so the pr-reviewer can work
Live NAS run showed the Grok pr-reviewer claim the review and fetch the diff,
then write it to /tmp and FAIL to read it back: opencode auto-denied
"external_directory (/tmp/*)" — its file tools refuse paths outside the project
cwd, and in headless serve/run mode an "ask" permission auto-rejects (no human).
Add permission.external_directory (default "allow", env
ROBOCO_GROK_EXTERNAL_DIR_PERMISSION) to the generated opencode.json. The
container is the sandbox and secret-scrub still blocks credential-file reads, so
allowing in-container external-dir reads is safe and unblocks legitimate scratch
use (e.g. the pr-reviewer grepping a large diff in /tmp).
Verified live against grok-build-0.1: with external_directory:"allow" the Read
tool reads a file outside cwd and returns its contents (no auto-reject); the
plain-string form is accepted by opencode 1.17.8.
Needs a rebuild of roboco-agent-grok + a pr-reviewer re-run on the NAS to confirm.
* refactor(grok): split eligibility out of _maybe_kill_wedged_grok (xenon C -> B)
CI complexity gate (make quality -> xenon --max-absolute B) flagged
_maybe_kill_wedged_grok at rank C — too many guard branches in one method.
Extract the kill-candidate decision into _wedged_grok_slug(task, last_heartbeat)
-> slug | None (recent-heartbeat / no-owner / not-ACTIVE / not-GROK all yield
None); _maybe_kill_wedged_grok now just kills + evicts the returned slug.
Behaviour is identical (same guards, same order) — the reaper watchdog tests
pass unchanged. xenon now passes on the full package; ruff + mypy clean.
* feat(grok): start the in-container SDK server + budget feed (Claude parity)
The keystone of the Grok parity work (CEO's "take Claude as baseline, create
what's missing" call): the one-shot Grok container now starts the same SDK
server the Claude path runs, so the per-verb circuit breaker (the flow/do MCP
servers already POST /verb/attempted to it), the per-session budget/loop
counters, the terminal-verb tracking, and the SessionEnd post-mortem all work
on Grok instead of being silently absent.
- entrypoint: launch roboco.agent_sdk.server (bare venv python, not `uv run`
which would re-sync the drifted clone lock and stall), wait for /health,
reset counters; run opencode WITHOUT exec so the script regains control to
run the post-mortem and the silent-exit substitute after the run returns.
- budget-feed.js: opencode plugin that gates on /budget/status in
tool.execute.before (halt/loop deny — the only place to stop a runaway
one-shot run; opencode has no PostToolUse-deny) and records the executed
tool + args-hash in tool.execute.after. Fail-open; bare-verb normalization
for MCP-namespaced terminal verbs.
- silent-exit substitute: on a graceful exit with no terminal verb the
entrypoint posts /terminal/force_substitute so the task isn't left stuck
claimed/in_progress (Stop-hook parity at the boundary).
- opencode_config: wire budget-feed into the plugin array; add
ROBOCO_OPENCODE_EXTRA_PLUGINS so per-image role tool plugins load scoped to
one role; read the per-role ROBOCO_GROK_EDIT_PERMISSION.
Targeted gate green (ruff/mypy/xenon + opencode_config tests; node --check on
the plugins; bash -n on the entrypoint).
* feat(grok): give the Grok Secretary its CEO-authority tools (blocker)
The Grok Secretary could chat but had zero directive tools — it could not read
company state or act on a CEO command, so it was non-functional. This is the
integration blocker.
- secretary-tools.js: opencode plugin registering read_company_state /
read_task / submit_directive via the Hooks.tool API, each calling
/api/secretary/* with the container's HMAC agent token — a direct port of the
Claude Secretary's SDK tools (secretary_driver.build_secretary_options). The
high-impact directive kinds stay gated server-side (queued for CEO confirm).
- agent-grok-secretary.Dockerfile: bake the plugin and scope it to this image
via ROBOCO_OPENCODE_EXTRA_PLUGINS, so only the Secretary carries CEO authority.
- grok_secretary_main: correct the docstring that falsely claimed the tools
reached the API "through the mounted MCP gateway" (there is no gateway mount;
they're an opencode plugin).
- secretary.md: name the three tools and restate the confirm-before-act gate.
Verified locally that opencode loads a file-path plugin importing
@opencode-ai/plugin and resolves the package; the live model-tool-call +
backend round-trip is flagged UNVERIFIED-LIVE for the NAS.
* feat(grok): give the Grok Intake its propose_draft tool (draft card)
The prompter prompt tells the model to call propose_draft when the spec is
ready, but on Grok that tool didn't exist — so no draft chunk, no panel draft
card, and the human couldn't launch a task from a Grok intake chat.
- intake-tools.js: opencode plugin registering propose_draft via Hooks.tool;
the execute() only ACKs — the driver (OpencodeServeSession.normalize ->
_is_propose_draft -> _draft_from_tool_input) intercepts the tool CALL and
emits the `draft` chunk the panel renders.
- agent-grok-prompter.Dockerfile: bake the plugin, scoped to this image via
ROBOCO_OPENCODE_EXTRA_PLUGINS (delivery roles never draft).
- test: a propose_draft tool part normalizes to a draft chunk (not a tool_use).
The live tool-call -> draft-card path is flagged UNVERIFIED-LIVE for the NAS.
* feat(grok): scope opencode edit/bash/external-dir permissions per role
Grok wrote ONE global permission block, so a Grok pr_reviewer (or qa / PM /
auditor) ran with edit=allow + bash=allow on untrusted PR content. Now the
permissions are derived per role, mirroring orchestrator._get_role_permissions
on the Claude path:
- edit — allow only roles that write code (role_config.allows_write:
developer / documenter); everyone else edit=deny.
- bash — allow only roles that legitimately run a shell (developer /
documenter / cell_pm / main_pm); the read-only reviewers (qa / pr_reviewer /
auditor) and the board get bash=deny. secret-scrub still guards the rest.
- external_directory — only the pr_reviewer reads scratch outside its cwd (the
/tmp diff); delivery roles get deny.
One-shot roles resolve these in GrokProvider._append_grok_env; the interactive
intake/secretary set edit=deny + bash=deny in the orchestrator (intake keeps
external-dir reads for sibling product repos, the secretary does not). The
Claude path is untouched — the permission env is a GROK-only contract.
Targeted gate green (ruff/mypy/xenon + provider + interactive-spawn tests).
* feat(grok): park the provider on an xAI 429 (break the respawn loop)
A one-shot grok run that hit an xAI 429 exited without a terminal verb; the
dispatcher then re-spawned the same task every tick (429 -> exit -> respawn), a
container/token/cost loop with no living agent to call i_am_blocked.
- entrypoint: detect a rate-limit signature in the run output and exit 75
(EX_TEMPFAIL); a rate-limited task is NOT substituted — it must be retried.
- _handle_stopped_container: on a grok exit 75, park the provider via the
rate-limit tracker (retry_after window) instead of crash-retrying, and don't
count it as a crash. The existing probe-resume loop clears the park after the
window (unknown-provider time-expiry fallback) and the task is retried.
- spawn_agent: a grok-only, fail-open guard skips the launch while the provider
is parked, so the dispatcher no-ops instead of looping. The Claude path is
untouched.
Targeted gate green (ruff/mypy/xenon + new rate-limit tests; bash -n on the
entrypoint).
* feat(grok): close the secret-scrub bash-guard parity gaps
secret-scrub.js (the opencode bash guard) was missing three rules the Claude
bash-guard hook has, leaving a Grok dev able to read secrets the Claude path
blocks:
- source / dot-source of a credential-bearing file (source .env, . ./.env,
.bashrc / .git-credentials / .netrc / /proc/*/environ).
- interpreter one-liner reading a credential file
(python -c "open('.env')", node -e "readFileSync('.git-credentials')").
- git-ops check now runs on a SKELETONIZED command (heredoc bodies + echo/printf
args stripped) so a README/heredoc that merely documents `git push` is no
longer mistaken for invoking it — a false-positive parity fix from the Claude
guard.
Functionally smoke-tested with node against the real plugin (git push denied;
echo/heredoc "git push" allowed; source/interpreter cred reads denied; normal
commands allowed). Live opencode firing stays flagged in the file header.
* fix(grok): record a usage session for interactive intake/secretary (M1+M7)
_spawn_intake_container / _spawn_secretary_container built the AgentInstance by
hand and never recorded an agent_spawn_sessions row, so the reap finalizer had
no usage_session_id to look up — every interactive session (Claude or Grok)
finalized at 0 tokens / $0 in the rollups. Record the session (task_id=None) and
pin its id on the instance, mirroring _launch_spawn; the GROK path reads
opencode.db by this id, the Claude path reads the transcript.
Also correct the grok_intake_main docstring (M7): it claimed the serve process
was "gateway-wired" with an "MCP gateway", but interactive intake mounts no
gateway — its only tool is propose_draft, registered by the intake-tools.js
plugin.
* fix(grok): surface a dead opencode-serve clearly instead of a zombie chat (M2)
If `opencode serve` died after the session opened, every subsequent turn failed
with an opaque httpx connection error while the container lingered. send() now
detects the exited subprocess (returncode set) and yields a clear error chunk +
turn_end so the panel shows a real "session ended — start a new chat" message;
the idle watchdog / a human reap then tears the container down.
* fix(grok): close the panel relay when the cost-cap kills an interactive chat (M4)
_enforce_grok_cost_budget killed + evicted a container directly. For the
interactive roles (intake/secretary) that left the panel SSE relay open with no
close sentinel, so the chat froze with no explanation. Add
PrompterLiveRegistry.close_by_agent (push a final error event, then close every
session bound to that agent) and call it from the cost-cap watchdog when the
killed agent is the intake or secretary, so the panel reports the chat ended on
the cost cap instead of hanging.
* fix(grok): make the opencode runtime actually load — proven live on grok-build-0.1
Live verification (opencode 1.17.8 + grok-build-0.1, funded key) showed the Grok
runtime was loading INERT, three ways:
1. The provider override `provider.xai.npm=@ai-sdk/openai` failed model
resolution (ProviderModelNotFoundError) — opencode can't resolve that package
from its module path. Worse, ANY custom `provider.xai` block (even just
options) breaks plugin-tool registration. opencode's BUILT-IN xai provider
drives grok-build-0.1 with working tool-calls, so emit NO provider block; the
key + base reach it via XAI_API_KEY / XAI_BASE_URL env (provider.options.apiKey
alone does NOT authenticate).
2. Plugins referenced by absolute path in the config `plugin:` array never
registered their hooks/tools. opencode 1.17.8 only registers from the plugin
AUTO-DISCOVERY dir (~/.config/opencode/plugin/). Bake all plugins there.
3. Plugins must use a NAMED export, not `export default`.
Changes:
- opencode_config: no `provider` block, no `plugin` array; drop the dead
XaiTarget + timeout machinery; build_opencode_config now takes a model string.
- GrokProvider / orchestrator interactive env: inject XAI_API_KEY + XAI_BASE_URL
(drop the now-unused OPENAI_*).
- secret-scrub / budget-feed / secretary-tools / intake-tools: named exports;
baked into /home/agent/.config/opencode/plugin/ (drop the EXTRA_PLUGINS env).
- agent-grok* Dockerfiles: plugin dir + agent ownership; drop the unneeded
@ai-sdk/openai global install.
Verified live end-to-end: grok-build-0.1 calls read_company_state AND
submit_directive through secretary-tools.js and the backend receives both with
the agent token; a tool.execute.before guard fires; built-in tool-calls work.
Targeted gate green (ruff/mypy/xenon + opencode_config/providers/interactive
tests; node --check the plugins).
* fix(grok): deliver intake draft via the relay + correct opencode-mechanism docs
Live end-to-end verification (opencode 1.17.8 + grok-build-0.1) of the WHOLE
integration, then fixes for what it surfaced:
1) Intake draft card (FUNCTIONAL): opencode's synchronous serve reply
(POST /session/:id/message) returns only [step-start, text, step-finish] — it
does NOT include tool-call parts, so the driver could never extract the
propose_draft draft. intake-tools.js now POSTs the draft straight to the
prompter-live relay (/api/prompter/live/{session}/events, the same endpoint
the driver's relay sink uses), so the panel renders the card regardless.
Verified live: grok calls propose_draft -> the relay receives the draft.
2) Correct misattributed opencode "bugs" (DOCS): earlier comments asserted as
general opencode behavior that a provider.xai block / npm override / config
plugin:-array "break" registration. Re-testing showed those were artifacts of
a PROJECT-level .opencode/opencode.json; from the GLOBAL config (which
opencode_config writes) the built-in provider, model resolution, the plugin
array AND the auto-discovery dir all work, and MCP gateway verbs register
(delivery agents verified). Reframed the comments as design choices (built-in
provider + XAI_API_KEY env + plugins baked in the auto-discovery dir with
named exports) and dropped the false claims.
3) Reasoning --variant: passing it does not error, but whether opencode applies a
named reasoning variant to grok-build-0.1 (no provider-defined variants) is
UNVERIFIED — comment softened from a "~54% cut" claim to best-effort,
measure-on-NAS.
Verified live this session: one-shot delivery (model + MCP verbs + plugins +
hooks), secretary tools (read_company_state + submit_directive -> backend with
token), intake draft (relay), grok built-in-provider tool-calling. Remaining
NAS-only: full container assembly (SDK :9000 startup, entrypoint hooks, 429
parking) + the --variant cost measurement. Gate green (ruff/mypy + 51 tests;
node --check the plugins).
* feat(grok): reap abandoned interactive chats (M3)
An interactive intake/secretary chat the human abandoned (closed the tab without
confirming or stopping) leaked its container until the orchestrator restarted —
the wedged-grok reaper is task-driven and these run task_id=None, and an SSE
disconnect intentionally does NOT reap (so a page reload can reconnect).
Reap by IDLE TIME, not connection state: PrompterLiveRegistry tracks
last_activity (bumped on every push/deliver = a turn), and the 60s sweeper
retires sessions idle past ROBOCO_INTERACTIVE_IDLE_REAP_SECONDS (default 1800;
0 disables) via reap_intake_session / reap_secretary_session. An active or
page-reloaded chat that keeps exchanging turns stays fresh and is never reaped;
board-review-parked sessions (task_id set) are exempt. Provider-agnostic — fixes
the leak for both Claude and Grok interactive.
Tests: idle-only reap (active/parked/closed excluded), activity bump keeps a
session alive, threshold 0 disables. Gate green (ruff/mypy/xenon + prompter_live).
* fix(panel): resolve agent names from the live roster so they never drift
A review task assigned to the pr-reviewer rendered as a truncated raw
UUID instead of its name. Root cause: the panel resolved assignees from a
hardcoded static roster in agent-utils.ts that had drifted — it never
gained the board-adjacent agents added backend-side (intake-1,
secretary-1, pr-reviewer-1). Their UUIDs hit no map entry, so
getAgentDisplayName fell through to the unknown-UUID branch and returned
agentId.slice(0, 8). Every assignee surface (task table, task detail,
subtasks, journals, communications, commit cards) shares that resolver, so
all of them showed the fragment.
Make the live /api/agents roster the source of truth instead of a static
duplicate that silently rots:
- agent-utils: add a runtime registry (registerAgentRoster) keyed by both
UUID and slug; resolveToSlug / getAgentDisplayName / isKnownAgent consult
it first. The static maps remain only as an offline / first-paint
fallback (now complete with the three agents).
- api/agents: surface the backend UUID on AgentDefinition (getAll/getOne
previously dropped it), so the registry can key by UUID.
- use-agents: add useAgentRosterSync (registers the live roster) and derive
useAgents from live definitions, falling back to the static roster.
- providers: mount the sync once inside QueryClientProvider.
Now any agent the backend knows about resolves, including ones added after
this change — the panel can no longer drift out of sync.
Tests: agent-utils unit tests cover the three agents end-to-end, a
live-roster-only agent (drift-proofing), live-overrides-static, and a
regression guard for the existing roster.
* fix(pr-review): post a COMMENT review when GitHub forbids self-review
A pr-reviewer review of an org-authored PR never reached GitHub. The agent
side ran correctly (claim → read-only diff → review → post_pr_review →
completed + CEO notify), but the GitHub publish 422'd with "Can not request
changes on your own pull request": the PR was authored by the same account
that owns the project PAT. post_pr_review posts best-effort after the DB
transition, so the failure was logged and swallowed — the task completed and
the CEO was notified "reviewed" while the PR showed no review.
GitHub forbids APPROVE / REQUEST_CHANGES on your own PR but DOES allow a
plain COMMENT review. The org's internal PRs (and any PR the PAT owner
opened) hit this. Retry once as a COMMENT review on the self-review 422 so
the review actually lands; the verdict is already stated in the body. The
external/fork-PR path (different author) is unchanged — REQUEST_CHANGES
succeeds there and the fallback never fires.
Tests: self-review 422 downgrades to COMMENT and returns the COMMENT result;
a failing COMMENT retry still surfaces GitError with no infinite loop; the
existing non-self 422 still raises.
* fix(grok): harden cost-guard, pin runtime, refresh stale plugin comments
Address review findings on the Grok provider work:
- budget-feed plugin failed open unconditionally, so a one-shot task agent
whose in-container SDK budget server went unreachable would run with the
cost cap unenforced. The entrypoint now exports ROBOCO_BUDGET_ENFORCE=1
(one-shot agents always start that server) and the plugin's pre-exec gate
fails CLOSED when the flag is set and the budget endpoint is unreachable,
halting an uncapped burn. Interactive serve agents (intake/secretary) set
no flag and keep failing open (they run no budget server by design).
- Pin opencode-ai to the live-verified 1.17.8 (was an unpinned global npm
install). Untrusted model output runs under it; bump the pin deliberately.
- Document the ROBOCO_GROK_* operator vars in .env.example (image, the three
opencode permissions, reasoning effort, idle-kill, cost ceiling).
- Refresh stale plugin comments: the MCP tool-name shape and the secretary
tool-registration path are confirmed live, and secret-scrub's load route is
the auto-discovery dir (not a config plugin: array). Keep the honest
not-yet-exercised caveat on secret-scrub's deny path and the reasoning
variant — those remain genuinely unverified.
* fix(grok): unbreak workspace-cwd agents, free trapped agents, stop self-PR review
Three bugs surfaced by the first live Grok lifecycle run:
- Dev/QA/doc agents crash-looped at startup with ModuleNotFoundError on
roboco.llm.providers. The entrypoint ran the opencode-config render from the
agent's workspace-clone cwd, whose own roboco/ dir shadows /app on the
sys.path front; a branch without the grok code lacks the providers package.
Render from /app so the installed package always resolves (the render has no
cwd dependency — writes global, reads ROBOCO_MCP_CONFIG).
- A budget/loop halt blocked EVERY tool, including i_am_idle, unclaim, and
i_am_blocked, so a halted agent could neither continue nor stop and flailed —
one billed model turn per blocked retry. The before-gate now always lets the
release verbs through so a halted agent can exit cleanly.
- The inbound reviewer ingested the org's OWN PRs (authored by the repo-owner
account), which can't take a REQUEST_CHANGES review (GitHub 422) and get
re-reviewed every poll. The normalizer flags author_is_owner and ingestion
skips them — the reviewer reviews only PRs the org did not author.
External/contributor PRs are unaffected.
Tests: owner-authored PR flagged + skipped; normalize shape covers the new
field. Gate green on the changed modules (ruff/mypy/xenon + 48 tests).
* feat(grok-cli): render config.toml + map per-role grok CLI flags
First piece of the Grok CLI provider that replaces the opencode runtime: a
pure, unit-tested module the agent entrypoint runs to translate the mounted
mcp-config.json into ~/.grok/config.toml ([mcp_servers]) and compute the
per-role 'grok -p' flags — subagent/shell/edit tool removal, raw-git-mutation
and rm-rf denies, reasoning effort — mirroring ClaudeCodeProvider's per-role
permissions with native grok flags instead of an opencode permission block +
JS guard plugins. Uses tomli_w. The rendered config + env injection are
validated live against grok-build (the model called the server through it).
* feat(grok-cli): grok CLI agent image + headless entrypoint
The roboco-agent-grok image now installs xAI's official grok CLI (Grok Build,
pinned 0.2.56) instead of opencode, authenticated by the SuperGrok subscription
via a mounted ~/.grok/auth.json (parity with the Claude ~/.claude mount, no
metered API key). The entrypoint renders ~/.grok/config.toml + per-role flags
from /app (the ModuleNotFound-shadowing lesson), runs grok -p headless with
--output-format json, keeps the prompt-injection guard, and exits 75 on a
rate-limit so the orchestrator parks the provider. No in-container SDK server or
budget-feed — native --max-turns + server-side terminal-substitute replace them.
* feat(grok-cli): GrokCliProvider — subscription auth mount, mirrors ClaudeCodeProvider
Replace the opencode GrokProvider with GrokCliProvider: reuses the orchestrator's
shared mount/auth/git assembly (gateway + identity) exactly like the Claude path,
mounts the host ~/.grok/auth.json read-only (SuperGrok subscription) instead of
injecting an xAI key, and sets the slim env the grok-cli entrypoint + renderer
read (ROBOCO_AGENT_ID for per-role flags, model, mcp-config, prompt). Provider
routing fields are blanked before the shared step so the grok endpoint is never
mislabelled ANTHROPIC_*. Per-role permission logic now lives in grok_cli_config,
so the provider is slim. Registry/orchestrator/exports updated; provider tests
rewritten for the CLI behavior (no XAI key, auth mount present/absent).
* feat(grok-cli): capture per-session token usage + notional cost
Grok runs on the SuperGrok subscription, but — exactly like Claude on Max — we
still record per-agent tokens and a notional cost for the dashboard. The grok
CLI writes a cumulative totalTokens per turn into
~/.grok/sessions/<cwd>/<session-id>/updates.jsonl (the grok analogue of the
Claude transcript / old opencode.db); the max is the session total. This reader
locates that file (url-encoded cwd), extracts the total, and prices it at the
output rate (no input/output split from the CLI; conservative + matches the
reasoning-at-output convention). Validated against a real grok-build session
(18253 tokens -> $0.0365). Entrypoint + finalize wiring follows.
* feat(grok-cli): wire usage capture into the run (session id + post-run extract)
The provider pins a fixed session id (ROBOCO_AGENT_SESSION_ID, reused from the
agent session id as on the Claude path); the entrypoint passes it to
'grok -p -s <id>' so the run's session store is locatable, then runs the usage
reader post-run (best-effort) to write the captured tokens + cost. The
orchestrator-side finalize that reads that file follows.
* feat(grok-cli): read captured usage at finalize; keep interactive serve working
The provider mounts the per-agent data dir and points the entrypoint's usage
file at it; the orchestrator's grok finalize reads that usage.json first (the
grok-CLI total, priced at the output rate) and falls back to opencode.db for the
still-opencode interactive intake/secretary path. Re-add _reasoning_effort_for to
grok.py as a clearly-temporary shim for that interactive path (it needs opencode's
"minimal" variant, distinct from the CLI's --effort) until it is converted too.
* feat(grok): convert interactive intake/secretary to the grok CLI; delete opencode
Move the last Grok runtime off opencode onto xAI's official `grok` CLI, for full
parity with the Claude path. The intake/secretary chat now runs per-turn headless
`grok -p` invocations that resume one session id (proven live: context carries
across runs), with streaming-json deltas mapped to the existing panel StreamChunk
kinds — the IntakeDriver loop, message source, relay, and idle reaper are reused
unchanged; only the SessionFactory differs (GrokCliSession replaces the
opencode-serve session).
- GrokCliSession + a pure, unit-tested streaming-json -> StreamChunk assembler
(thought coalesced to one block, text streamed live, end captures the session
id for -r, fenced-draft fallback, clear errors incl. rate-limit).
- intake propose_draft and secretary read_company_state/read_task/submit_directive
are now FastMCP servers (roboco-intake / roboco-secretary) wired into
~/.grok/config.toml, launched via `uv run --directory /app` to resolve the
installed package. The secretary tools reuse the shared backend helpers.
- Orchestrator: interactive spawn mounts the subscription auth + per-agent usage
dir (no metered xAI key, no permission env — grok flags carry per-role perms);
usage/cost now read a captured usage.json (drop the opencode.db reader, the
_opencode_db_path/_grok_usage_from_opencode methods, and the cost-cap's
opencode read). hosts["opencode"] -> hosts["grok_usage"]; OPENCODE_DATA_DIR ->
GROK_USAGE_DATA_DIR.
- Fix one-shot usage capture: `-s` does not pin the session id (grok generates
its own), so the entrypoint now reads the real id back from the JSON run log
and the reader uses it; usage is captured per-turn on the interactive path.
- Delete the opencode layer: opencode_config/opencode_usage/opencode_session, the
docker/grok/*.js plugins, the old one-shot entrypoint, and their tests.
- Compose (all three files), .env.example, and stale comments updated to the
grok-CLI runtime; add the SuperGrok auth mount + grok-usage dir.
Gate green: ruff, mypy (296 files), xenon, tests. NAS build/verify pending.
* fix(grok): deliver the role blueprint as grok's system prompt via ~/.grok/AGENTS.md
The blueprint was mounted at /app/system-prompt.md but never reached grok — a real
parity gap vs the Claude path (which passes --system-prompt-file). grok agents ran
only on the per-task prompt, missing their RoboCo role/org context.
Verified live on grok 0.2.56 that the obvious flags do NOT work headless:
`--system-prompt-override` and `--rules` are silently ignored under `grok -p`
(identical output with and without). What IS honoured is grok's instruction-file
discovery — and `$HOME/.grok/AGENTS.md` is loaded GLOBALLY regardless of --cwd
(a project AGENTS.md only loads from the cwd/project root, which would pollute the
agent's git workspace). Proven end to end: a blueprint written there makes grok
adopt the role ("I am the RoboCo intake interviewer ... -- intake-1").
write_agents_md() copies /app/system-prompt.md -> ~/.grok/AGENTS.md; the one-shot
render (grok_cli_config.main) and both interactive mains call it. No git pollution
(it lives in ~/.grok, not the workspace), and it covers repo-cwd and /app-cwd
roles alike. Reverted the non-working --system-prompt-override wiring.
* feat(grok): close the Claude-parity divergences (reasoning, subagents, web, bash-guard)
Bring the grok CLI to parity with the Claude path on the four deliberate
differences:
- Reasoning: drop the per-role `--effort low` default — Claude sets no per-role
thinking budget, so grok now uses the model default for every role. The
fleet-wide ROBOCO_GROK_REASONING_EFFORT override stays as a cost lever. (This
also un-caps intake-draft quality, the one that actually mattered.)
- Subagents: the intake interviewer may now fan out to subagents (parity with the
Claude intake's `Task` allowance); every other role still has `Agent` removed.
- Web: `--disable-web-search` for every role — no agent gets direct web (Claude's
tool set has none either); the roles that get web reach it through the gated
roboco-search MCP, unaffected.
- Bash command filtering: full parity, split by deny semantics. Verified live that
a grok PreToolUse hook deny CANCELS the run, while native `--deny` denies
GRACEFULLY (the agent gets a permission error and recovers). So:
* git network/branch/history ops -> native `--deny` (operational reflex; the
agent must recover, not drop the task). Expanded to the full bash-guard set.
* credential-exfil / identity-forgery / internal-API / env-dump patterns ->
the SAME bash-guard the Claude path runs, wired as a grok PreToolUse hook
(ROBOCO_GUARD_SKIP_GIT=1 so it leaves git to `--deny`). A hard cancel is the
right response there — no legitimate agent reads ~/.netrc or forges an
X-Agent-ID. One tolerance line (accept grok's camelCase `toolInput`) makes
the one tested script guard both runtimes; +5 grok cases (50/50 green).
Also cleaned stale internal task-number / smoke labels out of bash-guard-hook.sh.
* fix(grok): install grok CLI to ~/.grok/bin (its real default), not ~/.local/bin
The image build failed at `chown ... /home/agent/.local: No such file or
directory`. The grok installer's default is $HOME/.grok/bin — the binary lands at
~/.grok/bin/grok; ~/.local/bin/grok is only a convenience SYMLINK the installer
creates on macOS but not in the Linux container. So the Dockerfile referenced a
directory that never existed:
- PATH pointed at ~/.local/bin -> `grok` would not be found at runtime even if
the build had passed;
- chown targeted ~/.local -> the build aborted.
Point PATH + chown at ~/.grok/bin / ~/.grok. Also harden the install: download the
script to a file (a `curl | bash` pipe swallows a curl failure as a silent no-op)
and verify the binary installed and runs (`test -x` + `grok --version`), so a
broken install fails the build loudly instead of producing a grok-less image.
* fix(grok): address adversarial-review findings across the grok-CLI conversion
A 7-dimension adversarial review (find -> independently refute) surfaced 14 real
issues; fixed each:
Runtime bugs
- GrokCliSession.send drained stdout fully BEFORE stderr — a >64KB stderr burst
would deadlock the turn forever (spinner never clears). Drain stderr
concurrently, and add a per-turn watchdog (ROBOCO_GROK_TURN_TIMEOUT_SECONDS,
default 600s) that kills a wedged process and emits error+turn_end.
- Crash-restarted grok agents launched `grok -p ""` (empty prompt) — Claude gets
a scan-for-work fallback. Default the prompt in _spawn_container so every
dedicated provider gets it too.
- _grok_usage_json read /data/grok-usage unconditionally while its writers branch
compose-vs-local, so a local-mode agent finalized at $0 and the cost-cap was
inert. Single-source the path in a new _grok_usage_dir helper (read == write).
- GrokCliSession secretary role fell through to "unknown" (get_agent_role returns
a truthy sentinel, never None), defeating the ROBOCO_AGENT_ROLE fallback.
Parity / hardening
- --deny set was missing `git tag -d` / `git reflog delete` that the Claude
bash-guard blocks — added them (the "same set" claim is now true).
- Interactive mains now install the bash-guard hook too (defense-in-depth).
- Compose: collapse the GROK_AUTH_DIR / ROBOCO_HOST_GROK_DIR auth-mount pair into
one canonical var so a partial override can't silently break agent auth.
Docs / comments
- Panel routing card + architecture security doc no longer say Grok runs on the
deleted opencode runtime; orchestrator comments point at the renamed entrypoint.
Tests
- Cover the interactive _render_grok_config MCP wiring (ModuleNotFound guard +
secretary HMAC env), the cost-cap kill-failure + interactive relay-close paths,
the local-mode usage read, the role fallback, the turn timeout, and the new
git denies. (#13 — a separate grok "Write" tool — investigated: grok's only
built-in file-mutation tool is search_replace, already removed; no gap.)
Gate green: ruff, mypy, xenon, tests.
* fix(grok): declare tomli-w as a runtime dependency (agent image needs it)
The grok agent image failed at spawn with `ModuleNotFoundError: No module named
'tomli_w'` when rendering ~/.grok/config.toml. tomli_w was only a transitive dep
of a dev-extra package, so it was present in dev/orchestrator envs but excluded
from the agent image, which builds its venv with `uv sync --frozen --no-dev`.
grok_cli_config imports it at module load to serialize the MCP gateway config, so
without it a Grok agent gets no gateway verbs.
Promote tomli-w to a direct [project.dependencies] entry. Locked with
`--upgrade-package tomli-w` so only tomli-w is added — no incidental churn of the
8 unrelated packages a full re-resolve would have bumped.
* Updated uv.lock
* fix(grok): auto-approve tool execution (--always-approve) so headless agents can call tools
Live smoke caught every grok agent (Main PM, pr-reviewer, dev, …) ending its run
with stopReason=Cancelled and empty output the instant it reached for a tool. Root
cause: headless `grok -p` cannot approve a tool call without `--always-approve`
(grok's docs: required for unattended automation), and the per-role args didn't
pass it — so no agent could call a gateway verb, an edit, or an MCP tool, and the
run was cancelled.
Add `--always-approve` to grok_cli_args_for_role (one place → every role, one-shot
and interactive). Safety is unaffected: `--disallowed-tools` still removes tools
and `--deny` still hard-blocks command patterns regardless of approval (a denied
command returns a permission error and the agent recovers — verified live).
Proven in the rebuilt image side-by-side: without the flag a tool call yields
Cancelled/not-called; with the real rendered args it returns EndTurn and the MCP
tool actually runs. (My earlier in-image tool-calling check passed `--always-approve`
manually, which masked that the production args omitted it — fixed.)
* fix(pr-review): seed claim heartbeat so the grok reviewer isn't wedge-killed
pr_review_claim transitioned a review task pending -> in_progress but never
seeded last_heartbeat_at, unlike every sibling claim path (_finalize_claim,
qa_claim, doc claim). The reaper treats a NULL heartbeat as a stale claim, and
the GROK idle-kill watchdog bypasses the live-container skip on a NULL
heartbeat -- so the reviewer container was killed (Cancelled) before it could
post_pr_review, churning the task back to pending on a respawn loop. A Claude
reviewer was shielded by the live-instance skip; only GROK manifested it.
Seed the heartbeat at claim time, matching the established invariant. Verified
against a real Postgres (10/10 test_pr_review_db tests, incl. the new
last_heartbeat_at assertion).
* fix(grok): stream one-shot output live + capture real token usage
Two gaps the buffered run hid, both verified in the real image with mounted
SuperGrok auth:
- Observability: the entrypoint buffered grok's output to a temp file and only
cat it after the run, so `docker logs` was blank while the agent worked.
Switch the one-shot to --output-format streaming-json piped through tee:
grok flushes each thought/text event incrementally (confirmed token-by-token
live in-container), so the agent's reasoning shows in docker logs in real
time, parity with the Claude stream-json path. Read the session id back from
the NDJSON run log (the terminal `end` event) since -s does not pin it.
- Usage: total_tokens read 0 for every grok run. grok nests the cumulative
totalTokens on params.update._meta, but the reader looked at params._meta
(which only holds event ids); the unit fixture had the same wrong shape, so
the tests masked it. Read the real path (with params._meta / top-level
fallbacks) and fix the fixture to the real grok shape. Verified live:
usage.json now reports total_tokens=3262, cost_usd=0.006524 (was 0).
* fix(grok): validate agent_id before using it as a usage-dir path segment
CodeQL flagged a high-severity py/path-injection: agent_id flowed from
request-facing call sites into _grok_usage_dir() and on to read_text(), so a
value containing '..' or a separator could traverse the filesystem. Validate
agent_id against the slug/uuid allowlist ([A-Za-z0-9_-]+) at the single
chokepoint (_grok_usage_dir feeds both the mount and the finalize read);
anything else raises. Rejects traversal; accepts every real agent slug.
* fix(grok): use explicit-guard path sanitizer CodeQL recognizes as a barrier
The re.fullmatch allowlist from a35b640d was secure but CodeQL's py/path-injection
dataflow did not model the regex call as a barrier, so the high-severity alert
persisted on the analyzed merge. Switch _safe_agent_path_segment to explicit
guards (empty / '.' / '..' / '/' / '\\' / NUL) -- the barrier form the query
recognizes -- which still rejects every traversal vector. Drop the now-unused
re import.
* fix(api): validate agent_id at the orchestrator route boundary (path-injection)
CodeQL traces the py/path-injection from the request agent_id path param on the
orchestrator routes (stop/spawn/resolve-wait/mark-waiting/status) down to the
grok usage-dir read. Validate agent_id at the HTTP boundary with explicit
traversal guards (empty / '.' / '..' / '/' / '\\' / NUL) returning 422, so the
sanitized value is what flows downstream and the query sees a barrier at the
source. Runtime _grok_usage_dir keeps its guard as defense in depth for
non-HTTP callers.
* fix(grok): sanitize usage-dir agent_id with Path(...).name (CodeQL barrier)
Proven against the analyzed merge: CodeQL does not propagate a control-flow guard
through a helper's return value, so neither the route validator nor the
_safe_agent_path_segment guard cleared the py/path-injection alert. Reduce the
validated id to its final path component with Path(...).name -- a data-flow
sanitizer CodeQL models and propagates through the return -- at the single
source of truth (_grok_usage_dir), covering both the finalize read and the
mount/mkdir. The guard stays for fail-loud reject semantics; .name is the
recognized barrier (identity for a valid slug).
* fix(grok): containment-check the usage read against a fixed root (path-injection)
Three sanitizers failed to clear the CodeQL alert because the query does not
model them here: a regex guard, an explicit guard, and Path(...).name (verified
each against the analyzed merge). Replace with the barrier CodeQL does
recognize -- and that is also a genuine control -- at the read sink: resolve the
usage.json path and refuse it unless it is_relative_to the resolved usage root
(a fixed, untainted base from config, extracted as _grok_usage_root).
Note the github-advanced-security autofix proposed 'if usage_json.parent !=
usage_dir: return None', which is a no-op -- appending the constant 'usage.json'
never changes the parent, and it compares against the tainted dir, not a safe
base. This compares against the fixed root instead. The _safe_agent_path_segment
guard stays (fail-loud reject upstream, covers the mount/write side).
* fix(grok): sanitize the usage read with os.path.basename (CodeQL-modeled barrier)
Four prior barriers did not clear the py/path-injection alert (verified each
against the analyzed merge): a regex guard, an explicit guard, Path(...).name,
and an is_relative_to containment check. The one sanitizer CodeQL's query
documents -- os.path.basename -- was never actually tried: it was swapped for
Path(...).name to dodge ruff PTH119, and that pathlib form is not modeled.
Apply os.path.basename to the agent id in _grok_usage_json's own scope (the read
sink), so there is no recognition or interprocedural-propagation ambiguity, and
allow PTH119 for this file with a documented reason. The _safe_agent_path_segment
guard and the route 422 stay as the actual reject controls.
---------
Co-authored-by: Renn F <rennf93@users.noreply.github.com>
The internal-API rule only fired when the FIRST shell token was an HTTP
CLI (curl/wget/http/https/httpie). smoke-17 showed an agent reach the
orchestrator with hand-forged X-Agent-ID/X-Agent-Role headers via:
python3 << 'EOF'
import httpx
httpx.post("http://roboco-orchestrator:8000/api/v2/flow/developer/i_will_work_on",
headers={"X-Agent-ID": "<self>", "X-Agent-Role": "developer"})
EOF
The binary is python3 (slips the CLI check) and it imports httpx, not
roboco.* (slips the #164 import check). Only minimax's wrong endpoint
path prevented a real gateway bypass under a forged identity.
Add a language-agnostic rule: deny when the command pairs an HTTP-client
token (httpx/requests/urllib/aiohttp/http.client/net::http/fetch(/
node-fetch/axios/...) with a forbidden internal host, consistent with
the curl/wget sibling (inspects full $low incl. heredoc body). External
HTTP (pypi/docs/github) has no internal host so it still passes. The
stale "interpreter one-liners — out of scope" KNOWN GAP comment is
corrected; the variable-expansion gap remains documented.
11 new tests incl. the exact smoke-17 heredoc, requests/urllib/aiohttp/
node-fetch/Net::HTTP variants, and allow-cases (external host, client
import w/o host, pytest runner). make quality green.
The git network/auth deny rule matched its regex against the whole
command string, so heredoc bodies and echo/printf arguments that merely
documented git verbs (a README, a notes file) were treated as git
invocations and denied. This wedged smoke-13's dev: after wiping the
README via an Edit/Write fallback it could not restore it because every
`cat > README.md << EOF ... git commit ... EOF` was blocked.
The git-ops check now runs against a skeleton of the command with
heredoc bodies and echo/printf literal args stripped (both are data the
shell writes, never executed). Quoted args to a shell interpreter
(`bash -c "... && git fetch"`) ARE executed, are not echo/printf/heredoc
bodies, and so survive untouched — the hook's core purpose is preserved.
A sentinel prefix distinguishes a legitimately-empty skeleton from a
python failure (fail closed on failure). All other rules, including the
#164 import-bypass rule, still inspect the full command.
Smoke-12: be-dev-1 (minimax-m2.7) bypassed the entire MCP boundary by
running `uv run python3 -c "import os;
os.environ['ROBOCO_AGENT_ID']='...'; from roboco.mcp.flow_server
import open_pr; open_pr(...)"` from the Bash tool. This voided the
per-role tool manifest (role-scoping is meaningless if the agent can
import any server module in-process), forged agent identity via an
env-var rewrite, and ran choreographer code outside the gateway's
tracing + auth.
bash-guard-hook.sh now adds two deny rules:
1. Any python/uv/poetry/pipenv/pdm/hatch invocation that imports or
`-m`-runs roboco.* internals (mcp/services/runtime/foundation/
api/enforcement). The whole command string — heredoc body
included — is matched, so quoting/heredoc forms are covered.
2. Any assignment or export of ROBOCO_AGENT_ID (identity forgery).
Reading roboco source for context (cat/grep) is still allowed — the
block is on *executing* internals, not viewing them. Normal python
one-liners without roboco imports still pass.
19 bash-guard tests pass (10 prior + 9 new). Note: takes effect on
agent-image rebuild (hook ships in the agent container).
Smoke run 3 showed the bash-guard hook emitting 8+ lines on every
blocked shell-git op — enumerating every alternative MCP verb across
roboco-flow / roboco-do / roboco-git-readonly. That's repeated token
spend on every refused retry; the LLM doesn't need the full alt-list
inline, it has the role prompt + the MCP tool schema for that.
Trimmed to 2 lines: denial reason + a one-line pointer to the role's
State→Verb table. Test asserts <= 3 echo lines in any denial block.
Spec ref: docs/superpowers/specs/2026-05-12-post-smoke-3-fixes-design.md
section B5.
Previous regex used a greedy [^|]* and required at least one / before
the host, so 'curl roboco-orchestrator:8000/api' (no scheme, no slash)
slipped through. Split into two simpler checks: (a) line starts with
curl/wget/http/https/httpie, AND (b) line contains a forbidden host.
Probe-verified: scheme-ful, scheme-less, and protocol-relative forms
all denied; external URLs still allowed; GitHub-specific deny still
fires first.
Prompts told agents internal API calls were denied; the guard only
denied GitHub. Combined with task 4 (X-Agent-Role enforcement) this
closes the manifest-bypass loophole.
