2 Commits

Include renames

Author	SHA1	Message	Date
3 people	3849c1737e	feat(video): switch 0.19.0 renderer Remotion → HyperFrames (HTML-native, Apache-2.0) (#314) ... * feat(video): rewrite sidecar render core to HyperFrames (in place) * feat(video): convert motion compositions from Remotion TSX to HyperFrames HTML * refactor(video): rename render client to video_renderer_client (renderer-agnostic) * chore(video): rename remotion-renderer prose in test_video_pipeline docstrings * chore(video): rename sidecar to video-renderer + add system ffmpeg for HyperFrames * chore(video): rename stray remotion-renderer refs in sidecar + py docstrings (controller cleanup) * chore(video): fix stale Remotion API names in Dockerfile comment (controller cleanup) * docs(video): rewrite video-engine prose for HyperFrames + add map entry + folded prose fixes * docs(video): add trailing newline to docs/map/video-engine.md (controller cleanup) * chore(video): drop internal spec refs + minio/test suppressions (folded hygiene) * fix(video): reclaim outDir on createRenderJob throw + hide empty 4th highlight Final whole-branch review (Opus) triaged two FIX items from the SDD nits ledger; the rest ship as-is. - render.js: a synchronous throw from createRenderJob (post-mkdtemp, not awaited) left an empty outDir on disk — the outer catch only reclaimed extractDir. Reclaim outDir too when it exists, and correct the stale comment that claimed the out dir was never created. - {vertical,square}.html: the 4th highlights <li> lived in the DOM hidden only by JS, so a no-JS / failed-script render would show an empty bullet. Start it style="display:none" and reveal on populate, so an unscripted render shows nothing instead. Vitest smoke (release-announcement.test.js) 4/4 green; render.js syntax checked. Python suite untouched by this fix (JS/HTML only). * fix(video): type _override_db yield as AsyncSession | None T7 widened _build_app's db_session param to AsyncSession | None (to drop the 4x # type: ignore[arg-type] on the DB-independent _build_app(None, ...) calls) but left the inner _override_db fixture typed AsyncIterator[AsyncSession] — so 'yield db_session' yielded AsyncSession | None into a declared AsyncSession, and mypy failed at test_video_routes.py:177 ('Incompatible types in yield'). The DB-independent media tests pass db_session=None deliberately: their route uses a monkeypatched task service and never awaits the session, so yielding None is safe at runtime. Type the override's yield as AsyncSession | None to match — no cast, no # type: ignore, no assert, runtime behavior unchanged. The 3 media tests (3 passed) and the 19 db-gated tests (skipped locally) hold. * chore(gate): skip .superpowers scratch in markdown prose gate reflow_md.py walks the filesystem via rglob('*.md') and skips tooling dirs (.venv, .mypy_cache, .pytest_cache, ...) but not .superpowers/ — the superpowers SDD workflow's scratch dir (briefs, reports, progress ledger, all gitignored). A dev running SDD locally would hit a false markdown-prose gate failure on those transient files. Add .superpowers to SKIP_DIRS, consistent with the existing tooling-scratch exclusions. * fix(video): validate composition_id to close path traversal (CodeQL) compositionId flowed unvalidated from the POST body into path.join under extractDir/motion/compositions/, so a '../..'-style value could escape the composition dir (CodeQL: Uncontrolled data used in path expression). Validate at the trust boundary in server.js (/^[A-Za-z0-9_-]+$/) and add a path.resolve + startsWith containment check in render.js so it stays safe regardless of caller. * fix(mcp): send X-Agent-Token + X-Agent-Team from flow/do servers flow_server._build_headers and do_server._build_headers constructed only X-Agent-ID/Role/Correlation-ID, omitting X-Agent-Token and X-Agent-Team (unlike ApiClient._get_agent_headers used by the other MCP servers). Latent since the gateway refactor — surfaced when ROBOCO_AGENT_AUTH_REQUIRED=true was armed on the NAS, 401-ing every flow/do verb with 'Missing X-Agent-Token header'. Add both headers (mirroring ApiClient) so the HMAC gate passes. Tests assert the headers are now injected. * [video-engine] Per-project video_engine_enabled opt-in toggle Mirrors ci_watch_enabled (migration 048): the global ROBOCO_VIDEO_ENGINE_ENABLED flag arms the subsystem; the new projects.video_engine_enabled column (migration 063) opts a repo into authoring against its motion/ dir. VideoEngine._opted_in_project no-ops open_video_task at the single chokepoint covering all three trigger paths (on-release, on-spotlight, CEO on-demand) until the operator flips it in the panel edit-project dialog. Existing projects stay opted out (server_default=false). * fix(auth): send X-Agent-Token + X-Agent-Team from all agent->API call sites The prior fix (6ed4e139) covered the flow/do MCP servers but missed four other agent->orchestrator call sites that built the header dict by hand and omitted X-Agent-Token and/or X-Agent-Team. With ROBOCO_AGENT_AUTH_REQUIRED armed on the NAS, every one 401s: - agent_sdk/server.py: the session-end post-mortem flush (/api/journals/me/entries), A2A persistence + offline fallback (/api/a2a/*), and the stopped-without-transition auto-substitute (/api/tasks/auto-substitute) — all sent only X-Agent-ID/Role, so each 401'd 'Missing X-Agent-Token'. Add a shared _agent_headers() helper (mirroring flow_server._build_headers) and route all four through it. - agent_sdk/secretary_driver.py: _headers() sent the token but not the team, so the HMAC gate 401'd with signature mismatch (secretary is board-team; token signed with team='board', verified with team=''). Add the team header. - mcp/git_readonly.py: the read-only git MCP sent only X-Agent-ID/Role — no token, no team — so /api/git/* 401'd once auth was armed. Convert the static _HEADERS to a _headers() helper with team + token. - runtime/orchestrator.py: the cell-PM auto-submit self-API call acted as a PM with a hand-built {X-Agent-ID, X-Agent-Role} dict — no token, no team — 401ing under auth-required. Add _agent_api_headers(uuid, role) mirroring _system_api_headers, and use it. Tests: _agent_headers round-trip (token + team, team-omitted when None), _agent_api_headers carries a signed PM token + team. * [auth] Omit UNSIGNED self-call token in dev mode + video-engine test mypy fix _agent_api_headers sent the UNSIGNED sentinel when ROBOCO_AGENT_AUTH_SECRET was unset, but the dev-mode middleware rejects a presented-but-unverifiable token with 401 signature mismatch (while accepting a missing one). The cell-PM auto-submit self-call 401'd in every dev run, regressing test_auto_submit_cuts_the_pm_turn. Attach the token only when a secret is set. Also fix the FromClause.update mypy error in the per-project video-engine opt-out test (ORM row load + flush). * [auth] Omit UNSIGNED agent token at every agent->API call site The orchestrator injects ROBOCO_AGENT_TOKEN=UNSIGNED when the HMAC secret is unset at spawn. The API middleware rejects a presented-but-unverifiable token with 401 'signature mismatch' even in dev mode (auth not required), so forwarding UNSIGNED turned every flow/do/SDK/secretary/git verb into a 401 — the live pr_reviewer/i_am_idle signature-mismatch loop. Omit the header when the token is the UNSIGNED sentinel at all five agent-side header builders; dev accepts a missing token, prod 401s with 'Missing X-Agent-Token' (the clear respawn-with-secret signal). Add a structlog diagnostic on the middleware reject path so the next mismatch logs the exact (id, role, team, token_unsigned, auth_required) inputs. * [auth] Self-heal stale agent tokens at orchestrator startup A token is signed once at spawn. If ROBOCO_AGENT_AUTH_SECRET drifts afterwards (a .env change, a compose recreate that reloads the orchestrator's env without recreating agent containers, an image redeploy), the surviving agent keeps sending its old token and the middleware 401s every verb with 'signature mismatch'. The container stays alive heartbeating, so the reaper never reclaims it and no fresh agent spawns: the fleet stalls. _heal_stale_agent_tokens runs at startup (before _readopt_running_agents) and kills each running agent container whose baked-in token no longer verifies against the current secret, so normal dispatch re-spawns it with a freshly signed token. Inert when the secret is unset (dev): verify fails for every token without a secret, so the heal would kill the whole fleet without this gate. Best-effort: a probe failure leaves the container alone (the reaper still covers it). * [auth] Sign agent token over the UUID, not the slug (pr_reviewer 401 root cause) The token was signed over the agent slug (_append_agent_auth_env) while the MCP servers send X-Agent-ID as the agent UUID (_generate_mcp_config, since 453a7ae2 — gateway v1 parses X-Agent-ID as Annotated[UUID]). The middleware verified HMAC(uuid:role:team) against a slug-signed token → 'signature mismatch', token_unsigned=false. Latent for 2 months until 6ed4e139/53391f22 made the MCP servers forward the token. The c0328971 startup heal missed it: docker exec printenv reads the container-level ROBOCO_AGENT_ID (the slug), so the heal verified the slug-signed token against the slug → matched → didn't kill the stale container, which kept 401ing (its MCP server sends the UUID). Fix: sign the token over the UUID, set the container ROBOCO_AGENT_ID to the UUID too (so the SDK server — which inherits container env, not the MCP manifest env — sends UUID consistently), and resolve the container-env id to its UUID in _heal_stale_agent_tokens so pre-fix stale containers are evicted on next restart. Regression test: test_heal_kills_slug_env_container_with_slug_signed_token. * [scan] gate A2A/notification/stream agent-id deps under cloud auth (C1) * [scan] omit UNSIGNED agent token from MCP server headers (H1) * [scan] fail loud when cloud auth and nginx CEO-token are both armed (H2) * [scan] cache last-known-good auth-probe result in panel proxy (C2) * [respawn] Tripped breaker self-heals after a cooldown A DB-durable PM-respawn counter (migration 051 / e2f7097a) wedges forever once tripped: the only reset was a task status change, which can't happen while the breaker blocks the spawn. So a deploy that fixes the underlying loop (auth/prompt/schema) couldn't clear the wedge without manual DELETE surgery on respawn_tracker — the 2026-07-06 pr-reviewer-1 loop, where the auth fix cleared the 401 but count=63 survived restart and kept skipping the dispatcher spawn for an external-PR task. Freeze last_check at the trip tick and, after pm_respawn_trip_cooldown_seconds (default 300), let ONE spawn through. A still-wedged task re-trips after the threshold (bounded re-burn ~3 spawns per window); a fixed one advances and the status-change path fully resets. Restore re-stamps last_check to now, so a freshly restored row still trips immediately — durability preserved, which is why the migration-051 persistence tests still pass. * [scan] fix test_deps callsites for cloud-auth-gate signature change (C1 followup) * [scan] per-IP rate limit on /auth/login under cloud auth (L31) * [scan] Phase 1 auth/security fixes under 0.19.0 CHANGELOG * [scan] secretary token signs over real team (board) not empty — fixes /api/secretary/* 401 (L31-class) * [scan] LoginRateLimiter: key off X-Forwarded-For first hop + redis-down fail-open test nginx is the single entry point; request.client.host is the nginx peer IP, collapsing every external client into one limiter bucket (self-DoS amp). Read the downstream client IP from X-Forwarded-For (first hop) / X-Real-IP, falling back to the peer. Adds coverage for the XFF keying, the redis-down fail-open branch, and drops a redundant asyncio marker on a sync-TestClient test. * [scan] nits: describe login_max_attempts + replace cast with assert in get_current_agent_slug login_max_attempts was the only bare cloud-auth field; add a Field description matching the surrounding idiom. Replace cast('str', ctx.slug) with a runtime assert that fails loud if the cloud-auth ctx invariant breaks, and drop the now-unused cast import. * [scan] secretary token: use get_agent_team resolver + complete spawn-shutdown mock team (0dfd45ca followup) * [scan] require agent HMAC token under cloud_auth (close v1 flow/do header-trust) * [scan] _require_ceo accepts CEO session cookie under cloud_auth * [scan] HTTP require_panel_token accepts session cookie under cloud_auth * [scan] gate /api/settings behind panel token * [scan] gate unauthenticated /api read routes (agents/a2a-tasks/kanban/usage/rate-limits) * [scan] hoist deferred test imports to top-level (clear PLC0415) * [scan] Phase 1b e2e smoke + CHANGELOG * [scan] add_dependency rejects self-reference + cycle (M18) * [scan] WorkSessionService.create translates IntegrityError to ConflictError (H10) * [scan] _qa_or_doc_claim locks the task row FOR UPDATE (M19) * [scan] docs_complete + mark_pr_created lock the task row FOR UPDATE (H4) * [scan] gate complete() IN_PROGRESS on leaf/branchless only (H3) * [scan] _unclaim_from_blocked clears stale pre-block snapshot (H5) * [scan] admin_set_status terminal guard + skip revision bump under force (M20) * [scan] cell_pm_complete idempotent pre-check before merge (H7) * [scan] wrap gateway post-runner side effects in try/except (H6) * [scan] pass_qa/fail_qa accept AWAITING_QA only (L29) * [scan] mark_pr_created passes audit_agent_id (L30) * [scan] phase 2 e2e smoke - one scenario per finding * [scan] phase 2 quality gate ruff format + check: green mypy roboco/: green (357 files) pytest unit+integration: 6905 passed, 10 pre-existing DB-contamination failures (pass in isolation) e2e smoke: 11 passed, 4 cross-scenario workspace-contamination failures (all 6 state-machine scenarios pass individually) Quality-gate fixes: - move function-local imports to module top (PLC0415) - fix M19 regression: submit_for_qa clears active_claimant_id so the competing-claimant guard lets the QA claim through - fix H7 regression: _StubGit gains is_pr_merged_for_task - fix M19 unit tests: mock session.execute for the FOR UPDATE lock - e2e H3: notes >= 20 chars; e2e H5: rich i_will_work_on inputs + PM unclaims (block reassigns to PM) * [scan] move active_claimant_id clear into pass_qa/fail_qa + admin_set_status (M19 follow-on) Phase 2 opus whole-branch review found the M19 follow-on clear lived in the gateway wrappers (qa_pass/qa_fail) not the transition methods (pass_qa/fail_qa) themselves. The direct REST routes POST /pass-qa and POST /fail-qa call the transitions directly, bypassing the wrappers and leaving the QA's stale active_claimant_id set in AWAITING_DOCUMENTATION / NEEDS_REVISION — the competing-claimant guard then rejects the next legitimate documenter/QA claim. admin_set_status had the same gap for a non-blocked override into a review/queue state (IN_PROGRESS->AWAITING_QA left the dev's id, blocking qa_claim). Root-cause fix: move the clear INTO pass_qa and fail_qa (mirroring submit_for_qa), add a clear in admin_set_status when new_status in _REVIEW_QUEUE_STATES and from_status != BLOCKED, and drop the now-redundant clears + flushes from the qa_pass/qa_fail wrappers. Every caller is covered; the wrappers keep their actor-mismatch warnings. Covering tests: test_pass_qa_clears_active_claimant_for_doc_claim (asserts a subsequent doc_claim succeeds), test_fail_qa_clears_active_claimant, test_admin_set_status_into_review_queue_clears_active_claimant, test_admin_set_status_non_review_queue_keeps_active_claimant. Updated the two wrapper unit tests that asserted the wrapper clears (now the transition's job). * [C3] unindex_journal_entry + call from delete_entry JournalService.delete_entry deleted the DB row but never de-indexed the RAG chunks, so deleted/private journal content bled forever into RAG answers and claim-time briefings. Add OptimalService.unindex_journal_entry mirroring unindex_playbook (vector-store delete_by_source + tracking-row delete via get_db_context, both idempotent + best-effort), and call it from delete_entry after the row commit inside a try/except so a de-index failure never errors the delete. * [M25] learning_id hashes full content to avoid collision The memory distiller emits lessons with a fixed 'Problem: …' opening shape, so two distinct lessons whose first 100 chars match collided on learning_id = f"lrn-{md5(content[:100])[:12]}". replace_on_reingest then routed both to the same source URI and the second ingest's replace_chunks DELETE wiped the first lesson's chunks — silent data loss. Hash the full content (widening the hex slice 12→16) so distinct bodies get distinct ids and each retains its chunks. * [H13] reject non-internal local_llm_base_url at config load * [M28] bulk-insert learning broadcast instead of N+1 * [M27] mark_read/mark_all_read stamp only the unread rows seen at call time mark_read and mark_all_read used to zero the unread counter FIRST, then run a bulk UPDATE … WHERE read_at IS NULL that stamped every inbound unread row. A send_chat_message committing between the counter-zero and the UPDATE inserted a new read_at NULL row that the UPDATE then stamped as read — the new message was silently consumed while the counter stayed 0. Mirrors get_unread_messages (same file): SELECT the unread message IDs at call time, UPDATE exactly those IDs, then recompute the unread counter from the DB via the existing _reset_unread_counter helper. A message arriving mid-call is not in the selected ID set, so the UPDATE skips it and the recomputed counter keeps it unread. * [H12] dedup: exact to_agents predicate + purpose discriminator + ack DEL * [M23] playbook indexed_ok/indexed_at + startup reconcile of unindexed approved * [M24] RAG indexing dead-letter + janitor reclaim + failed_index_count health * [L23] institutional_memory_status sentinel distinguishes below-floor/empty/error/disabled * [L26] sweep_expired_notifications re-escalates stale unacked ack-required * [phase3] e2e smoke + CHANGELOG for 0.19.0 * [M24] _reindex_journal_entry honors is_private (C1 review fix) Dead-letter replay mirrors the original journal._schedule_rag_index path: a private entry is never indexed into the shared JOURNALS corpus, and a private learning is still recorded into LEARNINGS as non-shareable. Previously the replay always called index_journal_entry and skipped record_learning for private learnings, leaking private content on replay and dropping the legitimate non-shared learning. Three regression tests. * [H11] clone via git -c http.extraheader, not URL-embedded PAT * [H11] _sync_read_clone fetch via http.extraheader, not URL-embedded PAT Sibling site to the clone fix: the conventions read-clone refresh ran 'git fetch --tags <https://TOKEN@host> <branch>', exposing the PAT in the fetch argv on the orchestrator host. Mirrors the clone site's per-call '-c http.extraheader=Authorization: Basic …' prefix + bare URL. SSH URLs and tokenless public repos unchanged. * [H11] release_executor clone+push via http.extraheader; delete _inject_token_into_url * [H8] rebase_onto_base gates on clean tree like pull * [H9] _link_commit_to_task flushes, doesn't commit out-of-band * [M38] _pr_is_merged returns None on HTTPError; caller assumes merged * [M39] _cherry_unmerged_entry marker grep anchored to commit-prefix * [L1] thread actor_agent_id through update_pr_for_task * [H8] fix rebase test mocks for clean-tree gate H8 inserted a 'git status --porcelain' dirty-tree gate at the top of rebase_onto_base (mirroring pull). The 3 rebase control-flow tests mocked _run_git with a side_effect list matching the OLD call sequence (no leading status call), so every call shifted by one and the assertions missed. Prepend a clean-status result to each list so the gate passes and the fetch/checkout/reset/rebase/diff/abort/push sequence aligns. Verified: 16 passed (was 3 failed/13 passed post-H8, 16 passed pre-H8). * [L2] push --force-with-lease instead of bare --force * [L1] refresh stale workspace-resolution docstrings pr_target and _workspace_for_branch still documented the actor → assigned_to → created_by fallback chain that L1 removed from _resolve_workspace_agent_id. Update both to the post-L1 actor → assigned_to → None resolver (project.workspace_path as the final fallback) so a future reader doesn't rely on a fallback that no longer exists. * [M37] merge_pr locks the work_session row FOR UPDATE * [phase4] e2e smoke + CHANGELOG for 0.19.0 * [phase4] fix M37 test flake + document H8 skip The opus whole-branch review flagged the M37 concurrency tests as ~50% flaky: both asserted caller A wins the FOR UPDATE race, but which caller wins the lock is non-deterministic. When B won, the 'assert a_row.merged_by == a_merger' branch flipped false even though the production code (M37) was correct — exactly one merger recorded, audit trail intact. Assert the invariant instead: both rows COMPLETED, both report the same merged_by, value in {a_merger, b_merger}. Applied to both the unit test and the e2e twin. Also documents the H8 e2e skip in the module docstring (the report claimed it was documented there but it wasn't) and drops the internal 'Phase 4' label from the docstring header in favor of the public '0.19.0' version anchor. * [H24] wait_for_ci polls through the window on non-success * [H25,L34] release mutex orphan-sweep on start + shared redis client * [M1] tiktok _refresh commits rotated tokens in an independent session * [H25] drop new type:ignore in orphan-sweep test (constraint cleanup) * [M2] feature-spotlight re-arms when exploration stale past 2x interval with no live HoM spawn * [M6,M7] mark_seen after meaningful+project; persist since_id cursor in redis * [M3,M5] reject() guards COMPLETED; edited_body deferred into the single-flight lock * [M4] bound list_completed_video_tasks + ix_tasks_source_status_created index (migration 066) * [M8,M9,L9] pass head_sha to CI gate; _run_git 30s timeout; _commits_since split maxsplit 2 * [M10,L35] dedupe dep_update by (git_url, command); fold redundant per-project queries * [L36] gather ci_watch telemetry sweep instead of sequential iteration * [L11] document self_heal fingerprint is stable per-signal by design * [M11] engine-loop liveness watchdog: heartbeat + 2x-interval staleness alert * [M21] video render loop commits per-task, not one trailing commit * [M22] _detect_stuck_tasks skips held-CEO-source tasks * [L6] video_renderer_client._save writes temp + atomic rename * [phase5] e2e smoke + CHANGELOG for 0.19.0 * [M11] instrument x_mentions + roadmap engine loops with liveness heartbeats * [phase5] fix-wave: correct e2e M11 unit-test filename + strengthen failed-cycle heartbeat assertion * [C4] panel WS: shared /ws/system socket + long-tail retry + pong watchdog * [H15] video-post-queue caption derived per render (mirror x-post-queue) * [C4-fix] panel WS: discriminating long-tail tests + drop dead freeze block + evict dead shared conn on manual disconnect Finding 1 (Critical, websocket.test.ts): the two long-tail-retry tests fired onopen between close cycles, which reset reconnectAttempts to 0 each cycle, so they passed under the pre-fix 3-attempt gate. Rewrote both to NEVER fire onopen between closes, so attempts accumulates: test 1 asserts state stays 'reconnecting' past attempt 3 (old gate would flip 'disconnected' terminal); test 2 asserts a new socket is constructed within 30000ms at attempt 7 where uncapped 5000*1.5^7 ~= 85s (old uncapped code would leave the timer unexpired). Verified both FAIL on a reverted old-shape connection.ts and PASS on the fixed code. Finding 2 (Important, connection.ts): the 'if (raw >= cap) this.reconnectAttempts = exp' block was a no-op (exp was just read from the same field) and the unconditional increment afterwards grew the counter regardless. Deleted the dead block; kept the Math.min cap on the delay. Replaced the misleading ponytail comment with an accurate one: delay is capped, counter grows unbounded but delay is bounded. Finding 3 (Important, use-websocket.ts): manual disconnect() tore down the shared conn for all subscribers but left the dead (manualClose=true, never reconnects) entry in _sharedSockets, so a later mount hit the reuse branch, attached a subscriber, replayed 'disconnected', and never called connect(). Added a urlRef and _sharedSockets.delete(url) in the manual disconnect callback so a later mount reopens a fresh conn. * [H16] settings Save wired to settingsApi (persist + read back) * [H17] tasks page passes status/team/limit to useTasks (server-side filter) * [H18] useAgents roster re-derives on live-status change (statusEpoch in queryKey) * [M40] useMetrics reads agent counts from useAgentStatus cache (dedupe poll) * [H18] tighten useAgents statusEpoch comment (drop spec ref) * [M40] drop spec ref + tighten useMetrics comment * [M41] scorecard refetchInterval 60s -> 5min (25 req/min -> 5) * [M42] feature-flag off-transition confirm + pending-keys Set * [M43] X/TikTok credentials clear-behind confirm dialog * [M44] rate-limit syncFromApi merges (keep fresher hitAt) + A2A reconnect invalidation * [phase6] proxy.ts cookie-check comment + CHANGELOG Fixed entries * [phase6] drop stale WS pin-attempts comment + fix tasks-page lead-in * [H21] type DelegateRequest.estimated_complexity as Complexity (reject critical) * [H22] type SoftBlockRequest.resolver_type as BlockerResolverType (no silent AGENT fallback) * [H23] serialize TaskTable.documents into TaskResponse (DocRefResponse) * [L27] delete SubstituteRequest phantom suggested_role/suggested_team fields * [L14] Envelope.not_found defaults remediate (guide re-fetch + re-issue) * [L28] delete unused ListResponse generic (dead code; pagination deferred) * [H19] _delegate_static_guards allow cell_projects roots (cross-cell MegaTask) * [M13] MegaTask confirm-batch idempotency key from session_id (SETNX guard + result sidecar) * [M14] strip assigned_to from MegaTask drafts (no board-owned root-subtask deadlock) * [H20] thin_routes receiver-gate add/add_all/merge (no false block on set/cache.add) * [M16] tighten noqa code-capture to [A-Z0-9, ]+ (no false block on natural prose) * [M45] conventions read-clone force-refetch on read (no 30s stale map window) * [L25] conventions._resolve returns (root, sha); ORM mutated on the event loop * [M15] open_conventions_pr force-pushes disposable scaffold branch (no silent None) * [L24] roadmap cycle completion emits status-transition audit * [Phase7] CHANGELOG: 15 schema/conventions/MegaTask/API fixed (H21-H23,L27,L14,L28,H19,M13,M14,H20,M16,M45,L25,M15,L24) * [Phase7] lint gate hygiene: shorten docstring (E501), sort imports (I001), hoist AuditLogTable import (PLC0415) * [H14] Enable the GROK provider row in _apply_grok so routing reaches the GrokCliProvider * [M31] Route GROK active-token resolution to usage.json so live usage reflects grok agents * [M32] Pass cache read/write tokens to calculate_cost in the usage sweep so live cost reflects Anthropic cache spend * [M33] Park Ollama-Cloud rate limits via a marker map so a glm-5.2:cloud 429 parks instead of crash-respawning * [M34] Sweep orphan agent_spawn_sessions at startup so crashed-run tokens roll into usage/cost summaries * [L12] Persist revisit_resets (migration 067) so the PM-respawn breaker's revisit counter survives a restart * [L18] Date-gate the Sonnet-5 promo revert so billing returns to list rates after 2026-08-31 * [L20] Warn when ROBOCO_GROK_RUN_LOG yields no session id instead of silently falling back to a zero-usage env id * [phase8] CHANGELOG: LLM provider routing, usage capture, billing fixes * [phase8] Trailing ruff format hygiene (orchestrator marker tuples, token-sweep test signatures) * [phase8] Fix mypy: rename GROK-branch tokens var so transcript fallback stays reachable * [M35] Add an expiring agent-token format (iat/exp) with backward-compatible verify * [M35] Wire agent-token TTL at spawn (config + orchestrator + grok) so tokens are bounded * [M36] Add JWT jti claim and re-mint the sliding cookie only near expiry so a stolen cookie's exp is fixed * [M36] Redis jti revocation: read_token rejects revoked jtis and logout revokes the current jti * [phase9] CHANGELOG: bound agent tokens + sliding-cookie re-mint window + jti revocation * [scan-fix] mypy: type-annotate test files for make-quality gate CI's make quality runs mypy roboco/ tests/; the scan-fix program's local gate ran mypy roboco/ only, so test files were never type-checked. Fix all 67 errors across 23 test files with real annotations/casts/asserts/dead-code removal — no # type: ignore / # noqa added. * [e2e] Per-test DB isolation + dispatcher re-claim before PM complete * [scan] Regenerate verb tables for delegate Complexity type * [scan] Reduce 9 xenon C-ranks to B (auth, orchestrator, gateway, services) * [scan] Restore short-circuit time.time() in verify_agent_token (security path) --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com>	2026-07-07 10:09:23 +02:00
3 people	fa3e25e656	feat(grok): pluggable agent providers + Grok on the official grok CLI (#218) ... * feat(providers): pluggable agent providers + Grok (xAI) backend Add a roboco/llm/providers/ seam — an AgentProvider lifecycle ABC and a ProviderRegistry keyed by ModelProvider — so the orchestrator can drive agent backends other than Claude Code. The first non-Claude backend is GrokProvider for xAI's grok-build-0.1. xAI is OpenAI-compatible only (no Anthropic-Messages endpoint), so a Grok agent runs an OpenAI-protocol runtime pointed at https://api.x.ai/v1 rather than the ANTHROPIC_BASE_URL injection the other providers use. It reuses the orchestrator's existing mount/auth assembly, so it inherits the same MCP gateway + tool-manifest wiring as every other agent by construction, and passes its prompt via env (never an argv positional). The change is purely additive: only GROK routes through the registry; Anthropic / Ollama Cloud / self-hosted spawns run the existing _spawn_container path unchanged. Includes: - ModelProvider.GROK (migration 038) + a seeded Grok provider row (migration 039) + a grok-build-0.1 catalog entry - GET/PUT /api/providers/grok-key to store the xAI key (Fernet-encrypted, reusing the existing provider-key machinery) - ClaudeCodeProvider reference adapter over the current spawn - unit tests for the registry, GrokProvider (gateway wiring, no ANTHROPIC_* leak, prompt-injection safety, failure paths) and routing The dedicated roboco-agent-grok image and the exact OpenAI-protocol CLI invocation are the remaining piece to finalise with xAI. * feat(providers): native Grok runtime — opencode image, config gen, panel key Complete the native Grok (xAI) path so grok-build-0.1 runs as a real RoboCo agent, not just the provider seam. - roboco-agent-grok image (docker/agent-grok.Dockerfile): FROM agent-base + opencode (the OpenAI-protocol runtime). One image serves every role; role behaviour comes from the mounted manifest / mcp-config / system prompt, exactly as on the Claude path. - Entrypoint renders opencode.json at spawn from the GrokProvider env contract + the mounted Claude Code mcp-config.json (roboco.llm.providers.opencode_config): translates RoboCo's gateway servers (roboco-flow / roboco-do / ...) into opencode's mcp block, declares the xAI OpenAI-compatible provider + model, and wires permissions + instructions. Pure, unit-tested translation. - Orchestrator registers GrokProvider with the registry-qualified image (_qualify_agent_image) so it resolves in local and registry deploys. - Compose (both files + the registry compose) gain an agent-grok-image builder service. - Panel: a Grok (xAI) API key card on the AI Providers page, plus the grok ModelProvider value. KNOWN PARITY GAP (opencode runtime): the bash-guard PAT-scrub and the transcript-based usage/cost capture are Claude Code hooks and do not transfer to opencode. bash permission is operator-tunable (ROBOCO_GROK_BASH_PERMISSION) so a deployment can fail closed until a security/usage-parity opencode plugin lands. That plugin and live E2E validation are the remaining work to finalize with xAI. * ci(release): build + publish the roboco-agent-grok image Add roboco-agent-grok to the release workflow's image build/publish map so registry deploys carry the Grok runtime image (parity with every other agent image). Split from the feature commit because pushing a workflow change requires a workflow-scoped token. * fix(migration): commit the grok enum value before seeding (autocommit_block) CI's "Apply database migrations" failed with asyncpg UnsafeNewEnumValueUsageError: alembic runs the whole upgrade in a single transaction, so migration 039's INSERT used 'grok' in the same transaction that 038 added it — which Postgres forbids. Splitting into two migration files did not help (one transaction spans both). Wrap the ALTER TYPE ADD VALUE in op.get_context().autocommit_block() so the value commits before 039 (and any later migration) uses it. Still renders in offline --sql, so the enum-migration-parity test is unaffected. * feat(grok): price grok-build-0.1 + secret-scrub opencode plugin - pricing.py: add grok-build-0.1 rates ($1/1M input, $0.20 cached, $2/1M output), verified against xAI's published pricing. Grok is a priced non-Anthropic model, so cost computes the moment usage is captured. - secret-scrub.js: an opencode tool.execute.before plugin porting the security-critical bash-guard deny rules (git network ops, credential-file reads, /proc env, internal-host HTTP, roboco.* imports, ROBOCO_AGENT_ID forgery, env dumps, destructive rm) to the opencode runtime — restoring the guard the Claude Code hook can't provide there. Throwing denies the call (confirmed by opencode's env-protection example). Wired into the generated opencode.json plugin array + baked into the grok image. Deny logic verified via node (9 deny + 5 allow cases). UNVALIDATED against a live opencode runtime: confirm it fires in the live E2E spawn before a Grok dev-agent touches a real repo; the bash permission is operator-tunable as a second gate. Cost CAPTURE (distinct from pricing) is intentionally NOT built yet: opencode's plugin hooks expose model info but no token/usage object, so the capture path is unconfirmed and needs the live spawn to settle. * feat(grok): read opencode session usage for cost capture Confirmed by inspecting a local opencode run: opencode persists per-session usage in SQLite at ~/.local/share/opencode/opencode.db — the `session` table carries cost + tokens_input/output/reasoning/cache_read/cache_write. xAI's response usage object (prompt_tokens, completion_tokens, prompt_tokens_details.cached_tokens, completion_tokens_details.reasoning_tokens) maps directly onto those columns. Add opencode_usage.read_session_usage / cost_for_session: read the opencode DB and price the tokens via roboco.billing.pricing (our cost stays authoritative; opencode's own `cost` column is kept for reference). Tested against a fixture DB mirroring the real schema (single session, summed sessions, missing/empty DB). Remaining wiring (for the live spawn): mount the opencode data dir on grok spawn + call cost_for_session at reap to record the usage rollup. * fix(grok): correct opencode provider (Responses API), stdin, reasoning cost A live opencode run against api.x.ai/v1 surfaced three real bugs: 1. Provider package — grok-build-0.1 is driven via the OpenAI Responses API (opencode calls model.responses()). @ai-sdk/openai-compatible is chat/completions only and errors "responses is not a function". Switch the generated opencode.json provider + the grok image to @ai-sdk/openai. 2. Headless hang — `opencode run` blocks after init without a TTY; close stdin (`< /dev/null`) in the entrypoint so it proceeds to the model call. 3. Reasoning-token cost — grok-build-0.1 is a reasoning model; reasoning tokens bill as output but opencode stores them in a separate column. cost_for_session folds tokens_reasoning into output (else ~22x undercount). Verified end-to-end against a real session row (input=6120, output=1, reasoning=226, cache_read=1856): our pricing reproduces opencode's stored USD cost ($0.0069452) exactly. Tests anchored to that real row. * feat(grok): first-class xAI/Grok routing mode (UI + backend) The Routing-mode toggle had Anthropic / Ollama / Self-Hosted / Mix but no way to route the whole org to Grok. Add it end to end: - backend: apply_mode("grok") + _apply_grok (GLOBAL default -> grok-build-0.1) + derive_mode "grok" detection; ApplyModeRequest/ModeResponse accept "grok". - panel: a "Grok" routing-mode card (between Anthropic and Ollama, gated on the xAI key) + flipToGrok; a Grok group in the per-agent mix dropdown + catalogGrokOnly + a grok ProviderBadge variant; the mix-save key check and the AI-routing description now cover Grok. - tests: integration derive_mode/apply_mode "grok" cases (+ grok provider row in the fixture). Gated: ruff + mypy clean; panel typecheck + lint clean. * feat(grok): reasoning-effort by role (cut grok-build cost on cheap roles) grok-build-0.1 reasons heavily by default and reasoning bills at the output rate (a live "say ok" call emitted ~300 reasoning tokens, ~85% of its cost). Confirmed live that opencode's `--variant minimal` cuts reasoning ~54% (298 -> 136 tokens, same prompt). GrokProvider now picks reasoning effort by role: code-quality roles (developer, qa, pr_reviewer) keep full reasoning; coordination / docs / board roles (cell_pm, main_pm, documenter, product_owner, head_marketing, auditor, prompter, secretary) run "minimal". It's passed to opencode via the entrypoint's `--variant`. Operators can force one effort for ALL grok agents with the ROBOCO_GROK_REASONING_EFFORT env (minimal | high | max, or default/full). Tests cover the role map, the env override, and the spawn env wiring. * style(panel): show the Grok (xAI) key card above the Ollama card * fix(grok): stop opencode subagent-stream hang at the config layer The Grok pr_reviewer wedged in_progress forever: opencode's default agent ran with the subagent `task` tool enabled, spawned an Explore subagent on grok-build-0.1 whose model call opened an SSE stream that went idle, and the run hung with no timeout. - Hard-disable opencode's subagent `task` tool in the generated opencode.json. No RoboCo role uses opencode-internal subagents — work flows through the gateway verbs — so removing the tool kills the hang trigger outright. - Set provider.xai.options.timeout + chunkTimeout (operator-tunable via ROBOCO_GROK_REQUEST_TIMEOUT_MS / ROBOCO_GROK_CHUNK_TIMEOUT_MS) as the defence-in-depth backstop; chunkTimeout aborts an idle stream. - Bundle the permission + timeout + subagent knobs into an OpencodeGuards dataclass (keeps the builder under the arg-count gate). - Drop the dead ROBOCO_AGENT_TOOLS spawn env (it had no consumer); opencode tool restriction lives in the rendered config now. * feat(grok): reaper watchdog kills wedged opencode containers The heartbeat reaper deliberately skips a task whose assignee holds a live ACTIVE container, so a Claude agent deep in a long edit/test cycle isn't churned out from under live work. A wedged opencode container breaks that assumption: it stays ACTIVE while firing no gateway verb, so its heartbeat never advances and the live-instance skip would shield its task forever — the exact way the Grok pr_reviewer parked in_progress. Add a longer grok-idle kill threshold (ROBOCO_GROK_IDLE_KILL_SECONDS, default 900s, well past the stream chunk timeout). A GROK instance idle past it is force-removed (its logs dumped to disk first) and evicted from the instance registry, so the same reaper pass then releases the task. Only GROK runtimes are eligible — a quiet Claude agent keeps the heartbeat-skip protection. * feat(grok): guard interactive roles from GROK routes (interim) intake (prompter) and secretary run a held-open chat session driven by the Claude Agent SDK. GROK has no interactive runtime yet, and a GROK route for those slugs would be spawned with the route creds injected as ANTHROPIC_* against api.x.ai/v1 — the wrong protocol — producing a silent, empty reply (the blank intake we observed). Downgrade a GROK route for intake-1/secretary-1 to the Anthropic default with a logged warning. The one-shot delivery roles route to GROK unchanged. This guard is replaced by the real interactive fork once the opencode interactive driver lands. * feat(grok): capture one-shot Grok usage/cost from the opencode store A GROK agent runs opencode, not Claude Code: it has no SDK /usage/status server and writes no Claude transcript, so _resolve_final_token_usage found nothing and every Grok agent finalized at 0 tokens / $0 — the opencode_usage reader existed but had no caller. - Mount a per-agent opencode data dir ($DATA/opencode/<agent_id> → /home/agent/.local/share/opencode) so opencode.db is captured, and mount the same host dir into the orchestrator (/data/opencode) in all three compose files so the finalizer can read it back — the opencode analogue of the mounted Claude transcript. - _resolve_final_token_usage branches on provider_type: GROK reads opencode.db via opencode_usage (reasoning folded into output, billed at the output rate) and skips the SDK/transcript path. A 0-token read logs a WARNING so a silent mount failure isn't mistaken for a real zero-cost run. - ROBOCO_OPENCODE_DATA_DIR overrides the in-orchestrator path for local runs. * feat(grok): make interactive spawns first-class on AgentProvider (additive) The AgentProvider ABC modelled only the one-shot lifecycle (spawn/stop/ health_check/remove), so the interactive intake/secretary roles could never route through a provider. Add an opt-in interactive surface: - supports_interactive class flag (default False). - InteractiveSpawnSpec: the resolved AgentConfig + session id + role-specific image + optional HMAC token — everything a provider needs without importing orchestrator internals. - spawn_interactive(spec): a non-abstract default that declines via ProviderError, so every existing one-shot provider is unchanged. Pure scaffolding — no provider opts in yet (GrokProvider flips the flag when its interactive driver lands). Zero behavioural change. * feat(grok): Grok-native interactive runtime (opencode serve) — container side Builds the Grok analogue of the Claude intake/secretary live-session runtime, satisfying the same IntakeSession seam so the existing IntakeDriver loop, message source, relay, and StreamChunk panel contract are reused unchanged: - OpencodeServeSession: a held-open `opencode serve` session (context persists across turns) where each human turn is one synchronous POST /session/:id/ message; normalize_opencode_message maps the reply parts to text/thinking/ tool_use/draft/turn_end chunks (draft via a propose_draft tool part or the fenced roboco-draft fallback). Doc-verified against opencode's server API. - grok_intake_main / grok_secretary_main: container entrypoints mirroring the Claude mains but yielding an OpencodeServeSession; they render opencode.json (xAI provider + MCP + system prompt) first, then run the receiver + driver. - roboco-agent-grok-prompter / -secretary images (FROM roboco-agent-grok) + their builder services in all three compose files. UNVERIFIED-LIVE: the opencode serve flow + exact Part schema + draft path need a live run against grok-build-0.1 (the part mapping is defensive). The orchestrator wiring that routes a GROK intake/secretary route to these images is the next step (a design decision is open — see the handoff notes). * feat(grok): route interactive intake/secretary to opencode-serve images Wire the GROK interactive path the in-place way (matching how the interactive roles already choose ANTHROPIC_* per route), so a GROK route launches the Grok-native opencode-serve image instead of the Claude SDK-driver image: - _spawn_intake_container / _spawn_secretary_container pick the grok-prompter / grok-secretary image (ensuring the base→grok→interactive build chain) when the route is GROK, and stamp provider_type on the spec + AgentConfig so finalize routes usage to the opencode store. - _build_intake_run_cmd / _build_secretary_run_cmd inject OPENAI_* + the opencode store mount + system-prompt env for GROK via a shared _append_interactive_provider_env, keeping ANTHROPIC_* for every other provider. The intake's minimal mounts (no gateway MCP) are preserved, so Grok intake matches the Claude intake's tool surface (the spec). - Add a per-agent opencode store mount to the interactive host paths so interactive Grok usage/cost is captured like the one-shot path. Removes the interim Phase-0 routing guard (the real path supersedes it) and retires the unused AgentProvider.spawn_interactive/InteractiveSpawnSpec seam — the interactive roles have a bespoke assembly that the one-shot provider surface doesn't fit, so the fork lives in their own builders. UNVERIFIED-LIVE: end-to-end intake/secretary chat on Grok needs the stack up + opencode serve confirmed against grok-build-0.1. * feat(grok): surface intake/secretary in the mix-mode picker; doc guardrail parity - Panel: add intake-1 (prompter) and secretary-1 to the mix-mode per-agent routing list so an operator can assign Grok (or Claude) to the interactive roles from the UI; assigning a Grok model routes them to the opencode-serve image. tsc + eslint clean. - opencode_config: correct the now-stale parity note — bash-guard is ported (secret-scrub.js) and usage/cost is captured (opencode store); the remaining gap is the budget/loop/stop/prompt-injection hooks, which need a sidecar plugin (open decision), with ROBOCO_GROK_BASH_PERMISSION as the interim gate. * test(grok): mypy-clean the reaper watchdog + interactive spawn tests The CI mypy scope (roboco/ tests/) flagged test-only typing issues my per-file runs missed: direct method assignment (orch._remove_container = AsyncMock()) trips [method-assign], and a module-level dict[str,str] is invariant against the dict[str, str|None] the run-spec expects. - Use monkeypatch.setattr for _remove_container in the watchdog tests. - Annotate the shared _HOSTS as dict[str, str | None]. Production code unchanged; mypy roboco/ tests/ is green. * feat(grok): cost-ceiling kill-switch (budget-guardrail parity) Claude Code's per-agent token-budget hook fires against the SDK :9000 server; opencode exposes NO usage/budget hook to a plugin (confirmed against its plugin docs), so the budget kill-switch can't be a plugin/sidecar — the orchestrator enforces it instead. _enforce_grok_cost_budget runs each dispatch tick: for every ACTIVE GROK container it reads cumulative cost from the opencode store (the Phase-2 reader) and kills + evicts it past ROBOCO_GROK_MAX_COST_USD (0 = off), after which the reaper releases the freed task. This also catches a runaway loop that keeps firing verbs (so it evades the idle watchdog) but still burns cost. Covers the budget/runaway-burn slice of guardrail parity. The remaining Claude hooks (prompt-injection PRE-gate, stop-guard terminal-verb) have no blocking opencode equivalent — opencode's message/stop hooks are observe-only — and the interactive reasoning-variant has no opencode.json/serve knob (CLI-flag only); both are pinned for a live probe rather than shipped as a guess. * docs(grok): document Grok's reduced guardrail posture (honest, not blocking) Grok agents run on opencode, not Claude Code, so they do NOT have full guardrail parity — claiming otherwise would be false. Document it truthfully and keep them usable rather than blocking them. - Panel routing card: an amber caveat shown in Grok/Mix mode — command/ secret-exfil guard + cost cap apply to Grok, but the prompt-injection guard does NOT (opencode cannot block a turn); Anthropic/Ollama/Self-Hosted run through Claude Code with the full guard set; prefer those for agents that ingest untrusted or cross-agent content; Grok is safe for trusted work. - docs/self/architecture/llm-provider-security.md: the reference — the two runtimes, which provider uses which, the per-guardrail parity matrix, why the injection/stop gaps exist (opencode hooks are observe-only), and the routing recommendation (delivery roles handling untrusted content → a Claude-Code-runtime provider). Panel tsc + eslint clean. * fix(grok): make the live interactive path work — store perms, error surfacing, variant Found by actually running opencode serve locally (the path was doc-verified but never executed). Three fixes: 1. EACCES on the opencode store mount (the live intake crash): on Linux docker auto-creates a missing bind source as root:root, so the non-root agent user could not mkdir/write in /home/agent/.local/share/opencode and opencode died at boot. _ensure_opencode_data_dir pre-creates the per-agent dir 0777 before the mount (one-shot via the _GrokHost seam, interactive in both spawns). 2. Silent blank reply on a model error: opencode reports a turn failure in info.error with parts=[], NOT as a part — verified live (a bad xAI key returns info.error APIError). send() / normalize_opencode_message now surface it as an "error" StreamChunk so a failed turn is never blank (the original intake bug class). Confirmed live: the error now renders. 3. Reasoning variant on the serve path: the live OpenAPI shows the message body accepts a "variant" field (it is NOT CLI-only, as the docs implied), so the pin is unblocked. send() passes ROBOCO_GROK_VARIANT as the per-turn variant; the orchestrator sets it per-role (_reasoning_effort_for) for interactive Grok, the same lever as the one-shot --variant. opencode serve startup, POST /session, session-id extraction, the part-type mapping (text/reasoning/tool), and the error path are all validated against a live opencode 1.17.8. A real successful grok reply still needs a funded key. * fix(grok): pre-create agent-owned ~/.local in the grok image (opencode state EACCES) Running the built grok-prompter container surfaced a second EACCES the mechanism analysis missed: bind-mounting the opencode store at ~/.local/share/opencode makes docker create the intermediate ~/.local AS ROOT, so the non-root agent user then cannot create its sibling ~/.local/state and opencode dies at boot. Pre-create the ~/.local tree agent-owned in the image so the mount leaves the parents writable. Complements the orchestrator 0777 host-source pre-create (which covers the bind source on Linux). Verified live: with this fix the container starts clean, opencode serve opens the session, a POST /turn produces a real grok reply, and all chunks (thinking/text/turn_end) reach the relay endpoint. * feat(grok): prompt-injection guard for Grok (parity with the Claude hook) The injection guard is RoboCo's own hook (user-prompt-hook.sh), not a runtime built-in, so it can be recreated at our input boundary regardless of runtime — opencode's lack of a blocking pre-prompt hook is irrelevant. - prompt_guard.detect_injection: the deny patterns ported to reusable Python. - IntakeDriver._run_turn scans every interactive turn before sending it to the model and denies a match as an error chunk. Covers BOTH Grok (opencode) and the Claude SDK intake (which runs with setting_sources=[] and so never loaded the bash hook — it was unguarded too). - The one-shot grok entrypoint scans ROBOCO_INITIAL_PROMPT and refuses a poisoned task prompt (parity with the Claude UserPromptSubmit deny). - Broadened the pattern (Python + the bash hook, kept in sync) to catch the multi-qualifier canonical phrasing "ignore all previous instructions", which the single-qualifier original missed — without false-positiving on "ignore the linting rules" (an intermediate non-qualifier word breaks it). So Grok now has the command/secret-exfil guard (secret-scrub), the cost cap, AND the injection guard. Verified: 94 agent_sdk tests pass; bash + Python agree on detect/miss cases. * docs(grok): drop the security disclaimers — injection guard closes the gap With the prompt-injection guard now recreated for Grok (prior commit), the "Grok lacks the injection guard / prefer Claude for delivery roles" warning is no longer true, so remove it: - Panel routing card: replace the amber "prefer Claude / not safe" caveat with a neutral one-liner — Grok agents run on opencode; the command/secret-exfil guard, the prompt-injection guard, and the cost cap all apply. - docs/self/architecture/llm-provider-security.md: prompt-injection row flips to "yes" for Grok; intro + routing recommendation updated to "effective security parity, any agent (incl. delivery roles) can run on Grok"; the only remaining unported hook is the non-security stop-guard. - opencode_config docstring: the remaining gap is now just the stop-guard (budget + injection are covered). Panel tsc + eslint clean. * fix(grok): allow external-directory reads so the pr-reviewer can work Live NAS run showed the Grok pr-reviewer claim the review and fetch the diff, then write it to /tmp and FAIL to read it back: opencode auto-denied "external_directory (/tmp/*)" — its file tools refuse paths outside the project cwd, and in headless serve/run mode an "ask" permission auto-rejects (no human). Add permission.external_directory (default "allow", env ROBOCO_GROK_EXTERNAL_DIR_PERMISSION) to the generated opencode.json. The container is the sandbox and secret-scrub still blocks credential-file reads, so allowing in-container external-dir reads is safe and unblocks legitimate scratch use (e.g. the pr-reviewer grepping a large diff in /tmp). Verified live against grok-build-0.1: with external_directory:"allow" the Read tool reads a file outside cwd and returns its contents (no auto-reject); the plain-string form is accepted by opencode 1.17.8. Needs a rebuild of roboco-agent-grok + a pr-reviewer re-run on the NAS to confirm. * refactor(grok): split eligibility out of _maybe_kill_wedged_grok (xenon C -> B) CI complexity gate (make quality -> xenon --max-absolute B) flagged _maybe_kill_wedged_grok at rank C — too many guard branches in one method. Extract the kill-candidate decision into _wedged_grok_slug(task, last_heartbeat) -> slug | None (recent-heartbeat / no-owner / not-ACTIVE / not-GROK all yield None); _maybe_kill_wedged_grok now just kills + evicts the returned slug. Behaviour is identical (same guards, same order) — the reaper watchdog tests pass unchanged. xenon now passes on the full package; ruff + mypy clean. * feat(grok): start the in-container SDK server + budget feed (Claude parity) The keystone of the Grok parity work (CEO's "take Claude as baseline, create what's missing" call): the one-shot Grok container now starts the same SDK server the Claude path runs, so the per-verb circuit breaker (the flow/do MCP servers already POST /verb/attempted to it), the per-session budget/loop counters, the terminal-verb tracking, and the SessionEnd post-mortem all work on Grok instead of being silently absent. - entrypoint: launch roboco.agent_sdk.server (bare venv python, not `uv run` which would re-sync the drifted clone lock and stall), wait for /health, reset counters; run opencode WITHOUT exec so the script regains control to run the post-mortem and the silent-exit substitute after the run returns. - budget-feed.js: opencode plugin that gates on /budget/status in tool.execute.before (halt/loop deny — the only place to stop a runaway one-shot run; opencode has no PostToolUse-deny) and records the executed tool + args-hash in tool.execute.after. Fail-open; bare-verb normalization for MCP-namespaced terminal verbs. - silent-exit substitute: on a graceful exit with no terminal verb the entrypoint posts /terminal/force_substitute so the task isn't left stuck claimed/in_progress (Stop-hook parity at the boundary). - opencode_config: wire budget-feed into the plugin array; add ROBOCO_OPENCODE_EXTRA_PLUGINS so per-image role tool plugins load scoped to one role; read the per-role ROBOCO_GROK_EDIT_PERMISSION. Targeted gate green (ruff/mypy/xenon + opencode_config tests; node --check on the plugins; bash -n on the entrypoint). * feat(grok): give the Grok Secretary its CEO-authority tools (blocker) The Grok Secretary could chat but had zero directive tools — it could not read company state or act on a CEO command, so it was non-functional. This is the integration blocker. - secretary-tools.js: opencode plugin registering read_company_state / read_task / submit_directive via the Hooks.tool API, each calling /api/secretary/* with the container's HMAC agent token — a direct port of the Claude Secretary's SDK tools (secretary_driver.build_secretary_options). The high-impact directive kinds stay gated server-side (queued for CEO confirm). - agent-grok-secretary.Dockerfile: bake the plugin and scope it to this image via ROBOCO_OPENCODE_EXTRA_PLUGINS, so only the Secretary carries CEO authority. - grok_secretary_main: correct the docstring that falsely claimed the tools reached the API "through the mounted MCP gateway" (there is no gateway mount; they're an opencode plugin). - secretary.md: name the three tools and restate the confirm-before-act gate. Verified locally that opencode loads a file-path plugin importing @opencode-ai/plugin and resolves the package; the live model-tool-call + backend round-trip is flagged UNVERIFIED-LIVE for the NAS. * feat(grok): give the Grok Intake its propose_draft tool (draft card) The prompter prompt tells the model to call propose_draft when the spec is ready, but on Grok that tool didn't exist — so no draft chunk, no panel draft card, and the human couldn't launch a task from a Grok intake chat. - intake-tools.js: opencode plugin registering propose_draft via Hooks.tool; the execute() only ACKs — the driver (OpencodeServeSession.normalize -> _is_propose_draft -> _draft_from_tool_input) intercepts the tool CALL and emits the `draft` chunk the panel renders. - agent-grok-prompter.Dockerfile: bake the plugin, scoped to this image via ROBOCO_OPENCODE_EXTRA_PLUGINS (delivery roles never draft). - test: a propose_draft tool part normalizes to a draft chunk (not a tool_use). The live tool-call -> draft-card path is flagged UNVERIFIED-LIVE for the NAS. * feat(grok): scope opencode edit/bash/external-dir permissions per role Grok wrote ONE global permission block, so a Grok pr_reviewer (or qa / PM / auditor) ran with edit=allow + bash=allow on untrusted PR content. Now the permissions are derived per role, mirroring orchestrator._get_role_permissions on the Claude path: - edit — allow only roles that write code (role_config.allows_write: developer / documenter); everyone else edit=deny. - bash — allow only roles that legitimately run a shell (developer / documenter / cell_pm / main_pm); the read-only reviewers (qa / pr_reviewer / auditor) and the board get bash=deny. secret-scrub still guards the rest. - external_directory — only the pr_reviewer reads scratch outside its cwd (the /tmp diff); delivery roles get deny. One-shot roles resolve these in GrokProvider._append_grok_env; the interactive intake/secretary set edit=deny + bash=deny in the orchestrator (intake keeps external-dir reads for sibling product repos, the secretary does not). The Claude path is untouched — the permission env is a GROK-only contract. Targeted gate green (ruff/mypy/xenon + provider + interactive-spawn tests). * feat(grok): park the provider on an xAI 429 (break the respawn loop) A one-shot grok run that hit an xAI 429 exited without a terminal verb; the dispatcher then re-spawned the same task every tick (429 -> exit -> respawn), a container/token/cost loop with no living agent to call i_am_blocked. - entrypoint: detect a rate-limit signature in the run output and exit 75 (EX_TEMPFAIL); a rate-limited task is NOT substituted — it must be retried. - _handle_stopped_container: on a grok exit 75, park the provider via the rate-limit tracker (retry_after window) instead of crash-retrying, and don't count it as a crash. The existing probe-resume loop clears the park after the window (unknown-provider time-expiry fallback) and the task is retried. - spawn_agent: a grok-only, fail-open guard skips the launch while the provider is parked, so the dispatcher no-ops instead of looping. The Claude path is untouched. Targeted gate green (ruff/mypy/xenon + new rate-limit tests; bash -n on the entrypoint). * feat(grok): close the secret-scrub bash-guard parity gaps secret-scrub.js (the opencode bash guard) was missing three rules the Claude bash-guard hook has, leaving a Grok dev able to read secrets the Claude path blocks: - source / dot-source of a credential-bearing file (source .env, . ./.env, .bashrc / .git-credentials / .netrc / /proc/*/environ). - interpreter one-liner reading a credential file (python -c "open('.env')", node -e "readFileSync('.git-credentials')"). - git-ops check now runs on a SKELETONIZED command (heredoc bodies + echo/printf args stripped) so a README/heredoc that merely documents `git push` is no longer mistaken for invoking it — a false-positive parity fix from the Claude guard. Functionally smoke-tested with node against the real plugin (git push denied; echo/heredoc "git push" allowed; source/interpreter cred reads denied; normal commands allowed). Live opencode firing stays flagged in the file header. * fix(grok): record a usage session for interactive intake/secretary (M1+M7) _spawn_intake_container / _spawn_secretary_container built the AgentInstance by hand and never recorded an agent_spawn_sessions row, so the reap finalizer had no usage_session_id to look up — every interactive session (Claude or Grok) finalized at 0 tokens / $0 in the rollups. Record the session (task_id=None) and pin its id on the instance, mirroring _launch_spawn; the GROK path reads opencode.db by this id, the Claude path reads the transcript. Also correct the grok_intake_main docstring (M7): it claimed the serve process was "gateway-wired" with an "MCP gateway", but interactive intake mounts no gateway — its only tool is propose_draft, registered by the intake-tools.js plugin. * fix(grok): surface a dead opencode-serve clearly instead of a zombie chat (M2) If `opencode serve` died after the session opened, every subsequent turn failed with an opaque httpx connection error while the container lingered. send() now detects the exited subprocess (returncode set) and yields a clear error chunk + turn_end so the panel shows a real "session ended — start a new chat" message; the idle watchdog / a human reap then tears the container down. * fix(grok): close the panel relay when the cost-cap kills an interactive chat (M4) _enforce_grok_cost_budget killed + evicted a container directly. For the interactive roles (intake/secretary) that left the panel SSE relay open with no close sentinel, so the chat froze with no explanation. Add PrompterLiveRegistry.close_by_agent (push a final error event, then close every session bound to that agent) and call it from the cost-cap watchdog when the killed agent is the intake or secretary, so the panel reports the chat ended on the cost cap instead of hanging. * fix(grok): make the opencode runtime actually load — proven live on grok-build-0.1 Live verification (opencode 1.17.8 + grok-build-0.1, funded key) showed the Grok runtime was loading INERT, three ways: 1. The provider override `provider.xai.npm=@ai-sdk/openai` failed model resolution (ProviderModelNotFoundError) — opencode can't resolve that package from its module path. Worse, ANY custom `provider.xai` block (even just options) breaks plugin-tool registration. opencode's BUILT-IN xai provider drives grok-build-0.1 with working tool-calls, so emit NO provider block; the key + base reach it via XAI_API_KEY / XAI_BASE_URL env (provider.options.apiKey alone does NOT authenticate). 2. Plugins referenced by absolute path in the config `plugin:` array never registered their hooks/tools. opencode 1.17.8 only registers from the plugin AUTO-DISCOVERY dir (~/.config/opencode/plugin/). Bake all plugins there. 3. Plugins must use a NAMED export, not `export default`. Changes: - opencode_config: no `provider` block, no `plugin` array; drop the dead XaiTarget + timeout machinery; build_opencode_config now takes a model string. - GrokProvider / orchestrator interactive env: inject XAI_API_KEY + XAI_BASE_URL (drop the now-unused OPENAI_*). - secret-scrub / budget-feed / secretary-tools / intake-tools: named exports; baked into /home/agent/.config/opencode/plugin/ (drop the EXTRA_PLUGINS env). - agent-grok* Dockerfiles: plugin dir + agent ownership; drop the unneeded @ai-sdk/openai global install. Verified live end-to-end: grok-build-0.1 calls read_company_state AND submit_directive through secretary-tools.js and the backend receives both with the agent token; a tool.execute.before guard fires; built-in tool-calls work. Targeted gate green (ruff/mypy/xenon + opencode_config/providers/interactive tests; node --check the plugins). * fix(grok): deliver intake draft via the relay + correct opencode-mechanism docs Live end-to-end verification (opencode 1.17.8 + grok-build-0.1) of the WHOLE integration, then fixes for what it surfaced: 1) Intake draft card (FUNCTIONAL): opencode's synchronous serve reply (POST /session/:id/message) returns only [step-start, text, step-finish] — it does NOT include tool-call parts, so the driver could never extract the propose_draft draft. intake-tools.js now POSTs the draft straight to the prompter-live relay (/api/prompter/live/{session}/events, the same endpoint the driver's relay sink uses), so the panel renders the card regardless. Verified live: grok calls propose_draft -> the relay receives the draft. 2) Correct misattributed opencode "bugs" (DOCS): earlier comments asserted as general opencode behavior that a provider.xai block / npm override / config plugin:-array "break" registration. Re-testing showed those were artifacts of a PROJECT-level .opencode/opencode.json; from the GLOBAL config (which opencode_config writes) the built-in provider, model resolution, the plugin array AND the auto-discovery dir all work, and MCP gateway verbs register (delivery agents verified). Reframed the comments as design choices (built-in provider + XAI_API_KEY env + plugins baked in the auto-discovery dir with named exports) and dropped the false claims. 3) Reasoning --variant: passing it does not error, but whether opencode applies a named reasoning variant to grok-build-0.1 (no provider-defined variants) is UNVERIFIED — comment softened from a "~54% cut" claim to best-effort, measure-on-NAS. Verified live this session: one-shot delivery (model + MCP verbs + plugins + hooks), secretary tools (read_company_state + submit_directive -> backend with token), intake draft (relay), grok built-in-provider tool-calling. Remaining NAS-only: full container assembly (SDK :9000 startup, entrypoint hooks, 429 parking) + the --variant cost measurement. Gate green (ruff/mypy + 51 tests; node --check the plugins). * feat(grok): reap abandoned interactive chats (M3) An interactive intake/secretary chat the human abandoned (closed the tab without confirming or stopping) leaked its container until the orchestrator restarted — the wedged-grok reaper is task-driven and these run task_id=None, and an SSE disconnect intentionally does NOT reap (so a page reload can reconnect). Reap by IDLE TIME, not connection state: PrompterLiveRegistry tracks last_activity (bumped on every push/deliver = a turn), and the 60s sweeper retires sessions idle past ROBOCO_INTERACTIVE_IDLE_REAP_SECONDS (default 1800; 0 disables) via reap_intake_session / reap_secretary_session. An active or page-reloaded chat that keeps exchanging turns stays fresh and is never reaped; board-review-parked sessions (task_id set) are exempt. Provider-agnostic — fixes the leak for both Claude and Grok interactive. Tests: idle-only reap (active/parked/closed excluded), activity bump keeps a session alive, threshold 0 disables. Gate green (ruff/mypy/xenon + prompter_live). * fix(panel): resolve agent names from the live roster so they never drift A review task assigned to the pr-reviewer rendered as a truncated raw UUID instead of its name. Root cause: the panel resolved assignees from a hardcoded static roster in agent-utils.ts that had drifted — it never gained the board-adjacent agents added backend-side (intake-1, secretary-1, pr-reviewer-1). Their UUIDs hit no map entry, so getAgentDisplayName fell through to the unknown-UUID branch and returned agentId.slice(0, 8). Every assignee surface (task table, task detail, subtasks, journals, communications, commit cards) shares that resolver, so all of them showed the fragment. Make the live /api/agents roster the source of truth instead of a static duplicate that silently rots: - agent-utils: add a runtime registry (registerAgentRoster) keyed by both UUID and slug; resolveToSlug / getAgentDisplayName / isKnownAgent consult it first. The static maps remain only as an offline / first-paint fallback (now complete with the three agents). - api/agents: surface the backend UUID on AgentDefinition (getAll/getOne previously dropped it), so the registry can key by UUID. - use-agents: add useAgentRosterSync (registers the live roster) and derive useAgents from live definitions, falling back to the static roster. - providers: mount the sync once inside QueryClientProvider. Now any agent the backend knows about resolves, including ones added after this change — the panel can no longer drift out of sync. Tests: agent-utils unit tests cover the three agents end-to-end, a live-roster-only agent (drift-proofing), live-overrides-static, and a regression guard for the existing roster. * fix(pr-review): post a COMMENT review when GitHub forbids self-review A pr-reviewer review of an org-authored PR never reached GitHub. The agent side ran correctly (claim → read-only diff → review → post_pr_review → completed + CEO notify), but the GitHub publish 422'd with "Can not request changes on your own pull request": the PR was authored by the same account that owns the project PAT. post_pr_review posts best-effort after the DB transition, so the failure was logged and swallowed — the task completed and the CEO was notified "reviewed" while the PR showed no review. GitHub forbids APPROVE / REQUEST_CHANGES on your own PR but DOES allow a plain COMMENT review. The org's internal PRs (and any PR the PAT owner opened) hit this. Retry once as a COMMENT review on the self-review 422 so the review actually lands; the verdict is already stated in the body. The external/fork-PR path (different author) is unchanged — REQUEST_CHANGES succeeds there and the fallback never fires. Tests: self-review 422 downgrades to COMMENT and returns the COMMENT result; a failing COMMENT retry still surfaces GitError with no infinite loop; the existing non-self 422 still raises. * fix(grok): harden cost-guard, pin runtime, refresh stale plugin comments Address review findings on the Grok provider work: - budget-feed plugin failed open unconditionally, so a one-shot task agent whose in-container SDK budget server went unreachable would run with the cost cap unenforced. The entrypoint now exports ROBOCO_BUDGET_ENFORCE=1 (one-shot agents always start that server) and the plugin's pre-exec gate fails CLOSED when the flag is set and the budget endpoint is unreachable, halting an uncapped burn. Interactive serve agents (intake/secretary) set no flag and keep failing open (they run no budget server by design). - Pin opencode-ai to the live-verified 1.17.8 (was an unpinned global npm install). Untrusted model output runs under it; bump the pin deliberately. - Document the ROBOCO_GROK_* operator vars in .env.example (image, the three opencode permissions, reasoning effort, idle-kill, cost ceiling). - Refresh stale plugin comments: the MCP tool-name shape and the secretary tool-registration path are confirmed live, and secret-scrub's load route is the auto-discovery dir (not a config plugin: array). Keep the honest not-yet-exercised caveat on secret-scrub's deny path and the reasoning variant — those remain genuinely unverified. * fix(grok): unbreak workspace-cwd agents, free trapped agents, stop self-PR review Three bugs surfaced by the first live Grok lifecycle run: - Dev/QA/doc agents crash-looped at startup with ModuleNotFoundError on roboco.llm.providers. The entrypoint ran the opencode-config render from the agent's workspace-clone cwd, whose own roboco/ dir shadows /app on the sys.path front; a branch without the grok code lacks the providers package. Render from /app so the installed package always resolves (the render has no cwd dependency — writes global, reads ROBOCO_MCP_CONFIG). - A budget/loop halt blocked EVERY tool, including i_am_idle, unclaim, and i_am_blocked, so a halted agent could neither continue nor stop and flailed — one billed model turn per blocked retry. The before-gate now always lets the release verbs through so a halted agent can exit cleanly. - The inbound reviewer ingested the org's OWN PRs (authored by the repo-owner account), which can't take a REQUEST_CHANGES review (GitHub 422) and get re-reviewed every poll. The normalizer flags author_is_owner and ingestion skips them — the reviewer reviews only PRs the org did not author. External/contributor PRs are unaffected. Tests: owner-authored PR flagged + skipped; normalize shape covers the new field. Gate green on the changed modules (ruff/mypy/xenon + 48 tests). * feat(grok-cli): render config.toml + map per-role grok CLI flags First piece of the Grok CLI provider that replaces the opencode runtime: a pure, unit-tested module the agent entrypoint runs to translate the mounted mcp-config.json into ~/.grok/config.toml ([mcp_servers]) and compute the per-role 'grok -p' flags — subagent/shell/edit tool removal, raw-git-mutation and rm-rf denies, reasoning effort — mirroring ClaudeCodeProvider's per-role permissions with native grok flags instead of an opencode permission block + JS guard plugins. Uses tomli_w. The rendered config + env injection are validated live against grok-build (the model called the server through it). * feat(grok-cli): grok CLI agent image + headless entrypoint The roboco-agent-grok image now installs xAI's official grok CLI (Grok Build, pinned 0.2.56) instead of opencode, authenticated by the SuperGrok subscription via a mounted ~/.grok/auth.json (parity with the Claude ~/.claude mount, no metered API key). The entrypoint renders ~/.grok/config.toml + per-role flags from /app (the ModuleNotFound-shadowing lesson), runs grok -p headless with --output-format json, keeps the prompt-injection guard, and exits 75 on a rate-limit so the orchestrator parks the provider. No in-container SDK server or budget-feed — native --max-turns + server-side terminal-substitute replace them. * feat(grok-cli): GrokCliProvider — subscription auth mount, mirrors ClaudeCodeProvider Replace the opencode GrokProvider with GrokCliProvider: reuses the orchestrator's shared mount/auth/git assembly (gateway + identity) exactly like the Claude path, mounts the host ~/.grok/auth.json read-only (SuperGrok subscription) instead of injecting an xAI key, and sets the slim env the grok-cli entrypoint + renderer read (ROBOCO_AGENT_ID for per-role flags, model, mcp-config, prompt). Provider routing fields are blanked before the shared step so the grok endpoint is never mislabelled ANTHROPIC_*. Per-role permission logic now lives in grok_cli_config, so the provider is slim. Registry/orchestrator/exports updated; provider tests rewritten for the CLI behavior (no XAI key, auth mount present/absent). * feat(grok-cli): capture per-session token usage + notional cost Grok runs on the SuperGrok subscription, but — exactly like Claude on Max — we still record per-agent tokens and a notional cost for the dashboard. The grok CLI writes a cumulative totalTokens per turn into ~/.grok/sessions/<cwd>/<session-id>/updates.jsonl (the grok analogue of the Claude transcript / old opencode.db); the max is the session total. This reader locates that file (url-encoded cwd), extracts the total, and prices it at the output rate (no input/output split from the CLI; conservative + matches the reasoning-at-output convention). Validated against a real grok-build session (18253 tokens -> $0.0365). Entrypoint + finalize wiring follows. * feat(grok-cli): wire usage capture into the run (session id + post-run extract) The provider pins a fixed session id (ROBOCO_AGENT_SESSION_ID, reused from the agent session id as on the Claude path); the entrypoint passes it to 'grok -p -s <id>' so the run's session store is locatable, then runs the usage reader post-run (best-effort) to write the captured tokens + cost. The orchestrator-side finalize that reads that file follows. * feat(grok-cli): read captured usage at finalize; keep interactive serve working The provider mounts the per-agent data dir and points the entrypoint's usage file at it; the orchestrator's grok finalize reads that usage.json first (the grok-CLI total, priced at the output rate) and falls back to opencode.db for the still-opencode interactive intake/secretary path. Re-add _reasoning_effort_for to grok.py as a clearly-temporary shim for that interactive path (it needs opencode's "minimal" variant, distinct from the CLI's --effort) until it is converted too. * feat(grok): convert interactive intake/secretary to the grok CLI; delete opencode Move the last Grok runtime off opencode onto xAI's official `grok` CLI, for full parity with the Claude path. The intake/secretary chat now runs per-turn headless `grok -p` invocations that resume one session id (proven live: context carries across runs), with streaming-json deltas mapped to the existing panel StreamChunk kinds — the IntakeDriver loop, message source, relay, and idle reaper are reused unchanged; only the SessionFactory differs (GrokCliSession replaces the opencode-serve session). - GrokCliSession + a pure, unit-tested streaming-json -> StreamChunk assembler (thought coalesced to one block, text streamed live, end captures the session id for -r, fenced-draft fallback, clear errors incl. rate-limit). - intake propose_draft and secretary read_company_state/read_task/submit_directive are now FastMCP servers (roboco-intake / roboco-secretary) wired into ~/.grok/config.toml, launched via `uv run --directory /app` to resolve the installed package. The secretary tools reuse the shared backend helpers. - Orchestrator: interactive spawn mounts the subscription auth + per-agent usage dir (no metered xAI key, no permission env — grok flags carry per-role perms); usage/cost now read a captured usage.json (drop the opencode.db reader, the _opencode_db_path/_grok_usage_from_opencode methods, and the cost-cap's opencode read). hosts["opencode"] -> hosts["grok_usage"]; OPENCODE_DATA_DIR -> GROK_USAGE_DATA_DIR. - Fix one-shot usage capture: `-s` does not pin the session id (grok generates its own), so the entrypoint now reads the real id back from the JSON run log and the reader uses it; usage is captured per-turn on the interactive path. - Delete the opencode layer: opencode_config/opencode_usage/opencode_session, the docker/grok/*.js plugins, the old one-shot entrypoint, and their tests. - Compose (all three files), .env.example, and stale comments updated to the grok-CLI runtime; add the SuperGrok auth mount + grok-usage dir. Gate green: ruff, mypy (296 files), xenon, tests. NAS build/verify pending. * fix(grok): deliver the role blueprint as grok's system prompt via ~/.grok/AGENTS.md The blueprint was mounted at /app/system-prompt.md but never reached grok — a real parity gap vs the Claude path (which passes --system-prompt-file). grok agents ran only on the per-task prompt, missing their RoboCo role/org context. Verified live on grok 0.2.56 that the obvious flags do NOT work headless: `--system-prompt-override` and `--rules` are silently ignored under `grok -p` (identical output with and without). What IS honoured is grok's instruction-file discovery — and `$HOME/.grok/AGENTS.md` is loaded GLOBALLY regardless of --cwd (a project AGENTS.md only loads from the cwd/project root, which would pollute the agent's git workspace). Proven end to end: a blueprint written there makes grok adopt the role ("I am the RoboCo intake interviewer ... -- intake-1"). write_agents_md() copies /app/system-prompt.md -> ~/.grok/AGENTS.md; the one-shot render (grok_cli_config.main) and both interactive mains call it. No git pollution (it lives in ~/.grok, not the workspace), and it covers repo-cwd and /app-cwd roles alike. Reverted the non-working --system-prompt-override wiring. * feat(grok): close the Claude-parity divergences (reasoning, subagents, web, bash-guard) Bring the grok CLI to parity with the Claude path on the four deliberate differences: - Reasoning: drop the per-role `--effort low` default — Claude sets no per-role thinking budget, so grok now uses the model default for every role. The fleet-wide ROBOCO_GROK_REASONING_EFFORT override stays as a cost lever. (This also un-caps intake-draft quality, the one that actually mattered.) - Subagents: the intake interviewer may now fan out to subagents (parity with the Claude intake's `Task` allowance); every other role still has `Agent` removed. - Web: `--disable-web-search` for every role — no agent gets direct web (Claude's tool set has none either); the roles that get web reach it through the gated roboco-search MCP, unaffected. - Bash command filtering: full parity, split by deny semantics. Verified live that a grok PreToolUse hook deny CANCELS the run, while native `--deny` denies GRACEFULLY (the agent gets a permission error and recovers). So: * git network/branch/history ops -> native `--deny` (operational reflex; the agent must recover, not drop the task). Expanded to the full bash-guard set. * credential-exfil / identity-forgery / internal-API / env-dump patterns -> the SAME bash-guard the Claude path runs, wired as a grok PreToolUse hook (ROBOCO_GUARD_SKIP_GIT=1 so it leaves git to `--deny`). A hard cancel is the right response there — no legitimate agent reads ~/.netrc or forges an X-Agent-ID. One tolerance line (accept grok's camelCase `toolInput`) makes the one tested script guard both runtimes; +5 grok cases (50/50 green). Also cleaned stale internal task-number / smoke labels out of bash-guard-hook.sh. * fix(grok): install grok CLI to ~/.grok/bin (its real default), not ~/.local/bin The image build failed at `chown ... /home/agent/.local: No such file or directory`. The grok installer's default is $HOME/.grok/bin — the binary lands at ~/.grok/bin/grok; ~/.local/bin/grok is only a convenience SYMLINK the installer creates on macOS but not in the Linux container. So the Dockerfile referenced a directory that never existed: - PATH pointed at ~/.local/bin -> `grok` would not be found at runtime even if the build had passed; - chown targeted ~/.local -> the build aborted. Point PATH + chown at ~/.grok/bin / ~/.grok. Also harden the install: download the script to a file (a `curl | bash` pipe swallows a curl failure as a silent no-op) and verify the binary installed and runs (`test -x` + `grok --version`), so a broken install fails the build loudly instead of producing a grok-less image. * fix(grok): address adversarial-review findings across the grok-CLI conversion A 7-dimension adversarial review (find -> independently refute) surfaced 14 real issues; fixed each: Runtime bugs - GrokCliSession.send drained stdout fully BEFORE stderr — a >64KB stderr burst would deadlock the turn forever (spinner never clears). Drain stderr concurrently, and add a per-turn watchdog (ROBOCO_GROK_TURN_TIMEOUT_SECONDS, default 600s) that kills a wedged process and emits error+turn_end. - Crash-restarted grok agents launched `grok -p ""` (empty prompt) — Claude gets a scan-for-work fallback. Default the prompt in _spawn_container so every dedicated provider gets it too. - _grok_usage_json read /data/grok-usage unconditionally while its writers branch compose-vs-local, so a local-mode agent finalized at $0 and the cost-cap was inert. Single-source the path in a new _grok_usage_dir helper (read == write). - GrokCliSession secretary role fell through to "unknown" (get_agent_role returns a truthy sentinel, never None), defeating the ROBOCO_AGENT_ROLE fallback. Parity / hardening - --deny set was missing `git tag -d` / `git reflog delete` that the Claude bash-guard blocks — added them (the "same set" claim is now true). - Interactive mains now install the bash-guard hook too (defense-in-depth). - Compose: collapse the GROK_AUTH_DIR / ROBOCO_HOST_GROK_DIR auth-mount pair into one canonical var so a partial override can't silently break agent auth. Docs / comments - Panel routing card + architecture security doc no longer say Grok runs on the deleted opencode runtime; orchestrator comments point at the renamed entrypoint. Tests - Cover the interactive _render_grok_config MCP wiring (ModuleNotFound guard + secretary HMAC env), the cost-cap kill-failure + interactive relay-close paths, the local-mode usage read, the role fallback, the turn timeout, and the new git denies. (#13 — a separate grok "Write" tool — investigated: grok's only built-in file-mutation tool is search_replace, already removed; no gap.) Gate green: ruff, mypy, xenon, tests. * fix(grok): declare tomli-w as a runtime dependency (agent image needs it) The grok agent image failed at spawn with `ModuleNotFoundError: No module named 'tomli_w'` when rendering ~/.grok/config.toml. tomli_w was only a transitive dep of a dev-extra package, so it was present in dev/orchestrator envs but excluded from the agent image, which builds its venv with `uv sync --frozen --no-dev`. grok_cli_config imports it at module load to serialize the MCP gateway config, so without it a Grok agent gets no gateway verbs. Promote tomli-w to a direct [project.dependencies] entry. Locked with `--upgrade-package tomli-w` so only tomli-w is added — no incidental churn of the 8 unrelated packages a full re-resolve would have bumped. * Updated uv.lock * fix(grok): auto-approve tool execution (--always-approve) so headless agents can call tools Live smoke caught every grok agent (Main PM, pr-reviewer, dev, …) ending its run with stopReason=Cancelled and empty output the instant it reached for a tool. Root cause: headless `grok -p` cannot approve a tool call without `--always-approve` (grok's docs: required for unattended automation), and the per-role args didn't pass it — so no agent could call a gateway verb, an edit, or an MCP tool, and the run was cancelled. Add `--always-approve` to grok_cli_args_for_role (one place → every role, one-shot and interactive). Safety is unaffected: `--disallowed-tools` still removes tools and `--deny` still hard-blocks command patterns regardless of approval (a denied command returns a permission error and the agent recovers — verified live). Proven in the rebuilt image side-by-side: without the flag a tool call yields Cancelled/not-called; with the real rendered args it returns EndTurn and the MCP tool actually runs. (My earlier in-image tool-calling check passed `--always-approve` manually, which masked that the production args omitted it — fixed.) * fix(pr-review): seed claim heartbeat so the grok reviewer isn't wedge-killed pr_review_claim transitioned a review task pending -> in_progress but never seeded last_heartbeat_at, unlike every sibling claim path (_finalize_claim, qa_claim, doc claim). The reaper treats a NULL heartbeat as a stale claim, and the GROK idle-kill watchdog bypasses the live-container skip on a NULL heartbeat -- so the reviewer container was killed (Cancelled) before it could post_pr_review, churning the task back to pending on a respawn loop. A Claude reviewer was shielded by the live-instance skip; only GROK manifested it. Seed the heartbeat at claim time, matching the established invariant. Verified against a real Postgres (10/10 test_pr_review_db tests, incl. the new last_heartbeat_at assertion). * fix(grok): stream one-shot output live + capture real token usage Two gaps the buffered run hid, both verified in the real image with mounted SuperGrok auth: - Observability: the entrypoint buffered grok's output to a temp file and only cat it after the run, so `docker logs` was blank while the agent worked. Switch the one-shot to --output-format streaming-json piped through tee: grok flushes each thought/text event incrementally (confirmed token-by-token live in-container), so the agent's reasoning shows in docker logs in real time, parity with the Claude stream-json path. Read the session id back from the NDJSON run log (the terminal `end` event) since -s does not pin it. - Usage: total_tokens read 0 for every grok run. grok nests the cumulative totalTokens on params.update._meta, but the reader looked at params._meta (which only holds event ids); the unit fixture had the same wrong shape, so the tests masked it. Read the real path (with params._meta / top-level fallbacks) and fix the fixture to the real grok shape. Verified live: usage.json now reports total_tokens=3262, cost_usd=0.006524 (was 0). * fix(grok): validate agent_id before using it as a usage-dir path segment CodeQL flagged a high-severity py/path-injection: agent_id flowed from request-facing call sites into _grok_usage_dir() and on to read_text(), so a value containing '..' or a separator could traverse the filesystem. Validate agent_id against the slug/uuid allowlist ([A-Za-z0-9_-]+) at the single chokepoint (_grok_usage_dir feeds both the mount and the finalize read); anything else raises. Rejects traversal; accepts every real agent slug. * fix(grok): use explicit-guard path sanitizer CodeQL recognizes as a barrier The re.fullmatch allowlist from a35b640d was secure but CodeQL's py/path-injection dataflow did not model the regex call as a barrier, so the high-severity alert persisted on the analyzed merge. Switch _safe_agent_path_segment to explicit guards (empty / '.' / '..' / '/' / '\\' / NUL) -- the barrier form the query recognizes -- which still rejects every traversal vector. Drop the now-unused re import. * fix(api): validate agent_id at the orchestrator route boundary (path-injection) CodeQL traces the py/path-injection from the request agent_id path param on the orchestrator routes (stop/spawn/resolve-wait/mark-waiting/status) down to the grok usage-dir read. Validate agent_id at the HTTP boundary with explicit traversal guards (empty / '.' / '..' / '/' / '\\' / NUL) returning 422, so the sanitized value is what flows downstream and the query sees a barrier at the source. Runtime _grok_usage_dir keeps its guard as defense in depth for non-HTTP callers. * fix(grok): sanitize usage-dir agent_id with Path(...).name (CodeQL barrier) Proven against the analyzed merge: CodeQL does not propagate a control-flow guard through a helper's return value, so neither the route validator nor the _safe_agent_path_segment guard cleared the py/path-injection alert. Reduce the validated id to its final path component with Path(...).name -- a data-flow sanitizer CodeQL models and propagates through the return -- at the single source of truth (_grok_usage_dir), covering both the finalize read and the mount/mkdir. The guard stays for fail-loud reject semantics; .name is the recognized barrier (identity for a valid slug). * fix(grok): containment-check the usage read against a fixed root (path-injection) Three sanitizers failed to clear the CodeQL alert because the query does not model them here: a regex guard, an explicit guard, and Path(...).name (verified each against the analyzed merge). Replace with the barrier CodeQL does recognize -- and that is also a genuine control -- at the read sink: resolve the usage.json path and refuse it unless it is_relative_to the resolved usage root (a fixed, untainted base from config, extracted as _grok_usage_root). Note the github-advanced-security autofix proposed 'if usage_json.parent != usage_dir: return None', which is a no-op -- appending the constant 'usage.json' never changes the parent, and it compares against the tainted dir, not a safe base. This compares against the fixed root instead. The _safe_agent_path_segment guard stays (fail-loud reject upstream, covers the mount/write side). * fix(grok): sanitize the usage read with os.path.basename (CodeQL-modeled barrier) Four prior barriers did not clear the py/path-injection alert (verified each against the analyzed merge): a regex guard, an explicit guard, Path(...).name, and an is_relative_to containment check. The one sanitizer CodeQL's query documents -- os.path.basename -- was never actually tried: it was swapped for Path(...).name to dodge ruff PTH119, and that pathlib form is not modeled. Apply os.path.basename to the agent id in _grok_usage_json's own scope (the read sink), so there is no recognition or interprocedural-propagation ambiguity, and allow PTH119 for this file with a documented reason. The _safe_agent_path_segment guard and the route 422 stay as the actual reject controls. --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com>	2026-06-19 09:15:01 +02:00