mirror of
https://github.com/rennf93/roboco.git
synced 2026-08-03 07:23:24 +02:00
18eaacb9ea6ac6f8233701148727833c0706c55d
88
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 RoboCo Release Manager
|
18eaacb9ea | chore(release): 0.27.0 | ||
|
|
10f039c36f |
feat(eval): golden-task eval harness + doctrine cohort stamp (#655)
* fix(notifications): exponential backoff + CAS claim for expired-unacked re-escalation The sweep re-escalated every expired unacked ack-required notification on every ~60s tick, forever — the live incident: 3 fresh blocker escalations + Telegram DMs per minute from a static stale pile. Now each notification carries reescalation_count / last_reescalated_at / reescalation_delivered_count (migration 079): first fire at expiry, then doubling intervals from 1h capped at 24h, hard stop after ROBOCO_NOTIFICATION_MAX_REESCALATIONS (default 5) with one permanent log carrying attempts-vs-delivered so 'seen and ignored' is distinguishable from 'route never worked'. The due/wait/capped decision is a pure function in foundation/policy/communications.py. Per adversarial review, the attempt slot is claimed by compare-and-set (UPDATE ... WHERE reescalation_count = :n) BEFORE delivery — the previous draft leaned on the 60s dedup window, which never engages for BLOCKER_ESCALATION (_LOOP_PRONE_TYPES excludes it), so concurrent sweeps would have double-delivered. A lost claim skips delivery outright. Legacy rows read as count=0 and keep today's first-fire semantics. 61 tests incl. a two-session CAS race and a real alembic upgrade/downgrade round trip. * feat(budgets): per-task and per-project cost budgets (flag-gated) tasks.budget_usd + projects.monthly_budget_usd (migration 080, chained on 079; adds ix_agent_spawn_sessions_task_id since both enforcement seams filter on bare task_id). Behind ROBOCO_TASK_BUDGETS_ENABLED (default off, feature-flags card) — verifiably inert when off. Claim-time: a project-month-spend guard applies to WORK-STARTING claims only (i_will_work_on / i_will_plan) — per adversarial review, review/ doc/gate/inbound-PR claims are exempt so in-flight work can always finish reviewing and merging at cap. Spend counts closed sessions' estimated_cost_usd PLUS open sessions priced live from token snapshots (the original closed-only sum read parallel long sessions as $0). Sweep-side: the existing budget sweep also prices the active task's spend vs budget_usd (TaskType defaults when null); on breach the task is BLOCKED (HUMAN resolver, budget marker) BEFORE the graceful stop so the unclaim no-ops and the dispatcher never respawns onto it, and the CEO notification names both recovery steps. unblock on a budget-blocked task re-checks live spend and refuses while still over — no silent re-breach loop. Panel: budget inputs in both dialogs (0 rejected — a zero budget silently blocks everything), spend logic consolidated in TaskService.task_spend_usd. 42 new tests incl. a real-DB spend-query suite and a two-tick non-refire sweep test. * feat(eval): golden-task eval harness + doctrine cohort stamp roboco/eval: 6 BenchTaskSpec fixtures run through the real lifecycle in a disposable environment (the e2e_smoke harness's fake GitHub + local git origin + throwaway DB catalog — real isolation, not convention), scored deterministically (terminal status, revision_count, cycle time, tokens/cost via the agent_spawn_sessions task_id join) plus a local- model judge whose output is nested under a non_deterministic-marked object so cohort diffs don't read judge noise as regression. CLI: python -m roboco.eval run --role <slug> --cohort <name>. Source- checkout-only by declared posture (deptry-scoped ignore + a hard ImportError guard naming why; tests/ never ships in images or wheels). agent_spawn_sessions.doctrine_version (migration 081, chained on 080) is stamped at spawn-session finalize from the composed prompt layers — with the session's model column it identifies a cohort durably. Per adversarial review: bench runs patch the vault flags off (they were writing real markdown into the operator's vault), and the real-spawn OrchestratorStageSpawner is deliberately cut to NotImplementedError — spawned containers' MCP wiring resolves to the production orchestrator under real agent UUIDs, so real spawns wait for a dedicated follow-up; the injectable scripted spawner is the working path. Full suite 13852 passed / 94% coverage in the source worktree; deptry/mypy/xenon clean. --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com> |
||
|
RoboCo Release Manager
|
31418c9a32 | chore(release): 0.26.0 | ||
|
|
5f32d8760a |
feat(forge): Phase 4 — GitLab + Gitea repo-provisioning parity (#581)
GitLab's create_org_repo (services/forge/gitlab.py) replaces the Phase-3
synthetic 501 with a real implementation: resolves org (a group's full
path, subgroups included) to a numeric namespace id via GET
/groups/{path}, falling back to the token's own namespace on a 404
(personal-namespace projects); POSTs /projects with the
name/path/description/visibility/initialize_with_readme payload
(visibility mapped private->"private"/"internal"); reshapes the 201
onto the GitHub fields callers read (full_name/clone_url/html_url) and
GitLab's duplicate-path 400 "has already been taken" onto GitHub's 422
shape, text preserved. Gitea's create_org_repo was already real but
untested — added transport-level coverage.
GitHubProvisioningService (services/github_provisioning.py) is now
provider-aware: ROBOCO_PROVISIONING_PROVIDER (github default / gitlab /
gitea) and ROBOCO_PROVISIONING_HOST (self-hosted instance, required for
gitlab/gitea or the service stays disabled exactly like a missing
token/org) pick the target forge; the class/factory names stay
GitHub-flavored for backward compatibility (pitch.py and existing
imports untouched). A shared _is_already_exists() helper recognizes
GitHub's "already exists" (422), Gitea's (409/422 "already exists"),
and GitLab's reshaped "has already been taken" (422). The existing-repo
re-fetch now builds a provider-aware RepoRef (GitLab packs org/name
into the owner field; GitHub/Gitea keep the owner,repo pair). Default
behavior (no new env set) is byte-for-byte the Phase-1 GitHub path,
pinned by a regression test.
Gates: ruff format/check clean, mypy roboco/+tests/ clean (1235 files),
xenon A/A/B clean, targeted suite (forge + provisioning + pitch) 79/79
green.
Co-authored-by: Renn F <rennf93@users.noreply.github.com>
|
||
|
|
461a6e1ae7 |
feat(forge): Phase 1 — GitProvider seam, GitHub transport extracted (#571)
* feat(forge): Phase 0 — git_provider column + registration-time forge validation Pointing a project at a GitLab/Gitea git_url used to fail silently, several steps deep, at first PR. New pure policy module (foundation/policy/forge.py) detects the provider from the git_url host and validates at the ProjectService create/update chokepoint: github auto-detects and auto-stamps, explicit git_provider=github is the GitHub Enterprise escape hatch, gitlab/gitea are recognized-but-not-yet-supported, unknown hosts get a loud rejection with guidance. An update changing git_url does NOT inherit a stored auto-stamped provider (restating the override is required), so a host swap can't smuggle the escape hatch past validation. Migration 075 adds the nullable projects.git_provider column; the panel project dialogs show the detected forge. Phase 0 of the forge-providers spec. * feat(forge): Phase 1 — GitProvider seam, GitHub transport extracted roboco/services/forge/: base.py holds the pure contracts (RepoRef + GitProvider ABC, stdlib-only — a later GitLabProvider is implemented by reading this file alone), github.py the httpx transport (20 endpoint methods behind one shared request-plumbing helper set), registry.py the wiring (git_provider column -> provider, failing loud on gitlab/gitea). GitService keeps its exact public surface and all response classification; its 26 inline REST call sites route through a lazy _forge property (several suites build GitService via __new__, so an __init__-set attribute breaks them). github_provisioning and release_executor ride the same provider. Zero behavior change — the pre-existing suites pass unmodified; per-project provider resolution lands with the second provider. --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com> |
||
|
RoboCo Release Manager
|
3e8b3d82e6 | chore(release): 0.25.0 | ||
|
|
bb3b4b0c6d |
W6: Telegram notifications bridge (V1) (#524)
* feat(gateway): reviewer/PM collision map (W5)
The collision surface (intends_to_touch / adds_migration / touches_shared)
is authored at delegate time, consumed once by SequencingService to wire
dependency edges, then never shown to a reviewer again. This surfaces it:
- Pure builder (services/gateway/choreographer/collision.py): for a task
under review, the surfaced siblings (same parent) that would collide —
file-overlap globs or a shared migration chain (both adds_migration) —
with the overlapping globs and a declared-vs-actual drift check. No
DB/IO; callers fetch siblings (one indexed get_subtasks query, mig 069)
+ actual files (git). Caps: 10 siblings, 5 globs.
- Evidence envelopes: collision_context block injected into QA
claim_review, PR-gate claim_gate_review (both carry real touched files
so drift is populated), and the PM i_will_plan briefing (no actual
files at plan time, drift omitted). Best-effort — a failure omits the
block, never breaks the verb/briefing. Empty block omitted (zero token
cost via _EVIDENCE_OMIT_WHEN_EMPTY).
- Panel: GET /api/tasks/{id}/collision-map (declared surface + sibling
overlap; no drift — the panel route resolves no workspace) + a Collision
tab on the task detail (8th tab). Mock-mode returns an empty map.
- docs/map added to the RAG auto-index dirs so the collision-map concept
is fleet-retrievable; skipped gracefully if the dir is absent.
19 new tests (15 unit on the pure builder + 4 integration on the route).
Gate green: ruff/mypy/xenon (module rank A)/pytest 13000/coverage 94.81%,
panel typecheck/lint/516 tests.
* [w6-telegram] Add Telegram notifications bridge (V1)
CEO-facing Telegram DM bridge, flag-gated off by default
(ROBOCO_TELEGRAM_ENABLED). Mirrors the X-credentials / X-client pattern:
- TelegramCredentialsTable (migration 073) — singleton Fernet-encrypted
bot_token + chat_id, all-or-nothing set/clear; API never returns plaintext.
- TelegramClient ABC / NullTelegramClient (no-op, configured->False, never
raises) / LiveTelegramClient (httpx POST sendMessage) / build_telegram_client
factory (Null when creds unset).
- /telegram/credentials CEO-only routes (write-only, guard-decorated).
- Best-effort _notify_telegram fan-out from the two CEO-notify producers
(notify_ceo_of_escalation, notify_ceo_of_completion) — guarded by the flag,
never raises into the producer, carries a panel deep-link when
panel_base_url is set.
- panel credentials card (2 fields) nested in the Telegram feature-flag row.
- panel_base_url + telegram_timeout_seconds config fields.
V1 scope only: credentials + flag + panel card + client + one-line fan-out.
Out of scope (V2): inbound commands, a TelegramEngine background loop, a
dedup ledger, a bus subscription.
* [w6-telegram] fix: slave mypy/xenon regression (product tests + helper extract)
Pre-existing on slave from prior session's merges — no PR's CI caught them
(squash merges don't re-CI the result; each branch was based on older slave).
- test_product: _product helper returned MagicMock -> list invariant error;
cast to ProductTable, move import under TYPE_CHECKING.
- test_usage: svc.session.execute (AsyncSession) has no call_args_list;
cast to MagicMock at the two call sites.
- product.progress_for_products: xenon rank C -> extract module-level
_project_to_products_map helper (repo pattern: helper-extract).
---------
Co-authored-by: Renn F <rennf93@users.noreply.github.com>
|
||
|
Renn F
|
3338f88e1b | chore(release): 0.24.0 | ||
|
|
1114ee5ea0 |
[77719d3f] A2A team telemetry: coordination event notifications for 5 event types (#477)
* [13d03d5c] Add 5 coordination-event notification producers + wire at chokepoints (#472) (#474) * [13d03d5c] Add 5 coordination-event notification producer methods * [13d03d5c] Wire reassignment/collision/unblock/dependency-revival notifications * [13d03d5c] Wire stale-claim-reaped notification into orchestrator reaper * [13d03d5c] fix(runtime): guard reaper's UUID annotation + defensive attr access The stale-claim-reaped notification hook added a runtime-unquoted `UUID` type annotation (only imported under TYPE_CHECKING, so the module raised NameError on import) and a direct `t.assigned_to` attribute access that crashes against the minimal test doubles the existing reaper test suite uses. Quote the annotation and switch to getattr-defensive access, matching `_assignee_is_provider_parked`'s existing convention in the same file. * [13d03d5c] test(notification): unit coverage for 5 coordination-event producers One test per new send_* method (reassignment, collision-sequencing, unblock, dependency-revival, stale-claim-reaped) following the existing _FakeDb/_patch_db_context pattern, asserting subject/body/ related_task_id/priority/recipient-count, plus a no-recipients no-op case for reassignment. * [13d03d5c] test(task): prove reassign + unblock don't double-fire notifications Two chokepoint-level tests mocking NotificationService at its defining module: a repeated reassign() to the same already-current target skips the notification (guarded by comparing against the pre-mutation assignee), and a repeated unblock() on the same task only notifies once since the second call short-circuits on the status!=BLOCKED guard. * [13d03d5c] style(task): ruff format the collision-sequencing wiring block No behavior change — reflows the newly-added _notify_collision_sequencing call site to satisfy ruff format's line-length rules. * [13d03d5c] docs(backend): add coordination-event notification producers guide Documented the 5 new NotificationService producers (reassignment, collision-sequencing, unblock, dependency-revival, stale-claim-reaped) with fire conditions, double-fire prevention mechanisms, and implementation patterns. Updated backend README to link the new services guide for developers integrating new coordination events. --------- Co-authored-by: Backend Developer 1 <be-dev-1@roboco.tech> Co-authored-by: Backend Documenter <be-doc@roboco.tech> * [3ee8150b] Frontend: render coordination-event notifications + e2e smoke coverage (#475) * [69777c3a] test(e2e-smoke): add coverage for soft-block + unblock coordination notifications (#471) Co-authored-by: Frontend Developer 2 <fe-dev-2@roboco.tech> * [8eb82639] Render 5 coordination-event notification types with task deep-links (#470) * [8eb82639] feat(notifications): add APPROVAL type icon and deep-link component test Add missing APPROVAL member to the frontend NotificationType enum to match backend roboco/models/base.py, wire its icon into the existing typeIcons Record in the notifications page, and add a component test covering type rendering and the task deep-link. * [8eb82639] docs(notifications): document 5 coordination-event types and APPROVAL enum addition Added comprehensive reference guide explaining the 5 notification types (TASK_ASSIGNMENT, BLOCKER_ESCALATION, REVIEW_REQUEST, DOCUMENTATION_REQUEST, APPROVAL), their visual identities (icon + color), use cases, and deep-linking behavior to related tasks. Updated panel README with quick reference table. TypeScript Record pattern ensures exhaustive type coverage at build time. --------- Co-authored-by: Frontend Developer 1 <fe-dev-1@roboco.tech> Co-authored-by: Frontend Documenter <fe-doc@roboco.tech> --------- Co-authored-by: Frontend Developer 2 <fe-dev-2@roboco.tech> Co-authored-by: Frontend Developer 1 <fe-dev-1@roboco.tech> Co-authored-by: Frontend Documenter <fe-doc@roboco.tech> * [a27de2a8] fix(docs): reflow hard-wrapped notification-types.md to pass markdown gate (#479) (#481) The Python quality gate on assembled PR #477 was red because the newly added docs/frontend/components/notification-types.md (introduced by the frontend coordination-event rendering commit) had manually wrapped prose paragraphs, which scripts/reflow_md.py --check rejects as part of make quality. Reflowed the file with scripts/reflow_md.py --apply (whitespace only, no content change) so the check passes. ruff format/check, mypy, xenon, vulture, bandit, and the full pytest suite (10284 passed) all confirmed green on this commit; notification.py, task.py, and orchestrator.py are untouched. Co-authored-by: Backend Developer 1 <be-dev-1@roboco.tech> * [705419d5] Remove duplicate unblock notification and fix its dependent tests (#485) (#488) * [705419d5] fix(notifications): remove duplicate unblock notification, fix its tests The /unblock route was still calling delivery.notify_assignee_of_unblock() (TASK_ASSIGNMENT) after TaskService.unblock() already sent the send_unblock_notification() ALERT wired in by an earlier task — a real duplicate notification on every unblock. Delete the route-layer call and the now-dead NotificationDeliveryService.notify_assignee_of_unblock method, fix the integration test that mocked it, and fix/extend the e2e notification-coordination-events test to assert the persisted ALERT rows (exact subjects) for both the direct-unblock and dependency-revival producers instead of the old TASK_ASSIGNMENT assertion. * [705419d5] docs(backend): update coordination-events doc for unblock duplicate removal --------- Co-authored-by: Backend Developer 1 <be-dev-1@roboco.tech> Co-authored-by: Backend Documenter <be-doc@roboco.tech> * [6c142a73] docs(changelog): document restored coordination-event notification producers and add collision-sequencing double-fire test (#489) (#490) Co-authored-by: Backend Developer 2 <be-dev-2@roboco.tech> * [77719d3f] Seed system agent in e2e harness to fix unblock/dependency-revival notifications The e2e harness's seed_company omitted the system sentinel agent that production seeds via initial_data.py. The unblock and dependency-revival notification producers default to from_agent="system", which _resolve_agent_uuid looks up by slug in the DB. With no system row the resolver returns None and _create_notification silently skips the notification, so the two ALERT assertions got 0 rows instead of 1. The soft-block test passed because it uses NotificationDeliveryService which creates the notification directly with a real agent UUID as from_agent, bypassing the slug resolution path entirely. * [77719d3f] Use foundation UUID for system agent to avoid slug collision The first attempt seeded the system agent with a random UUID. Other tests (_seed_system_and_secretary, _seed_video_agents) check by the fixed foundation UUID via session.get(AgentTable, uuid); not finding it they INSERT their own system row, hitting ix_agents_slug. Using the foundation UUID makes their check find the seed_company row and skip. * [77719d3f] Fix dependency-revival notification event loop mismatch The dependency-revival test calls _unblock_dependents directly via stack.run_db, which creates a new asyncio event loop. Inside, _notify_dependency_revival -> NotificationService._create_notification opened its own session via get_db_context(), which reuses the singleton _DbHolder engine — bound to the FastAPI server's event loop. The asyncpg connection raised 'Future attached to a different loop' and the exception was silently caught + logged as a warning, so the notification never persisted and the test saw 0 rows. Fix: add an optional db_session parameter to _create_notification and the two send methods. When provided, use the caller's session directly and skip the internal commit (the caller owns the transaction). The TaskService's _notify_unblock and _notify_dependency_revival now pass self.session, keeping the notification in the same event loop + session as the task transition. * [77719d3f] Scope system-agent seeding to notification tests only Seeding the system sentinel in seed_company (commits 3bba7b32/617b7890) fixed the 0-notification bug but caused 3 i_documented gateway_timeout failures: every e2e test now paid notification-creation latency for system-origin notifications that were previously silently skipped, pushing the already-slow i_documented verb past its 120s timeout. Move system-agent seeding out of seed_company and into a scoped _seed_system_agent helper called only by the two coordination-event tests that exercise send_unblock_notification / send_dependency_revival_notification (both resolve from_agent='system' via DB lookup). dev_lifecycle and state_machine tests revert to the pre-fix behavior (system-origin notifications silently skipped, no extra latency). The event-loop fix (commit |
||
|
Renn F
|
179467943c | chore(release): 0.23.0 | ||
|
|
7f138d3bf5 |
[e4ed92d6] Video pipeline per-project requests, re-render action, composition preview (#403)
* [7f2c881a] Project-scope video pipeline + re-render + preview proxy (#386) (#396) * [7f2c881a] feat(video): scope on-demand video requests + render loop to project_id Require project_id on VideoRequestBody (404 when unresolvable or not opted into the video engine), thread it through VideoEngine.open_video_task via a shared resolve_authoring_project helper, and resolve the render loop's motion/ workspace from the authoring task's own project_id instead of the hardcoded self_heal_project_slug. * [7f2c881a] fix(video): cast task.id to UUID before VideoEngine.rerender calls mypy flagged task.id as sqlalchemy.sql.sqltypes.UUID[Any] rather than uuid.UUID in the three rerender tests; cast to UUID per the codebase's established idiom (cast("UUID", obj.id)) used elsewhere for the same SQLAlchemy Mapped-attribute inference gap. * [7f2c881a] docs(video): API endpoints for project-scoped requests, re-render, and preview proxy Add comprehensive API documentation for the new project-scoped video engine endpoints: - POST /api/video/request: on-demand video authoring scoped to project_id (breaking change) - POST /api/video/pipeline/{task_id}/rerender: CEO-triggered re-render with idempotency key clearing - GET /api/video/preview/{task_id}/{file_path}: CEO preview proxy with path-traversal confinement Document project-scoping architecture: authoring tasks and render loop now resolve from task's own project_id instead of hardcoded self_heal_project_slug. Add migration guide covering breaking change to VideoRequestBody schema (project_id now required), error handling changes (404 on unresolvable/non-opted-in projects), and client migration steps. --------- Co-authored-by: Backend Developer 1 <be-dev-1@roboco.tech> Co-authored-by: Backend Documenter <be-doc@roboco.tech> * [8f959c3b] docs(ux_ui): add project picker, re-render control, and composition preview panel spec (#381) (#398) Co-authored-by: UX/UI Developer 1 <ux-dev-1@roboco.tech> * [1fb5b5cb] Project picker, re-render button, and composition preview panel (#397) (#402) * [1fb5b5cb] feat(video): project picker, re-render button, and composition preview panel * [1fb5b5cb] docs(video): add comprehensive guide for project picker, re-render button, and composition preview panel --------- Co-authored-by: Frontend Developer 1 <fe-dev-1@roboco.tech> Co-authored-by: Frontend Documenter <fe-doc@roboco.tech> * [a512f364] Add video_engine_enabled to ProjectSummaryResponse (#412) (#414) * [a512f364] feat(api): surface video_engine_enabled on ProjectSummaryResponse * [a512f364] docs(api): document video_engine_enabled on ProjectSummaryResponse --------- Co-authored-by: Backend Developer 1 <be-dev-1@roboco.tech> Co-authored-by: Backend Documenter <be-doc@roboco.tech> * [03607ab9] Fix re-render control gating/placement and project picker filter (#434) * [f2f3e89f] Fix RerenderControl gating/placement across queue and strip views (#431) * [f2f3e89f] feat(video): widen RerenderControl gating and share it across queue/strip views Extracts RerenderControl into a shared panel/src/components/dashboard/ video-rerender-control.tsx component, widens its gate from render_status === 'failed' to source_task_id + composition_id present (matching what the backend rerender endpoint actually requires), adds a confirm dialog before firing the mutation, and wires the same component into video-pipeline-strip.tsx for still-in-flight rendering/render_failed rows. * [f2f3e89f] docs(video): enhance RerenderControl JSDoc with gating logic and usage examples Add comprehensive JSDoc to the RerenderControl component covering its purpose, gating logic (render for any source_task_id + composition_id, regardless of render_status), three visual button states (idle/loading/ error), confirm-dialog guard behavior, and usage examples for both video-post-queue.tsx and video-pipeline-strip.tsx contexts. Explains why the backend's rerender endpoint doesn't require a failed render and how the component prevents accidental re-renders. --------- Co-authored-by: Frontend Developer 1 <fe-dev-1@roboco.tech> Co-authored-by: Frontend Documenter <fe-doc@roboco.tech> * [404d8ed3] Filter project picker to video-engine-enabled projects (#432) * [404d8ed3] feat(panel): filter video-request project picker to opted-in projects Add video_engine_enabled to the client ProjectSummary type, give ProjectSelector a videoEngineOnly filter prop, default RequestVideoDialog's picker to the current video-enabled project with a friendly empty-state when none exist, and cover the filter with a new project-selector test. * [404d8ed3] docs(panel): add ProjectSelector component API reference with videoEngineOnly filter Document the reusable ProjectSelector component with its props, filtering behavior, and new videoEngineOnly filter for video-engine-enabled projects. Follows the existing component documentation pattern from page-refresh-provider. --------- Co-authored-by: Frontend Developer 2 <fe-dev-2@roboco.tech> Co-authored-by: Frontend Documenter <fe-doc@roboco.tech> * [519a4088] fix(panel): import missing RerenderControl in video-post-queue and correct stale doc (#436) Co-authored-by: Frontend Developer 1 <fe-dev-1@roboco.tech> --------- Co-authored-by: Frontend Developer 1 <fe-dev-1@roboco.tech> Co-authored-by: Frontend Documenter <fe-doc@roboco.tech> Co-authored-by: Frontend Developer 2 <fe-dev-2@roboco.tech> * [8e912c3e] Reflow hard-wrapped video UX design doc to pass quality gate (#439) * [ccfe2015] docs(ux_ui): reflow video request composition-controls spec to one line per paragraph (#438) Co-authored-by: UX/UI Developer 1 <ux-dev-1@roboco.tech> * [99c3ed9c] docs(backend): reflow hard-wrapped prose in video-engine-endpoints.md and video-project-scoping.md (#442) Co-authored-by: UX/UI Developer 1 <ux-dev-1@roboco.tech> * [c2e98fc0] docs(backend): strip stray trailing whitespace in video-engine-endpoints.md fence (#451) Co-authored-by: UX/UI Developer 1 <ux-dev-1@roboco.tech> * [9c7bc11a] Reflow all 3 hard-wrapped docs on this branch and verify quality gate (#457) * [9c7bc11a] test(scripts): guard reflow_md.py --check wiring into make quality * [9c7bc11a] docs(standards): document markdown reflow quality gate workflow and verification Added comprehensive guide explaining the one-logical-unit-per-line markdown prose standard, how the reflow check integrates into make quality, the three reflowed files (video-engine-endpoints.md, video-project-scoping.md, composition-controls spec), and the regression test added to ensure wiring stability. This task verifies all three ACs are satisfied: reflow_md.py --check exits 0, make quality passes (non-DB portions), and the three files are whitespace-only reflowed. --------- Co-authored-by: UX/UI Developer 1 <ux-dev-1@roboco.tech> Co-authored-by: UX/UI Documenter <ux-doc@roboco.tech> --------- Co-authored-by: UX/UI Developer 1 <ux-dev-1@roboco.tech> Co-authored-by: UX/UI Documenter <ux-doc@roboco.tech> * [002f0cdd] docs(rag): document reflow-check zero-diff troubleshooting path (#459) (#460) Co-authored-by: Backend Developer 1 <be-dev-1@roboco.tech> * [e4ed92d6] fix rerender missing-task test — assert the empty queue it creates The test never seeds; the trailing assertion expected a phantom video post. Broken since the branch's first commit but never executed — every earlier CI run short-circuited at a pre-pytest gate step. --------- Co-authored-by: Backend Developer 1 <be-dev-1@roboco.tech> Co-authored-by: Backend Documenter <be-doc@roboco.tech> Co-authored-by: UX/UI Developer 1 <ux-dev-1@roboco.tech> Co-authored-by: Frontend Developer 1 <fe-dev-1@roboco.tech> Co-authored-by: Frontend Documenter <fe-doc@roboco.tech> Co-authored-by: Frontend Developer 2 <fe-dev-2@roboco.tech> Co-authored-by: UX/UI Documenter <ux-doc@roboco.tech> Co-authored-by: Renn F <rennf93@users.noreply.github.com> |
||
|
|
15a3e87a2f |
feat(vault): Obsidian vault V1 — projection core, Auditor narrative, input loop (#458)
* fix(gateway): gate review diffs against the task's real parent branch (#444) The in-path PR-review gate's evidence diff (claim_gate_review) and the pr_pass conventions guard derived their diff base via parent_branch_for string surgery, which reuses the child branch's own team segment — wrong for every cross-team hop (a frontend child of a main_pm root derives a ref that never existed) and silently falls back to the repo default branch, so the reviewer judged the entire inherited base-branch content as the task's own work and failed acceptance criteria the task never touched. Bounced a live goals-tab fix three times, unfixable by branch surgery. The gate now resolves the base via resolve_parent_branch (the parent task's recorded branch_name, cross-team correct) and threads it as a new preferred_parent override through git.diff / list_changed_files / conventions_check_for_task — consulted only when no explicit base is given, so the pinned literal-base contract (base="HEAD~1") and every other diff caller (QA, doc, content) are byte-identical. Parent lookup fails open (derived-base fallback) like the other resolve_parent_branch call sites, and is skipped entirely while the conventions flag is off. Also excludes .uv-cache/ and .claude/ (agent worktrees, private uv cache) from the markdown prose scanner — both are repo-local tool dirs whose vendored/generated files tripped make reflow-check. Co-authored-by: Renn F <rennf93@users.noreply.github.com> * feat(vault): Obsidian vault V1 — projection core + input loop The vault is a rebuildable projection of the DB (never a source of truth), default-off behind ROBOCO_OBSIDIAN_VAULT_ENABLED + ROBOCO_VAULT_PATH. Projection core: VaultWriter materializes tasks/journals/A2A/agents as wikilinked markdown (id-suffixed stable filenames, alias-based links so renames never break, is_private journals excluded like the RAG corpus); event seams materialize on journal write and A2A send and touch task-note frontmatter at the status-transition chokepoint — all best-effort, a vault failure never blocks a verb. Shipped .obsidian config (Dataview, Kanban, team/status graph groups) + _meta dashboards; python -m roboco.vault rebuild/relocate (rebuild preserves the narrative section). Auditor narrative duty: curate_vault content verb (auditor-only, playbook-curation pattern) spawned by a dedicated root-completion hook with its own cooldown — fully separate from _dispatch_audit_work, whose scheduled-sweep/alert-producer revival belongs to the queued fleet task. Input loop: VaultIntakeEngine (ROBOCO_VAULT_INTAKE_*) watches the intake folder for #roboco-tagged notes and materializes each as ONE held draft (confirmed_by_human=False, Secretary-owned, source=vault_note, excluded by the dispatchers via _is_held_ceo_source) with local-model extraction and a deterministic fallback; vault_seen_notes ledger (migration 070) keyed on path+content-hash (the CEO-feedback callout is stripped before hashing so the engine's own append never self-triggers); per-cycle and open-draft caps. Nothing auto-starts. * fix(vault): integrate with master — re-chain migration 070 onto 069, mypy-clean tests The vault branch was cut from slave before the sequence-gate promotion, so migration 070 chained from 068 while master already carried 069 — two heads on merge. Master is now merged in and 070 revises 069. Vault test files also get mypy-clean mock idioms (monkeypatch.setattr over method assignment; await_args narrowed before access). * fix(scripts): dedupe SKIP_DIRS again after the master merge Master still carries the twin-merge duplicate (its dedupe hotfix is an unmerged PR); the merge re-imported it here. * fix(vault): reflow hard-wrapped prose in the vault asset templates * chore(config): exclude .uv-cache and .claude from deptry's scan scope Same repo-local tool dirs the prose scanner skips; the standalone deptry target walks the repo root and drowned in the cache's unpacked wheels. * fix(vault): board-review activation path for vault drafts + relocate graft The input loop's held-artifact posture was a dead end: vault_note drafts were unconditionally held by _is_held_ceo_source, owned by the verbless Secretary, hidden from the panel's approval surfaces by team, and the open-drafts cap counted them forever — the engine self-bricked after ten notes. Vault drafts now ride the intake board-review path instead: a tagged note becomes a PENDING Product-Owner-assigned Board draft (the exact confirm_live_draft board shape), the board reviews it, and only the CEO's approve_and_start makes it deliverable — never-auto-starts now rests on the board gate, proven by tests against the real dispatchers. The cap counts only drafts still awaiting the CEO (team==BOARD, non-terminal), so approval and cancellation both free it. relocate into an existing personal vault now grafts old_root/RoboCo as a direct child (refusing loudly if RoboCo/ already exists there) and adds only absent .obsidian/_meta files — a personal vault's config is never clobbered. An absent destination keeps the whole-tree move. --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com> |
||
|
Renn F
|
76a396b152 | chore(release): 0.22.0 | ||
|
Renn F
|
a322dc8f9f | chore(release): 0.21.0 | ||
|
Renn F
|
7b6eaa47c8 | chore(release): 0.20.0 | ||
|
|
0bf0cd69b3 |
fix(release): close the 0.19.0 scan findings — sandbox mongo tag, flow-verb timeout walls, video hardening (#329)
- mongo:8-alpine → mongo:8 (tag never existed; a mongo-opted project could spawn no agents) + a Docker Hub tag-existence e2e guard for every sandbox engine - flow-verb timeouts at both walls: shared SLOW_VERBS policy (i_am_done / submit_up / submit_root / open_pr / i_will_work_on get the 900s server budget); the MCP client now outlasts the server budget (+10s headroom, orchestrator-injected env) so agents receive the middleware's clean 504 envelope instead of dying at the old flat 30s client timeout - cancellation safety: the quality gate kills+reaps its child on CancelledError; create_pr records the PR via a shield-with-wait-out helper so the write can neither be skipped nor race get_db's rollback - video engine: renderer sidecar isolated on a render-only network, 2g/2cpu caps, 570s render watchdog with exit-on-hang, 512MB tar decompression cap, CEO notification on terminal render failure, reject under the approve mutex (fail-closed on Redis-down) - dead python-jose dependency removed (drops ecdsa and its unfixable Minerva advisory PYSEC-2026-1325); panel --font-mono now a real monospace stack Co-authored-by: Renn F <rennf93@users.noreply.github.com> |
||
|
|
2a9d9e25d9 |
feat(tasks): task-content guardrails — structured plans + constraints split (#328)
* feat(tasks): task-content guardrails — structured plans + constraints split Bound task PLANNING content the way journals/notes already are, fixing the poor task quality flagged 2026-07-07 (degenerate roots, over-decomposed leaves, descriptions bloated by an auto-attached conventions dump). Phase A — plan/AC guardrails (no migration): - _pm_sub_tasks_gate: cap sub_tasks at 7; per-subtask ceilings (title <=200, description <=600) enforced at both the Pydantic boundary and the gate. Dropped the min-2-roots and no-subtasks-on-code rules: both contradict the 2026-05-08 rule (test_cell_pm_can_plan_code_typed_parent_via_i_will_plan) and break legitimate single-cell roots. Long comment in the gate explains. - IWillPlanRequest: plan <=2000, approach <=800 (floor 150 kept), typed SubTaskCreate/RiskCreate/OpenQuestionCreate replacing loose list[dict]. - DelegateRequest + task_completeness: acceptance_criteria capped at 7 items, each <=200 chars. New FieldRule.MAX_LENGTH_LIST + _post_rule_reject helper (extracted to keep the gate under xenon B). - Routes dump typed models to dicts for the existing rich_plan shaper. Phase B — conventions split (migration 068): - New nullable tasks.constraints Text column; _attach_baseline_constraints now writes the ## Constraints block there instead of appending to description, so description is the human-authored instruction only. The conventions still reach the agent independently at spawn via the ambient block, so agent correctness is unaffected. - TaskResponse / Task model / panel Task type carry constraints; panel shows a read-only Constraints card. Field is optional on the TS type (backend returns null for flag-off / pre-migration rows). Tests: 5 new gate unit tests, 7 schema tests, 3 AC policy tests, 3 e2e smoke scenarios; 4 baseline-constraints integration tests updated. ruff/mypy/xenon clean; 10026 unit+foundation+e2e green; panel typecheck clean. Refs: plan breezy-imagining-kahn * test(tasks): use typed SubTaskCreate instead of dict literals in plan tests make quality runs mypy over tests/ (1079 files), not just roboco/ — the four sites passing dict literals to the now-typed sub_tasks: list[SubTaskCreate] field failed mypy. Construct SubTaskCreate directly; the typed model raising ValidationError IS the boundary the rejection tests assert. * fix(deps): drop unused python-jose — clears PYSEC-2026-1325 (ecdsa, no fix) CI's pip-audit went red on a freshly-published advisory PYSEC-2026-1325 against ecdsa 0.19.2 (no fix published — 0.19.2 is the latest). ecdsa is a transitive dep of python-jose, which is a DIRECT dep of roboco but is NOT imported anywhere in roboco/ or tests/ (grep-verified). The actual JWT path uses PyJWT (import jwt) + fastapi_users.jwt, not python-jose. So python-jose is a dead dependency. Removing it (deletion over an --ignore-vuln waiver) drops ecdsa + rsa + pyasn1 + their type stubs from the lockfile, eliminating the CVE at the source. deptry roboco/ stays clean (no missing-dep), mypy clean, auth + schema tests pass. Master CI was green 9h before this PR's run, so the advisory published in that window would red any run including master — this fix unblocks both. * chore(prompts): regenerate verb tables for typed plan sub_tasks Phase A's IWillPlanRequest schema change (sub_tasks/risks/open_questions from loose list[dict] to typed SubTaskCreate/RiskCreate/OpenQuestionCreate) made the auto-generated verb tables stale. Regenerated via scripts/regenerate_verb_tables.py — the diff is purely the signature reflection (list[str|str] -> list[SubTaskCreate], etc.). Required by the foundation-check gate (Makefile:559). --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com> |
||
|
|
3849c1737e |
feat(video): switch 0.19.0 renderer Remotion → HyperFrames (HTML-native, Apache-2.0) (#314)
* feat(video): rewrite sidecar render core to HyperFrames (in place)
* feat(video): convert motion compositions from Remotion TSX to HyperFrames HTML
* refactor(video): rename render client to video_renderer_client (renderer-agnostic)
* chore(video): rename remotion-renderer prose in test_video_pipeline docstrings
* chore(video): rename sidecar to video-renderer + add system ffmpeg for HyperFrames
* chore(video): rename stray remotion-renderer refs in sidecar + py docstrings (controller cleanup)
* chore(video): fix stale Remotion API names in Dockerfile comment (controller cleanup)
* docs(video): rewrite video-engine prose for HyperFrames + add map entry + folded prose fixes
* docs(video): add trailing newline to docs/map/video-engine.md (controller cleanup)
* chore(video): drop internal spec refs + minio/test suppressions (folded hygiene)
* fix(video): reclaim outDir on createRenderJob throw + hide empty 4th highlight
Final whole-branch review (Opus) triaged two FIX items from the SDD nits
ledger; the rest ship as-is.
- render.js: a synchronous throw from createRenderJob (post-mkdtemp, not
awaited) left an empty outDir on disk — the outer catch only reclaimed
extractDir. Reclaim outDir too when it exists, and correct the stale
comment that claimed the out dir was never created.
- {vertical,square}.html: the 4th highlights <li> lived in the DOM hidden
only by JS, so a no-JS / failed-script render would show an empty bullet.
Start it style="display:none" and reveal on populate, so an unscripted
render shows nothing instead.
Vitest smoke (release-announcement.test.js) 4/4 green; render.js syntax
checked. Python suite untouched by this fix (JS/HTML only).
* fix(video): type _override_db yield as AsyncSession | None
T7 widened _build_app's db_session param to AsyncSession | None (to drop the
4x # type: ignore[arg-type] on the DB-independent _build_app(None, ...) calls)
but left the inner _override_db fixture typed AsyncIterator[AsyncSession] —
so 'yield db_session' yielded AsyncSession | None into a declared AsyncSession,
and mypy failed at test_video_routes.py:177 ('Incompatible types in yield').
The DB-independent media tests pass db_session=None deliberately: their route
uses a monkeypatched task service and never awaits the session, so yielding
None is safe at runtime. Type the override's yield as AsyncSession | None to
match — no cast, no # type: ignore, no assert, runtime behavior unchanged.
The 3 media tests (3 passed) and the 19 db-gated tests (skipped locally) hold.
* chore(gate): skip .superpowers scratch in markdown prose gate
reflow_md.py walks the filesystem via rglob('*.md') and skips tooling dirs
(.venv, .mypy_cache, .pytest_cache, ...) but not .superpowers/ — the
superpowers SDD workflow's scratch dir (briefs, reports, progress ledger,
all gitignored). A dev running SDD locally would hit a false markdown-prose
gate failure on those transient files. Add .superpowers to SKIP_DIRS,
consistent with the existing tooling-scratch exclusions.
* fix(video): validate composition_id to close path traversal (CodeQL)
compositionId flowed unvalidated from the POST body into path.join
under extractDir/motion/compositions/, so a '../..'-style value could
escape the composition dir (CodeQL: Uncontrolled data used in path
expression). Validate at the trust boundary in server.js
(/^[A-Za-z0-9_-]+$/) and add a path.resolve + startsWith containment
check in render.js so it stays safe regardless of caller.
* fix(mcp): send X-Agent-Token + X-Agent-Team from flow/do servers
flow_server._build_headers and do_server._build_headers constructed
only X-Agent-ID/Role/Correlation-ID, omitting X-Agent-Token and
X-Agent-Team (unlike ApiClient._get_agent_headers used by the other
MCP servers). Latent since the gateway refactor — surfaced when
ROBOCO_AGENT_AUTH_REQUIRED=true was armed on the NAS, 401-ing every
flow/do verb with 'Missing X-Agent-Token header'. Add both headers
(mirroring ApiClient) so the HMAC gate passes. Tests assert the
headers are now injected.
* [video-engine] Per-project video_engine_enabled opt-in toggle
Mirrors ci_watch_enabled (migration 048): the global
ROBOCO_VIDEO_ENGINE_ENABLED flag arms the subsystem; the new
projects.video_engine_enabled column (migration 063) opts a repo into
authoring against its motion/ dir. VideoEngine._opted_in_project no-ops
open_video_task at the single chokepoint covering all three trigger
paths (on-release, on-spotlight, CEO on-demand) until the operator
flips it in the panel edit-project dialog. Existing projects stay
opted out (server_default=false).
* fix(auth): send X-Agent-Token + X-Agent-Team from all agent->API call sites
The prior fix (
|
||
|
|
4923ee3ff3 |
MinIO video storage (chunk 1: config+deps+compose) + event-loop perf fix (#308)
* feat(video): Phase A — VideoEngine origination spine + held-source gates
New default-off engine skeleton: opens a UX/UI authoring task (source=video, assigned to a ux-dev, LOW complexity to clear the dev-needs-subtasks guard) and materializes a held CEO-approval draft (source=video_post). Excludes video_post from all three held-source skip sites; adds the video_draft marker, six config flags, and the feature-flag entries. Origination + gate behavior unit-tested.
* refactor(orchestrator): fold _dispatch_dev_work skip chain into a helper
The per-source if/continue chain grew past xenon's --max-absolute B when the video_post held source joined it. Extract _is_non_dev_dispatch_source (every held-CEO source plus the two Board exploration sources) so the dev loop's skip is one flat call. Behavior-identical.
* feat(video): Phase B — propose_video do-tool (metadata-only, team-gated)
UX/UI dev records a video's composition ref + per-platform captions onto the authoring task's video_draft marker. Team-gated at runtime via _caller_team (Role.DEVELOPER can't tell a ux-dev from a be-dev). Resolves the caller's ACTIVE task via get_active_task_for_agent, not an oldest-first scan that would clobber a second open video task. Metadata only, no render. Wired through do_server + route + schema; added to _DEV_DO.
* feat(video): Phase D — render loop + RemotionRenderer client
Orchestrator-async _video_render_loop renders a completed authoring task's merged composition to MP4 (vertical + square) via the remotion-renderer sidecar and materializes the held video_post draft. RemotionRenderer tars the read-clone's motion/ source, POSTs it, and saves the returned MP4 bytes to a TASK-scoped local path (no shared volume; a composition is reused across videos so a composition-scoped path would clobber an earlier draft). Render failures bounded-retry (read-clone catch-up window, transient sidecar) up to a cap, then terminal-fail. Client tested vs a mock transport; loop vs a mock renderer + real DB.
* feat(video): Phase C — release / spotlight / on-demand video triggers
Three entry points open a UX/UI video-authoring task via VideoEngine.open_video_task: (1) a published release drafts a companion video — best-effort in ReleaseProposalService.approve, never fails the publish; script from the CHANGELOG via the local model with a template fallback. (2) propose_feature_spotlight gains optional wants_video/video_script — best-effort, gated on video_on_spotlight, default-off leaves the spotlight flow byte-for-byte unchanged. (3) POST /video/request (CEO-only) for an on-demand brief, with clean disabled/not_opened responses. All gated on video_engine_enabled.
* fix(video): savepoint-isolate video-task inserts (F042 poisoned session)
The best-effort try/except around open_video_task (release-publish + spotlight hooks) swallowed the Python exception, but a DBAPI error at the insert flush left the shared session must-rollback — so the caller's next commit (release finalize / request boundary) threw PendingRollbackError: the release stuck 'pending' after actually publishing, or the spotlight draft + HTTP response were lost. Wrap both inserts (open_video_task, _originate_video_post) in a begin_nested savepoint (the repo's established F042 pattern) so a DB error rolls back only the insert. open_video_task returns None (every caller already handles it); _originate_video_post propagates to the render loop's handler. Regression test: an insert FK error returns None with the session left usable. Dormant while the flags were off; armed on the NAS.
* feat(video): Phase G — motion/ package + remotion-renderer sidecar + compose
In-repo Remotion v4 motion/ package (ReleaseAnnouncement composition; calculateMetadata returns 1080x1920 vertical / 1080x1080 square from inputProps.orientation) + a credential-free remotion-renderer sidecar: untar the POSTed motion/ source, bundle (LRU-cached per source sha), selectComposition + renderMedia h264, stream the MP4 bytes back — matching the RemotionRenderer client contract. docker/remotion.Dockerfile on Debian (Chrome apt deps, build-time Chrome pre-warm, ffmpeg bundled in @remotion/renderer). Wired into both compose files (roboco_default only, shm_size 1gb, /health check) + the release publish matrix. Verified via a real local render of both cuts; the Debian docker build is the CEO's to run.
* chore(video): D-hardening — video_post source_task_id + render-loop docstring
Add a source_task_id back-reference to the video_post held-draft marker (traceability from a draft to its authoring task; also makes the render loop's two-key idempotency check wireable later). Fix the render-loop test's stale docstring ('never retried' -> bounded-retry). Both from the Phase D critic's non-blocking follow-ups.
* feat(video): Phase E1 — VideoPostService + heartbeat mutex (approve->post)
CEO-approve->post service: heartbeat-renewed Redis mutex (fail-closed, grace=ttl-2*heartbeat), re-read-in-lock double-post guard, per-platform durable commits (asyncio.shield-ed, settle-before-rollback on lock-loss), all writes inside the lock (captions validated pre-lock, applied in-lock — no stale whole-column clobber), idempotent, per-platform retry-skip. Poster interfaces (X/TikTok, mocked here). Reject + list-held-drafts. Survived 3 adversarial rounds; residual = a crash in the poster->commit window (CEO-gated low-freq, documented).
* fix(video): G-hardening — renderer leaks + Share Tech Mono brand font
Sidecar: give bundle() an explicit outDir tracked + deleted on LRU eviction (was leaking ~19MB remotion-webpack-bundle-* per source); res.on('close') cleanup so an aborted/retried download no longer leaks its remotion-out-* MP4 dir. Fonts: vendor Share Tech Mono (roboco-website brand font) as the display face (self-hosted woff2, 400-weight, headline fontWeight 700->400 to avoid faux-bold) + self-hosted Inter body — no gstatic fetch at render time (lsof-verified). Extras: composition_id whitelist (400) + Multer error middleware (400/413).
* feat(video): Phase E2 — X v2 + TikTok posters, tiktok_credentials, routes
LiveXVideoPoster (X v2 chunked media upload: init/append/finalize/STATUS-poll -> tweet w/ media_ids, OAuth1 signer reused). LiveTikTokPoster (OAuth2 inbox: init -> chunked PUT with asymmetric final chunk -> status-fetch; 401 -> refresh_token grant, rotated token persisted). tiktok_credentials Fernet singleton + migration 062 (single head). Routes: CEO approve/reject + list held drafts + write-only tiktok creds, wiring real posters into VideoPostService. Residual: a lock-loss right after a token-refresh flush can discard the rotated token (same rare CEO-gated class as the documented post->commit window).
* feat(video): Phase F — panel video-post queue + TikTok creds card + flags
video-post-queue.tsx: <video> MP4 preview with 9:16/1:1 cut switch, per-platform editable captions (280/2200 counters, over-limit disables approve), approve/reject, Request-a-video dialog. tiktok-credentials-card.tsx (4 write-only OAuth2 fields). feature-flags-card inlines TikTokCredentialsForm under video_engine_enabled. Mounted in command-center. tsc/eslint clean, 273 panel tests green. NOTE: needs the GET /video/posts/{id}/media route + mp4_paths on VideoPostResponse (folded into H) for the preview source.
* feat(video): Phase H — media route + e2e smoke + NAS arming + docs
GET /video/posts/{id}/media?cut= (CEO-gated FileResponse of the rendered MP4; closes the panel preview gap) + mp4_paths on VideoPostResponse. e2e smoke tests/e2e_smoke/test_video_pipeline.py (full flow, sidecar+X/TikTok mocked; asserts dispatcher skips, render-loop materialize, propose_video team-gate, approve idempotency). NAS arming: docker-compose.yml/.yaml ROBOCO_VIDEO_ENGINE_ENABLED/ON_RELEASE/ON_SPOTLIGHT default-on (.yaml resynced to .yml); registry stays off. CLAUDE.md video-engine section + CHANGELOG. Fixed 2 pre-existing route-test pollution leaks. Full suite 11763 passed.
* fix(video): auth-carrying preview, media route confinement, VideoPost type drift
Three fixes along the video preview path:
1. panel video preview auth: the <video> element was pointed straight at
GET /video/posts/{id}/media, but a native <video src> GET carries none
of axios's X-Agent-ID/X-Agent-Role headers — so in the default
header-trust deployment the request 401s. Fetch the cut via
videoApi.getMediaBlob (axios, responseType: blob) and drive <video>
off a URL.createObjectURL result instead. The object URL is revoked
on cut-change (the previous cut's URL) and on unmount, so neither
cut switches nor row teardown leak blob URLs.
2. backend media route confinement: GET /video/posts/{id}/media now
resolves mp4_path and refuses it with 404 when it falls outside
settings.video_output_dir. Defense-in-depth against any future
writer of mp4_paths serving files from arbitrary disk locations.
3. panel VideoPost type/comment drift: added mp4_paths to the
VideoPost interface (the committed VideoPostResponse already
carries it), and corrected the stale comment on videoMediaUrl
that claimed no route served the rendered bytes — the route has
existed since the media endpoint landed; the comment now describes
why getMediaBlob exists instead of a direct <video src>.
* Persist rendered videos to data in physical storage.
* ++
* docs(video): 0.18.0 CHANGELOG entry + RAG + map reference for video engine
- Move the video engine bullet from [Unreleased] into [0.18.0] and note
the ROBOCO_VIDEO_OUTPUT_DIR bind-mount persistence.
- Add docs/rag/architecture/video-engine.md (mirrors x-engine.md shape:
enable/disable, three triggers, render loop + sidecar, CEO gate, media
route confinement, credentials).
- Reference the video render loop in docs/map/orchestrator.md's engine list.
* chore(video): re-bump to 0.19.0 + sync registry compose defaults
Version was wrongly bumped to 0.18.0; 0.18.0 is an already-released
section. Restore its 2026-07-04 date and move the video-engine CHANGELOG
bullet into a new [0.19.0] - 2026-07-05 section above it. Bump
pyproject.toml, roboco/__init__.py, roboco/config.py (app_version),
panel/package.json, and the motion/README inputProps example to 0.19.0.
docker-compose.registry.yml: add ROBOCO_VIDEO_ENGINE_ENABLED /
_VIDEO_ON_RELEASE / _VIDEO_ON_SPOTLIGHT defaulted false (NAS arms them
true), and comment out the video-renders bind mount with a short note
so the public registry image ships video off by default. Structural
sync with docker-compose.yml maintained.
* fix(video): rate-limit /render + reflow motion/README
CodeQL flagged js/missing-rate-limiting on the renderer /render route.
The sidecar is container-network-only with one trusted caller (the
orchestrator, which renders cuts serially), so this limiter is a
retry-storm ceiling (30/min, well above legit render rate), not the
primary control. Also reflows motion/README.md hard-wrapped prose that
failed the markdown quality gate.
* fix(build): finish pnpm 11 migration + regen verb tables
The panel Docker image build failed on `pnpm install --frozen-lockfile`:
node:22-alpine's corepack resolved to its bundled pnpm 11, but
panel/package.json pinned packageManager to pnpm@10.25.0, and pnpm 11
refuses to run against that pin. The Dockerfiles were already written for
pnpm 11 (comments, CI=true, strictDepBuilds); the package.json pin was the
stale outlier. Finish the migration instead of working around it:
- panel/package.json: packageManager pnpm@10.25.0 -> pnpm@11.10.0; drop the
`pnpm` field (pnpm 11 ignores it — build approval lives in
panel/pnpm-workspace.yaml's allowBuilds). Lockfile unchanged (pnpm 11
accepts it as-is); frozen-lockfile verified.
- remotion-renderer/package.json: pin packageManager pnpm@11.10.0 for
determinism (was relying on corepack's implicit default); engines.node
>=22.13 (pnpm 11 requirement).
- docker/panel.Dockerfile + docker/remotion.Dockerfile: `corepack prepare
pnpm@11.10.0 --activate` so the build uses the pinned version explicitly
instead of trusting corepack's bundled default (which a future
node:22-alpine could change).
- .github/workflows/panel-ci.yml: Node 20 -> 22 (pnpm 11 requires
Node >=22.13; Node 20 fails the engines check).
Also regenerate agents/prompts/_generated/{developer,head_marketing,verbs}.md
— the video engine added propose_video and extended propose_feature_spotlight
(wants_video, video_script) but the verb tables weren't refreshed, failing
the foundation-check quality gate.
* chore(build): approve esbuild build script in remotion pnpm-workspace.yaml
pnpm 11 generated this file with a placeholder ('set this to true or false')
during install; resolve it to true so local dev of the renderer doesn't
re-prompt. esbuild's postinstall only verifies the prebuilt platform binary
(@esbuild/<platform> is installed as an optional dep), so approving it is
safe and silences the ERR_PNPM_IGNORED_BUILDS warning.
* fix(build): copy pnpm-workspace.yaml into panel + remotion images
pnpm 11 hard-errors with [ERR_PNPM_IGNORED_BUILDS] (exit 1) when a
dependency ships a postinstall script that isn't approved in
allowBuilds. Both Dockerfiles copied only package.json + pnpm-lock.yaml,
so the build-approval map in pnpm-workspace.yaml never made it into the
image — the remotion image build died on esbuild@0.28.1's postinstall.
Copy pnpm-workspace.yaml alongside the manifests in both images. In
panel, this also drops the --config.strictDepBuilds=false workaround:
with sharp and unrs-resolver now approved, their postinstalls run and
install the platform-specific binaries (previously skipped, leaving
sharp without its @img/sharp-* binary at runtime).
Verified locally: remotion + panel `pnpm install --frozen-lockfile`
exit 0 with the workspace file present; both exit 1 without it.
* fix(perf): offload conventions + release-readiness blocking I/O off the event loop
The orchestrator runs uvicorn and the orchestration background loops on a
single shared event loop, so any sync I/O anywhere — even inside a background
loop — blocks API responsiveness for its duration. Two call sites were missing
asyncio.to_thread wrappers:
- ConventionsService.get_map/health/restore called the sync _resolve
(`git rev-parse`), _read_committed_standard (file read + yaml parse), and
_derive (filesystem walk via derive_from_scan) inline. Reachable from
GET /api/projects/{id}/conventions and from the agent spawn-prepare path.
- ReleaseManagerEngine._production_assess called gather_snapshot inline —
multiple `subprocess.run` git calls + a filesystem walk, running inside the
release-manager background loop.
Wrap each blocking call in asyncio.to_thread at the async boundary. No
signature changes; helpers stay sync. Verified: targeted tests pass
(196 passed, 36 DB-skipped), ruff + format clean.
These were the only responsiveness gaps surfaced by the concurrency audit —
the rest of the heavy paths (agent spawn via `docker run -d`, video render
loop, git ops via the 16-worker ThreadPoolExecutor, workspace subprocess
calls) already offload correctly. No API/worker container split needed.
* feat(storage): add MinIO config + dep + compose (no-op, default-off)
Chunk 1 of the MinIO video-storage plan (§1, §2, §6). No behavior change:
minio_endpoint defaults to empty = disabled, the existing FileResponse serve
path is untouched (chunk 4 wires the serve path; chunk 2 adds the client).
- pyproject.toml: add `minio` (minio-py) to dependencies; regenerate uv.lock
(resolves minio v7.2.20 + pycryptodome transitive).
- roboco/config.py: add 5 settings fields after video_output_dir
(minio_endpoint/_access_key/_secret_key/_bucket/_region). Plain str Fields
matching the existing ROBOCO_ENCRYPTION_KEY style; no SecretStr, no
presign_ttl_seconds (YAGNI — we don't presign in phase 1).
- docker-compose.yml: add `minio` service (data network only, named
minio-data volume, host ports 19000/19001 for debugging, mc healthcheck)
and a one-shot `minio-init` service mirroring the ollama-init pattern
(mc alias set + mb -p, idempotent via || true). Add ROBOCO_MINIO_* env to
the orchestrator env block (endpoint, access/secret key, bucket, region).
- docker-compose.registry.yml: intentionally omit the minio/minio-init
services and leave ROBOCO_MINIO_* unset (NAS default-on, registry
default-off — the established pattern); comment added to the orchestrator
env block noting the omission.
* docs(storage): 0.19.0 CHANGELOG + RAG + map reference for MinIO chunk 1
Backfills the release-polish docs for MinIO chunk 1 (§10 of the plan):
- docker-compose.yaml synced to docker-compose.yml (the two NAS compose files
must stay byte-identical; .yml was edited in chunk 1, .yaml was stale).
- CHANGELOG [0.19.0]: Added (MinIO scaffolding) + Fixed (event-loop I/O offload).
- docs/rag/architecture/minio-storage.md: RAG doc mirroring video-engine.md.
- docs/map/deployment-tooling.md: one-line storage reference.
* MinIO chunk 2: minio_client module (singleton + unconfigured guard) (#309)
* feat(storage): minio_client module (singleton + unconfigured guard)
Chunk 2 of the MinIO plan (§3). roboco/services/minio_client.py adds:
- get_client(): singleton minio-py Minio from settings; returns None when
minio_endpoint is empty (the disabled path used by the chunk 3/4 guards).
Parses http://... endpoint into host:port + secure flag.
- put_object(bytes, key): no-ops when unconfigured; otherwise PUTs to
settings.minio_bucket with ContentType video/mp4.
- get_object_stream(key): yields object bytes for StreamingResponse; lets
S3Error propagate so the serve route (chunk 4) can fall back to disk.
Sync calls — every call site wraps in asyncio.to_thread (chunks 3/4). One
unit test covers the unconfigured guard + endpoint scheme parsing (mocks,
no real MinIO). Not yet wired into remotion_client._save or the media route.
* MinIO chunk 3: wire write path (remotion_client._save PUT) (#310)
* feat(storage): wire MinIO write path in remotion_client._save
Chunk 3 of the MinIO plan (§3). After the local mp4 write, _save PUTs the bytes
to MinIO under key = Path(mp4_path).name (already {render_key}-{orientation}.mp4),
guarded by minio_client.get_client() (None when minio_endpoint empty) and
wrapped in asyncio.to_thread. Local disk stays the source of truth for the
poster publish path (x_video_client/tiktok_client read mp4_path from disk);
the PUT is additive. _save still returns the local path str — mp4_paths,
marker, and schema unchanged. Disabled (local-only) when MinIO unconfigured.
One test: asserts put_object is called with the basename key when configured
and the local file is still written; existing test stays green via the
unconfigured-default path. Mocks only.
* fix(storage): make MinIO PUT non-fatal in remotion_client._save
A configured-but-down MinIO made put_object raise inside the worker thread,
failing the render and retry-looping a task whose local file was already
written. Local disk is the source of truth and the serve route falls back to
FileResponse on S3Error, so a failed durable-copy PUT must never fail the
render — log and continue; the next render re-attempts the PUT.
Adds test_save_swallows_minio_put_failure (PUT raises -> _save still returns
the local path and the local file is written). Extends the CHANGELOG write-
path bullet with the non-fatal guarantee.
* MinIO chunk 4: serve path (StreamingResponse + FileResponse fallback) (#311)
* feat(storage): serve MinIO via the media route (StreamingResponse + FileResponse fallback)
Chunk 4 of the MinIO plan (§4 — the crux). GET /api/video/posts/{id}/media
derives key = Path(mp4_path).name and, when minio_endpoint is set, returns a
StreamingResponse over minio_client.get_object_stream(key), keeping
_require_ceo so auth stays end-to-end (no presigned URLs). Falls back to
FileResponse on S3Error (old render not in MinIO) or when MinIO is
unconfigured — the panel's axios-blob flow is unchanged (same URL, headers,
body, just chunked). The confinement check is kept as defense-in-depth (the
key is a basename so traversal is impossible, but the check is cheap and
protects the poster path).
Two integration tests: configured serve path streams from a stubbed
get_object_stream (CEO 200, non-CEO 403); unconfigured fallback serves the
local file via FileResponse. Mocks only — no real MinIO.
* fix(storage): eager stat_object probe so the MinIO serve fallback actually fires
The chunk-4 route wrapped StreamingResponse(get_object_stream(key), ...) in a
try/except, but get_object_stream is a lazy generator — its client.get_object
call runs on the first next(), i.e. AFTER the route returned and Starlette
started streaming. An S3Error (NoSuchKey / MinIO down) there is uncatchable;
the try/except caught nothing and the FileResponse fallback never triggered.
Add minio_client.stat_object(key): an eager existence/readiness probe that
runs INSIDE the route's try/except, so a missing object or down MinIO raises
before the StreamingResponse starts and the fallback serves the local file.
stat-then-get is two round trips; a mid-stream failure after a successful stat
is a rare race the CEO can retry (documented ceiling).
Tests: the configured test now stubs stat_object; a new test asserts the
S3Error fallback serves the local file via FileResponse and that
get_object_stream is never called. RAG doc updated to record the eager-probe
correctness detail + the non-fatal PUT.
* docs(rag): mark MinIO deployment note landed (chunk 5) (#312)
Co-authored-by: Renn F <rennf93@users.noreply.github.com>
---------
Co-authored-by: Renn F <rennf93@users.noreply.github.com>
---------
Co-authored-by: Renn F <rennf93@users.noreply.github.com>
---------
Co-authored-by: Renn F <rennf93@users.noreply.github.com>
* fix(video): offload minio stat_object off the event loop
stat_object was called inline in the async media route, blocking the
shared event loop for one sync urllib3 round-trip per preview request —
contradicting minio_client's own 'every call site wraps in to_thread'
docstring and this PR's perf-fix theme. Wrap in asyncio.to_thread; the
try/except still catches S3Error (to_thread re-raises) so the
FileResponse fallback is unchanged. Also add the trailing newline to
the minio-storage RAG doc.
* Fix red CI
* Make CI green
---------
Co-authored-by: Renn F <rennf93@users.noreply.github.com>
|
||
|
|
e9d0e0bd48 |
feat(video): 0.19.0 video engine (Remotion) + preview auth + render persistence (#307)
* feat(video): Phase A — VideoEngine origination spine + held-source gates
New default-off engine skeleton: opens a UX/UI authoring task (source=video, assigned to a ux-dev, LOW complexity to clear the dev-needs-subtasks guard) and materializes a held CEO-approval draft (source=video_post). Excludes video_post from all three held-source skip sites; adds the video_draft marker, six config flags, and the feature-flag entries. Origination + gate behavior unit-tested.
* refactor(orchestrator): fold _dispatch_dev_work skip chain into a helper
The per-source if/continue chain grew past xenon's --max-absolute B when the video_post held source joined it. Extract _is_non_dev_dispatch_source (every held-CEO source plus the two Board exploration sources) so the dev loop's skip is one flat call. Behavior-identical.
* feat(video): Phase B — propose_video do-tool (metadata-only, team-gated)
UX/UI dev records a video's composition ref + per-platform captions onto the authoring task's video_draft marker. Team-gated at runtime via _caller_team (Role.DEVELOPER can't tell a ux-dev from a be-dev). Resolves the caller's ACTIVE task via get_active_task_for_agent, not an oldest-first scan that would clobber a second open video task. Metadata only, no render. Wired through do_server + route + schema; added to _DEV_DO.
* feat(video): Phase D — render loop + RemotionRenderer client
Orchestrator-async _video_render_loop renders a completed authoring task's merged composition to MP4 (vertical + square) via the remotion-renderer sidecar and materializes the held video_post draft. RemotionRenderer tars the read-clone's motion/ source, POSTs it, and saves the returned MP4 bytes to a TASK-scoped local path (no shared volume; a composition is reused across videos so a composition-scoped path would clobber an earlier draft). Render failures bounded-retry (read-clone catch-up window, transient sidecar) up to a cap, then terminal-fail. Client tested vs a mock transport; loop vs a mock renderer + real DB.
* feat(video): Phase C — release / spotlight / on-demand video triggers
Three entry points open a UX/UI video-authoring task via VideoEngine.open_video_task: (1) a published release drafts a companion video — best-effort in ReleaseProposalService.approve, never fails the publish; script from the CHANGELOG via the local model with a template fallback. (2) propose_feature_spotlight gains optional wants_video/video_script — best-effort, gated on video_on_spotlight, default-off leaves the spotlight flow byte-for-byte unchanged. (3) POST /video/request (CEO-only) for an on-demand brief, with clean disabled/not_opened responses. All gated on video_engine_enabled.
* fix(video): savepoint-isolate video-task inserts (F042 poisoned session)
The best-effort try/except around open_video_task (release-publish + spotlight hooks) swallowed the Python exception, but a DBAPI error at the insert flush left the shared session must-rollback — so the caller's next commit (release finalize / request boundary) threw PendingRollbackError: the release stuck 'pending' after actually publishing, or the spotlight draft + HTTP response were lost. Wrap both inserts (open_video_task, _originate_video_post) in a begin_nested savepoint (the repo's established F042 pattern) so a DB error rolls back only the insert. open_video_task returns None (every caller already handles it); _originate_video_post propagates to the render loop's handler. Regression test: an insert FK error returns None with the session left usable. Dormant while the flags were off; armed on the NAS.
* feat(video): Phase G — motion/ package + remotion-renderer sidecar + compose
In-repo Remotion v4 motion/ package (ReleaseAnnouncement composition; calculateMetadata returns 1080x1920 vertical / 1080x1080 square from inputProps.orientation) + a credential-free remotion-renderer sidecar: untar the POSTed motion/ source, bundle (LRU-cached per source sha), selectComposition + renderMedia h264, stream the MP4 bytes back — matching the RemotionRenderer client contract. docker/remotion.Dockerfile on Debian (Chrome apt deps, build-time Chrome pre-warm, ffmpeg bundled in @remotion/renderer). Wired into both compose files (roboco_default only, shm_size 1gb, /health check) + the release publish matrix. Verified via a real local render of both cuts; the Debian docker build is the CEO's to run.
* chore(video): D-hardening — video_post source_task_id + render-loop docstring
Add a source_task_id back-reference to the video_post held-draft marker (traceability from a draft to its authoring task; also makes the render loop's two-key idempotency check wireable later). Fix the render-loop test's stale docstring ('never retried' -> bounded-retry). Both from the Phase D critic's non-blocking follow-ups.
* feat(video): Phase E1 — VideoPostService + heartbeat mutex (approve->post)
CEO-approve->post service: heartbeat-renewed Redis mutex (fail-closed, grace=ttl-2*heartbeat), re-read-in-lock double-post guard, per-platform durable commits (asyncio.shield-ed, settle-before-rollback on lock-loss), all writes inside the lock (captions validated pre-lock, applied in-lock — no stale whole-column clobber), idempotent, per-platform retry-skip. Poster interfaces (X/TikTok, mocked here). Reject + list-held-drafts. Survived 3 adversarial rounds; residual = a crash in the poster->commit window (CEO-gated low-freq, documented).
* fix(video): G-hardening — renderer leaks + Share Tech Mono brand font
Sidecar: give bundle() an explicit outDir tracked + deleted on LRU eviction (was leaking ~19MB remotion-webpack-bundle-* per source); res.on('close') cleanup so an aborted/retried download no longer leaks its remotion-out-* MP4 dir. Fonts: vendor Share Tech Mono (roboco-website brand font) as the display face (self-hosted woff2, 400-weight, headline fontWeight 700->400 to avoid faux-bold) + self-hosted Inter body — no gstatic fetch at render time (lsof-verified). Extras: composition_id whitelist (400) + Multer error middleware (400/413).
* feat(video): Phase E2 — X v2 + TikTok posters, tiktok_credentials, routes
LiveXVideoPoster (X v2 chunked media upload: init/append/finalize/STATUS-poll -> tweet w/ media_ids, OAuth1 signer reused). LiveTikTokPoster (OAuth2 inbox: init -> chunked PUT with asymmetric final chunk -> status-fetch; 401 -> refresh_token grant, rotated token persisted). tiktok_credentials Fernet singleton + migration 062 (single head). Routes: CEO approve/reject + list held drafts + write-only tiktok creds, wiring real posters into VideoPostService. Residual: a lock-loss right after a token-refresh flush can discard the rotated token (same rare CEO-gated class as the documented post->commit window).
* feat(video): Phase F — panel video-post queue + TikTok creds card + flags
video-post-queue.tsx: <video> MP4 preview with 9:16/1:1 cut switch, per-platform editable captions (280/2200 counters, over-limit disables approve), approve/reject, Request-a-video dialog. tiktok-credentials-card.tsx (4 write-only OAuth2 fields). feature-flags-card inlines TikTokCredentialsForm under video_engine_enabled. Mounted in command-center. tsc/eslint clean, 273 panel tests green. NOTE: needs the GET /video/posts/{id}/media route + mp4_paths on VideoPostResponse (folded into H) for the preview source.
* feat(video): Phase H — media route + e2e smoke + NAS arming + docs
GET /video/posts/{id}/media?cut= (CEO-gated FileResponse of the rendered MP4; closes the panel preview gap) + mp4_paths on VideoPostResponse. e2e smoke tests/e2e_smoke/test_video_pipeline.py (full flow, sidecar+X/TikTok mocked; asserts dispatcher skips, render-loop materialize, propose_video team-gate, approve idempotency). NAS arming: docker-compose.yml/.yaml ROBOCO_VIDEO_ENGINE_ENABLED/ON_RELEASE/ON_SPOTLIGHT default-on (.yaml resynced to .yml); registry stays off. CLAUDE.md video-engine section + CHANGELOG. Fixed 2 pre-existing route-test pollution leaks. Full suite 11763 passed.
* fix(video): auth-carrying preview, media route confinement, VideoPost type drift
Three fixes along the video preview path:
1. panel video preview auth: the <video> element was pointed straight at
GET /video/posts/{id}/media, but a native <video src> GET carries none
of axios's X-Agent-ID/X-Agent-Role headers — so in the default
header-trust deployment the request 401s. Fetch the cut via
videoApi.getMediaBlob (axios, responseType: blob) and drive <video>
off a URL.createObjectURL result instead. The object URL is revoked
on cut-change (the previous cut's URL) and on unmount, so neither
cut switches nor row teardown leak blob URLs.
2. backend media route confinement: GET /video/posts/{id}/media now
resolves mp4_path and refuses it with 404 when it falls outside
settings.video_output_dir. Defense-in-depth against any future
writer of mp4_paths serving files from arbitrary disk locations.
3. panel VideoPost type/comment drift: added mp4_paths to the
VideoPost interface (the committed VideoPostResponse already
carries it), and corrected the stale comment on videoMediaUrl
that claimed no route served the rendered bytes — the route has
existed since the media endpoint landed; the comment now describes
why getMediaBlob exists instead of a direct <video src>.
* Persist rendered videos to data in physical storage.
* ++
* docs(video): 0.18.0 CHANGELOG entry + RAG + map reference for video engine
- Move the video engine bullet from [Unreleased] into [0.18.0] and note
the ROBOCO_VIDEO_OUTPUT_DIR bind-mount persistence.
- Add docs/rag/architecture/video-engine.md (mirrors x-engine.md shape:
enable/disable, three triggers, render loop + sidecar, CEO gate, media
route confinement, credentials).
- Reference the video render loop in docs/map/orchestrator.md's engine list.
* chore(video): re-bump to 0.19.0 + sync registry compose defaults
Version was wrongly bumped to 0.18.0; 0.18.0 is an already-released
section. Restore its 2026-07-04 date and move the video-engine CHANGELOG
bullet into a new [0.19.0] - 2026-07-05 section above it. Bump
pyproject.toml, roboco/__init__.py, roboco/config.py (app_version),
panel/package.json, and the motion/README inputProps example to 0.19.0.
docker-compose.registry.yml: add ROBOCO_VIDEO_ENGINE_ENABLED /
_VIDEO_ON_RELEASE / _VIDEO_ON_SPOTLIGHT defaulted false (NAS arms them
true), and comment out the video-renders bind mount with a short note
so the public registry image ships video off by default. Structural
sync with docker-compose.yml maintained.
* fix(video): rate-limit /render + reflow motion/README
CodeQL flagged js/missing-rate-limiting on the renderer /render route.
The sidecar is container-network-only with one trusted caller (the
orchestrator, which renders cuts serially), so this limiter is a
retry-storm ceiling (30/min, well above legit render rate), not the
primary control. Also reflows motion/README.md hard-wrapped prose that
failed the markdown quality gate.
* fix(build): finish pnpm 11 migration + regen verb tables
The panel Docker image build failed on `pnpm install --frozen-lockfile`:
node:22-alpine's corepack resolved to its bundled pnpm 11, but
panel/package.json pinned packageManager to pnpm@10.25.0, and pnpm 11
refuses to run against that pin. The Dockerfiles were already written for
pnpm 11 (comments, CI=true, strictDepBuilds); the package.json pin was the
stale outlier. Finish the migration instead of working around it:
- panel/package.json: packageManager pnpm@10.25.0 -> pnpm@11.10.0; drop the
`pnpm` field (pnpm 11 ignores it — build approval lives in
panel/pnpm-workspace.yaml's allowBuilds). Lockfile unchanged (pnpm 11
accepts it as-is); frozen-lockfile verified.
- remotion-renderer/package.json: pin packageManager pnpm@11.10.0 for
determinism (was relying on corepack's implicit default); engines.node
>=22.13 (pnpm 11 requirement).
- docker/panel.Dockerfile + docker/remotion.Dockerfile: `corepack prepare
pnpm@11.10.0 --activate` so the build uses the pinned version explicitly
instead of trusting corepack's bundled default (which a future
node:22-alpine could change).
- .github/workflows/panel-ci.yml: Node 20 -> 22 (pnpm 11 requires
Node >=22.13; Node 20 fails the engines check).
Also regenerate agents/prompts/_generated/{developer,head_marketing,verbs}.md
— the video engine added propose_video and extended propose_feature_spotlight
(wants_video, video_script) but the verb tables weren't refreshed, failing
the foundation-check quality gate.
* chore(build): approve esbuild build script in remotion pnpm-workspace.yaml
pnpm 11 generated this file with a placeholder ('set this to true or false')
during install; resolve it to true so local dev of the renderer doesn't
re-prompt. esbuild's postinstall only verifies the prebuilt platform binary
(@esbuild/<platform> is installed as an optional dep), so approving it is
safe and silences the ERR_PNPM_IGNORED_BUILDS warning.
* fix(build): copy pnpm-workspace.yaml into panel + remotion images
pnpm 11 hard-errors with [ERR_PNPM_IGNORED_BUILDS] (exit 1) when a
dependency ships a postinstall script that isn't approved in
allowBuilds. Both Dockerfiles copied only package.json + pnpm-lock.yaml,
so the build-approval map in pnpm-workspace.yaml never made it into the
image — the remotion image build died on esbuild@0.28.1's postinstall.
Copy pnpm-workspace.yaml alongside the manifests in both images. In
panel, this also drops the --config.strictDepBuilds=false workaround:
with sharp and unrs-resolver now approved, their postinstalls run and
install the platform-specific binaries (previously skipped, leaving
sharp without its @img/sharp-* binary at runtime).
Verified locally: remotion + panel `pnpm install --frozen-lockfile`
exit 0 with the workspace file present; both exit 1 without it.
---------
Co-authored-by: Renn F <rennf93@users.noreply.github.com>
|
||
|
Renn F
|
4393fab5c1 |
chore(release): 0.18.0 — bump pyproject.toml (canonical ref missed in 88db8ca6)
The 0.18.0 cut bumped config.py, __init__.py and panel/package.json but a truncated grep hid pyproject.toml's version=0.17.0, so |
||
|
Renn F
|
7cb00611e1 |
chore(comms): finalize #306 teardown — changelog + purge dangling refs
#306 removed the channels/sessions/messages subsystem but left dangling references. Agents were still told to call removed verbs at spawn, and a maintenance script referenced a dropped table. - prompts (roles/identities/teams/base): drop say/open_session/link_session/ channels() and the dead Channels sections; comms rows now teach A2A (dm + read_a2a); renumber the PM workflow steps the removed open_session step left behind - scripts/reset_runtime_state.sql: drop the dropped chunks_conversations table - models/events.py: mark the retired SESSION_*/MESSAGE_SENT enum members inert - mcp/{do,flow}_server.py: drop the dead SESSION_CLOSED error-map key - panel: drop channels_read/write from AgentPermissions; remove dead channel: KB source branch - pyproject.toml: refresh a ruff-exemption example off the removed verb kwargs - CHANGELOG: record #306 under [Unreleased] |
||
|
|
3ccc723cd4 |
v0.17.0 — Wave 3: sandbox DB, DB isolation, mobile UI, cloud auth, X account, roadmap engine (#303)
* feat(sandbox): throwaway per-agent Postgres/Redis sandbox containers
Orchestrator-provisioned sibling containers per agent spawn
(SandboxProvisioner, roboco/runtime/sandbox.py). Per-project opt-in via
projects.sandbox_services (migration 057); master switch
ROBOCO_SANDBOX_DB_ENABLED, default-off, armed in the NAS compose only.
When active, ROBOCO_TEST_DB_* / ROBOCO_TEST_REDIS_* point at the sandbox
and the prod-creds gate-env injection is suppressed (sandbox replaces,
never coexists). Sandbox lifetime tracks the agent container: teardown at
every removal path, orphan janitor at startup + each reaper tick with a
grace window for mid-flight spawns. The pre-spawn stale-clear spares the
just-provisioned sandbox; provision pre-clears stale same-named
containers from a crash-missed teardown.
Panel: per-project sandbox-service switches in the edit dialog + feature
flag card entry.
* docs: CLAUDE.md entry for the sandboxed dev DB/Redis subsystem
* feat(security): isolate prod Postgres/Redis from agent containers (roboco_data network)
Second user-defined bridge roboco_data carries postgres+redis only; the
orchestrator is multi-homed (default + data). Spawned agents and their
sandbox sidecars stay on roboco_default and can no longer resolve or
reach roboco-postgres:5432 / roboco-redis:6379 (redis has no auth —
membership is its only containment). Normal bridge, so host-published
ports (15432/16379) keep working. Applied to both build composes and
the registry compose; docker-compose.yml re-synced byte-identical with
docker-compose.yaml (it had drifted by the sandbox flag block).
ROBOCO_DB_NETWORK_ISOLATED (config default false, armed alongside the
topology) suppresses the legacy _append_gate_env prod-creds injection:
under isolation those creds dead-end, and unreachable creds are worse
than none. DB-needing projects opt into sandbox_services instead. The
flag is deliberately not a panel feature flag - it must travel with the
compose networks: stanzas.
Preserved by construction: agent<->agent A2A and orchestrator->agent SDK
polls on :9000, MCP->orchestrator on :8000, ollama reachability, docker
exec/inspect (daemon socket), host port publishing.
* feat(panel): full mobile responsiveness pass
Shared primitives: useIsMobile (useSyncExternalStore, hydration-safe,
memoized matchMedia subscribe), ResponsiveTable table->card switch below
md (single subtree mounted, no duplicated interactive rows), scrollable
snap TabsList in the base primitive (justify-center-safe so the first
tab stays reachable on overflow), persistent md:hidden bottom tab bar
(Overview/Tasks/Kanban/Chat, safe-area padded).
Applied: card lists for tasks/projects/products/work-sessions/sessions
+ the three raw metrics tables; CEO approval queue / release proposal /
playbook review action rows stack on narrow; command-center reorders
approvals above the fold on mobile; task-header metadata wraps;
Communications + A2A become URL-driven single-pane drill-downs below lg
(fixes the unconstrained-height ScrollArea bug) with dvh heights;
recharts label density/radius adapts via useIsMobile; git diff viewer
gets mobile font + wrap toggle; vh->dvh sweep; chat composers get
safe-area-inset padding; dashboard main p-4 md:p-6 + pb-20 for the bar.
Verified at 375px on the built app: bottom bar, drawer, approval-first
overview, swipeable kanban tab strip. All gates green (eslint, tsc,
vitest 249, next build 24/24 routes).
* feat(auth): cloud auth via FastAPI Users (default-off, single-user cookie session)
ROBOCO_CLOUD_AUTH_ENABLED (default off) lets the panel/API be exposed
beyond localhost without changing the CEO's local no-login flow while
off — get_agent_context and the WS gate are byte-for-byte unchanged in
off-mode. On: header-trust dies for humans — any agent-role claim (ceo
or a privileged PM/board role) with no valid HMAC token or session
cookie is 401, closing the header-spoof hole on the host-published
:8000 port for every role. The agent-fleet HMAC path and the system
self-PATCH keep working unmodified in both modes.
Single seeded CEO user (migration 058 users table, UserTable), no
registration router — idempotent env-driven upsert at startup by PK.
Cookie transport (httponly/secure/samesite=lax) + a JWTStrategy bound
to a fingerprint of the current password hash (rotating the password
invalidates every prior session). Sliding 30-day session: every
authenticated request re-mints the cookie, so an active session never
expires — no unexpected logouts.
Panel: (auth)/login page + proxy.ts (Next 16 rename of middleware; probes
/auth/status over the docker-internal URL, fails open to off) gate the
dashboard; client.ts gets withCredentials + 401->/login. nginx unchanged.
Review hardening: broadened the on-mode rejection from ceo-only to every
non-CEO role without a valid token (was only closed when
ROBOCO_AGENT_AUTH_REQUIRED was also armed); Next-16 proxy.ts rename to
clear the middleware deprecation warning.
* feat(x): RoboCo X account engine — HoM drafts, per-post CEO approval (default-off)
ROBOCO_X_ENGINE_ENABLED (default off, inert without creds). Mirrors the
ReleaseManagerEngine held-artifact shape: XEngine drafts a post when a
release publishes (via a draft_release_post seam on ReleaseProposalService
.approve) and drafts replies to meaningful mentions (dedicated poll loop,
x_seen_mentions dedup ledger, per-cycle/open caps). Drafting is
local-model-only, clamped to 280 chars. Nothing auto-posts — every tweet
is a held task (source x_post/x_reply, confirmed_by_human=False,
Secretary-owned, dispatcher-skipped) the CEO edits/approves/rejects in a
panel queue.
The four OAuth 1.0a secrets live Fernet-encrypted in a singleton
x_credentials row (migration 059, all-or-nothing, API returns only
has_credentials); decryption is server-side, agents never hold creds or
egress. Hand-rolled OAuth 1.0a HMAC-SHA1 signer, no new dependency;
NullXClient makes the unconfigured path a graceful no-op.
XPostService.approve (CEO-only) is the sole caller of post_tweet.
Review hardening: closed a double-post race — the approve path now
re-reads committed task state inside the Redis lock and commits COMPLETED
before releasing, so a concurrent approve that acquires the lock after the
winner released can't re-post (SET-NX is non-waiting, and the route-level
commit landed after the lock dropped). Added a regression test.
* feat(roadmap): board roadmap engine — PO proposes themed cycles, CEO approves per-item (default-off)
ROBOCO_ROADMAP_ENGINE_ENABLED (default off). Weekly, RoadmapEngine opens
ONE held exploration task (source=board_roadmap, confirmed_by_human=False,
Product-Owner-assigned), deduped to one open cycle. A dedicated one-shot
_dispatch_roadmap_exploration spawns the PO solo (not the two-reviewer
board path, which would also spawn HoM + fire Approve-&-Start). The PO
explores read-only (git/KB/metrics/releases/charter/web) and makes one
propose_roadmap call (PO-only content verb) authoring a themed cycle —
goal + 3-7 item drafts — persisted as a roadmap_cycle marker (no table,
no migration; head stays 059).
The CEO acts per-item in the panel roadmap queue: approve materializes a
BACKLOG task (source=roadmap, no assignee — never auto-starts), reject
records a reason; all-items-terminal completes the exploration task.
RoadmapService is idempotent per item. Dispatchers skip board_roadmap.
Includes a real SQLAlchemy dirty-check fix (deep-copy the JSON marker
before mutating, or the in-place edit + reassign compares equal to its
own baseline and the UPDATE is skipped).
Review hardening: create_task_from_draft now honors a draft-declared
source only from a {prompter, roadmap} whitelist — drafts are
LLM-authored, so an unbounded source could impersonate a privileged
origin (release_manager would even wedge that engine's dedup).
* chore(release): 0.17.0
Wave 3 — six default-off subsystems: sandboxed dev DB/Redis, prod
Postgres/Redis network isolation, full mobile UI pass, cloud auth
(FastAPI Users), the RoboCo X account engine, and the board roadmap
engine. Plus the waves 1+2 work already on master since 0.16.0.
Version bumped across the canonical set (config.py, __init__.py,
pyproject.toml, panel/package.json, uv.lock); CHANGELOG [Unreleased]
cut to [0.17.0]; docs/map delta added.
Compose: every optional feature armed :-true in the NAS composes, OFF
in the user-facing registry compose. Two opt-in exceptions default off
(CLOUD_AUTH — needs email/password/secret + TLS, would otherwise fail
startup; ROUTING_STRICT — fail-closed spawning). DB_NETWORK_ISOLATED
stays on in both (coupled to the roboco_data topology).
* chore(compose): arm cloud_auth + routing_strict ON in the NAS composes
Every feature defaults ON in the NAS composes per policy — these two
were wrongly left off. Both keep the ${VAR:-true} form so the operator
controls the real runtime via .env: cloud auth needs
ROBOCO_CLOUD_AUTH_EMAIL/_PASSWORD/_SECRET + TLS set there before a boot
(else startup fails loud), and routing_strict is fail-closed. Registry
compose keeps both off.
* fix(ci): reflow board.md prose (quality gate) + document v0.17.0 env creds
The roadmap section added hard-wrapped prose that failed the markdown
prose gate; reflowed (token-invariant). Also brought .env.example
current: cloud auth (now armed — needs SECRET or startup fails), routing
strict, the X engine (panel-entered OAuth), and web research.
* fix(ci): reduce cyclomatic complexity of five wave-3 blocks (xenon gate)
The wave-3 subagents introduced C-rank functions the CI xenon gate
rejects (my per-item reviews ran ruff/mypy/pytest but not xenon):
- sandbox.janitor_sweep -> extract _list_labeled_sandboxes /
_list_live_agent_containers / _prune_grace
- x_client.fetch_mentions -> extract _parse_mention_items
- x_engine.run_cycle -> extract _process_mentions
- orchestrator._dispatch_pm_work -> extract the source-skip into a
MODULE-level _is_held_ceo_source (module, not method, so the
wholesale-mocked dispatcher unit tests exercise the real logic)
- auth/seed.ensure_seed_user -> extract _apply_seed_updates (module avg -> A)
Behavior-preserving; full suite green (11902), xenon clean.
* fix(ci): declare pyjwt + fastapi-users-db-sqlalchemy as direct deps (deptry)
The cloud-auth code imports jwt and fastapi_users_db_sqlalchemy directly
but they were only transitive deps (via fastapi-users), which deptry
(quality gate, DEP003) rejects. Declared explicitly; deptry roboco/ clean.
Missed originally because local make quality stopped at earlier gates
before reaching deptry.
* feat(x): gate mention replies behind ROBOCO_X_REPLIES_ENABLED (default off)
Per CEO decision: the X engine should only post about releases by
default. Reading mentions needs a paid X API tier, so the mention-reply
half is now a deliberate opt-in on top of release posting.
New default-off flag x_replies_enabled gates the mentions poll loop
(_x_mentions_poll_loop) and XEngine.run_cycle; release-post drafting
(the release-proposal approve hook) is unaffected and still runs when
x_engine_enabled + credentials are set. Added to FEATURE_FLAGS + the
panel card. Tests: release posting works with replies off; run_cycle +
the poll loop are no-ops with replies off.
* fix: 401 only redirects to /login when cloud auth is on; panel-token strips .env quotes
Two bugs that together dead-ended login in secure mode:
- client.ts redirected to /login on ANY 401, so a mismatched panel
token (header-trust/secure mode, cloud auth off) bounced the user to a
login page whose backend route isn't mounted -> 404. Now it probes
/auth/status (bare fetch, no interceptor re-entry) and only redirects
when cloud_auth_enabled.
- make panel-token read the .env secret with grep|cut without stripping
surrounding quotes, so a quoted ROBOCO_AGENT_AUTH_SECRET produced a
token signed with the quotes included — which never verifies against
the orchestrator (docker-compose/pydantic unquote the secret). Now
strips surrounding single/double quotes.
* fix: git-log 500 on '|' in commit message; X queue shows an empty state
- GET /api/git/log 500'd (ValueError: Invalid isoformat) when a commit
SUBJECT contained a '|' (e.g. the 'curl|sh' lockdown commit): the
fixed '|' field delimiter let the subject's pipe shift the split so
author+date collapsed into one field. Switched to \x1f (Unit
Separator), which can't appear in commit content. Regression test with
a piped subject.
- The X Post Queue returned null when empty, so there was no visible
place for the X drafts. It now renders a discoverable empty state
pointing at Settings -> X credentials.
* docs: bring docs/rag + docs/map current for v0.17.0 (waves 1-3)
Agent-facing RAG corpus and codebase map updated for every feature in
the 0.17.0 span, code-verified:
- wave 3: sandbox DB, DB network isolation, cloud auth, X engine
(+ x_replies_enabled sub-flag), board roadmap engine — new RAG
architecture pages + role/tool/config-reference updates; new symbols,
migrations 057-059, panel surfaces, and the get_agent_context
dual-path across the map slices.
- waves 1-2: A2A live view + switchboard, prompter memory
(search_past_tasks), Secretary edit access + PM-lighter scope, the
PR-gate auto-submit turn cut (ROBOCO_PR_GATE_AUTO_SUBMIT_ENABLED).
- correctness fix: api-routes-schemas.md no longer claims the A2A admin
routes are reachable by any authenticated agent — they carry a
_require_ceo gate (wave 2c).
docs/internal, _front.md deltas, and the frozen _complete_map.md
snapshot untouched.
* fix(rag): atomic upsert for indexed-doc tracking (kills e2e segfault)
The indexed-document tracking write used check-then-insert in two paths
(IndexedDocumentRepository.upsert_batch and the file-source
_upsert_doc_record). Under concurrent indexing both callers saw no row
and both inserted, so the second violated uq_indexed_doc_source and
poisoned its transaction — surfacing in CI as the intermittent
_checkin_failed SIGSEGV on the failed connection's pool checkin.
Both paths now use INSERT ... ON CONFLICT DO UPDATE against the
constraint: coalesce keeps an existing title/preview when the new value
is empty (matching the old guards) and metadata is jsonb-merged. The
batch dedupes within itself first (ON CONFLICT can't touch a row twice
in one statement). expire_all after the Core upsert keeps same-session
ORM reads consistent with the merged DB row.
---------
Co-authored-by: Renn F <rennf93@users.noreply.github.com>
|
||
|
|
5936c2bdea |
Docs split Phase 1: docs.roboco.tech becomes canonical — redirect stubs, user tree removed, MkDocs retired (#299)
* docs: replace MkDocs deploy with static redirect stubs docs.roboco.tech (roboco-website) is now the canonical user-facing docs site (spec: docs/internal/specs/2026-07-03-docs-site-split.md). Every URL this repo's Pages site published needs to keep resolving, so scripts/gen_docs_redirects.py generates one meta-refresh + rel=canonical stub per page — derived from mkdocs.yml's nav while it still lists every page — into the committed docs-redirects/ directory. All 58 stub targets verify against the website repo's nav.ts (0 unmapped); the only rename is how-to/* -> tour/* per the spec's slug map. Rewrite .github/workflows/docs.yml to deploy docs-redirects/ directly instead of running `mkdocs build`. The user-facing docs/ tree and mkdocs.yml itself are untouched here — deleting them is the next step, gated on these stubs resolving correctly. * docs: delete the MkDocs user-facing tree, docs.roboco.tech is canonical Per docs/internal/specs/2026-07-03-docs-site-split.md decision (1): Material->MDX is not verbatim-portable, so repo A's user-facing docs are deleted rather than kept as a permanently-drifting mirror. Deletes index/get-started/company/how-to/panel/models/operations/optional/deploy/ api/troubleshooting plus images/videos/assets. KEEPS docs/rag/ (indexed agent corpus), docs/map/, docs/internal/, and the team buckets (backend/frontend/ux_ui) — none of these were ever in mkdocs.yml's nav. mkdocs.yml's entire nav mapped 1:1 onto the deleted tree, so pruning it "accordingly" leaves nothing — remove it outright, along with the now-dead `docs` optional-dependency group (mkdocs/mkdocs-material/mkdocstrings/ pymarkdownlnt — mkdocstrings was already unused, not wired into any mkdocs plugin), the matching deptry DEP002 ignore entries, .pymarkdown.json, and the serve-docs/build-docs/lint-docs/fix-docs Makefile targets (all scoped only to the deleted paths). Add regen-docs-redirects as the one remaining docs Makefile target. Repointed everything that linked into the deleted tree or the old Pages URL: README's hero video/gif and walkthrough links now hit the docs.roboco.tech-hosted copies (already duplicated there per the spec's ground truth), usage.md / deployment.md's jump-links, pyproject's Documentation URL, and CLAUDE.md's Blueprint Reference paragraph. * chore: sync uv.lock after removing the docs optional-dependency group Follow-up to the mkdocs.yml / docs extra removal — mkdocs, mkdocs-material, mkdocstrings, pymarkdownlnt, and their transitive-only dependencies drop out of the lockfile now that nothing in pyproject.toml declares them. --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com> |
||
|
|
d1cf6ecbf3 |
Wave 1: PR-gate turn cut, task search, trace timestamps, Secretary edits + e2e scenarios 2–3 (#295)
* feat(tests): e2e scenario 2 — the PM merge chain through the PR gate
Shared arcs extracted (arcs.py: canonical-company seeding + dev/qa/doc
segments); scenario 2 seeds a root->cell->dev hierarchy mid-flight, rides
the child through the scenario-1 arc into the cell branch (real squash
via the fake GitHub), then submit_up -> claim_gate_review/pr_pass ->
dispatcher re-claim (mirrored) -> PM complete merging cell->root. This is
the exact PM->reviewer->PM turn sequence the wave-1 turn cut shortens —
the BEFORE-net. Learned seams scripted: commit-subject validator (>=20
chars), reviewer learning-note gate, pr_pass clears ownership by design.
* feat(runtime): PR-gate turn cut — assembled parents auto-submit to the reviewer
When every child of an assembled parent is terminal, the closure
dispatcher now runs the real submit_up/submit_root through the internal
API as the owning PM (_try_auto_submit) instead of spawning the PM for
that turn — the submit's substance is deterministic gate code. Any gate
refusal falls back to the classic PM closure spawn; pr_fail routing and
the PM's final merge turn are unchanged; umbrellas never auto-submit.
ROBOCO_PR_GATE_AUTO_SUBMIT_ENABLED default-on; task.auto_submitted audit
row per cut. Proven by e2e scenario 2b (real API, real gates, real git)
against scenario 2 as the before-net.
* feat(notes): structured note sections carry a written_at trace stamp
Sections are overwrite-in-place, so without a stamp there was no way to
reconstruct WHEN a dev/qa/doc/reviewer note landed (CEO reMarkable item:
trace TIMESTAMPS). apply_structured_note stamps ISO written_at beside
the model fields; the panel notes tab renders it next to each card
title (pre-stamp rows render nothing). Progress updates, commits, and
journal entries already carried timestamps — this was the one gap.
* feat(tasks): server-side task search — title, details, and id prefix
The task list's search box only matched titles client-side, and the
trimmed summary payload deliberately carries no description — so
keyword/details/id search was impossible in the browser by design.
GET /tasks/summary gains q (ILIKE over title+description, id-prefix
match, composed with team/status and the view-permission scoping);
the panel debounces the box into the summary fetch and drops the
title-only client filter that would have hidden description matches.
* feat(wave-1): trace timestamps, real task search, Secretary task edits
- apply_structured_note stamps written_at per section; the panel notes
tab shows it (the one trace surface without a timestamp).
- GET /tasks/summary?q= searches title+description+id-prefix server-side
(summaries carry no description by design); panel debounces into the
fetch and drops the title-only client filter.
- Secretary control_task gains a CEO-gated edit action over the content
allowlist, and GET /secretary/tasks?q= resolves task names to ids for
the chat. PM-side expansion deferred per the CEO's 'not that much'.
* fix(workspace): dep-update probe scrubs the inherited venv pin
Under uv run the orchestrator's process tree carries VIRTUAL_ENV, and a
uv-based dep_update_command in the throwaway probe clone would target
that venv instead of the clone's — the same hazard _uv_subprocess_env
already guards on the install path.
* build: private per-repo uv cache — isolate from machine-wide uvx servers
Root cause of the recurring rich/pip/bandit rot, with evidence: uv cache
clean timed out on the ~/.cache/uv lock ('is another uv process
running?') — three uvx mcp-server-fetch processes (Claude Code fetch MCP,
one alive since Wednesday) share that cache and race repo syncs on it;
poisoned entries then survive venv rebuilds because rm -rf .venv never
touches the cache, and every re-link reproduces the breakage. UV_CACHE_DIR
now pins <repo>/.uv-cache (gitignored). The earlier UV_NO_SYNC
serialization stays as defense-in-depth but was not the whole story.
* feat(tests): e2e scenario 3 — pr_fail revision loop + root→CEO chain
3a: reviewer pr_fail with a concrete issue -> needs_revision ->
i_will_plan re-entry (full plan gates) -> real fix lands on the cell
branch (the unchanged-PR hard gate refuses resubmit until it does) ->
clean second pass -> merge. 3b: submit_root -> gate -> Main PM complete
escalates the root to the CEO -> the REAL approve-and-merge endpoint
squash-merges to the origin's master. Harness gains the tasks router, a
seeded CEO identity, origin_commit, and a fake GitHub whose head.sha is
recomputed live (real-GitHub semantics the unchanged gate reads). Seeds
now encode the real shape: delivery roots are team=main_pm and
planning-typed.
---------
Co-authored-by: Renn F <rennf93@users.noreply.github.com>
|
||
|
Renn F
|
6b5691b02a | chore(release): 0.16.0 | ||
|
|
1c87a4e4e4 |
Leak fixes, gate green again, uv/CI hardening, e2e lifecycle smoke harness (#294)
* test: align phase1 smoke mock with the armed team-match gate The |
||
|
|
0ca9d91b72 |
v0.16.0: fastapi-guard HTTP security layer — calibrated + scanner honeytrap (#290)
* [fastapi-guard] Phase 1a: gated config flags for the HTTP security layer Adds the ROBOCO_GUARD_* settings (all default-off / secure-default) for the upcoming fastapi-guard 7.2.0 hardening — guard_enabled (master switch), guard_fail_secure (fail-closed default; NAS overrides to false), guard_telemetry_enabled + guard_agent_api_key + guard_project_id (guard-agent telemetry, opt-in), guard_emergency + guard_emergency_whitelist (lockdown kill switch). Inert until consumed: nothing reads them yet, so the request path is unchanged. Foundation for v0.16.0. * [fastapi-guard] Phase 1b: security foundation module + gated wiring Add fastapi-guard 7.2.0 + guard-core 3.3.0 (bare, unpinned) and roboco/security.py: - build_security_config() from settings — behind-nginx real-IP (trusted_proxies + trust_x_forwarded_proto), HSTS/CSP headers, threat-ban + 404-sweep rules, redis-backed state, exclude_paths (/ws + health + docs), env-driven enforce_https, fail_secure (secure default), emergency lockdown, guard-agent telemetry (opt-in), passive-mode calibration switch. - guard_deco singleton (SecurityDecorator) for per-route decorators (Phase 2+). - Three custom content validators guard's WAF can't cover: prompt-injection / role-override, secret-exfil / credential-in-body, internal-SSRF. - apply_guard(app) + guarded_lifespan() wired into create_app AFTER settings. guard_passive_mode config flag added. Entirely gated by ROBOCO_GUARD_ENABLED (default off): create_app mounts nothing and returns the unchanged app when off (verified). make quality GREEN (cov 95.32%, pip-audit clean, import-linter 2/0). 12 new unit tests. * [fastapi-guard] Phase 2: critical-path decorators Apply guard decorators to the highest-value endpoints (metadata-only; enforced only when the middleware is mounted, so no-op while ROBOCO_GUARD_ENABLED is off): - provider keys (ollama/grok/self-hosted writes): strict rate_limit + max_request_size + block_clouds (no datacenter IP should touch secret writes). - settings write + release approve/reject (CEO-gated): strict rate_limit. - intake chat (prompter start/messages/events): rate_limit + max_request_size + custom_validation(prompt_injection_validator) — the prompt-facing free-text ingress gets the injection/role-override/secret-exfil content scan. make quality GREEN (cov 95.32%, contracts 2/0). App builds with guard off, decorators inert (verified). * [fastapi-guard] Phase 3: wide decorator coverage across ingress + sensitive routes Targeted-wide application (metadata-only; no-op until ROBOCO_GUARD_ENABLED). The global SecurityMiddleware already rate-limits + WAF-scans every request, so this adds the custom content validators on free-text ingress + tight limits on sensitive ops (not blanket per-route rate_limit on reads): - agent gateway do verbs (note/say/commit/dm/pitch/progress/draft_playbook/...): rate_limit + max_request_size + custom_validation(secret_exfil or prompt_injection). - a2a message/send + chat writes: rate_limit + size + prompt_injection. - optimal/RAG (kb/search, rag/query, mentor/ask, errors/decisions/standards/ learnings): prompt_injection on searches, secret_exfil on record writes; docs index → internal_ssrf. - tasks: create/update → prompt_injection; QA/doc/PM transitions → secret_exfil; CEO-gated verbs → tight rate_limit. - secretary chat → prompt_injection; research → internal_ssrf; orchestrator spawn/mutations → rate_limit; git ops + flow verbs → tight rate_limit. Pure GET/reads left to the global middleware. Applied via a Sonnet workflow, then verified: app builds with guard off (decorators inert), make quality GREEN (cov 95.37%, contracts 2/0). Decoy/honeypot-path surface deferred (needs verified guard ban-API integration — not rushed). * [fastapi-guard] Phase 5: arm the NAS composes in passive/log-only mode Arm ROBOCO_GUARD_ENABLED=true + ROBOCO_GUARD_PASSIVE_MODE=true + ROBOCO_GUARD_FAIL_SECURE=false on the two NAS composes (docker-compose.yaml + .yml). Passive = guard mounts and logs what it WOULD block but blocks nothing, so the next NAS deploy calibrates against real traffic; flip PASSIVE_MODE off after the false-positive review to enforce. fail_secure=false keeps a guard-internal error from 500ing the personal deploy. The registry (user-facing) compose is deliberately left unarmed so its published default stays conservative. Phase 4 (passive calibration) is the operational step this enables. * feat(security): Phase 3b — full-arsenal per-route guard enrichment Stack the applicable guard decorators per surface instead of the minimal rate_limit/max_request_size/custom_validation triad: content_type_filter on every JSON-body write, honeypot_detection form-traps on human-facing POSTs, block_clouds on key-writes + CEO release ops, behavior_analysis runaway-rate rules on the agent flow/do verbs, suspicious_detection + usage_monitor on the sensitive surfaces. Nine distinct decorators now applied thoughtfully per endpoint. All metadata-only — no-op while ROBOCO_GUARD_ENABLED is off. * fix(a2a): permit PR reviewer to deliver gate verdicts to the owning PM can_a2a_direct had no pr_reviewer rule, so a reviewer (team=board, or a cell team) fell through to the cell-member path and was cross-cell-denied when the in-path gate delivered a pr_fail change-request to main-pm (or a cross-cell cell-pm): "Cannot A2A main-pm ... Ask None to coordinate with None". The delivery is best-effort, so pr_fail still transitioned but the verdict never reached the owning PM — the blind-re-submit signal-gap the pr_fail fix closes. Add an explicit pr_reviewer handler: it may A2A only cell_pm / main_pm (its sole comms surface — everything else it posts on the PR itself), with a matching route hint. The cell reviewers kept same-team access by coincidence; this scopes every reviewer to PM-only, the correct model, with no other A2A caller affected. Refresh uv.lock to the current resolution. * feat(models): adopt Claude Sonnet 5 as the sonnet tier Point the 'sonnet' alias at claude-sonnet-5 (MODEL_MAP) and give pr_reviewer its own opus tier in ROLE_MODEL_MAP — it was falling through to the sonnet default, and the role gates untrusted external/fork PRs plus root→master, which warrants opus. Price claude-sonnet-5 at the promotional 33% off Sonnet 4.6 ($2.01 / $10.05, cache 0.201 / 0.5025) through 2026-08-31 via a dedicated pricing fragment that beats the bare 'sonnet' alias; revert to full rate when the promo ends. Bare 'sonnet' stays full-rate as a conservative fallback (prod prices the resolved claude-sonnet-5 id from the transcript). Update the model docs and the billing / usage / manifest / spawn tests. * feat(security): calibrate the guard WAF for RoboCo traffic + document the layer The first end-to-end run of the fastapi-guard layer showed active enforcement would block ~50% of legitimate agent traffic — RoboCo request bodies are code, SQL, diffs, file paths, HTML, and URLs, which the stock signature WAF reads as attacks. build_security_config now excludes RoboCo's free-text top-level body fields (derived from the real request models, including the free-form container fields whose nested prose is stringified and scanned) from WAF scanning, dropping the active-mode false-positive rate to zero while keeping the WAF on every non-excluded (id/enum/slug/branch) field and leaving the prompt-injection / secret-exfil / internal-SSRF validators — which run independently of the exclusion — fully in force. enable_penetration_detection is made explicit. Only excluded_detection_body_fields is reliable on guard 7.2.1: the per-route categories knob is bypassed for JSON bodies, and the body scanner excludes top-level keys only (scanning str(value) of every non-excluded field), so free-form container fields must be excluded wholesale. Adds tests/unit/test_security_middleware.py — the first end-to-end exercise of the middleware (mounts it, drives guard's lifespan, fires real requests): proves passive mode is log-only, active mode does not false-positive on realistic agent payloads, threats are still blocked inside excluded fields, and the WAF still fires on non-excluded fields. Docs: CHANGELOG (Unreleased); a user-facing Optional-subsystems page + nav + env reference for the HTTP security layer; the agent-facing RAG corpus (what it is + why a request could be blocked); and the roboco mapping (api-core-websocket / deployment-tooling / _complete_map). * feat(security): Surface N — scanner honeytrap (guard /api auto-ban + nginx edge-drop) Turns scanner probes against the scanner, in two layers matched to where traffic lands. Behind nginx only /api, /ws, /health, /ready reach the orchestrator, so guard can only see (and ban) scanner probes on those paths; the classic root probes (/.env, /wp-login.php, /phpmyadmin, /.git/config) hit the panel. So: - build_security_config's threat_ban_config gains recon / sensitive_file / cms_probing categories. A scanner probing those fingerprints on an /api path is detected on the URL-path scan; repeated probes from one IP trip an adaptive per-IP auto-ban (redis-backed, 24h). Only bans in active mode (passive logs the recon hit) and needs redis (the 24h ban exceeds the in-memory cap). The spec's decoy-route file is redundant — the WAF url-path scan bans regardless of a registered route — so it is intentionally omitted. - docker/nginx.conf drops the classic root scanner paths at the edge with 444 (connection closed, no response) before they reach the panel, anchored to known scanner fingerprints so /.well-known and every real panel/API route are untouched. Always on, independent of ROBOCO_GUARD_ENABLED. Tests: 2 unit (the exclusion set + the scanner-ban categories are present) and 2 integration (a decoy path is blocked in active mode, passes in passive). The nginx regex was validated against 15 scanner + 19 legit paths (0 false positives). Docs: CHANGELOG, the HTTP-security page, the roboco mapping, and the agent-facing RAG corpus. * Token optimization — per-role observability, compute policy, spawn preflight (#291) * test(models): lock the sonnet→claude-sonnet-5 MODEL_MAP invariant * feat(usage): surface cache tokens + cache_hit_rate in usage breakdowns * feat(usage): add per-role usage breakdown endpoint * feat(usage): add spawn-waste signal (per-role unproductive rate + respawn strikes) * feat(panel): surface per-role cost/cache + spawn-waste on the metrics page * feat(routing): Phase 2 per-role compute policy — qa→haiku, main_pm→sonnet, per-role effort env mechanism (default-inert) * feat(orchestrator): Phase 3 flag-gated spawn preflight — refuse non-gateway delivery roles (respawn-forever guard) * chore(compose): arm ROBOCO_SPAWN_PREFLIGHT_ENABLED on the NAS composes * docs: per-role usage observability, per-role compute policy, and spawn preflight --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com> * fix(panel): pin outputFileTracingRoot so the standalone build isn't broken by stray lockfiles * feat(routing): populate ROLE_EFFORT_MAP + wire the verified --effort flag (cell_pm/board/auditor to medium) * feat(gateway): omit empty context_briefing sections (Phase 4 payload compaction) * refactor(orchestrator): extract spawn chokepoint guards to restore xenon rank B on spawn_agent --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com> |
||
|
|
a8cb2470ba |
v0.15.0: Metrics granularity — per-member / per-task / org + CEO scorecards (#289)
* feat(metrics): capture per-session turns + tool_calls (phase 1)
Persist LLM iterations (turns) and tool invocations per agent spawn session,
the raw signal the granular per-member performance metrics build on (real
effort/iterations vs wall-clock).
- sum_transcript_usage returns a 5-tuple adding turns = unique assistant
message-id count; _usage_from_transcript + _resolve_active_tokens updated to
the 5-tuple (active-tokens keeps its 4-tuple contract by slicing).
- SDK: _SessionState.turns, set by /usage/sync; /usage/status (TokenUsageStatus)
now carries turns + tool_calls (= total_calls).
- orchestrator: new _resolve_final_turns_tools (SDK primary, transcript fallback
for turns only; Grok -> 0/0) wired into _finalize_spawn_session, which writes
turns + tool_calls to agent_spawn_sessions.
- migration 055 adds turns + tool_calls (BigInteger DEFAULT 0 -> historical/Grok
rows read 0, surfaced as n/a). Verified real alembic upgrade/downgrade.
Part of metrics-granularity (v0.15.0); recon-adjusted plan on disk.
* feat(metrics): pure compute_stage_effort helper (phase 2, part 1)
Foundation-layer overlap math (no DB): split each task status window into
active (merged wall-clock overlap of spawn stints — concurrent stints counted
once, so active <= window) vs wait (queue/review idle). Distinct from summed
effort. The per-task metrics service will feed it audit-log windows + spawn
stints. 9 unit tests (disjoint/nested/partial/merged/clamped/zero/multi-window).
* feat(metrics): per-task live metrics + GET /metrics/task/{id} (phase 2)
TaskMetrics dataclass + MetricsService.get_task_metrics: summed spawn effort
(vs wall-clock), turns/tool_calls/tokens/cost, per-stage active-vs-wait
(compute_stage_effort over audit windows x spawn stints), and who-caused-rework
(revision_count + named qa/pr fail events). Open stints and the open final
stage window close at completed_at for a terminal task (else now), so stages
don't grow past completion. Exposed at GET /dashboard/metrics/task/{task_id}
(404 if absent). Real-PG tests (compose/none/in-flight) + route tests (200/404).
* feat(metrics): CEO-as-member scorecard + ceo_reject audit regression (phase 3)
The human CEO is a measured member, read purely from audit_log (agent_role='ceo'
serializes from the CEO StrEnum): approval dwell (awaiting_ceo_approval -> a CEO
decision, incl. the coordination-root reject that lands in pending), unblock
dwell (blocked -> a CEO revive), and god-mode action count (every CEO-attributed
transition). CeoScorecard + MetricsService.get_ceo_scorecard (p50/p90 via
PERCENTILE_CONT, expanding IN for the decision sets) + GET
/dashboard/metrics/member/ceo (declared before any future member/{id} route).
The ceo_reject coordination-root audit gap the plan meant to close was already
closed by the gap-sweep (routes through admin_set_status -> agent_role='ceo'
audit); locked with a regression assertion in the existing coordination-reject
test. Real-PG tests: approval/unblock/godmode, non-ceo exclusion, empty->zeros.
* feat(metrics): audit instrumentation for escalations/blocked-others/idle (phase 4a)
The three extra per-member metrics that had no data source get durable,
in-session audit events (additive; never gate the underlying action):
- apply_escalation -> task.escalated (details.escalator_slug) on both the
normal block path and the pool-divert path -> escalations count.
- _unblock_dependents -> task.unblocked_dependents (details.count) on the
completed BLOCKER task, captured before the dependency edges are pruned ->
blocked-others count (sweeper attributes to the blocker's owner).
- mark_agent_idle -> agent.idle (details.agent_slug) -> idle/utilization (the
sweeper pairs an idle mark to the member's next spawn for idle duration).
(QA pass-rate needs no new event — reuses task.awaiting_documentation[qa] +
task.qa_fail.) Real-PG tests for each; 111 transition tests still green.
* feat(metrics): member_performance_daily rollup table + migration 056 (phase 4b)
The per-member scorecard rollup: one row per (date, member_kind, agent_slug),
CEO as a first-class member_kind='ceo' row (agent_slug='' NOT NULL so the
NULL-distinct UNIQUE keeps it unique). Full column set + the four CEO-approved
extras (qa_reviews_total/passed, escalations, blocked_others, idle_seconds) plus
blocked_seconds. Overwrite-upsert on (date, member_kind, agent_slug) for an
idempotent sweep. Migration 056 verified real up/down (24 cols, 4 indexes).
* feat(metrics): _sweep_member_performance rollup sweeper (phase 4c)
The daily per-member rollup sweep (mirrors _sweep_daily_rollup): a trailing
7-day, idempotent overwrite-upsert wired into _run_sweep. One focused query per
metric merges into a (date, agent_slug) accumulator — spawn effort/turns/tokens/
cost, completed/first-pass/revisions-received, revisions-caused (qa/pr fails),
QA pass-rate (passed + total), escalations (by escalator_slug), blocked-others
(unblocked_dependents by blocker owner), idle_seconds (idle mark -> next spawn),
blocked_seconds (blocked dwell) — plus one CEO row/day (approval/unblock dwell +
god-mode). Real-PG test asserts every facet + idempotency (a 2nd sweep
overwrites, never doubles); spawn-day != completion-day split is by-design.
* feat(metrics): member/org rollup scorecards + endpoints + live overlay (phase 5)
MemberScorecard + OrgScorecard with derived rates (FPY, effort-throughput,
turns/tool-calls per task, QA pass-rate, utilization) — all division-guarded to
None. get_member_scorecard reads member_performance_daily by slug and overlays
the member's live in-flight (non-terminal) tasks' effort via get_task_metrics
(disjoint by status: completion counts stay rollup-only, overlay only enriches
effort/turns/cost; includes_live_inflight flags it). get_org_scorecard
aggregates the cell (?team=) or whole org. Routes: GET /metrics/member/{agent_id}
(404 if absent, after the ceo literal route) + GET /metrics/org?team=. Real-PG
tests (derived rates, overlay no double-count, guards, org) + route tests.
* feat(metrics): granular CEO completion notification (phase 6)
There was no CEO completion notification at all (EventType.TASK_COMPLETED was
defined but never emitted). Add notify_ceo_of_completion in
NotificationDeliveryService — a granular body (real effort vs wall-clock +
stints/turns/tool-calls/revisions[QA/PR]/cost from get_task_metrics; degrades to
wall-clock-only, turns 'n/a', when there are no spawn sessions). Reuses the
existing ALERT type (no enum migration; the notificationtype PG enum is fixed at
001). ceo_approve now emits TASK_COMPLETED + fires the notification (best-effort
via _notify_completion — never blocks completion); complete() emits
TASK_COMPLETED too (closes the dead-code gap; the WS bridge can forward it).
Pure formatter tests + real-PG notification test.
* [metrics-granularity] Phase 7: panel Scorecards tab + dashboard overview
Add the CEO-facing metrics surfaces for the granularity feature:
- New "Scorecards" tab on the Metrics page: org rollup headline, the
CEO-as-member card (approval/unblock dwell + god-mode count), and a
per-member table (completed, first-pass yield, active effort, turns/task,
QA pass-rate, escalations, blocked-others, utilization). Each member row
self-fetches its rollup scorecard; live in-flight rows carry a "live" badge.
- New dashboard overview card (ScorecardOverviewPanel): org-wide 30-day
headline (completed, FPY, throughput/hr, active effort, cost) deep-linking
into the Scorecards tab.
- Plumbing: TaskMetrics/MemberScorecard/OrgScorecard/CeoScorecard types,
observability API client methods + empty fallbacks, and the four
useCeoScorecard/useMemberScorecard/useOrgScorecard/useTaskMetrics hooks.
Panel gate green: tsc, eslint, prettier, vitest (175 tests, +6 new).
* [metrics-granularity] test: make completion-notification robust to shared-DB CEO
test_notify_ceo_of_completion_creates_alert errored in the full suite (passed
in isolation): the session-scoped test DB is shared across the run, and the
sibling real-DB board-gate test commits a role=CEO agent (slug="ceo") without
cleanup — so my env fixture's hardcoded slug="ceo" insert hit a unique-constraint
violation, and a second role=CEO row would also make _get_ceo_agent()'s
scalar_one_or_none() raise. Reuse an existing CEO when present (the singleton the
production system actually has), else create one with a unique slug. Order-
independent. Also reflow test_metrics_instrumentation.py to ruff format.
* chore(release): 0.15.0
Metrics granularity: per-member/per-task/org + CEO-as-member scorecards,
turn/tool-call capture (migration 055), member_performance_daily rollup
(migration 056) with QA pass-rate / escalations / blocked-others / utilization,
per-task active-vs-wait metrics, granular completion notification, panel
Scorecards tab + dashboard Performance card, and the ceo_reject audit fix.
Version bump across the canonical set + CHANGELOG.
* [metrics-granularity] fix pre-tag audit findings (overlay double-count + panel error states)
Adversarial review before the v0.15.0 tag surfaced two real logical gaps:
- MAJOR (backend): the live in-flight overlay re-summed ALL sessions of every
non-terminal task via get_task_metrics, but _msweep_spawn already rolls up
every CLOSED session regardless of task status — so a closed session on a
still-open task was counted twice (rollup + overlay), permanently inflating a
member's effort/turns/tokens/cost on the common reap/respawn path. The overlay
now sums only OPEN sessions (ended_at IS NULL), which the closed-only rollup
can never contain — disjoint by construction. A just-closed session lands in
the rollup on the next ~60s sweep (no gap of note). Aggregated in SQL to mirror
_msweep_spawn. Regression test reproduces the double-count (turns 10→5).
- MAJOR (panel): the four new scorecard surfaces used `isLoading || !data` with
no isError branch, so a failed query span forever on a skeleton. They now
surface a load error. Tests added.
Also: OrgSummary active-effort formatting no longer round-trips hours→seconds→
hours; dashboard grid uses xl:grid-cols-4 (was 2xl) so 4 panels show at 1280px;
corrected the inaccurate "NULL distinct" CEO-row uniqueness comment (agent_slug
is NOT NULL; the '' tuple is simply distinct from agent rows).
make quality GREEN (cov 95.31%); panel GREEN (vitest 178).
* [metrics-granularity] fix: decode bytes stream message-id before XCLAIM
StreamEventBus._recover_stream passed the pending message id to XCLAIM via
str() on the raw bytes the client returns (redis client has no
decode_responses), producing "b'1782066556728-0'". Redis rejects that with
"Unrecognized XCLAIM option", so pending-message recovery threw on every
reclaim tick and unacked messages from crashed/slow consumers were never
reclaimed (leaking in the PEL on every stream, spamming the error log). Decode
via the existing _to_str helper — the fix the sibling claim path already uses.
Pre-existing in v0.14.0 (unrelated to metrics granularity); folded into this
release per CEO. TDD regression test + CHANGELOG entry. make quality GREEN.
---------
Co-authored-by: Renn F <rennf93@users.noreply.github.com>
|
||
|
|
15effce014 |
Chore: 141 Gaps fill-in (#283)
* Updated uv.lock
* Bunch of fixes we need to verify first..
* feat(megatask): per-cell project map root-subtasks (multi-project, multi-cell)
A MegaTask root-subtask can now target an ad-hoc per-cell project map — a
third targeting shape that mirrors the existing product fan-out root. In
RoboCo a project is per-cell (ProjectTable.assigned_cell); a monorepo is N
per-cell projects sharing one git_url. So 'multi-cell' IS 'multi-project',
and a task may mix per-cell projects across products or include OSS-library
projects not in any product.
Storage: migration 052 adds task_cell_projects (mirrors product_projects;
unique per (task, team)). TaskTable gains a cascade-delete cell_projects
relationship; TaskCreateRequest / TaskCreate / Task response carry the map.
Policy: batch.is_branchless_coordination + is_valid_batch_shape gain a
has_cell_projects param — a root-subtask targets exactly one of project /
product / cell-map; the umbrella still targets none. TaskService passes
has_cell_projects at every predicate call site and persists the rows in
create(). _ensure_branch_for_task cuts feature/main_pm/{root} per distinct
project in the map (via _distinct_projects_for_task); _require_target_or_umbrella
and _validate_batch_membership accept the map shape.
Fan-out: every distinct_project_ids site (task.py branch creation, routes
_project_for_complete + _resolve_project_for_merge, orchestrator
_ambient_projects_for_task, pr_review._project_slug_for, git._project_for_task)
generalizes to first-distinct-project-of-map-or-product. Choreographer
_resolve_subtask_project resolves a delegated subtask's cell from the parent's
cell map. The product-scoped _slugs_for_product intake helper is unchanged.
Intake: prompter._draft_cell_map extracts the per-cell map from the_work[].
_validate_batch_scope counts distinct projects across all drafts' cells
(>=2 min stays; one 2-cell draft satisfies it). create_task_from_draft
persists cell_projects for >=2-cell drafts (project_id/product_id None),
collapses a 1-cell map to the single-project shape, and leaves single-cell
top-level project_id drafts unchanged. _resolve_owning_team routes a
multi-cell map to Main PM (coordination root, like a product root — a cell
PM can't delegate cross-cell). propose_draft/propose_batch tool descriptions
declare the per-cell project_id (both Claude SDK + grok runtimes).
The umbrella stays branchless / pure-coordination / submit_root-rejected;
the CEO-escalation pr_number gate is not widened (the map root is
is_umbrella=False, mirroring a product root, so submit_root supplies it).
Single-cell root-subtasks and everything below them are byte-for-byte
unchanged. Un-run MegaTask waves (multi-cell drafts) become runnable.
* [feature] Panel per-cell project picker + pnpm format infra
MegaTask root-subtasks can fan out across cells (be+fe, fe+uxui). Since a
RoboCo project is per-cell (ProjectTable.assigned_cell), a monorepo is N
per-cell projects sharing one git_url — so multi-cell IS multi-project. The
batch-review card now shows one project Select per the_work entry, scoped to
that cell's repos, instead of one Select bound to a single top-level
project_id. confirmBatch validates each cell's project is in scope and the
batch still spans >=2 distinct projects.
- prompter.ts: CellWork gains optional project_id (the per-cell picker seam).
- batch-review-card.tsx: per-cell Selects (one per the_work entry), scoped to
the cell's projects; legacy single-cell drafts keep the one-Select path.
- use-prompter.ts: updateBatchDraftProject edits per-cell (entryIndex); confirmBatch validates every cell; batchFromEvent parses per-cell map.
Also adds the missing pnpm format infrastructure (the panel had no formatter
at all): prettier devDep + .prettierrc.json (default-style config: 80-col,
double-quote, semi, trailing-comma-all) + .prettierignore, plus format /
format:check scripts. Only the 3 changed files above were reformatted; the
~222 pre-existing non-compliant files are left untouched (a wholesale reformat
is a separate explicit decision, not bundled into this feature).
* [fix] MegaTask verification: migration 052 enum + async cell-map read
Two real bugs surfaced running the full gate against a containerized
Postgres (and the orchestrator boot log):
1. Migration 052 crashed a real orchestrator boot with
'type "team" already exists'. The generic sa.Enum(create_type=False)
does NOT set the postgres enum's create_type attribute, so op.create_table
(checkfirst=False) emitted a redundant CREATE TYPE against the pre-existing
team enum. Switched to postgresql.ENUM(create_type=False) — the postgres-
native enum whose create_type _check_for_name_in_memos actually reads, so
the CREATE TYPE is suppressed. Verified: 051->052 upgrade against a DB where
the team enum pre-existed (the exact path that crashed) now succeeds;
downgrade 052->051 drops the table and preserves the shared enum; fresh
upgrade head clean. (Migration 016 has the same latent sa.Enum pattern but
never re-runs in prod, so it's noted, not touched here.)
2. _ensure_branch_for_task read task.cell_projects (lazy=selectin to-many)
directly, tripping MissingGreenlet on a freshly-created/unqueried task —
which then poisoned the async session (PendingRollbackError). Replaced with
_task_has_cell_map: peeks InstanceState.unloaded (no IO) and reads the
already-loaded map, falling back to an awaited count query only when the
relationship is genuinely unloaded. Non-ORM stubs route to the plain
attribute. Fixes 2 integration tests; the 6 cell-map unit tests still pass.
Also: typed the self stub as Any in test_choreographer_subtask_project
(mypy tests/ wants Choreographer, not SimpleNamespace) — the codebase idiom.
Gate: ruff format/check clean; mypy roboco/ + tests/ clean; full pytest
10371 passed / 388 skipped against containerized pgvector:pg16; vulture clean.
Pre-existing xenon C-rank on reassign (from prior commit
|
||
|
|
5612375cba |
Feat/v0.13.0 (#270)
* feat(release): add release-manager feature flag (default off) * feat(release): change classification + semver-bump derivation * feat(release): readiness audit (changelog/version-ref/docs/migration/gate) * feat(release): release-manager engine proposes a gated release * feat(release): fail-closed release executor (bump, gate, publish) * feat(release): CEO approve/reject release-proposal surface * docs(release): document the gated release manager * feat(memory): add org-memory feature flags (default off) * feat(memory): add playbooks table + status enum + migration * feat(memory): playbook service with auditor curation transitions * feat(memory): playbooks RAG index plugin * feat(memory): index a playbook into RAG on approval * feat(memory): distill a high-signal lesson at task completion * feat(memory): keep private journal reflections out of the shared RAG corpus * feat(memory): draft_playbook verb + auditor curation verbs * fix(ci): resolve mypy tests/ errors blocking the gate (UUID casts, annotations) * feat(memory): auto-inject similar lessons/playbooks into the briefing * feat(memory): auditor playbook review queue (api + panel) * docs(memory): document the org-memory loop + playbook verbs * fix(provisioning): idempotent pitch provisioning (reuse product/project by slug on re-approval) * fix(memory): add chunks_playbooks to the chunk schema + isolate release route tests - Migration 030's CHUNK_TABLES was missing chunks_playbooks, breaking the IndexType<->migration parity guard once the PLAYBOOKS index landed. The upgrade is ALTER ... IF EXISTS so adding it is safe on any DB shape. - The release-route fixture's approve/reject paths call db.commit() (real behavior), so a held proposal outlived the per-test rollback and leaked into engine tests that read the global list_open_release_proposals(). Tear down source=release_manager rows after each test. - Make the gather_snapshot real-repo smoke version-agnostic (semver match) so it stops pinning the literal repo version. * chore(release): 0.13.0 * ++ --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com> |
||
|
|
153723406e |
Feat/autonomous maintenance (#264)
* feat(ci-watch): config flags Default-off CI-watch config (mirrors self_heal_*): ci_watch_enabled, ci_watch_default_workflow (ci.yml), ci_watch_interval_seconds (1800), ci_watch_max_open_tasks (3), ci_watch_max_per_cycle (1). Registers ci_watch_enabled in the panel FEATURE_FLAGS. 4 tests. * feat(ci-watch): per-project ci_watch_enabled/workflow (migration 048) Adds projects.ci_watch_enabled (bool NOT NULL default false) + projects.ci_watch_workflow (varchar null) — the per-project opt-in for multi-repo CI-watch. ProjectTable + Pydantic Project fields + migration 048 (off 047_ws_single_active). Real upgrade->downgrade->upgrade chain verified against a throwaway Postgres; 2 ORM round-trip tests. * feat(runtime): prune dangling agent images in the background sweeper Every agent-image rebuild orphans the prior build's layers as an untagged <none> image; across deploys these pile up (the operator hit ~80). The sweeper now runs 'docker image prune -f --filter dangling=true' (dangling only — a tagged image or one backing a running container is never dangling), throttled to settings.image_prune_interval_seconds (default 6h) and gated by image_prune_enabled (default on). Best-effort: any failure is logged, never raised into the sweeper. Mirrors the transcript-retention prune. 4 tests. * feat(ci-watch): source tag + open-task dedupe query CI_WATCH_SOURCE='ci_watch' + TaskService.list_open_ci_watch_tasks(git_url=None): non-terminal ci_watch tasks (the dedupe + open-cap basis), optionally scoped to one repo by git_url — a monorepo registers several cell-projects on one git_url, so dedupe keys on the repo, not the slug. 2 real-PG tests. * feat(ci-watch): multi-project CI telemetry fan-out MultiProjectCITelemetrySource.fetch(projects) reuses the hardened per-project get_latest_ci_conclusion for each opted-in project (passing its ci_watch_workflow or the configured default). Per-project isolation: a GitHub error or absent signal yields NO sample (unknown, never read as green) and never aborts the sweep; only a real conclusion yields a sample (fail→breach, pass→non-breach). self-heal source untouched. 3 tests + self-heal regression green. * feat(ci-watch): engine — fan-out, originate, dedupe, cap CiWatchEngine.run_cycle(projects) mirrors SelfHealEngine: assess via MultiProjectCITelemetrySource, open one PENDING ci_watch fix task per red repo (team=main_pm, assigned_to=main-pm, confirmed_by_human=True so it dispatches without an Approve-&-Start — the |
||
|
Renn F
|
9702955f0c |
chore(release): 0.11.1
Patch release bundling the post-0.11.0 run-hardening + PR-gate fixes: - PMs can re-claim needs_revision coordination roots (runtime/spec claim parity) - finished merges don't respawn-loop when the target branch is gone from origin - no phantom re-delegation from text-vs-id acceptance-criteria ref mismatch - PRECONDITION_OWNERSHIP surfaces as not_authorized, not a tracing gap - the spawn gate suppresses respawns for every parked provider, not just Grok - the Claude session limit is detected from the agent transcript so the park fires - the in-path PR-review gate lands its verdict on product-scoped (root->master) PRs - the gate persists its verdict to notes_structured.pr_review (no stale "passed") Bumps all canonical version refs (pyproject / uv.lock / panel package.json / __init__ / config.app_version + README / deployment / agent-image-tag examples). |
||
|
|
fe6c8e387f |
docs: sync prompts/RAG/CLAUDE + bump to 0.11.0 (run-hardening wave) (#254)
* docs: sync prompts/RAG/CLAUDE + bump to 0.11.0 for the run-hardening wave
Documentation + version sweep for everything shipped since
|
||
|
|
c09cf80b40 |
Feature/observability gateway health (#247)
* feat(observability): revision_count + audit_log query index (migration 045)
Adds tasks.revision_count (the O(1) rework counter — forward-only, existing
rows default 0) and the composite index audit_log(target_id, event_type,
timestamp) that powers the cycle-time and rework reconstruction queries.
Verified the real upgrade/downgrade/upgrade chain on a throwaway pgvector PG.
First task of the 0.10.0 observability dashboards.
* feat(observability): count reworks + attribute qa_fail/pr_fail to the rejector
Every transition into needs_revision increments tasks.revision_count at the
single audit chokepoint (exactly once per bounce, across all paths incl. pr_fail
and ceo_reject), so the rework rate is an O(1) read. A QA or PR-review bounce
also emits a named task.qa_fail / task.pr_fail audit event carrying the
rejector's agent_id, so the per-agent rework scorecard charges the rejection to
the reviewer who made it, not the developer who owns the task.
* feat(observability): cycle-time, bottleneck, rework, and scorecard metrics
MetricsService gains four read methods on the audit_log + tasks data: per-stage
cycle time reconstructed from the transition journey (excluding the named
qa_fail/pr_fail events), bottleneck distribution (cumulative dwell + live parked
counts), rework rate (overall/by-team/by-agent with rejector attribution + cost
via spawn-session task_id), and a fused per-agent/per-cell scorecard. Dataclass
models with to_dict(). Verified against a real Postgres journey.
* feat(observability): cycle-time/bottleneck/rework/scorecard read endpoints
Thin read-only routes on the dashboard router delegating to MetricsService:
/metrics/cycle-time, /metrics/bottlenecks, /metrics/rework, and
/metrics/scorecard/{agent,team}. 404 when an agent scorecard target is absent.
5 route tests (200 + shape + the agent-404 case).
* feat(panel): Delivery observability tab (cycle-time, bottlenecks, rework, scorecards)
A third Metrics tab built on the observability endpoints: a per-stage
cycle-time bar chart, a bottleneck panel (worst stage + cumulative dwell +
live parked counts), a rework panel (rate + by-team + by-agent attribution +
cost), and per-cell scorecards. Reuses Recharts + Card/Badge/Skeleton and the
React-Query hook pattern; observabilityApi mirrors usageApi with mock-mode
fallbacks. tsc + eslint clean; 113 panel tests pass.
* docs(observability): changelog + CLAUDE.md for the delivery dashboards
* feat(gateway-health): recover a broken-but-alive agent instead of protecting it
The verb-heartbeat cannot tell a quiet-healthy agent from one whose MCP gateway
is broken (a corrupted /app/.venv firing no verb) yet whose container is up — the
reaper's live-skip would shield it forever. The reaper now probes the gateway
out-of-band (docker exec: does the gateway venv import its deps?) and, once it
has been broken past gateway_health_grace_seconds (tolerating a transient probe
miss), kills + evicts the container so it falls through to release + respawn.
Probe-inconclusive or healthy spares the container. Gated by
gateway_health_enabled (default-on reliability fix; in the panel Feature Flags).
Defers the optional agent-side self-check + full registry re-adoption — the
reaper's docker-liveness fallback already recovers a broken-after-restart agent.
* docs(gateway-health): changelog + CLAUDE.md for broken-but-alive recovery
* docs(observability): user-facing docs for the Delivery dashboards + gateway-health
Documents the new Metrics -> Delivery tab (cycle-time, bottlenecks, rework with
rejector attribution, cell scorecards) in the panel guide and the operations
health-and-metrics guide, and adds the gateway-health env vars + an agent-gateway
recovery note. Published MkDocs site only; settings.md's default-off flag table
intentionally omits the default-on gateway-health flag (same as overload-break).
* chore(release): cut 0.10.0 (changelog section + version refs)
* fix(gateway): exempt PM coordinators from single-task claim guards
A Main/Cell PM plans and delegates many root tasks in parallel; the work
then runs in the delegated cells, not in the PM's own hands. But the
claim-time concurrency guards meant for developers — already_active and
paused (the latter firing after i_am_idle auto-pauses the PM's own
umbrella) — were applied to the PM too, so once it held one root it could
never plan a second: it thrashed between its claimed roots and respawned
forever, burning tokens for zero progress.
_run_claim_guards now skips already_active/paused for the coordinator PM
roles (_COORDINATOR_ROLES = {main_pm, cell_pm}); only unmet_dependency — a
real upstream sequence constraint, which parks the root back to pending —
still gates a PM. paused_tasks_guard also excludes the target task itself,
so a PM re-entering its own paused umbrella never self-blocks.
Tests: a coordinator plans a second root with one in_progress + one paused
sibling (full path + claimed-recovery path), the paused target exclusion,
and the developer guards still fire. Repurposed the pre-fix test that
asserted the now-removed PM block.
* fix(metrics): coerce SQL avg/extract hours aggregates to float (panel toFixed crash)
EXTRACT(epoch ...) returns numeric on PostgreSQL 14+, which asyncpg surfaces
as a Decimal; a Decimal serializes to a quoted JSON string, so the panel's
avg_cycle_hours.toFixed(1) (and the other hours fields) threw 'toFixed is not
a function' and blanked the Delivery tab.
A single _as_hours helper now rounds every SQL-averaged hours field to a real
float — avg_cycle_hours on the new scorecards plus the pre-existing
avg_completion_hours / avg_blocked_hours / longest_blocked_hours. Token and
cost fields were already float()-cast and are unaffected.
Regression test asserts _as_hours coerces Decimal -> float and preserves the
None/zero behavior.
* feat(panel): edit a task's sequence from the details page
A task's sequence (order within siblings, lower runs first) was display-only
with no way to change it from the UI, and TaskUpdate didn't carry the field
so PATCH couldn't set it either. The details page's Dependencies tab now has
an inline sequence editor mirroring the parent / dependency editors, and
PATCH /tasks/{id} accepts a sequence field (owner or privileged role) through
the existing generic update path.
* fix(mypy): green the full make-quality type gate
make quality runs 'mypy roboco/ tests/', which the per-module checks on the
0.10.0 branch never exercised. Two issues surfaced:
- The coordinator-exemption change added role_str to
Choreographer._run_claim_guards but not to the ChoreographerHelpers
protocol base, so the composed Choreographer had incompatible base-class
signatures. Sync the protocol signature.
- The gateway-health / stale-reaper tests stubbed methods by direct
assignment (orch._m = AsyncMock()) and typed their duck-typed task doubles
as object, tripping method-assign / assignment / attr-defined. Switch to
monkeypatch.setattr (keeping a local mock ref for the assertions) and type
the doubles as Any — no type: ignore.
Full mypy roboco/ tests/ clean (785 files); the 21 runtime tests pass.
* fix(metrics): static cycle-time SQL — clear bandit B608 (CI gate)
The cycle-time query interpolated an optional team clause into the text() SQL
via an f-string, which bandit flags as B608 (hardcoded SQL) and turned the
merge gate red. The team value was always a bound parameter, so it was a false
positive — but the f-string is the trigger. Rebuilt as one static query with
(CAST(:team AS text) IS NULL OR a.details->>'team' = :team) and an always-bound
team param (CAST, not ::text — SQLAlchemy's :param parser collides with
PostgreSQL's :: cast operator, which broke the query as a stray param).
Full make quality green vs a real pgvector PG (all 21 gate steps).
---------
Co-authored-by: Renn F <rennf93@users.noreply.github.com>
|
||
|
Renn F
|
6456322746 |
chore: bump version to 0.9.0
Bump the canonical version refs (pyproject, roboco.__init__, config.app_version, panel/package.json, uv.lock) plus the version-pin examples in the docs. The release tag + CHANGELOG date are deferred to the actual 0.9.0 cut. |
||
|
Renn F
|
28bb3b4374 |
docs: drop the unowned roboco.dev custom domain; serve on github.io
roboco.dev is not ours, so the docs.roboco.dev custom domain can never resolve. Remove the docs/CNAME and the custom-domain site_url, and point the advertised docs URL at the free GitHub Pages project URL (https://rennf93.github.io/roboco/) — no DNS required. |
||
|
Renn F
|
8e87506da4 |
docs: deploy via GitHub Pages Actions; serve at docs.roboco.dev
The gh-pages branch deploy (mkdocs gh-deploy --force) raced GitHub's built-in branch deployment and got canceled, and each force-push wiped the custom-domain CNAME. Switch to GitHub's official Pages Actions flow (build -> upload-pages-artifact -> deploy-pages) with a single 'pages' concurrency group, so there is one deterministic deployment and no branch to force-push. - Set the custom domain to docs.roboco.dev (site_url + a docs/CNAME that ships in the build artifact, so the domain persists across deploys). - Point the advertised docs URL at https://docs.roboco.dev across README, the usage/deployment stubs, the Makefile help, pyproject, and CLAUDE.md. - Requires a one-time Settings -> Pages -> Source = "GitHub Actions"; the gh-pages branch is no longer used. |
||
|
Renn F
|
2fb63fed1f |
docs: add the user-facing MkDocs documentation site
Build a complete user-facing documentation site (MkDocs Material) under docs/, served at roboco.dev/docs via a new gh-pages deploy workflow. - Sections: Get Started, The Company, the Tour, Operating the Panel, Choosing & Running Models, Cost & Observability, Optional Subsystems, Configure & Deploy, API Reference, Troubleshooting & Security (55 pages). - mkdocs.yml (Material theme; excludes the agent-facing rag/ corpus, internal scratch, and orphaned stub trees) and .github/workflows/docs.yml (mkdocs gh-deploy to gh-pages). - Retire the stale root usage.md and deployment.md to redirect stubs into the site. - Fix the docs tooling: add the pymarkdownlnt dependency + .pymarkdown.json, run serve-docs/lint-docs/fix-docs under the docs extra, add a build-docs strict gate. - Fix the roboco console-script entry point (cli, not the un-awaited async main). - README: correct the project-structure tree (optimal.py, alembic) and link the docs site. |
||
|
Renn F
|
c49dcebae4 |
fix(toolchain): raise requires-python floor to 3.13 so the resolver matches the runtime
RoboCo's code imports tomllib (3.11+) and the stack runs on 3.13, but requires-python declared >=3.10. The toolchain resolver picks the lowest satisfying version, so it provisioned agent workspaces of the self-hosted roboco-api project at Python 3.10 — an interpreter the suite cannot even be collected under, leaving the workspace .venv unusable and the gate running in an ad-hoc fallback env. Raising the floor to >=3.13 makes resolve_target_python return 3.13, matching the agent image. Re-locks to drop the now-unreachable 3.10-3.12 backports; a guard test pins the repo's own resolution to 3.13. |
||
|
|
16789c1ca7 |
Feature/architectural conventions standard (#243)
* feat(conventions): standard schema models + effective-map merge * feat(conventions): tree-sitter Python classifier + placement checks * feat(conventions): TS classifier, hygiene/custom checks, runner + CLI * feat(conventions): ROBOCO_CONVENTIONS_ENABLED flag + cache table + migration * feat(conventions): repo auto-scan + scaffold draft renderer * feat(conventions): ConventionsService (cache/baseline/ambient/scaffold/restore) * feat(conventions): auto-scaffold on project registration (flag-gated) * feat(conventions): TaskDescription.constraints + auto-baseline attach * feat(conventions): ambient architecture-map injection at spawn * test(conventions): subprocess CLI smoke for the agent-image entrypoint * feat(conventions): block i_am_done on block-level convention violations * feat(conventions): block pr_pass on unresolved convention violations * feat(conventions): surface convention findings into QA evidence * docs(prompts): convention awareness for PO/Intake/Dev/QA/PR-reviewer * feat(conventions): panel Conventions tab + flag toggle + parity * test(conventions): end-to-end block, fix, and waiver through the gate * refactor(conventions): extract pr_pass guards to keep pr_gate under the gate * style(conventions): format the baseline-constraints attach in task.create * test(conventions): type-annotate test helpers for the full mypy gate * build(conventions): ignore types-PyYAML in deptry (mypy-only type stub) * docs(conventions): document the standard in CLAUDE.md + PM prompt awareness * fix(conventions): baseline constraints are non-suppressible (dedup-append) * feat(conventions): scaffold on first workspace clone (threaded workspace) * feat(conventions): multi-project ambient map for PO/Intake (per-product) * feat(conventions): persist findings + violations-feed route (migration 044) * feat(conventions): panel violations feed in the Conventions tab * test(conventions): intake-spawn mock accepts the ambient layer kwarg * fix(docker): ollama-init best-effort pull, gate startup on cached models present A degraded/slow ollama registry made the model manifest re-check fail under set -e, so ollama-init exited 1 and blocked the orchestrator's service_completed_successfully gate — taking the whole stack down even though both models were already cached. Pulls are now best-effort; success is gated on the models being present, so a flaky registry can't down a cached deployment. * refactor(content): drop dead TaskDescription.with_baseline_constraints The structured baseline-merge helper had zero production callers. Project-task baseline constraints are attached by the wired string backstop (TaskService._attach_baseline_constraints), and a real task is free-form prose that cannot form a valid TaskDescription (requires a non-trivial objective + non-empty the_work), so the helper was unreachable from any live path — a leftover from the structured-merge -> string-append design pivot. Removing it leaves a single enforcement path. The constraints field itself stays: it is a member of the well-formed-spec schema (Objective / What This Builds / The Work / Notes / Constraints / Acceptance Criteria), rendered by render_markdown and unit-tested. --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com> |
||
|
|
01e10ad693 |
Feat/provider overload break (#242)
* feat(conventions): standard schema models + effective-map merge * feat(orchestrator): park provider on persistent server overload (529/500) A 429 rate limit already parks a provider — queue its spawns, probe until it recovers — but a persistent 529/500/503 overload had no such break: the run died and the orchestrator crash-retried straight back into the overload, burning tokens in a respawn loop. Generalize the park to provider-unavailability. On a non-graceful Anthropic agent exit, match the API's overload markers (overloaded_error / internal_server_error / "API Error: 5xx") against the dead container's own output and park the provider with kind="overloaded"; the existing spawn gate already queues any parked provider, and the probe-resume loop revives the task when it recovers. Grok keeps its exit-75 path; both now route through one _park_provider_unavailable helper. Markers are kept specific so an agent that merely writes about HTTP 500/529 can't trip the break. Fix the recovery probe to require a 2xx: it treated any non-429 as recovered, so a probe that itself got a 529 would have resumed agents straight back into the overload — wrong for the new path and for a 429 that lifts into a 5xx. Gated by ROBOCO_OVERLOAD_BREAK_ENABLED (default on; off => crash-retry). --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com> |
||
|
Renn F
|
9274cd4584 |
feat(toolchain): resolve target interpreter from requires-python
Pure resolver (services/toolchain.py) that derives the Python version an agent should provision a target workspace with. Defends against uv's resolution order — a .python-version file overrides requires-python during interpreter selection — by honoring the pin only when it satisfies requires-python, else resolving a concrete version from requires-python for the caller to pass via --python. This is the root cause behind the live guard-core-app failure (pin 3.13 vs packages needing 3.14). Promotes packaging to a direct dependency. |
||
|
Renn F
|
818333f626 | chore(release): 0.8.0 | ||
|
|
028b49161b |
[6d7fb817] chore(deps): upgrade pydantic-settings from 2.14.1 to 2.14.2 to fix GHSA-4xgf-cpjx-pc3j (#238) (#240) (#241)
- Add pydantic-settings>=2.14.2 constraint in pyproject.toml - Regenerate uv.lock: pydantic_settings-2.14.1 -> pydantic_settings-2.14.2 - Fix pre-existing xenon CC=12 in _fetch_latest_ci_run (self-heal CI signal commit introduced the complexity): extract HTTP retry loop into _get_ci_runs_response helper, bringing both methods to rank B All acceptance criteria verified: uv.lock shows 2.14.2, pip-audit --ignore-vuln CVE-2025-3000 exits 0 with no GHSA-4xgf-cpjx-pc3j mention, make gate (ruff/mypy/xenon) exits 0, no GHSA suppression in Makefile or pyproject.toml. Co-authored-by: Backend Developer 1 <be-dev-1@agents.roboco.dev> |
||
|
Renn F
|
01e082ff63 |
chore(release): 0.7.0
Roll up everything since 0.6.0 into the 0.7.0 CHANGELOG and bump the version (pyproject / __init__ / config.app_version). Rewrite the stale Unreleased Grok entry — which described the now-deleted opencode runtime — to the shipped reality: Grok agents on xAI's official grok CLI on a SuperGrok subscription, plus the token auto-refresh, the self-healing CI loop, the Company Scorecard, and the pr-reviewer / observability / usage / path-injection fixes. Also folds the uv.lock claude-agent-sdk spec sync (>=0.2.105) merged via #216. |
||
|
|
5ea2f82f64 |
chore(deps): update claude-agent-sdk requirement (#216)
Updates the requirements on [claude-agent-sdk](https://github.com/anthropics/claude-agent-sdk-python) to permit the latest version. Updates `claude-agent-sdk` to 0.2.105 - [Release notes](https://github.com/anthropics/claude-agent-sdk-python/releases) - [Changelog](https://github.com/anthropics/claude-agent-sdk-python/blob/main/CHANGELOG.md) - [Commits](https://github.com/anthropics/claude-agent-sdk-python/compare/v0.2.101...v0.2.105) --- updated-dependencies: - dependency-name: claude-agent-sdk dependency-version: 0.2.104 dependency-type: direct:production dependency-group: dependencies ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
|
fa3e25e656 |
feat(grok): pluggable agent providers + Grok on the official grok CLI (#218)
* feat(providers): pluggable agent providers + Grok (xAI) backend
Add a roboco/llm/providers/ seam — an AgentProvider lifecycle ABC and a
ProviderRegistry keyed by ModelProvider — so the orchestrator can drive
agent backends other than Claude Code.
The first non-Claude backend is GrokProvider for xAI's grok-build-0.1.
xAI is OpenAI-compatible only (no Anthropic-Messages endpoint), so a Grok
agent runs an OpenAI-protocol runtime pointed at https://api.x.ai/v1
rather than the ANTHROPIC_BASE_URL injection the other providers use. It
reuses the orchestrator's existing mount/auth assembly, so it inherits the
same MCP gateway + tool-manifest wiring as every other agent by
construction, and passes its prompt via env (never an argv positional).
The change is purely additive: only GROK routes through the registry;
Anthropic / Ollama Cloud / self-hosted spawns run the existing
_spawn_container path unchanged.
Includes:
- ModelProvider.GROK (migration 038) + a seeded Grok provider row
(migration 039) + a grok-build-0.1 catalog entry
- GET/PUT /api/providers/grok-key to store the xAI key (Fernet-encrypted,
reusing the existing provider-key machinery)
- ClaudeCodeProvider reference adapter over the current spawn
- unit tests for the registry, GrokProvider (gateway wiring, no
ANTHROPIC_* leak, prompt-injection safety, failure paths) and routing
The dedicated roboco-agent-grok image and the exact OpenAI-protocol CLI
invocation are the remaining piece to finalise with xAI.
* feat(providers): native Grok runtime — opencode image, config gen, panel key
Complete the native Grok (xAI) path so grok-build-0.1 runs as a real
RoboCo agent, not just the provider seam.
- roboco-agent-grok image (docker/agent-grok.Dockerfile): FROM agent-base
+ opencode (the OpenAI-protocol runtime). One image serves every role;
role behaviour comes from the mounted manifest / mcp-config / system
prompt, exactly as on the Claude path.
- Entrypoint renders opencode.json at spawn from the GrokProvider env
contract + the mounted Claude Code mcp-config.json
(roboco.llm.providers.opencode_config): translates RoboCo's gateway
servers (roboco-flow / roboco-do / ...) into opencode's mcp block,
declares the xAI OpenAI-compatible provider + model, and wires
permissions + instructions. Pure, unit-tested translation.
- Orchestrator registers GrokProvider with the registry-qualified image
(_qualify_agent_image) so it resolves in local and registry deploys.
- Compose (both files + the registry compose) gain an agent-grok-image
builder service.
- Panel: a Grok (xAI) API key card on the AI Providers page, plus the
grok ModelProvider value.
KNOWN PARITY GAP (opencode runtime): the bash-guard PAT-scrub and the
transcript-based usage/cost capture are Claude Code hooks and do not
transfer to opencode. bash permission is operator-tunable
(ROBOCO_GROK_BASH_PERMISSION) so a deployment can fail closed until a
security/usage-parity opencode plugin lands. That plugin and live E2E
validation are the remaining work to finalize with xAI.
* ci(release): build + publish the roboco-agent-grok image
Add roboco-agent-grok to the release workflow's image build/publish map so
registry deploys carry the Grok runtime image (parity with every other
agent image). Split from the feature commit because pushing a workflow
change requires a workflow-scoped token.
* fix(migration): commit the grok enum value before seeding (autocommit_block)
CI's "Apply database migrations" failed with asyncpg
UnsafeNewEnumValueUsageError: alembic runs the whole upgrade in a single
transaction, so migration 039's INSERT used 'grok' in the same transaction
that 038 added it — which Postgres forbids. Splitting into two migration
files did not help (one transaction spans both). Wrap the ALTER TYPE ADD
VALUE in op.get_context().autocommit_block() so the value commits before 039
(and any later migration) uses it. Still renders in offline --sql, so the
enum-migration-parity test is unaffected.
* feat(grok): price grok-build-0.1 + secret-scrub opencode plugin
- pricing.py: add grok-build-0.1 rates ($1/1M input, $0.20 cached, $2/1M
output), verified against xAI's published pricing. Grok is a priced
non-Anthropic model, so cost computes the moment usage is captured.
- secret-scrub.js: an opencode tool.execute.before plugin porting the
security-critical bash-guard deny rules (git network ops, credential-file
reads, /proc env, internal-host HTTP, roboco.* imports, ROBOCO_AGENT_ID
forgery, env dumps, destructive rm) to the opencode runtime — restoring the
guard the Claude Code hook can't provide there. Throwing denies the call
(confirmed by opencode's env-protection example). Wired into the generated
opencode.json plugin array + baked into the grok image.
Deny logic verified via node (9 deny + 5 allow cases). UNVALIDATED against a
live opencode runtime: confirm it fires in the live E2E spawn before a Grok
dev-agent touches a real repo; the bash permission is operator-tunable as a
second gate.
Cost CAPTURE (distinct from pricing) is intentionally NOT built yet: opencode's
plugin hooks expose model info but no token/usage object, so the capture path
is unconfirmed and needs the live spawn to settle.
* feat(grok): read opencode session usage for cost capture
Confirmed by inspecting a local opencode run: opencode persists per-session
usage in SQLite at ~/.local/share/opencode/opencode.db — the `session` table
carries cost + tokens_input/output/reasoning/cache_read/cache_write. xAI's
response usage object (prompt_tokens, completion_tokens,
prompt_tokens_details.cached_tokens, completion_tokens_details.reasoning_tokens)
maps directly onto those columns.
Add opencode_usage.read_session_usage / cost_for_session: read the opencode DB
and price the tokens via roboco.billing.pricing (our cost stays authoritative;
opencode's own `cost` column is kept for reference). Tested against a fixture DB
mirroring the real schema (single session, summed sessions, missing/empty DB).
Remaining wiring (for the live spawn): mount the opencode data dir on grok
spawn + call cost_for_session at reap to record the usage rollup.
* fix(grok): correct opencode provider (Responses API), stdin, reasoning cost
A live opencode run against api.x.ai/v1 surfaced three real bugs:
1. Provider package — grok-build-0.1 is driven via the OpenAI Responses API
(opencode calls model.responses()). @ai-sdk/openai-compatible is
chat/completions only and errors "responses is not a function". Switch the
generated opencode.json provider + the grok image to @ai-sdk/openai.
2. Headless hang — `opencode run` blocks after init without a TTY; close stdin
(`< /dev/null`) in the entrypoint so it proceeds to the model call.
3. Reasoning-token cost — grok-build-0.1 is a reasoning model; reasoning tokens
bill as output but opencode stores them in a separate column. cost_for_session
folds tokens_reasoning into output (else ~22x undercount).
Verified end-to-end against a real session row (input=6120, output=1,
reasoning=226, cache_read=1856): our pricing reproduces opencode's stored USD
cost ($0.0069452) exactly. Tests anchored to that real row.
* feat(grok): first-class xAI/Grok routing mode (UI + backend)
The Routing-mode toggle had Anthropic / Ollama / Self-Hosted / Mix but no way
to route the whole org to Grok. Add it end to end:
- backend: apply_mode("grok") + _apply_grok (GLOBAL default -> grok-build-0.1) +
derive_mode "grok" detection; ApplyModeRequest/ModeResponse accept "grok".
- panel: a "Grok" routing-mode card (between Anthropic and Ollama, gated on the
xAI key) + flipToGrok; a Grok group in the per-agent mix dropdown +
catalogGrokOnly + a grok ProviderBadge variant; the mix-save key check and
the AI-routing description now cover Grok.
- tests: integration derive_mode/apply_mode "grok" cases (+ grok provider row
in the fixture).
Gated: ruff + mypy clean; panel typecheck + lint clean.
* feat(grok): reasoning-effort by role (cut grok-build cost on cheap roles)
grok-build-0.1 reasons heavily by default and reasoning bills at the output
rate (a live "say ok" call emitted ~300 reasoning tokens, ~85% of its cost).
Confirmed live that opencode's `--variant minimal` cuts reasoning ~54%
(298 -> 136 tokens, same prompt).
GrokProvider now picks reasoning effort by role: code-quality roles (developer,
qa, pr_reviewer) keep full reasoning; coordination / docs / board roles
(cell_pm, main_pm, documenter, product_owner, head_marketing, auditor, prompter,
secretary) run "minimal". It's passed to opencode via the entrypoint's
`--variant`. Operators can force one effort for ALL grok agents with the
ROBOCO_GROK_REASONING_EFFORT env (minimal | high | max, or default/full).
Tests cover the role map, the env override, and the spawn env wiring.
* style(panel): show the Grok (xAI) key card above the Ollama card
* fix(grok): stop opencode subagent-stream hang at the config layer
The Grok pr_reviewer wedged in_progress forever: opencode's default agent ran
with the subagent `task` tool enabled, spawned an Explore subagent on
grok-build-0.1 whose model call opened an SSE stream that went idle, and the
run hung with no timeout.
- Hard-disable opencode's subagent `task` tool in the generated opencode.json.
No RoboCo role uses opencode-internal subagents — work flows through the
gateway verbs — so removing the tool kills the hang trigger outright.
- Set provider.xai.options.timeout + chunkTimeout (operator-tunable via
ROBOCO_GROK_REQUEST_TIMEOUT_MS / ROBOCO_GROK_CHUNK_TIMEOUT_MS) as the
defence-in-depth backstop; chunkTimeout aborts an idle stream.
- Bundle the permission + timeout + subagent knobs into an OpencodeGuards
dataclass (keeps the builder under the arg-count gate).
- Drop the dead ROBOCO_AGENT_TOOLS spawn env (it had no consumer); opencode
tool restriction lives in the rendered config now.
* feat(grok): reaper watchdog kills wedged opencode containers
The heartbeat reaper deliberately skips a task whose assignee holds a live
ACTIVE container, so a Claude agent deep in a long edit/test cycle isn't
churned out from under live work. A wedged opencode container breaks that
assumption: it stays ACTIVE while firing no gateway verb, so its heartbeat
never advances and the live-instance skip would shield its task forever — the
exact way the Grok pr_reviewer parked in_progress.
Add a longer grok-idle kill threshold (ROBOCO_GROK_IDLE_KILL_SECONDS, default
900s, well past the stream chunk timeout). A GROK instance idle past it is
force-removed (its logs dumped to disk first) and evicted from the instance
registry, so the same reaper pass then releases the task. Only GROK runtimes
are eligible — a quiet Claude agent keeps the heartbeat-skip protection.
* feat(grok): guard interactive roles from GROK routes (interim)
intake (prompter) and secretary run a held-open chat session driven by the
Claude Agent SDK. GROK has no interactive runtime yet, and a GROK route for
those slugs would be spawned with the route creds injected as ANTHROPIC_*
against api.x.ai/v1 — the wrong protocol — producing a silent, empty reply
(the blank intake we observed).
Downgrade a GROK route for intake-1/secretary-1 to the Anthropic default with
a logged warning. The one-shot delivery roles route to GROK unchanged. This
guard is replaced by the real interactive fork once the opencode interactive
driver lands.
* feat(grok): capture one-shot Grok usage/cost from the opencode store
A GROK agent runs opencode, not Claude Code: it has no SDK /usage/status
server and writes no Claude transcript, so _resolve_final_token_usage found
nothing and every Grok agent finalized at 0 tokens / $0 — the opencode_usage
reader existed but had no caller.
- Mount a per-agent opencode data dir ($DATA/opencode/<agent_id> →
/home/agent/.local/share/opencode) so opencode.db is captured, and mount the
same host dir into the orchestrator (/data/opencode) in all three compose
files so the finalizer can read it back — the opencode analogue of the
mounted Claude transcript.
- _resolve_final_token_usage branches on provider_type: GROK reads opencode.db
via opencode_usage (reasoning folded into output, billed at the output rate)
and skips the SDK/transcript path. A 0-token read logs a WARNING so a silent
mount failure isn't mistaken for a real zero-cost run.
- ROBOCO_OPENCODE_DATA_DIR overrides the in-orchestrator path for local runs.
* feat(grok): make interactive spawns first-class on AgentProvider (additive)
The AgentProvider ABC modelled only the one-shot lifecycle (spawn/stop/
health_check/remove), so the interactive intake/secretary roles could never
route through a provider. Add an opt-in interactive surface:
- supports_interactive class flag (default False).
- InteractiveSpawnSpec: the resolved AgentConfig + session id + role-specific
image + optional HMAC token — everything a provider needs without importing
orchestrator internals.
- spawn_interactive(spec): a non-abstract default that declines via
ProviderError, so every existing one-shot provider is unchanged.
Pure scaffolding — no provider opts in yet (GrokProvider flips the flag when
its interactive driver lands). Zero behavioural change.
* feat(grok): Grok-native interactive runtime (opencode serve) — container side
Builds the Grok analogue of the Claude intake/secretary live-session runtime,
satisfying the same IntakeSession seam so the existing IntakeDriver loop,
message source, relay, and StreamChunk panel contract are reused unchanged:
- OpencodeServeSession: a held-open `opencode serve` session (context persists
across turns) where each human turn is one synchronous POST /session/:id/
message; normalize_opencode_message maps the reply parts to text/thinking/
tool_use/draft/turn_end chunks (draft via a propose_draft tool part or the
fenced roboco-draft fallback). Doc-verified against opencode's server API.
- grok_intake_main / grok_secretary_main: container entrypoints mirroring the
Claude mains but yielding an OpencodeServeSession; they render opencode.json
(xAI provider + MCP + system prompt) first, then run the receiver + driver.
- roboco-agent-grok-prompter / -secretary images (FROM roboco-agent-grok) +
their builder services in all three compose files.
UNVERIFIED-LIVE: the opencode serve flow + exact Part schema + draft path need
a live run against grok-build-0.1 (the part mapping is defensive). The
orchestrator wiring that routes a GROK intake/secretary route to these images
is the next step (a design decision is open — see the handoff notes).
* feat(grok): route interactive intake/secretary to opencode-serve images
Wire the GROK interactive path the in-place way (matching how the interactive
roles already choose ANTHROPIC_* per route), so a GROK route launches the
Grok-native opencode-serve image instead of the Claude SDK-driver image:
- _spawn_intake_container / _spawn_secretary_container pick the
grok-prompter / grok-secretary image (ensuring the base→grok→interactive
build chain) when the route is GROK, and stamp provider_type on the spec +
AgentConfig so finalize routes usage to the opencode store.
- _build_intake_run_cmd / _build_secretary_run_cmd inject OPENAI_* + the
opencode store mount + system-prompt env for GROK via a shared
_append_interactive_provider_env, keeping ANTHROPIC_* for every other
provider. The intake's minimal mounts (no gateway MCP) are preserved, so
Grok intake matches the Claude intake's tool surface (the spec).
- Add a per-agent opencode store mount to the interactive host paths so
interactive Grok usage/cost is captured like the one-shot path.
Removes the interim Phase-0 routing guard (the real path supersedes it) and
retires the unused AgentProvider.spawn_interactive/InteractiveSpawnSpec seam —
the interactive roles have a bespoke assembly that the one-shot provider
surface doesn't fit, so the fork lives in their own builders.
UNVERIFIED-LIVE: end-to-end intake/secretary chat on Grok needs the stack up +
opencode serve confirmed against grok-build-0.1.
* feat(grok): surface intake/secretary in the mix-mode picker; doc guardrail parity
- Panel: add intake-1 (prompter) and secretary-1 to the mix-mode per-agent
routing list so an operator can assign Grok (or Claude) to the interactive
roles from the UI; assigning a Grok model routes them to the opencode-serve
image. tsc + eslint clean.
- opencode_config: correct the now-stale parity note — bash-guard is ported
(secret-scrub.js) and usage/cost is captured (opencode store); the remaining
gap is the budget/loop/stop/prompt-injection hooks, which need a sidecar
plugin (open decision), with ROBOCO_GROK_BASH_PERMISSION as the interim gate.
* test(grok): mypy-clean the reaper watchdog + interactive spawn tests
The CI mypy scope (roboco/ tests/) flagged test-only typing issues my per-file
runs missed: direct method assignment (orch._remove_container = AsyncMock())
trips [method-assign], and a module-level dict[str,str] is invariant against
the dict[str, str|None] the run-spec expects.
- Use monkeypatch.setattr for _remove_container in the watchdog tests.
- Annotate the shared _HOSTS as dict[str, str | None].
Production code unchanged; mypy roboco/ tests/ is green.
* feat(grok): cost-ceiling kill-switch (budget-guardrail parity)
Claude Code's per-agent token-budget hook fires against the SDK :9000 server;
opencode exposes NO usage/budget hook to a plugin (confirmed against its plugin
docs), so the budget kill-switch can't be a plugin/sidecar — the orchestrator
enforces it instead.
_enforce_grok_cost_budget runs each dispatch tick: for every ACTIVE GROK
container it reads cumulative cost from the opencode store (the Phase-2 reader)
and kills + evicts it past ROBOCO_GROK_MAX_COST_USD (0 = off), after which the
reaper releases the freed task. This also catches a runaway loop that keeps
firing verbs (so it evades the idle watchdog) but still burns cost.
Covers the budget/runaway-burn slice of guardrail parity. The remaining Claude
hooks (prompt-injection PRE-gate, stop-guard terminal-verb) have no blocking
opencode equivalent — opencode's message/stop hooks are observe-only — and the
interactive reasoning-variant has no opencode.json/serve knob (CLI-flag only);
both are pinned for a live probe rather than shipped as a guess.
* docs(grok): document Grok's reduced guardrail posture (honest, not blocking)
Grok agents run on opencode, not Claude Code, so they do NOT have full
guardrail parity — claiming otherwise would be false. Document it truthfully
and keep them usable rather than blocking them.
- Panel routing card: an amber caveat shown in Grok/Mix mode — command/
secret-exfil guard + cost cap apply to Grok, but the prompt-injection guard
does NOT (opencode cannot block a turn); Anthropic/Ollama/Self-Hosted run
through Claude Code with the full guard set; prefer those for agents that
ingest untrusted or cross-agent content; Grok is safe for trusted work.
- docs/self/architecture/llm-provider-security.md: the reference — the two
runtimes, which provider uses which, the per-guardrail parity matrix, why
the injection/stop gaps exist (opencode hooks are observe-only), and the
routing recommendation (delivery roles handling untrusted content → a
Claude-Code-runtime provider).
Panel tsc + eslint clean.
* fix(grok): make the live interactive path work — store perms, error surfacing, variant
Found by actually running opencode serve locally (the path was doc-verified but
never executed). Three fixes:
1. EACCES on the opencode store mount (the live intake crash): on Linux docker
auto-creates a missing bind source as root:root, so the non-root agent user
could not mkdir/write in /home/agent/.local/share/opencode and opencode died
at boot. _ensure_opencode_data_dir pre-creates the per-agent dir 0777 before
the mount (one-shot via the _GrokHost seam, interactive in both spawns).
2. Silent blank reply on a model error: opencode reports a turn failure in
info.error with parts=[], NOT as a part — verified live (a bad xAI key
returns info.error APIError). send() / normalize_opencode_message now surface
it as an "error" StreamChunk so a failed turn is never blank (the original
intake bug class). Confirmed live: the error now renders.
3. Reasoning variant on the serve path: the live OpenAPI shows the message body
accepts a "variant" field (it is NOT CLI-only, as the docs implied), so the
pin is unblocked. send() passes ROBOCO_GROK_VARIANT as the per-turn variant;
the orchestrator sets it per-role (_reasoning_effort_for) for interactive
Grok, the same lever as the one-shot --variant.
opencode serve startup, POST /session, session-id extraction, the part-type
mapping (text/reasoning/tool), and the error path are all validated against a
live opencode 1.17.8. A real successful grok reply still needs a funded key.
* fix(grok): pre-create agent-owned ~/.local in the grok image (opencode state EACCES)
Running the built grok-prompter container surfaced a second EACCES the
mechanism analysis missed: bind-mounting the opencode store at
~/.local/share/opencode makes docker create the intermediate ~/.local AS ROOT,
so the non-root agent user then cannot create its sibling ~/.local/state and
opencode dies at boot. Pre-create the ~/.local tree agent-owned in the image so
the mount leaves the parents writable. Complements the orchestrator 0777
host-source pre-create (which covers the bind source on Linux).
Verified live: with this fix the container starts clean, opencode serve opens
the session, a POST /turn produces a real grok reply, and all chunks
(thinking/text/turn_end) reach the relay endpoint.
* feat(grok): prompt-injection guard for Grok (parity with the Claude hook)
The injection guard is RoboCo's own hook (user-prompt-hook.sh), not a runtime
built-in, so it can be recreated at our input boundary regardless of runtime —
opencode's lack of a blocking pre-prompt hook is irrelevant.
- prompt_guard.detect_injection: the deny patterns ported to reusable Python.
- IntakeDriver._run_turn scans every interactive turn before sending it to the
model and denies a match as an error chunk. Covers BOTH Grok (opencode) and
the Claude SDK intake (which runs with setting_sources=[] and so never loaded
the bash hook — it was unguarded too).
- The one-shot grok entrypoint scans ROBOCO_INITIAL_PROMPT and refuses a
poisoned task prompt (parity with the Claude UserPromptSubmit deny).
- Broadened the pattern (Python + the bash hook, kept in sync) to catch the
multi-qualifier canonical phrasing "ignore all previous instructions", which
the single-qualifier original missed — without false-positiving on
"ignore the linting rules" (an intermediate non-qualifier word breaks it).
So Grok now has the command/secret-exfil guard (secret-scrub), the cost cap,
AND the injection guard. Verified: 94 agent_sdk tests pass; bash + Python agree
on detect/miss cases.
* docs(grok): drop the security disclaimers — injection guard closes the gap
With the prompt-injection guard now recreated for Grok (prior commit), the
"Grok lacks the injection guard / prefer Claude for delivery roles" warning is
no longer true, so remove it:
- Panel routing card: replace the amber "prefer Claude / not safe" caveat with
a neutral one-liner — Grok agents run on opencode; the command/secret-exfil
guard, the prompt-injection guard, and the cost cap all apply.
- docs/self/architecture/llm-provider-security.md: prompt-injection row flips to
"yes" for Grok; intro + routing recommendation updated to "effective security
parity, any agent (incl. delivery roles) can run on Grok"; the only remaining
unported hook is the non-security stop-guard.
- opencode_config docstring: the remaining gap is now just the stop-guard
(budget + injection are covered).
Panel tsc + eslint clean.
* fix(grok): allow external-directory reads so the pr-reviewer can work
Live NAS run showed the Grok pr-reviewer claim the review and fetch the diff,
then write it to /tmp and FAIL to read it back: opencode auto-denied
"external_directory (/tmp/*)" — its file tools refuse paths outside the project
cwd, and in headless serve/run mode an "ask" permission auto-rejects (no human).
Add permission.external_directory (default "allow", env
ROBOCO_GROK_EXTERNAL_DIR_PERMISSION) to the generated opencode.json. The
container is the sandbox and secret-scrub still blocks credential-file reads, so
allowing in-container external-dir reads is safe and unblocks legitimate scratch
use (e.g. the pr-reviewer grepping a large diff in /tmp).
Verified live against grok-build-0.1: with external_directory:"allow" the Read
tool reads a file outside cwd and returns its contents (no auto-reject); the
plain-string form is accepted by opencode 1.17.8.
Needs a rebuild of roboco-agent-grok + a pr-reviewer re-run on the NAS to confirm.
* refactor(grok): split eligibility out of _maybe_kill_wedged_grok (xenon C -> B)
CI complexity gate (make quality -> xenon --max-absolute B) flagged
_maybe_kill_wedged_grok at rank C — too many guard branches in one method.
Extract the kill-candidate decision into _wedged_grok_slug(task, last_heartbeat)
-> slug | None (recent-heartbeat / no-owner / not-ACTIVE / not-GROK all yield
None); _maybe_kill_wedged_grok now just kills + evicts the returned slug.
Behaviour is identical (same guards, same order) — the reaper watchdog tests
pass unchanged. xenon now passes on the full package; ruff + mypy clean.
* feat(grok): start the in-container SDK server + budget feed (Claude parity)
The keystone of the Grok parity work (CEO's "take Claude as baseline, create
what's missing" call): the one-shot Grok container now starts the same SDK
server the Claude path runs, so the per-verb circuit breaker (the flow/do MCP
servers already POST /verb/attempted to it), the per-session budget/loop
counters, the terminal-verb tracking, and the SessionEnd post-mortem all work
on Grok instead of being silently absent.
- entrypoint: launch roboco.agent_sdk.server (bare venv python, not `uv run`
which would re-sync the drifted clone lock and stall), wait for /health,
reset counters; run opencode WITHOUT exec so the script regains control to
run the post-mortem and the silent-exit substitute after the run returns.
- budget-feed.js: opencode plugin that gates on /budget/status in
tool.execute.before (halt/loop deny — the only place to stop a runaway
one-shot run; opencode has no PostToolUse-deny) and records the executed
tool + args-hash in tool.execute.after. Fail-open; bare-verb normalization
for MCP-namespaced terminal verbs.
- silent-exit substitute: on a graceful exit with no terminal verb the
entrypoint posts /terminal/force_substitute so the task isn't left stuck
claimed/in_progress (Stop-hook parity at the boundary).
- opencode_config: wire budget-feed into the plugin array; add
ROBOCO_OPENCODE_EXTRA_PLUGINS so per-image role tool plugins load scoped to
one role; read the per-role ROBOCO_GROK_EDIT_PERMISSION.
Targeted gate green (ruff/mypy/xenon + opencode_config tests; node --check on
the plugins; bash -n on the entrypoint).
* feat(grok): give the Grok Secretary its CEO-authority tools (blocker)
The Grok Secretary could chat but had zero directive tools — it could not read
company state or act on a CEO command, so it was non-functional. This is the
integration blocker.
- secretary-tools.js: opencode plugin registering read_company_state /
read_task / submit_directive via the Hooks.tool API, each calling
/api/secretary/* with the container's HMAC agent token — a direct port of the
Claude Secretary's SDK tools (secretary_driver.build_secretary_options). The
high-impact directive kinds stay gated server-side (queued for CEO confirm).
- agent-grok-secretary.Dockerfile: bake the plugin and scope it to this image
via ROBOCO_OPENCODE_EXTRA_PLUGINS, so only the Secretary carries CEO authority.
- grok_secretary_main: correct the docstring that falsely claimed the tools
reached the API "through the mounted MCP gateway" (there is no gateway mount;
they're an opencode plugin).
- secretary.md: name the three tools and restate the confirm-before-act gate.
Verified locally that opencode loads a file-path plugin importing
@opencode-ai/plugin and resolves the package; the live model-tool-call +
backend round-trip is flagged UNVERIFIED-LIVE for the NAS.
* feat(grok): give the Grok Intake its propose_draft tool (draft card)
The prompter prompt tells the model to call propose_draft when the spec is
ready, but on Grok that tool didn't exist — so no draft chunk, no panel draft
card, and the human couldn't launch a task from a Grok intake chat.
- intake-tools.js: opencode plugin registering propose_draft via Hooks.tool;
the execute() only ACKs — the driver (OpencodeServeSession.normalize ->
_is_propose_draft -> _draft_from_tool_input) intercepts the tool CALL and
emits the `draft` chunk the panel renders.
- agent-grok-prompter.Dockerfile: bake the plugin, scoped to this image via
ROBOCO_OPENCODE_EXTRA_PLUGINS (delivery roles never draft).
- test: a propose_draft tool part normalizes to a draft chunk (not a tool_use).
The live tool-call -> draft-card path is flagged UNVERIFIED-LIVE for the NAS.
* feat(grok): scope opencode edit/bash/external-dir permissions per role
Grok wrote ONE global permission block, so a Grok pr_reviewer (or qa / PM /
auditor) ran with edit=allow + bash=allow on untrusted PR content. Now the
permissions are derived per role, mirroring orchestrator._get_role_permissions
on the Claude path:
- edit — allow only roles that write code (role_config.allows_write:
developer / documenter); everyone else edit=deny.
- bash — allow only roles that legitimately run a shell (developer /
documenter / cell_pm / main_pm); the read-only reviewers (qa / pr_reviewer /
auditor) and the board get bash=deny. secret-scrub still guards the rest.
- external_directory — only the pr_reviewer reads scratch outside its cwd (the
/tmp diff); delivery roles get deny.
One-shot roles resolve these in GrokProvider._append_grok_env; the interactive
intake/secretary set edit=deny + bash=deny in the orchestrator (intake keeps
external-dir reads for sibling product repos, the secretary does not). The
Claude path is untouched — the permission env is a GROK-only contract.
Targeted gate green (ruff/mypy/xenon + provider + interactive-spawn tests).
* feat(grok): park the provider on an xAI 429 (break the respawn loop)
A one-shot grok run that hit an xAI 429 exited without a terminal verb; the
dispatcher then re-spawned the same task every tick (429 -> exit -> respawn), a
container/token/cost loop with no living agent to call i_am_blocked.
- entrypoint: detect a rate-limit signature in the run output and exit 75
(EX_TEMPFAIL); a rate-limited task is NOT substituted — it must be retried.
- _handle_stopped_container: on a grok exit 75, park the provider via the
rate-limit tracker (retry_after window) instead of crash-retrying, and don't
count it as a crash. The existing probe-resume loop clears the park after the
window (unknown-provider time-expiry fallback) and the task is retried.
- spawn_agent: a grok-only, fail-open guard skips the launch while the provider
is parked, so the dispatcher no-ops instead of looping. The Claude path is
untouched.
Targeted gate green (ruff/mypy/xenon + new rate-limit tests; bash -n on the
entrypoint).
* feat(grok): close the secret-scrub bash-guard parity gaps
secret-scrub.js (the opencode bash guard) was missing three rules the Claude
bash-guard hook has, leaving a Grok dev able to read secrets the Claude path
blocks:
- source / dot-source of a credential-bearing file (source .env, . ./.env,
.bashrc / .git-credentials / .netrc / /proc/*/environ).
- interpreter one-liner reading a credential file
(python -c "open('.env')", node -e "readFileSync('.git-credentials')").
- git-ops check now runs on a SKELETONIZED command (heredoc bodies + echo/printf
args stripped) so a README/heredoc that merely documents `git push` is no
longer mistaken for invoking it — a false-positive parity fix from the Claude
guard.
Functionally smoke-tested with node against the real plugin (git push denied;
echo/heredoc "git push" allowed; source/interpreter cred reads denied; normal
commands allowed). Live opencode firing stays flagged in the file header.
* fix(grok): record a usage session for interactive intake/secretary (M1+M7)
_spawn_intake_container / _spawn_secretary_container built the AgentInstance by
hand and never recorded an agent_spawn_sessions row, so the reap finalizer had
no usage_session_id to look up — every interactive session (Claude or Grok)
finalized at 0 tokens / $0 in the rollups. Record the session (task_id=None) and
pin its id on the instance, mirroring _launch_spawn; the GROK path reads
opencode.db by this id, the Claude path reads the transcript.
Also correct the grok_intake_main docstring (M7): it claimed the serve process
was "gateway-wired" with an "MCP gateway", but interactive intake mounts no
gateway — its only tool is propose_draft, registered by the intake-tools.js
plugin.
* fix(grok): surface a dead opencode-serve clearly instead of a zombie chat (M2)
If `opencode serve` died after the session opened, every subsequent turn failed
with an opaque httpx connection error while the container lingered. send() now
detects the exited subprocess (returncode set) and yields a clear error chunk +
turn_end so the panel shows a real "session ended — start a new chat" message;
the idle watchdog / a human reap then tears the container down.
* fix(grok): close the panel relay when the cost-cap kills an interactive chat (M4)
_enforce_grok_cost_budget killed + evicted a container directly. For the
interactive roles (intake/secretary) that left the panel SSE relay open with no
close sentinel, so the chat froze with no explanation. Add
PrompterLiveRegistry.close_by_agent (push a final error event, then close every
session bound to that agent) and call it from the cost-cap watchdog when the
killed agent is the intake or secretary, so the panel reports the chat ended on
the cost cap instead of hanging.
* fix(grok): make the opencode runtime actually load — proven live on grok-build-0.1
Live verification (opencode 1.17.8 + grok-build-0.1, funded key) showed the Grok
runtime was loading INERT, three ways:
1. The provider override `provider.xai.npm=@ai-sdk/openai` failed model
resolution (ProviderModelNotFoundError) — opencode can't resolve that package
from its module path. Worse, ANY custom `provider.xai` block (even just
options) breaks plugin-tool registration. opencode's BUILT-IN xai provider
drives grok-build-0.1 with working tool-calls, so emit NO provider block; the
key + base reach it via XAI_API_KEY / XAI_BASE_URL env (provider.options.apiKey
alone does NOT authenticate).
2. Plugins referenced by absolute path in the config `plugin:` array never
registered their hooks/tools. opencode 1.17.8 only registers from the plugin
AUTO-DISCOVERY dir (~/.config/opencode/plugin/). Bake all plugins there.
3. Plugins must use a NAMED export, not `export default`.
Changes:
- opencode_config: no `provider` block, no `plugin` array; drop the dead
XaiTarget + timeout machinery; build_opencode_config now takes a model string.
- GrokProvider / orchestrator interactive env: inject XAI_API_KEY + XAI_BASE_URL
(drop the now-unused OPENAI_*).
- secret-scrub / budget-feed / secretary-tools / intake-tools: named exports;
baked into /home/agent/.config/opencode/plugin/ (drop the EXTRA_PLUGINS env).
- agent-grok* Dockerfiles: plugin dir + agent ownership; drop the unneeded
@ai-sdk/openai global install.
Verified live end-to-end: grok-build-0.1 calls read_company_state AND
submit_directive through secretary-tools.js and the backend receives both with
the agent token; a tool.execute.before guard fires; built-in tool-calls work.
Targeted gate green (ruff/mypy/xenon + opencode_config/providers/interactive
tests; node --check the plugins).
* fix(grok): deliver intake draft via the relay + correct opencode-mechanism docs
Live end-to-end verification (opencode 1.17.8 + grok-build-0.1) of the WHOLE
integration, then fixes for what it surfaced:
1) Intake draft card (FUNCTIONAL): opencode's synchronous serve reply
(POST /session/:id/message) returns only [step-start, text, step-finish] — it
does NOT include tool-call parts, so the driver could never extract the
propose_draft draft. intake-tools.js now POSTs the draft straight to the
prompter-live relay (/api/prompter/live/{session}/events, the same endpoint
the driver's relay sink uses), so the panel renders the card regardless.
Verified live: grok calls propose_draft -> the relay receives the draft.
2) Correct misattributed opencode "bugs" (DOCS): earlier comments asserted as
general opencode behavior that a provider.xai block / npm override / config
plugin:-array "break" registration. Re-testing showed those were artifacts of
a PROJECT-level .opencode/opencode.json; from the GLOBAL config (which
opencode_config writes) the built-in provider, model resolution, the plugin
array AND the auto-discovery dir all work, and MCP gateway verbs register
(delivery agents verified). Reframed the comments as design choices (built-in
provider + XAI_API_KEY env + plugins baked in the auto-discovery dir with
named exports) and dropped the false claims.
3) Reasoning --variant: passing it does not error, but whether opencode applies a
named reasoning variant to grok-build-0.1 (no provider-defined variants) is
UNVERIFIED — comment softened from a "~54% cut" claim to best-effort,
measure-on-NAS.
Verified live this session: one-shot delivery (model + MCP verbs + plugins +
hooks), secretary tools (read_company_state + submit_directive -> backend with
token), intake draft (relay), grok built-in-provider tool-calling. Remaining
NAS-only: full container assembly (SDK :9000 startup, entrypoint hooks, 429
parking) + the --variant cost measurement. Gate green (ruff/mypy + 51 tests;
node --check the plugins).
* feat(grok): reap abandoned interactive chats (M3)
An interactive intake/secretary chat the human abandoned (closed the tab without
confirming or stopping) leaked its container until the orchestrator restarted —
the wedged-grok reaper is task-driven and these run task_id=None, and an SSE
disconnect intentionally does NOT reap (so a page reload can reconnect).
Reap by IDLE TIME, not connection state: PrompterLiveRegistry tracks
last_activity (bumped on every push/deliver = a turn), and the 60s sweeper
retires sessions idle past ROBOCO_INTERACTIVE_IDLE_REAP_SECONDS (default 1800;
0 disables) via reap_intake_session / reap_secretary_session. An active or
page-reloaded chat that keeps exchanging turns stays fresh and is never reaped;
board-review-parked sessions (task_id set) are exempt. Provider-agnostic — fixes
the leak for both Claude and Grok interactive.
Tests: idle-only reap (active/parked/closed excluded), activity bump keeps a
session alive, threshold 0 disables. Gate green (ruff/mypy/xenon + prompter_live).
* fix(panel): resolve agent names from the live roster so they never drift
A review task assigned to the pr-reviewer rendered as a truncated raw
UUID instead of its name. Root cause: the panel resolved assignees from a
hardcoded static roster in agent-utils.ts that had drifted — it never
gained the board-adjacent agents added backend-side (intake-1,
secretary-1, pr-reviewer-1). Their UUIDs hit no map entry, so
getAgentDisplayName fell through to the unknown-UUID branch and returned
agentId.slice(0, 8). Every assignee surface (task table, task detail,
subtasks, journals, communications, commit cards) shares that resolver, so
all of them showed the fragment.
Make the live /api/agents roster the source of truth instead of a static
duplicate that silently rots:
- agent-utils: add a runtime registry (registerAgentRoster) keyed by both
UUID and slug; resolveToSlug / getAgentDisplayName / isKnownAgent consult
it first. The static maps remain only as an offline / first-paint
fallback (now complete with the three agents).
- api/agents: surface the backend UUID on AgentDefinition (getAll/getOne
previously dropped it), so the registry can key by UUID.
- use-agents: add useAgentRosterSync (registers the live roster) and derive
useAgents from live definitions, falling back to the static roster.
- providers: mount the sync once inside QueryClientProvider.
Now any agent the backend knows about resolves, including ones added after
this change — the panel can no longer drift out of sync.
Tests: agent-utils unit tests cover the three agents end-to-end, a
live-roster-only agent (drift-proofing), live-overrides-static, and a
regression guard for the existing roster.
* fix(pr-review): post a COMMENT review when GitHub forbids self-review
A pr-reviewer review of an org-authored PR never reached GitHub. The agent
side ran correctly (claim → read-only diff → review → post_pr_review →
completed + CEO notify), but the GitHub publish 422'd with "Can not request
changes on your own pull request": the PR was authored by the same account
that owns the project PAT. post_pr_review posts best-effort after the DB
transition, so the failure was logged and swallowed — the task completed and
the CEO was notified "reviewed" while the PR showed no review.
GitHub forbids APPROVE / REQUEST_CHANGES on your own PR but DOES allow a
plain COMMENT review. The org's internal PRs (and any PR the PAT owner
opened) hit this. Retry once as a COMMENT review on the self-review 422 so
the review actually lands; the verdict is already stated in the body. The
external/fork-PR path (different author) is unchanged — REQUEST_CHANGES
succeeds there and the fallback never fires.
Tests: self-review 422 downgrades to COMMENT and returns the COMMENT result;
a failing COMMENT retry still surfaces GitError with no infinite loop; the
existing non-self 422 still raises.
* fix(grok): harden cost-guard, pin runtime, refresh stale plugin comments
Address review findings on the Grok provider work:
- budget-feed plugin failed open unconditionally, so a one-shot task agent
whose in-container SDK budget server went unreachable would run with the
cost cap unenforced. The entrypoint now exports ROBOCO_BUDGET_ENFORCE=1
(one-shot agents always start that server) and the plugin's pre-exec gate
fails CLOSED when the flag is set and the budget endpoint is unreachable,
halting an uncapped burn. Interactive serve agents (intake/secretary) set
no flag and keep failing open (they run no budget server by design).
- Pin opencode-ai to the live-verified 1.17.8 (was an unpinned global npm
install). Untrusted model output runs under it; bump the pin deliberately.
- Document the ROBOCO_GROK_* operator vars in .env.example (image, the three
opencode permissions, reasoning effort, idle-kill, cost ceiling).
- Refresh stale plugin comments: the MCP tool-name shape and the secretary
tool-registration path are confirmed live, and secret-scrub's load route is
the auto-discovery dir (not a config plugin: array). Keep the honest
not-yet-exercised caveat on secret-scrub's deny path and the reasoning
variant — those remain genuinely unverified.
* fix(grok): unbreak workspace-cwd agents, free trapped agents, stop self-PR review
Three bugs surfaced by the first live Grok lifecycle run:
- Dev/QA/doc agents crash-looped at startup with ModuleNotFoundError on
roboco.llm.providers. The entrypoint ran the opencode-config render from the
agent's workspace-clone cwd, whose own roboco/ dir shadows /app on the
sys.path front; a branch without the grok code lacks the providers package.
Render from /app so the installed package always resolves (the render has no
cwd dependency — writes global, reads ROBOCO_MCP_CONFIG).
- A budget/loop halt blocked EVERY tool, including i_am_idle, unclaim, and
i_am_blocked, so a halted agent could neither continue nor stop and flailed —
one billed model turn per blocked retry. The before-gate now always lets the
release verbs through so a halted agent can exit cleanly.
- The inbound reviewer ingested the org's OWN PRs (authored by the repo-owner
account), which can't take a REQUEST_CHANGES review (GitHub 422) and get
re-reviewed every poll. The normalizer flags author_is_owner and ingestion
skips them — the reviewer reviews only PRs the org did not author.
External/contributor PRs are unaffected.
Tests: owner-authored PR flagged + skipped; normalize shape covers the new
field. Gate green on the changed modules (ruff/mypy/xenon + 48 tests).
* feat(grok-cli): render config.toml + map per-role grok CLI flags
First piece of the Grok CLI provider that replaces the opencode runtime: a
pure, unit-tested module the agent entrypoint runs to translate the mounted
mcp-config.json into ~/.grok/config.toml ([mcp_servers]) and compute the
per-role 'grok -p' flags — subagent/shell/edit tool removal, raw-git-mutation
and rm-rf denies, reasoning effort — mirroring ClaudeCodeProvider's per-role
permissions with native grok flags instead of an opencode permission block +
JS guard plugins. Uses tomli_w. The rendered config + env injection are
validated live against grok-build (the model called the server through it).
* feat(grok-cli): grok CLI agent image + headless entrypoint
The roboco-agent-grok image now installs xAI's official grok CLI (Grok Build,
pinned 0.2.56) instead of opencode, authenticated by the SuperGrok subscription
via a mounted ~/.grok/auth.json (parity with the Claude ~/.claude mount, no
metered API key). The entrypoint renders ~/.grok/config.toml + per-role flags
from /app (the ModuleNotFound-shadowing lesson), runs grok -p headless with
--output-format json, keeps the prompt-injection guard, and exits 75 on a
rate-limit so the orchestrator parks the provider. No in-container SDK server or
budget-feed — native --max-turns + server-side terminal-substitute replace them.
* feat(grok-cli): GrokCliProvider — subscription auth mount, mirrors ClaudeCodeProvider
Replace the opencode GrokProvider with GrokCliProvider: reuses the orchestrator's
shared mount/auth/git assembly (gateway + identity) exactly like the Claude path,
mounts the host ~/.grok/auth.json read-only (SuperGrok subscription) instead of
injecting an xAI key, and sets the slim env the grok-cli entrypoint + renderer
read (ROBOCO_AGENT_ID for per-role flags, model, mcp-config, prompt). Provider
routing fields are blanked before the shared step so the grok endpoint is never
mislabelled ANTHROPIC_*. Per-role permission logic now lives in grok_cli_config,
so the provider is slim. Registry/orchestrator/exports updated; provider tests
rewritten for the CLI behavior (no XAI key, auth mount present/absent).
* feat(grok-cli): capture per-session token usage + notional cost
Grok runs on the SuperGrok subscription, but — exactly like Claude on Max — we
still record per-agent tokens and a notional cost for the dashboard. The grok
CLI writes a cumulative totalTokens per turn into
~/.grok/sessions/<cwd>/<session-id>/updates.jsonl (the grok analogue of the
Claude transcript / old opencode.db); the max is the session total. This reader
locates that file (url-encoded cwd), extracts the total, and prices it at the
output rate (no input/output split from the CLI; conservative + matches the
reasoning-at-output convention). Validated against a real grok-build session
(18253 tokens -> $0.0365). Entrypoint + finalize wiring follows.
* feat(grok-cli): wire usage capture into the run (session id + post-run extract)
The provider pins a fixed session id (ROBOCO_AGENT_SESSION_ID, reused from the
agent session id as on the Claude path); the entrypoint passes it to
'grok -p -s <id>' so the run's session store is locatable, then runs the usage
reader post-run (best-effort) to write the captured tokens + cost. The
orchestrator-side finalize that reads that file follows.
* feat(grok-cli): read captured usage at finalize; keep interactive serve working
The provider mounts the per-agent data dir and points the entrypoint's usage
file at it; the orchestrator's grok finalize reads that usage.json first (the
grok-CLI total, priced at the output rate) and falls back to opencode.db for the
still-opencode interactive intake/secretary path. Re-add _reasoning_effort_for to
grok.py as a clearly-temporary shim for that interactive path (it needs opencode's
"minimal" variant, distinct from the CLI's --effort) until it is converted too.
* feat(grok): convert interactive intake/secretary to the grok CLI; delete opencode
Move the last Grok runtime off opencode onto xAI's official `grok` CLI, for full
parity with the Claude path. The intake/secretary chat now runs per-turn headless
`grok -p` invocations that resume one session id (proven live: context carries
across runs), with streaming-json deltas mapped to the existing panel StreamChunk
kinds — the IntakeDriver loop, message source, relay, and idle reaper are reused
unchanged; only the SessionFactory differs (GrokCliSession replaces the
opencode-serve session).
- GrokCliSession + a pure, unit-tested streaming-json -> StreamChunk assembler
(thought coalesced to one block, text streamed live, end captures the session
id for -r, fenced-draft fallback, clear errors incl. rate-limit).
- intake propose_draft and secretary read_company_state/read_task/submit_directive
are now FastMCP servers (roboco-intake / roboco-secretary) wired into
~/.grok/config.toml, launched via `uv run --directory /app` to resolve the
installed package. The secretary tools reuse the shared backend helpers.
- Orchestrator: interactive spawn mounts the subscription auth + per-agent usage
dir (no metered xAI key, no permission env — grok flags carry per-role perms);
usage/cost now read a captured usage.json (drop the opencode.db reader, the
_opencode_db_path/_grok_usage_from_opencode methods, and the cost-cap's
opencode read). hosts["opencode"] -> hosts["grok_usage"]; OPENCODE_DATA_DIR ->
GROK_USAGE_DATA_DIR.
- Fix one-shot usage capture: `-s` does not pin the session id (grok generates
its own), so the entrypoint now reads the real id back from the JSON run log
and the reader uses it; usage is captured per-turn on the interactive path.
- Delete the opencode layer: opencode_config/opencode_usage/opencode_session, the
docker/grok/*.js plugins, the old one-shot entrypoint, and their tests.
- Compose (all three files), .env.example, and stale comments updated to the
grok-CLI runtime; add the SuperGrok auth mount + grok-usage dir.
Gate green: ruff, mypy (296 files), xenon, tests. NAS build/verify pending.
* fix(grok): deliver the role blueprint as grok's system prompt via ~/.grok/AGENTS.md
The blueprint was mounted at /app/system-prompt.md but never reached grok — a real
parity gap vs the Claude path (which passes --system-prompt-file). grok agents ran
only on the per-task prompt, missing their RoboCo role/org context.
Verified live on grok 0.2.56 that the obvious flags do NOT work headless:
`--system-prompt-override` and `--rules` are silently ignored under `grok -p`
(identical output with and without). What IS honoured is grok's instruction-file
discovery — and `$HOME/.grok/AGENTS.md` is loaded GLOBALLY regardless of --cwd
(a project AGENTS.md only loads from the cwd/project root, which would pollute the
agent's git workspace). Proven end to end: a blueprint written there makes grok
adopt the role ("I am the RoboCo intake interviewer ... -- intake-1").
write_agents_md() copies /app/system-prompt.md -> ~/.grok/AGENTS.md; the one-shot
render (grok_cli_config.main) and both interactive mains call it. No git pollution
(it lives in ~/.grok, not the workspace), and it covers repo-cwd and /app-cwd
roles alike. Reverted the non-working --system-prompt-override wiring.
* feat(grok): close the Claude-parity divergences (reasoning, subagents, web, bash-guard)
Bring the grok CLI to parity with the Claude path on the four deliberate
differences:
- Reasoning: drop the per-role `--effort low` default — Claude sets no per-role
thinking budget, so grok now uses the model default for every role. The
fleet-wide ROBOCO_GROK_REASONING_EFFORT override stays as a cost lever. (This
also un-caps intake-draft quality, the one that actually mattered.)
- Subagents: the intake interviewer may now fan out to subagents (parity with the
Claude intake's `Task` allowance); every other role still has `Agent` removed.
- Web: `--disable-web-search` for every role — no agent gets direct web (Claude's
tool set has none either); the roles that get web reach it through the gated
roboco-search MCP, unaffected.
- Bash command filtering: full parity, split by deny semantics. Verified live that
a grok PreToolUse hook deny CANCELS the run, while native `--deny` denies
GRACEFULLY (the agent gets a permission error and recovers). So:
* git network/branch/history ops -> native `--deny` (operational reflex; the
agent must recover, not drop the task). Expanded to the full bash-guard set.
* credential-exfil / identity-forgery / internal-API / env-dump patterns ->
the SAME bash-guard the Claude path runs, wired as a grok PreToolUse hook
(ROBOCO_GUARD_SKIP_GIT=1 so it leaves git to `--deny`). A hard cancel is the
right response there — no legitimate agent reads ~/.netrc or forges an
X-Agent-ID. One tolerance line (accept grok's camelCase `toolInput`) makes
the one tested script guard both runtimes; +5 grok cases (50/50 green).
Also cleaned stale internal task-number / smoke labels out of bash-guard-hook.sh.
* fix(grok): install grok CLI to ~/.grok/bin (its real default), not ~/.local/bin
The image build failed at `chown ... /home/agent/.local: No such file or
directory`. The grok installer's default is $HOME/.grok/bin — the binary lands at
~/.grok/bin/grok; ~/.local/bin/grok is only a convenience SYMLINK the installer
creates on macOS but not in the Linux container. So the Dockerfile referenced a
directory that never existed:
- PATH pointed at ~/.local/bin -> `grok` would not be found at runtime even if
the build had passed;
- chown targeted ~/.local -> the build aborted.
Point PATH + chown at ~/.grok/bin / ~/.grok. Also harden the install: download the
script to a file (a `curl | bash` pipe swallows a curl failure as a silent no-op)
and verify the binary installed and runs (`test -x` + `grok --version`), so a
broken install fails the build loudly instead of producing a grok-less image.
* fix(grok): address adversarial-review findings across the grok-CLI conversion
A 7-dimension adversarial review (find -> independently refute) surfaced 14 real
issues; fixed each:
Runtime bugs
- GrokCliSession.send drained stdout fully BEFORE stderr — a >64KB stderr burst
would deadlock the turn forever (spinner never clears). Drain stderr
concurrently, and add a per-turn watchdog (ROBOCO_GROK_TURN_TIMEOUT_SECONDS,
default 600s) that kills a wedged process and emits error+turn_end.
- Crash-restarted grok agents launched `grok -p ""` (empty prompt) — Claude gets
a scan-for-work fallback. Default the prompt in _spawn_container so every
dedicated provider gets it too.
- _grok_usage_json read /data/grok-usage unconditionally while its writers branch
compose-vs-local, so a local-mode agent finalized at $0 and the cost-cap was
inert. Single-source the path in a new _grok_usage_dir helper (read == write).
- GrokCliSession secretary role fell through to "unknown" (get_agent_role returns
a truthy sentinel, never None), defeating the ROBOCO_AGENT_ROLE fallback.
Parity / hardening
- --deny set was missing `git tag -d` / `git reflog delete` that the Claude
bash-guard blocks — added them (the "same set" claim is now true).
- Interactive mains now install the bash-guard hook too (defense-in-depth).
- Compose: collapse the GROK_AUTH_DIR / ROBOCO_HOST_GROK_DIR auth-mount pair into
one canonical var so a partial override can't silently break agent auth.
Docs / comments
- Panel routing card + architecture security doc no longer say Grok runs on the
deleted opencode runtime; orchestrator comments point at the renamed entrypoint.
Tests
- Cover the interactive _render_grok_config MCP wiring (ModuleNotFound guard +
secretary HMAC env), the cost-cap kill-failure + interactive relay-close paths,
the local-mode usage read, the role fallback, the turn timeout, and the new
git denies. (#13 — a separate grok "Write" tool — investigated: grok's only
built-in file-mutation tool is search_replace, already removed; no gap.)
Gate green: ruff, mypy, xenon, tests.
* fix(grok): declare tomli-w as a runtime dependency (agent image needs it)
The grok agent image failed at spawn with `ModuleNotFoundError: No module named
'tomli_w'` when rendering ~/.grok/config.toml. tomli_w was only a transitive dep
of a dev-extra package, so it was present in dev/orchestrator envs but excluded
from the agent image, which builds its venv with `uv sync --frozen --no-dev`.
grok_cli_config imports it at module load to serialize the MCP gateway config, so
without it a Grok agent gets no gateway verbs.
Promote tomli-w to a direct [project.dependencies] entry. Locked with
`--upgrade-package tomli-w` so only tomli-w is added — no incidental churn of the
8 unrelated packages a full re-resolve would have bumped.
* Updated uv.lock
* fix(grok): auto-approve tool execution (--always-approve) so headless agents can call tools
Live smoke caught every grok agent (Main PM, pr-reviewer, dev, …) ending its run
with stopReason=Cancelled and empty output the instant it reached for a tool. Root
cause: headless `grok -p` cannot approve a tool call without `--always-approve`
(grok's docs: required for unattended automation), and the per-role args didn't
pass it — so no agent could call a gateway verb, an edit, or an MCP tool, and the
run was cancelled.
Add `--always-approve` to grok_cli_args_for_role (one place → every role, one-shot
and interactive). Safety is unaffected: `--disallowed-tools` still removes tools
and `--deny` still hard-blocks command patterns regardless of approval (a denied
command returns a permission error and the agent recovers — verified live).
Proven in the rebuilt image side-by-side: without the flag a tool call yields
Cancelled/not-called; with the real rendered args it returns EndTurn and the MCP
tool actually runs. (My earlier in-image tool-calling check passed `--always-approve`
manually, which masked that the production args omitted it — fixed.)
* fix(pr-review): seed claim heartbeat so the grok reviewer isn't wedge-killed
pr_review_claim transitioned a review task pending -> in_progress but never
seeded last_heartbeat_at, unlike every sibling claim path (_finalize_claim,
qa_claim, doc claim). The reaper treats a NULL heartbeat as a stale claim, and
the GROK idle-kill watchdog bypasses the live-container skip on a NULL
heartbeat -- so the reviewer container was killed (Cancelled) before it could
post_pr_review, churning the task back to pending on a respawn loop. A Claude
reviewer was shielded by the live-instance skip; only GROK manifested it.
Seed the heartbeat at claim time, matching the established invariant. Verified
against a real Postgres (10/10 test_pr_review_db tests, incl. the new
last_heartbeat_at assertion).
* fix(grok): stream one-shot output live + capture real token usage
Two gaps the buffered run hid, both verified in the real image with mounted
SuperGrok auth:
- Observability: the entrypoint buffered grok's output to a temp file and only
cat it after the run, so `docker logs` was blank while the agent worked.
Switch the one-shot to --output-format streaming-json piped through tee:
grok flushes each thought/text event incrementally (confirmed token-by-token
live in-container), so the agent's reasoning shows in docker logs in real
time, parity with the Claude stream-json path. Read the session id back from
the NDJSON run log (the terminal `end` event) since -s does not pin it.
- Usage: total_tokens read 0 for every grok run. grok nests the cumulative
totalTokens on params.update._meta, but the reader looked at params._meta
(which only holds event ids); the unit fixture had the same wrong shape, so
the tests masked it. Read the real path (with params._meta / top-level
fallbacks) and fix the fixture to the real grok shape. Verified live:
usage.json now reports total_tokens=3262, cost_usd=0.006524 (was 0).
* fix(grok): validate agent_id before using it as a usage-dir path segment
CodeQL flagged a high-severity py/path-injection: agent_id flowed from
request-facing call sites into _grok_usage_dir() and on to read_text(), so a
value containing '..' or a separator could traverse the filesystem. Validate
agent_id against the slug/uuid allowlist ([A-Za-z0-9_-]+) at the single
chokepoint (_grok_usage_dir feeds both the mount and the finalize read);
anything else raises. Rejects traversal; accepts every real agent slug.
* fix(grok): use explicit-guard path sanitizer CodeQL recognizes as a barrier
The re.fullmatch allowlist from
|
||
|
Renn F
|
1d835ff50f |
chore(release): prepare v0.6.0
Bump the version to 0.6.0 across pyproject, the package, the config, and the panel, and add the 0.6.0 CHANGELOG entry: inbound external/internal PR review with a CEO decision queue and supersede, the panel feature-flags card, the required-cells decomposition gate, the CEO-rejected coordination-root deadlock fix, the panel UI pass, and registry-image deploy. Also refresh the locked dependencies, update the release-tag examples in the README and deployment docs, and correct the package docstring's agent count to 22. |
||
|
Renn F
|
99c2ac5c62 |
docs(changelog): cut RoboCo 0.5.0
Promote the Unreleased section to [0.5.0] - 2026-06-16 — AC/decomposition guardrails, per-dev sequenced code queues, the unified Business page, the 26 panel UI fixes, and the spawn/PR/ownership firefight fixes — and add the 0.5.0 compare link. Correct the Removed note: the /cockpit, /company-goals, /secretary, and /pitches panel routes are deleted (404), not redirected; the relocated strategy signals are served by the new GET /api/cockpit/signals endpoint. Bump the version 0.2.0 -> 0.5.0 across pyproject, __init__, config app_version, panel package.json, and the uv.lock self-entry — these had drifted unbumped since 0.3.0. |