mirror of
https://github.com/rennf93/roboco.git
synced 2026-08-03 07:23:24 +02:00
3849c1737e
* feat(video): rewrite sidecar render core to HyperFrames (in place)
* feat(video): convert motion compositions from Remotion TSX to HyperFrames HTML
* refactor(video): rename render client to video_renderer_client (renderer-agnostic)
* chore(video): rename remotion-renderer prose in test_video_pipeline docstrings
* chore(video): rename sidecar to video-renderer + add system ffmpeg for HyperFrames
* chore(video): rename stray remotion-renderer refs in sidecar + py docstrings (controller cleanup)
* chore(video): fix stale Remotion API names in Dockerfile comment (controller cleanup)
* docs(video): rewrite video-engine prose for HyperFrames + add map entry + folded prose fixes
* docs(video): add trailing newline to docs/map/video-engine.md (controller cleanup)
* chore(video): drop internal spec refs + minio/test suppressions (folded hygiene)
* fix(video): reclaim outDir on createRenderJob throw + hide empty 4th highlight
Final whole-branch review (Opus) triaged two FIX items from the SDD nits
ledger; the rest ship as-is.
- render.js: a synchronous throw from createRenderJob (post-mkdtemp, not
awaited) left an empty outDir on disk — the outer catch only reclaimed
extractDir. Reclaim outDir too when it exists, and correct the stale
comment that claimed the out dir was never created.
- {vertical,square}.html: the 4th highlights <li> lived in the DOM hidden
only by JS, so a no-JS / failed-script render would show an empty bullet.
Start it style="display:none" and reveal on populate, so an unscripted
render shows nothing instead.
Vitest smoke (release-announcement.test.js) 4/4 green; render.js syntax
checked. Python suite untouched by this fix (JS/HTML only).
* fix(video): type _override_db yield as AsyncSession | None
T7 widened _build_app's db_session param to AsyncSession | None (to drop the
4x # type: ignore[arg-type] on the DB-independent _build_app(None, ...) calls)
but left the inner _override_db fixture typed AsyncIterator[AsyncSession] —
so 'yield db_session' yielded AsyncSession | None into a declared AsyncSession,
and mypy failed at test_video_routes.py:177 ('Incompatible types in yield').
The DB-independent media tests pass db_session=None deliberately: their route
uses a monkeypatched task service and never awaits the session, so yielding
None is safe at runtime. Type the override's yield as AsyncSession | None to
match — no cast, no # type: ignore, no assert, runtime behavior unchanged.
The 3 media tests (3 passed) and the 19 db-gated tests (skipped locally) hold.
* chore(gate): skip .superpowers scratch in markdown prose gate
reflow_md.py walks the filesystem via rglob('*.md') and skips tooling dirs
(.venv, .mypy_cache, .pytest_cache, ...) but not .superpowers/ — the
superpowers SDD workflow's scratch dir (briefs, reports, progress ledger,
all gitignored). A dev running SDD locally would hit a false markdown-prose
gate failure on those transient files. Add .superpowers to SKIP_DIRS,
consistent with the existing tooling-scratch exclusions.
* fix(video): validate composition_id to close path traversal (CodeQL)
compositionId flowed unvalidated from the POST body into path.join
under extractDir/motion/compositions/, so a '../..'-style value could
escape the composition dir (CodeQL: Uncontrolled data used in path
expression). Validate at the trust boundary in server.js
(/^[A-Za-z0-9_-]+$/) and add a path.resolve + startsWith containment
check in render.js so it stays safe regardless of caller.
* fix(mcp): send X-Agent-Token + X-Agent-Team from flow/do servers
flow_server._build_headers and do_server._build_headers constructed
only X-Agent-ID/Role/Correlation-ID, omitting X-Agent-Token and
X-Agent-Team (unlike ApiClient._get_agent_headers used by the other
MCP servers). Latent since the gateway refactor — surfaced when
ROBOCO_AGENT_AUTH_REQUIRED=true was armed on the NAS, 401-ing every
flow/do verb with 'Missing X-Agent-Token header'. Add both headers
(mirroring ApiClient) so the HMAC gate passes. Tests assert the
headers are now injected.
* [video-engine] Per-project video_engine_enabled opt-in toggle
Mirrors ci_watch_enabled (migration 048): the global
ROBOCO_VIDEO_ENGINE_ENABLED flag arms the subsystem; the new
projects.video_engine_enabled column (migration 063) opts a repo into
authoring against its motion/ dir. VideoEngine._opted_in_project no-ops
open_video_task at the single chokepoint covering all three trigger
paths (on-release, on-spotlight, CEO on-demand) until the operator
flips it in the panel edit-project dialog. Existing projects stay
opted out (server_default=false).
* fix(auth): send X-Agent-Token + X-Agent-Team from all agent->API call sites
The prior fix (6ed4e139) covered the flow/do MCP servers but missed four
other agent->orchestrator call sites that built the header dict by hand
and omitted X-Agent-Token and/or X-Agent-Team. With ROBOCO_AGENT_AUTH_REQUIRED
armed on the NAS, every one 401s:
- agent_sdk/server.py: the session-end post-mortem flush
(/api/journals/me/entries), A2A persistence + offline fallback
(/api/a2a/*), and the stopped-without-transition auto-substitute
(/api/tasks/auto-substitute) — all sent only X-Agent-ID/Role, so each
401'd 'Missing X-Agent-Token'. Add a shared _agent_headers() helper
(mirroring flow_server._build_headers) and route all four through it.
- agent_sdk/secretary_driver.py: _headers() sent the token but not the
team, so the HMAC gate 401'd with signature mismatch (secretary is
board-team; token signed with team='board', verified with team='').
Add the team header.
- mcp/git_readonly.py: the read-only git MCP sent only X-Agent-ID/Role
— no token, no team — so /api/git/* 401'd once auth was armed. Convert
the static _HEADERS to a _headers() helper with team + token.
- runtime/orchestrator.py: the cell-PM auto-submit self-API call acted
as a PM with a hand-built {X-Agent-ID, X-Agent-Role} dict — no token,
no team — 401ing under auth-required. Add _agent_api_headers(uuid,
role) mirroring _system_api_headers, and use it.
Tests: _agent_headers round-trip (token + team, team-omitted when None),
_agent_api_headers carries a signed PM token + team.
* [auth] Omit UNSIGNED self-call token in dev mode + video-engine test mypy fix
_agent_api_headers sent the UNSIGNED sentinel when ROBOCO_AGENT_AUTH_SECRET
was unset, but the dev-mode middleware rejects a presented-but-unverifiable
token with 401 signature mismatch (while accepting a missing one). The
cell-PM auto-submit self-call 401'd in every dev run, regressing
test_auto_submit_cuts_the_pm_turn. Attach the token only when a secret is
set. Also fix the FromClause.update mypy error in the per-project
video-engine opt-out test (ORM row load + flush).
* [auth] Omit UNSIGNED agent token at every agent->API call site
The orchestrator injects ROBOCO_AGENT_TOKEN=UNSIGNED when the HMAC secret
is unset at spawn. The API middleware rejects a presented-but-unverifiable
token with 401 'signature mismatch' even in dev mode (auth not required),
so forwarding UNSIGNED turned every flow/do/SDK/secretary/git verb into a
401 — the live pr_reviewer/i_am_idle signature-mismatch loop. Omit the
header when the token is the UNSIGNED sentinel at all five agent-side
header builders; dev accepts a missing token, prod 401s with 'Missing
X-Agent-Token' (the clear respawn-with-secret signal). Add a structlog
diagnostic on the middleware reject path so the next mismatch logs the
exact (id, role, team, token_unsigned, auth_required) inputs.
* [auth] Self-heal stale agent tokens at orchestrator startup
A token is signed once at spawn. If ROBOCO_AGENT_AUTH_SECRET drifts
afterwards (a .env change, a compose recreate that reloads the
orchestrator's env without recreating agent containers, an image
redeploy), the surviving agent keeps sending its old token and the
middleware 401s every verb with 'signature mismatch'. The container
stays alive heartbeating, so the reaper never reclaims it and no fresh
agent spawns: the fleet stalls.
_heal_stale_agent_tokens runs at startup (before _readopt_running_agents)
and kills each running agent container whose baked-in token no longer
verifies against the current secret, so normal dispatch re-spawns it
with a freshly signed token. Inert when the secret is unset (dev):
verify fails for every token without a secret, so the heal would kill
the whole fleet without this gate. Best-effort: a probe failure leaves
the container alone (the reaper still covers it).
* [auth] Sign agent token over the UUID, not the slug (pr_reviewer 401 root cause)
The token was signed over the agent slug (_append_agent_auth_env) while the
MCP servers send X-Agent-ID as the agent UUID (_generate_mcp_config, since
453a7ae2 — gateway v1 parses X-Agent-ID as Annotated[UUID]). The middleware
verified HMAC(uuid:role:team) against a slug-signed token → 'signature
mismatch', token_unsigned=false. Latent for 2 months until 6ed4e139/53391f22
made the MCP servers forward the token.
The c0328971 startup heal missed it: docker exec printenv reads the
container-level ROBOCO_AGENT_ID (the slug), so the heal verified the
slug-signed token against the slug → matched → didn't kill the stale
container, which kept 401ing (its MCP server sends the UUID).
Fix: sign the token over the UUID, set the container ROBOCO_AGENT_ID to the
UUID too (so the SDK server — which inherits container env, not the MCP
manifest env — sends UUID consistently), and resolve the container-env id to
its UUID in _heal_stale_agent_tokens so pre-fix stale containers are evicted
on next restart. Regression test: test_heal_kills_slug_env_container_with_slug_signed_token.
* [scan] gate A2A/notification/stream agent-id deps under cloud auth (C1)
* [scan] omit UNSIGNED agent token from MCP server headers (H1)
* [scan] fail loud when cloud auth and nginx CEO-token are both armed (H2)
* [scan] cache last-known-good auth-probe result in panel proxy (C2)
* [respawn] Tripped breaker self-heals after a cooldown
A DB-durable PM-respawn counter (migration 051 / e2f7097a) wedges forever
once tripped: the only reset was a task status change, which can't happen
while the breaker blocks the spawn. So a deploy that fixes the underlying
loop (auth/prompt/schema) couldn't clear the wedge without manual DELETE
surgery on respawn_tracker — the 2026-07-06 pr-reviewer-1 loop, where the
auth fix cleared the 401 but count=63 survived restart and kept skipping
the dispatcher spawn for an external-PR task.
Freeze last_check at the trip tick and, after pm_respawn_trip_cooldown_seconds
(default 300), let ONE spawn through. A still-wedged task re-trips after the
threshold (bounded re-burn ~3 spawns per window); a fixed one advances and
the status-change path fully resets. Restore re-stamps last_check to now, so
a freshly restored row still trips immediately — durability preserved, which
is why the migration-051 persistence tests still pass.
* [scan] fix test_deps callsites for cloud-auth-gate signature change (C1 followup)
* [scan] per-IP rate limit on /auth/login under cloud auth (L31)
* [scan] Phase 1 auth/security fixes under 0.19.0 CHANGELOG
* [scan] secretary token signs over real team (board) not empty — fixes /api/secretary/* 401 (L31-class)
* [scan] LoginRateLimiter: key off X-Forwarded-For first hop + redis-down fail-open test
nginx is the single entry point; request.client.host is the nginx peer IP,
collapsing every external client into one limiter bucket (self-DoS amp).
Read the downstream client IP from X-Forwarded-For (first hop) / X-Real-IP,
falling back to the peer. Adds coverage for the XFF keying, the redis-down
fail-open branch, and drops a redundant asyncio marker on a sync-TestClient
test.
* [scan] nits: describe login_max_attempts + replace cast with assert in get_current_agent_slug
login_max_attempts was the only bare cloud-auth field; add a Field
description matching the surrounding idiom. Replace cast('str', ctx.slug)
with a runtime assert that fails loud if the cloud-auth ctx invariant
breaks, and drop the now-unused cast import.
* [scan] secretary token: use get_agent_team resolver + complete spawn-shutdown mock team (0dfd45ca followup)
* [scan] require agent HMAC token under cloud_auth (close v1 flow/do header-trust)
* [scan] _require_ceo accepts CEO session cookie under cloud_auth
* [scan] HTTP require_panel_token accepts session cookie under cloud_auth
* [scan] gate /api/settings behind panel token
* [scan] gate unauthenticated /api read routes (agents/a2a-tasks/kanban/usage/rate-limits)
* [scan] hoist deferred test imports to top-level (clear PLC0415)
* [scan] Phase 1b e2e smoke + CHANGELOG
* [scan] add_dependency rejects self-reference + cycle (M18)
* [scan] WorkSessionService.create translates IntegrityError to ConflictError (H10)
* [scan] _qa_or_doc_claim locks the task row FOR UPDATE (M19)
* [scan] docs_complete + mark_pr_created lock the task row FOR UPDATE (H4)
* [scan] gate complete() IN_PROGRESS on leaf/branchless only (H3)
* [scan] _unclaim_from_blocked clears stale pre-block snapshot (H5)
* [scan] admin_set_status terminal guard + skip revision bump under force (M20)
* [scan] cell_pm_complete idempotent pre-check before merge (H7)
* [scan] wrap gateway post-runner side effects in try/except (H6)
* [scan] pass_qa/fail_qa accept AWAITING_QA only (L29)
* [scan] mark_pr_created passes audit_agent_id (L30)
* [scan] phase 2 e2e smoke - one scenario per finding
* [scan] phase 2 quality gate
ruff format + check: green
mypy roboco/: green (357 files)
pytest unit+integration: 6905 passed, 10 pre-existing DB-contamination
failures (pass in isolation)
e2e smoke: 11 passed, 4 cross-scenario workspace-contamination failures
(all 6 state-machine scenarios pass individually)
Quality-gate fixes:
- move function-local imports to module top (PLC0415)
- fix M19 regression: submit_for_qa clears active_claimant_id so the
competing-claimant guard lets the QA claim through
- fix H7 regression: _StubGit gains is_pr_merged_for_task
- fix M19 unit tests: mock session.execute for the FOR UPDATE lock
- e2e H3: notes >= 20 chars; e2e H5: rich i_will_work_on inputs +
PM unclaims (block reassigns to PM)
* [scan] move active_claimant_id clear into pass_qa/fail_qa + admin_set_status (M19 follow-on)
Phase 2 opus whole-branch review found the M19 follow-on clear lived in
the gateway wrappers (qa_pass/qa_fail) not the transition methods
(pass_qa/fail_qa) themselves. The direct REST routes POST /pass-qa and
POST /fail-qa call the transitions directly, bypassing the wrappers and
leaving the QA's stale active_claimant_id set in AWAITING_DOCUMENTATION
/ NEEDS_REVISION — the competing-claimant guard then rejects the next
legitimate documenter/QA claim. admin_set_status had the same gap for a
non-blocked override into a review/queue state (IN_PROGRESS->AWAITING_QA
left the dev's id, blocking qa_claim).
Root-cause fix: move the clear INTO pass_qa and fail_qa (mirroring
submit_for_qa), add a clear in admin_set_status when
new_status in _REVIEW_QUEUE_STATES and from_status != BLOCKED, and drop
the now-redundant clears + flushes from the qa_pass/qa_fail wrappers.
Every caller is covered; the wrappers keep their actor-mismatch warnings.
Covering tests: test_pass_qa_clears_active_claimant_for_doc_claim
(asserts a subsequent doc_claim succeeds), test_fail_qa_clears_active_claimant,
test_admin_set_status_into_review_queue_clears_active_claimant,
test_admin_set_status_non_review_queue_keeps_active_claimant. Updated
the two wrapper unit tests that asserted the wrapper clears (now the
transition's job).
* [C3] unindex_journal_entry + call from delete_entry
JournalService.delete_entry deleted the DB row but never de-indexed the
RAG chunks, so deleted/private journal content bled forever into RAG
answers and claim-time briefings. Add OptimalService.unindex_journal_entry
mirroring unindex_playbook (vector-store delete_by_source + tracking-row
delete via get_db_context, both idempotent + best-effort), and call it
from delete_entry after the row commit inside a try/except so a de-index
failure never errors the delete.
* [M25] learning_id hashes full content to avoid collision
The memory distiller emits lessons with a fixed 'Problem: …' opening
shape, so two distinct lessons whose first 100 chars match collided on
learning_id = f"lrn-{md5(content[:100])[:12]}". replace_on_reingest then
routed both to the same source URI and the second ingest's replace_chunks
DELETE wiped the first lesson's chunks — silent data loss.
Hash the full content (widening the hex slice 12→16) so distinct bodies
get distinct ids and each retains its chunks.
* [H13] reject non-internal local_llm_base_url at config load
* [M28] bulk-insert learning broadcast instead of N+1
* [M27] mark_read/mark_all_read stamp only the unread rows seen at call time
mark_read and mark_all_read used to zero the unread counter FIRST, then run
a bulk UPDATE … WHERE read_at IS NULL that stamped every inbound unread row.
A send_chat_message committing between the counter-zero and the UPDATE
inserted a new read_at NULL row that the UPDATE then stamped as read — the
new message was silently consumed while the counter stayed 0.
Mirrors get_unread_messages (same file): SELECT the unread message IDs at
call time, UPDATE exactly those IDs, then recompute the unread counter from
the DB via the existing _reset_unread_counter helper. A message arriving
mid-call is not in the selected ID set, so the UPDATE skips it and the
recomputed counter keeps it unread.
* [H12] dedup: exact to_agents predicate + purpose discriminator + ack DEL
* [M23] playbook indexed_ok/indexed_at + startup reconcile of unindexed approved
* [M24] RAG indexing dead-letter + janitor reclaim + failed_index_count health
* [L23] institutional_memory_status sentinel distinguishes below-floor/empty/error/disabled
* [L26] sweep_expired_notifications re-escalates stale unacked ack-required
* [phase3] e2e smoke + CHANGELOG for 0.19.0
* [M24] _reindex_journal_entry honors is_private (C1 review fix)
Dead-letter replay mirrors the original journal._schedule_rag_index path:
a private entry is never indexed into the shared JOURNALS corpus, and a
private learning is still recorded into LEARNINGS as non-shareable.
Previously the replay always called index_journal_entry and skipped
record_learning for private learnings, leaking private content on replay
and dropping the legitimate non-shared learning. Three regression tests.
* [H11] clone via git -c http.extraheader, not URL-embedded PAT
* [H11] _sync_read_clone fetch via http.extraheader, not URL-embedded PAT
Sibling site to the clone fix: the conventions read-clone refresh ran
'git fetch --tags <https://TOKEN@host> <branch>', exposing the PAT in the
fetch argv on the orchestrator host. Mirrors the clone site's per-call
'-c http.extraheader=Authorization: Basic …' prefix + bare URL. SSH URLs
and tokenless public repos unchanged.
* [H11] release_executor clone+push via http.extraheader; delete _inject_token_into_url
* [H8] rebase_onto_base gates on clean tree like pull
* [H9] _link_commit_to_task flushes, doesn't commit out-of-band
* [M38] _pr_is_merged returns None on HTTPError; caller assumes merged
* [M39] _cherry_unmerged_entry marker grep anchored to commit-prefix
* [L1] thread actor_agent_id through update_pr_for_task
* [H8] fix rebase test mocks for clean-tree gate
H8 inserted a 'git status --porcelain' dirty-tree gate at the top of
rebase_onto_base (mirroring pull). The 3 rebase control-flow tests mocked
_run_git with a side_effect list matching the OLD call sequence (no
leading status call), so every call shifted by one and the assertions
missed. Prepend a clean-status result to each list so the gate passes
and the fetch/checkout/reset/rebase/diff/abort/push sequence aligns.
Verified: 16 passed (was 3 failed/13 passed post-H8, 16 passed pre-H8).
* [L2] push --force-with-lease instead of bare --force
* [L1] refresh stale workspace-resolution docstrings
pr_target and _workspace_for_branch still documented the actor →
assigned_to → created_by fallback chain that L1 removed from
_resolve_workspace_agent_id. Update both to the post-L1 actor →
assigned_to → None resolver (project.workspace_path as the final
fallback) so a future reader doesn't rely on a fallback that no
longer exists.
* [M37] merge_pr locks the work_session row FOR UPDATE
* [phase4] e2e smoke + CHANGELOG for 0.19.0
* [phase4] fix M37 test flake + document H8 skip
The opus whole-branch review flagged the M37 concurrency tests as
~50% flaky: both asserted caller A wins the FOR UPDATE race, but
which caller wins the lock is non-deterministic. When B won, the
'assert a_row.merged_by == a_merger' branch flipped false even
though the production code (M37) was correct — exactly one merger
recorded, audit trail intact. Assert the invariant instead: both
rows COMPLETED, both report the same merged_by, value in
{a_merger, b_merger}. Applied to both the unit test and the e2e
twin. Also documents the H8 e2e skip in the module docstring (the
report claimed it was documented there but it wasn't) and drops
the internal 'Phase 4' label from the docstring header in favor of
the public '0.19.0' version anchor.
* [H24] wait_for_ci polls through the window on non-success
* [H25,L34] release mutex orphan-sweep on start + shared redis client
* [M1] tiktok _refresh commits rotated tokens in an independent session
* [H25] drop new type:ignore in orphan-sweep test (constraint cleanup)
* [M2] feature-spotlight re-arms when exploration stale past 2x interval with no live HoM spawn
* [M6,M7] mark_seen after meaningful+project; persist since_id cursor in redis
* [M3,M5] reject() guards COMPLETED; edited_body deferred into the single-flight lock
* [M4] bound list_completed_video_tasks + ix_tasks_source_status_created index (migration 066)
* [M8,M9,L9] pass head_sha to CI gate; _run_git 30s timeout; _commits_since split maxsplit 2
* [M10,L35] dedupe dep_update by (git_url, command); fold redundant per-project queries
* [L36] gather ci_watch telemetry sweep instead of sequential iteration
* [L11] document self_heal fingerprint is stable per-signal by design
* [M11] engine-loop liveness watchdog: heartbeat + 2x-interval staleness alert
* [M21] video render loop commits per-task, not one trailing commit
* [M22] _detect_stuck_tasks skips held-CEO-source tasks
* [L6] video_renderer_client._save writes temp + atomic rename
* [phase5] e2e smoke + CHANGELOG for 0.19.0
* [M11] instrument x_mentions + roadmap engine loops with liveness heartbeats
* [phase5] fix-wave: correct e2e M11 unit-test filename + strengthen failed-cycle heartbeat assertion
* [C4] panel WS: shared /ws/system socket + long-tail retry + pong watchdog
* [H15] video-post-queue caption derived per render (mirror x-post-queue)
* [C4-fix] panel WS: discriminating long-tail tests + drop dead freeze block + evict dead shared conn on manual disconnect
Finding 1 (Critical, websocket.test.ts): the two long-tail-retry tests fired onopen between close cycles, which reset reconnectAttempts to 0 each cycle, so they passed under the pre-fix 3-attempt gate. Rewrote both to NEVER fire onopen between closes, so attempts accumulates: test 1 asserts state stays 'reconnecting' past attempt 3 (old gate would flip 'disconnected' terminal); test 2 asserts a new socket is constructed within 30000ms at attempt 7 where uncapped 5000*1.5^7 ~= 85s (old uncapped code would leave the timer unexpired). Verified both FAIL on a reverted old-shape connection.ts and PASS on the fixed code.
Finding 2 (Important, connection.ts): the 'if (raw >= cap) this.reconnectAttempts = exp' block was a no-op (exp was just read from the same field) and the unconditional increment afterwards grew the counter regardless. Deleted the dead block; kept the Math.min cap on the delay. Replaced the misleading ponytail comment with an accurate one: delay is capped, counter grows unbounded but delay is bounded.
Finding 3 (Important, use-websocket.ts): manual disconnect() tore down the shared conn for all subscribers but left the dead (manualClose=true, never reconnects) entry in _sharedSockets, so a later mount hit the reuse branch, attached a subscriber, replayed 'disconnected', and never called connect(). Added a urlRef and _sharedSockets.delete(url) in the manual disconnect callback so a later mount reopens a fresh conn.
* [H16] settings Save wired to settingsApi (persist + read back)
* [H17] tasks page passes status/team/limit to useTasks (server-side filter)
* [H18] useAgents roster re-derives on live-status change (statusEpoch in queryKey)
* [M40] useMetrics reads agent counts from useAgentStatus cache (dedupe poll)
* [H18] tighten useAgents statusEpoch comment (drop spec ref)
* [M40] drop spec ref + tighten useMetrics comment
* [M41] scorecard refetchInterval 60s -> 5min (25 req/min -> 5)
* [M42] feature-flag off-transition confirm + pending-keys Set
* [M43] X/TikTok credentials clear-behind confirm dialog
* [M44] rate-limit syncFromApi merges (keep fresher hitAt) + A2A reconnect invalidation
* [phase6] proxy.ts cookie-check comment + CHANGELOG Fixed entries
* [phase6] drop stale WS pin-attempts comment + fix tasks-page lead-in
* [H21] type DelegateRequest.estimated_complexity as Complexity (reject critical)
* [H22] type SoftBlockRequest.resolver_type as BlockerResolverType (no silent AGENT fallback)
* [H23] serialize TaskTable.documents into TaskResponse (DocRefResponse)
* [L27] delete SubstituteRequest phantom suggested_role/suggested_team fields
* [L14] Envelope.not_found defaults remediate (guide re-fetch + re-issue)
* [L28] delete unused ListResponse generic (dead code; pagination deferred)
* [H19] _delegate_static_guards allow cell_projects roots (cross-cell MegaTask)
* [M13] MegaTask confirm-batch idempotency key from session_id (SETNX guard + result sidecar)
* [M14] strip assigned_to from MegaTask drafts (no board-owned root-subtask deadlock)
* [H20] thin_routes receiver-gate add/add_all/merge (no false block on set/cache.add)
* [M16] tighten noqa code-capture to [A-Z0-9, ]+ (no false block on natural prose)
* [M45] conventions read-clone force-refetch on read (no 30s stale map window)
* [L25] conventions._resolve returns (root, sha); ORM mutated on the event loop
* [M15] open_conventions_pr force-pushes disposable scaffold branch (no silent None)
* [L24] roadmap cycle completion emits status-transition audit
* [Phase7] CHANGELOG: 15 schema/conventions/MegaTask/API fixed (H21-H23,L27,L14,L28,H19,M13,M14,H20,M16,M45,L25,M15,L24)
* [Phase7] lint gate hygiene: shorten docstring (E501), sort imports (I001), hoist AuditLogTable import (PLC0415)
* [H14] Enable the GROK provider row in _apply_grok so routing reaches the GrokCliProvider
* [M31] Route GROK active-token resolution to usage.json so live usage reflects grok agents
* [M32] Pass cache read/write tokens to calculate_cost in the usage sweep so live cost reflects Anthropic cache spend
* [M33] Park Ollama-Cloud rate limits via a marker map so a glm-5.2:cloud 429 parks instead of crash-respawning
* [M34] Sweep orphan agent_spawn_sessions at startup so crashed-run tokens roll into usage/cost summaries
* [L12] Persist revisit_resets (migration 067) so the PM-respawn breaker's revisit counter survives a restart
* [L18] Date-gate the Sonnet-5 promo revert so billing returns to list rates after 2026-08-31
* [L20] Warn when ROBOCO_GROK_RUN_LOG yields no session id instead of silently falling back to a zero-usage env id
* [phase8] CHANGELOG: LLM provider routing, usage capture, billing fixes
* [phase8] Trailing ruff format hygiene (orchestrator marker tuples, token-sweep test signatures)
* [phase8] Fix mypy: rename GROK-branch tokens var so transcript fallback stays reachable
* [M35] Add an expiring agent-token format (iat/exp) with backward-compatible verify
* [M35] Wire agent-token TTL at spawn (config + orchestrator + grok) so tokens are bounded
* [M36] Add JWT jti claim and re-mint the sliding cookie only near expiry so a stolen cookie's exp is fixed
* [M36] Redis jti revocation: read_token rejects revoked jtis and logout revokes the current jti
* [phase9] CHANGELOG: bound agent tokens + sliding-cookie re-mint window + jti revocation
* [scan-fix] mypy: type-annotate test files for make-quality gate
CI's make quality runs mypy roboco/ tests/; the scan-fix program's local
gate ran mypy roboco/ only, so test files were never type-checked. Fix all
67 errors across 23 test files with real annotations/casts/asserts/dead-code
removal — no # type: ignore / # noqa added.
* [e2e] Per-test DB isolation + dispatcher re-claim before PM complete
* [scan] Regenerate verb tables for delegate Complexity type
* [scan] Reduce 9 xenon C-ranks to B (auth, orchestrator, gateway, services)
* [scan] Restore short-circuit time.time() in verify_agent_token (security path)
---------
Co-authored-by: Renn F <rennf93@users.noreply.github.com>
588 lines
67 KiB
Markdown
588 lines
67 KiB
Markdown
# CLAUDE.md
|
||
|
||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||
|
||
## Licensing
|
||
|
||
RoboCo is licensed under **AGPL-3.0** (see `LICENSE`). Copyright (c) 2026 Renzo Franceschini. Do NOT reintroduce an MIT or other license reference anywhere (README, headers, package metadata) — the project is AGPL.
|
||
|
||
Contributions require a signed **Contributor License Agreement** (`CLA.md`), automated via the CLA Assistant workflow (`.github/workflows/cla.yml`). The CLA preserves the option to dual-license / offer a commercial edition later; keep copyright assignment language intact. See `CONTRIBUTING.md`.
|
||
|
||
## Project Overview
|
||
|
||
**RoboCo** is an AI Agentic Company - a virtual organization of 25 AI agents + 1 human CEO, designed to operate as a complete software development workforce. The system implements a structured organizational hierarchy with formal communication protocols, task management, and quality controls.
|
||
|
||
### Core Architecture
|
||
|
||
```
|
||
CEO (Renzo - Human)
|
||
|
|
||
+-- Intake (on-demand interviewer: chats only with the CEO to draft a task)
|
||
+-- Secretary (on-demand chief-of-staff: reads company state, runs gated CEO directives)
|
||
+-- PR Reviewer (read-only: the main reviewer — inbound external/fork + internal PRs, and the root→master in-path gate)
|
||
|
|
||
+-- Board (3 agents)
|
||
+-- Product Owner
|
||
+-- Head of Marketing
|
||
+-- Auditor (silent observer, reports to CEO)
|
||
|
|
||
+-- Main PM (coordinates all cells)
|
||
|
|
||
+-- Backend Cell (6 agents: 2 Devs, 1 QA, 1 PM, 1 Documenter, 1 PR Reviewer)
|
||
+-- Frontend Cell (6 agents: 2 Devs, 1 QA, 1 PM, 1 Documenter, 1 PR Reviewer)
|
||
+-- UX/UI Cell (6 agents: 2 Devs, 1 QA, 1 PM, 1 Documenter, 1 PR Reviewer)
|
||
```
|
||
|
||
### Hardware Infrastructure
|
||
|
||
- **Olares One (Powerhouse)**: Intel Ultra 9 + RTX 5090, runs Claude Code instances and AI inference - NOT YET ARRIVED
|
||
- **UGREEN NAS (Warehouse)**: 36TB RAID6, 128GB RAM, hosts PostgreSQL, Redis
|
||
- **Pi Cluster (Operations)**: Monitoring, notifications, smart home
|
||
|
||
## Development Standards
|
||
|
||
### Python (Backend)
|
||
```bash
|
||
# Package manager
|
||
uv
|
||
|
||
# Before any commit
|
||
uv run ruff format .
|
||
uv run ruff check .
|
||
uv run mypy roboco/
|
||
uv run pytest
|
||
|
||
# Coverage target: 80%
|
||
```
|
||
|
||
### TypeScript (Frontend)
|
||
```bash
|
||
# Package manager
|
||
pnpm
|
||
|
||
# Before any commit
|
||
pnpm format
|
||
pnpm lint
|
||
pnpm typecheck
|
||
pnpm test
|
||
|
||
# Coverage target: 80%
|
||
```
|
||
|
||
## Technology Stack
|
||
|
||
| Layer | Technology |
|
||
|-------|------------|
|
||
| API Framework | FastAPI |
|
||
| Database | PostgreSQL + asyncpg |
|
||
| Vector Store | PostgreSQL + pgvector (in-house engine) |
|
||
| RAG Engine | in-house (asyncpg + pgvector, hybrid retrieval) |
|
||
| Cache/Queue | Redis |
|
||
| Container Runtime | Docker + Docker Compose |
|
||
| Cloud LLM | Claude API (claude-opus-4-6) + xAI Grok (official `grok` CLI, SuperGrok subscription) |
|
||
| Local LLM | Ollama (glm-5.2:cloud for RAG/hybrid retrieval) |
|
||
| Embeddings | qwen3-embedding:0.6b (1024 dim) |
|
||
| Frontend | Next.js 16 + TypeScript + Tailwind + Radix UI (in `panel/`) |
|
||
| Edge / Proxy | nginx (single entry point on port 3000) |
|
||
|
||
## Multi-Agent Workspace Structure
|
||
|
||
Each agent gets their own git clone of a project, enabling parallel development without conflicts:
|
||
|
||
```
|
||
{ROBOCO_WORKSPACES_ROOT}/ # Default: /data/workspaces
|
||
+-- {project-slug}/
|
||
+-- {team}/
|
||
+-- {agent-slug}/
|
||
+-- [git repository]
|
||
```
|
||
|
||
**Example:**
|
||
```
|
||
/data/workspaces/
|
||
+-- roboco/
|
||
+-- backend/
|
||
| +-- be-dev-1/ # be-dev-1's workspace
|
||
| +-- be-dev-2/ # be-dev-2's workspace
|
||
+-- frontend/
|
||
+-- fe-dev-1/
|
||
+-- fe-dev-2/
|
||
```
|
||
|
||
Note: the Next.js control panel now lives at `roboco/panel/` inside this repo (no longer a separate `roboco-panel` project or workspace).
|
||
|
||
**Key Configuration (roboco/config.py):**
|
||
- `ROBOCO_WORKSPACES_ROOT`: Root directory for workspaces (default: `/data/workspaces`)
|
||
- `ROBOCO_WORKSPACE_AUTO_CLONE`: Auto-clone repos on first access (default: `true`)
|
||
- `ROBOCO_WORKSPACE_CLONE_TIMEOUT`: Clone timeout in seconds (default: `300`)
|
||
|
||
On a Python workspace, `WorkspaceService` runs `uv sync --extra dev` (not plain `uv sync`) so the clone's `.venv` carries the full gate toolchain (ruff/mypy/xenon/pytest) — the lint/type/complexity tools live in the `dev` **extra**, which plain `uv sync` skips. Without it an agent's `make quality` fails on `ruff: command not found` and the agent can't gate its own work.
|
||
|
||
Because the clone is shared across a dev's tasks, a **fresh claim** git-resets the workspace to a clean tree (`git reset --hard`) before checking out the new task's branch — discarding abandoned uncommitted cruft from a finished task while preserving all commits and the gitignored `.venv`. A resume short-circuits before this, so committed work is never reset.
|
||
|
||
## Git Workflow
|
||
|
||
### Branch Naming Convention
|
||
|
||
Branch names follow the pattern: `{type}/{team}/{task-hierarchy}`
|
||
|
||
**Types:** `feature`, `bug`, `chore`, `docs`, `hotfix`
|
||
|
||
**Task Hierarchy:** Uses `--` separator (not `/`) to avoid git ref conflicts.
|
||
|
||
**Examples:**
|
||
- Root task: `feature/backend/ABC12345`
|
||
- Subtask: `feature/backend/ABC12345--DEF67890`
|
||
- Sub-subtask: `feature/backend/ABC12345--DEF67890--GHI11111`
|
||
|
||
### Commit Format
|
||
|
||
Commits are automatically prefixed with the task ID:
|
||
|
||
```
|
||
[{task-id[:8]}] {message}
|
||
```
|
||
|
||
**Example:**
|
||
```
|
||
[ABC12345] Add user authentication endpoint
|
||
```
|
||
|
||
### Work Sessions
|
||
|
||
When a developer claims a task, a **WorkSession** is created that tracks:
|
||
- Branch name and base/target branches
|
||
- All commits made during the session
|
||
- Files modified
|
||
- PR number/URL when created
|
||
- Merge status and who merged
|
||
|
||
A task has at most **one active WorkSession**: re-claiming a task (pool release, reaper unclaim, escalation redirect) supersedes any prior agent's stale active session, enforced both at the service layer and by a DB partial-unique index (migration 047). Without it, duplicate active sessions made the one-row active lookup raise and crashed the claim/plan flow into a respawn loop.
|
||
|
||
A developer's clone is shared across all their tasks, so push and PR-head operate on the task's **recorded branch by name**, independent of the clone's current checkout — fixing the `BRANCH_MISMATCH` / "No commits between" failures when the clone was parked on a later task's branch. A missing local task-branch ref is first recovered from `origin/<branch>` before the push-by-name.
|
||
|
||
### Git Credentials
|
||
|
||
Git authentication is managed **per-project** through encrypted GitHub PATs:
|
||
|
||
- **Each project stores its own git token** - no global fallback
|
||
- **Tokens are encrypted at rest** using Fernet symmetric encryption
|
||
- **API never exposes tokens** - only returns `has_git_token: boolean`
|
||
- **Self-service via UI** - users set/update tokens in project settings
|
||
|
||
**Project fields:**
|
||
| Field | Description |
|
||
|-------|-------------|
|
||
| `git_token_encrypted` | Fernet-encrypted GitHub PAT (DB column) |
|
||
| `has_git_token` | Boolean indicator for API responses |
|
||
|
||
**Token flow:**
|
||
1. User creates project in UI, enters GitHub PAT
|
||
2. Token encrypted and stored in `projects.git_token_encrypted`
|
||
3. WorkspaceService decrypts token when cloning repos
|
||
4. GitService decrypts token for PR operations (gh CLI)
|
||
|
||
**HTTPS URLs require tokens** - attempting to clone without a token will raise `WorkspaceError`.
|
||
|
||
## Task Lifecycle
|
||
|
||
### Task States
|
||
|
||
The complete task lifecycle is defined in `roboco/foundation/policy/lifecycle.py` (`roboco/enforcement/task_lifecycle.py` is a backwards-compat shim over it):
|
||
|
||
```
|
||
backlog -> pending -> claimed -> in_progress -> [blocked|paused] -> verifying
|
||
| |
|
||
v v
|
||
awaiting_qa <------------------+ awaiting_documentation
|
||
| (needs_revision) | |
|
||
v | v
|
||
awaiting_documentation --------+ awaiting_pm_review
|
||
| |
|
||
v v
|
||
awaiting_pm_review awaiting_ceo_approval
|
||
| |
|
||
v v
|
||
completed completed
|
||
```
|
||
|
||
**In-path PR-review gate** (`awaiting_pr_review`): each assembled PR is reviewed before the PM merges. The cell PM's `submit_up` opens the cell→root PR and the Main PM's `submit_root` opens the root→master PR; both enter `awaiting_pr_review`, where a reviewer `pr_pass`es it on to `awaiting_pm_review` or `pr_fail`s it back to `needs_revision` — the merge-level reject the PM otherwise lacks. Leaf dev tasks and branchless coordination roots skip the gate.
|
||
|
||
**States:**
|
||
| State | Description |
|
||
|-------|-------------|
|
||
| `backlog` | PM setup phase - dependencies or session setup needed |
|
||
| `pending` | Ready for work - orchestrator can spawn agents |
|
||
| `claimed` | Agent has locked the task |
|
||
| `in_progress` | Active development |
|
||
| `blocked` | External dependency blocking progress |
|
||
| `paused` | Temporarily stopped (can resume) |
|
||
| `verifying` | Self-verification by developer |
|
||
| `needs_revision` | QA or CEO requested changes |
|
||
| `awaiting_qa` | Submitted for QA review — PR must already exist |
|
||
| `awaiting_documentation` | Documentation phase — PR already open from pre-QA; doc writes docs |
|
||
| `awaiting_pr_review` | In-path PR-review gate: a reviewer checks the assembled cell→root / root→master PR before the PM merges (assembled, PR-bearing tasks only) |
|
||
| `awaiting_pm_review` | Docs complete, PM reviews + merges |
|
||
| `awaiting_ceo_approval` | Major tasks escalated for CEO final approval |
|
||
| `completed` | Terminal state - work done and merged |
|
||
| `cancelled` | Terminal state - work cancelled |
|
||
|
||
### Role-Based Transitions
|
||
|
||
All status transitions are validated through the enforcement layer. Key restrictions:
|
||
|
||
| Transition | Allowed Roles |
|
||
|------------|---------------|
|
||
| `backlog` → `pending` (activate) | PM roles only |
|
||
| `pending` → `claimed` (claim) | Role must match task type (QA for awaiting_qa, etc.) |
|
||
| `claimed` → `pending` (unclaim) | Assignee or PM |
|
||
| `awaiting_qa` → `awaiting_documentation` (pass) | QA only |
|
||
| `awaiting_qa` → `needs_revision` (fail) | QA only |
|
||
| `awaiting_documentation` → `awaiting_pm_review` | Documenter or Developer (parallel completion) |
|
||
| `in_progress` → `awaiting_pr_review` (submit_up / submit_root) | PM roles (opens the assembled cell→root / root→master PR) |
|
||
| `awaiting_pr_review` → `awaiting_pm_review` (pr_pass) | PR reviewer only |
|
||
| `awaiting_pr_review` → `needs_revision` (pr_fail) | PR reviewer only |
|
||
| `awaiting_pm_review` → `completed` | PM roles only |
|
||
| `awaiting_pm_review` → `needs_revision` (request_changes) | PM roles only — the merge-level reject with concrete issues |
|
||
| `awaiting_pm_review` → `awaiting_ceo_approval` | PM roles only |
|
||
| `awaiting_ceo_approval` → `completed/needs_revision/cancelled` | CEO only |
|
||
| Any → `cancelled` | PM roles only |
|
||
|
||
**Unclaim Operation**: Agents can release claimed tasks back to the pool using `unclaim()`. This transitions `claimed` → `pending` and optionally reassigns to another agent.
|
||
|
||
**Board never owns a coordination root**: a Board role (Product Owner / Head of Marketing) is never assigned a Main-PM coordination root (delivery root or MegaTask root-subtask) via escalation or reassignment — Board roles have no `unblock` verb, so such a hand-off would deadlock. The transition is diverted to the pool for a role-matched Main-PM reclaim.
|
||
|
||
### Git Integration Requirements
|
||
|
||
All tasks follow git workflow. PR is created BEFORE QA review (not after) so QA can review the real PR diff on GitHub and downstream PM/CEO approval chain off a PR that already exists:
|
||
|
||
1. **claimed -> in_progress**: `branch_name` is auto-set on claim (hierarchical branches)
|
||
2. **verifying -> awaiting_qa** (submit-qa): Requires `self_verified`, `commits`, `pr_number` (PR open), and at least one `progress_updates` entry
|
||
3. **awaiting_qa -> awaiting_documentation** (pass-qa): Requires `pr_number` and substantive QA notes
|
||
4. **awaiting_documentation -> awaiting_pm_review**: Requires `docs_complete=True` (PR already exists from step 2 above)
|
||
5. **awaiting_pm_review -> awaiting_ceo_approval**: Must have `pr_number` set and all subtasks in a terminal state
|
||
|
||
### CEO Approval Workflow
|
||
|
||
Major tasks are escalated to CEO for final approval:
|
||
1. PM reviews and approves, escalates to `awaiting_ceo_approval`
|
||
2. CEO can:
|
||
- **Approve**: Merges PR, task -> `completed`
|
||
- **Request changes**: Task -> `needs_revision`
|
||
- **Cancel**: Task -> `cancelled`
|
||
|
||
## Data Models
|
||
|
||
### Core Models (roboco/models/)
|
||
|
||
| Model | Purpose |
|
||
|-------|---------|
|
||
| `Task` | Atomic unit of work with acceptance criteria |
|
||
| `Project` | Git repository configuration and CI/CD commands |
|
||
| `WorkSession` | Links agent work to task, tracks branch/commits/PR |
|
||
| `Agent` | AI agent with role, team, capabilities |
|
||
| `Notification` | Formal notification requiring acknowledgment |
|
||
| `Journal` | Agent personal log for reflections/learnings |
|
||
|
||
### Task Model Key Fields
|
||
|
||
```python
|
||
# Git configuration (all tasks follow git workflow)
|
||
task_type: TaskType # code, documentation, research, planning, design, administrative
|
||
project_id: UUID # Project this task works on (required)
|
||
branch_name: str # Branch for this task (auto-created on claim)
|
||
work_session_id: UUID # Active work session
|
||
|
||
# PR tracking (parallel execution in awaiting_documentation)
|
||
pr_number: int # GitHub/GitLab PR number
|
||
pr_url: str # Full URL to PR
|
||
docs_complete: bool # Documenter has finished
|
||
pr_created: bool # Developer has created PR
|
||
|
||
# Commits linked to task
|
||
commits: list[CommitRef] # All commits made for this task
|
||
```
|
||
|
||
## Communication Model
|
||
|
||
Agents coordinate via **task state + task detail fields**, not a channel/session backbone. Two comms primitives sit alongside that: **A2A** (`dm` + `read_a2a`, direct peer-to-peer, same-cell only — see `docs/rag/tools/a2a-tools.md`) for informal contact, and **Notifications** (`notify`, ack-required, sent by PMs/Board only) for formal signals.
|
||
|
||
Agent learnings (`note` scope='learning') broadcast as knowledge-share notifications only to other **agents** — the human / human-driven roles (CEO, prompter, secretary) are excluded, since agent knowledge-sharing is noise in a human's inbox.
|
||
|
||
## Key Principles
|
||
|
||
1. **Everything is a task** - All work is tracked and documented
|
||
2. **No work without a task** - Create task record first
|
||
3. **No task without acceptance criteria** - How do we know it's done?
|
||
4. **No closure without documentation** - Future agents need context
|
||
5. **Communication is constant** - Stream reasoning, log everything
|
||
6. **State is sacred** - If interrupted, state must be recoverable
|
||
7. **The Auditor sees all** - Quality monitored silently
|
||
8. **Commits linked to tasks** - Every commit references its task ID
|
||
9. **CEO approves major changes** - Escalation path for important work
|
||
|
||
## Agent Gateway
|
||
|
||
Agents do not call the API or per-domain MCP tools directly. They go through two thin MCP servers (`roboco-flow`, `roboco-do`) backed by the server-side **Choreographer** in `roboco/services/gateway/`. The Choreographer composes the existing services (TaskService, JournalService, GitService, etc.) into intent-verb sequences. Tracing, claim-locking, evidence assembly, and remediation hints are all centralized there.
|
||
|
||
Each agent gets a **spawn manifest** at `/app/tool-manifest.json` listing the verbs its role is allowed to call. The orchestrator builds the manifest from `roboco/services/gateway/role_config.py` and mounts it read-only into the agent container.
|
||
|
||
### Verb surface (canonical source: `lifecycle.intents_for_role`; every role also gets `i_am_idle`)
|
||
|
||
| Role | Flow verbs (beyond `i_am_idle`) |
|
||
|---------------|--------------------------------------------------------------------------------------------------|
|
||
| developer | `give_me_work`, `i_will_work_on`, `open_pr`, `i_am_done`, `i_am_blocked`, `resume`, `sync_branch`, `unclaim` |
|
||
| qa | `give_me_work`, `claim_review`, `pass_review`, `fail_review`, `i_am_blocked`, `resume`, `unclaim` |
|
||
| documenter | `give_me_work`, `claim_doc_task`, `i_documented`, `i_am_blocked`, `resume`, `unclaim` |
|
||
| cell_pm | `give_me_work`, `i_will_plan`, `delegate`, `complete`, `request_changes`, `submit_up`, `triage`, `unblock`, `escalate_up`, `reassign`, `resume`, `unclaim` |
|
||
| main_pm | `give_me_work`, `i_will_plan`, `delegate`, `complete`, `request_changes`, `submit_root`, `triage`, `triage_all`, `unblock`, `escalate_up`, `escalate_to_ceo`, `resume`, `unclaim` |
|
||
| pr_reviewer | `give_me_work`, `claim_pr_review`, `post_pr_review` (inbound external/fork PRs), `claim_gate_review`, `pr_pass`, `pr_fail` (in-path assembled-PR gate), `unclaim` |
|
||
| product_owner | `triage`, `escalate_to_ceo` |
|
||
| head_marketing| `triage`, `escalate_to_ceo` |
|
||
| auditor | `triage` (read-only — no `dm`) |
|
||
| prompter | (none beyond `i_am_idle` — not a delivery-lifecycle role; intake interviewer, human-only) |
|
||
| secretary | (none beyond `i_am_idle` — human-only chief-of-staff; reads company state + runs gated CEO directives) |
|
||
|
||
Content tools (do_server) — most roles: `commit`, `note`, `dm`, `read_a2a`, `evidence`. Delivery roles (developer / qa / documenter / cell_pm / main_pm) also get `draft_playbook` (draft a curated playbook for the KB). Product Owner additionally gets `propose_roadmap` (product_owner-only, authors the weekly board-roadmap cycle) and Head of Marketing additionally gets `propose_feature_spotlight` (head_marketing-only, drafts a feature-spotlight X post) — see "Board roadmap engine" / "RoboCo X account" below. Auditor is restricted to `note` (scope=reflect) + `evidence`, plus the playbook-curation verbs `approve_playbook` / `reject_playbook` / `archive_playbook` (a bounded, deliberate expansion — KB curation, not agent comms, so its no-`dm` restriction holds). The `pr_reviewer` posts its change-request on the PR itself (no agent comms). The `prompter` (intake) and `secretary` are restricted to `note` + `evidence` — human-only, no `dm`/`notify`. The `note`/journal write returns as soon as the entry is persisted; RAG indexing (Ollama embedding) runs fire-and-forget, so the tool no longer times out under concurrent load.
|
||
|
||
### MCP servers running per agent container
|
||
|
||
| Server | Purpose |
|
||
|----------------------|----------------------------------------------------------------------|
|
||
| `roboco-flow` | Intent verbs (give_me_work, i_am_done, claim_review, complete, ...) |
|
||
| `roboco-do` | Content tools (commit, note, dm, read_a2a, evidence) |
|
||
| `roboco-git-readonly`| Read-only git: status, log, diff, branches |
|
||
| `roboco-optimal` | RAG: `roboco_ask_mentor`, `roboco_kb_search` |
|
||
| `roboco-docs` | Project docs file management (selected roles) |
|
||
|
||
Every verb returns a standardized **Envelope**:
|
||
- ok: `{status, task_id, next, evidence?, context_briefing}`
|
||
- error: `{error, message, remediate, missing}`
|
||
|
||
The `next` field tells the agent what to call next; the `remediate` field on errors tells them exactly how to fix and retry. Agents should not guess state — trust the response. The verb runner re-checks the task after each composed atomic action and, on a concurrent mid-verb state change, fails fast with a clean `INVALID_STATE` (re-fetch + re-issue) rather than crashing on a `None` dereference.
|
||
|
||
## Agent Providers
|
||
|
||
Agent backends are pluggable. `roboco/llm/providers/` defines an `AgentProvider` lifecycle ABC (`base.py`) and a `ProviderRegistry` keyed by `ModelProvider` (`registry.py`), with `ClaudeCodeProvider` (default) and `GrokCliProvider`. The orchestrator resolves a provider at spawn from the agent's `ModelProvider`; when no dedicated provider is registered it falls back to the built-in Claude Code spawn. `ModelProvider` (`roboco/models/base.py`) is `ANTHROPIC` (default), `GROK`, `LOCAL`, `OLLAMA_CLOUD`, `OPENAI` (reserved). The seam is additive: only `GROK` routes through `GrokCliProvider`; Anthropic / Ollama Cloud / self-hosted spawns are unchanged, and every provider gets the same MCP gateway + tool-manifest wiring by construction.
|
||
|
||
**Grok runtime.** `GROK` agents run xAI's official `grok` CLI (model `grok-build`) authenticated by a **SuperGrok subscription**, not a metered API key — so a Grok workforce can't stall mid-task on out-of-credits. The host `~/.grok/auth.json` is mounted **read-only** into each agent (`GrokCliProvider._append_grok_auth_mount`; `ROBOCO_HOST_GROK_DIR` is the host mount source, set up once with `grok login`). It reaches parity with the Claude path by construction: same MCP gateway + manifest, per-role tool-removal and git-operation deny rules, a prompt-injection guard on the task prompt, headless tool auto-approval, and per-agent token/cost capture from the grok session store. It covers both one-shot delivery roles and the interactive Intake (Prompter) and Secretary chats (per-turn `grok -p` with session resume).
|
||
|
||
**Token auto-refresh.** The grok access token has a fixed ~6h server-set TTL and the CLI cannot refresh it headlessly — on an expired token it hangs forever at an interactive login prompt. The orchestrator mints a fresh token from the offline-access refresh token (xAI's OIDC `refresh_token` grant) before expiry and rewrites the shared `auth.json` in place (`roboco/llm/providers/grok_auth.py` `refresh_if_stale`, run once per dispatch tick; the orchestrator's `~/.grok` mount is read-write so it can rewrite it). As a backstop the agent entrypoint runs `python -m roboco.llm.providers.grok_auth --check` and refuses to start (exit 78) on a missing/expired token instead of hanging.
|
||
|
||
## Self-Healing & Feature Flags
|
||
|
||
**Self-healing CI loop (default-off).** RoboCo can watch its own repository's CI (a single named workflow) and, on a detected regression, open a fix task that is held out of dispatch until the CEO approves it (it terminates at `awaiting_ceo_approval`), then dispatch it through the normal delivery flow. It is dormant by default and armed by `ROBOCO_SELF_HEAL_ENABLED` plus a second opt-in `ROBOCO_SELF_HEAL_ORIGINATE_ENABLED`; origination is bounded by `ROBOCO_SELF_HEAL_MAX_OPEN_TASKS` / `_MAX_PER_CYCLE` so it can't flood the backlog. It never auto-merges or self-deploys (`roboco/services/self_heal_engine.py`).
|
||
|
||
**Multi-repo CI-watch (default-off).** The fan-out generalization of self-heal: instead of RoboCo's single own repo, it watches every project the operator opts into (`projects.ci_watch_enabled`, migration 048) and, on a red CI conclusion on that project's default branch, opens one fix task into that project's lifecycle that rides the normal delivery flow (+ PR-review gate) and never auto-merges. It reuses the exact hardened per-project `GitService.get_latest_ci_conclusion` (a missing signal is "unknown", never a false green; per-project errors are isolated and never abort the sweep), and is bounded + deduped per repo by `git_url` (a monorepo's cell-projects share one fix task) with per-cycle / rolling caps. Armed by `ROBOCO_CI_WATCH_ENABLED` (+ `_INTERVAL_SECONDS` / `_MAX_OPEN_TASKS` / `_MAX_PER_CYCLE` / `_DEFAULT_WORKFLOW`) and per-project `ci_watch_enabled` / `ci_watch_workflow`; `MultiProjectCITelemetrySource` (`roboco/services/telemetry/source.py`) + `CiWatchEngine` (`roboco/services/ci_watch_engine.py`) + a dedicated orchestrator `_ci_watch_loop`. The single-repo self-heal loop is untouched.
|
||
|
||
**Dependency-update bot (default-off).** A per-project engine mirroring the self-heal/CI-watch shape: weekly (default) it probes whether a dependency upgrade would change a project's lockfiles and, if so, opens one "update dependencies" task that rides the normal delivery flow (+ PR-review gate) and never auto-merges. Detection is read-only — `WorkspaceService.dry_upgrade_changes_lockfile` runs the project's `dep_update_command` (e.g. `uv lock --upgrade`) in a throwaway clone of the read clone and diffs the lockfile paths (`dep_update_paths`, or inferred `uv.lock`/`pnpm-lock.yaml`); the read clone is never mutated, nothing is committed/pushed, and a null/failing command originates nothing (fail-safe). A project participates only when `projects.dep_update_command` is set (migration 049); bounded + deduped per `git_url` with per-cycle/rolling caps. Armed by `ROBOCO_DEP_UPDATE_ENABLED` (+ `_INTERVAL_SECONDS` default 604800 / `_MAX_OPEN_TASKS` / `_MAX_PER_CYCLE`); `DepUpdateEngine` (`roboco/services/dep_update_engine.py`) + a dedicated `_dep_update_loop`.
|
||
|
||
**Gated release manager (default-off).** The autonomy that automates cutting a release up to the decision. A default-off background loop (`ReleaseManagerEngine` + `_release_manager_loop`) runs the deterministic readiness sweep (`ReleaseReadinessService.assess`, `roboco/services/release_readiness.py`) — diff-since-tag → conventional-commit classification → semver bump → version-reference completeness (the missed-ref guard) → CHANGELOG completeness → docs-drift (agent count) → migration single-head → gate state — and, past a threshold (`ROBOCO_RELEASE_MIN_COMMITS`, or any feat/security) with a green gate, originates ONE **release proposal** held for the CEO. The proposal is a `source='release_manager'` task owned by the Secretary, HELD (`confirmed_by_human=False`) and skipped by every dispatcher — acted on only by the CEO-gated routes, never delivered. The CEO approves or rejects-with-changes in the panel (`release-proposal-card.tsx`; `GET/POST /api/release/proposal{,/approve,/reject}`, CEO-only); approval runs the **fail-closed** `ReleaseExecutor` (`roboco/services/release_executor.py`): write the bumps across the canonical set (derived from the previous `chore(release):` commit) + the CHANGELOG entry, run `make quality` (abort before commit on red), commit `chore(release): X.Y.Z` (signed) + push, wait for green release-commit CI (abort before publish on red), then `gh release create vX.Y.Z`. Idempotent (an already-published version is a no-op) and never publishes without the CEO. Correctness is deterministic code, not agent judgment; the only generative step is the CHANGELOG prose, which the CEO reviews. Armed by `ROBOCO_RELEASE_MANAGER_ENABLED` (+ `ROBOCO_RELEASE_MIN_COMMITS` / `_INTERVAL_SECONDS`). Auto-deploy stays out of scope — publishing builds images; deploying to the NAS is the CEO's manual step.
|
||
|
||
**Organizational memory loop (default-off).** Closes the learn→reuse loop so agents stop cold-respawning blind. Three parts, all gated by `ROBOCO_ORG_MEMORY_ENABLED`: ① **capture** — at task completion `TaskService._completion_learnings_for` distills ONE high-signal lesson (Problem→Approach→Gotcha, ≤120 words) via the local model (`MemoryDistiller`, `roboco/services/memory_distiller.py`) instead of the noisy raw-notes/duration capture (flag-off keeps the legacy capture); journal indexing excludes `is_private` reflections from the shared corpus. ② **retrieve (keystone)** — on claim, `_briefing_for` injects `context_briefing["institutional_memory"]`: top-K (`ROBOCO_ORG_MEMORY_TOP_K`) relevance-floored (`ROBOCO_ORG_MEMORY_MIN_SCORE`) lessons + approved playbooks from a role-shaped query (`EvidenceRepo.similar_memory` over the LEARNINGS + PLAYBOOKS pgvector indexes); below the floor nothing is injected (no briefing bloat). ③ **playbooks** — a first-class curated procedure store: `PlaybookTable` (migration 050), the `PLAYBOOKS` OptimalService index, the `draft_playbook` content verb (delivery roles), Auditor `approve_playbook`/`reject_playbook`/`archive_playbook` curation (approval indexes it), and the panel review queue (`playbook-review-queue.tsx`; `/api/playbooks` Auditor/CEO routes). Distillation runs on the local model only — never a cloud LLM in the hot path; every step is best-effort (a failure never blocks completion or the briefing).
|
||
|
||
**Sandboxed dev DB/Redis (default-off).** Per-project opt-in (`projects.sandbox_services`, migration 057): when armed (`ROBOCO_SANDBOX_DB_ENABLED`), each opted-in project's agent spawn gets orchestrator-provisioned throwaway `postgres:16-alpine` / `redis:8-alpine` **sibling containers** (random per-sandbox creds, tmpfs pg data dir, memory/cpu-capped, labeled `roboco.sandbox=1`), injected as `ROBOCO_TEST_DB_*` / `ROBOCO_TEST_REDIS_*` **in place of** the legacy prod-creds gate-env injection (`_append_gate_env`, which points agents at RoboCo's own production Postgres under `ROBOCO_TOOLCHAIN_MATCH_ENABLED`) — sandbox replaces, never coexists with, prod creds. Lifetime tracks the agent container 1:1: teardown at every removal path plus an orphan janitor at startup + each reaper tick (grace-windowed so a sweep can't reap a sandbox whose spawn is still mid-flight; the pre-spawn stale-clear likewise spares the just-provisioned sandbox). Provisioning failure refuses the spawn (fail-loud); docker-in-agent stays structurally absent. `SandboxProvisioner` (`roboco/runtime/sandbox.py`), wired in the orchestrator spawn path.
|
||
|
||
**Cloud auth via FastAPI Users (default-off).** Lets the panel/API be safely exposed beyond localhost without touching the CEO's local no-login flow while off. Gated by `ROBOCO_CLOUD_AUTH_ENABLED` (+ `ROBOCO_CLOUD_AUTH_EMAIL` / `_PASSWORD` / `_SECRET` / `_COOKIE_MAX_AGE`; `Settings` fails loud at startup if the flag is on with no secret). Off: `get_agent_context` (`roboco/api/deps.py`) and the WS `_require_panel_token` gate (`roboco/api/websocket.py`) are byte-for-byte unchanged (header-trust). On: header-trust is dead for humans — any agent-role claim (`ceo` OR a privileged `main_pm`/`cell_pm`/board role) with no valid HMAC token or session cookie is 401, closing the header-spoof hole on the host-published `:8000` port for every role, not just `ceo` (real agents always carry a signed token, so they're unaffected); the agent-fleet HMAC path (and the orchestrator's `system` self-PATCH) keeps working unmodified in both modes; a valid session cookie authenticates as the single seeded CEO user. New `users` table (migration 058, `UserTable` in `roboco/db/tables.py`) backs FastAPI Users' `SQLAlchemyUserDatabase`; no registration router — `roboco/api/auth/seed.py` idempotently upserts exactly one row from env at startup (by primary key, so an email change renames the row instead of duplicating it). `roboco/api/auth/backend.py` wires a **cookie** transport (httponly, secure, samesite=lax) + a `JWTStrategy` subclass that binds each token to a fingerprint of the current `hashed_password`, so rotating the seeded password invalidates every prior session. Session lifetime is **sliding**: every authenticated request through `get_agent_context` re-mints + re-sets the cookie (`_slide_session_cookie`), so an active session never expires — only genuine inactivity past `cloud_auth_cookie_max_age` (default 30 days) logs out. `GET /api/auth/status` is always mounted (public); `/api/auth/login` + `/api/auth/logout` mount only when armed (`roboco/api/auth/routes.py`, mirroring `apply_guard`'s conditional mount). Panel: `(auth)/login/page.tsx` + `proxy.ts` (the Next 16 rename of `middleware.ts`; probes `/auth/status` over the docker-internal orchestrator URL, not through nginx, and fails open to "off" on any probe error/timeout) gate the `(dashboard)` group; `client.ts` adds `withCredentials` + a 401→`/login` redirect. nginx needs no changes (`/api/auth/*` rides the existing `/api/` proxy location) — but its own static `X-Agent-Token` injection (`ROBOCO_PANEL_AGENT_TOKEN`) is itself a valid HMAC credential that bypasses login when present, so a deployment arming cloud auth for real public exposure should leave that token unset (the two are alternative human-auth tiers, not layered).
|
||
|
||
**RoboCo X account (default-off).** The Head-of-Marketing voice on X (Twitter): drafts a post when a release publishes, drafts replies to meaningful mentions, and — a third, independent capability — periodically investigates RoboCo's own shipped features and drafts a spotlight for an under-publicized one. NOTHING auto-posts across any of the three; every tweet is held in a panel queue for the CEO to edit/approve. Gated by `ROBOCO_X_ENGINE_ENABLED` (+ `_MENTIONS_INTERVAL_SECONDS` / `_MENTIONS_MAX_PER_CYCLE` / `_MENTIONS_MIN_ENGAGEMENT` / `_MAX_OPEN_POSTS` / `X_ACCOUNT_USER_ID`); inert without credentials regardless. Mirrors the `ReleaseManagerEngine` held-artifact shape: `XEngine` (`roboco/services/x_engine.py`) originates a held task (`source` `x_post` / `x_reply` / `x_feature`, `confirmed_by_human=False`, Secretary-owned, skipped by every dispatcher) whose marker payload carries a body clamped to 280 chars. Release posts hook `ReleaseProposalService.approve`'s publish-success branch via a small `draft_release_post` seam; mentions ride a dedicated `_x_mentions_poll_loop` (no webhook infra exists) deduped by a `x_seen_mentions` ledger + per-cycle/open caps — both are **local-model-drafted** (never a cloud LLM in the hot path). The spotlight half is the one exception to "no agent spawn": gated by its own sub-switch `ROBOCO_X_FEATURE_SPOTLIGHT_ENABLED` (+ `_INTERVAL_SECONDS`, default 3 days) on top of `x_engine_enabled`, `_x_feature_spotlight_loop` opens a held PENDING exploration task (`source=x_feature_exploration`, team=Board, assigned to Head of Marketing, carrying a `x_seen_features` dedup-ledger snapshot marker) that `_dispatch_pm_work` routes (mirroring `ROADMAP_SOURCE`) to a one-shot real cloud-LLM spawn of the Head of Marketing — full read tools, investigates CHANGELOG.md/feature-flags/docs/map/charter/KB, calls the Head-of-Marketing-only `propose_feature_spotlight` do-tool exactly once, which marks the feature slug seen (`x_seen_features` table, migration 061) and materializes a brand-new `source=x_feature` held draft (completing the exploration task as a side effect — a deliberate asymmetry from `propose_roadmap`, which instead leaves its own task open). The four OAuth 1.0a secrets live Fernet-encrypted in a singleton `x_credentials` row (migration 059, all-or-nothing set/clear, mirroring the git-token pattern; the API only ever returns `has_credentials`) — decryption is server-side only, agents never hold creds or egress. `XPostService.approve` (CEO-only route) is the ONLY caller of `x_client.post_tweet`: it posts under a Redis single-flight lock, **re-reads the committed task state inside the lock and commits COMPLETED before releasing** so a concurrent approve can't double-post, and is idempotent (an already-posted draft is a no-op). The hand-rolled OAuth 1.0a HMAC-SHA1 signer (`roboco/services/x_client.py`) adds no dependency; a `NullXClient` makes the unconfigured path a graceful no-op (research-engine posture). All three draft kinds share one voice: `XEngine._voice_guide` reads the CEO-editable `company_goals.brand_voice` charter field (migration 061, panel-editable in Business → Goals) and appends it to a generic baseline (`_HOM_VOICE`) — the baseline alone until the CEO supplies a real sample. Panel: `x-post-queue.tsx` (editable draft + 280 counter, approve/reject, a `sourceMeta`-driven label/icon per source including "Feature spotlight") + `x-credentials-card.tsx` (4 write-only secret inputs).
|
||
|
||
**RoboCo video engine (default-off).** Bespoke motion-graphics videos (release announcements, feature spotlights, on-demand CEO briefs) authored by a UX/UI dev and distributed to X/TikTok — nothing renders or posts without the flags on, and nothing posts without an explicit CEO approval. Gated by `ROBOCO_VIDEO_ENGINE_ENABLED` (+ sub-switches `ROBOCO_VIDEO_ON_RELEASE` / `ROBOCO_VIDEO_ON_SPOTLIGHT`, and `_MAX_OPEN_POSTS` / `_RENDER_INTERVAL_SECONDS` / `_RENDER_TIMEOUT_SECONDS` / `_REQUEST_TIMEOUT_SECONDS` / `_OUTPUT_DIR`); a CEO on-demand brief rides `POST /video/request` regardless of the release/spotlight sub-switches. A project opts in via `projects.video_engine_enabled` (migration 063, mirroring `ci_watch_enabled`): the global flag arms the subsystem, the per-project flag opts a repo into authoring against its `motion/` — `VideoEngine._opted_in_project` no-ops `open_video_task` until the operator flips it in the panel's edit-project dialog. Two task kinds mirror the XEngine/ReleaseManagerEngine "originate a CEO-scoped artifact" shape but split across the real delivery lifecycle: `VideoEngine.open_video_task` (`roboco/services/video_engine.py`) opens a normal, ASSIGNED **authoring task** (`source=video`, `confirmed_by_human=True`, team=UX/UI, balanced across `ux-dev-1`/`ux-dev-2` by open-task count) that dispatches like any other pre-assigned code task — NOT held, NOT in any dispatcher's skip bucket. The assigned dev builds a HyperFrames HTML composition under `motion/compositions/<id>/` and calls the UX/UI-team-gated `propose_video` do-tool (metadata-only: composition id, input props, per-platform captions — every developer role carries the tool on their manifest, but the runtime `_caller_team` check rejects a be-dev/fe-dev) to stamp the task's `video_draft` marker, then commits + `open_pr` through the normal PR-review gate. Once that authoring task reaches `completed`, the orchestrator's `_video_render_loop` (bounded retry, `_MAX_VIDEO_RENDER_ATTEMPTS`) tars the merged `motion/` dir from the project's read-clone and POSTs it to the credential-free **video-renderer sidecar** (`VideoRenderer` in `roboco/services/video_renderer_client.py`, `ROBOCO_VIDEO_RENDERER_BASE_URL`) to render both the 9:16 and 1:1 cuts to MP4 (`video_output_dir`); on success `VideoEngine._originate_video_post` materializes a held **video-post draft** (`source=video_post`, `confirmed_by_human=False`, Secretary-owned, skipped by every dispatcher) carrying `mp4_paths` (`{vertical, square}` absolute paths) + the per-platform captions. The CEO reviews it in the panel's video queue (`video-post-queue.tsx`; `GET /video/posts` lists drafts including `mp4_paths` so the panel knows which cuts exist, `GET /video/posts/{id}/media?cut=vertical|square` streams the MP4 bytes for the preview player, CEO-gated throughout) and edits captions / approves / rejects. `VideoPostService.approve` (`roboco/services/video_post_service.py`) is the ONLY caller of the X-v2 (`XVideoPoster` in `x_video_client.py`) and TikTok inbox-upload (`TikTokPoster` in `tiktok_client.py`) posters; because a video upload + transcode/poll can run well past a minute, the critical section runs under a heartbeat-renewed Redis mutex (`heartbeat_mutex.py`, mirroring `ReleaseProposalService`'s release-execute lock shape) rather than a flat lock, commits each platform's posted-id durably before attempting the next (a partial failure never re-posts an already-succeeded platform on retry), and is idempotent (an already-`COMPLETED` draft returns the stored ids without calling a poster again). TikTok's four OAuth2 secrets live Fernet-encrypted in a singleton `tiktok_credentials` row (mirroring the git-token / `x_credentials` pattern; the API only ever returns `has_credentials`) — set via the panel's TikTok credentials card. `NullVideoRenderer` / `NullXVideoPoster` / `NullTikTokPoster` make every unconfigured leg a graceful no-op rather than a crash.
|
||
|
||
**Board roadmap engine (default-off).** The Board originating strategic work: on a weekly interval (`ROBOCO_ROADMAP_ENGINE_ENABLED` + `_INTERVAL_SECONDS` / `_MIN_ITEMS_PER_CYCLE` / `_MAX_ITEMS_PER_CYCLE`) `RoadmapEngine` (`roboco/services/roadmap_engine.py`) opens ONE held **exploration** task (`source="board_roadmap"`, `confirmed_by_human=False`, PENDING, Product-Owner-assigned, `Team.BOARD`), deduped to one open cycle at a time. A dedicated one-shot `_dispatch_roadmap_exploration` spawns the Product Owner **solo** — deliberately NOT `_handle_board_assigned_task` (which would also spawn Head of Marketing and fire the Approve-&-Start handoff, both wrong for a PO-authored cycle) — reusing the `_board_dispatched` one-shot tracker + respawn breaker, and short-circuiting once the cycle is authored. The PO explores (read-only git, KB/RAG, metrics, releases, charter, optional web research) and makes ONE `propose_roadmap` call (a content verb gated to `product_owner` only, `_ROADMAP_ROLES`; wired through the do_server/Choreographer like `pitch`) authoring a **themed cycle** — a one-line goal + 3-7 item drafts — persisted as a `roadmap_cycle` marker on the exploration task (no table/migration). The CEO acts per-item in the panel roadmap queue (`roadmap-review-queue.tsx`; `/api/roadmap/cycles{,/items/{id}/approve,/items/{id}/reject}`, CEO-only): approve materializes that item as a BACKLOG task (`source="roadmap"`, no assignee — never auto-starts; normal PM activation picks it up) via `PrompterService.create_task_from_draft`, reject records a reason; when every item is terminal the exploration task completes (`RoadmapService`, idempotent per item). Dispatchers skip `board_roadmap` (never delivery work). `create_task_from_draft` honors a draft-declared `source` only from a `{prompter, roadmap}` whitelist — an LLM-authored draft can't impersonate a privileged origin.
|
||
|
||
**Fable-mode (default-off).** Full opus-fable-playbook adoption: makes the fleet behave more like Fable 5 on the existing model tiers (the tiers stay — Fable 5 the model is not an option). Two levers, both gated by `ROBOCO_FABLE_MODE_ENABLED`: ① **doctrine** — `fable_doctrine_layer()` (`roboco/agents/factories/_base.py`) composes the vendored behavioral doctrine (`agents/prompts/doctrine/fable.md`, from `github.com/rennf93/opus-fable-playbook` MIT `output-styles/fable.md`, YAML frontmatter stripped) into `compose_prompt`'s layer tuple immediately after `base.md` — universal cross-role doctrine, the same tier as the base rules, ahead of role/team/identity layers so those keep their specificity precedence. ② **hooks** — 5 vendored scripts under `docker/scripts/fable-*.sh` (stop-gate, bash-discipline, honesty-nudge, prompt-nudge, precompact; `session-start.sh` deliberately SKIPPED — its doctrine card is redundant with ① and its output-style check is inapplicable here) are installed alongside RoboCo's own hooks, never replacing them: `AgentOrchestrator._fable_hook_groups()` appends them AFTER RoboCo's own per-event entries in the Claude-path settings.json (isolated into its own helper to protect `_generate_agent_settings`'s xenon budget); the grok path installs only `honesty-nudge` (`write_grok_fable_hooks`, `roboco/llm/providers/grok_cli_config.py`) — a deliberately conservative V1 scope, since a grok `PreToolUse`/`Stop` hook deny cancels the entire run (verified live) while `PostToolUse` never denies. Off by default: the spawn path (composed prompt, settings.json, grok hooks) is byte-for-byte unchanged when the flag is off. No new eval harness — measurement rides the existing rework/spawn-waste/`revision_count` dashboard (see "Delivery observability" below). Armed on the NAS deploy like the rest; left OFF in `docker-compose.registry.yml`.
|
||
|
||
**Ponytail (bundled with Fable-mode).** Rides `ROBOCO_FABLE_MODE_ENABLED` — no separate flag. Vendors the ponytail "lazy senior dev" build-laziness doctrine (`agents/prompts/doctrine/ponytail.md` + ethos sibling, MIT, Copyright (c) 2026 DietrichGebert — trimmed, YAML frontmatter stripped) into every composed system prompt via `ponytail_doctrine_layer` (`roboco/agents/factories/_base.py`), slotted immediately after the Fable doctrine layer and gated on the same flag. Role-scoped: developers (`AgentRole.DEVELOPER`) get the full ladder (YAGNI → reuse-in-this-codebase → stdlib → native-platform → installed-dep → one-line → minimal); every other role gets the ethos-only cut (`ponytail-ethos.md`) — the code-mechanics rungs are dropped so they can't leak into prose artifacts (task plans, review notes, docs). Both files carry a 5-point RoboCo preamble (the ethos sibling adds a 6th: free-text field obligations) that makes the ladder yield to the Architectural Conventions Standard (placement), the 80% coverage gate + QA review + self-verification, the per-team design bar, task hygiene (everything-is-a-task / commits-linked / state-is-sacred), and reviewer feedback (`needs_revision` / `pr_fail` / `request_changes`) — the overlap mitigation is scoping, not deletion, and it rides ponytail's own "when NOT to be lazy" clause. Developer intensity is tunable via `ROBOCO_PONYTAIL_INTENSITY` (`lite` / `full` / `ultra`, default `full`; `roboco/config.py` `ponytail_intensity`, a string value — not a feature flag): `full` enforces the ladder, `lite` builds what's asked and names the lazier alternative, `ultra` is YAGNI-extremist (deletion before addition, challenge the requirement). Non-developers get no dial — `ultra` is wrong for prose artifacts, so the ethos runs a fixed restrained stance. Prompt-only: no hooks, no grok-path changes — ponytail adds no hook surface, so bundling it under the Fable flag changes only the composed prompt, not the spawn hooks. The Fable flag's description in `roboco/config.py` names both doctrines.
|
||
|
||
**Feature flags / company-in-a-box.** Env-gated, default-off subsystems toggle from the panel's Settings → Feature Flags card (`panel/src/components/settings/feature-flags-card.tsx`) instead of hand-editing env: web research (`ROBOCO_RESEARCH_ENABLED`), the strategy engine (`ROBOCO_STRATEGY_ENGINE_ENABLED`), pitch provisioning (`ROBOCO_PROVISIONING_*`), external / internal PR review, the agent-runtime toolchain match (`ROBOCO_TOOLCHAIN_MATCH_ENABLED`), the architectural-conventions standard (`ROBOCO_CONVENTIONS_ENABLED`), gateway-health recovery (`ROBOCO_GATEWAY_HEALTH_ENABLED`), multi-repo CI-watch (`ROBOCO_CI_WATCH_ENABLED`), the dependency-update bot (`ROBOCO_DEP_UPDATE_ENABLED`), the gated release manager (`ROBOCO_RELEASE_MANAGER_ENABLED`), the organizational memory loop (`ROBOCO_ORG_MEMORY_ENABLED`), the sandboxed dev DB/Redis (`ROBOCO_SANDBOX_DB_ENABLED`), the RoboCo X account (`ROBOCO_X_ENGINE_ENABLED`), the RoboCo video engine (`ROBOCO_VIDEO_ENGINE_ENABLED`), the board roadmap engine (`ROBOCO_ROADMAP_ENGINE_ENABLED`), Fable-mode (`ROBOCO_FABLE_MODE_ENABLED`), and the self-heal flags above. Cloud auth (`ROBOCO_CLOUD_AUTH_ENABLED`) is deliberately NOT on this card — like `ROBOCO_DB_NETWORK_ISOLATED`, it's a compose/env-coupled flag a runtime toggle can't safely flip mid-session. A toggle persists in the settings store and takes effect on the next backend restart; an unset flag falls back to its environment / config default.
|
||
|
||
## Architectural Conventions Standard
|
||
|
||
**Per-project architectural standard (default-off).** Beyond the `make`-style gates (which check syntax/types/tests, not *where code lives*), each project can carry a repo-canonical `.roboco/conventions.yml` — an architecture map (which definition *kinds* belong in which modules), a toggleable rule set, custom regex rules, and waivers — so an agent cannot land a Pydantic model defined inside a router or a `# noqa` / `# type: ignore`. Placement of a *helper* (any top-level function) only **warns** — too blunt to hard-block; `thin_routes` doesn't count an explicit `db.commit()`; and a small allowlist of unavoidable framework suppressions (ruff `TC001`–`TC003`, pydantic `prop-decorator`) is exempt. Gated by `ROBOCO_CONVENTIONS_ENABLED`; fully inert when off. RoboCo itself ships a canonical `.roboco/conventions.yml`.
|
||
|
||
**Effective map.** Consumers read the *effective* map — auto-derived defaults (from a repo scan + `BUILTIN_RULES`, excluding `tests/`/`docs/` trees) overlaid by the committed file — so behaviour is identical whether the file is present, absent, or partial. `ConventionsService` (`roboco/services/conventions.py`) builds it, caches it per `(project, HEAD sha)` in `project_conventions_cache` (migration `043`), renders the per-task baseline constraints + the ambient prompt block, and scaffolds/restores the file via a PR (`GitService.open_conventions_pr`). The committed file + scan are read from a dedicated project-level **read clone** the service ensures on demand (`WorkspaceService.ensure_read_clone`, pinned to the default branch's HEAD) — the backfill that makes the standard resolve even for a project created before it existed, with no manual `workspace_path`. The schema lives in `roboco/foundation/policy/conventions/` (pure).
|
||
|
||
**Validator.** A single Python CLI, `python -m roboco.conventions check --root <repo> --files <a> <b> ...` (`roboco/conventions/`), uses tree-sitter (Python + TypeScript grammars, shipped in the agent image) to classify each changed definition and flag forbidden placements + hygiene + custom-rule matches as JSONL findings, after waiver filtering. Precision over recall (it abstains when uncertain so a `block` gate can't false-positive-strand a task) and fail-loud (a validator that cannot run exits 3 so the gate blocks, never silently passes).
|
||
|
||
**Threading + enforcement.** The standard reaches the work two ways: an ambient "Architectural Standard" block injected at spawn (`compose_prompt`) and an auto-attached `## Constraints` section on every project task (`TaskService.create`). Enforcement is deterministic: a `block`-level finding refuses `i_am_done` (dev pre-submit) and `pr_pass` (the in-path PR gate) with the offending `file:line` + fix hint; findings also surface in QA's `claim_review` evidence (`convention_findings`). A false positive is relieved by a `waiver` the dev commits in their branch — accountable, reviewed in the PR. The panel's per-project Conventions tab (in the edit-project dialog) shows the map + health and offers Save / Restore.
|
||
|
||
## Design Bar
|
||
|
||
**FE/UX-UI design bar (prompt-only, always on).** Frontend and UX/UI team agents carry a design-taste bar distilled from `Leonxlnx/taste-skill` (MIT) in their team prompts, so agent-authored UI stops defaulting to generic-AI layout/fonts/motion. It's a `## Design bar` section appended to `agents/prompts/teams/frontend.md` and `agents/prompts/teams/ux_ui.md`, reached by every cell role on those teams (dev/QA/PM/Documenter) via the team prompt layer, plus a pointer in the shared `agents/prompts/roles/developer.md` so `fe-dev`/`ux-dev` know to look for it without leaking the content into `be-dev`'s prompt. It covers three tuning dials — `DESIGN_VARIANCE` / `MOTION_INTENSITY` / `VISUAL_DENSITY` (1-10 each; dense product UI like the panel defaults to `2-3 / 2-3 / 7-8`) — plus typography/hierarchy, spacing/layout, motion, and "AI tells to avoid" rules, scoped to respect a project's existing design system (fonts, colors, radius) rather than silently override it. Prompt-only: `compose_prompt` itself is unchanged, no new verb/gate/state; guarded by `tests/unit/agents/test_design_bar_layer.py`.
|
||
|
||
## MegaTask (sequenced batch intake)
|
||
|
||
**MegaTask** lets the CEO describe several tasks in one Intake chat and ship them as one collision-aware, sequenced batch — even across projects that don't share a codebase (the motivating case: a SaaS app + its OSS core engine + a framework adapter). It is a **core capability, not a feature flag** (additive + opt-in by nature: proposed only when the CEO asks for several tasks; single-task intake is byte-for-byte unchanged), branded "MegaTask" on every user-facing surface while internal names stay technical (`batch_id`, `SequencingService`).
|
||
|
||
**The umbrella model.** A MegaTask's identity is a real **umbrella** task — branchless, no PR of its own — over N **root-subtasks**, each a real Main-PM coordination root with its own `project_id`, branch, and PR. Hierarchy: Umbrella (Main PM) → N Root-subtasks (Main PM) → Cell tasks (cell PMs) → Dev subtasks. One extra Main-PM layer on top of the normal model. The umbrella is the single board-review / CEO-approve / Main-PM-coordinate unit, so the batch plugs into the existing coordination-root flow for free (task tree, progress rollup, CEO queue).
|
||
|
||
**Identity predicate (single source of truth).** `roboco/foundation/policy/batch.py`: `is_batch_umbrella` (`batch_id` set AND `parent_task_id` None), `is_batch_root_subtask` (`batch_id` set AND parented), `is_branchless_coordination` ((no-project AND product) OR umbrella). Every git-exemption site consults it so the umbrella's exemptions can't drift: the orchestrator's `_is_coordination_task`, the claim→in_progress branch gate (`GitContext.is_coordination`), `_ensure_branch_for_task` (returns `""` for an umbrella), and the CEO-reject routing. `submit_root` hard-rejects an umbrella (it assembles no PR); umbrella completion reuses the existing branchless path (`all_subtasks_terminal`, PR waived → escalate to CEO).
|
||
|
||
**Sequencing.** The pure `SequencingService.analyze(surfaces, cell_of, cell_capacity)` (`roboco/services/sequencing.py`; schema in `roboco/foundation/policy/sequencing/`) turns each draft's collision surface — `intends_to_touch` (globs), `adds_migration`, `touches_shared` — into a dependency DAG + Kahn-layered **waves**: file-overlap serializes (more-important first by `(priority, idx)`), migration-adders chain serially, a shared-surface edit runs after each non-shared task it overlaps (file-overlap-conditioned), independent tasks run in parallel; cell-contention only warns. Correctness lives in code, not agent judgment. The columns `tasks.batch_id` + `intends_to_touch` / `adds_migration` / `touches_shared` are migration **046**.
|
||
|
||
**Intake + create path.** The intake chat can be scoped to a **MegaTask** (a multi-project picker → `StartLiveRequest.project_ids`); the orchestrator clones each repo (`_clone_intake_scope` / `_slugs_for_project_ids`, the multi-repo machinery products already used). The intake agent proposes the whole batch with one **`propose_batch`** tool call — wired on both runtimes (the Claude SDK driver emits one `batch` stream chunk; the grok `intake_server` POSTs a `batch` relay event). The panel's third intake scope accumulates it into a Review-MegaTask card → `POST /prompter/live/{session}/confirm-batch`. `PrompterService.confirm_live_batch` builds the umbrella + N root-subtasks (via `create_task_from_draft` + a `BatchPlacement`) and wires the analyzer edges through `add_dependency`. The Board route holds the root-subtasks in BACKLOG until `approve_and_start` releases them (`_activate_batch_root_subtasks`); the Main-PM route dispatches wave 0 at once. The Product Owner + Head of Marketing review the whole batch (their identity prompts carry a MegaTask section).
|
||
|
||
## Services
|
||
|
||
Core services in `roboco/services/`:
|
||
|
||
| Service | Purpose |
|
||
|---------|---------|
|
||
| `TaskService` | Task CRUD and state transitions |
|
||
| `WorkSessionService` | Git session management, PR lifecycle |
|
||
| `WorkspaceService` | Multi-agent workspace resolution and cloning |
|
||
| `ProjectService` | Project/repository management |
|
||
| `NotificationService` | Formal notifications |
|
||
| `JournalService` | Agent journals and entries |
|
||
| `OptimalService` | RAG queries (in-house pgvector engine) |
|
||
| `PermissionsService` | Role-based access control |
|
||
|
||
## Configuration
|
||
|
||
Key settings in `roboco/config.py` (env prefix: `ROBOCO_`):
|
||
|
||
```bash
|
||
# Database
|
||
ROBOCO_DATABASE_HOST=localhost
|
||
ROBOCO_DATABASE_PORT=5432
|
||
ROBOCO_DATABASE_USER=roboco
|
||
ROBOCO_DATABASE_PASSWORD=roboco
|
||
ROBOCO_DATABASE_NAME=roboco
|
||
|
||
# Redis
|
||
ROBOCO_REDIS_HOST=localhost
|
||
ROBOCO_REDIS_PORT=6379
|
||
|
||
# Security (REQUIRED)
|
||
# Generate with: python -c 'from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())'
|
||
ROBOCO_ENCRYPTION_KEY=<your-fernet-key>
|
||
|
||
# Workspaces
|
||
ROBOCO_WORKSPACES_ROOT=/data/workspaces
|
||
ROBOCO_WORKSPACE_AUTO_CLONE=true
|
||
ROBOCO_WORKSPACE_CLONE_TIMEOUT=300
|
||
|
||
# RAG (in-house pgvector engine)
|
||
ROBOCO_RAG_CHUNK_STRATEGY=fixed
|
||
ROBOCO_RAG_CHUNK_SIZE=512
|
||
ROBOCO_RAG_USE_HYDE=true
|
||
ROBOCO_RAG_USE_HYBRID_SEARCH=true
|
||
|
||
# AI/LLM
|
||
ROBOCO_DEFAULT_EMBEDDING_MODEL=qwen3-embedding:0.6b
|
||
ROBOCO_LOCAL_LLM_MODEL=glm-5.2:cloud
|
||
ROBOCO_LOCAL_LLM_BASE_URL=http://roboco-ollama:11434/v1
|
||
ROBOCO_OLLAMA_BASE_URL=http://roboco-ollama:11434
|
||
|
||
# Doctrine (bundled with fable-mode)
|
||
ROBOCO_PONYTAIL_INTENSITY=full # lite/full/ultra — developer ladder intensity (bundled with fable-mode)
|
||
```
|
||
|
||
## Docker Deployment
|
||
|
||
### Container Architecture
|
||
|
||
The system runs as Docker Compose services. All Dockerfiles live under `docker/` at the project root; every service uses `context: .` plus `dockerfile: docker/<name>.Dockerfile`.
|
||
|
||
| Service | Purpose | Healthcheck |
|
||
|---------|---------|-------------|
|
||
| `postgres` | PostgreSQL + pgvector | `pg_isready` |
|
||
| `redis` | Cache, sessions, event bus | `redis-cli ping` |
|
||
| `ollama` | Local LLM + embeddings | `ollama list` |
|
||
| `ollama-init` | Pulls models on startup | One-shot |
|
||
| `agent-base-image` / `agent-*-image` | Pre-built images spawned per agent | One-shot |
|
||
| `orchestrator` | API + agent spawner | Depends on all above |
|
||
| `panel` | Next.js control panel (internal, port 3000) | — |
|
||
| `nginx` | Reverse proxy fronting panel + orchestrator | — |
|
||
|
||
### Single Entry Point
|
||
|
||
`nginx` is the only externally-exposed service. It listens on `localhost:3000` and routes:
|
||
|
||
- `/api/*` and `/ws/*` → `orchestrator:8000`
|
||
- everything else → `panel:3000`
|
||
|
||
This avoids CORS since the browser sees one origin. The Next.js code uses relative URLs (`/api`, `/ws`) and lets nginx do the dispatch.
|
||
|
||
### Network topology (DB isolation)
|
||
|
||
Two user-defined bridges: `roboco_default` (the agent mesh — panel, nginx, ollama, every spawned agent container, and their sandbox DB/Redis sidecars) and `roboco_data` (postgres + redis ONLY). The orchestrator is the only multi-homed service (both networks), so agent containers cannot resolve or TCP-reach `roboco-postgres:5432` / `roboco-redis:6379` at all — network membership is the containment (redis has no auth). Agent↔agent A2A (`:9000`), orchestrator→agent SDK polls (`:9000`), MCP→orchestrator (`:8000`), and host-published ports (`15432`/`16379`/`11435`) are unaffected; `docker exec`/`inspect` paths ride the daemon socket, not the network. `ROBOCO_DB_NETWORK_ISOLATED` (config default `false`) is set `true` by the compose files that carry this topology and suppresses the legacy `_append_gate_env` prod-creds injection (unreachable creds are worse than none); DB-needing projects use the sandbox opt-in instead. The flag is deliberately NOT in the panel feature-flags card — it must travel with the compose `networks:` stanzas.
|
||
|
||
### WebSocket streams
|
||
|
||
The orchestrator exposes WebSocket endpoints under `/ws` (router in `roboco/api/websocket.py`, `ConnectionManager` + `broadcast_*` helpers):
|
||
|
||
| Endpoint | Purpose |
|
||
|----------|---------|
|
||
| `/ws/agents/{id}`, `/ws/notifications/{id}` | Per-resource live streams |
|
||
| `/ws/system` | Operator/system-wide stream (no per-agent keying) — the rate-limit lifecycle (`RATE_LIMIT_HIT` / `RATE_LIMIT_LIFTED`), live usage (`USAGE_SNAPSHOT`, pushed to the usage dashboard), and A2A message events (`a2a.message` frames) |
|
||
|
||
Server-side events reach these sockets through `roboco/api/websocket_bridge.py`, which subscribes to the `StreamEventBus` and forwards each event to the matching connections. To add a new live event: define an `EventType` (dotted value), publish it to the bus, add a `_handle_*` forwarder in `websocket_bridge`, and consume it on the panel via the `useWebSocket("/<endpoint>", …)` hook — do not stand up a parallel endpoint or client stack. `A2A_MESSAGE_SENT` is the worked example: `A2AService.send` publishes it (excerpt-capped payload), the bridge forwards it to `/ws/system` as an `a2a.message` frame, and the panel's `useA2ALiveStream` hook (a second consumer of that same shared `/ws/system` connection) consumes it to invalidate-on-frame.
|
||
|
||
### Rate limiting & usage
|
||
|
||
- **Provider rate limits** are tracked in Redis (`RateLimitStateTracker`, `roboco/services/gateway/`). On a provider 429 an agent calls `i_am_blocked(reason="rate_limited")`; the spawn gate then **queues** (never drops) further work for that provider, and a background probe-and-resume loop in the orchestrator clears the limit and revives parked agents when it lifts.
|
||
- **Provider overloads** reuse the same park-and-probe break. A persistent model-API overload (HTTP 529 / 500 / 503 — the SDK already retries transient ones) parks the provider exactly like a 429 instead of crash-retrying the agent straight back into the overload and burning tokens; the overload is detected orchestrator-side from the dead container's log markers, and the background loop revives the parked work when it recovers. The same break also catches the **Claude session-limit** 429 (the org's 5-hour usage window): an agent exiting with a 0-token session-limit rejection parks the provider and is auto-revived when the window resets, instead of fleet-wide crash-respawning straight back into the limit. Gated by `ROBOCO_OVERLOAD_BREAK_ENABLED` (default-on).
|
||
- **Gateway-health recovery** closes a blind spot in the stale-claim reaper: the heartbeat is bumped only by gateway verbs, so a broken-but-alive agent (a corrupted `/app/.venv` so no gateway tool imports) goes heartbeat-stale yet keeps its container up, and the reaper's live-skip would protect it forever. On a stale-heartbeat live container the reaper now probes the gateway out-of-band (`_probe_gateway_health` → `docker exec` the gateway venv imports) and, once broken past `ROBOCO_GATEWAY_HEALTH_GRACE_SECONDS` (a transient probe miss is tolerated), kills + evicts it (`_maybe_recover_broken_gateway`) so it falls through to release + respawn; healthy or inconclusive probes spare it. Gated by `ROBOCO_GATEWAY_HEALTH_ENABLED` (default-on). It is the third leg beside the shipped bash-guard `/app` block (prevents the self-corruption) and the reaper Docker-liveness fallback (stops over-reaping live containers).
|
||
- **PM coordinator concurrency.** A Main / Cell PM plans and delegates many root tasks in parallel — the actual work then runs in the delegated children/cells, not in the PM's own hands. The claim-time concurrency guards that keep a *developer* to one task at a time (`already_active` / `paused`, in `roboco/services/gateway/claim_guards.py`) are therefore **skipped for the coordinator PM roles** (`_COORDINATOR_ROLES = {main_pm, cell_pm}`, consulted in `_run_claim_guards`); only a genuine upstream **sequence dependency** (`unmet_dependency`, which parks the task back to `pending`) holds a PM's root back. Without this a single PM that claimed one root could never plan a second — it thrashed between its claimed roots and respawned forever, burning tokens for zero progress (the live `i_am_idle`-auto-paused-umbrella deadlock). The `paused` guard also excludes the target task itself, so a PM re-entering its own paused umbrella never self-blocks.
|
||
- **Orchestrator runtime-state durability.** The PM-respawn loop breaker (`_pm_respawn_tracker`, the `(agent_slug, task_id) → strike-count` circuit breaker) is **DB-durable** via the `respawn_tracker` table (migration 051): each gate mutation write-throughs fire-and-forget on the `_bg_tasks` set (`_schedule_respawn_persist` → `_persist_respawn_record`), and `restore_respawn_tracker()` repopulates it at `start()`, validating each row against live tasks (terminal/missing rows are evicted). Kept only in memory it reset to `count=1` on every restart and re-burned the whole strike threshold (4 spawns) against a still-wedged task. It mirrors the `WaitingRecordTable` / `restore_waiting_records` pattern: best-effort (a DB hiccup degrades to in-memory-only — it can only ever *suppress* a spawn, never manufacture one) and inert when the table is empty. The companion `_instances` registry is **reconciled-from-Docker** (not persisted) at startup via `_readopt_running_agents`, so the reaper's liveness path and the spawn gate's `_is_agent_active` check see surviving containers immediately after a restart.
|
||
- **Token usage** is captured per agent session from the Claude Code transcript via the SDK server's `/usage/sync` (hook → orchestrator finalize → `agent_spawn_sessions` → `daily_usage_rollups` → dashboard). Cost uses provider-aware pricing in `roboco/billing/pricing.py` (Anthropic priced; local/Ollama intentionally `$0`). The token sweep also publishes `USAGE_SNAPSHOT` to `/ws/system`, so the dashboard's "Token Usage & Cost" panel updates live and falls back to HTTP polling when the stream is down.
|
||
- **Delivery observability** (the panel's Metrics → "Delivery" tab) shows how work *flows*, computed by `MetricsService` from data already captured — no new feature flag. Per-stage cycle time and the bottleneck distribution are reconstructed from the `audit_log` transition journey (each generic `task.<status>` event marks entry into a status; the named `task.qa_fail`/`task.pr_fail` events are excluded from the reconstruction). Rework rate reads `tasks.revision_count` — incremented once per transition into `needs_revision` at the single chokepoint `TaskService._emit_status_transition_audit` — and attributes each bounce to the QA / PR-reviewer via those named audit events; rework cost joins `agent_spawn_sessions.task_id`. Read-only endpoints: `/dashboard/metrics/{cycle-time,bottlenecks,rework,scorecard/agent/{id},scorecard/team/{team}}`.
|
||
|
||
### Startup Sequence
|
||
|
||
The startup order is critical due to dependencies:
|
||
|
||
```
|
||
postgres ──┐
|
||
redis ─────┼──> ollama ──> ollama-init ──> orchestrator ──> panel ──> nginx
|
||
│ │ │
|
||
│ │ └── Pulls qwen3-embedding:0.6b, glm-5.2:cloud
|
||
│ └── Healthcheck: ollama list
|
||
└── Healthcheck: pg_isready, redis-cli ping
|
||
```
|
||
|
||
**Important timing notes:**
|
||
1. `ollama-init` pulls models (~30s for embedding model, ~2min for LLM)
|
||
2. Orchestrator waits for models before starting
|
||
3. FastAPI lifespan indexes documents using Ollama (~30-60s)
|
||
4. Orchestrator polls `/health` until API is ready before starting dispatcher
|
||
5. After orchestrator is up, `panel` (Next.js) builds/starts, then `nginx`
|
||
|
||
### Database migrations
|
||
|
||
Schema changes ship as Alembic migrations under `alembic/versions/`. Run:
|
||
|
||
```bash
|
||
docker compose exec orchestrator alembic upgrade head
|
||
```
|
||
|
||
after pulling any change that adds a new migration.
|
||
|
||
### Ollama Configuration
|
||
|
||
Ollama provides two APIs:
|
||
- `/v1/*` - OpenAI-compatible API (for LLM chat/completion)
|
||
- `/api/*` - Native Ollama API (for embeddings, model management)
|
||
|
||
The embedder uses `/api/embed` endpoint with the `qwen3-embedding:0.6b` model.
|
||
|
||
**Environment variables for Docker:**
|
||
```bash
|
||
ROBOCO_LOCAL_LLM_BASE_URL=http://roboco-ollama:11434/v1 # OpenAI-compat
|
||
ROBOCO_OLLAMA_BASE_URL=http://roboco-ollama:11434 # Native API
|
||
```
|
||
|
||
### Common Issues
|
||
|
||
| Symptom | Cause | Fix |
|
||
|---------|-------|-----|
|
||
| `404 /api/embed` | Model not pulled | Check `docker logs roboco-ollama-init` |
|
||
| `All connection attempts failed` | API not ready | Orchestrator starts before FastAPI lifespan completes |
|
||
| Healthcheck failing | Wrong endpoint | Use `ollama list` not `curl` |
|
||
|
||
## Blueprint Reference
|
||
|
||
The organizational structure, communication matrix, role descriptions, and access-control model are documented inline above and in the user-facing documentation site at **[docs.roboco.tech](https://docs.roboco.tech)** (the `roboco-website` repo — Next.js MDX, the canonical docs site as of the 2026-07-03 docs-site split: `docs/internal/specs/2026-07-03-docs-site-split.md`). This repo's old MkDocs-built user tree is gone; `.github/workflows/docs.yml` now only deploys the committed `docs-redirects/` stubs (meta-refresh + canonical) so every URL the old Pages site published keeps resolving, to docs.roboco.tech. `docs/rag/` remains the agent-facing RAG corpus (never published); `docs/map/` is the agent-facing exhaustive codebase map; `docs/internal/` holds specs and working notes; the old root `usage.md` / `deployment.md` now link straight to docs.roboco.tech.
|