diff --git a/docs/map/_complete_map.md b/docs/map/_complete_map.md index 07e9fa16..f813fa4d 100644 --- a/docs/map/_complete_map.md +++ b/docs/map/_complete_map.md @@ -644,228 +644,821 @@ Three workstreams plus a migration-chain catch-up (all reviewed; all default-off Slices touched: orchestrator (1, 2), runtime-providers (1), prompts-roles-taxonomy (1, 3), deployment-tooling (1), gateway-support + mcp-servers (2), product-strategy-research-pitch (2, already current from an earlier pass), db-migrations (2, 4). ---- - ## Purpose -The canonical source of truth for RoboCo's task state machine, per-role action/verb permissions, claim rules, team-match rules, and self-review prevention. The spec module is pure data + lookups (no DB, no I/O); the enforcement package re-exports a backwards-compat view of the same tables plus the git-workflow gates, SLA keys, A2A/journal/ownership access control. Import-time validators in _validate_lifecycle.py make a misconfigured spec fail fast at container start. +This slice is the agent-to-agent communication, audit forensics, journaling, structured-note persistence, message extraction, and role-based access-control backbone of RoboCo's service layer. A2AService builds Agent Cards and manages persistent slug-keyed A2A conversations/messages plus the legacy A2A-protocol task/notification path. AuditService best-effort persists security/lifecycle events to audit_log and backs the PM-respawn tracing-gap circuit breaker. JournalService owns agent journal CRUD, gateway scope→type entry writes, and fire-and-forget RAG indexing. content_notes is the single chokepoint that validates and persists structured task notes plus their derived TEXT mirrors. ExtractionService classifies raw LLM stream buffers into typed messages. PermissionService enforces notification/task/KB access from agents_config. ## Files | Path | Role | LOC | |---|---|---| -| /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py | Canonical lifecycle/permissions spec: Status, TaskType, Decision, Precondition, ActionSpec, IntentSpec, StatusTransition tables + can_invoke_action/intent lookups + CLAIM_RULES/ROLE_TEAM_RULES + PR_OPEN_STATES + UNMIGRATED debt set; import-time self-validates. | 1943 | -| /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/_validate_lifecycle.py | Import-time validators (BFS reachability, terminal exits, intent composition chains, claim-rule coverage, self_review symmetry, slug/team agreement, Status/TaskStatus ORM parity, UNMIGRATED subset); LifecycleSpecError aborts container start on a bad spec. | 356 | -| /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/task_lifecycle.py | Backwards-compat shim over the spec: derives VALID_TRANSITIONS / ROLE_RESTRICTED_TRANSITIONS (merging _LEGACY_OPERATIONAL_EDGES + _LEGACY_ROLE_GATES), predicate helpers, SLA keys, GitContext + validate_git_requirements (doc-phase / CEO-escalation / claim-branch gates). | 441 | -| /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/__init__.py | Package re-export aggregator: exposes task_lifecycle, a2a_access, journal_perms, task_ownership public symbols under one namespace. | 87 | -| /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/a2a_access.py | Agent-to-agent direct-message permission gate (delegates to roboco.agents_config.can_a2a_direct); A2AAccessDeniedError. | 91 | -| /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/journal_perms.py | Journal read permissions derived from foundation.policy.journaling ReadTier (ALL/ALL_CELLS/CELL_AND_PMS/CELL/OWN) + PROTECTED_JOURNALS; JournalAccessDeniedError. | 197 | -| /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/task_ownership.py | Task ownership + reassign + view + self-review guard (can_review_task); TaskOwnershipError. | 118 | +| roboco/services/a2a.py | A2A protocol + persistent conversation service: Agent Cards, task↔A2A conversion, legacy A2A task notifications, bidirectional response spawning, slug-keyed conversation/message CRUD, gateway send adapter, CEO admin/live-view surface (reply budget + org-wide read), CEO-DM offline-recipient wake | 2090 | +| roboco/services/audit.py | AuditService singleton: best-effort persist denial/lifecycle/agent events to audit_log, resolve actor role + slug→UUID at write time, tracing-gap query for respawn circuit breaker, recent-events query | 462 | +| roboco/services/journal.py | JournalService: journal/entry CRUD, gateway scope-string→JournalEntryType adapter, fire-and-forget RAG indexing (private excluded), tracing-gate existence checks, board review brief, growth analytics | 1029 | +| roboco/services/content_notes.py | Single chokepoint applying structured notes: validate via foundation ContentModel, store in notes_structured, regenerate derived TEXT mirror column (dev_notes/qa_notes/etc.) | 74 | +| roboco/services/extraction.py | ExtractionService + ExtractionPipeline: regex-pattern (and optional LLM/TOON) classification of raw agent LLM buffers into typed ExtractedMessages; transcription pipeline callback fan-out | 508 | +| roboco/services/permissions.py | PermissionService singleton + async helpers: notification scope (all/cell/board-chain), task-action and KB-action RBAC from agents_config; privileged/PM-role DB lookups | 425 | ## Key Symbols | Name | Kind | File:Line | Responsibility | |---|---|---|---| -| Status | StrEnum | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:40 | 15 task lifecycle statuses (backlog..cancelled) — the state machine alphabet. | -| TaskType | StrEnum | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:58 | 6 task types used by ActionSpec.allowed_task_types gating. | -| RejectionKind | Literal | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:67 | The 5 rejection flavors (not_authorized/invalid_state/tracing_gap/self_review/not_found) carried in Decision. | -| Decision | dataclass | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:76 | Frozen allow/reject result every consumer maps onto its envelope; invariants enforced in __post_init__; constructors allow()/reject()/tracing_gap(). | -| Precondition | dataclass | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:145 | Declarative gate-table row: a (task,agent,ctx)->bool predicate + remediate hint + missing_token + rejection_kind. | -| ActionSpec | dataclass | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:167 | Atomic action spec (allowed_roles, source_statuses, target_status, allowed_task_types, preconditions, self_review_block, needs_team_match). | -| IntentSpec | dataclass | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:187 | Gateway verb spec: composes tuple of action names, extra_preconditions, pre_side_effects/side_effects, next_hint callback. | -| StatusTransition | dataclass | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:212 | Row from STATUS_TRANSITIONS canon: source/target/triggering_by_action/role_constraint. | -| _STATUS_TRANSITIONS | tuple | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:231 | The canonical state-machine edge table (claim/start/block/qa/pr/complete/cancel/escalate/ceo edges incl. BLOCKED->PENDING and BLOCKED->AWAITING_CEO_APPROVAL). | -| _build_status_graph | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:374 | Derives source->frozenset(targets) view from _STATUS_TRANSITIONS. | -| STATUS_GRAPH | dict | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:382 | Derived state graph consumed by validators + enforcement shim. | -| _ATOMIC_ACTIONS | dict | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:414 | All 25 ActionSpecs (activate, claim, start, set_plan, block, unblock, pause, resume, submit_verification, submit_qa, qa_pass, qa_fail, pr_review_done, docs_complete, submit_for_review, pr_pass, pr_fail, complete, submit_pm_review, escalate_to_ceo, ceo_approve, ceo_reject, ceo_reject_to_pool, cancel, create_subtask). | -| CLAIM_RULES | dict | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:671 | Per-role claimable status sets; narrows the union claim ActionSpec.source_statuses. | -| ROLE_TEAM_RULES | dict | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:702 | Per-slug team binding (None=cross-cell/board) for needs_team_match enforcement. | -| _next_hint_pr_fail | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:789 | pr_fail next hint: steers Main-PM branch-bearing root to re-delegate (loop-breaker) vs dev-revise for cell/dev tasks. | -| Context | dataclass | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:849 | Caller-supplied per-request state (actor_id, plan, journal flags, original_developer_slug, notes, issues, files) fed to Precondition.check. | -| _p_non_terminal | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:894 | Precondition predicate: task not in COMPLETED/CANCELLED (F043 terminal-resurrection guard). | -| PRECONDITION_NON_TERMINAL | Precondition | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:942 | invalid_state precondition attached to escalate_up. | -| PR_OPEN_STATES | frozenset | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:963 | Lifecycle-owned canon of states a PR may be opened from (in_progress/verifying/awaiting_qa/awaiting_documentation/needs_revision); GitService derives its str set from this (F101). | -| PRECONDITION_PR_OPEN_STATE | Precondition | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:981 | invalid_state precondition attached to open_pr (parity with HTTP path). | -| _INTENT_VERBS | dict | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:994 | All ~30 gateway IntentSpecs (give_me_work, i_will_work_on, i_will_plan, delegate, open_pr, i_am_done, sync_branch, i_am_blocked, unclaim, reassign, resume, i_am_idle, claim_review, pass_review, fail_review, claim_pr_review, post_pr_review, claim_gate_review, pr_pass, pr_fail, claim_doc_task, i_documented, complete, escalate_up, escalate_to_ceo, submit_up, submit_root, unblock, triage, triage_all). | -| can_claim | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:1410 | Backward-compat wrapper around can_invoke_action('claim', ...). | -| _check_role_status_type | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:1427 | Role + source-status + task_type gate for an ActionSpec; returns rejection or None. | -| _check_self_review_and_preconditions | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:1469 | self_review_block check (original_developer_slug==actor_slug) + declarative precondition evaluation. | -| _check_claim_rules_narrow | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:1501 | Per-role CLAIM_RULES narrowing for the claim action; disambiguates not_authorized vs invalid_state. | -| can_invoke_action | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:1542 | Order-gated atomic action Decision: action exists -> role -> source status -> task_type -> self_review -> preconditions -> claim rules. | -| _check_intent_preconditions | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:1572 | Verb-level extra_preconditions gate; honors non-tracing rejection_kind (not_authorized/invalid_state) per F043 generalization. | -| can_invoke_intent | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:1604 | Verb Decision: role gate -> extra_preconditions -> FIRST composed action's can_invoke_action (or CLAIM_RULES narrowing for the claim_review/claim_doc_task/claim_gate_review special cases). | -| valid_next_verbs | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:1652 | Sorted list of verbs a role can state-applicably call on a task (preconditions evaluated lazily); used by envelope introspection. | -| composed_actions_for | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:1679 | Return the composes tuple for a verb (KeyError on unknown). | -| intents_for_role | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:1686 | Sorted tuple of verbs declared for a role; drives role_config.py MCP manifest build. | -| status_after | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:1696 | Post-action status or None (no transition / wrong source). | -| UNMIGRATED | frozenset | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:1710 | Known-debt set: legacy operational edges + role gates not yet absorbed into the spec (Phase 3 terminal invariant target = empty). | -| run_all_lifecycle_validators | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/_validate_lifecycle.py:349 | Runs all 14 import-time validators; first failure raises LifecycleSpecError. (14th added: _check_status_enum_parity cross-checks spec.Status against models.base.TaskStatus at import.) | -| reachable_from | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/_validate_lifecycle.py:40 | BFS over STATUS_GRAPH from a start status. | -| _check_status_reachability | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/_validate_lifecycle.py:67 | Every non-BACKLOG status reachable from PENDING. | -| _check_terminal_exits | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/_validate_lifecycle.py:81 | Every non-terminal status has a path to COMPLETED or CANCELLED. | -| _check_intent_chains | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/_validate_lifecycle.py:113 | Adjacent composes actions chain (prev.target_status in next.source_statuses). | -| _check_self_review_symmetry | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/_validate_lifecycle.py:162 | qa_pass/qa_fail/docs_complete/pr_pass/pr_fail agree on self_review_block. | -| _check_action_target_reachable_from_source | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/_validate_lifecycle.py:203 | ActionSpec transitions present in STATUS_GRAPH[source]. | -| _check_unmigrated_is_subset | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/_validate_lifecycle.py:256 | UNMIGRATED stays a subset of _KNOWN_UNMIGRATED_CONSUMERS. | -| validate_task_transition | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/task_lifecycle.py:187 | Legacy raising validator over VALID_TRANSITIONS + ROLE_RESTRICTED_TRANSITIONS (role gate only for transition-level pins). | -| can_agent_transition | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/task_lifecycle.py:225 | Non-raising variant of validate_task_transition. | -| is_terminal_state/is_waiting_state/is_active_state | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/task_lifecycle.py:242 | Hard-coded status-category predicates (NOT derived from the graph — drift risk). | -| ROLE_STATE_SLA_KEYS | dict | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/task_lifecycle.py:276 | (role,status)->settings-key map for stuck-task SLA sweep. | -| sla_seconds_for | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/task_lifecycle.py:285 | Resolve configured SLA seconds for (role,status) from settings. | -| GitContext | dataclass | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/task_lifecycle.py:316 | Git-state carrier (docs_complete/pr_created/pr_number/branch_name + is_coordination/is_external_review/is_umbrella exemption flags). | -| validate_git_requirements | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/task_lifecycle.py:342 | Doc-phase / CEO-escalation / claim-branch git gates; None short-circuits. | -| _LEGACY_OPERATIONAL_EDGES | dict | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/task_lifecycle.py:72 | Transitions the runtime exercises that the spec hasn't absorbed (unclaim/reaper/PM-self-complete/QA-direct/verifying-self-fail/PM-claim/revision-reentry). | -| _LEGACY_ROLE_GATES | dict | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/task_lifecycle.py:99 | Role pins for the legacy edges; UNION-merged with spec-derived ROLE_RESTRICTED_TRANSITIONS (overwrite once dropped pr_reviewer). | -| _build_role_restricted_transitions | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/task_lifecycle.py:156 | Merges spec role_constraint pins with _LEGACY_ROLE_GATES via union (not overwrite). | -| validate_a2a_access | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/a2a_access.py:38 | A2A direct-message gate (delegates to agents_config.can_a2a_direct). | -| can_read_journal | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/journal_perms.py:121 | Journal read decision by ReadTier + protected-journal + same-cell/owner-is-pm rules. | -| validate_task_ownership | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/task_ownership.py:35 | Ownership/reassign/view gate; non-PM must be the assignee. | -| can_review_task | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/task_ownership.py:102 | Self-review prevention: agent_id != task_developed_by. | +| A2AService | class | roboco/services/a2a.py:55 | Service layer for A2A protocol ops and persistent agent conversations; takes an AsyncSession | +| get_service_endpoint | staticmethod | roboco/services/a2a.py:69 | Build outbound callback URL; dials loopback when host binds 0.0.0.0/:: to avoid bandit B104 | +| build_system_agent_card | staticmethod | roboco/services/a2a.py:87 | Return the system-level Agent Card served at /.well-known/agent.json | +| build_agent_card | method | roboco/services/a2a.py:144 | Resolve agent by UUID or slug and return its AgentCard (None if missing) | +| _agent_to_card | method | roboco/services/a2a.py:172 | Map an AgentTable row to an AgentCard with role-keyed skills + bearer security scheme | +| task_to_a2a | method | roboco/services/a2a.py:265 | Canonical RoboCo TaskTable→A2ATask conversion with status mapping and metadata | +| get_task | method | roboco/services/a2a.py:320 | Fetch task by UUID string and return A2ATask or None | +| list_tasks | method | roboco/services/a2a.py:345 | Paginated task listing with has_more detection (fetches page_size+1) | +| _status_value_of | staticmethod | roboco/services/a2a.py:383 | Extract task status as a comparable string (enum .value or str); factored from cancel_task for xenon complexity gate | +| _apply_cancel_note | method | roboco/services/a2a.py:387 | Append actor-attributed cancellation note to task.dev_notes and flush; factored from cancel_task | +| cancel_task | method | roboco/services/a2a.py:408 | Cancel task and cascade to descendants; now takes agent_role (threaded into TaskService.cancel role gate) and actor_slug (recorded in cancellation note); rejects terminal states | +| discover_agents | method | roboco/services/a2a.py:442 | List AgentCards filtered by role/team/skill_tag | +| get_team_from_agent | staticmethod | roboco/services/a2a.py:485 | Map agent slug to Team enum via agents_config.get_agent_team (defaults BACKEND) | +| resolve_target_agent | staticmethod | roboco/services/a2a.py:496 | Resolve target agent slug from metadata (explicit target_agent or skill-based routing) | +| extract_message_text | staticmethod | roboco/services/a2a.py:523 | Split first text part into (title, description, full_text) | +| update_task_with_message | staticmethod | roboco/services/a2a.py:540 | Append A2A-protocol message text to task.dev_notes (legacy A2A thread store, not gateway conversations) | +| resolve_creator_agent | method | roboco/services/a2a.py:563 | Resolve creator AgentTable from from_agent slug or fall back to first main_pm | +| create_a2a_notification | method | roboco/services/a2a.py:619 | Legacy A2A peer-to-peer notification (requires task_id); requires from_agent present and target_agent resolvable (raises ValueError with distinct messages if either missing); enforces hierarchy unconditionally via validate_a2a_access (A2AAccessDeniedError), then parse_priority, delegates to NotificationService.send_a2a_notification | +| update_task_from_message | method | roboco/services/a2a.py:657 | Append response message to existing A2A task and notify/spawn original requester (bidirectional) | +| _lookup_requester_slug | staticmethod | roboco/services/a2a.py:700 | Reverse-lookup agent slug from creator UUID via static AGENT_UUIDS map | +| _publish_a2a_response_event | staticmethod | roboco/services/a2a.py:711 | Publish TASK_ASSIGNED event to event bus to spawn/notify the original requester; swallows errors | +| _notify_original_requester | method | roboco/services/a2a.py:741 | If task.dev_notes carries 'A2A Request' marker, publish response event to requester (skips self-response) | +| _canonical_pair | staticmethod | roboco/services/a2a.py:779 | Return two agent slugs in lexically-sorted order (conversation uniqueness key) | +| get_or_create_conversation | method | roboco/services/a2a.py:784 | Validate A2A access (validate_a2a_access), canonical-order lookup or create A2AConversationTable row | +| get_conversation | method | roboco/services/a2a.py:850 | Fetch conversation by ID only if agent_slug is a participant | +| get_conversation_admin | method | roboco/services/a2a.py:952 | Wave-2 CEO live view: like get_conversation but WITHOUT the participant check — returns any conversation by id for the org-wide read; None only if it truly doesn't exist | +| list_conversations | method | roboco/services/a2a.py:881 | List conversation summaries for an agent with optional status/with_agent/task_id filters; per-conv last-message preview query (N+1) | +| list_conversations_admin | method | roboco/services/a2a.py:1063 | Wave-2 CEO live view: list conversations across every agent pair (no participant filter), most-recent-first; backs GET /chat/admin/conversations | +| list_admin_pairs | method | roboco/services/a2a.py:1101 | Wave-2c switchboard: every agents_config.A2A_ALLOWED_PAIRS entry joined with its representative conversation (most-recently-updated when >1) via one bulk tuple_(agent_a,agent_b).in_() query — never N+1; backs GET /chat/admin/pairs | +| close_conversation | method | roboco/services/a2a.py:962 | Mark conversation CLOSED with optional resolution; participant-only | +| _enforce_ceo_reply_budget | method | roboco/services/a2a.py:1177 | Wave-2 reply-then-wait budget on the CEO's inbox — the one stateful gate the stateless can_a2a_direct matrix can't see. An agent may message the CEO only inside a conversation the CEO itself opened, and only up to the CEO's own message count there (rejects once agent_count >= ceo_count); no-op for CEO-authored sends or non-CEO conversations | +| send_chat_message | method | roboco/services/a2a.py:1225 | Send message in conversation; nil-UUID guard; dedup unread identical (conv,sender,kind,content); calls _enforce_ceo_reply_budget before persisting; bump unread for other side; reads skill from opts and persists it on the message row (nullable) | +| get_messages | method | roboco/services/a2a.py:1337 | Paginated chronological message list for a participant | +| get_messages_admin | method | roboco/services/a2a.py:1375 | Wave-2 CEO live view: like get_messages but WITHOUT the participant check — reads any conversation's transcript; [] only if the conversation truly doesn't exist | +| mark_read | method | roboco/services/a2a.py:1410 | Zero agent's per-side unread counter and bulk UPDATE read_at on inbound unread messages | +| mark_all_read | method | roboco/services/a2a.py:1451 | Agent-keyed bulk mark_read across all conversations with unread for this agent; returns count cleared | +| get_inbox_summary | method | roboco/services/a2a.py:1492 | Aggregate total unread, conversations with unread, pending + unanswered requires_response counts | +| list_pairs | method | roboco/services/a2a.py:1553 | Group conversations into unique agent pairs with rollup counts/unread/last_activity for frontend | +| _conv_to_model | method | roboco/services/a2a.py:1603 | A2AConversationTable→A2AConversation Pydantic model | +| _msg_to_model | method | roboco/services/a2a.py:1621 | A2AMessageTable→A2AChatMessage Pydantic model; now maps skill field (migration 054 adds nullable skill column on a2a_messages) | +| _resolve_slug_from_id | method | roboco/services/a2a.py:1642 | Lookup agent slug from UUID; raise ValueError if missing (gateway send adapter) | +| _get_conversation_for_reply_to_ceo | method | roboco/services/a2a.py:1652 | Wave-2: resolve the conversation for an agent replying to the CEO by direct lookup (bypasses get_or_create_conversation's validate-first gate, which would deny even a legitimate reply) — an existing pair conversation's mere presence proves the CEO opened it, since agents can never create one | +| send | method | roboco/services/a2a.py:1684 | Gateway adapter: resolve both ends to slugs, get_or_create_conversation (or _get_conversation_for_reply_to_ceo when replying to "ceo") + send_chat_message; publishes A2A_MESSAGE_SENT via _publish_a2a_message_sent afterward | +| _publish_a2a_message_sent | staticmethod | roboco/services/a2a.py:1742 | Wave-2: best-effort publish of A2A_MESSAGE_SENT (conversation_id/message_id/task_id/from_agent/to_agent/skill/body_excerpt/timestamp) to the event bus for the operator live view; a bus outage is logged and never rolls back the already-persisted message | +| _maybe_wake_ceo_recipient | method | roboco/services/a2a.py:1983 | CEO-authored send only (`from_slug == "ceo"`); gates on `is_spawnable_agent_slug` + the recipient role carrying `read_a2a` (else an unackable row would be immortal), dedups against an already-pending wake, then calls `send_a2a_notification(..., requires_ack=True)` so the row is visible to the orchestrator's `_dispatch_a2a_work` pending_ack_only poll. Called from `send_chat_message` and `interject_as_ceo`; best-effort, never breaks the send | +| _ack_pending_wake_notifications | method | roboco/services/a2a.py:2059 | Bulk-acknowledges this agent's pending CEO-DM wake notification(s); called from `mark_all_read`/`get_unread_messages` (the gateway's `read_a2a`) once the inbox is actually drained, so the wake row doesn't sit pending forever and permanently block the next dedup check | +| _AuditEvent | dataclass | roboco/services/audit.py:20 | Bundled fields for one audit row write (event_type, agent_id, target, severity, details) | +| _coerce_uuid | function | roboco/services/audit.py:36 | Best-effort coerce str/UUID to UUID; returns None for slugs/invalid | +| AuditService | class | roboco/services/audit.py:48 | SingletonService for audit logging; structured log + best-effort audit_log persistence | +| _persist | method | roboco/services/audit.py:77 | Write an audit row in its own session+commit; never propagate failures (observability must not block) | +| log_task_action_denial | method | roboco/services/audit.py:111 | Log a denied task action; resolves actual actor role from DB at write time over caller-supplied param; preserves non-UUID task_id sentinels (e.g. "N/A") in details["target_id_raw"] rather than silently coercing to NULL target_id | +| log_task_creation_denial | method | roboco/services/audit.py:163 | Log a pre-task-creation denial (no task row exists yet); records attempted payload under target_type="task_creation" with target_id=None — distinct from log_task_action_denial's NULL-target-id so role-escalation attempts are attributable | +| log_task_event | method | roboco/services/audit.py:204 | Log a task-lifecycle event (creation/transition) for TaskService chokepoint | +| log_event | method | roboco/services/audit.py:238 | Free-form generic audit event (e.g. gateway.rejected) for Choreographer forensics | +| log_agent_event | method | roboco/services/audit.py:273 | Log orchestrator agent event (spawned/stopped/stranded); resolves slug→UUID so agent_id is a real FK | +| _resolve_actor_role_from_db | method | roboco/services/audit.py:314 | Read agents.role for actor UUID at write time (DB authoritative over caller-supplied role) | +| _resolve_agent_id_by_slug | method | roboco/services/audit.py:302 | Static AGENT_UUIDS fast-path then DB lookup for slug→UUID; best-effort None on failure | +| has_recent_tracing_gap | method | roboco/services/audit.py:353 | Query audit_log for gateway.rejected/tracing_gap rows since cutoff; backs PM-respawn circuit breaker reset decision | +| get_recent_events | method | roboco/services/audit.py:399 | Fetch recent audit events as dicts (Auditor/CEO queries) with optional type/agent/severity filters | +| get_audit_service | function | roboco/services/audit.py:458 | Lazy singleton accessor for AuditService | +| JournalService | class | roboco/services/journal.py:64 | BaseService for journal/entry CRUD, gateway adapters, RAG indexing, tracing-gate existence checks | +| _get_optimal_service | method | roboco/services/journal.py:78 | Lazy-load OptimalService singleton (avoid circular import) | +| resolve_agent_id | method | roboco/services/journal.py:86 | Resolve UUID-or-slug string to agent UUID via repositories.resolve_agent_uuid | +| get_agent_slug | method | roboco/services/journal.py:102 | Reverse slug lookup via repositories.get_agent_slug | +| get_or_create_journal | method | roboco/services/journal.py:120 | Fetch or create a journal row for an agent (commits on create) | +| create_entry | method | roboco/services/journal.py:215 | Insert entry, bump journal metadata counters, commit; IntegrityError→rollback+None; schedule fire-and-forget RAG index | +| _schedule_rag_index | method | roboco/services/journal.py:324 | asyncio.create_task best-effort RAG index; skips private entries in shared JOURNALS index; strong-ref in _RAG_INDEX_TASKS | +| list_entries | method | roboco/services/journal.py:401 | Filtered/paginated entry listing (excludes private unless include_private) | +| board_review_brief | method | roboco/services/journal.py:455 | PO+HoM DECISION_LOG entries for a task (board handoff for CEO approval/intake redraft) | +| delete_entry | method | roboco/services/journal.py:494 | Delete entry and decrement journal counters (floored at 0) | +| add_task_reflection/add_decision_log/add_learning/add_struggle/add_general_entry | method | roboco/services/journal.py:532 | Convenience builders that get_or_create journal then create_entry via the journal model factories | +| get_growth_metrics | method | roboco/services/journal.py:702 | Compute learning/struggle/decision counts, resolution rate, learning frequency from entries_by_type + content scan | +| search_entries | method | roboco/services/journal.py:757 | Semantic RAG search over an agent's JOURNALS index; re-fetches entries and filters by owning journal | +| _has_entry_of_type | method | roboco/services/journal.py:835 | Existence check: agent has entry of type for task (backs tracing gates) | +| has_decision_for_task/latest_decision_at/has_note_for_task/has_learning_for_task/has_reflect_for_task/has_struggle_for_task | method | roboco/services/journal.py:854 | Per-type tracing-gate existence checks used by the Choreographer | +| has_recent_entry | method | roboco/services/journal.py:912 | Any entry within window (backs auditor i_am_idle session-scoped note obligation) | +| write_struggle/write_decision | method | roboco/services/journal.py:934 | Write-then-gate helpers deriving title from first content line for PM verbs | +| write_entry | method | roboco/services/journal.py:987 | Gateway adapter: scope string→JournalEntryType, get_or_create journal, create_entry | +| get_journal_service | function | roboco/services/journal.py:1027 | Factory: JournalService(db) | +| drain_rag_index_tasks | function | roboco/services/journal.py:51 | Test helper: await all in-flight background RAG index tasks | +| apply_structured_note | function | roboco/services/content_notes.py:57 | Validate payload via foundation ContentModel, store in notes_structured[content_type], regenerate TEXT mirror column; raises before any mutation | +| content_type_for_role | function | roboco/services/content_notes.py:45 | Map agent role to the note section content-type it authors via note(scope='handoff') | +| _MIRROR_COLUMN | module constant | roboco/services/content_notes.py:22 | content_type→derived TEXT mirror column (dev_notes/qa_notes/auditor_notes/doc_notes/pr_reviewer_notes/quick_context) | +| ExtractionService | class | roboco/services/extraction.py:134 | Pattern-based classifier turning raw agent LLM buffers into typed ExtractedMessages | +| extract | method | roboco/services/extraction.py:166 | Segment + classify content; emit ExtractedMessages with confidence + raw_excerpt | +| _segment_content | method | roboco/services/extraction.py:252 | Split on code blocks then double-newlines into paragraph segments | +| _classify_segment | method | roboco/services/extraction.py:280 | Score each MessageType by matched patterns; default REASONING@0.5 if none | +| _call_anthropic_with_retry | method | roboco/services/extraction.py:316 | Anthropic messages.create with MAX_RATE_LIMIT_RETRIES 429 backoff honoring Retry-After | +| extract_with_llm | method | roboco/services/extraction.py:361 | LLM/TOON classification (claude-3-haiku); falls back to pattern extract on any non-RateLimit error | +| ExtractionPipeline | class | roboco/services/extraction.py:459 | Wraps ExtractionService; process_buffer fans results to registered callbacks | +| PermissionService | class | roboco/services/permissions.py:130 | Singleton RBAC enforcement from agents_config: notifications, task actions, KB actions | +| can_send_notifications | method | roboco/services/permissions.py:255 | Whether role may call notify (foundation.NOTIFY_SENDER_ROLES) | +| can_notify | method | roboco/services/permissions.py:259 | Scope rules: all (main_pm/ceo), cell (cell_pm: PMs or same team), board-chain list, else False | +| can_perform_task_action | method | roboco/services/permissions.py:296 | CEO bypasses all; else TASK_PERMISSIONS with VIEW_OWN team restriction + VIEW_ALL fallback | +| can_perform_kb_action | method | roboco/services/permissions.py:333 | KB_PERMISSIONS role action check | +| check_all | method | roboco/services/permissions.py:357 | Comprehensive permission summary dict for an agent context | +| has_privileged_access | function | roboco/services/permissions.py:382 | Async DB check: agent role in PRIVILEGED_ROLES (CEO/Auditor/Main_PM); queries id OR slug | +| is_pm_role | function | roboco/services/permissions.py:410 | Async DB check: agent role in MANAGEMENT_ROLES (CEO/PO/CellPM/MainPM) | +| _get_notification_scope | function | roboco/services/permissions.py:106 | Return scope ('all'/'cell'/role list/[]) for a sender role | +| _get_agents_for_role_team | function | roboco/services/permissions.py:64 | All agent slugs matching a (role, team) pair from the precomputed lookup | ## Data Flow -Inputs: a (role, verb_or_action, task, Context) tuple. The Choreographer (services/gateway/choreographer/_impl.py + per-verb modules qa/pr_review/pr_gate) builds a Context (actor_id, plan, original_developer_slug, journal flags) per request, resolves the task row, and calls can_invoke_intent(role, verb, task, ctx). Control flow inside can_invoke_intent: verb lookup -> role gate -> _check_intent_preconditions (extra_preconditions, honoring per-precondition rejection_kind) -> if composes non-empty, can_invoke_action on the FIRST composed action only (subsequent actions chained by the runner after state transitions); if composes empty + verb in {claim_review, claim_doc_task, claim_gate_review}, _check_claim_rules_narrow; else allow. can_invoke_action order: action exists -> _check_role_status_type (role/source-status/task_type) -> _check_self_review_and_preconditions -> _check_claim_rules_narrow for claim. Output: a frozen Decision (allow / reject(kind,...) / tracing_gap(missing,remediate)) mapped by envelope.py onto the gateway Envelope (status/error/remediate/missing) and by valid_next_verbs onto the envelope's current_state introspection. Side-export: role_config.py calls intents_for_role at import to build per-role MCP tool manifests; GitService imports PR_OPEN_STATES to derive its HTTP PR-create str set; task_lifecycle shim derives VALID_TRANSITIONS/ROLE_RESTRICTED_TRANSITIONS consumed by TaskService._validate_and_set_status; validate_git_requirements consumed by TaskService transition path; enforcement.__init__ re-exports a2a/journal/ownership gates consumed by journal services. At module load the spec calls run_all_lifecycle_validators() which BFS-validates the graph + compositions + claim/team tables, aborting container start on inconsistency. +CONTROL FLOW: (1) A2A — HTTP routes in roboco/api/routes/a2a.py construct A2AService(db) per request for card discovery, task get/list/cancel, conversation CRUD, message send, mark-read, inbox/pairs; the gateway Choreographer/content_actions use A2AService.send (UUID→slug resolved) for directed agent messaging. Legacy A2A-protocol path: create_a2a_notification requires a task_id, requires both from_agent and target_agent to be present/resolvable (raises distinct ValueError if either missing), enforces hierarchy unconditionally via validate_a2a_access (raises A2AAccessDeniedError), parses priority via foundation.policy.communications.parse_priority, then delegates to NotificationService.send_a2a_notification (which now runs the loop-prone 60s Redis re-fire guard). Bidirectional responses: update_task_from_message appends to dev_notes and, if dev_notes contains the 'A2A Request' marker, _notify_original_requester publishes a TASK_ASSIGNED event to the StreamEventBus to spawn the offline requester. CEO live view (wave 2/2c): every send() publishes A2A_MESSAGE_SENT (_publish_a2a_message_sent) which websocket_bridge forwards to /ws/system as an a2a.message frame for the panel's /a2a switchboard; the CEO-only /chat/admin/* routes (_require_ceo) read across all conversations via get_conversation_admin/list_conversations_admin/get_messages_admin/list_admin_pairs (no participant check) and reply_as_ceo chimes in via the normal send() path, which routes a reply-to-CEO through _get_conversation_for_reply_to_ceo and gates every non-CEO send to the CEO through _enforce_ceo_reply_budget. CEO-DM wake: send_chat_message and interject_as_ceo both call _maybe_wake_ceo_recipient after persisting, which — for a CEO-authored send to a read_a2a-capable recipient only — creates an a2a_request NotificationTable row with requires_ack=True (a per-row override on CreateNotificationParams/send_a2a_notification, since A2A_REQUEST's type default is requires_ack=False) so the orchestrator's _dispatch_a2a_work pending_ack_only poll can see and spawn the offline recipient; the pending-row lookup doubles as dedup, and _ack_pending_wake_notifications closes it out once the recipient actually reads via read_a2a. Agent-to-agent DM never wakes — pull-only by design. (2) Audit — TaskService (log_task_event at every transition chokepoint), task routes (log_task_action_denial on 403 for action denials; log_task_creation_denial on 403 for pre-task create denials — distinct target_type="task_creation" with no task_id), the orchestrator (log_agent_event on spawn/stop + has_recent_tracing_gap for the PM-respawn circuit breaker) and the Choreographer (log_event for gateway.rejected) all call get_audit_service(); _persist opens its own session+commit so audit writes never roll back the caller's transaction. (3) Journal — gateway content_actions.note → JournalService.write_entry (scope string→type via foundation SCOPE_TO_TYPE), and write_struggle/write_decision for PM write-then-gate verbs; create_entry commits the row then _schedule_rag_index fires asyncio.create_task (strong-ref in _RAG_INDEX_TASKS) that calls OptimalService.index_journal_entry (skipped for is_private) and record_learning for LEARNING entries; tracing-gate existence checks (has_decision_for_task/latest_decision_at/has_note_for_task/...) feed the Choreographer's gate decisions. (4) content_notes — TaskService._set_structured_note and gateway content_actions handoff path call apply_structured_note(task, content_type, payload); it validates via foundation.policy.content.validate_content BEFORE mutating, reassigns notes_structured (to flag the JSON column dirty), and writes render_markdown() into the derived TEXT mirror column. (5) Extraction — app lifespan builds ExtractionPipeline(ExtractionService()); stream route process_buffer → ExtractionService.extract (regex classify) → callbacks store/broadcast messages; extract_with_llm is the optional Anthropic/TOON path. (6) Permissions — notification/task/KB routes and gateway kb_authz call PermissionService methods synchronously (no DB) from an AgentContext; has_privileged_access/is_pm_role are async DB lookups used by route deps. DATA: inputs are AsyncSession + UUIDs/slugs/payloads; outputs are Pydantic models (A2ATask, A2AConversation, Journal, JournalEntry), audit rows, structured note columns, ExtractedMessage lists, and bool permission decisions. ## Mermaid ```mermaid -stateDiagram-v2 - [*] --> backlog - backlog --> pending: activate (PM) - pending --> claimed: claim - claimed --> in_progress: start - in_progress --> blocked: block - in_progress --> paused: pause - blocked --> in_progress: unblock (PM) - blocked --> pending: unblock (PM, never-claimed) - paused --> in_progress: resume - in_progress --> verifying: submit_verification - verifying --> awaiting_qa: submit_qa - in_progress --> awaiting_pr_review: submit_for_review (PM, pre:create_pr) - awaiting_pr_review --> claimed: claim (pr_reviewer) - awaiting_pr_review --> awaiting_pm_review: pr_pass - awaiting_pr_review --> needs_revision: pr_fail - awaiting_qa --> awaiting_documentation: qa_pass - awaiting_qa --> needs_revision: qa_fail - awaiting_qa --> blocked: QA park (legacy) - awaiting_documentation --> awaiting_pm_review: docs_complete - in_progress --> awaiting_pm_review: submit_pm_review - awaiting_pm_review --> completed: complete (PM) - awaiting_pm_review --> awaiting_ceo_approval: escalate_to_ceo - awaiting_pm_review --> claimed: PM re-claim (legacy) - awaiting_pm_review --> needs_revision: PM reject (legacy) - blocked --> awaiting_ceo_approval: escalate_to_ceo - awaiting_ceo_approval --> completed: ceo_approve - awaiting_ceo_approval --> needs_revision: ceo_reject - needs_revision --> claimed: claim (dev/PM) - needs_revision --> in_progress: re-entry (legacy) - in_progress --> completed: pr_review_done (pr_reviewer, external PR) - claimed --> pending: unclaim (legacy) - in_progress --> pending: reaper (legacy) - pending --> claimed: claim (QA/doc/PM/pr_reviewer by status) - * --> cancelled: cancel (PM/CEO) +graph TD + subgraph Gateway + CH[Choreographer] -->|send UUID->slug| A2A + CA[content_actions] -->|note scope| JRN + CA -->|handoff| CN + CH -->|log_event gateway.rejected| AUD + CH -->|tracing gates: has_*_for_task| JRN + end + subgraph HTTP + RA[routes/a2a] --> A2A + RT[routes/tasks] --> AUD + RJ[routes/journals] --> JRN + RS[routes/stream] --> EX + end + subgraph Orchestrator + ORC -->|log_agent_event| AUD + ORC -->|has_recent_tracing_gap| AUD + ORC -->|board_review_brief| JRN + end + subgraph Services + A2A[A2AService] -->|create_a2a_notification| NOT[NotificationService] + A2A -->|cancel_task| TS[TaskService] + A2A -->|response event| BUS[StreamEventBus] + JRN[JournalService] -->|fire-forget| OPT[OptimalService RAG] + TS -->|apply_structured_note| CN[content_notes] + CN -->|validate_content| FC[foundation.policy.content] + AUD[AuditService] -->|own session commit| DB[(audit_log)] + JRN -->|commit| DB2[(journals/journal_entries)] + A2A -->|flush/commit| DB3[(a2a_conversations/a2a_messages)] + PERM[PermissionService] -->|can_notify| FND[foundation.NOTIFY_SENDER_ROLES] + end + EX[ExtractionService] -->|messages| CB[stream callbacks] + NOT -->|dedup + re-fire guard| DB4[(notifications)] ``` ## Logical Tree ``` -foundation-lifecycle -├── roboco/foundation/policy/lifecycle.py (canonical spec) -│ ├── Enums: Status, TaskType, RejectionKind -│ ├── Dataclasses: Decision, Precondition, ActionSpec, IntentSpec, StatusTransition, Context -│ ├── _STATUS_TRANSITIONS (edge table) -> STATUS_GRAPH (derived) -│ ├── _ATOMIC_ACTIONS (22 ActionSpecs) -│ ├── CLAIM_RULES (per-role claimable statuses) -│ ├── ROLE_TEAM_RULES (per-slug team binding) -│ ├── Precondition predicates + PRECONDITION_* constants (PLAN/COMMITS/NO_PR/OWNERSHIP/NON_TERMINAL/PR_OPEN_STATE) -│ ├── PR_OPEN_STATES (canon) -│ ├── _INTENT_VERBS (~30 IntentSpecs) + _next_hint_* helpers -│ ├── Lookups: can_claim, can_invoke_action, can_invoke_intent, valid_next_verbs, composed_actions_for, intents_for_role, status_after -│ ├── Internal helpers: _check_role_status_type, _check_self_review_and_preconditions, _check_claim_rules_narrow, _check_intent_preconditions -│ └── UNMIGRATED / _KNOWN_UNMIGRATED_CONSUMERS (debt fence) -├── roboco/foundation/_validate_lifecycle.py (import-time validators) -│ ├── LifecycleSpecError -│ ├── reachable_from (BFS) -│ └── 13 _check_* validators -> run_all_lifecycle_validators -└── roboco/enforcement/ (backwards-compat + access control) - ├── __init__.py (re-export aggregator) - ├── task_lifecycle.py - │ ├── _LEGACY_OPERATIONAL_EDGES / _LEGACY_ROLE_GATES - │ ├── VALID_TRANSITIONS / ROLE_RESTRICTED_TRANSITIONS (derived, union-merged) - │ ├── validate_task_transition / can_agent_transition / get_valid_transitions - │ ├── is_terminal_state / is_waiting_state / is_active_state (hard-coded) - │ ├── ROLE_STATE_SLA_KEYS / sla_seconds_for - │ └── GitContext / GitRequirementError / validate_git_requirements / check_parallel_completion - ├── a2a_access.py (A2A direct-message gate) - ├── journal_perms.py (ReadTier-based journal read gate) - └── task_ownership.py (ownership + reassign + self-review) +a2a-audit-journal-permissions +├── A2AService (a2a.py) +│ ├── Agent Card builders: build_system_agent_card, build_agent_card, _agent_to_card, discover_agents +│ ├── Task↔A2A conversion: task_to_a2a, get_task, list_tasks, cancel_task, _status_value_of, _apply_cancel_note +│ ├── Legacy A2A-protocol path: extract_message_text, update_task_with_message, create_a2a_notification, update_task_from_message, _notify_original_requester, _publish_a2a_response_event, resolve_creator_agent, resolve_target_agent +│ ├── Persistent conversations: get_or_create_conversation, get_conversation, list_conversations, close_conversation, _canonical_pair +│ ├── Chat messages: send_chat_message (dedup, _enforce_ceo_reply_budget), get_messages, mark_read, mark_all_read, get_inbox_summary, list_pairs +│ ├── CEO admin/live-view (wave 2/2c): get_conversation_admin, list_conversations_admin, list_admin_pairs, get_messages_admin, _get_conversation_for_reply_to_ceo, _publish_a2a_message_sent +│ ├── CEO-DM wake (wave 3): _maybe_wake_ceo_recipient (send_chat_message + interject_as_ceo), _ack_pending_wake_notifications (mark_all_read + get_unread_messages) +│ ├── Conversions: _conv_to_model, _msg_to_model +│ └── Gateway adapter: send, _resolve_slug_from_id, get_team_from_agent +├── AuditService (audit.py) +│ ├── Persistence: _persist (own session), _coerce_uuid +│ ├── Writers: log_task_action_denial, log_task_creation_denial, log_task_event, log_event, log_agent_event +│ ├── Resolvers: _resolve_actor_role_from_db, _resolve_agent_id_by_slug +│ ├── Queries: has_recent_tracing_gap, get_recent_events +│ └── Singleton: _AuditServiceHolder, get_audit_service +├── JournalService (journal.py) +│ ├── Journal CRUD: get_or_create_journal, get_journal, get_journal_by_agent +│ ├── Entry CRUD: create_entry, get_entry, list_entries, delete_entry +│ ├── RAG indexing: _schedule_rag_index, _RAG_INDEX_TASKS, drain_rag_index_tasks +│ ├── Convenience builders: add_task_reflection, add_decision_log, add_learning, add_struggle, add_general_entry +│ ├── Analytics: get_journal_stats, get_growth_metrics, search_entries +│ ├── Tracing-gate checks: _has_entry_of_type, has_decision_for_task, latest_decision_at, has_note_for_task, has_learning_for_task, has_reflect_for_task, has_struggle_for_task, has_recent_entry +│ ├── Write-then-gate: write_struggle, write_decision +│ ├── Gateway adapter: write_entry (scope→type), _SCOPE_TO_TYPE +│ └── Board: board_review_brief +├── content_notes (content_notes.py) +│ ├── apply_structured_note (validate→persist→mirror) +│ ├── content_type_for_role +│ └── _MIRROR_COLUMN / _ROLE_TO_CONTENT_TYPE maps +├── ExtractionService / ExtractionPipeline (extraction.py) +│ ├── Pattern lists: REASONING/DIALOGUE/DECISION/ACTION/BLOCKER/TECHNICAL +│ ├── extract / _segment_content / _classify_segment / _compile_patterns +│ ├── LLM path: extract_with_llm, _call_anthropic_with_retry (TOON) +│ └── ExtractionPipeline.process_buffer + on_message callbacks +└── PermissionService + helpers (permissions.py) + ├── Notification: can_send_notifications, can_notify, _can_role_send_notifications, _get_notification_scope, _BOARD_NOTIFY_TARGETS + ├── Task/KB: can_perform_task_action, get_task_actions, can_perform_kb_action, get_kb_actions + ├── Utility: get_permission_level, check_all + └── Async DB: has_privileged_access, is_pm_role, PRIVILEGED_ROLES, MANAGEMENT_ROLES ``` ## Dependencies -- Internal: roboco.foundation.identity (Role, Team), roboco.foundation.policy.journaling (ReadTier, ROLE_READ_TIERS, PROTECTED_JOURNALS), roboco.seeds.initial_data (AGENT_UUIDS, DEFAULT_AGENTS) for slug/team validators, roboco.agents_config (can_a2a_direct, get_a2a_route_hint, get_agent_cell/role/team), roboco.config.settings (SLA key resolution), roboco.exceptions (RobocoError, TaskLifecycleError) -- External: dataclasses (frozen dataclasses), enum.StrEnum, collections.deque (BFS validator), itertools.pairwise (intent chain validator), typing (Literal, TYPE_CHECKING, Callable, Any) +- Internal: roboco.agents_config (ALL_AGENTS, get_agent_skills, get_agent_team), roboco.config.settings (host, port, app_version, anthropic_api_key, pm_decision_window_seconds), roboco.db.tables (A2AConversationTable, A2AMessageTable, AgentTable, TaskTable, JournalTable, JournalEntryTable, AuditLogTable), roboco.db.base.get_session_factory, roboco.enforcement.validate_a2a_access, roboco.events (Event, EventType, get_event_bus), roboco.foundation.policy.communications (parse_priority, NOTIFY_SENDER_ROLES, ACK_REQUIRED_BY_TYPE), roboco.foundation.policy.content (ContentModel, validate_content), roboco.foundation.policy.journaling (SCOPE_TO_TYPE), roboco.foundation.identity (Role, PM_ROLES, is_spawnable_agent_slug), roboco.agents_config.get_agent_role, roboco.services.gateway.role_config.get_role_config (local import — cycles back into this module at module scope), roboco.services.notification_delivery.get_notification_delivery_service, roboco.models (NotificationPriority, NotificationType), roboco.models.a2a, models.audit, models.base, models.journal, models.message, models.extraction, models.optimal, models.permissions, roboco.seeds.initial_data.AGENT_UUIDS, roboco.services.base (SingletonService, BaseService), roboco.services.task.TaskService, roboco.services.notification.NotificationService, roboco.services.optimal.OptimalService / get_optimal_service, roboco.services.repositories (resolve_agent_uuid, get_agent_slug), roboco.services.exceptions (RateLimitError, MAX_RATE_LIMIT_RETRIES), roboco.llm.ToonAdapter, roboco.utils.converters (require_uuid, to_python_uuid) +- External: sqlalchemy (select, update, or_, and_, func, AsyncSession), structlog, anthropic (AsyncAnthropic, RateLimitError), asyncio, ipaddress, re, uuid, dataclasses, datetime ## Entry Points | Name | File | Trigger | |---|---|---| -| import of roboco.foundation.policy.lifecycle | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py | module load — fires run_all_lifecycle_validators() at the bottom; a bad spec aborts the orchestrator container start | -| Choreographer verb dispatch (can_invoke_intent) | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/gateway/choreographer/_impl.py | every gateway flow verb call — role/verb/task/Context decision before any state mutation | -| role_config.py manifest build (intents_for_role) | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/gateway/role_config.py | import — derives per-role MCP tool manifest from the spec | -| envelope introspection (valid_next_verbs) | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/gateway/envelope.py | every Envelope populate — surfaces current_state + applicable verbs to the agent | -| TaskService transition (validate_task_transition / validate_git_requirements / VALID_TRANSITIONS) | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/task.py | TaskService._validate_and_set_status on every status write | -| GitService HTTP PR-create (PR_OPEN_STATES) | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/git.py | HTTP POST /api/v1/.../prs — derives its str-set gate from the lifecycle canon | -| stuck-task SLA sweep (sla_seconds_for / ROLE_STATE_SLA_KEYS) | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/runtime/orchestrator.py | orchestrator sweep tick | - -## Config Flags -- agent_sla_developer_in_progress -- agent_sla_developer_verifying -- agent_sla_qa_claimed -- agent_sla_documenter_claimed -- agent_sla_cell_pm_claimed (resolved by sla_seconds_for via roboco.config.settings) — no ROBOCO_* feature flag directly gates this slice; it is the always-on policy core - +| HTTP routes /api/a2a/* | roboco/api/routes/a2a.py | Agent card, task, conversation, message, inbox REST endpoints construct A2AService(db) per request | +| HTTP routes /api/a2a/chat/admin/* (CEO-only) | roboco/api/routes/a2a.py | Wave-2/2c: org-wide live view — list_admin_conversations, list_admin_pairs (switchboard), list_admin_chat_messages, reply_as_ceo — all gated by _require_ceo, delegating to A2AService.list_conversations_admin / list_admin_pairs / get_messages_admin / send | +| Gateway content_actions.note / handoff | roboco/services/gateway/content_actions.py | Agent note/dm verbs → JournalService.write_entry + content_type_for_role + apply_structured_note | +| Choreographer send | roboco/services/gateway/choreographer/ | Directed A2A verb → A2AService.send (UUID→slug) | +| Choreographer tracing gates | roboco/services/gateway/choreographer/ | i_will_work_on / delegate / submit gates query JournalService.has_*_for_task and latest_decision_at | +| TaskService transition chokepoint | roboco/services/task.py | Every status transition → AuditService.log_task_event + log_task_action_denial; structured note writes → apply_structured_note | +| Orchestrator spawn/stop + respawn breaker | roboco/runtime/orchestrator.py | Agent spawned/stopped/stranded → AuditService.log_agent_event; PM-respawn strike decision → has_recent_tracing_gap; board review → board_review_brief | +| HTTP routes /api/journals, /api/tasks board-review | roboco/api/routes/journals.py | Journal/entry/list/search/board-review-brief REST via get_journal_service(db) | +| HTTP route /api/stream process_buffer | roboco/api/routes/stream.py | Transcription buffer ready → ExtractionPipeline.process_buffer | +| FastAPI lifespan | roboco/api/app.py | App startup constructs ExtractionPipeline(ExtractionService()) on _AppServices | +| Route deps + notification/KB routes | roboco/api/deps.py | Per-request service bag wires A2A/Journal/Audit/Permission; routes call PermissionService + has_privileged_access/is_pm_role | +| CLI / direct module use | roboco/services/extraction.py | extract_with_llm runnable standalone (Anthropic key) | ## Gotchas -- can_invoke_intent only checks the FIRST composed action's source-status; subsequent actions in a multi-step composition (e.g. i_will_work_on = claim,set_plan,start) are NOT pre-checked — the verb runner must execute them in order and rollback on mid-composition failure, or the task is left stranded (e.g. claimed but never started). -- verbs with composes=() and not in the claim_review/claim_doc_task/claim_gate_review special-case list have NO source-status gate at the spec layer — their entire gate is extra_preconditions + the handler. sync_branch, open_pr, unclaim, reassign, escalate_up, give_me_work, i_am_idle, triage, triage_all all rely on this. A new empty-composes verb that transitions state without a NON_TERMINAL/source-status precondition can resurrect a terminal task (the F043 class of bug). -- _check_intent_preconditions drops the `missing` list for non-tracing rejection kinds (not_authorized/invalid_state) — Decision.reject sets missing=[]. Agents doing exact-string checks on `missing` will not see tokens for ownership/terminal/PR-open-state failures. -- enforcement/task_lifecycle.is_terminal_state/is_waiting_state/is_active_state are hard-coded string sets, NOT derived from STATUS_GRAPH; they now partition the full Status enum (ef33d56c added awaiting_pr_review; a2d2ef47 added backlog+pending to is_waiting_state so the three predicates cover every Status member), and test_status_classification_covers_every_enum_member in tests/unit/enforcement/test_task_lifecycle.py asserts full coverage — a new Status member that falls through all three predicates will now fail the build instead of silently miscategorizing. The general drift risk (hard-coded vs graph-derived) remains. -- _build_role_restricted_transitions UNION-merges legacy role gates with spec pins; an earlier overwrite silently dropped pr_reviewer from (in_progress, completed). Any new spec role_constraint on an edge that also has a _LEGACY_ROLE_GATES entry must be tested for the union, not overwrite. -- UNMIGRATED must stay a subset of _KNOWN_UNMIGRATED_CONSUMERS or import fails — extending known debt requires editing both frozensets. -- Decision is frozen; the allowed=True path forbids missing/remediate, the allowed=False path forbids rejection_kind=None — building one outside the classmethods via direct __init__ is supported but the invariants are enforced in __post_init__. -- STATUS_GRAPH includes a self-loop-ish edge: cancel is generated for every non-terminal source, so every non-terminal status has CANCELLED as a target — validators pass because of this; removing the cancel generator without re-adding per-source edges would break _check_terminal_exits. -- the spec module imports Role/Team from foundation.identity and _validate_lifecycle at the BOTTOM (after table defs); validators defer-import the spec to break the cycle — pyproject PLC0415 exemption exists for this. -- BLOCKED has two unblock targets (IN_PROGRESS for previously-claimed, PENDING for never-claimed) — the ActionSpec.unblock source_statuses is just {BLOCKED} with target IN_PROGRESS; the BLOCKED->PENDING edge lives only in _STATUS_TRANSITIONS/STATUS_GRAPH and is exercised by the legacy shim, not by an ActionSpec. -- ROLE_TEAM_RULES is keyed by slug, not role — adding a new agent slug without a row means needs_team_match enforcement falls back to None (any team) for that slug silently. +- a2a.send_chat_message dedup: suppresses an identical (conversation, sender, kind, content) message while a prior one is still unread — protects against respawn re-emits, but a genuinely repeated urgent message is also collapsed until the recipient reads. Keyed on content equality, so rewording defeats it (intended). +- a2a.get_or_create_conversation canonical ordering (_canonical_pair, lexically smaller first) is the uniqueness key; a non-canonical pair lookup will miss the existing row and create a duplicate. validate_a2a_access is enforced BEFORE the canonical swap. +- a2a.create_a2a_notification requires a task_id (raises ValueError without one), requires from_agent present (raises ValueError "requires a 'from_agent' in metadata") and target_agent resolvable (raises ValueError "could not resolve a target agent"), then enforces hierarchy unconditionally via validate_a2a_access (raises A2AAccessDeniedError + route_hint — no longer a bare ValueError indistinguishable from the missing-field errors); it does NOT create a conversation row — it goes through NotificationService, which now runs the 60s Redis loop-prone re-fire guard (all_recipients_recently_notified) that can silently drop a legitimate A2A notification. +- a2a._notify_original_requester only fires when task.dev_notes contains the literal 'A2A Request' marker; the marker is written by the legacy A2A-protocol path (update_task_with_message), NOT by the gateway conversation path, so gateway A2A messages never trigger requester re-spawn via this path. +- a2a.list_conversations runs an N+1 query (last-message preview per conversation); fine at low volume but unbounded by the 50-row limit can cost on heavy agents. +- a2a.mark_read zeroes the conv's per-side counter and bulk-updates read_at on inbound unread messages in the SAME session; if the caller never commits, the read state is lost. +- a2a._enforce_ceo_reply_budget is the only stateful check in an otherwise-stateless access model (can_a2a_direct blocks conversation *creation* unconditionally, not individual sends); it counts messages per conversation on every send, so a very long-running CEO thread pays an extra COUNT query pair per message. +- a2a._get_conversation_for_reply_to_ceo treats conversation existence itself as proof of CEO authorization (agents can never create a CEO conversation) — if that invariant is ever broken elsewhere (e.g. a future seed/migration inserting one directly), an agent could reply into a CEO thread it was never actually invited to. +- The CEO admin/live-view routes (get_conversation_admin, list_conversations_admin, get_messages_admin) intentionally skip the participant check that every non-admin read enforces; they are safe only because the routes themselves are behind _require_ceo — a missing or misapplied _require_ceo on any new admin route would expose every agent's A2A transcript. +- a2a._maybe_wake_ceo_recipient only fires for `from_slug == "ceo"` — agent-to-agent `dm` never wakes an offline recipient, deliberately, so ordinary same-cell chatter can't burn spawns. It also skips a recipient whose role manifest lacks `read_a2a` (auditor, pr_reviewer, prompter, secretary): a wake row that role could never ack would sit pending forever and permanently suppress the dedup pre-check for that recipient going forward. +- a2a._maybe_wake_ceo_recipient's dedup is a pending-notification lookup (`pending_ack_only=True`, `type_filter=A2A_REQUEST`), not a separate dedup table — it relies on `_ack_pending_wake_notifications` actually clearing the row once the recipient reads (`read_a2a`/`get_unread_messages`/`mark_all_read`). A recipient that never reads keeps the wake row pending forever, so a second CEO message to them creates no new wake notification (silently, by design) but also never re-spawns them via this path a second time. +- Before wave 3, `_dispatch_a2a_work`'s `pending_ack_only=True` poll (see docs/map/orchestrator.md) was structurally unable to see ANY a2a_request row, CEO or not, because `A2A_REQUEST`'s `ACK_REQUIRED_BY_TYPE` default is `requires_ack=False`. `send_a2a_notification` gained a `requires_ack` kwarg and `CreateNotificationParams` a per-row `requires_ack` override (`roboco/models/notification.py`) so `_maybe_wake_ceo_recipient` alone can opt its row in; every other `send_a2a_notification` caller (including the legacy `create_a2a_notification` path) still defaults to `requires_ack=False` and remains invisible to that poll. +- audit._persist opens its OWN session and commits independently — audit writes survive caller rollback (good) but mean audit rows can exist for operations that were later rolled back (forensic skew). Failures are logged, never raised. +- audit.log_task_action_denial resolves the actor's role from agents.role at write time, overriding the caller-supplied agent_role param (DB authoritative) — a stale caller param is silently replaced, which can surprise tests asserting the supplied role. +- audit.has_recent_tracing_gap filters details->>'reason' == 'tracing_gap' via JSONB; any row whose details JSON lacks that key or uses a different reason string is invisible to the circuit breaker (it will fall back to strike counting). +- journal.get_or_create_journal COMMITS on create (not flush) — calling it inside an outer unit-of-work will prematurely commit the outer transaction's pending state. +- journal.create_entry commits the entry then schedules RAG indexing fire-and-forget; on IntegrityError it rolls back and returns None (callers must handle None, not raise). The RAG index task holds a strong ref in _RAG_INDEX_TASKS; a RuntimeError when no event loop is running silently skips indexing. +- journal._schedule_rag_index SKIPS index_journal_entry for is_private entries (shared JOURNALS index would leak private reflections), but STILL records a private LEARNING via record_learning with shareable=False — two different sinks with two different privacy rules. +- journal.latest_decision_at backs the pm_decision_window_seconds windowed gate; the window is read by the Choreographer from settings, not enforced here — drift between this query and the choreographer's cutoff can admit or reject a decision based on clock skew. +- content_notes.apply_structured_note raises ContentValidationError BEFORE any mutation (no partial write), but it reassigns task.notes_structured = structured (a new dict) to flag the JSONB column dirty — in-place mutation of the existing dict would NOT mark it dirty and the write would be lost on commit. +- content_notes._MIRROR_COLUMN maps only 6 content types; a content type absent from the map is stored structured-only with NO TEXT mirror, so any legacy reader of the TEXT column sees stale/empty for that type. +- extraction._classify_segment defaults to REASONING@0.5 when no pattern matches — every unclassified utterance becomes 'reasoning', inflating reasoning counts. +- extraction.extract_with_llm hardcodes model='claude-3-haiku-20240307' (stale model id) and only falls back to pattern extract on non-RateLimit errors; a persistent 429 re-raises RateLimitError after MAX_RATE_LIMIT_RETRIES. +- permissions.can_perform_task_action CEO-bypasses ALL actions including team-scoped VIEW_OWN — the CEO operates across teams by design (panel), but any route gating through this helper lets the CEO through. +- permissions.has_privileged_access / is_pm_role query by (id == agent_id) OR (slug == str(agent_id)) because the CEO uses a UUID-style slug; passing a non-UUID slug that happens to collide with a slug column value can match the wrong row. ## Drift from CLAUDE.md -- CLAUDE.md 'Role-Based Transitions' table lists escalate_to_ceo only from awaiting_pm_review; the spec also declares BLOCKED -> AWAITING_CEO_APPROVAL via escalate_to_ceo (lifecycle.py:334-339, ActionSpec source_statuses includes BLOCKED at line 615). The BLOCKED->AWAITING_CEO_APPROVAL edge is missing from the doc table. -- CLAUDE.md verb table for developer lists resume/unclaim but omits i_am_idle (every role gets i_am_idle per spec line 1150); the doc does state 'every role also gets i_am_idle' in prose, so this is partial drift — the per-role table column omits it. -- CLAUDE.md says 'roboco/enforcement/task_lifecycle.py is a backwards-compat shim over it' — true for the transition tables, but the shim ALSO owns GitContext/validate_git_requirements/SLA tables (not a pure view); the doc undersells the shim's owned symbols. -- enforcement/__init__.py __all__ does NOT export sla_seconds_for, check_parallel_completion, or ROLE_STATE_SLA_KEYS even though task_lifecycle.py's own __all__ does — consumers must import from the submodule, not the package. Not a CLAUDE.md drift but a surface inconsistency. +- CLAUDE.md Services table lists 'PermissionsService' (plural) as the RBAC service; the actual class is 'PermissionService' (singular) at roboco/services/permissions.py:130. The re-export PM_ROLES and the async helpers has_privileged_access/is_pm_role are not mentioned in CLAUDE.md. +- CLAUDE.md's notification table says notifications are 'sent by PMs/Board only'; permissions.py matches this (_can_role_send_notifications excludes AUDITOR) — no behavioral drift. +- CLAUDE.md describes the A2A/conversation surface only via the gateway Choreographer; it does not document the legacy A2A-protocol path (create_a2a_notification / update_task_with_message / dev_notes 'A2A Request' marker / TASK_ASSIGNED re-spawn) which still lives in A2AService. +- CLAUDE.md says journal 'note' write returns immediately and RAG indexing is fire-and-forget — journal.py:324 _schedule_rag_index matches this exactly (no drift). CLAUDE.md also says 'journal indexing excludes is_private reflections from the shared corpus' — journal.py:343 matches (skips index_journal_entry when is_private). No drift. +- CLAUDE.md does not mention AuditService.has_recent_tracing_gap (the PM-respawn circuit-breaker query) or that audit._persist uses its own session/commit — both are undocumented audit behaviors. + + +## Regression Risks + +| Title | File:Line | Claim | Severity | +|---|---|---|---| +| A2A legacy notification suppressed by new loop-prone re-fire guard | roboco/services/a2a.py:640 | create_a2a_notification delegates to NotificationService.send_a2a_notification. Since 3aff6e04, send_a2a_notification runs all_recipients_recently_notified (60s Redis SET-NX) for loop-prone types before creating the notification. A legitimate A2A peer notification re-sent within 60s (e.g. after a real state change, not a respawn loop) can be silently dropped, so the target agent is never notified/spawned. The A2A path has no awareness of which notification_type it produces being loop-prone, and cannot bypass the guard. | medium | +| PrReviewContent schema change could invalidate stale structured-note payloads | roboco/services/content_notes.py:65 | 15effce0 added optional 'issues' and 'head_sha' fields to PrReviewContent (foundation.policy.content.models). apply_structured_note calls validate_content which routes to PrReviewContent. The new fields have defaults (empty list / None) so existing payloads still validate and existing TEXT mirrors regenerate unchanged (Issues section only appended when issues present). Low risk, but any code that round-trips notes_structured.pr_review and assumed the exact key set may now encounter new keys. | low | +| Journal fire-and-forget RAG index silent on no-event-loop | roboco/services/journal.py:367 | _schedule_rag_index catches RuntimeError (no running event loop) and returns silently — no indexing, no log. Unchanged since baseline, but if a caller (e.g. a sync test or a non-async lifespan hook) writes a journal entry outside a running loop, the entry is persisted with zero RAG indexing and no warning. Combined with the new private-learning dual-sink rule this is easy to mis-verify. | low | +| Audit actor-role override can mismatch test expectations | roboco/services/audit.py:126 | log_task_action_denial resolves actual role from agents.role and overrides the caller-supplied agent_role. If a test seeds an agent with a role but passes a different agent_role param and asserts the persisted details.agent_role, it will see the DB role, not the param. Unchanged since baseline but a known foot-gun for regression tests added in the 15effce0 gap-fill batch. | low | + +## Changes Since Baseline + +> Post-snapshot updates (since 2026-06-29): +> - **b49337e7** `[chore] route-layer force gate + privileged-field gate + pre-task audit attribution` — audit.py: added `log_task_creation_denial` (target_type="task_creation", no task_id) as distinct from `log_task_action_denial`; `log_task_action_denial` now preserves non-UUID task_id sentinels in `details["target_id_raw"]` instead of silently dropping to NULL. +> - **d8a5bb48** `[chore] a2a service hierarchy gate (typed, unconditional) + persist skill on message row` — a2a.py: `create_a2a_notification` hierarchy gate is now unconditional (raises distinct ValueError if from_agent missing or target unresolvable, then calls `validate_a2a_access` raising typed A2AAccessDeniedError + route_hint instead of bare ValueError); `send_chat_message` reads and persists `skill` from opts on the message row (migration 054 adds nullable skill column on a2a_messages); `_msg_to_model` maps skill field; `send()` docstring updated. +> - **5bec3ec5** `[chore] a2a-routes: authenticate send_message responder + gate cancel task (PM-only)` — a2a.py: `cancel_task` gains `agent_role` (threaded into TaskService.cancel role gate) and `actor_slug` (recorded in cancellation note) params; the route now requires PM/management auth and passes the authenticated slug. +> - **b3558d4e** `[chore] complexity: split 5 C-rank blocks to <=B for xenon gate` — a2a.py: `cancel_task` factored into helpers `_status_value_of` (line 383) and `_apply_cancel_note` (line 387); no behavior change. +> - **da563487** `Wave 2 features: A2A live view (CEO chime-in + reply budget) and prompter memory (#297)` — a2a.py grows by ~250 lines: adds the CEO admin/live-view surface (`get_conversation_admin`, `list_conversations_admin`, `get_messages_admin`, `_enforce_ceo_reply_budget`, `_get_conversation_for_reply_to_ceo`) and the `A2A_MESSAGE_SENT` publish (`_publish_a2a_message_sent`, called from `send`) for the operator's org-wide watch view; `roboco/models/events.py` adds `EventType.A2A_MESSAGE_SENT`; `websocket_bridge.py` adds `_handle_a2a_message_event` forwarding it to `/ws/system` as an `a2a.message` frame. `routes/a2a.py` adds the CEO-gated `/chat/admin/conversations`, `/chat/admin/conversations/{id}/messages`, `/chat/admin/conversations/{id}/reply` routes (`_require_ceo`). +> - **876e19b3** `A2A switchboard (pair cards), Secretary/PM task access + closed over-permission hole, MegaTask conventions fix (#298)` — a2a.py adds `list_admin_pairs` (the switchboard's one-bulk-query pair+conversation join over `agents_config.A2A_ALLOWED_PAIRS`); `routes/a2a.py` adds the CEO-gated `/chat/admin/pairs` route. This commit also tightened `roboco/api/routes/tasks.py` (`_pm_editor_scope` / `_enforce_pm_lighter_fields`, out of this slice) and gave `SecretaryService` its `edit` directive action — see `docs/map/intake-secretary.md`. +> - **Wave 3** (2026-07-17, branch `feature/wave-3-a2a-ceo`, PR #547) — CEO-authored A2A DMs (panel "New DM" composer or `interject_as_ceo`) now wake an offline recipient: `_maybe_wake_ceo_recipient` (new) + `_ack_pending_wake_notifications` (new), gated to `read_a2a`-capable roles, reusing the legacy `a2a_request` NotificationTable row with a new per-row `requires_ack=True` override so it's finally visible to `_dispatch_a2a_work`'s `pending_ack_only` poll (previously structurally dead for every `a2a_request` row — see Gotchas). `send_a2a_notification` gains `requires_ack: bool = False` and an optional (`str | None`) `task_id`; `CreateNotificationParams` gains `requires_ack: bool | None = None`. Agent-to-agent `dm` is unaffected — still pull-only, no wake. Companion panel work (New-DM dialog, CEO direct-thread composer) is in `docs/map/panel.md`; the docs scrub that removed CEO-DM teaching from `docs/rag`/`agents/prompts` landed on the same branch (`ee620cf3`). +> - **`56b6693e`** ("security-hygiene-sweep") — zero code change in this slice's own files, but a behavior change flows through it: `agents_config.can_a2a_direct`'s CEO branch now consults `_check_ceo_a2a` (refuses `NO_COMMS_ROLES` targets — see `docs/map/foundation-policy-misc.md` / `docs/map/prompts-roles-taxonomy.md`), and `get_or_create_conversation`'s existing `validate_a2a_access(from_agent, to_agent)` call (line 784, unchanged) already delegates straight into `can_a2a_direct`. So a CEO DM to `auditor`/`pr_reviewer`/`prompter`/`secretary` now raises `A2AAccessDeniedError` at conversation-CREATION time — superseding the previous symptom-level mitigation, which let the conversation exist and only suppressed `_maybe_wake_ceo_recipient`'s wake notification (that independent manifest check still no-ops too, now unreachable via the normal send path but exercised directly by its own test as defense-in-depth). + +## Health +This slice is mature and internally consistent: the six services have clear separation of concerns (A2A transport/conversation, audit forensics, journal CRUD+RAG, note persistence chokepoint, stream extraction, RBAC), and the gateway/HTTP/orchestrator entry points map cleanly onto them. The code is defensive in the right places — audit._persist is best-effort with its own session, journal RAG indexing is fire-and-forget with a strong-ref guard, content_notes validates before mutating, and A2A conversation dedup prevents respawn re-emit storms. The main integrity concerns are cross-layer, not in-slice: (1) the new 60s Redis loop-prone notification re-fire guard (3aff6e04) sits between A2A's create_a2a_notification and delivery and can silently drop legitimate A2A notifications; (2) the legacy A2A-protocol path (dev_notes 'A2A Request' marker, TASK_ASSIGNED re-spawn) is undocumented in CLAUDE.md and coexists with the gateway conversation path, a known source of future confusion. No in-slice file changed since the fd10cc86 baseline, so there is no direct regression surface; the risks above are all dependency-mediated. Recommend a regression test that an A2A notification fired twice within 60s for genuinely different reasons still delivers. + +## Purpose +This slice is the packaging, build, and runtime-tooling layer of RoboCo: the Docker compose topologies (build-from-source and pre-built registry), per-service Dockerfiles (orchestrator + a family of agent role images extending a shared agent-base, plus the Next.js panel and nginx proxy), the uv/pnpm dependency manifests, the Makefile quality-gate and ops targets, the Pydantic-settings config (roboco/config.py) that every service reads, the bootstrap/CLI entrypoints that start the orchestrator, the structured-logging + exception hierarchy wired across the app, and the ops/CI helper scripts (lifecycle-artifact regeneration, postgres enum-parity verification, markdown reflow, runtime-state reset) and per-agent Claude/Grok hook scripts. + +## Files + +| Path | Role | LOC | +|---|---|---| +| docker-compose.yaml | Build-from-source compose: postgres/redis/ollama/ollama-init, 14 agent-*-image builders, orchestrator, panel, nginx, `backup` pg_dump sidecar on the `roboco_data` DB-isolation network; NAS prod env vars + volume mounts, `ROBOCO_OBSIDIAN_VAULT_ENABLED`/`ROBOCO_VAULT_INTAKE_ENABLED` default `true` | 556 | +| docker-compose.yml | Byte-identical copy of docker-compose.yaml (kept for the canonical name compose picks up by default) | 556 | +| docker-compose.registry.yml | Pull-and-run compose using pre-built GHCR/Docker Hub images (ROBOCO_REGISTRY + ROBOCO_VERSION); infra services byte-identical to build compose (incl. `backup` + vault defaults), agent-* services are one-shot pre-pulls, own `roboco_data` network | 334 | +| Makefile | Ops + quality targets: infra, dev/run/orchestrator, quality gate (ruff/mypy/pytest/xenon/radon/vulture/bandit/pip-audit/deptry/lint-imports/alembic/foundation-check), per-Python test matrix, docs, lifecycle regen | 548 | +| pyproject.toml | Project + dependency manifest: requires-python >=3.13,<3.15, deps, dev/docs extras, console scripts, ruff/mypy/pytest/coverage/vulture/bandit/radon/xenon/deptry/importlinter config | 451 | +| roboco/config.py | Pydantic Settings (env prefix ROBOCO_, cached via lru_cache); every tunable: DB, Redis, RAG, LLM/Ollama, workspaces, agent guardrails, gateway thresholds, autonomy-engine flags | 1022 | +| roboco/__init__.py | Package root: __version__ + re-exports settings, exceptions, logging helpers | 39 | +| roboco/bootstrap.py | Async bootstrap: DB init, Redis event bus, websocket bridge, orchestrator construction, uvicorn API server task, wait-for-ready poll, optional agent spawn, graceful shutdown | 161 | +| roboco/cli.py | argparse CLI wrapper around bootstrap.main / db-only; the ENTRYPOINT invoked by `python -m roboco.cli` | 59 | +| roboco/logging.py | structlog setup (dev ConsoleRenderer / prod JSONRenderer), secret-redaction processor, rotating file handler under /data/logs, LogContext context-manager | 252 | +| roboco/exceptions.py | Exception hierarchy (RobocoError base + NotFound/Validation/InvalidState/Permission/Auth/Task/TaskLifecycle/Agent/Notification/Service/Database/Git/MergeConflict/GitCommand/GitTimeout); includes TaskLifecycle transition hints and git-secret scrubbing | 497 | +| docker/orchestrator.Dockerfile | Multi-stage build: uv venv builder (python:3.13-slim) + runner with docker-cli/git/make/node/npm/pnpm, ENTRYPOINT python -m roboco.cli | 99 | +| docker/agent-base.Dockerfile | Shared agent runtime: uv venv + Node 22 + @anthropic-ai/claude-code, agent user, hook scripts, safe.directory *, ENTRYPOINT claude | 108 | +| docker/agent-pm.Dockerfile | PM agent — FROM roboco-agent-base, no extra tools (MCP-only) | 11 | +| docker/agent-dev-be.Dockerfile | Backend dev — adds postgresql-client + redis-tools on top of base | 18 | +| docker/agent-dev-fe.Dockerfile | Frontend dev — adds Playwright system deps + pnpm + chromium browser | 37 | +| docker/agent-qa-be.Dockerfile | Backend QA — adds postgresql-client on top of base | 18 | +| docker/agent-qa-fe.Dockerfile | Frontend QA — pnpm + `chromium-headless-shell` (not full chromium, under `/app/.playwright`) + `@playwright/mcp` for browser-based verification, wrapped by `playwright-mcp-entrypoint.sh` | 36 | +| docker/agent-ux.Dockerfile | UX/UI agent (shared by ux-dev + ux-qa) — FROM base + the same `chromium-headless-shell` + `@playwright/mcp` as agent-qa-fe (role-gated at registration, not image-gated, so ux-dev never sees the MCP server despite sharing the image) | 13 | +| docker/agent-doc.Dockerfile | Documenter — FROM base, no extra tools | 12 | +| docker/agent-prompter.Dockerfile | Intake (Prompter) — persistent Claude Agent SDK session, ENTRYPOINT python -m roboco.agent_sdk.intake_main | 17 | +| docker/agent-secretary.Dockerfile | Secretary — persistent Claude Agent SDK session with gated CEO-authority tools, ENTRYPOINT python -m roboco.agent_sdk.secretary_main | 19 | +| docker/agent-pr-reviewer.Dockerfile | PR Reviewer — FROM base, keeps claude entrypoint; read-only reviewer dispatched per review task | 16 | +| docker/agent-grok.Dockerfile | Grok runtime — base + official grok CLI 0.2.56 install, ENTRYPOINT grok-cli-agent-entrypoint.sh | 48 | +| docker/agent-grok-prompter.Dockerfile | Grok intake — FROM grok, ENTRYPOINT python -m roboco.agent_sdk.grok_intake_main, EXPOSE 9000 | 23 | +| docker/agent-grok-secretary.Dockerfile | Grok secretary — FROM grok, ENTRYPOINT python -m roboco.agent_sdk.grok_secretary_main, EXPOSE 9000 | 23 | +| docker/panel.Dockerfile | Multi-stage Next.js build (node:22-alpine, pnpm, shamefully-hoist), non-root nextjs runtime serving server.js on :3000 | 76 | +| docker/postgres-pgvector.Dockerfile | Example custom pgvector build (pg17) — currently unused; compose uses pgvector/pgvector:pg16 image directly | 15 | +| docker/nginx.conf | nginx default.conf template: /health /ready /api/ /ws/ -> orchestrator (with X-Agent-Token header), everything else -> panel | 67 | +| docker/postgres-init/01-create-extensions.sql | First-init SQL: CREATE EXTENSION IF NOT EXISTS vector + availability check (used by the pgvector image entrypoint) | 17 | +| docker/scripts/backup-entrypoint.sh | `backup` sidecar's own loop (not a Claude hook): `pg_dump -Fc` on start + every `BACKUP_INTERVAL_SECONDS` (default 24h), `.tmp`-suffix-then-rename, prune to newest `BACKUP_KEEP` (default 14) by mtime; runs on the data-only network | 46 | +| docker/scripts/playwright-mcp-entrypoint.sh | Wrapper entrypoint for `@playwright/mcp`: points it at the image's own baked `chromium-headless-shell` instead of letting the MCP package download a second browser | 20 | +| docker/scripts/sdk-startup-hook.sh | Claude SessionStart hook: start SDK server on :9000 with UV_PROJECT_ENVIRONMENT=/app/.venv + --no-sync, reset budget, print briefing/precompact | 58 | +| docker/scripts/a2a-check-hook.sh | PostToolUse hook: poll SDK /inbox/count and remind the agent of pending A2A messages (always exits 0) | 30 | +| docker/scripts/bash-guard-hook.sh | PreToolUse Bash guard: deny compound shell git network ops, PR/merge bypass, credential exfil, and uv run --active / uv targeting /app/.venv (exit 2 to deny) | 354 | +| docker/scripts/post-tool-budget-hook.sh | PostToolUse: per-session budget counter + loop detector via SDK /budget/tool_called; halts on hard cap, denies on loop+halt | 85 | +| docker/scripts/usage-report-hook.sh | PostToolUse+Stop: fire-and-forget POST /usage/sync with the transcript_path so the SDK parses token usage | 36 | +| docker/scripts/stop-hook.sh | Stop hook: block ungraceful exits (exit 2 first time, role-specific terminal-verb reminder); auto-substitute after exceeding allowance | 71 | +| docker/scripts/user-prompt-hook.sh | UserPromptSubmit: prompt-injection guard (deny classic jailbreak patterns, exit 2) + budget nudge | 72 | +| docker/scripts/pre-compact-hook.sh | PreCompact: snapshot budget + terminal status to /tmp/roboco-precompact-.md for session resume | 48 | +| docker/scripts/session-end-hook.sh | SessionEnd: post a reflective journal post-mortem (tool count, halt/loop, last terminal tool) to the SDK | 49 | +| docker/scripts/fable-{stop-gate,bash-discipline,honesty-nudge,prompt-nudge,precompact}-hook.sh | 5 vendored fable-mode hook scripts (from `opus-fable-playbook` v0.1.3), installed only when `fable_mode_enabled`: Stop/SubagentStop turn-discipline gate, PreToolUse[Bash] read-tool discipline, PostToolUse[Bash] honesty nudge (the one also ported to grok), UserPromptSubmit shape-matched reminder, PreCompact survival-list injection; all fail-open | ~200 | +| docker/scripts/grok-cli-agent-entrypoint.sh | Grok runtime entrypoint: render ~/.grok/config.toml, prompt-guard, symlink auth.json from RO mount, grok_auth --check (exit 78 on stale), run grok -p streaming-json, capture usage, exit 75 on 429/quota | 112 | +| docker/scripts/tests/bash-guard-tests.sh | bash-guard-hook test harness: run_case allow/deny table incl. the new /app venv-protection cases | 7451 | +| scripts/build_lifecycle_artifacts.py | Deterministic regeneration of lifecycle artifacts (intent-verbs.md, status-transitions.md, panel/lib/lifecycle.json, per-role prompt fragments) from foundation.policy.lifecycle | 57 | +| scripts/regenerate_verb_tables.py | Regenerate agents/prompts/_generated/verbs.md + per-role verb tables from role_config ROLE_CONFIGS + Pydantic flow/do schemas (skips driver-based prompter/secretary) | 233 | +| scripts/verify_postgres_enums.py | Foundation drift gate: compare postgres agentrole/team enum labels to foundation identity; exit 0 on match/skip(unreachable or unmigrated), 1 on drift | 127 | +| scripts/reflow_md.py | Reflow hard-wrapped markdown prose to one line per paragraph (token-invariant safety check); --apply / --check modes for the CI gate | 214 | +| scripts/reset_runtime_state.sh | Host/container smoke-test reset: stop agent containers, run reset_runtime_state.sql, FLUSH Redis, optional FULL_RESET data wipe, per-workspace git hard-reset + stray-branch prune | 263 | +| scripts/reset_runtime_state.sql | Transactional DELETE of runtime tables (tasks/.../a2a_*) preserving agents/projects/alembic_version; resets agents.metrics | 159 | + +## Key Symbols + +| Name | Kind | File:Line | Responsibility | +|---|---|---|---| +| Settings | class | roboco/config.py:13 | Pydantic BaseSettings; all ROBOCO_ env-backed tunables + computed fields (database_url, redis_url, internal_api_url, rag_store_url) | +| get_settings | function | roboco/config.py:1015 | lru_cache-backed singleton factory returning the cached Settings instance | +| internal_api_url | property | roboco/config.py:57 | computed_field: service-to-service API base URL (api_url override or http://host:port/api, maps 0.0.0.0 to 127.0.0.1) | +| database_url | property | roboco/config.py:85 | computed_field: asyncpg connection URL | +| database_url_sync | property | roboco/config.py:93 | computed_field: sync psycopg URL for Alembic | +| redis_url | property | roboco/config.py:110 | computed_field: Redis connection URL (optional password) | +| _BootstrapHolder | class | roboco/bootstrap.py:27 | Module-level holder for the singleton AgentOrchestrator instance | +| _run_api_server | function | roboco/bootstrap.py:33 | Build a uvicorn.Config for roboco.api.app:app and serve it (no reload in prod/container) | +| _wait_for_api_ready | function | roboco/bootstrap.py:46 | Poll http://127.0.0.1:port/health until 200 (up to max_wait) so the orchestrator starts only after lifespan indexing completes | +| main | function | roboco/bootstrap.py:69 | Async entrypoint: bootstrap_database, init Redis event bus + handlers, websocket bridge, orchestrator, DI wiring, API task, ready poll, optional spawns, graceful shutdown | +| parse_args | function | roboco/cli.py:15 | argparse: --skip-db, --skip-orchestrator, --spawn, --db-only | +| cli | function | roboco/cli.py:41 | Console-script entry: dispatch db-only or full bootstrap via asyncio.run | +| add_app_context | function | roboco/logging.py:25 | structlog processor injecting app/version/environment into every event dict | +| _redact_secrets | function | roboco/logging.py:60 | Regex-replace GitHub PAT / bearer / embedded-URL-credential shapes with | +| redact_event_dict | function | roboco/logging.py:70 | structlog processor running last; redacts every value in the event dict | +| setup_logging | function | roboco/logging.py:86 | Configure structlog (dev ConsoleRenderer / prod JSONRenderer), stdlib root logger, rotating file handler under /data/logs, quiet noisy libs | +| _resolve_log_dir | function | roboco/logging.py:169 | Resolve log dir: ROBOCO_LOG_DIR > /data/logs > ${ROBOCO_DATA_DIR:-./data}/logs > None | +| get_logger | function | roboco/logging.py:192 | Return a configured structlog BoundLogger | +| LogContext | class | roboco/logging.py:210 | Context manager binding/unbinding contextvars for scoped log context | +| log_operation | function | roboco/logging.py:231 | Build a structured-log context dict for an operation | +| RobocoError | class | roboco/exceptions.py:13 | Base exception with message/code/details and to_dict() for API responses | +| NotFoundError | class | roboco/exceptions.py:50 | Resource not found (code NOT_FOUND) | +| ValidationError | class | roboco/exceptions.py:77 | Input validation failure (code VALIDATION_ERROR) | +| InvalidStateError | class | roboco/exceptions.py:98 | Operation not allowed in current state (code INVALID_STATE) | +| PermissionDeniedError | class | roboco/exceptions.py:128 | Agent lacks permission (code PERMISSION_DENIED) | +| AuthenticationError | class | roboco/exceptions.py:151 | Auth required (code AUTHENTICATION_REQUIRED) | +| TaskError | class | roboco/exceptions.py:171 | Base task error (code TASK_ERROR), carries task_id | +| TaskLifecycleError | class | roboco/exceptions.py:191 | Invalid transition; _TRANSITION_HINTS table appends procedural tool-call hints for common footguns | +| AgentError | class | roboco/exceptions.py:273 | Base agent error carrying agent_id | +| NotificationError | class | roboco/exceptions.py:364 | Notification error base | +| ServiceError | class | roboco/exceptions.py:381 | External service error carrying service name | +| DatabaseError | class | roboco/exceptions.py:400 | Database operation failed | +| GitError | class | roboco/exceptions.py:424 | Base git operation error | +| _scrub_git_secrets | function | roboco/exceptions.py:439 | Redact creds a git command may echo into stderr (URL creds, Authorization basic, PATs) | +| MergeConflictError | class | roboco/exceptions.py:456 | PR merge refused for conflict; routes completion to conflict resolution | +| GitCommandError | class | roboco/exceptions.py:467 | git command failed; surfaces a secret-free stderr tail in .message | +| GitTimeoutError | class | roboco/exceptions.py:487 | git command timed out after Ns | +| write | function | scripts/build_lifecycle_artifacts.py:24 | mkdir -p + write_text, print relative path | +| main | function | scripts/build_lifecycle_artifacts.py:30 | Render intent-verbs.md, status-transitions.md, panel/lib/lifecycle.json, per-role lifecycle-{role}.md via foundation._generators | +| fetch_enum_values | function | scripts/verify_postgres_enums.py:26 | Query pg_enum for a type's labels | +| type_exists | function | scripts/verify_postgres_enums.py:40 | True iff a postgres enum type exists | +| should_skip_for_unmigrated | function | scripts/verify_postgres_enums.py:49 | Skip only when BOTH agentrole and team absent (DB not migrated); partial = drift | +| enum_drift | function | scripts/verify_postgres_enums.py:59 | Compare DB enum sets vs foundation Role/Team; return (has_drift, messages) | +| main | function | scripts/verify_postgres_enums.py:82 | Connect via asyncpg, fetch enums, skip-or-report drift against foundation identity (exit 0 skip/match, 1 drift) | +| reflow | function | scripts/reflow_md.py:141 | Token-invariant reflow of markdown prose (paragraphs, list items, front matter, code fences verbatim) | +| in_scope | function | scripts/reflow_md.py:162 | Skip SKIP_DIRS / EXCLUDE_PREFIXES (docs/internal, agents/prompts/_generated) / EXCLUDE_GLOBS | +| main | function | scripts/reflow_md.py:171 | Walk *.md, reflow, enforce token-invariant safety, --apply / --check (exit 1 on wrapped files) | +| _render_role_section | function | scripts/regenerate_verb_tables.py:138 | Render one role's flow + do verb tables from role_config + schemas | +| main | function | scripts/regenerate_verb_tables.py:181 | Write agents/prompts/_generated/verbs.md + per-role {role}.md, skipping driver-based prompter/secretary | +| _reset_workspace | function | scripts/reset_runtime_state.sh:177 | Per-clone git hard-reset to default branch + clean -fdx + delete stray feature branches (as owner) | +| _resolve_workspaces_root | function | scripts/reset_runtime_state.sh:154 | Resolve WORKSPACES_ROOT from env, then NAS /volume1/roboco/data/workspaces, then /data/workspaces | + +## Data Flow +Compose brings up infra (postgres/redis/ollama + ollama-init verifying models) then the orchestrator image, whose ENTRYPOINT `python -m roboco.cli` calls `roboco.cli.cli` -> `asyncio.run(bootstrap.main(...))`. bootstrap.main reads `roboco.config.settings` (env-injected by compose), runs `bootstrap_database()` (Alembic/migrations via the sync URL), inits the Redis event bus, starts the websocket bridge, constructs `AgentOrchestrator`, sets it on the API deps, launches uvicorn on `roboco.api.app:app`, polls `/health`, then `orchestrator.start()`. The orchestrator spawns per-role agent containers from the agent-*-image family (or registry pre-builts when ROBOCO_AGENT_IMAGE_REGISTRY set); each agent container runs `claude` (or the grok-cli entrypoint / agent_sdk driver for prompter/secretary/grok-*), with host volumes (workspaces, manifests, briefings, grok-usage, logs, ~/.claude, ~/.grok) bind-mounted in. Inside an agent container, Claude Code's hook system calls the docker/scripts/* hooks, which POST to the in-container SDK server on :9000 (budget/terminal/usage/inbox); the usage-report + post-tool-budget hooks feed token/loop state back to the SDK, and stop-hook forces a terminal verb before exit. nginx (port 3000) proxies /api and /ws to the orchestrator with the CEO panel token injected, and everything else to the Next.js panel. The Makefile `quality`/`foundation-check` gates run offline (ruff/mypy/pytest/xenon/.../verify_postgres_enums/build_lifecycle_artifacts) and are the merge barrier. reset_runtime_state.sh/.sql wipe runtime rows between smoke runs while preserving org scaffolding. + +## Mermaid +```mermaid +graph TD + subgraph Compose["docker-compose.{yml,yaml,registry.yml}"] + PG["postgres pgvector:pg16"] + REDIS["redis:8-alpine"] + OLL["ollama:latest"] + OLLI["ollama-init (curl pull+verify models)"] + ORC["orchestrator (python -m roboco.cli)"] + PN["panel (node server.js)"] + NG["nginx :3000"] + end + OLL --> OLLI + OLLI -->|service_completed_successfully| ORC + PG -->|service_healthy| ORC + REDIS -->|service_healthy| ORC + ORC -->|spawns| AGENT["agent containers (claude/grok entrypoint)"] + PN --> NG + ORC --> NG + NG -->|/api /ws /health| ORC + NG -->|everything else| PN + ORC -->|docker.sock + host volumes| AGENT + AGENT -->|hooks POST :9000| SDK["in-container SDK server"] + SDK -->|usage/budget/terminal| AGENT + + subgraph Bootstrap["roboco/bootstrap.py + cli.py"] + CLI["cli.cli (argparse)"] --> MAIN["bootstrap.main (async)"] + MAIN --> BD["bootstrap_database()"] + MAIN --> EB["init_event_bus (Redis Streams)"] + MAIN --> WSB["start_websocket_bridge()"] + MAIN --> ORCH["AgentOrchestrator()"] + MAIN --> UV["uvicorn roboco.api.app:app"] + MAIN --> READY["_wait_for_api_ready (/health poll)"] + READY --> START["orchestrator.start()"] + end + + subgraph Quality["Makefile gates"] + Q["make quality"] --> RUFF[ruff] --> MYPY[mypy] --> PYT[pytest cov>=80] --> XEN[xenon] --> RAD[radon] --> VUL[vulture] --> BAND[bandit] --> PIPA[pip-audit] --> DEPT[deptry] --> AL[alembic --sql] --> LINTI[lint-imports] --> FC["make foundation-check"] + FC --> IDV["foundation._validate"] --> TVP["tracing verb parity"] --> JC["journaling consumers"] --> CC["communications consumers"] --> LART["make lifecycle"] --> ENUM["verify_postgres_enums.py"] + end +``` + +## Logical Tree +``` +deployment-tooling +├─ Compose topologies +│ ├─ docker-compose.yaml / .yml (build-from-source) +│ │ ├─ infra: postgres, redis, ollama, ollama-init, backup (pg_dump sidecar, always on) +│ │ ├─ agent-*-image builders (14, FROM docker/agent-*.Dockerfile) +│ │ ├─ orchestrator (build context=.) +│ │ ├─ panel (build) +│ │ └─ nginx (image) — single :3000 entry, ROBOCO_PANEL_AGENT_TOKEN envsubst +│ └─ docker-compose.registry.yml (pre-built images via ROBOCO_REGISTRY/ROBOCO_VERSION) +│ ├─ infra (byte-identical) +│ ├─ agent-* one-shot pre-pulls +│ ├─ orchestrator (image, ROBOCO_AGENT_IMAGE_REGISTRY/TAG) +│ └─ panel + nginx (image) +├─ Dockerfiles (docker/) +│ ├─ orchestrator.Dockerfile (builder + runner w/ docker-cli, git, make, node, pnpm, uv) +│ ├─ agent-base.Dockerfile (venv + Node22 + claude-code + hooks, USER agent) +│ │ └─ docker/scripts/*.sh (sdk-startup, a2a-check, bash-guard, post-tool-budget, usage-report, stop, user-prompt, pre-compact, session-end, + 5 default-off fable-*.sh gated by fable_mode_enabled) +│ ├─ role images FROM agent-base: pm, dev-be, dev-fe, qa-be, qa-fe, ux, doc, prompter, secretary, pr-reviewer +│ ├─ grok family: agent-grok.Dockerfile (+ grok CLI 0.2.56, grok-cli-agent-entrypoint.sh) +│ │ └─ agent-grok-prompter / agent-grok-secretary (FROM grok, agent_sdk drivers) +│ ├─ panel.Dockerfile (Next.js standalone, non-root nextjs) +│ ├─ postgres-pgvector.Dockerfile (example, unused) +│ ├─ nginx.conf (proxy template) +│ └─ postgres-init/01-create-extensions.sql +├─ Python runtime core +│ ├─ roboco/config.py (Settings + computed fields + get_settings lru_cache) +│ ├─ roboco/__init__.py (version + re-exports) +│ ├─ roboco/cli.py (argparse -> bootstrap.main / db-only) +│ ├─ roboco/bootstrap.py (DB+bus+orchestrator+uvicorn+ready-poll) +│ ├─ roboco/logging.py (structlog + secret redaction + file rotation + LogContext) +│ └─ roboco/exceptions.py (hierarchy + transition hints + git secret scrub) +├─ Build/quality manifest +│ └─ pyproject.toml (deps, dev/docs extras, ruff/mypy/pytest/coverage/vulture/bandit/radon/xenon/deptry/importlinter/roboco.commits) +├─ Makefile (infra, run, quality gate, per-Python test matrix, docs, lifecycle, foundation-check) +└─ Ops/CI scripts (scripts/) + ├─ build_lifecycle_artifacts.py (render lifecycle artifacts) + ├─ regenerate_verb_tables.py (per-role verb tables from schemas) + ├─ verify_postgres_enums.py (foundation enum parity gate) + ├─ reflow_md.py (markdown prose reflow CI gate) + ├─ reset_runtime_state.sh + .sql (smoke-test state wipe + workspace git reset) + └─ docker/scripts/tests/bash-guard-tests.sh (hook deny/allow table) +``` + +## Dependencies +- Internal: roboco.api.app, roboco.api.deps, roboco.api.websocket, roboco.api.websocket_bridge, roboco.db (bootstrap_database), roboco.events (init_event_bus, register_default_handlers, set_event_context), roboco.runtime (AgentOrchestrator, set_reasoning_stream_callback), roboco.services.notification.NotificationService, roboco.foundation._generators, roboco.foundation.policy.lifecycle.Role, roboco.foundation.identity (Role, Team), roboco.foundation._validate, roboco.foundation.policy.lifecycle, roboco.api.schemas.v1.flow / .do, roboco.services.gateway.role_config.ROLE_CONFIGS, roboco.agent_sdk (intake_main/secretary_main/grok_*_main referenced by Dockerfiles), roboco.llm.providers.grok_cli_config / grok_auth / grok_cli_usage (referenced by grok entrypoint), roboco.agent_sdk.prompt_guard, roboco.agents_config.issue_panel_token (Makefile panel-token) +- External: python>=3.13,<3.15, pydantic / pydantic-settings, fastapi / uvicorn[standard] / websockets / sse-starlette, sqlalchemy[asyncio] / asyncpg / alembic, redis / hiredis, anthropic / openai / tiktoken / claude-agent-sdk, mcp / tomli-w, httpx / python-multipart / python-jose[cryptography] / passlib[bcrypt] / tenacity / structlog, cryptography / packaging / pyyaml / tree-sitter(-python/-typescript), docker (compose, cli, daemon socket mount), nginx:alpine, pgvector/pgvector:pg16, ollama/ollama:latest, curlimages/curl:latest, redis:8-alpine, node:22-alpine (panel), python:3.13-slim-bookworm (orchestrator + agent-base), @anthropic-ai/claude-code, pnpm, Playwright, chromium, xAI grok CLI 0.2.56, uv (astral), ruff, mypy, pytest(-asyncio/-cov/-xdist), vulture, bandit, pip-audit, radon, xenon, deptry, import-linter, mkdocs-material, pymarkdownlnt, make, git, jq + +## Entry Points + +| Name | File | Trigger | +|---|---|---| +| python -m roboco.cli / roboco console script | roboco/cli.py | orchestrator container ENTRYPOINT (docker/orchestrator.Dockerfile:98); `make orchestrator`; `make dev`; `make db-init` (--db-only) | +| roboco-bootstrap console script | roboco/bootstrap.py | pyproject [project.scripts] alias (points at roboco.bootstrap:cli which does not exist — see drift) | +| uvicorn roboco.api.app:app | roboco/bootstrap.py | spawned as asyncio task inside bootstrap.main (make api / make run run it directly) | +| make quality / foundation-check / lifecycle | Makefile | CI merge gate + local pre-submit | +| scripts/build_lifecycle_artifacts.py | scripts/build_lifecycle_artifacts.py | make lifecycle (foundation-check runs it) | +| scripts/verify_postgres_enums.py | scripts/verify_postgres_enums.py | make foundation-check (final step) | +| scripts/reflow_md.py --check | scripts/reflow_md.py | make reflow-check / make quality | +| scripts/regenerate_verb_tables.py | scripts/regenerate_verb_tables.py | manual after role_config / schema change (not wired into make quality) | +| scripts/reset_runtime_state.sh | scripts/reset_runtime_state.sh | manual smoke-test reset (host or ssh into NAS) | +| docker compose up | docker-compose.yaml | operator deploy (build) or -f docker-compose.registry.yml up (pull) | + +## Config Flags +- ROBOCO_DATABASE_HOST/PORT/USER/PASSWORD/NAME +- ROBOCO_REDIS_HOST/PORT/DB/PASSWORD +- ROBOCO_HOST/PORT/API_URL/CORS_ORIGINS/CORS_ALLOW_CREDENTIALS +- ROBOCO_ENCRYPTION_KEY (required by compose) +- ROBOCO_AGENT_AUTH_SECRET / ROBOCO_AGENT_AUTH_REQUIRED +- ROBOCO_ENVIRONMENT (development|staging|production; selects structlog renderer) +- ROBOCO_DEBUG +- ROBOCO_LOCAL_LLM_MODEL / ROBOCO_LOCAL_LLM_BASE_URL / ROBOCO_OLLAMA_BASE_URL / ROBOCO_DEFAULT_EMBEDDING_MODEL / ROBOCO_EMBEDDING_DIMENSIONS +- ROBOCO_RAG_CHUNK_STRATEGY/SIZE/OVERLAP/USE_HYDE/USE_HYBRID_SEARCH/AUTO_UPDATE_* +- ROBOCO_WORKSPACES_ROOT / ROBOCO_WORKSPACE_AUTO_CLONE / ROBOCO_WORKSPACE_CLONE_TIMEOUT / ROBOCO_WORKSPACE_REFRESH_FETCH_TIMEOUT_SECONDS / ROBOCO_WORKSPACE_INSTALL_DEV_DEPS / ROBOCO_WORKSPACE_DEP_INSTALL_TIMEOUT_SECONDS +- ROBOCO_AGENT_IMAGE_REGISTRY / ROBOCO_AGENT_IMAGE_TAG (registry vs local build) +- ROBOCO_HOST_PROJECT_DIR / ROBOCO_HOST_CLAUDE_DIR / ROBOCO_HOST_GROK_DIR / ROBOCO_HOST_DATA_DIR / ROBOCO_PUBLIC_BASE_URL +- ROBOCO_MANIFEST_HOST_DIR +- ROBOCO_CLAIM_STALE_SECONDS / ROBOCO_STALE_CLAIM_REAP_SECONDS / ROBOCO_PM_CLOSURE_RECENTLY_PAUSED_SECONDS / ROBOCO_GROK_IDLE_KILL_SECONDS / ROBOCO_GROK_MAX_COST_USD / ROBOCO_INTERACTIVE_IDLE_REAP_SECONDS / ROBOCO_CLAIMED_NO_AGENT_GRACE_SECONDS / ROBOCO_PM_DECISION_WINDOW_SECONDS +- ROBOCO_SPAWN_COOLDOWN_SECONDS / ROBOCO_ROLE_SPAWN_RATE_PER_MINUTE +- ROBOCO_TOOLCHAIN_MATCH_ENABLED +- ROBOCO_ROUTING_STRICT +- ROBOCO_OVERLOAD_BREAK_ENABLED +- ROBOCO_GATEWAY_HEALTH_ENABLED / ROBOCO_GATEWAY_HEALTH_GRACE_SECONDS +- ROBOCO_CONVENTIONS_ENABLED +- ROBOCO_GUARD_ENABLED / _PASSIVE_MODE / _FAIL_SECURE / _TELEMETRY_ENABLED / _AGENT_API_KEY / _PROJECT_ID / _EMERGENCY / _EMERGENCY_WHITELIST (fastapi-guard HTTP security layer, `roboco/security.py`; local branch `feature/fastapi-guard-hardening`, not on master) +- ROBOCO_RESEARCH_ENABLED / ROBOCO_RESEARCH_PROVIDER / ROBOCO_RESEARCH_API_KEY / ROBOCO_RESEARCH_*_QUOTA +- ROBOCO_PROVISIONING_ENABLED / ROBOCO_PROVISIONING_TOKEN / ROBOCO_PROVISIONING_ORG / ROBOCO_GITHUB_API_BASE_URL +- ROBOCO_STRATEGY_ENGINE_ENABLED / _INTERVAL_SECONDS / _STRANDED_BLOCKED_MINUTES +- ROBOCO_EXTERNAL_PR_ENABLED / _POLL_INTERVAL_SECONDS / _AUTHOR_ALLOWLIST / _REQUIRE_HUMAN_CONFIRM +- ROBOCO_INTERNAL_PR_ENABLED +- ROBOCO_SELF_HEAL_ENABLED / _ORIGINATE_ENABLED / _PROJECT_SLUG / _CI_WORKFLOW / _INTERVAL_SECONDS / _MAX_OPEN_TASKS / _MAX_PER_CYCLE / _NOTIFY_DEDUPE_SECONDS +- ROBOCO_CI_WATCH_ENABLED / _DEFAULT_WORKFLOW / _INTERVAL_SECONDS / _MAX_OPEN_TASKS / _MAX_PER_CYCLE +- ROBOCO_DEP_UPDATE_ENABLED / _INTERVAL_SECONDS / _MAX_OPEN_TASKS / _MAX_PER_CYCLE +- ROBOCO_RELEASE_MANAGER_ENABLED / _MIN_COMMITS / _INTERVAL_SECONDS / _CI_WORKFLOW +- ROBOCO_ORG_MEMORY_ENABLED / _TOP_K / _MIN_SCORE +- ROBOCO_SANDBOX_DB_ENABLED — sandboxed per-agent-spawn engine provisioner (`roboco/runtime/sandbox.py` + registry in `roboco/models/sandbox.py`); a project also needs its `sandbox_services` column set +- ROBOCO_DB_NETWORK_ISOLATED — set true only by the compose topology carrying the `roboco_data` data-only network; suppresses the legacy prod-creds gate-env injection +- ROBOCO_CLOUD_AUTH_ENABLED / _EMAIL / _PASSWORD / _SECRET / _COOKIE_MAX_AGE — FastAPI Users cookie login for the single seeded CEO; `Settings` fails loud at startup if armed with no secret +- ROBOCO_X_ENGINE_ENABLED / _MENTIONS_INTERVAL_SECONDS / _MENTIONS_MAX_PER_CYCLE / _MENTIONS_MIN_ENGAGEMENT / _MAX_OPEN_POSTS / ROBOCO_X_ACCOUNT_USER_ID / _REQUEST_TIMEOUT_SECONDS — the X (Twitter) engine; inert without stored OAuth 1.0a credentials regardless of the flag +- ROBOCO_ROADMAP_ENGINE_ENABLED / _INTERVAL_SECONDS (default 604800) / _MIN_ITEMS_PER_CYCLE / _MAX_ITEMS_PER_CYCLE — the board roadmap engine +- ROBOCO_X_FEATURE_SPOTLIGHT_ENABLED / _INTERVAL_SECONDS (default 259200/3d) — X-engine feature-spotlight sub-switch (requires ROBOCO_X_ENGINE_ENABLED also on), default off +- ROBOCO_OBSIDIAN_VAULT_ENABLED / ROBOCO_VAULT_PATH (default `/data/vault`) — Obsidian vault V1 projection master switch, config-default off but both compose files set it `true`; ROBOCO_VAULT_INTAKE_ENABLED / _INTERVAL_SECONDS / _DIR / _MAX_PER_CYCLE / _MAX_OPEN_DRAFTS — the independently-gated `#roboco`-tag inbox watcher +- ROBOCO_FABLE_MODE_ENABLED — opus-fable-playbook adoption (doctrine layer in the composed prompt + 5 Claude-path hook scripts + 1 grok-path hook), default off; off = byte-for-byte unchanged spawn path +- ROBOCO_MINIO_ENDPOINT / _ACCESS_KEY / _SECRET_KEY / _BUCKET / _REGION — MinIO object storage (default-off; empty endpoint = disabled, media route falls back to `FileResponse`; when set, `video_renderer_client._save` PUTs each render to MinIO after the local write and `GET /api/video/posts/{id}/media` streams it via `StreamingResponse` over `minio_client.get_object_stream`, key = basename, `_require_ceo` kept so auth stays end-to-end — no presigned URLs; `S3Error` falls back to `FileResponse`); NAS compose runs `minio` + `minio-init` on the `data` network with a named `minio-data` volume, registry compose omits MinIO; see `docs/rag/architecture/minio-storage.md` +- ROBOCO_TRANSCRIPT_RETENTION_DAYS / ROBOCO_TRANSCRIPT_PRUNE_ENABLED / _INTERVAL_SECONDS +- ROBOCO_IMAGE_PRUNE_ENABLED / _INTERVAL_SECONDS +- ROBOCO_GIT_COMMAND_TIMEOUT_SECONDS / _COMMIT_TIMEOUT_SECONDS / _NETWORK_TIMEOUT_SECONDS +- ROBOCO_PROTECTED_GIT_URLS +- ROBOCO_AGENT_TOOL_CALL_WARN/HALT / ROBOCO_AGENT_LOOP_THRESHOLD/WINDOW / ROBOCO_AGENT_STOP_ATTEMPT_ALLOWANCE / ROBOCO_AGENT_SLA_* / ROBOCO_CLAUDE_STUCK_KILL_SECONDS +- ROBOCO_QA_NOTES_MIN_CHARS / DOCS / DEV / PR_REVIEWER / QUICK_CONTEXT_MIN_CHARS +- ROBOCO_COMMIT_SUBJECT_MIN_CHARS / COMMIT_BANNED_WORDS +- ROBOCO_LOG_DIR / ROBOCO_DATA_DIR (logging.py + compose volume resolution) +- ROBOCO_REGISTRY / ROBOCO_VERSION (registry compose image selection) +- ROBOCO_PANEL_AGENT_TOKEN / NGINX_ENVSUBST_FILTER (nginx) +- ROBOCO_AGENT_ROLE / ROBOCO_AGENT_ID / ROBOCO_SDK_PORT / ROBOCO_SDK_URL / ROBOCO_INITIAL_PROMPT / ROBOCO_AGENT_MODEL / ROBOCO_GROK_ARGS_FILE / ROBOCO_WORKSPACE / ROBOCO_MCP_CONFIG / ROBOCO_GUARD_SKIP_GIT (hook scripts) +- OLLAMA_API_KEY (compose) + + +## Gotchas +- docker-compose.yml and docker-compose.yaml are byte-identical duplicates maintained in lockstep — the registry compose comment explicitly warns to mirror service changes across both. Drift between them is a silent footgun. +- pyproject.toml requires-python is >=3.13,<3.15 but the Makefile test matrix and PYTHON_VERSIONS span 3.10–3.14 and DEFAULT_PYTHON=3.10; `make test-3.10` will build an image the project cannot actually install into (ruff target-version py313, mypy python_version=3.13). The matrix is stale relative to the floor. +- orchestrator.Dockerfile pins the builder to python:3.13-slim-bookworm and forces UV_PYTHON_PREFERENCE=only-system so the venv symlinks survive the COPY to the runner; switching the base image Python (e.g. to 3.14) without re-validating the uv-managed-python path would brick the runner. +- The orchestrator container runs as root (no USER directive) so it can chown cloned workspaces to uid 1000 and set git safe.directory '*'; agent-base runs as `agent` (created with useradd -m). Crossing the uid boundary (orchestrator root-owned repo in an agent container) is what safe.directory '*' papers over — removing it re-introduces 'dubious ownership'. +- ollama-init deliberately runs WITHOUT `set -e` and gates success on model PRESENCE (/api/tags grep), not on the pull succeeding — a degraded registry used to block the orchestrator's service_completed_successfully gate. Any 'fix' that adds set -e or makes the pull mandatory re-breaks cold caches. +- sdk-startup-hook sets UV_PROJECT_ENVIRONMENT=/app/.venv AND passes --no-sync because bare `uv run` from a workspace cwd re-syncs the image venv against the clone's drifted lock (multi-minute stall). The generated mcp-config.json must keep both in sync — changing one without the other re-stalls the SDK bring-up. +- bash-guard-hook denies `uv run --active` and any `uv run`/`uvx` targeting /app — VIRTUAL_ENV is no longer baked globally in the agent image (removed in 7be10057; was the be-dev-1 2026-06-29 root cause). With VIRTUAL_ENV absent, --active finds no active env and errors rather than retargeting /app/.venv; an explicit /app target still bricks the gateway venv. A bare `uv run` (workspace venv) is the only allowed form. +- grok-cli-agent-entrypoint symlinks ~/.grok/auth.json -> /home/agent/.grok-auth-ro/auth.json because a single-file bind mount pins the inode and the atomic auth.json refresh never reached a running container; the directory RO mount sees the host rename. Breaking the symlink (or a stale baked stub auth.json) makes grok hang at an interactive login prompt forever. +- grok exit codes are load-bearing: 78 = expired/missing auth (EX_CONFIG, refuse to start), 75 = 429/quota (EX_TEMPFAIL, orchestrator parks the provider). The orchestrator's _handle_stopped_container branches on these; reusing them for other semantics would misroute. +- logging._resolve_log_dir prefers /data/logs only if it OR its parent exists; on a host run with no /data it falls back to ./data/logs. A host-side `./logs` at repo root is NOT the same directory and would duplicate logs. +- verify_postgres_enums.py now embeds skip semantics (exit 0 on unreachable OR both-enum-types-absent) so the Makefile must NOT pipe it through `|| echo skipped` — that masks a real drift exit(1) as a skip. The baseline did exactly that; 15effce0 removed the `|| echo`. +- reset_runtime_state.sql DELETEs in a hand-ordered FK-safe array and guards each with information_schema EXISTS — a new runtime table referencing tasks/journals must be added to drop_order or the DELETE will FK-violate. A partial schema (some a2a_* absent) is tolerated by the IF EXISTS guard. +- reset_runtime_state.sh deletes every local branch except the resolved default (main/master/fallback) per workspace; if a workspace's default can't be resolved (no origin, no main, no master) it SKIPs rather than nukes — safe, but leaves stale branches that can collide with next-run branch creation. +- TaskLifecycleError._TRANSITION_HINTS is a hand-maintained (current_status,target_status)->hint map; adding a new lifecycle state/transition without a matching hint means weak models fall back to guessing the tool sequence (the exact failure the hints exist to prevent). +- ROBOCO_CLAIM_STALE_SECONDS and ROBOCO_STALE_CLAIM_REAP_SECONDS are intentionally distinct fields — claim_stale_seconds drives trigger_filter spawn queueing, stale_claim_reap_seconds drives the reaper's release. Splitting them opens a duplicate-spawn window; merging them delays spawn decisions. NAS compose raises both to 1800. +- pyproject [project.scripts] declares `roboco-bootstrap = roboco.bootstrap:cli` but roboco/bootstrap.py defines NO `cli` symbol (only `main`); the canonical entry is `roboco = roboco.cli:cli`. The bootstrap-script entry is dead/broken (see drift). +- The Dockerfiles COPY pyproject.toml uv.lock README.md into /app for the uv sync layer; a missing/stale uv.lock at build time breaks the --frozen sync. The Makefile's `make upgrade` re-locks but does not rebuild images. +- Both NAS composes (`docker-compose.yml`/`.yaml`) set `ROBOCO_GUARD_ENABLED=true` / `ROBOCO_GUARD_PASSIVE_MODE=true` / `ROBOCO_GUARD_FAIL_SECURE=false` — the fastapi-guard HTTP security layer runs in detect-and-log calibration mode on the NAS, never blocking; `docker-compose.registry.yml` does not set these three and stays off (`guard_enabled` default `false`). Flipping `_PASSIVE_MODE` to `false` to enforce is a deliberate later step, not part of this arming. +- Both NAS composes now also arm `ROBOCO_SANDBOX_DB_ENABLED` / `ROBOCO_DB_NETWORK_ISOLATED` / `ROBOCO_CLOUD_AUTH_ENABLED` / `ROBOCO_X_ENGINE_ENABLED` / `ROBOCO_ROADMAP_ENGINE_ENABLED` all `${VAR:-true}` (config default is `false` for every one), alongside the pre-existing `${VAR:-true}` flips for `ROBOCO_SELF_HEAL_ENABLED`, `ROBOCO_PROVISIONING_ENABLED`, `ROBOCO_TRANSCRIPT_PRUNE_ENABLED`, and `ROBOCO_ROUTING_STRICT` — this is a personal-deploy posture (override any via `.env`), not the published conservative default. `docker-compose.registry.yml` keeps `ROBOCO_SELF_HEAL_ENABLED:-false` (and does not set `SANDBOX_DB`/`CLOUD_AUTH`/`X_ENGINE`/`ROADMAP_ENGINE` at all, so they fall through to the config `False` default) — it arms only `ROBOCO_DB_NETWORK_ISOLATED:-true`, with its own `roboco_data` network in the `networks:` stanza, so the registry compose ships DB-isolated but otherwise conservative. +- Surface N (scanner honeytrap) is two-layered because nginx only proxies `/api|/ws|/health|/ready` to the orchestrator: `docker/nginx.conf` has a `location ~*` block that `return 444`s the classic root scanner paths (`/.env`, `/.git`, `/wp-login.php`, `/phpmyadmin`, `actuator`, `cgi-bin`, `vendor/`, …) at the edge before they reach the panel (anchored to scanner fingerprints; `/.well-known` + real routes untouched; always on), while `roboco/security.py`'s `_THREAT_BAN_CONFIG` gained `recon`/`sensitive_file`/`cms_probing` so `/api`-path probes that DO reach guard trip an adaptive per-IP redis auto-ban (active mode only; passive logs). +- `.github/workflows/release.yml` must build every image `docker-compose.registry.yml` pulls: `roboco-agent-base` and `roboco-agent-grok` build first (outside the loop — the Grok-family images and two interactive Grok roles `FROM` them respectively, so they must exist in the daemon before the loop runs), then a 15-entry `IMAGES` associative array covers the rest — 17 images total. The two Grok sub-images (`roboco-agent-grok-prompter` / `roboco-agent-grok-secretary`) were previously missing from the loop and were never published, so a fresh registry-compose pull 404'd on them; adding an image anywhere in the compose files without a matching `IMAGES` entry (or the two pre-loop builds) silently re-breaks a cold registry pull. + + +## Drift from CLAUDE.md +- CLAUDE.md lists `agent-base-image / agent-*-image` as 'Pre-built images spawned per agent' under the services table, but the build compose defines them as one-shot build-and-exit builder services (entrypoint echo) that merely tag the image; the orchestrator spawns containers FROM those images later. The 'Pre-built images spawned per agent' phrasing conflates the build step with the spawn step. +- CLAUDE.md's Docker compose services table omits the panel and nginx services that are present in both compose files (it only lists postgres/redis/ollama/ollama-init/agent images/orchestrator). +- CLAUDE.md states the orchestrator container 'Depends on all above' and the startup sequence shows `orchestrator -> panel -> nginx`, but compose has panel depend_on orchestrator and nginx depend_on panel+orchestrator — the panel/orchestrator build order is reversed in the prose vs the compose depends_on graph (orchestrator does NOT depend on panel/nginx). +- pyproject.toml [project.scripts] declares `roboco-bootstrap = "roboco.bootstrap:cli"` but roboco/bootstrap.py has no `cli` callable (only `main` and `__main__` that imports roboco.cli.cli). The documented console script would fail to resolve; only `roboco = "roboco.cli:cli"` is valid. +- CLAUDE.md (Configuration section) still documents `ROBOCO_LOCAL_LLM_MODEL=glm-5:cloud` while config.py and all three compose files now default to `glm-5.2:cloud` (changed in 15effce0). The doc and code are out of sync on the model name. +- CLAUDE.md lists the agent image family but not the grok-prompter / grok-secretary images or the pr-reviewer image that exist in compose; the documented image set is incomplete vs the 14 builder services actually declared. +- CLAUDE.md says healthcheck for the orchestrator is not listed (table shows '—' for agent images and panel), which matches compose (no healthcheck on orchestrator/panel/nginx), but the startup-sequence prose implies an orchestrator healthcheck-driven dependency that does not exist — depends_on uses postgres/redis/ollama/ollama-init/agent-base-image conditions, not the orchestrator's own /health (that is polled in-process by bootstrap._wait_for_api_ready). ## Changes Since Baseline | SHA | Subject | Impact | |---|---|---| -| e202ce39 | Make main_pm + task_type=code impossible | In this slice: prose-only — added _next_hint_pr_fail (branch-aware re-delegate hint for Main-PM roots vs dev-revise for cell/dev) and rewrote submit_root description to 'branch-keyed not task_type-keyed'. The actual main_pm+code impossibility is enforced elsewhere (choreographer), not via ActionSpec.allowed_task_types here. | -| 250be5c2 | sync_branch dev verb — gate-level branch rebase (Phase B1) | Added IntentSpec sync_branch (dev-only, composes=(), extra_preconditions=OWNERSHIP, no DB transition) + _next_hint_synced helper. New empty-composes verb with no source-status gate; relies entirely on the handler's no-branch/protected-base guards. | -| 2f322286 | [F043] guard escalate_up against resurrecting terminal tasks | Added PRECONDITION_NON_TERMINAL (invalid_state) to escalate_up.extra_preconditions; generalized _check_intent_preconditions to honor any non-tracing rejection_kind (previously only not_authorized was special-cased, everything else was tracing_gap). Now invalid_state preconditions produce Decision.reject instead of tracing_gap. | -| c34e978f | [F101] enforce PR-open state gate on gateway open_pr | Added PR_OPEN_STATES frozenset (in_progress/verifying/awaiting_qa/awaiting_documentation/needs_revision) + PRECONDITION_PR_OPEN_STATE (invalid_state) inserted into open_pr.extra_preconditions between OWNERSHIP and COMMITS. Closes the gateway parity gap: open_pr (composes=()) previously had no source-status gate; now rejects claim/paused/blocked/terminal. | +| 15effce0 | Chore: 141 Gaps fill-in (#283) — squash of megatask per-cell project map + multi-fix | Bumped version 0.13.0 -> 0.14.0 in pyproject.toml, roboco/__init__.py, config.app_version, and the agent_image_tag doc example; renamed the local LLM model glm-5:cloud -> glm-5.2:cloud in config.py default and in all three compose files' ollama-init pull/verify + ROBOCO_LOCAL_LLM_MODEL env; rewrote verify_postgres_enums.py to embed skip semantics (exit 0 on unreachable/unmigrated, 1 on drift) with new type_exists/should_skip_for_unmigrated/enum_drift helpers and removed the Makefile's masking `// echo skipped`; added two bash-guard-hook deny rules for `uv run --active` and any uv run/uvx targeting /app/.venv (with matching bash-guard-tests cases) to prevent the be-dev-1 venv-brick; updated grok-cli-agent-entrypoint.sh to symlink ~/.grok/auth.json from a read-only host directory mount (F005) so the atomic auth refresh reaches running containers. | -> Post-snapshot updates (since 2026-06-29): 15effce0 + 536bbb64 (141-gap + logical-gap sweeps) added SYNC_BRANCH_STATES + PRECONDITION_SYNC_BRANCH_STATE to sync_branch.extra_preconditions (closes medium-risk source-status gap), PRECONDITION_ROOT_NOT_CODE to submit_root.extra_preconditions (closes low-risk spec/prose gap), and ceo_reject_to_pool ActionSpec (AWAITING_CEO_APPROVAL→PENDING CEO reject-to-pool path). 16b71be8 ([sweep] lifecycle: 6 gaps) fixed cancel CEO gate, claim_pr_review gate, needs_team_match enforcement, valid_next_verbs narrowing, pr_reviewer unclaim (unclaim.allowed_roles now includes Role.PR_REVIEWER), and complete side_effect ordering. ef33d56c ([chore] lifecycle-enforcement validators + status-class) dropped the spurious VERIFYING→awaiting_documentation legacy edge from _LEGACY_OPERATIONAL_EDGES, added awaiting_pr_review to is_waiting_state, and overhauled _check_status_enum_coverage from a tautology to a real bidirectional check + split _check_terminal_exits into COMPLETED-path + cancel-exit check + added new _check_status_enum_parity validator (spec.Status vs models.base.TaskStatus ORM parity). a2d2ef47 ([sweep] enforcement status-class partition) added backlog + pending to is_waiting_state completing the terminal/active/waiting partition + added the coverage-invariant test. b3558d4e ([chore] complexity) extracted _check_team_match helper from can_invoke_action (no behavior change). +> Post-snapshot updates (since 2026-06-29): 7be10057 `[bug] agent image: stop baking VIRTUAL_ENV=/app/.venv` — removed `VIRTUAL_ENV=/app/.venv` from the global ENV in agent-base.Dockerfile; updated bash-guard-hook.sh comment/deny message to reflect that `--active` now errors (no active env) rather than retargeting /app/.venv. 536bbb64 `Chore/all/logical gaps sweep (#286)` — added `routing_strict` (ROBOCO_ROUTING_STRICT), `self_heal_notify_dedupe_seconds` (ROBOCO_SELF_HEAL_NOTIFY_DEDUPE_SECONDS), and `claude_stuck_kill_seconds` (ROBOCO_CLAUDE_STUCK_KILL_SECONDS) to config.py; minor type-annotation strip fix in scripts/regenerate_verb_tables.py. 2759edf7 `[B-REL] release executor` — added `release_ci_workflow` (ROBOCO_RELEASE_CI_WORKFLOW) to config.py, decoupled from self_heal_ci_workflow. + +> **Local branch (not on master, NOT deployed):** `feature/fastapi-guard-hardening` landed `ROBOCO_GUARD_ENABLED` / `_PASSIVE_MODE` / `_FAIL_SECURE` / `_TELEMETRY_ENABLED` / `_AGENT_API_KEY` / `_PROJECT_ID` / `_EMERGENCY` / `_EMERGENCY_WHITELIST` in `config.py` (6 commits, `896532a3`..`99ee666e`) and set both NAS composes' `ROBOCO_GUARD_ENABLED=true` / `ROBOCO_GUARD_PASSIVE_MODE=true` / `ROBOCO_GUARD_FAIL_SECURE=false` (`c496b677`, Phase 5); `docker-compose.registry.yml` is untouched and stays off. See api-core-websocket for the `roboco/security.py` module + `create_app` wiring detail. + +> **v0.18.0** (2026-07-04): Fable mode — `agent-base.Dockerfile` now `COPY`s 5 vendored `docker/scripts/fable-*.sh` hook scripts (stop-gate, bash-discipline, honesty-nudge, prompt-nudge, precompact), installed at spawn time only when `ROBOCO_FABLE_MODE_ENABLED` is set (default off; `session-start.sh` from the upstream `opus-fable-playbook` was deliberately not ported — redundant with the doctrine layer). X feature-spotlight adds `ROBOCO_X_FEATURE_SPOTLIGHT_ENABLED`/`_INTERVAL_SECONDS` to `config.py` as a sub-switch of the existing X-engine flag. ## Regression Risks | Title | File:Line | Claim | Severity | |---|---|---|---| -| ~~sync_branch has no source-status gate — can be called on terminal/paused/blocked tasks~~ **RESOLVED** | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:1225 | FIXED post-snapshot (commits 536bbb64/15effce0): PRECONDITION_SYNC_BRANCH_STATE (rejection_kind=invalid_state) added to sync_branch.extra_preconditions; gates the verb to SYNC_BRANCH_STATES={claimed,in_progress,verifying,needs_revision}. Terminal, paused, and blocked tasks now receive Decision.reject(invalid_state). | medium | -| open_pr now rejects `claimed` — may break dev flows that open PR before start | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:963 | PR_OPEN_STATES excludes CLAIMED. The intent composition i_will_work_on (claim,set_plan,start) transitions claim->in_progress atomically, so the normal path is fine, but any path that opens a PR while still in CLAIMED (e.g. a dev who calls open_pr before i_will_work_on finishes, or a custom flow) now receives invalid_state instead of the previous silent accept. This is the intended F101 parity fix, but it is a behavior tightening — verify no caller relied on claim-state PR opens. | low | -| _check_intent_preconditions generalization may misroute existing not_authorized callers | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:1572 | Before F043, only not_authorized was special-cased and the message was first_missing.remediate. After generalization, ALL non-tracing rejection kinds (not_authorized, invalid_state) take the Decision.reject path with message=remediate. Behavior for not_authorized is unchanged (still remediate-as-message), but any caller that introspected rejection_kind=='tracing_gap' to mean 'a precondition failed' will now miss invalid_state/ownership failures. The envelope's missing[] field is also empty for these. Low-medium risk for consumers that branch on rejection_kind. | low | -| pr_fail hint now branches on team==MAIN_PM and branch_name — non-Main-PM branch-bearing tasks get dev-revise | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:789 | _next_hint_pr_fail returns the re-delegate hint only when team==MAIN_PM.value AND branch is set. A cell-PM coordination root with a branch (rare but possible) falls through to 'dev will revise', which is wrong for a PM-owned assembled PR. Low impact (hint text only, not a gate) but could mislead a cell PM into waiting instead of re-delegating. | low | -| ~~submit_root description claims 'branch-keyed, not task_type-keyed' but no allowed_task_types gate enforces it~~ **RESOLVED** | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:1502 | FIXED post-snapshot (commits 536bbb64/15effce0): PRECONDITION_ROOT_NOT_CODE (invalid_state, key="root_not_code") added to submit_root.extra_preconditions; rejects when task.task_type==CODE. The spec now enforces the constraint the prose described. Defense-in-depth vs the creation-path choreographer guard. | low | -| CLAIM_RULES lets PMs claim NEEDS_REVISION — combined with pr_fail reassign can create a PM/dev tug-of-war | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:687 | CELL_PM/MAIN_PM claim rules include NEEDS_REVISION (added so PM-owned coordination tasks have an actor after pr_fail/ceo_reject). give_me_work only offers assigned tasks, so normally safe, but a PM re-claiming a NEEDS_REVISION dev leaf (if misassigned) would bypass the dev-revise path. The comment documents this is scoped by the same mechanism as dev leaf-revision; risk is in assignment correctness upstream, not the spec itself. Low risk. | low | +| LLM model rename breaks cached ollama deployments / stale env | docker-compose.yaml:86 | The ollama-init verify now greps for `glm-5.2` exactly. A NAS volume that only has the old `glm-5:cloud` model cached (no network, or a slow registry) will hit the FATAL exit and block the orchestrator's service_completed_successfully gate — whereas the old `glm-5` grep would have passed. Operators with a pre-existing cached `glm-5:cloud` and no pull will fail to boot until the new model is pulled. | high | +| verify_postgres_enums skip-on-unmigrated masks a real enum drift on a partially-migrated DB | scripts/verify_postgres_enums.py:49 | should_skip_for_unmigrated returns True only when BOTH agentrole and team are absent. A partial schema where one enum type exists and the other does not correctly falls through to enum_drift (exit 1). But if a future migration drops one type temporarily, a DB that previously had both now reports drift instead of skipping — the gate could fail CI on a transitional schema. Low likelihood but the behavior change (baseline exited 1 on any connection failure; now exits 0) shifts the failure mode from 'hard fail' to 'skip' for unreachable postgres, which a CI without a migrated DB used to surface as a real signal. | medium | +| bash-guard `uv run --active` deny may false-positive on legitimate workspace commands | docker/scripts/bash-guard-hook.sh:341 | The rule denies any `uv run` containing the literal `--active` token anywhere in the command. A command like `uv run --active pytest` is denied even if the agent's intent was a workspace venv (VIRTUAL_ENV is no longer image-baked — 7be10057 — so `--active` simply errors, but the deny still fires first). Conversely the second rule's regex for /app targets (cd /app, --project /app, --directory /app, UV_PROJECT_ENVIRONMENT=/app) could match a benign command that mentions /app in an unrelated argument (e.g. a path under /app/data). The deny is a hard exit 2 with no override. | medium | +| grok auth.json symlink assumes the host directory mount path | docker/scripts/grok-cli-agent-entrypoint.sh:44 | The entrypoint unconditionally `rm -f /home/agent/.grok/auth.json` then symlinks to /home/agent/.grok-auth-ro/auth.json. If the orchestrator's mount layout changes (e.g. the RO dir is not mounted at .grok-auth-ro, or a future image bakes a real auth.json that should not be replaced), the symlink breaks and grok_auth --check exits 78 on every spawn. The rm -f also removes any pre-baked stub with no fallback. | medium | +| Makefile foundation-check no longer tolerates verify_postgres_enums failure | Makefile:540 | The `// echo skipped` was removed, so any non-zero exit from verify_postgres_enums now fails the gate. Correct by design (drift must fail), but if the script's asyncpg connect raises an exception type not in (OSError, asyncpg.PostgresError) — e.g. a permissions error classified differently — the unhandled exception exits non-zero and fails CI where the baseline would have masked it as a skip. | low | +| Version 0.14.0 bump without a corresponding release tag / image build | pyproject.toml:3 | pyproject, __init__.py, and config.app_version all say 0.14.0 but the branch feature/metrics-granularity is NOT deployed and memory notes say metrics-granularity was stopped mid-Phase-1. If a registry image is built from this tree it will be tagged 0.14.0 while the deployed NAS is on 0.14.0-memory-but-actually-0.13-ish. The agent_image_tag doc example ('0.14.0') could mislead an operator into pulling an unbuilt tag. | low | ## Health -The slice is structurally healthy and well-fenced: the spec is pure data + lookups with no I/O, frozen dataclasses enforce Decision invariants, 14 import-time validators (including the new Status/TaskStatus ORM-parity check) make a misconfigured spec fail fast at container start, and the enforcement shim clearly fences its _LEGACY_* additions (UNMIGRATED debt tracked, union-merge prevents the pr_reviewer-drop regression). Post-snapshot commits resolved the two open medium/low risks: sync_branch now has a source-status gate (PRECONDITION_SYNC_BRANCH_STATE) and submit_root now spec-enforces the not-code constraint (PRECONDITION_ROOT_NOT_CODE). The remaining integrity concerns are behavioral: (1) the empty-composes verb pattern (unclaim, reassign, escalate_up, give_me_work, i_am_idle, triage) still has NO source-status gate at the spec layer — each new empty-composes verb that touches state must add its own NON_TERMINAL/source-status precondition; (2) can_invoke_intent only pre-checks the FIRST composed action, leaving mid-composition rollback to the runner; (3) is_terminal_state/is_waiting_state/is_active_state are hard-coded string sets not derived from STATUS_GRAPH, though a build-time coverage test now catches any new Status member that falls through the partition. No active logic bugs found; the slice is the canonical source its consumers trust. +This slice is the load-bearing deployment surface and it is internally coherent: the compose topologies are kept in lockstep (with an explicit reminder), the Dockerfiles form a clean FROM-chain (agent-base -> role images -> grok variants), config.py is the single env-backed source of truth that bootstrap.py wires through, and the Makefile gate is comprehensive and deterministic. The main integrity concerns are (1) the byte-identical docker-compose.yml/.yaml pair that must be edited together or they silently drift, (2) the stale `roboco-bootstrap = roboco.bootstrap:cli` console script that points at a non-existent symbol, (3) the Makefile test matrix advertising Python 3.10-3.14 while pyproject requires >=3.13 (the 3.10/3.11/3.12 targets will fail to build), (4) the LLM model rename glm-5:cloud -> glm-5.2:cloud that will block boot on a NAS with only the old cached model, and (5) the CLAUDE.md configuration doc still referencing the old model name. The recent 15effce0 changes are well-scoped (version bump, model rename, enum-gate semantics fix, bash-guard /app venv protection, grok auth symlink) and each ships with matching tests or clear failure semantics, but the model-rename boot risk on cached deployments and the dead console-script entry are the items most likely to bite an operator. No state-machine holes or concurrency regressions were introduced in this slice by the baseline diff. + +## Purpose +Three default-off background "engine" services that watch CI / dependencies and originate a single PENDING fix task into the normal delivery lifecycle, then stop. SelfHealEngine watches RoboCo's OWN repo CI and (behind a second opt-in) opens a CEO-held fix task; CiWatchEngine fans that out to every opted-in project; DepUpdateEngine probes whether a dependency upgrade would change lockfiles and opens an "update dependencies" task. All three are detect+originate only — none ever start, approve, merge, or deploy; they flush writes and the orchestrator loop owns the commit. + +## Files + +| Path | Role | LOC | +|---|---|---| +| /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/self_heal_engine.py | Single-repo self-heal: detect a regression in RoboCo's own CI via telemetry, notify CEO, optionally open a HELD PENDING fix task | 310 | +| /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/ci_watch_engine.py | Multi-repo CI-watch: for each opted-in project whose CI is red, open one READY-to-start PENDING fix task (deduped per git_url) and notify the cell PM | 190 | +| /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/dep_update_engine.py | Dependency-update bot: probe each opted-in project's lockfile for changes and open one READY-to-start PENDING update task per repo | 138 | +| /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/runtime/orchestrator.py | Owns the three background loops (_self_heal_loop, _ci_watch_loop, _dep_update_loop) that construct the engines, call run_cycle, and commit the session | 0 | +| /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/task.py | Provides SELF_HEAL/CI_WATCH/DEP_UPDATE source tags, list_open_*_tasks dedupe queries, extract_self_heal_fingerprint, and the give_me_work self-heal hold filter | 0 | +| /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/telemetry/source.py | TelemetrySource protocol + GitHubCITelemetrySource (single repo) + MultiProjectCITelemetrySource (fan-out) feeding breach samples to the engines | 0 | + +## Key Symbols + +| Name | Kind | File:Line | Responsibility | +|---|---|---|---| +| RegressionObservation | dataclass | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/self_heal_engine.py:54 | Frozen record of one detected regression: fingerprint, signal/repo names, summary/detail/raw_ref | +| _fingerprint | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/self_heal_engine.py:65 | Stable 16-char sha256 prefix of the signal name — the dedupe key for open self-heal fix tasks | +| _NOTIFY_DEDUPE_KEY_PREFIX | constant | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/self_heal_engine.py:70 | Module-level Redis key prefix for per-fingerprint CEO-notify dedupe ("self_heal:notified:") | +| SelfHealEngine | class | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/self_heal_engine.py:73 | Detect regressions in RoboCo's own repo, notify CEO, optionally originate a HELD fix task | +| SelfHealEngine.assess | method | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/self_heal_engine.py:84 | Read telemetry samples, return RegressionObservations for breaches; pure, no side effects | +| SelfHealEngine.run_cycle | method | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/self_heal_engine.py:103 | Gate on self_heal_enabled, assess, notify CEO per obs (deduped per fingerprint via Redis), optionally originate; returns observations; flushes, caller commits | +| SelfHealEngine._open_self_heal_task_ids_by_fp | method | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/self_heal_engine.py:145 | Map each open self-heal task's fingerprint to its task id; best-effort (returns {} on DB error) — used to link CEO alert to fix task and corroborate notify dedupe | +| SelfHealEngine._already_notified | method | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/self_heal_engine.py:166 | Fail-open Redis check: True when the fingerprint was CEO-notified this episode (a Redis outage returns False so the notify fires anyway) | +| SelfHealEngine._mark_notified | method | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/self_heal_engine.py:184 | Record that this fingerprint was CEO-notified; sets a Redis key with self_heal_notify_dedupe_seconds TTL; best-effort (failure swallowed) | +| SelfHealEngine._dedupe_key | staticmethod | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/self_heal_engine.py:207 | Build the Redis key _NOTIFY_DEDUPE_KEY_PREFIX + fingerprint | +| SelfHealEngine._originate | method | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/self_heal_engine.py:210 | Open one PENDING HELD (confirmed_by_human=False) fix task per NEW regression, bounded by per-cycle/rolling caps + fingerprint dedupe | +| get_self_heal_engine | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/self_heal_engine.py:306 | Factory: construct SelfHealEngine bound to a session with optional test source | +| _cell_pm_slug_for | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/ci_watch_engine.py:45 | Resolve the cell-PM agent slug owning a team (e.g. Team.BACKEND -> 'be-pm'), or None | +| CiWatchEngine | class | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/ci_watch_engine.py:53 | Open a fix task per opted-in project whose CI is red; never merges | +| CiWatchEngine.run_cycle | method | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/ci_watch_engine.py:62 | Gate on ci_watch_enabled, fetch breaches for the watch set, originate fix tasks; returns opened tasks; flushes, caller commits | +| CiWatchEngine._originate | method | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/ci_watch_engine.py:78 | Open one ci_watch fix task per NEW red repo bounded by caps; notify the cell PM best-effort per opened task | +| CiWatchEngine._notify_cell_pm | method | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/ci_watch_engine.py:108 | Best-effort ack notification to the red project's cell PM; failure never rolls back origination | +| CiWatchEngine._should_open | method | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/ci_watch_engine.py:136 | True when project resolves and has no open ci_watch task for its git_url (monorepo dedupe) | +| CiWatchEngine._open_fix_task | method | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/ci_watch_engine.py:160 | Create the PENDING READY-to-start (confirmed_by_human=True) Main-PM coordination root fix task | +| get_ci_watch_engine | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/ci_watch_engine.py:198 | Factory: construct CiWatchEngine bound to a session with optional test source | +| DepUpdateEngine | class | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/dep_update_engine.py:39 | Open an update-dependencies task per opted-in project with lockfile changes available | +| DepUpdateEngine.run_cycle | method | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/dep_update_engine.py:48 | Gate on dep_update_enabled, probe each project, open tasks bounded by caps; returns opened tasks; flushes, caller commits | +| DepUpdateEngine._eligible | method | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/dep_update_engine.py:81 | Cheap checks (command set, id present, per-git_url dedupe) then expensive read-only lockfile probe; returns eligibility bool | +| DepUpdateEngine._open_task | method | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/dep_update_engine.py:96 | Create the PENDING READY-to-start (confirmed_by_human=True) Main-PM coordination root dep-update task | +| get_dep_update_engine | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/dep_update_engine.py:133 | Factory: construct DepUpdateEngine bound to a session with optional test workspace probe | + +## Data Flow +Each engine is constructed per-cycle by its orchestrator loop inside a `get_db_context()` session. SelfHealEngine pulls breach samples from a TelemetrySource (GitHubCITelemetrySource for RoboCo's own repo); CiWatchEngine's MultiProjectCITelemetrySource.fetch(projects) takes the watch set the orchestrator loaded; DepUpdateEngine does NOT use telemetry — it calls WorkspaceService.dry_upgrade_changes_lockfile(project) (read-only probe in a throwaway clone). On a breach, each engine calls TaskService.list_open_*_tasks (optionally scoped by git_url for ci_watch/dep_update) to dedupe, checks per-cycle + rolling open-task caps against settings, resolves the target project (self_heal resolves by slug via ProjectService.get_by_slug; ci_watch/dep_update already have the project row), then calls TaskService.create(TaskCreateRequest) with the matching source tag (SELF_HEAL_SOURCE / CI_WATCH_SOURCE / DEP_UPDATE_SOURCE), team=MAIN_PM, assigned_to the main-pm agent UUID, status=PENDING. The self-heal task is HELD (confirmed_by_human=False) and carries a fingerprint via markers.set_self_heal_fingerprint so later cycles see it as already-open; ci_watch and dep_update tasks are READY (confirmed_by_human=True). Self-heal always notifies the CEO via NotificationService.send_ack_notification; ci_watch notifies the red project's cell PM best-effort; dep_update notifies no one. Each engine only flushes; the orchestrator loop commits. The created tasks then ride the normal delivery lifecycle — give_me_work offers ci_watch/dep_update tasks immediately, but excludes source=self_heal + confirmed_by_human=False tasks until the CEO's approve_and_start flips the flag (task.py line ~7304). + +## Mermaid +```mermaid +graph TD + subgraph Orchestrator loops + SHL[_self_heal_loop] --> SH[SelfHealEngine.run_cycle] + CWL[_ci_watch_loop] --> CWR[_run_ci_watch_cycle] --> CW[CiWatchEngine.run_cycle] + DUL[_dep_update_loop] --> DUR[_run_dep_update_cycle] --> DU[DepUpdateEngine.run_cycle] + end + TS[TelemetrySource / MultiProjectCITelemetrySource] -->|breach samples| SH + TS -->|breach samples| CW + WS[WorkspaceService.dry_upgrade_changes_lockfile] -->|lockfile changed?| DU + SH -->|notify| CEO[CEO ack notif] + CW -->|notify best-effort| CELLPM[Cell PM ack notif] + SH --> TSVC[TaskService.create source=self_heal HELD] + CW --> TSVC2[TaskService.create source=ci_watch READY] + DU --> TSVC3[TaskService.create source=dep_update READY] + TSVC --> DB[(TaskTable)] + TSVC2 --> DB + TSVC3 --> DB + DB -->|list_open_*_tasks dedupe + caps| SH + DB -->|list_open_*_tasks dedupe + caps| CW + DB -->|list_open_*_tasks dedupe + caps| DU + DB -->|give_me_work hold filter| GMW[give_me_work: self_heal+unconfirmed excluded] + GMW --> CEO_APPROVE[CEO approve_and_start flips confirmed_by_human] + CEO_APPROVE --> NORMAL[Normal delivery lifecycle: dev->QA->PR review->CEO merge] + CW_TASK[ci_watch/dep_update PENDING task] --> NORMAL + settings_self_heal[self_heal_enabled + _originate_enabled] -.gate.-> SH + settings_cw[ci_watch_enabled] -.gate.-> CW + settings_du[dep_update_enabled] -.gate.-> DU +``` + +## Logical Tree +``` +engines-heal-ciwatch-depupdate + SelfHealEngine (roboco/services/self_heal_engine.py) + RegressionObservation (frozen dataclass: fingerprint, signal_name, repo_hint, summary, detail, raw_ref) + _fingerprint(signal_name) -> 16-char sha256 prefix + _NOTIFY_DEDUPE_KEY_PREFIX = "self_heal:notified:" + __init__(session, source=None) -> binds TelemetrySource + assess() -> [RegressionObservation] for breaches (pure) + run_cycle() -> gates on self_heal_enabled; assess; notify CEO deduped per fingerprint via Redis; optionally _originate + _open_self_heal_task_ids_by_fp() -> {fingerprint: task_id} for open self-heal tasks; best-effort + _already_notified(fingerprint) -> bool; fail-open Redis check + _mark_notified(fingerprint) -> set Redis key with notify_dedupe_seconds TTL; best-effort + _dedupe_key(fingerprint) -> Redis key string + _originate(observations) -> dedupe by fingerprint + caps; create HELD PENDING task; set_self_heal_fingerprint + get_self_heal_engine(session, source=None) + CiWatchEngine (roboco/services/ci_watch_engine.py) + _cell_pm_slug_for(team) -> cell PM slug + __init__(session, source=None) -> binds MultiProjectCITelemetrySource + run_cycle(projects) -> gates on ci_watch_enabled; fetch breaches; _originate + _originate(breaches, by_slug) -> per-repo dedupe + caps; _open_fix_task + _notify_cell_pm + _notify_cell_pm(project, sample) -> best-effort ack to cell PM + _should_open(task_svc, project) -> project resolves + no open task for git_url + _open_fix_task(task_svc, project, sample) -> create READY PENDING Main-PM root + get_ci_watch_engine(session, source=None) + DepUpdateEngine (roboco/services/dep_update_engine.py) + __init__(session, workspace=None) -> binds WorkspaceService + run_cycle(projects) -> gates on dep_update_enabled; per-project probe; _open_task + _eligible(task_svc, project) -> command set + id + git_url dedupe + dry_upgrade_changes_lockfile + _open_task(task_svc, project) -> create READY PENDING Main-PM root + get_dep_update_engine(session, workspace=None) + Orchestrator loops (roboco/runtime/orchestrator.py) + _self_heal_loop -> get_self_heal_engine(db).run_cycle() + db.commit() + _ci_watch_loop -> _run_ci_watch_cycle -> _load_ci_watch_set (one per (repo, effective workflow)) + get_ci_watch_engine(db).run_cycle(watch_set) + db.commit() + _dep_update_loop -> _run_dep_update_cycle -> _load_dep_update_set + get_dep_update_engine(db).run_cycle(projects) + db.commit() +``` + +## Dependencies +- Internal: roboco.config.settings, roboco.foundation.identity (AGENTS, Role), roboco.foundation.policy.content.markers (set_self_heal_fingerprint), roboco.models.base (Complexity, TaskNature, TaskStatus, TaskType, Team), roboco.services.base.BaseService, roboco.services.notification.NotificationService, roboco.services.project.get_project_service, roboco.services.task (TaskService, TaskCreateRequest, SELF_HEAL_SOURCE, CI_WATCH_SOURCE, DEP_UPDATE_SOURCE, extract_self_heal_fingerprint, get_task_service), roboco.services.telemetry (get_ci_telemetry_source), roboco.services.telemetry.source (get_multi_ci_telemetry_source), roboco.services.workspace.get_workspace_service (dry_upgrade_changes_lockfile), roboco.runtime.orchestrator (the three loops), roboco.db.get_db_context +- External: sqlalchemy.ext.asyncio.AsyncSession, asyncio, hashlib, dataclasses, typing + +## Entry Points + +| Name | File | Trigger | +|---|---|---| +| _self_heal_loop | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/runtime/orchestrator.py | asyncio.create_task at orchestrator start() (line 1012); sleeps self_heal_interval_seconds, opens a DB session, calls SelfHealEngine.run_cycle, commits; early-returns when self_heal_enabled is False | +| _ci_watch_loop | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/runtime/orchestrator.py | asyncio.create_task at orchestrator start() (line 1013); sleeps ci_watch_interval_seconds, runs _run_ci_watch_cycle (loads watch set, runs CiWatchEngine.run_cycle, commits); early-returns when ci_watch_enabled is False | +| _dep_update_loop | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/runtime/orchestrator.py | asyncio.create_task at orchestrator start() (line 1014); sleeps dep_update_interval_seconds, runs _run_dep_update_cycle (loads eligible projects, runs DepUpdateEngine.run_cycle, commits); early-returns when dep_update_enabled is False | + +## Config Flags +- ROBOCO_SELF_HEAL_ENABLED (self_heal_enabled) — master switch for the self-heal loop +- ROBOCO_SELF_HEAL_ORIGINATE_ENABLED (self_heal_originate_enabled) — second opt-in: actually open a fix task (notify-only otherwise) +- ROBOCO_SELF_HEAL_PROJECT_SLUG (self_heal_project_slug) — the single repo self-heal targets +- ROBOCO_SELF_HEAL_CI_WORKFLOW (self_heal_ci_workflow) — CI workflow name for the self-heal telemetry source +- ROBOCO_SELF_HEAL_INTERVAL_SECONDS (self_heal_interval_seconds) — loop period +- ROBOCO_SELF_HEAL_MAX_OPEN_TASKS (self_heal_max_open_tasks) — rolling open-task cap +- ROBOCO_SELF_HEAL_MAX_PER_CYCLE (self_heal_max_per_cycle) — per-cycle origination cap +- ROBOCO_SELF_HEAL_NOTIFY_DEDUPE_SECONDS (self_heal_notify_dedupe_seconds, default 7200) — per-fingerprint CEO-notify dedupe window; a regression that stays red notifies once per episode, not every tick; the key expires after this window so a recurrence notifies again; fail-open (Redis outage still lets the notify through) +- ROBOCO_CI_WATCH_ENABLED (ci_watch_enabled) — master switch for multi-repo CI-watch +- ROBOCO_CI_WATCH_DEFAULT_WORKFLOW (ci_watch_default_workflow) — fallback workflow when a project sets none +- ROBOCO_CI_WATCH_INTERVAL_SECONDS (ci_watch_interval_seconds) +- ROBOCO_CI_WATCH_MAX_OPEN_TASKS (ci_watch_max_open_tasks) +- ROBOCO_CI_WATCH_MAX_PER_CYCLE (ci_watch_max_per_cycle) +- ROBOCO_DEP_UPDATE_ENABLED (dep_update_enabled) — master switch for the dep-update bot +- ROBOCO_DEP_UPDATE_INTERVAL_SECONDS (dep_update_interval_seconds, default 604800 = weekly) +- ROBOCO_DEP_UPDATE_MAX_OPEN_TASKS (dep_update_max_open_tasks) +- ROBOCO_DEP_UPDATE_MAX_PER_CYCLE (dep_update_max_per_cycle) +- per-project projects.ci_watch_enabled / ci_watch_workflow / dep_update_command / assigned_cell (DB columns) + + +## Gotchas +- SelfHealEngine.run_cycle dedupes CEO notifications per fingerprint via Redis (_already_notified / _mark_notified): a regression that stays red across cycles pings the CEO once per episode, not every tick. The check fails open — a Redis outage returns False so the notify still fires (never a swallowed regression). The dedupe key TTL is self_heal_notify_dedupe_seconds (default 7200s), so a regression that clears and recurs within the window is not re-notified (expected: a cleared regression lifts the red signal and a new episode resets the key). The notification layer's purpose-dedup is now a belt-and-suspenders rather than the sole guard. +- SelfHealEngine._originate dedupes by the fingerprint carried in orchestration_markers (extract_self_heal_fingerprint); ci_watch and dep_update instead dedupe by git_url via list_open_*_tasks(git_url=...). The two mechanisms are independent — a self-heal task and a ci_watch task for the same repo are NOT deduped against each other (different source tags). +- Self-heal tasks are created HELD (confirmed_by_human=False) and excluded from give_me_work until the CEO approves; ci_watch and dep_update tasks are created READY (confirmed_by_human=True) and dispatch immediately. A wrong flag here would either strand a fix or auto-dispatch a held one. +- The engines only flush; the orchestrator loop commits. An exception between flush and commit (or a crashed loop iteration logged but swallowed at orchestrator.py 'cycle failed') loses the opened task rows — but they were already flushed into the session that is rolled back on the next get_db_context exit. +- CiWatchEngine._should_open now dedupes per (git_url, effective workflow): a same-workflow monorepo (several cell-projects on one repo) still collapses to one fix task, but two RED workflows of the same repo each get their own fix task (#44, fixed in 536bbb64). The effective workflow is ci_watch_workflow falling back to ci_watch_default_workflow; an empty-string ci_watch_workflow is treated as NULL via SQL NULLIF (d34bc1a7) so it correctly collapses to the default rather than opening a spurious second task. +- DepUpdateEngine._eligible orders cheap checks (command set, id, git_url dedupe) before the expensive dry_upgrade_changes_lockfile probe — but the per-cycle and rolling caps in run_cycle are checked BEFORE _eligible, so a project that fails eligibility still consumed a loop slot but did not consume a cap slot. +- _fingerprint hashes only signal_name (which 'already encodes the repo') — if two distinct regressions share a signal_name on the same repo they collide and the second is deduped away. +- Cell PM notification (ci_watch) is best-effort and catches Exception broadly; a notification failure logs a warning but never rolls back the already-flushed task, so a fix task can exist with no PM ping. +- _cell_pm_slug_for iterates _foundation.AGENTS.values() looking for role==CELL_PM and team==team; if the org config has no cell PM for an assigned_cell, the notification is silently skipped (pm_slug None -> return). +- SelfHealEngine.assess is pure but run_cycle constructs NotificationService() with no session — relies on NotificationService resolving its own session; if it ever needs the engine's session the wiring would break. + + +## Drift from CLAUDE.md +- CLAUDE.md says ci_watch 'reuses the exact hardened per-project GitService.get_latest_ci_conclusion' — the engines themselves do not call GitService; they consume breaches via MultiProjectCITelemetrySource.fetch(projects) in roboco/services/telemetry/source.py. The GitService call is inside the telemetry source, not in ci_watch_engine.py. Minor framing drift, not a code bug. +- CLAUDE.md says dep-update 'Detection is read-only ... WorkspaceService.dry_upgrade_changes_lockfile runs the project's dep_update_command in a throwaway clone of the read clone'. The engine calls self._workspace.dry_upgrade_changes_lockfile(project) but the engine file itself does not reference a 'read clone'; that detail lives in WorkspaceService. Accurate at the system level, not visible in this slice. +- CLAUDE.md states self-heal 'terminates at awaiting_ceo_approval'. The engine itself only creates a PENDING confirmed_by_human=False task; the awaiting_ceo_approval terminal is reached later by the normal lifecycle, not by any code in self_heal_engine.py. Consistent but the engine does not enforce the terminal state itself. + + +## Changes Since Baseline + +| SHA | Subject | Impact | +|---|---|---| +| 15effce0 | Chore: 141 Gaps fill-in (#283) | Single commit touching all three engine files (self_heal +45/-, ci_watch 20 lines tweaked, dep_update 14 lines). Docstring/comment tightening and minor structural cleanup across the three engines — no behavior change to the originate/dedupe/cap logic. Diffstat: 46 insertions, 33 deletions across the three files. | + +> Post-snapshot updates (since 2026-06-29): +> - **536bbb64** (Chore/all/logical gaps sweep, #286) — two behavior changes to engine files: (1) self_heal_engine.py: added per-fingerprint Redis CEO-notify dedupe (_NOTIFY_DEDUPE_KEY_PREFIX constant + _open_self_heal_task_ids_by_fp / _already_notified / _mark_notified / _dedupe_key methods); run_cycle now skips a CEO ping when the fingerprint was already notified this episode; also links the alert to the open fix task via task_id. LOC grew 227→310. (2) ci_watch_engine.py: _should_open now dedupes per (git_url, effective workflow) instead of just git_url — two red workflows of one monorepo each get their own fix task. +> - **d34bc1a7** ([chore] ci-watch/dep-update dedupe: normalize git_url + treat empty-string workflow as default, #148 #1267) — touched task.py and orchestrator.py (NOT the engine files directly): list_open_ci_watch_tasks and list_open_dep_update_tasks now normalize git_url via repo_key SQL mirror (regexp_replace/rtrim/lower) so URL accidentals (.git suffix, trailing slash, case) don't defeat the one-open-task-per-repo invariant; ci_watch workflow dedupe wraps with NULLIF so an empty-string ci_watch_workflow collapses to the default instead of opening a duplicate task every red cycle. + +## Regression Risks + +| Title | File:Line | Claim | Severity | +|---|---|---|---| +| ~~Self-heal CEO notification spam — no engine-level dedupe~~ **RESOLVED 536bbb64** | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/self_heal_engine.py:103 | ~~run_cycle notifies the CEO for EVERY observation EVERY cycle while a regression stays red.~~ Fixed in 536bbb64 (logical-gaps sweep): run_cycle now dedupes per fingerprint via Redis (_already_notified / _mark_notified with self_heal_notify_dedupe_seconds TTL, default 7200s). A persistent red regression pings the CEO once per episode; the check fails open (Redis outage = notify fires anyway). | medium | +| ~~ci_watch per-(repo,workflow) collapse vs per-git_url dedupe under-counts multi-workflow monorepos~~ **RESOLVED 536bbb64** | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/ci_watch_engine.py:137 | ~~_should_open dedupes by git_url only~~ Fixed in 536bbb64: _should_open now dedupes per (git_url, effective workflow), so two red workflows of one monorepo each get their own fix task. The d34bc1a7 companion normalizes git_url with repo_key in the DB query and adds NULLIF for empty-string workflows so the SQL matches Python truthiness collapse. | medium | +| Cap-check ordering in dep_update lets a non-eligible project consume a loop slot but not a cap slot | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/dep_update_engine.py:60 | run_cycle checks per-cycle/rolling caps BEFORE _eligible; a project that fails eligibility (no lockfile change) does not increment open_count, so caps are only consumed by real originations. Correct, but means the expensive dry_upgrade_changes_lockfile probe runs on every eligible project each cycle regardless of how many tasks already opened this cycle until the cap is hit — minor wasted probe cost, not a correctness bug. | low | +| ci_watch cell-PM notify swallows all exceptions after task creation | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/ci_watch_engine.py:129 | _notify_cell_pm catches Exception broadly and only logs a warning. A task is already flushed before the notify, so a notification failure leaves an orphan fix task with no PM ping. Best-effort by design, but the broad except could mask a persistent notification-service outage as a series of warnings. | low | +| Self-heal fingerprint collision on shared signal_name | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/self_heal_engine.py:65 | _fingerprint hashes only signal_name. Two distinct regressions on the same repo with the same signal_name collide; the second is deduped away and never gets a fix task. Unlikely in practice but a latent correctness gap. | low | + +## Health +All three engines are small, single-purpose, and follow a deliberately conservative pattern: gate on a default-off flag, read-only detect, bounded+deduped originate of one PENDING task, flush-only (caller commits), never start/approve/merge/deploy. The safety invariants (self_heal HELD behind CEO approve; ci_watch/dep_update READY but ride normal gates; per-git_url dedupe for monorepos; per-cycle + rolling caps) are intact and consistent with CLAUDE.md. Two medium risks present at baseline are now resolved: CEO notify spam (536bbb64 added Redis per-fingerprint dedupe) and multi-workflow monorepo under-count (536bbb64 changed _should_open to dedupe per (git_url, workflow); d34bc1a7 hardened the SQL to normalize git_url accidentals and treat empty-string workflow as NULL). Remaining standing risks are low-severity: fingerprint collision on shared signal_name, dep_update cap-check ordering (non-eligibles consume a loop slot not a cap slot), and ci_watch cell-PM notify swallowing all exceptions. Health is good. + +## See also +- `docs/map/engine-docs-sync.md` — a sibling originate-only engine that opens a docs-update task on release publish (release-triggered, no background loop). ## Purpose The pure policy + deterministic analyzer behind MegaTask (sequenced batch intake). batch.py is the single source of truth for umbrella/root-subtask identity and git-exemption predicates every layer consults. sequencing.models.py carries the DraftSurface/SequencePlan dataclasses. services/sequencing.py turns declared collision surfaces into a dependency DAG + Kahn-layered waves, and exposes the dev-task collision-DAG and multi-level (cell-task wave-chain + by-osmosis) edge helpers the choreographer wires through add_dependency. @@ -1291,6 +1884,227 @@ foundation-conventions-identity ## Health This slice is a pure, dependency-light foundation layer with no IO/DB and strong fail-fast integrity (six import-time validators abort container start on any roster inconsistency). The single change since baseline (5bb13c84) is purely additive — a new safe-lookup helper alongside the existing raising one — and does not alter any pre-existing symbol or behavior, so regression surface is small. The main latent risk is contractual: role_for_slug_or_none returns None and relies on every caller treating None as "not human-only / proceed", a convention enforced only by docstring and review, not by the type checker at use sites. The conventions sub-package is a clean schema + three-field merge with documented per-field precedence and a permissive (extra=ignore) base model for forward-compat. _generators is deterministic and CI-gated by byte-equality. Overall integrity is high; the slice is well-factored and the only watch item is keeping the role_for_slug vs role_for_slug_or_none discipline intact as new dispatcher call sites are added. +## Purpose +The canonical source of truth for RoboCo's task state machine, per-role action/verb permissions, claim rules, team-match rules, and self-review prevention. The spec module is pure data + lookups (no DB, no I/O); the enforcement package re-exports a backwards-compat view of the same tables plus the git-workflow gates, SLA keys, A2A/journal/ownership access control. Import-time validators in _validate_lifecycle.py make a misconfigured spec fail fast at container start. + +## Files + +| Path | Role | LOC | +|---|---|---| +| /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py | Canonical lifecycle/permissions spec: Status, TaskType, Decision, Precondition, ActionSpec, IntentSpec, StatusTransition tables + can_invoke_action/intent lookups + CLAIM_RULES/ROLE_TEAM_RULES + PR_OPEN_STATES + UNMIGRATED debt set; import-time self-validates. | 1943 | +| /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/_validate_lifecycle.py | Import-time validators (BFS reachability, terminal exits, intent composition chains, claim-rule coverage, self_review symmetry, slug/team agreement, Status/TaskStatus ORM parity, UNMIGRATED subset); LifecycleSpecError aborts container start on a bad spec. | 356 | +| /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/task_lifecycle.py | Backwards-compat shim over the spec: derives VALID_TRANSITIONS / ROLE_RESTRICTED_TRANSITIONS (merging _LEGACY_OPERATIONAL_EDGES + _LEGACY_ROLE_GATES), predicate helpers, SLA keys, GitContext + validate_git_requirements (doc-phase / CEO-escalation / claim-branch gates). | 441 | +| /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/__init__.py | Package re-export aggregator: exposes task_lifecycle, a2a_access, journal_perms, task_ownership public symbols under one namespace. | 87 | +| /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/a2a_access.py | Agent-to-agent direct-message permission gate (delegates to roboco.agents_config.can_a2a_direct); A2AAccessDeniedError. | 91 | +| /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/journal_perms.py | Journal read permissions derived from foundation.policy.journaling ReadTier (ALL/ALL_CELLS/CELL_AND_PMS/CELL/OWN) + PROTECTED_JOURNALS; JournalAccessDeniedError. | 197 | +| /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/task_ownership.py | Task ownership + reassign + view + self-review guard (can_review_task); TaskOwnershipError. | 118 | + +## Key Symbols + +| Name | Kind | File:Line | Responsibility | +|---|---|---|---| +| Status | StrEnum | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:40 | 15 task lifecycle statuses (backlog..cancelled) — the state machine alphabet. | +| TaskType | StrEnum | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:58 | 6 task types used by ActionSpec.allowed_task_types gating. | +| RejectionKind | Literal | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:67 | The 5 rejection flavors (not_authorized/invalid_state/tracing_gap/self_review/not_found) carried in Decision. | +| Decision | dataclass | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:76 | Frozen allow/reject result every consumer maps onto its envelope; invariants enforced in __post_init__; constructors allow()/reject()/tracing_gap(). | +| Precondition | dataclass | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:145 | Declarative gate-table row: a (task,agent,ctx)->bool predicate + remediate hint + missing_token + rejection_kind. | +| ActionSpec | dataclass | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:167 | Atomic action spec (allowed_roles, source_statuses, target_status, allowed_task_types, preconditions, self_review_block, needs_team_match). | +| IntentSpec | dataclass | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:187 | Gateway verb spec: composes tuple of action names, extra_preconditions, pre_side_effects/side_effects, next_hint callback. | +| StatusTransition | dataclass | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:212 | Row from STATUS_TRANSITIONS canon: source/target/triggering_by_action/role_constraint. | +| _STATUS_TRANSITIONS | tuple | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:231 | The canonical state-machine edge table (claim/start/block/qa/pr/complete/cancel/escalate/ceo edges incl. BLOCKED->PENDING and BLOCKED->AWAITING_CEO_APPROVAL). | +| _build_status_graph | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:374 | Derives source->frozenset(targets) view from _STATUS_TRANSITIONS. | +| STATUS_GRAPH | dict | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:382 | Derived state graph consumed by validators + enforcement shim. | +| _ATOMIC_ACTIONS | dict | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:414 | All 25 ActionSpecs (activate, claim, start, set_plan, block, unblock, pause, resume, submit_verification, submit_qa, qa_pass, qa_fail, pr_review_done, docs_complete, submit_for_review, pr_pass, pr_fail, complete, submit_pm_review, escalate_to_ceo, ceo_approve, ceo_reject, ceo_reject_to_pool, cancel, create_subtask). | +| CLAIM_RULES | dict | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:671 | Per-role claimable status sets; narrows the union claim ActionSpec.source_statuses. | +| ROLE_TEAM_RULES | dict | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:702 | Per-slug team binding (None=cross-cell/board) for needs_team_match enforcement. | +| _next_hint_pr_fail | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:789 | pr_fail next hint: steers Main-PM branch-bearing root to re-delegate (loop-breaker) vs dev-revise for cell/dev tasks. | +| Context | dataclass | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:849 | Caller-supplied per-request state (actor_id, plan, journal flags, original_developer_slug, notes, issues, files) fed to Precondition.check. | +| _p_non_terminal | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:894 | Precondition predicate: task not in COMPLETED/CANCELLED (F043 terminal-resurrection guard). | +| PRECONDITION_NON_TERMINAL | Precondition | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:942 | invalid_state precondition attached to escalate_up. | +| PR_OPEN_STATES | frozenset | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:963 | Lifecycle-owned canon of states a PR may be opened from (in_progress/verifying/awaiting_qa/awaiting_documentation/needs_revision); GitService derives its str set from this (F101). | +| PRECONDITION_PR_OPEN_STATE | Precondition | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:981 | invalid_state precondition attached to open_pr (parity with HTTP path). | +| _INTENT_VERBS | dict | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:994 | All ~30 gateway IntentSpecs (give_me_work, i_will_work_on, i_will_plan, delegate, open_pr, i_am_done, sync_branch, i_am_blocked, unclaim, reassign, resume, i_am_idle, claim_review, pass_review, fail_review, claim_pr_review, post_pr_review, claim_gate_review, pr_pass, pr_fail, claim_doc_task, i_documented, complete, escalate_up, escalate_to_ceo, submit_up, submit_root, unblock, triage, triage_all). | +| can_claim | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:1410 | Backward-compat wrapper around can_invoke_action('claim', ...). | +| _check_role_status_type | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:1427 | Role + source-status + task_type gate for an ActionSpec; returns rejection or None. | +| _check_self_review_and_preconditions | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:1469 | self_review_block check (original_developer_slug==actor_slug) + declarative precondition evaluation. | +| _check_claim_rules_narrow | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:1501 | Per-role CLAIM_RULES narrowing for the claim action; disambiguates not_authorized vs invalid_state. | +| can_invoke_action | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:1542 | Order-gated atomic action Decision: action exists -> role -> source status -> task_type -> self_review -> preconditions -> claim rules. | +| _check_intent_preconditions | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:1572 | Verb-level extra_preconditions gate; honors non-tracing rejection_kind (not_authorized/invalid_state) per F043 generalization. | +| can_invoke_intent | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:1604 | Verb Decision: role gate -> extra_preconditions -> FIRST composed action's can_invoke_action (or CLAIM_RULES narrowing for the claim_review/claim_doc_task/claim_gate_review special cases). | +| valid_next_verbs | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:1652 | Sorted list of verbs a role can state-applicably call on a task (preconditions evaluated lazily); used by envelope introspection. | +| composed_actions_for | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:1679 | Return the composes tuple for a verb (KeyError on unknown). | +| intents_for_role | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:1686 | Sorted tuple of verbs declared for a role; drives role_config.py MCP manifest build. | +| status_after | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:1696 | Post-action status or None (no transition / wrong source). | +| UNMIGRATED | frozenset | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:1710 | Known-debt set: legacy operational edges + role gates not yet absorbed into the spec (Phase 3 terminal invariant target = empty). | +| run_all_lifecycle_validators | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/_validate_lifecycle.py:349 | Runs all 14 import-time validators; first failure raises LifecycleSpecError. (14th added: _check_status_enum_parity cross-checks spec.Status against models.base.TaskStatus at import.) | +| reachable_from | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/_validate_lifecycle.py:40 | BFS over STATUS_GRAPH from a start status. | +| _check_status_reachability | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/_validate_lifecycle.py:67 | Every non-BACKLOG status reachable from PENDING. | +| _check_terminal_exits | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/_validate_lifecycle.py:81 | Every non-terminal status has a path to COMPLETED or CANCELLED. | +| _check_intent_chains | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/_validate_lifecycle.py:113 | Adjacent composes actions chain (prev.target_status in next.source_statuses). | +| _check_self_review_symmetry | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/_validate_lifecycle.py:162 | qa_pass/qa_fail/docs_complete/pr_pass/pr_fail agree on self_review_block. | +| _check_action_target_reachable_from_source | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/_validate_lifecycle.py:203 | ActionSpec transitions present in STATUS_GRAPH[source]. | +| _check_unmigrated_is_subset | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/_validate_lifecycle.py:256 | UNMIGRATED stays a subset of _KNOWN_UNMIGRATED_CONSUMERS. | +| validate_task_transition | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/task_lifecycle.py:187 | Legacy raising validator over VALID_TRANSITIONS + ROLE_RESTRICTED_TRANSITIONS (role gate only for transition-level pins). | +| can_agent_transition | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/task_lifecycle.py:225 | Non-raising variant of validate_task_transition. | +| is_terminal_state/is_waiting_state/is_active_state | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/task_lifecycle.py:242 | Hard-coded status-category predicates (NOT derived from the graph — drift risk). | +| ROLE_STATE_SLA_KEYS | dict | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/task_lifecycle.py:276 | (role,status)->settings-key map for stuck-task SLA sweep. | +| sla_seconds_for | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/task_lifecycle.py:285 | Resolve configured SLA seconds for (role,status) from settings. | +| GitContext | dataclass | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/task_lifecycle.py:316 | Git-state carrier (docs_complete/pr_created/pr_number/branch_name + is_coordination/is_external_review/is_umbrella exemption flags). | +| validate_git_requirements | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/task_lifecycle.py:342 | Doc-phase / CEO-escalation / claim-branch git gates; None short-circuits. | +| _LEGACY_OPERATIONAL_EDGES | dict | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/task_lifecycle.py:72 | Transitions the runtime exercises that the spec hasn't absorbed (unclaim/reaper/PM-self-complete/QA-direct/verifying-self-fail/PM-claim/revision-reentry). | +| _LEGACY_ROLE_GATES | dict | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/task_lifecycle.py:99 | Role pins for the legacy edges; UNION-merged with spec-derived ROLE_RESTRICTED_TRANSITIONS (overwrite once dropped pr_reviewer). | +| _build_role_restricted_transitions | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/task_lifecycle.py:156 | Merges spec role_constraint pins with _LEGACY_ROLE_GATES via union (not overwrite). | +| validate_a2a_access | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/a2a_access.py:38 | A2A direct-message gate (delegates to agents_config.can_a2a_direct). | +| can_read_journal | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/journal_perms.py:121 | Journal read decision by ReadTier + protected-journal + same-cell/owner-is-pm rules. | +| validate_task_ownership | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/task_ownership.py:35 | Ownership/reassign/view gate; non-PM must be the assignee. | +| can_review_task | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/enforcement/task_ownership.py:102 | Self-review prevention: agent_id != task_developed_by. | + +## Data Flow +Inputs: a (role, verb_or_action, task, Context) tuple. The Choreographer (services/gateway/choreographer/_impl.py + per-verb modules qa/pr_review/pr_gate) builds a Context (actor_id, plan, original_developer_slug, journal flags) per request, resolves the task row, and calls can_invoke_intent(role, verb, task, ctx). Control flow inside can_invoke_intent: verb lookup -> role gate -> _check_intent_preconditions (extra_preconditions, honoring per-precondition rejection_kind) -> if composes non-empty, can_invoke_action on the FIRST composed action only (subsequent actions chained by the runner after state transitions); if composes empty + verb in {claim_review, claim_doc_task, claim_gate_review}, _check_claim_rules_narrow; else allow. can_invoke_action order: action exists -> _check_role_status_type (role/source-status/task_type) -> _check_self_review_and_preconditions -> _check_claim_rules_narrow for claim. Output: a frozen Decision (allow / reject(kind,...) / tracing_gap(missing,remediate)) mapped by envelope.py onto the gateway Envelope (status/error/remediate/missing) and by valid_next_verbs onto the envelope's current_state introspection. Side-export: role_config.py calls intents_for_role at import to build per-role MCP tool manifests; GitService imports PR_OPEN_STATES to derive its HTTP PR-create str set; task_lifecycle shim derives VALID_TRANSITIONS/ROLE_RESTRICTED_TRANSITIONS consumed by TaskService._validate_and_set_status; validate_git_requirements consumed by TaskService transition path; enforcement.__init__ re-exports a2a/journal/ownership gates consumed by journal services. At module load the spec calls run_all_lifecycle_validators() which BFS-validates the graph + compositions + claim/team tables, aborting container start on inconsistency. + +## Mermaid +```mermaid +stateDiagram-v2 + [*] --> backlog + backlog --> pending: activate (PM) + pending --> claimed: claim + claimed --> in_progress: start + in_progress --> blocked: block + in_progress --> paused: pause + blocked --> in_progress: unblock (PM) + blocked --> pending: unblock (PM, never-claimed) + paused --> in_progress: resume + in_progress --> verifying: submit_verification + verifying --> awaiting_qa: submit_qa + in_progress --> awaiting_pr_review: submit_for_review (PM, pre:create_pr) + awaiting_pr_review --> claimed: claim (pr_reviewer) + awaiting_pr_review --> awaiting_pm_review: pr_pass + awaiting_pr_review --> needs_revision: pr_fail + awaiting_qa --> awaiting_documentation: qa_pass + awaiting_qa --> needs_revision: qa_fail + awaiting_qa --> blocked: QA park (legacy) + awaiting_documentation --> awaiting_pm_review: docs_complete + in_progress --> awaiting_pm_review: submit_pm_review + awaiting_pm_review --> completed: complete (PM) + awaiting_pm_review --> awaiting_ceo_approval: escalate_to_ceo + awaiting_pm_review --> claimed: PM re-claim (legacy) + awaiting_pm_review --> needs_revision: PM reject (legacy) + blocked --> awaiting_ceo_approval: escalate_to_ceo + awaiting_ceo_approval --> completed: ceo_approve + awaiting_ceo_approval --> needs_revision: ceo_reject + needs_revision --> claimed: claim (dev/PM) + needs_revision --> in_progress: re-entry (legacy) + in_progress --> completed: pr_review_done (pr_reviewer, external PR) + claimed --> pending: unclaim (legacy) + in_progress --> pending: reaper (legacy) + pending --> claimed: claim (QA/doc/PM/pr_reviewer by status) + * --> cancelled: cancel (PM/CEO) +``` + +## Logical Tree +``` +foundation-lifecycle +├── roboco/foundation/policy/lifecycle.py (canonical spec) +│ ├── Enums: Status, TaskType, RejectionKind +│ ├── Dataclasses: Decision, Precondition, ActionSpec, IntentSpec, StatusTransition, Context +│ ├── _STATUS_TRANSITIONS (edge table) -> STATUS_GRAPH (derived) +│ ├── _ATOMIC_ACTIONS (22 ActionSpecs) +│ ├── CLAIM_RULES (per-role claimable statuses) +│ ├── ROLE_TEAM_RULES (per-slug team binding) +│ ├── Precondition predicates + PRECONDITION_* constants (PLAN/COMMITS/NO_PR/OWNERSHIP/NON_TERMINAL/PR_OPEN_STATE) +│ ├── PR_OPEN_STATES (canon) +│ ├── _INTENT_VERBS (~30 IntentSpecs) + _next_hint_* helpers +│ ├── Lookups: can_claim, can_invoke_action, can_invoke_intent, valid_next_verbs, composed_actions_for, intents_for_role, status_after +│ ├── Internal helpers: _check_role_status_type, _check_self_review_and_preconditions, _check_claim_rules_narrow, _check_intent_preconditions +│ └── UNMIGRATED / _KNOWN_UNMIGRATED_CONSUMERS (debt fence) +├── roboco/foundation/_validate_lifecycle.py (import-time validators) +│ ├── LifecycleSpecError +│ ├── reachable_from (BFS) +│ └── 13 _check_* validators -> run_all_lifecycle_validators +└── roboco/enforcement/ (backwards-compat + access control) + ├── __init__.py (re-export aggregator) + ├── task_lifecycle.py + │ ├── _LEGACY_OPERATIONAL_EDGES / _LEGACY_ROLE_GATES + │ ├── VALID_TRANSITIONS / ROLE_RESTRICTED_TRANSITIONS (derived, union-merged) + │ ├── validate_task_transition / can_agent_transition / get_valid_transitions + │ ├── is_terminal_state / is_waiting_state / is_active_state (hard-coded) + │ ├── ROLE_STATE_SLA_KEYS / sla_seconds_for + │ └── GitContext / GitRequirementError / validate_git_requirements / check_parallel_completion + ├── a2a_access.py (A2A direct-message gate) + ├── journal_perms.py (ReadTier-based journal read gate) + └── task_ownership.py (ownership + reassign + self-review) +``` + +## Dependencies +- Internal: roboco.foundation.identity (Role, Team), roboco.foundation.policy.journaling (ReadTier, ROLE_READ_TIERS, PROTECTED_JOURNALS), roboco.seeds.initial_data (AGENT_UUIDS, DEFAULT_AGENTS) for slug/team validators, roboco.agents_config (can_a2a_direct, get_a2a_route_hint, get_agent_cell/role/team), roboco.config.settings (SLA key resolution), roboco.exceptions (RobocoError, TaskLifecycleError) +- External: dataclasses (frozen dataclasses), enum.StrEnum, collections.deque (BFS validator), itertools.pairwise (intent chain validator), typing (Literal, TYPE_CHECKING, Callable, Any) + +## Entry Points + +| Name | File | Trigger | +|---|---|---| +| import of roboco.foundation.policy.lifecycle | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py | module load — fires run_all_lifecycle_validators() at the bottom; a bad spec aborts the orchestrator container start | +| Choreographer verb dispatch (can_invoke_intent) | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/gateway/choreographer/_impl.py | every gateway flow verb call — role/verb/task/Context decision before any state mutation | +| role_config.py manifest build (intents_for_role) | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/gateway/role_config.py | import — derives per-role MCP tool manifest from the spec | +| envelope introspection (valid_next_verbs) | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/gateway/envelope.py | every Envelope populate — surfaces current_state + applicable verbs to the agent | +| TaskService transition (validate_task_transition / validate_git_requirements / VALID_TRANSITIONS) | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/task.py | TaskService._validate_and_set_status on every status write | +| GitService HTTP PR-create (PR_OPEN_STATES) | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/git.py | HTTP POST /api/v1/.../prs — derives its str-set gate from the lifecycle canon | +| stuck-task SLA sweep (sla_seconds_for / ROLE_STATE_SLA_KEYS) | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/runtime/orchestrator.py | orchestrator sweep tick | + +## Config Flags +- agent_sla_developer_in_progress +- agent_sla_developer_verifying +- agent_sla_qa_claimed +- agent_sla_documenter_claimed +- agent_sla_cell_pm_claimed (resolved by sla_seconds_for via roboco.config.settings) — no ROBOCO_* feature flag directly gates this slice; it is the always-on policy core + + +## Gotchas +- can_invoke_intent only checks the FIRST composed action's source-status; subsequent actions in a multi-step composition (e.g. i_will_work_on = claim,set_plan,start) are NOT pre-checked — the verb runner must execute them in order and rollback on mid-composition failure, or the task is left stranded (e.g. claimed but never started). +- verbs with composes=() and not in the claim_review/claim_doc_task/claim_gate_review special-case list have NO source-status gate at the spec layer — their entire gate is extra_preconditions + the handler. sync_branch, open_pr, unclaim, reassign, escalate_up, give_me_work, i_am_idle, triage, triage_all all rely on this. A new empty-composes verb that transitions state without a NON_TERMINAL/source-status precondition can resurrect a terminal task (the F043 class of bug). +- _check_intent_preconditions drops the `missing` list for non-tracing rejection kinds (not_authorized/invalid_state) — Decision.reject sets missing=[]. Agents doing exact-string checks on `missing` will not see tokens for ownership/terminal/PR-open-state failures. +- enforcement/task_lifecycle.is_terminal_state/is_waiting_state/is_active_state are hard-coded string sets, NOT derived from STATUS_GRAPH; they now partition the full Status enum (ef33d56c added awaiting_pr_review; a2d2ef47 added backlog+pending to is_waiting_state so the three predicates cover every Status member), and test_status_classification_covers_every_enum_member in tests/unit/enforcement/test_task_lifecycle.py asserts full coverage — a new Status member that falls through all three predicates will now fail the build instead of silently miscategorizing. The general drift risk (hard-coded vs graph-derived) remains. +- _build_role_restricted_transitions UNION-merges legacy role gates with spec pins; an earlier overwrite silently dropped pr_reviewer from (in_progress, completed). Any new spec role_constraint on an edge that also has a _LEGACY_ROLE_GATES entry must be tested for the union, not overwrite. +- UNMIGRATED must stay a subset of _KNOWN_UNMIGRATED_CONSUMERS or import fails — extending known debt requires editing both frozensets. +- Decision is frozen; the allowed=True path forbids missing/remediate, the allowed=False path forbids rejection_kind=None — building one outside the classmethods via direct __init__ is supported but the invariants are enforced in __post_init__. +- STATUS_GRAPH includes a self-loop-ish edge: cancel is generated for every non-terminal source, so every non-terminal status has CANCELLED as a target — validators pass because of this; removing the cancel generator without re-adding per-source edges would break _check_terminal_exits. +- the spec module imports Role/Team from foundation.identity and _validate_lifecycle at the BOTTOM (after table defs); validators defer-import the spec to break the cycle — pyproject PLC0415 exemption exists for this. +- BLOCKED has two unblock targets (IN_PROGRESS for previously-claimed, PENDING for never-claimed) — the ActionSpec.unblock source_statuses is just {BLOCKED} with target IN_PROGRESS; the BLOCKED->PENDING edge lives only in _STATUS_TRANSITIONS/STATUS_GRAPH and is exercised by the legacy shim, not by an ActionSpec. +- ROLE_TEAM_RULES is keyed by slug, not role — adding a new agent slug without a row means needs_team_match enforcement falls back to None (any team) for that slug silently. + + +## Drift from CLAUDE.md +- CLAUDE.md 'Role-Based Transitions' table lists escalate_to_ceo only from awaiting_pm_review; the spec also declares BLOCKED -> AWAITING_CEO_APPROVAL via escalate_to_ceo (lifecycle.py:334-339, ActionSpec source_statuses includes BLOCKED at line 615). The BLOCKED->AWAITING_CEO_APPROVAL edge is missing from the doc table. +- CLAUDE.md verb table for developer lists resume/unclaim but omits i_am_idle (every role gets i_am_idle per spec line 1150); the doc does state 'every role also gets i_am_idle' in prose, so this is partial drift — the per-role table column omits it. +- CLAUDE.md says 'roboco/enforcement/task_lifecycle.py is a backwards-compat shim over it' — true for the transition tables, but the shim ALSO owns GitContext/validate_git_requirements/SLA tables (not a pure view); the doc undersells the shim's owned symbols. +- enforcement/__init__.py __all__ does NOT export sla_seconds_for, check_parallel_completion, or ROLE_STATE_SLA_KEYS even though task_lifecycle.py's own __all__ does — consumers must import from the submodule, not the package. Not a CLAUDE.md drift but a surface inconsistency. + + +## Changes Since Baseline + +| SHA | Subject | Impact | +|---|---|---| +| e202ce39 | Make main_pm + task_type=code impossible | In this slice: prose-only — added _next_hint_pr_fail (branch-aware re-delegate hint for Main-PM roots vs dev-revise for cell/dev) and rewrote submit_root description to 'branch-keyed not task_type-keyed'. The actual main_pm+code impossibility is enforced elsewhere (choreographer), not via ActionSpec.allowed_task_types here. | +| 250be5c2 | sync_branch dev verb — gate-level branch rebase (Phase B1) | Added IntentSpec sync_branch (dev-only, composes=(), extra_preconditions=OWNERSHIP, no DB transition) + _next_hint_synced helper. New empty-composes verb with no source-status gate; relies entirely on the handler's no-branch/protected-base guards. | +| 2f322286 | [F043] guard escalate_up against resurrecting terminal tasks | Added PRECONDITION_NON_TERMINAL (invalid_state) to escalate_up.extra_preconditions; generalized _check_intent_preconditions to honor any non-tracing rejection_kind (previously only not_authorized was special-cased, everything else was tracing_gap). Now invalid_state preconditions produce Decision.reject instead of tracing_gap. | +| c34e978f | [F101] enforce PR-open state gate on gateway open_pr | Added PR_OPEN_STATES frozenset (in_progress/verifying/awaiting_qa/awaiting_documentation/needs_revision) + PRECONDITION_PR_OPEN_STATE (invalid_state) inserted into open_pr.extra_preconditions between OWNERSHIP and COMMITS. Closes the gateway parity gap: open_pr (composes=()) previously had no source-status gate; now rejects claim/paused/blocked/terminal. | + +> Post-snapshot updates (since 2026-06-29): 15effce0 + 536bbb64 (141-gap + logical-gap sweeps) added SYNC_BRANCH_STATES + PRECONDITION_SYNC_BRANCH_STATE to sync_branch.extra_preconditions (closes medium-risk source-status gap), PRECONDITION_ROOT_NOT_CODE to submit_root.extra_preconditions (closes low-risk spec/prose gap), and ceo_reject_to_pool ActionSpec (AWAITING_CEO_APPROVAL→PENDING CEO reject-to-pool path). 16b71be8 ([sweep] lifecycle: 6 gaps) fixed cancel CEO gate, claim_pr_review gate, needs_team_match enforcement, valid_next_verbs narrowing, pr_reviewer unclaim (unclaim.allowed_roles now includes Role.PR_REVIEWER), and complete side_effect ordering. ef33d56c ([chore] lifecycle-enforcement validators + status-class) dropped the spurious VERIFYING→awaiting_documentation legacy edge from _LEGACY_OPERATIONAL_EDGES, added awaiting_pr_review to is_waiting_state, and overhauled _check_status_enum_coverage from a tautology to a real bidirectional check + split _check_terminal_exits into COMPLETED-path + cancel-exit check + added new _check_status_enum_parity validator (spec.Status vs models.base.TaskStatus ORM parity). a2d2ef47 ([sweep] enforcement status-class partition) added backlog + pending to is_waiting_state completing the terminal/active/waiting partition + added the coverage-invariant test. b3558d4e ([chore] complexity) extracted _check_team_match helper from can_invoke_action (no behavior change). + +## Regression Risks + +| Title | File:Line | Claim | Severity | +|---|---|---|---| +| ~~sync_branch has no source-status gate — can be called on terminal/paused/blocked tasks~~ **RESOLVED** | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:1225 | FIXED post-snapshot (commits 536bbb64/15effce0): PRECONDITION_SYNC_BRANCH_STATE (rejection_kind=invalid_state) added to sync_branch.extra_preconditions; gates the verb to SYNC_BRANCH_STATES={claimed,in_progress,verifying,needs_revision}. Terminal, paused, and blocked tasks now receive Decision.reject(invalid_state). | medium | +| open_pr now rejects `claimed` — may break dev flows that open PR before start | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:963 | PR_OPEN_STATES excludes CLAIMED. The intent composition i_will_work_on (claim,set_plan,start) transitions claim->in_progress atomically, so the normal path is fine, but any path that opens a PR while still in CLAIMED (e.g. a dev who calls open_pr before i_will_work_on finishes, or a custom flow) now receives invalid_state instead of the previous silent accept. This is the intended F101 parity fix, but it is a behavior tightening — verify no caller relied on claim-state PR opens. | low | +| _check_intent_preconditions generalization may misroute existing not_authorized callers | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:1572 | Before F043, only not_authorized was special-cased and the message was first_missing.remediate. After generalization, ALL non-tracing rejection kinds (not_authorized, invalid_state) take the Decision.reject path with message=remediate. Behavior for not_authorized is unchanged (still remediate-as-message), but any caller that introspected rejection_kind=='tracing_gap' to mean 'a precondition failed' will now miss invalid_state/ownership failures. The envelope's missing[] field is also empty for these. Low-medium risk for consumers that branch on rejection_kind. | low | +| pr_fail hint now branches on team==MAIN_PM and branch_name — non-Main-PM branch-bearing tasks get dev-revise | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:789 | _next_hint_pr_fail returns the re-delegate hint only when team==MAIN_PM.value AND branch is set. A cell-PM coordination root with a branch (rare but possible) falls through to 'dev will revise', which is wrong for a PM-owned assembled PR. Low impact (hint text only, not a gate) but could mislead a cell PM into waiting instead of re-delegating. | low | +| ~~submit_root description claims 'branch-keyed, not task_type-keyed' but no allowed_task_types gate enforces it~~ **RESOLVED** | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:1502 | FIXED post-snapshot (commits 536bbb64/15effce0): PRECONDITION_ROOT_NOT_CODE (invalid_state, key="root_not_code") added to submit_root.extra_preconditions; rejects when task.task_type==CODE. The spec now enforces the constraint the prose described. Defense-in-depth vs the creation-path choreographer guard. | low | +| CLAIM_RULES lets PMs claim NEEDS_REVISION — combined with pr_fail reassign can create a PM/dev tug-of-war | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/foundation/policy/lifecycle.py:687 | CELL_PM/MAIN_PM claim rules include NEEDS_REVISION (added so PM-owned coordination tasks have an actor after pr_fail/ceo_reject). give_me_work only offers assigned tasks, so normally safe, but a PM re-claiming a NEEDS_REVISION dev leaf (if misassigned) would bypass the dev-revise path. The comment documents this is scoped by the same mechanism as dev leaf-revision; risk is in assignment correctness upstream, not the spec itself. Low risk. | low | + +## Health +The slice is structurally healthy and well-fenced: the spec is pure data + lookups with no I/O, frozen dataclasses enforce Decision invariants, 14 import-time validators (including the new Status/TaskStatus ORM-parity check) make a misconfigured spec fail fast at container start, and the enforcement shim clearly fences its _LEGACY_* additions (UNMIGRATED debt tracked, union-merge prevents the pr_reviewer-drop regression). Post-snapshot commits resolved the two open medium/low risks: sync_branch now has a source-status gate (PRECONDITION_SYNC_BRANCH_STATE) and submit_root now spec-enforces the not-code constraint (PRECONDITION_ROOT_NOT_CODE). The remaining integrity concerns are behavioral: (1) the empty-composes verb pattern (unclaim, reassign, escalate_up, give_me_work, i_am_idle, triage) still has NO source-status gate at the spec layer — each new empty-composes verb that touches state must add its own NON_TERMINAL/source-status precondition; (2) can_invoke_intent only pre-checks the FIRST composed action, leaving mid-composition rollback to the runner; (3) is_terminal_state/is_waiting_state/is_active_state are hard-coded string sets not derived from STATUS_GRAPH, though a build-time coverage test now catches any new Status member that falls through the partition. No active logic bugs found; the slice is the canonical source its consumers trust. + ## Purpose The "misc" foundation-policy slice holds the pure, service-agnostic rule catalogs that the gateway/services layer composes: notification/urgency rules (communications.py), journal scope + read-tier permissions (journaling.py), task completeness field rules + placeholder denylists (task_completeness.py), the verb→required-set tracing gate table (tracing.py), per-agent budget/loop/circuit-breaker thresholds (agent_loop.py), and the structured agent-content schema (content/ — typed PR-review/QA/doc/dev/auditor/resumption/task-description models with validators). It is data + validators, no I/O, no DB — the single source of truth that enforcement/services consume. @@ -1315,7 +2129,8 @@ The "misc" foundation-policy slice holds the pure, service-agnostic rule catalog |---|---|---|---| | parse_priority | function | roboco/foundation/policy/communications.py:23 | Resolve a NotificationPriority from raw_priority string or legacy urgent flag; unknown→NORMAL | | NOTIFY_SENDER_ROLES | constant | roboco/foundation/policy/communications.py:51 | frozenset of roles permitted to call notify() (CELL_PM, MAIN_PM, PRODUCT_OWNER, HEAD_MARKETING, CEO) | -| ACK_REQUIRED_BY_TYPE | constant | roboco/foundation/policy/communications.py:66 | NotificationType→requires_ack mapping (action-required vs informational) | +| NO_COMMS_ROLES | constant | roboco/foundation/policy/communications.py:66 | frozenset of roles with NO agent-comms surface at all (AUDITOR, PR_REVIEWER, PROMPTER, SECRETARY) — the canonical set both `content_actions.dm()`'s sender-side guard and `agents_config.can_a2a_direct`'s CEO-target-side check consume, so the two enforcement points can't drift apart | +| ACK_REQUIRED_BY_TYPE | constant | roboco/foundation/policy/communications.py:80 | NotificationType→requires_ack mapping (action-required vs informational) | | Scope | enum | roboco/foundation/policy/journaling.py:21 | Journal entry scope StrEnum (note/decision/reflect/learning/struggle) | | SCOPE_TO_TYPE | constant | roboco/foundation/policy/journaling.py:32 | Scope→JournalEntryType single-source mapping | | ReadTier | enum | roboco/foundation/policy/journaling.py:41 | Journal read-breadth StrEnum (own/cell/cell_and_pms/all_cells/all) | @@ -1409,7 +2224,7 @@ The "misc" foundation-policy slice holds the pure, service-agnostic rule catalog | _extract_strs_from_dict | function | roboco/foundation/policy/content/validators.py:128 | Resolve dict element to strings via first recognized text key else bare string values | ## Data Flow -This slice is pure policy/data: no I/O, no DB, no async. Control flows IN from the gateway/services layer at call sites. (1) communications.py: notification_delivery.py / notification.py import ACK_REQUIRED_BY_TYPE; agents_config.py / services/permissions.py / gateway/content_actions.py import NOTIFY_SENDER_ROLES for the notify-sender allowlist; a2a.py imports parse_priority to resolve A2A urgency tristate. (2) journaling.py: JournalService + enforcement/journal_perms.py import SCOPE_TO_TYPE, ROLE_READ_TIERS, PROTECTED_JOURNALS; gateway content_actions imports Scope for the note scope validation. (3) task_completeness.py: TaskService.create / PrompterService.create_task_from_draft / choreographer._impl._create_subtask_from_inputs call check(TASK_AT_CREATE, task) and the fill_* helpers; on failure raise TaskCompletenessError → gateway returns Envelope.incomplete_input with field_hints. (4) tracing.py: choreographer/_impl.py, qa.py, doc.py, pr_gate.py, pr_review.py call requirements_for(verb) then check_requirements(task=..., requirements=..., ctx=GateContext(...)) before allowing a state transition; failure → Envelope.tracing_gap with missing list. The parity test test_every_intent_verb_has_a_tracing_decision asserts every intent verb is in either VERB_REQUIREMENTS or VERBS_WITHOUT_TRACING. (5) agent_loop.py: agent_sdk/server.py reads DEFAULT_BUDGET thresholds (env-overridable) and retry_limit_for(verb); orchestrator.py reads pm_respawn_max_unproductive + pm_respawn_max_tracing_resets for the respawn circuit breaker. (6) content/: gateway content_actions / choreographer / prompter / intake_driver / flow_server call validate_content(content_type, payload) → model instance → render_markdown() written to Task note columns + PR comment bodies; pr_review_conflict guards the GitHub review event against findings; markers.py accessors are called across task.py, orchestrator, self_heal_engine, release_proposal, release_manager_engine to read/write orchestration_markers. coerce_str_list is applied at the intake/flow boundary (intake_driver, flow_server, flow schemas, prompter) BEFORE payloads reach the models, so models only need coerce_to_list as a backstop. +This slice is pure policy/data: no I/O, no DB, no async. Control flows IN from the gateway/services layer at call sites. (1) communications.py: notification_delivery.py / notification.py import ACK_REQUIRED_BY_TYPE; agents_config.py / services/permissions.py / gateway/content_actions.py import NOTIFY_SENDER_ROLES for the notify-sender allowlist; a2a.py imports parse_priority to resolve A2A urgency tristate; `agents_config.py`'s `_check_ceo_a2a` (consulted from `can_a2a_direct`'s CEO branch) and `gateway/content_actions.py`'s `_NO_COMMS_ROLES` (the `dm()` sender-side guard, now `frozenset(r.value for r in NO_COMMS_ROLES)` instead of an independently hand-maintained literal) both import NO_COMMS_ROLES. (2) journaling.py: JournalService + enforcement/journal_perms.py import SCOPE_TO_TYPE, ROLE_READ_TIERS, PROTECTED_JOURNALS; gateway content_actions imports Scope for the note scope validation. (3) task_completeness.py: TaskService.create / PrompterService.create_task_from_draft / choreographer._impl._create_subtask_from_inputs call check(TASK_AT_CREATE, task) and the fill_* helpers; on failure raise TaskCompletenessError → gateway returns Envelope.incomplete_input with field_hints. (4) tracing.py: choreographer/_impl.py, qa.py, doc.py, pr_gate.py, pr_review.py call requirements_for(verb) then check_requirements(task=..., requirements=..., ctx=GateContext(...)) before allowing a state transition; failure → Envelope.tracing_gap with missing list. The parity test test_every_intent_verb_has_a_tracing_decision asserts every intent verb is in either VERB_REQUIREMENTS or VERBS_WITHOUT_TRACING. (5) agent_loop.py: agent_sdk/server.py reads DEFAULT_BUDGET thresholds (env-overridable) and retry_limit_for(verb); orchestrator.py reads pm_respawn_max_unproductive + pm_respawn_max_tracing_resets for the respawn circuit breaker. (6) content/: gateway content_actions / choreographer / prompter / intake_driver / flow_server call validate_content(content_type, payload) → model instance → render_markdown() written to Task note columns + PR comment bodies; pr_review_conflict guards the GitHub review event against findings; markers.py accessors are called across task.py, orchestrator, self_heal_engine, release_proposal, release_manager_engine to read/write orchestration_markers. coerce_str_list is applied at the intake/flow boundary (intake_driver, flow_server, flow schemas, prompter) BEFORE payloads reach the models, so models only need coerce_to_list as a backstop. ## Mermaid ```mermaid @@ -1554,6 +2369,8 @@ foundation/policy (misc slice) | 3a4a3fe5 | reduce xenon C-rank blocks to A (behavior-preserving) | Extracted _extract_strs_from_dict helper from inline logic in validators._extract_strs — pure move-and-call refactor, no control flow / return values / side effects changed | > Post-snapshot updates (since 2026-06-29): a squash-merge commit landed on this slice. `15effce0` (Chore: 141 Gaps fill-in, 2026-06-29) squash-merged the pre-snapshot baseline commits (53d60da3/e202ce39/e52fd05d/cf7603f3/3a4a3fe5) into a single PR — those hashes no longer exist in history but their effects are fully reflected in the Key Symbols above (coerce_str_list/PrReviewContent.issues+head_sha/sync_branch in VERBS_WITHOUT_TRACING). No other symbols in this slice changed. +> +> `56b6693e` ("security-hygiene-sweep"): adds `NO_COMMS_ROLES` (communications.py:66) — the four no-agent-comms-surface roles (auditor/pr_reviewer/prompter/secretary), extracted as the single canonical set two previously-independent hand-maintained literals now both derive from: `content_actions.py`'s `_NO_COMMS_ROLES` (the `dm()` sender-side guard) and a new `agents_config.py` helper, `_check_ceo_a2a` (called from `can_a2a_direct`'s CEO branch — previously an unconditional `True` for any CEO-initiated target; the "one asymmetric rule" now itself refuses a no-comms target). A downstream consequence with no separate code edit: `agents_config.py`'s statically-derived `A2A_ALLOWED_PAIRS` shrank 93→88 (`ceo` group 23→18). ## Regression Risks @@ -1568,1129 +2385,6 @@ foundation/policy (misc slice) ## Health This slice is in good shape: it is pure data + validators with no I/O or concurrency of its own, well-documented intent at each site, and a parity test guarding the tracing table against drift. The recent changes are small, additive, and mostly behavior-preserving. The main latent risk is a naming-convention footgun (agent_loop 'pass'/'fail' keys vs tracing 'pass_review'/'fail_review'; sync_branch missing from VERB_RETRY_LIMITS) — not an active bug today, but one rename away from silently degrading. The markers accessors correctly handle SQLAlchemy dirty tracking via dict reassignment, and the content models correctly delegate SDK-wrapper flattening to the intake boundary via coerce_str_list. No active regressions observed in the slice itself. -# models slice - -## Purpose - -The Pydantic/dataclass domain surface of RoboCo — the typed contract the API, services, orchestrator, and agent runtimes all speak. These are **not** the ORM tables (`roboco/db/tables.py` owns persistence); models here are request/response schemas, domain aggregates, runtime DTOs, and enums that get crossed into SQLAlchemy rows by the service layer. Validation (`RobocoBase`: `extra="forbid"`, `validate_assignment`, `use_enum_values`) lives here, and several files (`agents.py`, `runtime.py`, `optimal.py`, `metrics.py`, `llm.py`, `audit.py`, `dashboard.py`, `transcription.py`, `extraction.py`, `permissions.py`) are pure dataclasses/StrEnums with no Pydantic model at all — runtime value types the orchestrator and services pass around. - -## Files - -| Path | Role | approx LOC | -|------|------|------------| -| `__init__.py` | Public re-export surface (`Agent`, `Task`, `Notification`, `Journal`, `CommitRef`, enums, `get_column_config`, …) | 149 | -| `base.py` | All shared StrEnums (`TaskStatus`, `TaskType`, `AgentStatus`, `ModelProvider`, `JournalEntryType`, …) + `RobocoBase`/`TimestampMixin` + `AgentRole`/`Team` aliases to `foundation.identity` | 275 | -| `task.py` | `Task` aggregate + `CommitRef`/`DocRef`/`ProgressUpdate`/`Checkpoint`/`SubTask`/`TaskPlan` + `TaskCreate`/`TaskUpdate`/`TaskCreateRequest` | 502 | -| `a2a.py` | A2A protocol wire models (`AgentCard`, `A2ATask`, `A2AMessage`, parts) + persistent conversation models (`A2AConversation`, `A2AChatMessage`) + state mappers | 592 | -| `agents.py` | Agent **runtime** domain types — per-role phase enums (`DevTaskPhase`, `QATaskPhase`, `CellPMPhase`, …), `AgentConfig`/`AgentState`, `TaskContext`/`ReviewContext`/`DocContext`, `AuditFlag`/`AuditReport` | 405 | -| `journal.py` | `Journal`/`JournalEntry` + 5 factory param dataclasses + `create_*_entry` factories + `JournalStats`/`GrowthMetrics` | 374 | -| `permissions.py` | `PermissionLevel` IntEnum, `ROLE_LEVELS` (built from `agents_config`), `AgentContext`, `COMMUNICATION_MATRIX`, `TASK_PERMISSIONS`, `KB_PERMISSIONS` | 284 | -| `optimal.py` | RAG domain types — `IndexType`, `SearchResult`/`SearchOutcome`/`RAGResponse`, `ErrorPattern`/`Decision`/`Standard`, `MentorResponse`, `CodeReviewResult` | 275 | -| `metrics.py` | `VelocityMetrics`/`BlockerMetrics`/`TeamMetrics`/`AgentMetrics` + v0.10.0 observability: `StageTiming`/`StageBottleneck`/`BottleneckReport`/`AgentReworkRate`/`ReworkReport`/`Scorecard` | 249 | -| `events.py` | `EventType` StrEnum + `Event` dataclass (JSON round-trip) + `NotificationServiceProtocol`/`OrchestratorAccessProtocol`/`EventContext` DI container | 199 | -| `handoff.py` | `DocumenterHandoff` + `CodeSample`/`DocumentationItem`/`ConversationRef` + `HandoffCreate` (RESERVED — see file header) | 202 | -| `project.py` | `Project` + `BranchReason` + `ProjectCreate`/`ProjectUpdate` (CI-watch, dep-update, quality_command fields) | 178 | -| `agent.py` | `Agent` API model + `ModelConfig`/`AgentPermissions`/`AgentMetrics` + `AgentCreate`/`AgentUpdate` | 172 | -| `kanban.py` | `KanbanBoard`/`KanbanColumn`/`KanbanCard`/`KanbanSwimlane` + per-role column configs + `get_column_config` | 159 | -| `llm_catalog.py` | `CatalogEntry` + `MODEL_CATALOG`/`MODEL_CATALOG_BY_NAME`/`provider_type_for_model` + `OLLAMA_DEFAULT_MODEL` (Settings dropdown source of truth) | 103 | -| `message.py` | `ExtractedMessage` + `RawStream` | 137 | -| `runtime.py` | Orchestrator runtime types — `OrchestratorAgentState`, `SpawnGitContext`, `OrchestratorAgentConfig`, `AgentInstance`, `WaitingRecord`, `MODEL_MAP`, `ROLE_MODEL_MAP` | 128 | -| `transcription.py` | `StreamBuffer` (flush heuristic) + `TranscriptionConfig` | 119 | -| `llm.py` | `LLMUsage`/`ToonConfig`/`EncodedBlock`/`ToonMetrics` (token + TOON serialization metrics) | 118 | -| `notification.py` | `Notification` + `NotificationCreate` + `CreateNotificationParams` (gains a per-row `requires_ack: bool \| None` override, wave 3) | 121 | -| `work_session.py` | `WorkSession` + `WorkSessionStatus` + `WorkSessionCreate`/`WorkSessionUpdate` | 117 | -| `pitch.py` | `Pitch` + `PitchStatus` + `PitchCreate` (Board proposal → provisioning) | 73 | -| `extraction.py` | `ExtractionContext`/`ExtractionResult`/`ExtractionConfig` | 68 | -| `product.py` | `Product` + `ProductCellMapping` (cell→Project map; validator enforces cell-only) + create/update DTOs | 69 | -| `playbook.py` | `Playbook` + `PlaybookCreate`/`PlaybookUpdate` (curated procedure; `from_attributes` for ORM load) | 58 | -| `audit.py` | `AuditEventType` StrEnum + `PermissionDenialContext`/`StateTransitionDenialContext` dataclasses | 58 | -| `dashboard.py` | `FlagData`/`ReportData`/`TeamHealthData`/`AuditQueueItem`/`CreateFlagParams`/`DashboardStorage` | 96 | -| `secretary.py` | `DirectiveKind`/`DirectiveStatus` StrEnums + `GATED_KINDS` frozenset | 41 | -| `README.md` | Architecture doc for the models package | ~250 | - -## Key Symbols - -| Name | Kind | File:Line | Responsibility | -|------|------|-----------|----------------| -| `Task` | Pydantic model | task.py:132 | Atomic unit of work; carries status, branch, PR, batch surface, ACs, gateway lock, structured notes | -| `TaskStatus` | StrEnum | base.py:31 | 15-state lifecycle (backlog→pending→claimed→…→completed/cancelled) | -| `TaskType` | StrEnum | base.py:63 | code/documentation/research/planning/design/administrative | -| `TaskNature` | StrEnum | base.py:74 | technical/non_technical | -| `CommitRef` | Pydantic model | task.py:31 | Git commit reference (hash/message/timestamp/author) | -| `DocRef` | Pydantic model | task.py:42 | Document reference with version + author trail | -| `TaskPlan` | Pydantic model | task.py:109 | Approach + ordered `SubTask`s + risks/open_questions | -| `TaskCreate` | Pydantic schema | task.py:350 | Request schema; `_exactly_one_target` validator (project_id / product_id / cell_projects) | -| `TaskCreateRequest` | dataclass | task.py:456 | Service-layer create params mirroring `TASK_AT_CREATE` (no silent defaults) | -| `Agent` | Pydantic model | agent.py:79 | API agent model (role, team, model config, permissions, metrics, journal_id) | -| `AgentRole` | alias | base.py:23 | `= identity.Role` — canonical Role enum lives in `foundation/identity` | -| `Team` | alias | base.py:24 | `= identity.Team` — canonical Team enum lives in `foundation/identity` | -| `ModelProvider` | StrEnum | base.py:197 | anthropic/ollama_cloud/openai/local/grok | -| `ModelConfig` | Pydantic model | agent.py:27 | provider + name + fallback + temperature + max_tokens | -| `AgentPermissions` | Pydantic model | agent.py:45 | can_notify | -| `WorkSession` | Pydantic model | work_session.py:25 | Branch/PR/merge tracking for a (project, task, agent) work episode | -| `WorkSessionStatus` | StrEnum | work_session.py:17 | active/completed/abandoned | -| `Project` | Pydantic model | project.py:47 | Git repo config + CI/dep-update/`sandbox_services` opt-ins + `assigned_cell` | -| `BranchReason` | StrEnum | project.py:18 | feature/bug/chore/docs/hotfix (branch-name prefixes) | -| `ExtractedMessage` | Pydantic model | message.py:51 | Stored message with embedding, edit_history, mentions, task_id | -| `MessageType` | StrEnum | base.py:128 | reasoning/dialogue/decision/action/blocker/technical | -| `RawStream` | Pydantic model | message.py:32 | Ephemeral WebSocket chunk payload | -| `Notification` | Pydantic model | notification.py:26 | Formal signal requiring ACK; from/to_agents, acked_by, acked_at | -| `NotificationType` | StrEnum | base.py:139 | task_assignment/priority_change/blocker_escalation/review_request/…/a2a_request | -| `Journal` | Pydantic model | journal.py:75 | Agent's personal journal; entries_by_type, latest_summary | -| `JournalEntry` | Pydantic model | journal.py:25 | Reflection/learning/struggle/decision entry with embedding + `is_private` | -| `JournalEntryType` | StrEnum | base.py:172 | task_reflection/decision_log/learning/struggle/general | -| `Playbook` | Pydantic model | playbook.py:17 | Curated procedure (draft→approved/archived); `from_attributes` for ORM load | -| `PlaybookStatus` | StrEnum | base.py:112 | draft/approved/archived | -| `AuditEventType` | StrEnum | audit.py:13 | permission_denied/unauthorized_access/role_changed/pm_override/… | -| `A2ATask` | Pydantic model | a2a.py:265 | A2A protocol work unit (maps to internal TaskTable) | -| `A2AMessage` | Pydantic model | a2a.py:212 | A2A communication turn with `Part` union (text/file/data/artifact) | -| `AgentCard` | Pydantic model | a2a.py:103 | A2A agent discovery card (published at /.well-known/agent.json) | -| `A2AConversation` | Pydantic model | a2a.py:468 | Persistent agent-pair conversation (canonical agent_a Post-snapshot updates (since 2026-06-29): 4 additional commits touched `roboco/models/`. -> - **c71f9b3b** `[chore] logical-gaps: kanban board column coverage + status-class fixes` — `kanban.py`: `DEV_COLUMNS` expanded from 7 to 15 entries (all `TaskStatus` values now covered); `QA_COLUMNS` dropped the `VERIFYING→"In Review"` entry (VERIFYING is the dev's self-check, not a QA state); `PM_COLUMNS` widened to include all gate/revision/paused/cancelled/backlog columns. No model API surface change; internal column configs only. -> - **d8a5bb48** `[chore] logical-gaps: a2a service hierarchy gate + persist skill on message row` — `a2a.py`: `A2AChatMessage` gained `skill: str | None` field (migration 054 adds the DB column). The `send()` verb now persists which capability the A2A is about so the receiver and inbox can surface it. -> - **536bbb64** `[chore] logical-gaps sweep (#286)` — `task.py`: `DocRef` gained `commit_status: str | None` (whether the doc reached the project repo: `committed`/`skipped`/`failed`); this 8-line insertion shifts all subsequent task.py line numbers by +8 vs the baseline annotations above. `playbook.py`: `Playbook` gained `archived_by: UUID | None` and `archived_at: datetime | None` to support the archive curation path. -> - **76ce53e3** `[fix] chat: wire live message delivery end-to-end (MESSAGE_SENT)` — `events.py`: added `EventType.MESSAGE_SENT = "message.sent"`; the bridge forwarded it to `/ws/sessions/{id}` and `/ws/channels/{id}` subscribers. **Now removed** by the comms-subsystem teardown (`docs/internal/specs/2026-07-03-comms-teardown-trace.md`) — the `/ws/sessions`/`/ws/channels` bridge and the tables it served are gone; `EventType.MESSAGE_SENT` itself is left as a dead/inert enum member, not deleted. - -## Regression Risks - -| Title | File:Line | Claim | Severity | -|-------|-----------|-------|----------| -| `TaskCreate._exactly_one_target` 3-way validator | task.py:405 | Tests/clients asserting the old 2-way error message ("a task needs either a project_id… or a product_id…") will fail against the new message ("a task needs exactly one target: … or cell_projects …"). Any caller that constructed a `TaskCreate` with both `project_id` and `product_id` was already rejected; callers passing `cell_projects` + another target are newly rejected. | Medium | -| `Task.cell_projects` requires migration 052 | task.py:179 | The `cell_projects` field exists on the model regardless of DB state, but persistence (`TaskTable.cell_projects` relationship + `task_cell_projects` table) needs migration 052. On a DB where 052 hasn't run, `TaskService.create` with a non-empty `cell_projects` will fail at insert. | Medium | -| `SpawnGitContext.task_short_id` consumer parity | runtime.py:38 | The field is additive with a `None` default, but every spawn-path consumer that should route the agent into the per-task worktree must read it; a missed consumer silently falls back to the clone root (the old behavior). | Low | -| Ollama catalog defaults vs persisted assignments | llm_catalog.py:103 | `OLLAMA_DEFAULT_MODEL` changed, but spawn reads persisted `model_assignments` rows — so the default only applies when no row exists. An operator who deletes the `model_assignments` rows (the documented "kill stale fleet model" procedure) will now get kimi-k2.7-code instead of minimax-m3. Verify the tag actually works on the Ollama Cloud plan before relying on this. (`OLLAMA_ROLE_DEFAULTS` was removed 2026-07-17 as dead code — it was never consulted by routing.) | Low | -| `ProductCellMapping` import cycle risk | task.py:24 | `task.py` now imports from `product.py` at module load. `product.py` imports only from `foundation.identity` and `base.py` — no cycle today, but a future `product.py` → `task.py` import would create one. | Low | -| `AgentRole`/`Team` alias removal pending | base.py:21–24 | The aliases to `foundation.identity` are a migration shim ("Removed in Phase 4 housekeeping"). Consumers still importing `AgentRole`/`Team` from `roboco.models.base` will break when the shim is removed. | Low | - -## Health - -The models package is coherent and well-layered: a single `RobocoBase` config drives consistency, enums are centralized in `base.py` (with the `AgentRole`/`Team` alias-to-`foundation.identity` migration clearly commented), and the API/Pydantic vs runtime/dataclass split is explicit (`agent.py` vs `agents.py`, with a docstring calling it out). The recent MegaTask per-cell-map change is small, additive, and validator-guarded; the main follow-on risk is migration 052 parity and test assertions on the renamed validator message — both mechanical. Two long-standing cleanliness items linger: `handoff.py` is a reserved-but-unwired model (file header says so), and several files (`dashboard.py`, `transcription.py`, `extraction.py`) are pure dataclasses that read as service-layer DTOs rather than domain models — harmless but slightly muddies the "models = typed contract surface" framing. Enum parity with the ORM (`tables.py`) is enforced by the test gate. No blocking issues; the package is in good shape. - -# db-migrations slice - -## Purpose -The DB layer is async SQLAlchemy 2.0 over PostgreSQL+asyncpg, with pgvector for the in-house RAG engine. Schema evolution is owned by an Alembic chain (001→061) that runs on every boot via `init_db()`; `Base.metadata.create_all` is no longer the source of truth — migration 017 reconciled the drift the other way. The ORM tables live in one fat module `roboco/db/tables.py` (~2.5k lines, 38 tables). - -## Files - -| Path | Role | -|------|------| -| `roboco/db/__init__.py` | Re-exports `Base`, session helpers, `bootstrap_database`, table classes. | -| `roboco/db/base.py` | `Base` (DeclarativeBase + naming convention), `get_engine`, `get_session_factory`, `get_db` (FastAPI dep), `get_db_context`, `run_migrations`, `init_db`, `_db_has_*` probes. Stamps a pre-Alembic DB at 001 then upgrades head. | -| `roboco/db/tables.py` | All 37 ORM table classes (single module). | -| `roboco/db/seed.py` | `bootstrap_database()` — runs `init_db` then seeds agents. | -| `alembic/env.py` | Async Alembic env; imports `roboco.db.tables` to register metadata, overrides `sqlalchemy.url` from settings, `compare_type` + `compare_server_default` on. | -| `alembic.ini` | Standard config; `script_location=alembic`, `prepend_sys_path=.`, no URL (set in env.py). | -| `alembic/versions/` | 61 migration files 001..061 (two share number 026 — chained, not a collision). | - -## Key Symbols - -| Name | Kind | File:Line | Responsibility | -|------|------|-----------|----------------| -| `Base` | class | db/base.py:38 | DeclarativeBase + MetaData naming convention. | -| `get_engine` | fn | db/base.py:46 | Lazy singleton async engine (pool_pre_ping). | -| `get_db` | fn | db/base.py:70 | FastAPI async session dependency. | -| `get_db_context` | fn | db/base.py | Out-of-request async session context. | -| `init_db` | fn | db/base.py:180 | Boot entry: stamp pre-Alembic DB at 001 then `run_migrations`. | -| `run_migrations` | fn | db/base.py:141 | Runs `alembic upgrade head` via `command.upgrade` in a thread. | -| `bootstrap_database` | fn | db/seed.py:282 | `init_db` + seed default agents. | -| `TaskTable` | class | tables.py:157 | Core task entity (largest table, drives lifecycle). | -| `WorkSessionTable` | class | tables.py:798 | Per-claim session; single-active enforced by 047 partial-unique index. | -| `AgentTable` | class | tables.py:95 | Agent identity, role, team, model provider assignment. | -| `ProjectTable` | class | tables.py:475 | Git repo config + CI/watch/dep-update/quality_command/`sandbox_services` (057) cols. | -| `AuditLogTable` | class | tables.py:1940 | Transition journey; `details` JSONB (010); composite query index (045). | -| `AgentSpawnSessionTable` | class | tables.py:2170 | Per-spawn token totals; feeds usage dashboard. | -| `ProjectConventionsCacheTable` | class | tables.py:2442 | Effective conventions map per (project, HEAD sha). | -| `PlaybookTable` | class | tables.py:721 | Curated procedures (draft→approved→indexed). | -| `RespawnTrackerTable` | class | tables.py:1902 | Durable PM-respawn circuit breaker mirror. | -| `TaskCellProjectTable` | class | tables.py:632 | Per-cell project map for a MegaTask root-subtask (052). | -| `WaitingRecordTable` | class | tables.py:1872 | Persisted dispatcher waiting records (restore at start). | -| `IndexedDocumentTable` | class | tables.py:1651 | RAG corpus docs (added to chain by 017). | -| `UserTable` | class | tables.py:2603 | Cloud-auth (FastAPI Users) single seeded CEO login row (058). | -| `XCredentialsTable` | class | tables.py:2650 | Singleton Fernet-encrypted OAuth 1.0a secrets for the X engine (059). | -| `XSeenMentionTable` | class | tables.py:2675 | X mentions-poll dedup ledger, keyed by mention id (059). | -| `XSeenFeatureTable` | class | tables.py:2264 | X feature-spotlight dedup ledger, keyed by feature slug (061). | -| `run_async_migrations` | fn | env.py | Async online migration runner (NullPool). | - -## Migration Chain - -| Num | File | What it adds/changes | -|-----|------|---------------------| -| 001 | 001_initial_schema.py | All initial tables (agents, tasks, work_sessions, notifications, journals, audit_log, a2a_*); also originally created channels/sessions/messages, later dropped by the comms-teardown migration. | -| 002 | 002_persistence_tables.py | Persistence tables + `NotificationType.APPROVAL`. | -| 003 | 003_blocker_resolver_type.py | `tasks.blocker_resolver_type` + `blockerresolvertype` enum. | -| 004 | 004_provider_routing.py | `provider_configs` + `model_assignments`; `modelprovider`/`assignmentscope` enums (create_type=False). | -| 005 | 005_blocker_raised_by.py | `tasks.blocker_raised_by`. | -| 006 | 006_gateway_columns.py | Gateway cols: claimant lock, heartbeat, pre-block snapshot, AC status, qa evidence flag. | -| 007 | 007_gateway_triggers_table.py | `gateway_triggers` (dispatcher decision log). | -| 008 | 008_align_skills.py | No-op (skills alignment done statically). | -| 009 | 009_enum_reconcile.py | Reconcile every postgres enum with ORM StrEnum (lowercase) + new members. | -| 010 | 010_audit_log_details_jsonb.py | `audit_log.details` JSON→JSONB. | -| 011 | 011_drop_quarantined_state.py | Drop `quarantined` from taskstatus enum (phantom state, audit D15). | -| 012 | 012_align_agentrole_foundation.py | Add agentrole/team enum values foundation declares. | -| 013 | 013_drop_role_enum.py | Drop stray `role` enum (smoke run 2). | -| 014 | 014_drop_pm_approvals.py | Drop unused `tasks.pm_approvals`. | -| 015 | 015_drop_task_execution_outputs.py | Drop unused `execution_log`/`outputs`. | -| 016 | 016_add_products_and_task_product_id.py | `products` + `product_projects`; `tasks.product_id` (team enum create_type=False). | -| 017 | 017_reconcile_orm_schema_drift.py | Add ORM tables/columns the chain never had (e.g. `indexed_documents`). | -| 018 | 018_task_project_id_nullable.py | `tasks.project_id` nullable (board/fan-out tasks carry product_id). | -| 019 | 019_seed_default_providers.py | Idempotent seed of default model providers. | -| 020 | 020_backfill_enum_values.py | Backfill ORM enum values the chain never added. | -| 021 | 021_task_board_review_complete.py | `tasks.board_review_complete` (board-review handoff flag). | -| 022 | 022_default_branch_master.py | Flip `projects.default_branch` default `main`→`master`. | -| 023 | 023_prompter_tracking_columns.py | `tasks.source` + `confirmed_by_human` (prompter origin). | -| 024 | 024_add_prompter_tables.py | `prompter_sessions`, `prompter_messages`, `task_drafts`. | -| 025 | 025_agentrole_prompter.py | Add `prompter` to agentrole enum. | -| 026a | 026_completed_dependency_ids.py | `tasks.completed_dependency_ids`. | -| 026b | 026_token_usage_tables.py | `agent_spawn_sessions` + `token_usage_snapshots` (chained off 026a). | -| 027 | 027_system_settings.py | `system_settings` key-value store. | -| 028 | 028_seed_self_hosted_provider.py | Seed Self-Hosted (Ollama LOCAL) provider row. | -| 029 | 029_project_quality_command.py | `projects.quality_command` (fast pre-submit gate). | -| 030 | 030_rag_chunks_content_schema.py | Align RAG chunk tables with vector-store schema. | -| 031 | 031_rag_chunks_fulltext.py | tsvector + GIN index on every chunk table (hybrid retrieval). | -| 032 | 032_company_goals.py | `company_goals` singleton charter. | -| 033 | 033_pitches.py | `pitches` (Board proposals → auto-provision). | -| 034 | 034_agentrole_secretary.py | Add `secretary` to agentrole enum. | -| 035 | 035_secretary_directives.py | `secretary_directives` (command audit + gate queue). | -| 036 | 036_ac_ids_and_parent_refs.py | Per-criterion AC ids + child→parent AC linkage. | -| 037 | 037_agentrole_pr_reviewer.py | Add `pr_reviewer` to agentrole enum. | -| 038 | 038_modelprovider_grok.py | Add `grok` to modelprovider enum. | -| 039 | 039_seed_grok_provider.py | Seed Grok (xAI) provider row. | -| 040 | 040_awaiting_pr_review.py | Add `awaiting_pr_review` to taskstatus enum (PR-review gate). | -| 041 | 041_structured_content_columns.py | `pr_reviewer_notes`, machine-marker split, structured content cols. | -| 042 | 042_worksession_toolchain.py | `work_sessions` toolchain matching cols. | -| 043 | 043_conventions_cache.py | `project_conventions_cache`. | -| 044 | 044_convention_findings.py | `project_convention_findings` (violations feed). | -| 045 | 045_observability_rework.py | `tasks.revision_count` + audit_log composite query index. | -| 046 | 046_batch_intake.py | `tasks.batch_id` + collision descriptors (intends_to_touch, adds_migration, touches_shared). | -| 047 | 047_ws_single_active.py | Partial-unique index: one ACTIVE work_session per task. | -| 048 | 048_ci_watch_project_cols.py | Per-project CI-watch opt-in cols. | -| 049 | 049_dep_update_project_cols.py | Per-project dep-update bot opt-in cols. | -| 050 | 050_playbooks.py | `playbooks` table (curated procedures). | -| 051 | 051_respawn_tracker.py | `respawn_tracker` (durable PM-respawn counter). | -| 052 | 052_task_cell_projects.py | `task_cell_projects` (per-cell project map for MegaTask root-subtask; reuses team enum create_type=False). | -| 053 | 053_playbook_archived_attr.py | `playbooks.archived_by` (UUID) + `playbooks.archived_at` (DateTime) — distinct retirement attribution; keeps `approved_by`/`approved_at` as approval-only provenance. | -| 054 | 054_a2a_message_skill.py | `a2a_messages.skill` (String 100, nullable) — persists the capability a directed A2A message concerns; was silently dropped on send. | -| 055 | 055_spawn_session_turns_tool_calls.py | `agent_spawn_sessions.turns` + `.tool_calls` (BigInteger, DEFAULT 0) — per-stint LLM iterations + tool invocations for the granular per-member performance metrics. | -| 056 | 056_member_perf_daily.py | `member_performance_daily` — one row per (date, member_kind, agent_slug) scorecard rollup (incl. CEO as `member_kind='ceo'`). | -| 057 | 057_project_sandbox_services.py | `projects.sandbox_services` (ARRAY(String), nullable) — per-project opt-in for the sandboxed per-agent-spawn engine provisioner (postgres / redis / mongo via the `SANDBOX_ENGINES` registry). | -| 058 | 058_cloud_auth_users.py | `users` table (FastAPI Users schema) — the single seeded CEO login for cloud auth (`ROBOCO_CLOUD_AUTH_ENABLED`, default off). | -| 059 | 059_x_credentials.py | `x_credentials` (singleton Fernet-encrypted OAuth 1.0a secrets) + `x_seen_mentions` (mentions-poll dedup ledger) — the X (Twitter) engine (`ROBOCO_X_ENGINE_ENABLED`, default off). | -| 060 | 060_drop_messaging.py | Drops the channels/groups/sessions/session_tasks/messages subsystem (comms teardown — A2A is now the sole directed-message channel): `journal_entries.session_id` column, the 5 tables, and 4 enum types (`messagetype`/`sessionstatus`/`sessionscope`/`channeltype`); one-way (`downgrade()` raises `NotImplementedError`). | -| 061 | 061_x_feature_spotlight.py | `x_seen_features` (feature-spotlight dedup ledger, keyed by feature slug) + `company_goals.brand_voice` (Text, CEO-authored brand-voice sample, feeds `_voice_guide`) — X feature-spotlight (`ROBOCO_X_FEATURE_SPOTLIGHT_ENABLED`, default off, sub-switch of `x_engine_enabled`). | -| 062-072 | *(not yet reflected in this table — pre-existing gap, out of scope for this pass)* | Vault V1/V2, revision-findings ledger, sandbox extensions, and other slices landed migrations in this range; see `alembic/versions/` directly until this table is backfilled. | -| 073 | 073_project_environments.py | `projects.environments` (nullable JSONB) — the per-project ordered environment ladder (`list[{name, branch}]`, index 0 = head rung, index -1 = prod rung) that replaces `default_branch` as the source of truth for a project's PR target and release target. Additive: a null value falls back to a degenerate single-branch ladder synthesized from `default_branch` at read time (`roboco/models/env_branches.py`), so existing projects are unaffected until the CEO declares a real ladder. | -| 074 | 074_telegram_credentials.py | `telegram_credentials` (singleton Fernet-encrypted `bot_token_encrypted` + `chat_id_encrypted`, mirrors `x_credentials`) — the Telegram notifications bridge (`ROBOCO_TELEGRAM_ENABLED`, default off). | - -## Data Flow -On boot, `init_db()` probes for application tables and `alembic_version`; if a pre-Alembic DB exists it stamps it at revision 001, then always runs `run_migrations()` → `alembic upgrade head` (in a thread via `asyncio.to_thread`). `env.py` imports `roboco.db.tables` so `Base.metadata` is fully populated, overrides `sqlalchemy.url` from `settings.database_url`, and runs online with an async NullPool engine. `compare_type` + `compare_server_default` are on so autogenerate drift is detectable. `tables.py` classes are the ORM mapping the migrations build; the domain layer reads them through `roboco/models/` dataclasses, not the tables directly. - -## Mermaid - -```mermaid -graph LR - 001-->002-->003-->004-->005-->006-->007-->008-->009-->010 - 010-->011-->012-->013-->014-->015-->016-->017-->018-->019 - 019-->020-->021-->022-->023-->024-->025-->026a-->026b-->027 - 027-->028-->029-->030-->031-->032-->033-->034-->035-->036 - 036-->037-->038-->039-->040-->041-->042-->043-->044-->045 - 045-->046-->047-->048-->049-->050-->051-->052-->053-->054 - 054-->055-->056-->057-->058-->059-->060-->061 -``` - -## Logical Tree - -``` -Migration chain 001..059 -├── Initial schema -│ └── 001 initial schema (agents, tasks, work_sessions, notifications, journals, audit_log, a2a_*; also originally channels/sessions/messages, later dropped by the comms-teardown migration) -├── Persistence -│ └── 002 persistence tables + NotificationType.APPROVAL -├── Blocker metadata -│ ├── 003 blocker_resolver_type + blockerresolvertype enum -│ └── 005 blocker_raised_by -├── Provider routing & model assignments -│ ├── 004 provider_configs + model_assignments (modelprovider/assignmentscope enums) -│ ├── 019 seed default model providers -│ ├── 028 seed Self-Hosted (Ollama LOCAL) provider -│ ├── 038 add grok to modelprovider enum -│ └── 039 seed Grok (xAI) provider -├── Gateway -│ ├── 006 gateway columns (claimant lock, heartbeat, pre-block snapshot, AC status, qa evidence) -│ └── 007 gateway_triggers (dispatcher decision log) -├── Enum reconcile / widening -│ ├── 009 reconcile postgres enums with ORM StrEnum -│ ├── 011 drop quarantined from taskstatus enum -│ ├── 012 align agentrole/team enums with foundation -│ ├── 013 drop stray role enum -│ ├── 020 backfill ORM enum values -│ ├── 025 add prompter to agentrole enum -│ ├── 034 add secretary to agentrole enum -│ └── 037 add pr_reviewer to agentrole enum -├── Audit log -│ ├── 010 audit_log.details JSON→JSONB -│ └── 045 tasks.revision_count + audit_log composite query index -├── Cleanup / drops -│ ├── 014 drop unused tasks.pm_approvals -│ ├── 015 drop unused execution_log/outputs -│ └── 008 no-op (skills alignment done statically) -├── Products -│ ├── 016 products + product_projects; tasks.product_id (team enum create_type=False) -│ └── 018 tasks.project_id nullable -├── ORM drift reconcile -│ └── 017 add ORM tables/columns the chain never had (indexed_documents) -├── Board review -│ └── 021 tasks.board_review_complete -├── Project defaults -│ └── 022 flip projects.default_branch default main→master -├── Prompter tracking -│ ├── 023 tasks.source + confirmed_by_human -│ └── 024 prompter_sessions, prompter_messages, task_drafts -├── Dependency / token usage -│ ├── 026a tasks.completed_dependency_ids -│ └── 026b agent_spawn_sessions + token_usage_snapshots (chained off 026a) -├── System settings -│ └── 027 system_settings key-value store -├── Project quality -│ └── 029 projects.quality_command (fast pre-submit gate) -├── RAG -│ ├── 030 align RAG chunk tables with vector-store schema -│ └── 031 tsvector + GIN index on chunk tables (hybrid retrieval) -├── Strategy / provisioning -│ ├── 032 company_goals singleton charter -│ └── 033 pitches (Board proposals → auto-provision) -├── Secretary -│ └── 035 secretary_directives (command audit + gate queue) -├── Acceptance criteria -│ └── 036 per-criterion AC ids + child→parent AC linkage -├── PR review -│ ├── 040 add awaiting_pr_review to taskstatus enum -│ └── 041 pr_reviewer_notes, machine-marker split, structured content cols -├── Worksession toolchain -│ └── 042 work_sessions toolchain matching cols -├── Conventions standard -│ ├── 043 project_conventions_cache -│ └── 044 project_convention_findings (violations feed) -├── MegaTask / batch intake -│ ├── 046 tasks.batch_id + collision descriptors -│ └── 052 task_cell_projects (per-cell project map for MegaTask root-subtask) -├── WorkSession single-active -│ └── 047 partial-unique index: one ACTIVE work_session per task -├── Autonomous maintenance -│ ├── 048 per-project CI-watch opt-in cols -│ └── 049 per-project dep-update bot opt-in cols -├── Organizational memory -│ ├── 050 playbooks table (curated procedures) -│ └── 053 playbooks.archived_by + archived_at (distinct retirement attribution from approval) -├── Orchestrator runtime durability -│ └── 051 respawn_tracker (durable PM-respawn counter) -├── A2A messaging -│ └── 054 a2a_messages.skill (nullable; persists directed-A2A capability context) -├── Per-member performance metrics -│ ├── 055 agent_spawn_sessions.turns + .tool_calls (DEFAULT 0) -│ └── 056 member_performance_daily (per date/member_kind/agent_slug rollup) -├── Sandboxed dev DB/Redis -│ └── 057 projects.sandbox_services (per-project opt-in array) -├── Cloud auth -│ └── 058 users (FastAPI Users; single seeded CEO login) -├── X (Twitter) engine -│ ├── 059 x_credentials (singleton encrypted OAuth 1.0a) + x_seen_mentions (dedup ledger) -│ └── 061 x_seen_features (feature-spotlight dedup ledger) + company_goals.brand_voice -└── Comms teardown - └── 060 drop channels/groups/sessions/session_tasks/messages + journal_entries.session_id (A2A is now the sole directed-message channel; one-way, no downgrade) -``` - -## Dependencies -- PostgreSQL 15+ (NULLS NOT DISTINCT) — actually pgvector image on PG 16. -- `pgvector` extension for RAG cosine similarity (`chunks_*` tables, `indexed_documents`). -- `asyncpg` driver; SQLAlchemy 2.0 async. -- Alembic; migrations run on every orchestrator boot. - -## Entry Points -- `init_db()` / `run_migrations()` in `roboco/db/base.py` — boot-time `alembic upgrade head`. -- `bootstrap_database()` in `roboco/db/seed.py` — init + seed. -- `alembic upgrade head` (manual, in orchestrator container). -- `conftest` (tests) — ephemeral DB per test; runs migrations or `create_all` depending on PG availability. - -## Config Flags -- `ROBOCO_DATABASE_*` (host/port/user/password/name) — `settings.database_url`. -- `ROBOCO_DATABASE_ECHO`, pool size/timeout/recycle. -- No DB-specific feature flag; migrations always run. Feature flags (`ROBOCO_CONVENTIONS_ENABLED`, `ROBOCO_CI_WATCH_ENABLED`, `ROBOCO_DEP_UPDATE_ENABLED`, `ROBOCO_RELEASE_MANAGER_ENABLED`, `ROBOCO_ORG_MEMORY_ENABLED`) gate *use* of tables the migrations already added. - -## Gotchas -- **`sa.Enum(create_type=False)` is silently ignored** — the flag only works on `postgresql.ENUM`, not generic `sa.Enum`. Using `sa.Enum` re-emits CREATE TYPE and fails with "type already exists". 001/016/052 carry the live gotcha comment; 004/016 use the correct `postgresql.ENUM(create_type=False)`. 001 itself uses `sa.Enum(..., create_type=False)` in spots — latent on a clean re-apply. -- **Enum-parity gate can false-green** — `make quality` runs `scripts/verify_postgres_enums.py` only against a migrated DB; an empty `roboco` DB (conftest ephemeral) or `|| echo` masking hides drift. Fixed in 957fb522 but the gate is only as good as the DB it points at. -- **016 latent** — the `team` enum member list under create_type=False is the *original* set, not the later-widened set; inert but misleading. -- **Two files numbered 026** — not a collision: `026_token_usage_tables` chains off `026_completed_dependency_ids`. Renaming is risky (breaks down_revision refs). -- **052 reuses the `team` enum** with `create_type=False` correctly — no new enum added; safe. -- **017 reconciled drift the other way** — added ORM tables the chain had missed; `create_all` is no longer authoritative. - -## Drift from CLAUDE.md -- CLAUDE.md says "52 migrations 001..052" — now stale; chain is 001..059 (59 files). Does not mention the two 026 files (chained, not a conflict). -- CLAUDE.md cites migrations 043/046/047/048/049/050/051 by number in feature sections — all present and consistent. -- No factual drift found in the DB layer description. - -## Changes Since Baseline -`git log fd10cc862c2020b3f639cdb686d427b0198a2441..HEAD -- alembic/ roboco/db/`: -- `15effce0` Chore: 141 Gaps fill-in (#283) — adds migration 052 (`task_cell_projects`) + `TaskCellProjectTable`; logic-touching. - -(Only one commit in range touches these paths.) - -> Post-snapshot updates (since 2026-06-29): `536bbb64` (Chore/all/logical gaps sweep #286) — adds migration 053 (`playbooks.archived_by`/`archived_at`), two new columns on `PlaybookTable`; `d8a5bb48` ([chore] a2a hierarchy gate + skill persist) — adds migration 054 (`a2a_messages.skill`), one new column on `A2AMessageTable`, wired through `send_chat_message` and the A2AChatMessage model. -> -> Delta 2026-07-03 (v0.17.0, 5 features): `055_spawn_session_turns_tool_calls` (`agent_spawn_sessions.turns`/`.tool_calls`) + `056_member_perf_daily` (`member_performance_daily`) predate this wave but were never appended to this doc; `057_project_sandbox_services` adds `projects.sandbox_services` (sandboxed dev DB/Redis/Mongo, `ROBOCO_SANDBOX_DB_ENABLED`); `058_cloud_auth_users` adds `users` (`UserTable`, cloud auth, `ROBOCO_CLOUD_AUTH_ENABLED`); `059_x_credentials` adds `x_credentials` (`XCredentialsTable`) + `x_seen_mentions` (`XSeenMentionTable`) (X engine, `ROBOCO_X_ENGINE_ENABLED`). Chain head is now 059. Mongo rides existing 057 (no new migration) — it's just another entry in the `SANDBOX_ENGINES` registry. -> -> Delta 2026-07-04 (v0.18.0): `060_drop_messaging` (the comms-teardown migration — drops `messages`/`session_tasks`/`sessions`/`groups`/`channels` + 4 enum types + `journal_entries.session_id`; A2A is now the sole directed-message channel; one-way, `downgrade()` raises `NotImplementedError`) had already landed on master but was never appended to this doc; `061_x_feature_spotlight` adds `x_seen_features` (`XSeenFeatureTable`) + `company_goals.brand_voice` (X feature-spotlight, `ROBOCO_X_FEATURE_SPOTLIGHT_ENABLED`, sub-switch of `x_engine_enabled`). Chain head is now 061. ORM table count is now 38 (verified via `grep -c '^class .*Table' roboco/db/tables.py`), up from this doc's previously-stated 37 (that figure predates 055-061 and was never recomputed). - -## Regression Risks - -| Title | File:Line | Claim | Severity | -|-------|-----------|-------|----------| -| sa.Enum create_type silently dropped | alembic/versions/001_initial_schema.py:127,312 | 001 uses `sa.Enum(..., create_type=False)` which is a no-op on the generic Enum; a fresh re-apply on a clean DB can double-emit CREATE TYPE. | High | -| Enum-parity gate false-green | Makefile:540 + scripts/verify_postgres_enums.py | Gate skips on no-migrated-DB; an empty/mismatched `roboco` DB hides postgres-enum drift until a smoke run. | High | -| 016 team enum stale member list | alembic/versions/016_add_products_and_task_product_id.py:38 | `postgresql.ENUM(create_type=False)` member list is frozen at the original set; inert but masks later widening. | Medium | -| Missing pgvector extension blocks RAG | roboco/db/tables.py (chunks_*/indexed_documents) | Migrations assume pgvector installed; on a plain PG the vector columns fail and init_db aborts. | High | -| Two 026 files — rename hazard | alembic/versions/026_*.py | Renaming either 026 file breaks `down_revision` chain; autogenerate may mis-order. | Medium | -| 047 partial-unique index assumes single-active | alembic/versions/047_ws_single_active.py | A duplicate ACTIVE session raises on the partial-unique index; service-layer guard must run first or claim crashes. | Medium | -| 052 reuses team enum — order-dependent | alembic/versions/052_task_cell_projects.py:44 | Depends on `team` enum already existing (from 001/016); a partial chain replay to 052 without 016 would fail. | Low | -| Single-head violation on re-apply | alembic/versions/017_reconcile_orm_schema_drift.py | 017 adds tables/columns that `create_all` had created; on a DB built by `create_all` then stamped, 017 may double-create. | Medium | -| Migration 060 is a one-way removal with no downgrade | alembic/versions/060_drop_messaging.py:57-61 | `downgrade()` raises `NotImplementedError` — recreating channels/groups/sessions/session_tasks/messages + 4 enum types would need the full original schema. Any rollback plan for a bad deploy past 060 must restore from a pre-060 DB backup, not `alembic downgrade`. | Low | - -## Health -The chain is linear and complete (001→061), with `init_db` running `upgrade head` on every boot so deployed schemas stay current. The two structural risks are the `sa.Enum(create_type=False)` no-op in 001 (latent on clean re-applies) and the enum-parity gate's dependence on a populated migrated DB. New migrations consistently use the `postgresql.ENUM(create_type=False)` pattern and `ALTER TYPE ... ADD VALUE IF NOT EXISTS` for enum widening, so recent additions are safe. - -# RoboCo Slice Map — `api-core-websocket` - -Slice key: `api-core-websocket` Repo root: `/Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco` Baseline commit: `fd10cc862c2020b3f639cdb686d427b0198a2441` Head: `15effce0` (2026-06-29, "Chore: 141 Gaps fill-in (#283)") - -## Purpose - -The FastAPI application shell, request pipeline, and real-time WebSocket fan-out layer for RoboCo. `app.py` builds the ASGI app, wires ~40 route routers, and runs the async lifespan (DB migrations, feature-flag overlay, transcription/extraction/RAG/learning service init, ordered shutdown). `middleware.py` adds correlation IDs, request logging, and a full exception-handler chain mapping domain/service/HTTP errors to structured JSON. `websocket.py` + `websocket_bridge.py` own the live panel streams (agents, notifications, system) with per-connection bounded send queues and an event-bus bridge. `deps.py` is the dependency-injection spine: agent header auth, role-gate helpers, and Choreographer/ContentActions wiring. `utils/` provides route-layer error factories and get-or-404/ownership helpers. `middleware_docs.py` enforces the docs-path permission matrix. `roboco/security.py` (outside `api/` but wired here) supplies the optional fastapi-guard HTTP security layer: `apply_guard(app)` mounts `SecurityMiddleware` last — outermost — in `create_app`, and `guarded_lifespan(lifespan)` wraps the async lifespan, both gated by `ROBOCO_GUARD_ENABLED` (default off, byte-for-byte unchanged request path while off). - -## Files - -| Path | Role | approx LOC | -|------|------|-----------| -| `roboco/api/app.py` | FastAPI app factory + async lifespan (startup/shutdown ordering) | ~490 | -| `roboco/api/deps.py` | DI: agent header auth, role gates, Choreographer/ContentActions builders, pagination | ~611 | -| `roboco/api/middleware.py` | Correlation-id + request-logging middleware, exception handlers, 422 secret-scrub + UUID remediation | ~469 | -| `roboco/api/middleware_docs.py` | Docs-path access-control matrix (read/write per role/team/slug) | ~330 | -| `roboco/api/websocket.py` | WS routes + ConnectionManager with bounded per-connection send queues, panel-token gate, idle timeout | ~692 | -| `roboco/api/websocket_bridge.py` | Event-bus → WS forwarders (notifications, agents, rate-limit, usage, A2A) | ~252 | -| `roboco/api/utils/__init__.py` | Re-export surface for error factories + resource helpers | ~43 | -| `roboco/api/utils/errors.py` | HTTPException factories + `handle_service_error` + `service_error_handler` decorator | ~214 | -| `roboco/api/utils/resources.py` | `get_or_404`, `get_by_field_or_404`, `require_ownership`/`require_recipient`/`require_membership` | ~180 | -| `roboco/api/__init__.py` | Deliberately does NOT re-export `app` (circular-import guard, documented) | ~14 | -| `roboco/security.py` | fastapi-guard 7.2.1 / guard-core 3.3.0 HTTP security layer: `SecurityMiddleware` + `guard_deco` (`SecurityDecorator`) singleton, gated by `ROBOCO_GUARD_ENABLED` (default off); wired into `create_app` via `apply_guard`/`guarded_lifespan` | ~407 | - -## Key Symbols - -| Name | Kind | File:Line | Responsibility | -|------|------|-----------|----------------| -| `lifespan` | async ctx mgr | app.py:82 | Startup: migrations, flag overlay, transcription/extraction/optimal/learning init; Shutdown: stop orchestrator BEFORE close_db, then close optimal, then DB | -| `create_app` | func | app.py:196 | Build FastAPI, add CORS + custom middleware, mount ~40 routers + ws_router at `/ws` | -| `app` | module attr | app.py:489 | The default ASGI instance (`roboco.api.app:app` entrypoint) | -| `_AppServices` | class | app.py:74 | Holder for transcription/extraction singletons set in lifespan | -| `DbSession` | type alias | deps.py:46 | `Annotated[AsyncSession, Depends(get_db)]` | -| `resolve_agent_id` | func | deps.py:49 | Resolve UUID-or-slug → UUID, 400 on miss | -| `_ServiceHolder` | class | deps.py:74 | Singleton store for PermissionService + orchestrator | -| `set_orchestrator`/`clear_orchestrator`/`get_orchestrator`/`get_orchestrator_or_none` | funcs | deps.py:91-119 | Global orchestrator accessors; `get_orchestrator` 503s when unset, `_or_none` used by shutdown | -| `get_current_agent_id`/`get_current_agent_slug`/`get_optional_agent_id` | funcs | deps.py:125-208 | Header-based agent identity (UUID or slug) | -| `_auth_required` | func | deps.py:211 | Reads `ROBOCO_AGENT_AUTH_REQUIRED` env | -| `_check_agent_auth_token` | func | deps.py:217 | HMAC token enforcement (required in prod, optional-but-verified in dev) | -| `require_panel_token` | func | deps.py:251 | CEO-HMAC gate for live-chat bridges (HTTP analog of WS gate) | -| `_resolve_agent_identity` | func | deps.py:277 | Returns `(agent_id, slug)`, special-casing `system` role | -| `_coerce_agent_role`/`_coerce_agent_team` | funcs | deps.py:298/324 | Parse role/team headers with DB fallback for role | -| `_header_trust_agent_context` | func | deps.py:340 | The original `get_agent_context` body verbatim (header-trust); the OFF-mode path, and also what a valid agent HMAC token delegates to when cloud auth is ON | -| `_slide_session_cookie` | func | deps.py:379 | Re-mints + re-sets the session cookie on every cookie-authenticated request — the sliding 30-day window (only inactivity past `cloud_auth_cookie_max_age` logs out) | -| `_cloud_auth_agent_context` | func | deps.py:390 | Dual-path enforcement when `cloud_auth_enabled`: a valid HMAC token (any role) delegates to `_header_trust_agent_context`; otherwise a non-CEO role claim is rejected outright, and the CEO must present a valid session cookie via `resolve_session_user` | -| `get_agent_context` | func | deps.py:452 | Builds `AgentContext` from X-Agent-* headers + token (+ the `roboco_session` cookie); byte-for-byte header-trust when `cloud_auth_enabled` is False, else delegates to `_cloud_auth_agent_context` | -| `require_pm_or_above`/`require_developer_or_above`/`require_cell_access` | funcs | deps.py:403/428/443 | Coarse role-gate guards (403) | -| `require_ceo_role` | func | deps.py:412 | Single CEO-check: raises 403 unless `role` is CEO; accepts `AgentRole`/`Role`/lowercase string; unifies orchestrator-router + release-handler CEO gates into one source of truth (`536bbb64`) | -| `require_notification_permission`/`require_task_action` | dep factories | deps.py:490/508 | PermissionService-backed dependency factories | -| `get_choreographer`/`get_content_actions` | funcs | deps.py:545/575 | Build Choreographer / ContentActions with all service deps + orchestrator/bus | -| `get_pagination` | func | deps.py:599 | Clamp limit 1-100, offset ≥0 | -| `CorrelationIdMiddleware` | class | middleware.py:54 | X-Correlation-ID in/out + structlog bind | -| `RequestLoggingMiddleware` | class | middleware.py:96 | Log request/response + `X-Response-Time-Ms` header | -| `get_status_code`/`roboco_exception_handler`/`service_exception_handler`/`rate_limit_exception_handler`/`generic_exception_handler`/`http_exception_handler`/`request_validation_handler` | funcs | middleware.py:143-441 | Exception → structured JSONResponse chain | -| `_SERVICE_ERROR_STATUS` | const | middleware.py:191 | Maps service exception types → HTTP status | -| `_uuid_field_remediation` | func | middleware.py:343 | Actionable hint when an agent sends an 8-char task prefix as UUID | -| `_SECRET_FIELD_NAMES`/`_scrub_secrets` | const/func | middleware.py:372/389 | Redact credential fields from 422 log bodies | -| `setup_middleware` | func | middleware.py:444 | Register exception handlers + middleware in order | -| `DOCS_PERMISSIONS` | const | middleware_docs.py:55 | Path-prefix → read/write role matrix | -| `check_docs_access`/`require_docs_access`/`get_allowed_docs_paths` | funcs | middleware_docs.py:222/265/303 | Docs path permission checks | -| `_fast_path_access_decision` | func | middleware_docs.py:203 | CEO/auditor/main_pm short-circuit | -| `ConnectionManager` | class | websocket.py:82 | Tracks all WS subscriptions + per-connection send queues | -| `_ClientConnection` | class | websocket.py:45 | Bounded outbound queue + sender task holder | -| `manager` | singleton | websocket.py:334 | Global ConnectionManager | -| `_require_panel_token` | func | websocket.py:61 | WS upgrade CEO-HMAC gate (returns False to close) | -| `validate_agent_exists` | func | websocket.py:337 | DB existence check for claimed agent_id | -| `_register_sender`/`_run_sender`/`_enqueue_or_send`/`_send_with_timeout` | methods | websocket.py:122/129/244/270 | Non-blocking fan-out plumbing | -| `connect_agent`/`connect_notifications`/`connect_system` | methods | websocket.py:187/213/224 | Accept + register per stream type | -| `disconnect` | method | websocket.py:230 | Remove socket from all sets + cancel sender task | -| `broadcast_to_agent_watchers`/`broadcast_system` | methods | websocket.py:310/332 | Fan-out enqueues | -| `agent_stream`/`notification_stream`/`system_stream` | routes | websocket.py:445/573/625 | WS endpoints under `/ws` | -| `broadcast_agent_chunk`/`broadcast_notification` | funcs | websocket.py:650/665 | Helper broadcasters for external callers | -| `IDLE_TIMEOUT_SECONDS`/`MAX_SEND_QUEUE`/`SEND_TIMEOUT_SECONDS` | consts | websocket.py:35/41/42 | 90s / 256 / 10s tunables | -| `register_websocket_bridge_handlers`/`start_websocket_bridge` | funcs | websocket_bridge.py:169/203 | Subscribe bus handlers | -| `_handle_notification_sent`/`_handle_agent_event`/`_handle_rate_limit_event`/`_handle_usage_event`/`_handle_a2a_message_event` | funcs | websocket_bridge.py | Event-bus → WS forwarders (`_handle_a2a_message_event` fans `A2A_MESSAGE_SENT` to `/ws/system` as an `a2a.message` frame — the CEO's live view of every agent-to-agent chat) | -| `_RATE_LIMIT_WS_TYPES`/`_USAGE_WS_TYPES` | consts | websocket_bridge.py:18/23 | EventType → panel `type` string maps | -| `not_found`/`forbidden`/`unauthorized`/`validation_error`/`conflict`/`service_unavailable` | funcs | utils/errors.py:29-142 | HTTPException factories | -| `handle_service_error`/`service_error_handler` | func/deco | utils/errors.py:150/191 | ServiceError → HTTPException translation | -| `get_or_404`/`get_by_field_or_404` | funcs | utils/resources.py:17/59 | Generic get-or-404 helpers | -| `require_ownership`/`require_recipient`/`require_membership` | funcs | utils/resources.py:96/130/156 | Authorization checks | -| `apply_guard` | func | security.py:378 | Mounts `SecurityMiddleware` on `app` + sets `app.state.guard_decorator`; no-op unless `settings.guard_enabled` | -| `guarded_lifespan` | func | security.py:399 | Wraps `lifespan` with guard's `make_lifespan` (redis/geo/agent init) when armed; passthrough when off | -| `build_security_config` | func | security.py:329 | Assembles the global `SecurityConfig` from settings: passive_mode, fail_secure, enforce_https, WAF calibration fields | -| `security_config` / `guard_deco` | module singletons | security.py:374-375 | Built once at import (pure, no I/O); `guard_deco` is the `SecurityDecorator` route files decorate with `@guard_deco.` | -| `prompt_injection_validator`/`secret_exfil_validator`/`internal_ssrf_validator` | async funcs | security.py:116/128/139 | Custom `@guard_deco.custom_validation` content checks the signature WAF can't cover; each returns a generic 400 (no rule detail leaked) | -| `_WAF_FREETEXT_BODY_FIELDS` | const | security.py:211 | Top-level free-text body-field exclusion set (`excluded_detection_body_fields`) — the WAF calibration; includes free-form container fields (plan/risks/findings/section/payload/...) whose nested prose is stringified and scanned | - -## Data Flow - -**HTTP request**: nginx → ASGI `app` → `CorrelationIdMiddleware` (binds correlation_id + path/method to structlog) → `RequestLoggingMiddleware` (start timer) → route. Route resolves `CurrentAgentContext` via `get_agent_context` (headers + HMAC verify + identity/role/team resolution), plus service deps from `get_choreographer`/`get_content_actions`. When `ROBOCO_CLOUD_AUTH_ENABLED` is off (default) this is byte-for-byte the historical header-trust path (`_header_trust_agent_context`). When on, `_cloud_auth_agent_context` enforces a dual path: a request carrying a valid `X-Agent-Token` HMAC (any role — the agent fleet + the orchestrator's own `system` self-PATCH) is verified then delegated to the same header-trust resolution; a request with no valid token and a non-CEO role claim is rejected outright (closes the LAN header-spoof hole); the CEO alone may instead authenticate via the `roboco_session` cookie (`resolve_session_user`, `roboco.api.auth.session`), which is re-minted on every authenticated request (`_slide_session_cookie`) for a sliding 30-day window. On exception, the handler chain maps: `RequestValidationError` → 422 (scrubbed log + UUID remediation hint), `HTTPException` → standardized error code, `RobocoError` → domain status, `ServiceError` → parallel-hierarchy status, `RateLimitError` → 429 + `Retry-After`, `Exception` → 500. Response gains `X-Correlation-ID` + `X-Response-Time-Ms`. When `ROBOCO_GUARD_ENABLED` is on, `SecurityMiddleware` (mounted last in `create_app`, so outermost) runs before any of this: rate/size/WAF/custom-validator checks either block the request (enforce mode) or only log the detection (`guard_passive_mode`, the calibration posture) ahead of the correlation-id middleware; off by default, the whole path is unchanged. - -**Lifespan startup**: `init_db` (alembic upgrade + create_all fallback) → `apply_persisted_feature_flags` (panel settings overlay, best-effort) → `TranscriptionService.start()` + `ExtractionPipeline` → `get_optimal_service()` (BLOCKS 30-90s for RAG) → `LearningPropagationService.initialize(optimal)`. `app.state.*` holds singletons. **Shutdown**: stop orchestrator (drains bg DB writes) → `close_optimal_service` → `close_db`. The orchestrator-stop-before-DB order is load-bearing. - -**WebSocket**: panel → `wss://.../ws/{kind}/{id}` → `_require_panel_token` (CEO HMAC, except `/ws/system`) → query-param `agent_id`/`viewer_id` UUID parse (+ DB existence check for agent/notifications) → `manager.connect_*` accepts, registers in subscription set, spawns per-connection `_run_sender` task draining a bounded queue. Receive loop `await asyncio.wait_for(receive_text(), IDLE_TIMEOUT_SECONDS)`; "ping"→"pong"; timeout/disconnect → `finally disconnect`. **Broadcasts**: service code calls `manager.broadcast_to_*` → `_enqueue_or_send` per conn → either `conn.queue.put_nowait` (drop+warn on full) or legacy fire-and-forget `_send_with_timeout`. **Event-bus bridge**: `StreamEventBus` publishes `NOTIFICATION_SENT`/`AGENT_*`/`RATE_LIMIT_*`/`USAGE_SNAPSHOT`/`A2A_MESSAGE_SENT` → `websocket_bridge` handlers → `manager.broadcast_*` → panel. `A2A_MESSAGE_SENT` (published on every agent-to-agent DM) fans out to `/ws/system` as an `a2a.message` frame carrying conversation_id/message_id/task_id/from_agent/to_agent/skill + a capped body excerpt — the CEO's live view of every agent-to-agent chat, and the canonical pattern for wiring a new live event to the panel. - -## Mermaid - -```mermaid -sequenceDiagram - participant Panel - participant Nginx - participant ASGI as app (FastAPI) - participant MW as middleware - participant Route as route + deps - participant Chor as Choreographer - participant Bus as StreamEventBus - participant Mgr as ConnectionManager - participant WS as /ws/* client - - Note over ASGI: lifespan startup - ASGI->>ASGI: init_db (alembic) - ASGI->>ASGI: apply_persisted_feature_flags - ASGI->>ASGI: transcription + extraction + optimal(RAG) + learning - - Panel->>Nginx: HTTPS /api/* (X-Agent-Token) - Nginx->>ASGI: forward - ASGI->>MW: CorrelationIdMiddleware (bind cid) - MW->>MW: RequestLoggingMiddleware (start timer) - MW->>Route: dispatch - Route->>Route: get_agent_context (HMAC verify) - Route->>Chor: get_choreographer(db) - Chor-->>Route: Envelope - Route-->>MW: response - MW-->>Nginx: + X-Correlation-ID, X-Response-Time-Ms - Nginx-->>Panel: response - - Panel->>Nginx: wss /ws/system - Nginx->>ASGI: upgrade - ASGI->>Mgr: _require_panel_token -> connect_system - Mgr->>Mgr: _register_sender (queue + task) - Bus-->>ASGI: A2A_MESSAGE_SENT event - ASGI->>Mgr: _handle_a2a_message_event - Mgr->>WS: enqueue -> send_text (a2a.message) - - Note over ASGI: lifespan shutdown - ASGI->>Chor: orchestrator.stop() (drains DB writes) - ASGI->>ASGI: close_optimal_service - ASGI->>ASGI: close_db -``` - -## Logical Tree - -``` -roboco/api/ -├── __init__.py # no re-export of app (circular-import guard) -├── app.py -│ ├── _AppServices # transcription/extraction holders -│ ├── lifespan() # startup + shutdown ordering -│ └── create_app() -> app # ~40 routers + ws_router at /ws -├── deps.py -│ ├── _ServiceHolder # permission_service + orchestrator singletons -│ ├── agent identity (get_current_agent_id / slug / optional / context) -│ ├── _check_agent_auth_token / require_panel_token (HMAC) -│ ├── role gates (require_pm_or_above / developer_or_above / cell_access) -│ ├── permission dep factories (notification, task action) -│ ├── get_choreographer / get_content_actions -│ └── get_pagination -├── middleware.py -│ ├── CorrelationIdMiddleware -│ ├── RequestLoggingMiddleware -│ ├── exception handlers (RequestValidationError, HTTPException, RobocoError, -│ │ ServiceError, RateLimitError, Exception) -│ ├── _scrub_secrets / _uuid_field_remediation -│ └── setup_middleware() -├── middleware_docs.py -│ ├── DOCS_PERMISSIONS matrix -│ ├── check_docs_access / require_docs_access -│ └── get_allowed_docs_paths -├── websocket.py -│ ├── _ClientConnection (queue + sender) -│ ├── _require_panel_token -│ ├── ConnectionManager (agent/notification/system sets + senders) -│ ├── manager singleton -│ ├── validate_agent_exists -│ ├── routes: /agents/{id} /notifications/{id} /system -│ └── broadcast_agent_chunk / broadcast_notification helpers -├── websocket_bridge.py -│ ├── _handle_notification_sent / _handle_agent_event / _handle_a2a_message_event -│ ├── _handle_rate_limit_event / _handle_usage_event -│ └── register_websocket_bridge_handlers / start_websocket_bridge -└── utils/ - ├── __init__.py # re-exports - ├── errors.py # HTTPException factories + service_error_handler - └── resources.py # get_or_404 + ownership/recipient/membership -``` - -## Dependencies - -**Internal**: `roboco.config.settings`; `roboco.db.base` (init_db/close_db/get_db/get_session_factory); `roboco.db.tables.AgentTable`; `roboco.foundation.identity` (BOARD_ROLES/DEV_ROLES/PM_ROLES/Role); `roboco.models` (AgentRole/Team); `roboco.runtime.AgentOrchestrator`; `roboco.agents_config` (CEO_AGENT_ID, verify_agent_token, AGENT_ROLE_MAP/AGENT_TEAM_MAP, ALL_DOCS, _resolve_to_slug); `roboco.events` (Event/EventType/get_event_bus); `roboco.exceptions` (RobocoError tree); `roboco.services.base` (ServiceError tree); `roboco.services.exceptions.RateLimitError`; `roboco.services.{permissions,task,work_session,git,workspace,journal,a2a,product,notification,notification_delivery,audit,settings,extraction,learning,optimal,transcription}`; `roboco.services.gateway.{choreographer,content_actions,evidence_repo}`; `roboco.services.repositories` (resolve_agent_uuid/resolve_agent_identity); `roboco.api.schemas.{optimal.PaginationParams,common.ErrorCode}`; ~40 `roboco.api.routes.*` routers; `roboco.api.routes.v1.*` flow modules. - -**External**: `fastapi` (FastAPI, APIRouter, WebSocket, HTTPException, Depends, Header, status), `starlette.middleware.base.BaseHTTPMiddleware`, `starlette` responses, `sqlalchemy` (select, async session), `structlog`, `pydantic` (via schemas), `asyncio`, `uuid`, `json`, `time`, `contextlib`. (`httpx` was REMOVED from websocket.py in the baseline→head diff — the self-call `validate_channel_access` is gone.) - -## Entry Points - -- `roboco.api.app:app` — the ASGI instance uvicorn/gunicorn serves; `create_app()` called at import. -- `lifespan` — FastAPI async context manager; runs on startup/shutdown. -- `setup_middleware(app)` — called from `create_app` after CORS. -- `register_websocket_bridge_handlers()` / `start_websocket_bridge()` — called by orchestrator/bootstrap after the event bus is up (NOT in `lifespan`; the lifespan does not register bridge handlers). -- WS routes — invoked by the panel via `wss://.../ws/{agents|notifications|system}/...`, mounted at prefix `/ws` in `create_app`. -- `manager` singleton — imported directly by services and the bridge to broadcast. -- DI deps — resolved per-request by FastAPI (`get_agent_context`, `get_choreographer`, `get_content_actions`, `get_pagination`, `require_*` factories). -- `require_panel_token` — HTTP dep on the live intake/secretary chat bridges. - -## Config Flags - -- `ROBOCO_AGENT_AUTH_REQUIRED` — gates HMAC token enforcement (deps.py:211, websocket.py:71, middleware docstring app.py:94). Unset → header-trust/dev mode; set `true`/`1`/`yes` → strict. -- `ROBOCO_AGENT_AUTH_SECRET` — the HMAC secret consumed by `verify_agent_token` (read inside `roboco.agents_config`). -- `ROBOCO_CLOUD_AUTH_ENABLED` (+ `_EMAIL`/`_PASSWORD`/`_SECRET`/`_COOKIE_MAX_AGE`, default off) — `deps.get_agent_context`'s dual-path switch (`_cloud_auth_agent_context` vs byte-for-byte `_header_trust_agent_context`); the login/logout FastAPI Users router is mounted by `roboco.api.auth.routes.mount_cloud_auth` only when true, but `/api/auth/status` is always mounted (public probe for the panel's `proxy.ts`). -- `ROBOCO_DATABASE_*`, `ROBOCO_REDIS_*` — read transitively via `settings` / `init_db`. -- `settings.cors_origins` / `settings.cors_allow_credentials` — CORS middleware config (app.py:218). -- `settings.app_version` / `settings.environment` / `settings.debug` — logged at startup; docs/redoc URLs are unconditional (the `if settings.debug` is commented out, app.py:207-208). -- `settings.host` / `settings.port` — no longer used in websocket.py (the httpx self-call was removed); still referenced elsewhere. -- `ROBOCO_GUARD_ENABLED` / `_PASSIVE_MODE` / `_FAIL_SECURE` / `_TELEMETRY_ENABLED` / `_AGENT_API_KEY` / `_PROJECT_ID` / `_EMERGENCY` / `_EMERGENCY_WHITELIST` — read by `roboco/security.py`, wired into `create_app` via `apply_guard(app)` (app.py:234) + `guarded_lifespan(lifespan)` (app.py:212); `ROBOCO_ENVIRONMENT` additionally drives `enforce_https` (production only). -- Otherwise no direct ROBOCO_* feature flags live in this slice; the lifespan applies persisted flag overlays via `apply_persisted_feature_flags` but does not itself read individual subsystem flags. - -## Gotchas - -- **~~`/ws/system` was the only WS endpoint WITHOUT `_require_panel_token`~~ — RESOLVED (`536bbb64`)**: `/ws/system` now calls `_require_panel_token` before subscribing (websocket.py:621); all `/ws/*` endpoints are consistently gated (five at the time of `536bbb64`; three after the comms-subsystem teardown removed `/ws/channels` + `/ws/sessions`). In strict mode a missing token closes with `WS_1008_POLICY_VIOLATION`; a forged token is rejected even in dev mode. -- **`websocket.py` module docstring (lines 8-13) is STALE** — it claims WS "validates agent_id via query params and verify the agent exists in the database. In production, this should be enhanced with proper token-based authentication." Actual security is now the CEO HMAC panel token via `_require_panel_token`, and `validate_agent_exists` is only called on `/agents`, `/notifications`. The docstring misleads. -- **`get_choreographer` passes `stream_bus=None` when no orchestrator is set** (deps.py:557) — fine, but means the rate-limit park path is inert during the startup window before bootstrap sets the orchestrator. -- **`_check_agent_auth_token` dev mode**: a missing token is allowed; a presented-but-invalid token is rejected. The header-trust warning at app.py:94-102 is the only signal. In dev, any reachable client can act as any role (including `ceo`) by setting headers. -- **`_resolve_agent_identity` `system` role special-case** (deps.py:281) returns `UUID(x_agent_id)` with NO DB lookup. Combined with dev-mode no-auth, a caller can claim `role=system` with an arbitrary UUID and bypass agent resolution entirely. `ROBOCO_CLOUD_AUTH_ENABLED` does not add a DB lookup here either — it only requires that a `system`/any-role claim carry a *verified* HMAC token first (`_cloud_auth_agent_context` rejects any non-CEO role claim without one), so the identity-bypass itself is unchanged, just gated behind a valid signature. -- **`_coerce_agent_role` falls back to the DB role when the header isn't a valid enum** (deps.py:298). The X-Agent-Role header is therefore advisory when malformed — the authoritative role is the agent row's. Good for safety, but means a caller cannot escalate by header alone (the DB role wins). -- **`request_validation_handler` returns the UNSCRUBBED body to the client** (middleware.py:434). Only the server log is scrubbed (`_scrub_secrets`); the 422 response echoes whatever the client sent, including any secret fields. By design (the client sent them), but worth knowing. -- **`_run_sender` self-cancels on send error**: on a hard send error it calls `self.disconnect(ws)` which cancels `conn.sender` — the very task currently running (websocket.py:155). It returns immediately after, so the cancellation lands on an already-returning task; harmless in practice but a subtle self-cancel. -- **Lifespan shutdown order is load-bearing**: orchestrator.stop() MUST run before close_db (app.py:170-186). Reverting this order silently drops final audit-log rows + respawn_tracker upserts + agent-state finalizes. `stop()` is idempotent (bootstrap's finally re-calls it). -- **`ConnectionManager` sets are NOT mutated under a lock** — relies on asyncio single-threadedness. A broadcast iterating a set while `disconnect` mutates it is safe within one event loop, but `_run_sender`'s `disconnect(ws)` is called from a different task than the receive loop's `finally disconnect`, so two tasks can concurrently mutate the same set. `set.discard` is safe but iteration-during-mutation could raise `RuntimeError: Set changed size during iteration` in pathological cases. -- **`broadcast_notification` (websocket.py:665) bypasses the `broadcast_to_*` pattern** and reaches into `manager._enqueue_or_send` directly with a pre-serialized `data` string, while `broadcast_to_*` serialize inside. Inconsistent but works. -- **`roboco/api/__init__.py` deliberately does NOT re-export `app`** — importing `roboco.api.schemas.X` must not transitively load the FastAPI app + routes (circular-import cycle). The entrypoint imports `roboco.api.app:app` directly. Do not "helpfully" re-export here. -- **`docs_url`/`redoc_url` are unconditional** (app.py:207-208) — the `if settings.debug` gating is commented out, so `/docs` and `/redoc` are always served. -- **`apply_persisted_feature_flags` is best-effort** (app.py:115-121) — a DB failure logs a warning and continues with env defaults; startup is never blocked. -- **fastapi-guard is a genuine no-op when off** (`ROBOCO_GUARD_ENABLED` default `false`) — `apply_guard` returns before `add_middleware`, so `create_app`'s request path is byte-for-byte unchanged; the per-route `@guard_deco.*` decorators across ~21 route files are harmless because the decorator only takes effect once `app.state.guard_decorator` is set by `apply_guard` (security.py:388). -- **`excluded_detection_body_fields` is the only reliable WAF-calibration knob on guard 7.2.1** — the per-route `categories`/`enabled_detection_categories` config is bypassed for JSON bodies, and the body scanner excludes TOP-LEVEL keys only, scanning `str(value)` of every non-excluded field (the whole stringified subtree). A free-form container field (e.g. `plan`, `findings`) must therefore be excluded wholesale or its nested prose still trips the WAF. - -## Drift from CLAUDE.md - -- `CLAUDE.md` "WebSocket streams" section lists `/ws/agents/{id}`, `/ws/notifications/{id}`, `/ws/system` — **matches** `websocket.py` routes. No drift: both sides now agree the surviving set is these three. `/ws/channels/{id}` and `/ws/sessions/{id}` are removed from `websocket.py` itself as part of the comms-subsystem teardown (a real route deletion, not a stale doc omission). -- `CLAUDE.md` says `/ws/system` carries "the rate-limit lifecycle (`RATE_LIMIT_HIT` / `RATE_LIMIT_LIFTED`) and live usage (`USAGE_SNAPSHOT`)" — **matches** `websocket_bridge.py:138-166` (`_handle_rate_limit_event` + `_handle_usage_event`). No drift. -- `CLAUDE.md` says "To add a new live event: define an `EventType`, publish it to the bus, add a `_handle_*` forwarder in `websocket_bridge`, and consume it on the panel via the `useWebSocket(...)` hook" — **matches** the `_handle_*` pattern. CLAUDE.md's worked example is now `A2A_MESSAGE_SENT` → `_handle_a2a_message_event` → `/ws/system` (replacing the retired `MESSAGE_SENT`/`useSessionStream` example), consistent with this slice's implementation. No drift. -- `CLAUDE.md` "Startup Sequence" says "FastAPI lifespan indexes documents using Ollama (~30-60s)" — `app.py:137-147` initializes `OptimalService` (RAG) with a 30-90s blocking init and logs "OptimalService (RAG) initialized successfully"; the "indexes documents" framing is approximate but consistent. No material drift. -- `CLAUDE.md` does NOT document the `_require_panel_token` CEO-HMAC WS gate or the HTTP `require_panel_token` dep. The `websocket.py` module docstring (lines 8-13) describes the OLD query-param model, contradicting the actual HMAC-token implementation. The CLAUDE.md "Agent Gateway" section's "Agents do not call the API or per-domain MCP tools directly" is consistent with `/ws/*` being operator-only. **Drift: in-file docstring vs actual code (websocket.py:8-13); CLAUDE.md itself is silent on WS auth, so no CLAUDE.md contradiction.** -- `CLAUDE.md` "Orchestrator runtime-state durability" notes the respawn_tracker DB-durable writes are drained on `stop()` — `app.py:170-186` implements the required ordering (stop before close_db). Consistent. -- `CLAUDE.md` "Feature flags / company-in-a-box" says flags "toggle from the panel's Settings → Feature Flags card ... A toggle persists in the settings store and takes effect on the next backend restart" — `app.py:115-121` applies them in lifespan. Consistent. -- `CLAUDE.md` does not mention the `CorrelationIdMiddleware` / `RequestLoggingMiddleware` / exception-handler chain by name; `middleware.py` is the implementation of the implied "structured error" contract. No contradiction. -- `CLAUDE.md`'s "Feature flags / company-in-a-box" list of env-gated default-off subsystems does not mention `ROBOCO_GUARD_ENABLED` / the fastapi-guard HTTP security layer (`roboco/security.py`, wired here via `apply_guard`/`guarded_lifespan`); the doc is silent rather than contradictory. - -Net: **no direct contradictions with CLAUDE.md**; the one stale security docstring lives in `websocket.py` itself. - -## Changes Since Baseline - -Only ONE commit in `fd10cc86..HEAD` touched this slice: `15effce0` "Chore: 141 Gaps fill-in (#283)" (2026-06-29). - -Diff stat: `app.py +20`, `deps.py +44`, `middleware.py +49`, `websocket.py +309/-66` (net), `middleware_docs.py` / `websocket_bridge.py` / `utils/*` UNCHANGED. - -> **Post-snapshot update (2026-07-01, chat-subsystem live-delivery work `76ce53e3`):** `websocket_bridge.py` is no longer unchanged — it gained `_handle_message_event` (forwards `EventType.MESSAGE_SENT` to `/ws/sessions/{id}` + `/ws/channels/{id}` as a `message.new` frame) and a `MESSAGE_SENT` subscription in `register_websocket_bridge_handlers`. This is the live transcript-update path that was previously dead (`send_message` never broadcast). **Now removed** by the comms-subsystem teardown (`docs/internal/specs/2026-07-03-comms-teardown-trace.md`): the `/ws/sessions` + `/ws/channels` routes, the `_handle_message_event` forwarder, and its `MESSAGE_SENT` subscription are all gone; `EventType.MESSAGE_SENT` itself is left as a dead/inert enum member, not deleted. The surviving bridge worked example is `_handle_a2a_message_event` (`EventType.A2A_MESSAGE_SENT` → `/ws/system` as an `a2a.message` frame). - -> **Post-snapshot update (2026-07-01, logical-gap sweep `536bbb64`):** `deps.py` gained `require_ceo_role` (deps.py:412) — single source-of-truth CEO-role check shared by the orchestrator router gate and the release handler, replacing two diverged inline comparisons. `websocket.py` gated `/ws/system` with `_require_panel_token` (websocket.py:621), closing the medium regression risk; all `/ws/*` endpoints are now consistently gated (five at the time of this commit; the comms-subsystem teardown since removed `/ws/channels` + `/ws/sessions`, leaving `/ws/agents`, `/ws/notifications`, `/ws/system`). - -> **Local branch (not on master, NOT deployed):** `feature/fastapi-guard-hardening` (6 fastapi-guard commits `896532a3`..`99ee666e`, branched off `ab69851d`, plus 2 unrelated bundled commits) adds `roboco/security.py` and wires it into this slice — `apply_guard(app)` mounts `SecurityMiddleware` last in `create_app` (app.py:234) and `guarded_lifespan(lifespan)` wraps the async lifespan (app.py:212), both gated by `ROBOCO_GUARD_ENABLED` (default off, byte-for-byte unchanged request path when off). Per-route `@guard_deco.*` decorators (rate_limit/max_request_size/content_type_filter/behavior_analysis/block_clouds/honeypot_detection/usage_monitor/suspicious_detection/custom_validation — 9 kinds) are applied across 21 route files outside this slice (api-routes-schemas + v1 flow/do). `build_security_config` also carries a WAF false-positive calibration: `excluded_detection_body_fields` (75 free-text top-level body fields, including container fields like plan/risks/findings/section/payload) plus `enable_penetration_detection=True`, dropping active-mode false positives on RoboCo's own code/SQL/diff/URL-bearing traffic to zero while leaving the three custom validators and the WAF on non-excluded (id/enum/slug/branch) fields fully in force. New tests: `tests/unit/test_security.py` (unit) + `tests/unit/test_security_middleware.py` (integration — mounts the real middleware end-to-end). Both NAS composes (`docker-compose.yml`/`.yaml`) arm the layer passive/log-only (`c496b677`, Phase 5) — see deployment-tooling. - -Logic-touching changes in that commit, scoped to this slice: - -| Change | File:Line | IMPACT | -|--------|-----------|--------| -| Lifespan shutdown now stops orchestrator BEFORE close_db (was: DB closed first, only bootstrap's finally stopped orchestrator) | app.py:170-186 | Final audit-log rows / respawn_tracker upserts / agent-state finalizes no longer silently dropped on shutdown. `get_orchestrator_or_none()` added so lifespan doesn't 503 when no orchestrator is wired (tests/skip_orchestrator). | -| `get_orchestrator_or_none` + `clear_orchestrator` added | deps.py:96-119 | New accessor for shutdown + test teardown; `get_orchestrator` still 503s. | -| `require_panel_token` HTTP dep added (CEO HMAC for live-chat bridges) | deps.py:251-274 | New gate; mirrors WS `_require_panel_token`. Browser EventSource can't set headers, so token-only. | -| `CEO_AGENT_ID` import added to deps.py | deps.py:18 | Required by `require_panel_token` + reused by WS gate. | -| `_SECRET_FIELD_NAMES` + `_scrub_secrets` added; 422 log scrubs credential fields | middleware.py:372-404, 426 | Plaintext GitHub PAT / provider API key / bearer tokens no longer dumped to structlog on a 422. Response body unchanged. | -| WS panel-token gate (`_require_panel_token`) added to channel/agent/session/notifications streams | websocket.py:61, 371/439/503/567 | `/ws/*` now CEO-HMAC-gated in prod; dev allows missing token but rejects forged. **`/ws/system` was NOT gated** (inconsistency). | -| Per-connection bounded send queue + sender task (`_ClientConnection`, `_register_sender`, `_run_sender`, `_enqueue_or_send`, `_send_with_timeout`) | websocket.py:45-156, 244-281 | Replaced `asyncio.gather(*[conn.send_text(data)])` with non-blocking enqueue. One slow client can no longer back-pressure the fan-out; full queue drops+warns. `_run_sender` reaps dead sockets on hard send error. | -| `IDLE_TIMEOUT_SECONDS` (90s) `asyncio.wait_for` on `receive_text` | websocket.py:35, 400/471/534/586/623 | Half-open sockets from dead containers no longer block the receive loop forever; `TimeoutError` → `finally disconnect`. | -| `httpx` self-call `validate_channel_access` REMOVED; `settings` import dropped from websocket.py | websocket.py (was) | The channel WS no longer makes an HTTP round-trip to `/api/permissions/check` on the local server (deadlocks/latency risk gone). Channel access now gated only by panel token. | -| `structlog` logger (`log`) replaced the old logger; `validate_agent_exists` kept for agent/session/notifications | websocket.py:30, 337 | Logging consistent with rest of API. | -| `system_stream` endpoint + `broadcast_system` + `connect_system` (already present pre-baseline) — the commit RETAINED the no-token path for `/ws/system` | websocket.py:208-212, 316-322, 608 | Operator stream stays ungated; rate-limit/usage telemetry reachable without panel token. | - -`middleware_docs.py` and `utils/*` are byte-for-byte unchanged since baseline. `websocket_bridge.py` was unchanged at the snapshot but has since been edited by the chat-subsystem live-delivery work (the `_handle_message_event` forwarder + `MESSAGE_SENT` subscription — see the post-snapshot note above). - -## Regression Risks - -| Title | File:Line | Claim | Severity | -|-------|-----------|-------|----------| -| `/ws/system` ungated while siblings require panel token | websocket.py:608 | ~~The rate-limit/usage operator stream has no `_require_panel_token` call; if nginx does not edge-gate `/ws/system`, any reachable client gets RATE_LIMIT_HIT/LIFTED + USAGE_SNAPSHOT telemetry. The other four streams gate.~~ **RESOLVED (`536bbb64`)**: `/ws/system` now calls `_require_panel_token` (websocket.py:621); all `/ws/*` endpoints are consistently gated (five then; the comms-subsystem teardown since removed `/ws/channels` + `/ws/sessions`, leaving three). | ~~medium~~ resolved | -| `_run_sender` self-cancels its own task on send error | websocket.py:155 | `self.disconnect(ws)` pops `conn.sender` and calls `.cancel()` on the task currently running `_run_sender`. The immediate `return` mitigates, but a cancellation landing on a returning task is a subtle race; under heavy churn could mask a later send or trip a CancelledError in an `except Exception` handler. | low | -| Concurrent set mutation between sender-task `disconnect` and receive-loop `finally disconnect` | websocket.py:155 / 425 | Two tasks (sender + receive loop) can call `disconnect(ws)` on the same socket concurrently. `set.discard` is idempotent, but a `broadcast_to_*` iterating the same set on the loop thread during the sender-task's `disconnect` could raise `Set changed size during iteration`. Single-loop asyncio makes this rare but not impossible. | low | -| `validate_agent_exists` opens its own DB session via `async for db in get_db()` per WS upgrade | websocket.py:347 | Each `/agents`, `/notifications` upgrade grabs a session just to confirm the viewer exists — extra session pressure under many concurrent panel connections; the check is also bypassable in dev (no token). | low | -| Lifespan shutdown hangs if `orchestrator.stop()` blocks | app.py:184 | The new stop-before-close-db order is correct, but a hung `stop()` now blocks `close_optimal_service` + `close_db` (wrapped in try/except, so a hang — not an exception — is the failure mode). Pre-baseline, a hung stop only affected bootstrap's finally. | low | -| 422 response body still echoes unscrubbed secrets | middleware.py:434 | `_scrub_secrets` only scrubs the LOG, not the response. A client sending a `git_token` that fails validation gets it back in the 422 `body` field. By design but a leak surface if logs/responses are captured. | low | -| `_resolve_agent_identity` `system` role bypasses DB lookup | deps.py:281 | `role=system` returns `UUID(x_agent_id)` with no DB check. In dev (no auth) a caller can claim `system` with any UUID and get an `AgentContext` for a non-existent agent. Auth-required mode still needs a valid HMAC, mitigating prod. | low | -| `_enqueue_or_send` legacy fallback creates unbounded fire-and-forget tasks | websocket.py:266 | For an unregistered socket (shouldn't happen since every `connect_*` registers), a `_send_with_timeout` task is spawned per message with no queue cap. Only the legacy path; held in `_pending_sends` for GC safety. | low | -| `broadcast_notification` double-iterates notification connections | websocket.py:665-691 | It reads `manager.notification_connections.get(agent_id)` then calls `manager._enqueue_or_send` per conn, bypassing the `broadcast_to_*` serializer. If a disconnect happens between the snapshot and the enqueue, the enqueue targets a dead socket whose sender was cancelled — `_enqueue_or_send` falls to the legacy fire-and-forget path. Low. | low | -| Stale module docstring misleads future edits | websocket.py:8-13 | Describes query-param agent validation as the security model; actual model is CEO HMAC. A future edit trusting the docstring could weaken the gate. | low | - -## Health - -This slice is the API spine and is in **good structural shape**. The baseline→head refactor (`15effce0`) materially hardened it: the WebSocket fan-out no longer back-pressures on a slow client (bounded per-connection queues + sender tasks), half-open sockets are reaped by an idle timeout, the lifespan shutdown ordering fix stops silent final-write drops, 422 logs no longer leak credentials, and the WS + HTTP panel-token gates close the operator-only invariant. Post-snapshot, `536bbb64` closed the remaining medium risk by gating `/ws/system` (all `/ws/*` endpoints are now consistently gated — five then, three after the comms-subsystem teardown) and added `require_ceo_role` as a single-source CEO check; `76ce53e3` wired the live message-delivery path (`_handle_message_event`, since removed by the teardown). The event-bus bridge is clean and follows the documented `_handle_*` extension pattern. The main remaining integrity concerns are minor: the `websocket.py` module docstring is stale vs the HMAC implementation, and the concurrent-set-mutation window between the sender task's error-path `disconnect` and the receive loop's `finally disconnect` is theoretically present under asyncio single-threadedness. No logic-touching change since baseline looks broken; the regression risks above are edge-case and mostly low severity. The unchanged files (`middleware_docs.py`, `utils/*`) are stable; `websocket_bridge.py` gained `_handle_message_event` (`76ce53e3`, since removed by the comms-subsystem teardown — the surviving forwarder is `_handle_a2a_message_event`). Recommended follow-ups: refresh the `websocket.py` docstring, and consider a lock or snapshot-iteration in `ConnectionManager` broadcast paths. - -# Slice: api-routes-schemas - -## Purpose -The FastAPI surface of RoboCo: every HTTP route under `roboco/api/routes/` (the operator/panel `api/*` CRUD + dashboard/orchestrator/a2a/live bridges) and the agent-gateway `api/v1/flow/*` (intent verbs) + `api/v1/do/*` (content tools), with Pydantic request/response schemas under `roboco/api/schemas/`. Routes are thin handlers that resolve services via `Depends` and return typed responses; all agent-gateway verbs funnel through the Choreographer. - -## Files - -| Path | Role | -|------|------| -| roboco/api/routes/health.py | Liveness/readiness (DB + Redis probes). | -| roboco/api/routes/agents.py | List/get agents. | -| roboco/api/routes/notifications.py | Notification list/ack/send. | -| roboco/api/routes/stream.py | Agent stream chunk/complete/extract + permissions. | -| roboco/api/routes/journals.py | Journal entries, search, growth stats. | -| roboco/api/routes/kanban.py | Per-team kanban boards + main-pm/board/stats. | -| roboco/api/routes/cockpit.py | Cockpit summary/signals. | -| roboco/api/routes/company_goals.py | Company goals get/put. | -| roboco/api/routes/settings.py | Settings + feature-flags get/set. | -| roboco/api/routes/dashboard.py | CEO/auditor/kanban/metrics/agents/activity dashboards. | -| roboco/api/routes/tasks.py | Task CRUD + lifecycle transitions (claim/start/verify/qa/complete...). | -| roboco/api/routes/work_session.py | Work-session list/commit/files/PR/merge/complete/abandon. | -| roboco/api/routes/git.py | Per-project git status/log/diff/commit/push/PR/rebase/branch-cleanup sweep. | -| roboco/api/routes/project.py | Project CRUD + workspace/sync/access + conventions. | -| roboco/api/routes/product.py | Product CRUD. | -| roboco/api/routes/optimal.py | RAG: kb/search, rag/query, mentor/ask, learnings, decisions, review. | -| roboco/api/routes/research.py | Web search/fetch. | -| roboco/api/routes/orchestrator.py | CEO-gated spawn/stop/resolve-wait/mark-waiting + status. | -| roboco/api/routes/a2a.py | Agent-to-agent inbox/conversations/tasks + SSE streams. | -| roboco/api/routes/prompter_live.py | Live Intake chat (start/stream/messages/confirm/confirm-batch). | -| roboco/api/routes/secretary.py | Company state + CEO directives confirm/reject. | -| roboco/api/routes/secretary_live.py | Live Secretary chat (start/stream/messages/stop/events). | -| roboco/api/routes/release.py | CEO-only release proposal approve/reject. | -| roboco/api/routes/playbooks.py | Playbook approve/reject/archive (Auditor/CEO). | -| roboco/api/routes/pitch.py | Pitch create/list/approve/reject. | -| roboco/api/routes/provider.py | Provider catalog + ollama/grok/self-hosted key + mode. | -| roboco/api/routes/usage.py | Token usage summary/time-series/by-agent/team/model/role/sessions, cache-efficiency, spawn-waste (per-role unproductive-spawn rate + respawn strikes). | -| roboco/api/routes/system.py | System-wide info. | -| roboco/api/routes/docs.py | Project docs write/read/list/delete. | -| roboco/api/routes/x.py | X (Twitter) engine — CEO-only: list/approve/reject held draft posts + set/status OAuth 1.0a credentials. | -| roboco/api/routes/roadmap.py | Board roadmap engine — CEO-only: list open cycles + per-item approve/reject. | -| roboco/api/auth/ | Cloud auth (FastAPI Users, default off): `backend.py` (cookie transport + password-fingerprint-bound JWT strategy), `manager.py` (`UserManager` + DI chain), `session.py` (`resolve_session_user`, shared by the HTTP dual-path and the WS panel-token gate), `seed.py` (idempotent single seeded CEO login upsert), `routes.py` (always-public `/auth/status` + conditional login/logout mount). | -| roboco/api/routes/v1/_role_dep.py | Per-role HMAC guards + `envelope_to_response` helper. | -| roboco/api/routes/v1/do.py | Content verbs `/api/v1/do/*` (commit/note/say/dm/evidence/playbook...). | -| roboco/api/routes/v1/flow_dev.py | Developer flow verbs. | -| roboco/api/routes/v1/flow_qa.py | QA flow verbs (claim/pass/fail_review). | -| roboco/api/routes/v1/flow_doc.py | Documenter flow verbs. | -| roboco/api/routes/v1/flow_cell_pm.py | Cell-PM flow verbs (delegate/submit_up/triage/complete...). | -| roboco/api/routes/v1/flow_main_pm.py | Main-PM flow verbs (submit_root/triage_all/escalate_to_ceo...). | -| roboco/api/routes/v1/flow_board.py | Board (product_owner/head_marketing) triage/escalate_to_ceo. | -| roboco/api/routes/v1/flow_auditor.py | Auditor triage/i_am_idle. | -| roboco/api/routes/v1/flow_pr_reviewer.py | PR-reviewer verbs incl. gate pr_pass/pr_fail. | -| roboco/api/schemas/*.py | Per-domain Pydantic request/response models (one per route file). | -| roboco/api/schemas/v1/flow.py | All flow-verb request bodies + `StrList` coercion validator. | -| roboco/api/schemas/v1/do.py | All do-verb request bodies. | - -## Key Endpoints - -| Method | Path | Handler (file) | Auth/Role | -|--------|------|----------------|-----------| -| GET | /api/health, /api/ready | health.py | none | -| GET | /api/dashboard/ceo | dashboard.py | agent context | -| GET | /api/dashboard/metrics/{cycle-time,bottlenecks,rework,scorecard/*} | dashboard.py | agent context | -| GET/POST/PATCH/DELETE | /api/tasks, /api/tasks/{id}/{claim,start,verify,submit-qa,pass-qa,fail-qa,complete,cancel,escalate-to-ceo} | tasks.py | agent context + `require_task_action` | -| GET | /api/tasks/summary?q= (list_tasks_summary) | tasks.py | agent context — trimmed list-view rows; server-side title/description/id-prefix search via `TaskService.search_tasks` when `q` is set (wave 1, `d1cf6ecb`) | -| GET/POST | /api/orchestrator/{status,agents/{id},waiting} ; /spawn,/stop,/resolve-wait,/mark-waiting | orchestrator.py | `_require_ceo` (HMAC) | -| POST | /api/a2a/{send,send-stream} ; /chat/conversations ; /tasks/{id}/cancel | a2a.py | `require_any_authenticated_agent` | -| GET/POST | /api/a2a/chat/admin/{conversations,pairs,conversations/{id}/messages,conversations/{id}/reply} | a2a.py | `_require_ceo` (org-wide live view + reply-as-CEO; wave 2 `da563487` / wave 2c `876e19b3`) | -| POST | /api/prompter/live, /live/{id}/{stream,status,messages,stop,confirm,confirm-batch} | prompter_live.py | `require_panel_token` (CEO HMAC) | -| GET | /api/prompter/live/{id}/search-tasks | prompter_live.py | session-aliveness check (no agent identity) — intake's `search_past_tasks` tool (wave 1, `d1cf6ecb`) | -| POST | /api/secretary/live, /live/{id}/{stream,messages,stop,events} ; /api/secretary/{state,directives} | secretary*.py | panel token / agent ctx | -| GET | /api/secretary/tasks?q= (search_tasks) | secretary.py | agent ctx, Secretary or CEO role — resolve a task NAME to id(s) for a directive (wave 1, `d1cf6ecb`) | -| GET/POST | /api/release/proposal, /proposal/approve, /proposal/reject | release.py | `_require_ceo` (agent.role==CEO) | -| GET/POST | /api/playbooks, /{id}/{approve,reject,archive} | playbooks.py | agent context (Auditor/CEO) | -| GET/POST | /api/x/posts, /posts/{id}/{approve,reject}, /credentials | x.py | `require_ceo_role` (agent context) | -| GET/POST | /api/roadmap/cycles, /cycles/{id}/items/{id}/{approve,reject} | roadmap.py | `require_ceo_role` (agent context) | -| GET/POST | /api/auth/status (always), /auth/login, /auth/logout (mounted only when `cloud_auth_enabled`) | auth/routes.py | none (status) / FastAPI Users cookie login | -| GET/POST/PUT/DELETE | /api/projects, /{id}/conventions, /workspace, /sync | project.py | agent context | -| POST | /api/git/branches/cleanup | git.py | agent context, PM/CEO role-gated like `/rebase`; rate-limit 5/60 — cursor-resumable stale-branch sweep, `GitBranchCleanupRequest`/`Response` (wave 2, open PR #548) | -| POST | /api/v1/flow/developer/{give_me_work,i_will_work_on,open_pr,i_am_done,unclaim,resume,sync_branch} | flow_dev.py | `require_dev` (role + HMAC) | -| POST | /api/v1/flow/qa/{claim_review,pass_review,fail_review} | flow_qa.py | `require_qa` | -| POST | /api/v1/flow/cell_pm/{delegate,submit_up,complete,triage,unblock,reassign} | flow_cell_pm.py | `require_cell_pm` | -| POST | /api/v1/flow/main_pm/{submit_root,triage_all,escalate_to_ceo,complete} | flow_main_pm.py | `require_main_pm` | -| POST | /api/v1/flow/pr_reviewer/{claim_pr_review,claim_gate_review,pr_pass,pr_fail,post_pr_review} | flow_pr_reviewer.py | `require_pr_reviewer` | -| POST | /api/v1/do/{commit,note,say,dm,notify,evidence,draft_playbook,approve_playbook,...} | do.py | `require_any_authenticated_agent` (HMAC, any role) | -| GET | /ws/{agents,notifications,system}/{id} | websocket.py | WS panel/HMAC token | - -## Key Symbols - -| Name | Kind | File:Line | Responsibility | -|------|------|-----------|----------------| -| `require_any_authenticated_agent` | dep | v1/_role_dep.py | HMAC-verify X-Agent-ID/role/team token; router-level guard on do + a2a. | -| `require_` (require_dev/qa/...) | dep | v1/_role_dep.py | Per-role guard: HMAC + role assertion, applied as router dependency. | -| `envelope_to_response` | fn | v1/_role_dep.py | Convert Choreographer `Envelope` to JSON, set status from `envelope.status`. | -| `_check_agent_auth_token` | fn | api/deps.py:217 | Core HMAC verify; rejects invalid tokens even in dev; required-only in prod. | -| `require_panel_token` | dep | api/deps.py:251 | CEO-signed HMAC gate for live-chat bridges (HTTP analog of WS gate). | -| `CurrentAgentContext` | dep | api/deps.py:376 | Resolves agent from headers + HMAC, injects `AgentContext`. | -| `_require_ceo` | dep | routes/orchestrator.py:37 | Router-level CEO-HMAC guard on orchestrator control routes. | -| `_validated_agent_id` | fn | routes/orchestrator.py:99 | Path-injection guard (rejects empty/`.`/`..`/`/`/`\`/NUL) then normalizes via `_resolve_to_slug` — spawn/stop/status/resolve-wait/mark-waiting accept either a DB UUID or a slug and address the runtime container by the resolved slug; an unknown UUID passes through unchanged. | -| `setup_middleware` | fn | api/middleware.py | Register exception handlers (422 scrub, HTTP, RobocoError, generic). | -| `request_validation_handler` | fn | api/middleware.py:407 | Log 422 body (secrets scrubbed) + uuid remediate hint. | -| `_scrub_secrets` | fn | api/middleware.py:389 | Deep-redact known secret fields from logged 422 bodies. | -| `StrList` | type | schemas/v1/flow.py:20 | `list[str]` with `coerce_str_list` BeforeValidator (XML-nested LLM lists). | -| `Choreographer` | svc | services/gateway/choreographer.py | Composes service intents behind every flow verb. | -| `ContentActions` | svc | services/gateway/ | Composes do-verb content actions (commit/note/say/...). | -| `router` (do) | router | v1/do.py:38 | `/api/v1/do` router, `require_any_authenticated_agent` dep. | -| `router` (flow_dev) | router | v1/flow_dev.py:24 | `/api/v1/flow/developer` router, `require_dev` dep. | - -## Data Flow -Request hits nginx (port 3000) -> FastAPI app (`api/app.py`) registers routers under `/api/*` plus `/api/v1/flow/*` and `/api/v1/do/*`. Middleware chain (CorrelationId -> RequestLogging) attaches a correlation ID and logs; exception handlers intercept 422/HTTP/RobocoError/generic. Router-level `Depends` resolves `DbSession` + agent context (HMAC-verified from `X-Agent-*` headers) and, on agent-gateway routes, the role guard. The thin handler pulls a service via `Depends` (TaskService, Choreographer, ContentActions, GitService, OptimalService, ReleaseProposalService...) and returns a typed Pydantic response; flow/do verbs return the Choreographer `Envelope` via `envelope_to_response`. SSE (`EventSourceResponse`) is used for live-chat streams and a2a send-stream. - -## Mermaid -```mermaid -graph TD - app[FastAPI app.py] - app -->|/api/*| ops[Operator/Panel routes] - app -->|/api/v1/flow/*| flow[Flow routers] - app -->|/api/v1/do/*| do[do router] - app -->|/ws/*| ws[websocket.py] - ops --> tasks[tasks.py -> TaskService] - ops --> dash[dashboard.py -> MetricsService] - ops --> orch[orchestrator.py -> AgentOrchestrator CEO-gate] - ops --> a2a[a2a.py -> A2AService SSE] - ops --> livep[prompter_live.py -> PrompterService SSE panel-token] - ops --> lives[secretary_live.py -> SecretaryService SSE] - ops --> rel[release.py -> ReleaseProposalService CEO-gate] - flow --> fdev[flow_dev -> Choreographer.give_me_work/i_will_work_on/...] - flow --> fqa[flow_qa -> Choreographer.claim_review/pass/fail] - flow --> fpm[flow_cell_pm/main_pm -> Choreographer.delegate/submit_up/submit_root] - flow --> fpr[flow_pr_reviewer -> Choreographer.pr_pass/pr_fail] - do --> doR[do.py -> ContentActions.commit/note/say/dm/evidence] - flow -.->|HMAC role guard| _role_dep[_role_dep.py] - do -.->|HMAC any-role guard| _role_dep - orch -.->|HMAC CEO guard| deps[deps._require_ceo] - livep -.->|panel HMAC| deps2[deps.require_panel_token] - ws --> cm[ConnectionManager -> StreamEventBus] -``` - -## Logical Tree -``` -roboco/api/ -├── routes/ -│ ├── operator-panel (api/*) -│ │ ├── health.py liveness/readiness -│ │ ├── agents.py agent list/get -│ │ ├── notifications.py notification ack/send -│ │ ├── stream.py agent stream chunks/extract -│ │ ├── journals.py journal entries + growth -│ │ ├── kanban.py kanban boards -│ │ ├── cockpit.py cockpit summary/signals -│ │ ├── company_goals.py company goals -│ │ ├── settings.py settings + feature-flags -│ │ ├── dashboard.py CEO/auditor/metrics dashboards -│ │ ├── tasks.py task CRUD + lifecycle -│ │ ├── work_session.py work-session/PR/merge -│ │ ├── git.py per-project git ops -│ │ ├── project.py project CRUD + conventions -│ │ ├── product.py product CRUD -│ │ ├── optimal.py RAG kb/query/mentor -│ │ ├── research.py web search/fetch -│ │ ├── docs.py project docs -│ │ ├── system.py system info -│ │ └── usage.py token usage -│ ├── ceo-gated / live bridges -│ │ ├── orchestrator.py CEO spawn/stop/mark-waiting -│ │ ├── release.py release proposal approve/reject -│ │ ├── playbooks.py playbook curation -│ │ ├── pitch.py pitch approve/reject -│ │ ├── x.py X engine post queue approve/reject + credentials -│ │ ├── roadmap.py board roadmap cycle item approve/reject -│ │ ├── a2a.py agent-to-agent + SSE -│ │ ├── prompter_live.py live Intake chat -│ │ ├── secretary.py company state + directives -│ │ ├── secretary_live.py live Secretary chat -│ │ └── provider.py provider catalog/keys -│ └── v1/ (agent-gateway) -│ ├── _role_dep.py HMAC role guards + envelope helper -│ ├── do.py /api/v1/do/* content verbs -│ ├── flow_dev.py developer flow verbs -│ ├── flow_qa.py QA flow verbs -│ ├── flow_doc.py documenter flow verbs -│ ├── flow_cell_pm.py cell-PM flow verbs -│ ├── flow_main_pm.py main-PM flow verbs -│ ├── flow_board.py board flow verbs -│ ├── flow_auditor.py auditor flow verbs -│ └── flow_pr_reviewer.py PR-reviewer flow verbs -├── auth/ (cloud auth, default off — ROBOCO_CLOUD_AUTH_ENABLED) -│ ├── backend.py cookie transport + password-fingerprint-bound JWT strategy -│ ├── manager.py UserManager + get_user_db/get_user_manager DI chain -│ ├── session.py resolve_session_user (shared HTTP + WS cookie validation) -│ ├── seed.py ensure_seed_user / ensure_seed_user_startup (single CEO row) -│ └── routes.py always-public /status + conditional login/logout mount -└── schemas/ - ├── *.py per-domain Pydantic models - └── v1/ - ├── flow.py flow-verb bodies + StrList - └── do.py do-verb bodies -``` - -## Dependencies -- FastAPI + sse-starlette (SSE), pydantic v2. -- `roboco/api/deps.py` — shared deps (DbSession, agent context, HMAC, orchestrator). -- `roboco/api/middleware.py` — exception handlers + correlation/log middleware. -- `roboco/services/*` — TaskService, GitService, OptimalService, AgentOrchestrator, Choreographer, ContentActions, ReleaseProposalService, PrompterService, SecretaryService, MetricsService, XPostService, XCredentialsService, RoadmapService, etc. -- `roboco/api/auth/*` (`auth_backend`, `get_user_manager`, `resolve_session_user`, `mount_cloud_auth`) — cloud auth, consumed by `deps.get_agent_context`'s dual-path and the WS panel-token gate. -- `roboco/foundation/identity.py` (`Role`) + `roboco/agents_config.py` (`verify_agent_token`, `CEO_AGENT_ID`). -- `roboco/api/websocket.py` + `websocket_bridge.py` (WS event forwarding). - -## Entry Points -- `roboco/api/app.py` `create_app()` builds the FastAPI app, mounts all routers under `/api` (prefix) + `/ws` (WS router). -- `roboco/api/routes/v1/_role_dep.py` is imported by every flow router + do + a2a for HMAC/role guards and `envelope_to_response`. -- `roboco/api/routes/orchestrator.py` router constructed with `dependencies=[Depends(_require_ceo)]` (router-wide CEO gate). - -## Config Flags -- Auth-gate mode: `_auth_required()` (env-driven; HMAC mandatory in prod-ish, optional in dev) — `api/deps.py`. -- Feature-flag routes are inert when their backing engine is off: `release.py` (ROBOCO_RELEASE_MANAGER_ENABLED), `prompter_live.py` MegaTask batch, `optimal.py` learnings (ROBOCO_ORG_MEMORY_ENABLED), `research.py` (ROBOCO_RESEARCH_ENABLED), `provider.py` grok/self-hosted (ROBOCO_GROK / self-hosted), CI-watch/dep-update originate elsewhere but surface via orchestrator/tasks. - -## Gotchas -- `do` + `a2a` routers are token-only (any authenticated role), not role-asserted — any signed agent can call any content verb; service-layer scope is the only gate. -- `request_validation_handler` scrubs secrets from the **log** but the 422 **response body echoes the client's submission unchanged** (comment explicit) — secrets can still leak to the caller if the caller is not the legitimate owner. -- SSE live-chat bridges open one session per query/stream and rely on `require_panel_token` (CEO HMAC injected by nginx); a missing/invalid token in dev mode is tolerated (`_auth_required()` false) — prod must arm it. -- `StrList` BeforeValidator is load-bearing: without it the Claude SDK's XML-nested list input crashes `i_will_plan`/`delegate` with 422 (MegaTask memory Bug 3). -- `orchestrator.py` and `release.py` use two different `_require_ceo` implementations (HMAC header vs `agent.role==CEO` from context) — keep their semantics aligned. -- WS endpoints live on `/ws/*` (separate router in `websocket.py`), not under `/api`; the bridge subscribes to `StreamEventBus` and forwards per resource-id. -- `/api/tasks` PATCH is not a single admin surface: `_pm_editor_scope` (tasks.py:256) routes cell_pm/main_pm to a content-only allowlist (`_PM_LIGHTER_UPDATE_FIELDS`: title/description/acceptance_criteria/priority, zero status changes) enforced by `_enforce_pm_lighter_fields` (tasks.py:278), while CEO/Board/Auditor keep the unrestricted admin bypass; a cell_pm editing a task outside its own team 403s before the field check even runs. -- `GET /api/tasks/summary` and `GET /api/secretary/tasks` both call the same `TaskService.search_tasks` (ILIKE title/description + id-prefix) but through different auth (agent-context view-scope vs Secretary-or-CEO role check) and different response shapes (trimmed `TaskSummaryResponse` vs a hand-built dict list) — don't assume one route's pagination/limit semantics apply to the other. - -## Drift from CLAUDE.md -- CLAUDE.md lists `pr_pass`/`pr_fail` under `pr_reviewer` verbs and the in-path gate; code matches (`flow_pr_reviewer.py` exposes `claim_gate_review`, `pr_pass`, `pr_fail`). No drift found. -- CLAUDE.md says agent comms use `dm`/`read_a2a`/`notify` via do_server (the channel/session `say`/`open_session`/`link_session` surface was removed in the comms-subsystem teardown); code matches. No drift. -- CLAUDE.md lists `sync_branch` as a developer verb; present in `flow_dev.py:125`. No drift. -- CLAUDE.md's verb table omits `flow_pr_reviewer.post_pr_review` (external PR comment) — present in code; additive, not contradictory. -- None material. - -## Changes Since Baseline -`git log fd10cc86..HEAD -- roboco/api/routes/ roboco/api/schemas/`: -- `15effce0` Chore: 141 Gaps fill-in (#283) — broad route/schema hardening pass (the only logic-touching commit in range at the time this section was last refreshed). - -> Post-snapshot: many further commits touch this slice (536bbb64, df87fcf0, a8cb2470, 0ca9d91b, cfde4369, 0f1ed3cc, 1c87a4e4, and the three below) — only the wave-1/2/2c ones relevant to this pass are itemized; a full re-audit of the intervening route/schema history is still owed. -> - `d1cf6ecb` Wave 1 (#295) — adds `GET /api/tasks/summary?q=` search (`TaskService.search_tasks`), `GET /api/prompter/live/{id}/search-tasks` (intake memory), `GET /api/secretary/tasks?q=` (Secretary task-by-name lookup), and the Secretary `edit` directive action. -> - `da563487` Wave 2 (#297) — adds the CEO-only `/api/a2a/chat/admin/{conversations,conversations/{id}/messages,conversations/{id}/reply}` routes (`_require_ceo`) for the A2A live view + reply-as-CEO. -> - `876e19b3` Wave 2c (#298) — adds `/api/a2a/chat/admin/pairs` (the switchboard, same `_require_ceo` gate); tightens `/api/tasks` PATCH so cell/main PM roles get a content-only field allowlist instead of the unrestricted CEO/Board/Auditor admin bypass (`_pm_editor_scope` / `_enforce_pm_lighter_fields`, `roboco/api/routes/tasks.py:256,278`) — closes an over-permission hole where PM identities could edit any-team tasks via the ASSIGN-holding bypass. -> - `637c75dc` (2026-07-17, PR #546, "wave-1 quick wins") fix(api): normalize agent UUID to slug at the orchestrator route boundary — `_validated_agent_id` now also calls `_resolve_to_slug` after its path-injection checks, so a caller-supplied DB UUID (e.g. from the panel) resolves to the canonical slug before spawn/stop/status/resolve-wait/mark-waiting address the runtime, fixing UUID-named containers and registry misses. -> - (open PR #548, branch `feature/wave-2-hygiene-charts`, 2026-07-17) adds `POST /api/git/branches/cleanup` (PM/CEO role-gated like `/rebase`, rate-limit 5/60) + `GitBranchCleanupRequest`/`GitBranchCleanupResponse` schemas — cursor-resumable sweep of terminal tasks' remote+local branches, backing a confirm-dialog button on the panel Git page. - -## Regression Risks - -| Title | File:Line | Claim | Severity | -|-------|-----------|-------|----------| -| do/a2a any-role token gate | v1/do.py:43, a2a.py:114 | `require_any_authenticated_agent` only verifies HMAC + that the agent exists; it does NOT assert the role matches the verb's intended role family — a QA-signed token could call `do/commit`, or any agent could call the participant-scoped `a2a` routes (send/conversations) for a pair it has no policy access to (only the gateway's `can_a2a_direct`/`validate_a2a_access` matrix, a service-layer check, stops it). Service-layer scope is the sole guard on these paths; a missed service check = privilege escape. **Correction:** the `/chat/admin/*` routes (org-wide live view + reply-as-CEO) are NOT on this gate — they carry their own router-level `_require_ceo` guard, added in wave 2 (`da563487`) and extended to `/chat/admin/pairs` in wave 2c (`876e19b3`); a non-CEO agent 403s before reaching the service layer on those. | High | -| 422 response echoes secrets | middleware.py:407 | `_scrub_secrets` redacts only the **log** body; the JSON response still contains `body` with the caller's original secret fields. A 422 on `git_token`/`api_key` returns the secret back to the client (and to any MITM/log of the response). | High | -| orchestrator CEO gate vs release CEO gate divergence | orchestrator.py:37 vs release.py:32 | Two independent `_require_ceo` implementations: orchestrator uses HMAC header verification, release uses `agent.role == CEO` from `CurrentAgentContext`. If one path's HMAC/context resolution drifts, the two CEO surfaces enforce different identities. | Medium | -| SSE transport errors swallowed | prompter_live.py:122, secretary_live.py:61, a2a.py:195 | `EventSourceResponse` streams run long-lived; a Choreographer/orchestrator raise mid-stream is caught by `contextlib` suppress but can drop the stream silently without a terminal event to the panel. | Medium | -| Cross-repo PR collision via /api/work-sessions/{id}/pr/merge | work_session.py:259 | PR merge by global `pr_number` (no project_id scoping in the route signature) — the same class of cross-repo collision already fixed in `cell_pm_complete` could recur if this endpoint is wired to merge. | Medium | -| Dashboard/metrics endpoints role-gating | dashboard.py:58+ | `/ceo`, `/auditor`, `/scorecard/*` rely on `CurrentAgentContext` but the route-level gating is weak (no explicit `require_pm_or_above`); a non-CEO agent calling `/dashboard/ceo` is filtered only by service-layer logic, not the router. | Medium | -| WS panel-token vs agent-token dual gate | websocket.py / deps.py | `/ws/*` endpoints use a WS-specific `_require_panel_token` for panel streams but agent-id keying for `/ws/agents/{id}`; mismatched HMAC secret rotation between the two could grant panel read of agent streams or vice-versa. | Low-Med | -| flow `i_will_plan` StrList crash recurrence | schemas/v1/flow.py:20 | If a new LLM-authored `list[str]` field is added to a flow schema without `StrList`, the SDK XML-nesting crash reappears (silent 422 loop). Reviewer-only by inspection. | Low-Med | - -## Health -The route layer is thin, consistently organized (one router per domain, one schema file per router), and the agent-gateway HMAC guard is centralized in `_role_dep.py` + `deps.py`. Main risks are the any-role `do`/`a2a` gate (relies on service-layer scope), the 422 response echoing secrets, and the two divergent CEO guards — all addressable without structural change. SSE live-chat streams are the fragile transport path. - ## Purpose The support layer of the agent gateway: pure/cheap components the Choreographer composes into intent-verb sequences. Envelope is the wire contract; claim_guards/claimant_lock/trigger_filter gate concurrency and spawn decisions; content_actions smart-wraps the do-tools (commit/note/dm/notify/evidence/progress/pr_update/playbook/pitch); evidence_builder/evidence_repo assemble briefings; role_config is the per-role verb/tool manifest source; rate_limit_tracker persists provider park state in Redis; quality_gate runs the pre-submit fast checks; merge_chain resolves PR targets; commit_validator/remediation/kb_authz are small policy shims. None of these own the verb state machine — they are invoked BY the Choreographer and the MCP route handlers. @@ -3028,6 +2722,8 @@ gateway-support > Post-snapshot updates (since 2026-06-29): commit 536bbb64 ("Chore/all/logical gaps sweep #286") touched content_actions.py and rate_limit_tracker.py in this slice. (1) rate_limit_tracker: activate() is now a Lua atomic merge (_ACTIVATE_RATE_LIMIT) that carries over the previous probe_failures count — the increment-vs-activate race is closed. (2) content_actions: _curate_playbook wraps the gating session.commit() in try/except PendingRollbackError; a poisoned session now returns a clean Envelope.invalid_state instead of 500-ing, and an uncommitted playbook cannot fall through to the RAG index. > > **v0.18.0** (2026-07-04): X feature-spotlight adds `ContentActions.propose_feature_spotlight` (content_actions.py:1238) + `_FEATURE_SPOTLIGHT_ROLES = frozenset({"head_marketing"})` (line 313), mirroring `pitch`'s Board-gated-verb shape. **Gap found (static read, not live-verified):** this verb has no wrapper function or `_TOOLS` entry in `roboco/mcp/do_server.py` (unlike `propose_roadmap`, which has both), so `_register_tools()`'s manifest∩`_TOOLS` intersection never exposes it to a spawned Head-of-Marketing agent despite the role-config grant — see mcp-servers.md Regression Risks. +> +> **`56b6693e`** ("security-hygiene-sweep"): `content_actions.py:113`'s `_NO_COMMS_ROLES` (the `dm()` sender-side guard listed in the frozenset table above) changes from an independently hand-maintained literal to `frozenset(r.value for r in _comms.NO_COMMS_ROLES)` — derived from the new canonical `roboco.foundation.policy.communications.NO_COMMS_ROLES` (see `docs/map/foundation-policy-misc.md`), the SAME set `agents_config.py`'s CEO-target A2A check now also consumes, so the two enforcement points can't drift apart. No behavior change (the role set — auditor/pr_reviewer/prompter/secretary — is identical); this is a single-source-of-truth refactor. ## Regression Risks @@ -3045,1625 +2741,450 @@ gateway-support ## Health This slice is a mature, well-documented support layer with clear seams (pure helpers vs DB-touching services vs the wire Envelope). The single baseline-to-HEAD commit (15effce0) is a coherent set of bug fixes and defence-in-depth expansions: the blocked-task claim guard, the quality_gate zombie-reap + fail-closed fix, the atomic Lua probe-counter, the top-level resumption fields, the expanded no-comms gate, the prompter/secretary notify refusal, and the archive_playbook bug fix with commit-before-index ordering. The main integrity concerns are behavioral, not structural: the blocked-claim change and the prompter/secretary notify refusal are intended but could break orchestration paths that relied on the old permissiveness; the archive_playbook fix changes the archived-playbook observable state; and the rate_limit race window is narrowed (increment-vs-increment atomic) but not closed (increment-vs-activate still overwrites). The Envelope contract is stable and the introspection path correctly degrades. Regression risk is moderate and concentrated in content_actions.py and the two claim/spawn guards, all of which have clear test coverage expectations per the commit message. -# RoboCo Slice Map — `mcp-servers` - -Scope: `roboco/mcp/` (every server file + `schemas/` + `utils.py`). Repo root: `/Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco`. - ## Purpose - -The `roboco/mcp` package is the agent-side MCP gateway: a set of `FastMCP` server processes that run **inside each agent container** and expose the RoboCo intent-verb / content-tool / RAG / docs / git-readonly / intake / secretary / web-research surfaces to the Claude Code or grok-CLI runtime as MCP tools. They are thin bridges — every tool either POSTs to the orchestrator's HTTP gateway (`/api/v1/flow/*`, `/api/v1/do/*`, `/api/git/*`, `/optimal/*`, `/docs/*`, `/research/*`, `/api/secretary/*`, `/api/prompter/live/*`) or, for the flow/do path, additionally forwards rejections to a per-container SDK loopback (`ROBOCO_SDK_URL`) that runs the per-verb circuit breaker. The orchestrator (not the MCP layer) is the authority for role scoping, state transitions, and git-side effects; the MCP layer only shapes calls, classifies rejections, and substitutes `circuit_open` envelopes when the breaker trips. - -## Files - -| Path | Role | approx LOC | -|------|------|------------| -| `roboco/mcp/__init__.py` | Package docstring only — deliberately import-free so `python -m roboco.mcp.` does not pull sibling modules (esp. `optimal_server`'s pgvector/ollama stack, ~6s startup). | 23 | -| `roboco/mcp/utils.py` | Shared HTTP helpers: `_get_agent_headers`, `format_error_response`, `ApiResponse`, `ApiClient` (async httpx wrapper). Used by `optimal_server`, `docs_server`, `search_server`. | 383 | -| `roboco/mcp/schemas/__init__.py` | Pydantic input models. After Phase-4 T9 deletions only `WriteDocInput` remains. | 35 | -| `roboco/mcp/flow_server.py` | `roboco-flow` MCP server — intent verbs (lifecycle). Manifest-scoped registration, role-scoped path `/api/v1/flow//`, per-verb circuit breaker + 404-route synthesis. | 1028 | -| `roboco/mcp/do_server.py` | `roboco-do` MCP server — content tools (commit, note, pitch, propose_roadmap, dm, notify, evidence, progress, playbook curation, pr_update). Manifest-scoped; fixed `/api/v1/do/` path; mirror circuit breaker. | 976 | -| `roboco/mcp/optimal_server.py` | `roboco-optimal` MCP server — RAG / KB / mentor / error / decision / standards / learnings / index-mgmt / proactive-context. Factory `create_optimal_mcp_server(agent_id)`; calls `/optimal/*`. | 1102 | -| `roboco/mcp/docs_server.py` | `roboco-docs` MCP server — docs write/read/list/delete via `/docs/*`. Factory `create_docs_mcp_server(agent_id)`. RAG-based dedup on write. | 251 | -| `roboco/mcp/git_readonly.py` | `roboco-git-readonly` MCP server — four read-only git views (status/log/diff/branch list) via `/api/git/*`. No breaker, no manifest. | 123 | -| `roboco/mcp/intake_server.py` | `roboco-intake` MCP server — grok intake path only: `propose_draft` / `propose_batch` POST directly to the prompter-live relay. | 215 | -| `roboco/mcp/secretary_server.py` | `roboco-secretary` MCP server — grok secretary path only: `read_company_state` / `read_task` / `submit_directive`, delegating to `agent_sdk.secretary_driver`. | 64 | -| `roboco/mcp/search_server.py` | `roboco-search` MCP server — `web_search` / `web_fetch` via `/research/*` (provider key stays server-side). Factory `create_search_mcp_server(agent_id)`. | 130 | - -(Excluded: `__pycache__/`, `.DS_Store`.) - -## Key Symbols - -| Name | Kind | File:Line | Responsibility | -|------|------|-----------|----------------| -| `mcp` (flow) | `FastMCP` | `flow_server.py:252` | Server instance `roboco-flow`; tools registered onto it at import time. | -| `mcp` (do) | `FastMCP` | `do_server.py:220` | Server instance `roboco-do`. | -| `mcp` (git-readonly) | `FastMCP` | `git_readonly.py:31` | Server instance `roboco-git-readonly`. | -| `mcp` (intake) | `FastMCP` | `intake_server.py:32` | Server instance `roboco-intake`. | -| `mcp` (secretary) | `FastMCP` | `secretary_server.py:30` | Server instance `roboco-secretary`. | -| `StrList` | type alias | `flow_server.py:39` | `Annotated[list[str], BeforeValidator(coerce_str_list)]` — tolerates Claude SDK's nested XML-ish tool-input shapes before MCP validation rejects. | -| `_CIRCUIT_REJECTION_KINDS` | frozenset | `flow_server.py:66`, `do_server.py:50` | The 4 breaker-counted kinds: `tracing_gap`, `invalid_state`, `not_authorized`, `incomplete_input`. | -| `_DICT_ERROR_CODE_MAP` | dict | `flow_server.py:89`, `do_server.py:73` | Exact code→kind map for known RobocoError codes (e.g. `AUTHENTICATION_REQUIRED`→`not_authorized`). Unknown codes fall through to a substring branch for forward-compat. Added in 536bbb64 to fix AUTHENTICATION_REQUIRED mis-routing (#161). | -| `_classify_dict_error_code` | func | `flow_server.py:110`, `do_server.py:94` | Map a dict-shaped `error.code` to a counted breaker kind: consults `_DICT_ERROR_CODE_MAP` first (exact), then substring fallback for unknown codes; NOT_FOUND → None. | -| `_remediate_for_kind` | func | `flow_server.py:130`, `do_server.py:150` | Synthesize a directed recovery hint string for each counted kind (not_found / incomplete_input / not_authorized / invalid_state). Used by `_normalize_exception_envelope`. | -| `_normalize_exception_envelope` | func | `flow_server.py:164`, `do_server.py:180` | Lift a dict-`error` exception-handler body or 422 `detail` list into Envelope wire format (string kind + message + remediate + missing). Returns None when payload is already a valid Envelope. Added in 0d714b6c (#232). | -| `_classify_rejection` | func | `flow_server.py:216`, `do_server.py:114` | Classify all 3 rejection shapes (string kind / dict error / 422 `detail`) → counted kind or None. Guards the `dict in frozenset` `TypeError`. | -| `_build_headers` | func | `flow_server.py:256`, `do_server.py:224` | Per-call headers: `X-Agent-ID`, `X-Agent-Role`, fresh `X-Correlation-ID` (UUID per MCP call). | -| `_post` (flow) | func | `flow_server.py:272` | POST to orchestrator; normalize exception bodies via `_normalize_exception_envelope`; synthesize `invalid_state` on bare 404 missing route; `not_found` on descriptive 404 detail; `transport_error` on non-JSON; forward rejection to breaker. | -| `_post` (do) | func | `do_server.py:238` | Mirror of flow `_post` for content tools. | -| `_verb_from_path` | func | `flow_server.py:384`, `do_server.py:339` | Extract verb name from path for breaker reporting. | -| `_record_and_check_circuit` | func | `flow_server.py:394`, `do_server.py:349` | Forward a rejection to `SDK_URL/verb/attempted`; if SDK says `open`, replace the payload with `circuit_envelope` (dict-copied, original nested as `inner`, task_id/correlation_id lifted to top level). Best-effort (fail-open). | -| `_ROLE_TO_ROUTE_PREFIX` | dict | `flow_server.py:476` | Maps `product_owner`/`head_marketing` → `board` route segment; every other role passes through unchanged. | -| `_role_path` | func | `flow_server.py:483` | Build `/api/v1/flow//` path. | -| `_TOOLS` (flow) | dict | `flow_server.py:879` | Verb name → Python impl map (27 verbs). `pass`/`fail` keys bridge the `pass_review`/`fail_review` IntentSpec names. | -| `_INTENT_TO_PUBLIC` | dict | `flow_server.py:929` | `pass_review`→`pass`, `fail_review`→`fail` — fixes the dogfood gap where QA tools were silently dropped. | -| `_load_manifest_flow_tools` | func | `flow_server.py:935` | Read `/app/tool-manifest.json` `flow_tools`; None if missing/unreadable. | -| `_register_tools` (flow) | func | `flow_server.py:965` | Raise `RuntimeError` if manifest missing and `ROBOCO_ALLOW_FULL_TOOLSET` not set; if set, registers full tool set as dev/test escape hatch. | -| `_REGISTERED_TOOLS` (flow) | var | `flow_server.py:1024` | Import-time registration side effect. | -| `_TOOLS` (do) | dict | `do_server.py:863` | Tool name → impl map (21 content tools, incl. `propose_roadmap`). | -| `_load_manifest_do_tools` | func | `do_server.py:866` | Read manifest `do_tools` list. | -| `_register_tools` (do) | func | `do_server.py:896` | Manifest-scoped registration; raise `RuntimeError` if manifest missing (unless `ROBOCO_ALLOW_FULL_TOOLSET` set). | -| `give_me_work` … `i_am_idle` | verb funcs | `flow_server.py:491–625` | Dev verbs. | -| `claim_review` / `pass_review` / `fail_review` | verb funcs | `flow_server.py:628–653` | QA verbs (registered as `pass`/`fail`). | -| `claim_pr_review` / `post_pr_review` / `claim_gate_review` / `pr_pass` / `pr_fail` | verb funcs | `flow_server.py:656–876` | PR-reviewer verbs (inbound + in-path gate). | -| `i_will_plan` / `delegate` / `submit_up` / `submit_root` | verb funcs | `flow_server.py:759–856` | PM coordination verbs. | -| `note` | verb func | `do_server.py:428` | Journal entry + handoff section writer (top-level `done`/`next` strings — the meltdown-#1 fix). | -| `commit` / `dm` / `notify` / `evidence` | verb funcs | `do_server.py:423–598` | Core content tools. | -| `propose_roadmap` | verb func | `do_server.py:531` | Product Owner (board-roadmap-only, `_PRODUCT_OWNER_DO`): propose a themed roadmap cycle (goal + 3-7 item drafts) exactly once per exploration task. | -| `draft_playbook` / `approve_playbook` / `reject_playbook` / `archive_playbook` | verb funcs | `do_server.py:600–644` | Playbook curation (delivery + Auditor). | -| `progress` / `notify_list` / `notify_get` / `notify_ack` / `pr_update` / `read_messages` | verb funcs | `do_server.py:646–840` | Wave-1 parity content tools. | -| `create_optimal_mcp_server` | factory | `optimal_server.py:1068` | Build `roboco-optimal-{agent_id}` server; registers 8 tool groups. | -| `roboco_kb_search` / `roboco_rag_query` / `roboco_kb_stats` | tools | `optimal_server.py:70–213` | Search / RAG / stats. | -| `roboco_ask_mentor` | tool | `optimal_server.py:371` | Primary conversational RAG tool (65s timeout). | -| `roboco_search_error` / `roboco_record_error_solution` | tools | `optimal_server.py:439–535` | Error-pattern memory. | -| `roboco_check_decision` / `roboco_record_decision` | tools | `optimal_server.py:542–625` | Decision memory (`RecordDecisionInput` pydantic model at L33). | -| `roboco_get_standards` / `roboco_validate_action` / `roboco_review_code` | tools | `optimal_server.py:632–765` | Standards + validation. | -| `roboco_record_learning` / `roboco_search_learnings` | tools | `optimal_server.py:772–875` | Learnings. | -| `roboco_clear_index` / `roboco_reindex_all` / `roboco_index_status` | tools | `optimal_server.py:882–993` | Index admin. | -| `roboco_get_proactive_context` | tool | `optimal_server.py:1000` | Stored-then-fresh proactive context for a task. | -| `normalize_index_types` | func | `optimal_server.py:53` | Map legacy `docs` alias → `documentation` before route's `IndexType(...)` conversion. | -| `create_docs_mcp_server` | factory | `docs_server.py:160` | Build `roboco-docs-{agent_id}` server (write/read/list/delete). | -| `WriteDocInput` | pydantic | `schemas/__init__.py:12` | Docs write input (task_id, filename, doc_type, title, content). | -| `roboco_git_status` / `roboco_git_log` / `roboco_git_diff` / `roboco_git_branch_list` | tools | `git_readonly.py:45–119` | Read-only git views via `/api/git/*`. | -| `propose_draft` / `propose_batch` | tools | `intake_server.py:104–215` | Grok intake: POST draft/batch to prompter-live relay; `propose_batch` drops malformed (no string `title` or `name`) entries via `_normalize_batch_drafts` and refuses empty batches. | -| `post_draft` / `post_batch` / `_post_event` | funcs | `intake_server.py:41–102` | Relay POST helpers (never raise; unit-testable with `httpx.MockTransport`). `_post_event` now captures relay response body under `detail` on non-success so the grok agent sees the real reason (0d714b6c). | -| `_draft_title` | func | `intake_server.py:135` | Extract a string title from a batch draft dict, accepting `title` or `name` key; returns None if neither is a string. | -| `_normalize_batch_drafts` | func | `intake_server.py:146` | Filter + normalize MegaTask batch drafts: drops title-less/name-less entries, normalizes name-only drafts onto `title` key. Returns `(well_formed, dropped_count)`. | -| `read_company_state` / `read_task` / `submit_directive` | tools | `secretary_server.py:33–60` | Secretary CEO-authority tools; delegate to `agent_sdk.secretary_driver`. | -| `ApiClient` | class | `utils.py:115` | Async httpx client with agent headers, base URL `settings.internal_api_url`, `get/post/put/patch/delete` + `*_or_error` tuples. | -| `ApiResponse` | class | `utils.py:82` | Response wrapper (`ok`, `status_code`, `json`, `text`, `is_status`). | -| `_get_agent_headers` | func | `utils.py:27` | `X-Agent-ID`/`X-Agent-Role`/`X-Agent-Team`/`X-Agent-Token` (HMAC token from `ROBOCO_AGENT_TOKEN`). | -| `format_error_response` | func | `utils.py:52` | Wraps `roboco.api.schemas.common.error_response`. | -| `create_search_mcp_server` | factory | `search_server.py:83` | `roboco-search-{agent_id}` with `web_search`/`web_fetch`. | - -## Data Flow - -Every server is launched as its own subprocess (`uv run --no-sync python -m roboco.mcp. [agent_id]`) by the orchestrator's `_generate_mcp_config` (for flow/do/git-readonly/optimal/docs/search) or by the grok intake/secretary mains (for intake/secretary). At import time the flow/do servers read `/app/tool-manifest.json` (env `ROBOCO_TOOL_MANIFEST_PATH`) and register only the verbs/tools listed for the role — refusing to start if the manifest is missing, unless `ROBOCO_ALLOW_FULL_TOOLSET` is set (dev/test only; the all-tools fallback was the original bug that let PMs see dev verbs and 404). The optimal/docs/search/intake/secretary/git-readonly servers register their full tool surface unconditionally (role gating is server-side at the route). - -When an agent calls a tool: - -1. **flow/do** — `_post` builds headers (fresh `X-Correlation-ID` per call), POSTs the JSON body to the orchestrator at `/api/v1/flow//` or `/api/v1/do/`. On 404 (a manifest-advertised verb with no matching route) it synthesizes an `invalid_state` Envelope so the agent gets a `remediate` hint instead of a raw `detail` body. On non-JSON body it synthesizes `transport_error`. Otherwise the Envelope is surfaced as-is (success or rejection). -2. **breaker path** — `_classify_rejection` determines whether the envelope is a counted rejection (string kind / dict `error.code` / 422 `detail` list). If counted, `_record_and_check_circuit` POSTs to the local SDK at `ROBOCO_SDK_URL/verb/attempted` (2s timeout). If the SDK returns `open=true`, the original rejection is **replaced** by the SDK's `circuit_envelope` before returning to the agent — stopping retry storms. SDK unreachable → fail-open (return original payload + log). -3. **optimal/docs/search** — `ApiClient` (async httpx) calls the orchestrator's `/optimal/*`, `/docs/*`, `/research/*` routes with `X-Agent-*` headers; shapes the response into a tool-specific dict (status, results, hints). -4. **git-readonly** — `_get` does a synchronous httpx GET to `/api/git/*` with `X-Agent-ID`/`X-Agent-Role`; `raise_for_status` propagates HTTP errors. -5. **intake** — `propose_draft`/`propose_batch` POST directly to `/api/prompter/live/{session}/events` (the prompter-live relay) because grok's `streaming-json` output does not surface tool-call events. Returns a human-readable string (not an Envelope). -6. **secretary** — the three tools delegate to `agent_sdk.secretary_driver._do_*` helpers (shared with the Claude SDK path) and `json.dumps` the result. - -## Mermaid - -```mermaid -graph LR - subgraph AgentContainer["Agent container (per spawn)"] - CC["Claude Code / grok CLI runtime"] - FS["roboco-flow MCP"] - DS["roboco-do MCP"] - GR["roboco-git-readonly MCP"] - OP["roboco-optimal MCP"] - DOC["roboco-docs MCP (conditional)"] - SRCH["roboco-search MCP (conditional)"] - INT["roboco-intake MCP (grok only)"] - SEC["roboco-secretary MCP (grok only)"] - SDK["per-verb SDK loopback :9000"] - end - - CC -->|MCP tool call| FS - CC -->|MCP tool call| DS - CC -->|MCP tool call| GR - CC -->|MCP tool call| OP - CC -->|MCP tool call| DOC - CC -->|MCP tool call| SRCH - CC -->|MCP tool call| INT - CC -->|MCP tool call| SEC - - FS -->|POST /api/v1/flow//| ORC["Orchestrator HTTP gateway"] - DS -->|POST /api/v1/do/| ORC - GR -->|GET /api/git/*| ORC - OP -->|POST /optimal/*| ORC - DOC -->|POST /docs/*| ORC - SRCH -->|POST /research/*| ORC - INT -->|"POST /api/prompter/live/{s}/events"| ORC - SEC -->|POST /api/secretary/*| ORC - - FS -.->|rejection → /verb/attempted| SDK - DS -.->|rejection → /verb/attempted| SDK - SDK -.->|open=true → circuit_envelope| FS - SDK -.->|open=true → circuit_envelope| DS - - ORC -->|Envelope 2xx/4xx| FS - ORC -->|Envelope 2xx/4xx| DS -``` - -## Logical Tree - -``` -roboco/mcp/ -├── __init__.py # import-free package docstring (avoid sibling-load startup tax) -├── utils.py -│ ├── _get_agent_headers() # X-Agent-ID/Role/Team/Token (HMAC) -│ ├── format_error_response() -│ ├── ApiResponse # ok / status_code / json / text / is_status -│ └── ApiClient # async httpx; get/post/put/patch/delete + *_or_error -├── schemas/__init__.py -│ └── WriteDocInput # only survivor of Phase-4 T9 deletions -├── flow_server.py # roboco-flow (intent verbs) -│ ├── StrList # BeforeValidator(coerce_str_list) — SDK XML-ish input -│ ├── _CIRCUIT_REJECTION_KINDS / _DICT_ERROR_CODE_MAP / _classify_dict_error_code / _classify_rejection -│ ├── _remediate_for_kind / _normalize_exception_envelope -│ ├── _build_headers / _post / _verb_from_path / _record_and_check_circuit -│ ├── _ROLE_TO_ROUTE_PREFIX (PO/HM → board) / _role_path -│ ├── dev verbs: give_me_work, i_will_work_on, open_pr, i_am_done, i_am_blocked, unclaim, reassign, resume, sync_branch, i_am_idle -│ ├── QA verbs: claim_review, pass_review(→pass), fail_review(→fail) -│ ├── PR-reviewer verbs: claim_pr_review, post_pr_review, claim_gate_review, pr_pass, pr_fail -│ ├── Doc verbs: claim_doc_task, i_documented -│ ├── PM verbs: triage, triage_all, unblock, complete, escalate_up, i_will_plan, delegate, submit_up, submit_root -│ ├── Board/Main-PM: escalate_to_ceo -│ ├── _TOOLS / _INTENT_TO_PUBLIC -│ └── _load_manifest_flow_tools / _register_tools (fails loud if no manifest; ROBOCO_ALLOW_FULL_TOOLSET escape hatch) -├── do_server.py # roboco-do (content tools) -│ ├── mirror breaker machinery (_CIRCUIT_REJECTION_KINDS, _DICT_ERROR_CODE_MAP, _classify_*, _remediate_for_kind, _normalize_exception_envelope, _record_and_check_circuit) -│ ├── commit, note (handoff done/next), pitch, propose_roadmap, dm, notify, evidence -│ ├── progress, notify_list/get/ack, pr_update, read_messages -│ ├── draft_playbook, approve_playbook, reject_playbook, archive_playbook -│ └── _TOOLS / _load_manifest_do_tools / _register_tools (fails loud; ROBOCO_ALLOW_FULL_TOOLSET escape hatch) -├── optimal_server.py # roboco-optimal (RAG/KB) -│ ├── RecordDecisionInput, normalize_index_types (docs→documentation) -│ ├── _register_search_tools (kb_search, rag_query, kb_stats) -│ ├── _register_indexing_tools (index_code, index_docs) -│ ├── _register_utility_tools (tokens_estimate) -│ ├── _register_mentor_tools (ask_mentor) -│ ├── _register_error_tools (search_error, record_error_solution) -│ ├── _register_decision_tools (check_decision, record_decision) -│ ├── _register_standards_tools (get_standards, validate_action, review_code) -│ ├── _register_learning_tools (record_learning, search_learnings) -│ ├── _register_index_management_tools (clear_index, reindex_all, index_status) -│ ├── _register_proactive_tools (get_proactive_context) -│ └── create_optimal_mcp_server(agent_id) -├── docs_server.py # roboco-docs -│ ├── _handle_write/read/list/delete -│ └── create_docs_mcp_server(agent_id) -├── git_readonly.py # roboco-git-readonly (4 read-only tools) -├── intake_server.py # roboco-intake (grok only) -│ ├── _post_event / post_draft / post_batch -│ ├── _draft_title / _normalize_batch_drafts # title-or-name filter + name→title normalization -│ └── propose_draft / propose_batch (MegaTask) -├── secretary_server.py # roboco-secretary (grok only; delegates to secretary_driver) -└── search_server.py # roboco-search (web research, Board+PM) - ├── _handle_search / _handle_fetch - └── create_search_mcp_server(agent_id) -``` - -## Dependencies - -**Internal (roboco):** -- `roboco.config.settings` — `internal_api_url` (utils), `research_enabled` (orchestrator mount gate). -- `roboco.agents_config` — `get_agent_role`, `get_agent_team` (utils headers). -- `roboco.api.schemas.common.error_response` (utils `format_error_response`). -- `roboco.foundation.policy.content.validators.coerce_str_list` (flow `StrList`). -- `roboco.agent_sdk.secretary_driver` — `_do_read_state` / `_do_read_task` / `_do_submit_directive` (secretary server). -- `roboco.mcp.schemas.WriteDocInput` (docs server). -- `roboco.mcp.utils.ApiClient` / `format_error_response` (optimal, docs, search). - -**External:** -- `mcp.server.fastmcp.FastMCP` — MCP server framework (all servers). -- `pydantic` (`BaseModel`, `Field`, `BeforeValidator`, `Annotated`) — input validation. -- `httpx` — sync (flow/do/git-readonly) + async (utils ApiClient, intake) HTTP. -- `structlog` — logging (flow/do). -- `fastapi.status` — HTTP status constants (optimal). - -**Downstream consumers:** -- `roboco.runtime.orchestrator._generate_mcp_config` — mounts flow/do/git-readonly/optimal (always), docs (docs_roles), search (research_roles + `research_enabled`). -- `roboco.agent_sdk.grok_intake_main` / `grok_secretary_main` — mount intake/secretary for the grok path. -- `roboco.runtime.spawn_manifest` — writes `/app/tool-manifest.json` (the `flow_tools`/`do_tools` lists the flow/do servers read at import). - -## Entry Points - -- `python -m roboco.mcp.flow_server` — `mcp.run()` at `flow_server.py:1028`. Env required: `ROBOCO_AGENT_ID`, `ROBOCO_AGENT_ROLE`; reads `ROBOCO_ORCHESTRATOR_URL`, `ROBOCO_SDK_URL`, `ROBOCO_TOOL_MANIFEST_PATH`. -- `python -m roboco.mcp.do_server` — `mcp.run()` at `do_server.py:954`. Same env as flow. -- `python -m roboco.mcp.git_readonly` — `mcp.run()` at `git_readonly.py:123`. Env: `ROBOCO_AGENT_ID`, `ROBOCO_AGENT_ROLE`, `ROBOCO_ORCHESTRATOR_URL`. -- `python -m roboco.mcp.optimal_server ` — `server.run()` at `optimal_server.py:1102`. Positional `agent_id` arg. -- `python -m roboco.mcp.docs_server ` — `server.run()` at `docs_server.py:251`. -- `python -m roboco.mcp.search_server ` — `server.run()` at `search_server.py:130`. -- `python -m roboco.mcp.intake_server` — `mcp.run()` at `intake_server.py:215`. Env: `ROBOCO_API_URL`, `ROBOCO_PROMPTER_SESSION_ID`. Mounted by `grok_intake_main`, NOT by the orchestrator. -- `python -m roboco.mcp.secretary_server` — `mcp.run()` at `secretary_server.py:64`. Env: `ROBOCO_API_URL`, `ROBOCO_AGENT_ID`, `ROBOCO_AGENT_ROLE`, `ROBOCO_AGENT_TOKEN`. Mounted by `grok_secretary_main`, NOT by the orchestrator. - -Invocation is one subprocess per agent container (the orchestrator writes `roboco-mcp-{agent_id}.json` into `/app/mcp-configs` and the runtime launches each `mcpServers` entry with `uv run --no-sync` pinned to `/app/.venv`). - -## Config Flags - -Env vars read in this slice (all `ROBOCO_*`): - -| Flag / env | Where | Purpose | -|------------|-------|---------| -| `ROBOCO_AGENT_ID` | flow, do, git-readonly (required) | Agent identity for `X-Agent-ID` header + role-path. | -| `ROBOCO_AGENT_ROLE` | flow, do, git-readonly (required) | Role for `X-Agent-Role` + flow route prefix. | -| `ROBOCO_AGENT_TOKEN` | utils `_get_agent_headers` | HMAC agent token injected by orchestrator at spawn; sent as `X-Agent-Token`. | -| `ROBOCO_ORCHESTRATOR_URL` | flow, do, git-readonly | Orchestrator base URL (default `http://roboco-orchestrator:8000`). | -| `ROBOCO_SDK_URL` | flow, do | Per-container SDK loopback for the breaker (default `http://localhost:9000`). | -| `ROBOCO_TOOL_MANIFEST_PATH` | flow, do | Path to the spawn manifest (default `/app/tool-manifest.json`). | -| `ROBOCO_API_URL` | intake, secretary | Orchestrator base URL for the grok-path servers. | -| `ROBOCO_PROMPTER_SESSION_ID` | intake | The live intake session id; without it `propose_draft`/`propose_batch` return a no-op string. | -| `ROBOCO_PROJECT_SLUG` / `ROBOCO_BRANCH` | set by orchestrator into `mcp_env` (consumed indirectly by `/api/git/*`) | Git context. | -| `settings.research_enabled` | orchestrator mount gate for `roboco-search` (not read inside the slice) | Web-research server armed only when true AND role in research_roles. | -| `settings.internal_api_url` | utils `ApiClient.base_url` | Base URL for optimal/docs/search async calls. | -| `ROBOCO_ALLOW_FULL_TOOLSET` | flow `_register_tools`, do `_register_tools` | Dev/test escape hatch: when set, a missing manifest registers the full tool set instead of raising `RuntimeError`. Never set in production — the full-toolset path was the original bug this policy replaced. | - -No default-off feature flag is armed *inside* this slice; the only flag-gated server here is `roboco-search` (gated upstream by `ROBOCO_RESEARCH_ENABLED` in the orchestrator mount). - -## Gotchas - -- **Import-free `__init__.py` is load-bearing.** Re-exporting server factories here would force `optimal_server` (pgvector/ollama stack, ~6s) to load on every `python -m roboco.mcp.` and time out the MCP init — symptom: "roboco-flow/do tools never register". -- **flow/do refuse to start without the manifest** unless `ROBOCO_ALLOW_FULL_TOOLSET` is set. A missing `/app/tool-manifest.json` raises `RuntimeError` at import (production path). Previously the fallback registered all verbs, letting PMs call dev verbs at wrong URLs (404 storm). Local test runs without the bind mount can either set `ROBOCO_TOOL_MANIFEST_PATH` to a real file or set `ROBOCO_ALLOW_FULL_TOOLSET` to skip the hard-fail; the latter must never reach production containers. -- **`pass`/`fail` are Python keywords.** The IntentSpec layer uses `pass_review`/`fail_review`; the MCP layer exposes the public names `pass`/`fail`. `_INTENT_TO_PUBLIC` bridges the two. Forgetting this bridge silently drops QA tools from the palette (the dogfood bug that motivated it). -- **Dict-shaped `error` crashes a naive breaker.** `error in frozenset` raises `TypeError: unhashable type: 'dict'` when FastAPI exception handlers return `error` as a dict. `_classify_rejection` uses `isinstance` checks first — never a `dict in frozenset` membership test. -- **404 handling has three cases (updated 536bbb64).** (1) Bare default FastAPI 404 (`{"detail": "Not Found"}`) → missing route, synthesized as `invalid_state` with a wiring-gap remediate. (2) 404 carrying an `error` field → surfaced as-is (proxy re-status edge case). (3) 404 with a *descriptive* `detail` string → surfaced as `not_found` with a re-fetch remediate (#61). Previously only cases 1 and 2 existed and a descriptive 404 was mis-synthesized as `invalid_state`. -- **`StrList` is not just cosmetic.** A bare `list[str]` annotation hard-rejects the Claude SDK's nested `[[["…"]]]` / `[{item: {$text}}]` tool-input shapes at MCP validation *before* the verb body runs — surfacing as a confusing `1 validation error for i_will_planArguments…`. The `BeforeValidator` flattens first. -- **Breaker is fail-open.** SDK unreachable/slow/malformed → return the original rejection. The breaker is a safety net only; it must never break the gateway path. `_SDK_TIMEOUT=2.0` is tight by design. -- **`note(scope='handoff')` top-level `done`/`next` are the load-bearing fields for PM resumption.** Passing an empty `section={}` used to crash the minimax PMs (`done Field required` → `note circuit_open` → tracing gate blocked `delegate`). The MCP signature now has `done`/`next` as discrete string params. Do not pass `section={}`. -- **`propose_batch` filters and refuses empty batches.** Drafts without a string `title` OR `name` are dropped (via `_draft_title`); a `name`-only draft is normalized onto `title` before posting; if all are dropped it returns an error string instead of POSTing (would silently vanish on the panel side). `dropped` count is sent to the relay. Previously only `title` was accepted — `name`-only drafts were silently dropped even if well-formed (536bbb64). -- **intake/secretary are NOT mounted by the orchestrator.** They are mounted by `grok_intake_main`/`grok_secretary_main` for the grok path only. The orchestrator's `_generate_mcp_config` only knows flow/do/git-readonly/optimal/docs/search. -- **`optimal_server` positional `agent_id` is mandatory.** `python -m roboco.mcp.optimal_server` with no arg prints usage and exits 1. Same for docs/search. -- **`normalize_index_types`** maps the legacy `docs` alias to `documentation` before the route's `IndexType(...)` conversion — without it agents passing `index_types=["docs"]` get a 400. -- **git-readonly uses `raise_for_status`.** Unlike flow/do (which surface 4xx Envelopes), git-readonly propagates HTTP errors as exceptions. A non-200 from `/api/git/*` surfaces to the agent as a transport error, not an Envelope. -- **`X-Correlation-ID` is minted per MCP call** in flow/do (not per session). The orchestrator's `CorrelationIdMiddleware` accepts it as the inbound id and binds structlog + audit row to it. - -## Drift from CLAUDE.md - -CLAUDE.md "MCP servers running per agent container" table lists 5 servers (`roboco-flow`, `roboco-do`, `roboco-git-readonly`, `roboco-optimal`, `roboco-docs`). The actual `roboco/mcp/` directory contains **8** server modules: - -- `roboco/mcp/intake_server.py` (roboco-intake) — omitted from the CLAUDE.md table. Mounted by `grok_intake_main` for the grok intake path, not by `_generate_mcp_config`. -- `roboco/mcp/secretary_server.py` (roboco-secretary) — omitted from the CLAUDE.md table. Mounted by `grok_secretary_main` for the grok secretary path. -- `roboco/mcp/search_server.py` (roboco-search) — omitted from the CLAUDE.md table. Mounted by `_generate_mcp_config` (orchestrator.py:2915) only when `settings.research_enabled` AND role in `(cell_pm, main_pm, product_owner, head_marketing)`. - -CLAUDE.md "roboco-optimal" row says the server exposes `roboco_ask_mentor`, `roboco_kb_search` only. The actual `optimal_server.py` registers **18** tools (search/rag/stats/index_code/index_docs/tokens_estimate/ask_mentor/search_error/record_error_solution/check_decision/record_decision/get_standards/validate_action/review_code/record_learning/search_learnings/clear_index/reindex_all/index_status/get_proactive_context). Understatement, not contradiction. - -CLAUDE.md "roboco-do" row lists `commit, note, say, dm, evidence` and (in the Agent Gateway section) `draft_playbook` for delivery roles + `approve_playbook`/`reject_playbook`/`archive_playbook` for the Auditor. The actual `do_server.py` `do_tools` registry also contains `pitch`, `propose_roadmap` (Product Owner only), `progress`, `notify`, `notify_list`, `notify_get`, `notify_ack`, `pr_update`, `read_messages`, `read_a2a` — none mentioned in CLAUDE.md (`do_tools` is 18 entries, not the 5 named). Understatement. - -CLAUDE.md says the `note`/journal write "returns as soon as the entry is persisted; RAG indexing runs fire-and-forget." The MCP `note` tool itself is synchronous w.r.t. the orchestrator (a single POST); the fire-and-forget behavior is server-side, not visible in this slice — consistent, not drift. - -No contradicted claims found in this slice; the drift is omission (3 servers, ~13 do-tools, ~16 optimal tools not listed). - -## Changes Since Baseline - -Baseline: `fd10cc862c2020b3f639cdb686d427b0198a2441`. Commands: - -``` -git log --oneline fd10cc862c..HEAD -- roboco/mcp/ -git diff --stat fd10cc862c..HEAD -- roboco/mcp/ -``` - -Diff stat: `do_server.py +140/-? `, `flow_server.py +172/-?`, `intake_server.py +20/-?` (3 files, +298/-34). - -Only **one** commit touched this slice since baseline: - -- **`15effce0` — "Chore: 141 Gaps fill-in (#283)"** (merged PR, 2026-06-29). IMPACT on this slice: - - **`flow_server.py`**: added `StrList` (`BeforeValidator(coerce_str_list)`) so the Claude SDK's nested XML-ish tool-input shapes flatten before MCP validation; added `_MISSING_ROUTE_STATUS` 404 handling that synthesizes an `invalid_state` Envelope for manifest-registered verbs whose HTTP route is missing; added `_classify_dict_error_code` + `_classify_rejection` so dict-shaped `error` (FastAPI exception handlers) and 422 `detail`-list rejections count toward the per-verb circuit breaker (previously bypassed → unbounded retries); guarded against `TypeError: unhashable type: 'dict'`. - - **`do_server.py`**: mirrored the same breaker machinery (the dogfood gap: `note(scope='decision')` had looped 8× returning `incomplete_input` with no breaker) + the same 404 missing-route synthesis. - - **`intake_server.py`**: docstring-only change — `propose_draft`/`propose_batch` tool descriptions now declare the per-cell `project_id` on `the_work[]` entries (MegaTask multi-cell fan-out). No logic change in intake. - -No other commits in this slice since baseline. - -> Post-snapshot updates (since 2026-06-29): -> -> - **`536bbb64` — "Chore/all/logical gaps sweep (#286)"** (merged 2026-06-30). IMPACT on this slice: -> - **`flow_server.py` + `do_server.py`**: added `_DICT_ERROR_CODE_MAP` (exact code→kind map, replacing pure-substring classification; closes AUTHENTICATION_REQUIRED mis-routing #161); added descriptive-404 carve-out in `_post` (surfaced as `not_found` instead of `invalid_state` for real resource-not-found 404s, #61); the circuit_open substitution now dict-copies the SDK envelope and nests the original rejection as `inner` (#60); `_register_tools` now accepts `ROBOCO_ALLOW_FULL_TOOLSET` as a dev/test escape hatch instead of always raising `RuntimeError` (#162). -> - **`intake_server.py`**: `propose_batch` now accepts `name` as a fallback for `title` (via new `_draft_title` / `_normalize_batch_drafts` helpers); `name`-only drafts are normalized onto `title` before posting rather than silently dropped (#163). -> - **`docs_server.py`**: `_handle_write` now surfaces a `commit_status == "failed"` outcome in the tool return string, telling the documenter to warn the cell PM when the doc could not be committed to the project repo (#34). -> -> - **`0d714b6c` — "[chore] mcp-servers: normalize exception bodies to Envelope + lift task_id/correlation_id on circuit_open"** (committed 2026-06-30). IMPACT on this slice: -> - **`flow_server.py` + `do_server.py`**: added `_remediate_for_kind` and `_normalize_exception_envelope`; the non-404 JSON path in `_post` now normalizes dict-`error` exception-handler bodies and 422 `detail` lists into the Envelope wire format so agents get `remediate`/`next` instead of raw exception bodies (#232); `_record_and_check_circuit` lifts `task_id`/`correlation_id` from the original rejection onto the circuit_open envelope top level (#359). -> - **`intake_server.py`**: `_post_event` captures relay response body under `detail` on non-success so the grok intake agent sees the real failure reason instead of an opaque `http_422` token (#57). -> -> - **v0.18.0** (2026-07-04): No commit touched `roboco/mcp/` for the X feature-spotlight workstream — `do_server.py`'s `_TOOLS` dict (21 content tools) is unchanged, which is itself the finding: `propose_feature_spotlight` was wired at the role-config + content-actions layers but never registered here. See Regression Risks below. - -## Regression Risks - -| Title | File:Line | Claim | Severity | -|-------|-----------|-------|----------| -| Breaker substitution could mask a real, fixable rejection | `flow_server.py:394`, `do_server.py:349` | **Partially mitigated (536bbb64):** the circuit_open envelope is now a dict-copy of the SDK's envelope with the original rejection nested as `inner` (preserving its kind/message/remediate). The agent sees `circuit_open` at the top level but the underlying rejection survives for ops debugging. The core risk remains: if the breaker trips on a mis-counted storm, the agent stops instead of retrying. | medium | -| Dict-error classification substring fallback → mis-routing risk for unknown codes | `flow_server.py:110`, `do_server.py:94` | **Partially mitigated (536bbb64):** `_classify_dict_error_code` now consults `_DICT_ERROR_CODE_MAP` first (exact match for all known RobocoError codes, closing the AUTHENTICATION_REQUIRED mis-routing bug #161). Only codes NOT in the map fall through to the substring branch. A novel code that accidentally contains `DENIED`/`AUTH`/`PERMISSION` but is semantically different would still mis-route. Risk is now confined to future unknown codes only. | low | -| 404 synthesis assumptions | `flow_server.py:272`, `do_server.py:238` | **Partially mitigated (536bbb64 #61):** a third 404 case was added: a 404 with a *descriptive* `detail` string (not the bare FastAPI default `"Not Found"`) is now surfaced as `not_found`, not `invalid_state`. Residual risk: a future route that returns a bare 404 with no `detail`/`error` field for a real resource-not-found would still synthesize `invalid_state`. The two carve-outs (`error` field → as-is; descriptive `detail` → `not_found`) cover the known cases. | low | -| `_register_tools` raises at import if manifest missing — local dev breakage | `flow_server.py:965`, `do_server.py:896` | **Mitigated (536bbb64):** `ROBOCO_ALLOW_FULL_TOOLSET` env var added as a dev/test escape hatch that bypasses the `RuntimeError` and registers the full tool set. Production behaviour is unchanged (ROBOCO_ALLOW_FULL_TOOLSET is not set in the orchestrator manifest). The mitigant must not leak into production containers. | low | -| `propose_batch` well-formed filter could silently drop intended drafts | `intake_server.py:146` | **Partially mitigated (536bbb64 #163):** `_draft_title` now accepts `name` as a fallback for `title`, and `_normalize_batch_drafts` normalizes `name`-only drafts onto `title` before posting. Residual risk: a draft using a different key (e.g. `label`) is still dropped silently; the `dropped` count is sent but the CEO may not notice. | low | -| Breaker SDK timeout (2s) may be too tight under load | `flow_server.py:55`, `do_server.py:37` | `_SDK_TIMEOUT=2.0` for the loopback `/verb/attempted` POST. Under container CPU contention the SDK could exceed 2s and the breaker fails open (returns original payload) — re-introducing the unbounded-retry condition the breaker was added to stop. Fail-open is safe but defeats the protection. | low | -| Circuit-breaker forwarding does not pass `task_id` for content tools that lack one | `do_server.py:379` | `body.get("task_id")` is sent to the SDK. Several do-tools (`read_messages`, `notify_list`) have no `task_id` — the breaker records `task_id=None`, so the per-verb (not per-task) breaker still works, but any future per-task breaker logic would mis-attribute these. | low | -| `propose_feature_spotlight` granted by role_config but not registered in `do_server.py` | `do_server.py:785-888`, `roboco/services/gateway/role_config.py:120-123` | v0.18.0's Head-of-Marketing-only content verb (`ContentActions.propose_feature_spotlight`) has no wrapper function or `_TOOLS` entry here, unlike `propose_roadmap` which has both (`do_server.py:529`, `:789`). `_register_tools()` only registers the intersection of the manifest's granted verbs and `_TOOLS` (`unknown = [verb for verb in allowed if verb not in _TOOLS]` silently drops anything else), so a spawned Head-of-Marketing agent cannot actually call this tool via MCP despite the role-config grant. Found via static read (grep for the verb name across `do_server.py` returns zero hits) — not reproduced against a live spawn. | Medium | - -## Health - -The slice is coherent and well-defended: the flow/do servers share a near-identical, heavily-commented breaker/404-synthesis contract (with the duplication acknowledged in comments as a deliberate mirror), the manifest-gated registration is fail-loud and blocks the off-role-verb class, and the `StrList` / dict-error / 404 fixes added in `15effce0` close real observed retry-storm and validation-rejection loops. The main integrity concerns are (a) the duplicated breaker logic across flow/do is a drift hazard — a future change to `_CIRCUIT_REJECTION_KINDS` or classification must be applied in both files or the two servers diverge; (b) CLAUDE.md's server table is stale (3 servers + many tools unlisted), which could mislead a reader into thinking intake/secretary/search are not agent-facing MCP servers; (c) the breaker's fail-open posture is correct but means the protection is only as good as the SDK loopback staying responsive within 2s. No correctness bugs observed; the slice is fit for purpose. - -# Choreographer Slice Map - -## Purpose -The Choreographer is the server-side composition layer that turns agent intent-verbs (`give_me_work`, `i_will_work_on`, `i_am_done`, `delegate`, `submit_up`, `submit_root`, `complete`, …) into ordered sequences of atomic TaskService / GitService actions. It owns the precondition gates the lifecycle spec does not model (concurrency invariants, tracing/progress gates, free-text soup, conventions, behind-base, unchanged-PR loop-stoppers) and wraps every composed mutation in a SAVEPOINT via `VerbRunner`. Every verb returns a standardized `Envelope` (`ok` / `error` + `next` + `remediate` + `context_briefing`). - -## Files - -| Path | Role | -|------|------| -| `roboco/services/gateway/choreographer/_impl.py` | The `_LegacyChoreographer` / `Choreographer` class — all verb bodies + guard helpers (~6.9k lines). | -| `roboco/services/gateway/choreographer/_protocol.py` | `ChoreographerHelpers` — TYPE_CHECKING-only stub of helpers role mixins call on `self`, so mypy sees typed signatures (runtime `object`). | -| `roboco/services/gateway/choreographer/_verb_runner.py` | `VerbRunner` — composed-actions runner; wraps `composes` in `session.begin_nested()` SAVEPOINT, runs `pre_side_effects` / `side_effects` outside. | - -## Key Symbols (landmarks only) - -| Name | Kind | File:Line | Responsibility | -|------|------|-----------|----------------| -| `Choreographer` | class | `_impl.py:339` | Composed entry point; deps-injected; exposes verbs + helpers. | -| `ChoreographerDeps` | dataclass | `_impl.py:207` | Dependency injection container (task, work_session, git, a2a, journal, audit, evidence_repo, messaging, product, orchestrator, stream_bus). | -| `_COORDINATOR_ROLES` | ClassVar | `_impl.py:914` | `{main_pm, cell_pm}` — exempt from `already_active`/`paused` claim guards + advisory lock. | -| `give_me_work` | async verb | `_impl.py:766` | Picks next task for agent + builds briefing (institutional memory injected here). | -| `_briefing_for` | async helper | `_impl.py:813` | Builds `context_briefing`. Claim-scoped: `full=True` (context-acquisition verbs only — give_me_work/claims/plan/resume/triage) carries the heavy sections via `_heavy_briefing_sections` (company_goals, recent_team_activity, blockers, task_handoff, institutional_memory); every other verb gets slim signals-only (unread a2a/mentions/notifications + metadata gaps). `include_company_goals=True` is a narrower, cheap-only opt-in (`_resolve_company_goals`) that fetches just the company_goals singleton without the rest of `full`'s heavy sections — used by `board_triage`'s idle branch so the CEO's charter (brand_voice/north_star) still reaches the Product Owner's roadmap-exploration / Head of Marketing's feature-spotlight-exploration one-shot spawns, whose directly-assigned exploration task is never itself a strategic root awaiting PM review (so the `full=True` branch never fires for it). AC coverage stays independent of `full`. | -| `_run_claim_guards` | async helper | `_impl.py:916` | already_active / paused / unmet_dependency (with re-check race narrowing) + `_lane_claim_guard`; `skip_dev_guards=False` param skips dev-only guards for pr_reviewer gate claims (claim_gate_review). | -| `_lane_claim_guard` | async helper | `_impl.py:977` | Out-of-order-start barrier: refuse code leaf behind an earlier open same-assignee sibling. Fail-closed on lookup error. | -| `_claim_plan_start_gate` | async helper | `_impl.py:1179` | spec gate → advisory claim lock (non-PM) → behavioral guards. | -| `_claim_plan_start_run` | async helper | `_impl.py:1251` | `runner.run_intent(verb)` + ensure_work_session + `_touch`. | -| `i_will_work_on` | async verb | `_impl.py:1326` | Dev claim+plan+start path; routes re-entry vs fresh claim. | -| `open_pr` | async verb | `_impl.py:1571` | Pre-flight + `run_intent("open_pr")` (push_branch + create_pr side effects). | -| `i_am_done` | async verb | `_impl.py:1771` | Dev pre-submit; runs `_i_am_done_gate` then `run_intent("i_am_done")`. | -| `_i_am_done_gate` | async helper | `_impl.py:1885` | Ordered gate chain: tracing → submit_qa fields → push → behind_base → quality → toolchain → conventions; then write AC status. | -| `_behind_base_gate` | async helper | `_impl.py:2154` | Refuse submit when branch behind its base (sibling PR merged); fail-open on git error. | -| `_toolchain_broken_guard` | async helper | `_impl.py:1939` | Block delivery gate when agent workspace can't run suite; `reviewer=True` for `pr_pass`. | -| `_conventions_guard` | async helper | `_impl.py:2043` | Run architectural-conventions validator; `block` finding refuses gate. | -| `_pm_task_type_error` | static method | `_impl.py:4752` | Reject a code/non-planning task_type delegated to a PM (cell or main); extracted from `_validate_assignee_task_type` to keep that dispatcher under complexity budget. | -| `i_am_blocked` | async verb | `_impl.py:3086` | Rate-limit parking vs generic block; `run_intent("i_am_blocked")`. | -| `_handle_rate_limited_parking` | async helper | `_impl.py:2983` | Park provider on 429/overload/session-limit. | -| `unclaim` | async verb | `_impl.py:3184` | Release claimed task → pending (optional reassign). | -| `reassign` | async verb | `_impl.py:3343` | Reassign task with `_validate_reassign`. | -| `resume` | async verb | `_impl.py:3422` | Resume paused/blocked task. | -| `sync_branch` | async verb | `_impl.py:3528` | Gate-level rebase verb (new since baseline). | -| `i_am_idle` | async verb | `_impl.py:3678` | Idle signal; auto-pause in_progress tasks; pending-assignment / PM review / auditor guards. | -| `i_will_plan` | async verb | `_impl.py:4071` | PM plan verb; `_pm_sub_tasks_gate` enforces substantive approach/sub_tasks. | -| `delegate` | async verb | `_impl.py:4173` | PM creates subtask; sizing, sibling-dedup, spine-cap, lifecycle guards; `_create_subtask_from_inputs`. | -| `submit_up` | async verb | `_impl.py:5348` | Cell PM opens cell→root PR + enters `awaiting_pr_review`; unchanged-PR guard. | -| `submit_root` | async verb | `_impl.py:6260` | Main PM opens root→master PR + enters gate; umbrella hard-reject + unchanged-PR guard. | -| `_submit_root_unchanged_pr_guard` | async helper | `_impl.py:6149` | Loop-stopper: refuse re-submit when PR head SHA == last `pr_fail` SHA. Fail-open on ambiguity. | -| `_current_pr_head_sha` | async helper | `_impl.py:6196` | Best-effort current PR head SHA via `_project_slug_for` + `git.get_pr_head_sha`. | -| `complete` | async verb | `_impl.py:6599` | Role-dispatch to `cell_pm_complete` / `main_pm_complete`; umbrella-in-progress bypasses spec gate. | -| `main_pm_complete` | async verb | `_impl.py:6496` | Main PM merge + escalate to CEO (never merges master itself). | -| `escalate_to_ceo` | async verb | `_impl.py:6844` | Escalate to `awaiting_ceo_approval`. | -| `VerbRunner.run_intent` | async method | `_verb_runner.py:37` | pre_side_effects → SAVEPOINT(composes) → side_effects; intermediate-None raises INVALID_STATE. | -| `VerbRunner._do_pr_merge` | async handler | `_verb_runner.py:257` | `pr_merge` with `project_id` scoping (cross-repo collision fix) + `resolve_parent_branch`. | -| `ChoreographerHelpers` | stub class | `_protocol.py:31` | TYPE_CHECKING-only typed view of `self` helpers for role mixins. | - -## Data Flow -An MCP `flow/*` call hits the orchestrator → the role-specific gateway verb → `Choreographer.(agent_id, task_id, ...)`. The verb fetches the task (`self.task.get`), builds a briefing (`_briefing_for`), runs the spec gate (`spec.can_invoke_intent(role, verb, t, ctx)`) and any verb-specific preflight guards (free-text soup, claim guards, conventions, behind-base, unchanged-PR). On rejection it emits via `_emit_rejection` with `next`/`remediate`. On allow it calls `VerbRunner.run_intent(verb, t, agent, ctx)`: `pre_side_effects` (e.g. `create_root_pr` for `submit_root`) run OUTSIDE the SAVEPOINT; then `session.begin_nested()` wraps the composed atomic actions (`claim`/`set_plan`/`start`/`submit_qa`/…) re-fetching the task after each; then `side_effects` (push_branch / create_pr / pr_merge) run after the savepoint commits. An intermediate `None` from a composed action raises `INVALID_STATE` (concurrent transition); a trailing `None` flows out as the verb result. The verb wraps the final task in `Envelope.ok(status, next_hint, briefing)` with `.with_introspection(task, role)`. - -## Mermaid -```mermaid -sequenceDiagram - participant Agent - participant Gateway as MCP flow verb - participant Ch as Choreographer - participant Spec as lifecycle spec - participant VR as VerbRunner - participant TS as TaskService - participant Git as GitService - Agent->>Gateway: verb(agent_id, task_id, notes) - Gateway->>Ch: verb(agent_id, task_id, notes) - Ch->>TS: task.get(task_id) - Ch->>Ch: _briefing_for(...) - Ch->>Spec: can_invoke_intent(role, verb, t, ctx) - alt not allowed - Ch-->>Agent: Envelope error (from_decision) + remediate - else allowed - Ch->>Ch: preflight guards (soup / claim / conventions / behind_base / unchanged_pr) - alt guard rejects - Ch-->>Agent: Envelope invalid_state + remediate - else pass - Ch->>VR: run_intent(verb, t, agent, ctx) - VR->>Git: pre_side_effects (create_root_pr) - VR->>TS: session.begin_nested() (SAVEPOINT) - loop composes - VR->>TS: atomic action (claim / set_plan / start / submit_qa / ...) - TS-->>VR: updated task (None on concurrent source-state mismatch) - alt intermediate None - VR-->>Ch: raise INVALID_STATE - end - end - VR->>TS: commit savepoint - VR->>Git: side_effects (push_branch / create_pr / pr_merge) - VR-->>Ch: final task - Ch->>TS: ensure_work_session / _touch - Ch-->>Agent: Envelope ok(status, next, briefing) - end - end -``` - -## Logical Tree -``` -Choreographer (composed class, _impl.py) -├── Deps: task / work_session / git / a2a / journal / audit / evidence_repo / messaging / product / orchestrator / stream_bus -├── Verb bodies -│ ├── Dev: give_me_work, i_will_work_on, open_pr, i_am_done, i_am_blocked, unclaim, resume, sync_branch, i_am_idle -│ ├── PM: i_will_plan, delegate, submit_up, submit_root, complete→{cell_pm_complete, main_pm_complete}, escalate_up, triage, triage_all, unblock, reassign, pm_give_me_work -│ └── Board/CEO: escalate_to_ceo -├── Guard helpers -│ ├── _run_claim_guards → already_active / paused / unmet_dependency / _lane_claim_guard -│ ├── _i_am_done_gate chain → tracing / submit_qa_fields / push / behind_base / quality / toolchain / conventions -│ ├── _submit_up_guard / _submit_up_unchanged_pr_guard / _submit_root_unchanged_pr_guard -│ ├── _guard_free_text / _free_text_soup / _soup_or_decision_env -│ └── _conventions_guard / _toolchain_broken_guard -└── VerbRunner (_verb_runner.py) - ├── pre_side_effects → {create_root_pr} - ├── composes (SAVEPOINT) → {claim, set_plan, start, submit_qa, qa_pass, qa_fail, docs_complete, complete, submit_pm_review, submit_for_review, pr_pass, pr_fail, escalate_to_ceo, block, unblock, resume, pr_review_done} - └── side_effects → {push_branch, create_pr, create_root_pr, pr_merge} -``` - -## Dependencies -- **Internal services**: `TaskService`, `WorkSessionService`, `GitService`, `EvidenceRepo`/`evidence_builder`, `a2a`, `JournalService`, `AuditService`, `ProjectService`/product, orchestrator handle, `StreamEventBus`. -- **Policy (pure)**: `roboco.foundation.policy.lifecycle` (`can_invoke_intent`, `Context`, `Role`, `_INTENT_VERBS`), `foundation.policy.batch` (`is_batch_umbrella`), `foundation.policy.content` markers + `reject_trivial`. -- **Gateway helpers**: `claim_guards` (`already_active_guard`, `paused_tasks_guard`, `unmet_dependency_guard`), `merge_chain.resolve_parent_branch`, `envelope.Envelope`, `remediation` hints. -- **External**: `structlog`, `asyncpg` (via task_service session / SAVEPOINT), `redis` (rate-limit parking), `git`/`gh` CLI (via git_service). - -## Entry Points -- MCP `roboco-flow` server → per-role verb methods (manifest-driven allowlist from `role_config.py`). -- Orchestrator `/api/v1/flow/*` REST endpoints → same `Choreographer` methods. -- Internal cross-verb calls (e.g. `complete` → `cell_pm_complete` / `main_pm_complete`; `_claim_plan_start_run` shared by `i_will_work_on` + `i_will_plan`). - -## Config Flags -- `ROBOCO_TOOLCHAIN_MATCH_ENABLED` — gates `_toolchain_broken_guard` (default-off). -- `ROBOCO_CONVENTIONS_ENABLED` — gates `_conventions_guard` (default-off). -- `ROBOCO_OVERLOAD_BREAK_ENABLED` — rate-limit/overload parking path (default-on). -- `ROBOCO_ORG_MEMORY_ENABLED` — institutional-memory injection in `_briefing_for` (default-off). -- `ROBOCO_GATEWAY_HEALTH_ENABLED` — reaper gateway-health probe (default-on; not directly in choreographer but feeds the parking path). - -## Gotchas -- The SAVEPOINT wraps only DB atomic actions; `pre_side_effects` (e.g. `create_root_pr`) run BEFORE the savepoint and are NOT rolled back if a later composed action raises — they are idempotent by contract. -- `side_effects` (push/create_pr/pr_merge) run AFTER savepoint commit and are idempotent + retryable; a crash between commit and side-effect leaves the PR uncreated (recovered by re-issue / `open_pr` parity). -- `VerbRunner` only raises on an INTERMEDIATE `None` composed action; a trailing `None` is the verb's own "no transition" result and each verb body must handle it (submit_root_finalize, _claim_plan_start_run do; a verb that forgets will None-deref). -- `_lane_claim_guard` fail-closed on lookup error — a DB hiccup rejects the claim (calls `release_dependency_blocked_claim`); acceptable but can briefly bounce a dev. -- `_submit_*_unchanged_pr_guard` FAILS OPEN on every ambiguous case (no recorded sha, no project slug, git error, closed PR) — only exact-unchanged is hard-blocked; a regression in `_current_pr_head_sha` resolver silently re-opens the loop. -- `complete` bypasses the spec gate for an in-progress batch umbrella (`_is_umbrella_in_progress`) and relies on `main_pm_complete`'s own guards — a mis-classified umbrella could skip the AWAITING_PM_REVIEW status constraint. - -## Drift from CLAUDE.md -- CLAUDE.md verb table lists `submit_root` for `main_pm` and `submit_up` for `cell_pm` — matches code (`_impl.py:5348`, `6260`). No drift. -- CLAUDE.md: "PM coordinator concurrency … claim-time concurrency guards skipped for `_COORDINATOR_ROLES`" — matches (`_impl.py:939`, `1234`). No drift. -- CLAUDE.md verb surface omits `sync_branch` from the developer list — code has `sync_branch` at `_impl.py:3528` (added since baseline; memory note `project_sync_branch_tracing_gap.md` flags it). Minor doc drift. -- CLAUDE.md: "PR is created BEFORE QA review" — `i_am_done` gate chain pushes + creates PR context but the actual `create_pr` side-effect runs in `open_pr`/`submit_up`/`submit_root`, not `i_am_done`; consistent with the described flow. No drift. -- CLAUDE.md: "only the CEO merges master; Main PM ready root PR → awaiting_ceo_approval (does NOT merge)" — `main_pm_complete` escalates; `VerbRunner._do_pr_merge` exists for cell-level merges and `create_root_pr` opens but the root merge is CEO-gated. Consistent. No drift. - -## Changes Since Baseline -`git log --oneline fd10cc862c2020b3f639cdb686d427b0198a2441..HEAD -- roboco/services/gateway/choreographer/` → 2 commits touching these files (+814/−89): - -1. `15effce0` — 141 Gaps fill-in (#283): added out-of-order-start guards (`_lane_claim_guard`, `_behind_base_gate`, `sync_branch`), unchanged-PR loop-stoppers (`_submit_root_unchanged_pr_guard`, `_submit_up_unchanged_pr_guard`, `_current_pr_head_sha`), `project_id` scoping on `pr_merge` (cross-repo collision fix), umbrella-in-progress bypass in `complete`, `_submit_root_finalize` None-guard, reviewer flag on `_toolchain_broken_guard`. -2. `3aff6e04` — Close gaps (#285): follow-on touch-ups in the same areas (per-cell project map root-subtask support umbrella handling). - -> Post-snapshot updates (since 2026-06-29): -> - `536bbb64` — logical-gaps sweep: `_run_claim_guards` gains `skip_dev_guards: bool = False` param (pr_reviewer `claim_gate_review` calls skip dev-only already_active/paused/lane guards; dependency guard still runs). `_pm_task_type_error` extracted as new `@staticmethod` on `Choreographer` (`_impl.py:4752`) from `_validate_assignee_task_type` — fixes Main PM omission from the PM-cannot-own-code delegate gate. `_submit_*_unchanged_pr_guard` now logs a `warning` on head_sha resolver failure so fail-open behavior is observable. Matching `skip_dev_guards` stub added to `_protocol.py`. `_impl.py` +78/−0 lines; `_protocol.py` +1 line. -> - `0e7674af` — verb_runner trailing-None side-effect guard + actor_agent_id threading: `run_intent` now skips the `side_effects` loop when the trailing composed action returns `None` (prevents `_do_push_branch(None)` / `_do_pr_merge(None)` AttributeError crash, converting it to a clean caller-handled `None`). `_do_push_branch`, `_do_create_pr`, `_do_create_root_pr`, and `_do_escalate_to_ceo` all forward `actor_agent_id=agent.id` into `git_service` / `task_service`. `main_pm_complete` escalate path in `_impl.py` also forwards `actor_agent_id`. `_verb_runner.py` +52/−8 lines; `_impl.py` +8/−2 lines. - -## Regression Risks - -| Title | File:Line | Claim | Severity | -|-------|-----------|-------|----------| -| Intermediate-None contract depends on every verb body handling trailing None | `_verb_runner.py:89` + `_impl.py:1277,6358` | A verb that forgets the trailing-None guard None-derefs `t.status`; submit_root + claim_plan_start handle it, but any NEW verb using `run_intent` with a possibly-None last action inherits the trap. **Side-effect crash path closed (0e7674af)**: `run_intent` now skips the `side_effects` loop on a trailing `None`, so `_do_push_branch(None)` no longer crashes. Verb bodies still need to handle `None` return for their own error messaging but will not AttributeError. Risk reduced: runner-level crash path fixed; verb-body None-deref in messaging remains a code-discipline risk. | High → Medium | -| `pr_merge` project_id scoping assumes `task.project_id` is non-None | `_verb_runner.py:263` | `project_id=task.project_id` — if a coordination/umbrella task ever reaches `pr_merge` with `project_id=None`, the cross-repo collision guard silently matches nothing or None-keys the scoping; could merge the wrong PR or no-op. | High | -| `_submit_*_unchanged_pr_guard` fails open on resolver regressions | `_impl.py:6175,6240` | Any future break in `_current_pr_head_sha` / `_project_slug_for` / `git.get_pr_head_sha` makes the loop-stopper a no-op, re-opening the 2026-06-27 pr_fail re-submit loop. **Partially mitigated (536bbb64)**: `_current_pr_head_sha` now emits a `structlog.warning` on resolver failure so the fail-open path is observable in logs; the underlying fail-open behavior is intentional and unchanged. | High | -| `_lane_claim_guard` calls `release_dependency_blocked_claim` on lookup error | `_impl.py:988,996` | Fail-closed path releases the claim before returning the error envelope; if the lookup error is transient the dev is bounced + work-session abandoned even though the lane was actually free. | Medium | -| `complete` umbrella-in-progress bypass skips the spec AWAITING_PM_REVIEW status check | `_impl.py:6657` | `_is_umbrella_in_progress` mis-classification (e.g. a non-batch branchless task with matching predicates) lets a non-awaiting_pm_review task reach `main_pm_complete`/`cell_pm_complete`. | Medium | -| `_run_claim_guards` dependency re-check narrows but does not close the race | `_impl.py:965-967` | The re-check returns None (skip release) when fresh read sees deps met, but the window between re-check and the caller's claim is still unlocked; a concurrent terminal transition there is benign (monotonic), but a non-monotonic future status could re-open. | Low | -| `submit_root` runs `create_root_pr` as a pre_side_effect OUTSIDE the savepoint | `_verb_runner.py:67` + `_impl.py:6339` | If a later composed `submit_for_review` raises, the root→master PR is already opened and NOT rolled back; re-issue is idempotent-by-contract but a non-idempotent future pre_side_effect would leak. | Medium | -| `_i_am_done_gate` writes AC criteria status AFTER all gates pass but BEFORE `run_intent` | `_impl.py:1910` | `_write_criteria_status` runs in the gate phase; if `run_intent("i_am_done")` then raises, the AC status rows persist for a task that did not transition — a stale write the next attempt must overwrite. | Low | - -## Health -The slice is structurally sound: the SAVEPOINT boundary, intermediate-None INVALID_STATE guard, and role-exempt coordinator concurrency model are coherent and well-documented. The highest-temperature areas are the new (since baseline) fail-open loop-stoppers and fail-closed lane guard — both correct by design but tightly coupled to resolvers (`_current_pr_head_sha`, `has_earlier_incomplete_code_sibling`) whose regressions silently revert the protection. The 814-line delta is concentrated in guard additions rather than control-flow rewrites, so baseline behavior is largely preserved; the main residual risk is verb-body discipline around the trailing-None contract for any future verb. - -# task-service slice - -## Purpose -`TaskService` is the authoritative owner of the task lifecycle: CRUD, hierarchical create (incl. MegaTask umbrella + root-subtasks), claim/locking, every status transition, completion/CEO approval/cancellation, dependency DAG wiring, rework routing, and completion-time learning capture. All status writes funnel through `_validate_and_set_status` + `_emit_status_transition_audit` so the audit journey and the `revision_count` rework counter stay in lockstep with real task state. - -## Files - -| Path | Role | -|------|------| -| `roboco/services/task.py` | Single 8.7k-line service module implementing TaskService + a few internal dataclass containers (`_CompletionSnapshot`, `SoftBlockInput`, `SoftBlockInfo`, `GatewayAgentView`). | - -## Key Symbols - -| Name | Kind | File:Line | Responsibility | -|------|------|-----------|----------------| -| `_validate_and_set_status` | method | task.py:548 | Single chokepoint: validate transition + git requirements, set status, poke dispatcher, emit audit. | -| `_emit_status_transition_audit` | method | task.py:652 | Write `task.` audit row in caller session; bump `revision_count` on entry into `needs_revision`. | -| `_alert_auditor_of_rework` | method | task.py:1019 | Best-effort helper that asks `NotificationDeliveryService` to send a HIGH `ALERT` to the auditor when a task enters `needs_revision`. Called from `fail_qa`, `pr_fail`, and `request_changes` immediately after `await self.session.flush()` so the transition row is visible before the alert is dispatched. | -| `create` | method | task.py:864 | New task; depth/batch/AC validation; branchless/umbrella flags; baseline constraints attachment; (V2) vault materialize-on-create. | -| `_attach_baseline_constraints` | method | task.py:971 | Append conventions baseline constraints to task prompt (gated `conventions_enabled`). | -| `_materialize_vault_note` | method | task.py:910 | V2: best-effort vault seam called from `create` — assembles + writes a deterministic task note (narrative placeholder) so a task is visible in the vault from the moment it exists, not just at Auditor curation/rebuild. Gated `obsidian_vault_enabled`; swallows + logs any failure. | -| `list_updated_since` | method | task.py:7101 | V2: tasks touched (`COALESCE(updated_at, created_at)`) since a timestamp, ascending, paged — the vault janitor's changed-task re-projection set. | -| `list_archive_candidates` | method | task.py:7124 | V2: terminal tasks whose terminal timestamp falls in `[after, before)`, ascending, paged — the vault janitor's archival-pass candidate window (watermark-bounded so a sweep never rescans the whole archive). | -| `sample_stale_tasks` | method | task.py:7154 | V2: random sample of tasks last touched before a cutoff — the vault janitor's drift-verification sample. | -| `activate` | method | task.py:1577 | `backlog→pending` (PM only); batch-shape guard. | -| `_ensure_branch_for_task` | method | task.py:1675 | Branch resolution for claim; `""` for branchless/umbrella. | -| `_auto_create_branch` | method | task.py:1833 | Cut hierarchical branch + per-task worktree add (F123). | -| `_remove_task_worktree` | method | task.py:1913 | Low-level worktree removal by task id. | -| `admin_set_status` | method | task.py:2060 | Privileged override (bypass validator); restores pre-block owner; still emits audit. Post-#2176: the blocked→pending/in_progress restore path now attributes the audit row to the admin actor (not the restored owner) and emits a `task.admin_override` row (`forced=False, restore=True`) independent of the `force` flag. | -| `_restore_block_ownership` | method | task.py:8526 | Factored out of `_apply_pre_block_restore` (b3558d4e complexity split): applies snapshotted status/owner restore (branchless in_progress→pending divert), returns `(pre_status, restored_status, restored_owner)`. | -| `_emit_admin_override_audit` | method | task.py:8555 | Factored out of `_apply_pre_block_restore`: writes `task.admin_override` audit row for admin-triggered blocked restores (`forced=False, restore=True`). | -| `claim` | method | task.py:3112 | `FOR UPDATE` lock + `_validate_claim_preconditions` + `_finalize_claim`; calls `_validate_and_set_status(claimed)`. | -| `_validate_claim_preconditions` | method | task.py:2883 | Per-claim validator chain: status, `_claim_blocked_by_sequencing` (dependency + sequence), team, pre-assignment theft, self-review. | -| `_claim_blocked_by_sequence` | method | task.py:2805 | Strict sibling-sequence gate: a PENDING/`needs_revision` task with parent + effective `sequence` (`COALESCE(sequence, 0)`) N is held while any same-parent sibling with a strictly lower effective sequence is non-terminal — assignee-blind, independent of `dependency_ids`. Ties run parallel; cancelled siblings never block. | -| `_claim_blocked_by_dependencies` | method | task.py:2781 | `unmet_dependency` TIMING gate: refuses claim while any `dependency_ids` entry is non-terminal. | -| `is_pending_claim_blocked` | method | task.py:2864 | Read-only wrapper over `_claim_blocked_by_sequencing` (dependency OR sequence) so the orchestrator dispatcher can filter a doomed claim before attempting it (`_pending_claim_blocked` in orchestrator.py). | -| `stamp_wave_sequence` | method | task.py:7452 | Stamps a freshly delegated subtask's `sequence` as `1 + max(sequence of each same-parent dependency target)`, or `0` when independent — so independent siblings tie (parallel under the sequence gate) while colliding/ordered work ascends. Runs POST-wiring (after the collision DAG / cross-cell edges land); PM-authored sequences are never rewritten. | -| `_apply_dependency_lineage` / `_merge_one_dependency` | method | task.py:2308 / 2337 | Claim-time content assist (not a gate): merges each same-repo dependency's landed work into a freshly cut branch when it lies outside the branch's own ancestor chain (`GitService.merge_dependency_lineage`); a real conflict aborts the merge and stamps a `dependency_lineage_conflict` transition note instead of failing the claim. | -| `_finalize_claim` | method | task.py:2925 | Work-session create/inherit, branch cut, proactive-context injection. | -| `_inject_proactive_context` | method | task.py:3154 | Briefing injection at claim (institutional memory when `org_memory_enabled`). | -| `_completion_learnings_for` | method | task.py:2798 | Distill one lesson (ON) vs legacy raw capture (OFF). | -| `_extract_completion_learnings` | method | task.py:2837 | Fire-and-forget learning record + RAG indexing. | -| `start` | method | task.py:3354 | `claimed→in_progress`. | -| `unclaim_for_agent` / `_force_unclaim_to_pending` | method | task.py:3579 / 3507 | Release claim to pool; abandon stale work session. | -| `block` / `soft_block` / `unblock` | method | task.py:3760 / 3823 / 3897 | Snapshot pre-block owner; restore on unblock. | -| `submit_for_qa` | method | task.py:4065 | `verifying→awaiting_qa`; clears claimed_by (passes explicit audit_agent_id). | -| `pass_qa` / `fail_qa` | method | task.py:4112 / 4187 | QA verdict; `fail_qa` routes back to original dev (marker → work-session fallback), then calls `_alert_auditor_of_rework` after flush to emit a best-effort auditor rework ALERT. | -| `_resolve_revision_dev` | method | task.py:4301 | Work-session fallback when `original_developer` marker missing. | -| `docs_complete` | method | task.py:4336 | `awaiting_documentation→awaiting_pm_review` (parallel completion). | -| `request_changes` | method | task.py:9975 | PM merge-review request-changes path; transitions to `needs_revision`, then calls `_alert_auditor_of_rework` after flush to emit a best-effort auditor rework ALERT. | -| `submit_for_pm_review` / `complete` | method | task.py:4690 / 4882 | PM review submit + completion / CEO escalation chain. | -| `_apply_complete_approval_chain` | method | task.py:4811 | Leaf→completed vs root→awaiting_ceo_approval. | -| `_assert_pr_merged_for_complete` | method | task.py:4845 | PR-merged gate before `complete`. | -| `apply_escalation` | method | task.py:4942 | `in_progress→blocked` direct status set + audit emit (bypasses validator by design). | -| `escalate_to_ceo` | method | task.py:5064 | `awaiting_pm_review→awaiting_ceo_approval`; gained `actor_agent_id: UUID | None = None` param (stamped as `audit_agent_id` so the transition row attributes to the specific PM/Board agent, not just the role). | -| `ceo_approve` | method | task.py:5146 | CEO merges then approves; `awaiting_ceo_approval→completed`. | -| `ceo_reject` | method | task.py:5414 | Reject → `needs_revision` (dev) or `pending` (branchless root via admin_set_status); now validates `reason` (`reject_trivial` — previously an uncaught Pydantic error could 500 on empty/trivial input) and inserts one `origin=ceo` Finding onto the revision-findings ledger; the branchless-root path manually bumps `revision_count` + emits `task.ceo_reject` since it skips `_emit_status_transition_audit`. See `docs/map/review-findings.md`. | -| `_delete_task_branch_best_effort` | method | task.py:6726 | Cancel-path cleanup: remote branch delete + `_remove_task_worktree_best_effort(force_branch_delete=True)`; skipped once branch is unset. | -| `_remove_task_worktree_best_effort` | method | task.py:6767 | Shared worktree+local-branch+previews cleanup called by both cancel and terminal paths; force-deletes the local branch ref unless it's an environment-ladder rung (`effective_environments`). | -| `_cleanup_task_previews_best_effort` | method | task.py:6804 | `rmtree` the task's `.previews/{task8}` video-render dir; path-containment-checked against the project workspace dir before deleting. | -| `_remove_task_worktree_on_terminal` | method | task.py:6829 | Best-effort worktree + local-branch (force `-D`, squash-merge is never an ancestor) + previews cleanup on complete/ceo_approve; no-op for branchless. | -| `cancel` | method | task.py:5644 | Cascade-cancel descendants through the validator. | -| `reassign` / `reassign_active_claim` | method | task.py:7657 / 7807 | Reassignment with Board/Main-PM diversion guards. | -| `pr_pass` / `pr_fail` | method | task.py:8100 / 8137 | In-path PR-review gate verdicts; `pr_fail` transitions to `needs_revision`, then calls `_alert_auditor_of_rework` after flush to emit a best-effort auditor rework ALERT. | -| `list_open_docs_sync_tasks` | method | task.py:1580 | Returns open `source=docs_sync` tasks, optionally scoped to one release version via the `docs_sync_release_version` marker. The version predicate is applied in SQL so dedupe/cap checks do not haul every open row into Python. | - -## Data Flow -Request → `TaskService` loads `TaskTable` (`get`/`_load_task_or_raise`) → validates role/transition (`validate_task_transition`) + git reqs (`validate_git_requirements`, branchless/umbrella/external-review exempt) → mutates columns → `_emit_status_transition_audit` writes `AuditLogTable` row + bumps `revision_count` in the same session → pokes orchestrator `trigger_dispatch()` → fires fire-and-forget background tasks (RAG indexing, learning distillation, worktree cleanup, work-session close). Terminal states trigger `_unblock_dependents` to revive waiting tasks. - -## Mermaid -```mermaid -stateDiagram-v2 - [*] --> backlog: create - backlog --> pending: activate (PM) - pending --> claimed: claim (role-matched) - claimed --> in_progress: start - claimed --> pending: unclaim - in_progress --> blocked: block - blocked --> in_progress: unblock(restore) - in_progress --> verifying: submit_for_verification - verifying --> awaiting_qa: submit_for_qa (PR open) - awaiting_qa --> awaiting_documentation: pass_qa - awaiting_qa --> needs_revision: fail_qa - awaiting_documentation --> awaiting_pm_review: docs_complete - in_progress --> awaiting_pr_review: submit_up/submit_root (PM) - awaiting_pr_review --> awaiting_pm_review: pr_pass - awaiting_pr_review --> needs_revision: pr_fail - awaiting_pm_review --> completed: complete (leaf) - awaiting_pm_review --> awaiting_ceo_approval: escalate_to_ceo (root) - awaiting_ceo_approval --> completed: ceo_approve - awaiting_ceo_approval --> needs_revision: ceo_reject (dev) - awaiting_ceo_approval --> pending: ceo_reject (branchless root) - needs_revision --> claimed: re-claim - completed --> [*] - cancelled --> [*] -``` - -## Logical Tree -- TaskService - - State core: `_validate_and_set_status`, `_emit_status_transition_audit`, `admin_set_status`, `_restore_block_ownership`, `_emit_admin_override_audit` - - Create/shape: `create`, `_validate_parent_depth`, `_validate_batch_membership`, `activate` - - Branch/worktree: `_ensure_branch_for_task`, `_auto_create_branch`, `_delete_task_branch_best_effort`, `_remove_task_worktree*`, `_cleanup_task_previews_best_effort` - - Claim: `claim`, `_validate_claim_preconditions`, `_claim_blocked_by_sequence`, `_claim_blocked_by_dependencies`, `_finalize_claim`, `_apply_dependency_lineage`, `_inject_proactive_context`, `acquire_*_lock` - - Lifecycle verbs: `start`, `block*`, `unblock`, `pause`, `resume`, `submit_for_qa`, `pass_qa`, `fail_qa`, `docs_complete`, `submit_for_pm_review` - - Completion: `complete`, `_apply_complete_approval_chain`, `ceo_approve`, `ceo_reject`, `cancel` - - Rework routing: `fail_qa`, `_resolve_revision_dev`, `ceo_reject` - - Learning/indexing: `_completion_learnings_for`, `_extract_completion_learnings`, `_trigger_completion_hooks`, `_index_*_background` - - Dependencies/sequencing: `add_dependency`, `wire_sibling_collision_dag`, `wire_cell_task_wave_chain`, `_unblock_dependents` - - Reassign/escalate: `reassign*`, `escalate*`, `_maybe_divert_*` - - PR gate: `pr_gate_claim`, `submit_for_review`, `pr_pass`, `pr_fail` - - Queries: `list_*`, `count_*`, `*_ac_coverage`, `all_subtasks_terminal` - -## Dependencies -- `roboco.foundation.policy.lifecycle` (transitions, role restrictions, git requirements, `is_branchless_coordination`, `is_batch_umbrella`) -- `roboco.foundation.policy.batch` / `sequencing` (batch predicates, sibling DAG) -- `roboco.services.work_session` (close/abandon), `roboco.services.workspace`, `roboco.services.learning`, `roboco.services.memory_distiller` -- `roboco.services.conventions` (`_attach_baseline_constraints`) -- `roboco.db.tables` (`TaskTable`, `AuditLogTable`, `WorkSessionTable`, `ProjectTable`) -- `roboco.api.deps.get_orchestrator` (lazy; dispatch poke), `roboco.config.settings` -- Markers / `extract_original_developer` helpers - -## Entry Points -- `TaskService.create` / `create_subtask` — task creation (orchestrator intake, batch confirm, gateway delegate). -- `TaskService.claim` — gateway `give_me_work` / `i_will_work_on` / `claim_review` / `claim_doc_task`. -- Lifecycle verbs (`start`, `submit_for_qa`, `pass_qa`/`fail_qa`, `docs_complete`, `submit_for_pm_review`, `complete`, `cancel`, `pr_pass`/`pr_fail`, `escalate_*`, `ceo_approve`/`ceo_reject`) — all gateway flow verbs. -- `admin_set_status` — operator PATCH + orchestrator auto-recover. -- `wire_*` / `add_dependency` — `SequencingService` / `BatchPlacement`. - -## Config Flags -- `ROBOCO_ORG_MEMORY_ENABLED` — `_completion_learnings_for` swaps raw capture for one distilled lesson (task.py:2810). -- `ROBOCO_CONVENTIONS_ENABLED` — `_attach_baseline_constraints` skipped when off (task.py:1000). -- (Indirect, via called services) `ROBOCO_SELF_HEAL_*`, `ROBOCO_CI_WATCH_*`, `ROBOCO_DEP_UPDATE_*`, `ROBOCO_RELEASE_MANAGER_*` gate the `list_open_*`/`list_open_release_proposals` query paths. - -## Gotchas -- `_emit_status_transition_audit` writes the audit row in the CALLER's session — callers that clear `claimed_by` before transitioning MUST pass `audit_agent_id` or the row lands unattributed (task.py:688). -- `apply_escalation` (task.py:4942) sets `task.status` directly and calls `_emit_status_transition_audit` deliberately bypassing the strict validator (blocked is a terminal-ish hold) — only audited privileged-style path besides `admin_set_status`. -- `fail_qa` accepts `claimed`/`in_progress` (QA is mid-review); the `original_developer` marker is unreliable — the work-session fallback (`_resolve_revision_dev`) is load-bearing (task.py:4248). -- Branchless/umbrella/external-review tasks are exempt from the branch gate inside `GitContext` (task.py:597-611); umbrella is also exempt from the `awaiting_pm_review→awaiting_ceo_approval` pr_number gate. -- `complete()` requires PR merged (`_assert_pr_merged_for_complete`) EXCEPT branchless roots; `ceo_approve` separately checks `work_session.pr_status=="merged"` and refuses otherwise. -- Background indexing/learning/cleanup tasks are tracked on `self._background_tasks` and are best-effort — a failure never blocks the transition. -- The sequence gate (`_claim_blocked_by_sequence`) is enforced ONLY in `_validate_claim_preconditions`, i.e. inside `claim` itself — both the gateway claim verbs AND the orchestrator's raw dispatch claim cross it because they both funnel through `TaskService.claim`, unlike the pre-#382 dependency gate which briefly lived only on the gateway side. Any future claim path that bypasses `TaskService.claim` (a raw `admin_set_status`, for instance) does NOT get sequence enforcement. -- `_apply_dependency_lineage` is scoped to SAME-REPO dependencies only (`dep_task.project_id != ctx.project.id` short-circuits) — a cross-repo dependency edge (e.g. a MegaTask root-subtask in another project) has no shared git history to merge and is silently skipped; the dependency TIMING gate still holds the claim regardless of repo. -- `TaskTable.orchestration_markers` is generic `JSON`, not `JSONB`. Any SQL predicate on a marker key must use `.as_string()` (or the JSON dialect's generic comparator), not `.astext`, which is JSONB-only and raises `AttributeError` at compile time. `list_open_docs_sync_tasks(version=...)` at task.py:1596 is the current example; the inline comment records the rationale. -- Both cancel and terminal-completion now force-delete (`-D`) the task's LOCAL branch ref in the assignee's clone alongside the worktree — a completed task's PR was squash-merged (its local ref is never an ancestor of base, so a "safe" `-d` refuses unconditionally) and a cancelled task's work is discarded by decision, so the ref is spent either way. Skipped when the branch name coincides with an environment-ladder rung (`effective_environments`), which outlives any one task. - -## Drift from CLAUDE.md -- CLAUDE.md states ceo_reject "~4779 skips _validate_and_set_status in branchless path". Actual: branchless branch of `ceo_reject` is at task.py:5488 and routes through `admin_set_status` (which DOES emit audit at task.py:2100). The non-branchless branch DOES call `_validate_and_set_status` (task.py:5461). No audit gap — the line reference is stale. -- CLAUDE.md "PR is created BEFORE QA review" — `submit_for_qa` enforces `pr_number` via `validate_git_requirements` (consistent, no drift). -- CLAUDE.md verb table lists `pr_reviewer` `pr_pass`/`pr_fail` — present at task.py:8100/8137 (consistent). - -## Changes Since Baseline -`git log fd10cc86..HEAD -- roboco/services/task.py`: -- `15effce0` Chore: 141 Gaps fill-in (#283) — bulk gap closure; transition audit chokepoint + `revision_count` centralization (task.py:685-706), branchless/umbrella git-context exemptions, fail_qa work-session fallback, ceo_reject branchless routing. -- `3aff6e04` Chore: Close gaps (#285) — follow-on gap close (worktree-on-terminal cleanup F123 Phase C, escalation audit emit, rework routing hardening). - -> Post-snapshot updates (since 2026-06-29): `20f1f9ba` admin_set_status: thread actor_id/actor_role into `_apply_pre_block_restore`; blocked→pending/in_progress restore now attributes the audit row to the admin actor (not the restored owner) and emits a `task.admin_override` row (forced=False, restore=True) independent of the force flag. `b3558d4e` complexity: extract `_restore_block_ownership` (line 8526) + `_emit_admin_override_audit` (line 8555) from `_apply_pre_block_restore` — no behavior change, splits a C-rank block for the xenon gate. `0e7674af` escalate_to_ceo gains `actor_agent_id: UUID | None = None` param stamped as audit_agent_id; push_branch / create_pr / create_root_pr / escalate_to_ceo side-effect handlers in the verb runner now forward actor_agent_id (was dropped, causing wrong workspace or role-only audit attribution). `8f3f4236` (#452) "sequence is the bar" — adds `_claim_blocked_by_sequence` + `_validate_claim_preconditions` wiring, `stamp_wave_sequence` (replacing a raw per-sibling delegation ordinal), and migration 069 (`tasks.parent_task_id` index, the sibling probe's hot path). `f2834cf5` (#466) adds `_apply_dependency_lineage`/`_merge_one_dependency`, called from `_create_branch_in_project` right after a fresh branch cut. `61e00832` (PR #492) added `_alert_auditor_of_rework()` and invoked it from `fail_qa`, `pr_fail`, and `request_changes` after each transition to `needs_revision`, wiring the reactive auditor ALERT path. `f6c75237` (PR #509) restored those `_alert_auditor_of_rework()` calls after they were accidentally deleted by the docs-sync PR: all three call sites now dispatch the alert immediately after `await self.session.flush()` so the `needs_revision` transition row is committed before the auditor notification is created. The same commit also changed the descendant-traversal casts in `_supersede_replacement_landed` and `get_all_descendants`, but it used `cast(UUID, child.id)` with a scoped `# noqa: TC006` and `child.id` with a `# type: ignore[arg-type]`, respectively. `e4b7dd0f` / PR #511 reverted those two cast regressions to the preferred string-literal form `cast('UUID', child.id)` with no lint or type suppression, leaving `DOCS_SYNC_SOURCE` and `list_open_docs_sync_tasks` untouched. -> -> (uncommitted, branch `feature/findings-ledger`, 2026-07-11) Revision-findings ledger: `_audit_events_for` (task.py:997) gains `task.request_changes` (agent_role `cell_pm`/`main_pm`) and `task.ceo_reject` (agent_role `ceo`) branches alongside the existing `task.qa_fail`/`task.pr_fail`; `ceo_reject` gains reason validation + a ledger `Finding` insert (see above); `qa_fail` and `request_changes` drop their raw `dev_notes` appends (the mirror-column data-loss bug) in favor of the ledger + a structured note. Full detail: `docs/map/review-findings.md`. -> -> (open PR #548, branch `feature/wave-2-hygiene-charts`, 2026-07-17) Local branch refs stop leaking: `_delete_task_branch_best_effort`/`_remove_task_worktree_on_terminal` now also force-delete the assignee's local branch ref (via new `WorkspaceService.delete_local_branch`) and rmtree the task's `.previews/{task8}` video-preview dir, both skipped for environment-ladder rungs. See `docs/map/worksession-git.md` for the paired `GitService.cleanup_stale_branches` sweep. - -## Regression Risks - -| Title | File:Line | Claim | Severity | -|-------|-----------|-------|----------| -| `ceo_approve` skips work-session close | task.py:5146 | `ceo_approve` calls `_remove_task_worktree_on_terminal` but NOT `_close_work_session_for_task` (only `complete()` at 4934 does). Approved-via-CEO tasks leave the WorkSession row not marked `completed`/closed → reporting/session-resolution drift. | High | -| `ceo_approve` skips full completion hooks | task.py:5200-5210 | Only fires `_extract_completion_learnings` manually; skips `_trigger_completion_hooks` so code-changes RAG indexing + decision indexing never run for CEO-approved (root) tasks. | Medium | -| `apply_escalation` bypasses validator | task.py:4942 | Sets `task.status` directly then emits audit; a caller passing a wrong target status would skip `validate_task_transition`/git-req checks. Relies on call-site discipline. | Medium | -| `fail_qa` route depends on unreliable marker | task.py:4228-4272 | Fast path reads `original_developer` marker; if absent, falls to `_resolve_revision_dev`. If both miss (no dev work session, e.g. parent-only edit) task is unassigned to pool → PM may grab a dev task (the original 2026-06-27 loop). | High | -| Branchless `ceo_reject` uses `admin_set_status` | task.py:5488 | Bypasses strict validator (intended) but `awaiting_ceo_approval→pending` is not in `VALID_TRANSITIONS`; any future tightening of admin override could wedge coordination-root rejection. | Medium | -| `revision_count` bump is in audit helper only | task.py:702-706 | Any future transition path that sets `task.status` directly WITHOUT calling `_emit_status_transition_audit` (mirroring `apply_escalation`'s pattern) would silently skip the rework counter — metric drift. | Medium | -| `_remove_task_worktree_on_terminal` silent-fail | task.py:5614-5627 | Cleanup failure is logged-warning only; on recurring FS/permission error worktrees leak indefinitely with no operator signal beyond logs. | Low | -| Concurrent mid-verb state change | task.py:548 | `_validate_and_set_status` does not re-fetch the task after validation; a concurrent committer could flip status between load and set, producing an invalid edge that the validator already passed. Mitigated upstream by verb-runner savepoints, not here. | Medium | -| `cancel` cascade swallows role violations | task.py:5679-5690 | Descendants that fail role validation are skipped (warning), so a cancel can leave non-terminal descendants orphaned in `awaiting_ceo_approval` (only CEO may cancel those). | Medium | -| `submit_for_qa` clears `claimed_by` before transition | task.py:568-572, 4065 | Relies on `audit_agent_id` being passed to attribute the row to the dev; if a future caller forgets, the `awaiting_qa` audit row lands `agent_id=NULL`. | Low | - -## Health -`TaskService` is the most load-bearing service and the most hardened: the audit chokepoint, `revision_count` centralization, branchless/umbrella exemptions, and worktree-on-terminal cleanup all landed in the two recent gap-closure commits. The residual risk is concentrated in the two CEO-path asymmetries (`ceo_approve` not closing the work session / not running the full completion hooks) and in `fail_qa`/`ceo_reject` rework routing, which depends on the unreliable `original_developer` marker and a work-session fallback that has no guarantee a developer session exists. - -# RoboCo Slice Map — `worksession-git` - -Scope key: `worksession-git` Repo root: `/Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco` Files in scope: -- `roboco/services/work_session.py` -- `roboco/services/git.py` -- `roboco/templates/git/` (`__init__.py`, `branch.py`, `commit.py`, `constants.py`, `pr_internal.py`, `pr_root.py`) - -## Purpose - -This slice is the git substrate every delivery agent works on. `GitService` runs all git subprocesses (status/commit/branch/push/rebase), mints branches + commit messages + PR bodies from templates, and drives the GitHub REST API for PR create/merge/close. `WorkSessionService` persists the per-claim row that links an agent to a task's branch/commits/PR and enforces the single-active-per-task invariant. The `roboco/templates/git/` package is the pure rendering layer for branch names, commit messages, and internal/root PR bodies. Together they are the boundary between the task lifecycle and the actual git history on disk + GitHub. - -## Files - -| Path | Role | approx LOC | -|------|------|------------| -| `roboco/services/work_session.py` | WorkSession CRUD, commit/file tracking, PR-lifecycle record, single-active invariant | 685 | -| `roboco/services/git.py` | Git subprocess execution, branch/commit/PR/rebase/merge/sync, GitHub REST API, conventions validator runner | 4596 | -| `roboco/templates/git/__init__.py` | Package re-exports for branch/commit/PR templates | 48 | -| `roboco/templates/git/branch.py` | Hierarchical branch name builder + root-task resolver | 131 | -| `roboco/templates/git/constants.py` | `BRANCH_TYPES`, `COMMIT_TYPES`, `MAX_TASK_DEPTH`, length constants | 52 | -| `roboco/templates/git/commit.py` | `CommitContext` + `build_commit_message` (traceability links) | 114 | -| `roboco/templates/git/pr_internal.py` | Internal (subtask→parent) PR title/body builder | 140 | -| `roboco/templates/git/pr_root.py` | Root (→master, CEO-level) PR title/body builder with task tree | 245 | - -## Key Symbols - -| Name | Kind | File:Line | Responsibility | -|------|------|-----------|----------------| -| `WorkSessionService` | class | work_session.py:29 | Session lifecycle + single-active invariant | -| `WorkSessionService.create` | method | work_session.py:50 | Validate project/task, refuse duplicate, supersede stale ACTIVE, insert row | -| `WorkSessionService.get_active_for_task` | method | work_session.py:184 | Most-recent ACTIVE row (resilient to dup-rows defect) | -| `WorkSessionService.supersede_active_sessions_for_task` | method | work_session.py:247 | ABANDON every other ACTIVE session for a task (single-active) | -| `WorkSessionService.add_commit` | method | work_session.py:363 | Append dedup'd commit SHA to session.commits | -| `WorkSessionService.create_pr` | method | work_session.py:427 | Record pr_number/pr_url/pr_created_at | -| `WorkSessionService.merge_pr` | method | work_session.py:488 | Record merge + COMPLETED; idempotent active-guard (F062) | -| `WorkSessionService.close` | method | work_session.py:618 | Idempotent COMPLETED on task completion | -| `WorkSessionService.abandon` | method | work_session.py:577 | ABANDONED + ended_at (non-active → warning + None) | -| `WorkSessionService.has_unpushed_commits` | method | work_session.py:662 | PR-existence proxy for unpushed commit detection | -| `WorkSessionService.task_team_for_session` | method | work_session.py:147 | Return the task's team (cell) for a given session; used by route layer's PM cell-ownership check on `merge_pr` | -| `get_work_session_service` | factory | work_session.py:682 | Construct service from AsyncSession | -| `GitService` | class | git.py:234 | All git operations + GitHub API | -| `_GIT_EXECUTOR` | module const | git.py:116 | Dedicated ThreadPoolExecutor for git subprocesses (16 workers) | -| `resolve_git_dir` | func | git.py:129 | Resolve `.git` dir for clone OR linked worktree | -| `_remove_stale_git_locks` | func | git.py:156 | Best-effort clear orphaned `.git/**/*.lock` after timeout SIGKILL | -| `_select_ci_head_run` | func | git.py:220 | Pick CI run matching current HEAD (anti-stale-green) | -| `GitService._run_git` | method | git.py:247 | Run git subprocess in dedicated pool, token header, chown-back, lock cleanup | -| `GitService._token_for_project` | method | git.py:344 | Decrypted project PAT (logs loudly on key-rotation failure) | -| `GitService.get_workspace` | method | git.py:387 | Resolve/clone agent workspace (auto_clone aware) | -| `GitService.get_status` | method | git.py:499 | Porcelain status + ahead/behind | -| `GitService._classify_porcelain` | static | git.py:439 | Split porcelain into staged/unstaged/untracked (column-safe) | -| `GitService._parse_git_url` | static | git.py:557 | Extract (owner,repo) from tokened/https/ssh GitHub URL | -| `GitService.create_commit` | method | git.py:640 | Stage + commit with template message + worktree ensure | -| `GitService._worktree_for_task` | static | git.py:747 | Per-task worktree path `{clone_root}/.worktrees/{task_id[:8]}` (F123) | -| `GitService._ensure_worktree_for_commit` | method | git.py:756 | Re-attach a pruned worktree before cwd-dependent op | -| `GitService._assert_on_task_branch` | method | git.py:771 | Recover drifted clone onto task branch (never discards work) | -| `GitService.commit_for_task` | method | git.py:875 | Agent-facing commit verb backing the `commit` content tool | -| `GitService.create_branch` | method | git.py:989 | Build branch name, fetch base, `worktree add` (F123), push -u | -| `GitService.create_branch_for_task` | method | git.py:1188 | Resolve workspace/team, create branch, commit DB | -| `GitService.checkout_branch_for_agent` | method | git.py:1276 | Allowlist-bounded checkout for agent verb | -| `GitService.push_for_task` | method | git.py:1428 | Push the task's recorded branch by name (clone-checkout-independent) | -| `GitService.push_task_branch` | method | git.py:1467 | Gateway branch-keyed push | -| `GitService.create_pull_request` | method | git.py:2132 | Open PR via GitHub API (legacy project-scoped) | -| `GitService.create_pr_for_task` | method | git.py:2639 | Agent-facing open_pr verb | -| `GitService.update_pr_for_task` | method | git.py:2506 | Patch PR title/body; 404→typed GitError | -| `GitService.get_pr_head_sha` | method | git.py:2451 | PR head SHA for pr_fail re-submit loop guard (fail-open) | -| `GitService.get_latest_ci_conclusion` | method | git.py:1929 | Per-project CI signal (unknown never false-green) | -| `GitService.get_pr_ci_status` | method | git.py:2662 | CI status of a PR's current head commit for the in-path pr_pass gate; returns {state, failing_checks?, head_sha} or None on config gaps (fail-open) | -| `GitService._ci_status_prereqs` | method | git.py:2698 | Resolve (owner, repo, auth headers, head_sha) for CI-status lookup or None on any gap | -| `GitService._fetch_check_runs` | method | git.py:2730 | GET check-runs for head_sha; None on any API failure | -| `GitService._classify_check_runs` | method | git.py:2770 | State classification: success/failure/pending from check-run conclusions list | -| `GitService._classify_zero_check_runs` | method | git.py:2790 | State classification when zero check-runs exist: pending_not_scheduled or no_ci_configured (depends on workflow count) | -| `GitService.list_open_prs` | method | git.py:1844 | Normalized open-PR list | -| `GitService.post_pr_review` | method | git.py:2329 | Post reviewer comments via GitHub API | -| `GitService.merge_pull_request` | method | git.py:2914 | GitHub merge API + method fallback + already-merged disambiguation | -| `GitService.merge_pr_for_task` | method | git.py:3046 | Role-gated merge + recorded-PR verification + auto-complete | -| `GitService._assert_merge_role` | method | git.py:2989 | PM/CEO approval-chain role gate | -| `GitService.pr_merge` | method | git.py:3616 | Gateway merge: project_id-scoped, parent row lock, 409 retry, CEO-only default guard | -| `GitService._merge_with_retry` | method | git.py:3555 | Single 409 retry + already-merged disambiguation → MergeConflictError | -| `GitService._lock_parent_task_for_merge` | method | git.py:3503 | SELECT FOR UPDATE on parent task (sibling merge serialization) | -| `GitService._resolve_merger_id` | static | git.py:3530 | merged_by attribution: actor→assigned→created→UUID(0) | -| `GitService.rebase_onto_base` | method | git.py:3733 | Rebase primitive: rebased/superseded/conflicts classification | -| `GitService.rebase_pr_for_task` | method | git.py:3792 | PR-keyed rebase via PR refs (project_id scoped) | -| `GitService.sync_task_branch` | method | git.py:3847 | Task-keyed rebase through dev `sync_branch` verb (pre-PR) | -| `GitService.is_behind_base` | method | git.py:3889 | `(behind, ahead)` counts for i_am_done submit gate | -| `GitService.close_pull_request` | method | git.py:3940 | Close superseded PR + optional comment + branch cleanup (idempotent) | -| `GitService._delete_remote_branch_best_effort` | method | git.py:3608 | Best-effort remote delete; skips main/master/develop + open-dependent-PR branches; returns `bool` (issued vs skipped/failed) | -| `GitService.delete_task_branch` | method | git.py:3671 | Cancel-path remote branch delete; chokepoint for the environment-ladder skip (`effective_environments`) so a task's `branch_name` can never collide-delete a ladder rung; returns `bool` | -| `GitService.cleanup_stale_branches` | method | git.py:3711 | `POST /git/branches/cleanup` backing sweep: terminal (completed/cancelled) tasks' branches, remote (`delete_task_branch`) + local force-delete in the assignee's clone; capped 200/call, cursor-resumable | -| `GitService._stale_branch_window` | method | git.py:3777 | One deterministic `ORDER BY id` window of sweep candidates; ladder rungs excluded from results but still advance the cursor | -| `GitService._cleanup_one_stale_branch` | method | git.py:3810 | Per-branch remote+local delete for one sweep candidate; raises on unexpected failure so the caller's try/except counts it as an error | -| `GitService.pr_target` | method | git.py:4021 | Return PR base branch (project_id scoped) | -| `GitService.create_pr` | method | git.py:3418 | Branch-keyed open PR (gateway path; ensures base on remote) | -| `GitService._record_pr_atomically` | method | git.py:2601 | Atomic pr_number/url write to task | -| `GitService.run_pre_submit_quality_gate` | method | git.py:3208 | `make quality` gate before submit | -| `GitService.conventions_check_for_task` | method | git.py:4368 | Run conventions validator on changed files (fail-closed) | -| `GitService._run_conventions_validator` | method | git.py:4408 | Subprocess `python -m roboco.conventions` with 120s cap | -| `GitService.open_conventions_pr` | method | git.py:4456 | Scaffold `.roboco/conventions.yml` on a branch + open PR | -| `GitService.diff` / `list_changed_files` / `read_file_at_branch` | methods | git.py:4192/4225/4259 | Read-only git queries (gateway + routes) | -| `GitService.commit` | method | git.py:4286 | Gateway content-verb commit (branch-keyed) | -| `get_git_service` | factory | git.py:4594 | Construct GitService from AsyncSession | -| `build_branch_name` | func | templates/git/branch.py:37 | `{type}/{team}/{root}--{sub}--...` up to MAX_TASK_DEPTH | -| `get_root_task_id` | func | templates/git/branch.py:97 | Walk parent chain to root | -| `BranchNameError` | exc | templates/git/branch.py:33 | Bad type / missing task / over-depth | -| `build_commit_message` | func | templates/git/commit.py:63 | Rich commit msg with task/root/agent/session links | -| `CommitContext` | dataclass | templates/git/commit.py:34 | Validated commit-message input | -| `build_pr_body_internal` / `build_pr_title_internal` | funcs | templates/git/pr_internal.py:75/130 | Subtask→parent PR rendering | -| `build_pr_body_root` / `build_pr_title_root` | funcs | templates/git/pr_root.py:167/235 | Root PR rendering with task tree + AC checklist | -| `MAX_TASK_DEPTH` | const | templates/git/constants.py:45 | 4 (umbrella→root→cell→dev) | -| `BRANCH_TYPES` / `COMMIT_TYPES` | consts | templates/git/constants.py:10/21 | Allowed prefixes | - -## Data Flow - -A developer claims a task → the orchestrator/choreographer calls `create_branch_for_task` → `build_branch_name` walks the task parent chain (`TaskService.get`) up to `MAX_TASK_DEPTH=4`, joins `--`-separated 8-char UUID prefixes, and yields `{type}/{team}/{root}--{sub}--...`. `create_branch` fetches only the needed refs from origin, runs `git worktree add` under `{clone_root}/.worktrees/{task_id[:8]}` (F123 per-task isolation), force-pushes the branch with `-u`, and stores `branch_name` on the task. A `WorkSession` row is created (`WorkSessionService.create`), first superseding any other agent's stale ACTIVE session on that task. - -The agent commits via the `commit` content verb → `GitService.commit` (or `commit_for_task` route), which ensures the worktree is attached, asserts the workspace is on the task branch (recovering a drifted clone), stages, runs `build_commit_message` (`CommitContext` → header + metadata + links), commits, then best-effort links the SHA to the task + work session (`_link_commit_to_task`). Every `_run_git` call re-chowns the tree to the agent uid and clears orphaned lock files on timeout. - -`open_pr` → `create_pr` resolves the task by branch name, ensures the parent branch exists on origin, POSTs the PR via GitHub REST, and atomically records `pr_number`/`pr_url` on the task (`_record_pr_atomically`) and work session (`create_pr`). PR title/body come from `task.title`/`task.description` for gateway PRs; the rich `build_pr_body_root`/`_internal` templates are used by the older `create_pull_request` path. - -Merge: a cell PM `complete`/`submit_up` → `pr_merge` (gateway) scopes the task lookup by `project_id` (cross-repo PR-number collision guard), takes a `SELECT FOR UPDATE` lock on the parent task, calls `_merge_with_retry` (squash; on 409 re-syncs target + retries once; on 405 disambiguates already-merged vs real conflict → `MergeConflictError`), deletes the PR branch, syncs the local target, and records the merge on the work session (`merge_pr`, idempotent). The CEO-only root→master merge goes through `merge_pr_for_task` (role-gated, recorded-PR verification) → `merge_pull_request`. The CEO-merge never targets the default branch via `pr_merge` (the `target == default_branch` guard refuses it) — `default_branch` here is `_project_default_branch`, which now resolves via `roboco.models.env_branches.head_branch(project)` (the env-ladder head rung) rather than reading `project.default_branch` directly; a project with no declared ladder resolves to the same value via the read-time shim. - -Behind-base recovery: `is_behind_base` feeds the `i_am_done` submit gate; on a non-zero behind, the dev calls `sync_branch` → `sync_task_branch` → `rebase_onto_base` (rebased/superseded/conflicts). On a merge conflict, the choreographer calls `rebase_pr_for_task`, then either re-merges or `close_pull_request`s a superseded PR. `get_pr_head_sha` feeds the `submit_root` re-submit loop guard. - -## Mermaid - -```mermaid -stateDiagram-v2 - [*] --> Active: create() (supersede stale) - Active --> Active: add_commit / add_files_modified - Active --> Active: create_pr (pr_number set) - Active --> Completed: merge_pr (idempotent active-guard) - Active --> Completed: close() (task completion) - Active --> Abandoned: abandon() / supersede_active_sessions_for_task - Completed --> [*]: terminal - Abandoned --> [*]: terminal - note right of Active - single-active per task enforced at - create + DB partial-unique index (mig 047) - end note -``` - -```mermaid -sequenceDiagram - participant G as Choreographer/Gateway - participant GS as GitService - participant GH as GitHub REST API - participant WS as WorkSessionService - participant DB as DB (TaskTable) - - G->>GS: pr_merge(pr_number, target, project_id) - GS->>DB: SELECT task WHERE pr_number AND project_id (scoped) - GS->>DB: SELECT FOR UPDATE parent_task (serialize siblings) - GS->>GH: PUT /pulls/{n}/merge (squash) - alt 409 conflict - GS->>GS: _sync_target_branch (re-pull) - GS->>GH: PUT /pulls/{n}/merge (retry once) - end - alt non-success - GS->>GH: GET /pulls/{n} (already-merged?) - opt already merged - GS-->>G: idempotent success - end - opt real conflict - GS-->>G: raise MergeConflictError - end - end - GS->>GH: DELETE PR branch (best-effort) - GS->>GS: _sync_target_branch_best_effort - GS->>WS: merge_pr(session_id, merger_id) - WS->>WS: guard status==ACTIVE else return unchanged - WS-->>GS: COMPLETED + merged_by - GS-->>G: {"merge_commit_sha": ...} -``` - -## Logical Tree - -``` -roboco/ -├── services/ -│ ├── work_session.py -│ │ └── WorkSessionService (BaseService) -│ │ ├── create / get / get_or_raise / update -│ │ ├── get_active_for_task(_and_agent) -│ │ ├── supersede_active_sessions_for_task # single-active invariant -│ │ ├── list_by_agent / list_by_project / list_active_sessions -│ │ ├── add_commit / add_files_modified -│ │ ├── create_pr / update_pr_status / merge_pr -│ │ ├── complete / abandon / close -│ │ └── files_changed / has_unpushed_commits # gateway backfill -│ └── git.py -│ ├── _GIT_EXECUTOR (ThreadPoolExecutor, 16) -│ ├── resolve_git_dir / _remove_stale_git_locks / _select_ci_head_run -│ └── GitService (BaseService) -│ ├── _run_git (token, timeout, chown, lock-cleanup) -│ ├── _token_for_project / _token_for_workspace / get_workspace -│ ├── status: get_status / get_current_branch / _classify_porcelain / _ahead_behind -│ ├── commit: create_commit / commit_for_task / commit (gateway) / _link_commit_to_task -│ ├── worktree (F123): _worktree_for_task / _ensure_worktree_for_commit / _assert_on_task_branch -│ ├── branch: create_branch / create_branch_for_task / create_branch_from_pr_head / checkout* -│ ├── push/pull/fetch/rebase: push_for_task / push_task_branch / pull / fetch / rebase -│ ├── PR context: _build_root_pr_context / _build_internal_pr_context / _generate_pr_title_body -│ ├── PR list/find: _find_existing_pr / list_open_prs / _fetch_open_prs / _normalize_open_pr -│ ├── PR create: create_pull_request / create_pr_for_task / create_pr (branch-keyed) -│ ├── PR update/review: update_pr_for_task / post_pr_review / _patch_pr_title_body -│ ├── PR read: get_pr_diff / get_pr_head_sha / pr_target -│ ├── CI: get_latest_ci_conclusion / _get_ci_runs_response / _fetch_latest_ci_run -│ ├── merge: merge_pull_request / merge_pr_for_task / pr_merge / _merge_with_retry -│ │ _assert_merge_role / _lock_parent_task_for_merge / _resolve_merger_id -│ │ _pr_is_merged / _auto_complete_on_merge / _first_allowed_merge_method -│ ├── branch cleanup: _delete_remote_branch_best_effort / _delete_pr_branch_best_effort -│ │ delete_task_branch / _branch_has_open_dependents -│ │ cleanup_stale_branches / _stale_branch_window / _cleanup_one_stale_branch (sweep) -│ ├── rebase/sync: rebase_onto_base / rebase_pr_for_task / sync_task_branch / is_behind_base -│ ├── close: close_pull_request -│ ├── quality: run_pre_submit_quality_gate / toolchain_status_for_task / _fast_gate_commands -│ ├── conventions: conventions_check_for_task / _run_conventions_validator / open_conventions_pr -│ ├── read-only: diff / list_changed_files / read_file_at_branch / _ref_exists / _resolve_diff_base -│ └── helpers: _task_for_branch / _project_for_task / _workspace_for_branch / _token_for_branch ... -└── templates/git/ - ├── __init__.py # re-exports - ├── constants.py # BRANCH_TYPES / COMMIT_TYPES / MAX_TASK_DEPTH=4 - ├── branch.py # build_branch_name / get_root_task_id / BranchNameError - ├── commit.py # CommitContext / build_commit_message / CommitMessageError - ├── pr_internal.py # InternalPRContext / build_pr_body_internal / build_pr_title_internal - └── pr_root.py # RootPRContext / SubtaskInfo / build_pr_body_root / build_pr_title_root -``` - -## Dependencies - -Internal: -- `roboco.config.settings` (timeouts, URLs, workspace root, auto_clone) -- `roboco.exceptions` (`GitCommandError`, `GitError`, `GitTimeoutError`, `MergeConflictError`) -- `roboco.foundation.policy.lifecycle` (role/intent parity) -- `roboco.models.base` (`AgentRole`, `TaskStatus`) -- `roboco.models.work_session` (`WorkSessionCreate/Update/Status`) -- `roboco.db.tables` (`ProjectTable`, `TaskTable`, `WorkSessionTable`) -- `roboco.services.base` (`BaseService`, `NotFoundError`, `ConflictError`, `ValidationError`, `UnauthorizedError`, `ServiceError`) -- `roboco.services.project` / `roboco.services.task` / `roboco.services.workspace` (composed for clone/workspace/branch resolution) -- `roboco.services.gateway.quality_gate` (`GateResult`, `run_quality_commands`) -- `roboco.templates.git` (all template builders) -- `roboco.utils.converters.require_uuid`, `roboco.utils.crypto.EncryptionError` -- `roboco.api.schemas.git` (TYPE_CHECKING only — runtime duck-typed) - -External: -- `sqlalchemy` / `sqlalchemy.ext.asyncio` -- `httpx` (GitHub REST API) -- `asyncio`, `subprocess`, `concurrent.futures.ThreadPoolExecutor` -- `dataclasses`, `pathlib`, `uuid`, `base64`, `re`, `json`, `time`, `os`, `sys` - -## Entry Points - -- **HTTP routes** (`roboco/api/routes/git.py`): `get_status`, `log`, `diff`, `commit_for_task`, `push_for_task`, `create_branch_for_task`, `checkout_branch_for_agent`, `create_pr_for_task`, `merge_pr_for_task`, `pull`, `fetch`, `rebase`, `cleanup_stale_branches` (`POST /git/branches/cleanup`, PM/CEO role-gated like `/rebase`, rate-limit 5/60) — all construct via `get_git_service(db)`. -- **HTTP routes** (`roboco/api/routes/tasks.py:253`): `get_git_service` for task-scoped git. -- **Gateway Choreographer** (`roboco/services/gateway/choreographer/`): - - `_verb_runner._do_pr_merge` → `pr_merge` - - `_impl` → `conventions_check_for_task` (i_am_done + pr_pass gates), `is_behind_base` (submit gate), `sync_task_branch` (sync_branch verb), `pr_merge` / `rebase_pr_for_task` / `close_pull_request` (merge-conflict resolution), `get_pr_head_sha` (submit_root re-submit guard) - - `pr_gate.py` → `get_pr_head_sha` - - `qa.py` → `conventions_check_for_task` -- **WorkspaceService** calls `ensure_worktree` / `ensure_worktree_for_resume` (F123). -- **Lifespan/CLI**: none direct; `git_service` is constructed per-request via `deps.py` (`git=GitService(db_session)`). - -## Config Flags - -| Flag / setting | Source | Used for | -|----------------|--------|----------| -| `ROBOCO_GIT_EXECUTOR_WORKERS` (env, default 16) | `os.environ` at git.py:117 | Dedicated git subprocess pool size | -| `settings.git_command_timeout_seconds` | config.py:728 | Default `_run_git` timeout | -| `settings.git_commit_timeout_seconds` | config.py:737 | Staging/commit large changeset timeout | -| `settings.git_network_timeout_seconds` | config.py:748 | fetch/pull/push/ls-remote timeout | -| `settings.workspaces_root` | config.py:603 | Workspace path root (token derivation) | -| `settings.workspace_auto_clone` | config.py:607 | `get_workspace` auto-clone branch | -| `settings.public_base_url` | config.py:827 | Commit/PR template link base (`+ /api`) | -| `settings.internal_api_url` | config.py:57 | Internal PR body link base | -| `settings.github_api_base_url` | config.py:327 | GitHub REST API base (PR/CI/review) | -| `settings.release_ci_workflow` | config.py:597 | Named workflow file (e.g. `ci.yml`) used by the release CI gate in `get_latest_ci_conclusion`; always resolves a named workflow so the gate never degrades to the imprecise all-workflows mode | - -Module-level tunables (not env): `_SLOW_GIT_OP_MS=5000`, `_CI_RUN_WINDOW=20`, `_CI_FETCH_ATTEMPTS=3`, `_CI_FETCH_BACKOFF_SECONDS=0.5`, `_CI_RETRYABLE_STATUS`, `_CONVENTIONS_VALIDATOR_TIMEOUT_SECONDS=120`, `_GH_UNPROCESSABLE=422`, `_HTTP_NOT_FOUND=404`, `_HTTP_CONFLICT=409`. - -## Gotchas - -- **Token transport split**: git-over-HTTPS uses HTTP **Basic** (`x-access-token:...`) via `http.extraheader` (git.py:289); the GitHub REST API uses **Bearer**. Swapping them causes silent credential-prompt failures. -- **`pr_number` is NOT repo-scoped in `tasks.pr_number`** — every gateway merge/close path (`pr_merge`, `close_pull_request`, `pr_target`, `rebase_pr_for_task`) requires `project_id` to scope the task lookup. The route `merge_pr_for_task` instead verifies `data.pr_number == task.pr_number` (recorded PR is source of truth). -- **`get_active_for_task` returns the most-recent ACTIVE row**, not `scalar_one_or_none` — the historical duplicate-ACTIVE defect would otherwise raise `MultipleResultsFound`. The invariant is also enforced at `create` and by a DB partial-unique index (migration 047). -- **`merge_pr` idempotency guard (F062)**: a terminal session is returned unchanged; `complete`/`abandon` on a non-active session return `None` (warning). `close` returns the session unchanged. Behavior differs between these three on terminal state — `merge_pr`/`close` are silent, `complete`/`abandon` warn-and-None. -- **F123 per-task worktrees**: commit/checkout/rebase/conventions MUST run inside `{clone_root}/.worktrees/{task_id[:8]}`, not the clone root (which sits on the default branch). `_worktree_for_task` + `_ensure_worktree_for_commit` are the seam; forgetting them makes checkout fail with "already checked out at ''" or false-passes the conventions validator. -- **`create_branch` runs `reset --hard` on a no-commit branch** in the worktree (git.py:1104) — safe because `unique == 0`, but a branch carrying real work is left as-is. The fresh-claim path only; resume short-circuits before it. -- **`_run_git` re-chowns the tree** after every op (root → agent uid 1000); without it the agent's next commit fails with "unable to append to .git/logs/refs/...". -- **Porcelain parsing** uses `splitlines()` not `strip().split("\n")` — strip eats the leading space on ` D file` and false-stages deletions. -- **`get_current_branch` raises on detached HEAD** instead of returning `""` — the empty string used to leak "(HEAD detached at ...)" into `checkout -b`. -- **`MAX_TASK_DEPTH=4`** (was 3) — MegaTask's umbrella→root→cell→dev needs 4; validator rejects a child whose depth would *reach* MAX_TASK_DEPTH, so 4 permits dev at depth 3. -- **Branch name uses 8-char UUID prefix** (`_SHORT_ID_LEN=8`), not full UUID — full UUIDs produced 140-char branch names. -- **`rebase_onto_base` force-pushes with `--force-with-lease`** only the head branch; never touches base. `superseded` (unique==0) means close-without-merge. -- **`is_behind_base` raises on git failure**; the i_am_done gate fail-opens on the raised error so a flaky fetch can't strand the task. -- **Conventions validator fails closed** (`could_not_run=True` blocks submit) on resolution error / timeout / non-zero exit; branchless + no-changed-files fail open. -- **`_assert_on_task_branch` never discards work** — it does `checkout`, not `reset --hard`, to preserve a resumed agent's unpushed commits. -- **CEO-only master merge**: `pr_merge` refuses `target == default_branch` for agents; only `merge_pr_for_task` (CEO role-gated from `awaiting_ceo_approval`) may merge to master. `default_branch` resolves through the env-ladder head rung (`_project_default_branch` → `head_branch(project)`), not the raw `projects.default_branch` column. -- **`cleanup_stale_branches` cursor is required, not optional**: task rows never change as a side effect of the sweep (unlike, say, a queue that drains), so a repeat call with no `after_cursor` re-scans the identical first 200-row window forever instead of progressing. `_stale_branch_window` still advances the cursor past ladder-rung rows even though they're excluded from `candidates`, so `truncated` can't false-negative when a rung lands inside the window. -- **Local branch delete in the sweep is always `force=True`** (`-D`) regardless of completed vs cancelled — a completed task's PR was squash-merged, so its local ref is never an ancestor of base and a "safe" `-d` would refuse every single candidate. - -## Drift from CLAUDE.md - -- **CLAUDE.md "WorkSessionService" table** claims the service handles "Git session management, PR lifecycle" — accurate. No drift. -- **CLAUDE.md says** `ROBOCO_WORKSPACE_CLONE_TIMEOUT=300` is a WorkspaceService config; not referenced in this slice (lives in `workspace.py`). No drift in scope. -- **CLAUDE.md verb table** lists `sync_branch` for developers and `/rebase` for PM/CEO. The code matches: `sync_task_branch` is the dev path, `rebase_pr_for_task` the PR-keyed path. No drift. -- **CLAUDE.md** states "A task has at most one active WorkSession ... enforced both at the service layer and by a DB partial-unique index (migration 047)." Code matches (`supersede_active_sessions_for_task` + `get_active_for_task` resilient return). No drift. -- **CLAUDE.md** "PR is created BEFORE QA review" — `create_pr`/`create_pr_for_task` only sets `pr_number`; QA pass requires it. Matches. -- **Minor doc vs code**: CLAUDE.md commit-format example is `[{task-id[:8]}] {message}` (single ID), but `build_commit_message` (templates/git/commit.py:80) emits `[{root_short}:{task_short}] {type}({scope}): {desc}` — a richer two-ID header. The doc undersells the actual format; not a bug, but the template header is not the literal `[{task-id[:8]}]` the doc shows. -- **CLAUDE.md** lists `merge_pull_request`-style PM merges; the agent-facing path is now `pr_merge` (gateway) with parent-row locking + 409 retry, which the doc does not describe. Additive (the route `merge_pr_for_task` still exists) — doc is incomplete rather than wrong. - -## Changes Since Baseline - -Baseline: `fd10cc862c2020b3f639cdb686d427b0198a2441` (master tip before the metrics-granularity branch). Diff stat: `git.py +614/-107`-ish, `work_session.py +11`, `branch.py +9/-`, `constants.py +12/-`. Commits touching these files: `15effce0` (#283 "141 Gaps fill-in"), `3aff6e04` (#285 "Close gaps"). - -| Commit | IMPACT (one line) | -|--------|-------------------| -| `15effce0` (141 Gaps fill-in) | Added `resolve_git_dir` + worktree-aware `_remove_stale_git_locks`; added F123 `_worktree_for_task`/`_ensure_worktree_for_commit` and routed commit/checkout/rebase/conventions into the per-task worktree; added `get_pr_head_sha` (pr_fail re-submit guard); added `sync_task_branch` + `is_behind_base` (dev sync_branch verb + i_am_done behind gate); added `rebase_onto_base`/`rebase_pr_for_task` (merge-conflict resolver); added `close_pull_request` (superseded-PR close); added `pr_merge` with `project_id` scoping + parent-row lock + 409 retry + CEO-only default-branch guard + already-merged disambiguation; added `_merge_with_retry`/`_pr_is_merged`/`_resolve_merger_id`/`_lock_parent_task_for_merge`; added conventions validator runner (`conventions_check_for_task`/`_run_conventions_validator`/`open_conventions_pr`); raised `MAX_TASK_DEPTH` 3→4 (MegaTask depth-cap fix); `WorkSessionService.merge_pr` idempotent active-guard (F062). | -| `3aff6e04` (Close gaps) | Same mega-commit (the PR body is identical — #285 is the merge closure of the #283 batch); the in-scope file deltas are the same set of additions. No additional logic change to these files beyond what #283 listed. | - -> Post-snapshot updates (since 2026-06-29): `536bbb64` (Chore/all/logical gaps sweep #286 — closed regression risks #108 and #109 in this slice: `_merge_with_retry` now falls back to a permitted merge method on 405 via `_first_allowed_merge_method` before raising `MergeConflictError`; `close_pull_request` default flipped to `delete_branch=False`, choreographer caller now passes `delete_branch=True` explicitly). `00513399` ([bug] push_branch — `push_branch(branch_name)` now passes `branch=branch_name` to `self.push()` so the gateway `open_pr` path pushes the actual named task branch rather than the clone root's current checkout; fixes the "No commits between" 422 → `i_am_blocked` wedge observed in the F123 per-worktree model). `2759edf7` ([B-REL] release executor — added `_CiRunQuery` dataclass at git.py:241 to bundle per-project CI-fetch inputs; `get_latest_ci_conclusion` and `_fetch_latest_ci_run` now accept an optional `head_sha` so the release CI gate polls a specific release commit's own run rather than branch-latest; `settings.release_ci_workflow` config flag added). `69071030` ([chore] work-session-routes — added `WorkSessionService.task_team_for_session` helper (route layer PM cell-ownership check for `merge_pr`); route layer now stamps `merged_by` from the authenticated caller rather than the request body — `WorkSessionService.merge_pr` signature is unchanged, but `MergePRRequest` schema dropped `merged_by` field). -> -> (open PR #548, branch `feature/wave-2-hygiene-charts`, 2026-07-17) Local branch refs stop leaking alongside remote ones: `delete_task_branch` now also skips environment-ladder rungs (previously only the remote-delete's own main/master/develop guard existed) and returns `bool`; new `cleanup_stale_branches` + `_stale_branch_window` + `_cleanup_one_stale_branch` back a PM/CEO-only `POST /git/branches/cleanup` sweep of terminal tasks' remote+local branches, exposed as a confirm-dialog button on the panel Git page. See `docs/map/task-service.md` for the paired per-task reap at cancel/completion and `docs/map/workspace.md` for the new `WorkspaceService.delete_local_branch` primitive both routes share. - -## Regression Risks - -| Title | File:Line | Claim | Severity | -|-------|-----------|-------|----------| -| `merge_pr` idempotent guard silently drops retried merge attribution | work_session.py:517 | A retried merge after a successful-but-unconfirmed GitHub merge returns the *existing* COMPLETED row unchanged, so the retry's `merged_by`/`pr_merged_at` are NOT updated. Correct for audit-trail integrity, but a caller that relied on `merge_pr` returning a freshly-updated row (e.g. reading `pr_merged_at` as "now") now gets the original timestamp. Low risk but a behavior change worth a regression test. | low | -| `pr_merge` CEO-only guard refuses default-branch target | git.py:3671 | If a non-root PR's target legitimately resolves to the repo default branch (e.g. a mis-configured `default_branch` or a single-branch repo), `pr_merge` now hard-fails with `UnauthorizedError`. Any cell that previously merged a leaf PR into master via this path (shouldn't happen per policy, but the route existed) is now blocked. Since the env-branches ladder (#534) the comparison target is the env-ladder **head** rung (`_project_default_branch` → `head_branch(project)`), not necessarily prod — a project that declares head != prod has this guard protect the head rung, not the CEO's actual prod branch. | medium | -| `pr_merge` parent-row `SELECT FOR UPDATE` can deadlock | git.py:3503/3683 | Two PMs merging sibling subtasks of *different* parents that share an ancestor, under并发 lock ordering, could deadlock if lock acquisition order diverges. Lock is only on the immediate parent, so risk is bounded, but a deadlock surfaces as a rollback (not a clean 409 retry). | medium | -| `create_branch` `reset --hard` on zero-commit branch inside worktree | git.py:1104 | If `rev-list --count {base_ref}..{branch_name}` returns `0` for a branch that actually carries work (e.g. base_ref mis-resolved to a ref that already contains the work), the worktree is hard-reset and uncommitted work in the worktree is lost. The `unique==0` check is the only guard; a stale `base_ref` could trip it. | medium | -| F123 worktree routing — commit/conventions/rebase run in worktree, merge sync runs in clone root | git.py:3696/3785 | `pr_merge` calls `_sync_target_branch_best_effort(workspace,...)` with the clone-root workspace (from `get_workspace`), not the per-task worktree. If the target branch is checked out in a worktree, the sync's `checkout` of target in the clone root fails ("already checked out"). Best-effort swallows it, but the local target ref may stay stale for the next sibling merge. | medium | -| ~~`_merge_with_retry` retries on 409 only; 405 falls through to already-merged check then `MergeConflictError`~~ | git.py:3597 | **FIXED** (`536bbb64` #108) — `_merge_with_retry` now falls back to a permitted merge method (via `_first_allowed_merge_method`, exclude='squash') on 405 before raising `MergeConflictError`, mirroring the CEO `merge_pull_request` path. A 405 with no permitted fallback or a second 405 still falls through to disambiguation/`MergeConflictError`. | ~~medium~~ resolved | -| `is_behind_base` raises on fetch failure; gate fail-opens | git.py:3922/3938 | A flaky origin fetch makes `is_behind_base` raise; the i_am_done gate catches it and fail-opens, letting a behind branch submit. The merge layer's own behind check is the backstop, but a genuinely-behind branch can reach QA. Documented, but a regression in the "gate is authoritative" expectation. | low | -| `MAX_TASK_DEPTH` 3→4 changes branch-name length + validation | constants.py:45 / branch.py:71 | Any pre-existing task hierarchy at depth 4 that was previously rejected now builds a 4-segment branch name; tasks created under the old cap that stored a shorter branch are unaffected, but new subtasks of a previously-maxed tree now cut branches where they couldn't before — could surface latent assumptions in downstream consumers parsing branch names. | low | -| ~~`close_pull_request` deletes branch on close by default~~ | git.py:4005 | **FIXED** (`536bbb64` #109) — default flipped to `delete_branch=False` (opt-in deletion); the choreographer supersede caller now passes `delete_branch=True` explicitly when it wants deletion, matching the orchestrator supersede path. A superseded PR's branch is preserved by default. | ~~medium~~ resolved | -| Conventions validator fail-closed on resolution error | git.py:4392 | A workspace resolution failure (missing clone, diff error) returns `could_not_run=True`, which the block-gate treats as a hard refuse. A transient workspace/clone issue can now block `i_am_done`/`pr_pass` where previously the gate would have passed. Intentional but a new stranding vector. | low | -| `get_pr_head_sha` fail-open returns None on any error | git.py:2496 | The `submit_root` re-submit loop guard only hard-blocks on an *exact* unchanged head SHA; any GitHub error / closed PR returns None and the guard passes, so a flaky API call lets a weak coordinator re-submit the same failed PR. Documented fail-open, but a regression vs. a strict gate. | low | - -## Health - -Integrity is **good and actively hardened**. The slice carries the scars of multiple live meltdowns (single-active work-session defect, pr_fail re-submit loop, cell_pm merge block<->unblock, MegaTask depth cap, cross-repo PR-number collision) and each is closed with a deterministic guard plus a comment explaining the failure mode. The F123 per-task-worktree routing is consistently threaded through commit/checkout/rebase/conventions, and the merge path has layered defenses (parent-row lock, 409 retry, already-merged disambiguation, CEO-only master guard). Two formerly-medium risks in the merge path have since been closed: `_merge_with_retry` now has the 405 method-fallback that `merge_pull_request` has (`536bbb64`), and `close_pull_request` now defaults to `delete_branch=False` (`536bbb64`). The remaining residual risk is **`pr_merge`'s post-merge target sync** running in the clone root (not the worktree) and best-effort-swallowing a checkout conflict — the local target ref may stay stale for the next sibling merge. Test coverage of the work-session lifecycle is solid; the newer `pr_merge`/`sync_task_branch`/`rebase_onto_base`/`close_pull_request` quartet deserves the most scrutiny on any future change. No outright bugs found; the drift vs CLAUDE.md is documentation undersell (commit header format, gateway merge-path description), not behavioral mismatch. - -# workspace slice - -## Purpose -WorkspaceService manages the per-agent git clone layout under {workspaces_root}/{project}/{team}/{agent}/, plus the F123 per-task linked worktrees under {clone_root}/.worktrees/{task}/. It clones, refresh-fetches, repairs ownership, installs dev deps, scaffolds the conventions standard on first clone, maintains a project-level read clone for the conventions engine, and runs the read-only dep-upgrade probe. It is the filesystem/git-clone substrate every agent spawn and every git verb eventually lands on. +The CEO-facing intake and chief-of-staff slice. PrompterService turns a confirmed live-intake structured draft (or a MegaTask batch of drafts) into real Task rows, routing ownership/team and sequencing collision-free waves. PrompterLiveRegistry is the in-process bridge that relays a live chat between a spawned prompter/secretary container and the panel (SSE stream + turn delivery + park/idle lifecycle). SecretaryService reads company state and executes or gates the CEO's directives (relay/announce/charter/pitch/task-control), recording every directive auditably. ## Files | Path | Role | LOC | |---|---|---| -| /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/workspace.py | WorkspaceService + helpers: clone/own/refresh/install deps, per-task worktrees, read clone, dep-upgrade probe | 1757 | +| roboco/services/prompter.py | PrompterService: create tasks from confirmed intake drafts (single + MegaTask batch), route owning team, sequence drafts into waves; plus pure description/readiness helpers, the wave-1/2 prompter-memory history-digest builders, and compact task-search row rendering | 1313 | +| roboco/services/prompter_live.py | PrompterLiveRegistry: process-wide singleton bridging live intake/secretary chat between panel (SSE) and spawned container (HTTP turn), with open/close/park/idle-reap lifecycle | 234 | +| roboco/services/secretary.py | SecretaryService: read company state + submit/confirm/reject gated CEO directives (relay/announce/charter/pitch/task-control incl. wave-1 full-content `edit` + claim-aware reassignment), persisted in secretary_directives | 418 | ## Key Symbols | Name | Kind | File:Line | Responsibility | |---|---|---|---| -| _chown_entry | function | roboco/services/workspace.py:84 | Chown one entry to (AGENT_UID, AGID); return True on success or already-correct | -| _make_owner_and_group_rw | function | roboco/services/workspace.py:95 | Best-effort chmod ensuring owner+group rw (+x for dirs) for ACL-inheriting NAS volumes | -| _own_and_grant_rw | function | roboco/services/workspace.py:122 | Chown + grant rw on one entry; return 1 if chown failed | -| _iter_ownable_entries | function | roboco/services/workspace.py:129 | Yield workspace root + every entry, pruning heavy gitignored trees (_PRUNE_DIRS) so os.walk stays fast | -| _ensure_agent_owned | function | roboco/services/workspace.py:145 | Chown + group-write the whole working tree (pruned) so uid-1000 agent can read+write .git and sources | -| _resolve_clone_root | function | roboco/services/workspace.py:184 | Given a worktree path, return its clone root (parent.parent when under .worktrees/); pure path logic | -| _uv_subprocess_env | function | roboco/services/workspace.py:198 | Env for orchestrator-side uv subprocess: pin UV_PYTHON_INSTALL_DIR to /.uv-python so fetched CPython lands on the mount; also pops VIRTUAL_ENV and UV_PROJECT_ENVIRONMENT to drop the image-baked /app/.venv pin (9faf2763) | -| _monotonic | function | roboco/services/workspace.py:223 | Thin wrapper over time.monotonic so tests can patch it without breaking asyncio's own clock | -| _ensure_lock_for | function | roboco/services/workspace.py:244 | Return (lazy-create) the per (project_slug, agent_slug) asyncio.Lock serializing ensure_workspace/concurrent clones | -| _inject_token_into_url | function | roboco/services/workspace.py:254 | Embed a GitHub PAT into an HTTPS git URL for clone/fetch auth; pass-through for SSH and already-tokenized URLs | -| WorkspaceError | class | roboco/services/workspace.py:283 | Exception raised on workspace/clone/worktree failures | -| _lockfile_digest | function | roboco/services/workspace.py:306 | SHA-256 over present lockfiles (uv.lock/pnpm-lock.yaml/package-lock.json/package.json) for idempotent dev-deps install | -| _detect_dep_commands | function | roboco/services/workspace.py:332 | Detect ecosystem + return (label, argv) install commands: uv sync --extra dev (optionally --python X), pnpm/npm | -| WorkspaceService | class | roboco/services/workspace.py:374 | Service: per-agent workspace path math, clone/own/refresh, worktrees, read clone, dep probe, dev-deps install | -| WorkspaceService.get_workspace_path | method | roboco/services/workspace.py:399 | Compute {root}/{project}/{team}/{agent}/ path; raise if team is None | -| WorkspaceService.get_clone_root_path | method | roboco/services/workspace.py:433 | Same as get_workspace_path; named separately to express clone-root vs worktree intent | -| WorkspaceService.get_worktree_path | method | roboco/services/workspace.py:447 | Per-task worktree path {clone_root}/.worktrees/{task_short_id}; raise on empty id | -| WorkspaceService._clone_root_default_branch | staticmethod | roboco/services/workspace.py:486 | Read origin/HEAD to get the default branch name ("main"); returns "" when unresolvable (cfe725da) | -| WorkspaceService._park_clone_root_off_branch | staticmethod | roboco/services/workspace.py:501 | Restore F123 invariant before worktree add: if clone root HEAD is the task branch, move back to default branch or detach so the branch ref is free for the worktree (cfe725da) | -| WorkspaceService._worktree_git | staticmethod | roboco/services/workspace.py:469 | Run git -C capturing output, check optional | -| WorkspaceService._link_shared_venv | staticmethod | roboco/services/workspace.py:534 | Symlink worktree/.venv -> ../../.venv only if clone-root .venv exists; idempotent via lexists guard | -| WorkspaceService.ensure_worktree | method | roboco/services/workspace.py:555 | git worktree add -b (or reuse existing branch); calls _park_clone_root_off_branch first; link venv; chown worktree + clone root | -| WorkspaceService.ensure_worktree_for_resume | method | roboco/services/workspace.py:591 | Re-add a pruned worktree on resume (no -b; branch ref survives); calls _park_clone_root_off_branch first; idempotent; link venv + chown | -| WorkspaceService._fetch_branch_ref | method | roboco/services/workspace.py:613 | Token-aware git fetch origin into clone_root; best-effort (never raises); used by ensure_worktree_self_heal (536bbb64) | -| WorkspaceService.ensure_worktree_self_heal | method | roboco/services/workspace.py:671 | Orchestrator spawn-time chokepoint: re-attaches a per-task worktree after clone vanished (redeploy/disk loss); fetches branch ref from origin when the local ref is absent after a re-clone, then delegates to ensure_worktree (536bbb64) | -| WorkspaceService.remove_worktree | method | roboco/services/workspace.py:733 | Best-effort git worktree remove --force + prune; no-op if gone (cancel/terminal/reaper evict) | -| WorkspaceService.delete_local_branch | method | roboco/services/workspace.py:787 | Best-effort `git branch -d/-D ` in a clone; never raises; skips main/master/develop/empty (mirrors GitService._delete_remote_branch_best_effort); callers run it AFTER remove_worktree (a still-checked-out branch refuses) | -| WorkspaceService.resolve_workspace | method | roboco/services/workspace.py:745 | Look up agent (UUID or slug) -> team+slug -> workspace path; default team BACKEND | -| WorkspaceService._lookup_agent_or_raise | method | roboco/services/workspace.py:787 | Find agent by UUID or slug; raise WorkspaceError if missing | -| WorkspaceService._is_workspace_healthy | staticmethod | roboco/services/workspace.py:806 | True only if .git exists AND has HEAD + objects/ (rejects stub clones) | -| WorkspaceService._prune_broken_refs | staticmethod | roboco/services/workspace.py:816 | Delete .bak ref debris + loose refs whose content is neither sha nor symref before a fetch; best-effort | -| WorkspaceService._fetch_origin_best_effort | staticmethod | roboco/services/workspace.py:854 | Scoped credential-less git fetch of current+default branch with 30s TTL; downgrades expected auth-fail to DEBUG | -| WorkspaceService._resolve_git_token | staticmethod | roboco/services/workspace.py:947 | Decrypt project git token; raise WorkspaceError on decrypt failure or HTTPS-with-no-token | -| WorkspaceService.ensure_workspace | method | roboco/services/workspace.py:969 | Idempotent ensure: healthy short-circuit (own+fetch+install_deps) or rmtree partial then clone+scaffold; per (project,agent) lock | -| WorkspaceService._maybe_scaffold_conventions | method | roboco/services/workspace.py:1101 | Flag-gated once-per-process scaffold of .roboco/conventions.yml on a project's first clone; swallows all failures | -| WorkspaceService.ensure_read_clone | method | roboco/services/workspace.py:1133 | Ensure project-level read clone at {root}/{project}/_meta/conventions, hard-reset to `origin/` (the env-ladder head rung via `roboco.models.env_branches.head_branch`, shimmed from `default_branch` when no ladder is declared); 30s TTL fetch | -| WorkspaceService._read_clone_token | staticmethod | roboco/services/workspace.py:1184 | Decrypt project token for read-clone refresh; return None on failure (public repos ok) | -| WorkspaceService._sync_read_clone | staticmethod | roboco/services/workspace.py:1200 | Token-authed fetch + checkout + reset --hard FETCH_HEAD on the read clone; best-effort | -| WorkspaceService._clone_repo | method | roboco/services/workspace.py:1239 | git clone --branch --no-tags (no --single-branch) + configure identity/fileMode + scrub PAT + leak-check + chown + install_dev_deps; rmtree on any failure | -| WorkspaceService.install_dev_deps | method | roboco/services/workspace.py:1424 | Idempotent dev-deps install via lockfile digest marker; runs detected cmds, chowns results, records toolchain marker | -| WorkspaceService._resolve_toolchain_target | staticmethod | roboco/services/workspace.py:1478 | Return target Python version when toolchain_match_enabled, else None | -| WorkspaceService._record_toolchain | method | roboco/services/workspace.py:1486 | Run pytest --collect-only smoke under target python and write .git/.roboco-toolchain marker JSON | -| WorkspaceService._run_toolchain_smoke | staticmethod | roboco/services/workspace.py:1504 | Return ok/broken/unknown from pytest collect-only under the target interpreter (precision over recall) | -| WorkspaceService.read_toolchain_status | staticmethod | roboco/services/workspace.py:1546 | Read (python, status) from the toolchain marker; (None,None) when absent/unreadable | -| WorkspaceService._dep_install_cache_hit | staticmethod | roboco/services/workspace.py:1562 | True when stored digest equals current lockfile digest (skip install) | -| WorkspaceService._run_dep_install | staticmethod | roboco/services/workspace.py:1575 | Run one install command in a thread; swallow FileNotFoundError/timeout/OSError; return True only on exit 0 | -| WorkspaceService.dry_upgrade_changes_lockfile | method | roboco/services/workspace.py:1634 | Read-only dep-upgrade probe: local --no-hardlinks clone of read clone under lock, run dep_update_command, report dirty lockfile paths; fail-safe False | -| WorkspaceService._clone_local_into | staticmethod | roboco/services/workspace.py:1692 | git clone --local --no-hardlinks of read clone into throwaway dir (independent copy) | -| WorkspaceService._probe_lockfile_on_clone | staticmethod | roboco/services/workspace.py:1716 | Run upgrade via shlex.split (no shell) + git status --porcelain on lock paths; False on non-zero | -| WorkspaceService.workspace_exists | method | roboco/services/workspace.py:1752 | Bool: workspace resolved and .git exists | -| WorkspaceService.list_workspaces | method | roboco/services/workspace.py:1764 | Scan {root}/{project}/*/* for dirs containing .git; return info dicts | -| WorkspaceService._resolve_branch_to_project_slug | method | roboco/services/workspace.py:1797 | Look up task by branch_name -> project slug; raise if no task or project missing | -| WorkspaceService.fetch_branch_for_inspection | method | roboco/services/workspace.py:1823 | Ensure workspace for QA/Doc/PM, git fetch origin with token http.extraheader; re-chown; return workspace path | -| WorkspaceService.delete_workspace | method | roboco/services/workspace.py:1897 | rmtree the resolved workspace; True if deleted, False if absent | -| get_workspace_service | function | roboco/services/workspace.py:1930 | Factory: WorkspaceService(session) | +| ReadinessTag | dataclass | roboco/services/prompter.py:71 | Parsed contents of an assistant turn's trailing roboco-meta JSON block (covered, ready, scale) | +| BatchPlacement | dataclass | roboco/services/prompter.py:80 | Where a draft sits in a MegaTask batch (parent_task_id, batch_id, sequence, team_override) | +| PrompterService | class | roboco/services/prompter.py:97 | Create tasks from confirmed intake drafts; pure draft/description helpers + DB-backed create | +| PrompterService._session | property | roboco/services/prompter.py:108 | Return the AsyncSession or raise ServiceError if constructed without one | +| PrompterService._assignee_is_board | method | roboco/services/prompter.py:118 | True if agent_id is a board/advisory role (PO / HoM / Auditor) | +| PrompterService._validate_draft_target | staticmethod | roboco/services/prompter.py:125 | A draft targets exactly one of project/product/per-cell-map, or none for an umbrella | +| PrompterService._resolve_owning_team | method | roboco/services/prompter.py:163 | Route owning team: team_override wins; if no product: multi-cell map (≥2 cells) -> MAIN_PM else lead cell; if product: board assignee -> BOARD else MAIN_PM (product/board routing checked BEFORE multi-cell force) | +| PrompterService._validate_and_coerce_draft | method | roboco/services/prompter.py:196 | Validate title+AC, coerce list fields (acceptance_criteria/what_this_builds/notes/the_work[].items) to list[str] in place | +| PrompterService._resolve_draft_assignee | method | roboco/services/prompter.py:243 | Explicit confirm-button assignment wins; else fall back to draft.assigned_to UUID | +| PrompterService._coerce_pm_code_to_planning | method | roboco/services/prompter.py:256 | Coerce code->planning when owner is a coordination PM role; two layers: team-based (main_pm_cannot_own_code) then assignee-based (pm_cannot_own_code); issue-resolution carve-out never applies for new intake tasks | +| PrompterService.create_task_from_draft | method | roboco/services/prompter.py:293 | Operate on a _copy_draft copy (caller never mutated), compose description, validate target, coerce enums, route team, coerce PM+code->planning via _coerce_pm_code_to_planning, persist via TaskService.create | +| PrompterService.confirm_live_draft | method | roboco/services/prompter.py:368 | Confirm a live-intake single draft -> create at PENDING assigned to product-owner (board) or main-pm route; return task id | +| PrompterService._sequence_drafts | method | roboco/services/prompter.py:419 | Build DraftSurface list and run SequencingService.analyze into waves; SequencingError -> ValidationError 400 | +| PrompterService.preview_batch | method | roboco/services/prompter.py:459 | Compute MegaTask waves+warnings WITHOUT creating (panel pre-confirm preview) | +| PrompterService._validate_batch_scope | staticmethod | roboco/services/prompter.py:473 | Each draft targets scoped repos via cell map or top-level project_id; union across drafts spans >=2 distinct projects | +| PrompterService.confirm_live_batch | method | roboco/services/prompter.py:524 | Create MegaTask umbrella + N sequenced root-subtasks, wire dependency edges; return umbrella_id/root_ids/waves/warnings | +| PrompterService.update_live_draft | method | roboco/services/prompter.py:628 | Apply a board-informed re-draft to an existing task in place; route via approve_and_start or re-board (clear board_review_complete) | +| PrompterService._resolve_uuid_field | staticmethod | roboco/services/prompter.py:676 | Parse draft_data[key] as UUID; None if absent, ValidationError if malformed | +| PrompterService._lead_cell_team | staticmethod | roboco/services/prompter.py:690 | Owner of a single-cell task: first valid Team in the_work, else default | +| PrompterService._coerce_draft_enums | staticmethod | roboco/services/prompter.py:704 | Coerce team/task_type/nature/complexity to valid enums; default on invalid/missing so confirm never hard-fails | +| PrompterService._coerce_priority | staticmethod | roboco/services/prompter.py:734 | Coerce priority (word or number) to int 0-3, default 2 | +| parse_readiness | function | roboco/services/prompter.py:786 | Split assistant reply into (clean_text, ReadinessTag) from trailing roboco-meta JSON fence | +| _as_work_entry | function | roboco/services/prompter.py:818 | Normalize a the_work entry (bare string -> {team:str}) so .get works on all entries | +| _cell_teams | function | roboco/services/prompter.py:835 | Distinct cell team values present in the_work, in order | +| _draft_cell_map | function | roboco/services/prompter.py:846 | Per-cell (team, project_id) map from the_work entries; de-duped by team; the multi-cell MegaTask root-subtask seam | +| derive_scale | function | roboco/services/prompter.py:882 | 'multi' when >1 cell participates, else 'single' | +| _clean_list | function | roboco/services/prompter.py:947 | coerce_str_list wrapper: trimmed non-empty string items, extracting dict-wrapped text | +| _copy_draft | function | roboco/services/prompter.py:956 | Shallow copy of draft dict with the_work unit dicts also copied, so _validate_and_coerce_draft cannot mutate the caller's dict | +| _text | function | roboco/services/prompter.py:896 | Trimmed string from a possibly-missing scalar | +| _bullets | function | roboco/services/prompter.py:901 | Render a markdown bullet list | +| _cell_label | function | roboco/services/prompter.py:906 | Display label for a team value | +| _render_work_entry | function | roboco/services/prompter.py:911 | Render one cell's slice: bold heading + summary + deliverable bullets | +| _render_the_work | function | roboco/services/prompter.py:926 | Render The Work section, prepending a board-led lead line when multi-cell | +| _section | function | roboco/services/prompter.py:938 | Append a markdown section when its body is non-empty | +| format_board_briefing | function | roboco/services/prompter.py:944 | Render board review entries into a markdown briefing to seed a re-draft intake session | +| compose_redraft_message | function | roboco/services/prompter.py:969 | Seed message for a re-draft session: current draft + board feedback | +| compose_description | function | roboco/services/prompter.py:985 | Deterministically build the markdown description from structured fields; fall back to model description if too sparse (<20 chars) | +| _compose_umbrella_draft | function | roboco/services/prompter.py:1015 | Build the branchless umbrella draft from batch + wave plan; task_type=planning | +| get_prompter_service | function | roboco/services/prompter.py:1059 | Factory: construct PrompterService with optional db session | +| LiveIntakeSession | dataclass | roboco/services/prompter_live.py:39 | One live chat: session_id, agent_id, asyncio queue, closed flag, parked task_id, last_activity timestamp | +| PrompterLiveRegistry | class | roboco/services/prompter_live.py:58 | Tracks live intake/secretary sessions; bridges panel<->container via push/stream/deliver; lifecycle open/close/park | +| PrompterLiveRegistry.open | method | roboco/services/prompter_live.py:68 | Register a live session; idempotent (returns existing un-closed session instead of orphaning its SSE queue) | +| PrompterLiveRegistry.get | method | roboco/services/prompter_live.py:88 | Return the session or None | +| PrompterLiveRegistry.is_alive | method | roboco/services/prompter_live.py:91 | True when a live un-closed session exists (panel reload reconnect decision) | +| PrompterLiveRegistry.close | method | roboco/services/prompter_live.py:101 | End a session: pop, mark closed, push _CLOSE sentinel to unblock the SSE stream | +| PrompterLiveRegistry.close_by_agent | method | roboco/services/prompter_live.py:110 | Close every live session bound to agent_id (forced kill); optional final error event; returns closed ids | +| PrompterLiveRegistry.park | method | roboco/services/prompter_live.py:129 | Mark session parked awaiting board review of task_id (keeps it alive for in-context re-draft) | +| PrompterLiveRegistry.find_by_task | method | roboco/services/prompter_live.py:148 | Return the live un-closed session parked for task_id, if any | +| PrompterLiveRegistry.push | method | roboco/services/prompter_live.py:157 | Queue one agent event for SSE; bump last_activity; False if no/gone session | +| PrompterLiveRegistry.idle_session_ids | method | roboco/services/prompter_live.py:166 | Return (session_id, agent_id) idle past threshold; excludes closed and board-parked sessions | +| PrompterLiveRegistry.stream | method | roboco/services/prompter_live.py:185 | Async generator yielding queued events until _CLOSE sentinel | +| PrompterLiveRegistry.deliver | method | roboco/services/prompter_live.py:198 | POST the human's text to the container's /turn receiver; bump last_activity; debug-log transient failures | +| get_live_registry | function | roboco/services/prompter_live.py:230 | Process-wide singleton accessor (lazily instantiates PrompterLiveRegistry) | +| SecretaryService | class | roboco/services/secretary.py:54 | Read company state + execute/queue CEO directives; BaseService subclass bound to a session | +| SecretaryService.read_company_state | method | roboco/services/secretary.py:63 | Aggregate goals + task counts + proposed pitches + pending directives for the CEO dashboard | +| SecretaryService.read_task | method | roboco/services/secretary.py:79 | Read a single task's id/title/status/team/assignee/description or NotFoundError | +| SecretaryService.get_directive | method | roboco/services/secretary.py:96 | Fetch a directive row by id or None | +| SecretaryService.list_directives | method | roboco/services/secretary.py:104 | List directives ordered by requested_at desc, optional status filter | +| SecretaryService.submit_directive | method | roboco/services/secretary.py:115 | Validate payload; persist row; if gated -> notify CEO pending + return; else run immediately | +| SecretaryService.confirm_directive | method | roboco/services/secretary.py:134 | CEO confirms a pending directive: set decided_by, run it | +| SecretaryService.reject_directive | method | roboco/services/secretary.py:142 | CEO rejects a pending directive: REJECTED + decided_by/at + result reason | +| SecretaryService.to_dict | staticmethod | roboco/services/secretary.py:153 | Serialize a directive row to a dict for API response | +| SecretaryService._pending_or_raise | method | roboco/services/secretary.py:171 | Fetch directive; NotFoundError if missing, ConflictError if not PENDING | +| SecretaryService._validate_payload | staticmethod | roboco/services/secretary.py:182 | Require the per-kind payload keys from _REQUIRED_PAYLOAD else ValidationError | +| SecretaryService._run | method | roboco/services/secretary.py:188 | Execute a directive: set result, EXECUTED on success, FAILED+error message on caught domain errors | +| SecretaryService._execute | method | roboco/services/secretary.py:246 | Dispatch by kind: relay/announce deliver via notification, update_charter upsert, approve_pitch approve+provision, else _control_task | +| SecretaryService._EDITABLE_TASK_FIELDS | ClassVar[frozenset] | roboco/services/secretary.py:278 | Wave-1: the full content-field allowlist the Secretary may edit on CEO confirmation (title/description/acceptance_criteria/priority/team/estimated_complexity/nature/assigned_to) — status is deliberately excluded (its own audited start/cancel/override path); git fields (branch/PR) are never editable | +| SecretaryService._control_task | method | roboco/services/secretary.py:302 | Task control action dispatch: **edit** (wave-1, full content surface via _edit_task), start (approve_and_start), cancel, or override status with CEO as actor | +| SecretaryService._edit_task | method | roboco/services/secretary.py:325 | Wave-1: apply the Secretary's edit — allowlisted content fields go through TaskService.update after enum coercion (team/estimated_complexity/nature); assigned_to is popped out and routed through claim-aware reassignment (_reassign_task) instead of a plain field set | +| SecretaryService._reassign_task | method | roboco/services/secretary.py:362 | Route an edit's reassignment through claim-aware paths: reassign_active_claim (reseeds heartbeat) when the task is claimed/in_progress, else the general reassign (review-state handoffs, or explicit unassign) — never a naive setattr on assigned_to | +| SecretaryService._resolve_assignee | method | roboco/services/secretary.py:384 | Resolve an edit's assigned_to to a UUID: accepts None (unassign), a UUID string, or an agent slug (same convention as the CEO chat's REST PATCH path) | +| SecretaryService._notify_ceo_pending | method | roboco/services/secretary.py:403 | Send an ack notification to the CEO that a gated directive awaits confirmation | +| get_secretary_service | function | roboco/services/secretary.py:416 | Factory: construct SecretaryService bound to a session | +| build_history_digest | function | roboco/services/prompter.py:1221 | Wave-1/2 prompter memory: render a chronological digest of recent tasks (top `limit`, reversed to oldest-first for a timeline read) into markdown bullet lines; empty input -> "" | +| project_history_digest | function | roboco/services/prompter.py:1236 | One project's rendered history digest via `TaskService.list_recent_for_project`; None if the project has no tasks | +| history_digest_layer | function | roboco/services/prompter.py:1253 | Ambient task-history-digest block for the in-scope project(s), one sub-block per project (headed by slug when >1 — the MegaTask case); None when no in-scope project has any tasks (no empty-header noise) | +| compact_task_rows | function | roboco/services/prompter.py:1286 | Render TaskTable rows into the compact id/title/status/team/priority dicts returned by the intake `search_past_tasks` HTTP route | +| TaskService.list_recent_for_project | method | roboco/services/task.py:6361 | Recent non-cancelled tasks for a project ordered by coalesce(completed_at, updated_at, created_at) desc — backs the prompter's per-project history digest so a just-touched task surfaces ahead of an old completed one; CANCELLED tasks are excluded (abandoned work is not precedent) | +| TaskService.search_tasks | method | roboco/services/task.py:6384 | Case-insensitive ILIKE search over title/description + id-prefix match; backs the panel's task search bar (GET /tasks/summary?q=), the Secretary's task-by-name lookup (GET /secretary/tasks?q=), and the intake `search_past_tasks` tool | +| search_past_tasks (route) | route | roboco/api/routes/prompter_live.py:376 | GET /live/{session}/search-tasks: session-aliveness-gated (mirrors /events' trust boundary — the intake container has no agent identity) bounded compact search calling TaskService.search_tasks + compact_task_rows | +| query_past_tasks / format_search_results | function | roboco/mcp/intake_server.py:108,145 | Shared HTTP-call + bounding + rendering logic for `search_past_tasks`, module-level so both the grok MCP tool and the Claude SDK in-process tool call the exact same implementation | +| search_past_tasks (grok MCP tool) | mcp tool | roboco/mcp/intake_server.py:161 | Grok-CLI intake's "have we done something like this before?" tool; reads ROBOCO_PROMPTER_SESSION_ID, delegates to query_past_tasks + format_search_results | +| _search_past_tasks (Claude SDK in-process tool) | tool | roboco/agent_sdk/intake_driver.py:512 | Claude SDK driver's in-process parity tool for the same feature — imports query_past_tasks/format_search_results from intake_server.py directly (one implementation, both runtimes) | ## Data Flow -Inputs: an AsyncSession, a project_slug, an agent_id (UUID or slug), optionally a git_url/default_branch/force. The orchestrator, GitService, TaskService, conventions service, dep_update_engine, and gateway content_actions all obtain a WorkspaceService via get_workspace_service(session) (or WorkspaceService(db) directly in the spawn path). Control flow on ensure_workspace: _lookup_agent_or_raise -> get_workspace_path -> acquire per-(project,agent) asyncio.Lock -> if _is_workspace_healthy (.git+HEAD+objects): _ensure_agent_owned (to_thread), prune broken refs, scoped _fetch_origin_best_effort (30s TTL, force override), re-chown, install_dev_deps (digest-cache hit short-circuits), return. Else: rmtree any partial/stub dir, ProjectService.get_by_slug, resolve the clone target branch via `head_branch(project)` (the env-ladder head rung — `roboco.models.env_branches`, shimmed from `default_branch` when no ladder is declared), _resolve_git_token (decrypt PAT; raise on HTTPS-with-no-token), _clone_repo (git clone --branch --no-tags, configure identity/fileMode, scrub PAT from remote URL, _assert_no_pat_leak scanning .git/** for ghp_/github_pat_/x-access-token, chown, install_dev_deps), _maybe_scaffold_conventions (once-per-process, flag-gated). Per-task worktree path: get_clone_root_path + get_worktree_path (.worktrees/{task_short_id}); ensure_worktree runs git worktree add -b (or reuses an existing branch ref), _link_shared_venv (symlink to clone-root .venv only if it exists), chowns both worktree and clone root. GitService.create_branch calls ensure_worktree; commit/rebase paths call ensure_worktree_for_resume via GitService._ensure_worktree_for_commit; the orchestrator's _ensure_worktree_before_spawn calls ensure_worktree_self_heal (post-536bbb64) which first fetches the branch ref from origin if the local ref is absent after a re-clone, then delegates to ensure_worktree; TaskService.complete/cancel call remove_worktree. ensure_read_clone is called by ConventionsService for the project-level read clone at _meta/conventions, hard-reset to origin/default. dry_upgrade_changes_lockfile (dep_update_engine) clones the read clone --local --no-hardlinks into a throwaway under the read-clone lock, runs dep_update_command, and checks git status --porcelain on lockfile paths. Outputs: workspace Path (and side effects: on-disk clone/worktree, .venv symlink, .git/.roboco-dep-install + .git/.roboco-toolchain markers, root-owned refs re-chowned to agent uid). All git/subprocess work runs via asyncio.to_thread; tokens are injected only transiently into argv (never written to .git/config) and scrubbed post-clone. +Intake: the orchestrator spawns a prompter container and calls PrompterLiveRegistry.open(session_id, INTAKE_AGENT_ID); the panel SSE endpoint calls stream() and the message endpoint calls deliver() -> POST http://roboco-agent-{agent_id}:{SDK_PORT}/turn. The container driver POSTs normalized StreamChunks to the relay push() endpoint. When the agent emits a roboco-meta fence, parse_readiness extracts ReadinessTag (covered/ready/scale) used by the orchestrator to decide proposal readiness. The CEO confirms via panel: confirm_live_draft (single) or confirm_live_batch (MegaTask) -> PrompterService.create_task_from_draft -> TaskService.create (DB). For a batch, _sequence_drafts (pure SequencingService.analyze) computes waves/edges, the umbrella is created branchless via _compose_umbrella_draft, N root-subtasks are created with BatchPlacement(parent=umbrella, batch_id, sequence=wave_index), then TaskService.add_dependency wires each edge (b depends on a). preview_batch returns the same waves without creating. update_live_draft applies board feedback to an existing task (update + approve_and_start or re-board). On board review, registry.park(session_id, task_id) keeps the chat alive; find_by_task recovers it for the re-draft injection. Idle sweep: orchestrator calls idle_session_ids(threshold) and close()s abandoned chats; close_by_agent fires on forced kill. Secretary: routes /api/secretary/* call read_company_state/read_task/submit_directive/confirm_directive/reject_directive -> SecretaryService -> TaskService/CompanyGoalsService/PitchService/NotificationService (relay/announce deliver via notification to the target agent(s)); gated kinds (UPDATE_CHARTER/CONTROL_TASK/APPROVE_PITCH/ANNOUNCE) persist PENDING + notify CEO, then confirm_directive runs _execute with the CEO as actor; RELAY_MESSAGE runs immediately. A CONTROL_TASK directive's payload["action"] fans out inside _control_task: "edit" (wave-1) is the new full-content path — _edit_task applies the allowlisted fields via TaskService.update after enum coercion, and pops assigned_to for claim-aware reassignment (_reassign_task) rather than a plain field set; "start"/"cancel"/"override" are the pre-existing status-only actions. The CEO refers to tasks by NAME in the Secretary chat, so GET /api/secretary/tasks?q= (TaskService.search_tasks) resolves a name to a concrete id before a directive targets it. Prompter memory (wave 1/2): at intake spawn, the orchestrator's `_resolve_history_digest_ambient` calls `history_digest_layer`, which fans out `project_history_digest` per in-scope project — each pulling `TaskService.list_recent_for_project` and rendering it via `build_history_digest` — into one ambient "Recent tasks" block injected into the spawn prompt; mid-conversation, the intake agent's `search_past_tasks` tool (routed through `query_past_tasks`/`format_search_results` in roboco/mcp/intake_server.py, shared byte-for-byte with the Claude SDK's in-process `_search_past_tasks` tool in roboco/agent_sdk/intake_driver.py) hits GET /live/{session}/search-tasks -> TaskService.search_tasks -> compact_task_rows. ## Mermaid ```mermaid -flowchart TD - caller["Callers: orchestrator spawn, GitService, TaskService, ConventionsService, DepUpdateEngine, gateway content_actions"] - factory["get_workspace_service(session)"] - WS["WorkspaceService"] - caller --> factory --> WS - - subgraph ensure["ensure_workspace (per project+agent lock)"] - health{"_is_workspace_healthy?
.git+HEAD+objects"} - own1["_ensure_agent_owned"] - prune["_prune_broken_refs"] - fetch["_fetch_origin_best_effort
scoped, 30s TTL, force override"] - deps["install_dev_deps
digest-cache hit -> skip"] - health -->|yes| own1 --> prune --> fetch --> own2["re-chown after fetch"] --> deps --> ret1["return workspace"] - health -->|no| rm["rmtree partial/stub"] --> proj["ProjectService.get_by_slug"] - proj --> tok["_resolve_git_token (decrypt PAT)"] - tok --> clone["_clone_repo
clone --no-tags --branch
configure+scrub PAT
leak-check + chown + install_dev_deps"] - clone --> scaffold["_maybe_scaffold_conventions
once-per-process, flag-gated"] --> ret2["return workspace"] +sequenceDiagram + participant CEO + participant Panel + participant Orchestrator + participant Reg as PrompterLiveRegistry + participant Container as prompter container + participant PS as PrompterService + participant TS as TaskService + CEO->>Orchestrator: start intake chat + Orchestrator->>Reg: open(session_id, INTAKE_AGENT_ID) + Panel->>Reg: stream(session_id) (SSE) + CEO->>Panel: type message + Panel->>Reg: deliver(session_id, text) + Reg->>Container: POST /turn + Container->>Reg: push(session_id, StreamChunk) + Reg->>Panel: yield event + CEO->>Panel: confirm draft (board/main_pm) + Panel->>PS: confirm_live_draft(draft, agent_id, route) + PS->>PS: create_task_from_draft + PS->>TS: create(TaskCreateRequest, confirmed_by_human=True) + TS-->>Panel: task_id + alt MegaTask + Panel->>PS: confirm_live_batch(title, drafts, project_ids, route) + PS->>PS: _sequence_drafts -> waves + edges + PS->>TS: create umbrella (branchless) + loop each draft + PS->>TS: create root-subtask (BatchPlacement) + end + loop each edge (a,b) + PS->>TS: add_dependency(b, a) + end end - WS --> ensure + Orchestrator->>Reg: park(session_id, task_id) (board review) + Orchestrator->>Reg: idle_session_ids(threshold) -> close() abandoned +``` - subgraph wt["Per-task worktrees (F123)"] - gwp["get_worktree_path
{clone_root}/.worktrees/{task}"] - ew["ensure_worktree
worktree add -b branch base"] - ewr["ensure_worktree_for_resume
re-add pruned, no -b"] - rmw["remove_worktree
worktree remove --force + prune"] - link["_link_shared_venv
symlink -> ../../.venv if exists"] - ew --> link --> chown2["_ensure_agent_owned x2"] - ewr --> link - end - GitService --> ew - GitService --> ewr - orchestrator --> ewr - TaskService --> rmw - - subgraph rc["Read clone + dep probe"] - erc["ensure_read_clone
_meta/conventions, hard-reset origin/default, 30s TTL"] - src["_sync_read_clone
token-authed fetch + reset --hard FETCH_HEAD"] - dry["dry_upgrade_changes_lockfile
local --no-hardlinks clone under lock
run dep_update_command -> dirty?"] - erc --> src - dry --> erc - end - ConventionsService --> erc - DepUpdateEngine --> dry +```mermaid +stateDiagram-v2 + direction LR + [*] --> Pending: submit_directive (gated) + Pending --> Executed: confirm_directive (_run ok) + Pending --> Rejected: reject_directive + Pending --> Failed: _run raised domain error + [*] --> Executed: submit_directive (RELAY_MESSAGE, direct) + Executed --> [*] + Rejected --> [*] + Failed --> [*] ``` ## Logical Tree ``` -WorkspaceService slice -+-- Module-level helpers -| +-- _chown_entry / _make_owner_and_group_rw / _own_and_grant_rw -| +-- _iter_ownable_entries (prunes _PRUNE_DIRS) -| +-- _ensure_agent_owned (whole-tree chown+chmod, best-effort) -| +-- _resolve_clone_root (worktree -> clone root path logic) -| +-- _uv_subprocess_env (UV_PYTHON_INSTALL_DIR pin) -| +-- _monotonic (test-patchable clock) -| +-- _ensure_lock_for (per project+agent asyncio.Lock) -| +-- _inject_token_into_url (PAT into HTTPS URL) -| +-- _lockfile_digest / _detect_dep_commands -| +-- markers: _DEP_INSTALL_MARKER, _TOOLCHAIN_MARKER -+-- WorkspaceError -+-- WorkspaceService -| +-- Path math: get_workspace_path / get_clone_root_path / get_worktree_path -| +-- Worktree ops: _clone_root_default_branch / _park_clone_root_off_branch / _worktree_git / _link_shared_venv / ensure_worktree / ensure_worktree_for_resume / _fetch_branch_ref / ensure_worktree_self_heal / remove_worktree / delete_local_branch -| +-- Agent lookup: resolve_workspace / _lookup_agent_or_raise -| +-- Health + refs: _is_workspace_healthy / _prune_broken_refs / _fetch_origin_best_effort -| +-- Token: _resolve_git_token / _read_clone_token -| +-- Clone + ensure: ensure_workspace / _clone_repo / _maybe_scaffold_conventions -| +-- Read clone: ensure_read_clone / _sync_read_clone -| +-- Dev deps + toolchain: install_dev_deps / _resolve_toolchain_target / _record_toolchain / _run_toolchain_smoke / read_toolchain_status / _dep_install_cache_hit / _run_dep_install -| +-- Dep-update probe: dry_upgrade_changes_lockfile / _clone_local_into / _probe_lockfile_on_clone -| +-- Misc: workspace_exists / list_workspaces / _resolve_branch_to_project_slug / fetch_branch_for_inspection / delete_workspace -+-- get_workspace_service (factory) +intake-secretary + PrompterService (roboco/services/prompter.py) + Draft validation & coercion + _validate_and_coerce_draft + _validate_draft_target (project/product/cell-map/umbrella) + _coerce_draft_enums (team/task_type/nature/complexity) + _coerce_priority + _resolve_uuid_field + _resolve_draft_assignee + Team routing + _resolve_owning_team + _assignee_is_board + _lead_cell_team + Task creation + create_task_from_draft (single + placement) + confirm_live_draft (board / main_pm route) + update_live_draft (re-draft in place) + MegaTask batch + _sequence_drafts -> SequencingService.analyze + preview_batch (no-create preview) + _validate_batch_scope (>=2 distinct projects, in-scope) + confirm_live_batch (umbrella + N root-subtasks + edges) + _compose_umbrella_draft + Pure helpers + parse_readiness, compose_description, format_board_briefing, compose_redraft_message + _as_work_entry, _cell_teams, _draft_cell_map, derive_scale, _clean_list, _text, _bullets, _cell_label, _render_work_entry, _render_the_work, _section + Prompter memory (wave 1/2): build_history_digest, project_history_digest, history_digest_layer, compact_task_rows + Dataclasses: ReadinessTag, BatchPlacement + PrompterLiveRegistry (roboco/services/prompter_live.py) + LiveIntakeSession dataclass (queue, closed, task_id, last_activity) + Lifecycle: open, get, is_alive, close, close_by_agent, park, find_by_task + Agent->panel: push, stream, idle_session_ids + Panel->agent: deliver (POST /turn) + Singleton: _RegistryHolder, get_live_registry + SecretaryService (roboco/services/secretary.py) + Reads: read_company_state, read_task + Directives: get_directive, list_directives, submit_directive, confirm_directive, reject_directive, to_dict + Internals: _pending_or_raise, _validate_payload, _run, _execute, _control_task, _notify_ceo_pending + Task edit (wave-1): _EDITABLE_TASK_FIELDS, _edit_task, _reassign_task (claim-aware), _resolve_assignee (uuid-or-slug) ``` ## Dependencies -- Internal: roboco.config.settings, roboco.db.tables.AgentTable, roboco.db.tables.TaskTable, roboco.logging.get_logger, roboco.models.base.Team, roboco.models.env_branches.head_branch (env-ladder head-rung resolver backing the clone target in ensure_workspace and ensure_read_clone), roboco.services.toolchain.resolve_target_python, roboco.services.project.get_project_service / ProjectService, roboco.services.conventions.get_conventions_service / ConventionsService, roboco.utils.crypto.EncryptionError, roboco.db.base.get_db_context (orchestrator spawn path) -- External: asyncio, contextlib, json, math, os, re, shlex, shutil, subprocess, tempfile, time, pathlib.Path, uuid.UUID, collections.abc.Iterator, sqlalchemy.ext.asyncio.AsyncSession, sqlalchemy.select, hashlib (lazy in _lockfile_digest), stat (lazy in _make_owner_and_group_rw), base64 (lazy in fetch_branch_for_inspection) +- Internal: roboco.services.task.get_task_service / TaskService, roboco.services.sequencing.SequencingService, roboco.services.company_goals.get_company_goals_service, roboco.services.pitch.get_pitch_service, roboco.services.notification.NotificationService, roboco.services.base.BaseService/NotFoundError/ConflictError/ValidationError/ServiceError, roboco.db.tables.AgentTable/TaskTable/SecretaryDirectiveTable, roboco.foundation.identity.CELL_TEAMS/AGENTS/role_for_uuid_or_none, roboco.foundation.policy.batch.is_batch_umbrella/main_pm_cannot_own_code/pm_cannot_own_code, roboco.foundation.policy.content.validators.coerce_str_list, roboco.foundation.policy.sequencing.models.DraftSurface/SequencePlan/SequencingError, roboco.foundation.policy.lifecycle (TaskStatus source), roboco.models.base (AgentRole/Complexity/TaskNature/TaskStatus/TaskType/Team), roboco.models.product.ProductCellMapping, roboco.models.task.TaskCreateRequest, roboco.models.secretary (DirectiveKind/DirectiveStatus/GATED_KINDS), roboco.seeds.initial_data.AGENT_UUIDS, roboco.utils.converters.require_uuid +- External: sqlalchemy (select, AsyncSession), structlog, httpx, asyncio, contextlib, json, re, dataclasses, uuid, datetime ## Entry Points | Name | File | Trigger | |---|---|---| -| ensure_workspace | roboco/services/workspace.py | GitService.create_branch_for_task / push / PR ops; orchestrator spawn ensure; gateway content_actions; called transitively by many verbs | -| ensure_worktree | roboco/services/workspace.py | GitService.create_branch_for_task on fresh claim (worktree add -b ) | -| ensure_worktree_for_resume | roboco/services/workspace.py | GitService._ensure_worktree_for_commit (commit/rebase paths) | -| ensure_worktree_self_heal | roboco/services/workspace.py | orchestrator._ensure_worktree_before_spawn before -w container launch (replaces the former ensure_worktree_for_resume call there; handles vanished clones + missing branch refs) | -| remove_worktree | roboco/services/workspace.py | TaskService terminal/cancel paths + claim-rollback (mid-claim failure) | -| delete_local_branch | roboco/services/workspace.py | TaskService terminal/cancel paths (right after remove_worktree) + GitService.cleanup_stale_branches sweep | -| ensure_read_clone | roboco/services/workspace.py | ConventionsService.scaffold/effective-map reads (project-level conventions metadata) | -| dry_upgrade_changes_lockfile | roboco/services/workspace.py | DepUpdateEngine periodic probe loop | -| fetch_branch_for_inspection | roboco/services/workspace.py | gateway content_actions (QA/Documenter/PM need to read a dev branch) | -| read_toolchain_status | roboco/services/workspace.py | GitService spawn-time toolchain runnability check | -| list_workspaces / workspace_exists / delete_workspace | roboco/services/workspace.py | admin API routes / project service maintenance | +| POST /api/prompter/live/{session}/confirm-draft (confirm_live_draft) | roboco/api/routes/prompter_live.py | panel confirm button -> PrompterService.confirm_live_draft | +| POST /api/prompter/live/{session}/confirm-batch (confirm_live_batch) | roboco/api/routes/prompter_live.py | panel MegaTask confirm -> PrompterService.confirm_live_batch | +| GET /api/prompter/live/preview-batch (preview_batch) | roboco/api/routes/prompter_live.py | panel pre-confirm preview -> PrompterService.preview_batch | +| POST /api/prompter/live/{session}/redraft (update_live_draft) | roboco/api/routes/prompter_live.py | panel re-draft confirm -> PrompterService.update_live_draft | +| relay push/stream/deliver/is_alive endpoints | roboco/api/routes/prompter_live.py + secretary_live.py | panel SSE + message POST over PrompterLiveRegistry | +| orchestrator live-intake spawn/reap/idle hooks | roboco/runtime/orchestrator.py | _spawn_intake_container / _spawn_secretary_container / idle-reap sweep / board-review park / close_by_agent on kill | +| GET /api/prompter/live/{session}/search-tasks (search_past_tasks) | roboco/api/routes/prompter_live.py | Intake agent's `search_past_tasks` tool -> TaskService.search_tasks + compact_task_rows; session-aliveness-gated | +| POST /api/secretary/state, /task, /directive, /directive/{id}/confirm\|reject | roboco/api/routes/secretary.py | Secretary panel surface -> SecretaryService reads + directive lifecycle | +| GET /api/secretary/tasks?q= (search_tasks) | roboco/api/routes/secretary.py | Secretary or CEO resolves a task NAME to concrete id(s) -> TaskService.search_tasks, for targeting a `control_task` directive | ## Config Flags -- ROBOCO_WORKSPACES_ROOT (settings.workspaces_root; default /data/workspaces) -- ROBOCO_WORKSPACE_AUTO_CLONE (settings.workspace_auto_clone; default true) -- ROBOCO_WORKSPACE_CLONE_TIMEOUT (settings.workspace_clone_timeout; default 300s) - bounds git clone + fetch_branch_for_inspection fetch -- ROBOCO_WORKSPACE_REFRESH_FETCH_TIMEOUT_SECONDS (settings.workspace_refresh_fetch_timeout_seconds; default 60s) - bounds the healthy-clone scoped refresh fetch -- ROBOCO_WORKSPACE_INSTALL_DEV_DEPS (settings.workspace_install_dev_deps; default true) - gates post-clone dev-dep install -- ROBOCO_WORKSPACE_DEP_INSTALL_TIMEOUT_SECONDS (settings.workspace_dep_install_timeout_seconds; default 600s) - bounds uv sync / pnpm install / toolchain smoke / dep-upgrade probe -- ROBOCO_TOOLCHAIN_MATCH_ENABLED (settings.toolchain_match_enabled; default off) - gates provisioning against the target project's declared Python + the runnability smoke marker -- ROBOCO_CONVENTIONS_ENABLED (settings.conventions_enabled; default off) - gates the first-clone conventions scaffold PR -- ROBOCO_AGENT_UID / ROBOCO_AGENT_GID (env; default 1000/1000) - the agent container user the workspace is chowned to +- ROBOCO_WORKSPACE_AUTO_CLONE / ROBOCO_WORKSPACE_CLONE_TIMEOUT (intake multi-repo clone scope: _clone_intake_scope, indirectly via orchestrator) +- ROBOCO_SELF_HEAL_ORIGINATE_ENABLED etc. do NOT gate this slice +- No direct ROBOCO_* flag in these three files; intake is a core capability (not feature-flagged), secretary is always-on; MegaTask is additive core, not gated ## Gotchas -- The per-(project,agent) asyncio.Lock (_ENSURE_WORKSPACE_LOCKS) is process-local only. Across orchestrator processes (or restarts) two coroutines can still race the .git-exists check; the rmtree-partial-then-clone path assumes single-process serialization. -- _is_workspace_healthy requires .git + HEAD + objects/ — a stub clone from a failed `git clone` (only FETCH_HEAD) is intentionally rejected and re-cloned. A regression that loosens this check re-mounts agents on broken clones. -- _fetch_origin_best_effort is credential-LESS (token was scrubbed from .git/config by _clone_repo). For PRIVATE repos the refresh fetch silently fails (downgraded to DEBUG) and the workspace stays at clone-time refs until the next token-bearing operation (create_branch / fetch_branch_for_inspection). Stale-base risk for private repos. -- _ensure_agent_owned is called TWICE in the healthy path (before and after the fetch) because the root-side fetch writes root-owned pack/refs under .git/objects and .git/refs — skipping the second chown leaves the agent unable to update refs. -- 30s TTL fetch caches (_fetch_cache instance attr, _read_clone_synced module attr) are keyed by str(workspace); a Path that resolves to the same dir via a different route (worktree vs clone root) would not share a cache entry. -- _link_shared_venv only symlinks if clone_root/.venv EXISTS (F-fix 0f7d6929). On the very first claim, install_dev_deps provisions the venv AFTER ensure_worktree already ran — the worktree add path can run before .venv exists, so the symlink is skipped and a later ensure (resume/commit) self-heals it. If no later ensure fires, uv re-syncs a worktree-local venv (the bug the F-fix mitigated but did not fully close — recovery of an already-clobbered worktree venv is out of scope). -- _PRUNE_DIRS excludes .venv/node_modules from the chown walk for speed, but .uv-python is intentionally NOT pruned (so the fetched CPython is chowned). If .uv-python grows huge on a monorepo, the walk slows. -- _clone_repo does NOT pass --single-branch: agents/QA/doc must fetch peer feature branches. A regression adding --single-branch would silently break `checkout origin/feature/...`. -- _assert_no_pat_leak scans .git/** for ghp_/github_pat_/x-access-token bytes and rm-trees the workspace on any hit. Binary pack files are read as bytes; a coincidental byte sequence in a blob is unlikely but theoretically possible — false positives destroy the workspace. -- Both clone-failure except branches (CalledProcessError, TimeoutExpired) rmtree the workspace before raising (F063 bb94e6ba). If rmtree itself fails (busy mount), the half-configured clone with the PAT in .git/config could survive — the leak-check did not run. -- _maybe_scaffold_conventions uses a process-wide _SCAFFOLD_ATTEMPTED set: the scaffold is attempted at most once per project per orchestrator process. A first-clone failure is never retried within the same process lifetime. -- dry_upgrade_changes_lockfile holds the read-clone lock only for the local clone step, then releases it before the upgrade runs. The tiny gap between ensure_read_clone releasing and the probe re-acquiring is safe only because any concurrent _sync_read_clone completes under the lock first — a future change that interleaves could race. -- get_workspace_path raises WorkspaceError if team is None rather than producing a literal 'None' segment; resolve_workspace falls back to Team.BACKEND when agent.team is falsy — agents missing a team silently land under backend/. -- fetch_branch_for_inspection reuses workspace_clone_timeout (300s) for a single-branch fetch, not the shorter refresh timeout — a hung remote blocks the QA/Doc verb for 5 minutes. -- delete_local_branch only detaches the ref; remove_worktree only detaches the worktree. Callers MUST run remove_worktree first — `git branch -d/-D` refuses a branch still checked out elsewhere in the clone (the worktree). Skipping the order silently no-ops the branch delete (check=False swallows the refusal). - - -## Drift from CLAUDE.md -- CLAUDE.md (Git Credentials / Token flow) says 'HTTPS URLs require tokens - attempting to clone without a token will raise WorkspaceError' — matches _resolve_git_token (line 787). No drift. -- CLAUDE.md (Multi-Agent Workspace Structure) says a Python workspace runs `uv sync --extra dev` (not plain `uv sync`) so the dev extra is present — matches _detect_dep_commands (line 359). No drift. -- CLAUDE.md (Work Sessions / fresh claim) says a fresh claim git-resets the workspace to a clean tree (`git reset --hard`) before checking out the new branch. ACTUAL code: F123 (67107f8a) replaced that reset+checkout with per-task `git worktree add` under {clone_root}/.worktrees/{task}/ — the reset --hard no longer runs on fresh claim. This is a real drift between the doc narrative and the post-F123 code. -- CLAUDE.md (Architectural Conventions Standard) says the read clone is pinned to the default branch's HEAD via WorkspaceService.ensure_read_clone — matches (line 958). No drift. -- CLAUDE.md (Dependency-update bot) says WorkspaceService.dry_upgrade_changes_lockfile runs dep_update_command in a throwaway clone of the READ CLONE and the read clone is never mutated — matches (line 1459). No drift. +- PrompterLiveRegistry.open is deliberately idempotent: a second open for an un-closed session returns the existing one instead of swapping the queue, because stream() captures the queue once and a fresh queue would strand the browser SSE on the old one while events push to the new one. +- Registry is a process-wide singleton held on _RegistryHolder (not a `global`); orchestrator is single-process and holds container state in memory — the relay is in-process only, not cross-process. +- deliver() logs transient POST failures at DEBUG (not ERROR) because the opening-message delivery retries until the container receiver is up; callers surface real failure (the /messages route 404s, _deliver_when_ready warns once after N tries). +- park() keeps a session alive (opposite of close) so board feedback can be injected in-context for an in-place re-draft; idle_session_ids explicitly excludes task_id-set (parked) sessions from idle reaping. +- TaskService is imported lazily inside create_task_from_draft / confirm_live_batch / update_live_draft to avoid circular imports. +- PM + code is structurally impossible: intake coerces code->planning via `_coerce_pm_code_to_planning`, which has two layers — team-based (main_pm_cannot_own_code) and assignee-based (pm_cannot_own_code for any PM assignee on a cell team). The umbrella is task_type=planning. TaskService.create is the backstop for non-intake HTTP paths. +- AGENTS['ceo'].uuid is captured at import time as _CEO_ID in secretary.py — CEO identity is a fixed seed uuid, not a DB lookup. +- _draft_cell_map de-dupes by team (first mapping wins) because task_cell_projects is unique per (task, team); a second the_work entry for the same cell is silently dropped. +- _compose_umbrella_draft produces a draft with NO project_id/product_id (branchless); _validate_draft_target's umbrella branch hard-rejects any target on it. +- Secretary _run catches ConflictError/NotFoundError/ValidationError/ValueError/KeyError -> FAILED with `error: {exc}` in result; any other exception propagates (no rollback of the flush). +- GATED_KINDS = {UPDATE_CHARTER, CONTROL_TASK, APPROVE_PITCH, ANNOUNCE}; only RELAY_MESSAGE runs immediately on submit_directive — ANNOUNCE is gated (needs CEO confirm), despite being a 'post a message' shape. +- compose_description falls back to the raw model description if the composed body is < _MIN_DESCRIPTION_LEN (20) chars — so a too-sparse structured draft still clears the schema minimum. +- SecretaryService._edit_task never touches status — _EDITABLE_TASK_FIELDS deliberately excludes it (status rides the separate audited start/cancel/override actions), so an "edit" directive that also needs a status change requires a second CONTROL_TASK directive. +- SecretaryService._reassign_task branches on the task's CURRENT status at call time: claimed/in_progress goes through reassign_active_claim (reseeds the heartbeat so the new assignee isn't immediately stale to the reaper), everything else falls through to the general reassign — a caller relying on one code path for both is testing the wrong branch depending on task state. +- history_digest_layer / build_history_digest return None / "" respectively on no data — a brand-new project or a board-level (no-project) spawn injects nothing into the ambient prompt (no empty "Recent tasks" header noise), which also means there is no explicit signal in the prompt that the digest was even attempted. +- search_past_tasks (both the grok MCP tool and the Claude SDK in-process tool) reads ROBOCO_PROMPTER_SESSION_ID from the environment and calls the session-scoped HTTP route — a tool call with no live session (or a session the registry has already closed) returns a plain string error, not an exception, so a stale intake container can call it silently forever without a hard failure surfacing. ## Changes Since Baseline | SHA | Subject | Impact | |---|---|---| -| bb94e6ba | [F063] workspace._clone_repo: rmtree half-configured clone on failure | Both clone-failure except branches (CalledProcessError, TimeoutExpired) now shutil.rmtree the workspace before raising WorkspaceError, so a half-configured clone with the PAT still in .git/config cannot survive and be re-mounted by the next ensure_workspace health short-circuit. | -| c3057bb3 | Updated domain | Trivial: git config user.email domain bump in _clone_repo's _configure_git (now {slug}@roboco.tech). No behavior change beyond commit author email. | -| 1a773e45 | [F116] hold the read-clone lock across the dep-probe local clone | dry_upgrade_changes_lockfile split into _clone_local_into (run UNDER the _meta-conventions lock) + _probe_lockfile_on_clone (lock-free on the independent copy), closing a race where a concurrent ensure_read_clone hard-reset could mutate the read clone mid-clone. | -| 3441e371 | [sweep] strip Fxxx audit-ID tokens + trim bloated comments/docstrings | Comment/docstring prose only — no code-line edits in workspace.py. Reduced narrative bulk; no behavioral change. | -| 67107f8a | [F123] per-task git worktrees — coordinator PM roots no longer clobber each other | Major: added get_clone_root_path/get_worktree_path/ensure_worktree/ensure_worktree_for_resume/remove_worktree/_link_shared_venv/_worktree_git + _resolve_clone_root + _uv_subprocess_env worktree-awareness. Replaced the fresh-claim `git reset --hard` + `checkout -b` with `git worktree add` under {clone_root}/.worktrees/{task}/ so a coordinator PM holding multiple in_progress roots no longer clobbers one root's working tree by checking out another's branch. .venv symlinked from worktree to clone root; .uv-python gitignored. | -| 0f7d6929 | [F-fix] gate the worktree .venv symlink on the clone-root venv existing | _link_shared_venv now no-ops when clone_root/.venv does not yet exist (instead of dangling a symlink), so uv no longer errors or silently re-syncs a worktree-local venv in the near-zero gap before install_dev_deps provisions the clone-root venv. A later ensure self-heals the link. | +| 15effce0 | feat(megatask): per-cell project map root-subtasks (multi-project, multi-cell) + main_pm+code impossibility + re-draft/batch hardening | Only commit touching this slice since baseline (prompter.py +228/-55; prompter_live.py and secretary.py unchanged). Adds the ad-hoc per-cell project map as a third draft target shape: _draft_cell_map, _MULTI_CELL_MIN, has_cell_projects param on _validate_draft_target, cell_projects on TaskCreateRequest, _validate_batch_scope counting per-cell pids. Adds main_pm_cannot_own_code coercion (code->planning) and switches umbrella task_type CODE->PLANNING. Extracts _validate_and_coerce_draft (coerces list fields via coerce_str_list) and _resolve_draft_assignee. Adds _as_work_entry to tolerate bare-string the_work entries. Changes _clean_list to use coerce_str_list (extracts dict-wrapped text instead of str(dict)). | -> Post-snapshot updates (since 2026-06-29): 5 commits touched workspace.py. (1) 9faf2763 [hotfix] strip VIRTUAL_ENV + UV_PROJECT_ENVIRONMENT from _uv_subprocess_env so workspace uv calls stop warning about the image-baked /app/.venv pin. (2) cfe725da [hotfix] worktree: clone root left on the task branch caused fatal "already checked out" on every worktree add re-dispatch — added _clone_root_default_branch + _park_clone_root_off_branch; ensure_worktree and ensure_worktree_for_resume now call _park_clone_root_off_branch before the add to restore the F123 invariant. (3) 536bbb64 (logical-gap sweep PR#286) added _fetch_branch_ref + ensure_worktree_self_heal: the orchestrator's _ensure_worktree_before_spawn now calls ensure_worktree_self_heal instead of bare ensure_worktree_for_resume so a vanished clone (redeploy/disk loss) that left no local branch ref recovers the pushed commits from origin before re-attaching. (4) 3aff6e04 and 15effce0 (gap-fill PRs #285/#283) contributed earlier worktree + dep-probe plumbing (the _clone_local_into / _probe_lockfile_on_clone split already captured in the baseline). +> Post-snapshot updates (since 2026-06-29): 536bbb64 (Chore/all/logical gaps sweep, PR#286, 2026-06-30) touched prompter.py only (prompter_live.py and secretary.py still unchanged). Key changes: (1) fixes Risk #1 — 1-cell map branch now conditioned on `resolved_project_id is None and resolved_product_id is None` so a top-level target is no longer silently dropped; (2) fixes Risk #2 — `_draft_cell_map` now raises `ValidationError` on a malformed project_id instead of silently continuing; (3) fixes Risk #4 — `create_task_from_draft` calls `_copy_draft` first so `_validate_and_coerce_draft` never mutates the caller's dict; (4) fixes Risk #5 — product/board routing is now checked BEFORE the multi-cell map force (multi-cell is inside the `if resolved_product_id is None:` branch); (5) extracts code->planning coercion into `_coerce_pm_code_to_planning`, extending it to cover PM assignees on any team (via the new `pm_cannot_own_code` helper imported from `roboco.foundation.policy.batch`); (6) adds `_copy_draft` module-level function. LOC grew from ~1066 to 1142. > -> Further post-snapshot update (#534, env-branches ladder): `ensure_workspace`'s fresh-clone branch and `ensure_read_clone` both resolve their target branch via `roboco.models.env_branches.head_branch(project)` — the env-ladder's head rung — instead of reading `project.default_branch` directly. A project with no declared ladder resolves to the identical `default_branch` value via the read-time shim, so this is behavior-preserving until the CEO declares a real ladder in the panel. +> `d1cf6ecb` Wave 1: PR-gate turn cut, task search, trace timestamps, Secretary edits + e2e scenarios 2–3 (#295) — secretary.py gains the full `edit` action (`_EDITABLE_TASK_FIELDS`, `_edit_task`, `_reassign_task`, `_resolve_assignee`) on `_control_task`; prompter.py gains the prompter-memory digest builders (`build_history_digest`, `project_history_digest`, `history_digest_layer`, `compact_task_rows`) plus the `TaskService.list_recent_for_project` / `search_tasks` backing queries; adds the `GET /live/{session}/search-tasks` route and the `search_past_tasks` MCP tool + Claude-SDK in-process parity tool. First commit to touch secretary.py since baseline. > -> (open PR #548, branch `feature/wave-2-hygiene-charts`, 2026-07-17) Added `delete_local_branch` (line 787) so `TaskService`'s cancel/terminal-completion cleanup and `GitService.cleanup_stale_branches` can reap a spent local branch ref, not just the worktree — previously every task an agent ever claimed leaked a permanent `refs/heads/{branch}` in that agent's clone. +> `da563487` Wave 2 features: A2A live view (CEO chime-in + reply budget) and prompter memory (#297) / `876e19b3` A2A switchboard + Secretary/PM task access + closed over-permission hole (#298) — no further changes to prompter.py/prompter_live.py/secretary.py beyond wave 1 above; these two commits' Secretary/PM-access work landed in `roboco/api/routes/tasks.py` (`_pm_editor_scope` / `_enforce_pm_lighter_fields`, closing the PM-role unrestricted-admin hole — out of this slice, see `docs/map/api-routes-schemas.md`) and their A2A work is entirely in `docs/map/a2a-audit-journal-permissions.md`. +> +> **"prompter-memory" (wave 2 tweak)**: `TaskService.list_recent_for_project` (task.py:6361) now excludes `TaskStatus.CANCELLED` — abandoned work is no longer precedent an intake agent's history digest treats as shipped or in-flight. Companion prompt guidance: `agents/prompts/roles/prompter.md` gains a `## Task history — don't propose what's already been done` section instructing the agent to check the ambient `## Task History` digest and the `search_past_tasks` tool before drafting — avoid duplicates (name a shipped/in-flight precedent by short id instead of quietly re-drafting it), cite precedent in a follow-up task's `notes`/`what_this_builds`, and let MegaTask sequencing be informed by (not overridden by) observed staging patterns. Informational only — the sequencing analyzer (`SequencingService`) keeps ownership of actual ordering. ## Regression Risks | Title | File:Line | Claim | Severity | |---|---|---|---| -| Worktree .venv symlink self-heal depends on a later ensure firing | roboco/services/workspace.py:534 | ensure_worktree (fresh claim) runs _link_shared_venv BEFORE install_dev_deps provisions clone_root/.venv, so the symlink is skipped on the first claim. The shared-venv optimization only self-heals if a later ensure (resume/commit via _ensure_worktree_for_commit) re-runs _link_shared_venv. If the agent commits via a path that does not re-invoke ensure and uv re-syncs a worktree-local .venv first, the lexists guard prevents replacing the real dir and the worktree is stuck with a duplicated venv. The F-fix mitigated the dangling-symlink case but did not close the already-clobbered-venv recovery (explicitly out of scope per commit msg). | medium | -| ensure_worktree reuses an existing branch ref without validating it points at base | roboco/services/workspace.py:570 | When branch_exists is True (re-claim after rollback) ensure_worktree runs `worktree add ` with no -b and no base. If the surviving branch ref was left at an unexpected commit (e.g. a prior partial rebase, or a force-pushed-and-locally-stale ref), the worktree is created at that commit, not at the intended base. The caller (GitService.create_branch_for_task) assumes a fresh branch at base; a stale ref could spawn the agent on the wrong HEAD. | medium | -| ensure_worktree_for_resume silently re-adds a worktree whose branch was force-updated remotely | roboco/services/workspace.py:591 | On resume via GitService._ensure_worktree_for_commit, ensure_worktree_for_resume re-adds the worktree from the surviving local branch ref (no fetch, no base). If the branch was force-pushed remotely while the agent was down and the local ref is stale, the agent resumes on the old commits with no warning. NOTE: the orchestrator spawn path (cfe725da/536bbb64) now calls ensure_worktree_self_heal instead, which fetches the branch ref from origin before re-attaching — the spawn path is resolved. The GitService commit path still uses ensure_worktree_for_resume without a fetch. | medium | -| PAT-leak scan cannot run if .git was wiped by a prior failed rmtree | roboco/services/workspace.py:1391 | _assert_no_pat_leak (line 1352) guards on `git_dir.exists()` and returns early if not. If a catastrophic clone left .git partially absent but the auth URL written elsewhere (e.g. into .git/config before .git/objects was created), the early return means the leak check is skipped. Combined with the F063 rmtree-on-failure this is low risk, but a rmtree that fails silently (ignore_errors=True at line 1382 only triggers on leak detection, not on the failure branches) could leave a tokenized .git/config. | low | -| dry_upgrade probe lock gap could race a future interleaved sync | roboco/services/workspace.py:1673 | The dep-update probe acquires the _meta-conventions lock only around _clone_local_into (line 1676) and releases it before _probe_lockfile_on_clone (line 1679). The commit msg argues this is safe because any concurrent _sync_read_clone completes under the lock first. This holds ONLY because _sync_read_clone is the sole other holder; if a future change adds a third concurrent mutator of the read clone that interleaves between the release and re-acquire (none today), the local clone could read a half-mutated source. Fragile invariant documented only in the commit, not enforced. | low | -| _fetch_origin_best_effort TTL cache not shared between clone root and worktree paths | roboco/services/workspace.py:1038 | _fetch_cache is keyed by str(workspace) on the instance. ensure_workspace is called with the clone-root path, but a worktree-path caller (none currently call ensure_workspace directly with a worktree path, but _resolve_clone_root exists to support worktree-aware uv env) would get a separate cache entry. Not a current bug, but a future worktree-aware ensure_workspace call could double-fetch. | low | -| _ensure_agent_owned walk excludes .venv/node_modules but agent may need to write them | roboco/services/workspace.py:67 | _PRUNE_DIRS skips .venv, node_modules, .next etc. from the chown walk for speed. The agent normally owns these (it created them) and the symlinked worktree .venv points to the clone-root .venv which IS walked (it is not under a pruned name at clone root). But a worktree-local .venv created by uv when the symlink was missing (regression risk #1) would NOT be chowned, leaving the agent unable to write into it. Edge case, low severity. | low | +| ~~1-cell map silently drops product_id and top-level project_id~~ **RESOLVED 536bbb64** | roboco/services/prompter.py:346 | ~~When _draft_cell_map returns exactly 1 entry, create_task_from_draft overwrites resolved_project_id with cell_map[0][1] and forces resolved_product_id=None.~~ Fixed: the 1-cell branch is now guarded by `resolved_project_id is None and resolved_product_id is None`; a top-level target is preserved over a redundant 1-cell map. | ~~medium~~ fixed | +| ~~Invalid project_id in a multi-cell map silently collapses the shape~~ **RESOLVED 536bbb64** | roboco/services/prompter.py:926 | ~~_draft_cell_map skips any the_work entry whose project_id fails UUID(str(pid)) (try/except continues).~~ Fixed: _draft_cell_map now raises `ValidationError` (clean 400) for a present-but-malformed project_id instead of silently continuing; the human is prompted to re-enter it. | ~~medium~~ fixed | +| Umbrella target gate is a behavior tightening that could reject previously-tolerated drafts | roboco/services/prompter.py:139 | The rewritten _validate_draft_target now hard-rejects an umbrella (is_batch_umbrella) that carries ANY target (project/product/cell-map). Before this commit an umbrella with only a project_id (no product_id) would not raise. Internal _compose_umbrella_draft never sets a project_id so the happy path is safe, but any external caller that builds a BatchPlacement(is_umbrella=True) draft with a stray project_id now gets a ValidationError instead of silent acceptance. | low | +| _clean_list semantics changed: dict-wrapped items now extracted instead of str(dict) | roboco/services/prompter.py:887 | _clean_list now delegates to coerce_str_list, which extracts text from dict-wrapped items (e.g. {'$text': ...}) instead of rendering `str(dict)`. This changes the rendered description text for any draft whose list fields contain dict-wrapped items. If coerce_str_list returns an unexpected shape for a non-string non-dict item (e.g. a list-of-lists), the rendered bullets / intended_to_touch / batch-scope counting could differ from the prior behavior. | low | +| ~~_validate_and_coerce_draft mutates the caller's draft dict in place~~ **RESOLVED 536bbb64** | roboco/services/prompter.py:207 | ~~_validate_and_coerce_draft overwrites draft_data fields in place; create_task_from_draft and confirm_live_batch callers were guarded by dict() copies but update_live_draft was not.~~ Fixed: create_task_from_draft now calls `_copy_draft(draft_data)` first (deep-copies the_work unit dicts too); the remaining concern for update_live_draft (no _validate_and_coerce_draft call) is unchanged. | ~~medium~~ partially fixed | +| ~~Multi-cell map team routing precedes product/board routing~~ **RESOLVED 536bbb64** | roboco/services/prompter.py:163 | ~~_resolve_owning_team checked multi-cell before product/board.~~ Fixed: product/board routing is now checked first (`if resolved_product_id is None:` gates the multi-cell path); a product draft with a ≥2-cell the_work map stays on the board-review path as required. The representation limit (product + cell-map not simultaneously expressible) is intentional, not a bug. | ~~low~~ fixed | ## Health -WorkspaceService is a mature, heavily-instrumented slice with strong defensive hygiene: per-(project,agent) asyncio locks, partial-clone detection + rmtree, a real .git+HEAD+objects health check, scoped + TTL-cached refresh fetches, PAT injection that is never persisted to .git/config, a belt-and-suspenders leak scan that destroys the workspace on any hit, idempotent lockfile-digest-gated dev-deps install, and F123 per-task worktrees that eliminated the coordinator-PM clobber. The F063 + F116 + F123 + F-fix wave closed real deploy-blocker races (PAT leak on half-configured clone, read-clone mid-clone race, root-clobber, dangling venv symlink). Residual risk is concentrated in the worktree venv-symlink timing (first-claim skip depends on a later ensure to self-heal), the resume path reusing a possibly-stale local branch ref without a freshness check, and the process-local (not cross-process) ensure-workspace lock. The slice diverges from CLAUDE.md's "fresh claim git reset --hard" narrative — by design, post-F123 — and that doc drift should be reconciled. Overall integrity is high; the regression risks are edge-case rather than core-path. - -# RoboCo Slice Map — `support-services` - -Slice key: `support-services` Repo root: `/Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco` Scope: `roboco/services/{agent,health,settings,toolchain,provider,llm,proactive,transcription,base,exceptions}.py`, `roboco/events/`, `roboco/seeds/`, `roboco/utils/` +The slice is coherent and well-defended. prompter_live.py remains unchanged since baseline and reads as a clean, focused singleton with correct lifecycle semantics (idempotent open, sentinel-based stream close, park-vs-close distinction). prompter.py and secretary.py both changed in wave 1 (`d1cf6ecb`): prompter.py gained the per-cell MegaTask map shape, the main_pm+code->planning coercion that closes the 2026-06-27 meltdown class, AND the prompter-memory digest builders (history digest + compact task search) — its validation is stricter and coercion is robust against LLM-emitted shapes (bare-string the_work, dict-wrapped list items, word-valued priority). secretary.py gained a genuinely new capability (the full-content `edit` directive action with claim-aware reassignment), its first change since baseline; the split between the allowlisted content fields and the status-only start/cancel/override actions is clean and the reassignment logic correctly branches on claim state. The main integrity concerns are two pre-existing silent-collapse paths in the cell-map handling: a 1-cell map silently drops product_id/top-level project_id, and a malformed project_id in a multi-cell map silently collapses the shape to single-cell — neither raises, so an LLM producing a slightly-off draft will create a mis-shaped task instead of a clean 400. The update_live_draft path skips the new _validate_and_coerce_draft guard, so re-drafts are not protected against empty-after-coercion AC. No drift from CLAUDE.md was found; the MegaTask umbrella is branchless/planning, ANNOUNCE is gated, and single-task intake is preserved. Overall the slice is healthy but the silent-collapse edges warrant a hardening pass to convert them into ValidationErrors. ## Purpose - -Cross-cutting support layer beneath the delivery services: the service-base/error hierarchy every service inherits, Fernet crypto + UUID converters, the Redis-Streams event bus + workflow trigger handlers, the static seed data that bootstraps agents, infrastructure health probes, the runtime-editable settings/feature-flag store, agent-record lookups, per-project Python interpreter resolution, provider-config CRUD + per-agent model routing, proactive knowledge injection, and raw-stream transcription buffering. None of these own the task lifecycle; they are the plumbing the lifecycle services, orchestrator, and API routes compose on top. +This slice implements RoboCo's formal-notification backbone: NotificationService is the typed notification factory (blocker, QA-ready, A2A, board-review, ack), NotificationDeliveryService handles delivery (transactional-outbox bus publish), ACK tracking, expiry sweeps, PM/CEO task-handoff notifications, and the best-effort Telegram DM bridge (`_notify_telegram`), and notification_dedup is a bounded Redis SET-NX re-fire guard for loop-prone notification types. Together they turn lifecycle events into both a durable DB record and a real-time push, with multiple dedup layers (Redis re-fire window + DB purpose-dedup) to keep agent inboxes from flooding under coordinator loops. Ack-required notifications now also carry a TTL: `_create_notification` stamps `expires_at` (`settings.notification_ack_ttl_hours`, default 48) so the previously dead-on-arrival `sweep_expired_notifications` query finally matches rows. The Telegram side has grown into a full two-way cockpit: V1 (outbound-only DMs on escalation/completion) plus V2's `TelegramInboundEngine` (`telegram_inbound.py`) — a poll loop that turns the CEO's Telegram replies/button-taps into the same CEO-gated service calls the HTTP routes make; V3 (the Mini App sign-in) is a pure HTTP route + validator, mapped in `api-routes-schemas.md` / `support-services.md` / `panel.md`; **V4** adds `TgCockpitService` (`tg_cockpit.py`, backing the Mini App's "Today" brief) and `telegram_bridge.py` (bridging `/secretary`/`/newtask` chat commands into the same live-chat runtimes the panel drives) — both are in this slice's files now. ## Files -| Path | Role | approx LOC | +| Path | Role | LOC | |---|---|---| -| `roboco/services/base.py` | `BaseService` (session-bound) + `SingletonService` + `SingletonHolder[T]` + `ServiceError` hierarchy (NotFound/Validation/Conflict/Unauthorized/ServiceUnavailable) | 226 | -| `roboco/services/exceptions.py` | LLM-provider rate-limit exception + `Retry-After` parser + retry constants | 81 | -| `roboco/services/agent.py` | Thin read-side `AgentService` over `AgentTable` (list/get by uuid/slug/raise) | 74 | -| `roboco/services/health.py` | `check_database` / `check_redis` infrastructure probes backing `/health` | 32 | -| `roboco/services/settings.py` | `SettingsService` CRUD over `system_settings` + `FEATURE_FLAGS` registry + startup overlay onto `roboco.config.settings` | 165 | -| `roboco/services/toolchain.py` | Pure resolver: target project's Python interpreter from `pyproject.toml` / `.python-version` | 117 | -| `roboco/services/provider.py` | `ProviderService` CRUD for `provider_configs` rows + Fernet-encrypted token tri-state updates | 229 | -| `roboco/services/llm.py` | `ModelRoutingService`: resolve (provider, model) per agent spawn; assignment CRUD; mode apply (anthropic/grok/ollama/self_hosted/mix); Ollama probe | 599 | -| `roboco/services/proactive.py` | `ProactiveKnowledgeService`: assemble RAG context packages on task-claim / session-start | 542 | -| `roboco/services/transcription.py` | `TranscriptionService`: buffer raw LLM stream chunks into extractable segments | 278 | -| `roboco/events/__init__.py` | Public re-exports for the event system | 41 | -| `roboco/events/bus.py` | Backward-compat shim: `EventBus = StreamEventBus`, `get_event_bus`, `init_event_bus` | 57 | -| `roboco/events/handlers.py` | Workflow trigger handlers (task status → notifications, QA result, blocker, question, auditor spawn) + `register_default_handlers` | 419 | -| `roboco/events/stream_bus.py` | `StreamEventBus`: Redis Streams durable event bus with consumer groups, ACK, pending recovery, periodic reclaim loop, dead-letter for undecodable messages | 604 | -| `roboco/seeds/__init__.py` | Re-exports seed constants | 25 | -| `roboco/seeds/initial_data.py` | Static seed data (agents) derived from `foundation` catalogs | 332 | -| `roboco/utils/__init__.py` | Re-exports crypto + converter helpers | 23 | -| `roboco/utils/converters.py` | `InvalidIdentifierError` + `require_uuid` / `to_python_uuid` / `to_python_uuid_list` + `repo_key` | 99 | -| `roboco/utils/crypto.py` | Fernet `encrypt_token` / `decrypt_token` / `is_encryption_configured` + `EncryptionError` | 111 | - -## Key Symbols - -| Name | Kind | File:Line | Responsibility | -|---|---|---|---| -| `BaseService` | class | `services/base.py:116` | Session-bound service base; holds `self.session` + `self.log` (structlog bound to `service_name`) | -| `SingletonService` | class | `services/base.py:154` | Stateless singleton base with logger only | -| `SingletonHolder[T]` | generic class | `services/base.py:184` | PEP-695 generic singleton holder (get/set/clear/is_initialized) | -| `ServiceError` | class | `services/base.py:25` | Base service exception with `message` + `details` | -| `NotFoundError` | class | `services/base.py:34` | 404-bound; carries `resource_type` / `resource_id` | -| `ConflictError` | class | `services/base.py:64` | 409-bound; carries `resource_type` | -| `ValidationError` | class | `services/base.py:51` | 400-bound; carries `field` | -| `RateLimitError` | class | `services/exceptions.py:31` | Raised after all 429 retries exhausted; carries `provider` + `retry_after` | -| `parse_retry_after_header` | func | `services/exceptions.py:65` | Numeric `Retry-After` → float seconds (HTTP-date unsupported) | -| `MAX_RATE_LIMIT_RETRIES` | const | `services/exceptions.py:20` | `= 5` | -| `AgentService` | class | `services/agent.py:19` | Read-only agent queries (list/get_by_uuid/get_by_slug/get_by_uuid_or_slug_or_raise) | -| `check_database` | func | `services/health.py:14` | Opens a DB context, `SELECT 1`, returns `(msg, ok)` | -| `check_redis` | func | `services/health.py:24` | `redis.from_url(settings.redis_url).ping()` then close | -| `SettingsService` | class | `services/settings.py:83` | KV CRUD on `system_settings` (get/get_int/get_bool/set/all); `set` validates + flushes, caller commits | -| `FEATURE_FLAGS` | tuple | `services/settings.py:46` | Panel-tunable flag registry `(key, label)`; maps to `Settings` bool attrs of same name | -| `validate_setting` | func | `services/settings.py:75` | Reject unknown keys + run per-key validator | -| `apply_persisted_feature_flags` | func | `services/settings.py:146` | Startup overlay: stored flag value → `setattr(settings, key, bool)`; returns overridden keys | -| `feature_flag_effective_values` | func | `services/settings.py:131` | Stored override else env default; backs Settings panel card | -| `SettingValidationError` | class | `services/settings.py:22` | Unknown/invalid setting on write | -| `resolve_target_python` | func | `services/toolchain.py:103` | Returns `ResolvedPython(version, source)` or None; honors `.python-version` only if it satisfies `requires-python` | -| `satisfies` | func | `services/toolchain.py:48` | PEP 440 membership test (empty specifier = any) | -| `ResolvedPython` | dataclass | `services/toolchain.py:40` | Frozen `(version, source)` result | -| `ProviderService` | class | `services/provider.py:61` | CRUD for `provider_configs`; tri-state token update; 409 on delete-with-assignments | -| `ProviderCreate` / `ProviderUpdate` | dataclasses | `services/provider.py:32,43` | Service-boundary shapes; `ProviderUpdate.auth_token` tri-state + `clear_auth_token` | -| `get_decrypted_token` | method | `services/provider.py:211` | Decrypt provider token or None; raises `EncryptionError` on bad key | -| `ModelRoutingService` | class | `services/llm.py:119` | Per-agent route resolution + assignment CRUD + mode apply | -| `AgentRoute` | dataclass | `services/llm.py:95` | Frozen resolved route `(provider_id, type, base_url, auth_token, model_name)`; None base_url/token = Anthropic default | -| `resolve_for_agent` | method | `services/llm.py:124` | Precedence ladder agent>role>global; never raises — downgrades to legacy Anthropic path | -| `probe_ollama_tags` | func | `services/llm.py:63` | `{base_url}/api/tags` probe; never raises, returns `([], error)` | -| `upsert_assignment` | method | `services/llm.py:241` | Insert-or-update by `(scope, scope_value)`; routes non-catalog names to LOCAL; auto-enables LOCAL provider | -| `apply_mode` | method | `services/llm.py:418` | Wipe role/global rows (AGENT_SLUG pins preserved) + set GLOBAL for anthropic/grok/ollama/self_hosted; per-agent map for mix | -| `derive_mode` | method | `services/llm.py:314` | Settings UI label from current assignments | -| `set_ollama_api_key` / `set_grok_api_key` | methods | `services/llm.py:340,360` | Encrypt+enable / clear+disable on the seeded provider row | -| `ProactiveKnowledgeService` | class | `services/proactive.py:90` | Builds `ContextPackage` from multiple RAG indexes on claim/session | -| `ContextPackage` | dataclass | `services/proactive.py:27` | Aggregates similar_tasks/learnings/code_patterns/standards/decisions/known_issues + summary | -| `on_task_claimed` / `on_session_started` | methods | `services/proactive.py:116,201` | Best-effort multi-index search; each source wrapped in try/except | -| `get_proactive_service` | func | `services/proactive.py:534` | Singleton holder; lazy-inits with `OptimalService` | -| `TranscriptionService` | class | `services/transcription.py:28` | Per-(agent,session) `StreamBuffer` map; periodic flush task; callback registration | -| `process_chunk` | method | `services/transcription.py:120` | Append chunk, return buffer if ready-for-extraction else None | -| `_periodic_flush` | method | `services/transcription.py:225` | Background loop: sleep `flush_interval_seconds`, yield ready buffers to callbacks | -| `StreamEventBus` | class | `events/stream_bus.py:35` | Redis Streams bus: `xadd` trim, `xreadgroup` block=5000, ACK-on-success, `xclaim` recovery, periodic `_reclaim_loop`, dead-letter for undecodable messages | -| `DEAD_LETTER_STREAM` | class attr | `events/stream_bus.py:51` | `"roboco:stream:dead-letter"` — undecodable messages are parked here before ACK for operator inspection | -| `publish` / `publish_task_event` | methods | `events/stream_bus.py:152,191` | `xadd` to category-grouped stream, returns message id | -| `recover_pending` | method | `events/stream_bus.py:534` | `xpending_range` + `xclaim` idle≥60s messages; called at startup and periodically by `_reclaim_loop` | -| `_reclaim_loop` | method | `events/stream_bus.py:253` | Background task spawned alongside `_listen_loop`; re-runs `recover_pending` every 60s so runtime handler failures are retried without waiting for a restart | -| `_handle_message` | method | `events/stream_bus.py:434` | Decode (poison-pill: dead-letter+ACK on `Event.from_json` failure) → dispatch → ACK iff all handlers succeeded (else stays pending for reclaim) | -| `_dead_letter` | method | `events/stream_bus.py:403` | Best-effort write to `DEAD_LETTER_STREAM`; never blocks ACK on publish failure | -| `get_stream_event_bus` / `init_stream_event_bus` | funcs | `events/stream_bus.py:577,584` | Singleton + connect + optional startup pending recovery | -| `handle_task_status_change` | func | `events/handlers.py:141` | Routes task.* events to PM/QA/Documenter/developer notifications | -| `handle_auditor_spawn` | func | `events/handlers.py:332` | One-shot auditor spawn on blocked/cancelled/awaiting_ceo_approval; failures swallowed | -| `register_default_handlers` | func | `events/handlers.py:372` | Subscribes all default handlers on the bus | -| `set_event_context` / `get_event_context` | funcs | `events/handlers.py:26,37` | DI of `notification_service` + `orchestrator` into module-global `_context` | -| `DEFAULT_AGENTS` | const | `seeds/initial_data.py:156` | Composed from `foundation.AGENTS` + presentation names; system sentinel appended literally (team=None) | -| `AGENT_UUIDS` / `CEO_AGENT_ID` | consts | `seeds/initial_data.py:83,173` | String-keyed UUID map for legacy consumers | -| `encrypt_token` / `decrypt_token` | funcs | `utils/crypto.py:40,67` | Fernet symmetric encrypt/decrypt; `EncryptionError` on empty/bad key | -| `is_encryption_configured` | func | `utils/crypto.py:102` | True iff `settings.encryption_key` yields a valid Fernet | -| `InvalidIdentifierError` | class | `utils/converters.py:11` | `ValueError` subclass raised by `require_uuid` on None or unparseable input; typed so callers can distinguish a bad identifier instead of broad-catching (#25) | -| `require_uuid` | func | `utils/converters.py:21` | Coerce to `UUID`, raise `InvalidIdentifierError` (a `ValueError` subclass) on None or bad input | -| `repo_key` | func | `utils/converters.py:47` | Normalize a git URL to a case/`.git`-suffix/trailing-slash insensitive key for ci_watch/dep_update dedupe (#1267) | -| `to_python_uuid` / `to_python_uuid_list` | funcs | `utils/converters.py:61,81` | None-safe SQLAlchemy UUID coercion | +| roboco/services/notification.py | Typed notification factory (blocker/QA/docs/handoff/A2A/board-review/ack) with slug→UUID recipient resolution, DB purpose-dedup + Redis re-fire guard, owns its own DB context and commit | 943 | +| roboco/services/notification_dedup.py | Bounded Redis SET-NX re-fire guard for loop-prone notification types (TASK_ASSIGNMENT/REVIEW_REQUEST/DOCUMENTATION_REQUEST/BROADCAST); 60s TTL, fail-open | 91 | +| roboco/services/notification_delivery.py | Delivery (transactional-outbox deferred bus publish), ACK/read tracking, expiry sweep, PM/CEO task-handoff notifications (notify_pm_of_block, escalate_and_notify, etc.), API-facing list/CRUD, `_notify_telegram` best-effort CEO DM fan-out (V2: `actionable=True` on escalation attaches an Approve/Reject/Open inline keyboard) | 1305 | +| roboco/services/telegram_client.py | Bot API client ABC + `NullTelegramClient` (unconfigured, never egresses) + `LiveTelegramClient`: `send_message` (reply_markup/reply_to_message_id), V2 additions `get_updates` (long-poll), `answer_callback_query`, `edit_message_reply_markup`, `edit_message_text` | 247 | +| roboco/services/telegram_credentials.py | Singleton Fernet-encrypted `bot_token`/`chat_id` CRUD (mirrors `x_credentials.py`); decrypts server-side only, API returns `has_credentials` only | 109 | +| roboco/services/telegram_inbound.py | V2: `TelegramInboundEngine` — getUpdates poll cycle (offset persisted as `telegram_last_update_id` in system_settings), chat-id AND sender-id authorization, `BOT_COMMANDS` registry driving both `/help` and a once-per-process `setMyCommands` sync, `/status`/`/queue`/`/task`/`/agents`/`/usage`/`/blocked`/`/secretary`/`/newtask`/`/end` command router, `apv|rej::` callback codec, force_reply reject/approve-notes state machine (in-memory `_PENDING_REPLIES`, TTL), per-kind dispatch to the SAME service methods the CEO-gated HTTP routes call (task/release/xpost/video/roadmap), `via=telegram` audit rows | 1295 | +| roboco/services/tg_cockpit.py | V4: `TgCockpitService` — DB-only, one-round-trip aggregate for the Mini App home screen (`today()`) and the bot's `/agents` command (`fleet()`); no live GitHub calls, no orchestrator singleton | 217 | +| roboco/services/telegram_bridge.py | V4: bridges `/secretary`/`/newtask` Telegram chat into the SAME in-process live-chat runtimes the panel drives — a per-chat consumer task drains `PrompterLiveRegistry.stream`, forwards `turn_end`/`draft`/`batch`/`error` events as Telegram messages, and routes a `draft` event's Send-to-Board confirm through `PrompterService.confirm_live_draft` + registry `park` | 291 | ## Data Flow - -**Spawn routing.** The orchestrator (`runtime/orchestrator.py:3294`) opens a DB session, calls `get_model_routing_service(db).resolve_for_agent(agent_slug)`. `ModelRoutingService` walks `model_assignments` (AGENT_SLUG → ROLE → GLOBAL), joins the `provider_configs` row, decrypts any token via `ProviderService.get_decrypted_token` (Fernet from `utils/crypto`), probes LOCAL providers via `probe_ollama_tags`, and returns an `AgentRoute`. On any failure (decrypt, unreachable server, missing row) it falls back to `_legacy_route` (role → `MODEL_MAP` short name, Anthropic, mounted `~/.claude`). The orchestrator injects `ANTHROPIC_*`/base_url/auth env into the container only when the route is non-Anthropic. - -**Settings/flags.** At FastAPI lifespan (`api/app.py:117`), `apply_persisted_feature_flags(db)` reads each `FEATURE_FLAGS` key from `system_settings` and `setattr`s the live `roboco.config.settings` singleton so the rest of the app reads panel choices; an unset key keeps the env default. The Settings panel reads effective values via `feature_flag_effective_values` and writes via `SettingsService.set` (validates → upsert → flush; route commits). - -**Events.** `bootstrap.py:92` calls `init_event_bus()` → `init_stream_event_bus` → `connect()` + `recover_pending()` (reclaims idle ≥60s messages from crashed consumers via `xclaim`). `register_default_handlers` subscribes task/session/handoff/QA/blocker/question/auditor handlers. Publishers call `bus.publish(Event)` → `xadd` to `roboco:stream:{category}` (trimmed to 10000). `start_listening` spawns both `_listen_loop` and `_reclaim_loop`; the reclaim loop re-runs `recover_pending` every 60s so a runtime handler failure is retried without waiting for a restart. `_listen_loop` blocks on `xreadgroup` (count=10, block=5000); `_handle_message` first tries `Event.from_json` in its own try/except — an undecodable payload (unknown `EventType`, bad UUID/timestamp, malformed JSON) is dead-lettered to `DEAD_LETTER_STREAM` then ACKed so a poison pill never wedges the stream. Successfully decoded events are dispatched via `asyncio.gather` with a per-(event.id, handler) SET-NX idempotency guard (`_run_handler_guarded`), and the message is ACKed only if every handler succeeded — failed handlers leave the message pending for the reclaim loop. `_run_handler_guarded` catches `BaseException` (including `asyncio.CancelledError`) so a mid-flight cancellation clears the idempotency marker and allows replay. Handlers use the injected `_context` (notification_service + orchestrator). - -**Proactive injection.** `TaskService.claim_task` (`services/task.py:2548`) lazily calls `get_proactive_service()`, which singleton-inits with `OptimalService`. `on_task_claimed` runs five best-effort RAG searches (similar tasks, learnings, standards, decisions, known issues), each in its own try/except, builds a `ContextPackage` + summary. The `code_patterns` field is retained in `ContextPackage` for API/schema back-compat but is always empty — `_find_code_patterns` was removed. `api/routes/optimal.py:1255` exposes `get_context_for_task`/`get_context_for_session` which own DB lookups so routes don't query directly. - -**Transcription.** `api/app.py:124` constructs `TranscriptionService()` at lifespan; `ExtractionService` (`services/extraction.py`) composes it. `process_chunk` appends to the per-(agent,connection) `StreamBuffer`; when `is_ready_for_extraction` (min chars / idle threshold / max chars) it returns the buffer for extraction. A `_periodic_flush` background task wakes every `flush_interval_seconds` and pushes ready buffers to registered callbacks. - -**Seeds.** `db/seed.py` iterates `DEFAULT_AGENTS` to populate the DB on bootstrap; `foundation/_validate_lifecycle.py:237` validates against `DEFAULT_AGENTS`. It derives from `foundation.AGENTS` so adding an agent is a single foundation edit. - -**Toolchain.** `WorkspaceService` (`services/workspace.py:1308`) calls `resolve_target_python(workspace)` to pick the interpreter for `uv --python` when provisioning a target project's clone. - -**Health.** `api/routes/health.py:41` calls `check_database()` + `check_redis()` for `/health`. +Two create-and-deliver paths exist. (A) NotificationService._create_notification (notification.py) opens its OWN get_db_context, resolves sender + recipients to UUIDs via _resolve_agent_uuid, runs the Redis re-fire guard (all_recipients_recently_notified), then DB purpose-dedup (ack-required types only, same sender+type+task+overlapping recipients not yet acked), builds NotificationTable with requires_ack from ACK_REQUIRED_BY_TYPE, flushes, calls NotificationDeliveryService.deliver (which defers NOTIFICATION_SENT bus events to after_commit), and finally commits — the commit triggers the deferred bus drain. (B) NotificationDeliveryService._persist_and_deliver (notification_delivery.py) is used by the task-handoff helpers (notify_pm_of_block, escalate_and_notify, etc.): it runs inside the CALLER's open transaction, applies only the Redis re-fire guard (no DB purpose-dedup), adds+flushes+delivers, and leaves the commit to the caller (api/routes/tasks.py). Sweeper loops in the orchestrator call sweep_expired_notifications periodically. Real-time push: deliver defers per-recipient NOTIFICATION_SENT events; the after_commit listener schedules _drain_pending_publishes which publishes to the StreamEventBus; websocket_bridge forwards to /ws/notifications/{id} sockets. ACKs flow acknowledge → acked_by/read_by mutation + NOTIFICATION_ACKED event. ## Mermaid - ```mermaid -graph TD - subgraph "Model routing (spawn time)" - ORC["orchestrator.resolve_model_route"] --> MRS["ModelRoutingService.resolve_for_agent"] - MRS -->|"agent>role>global"| FIND["_find_assignment"] - FIND --> ROW["model_assignments row"] - ROW --> PROV["ProviderConfigTable"] - PROV -->|"token?"| PS["ProviderService.get_decrypted_token"] - PS --> CRYPTO["utils/crypto Fernet"] - PROV -->|"LOCAL?"| PROBE["probe_ollama_tags /api/tags"] - PROBE -->|unreachable| LEG["_legacy_route ANTHROPIC"] - PS -->|decrypt fail| LEG - MRS -->|ok| ROUTE["AgentRoute"] - LEG --> ROUTE - end -``` - -```mermaid -sequenceDiagram - participant B as bootstrap.py - participant S as StreamEventBus - participant R as Redis Streams - participant H as Handlers - participant N as NotificationService - B->>S: init_event_bus(consumer_name) - S->>R: connect + xpending/xclaim (recover_pending) - S->>S: register_default_handlers - loop listen - S->>R: xreadgroup(block=5000, count=10) - R-->>S: messages - S->>H: _dispatch_event (gather) - H->>N: send_*_notification - H-->>S: ok/exception - alt all succeeded - S->>R: xack - else any failed - S->>S: leave pending (reclaim later) +flowchart TD + subgraph Notification + TYPED["send_blocker / send_qa_ready / send_a2a / send_ack ..."] --> CN["_create_notification"] + CN --> RES["_resolve_agent_uuid (sender+recipients)"] + CN --> RD["all_recipients_recently_notified
Redis SET-NX 60s"] + CN --> DD["DB purpose-dedup
ack-required only"] + CN --> NT["NotificationTable (requires_ack=ACK_REQUIRED_BY_TYPE)"] + CN --> DLV["deliver"] + HANDOFF["notify_pm_of_block / escalate_and_notify / notify_ceo_of_escalation"] --> PAD["_persist_and_deliver"] + PAD --> RD + PAD --> NT2["NotificationTable"] + PAD --> DLV end - end -``` -```mermaid -graph LR - subgraph "Settings overlay" - PANEL["Settings panel"] -->|POST| API["api/routes/settings"] - API --> SS["SettingsService.set"] - SS --> VALID["validate_setting"] - SS --> DB[("system_settings")] - LIFESPAN["api/app.py lifespan"] --> APPLY["apply_persisted_feature_flags"] - APPLY --> DB - APPLY -->|"setattr(key,bool)"| CFG["roboco.config.settings singleton"] - CFG --> CONSUMERS["all flag-gated code"] - end + subgraph Delivery + DLV -->|"in-tx"| DA["delivered_at = now"] + DLV --> DEF["defer_bus_publish
per-recipient NOTIFICATION_SENT"] + DEF -.->|"after_commit"| DRN["_drain_pending_publishes"] + DEF -.->|"after_rollback"| DIS["_discard_pending_publishes (no phantom)"] + DRN --> BUS["StreamEventBus"] + BUS --> WS["websocket_bridge → /ws/notifications/{id}"] + ACK["acknowledge"] --> AB["acked_by / read_by / acked_at"] + ACK --> BUS2["NOTIFICATION_ACKED event"] + end + + RD --> REDIS[("Redis SET-NX")] + DD --> PG[("PostgreSQL notifications")] + NT --> PG + DRN --> REDIS ``` ## Logical Tree - ``` -support-services -├── services/ -│ ├── base.py # BaseService / SingletonService / SingletonHolder[T] / ServiceError* -│ ├── exceptions.py # RateLimitError + Retry-After parser -│ ├── agent.py # AgentService (read-only) -│ ├── health.py # check_database / check_redis -│ ├── settings.py # SettingsService + FEATURE_FLAGS + startup overlay -│ ├── toolchain.py # resolve_target_python (pure) -│ ├── provider.py # ProviderService (provider_configs CRUD + Fernet tokens) -│ ├── llm.py # ModelRoutingService (spawn routing + modes + Ollama probe) -│ ├── proactive.py # ProactiveKnowledgeService (RAG context packages) -│ └── transcription.py # TranscriptionService (stream buffering) -├── events/ -│ ├── __init__.py # public re-exports -│ ├── bus.py # EventBus = StreamEventBus (compat shim) -│ ├── handlers.py # workflow trigger handlers + register_default_handlers -│ └── stream_bus.py # Redis Streams durable bus (xadd/xreadgroup/xack/xclaim) -├── seeds/ -│ ├── __init__.py -│ └── initial_data.py # DEFAULT_AGENTS from foundation -└── utils/ - ├── __init__.py - ├── converters.py # UUID coercion - └── crypto.py # Fernet encrypt/decrypt +notification +├── notification.py (NotificationService) +│ ├── _resolve_agent_uuid (slug/UUID → UUID; 'system' seed) +│ ├── send_blocker / send_stuck_agent / send_qa_ready / send_docs_ready +│ ├── send_handoff / send_qa_failed / send_board_review_complete +│ ├── send_external_pr_reviewed / send_ack / send_a2a (tristate priority; `requires_ack` kwarg overrides the A2A_REQUEST type default of False — only the A2A CEO-DM wake path sets it True) +│ ├── _notification_type_label / _resolve_recipients +│ └── _create_notification (own DB context, re-fire guard, DB dedup, requires_ack, commit) +├── notification_dedup.py +│ ├── _LOOP_PRONE_TYPES (4 types) +│ ├── _DEDUP_TTL_SECONDS (60) +│ ├── _key (type:from:recipient:task) +│ └── all_recipients_recently_notified (per-recipient SET-NX, fail-open) +└── notification_delivery.py (NotificationDeliveryService) + ├── Transactional outbox (F107) + │ ├── defer_bus_publish (enqueue + register listeners) + │ ├── _schedule_pending_publishes (after_commit → loop.create_task) + │ ├── _drain_pending_publishes (best-effort publish) + │ └── _discard_pending_publishes (after_rollback) + ├── EscalationError / EscalationOutcome / BlockerDetails + ├── deliver (delivered_at in-tx, defer per-recipient events) + ├── get_notification / _notification_is_fully_acked / _log_expired_notification + ├── sweep_expired_notifications (log stale unacked) + ├── get_pending_for_agent / get_unacknowledged_for_agent / get_notification_count + ├── acknowledge / mark_read / bulk_acknowledge / get_ack_status / get_delivery_summary + ├── Task-handoff / audit-bridge notifications + │ ├── notify_pm_of_block / notify_pm_of_docs_complete / notify_pm_of_review_submission + │ ├── notify_assignee_of_unblock / notify_assignee_of_ceo_rejection + │ ├── escalate_and_notify (EscalationError/EscalationOutcome) + │ ├── notify_ceo_of_escalation + │ ├── notify_auditor_of_rework (ALERT to the auditor agent on needs_revision) + │ └── _persist_and_deliver (re-fire guard only, caller commits) + ├── Recipient helpers: _resolve_team_pm / _resolve_pm_for_agent_or_team / _get_agent_by_id/slug / _get_ceo_agent / _get_auditor_agent + ├── API-facing: list_system_notifications / list_for_agent / get_for_recipient_and_mark_read / acknowledge_for_recipient / mark_read_for_recipient + └── _notify_telegram (best-effort CEO DM; actionable=True on escalation attaches build_action_keyboard) +telegram_client.py (TelegramClient ABC / NullTelegramClient / LiveTelegramClient) +├── V1: send_message (reply_markup, reply_to_message_id) +└── V2: get_updates (long-poll) / answer_callback_query / edit_message_reply_markup / edit_message_text +telegram_inbound.py (TelegramInboundEngine, V2) +├── run_cycle (getUpdates offset cursor, dispatch each update, advance+persist offset) +├── _handle_message (chat+sender auth, force_reply pending-consume, command dispatch) +├── _dispatch_command (/status /queue /task /help) +├── _handle_callback (chat+sender auth, parse_callback, needs_reply branch → _prompt_for_reply, else _dispatch_approve) +├── _dispatch_approve / _dispatch_reject (per-kind handler dict: task/release/xpost/video/roadmap) +│ └── each handler calls the SAME service method the CEO-gated HTTP route calls; _mark_audit stamps a via=telegram AuditLogTable row +└── _finish_action (clears the buttoned message's keyboard, stamps the outcome) ``` ## Dependencies - -**Internal (downstream):** -- `roboco.config.settings` (encryption key, redis_url, feature-flag defaults) — `crypto`, `stream_bus`, `health`, `settings` -- `roboco.db.tables` (`AgentTable`, `ProviderConfigTable`, `ModelAssignmentTable`, `SystemSettingTable`, `TaskTable`) — `agent`, `provider`, `llm`, `settings`, `proactive` -- `roboco.db.base.get_db_context` — `health`, `proactive` -- `roboco.models.base` (`AgentRole`, `Team`, `ModelProvider`, `AssignmentScope`) — `agent`, `provider`, `llm` -- `roboco.models.events` (`Event`, `EventType`, `EventContext`, protocols) — `events/*` -- `roboco.models.llm_catalog` (`MODEL_CATALOG_BY_NAME`, `OLLAMA_DEFAULT_MODEL`) — `llm` -- `roboco.models.runtime` (`MODEL_MAP`, `ROLE_MODEL_MAP`) — `llm` -- `roboco.models.optimal` (`IndexType`, `QueryContext`, `SearchResult`) — `proactive` -- `roboco.models.message.RawStream`, `roboco.models.transcription.*` — `transcription` -- `roboco.agents_config.get_agent_role` — `llm` -- `roboco.foundation.identity` — `seeds` -- `roboco.services.optimal.get_optimal_service` — `proactive` (lazy) -- `roboco.logging.get_logger` — `crypto` - -**External:** -- `sqlalchemy` / `sqlalchemy.ext.asyncio` — ORM sessions -- `redis.asyncio` — streams bus, health probe -- `cryptography.fernet` — token encryption -- `httpx` — Ollama probe -- `structlog` — logging everywhere -- `packaging` (`Version`, `SpecifierSet`) — `toolchain` -- `tomllib` — `toolchain` pyproject parse +- Internal: roboco.config.settings (redis_url), roboco.db.tables (NotificationTable, AgentTable, TaskTable), roboco.db.base.get_db_context, roboco.events (Event, EventType, get_event_bus), roboco.foundation.policy.communications.ACK_REQUIRED_BY_TYPE, roboco.models.base (NotificationPriority, NotificationType, AgentRole), roboco.models.notification.CreateNotificationParams, roboco.services.base (BaseService, ConflictError, NotFoundError), roboco.services.permissions.has_privileged_access, roboco.services.repositories.get_agent_slug, roboco.agents_config (get_escalation_target, get_pm_for_agent, get_pm_for_team), roboco.utils.converters (require_uuid, to_python_uuid) +- External: sqlalchemy (select, and_, or_, func, event, joinedload, selectinload, with_for_update, IntegrityError, AsyncSession), redis.asyncio (from_url, set NX EX, aclose), asyncio (create_task, get_running_loop), structlog, datetime (UTC, datetime, timedelta), uuid.UUID, dataclasses +- telegram_inbound.py additionally imports: roboco.services.release_proposal (dispatch_approve, get_release_proposal_service, TaskAlreadyCompletedError), roboco.services.roadmap_service, roboco.services.task.get_task_service, roboco.services.telegram_credentials, roboco.services.tiktok_client/tiktok_credentials, roboco.services.video_post_service (VideoPostService, TaskAlreadyCompletedError, VideoCaptionTooLongError), roboco.services.x_credentials, roboco.services.x_post_service (TaskAlreadyCompletedError, XPostBodyTooLongError), roboco.services.x_video_client, roboco.foundation.policy.content.validators.reject_trivial, roboco.seeds.initial_data.AGENT_UUIDS ## Entry Points -| Symbol | Invoked from | Trigger | +| Name | File | Trigger | |---|---|---| -| `apply_persisted_feature_flags` | `api/app.py:117` | FastAPI lifespan (after DB ready) | -| `get_settings_service` | `api/routes/settings.py`, `runtime/orchestrator.py:5910` | `GET/POST /api/settings`; orchestrator transcript-retention read | -| `get_model_routing_service().resolve_for_agent` | `runtime/orchestrator.py:3301` | Each agent spawn | -| `ModelRoutingService.*` assignment/mode ops | `api/routes/provider.py` | `GET/POST/DELETE /api/providers/*` | -| `ProviderService.*` | `api/routes/provider.py` | provider CRUD routes | -| `get_agent_service` | `api/routes/agents.py`, `services/task.py`, `services/pitch.py` | `/api/agents`; task/pitch main-pm lookup | -| `check_database` / `check_redis` | `api/routes/health.py:41` | `GET /health` | -| `resolve_target_python` | `services/workspace.py:1308` | Workspace provisioning (clone/claim) | -| `init_event_bus` / `register_default_handlers` / `set_event_context` | `bootstrap.py:92-96` | App bootstrap | -| `bus.publish` / `publish_task_event` | services, orchestrator, websocket_bridge | Status transitions, live events | -| `get_proactive_service` | `services/task.py:2548`, `api/routes/optimal.py:1257` | claim_task, `/api/optimal/context` | -| `TranscriptionService` | `api/app.py:124`, `services/extraction.py` | Lifespan construct; extraction pipeline | -| `DEFAULT_AGENTS` | `db/seed.py`, `foundation/_validate_lifecycle.py` | DB bootstrap + lifecycle validation | -| `encrypt_token` / `decrypt_token` | `provider`, `llm` (via ProviderService), ProjectService | Token persist / read | +| NotificationService.send_*_notification | roboco/services/notification.py | TaskService / orchestrator lifecycle transitions (blocker, qa-ready, docs, a2a, board-review) | +| NotificationService.send_ack_notification | roboco/services/notification.py | gateway `notify` content verb (PM/Board only) | +| NotificationDeliveryService.notify_pm_of_block / escalate_and_notify / notify_ceo_of_escalation / notify_auditor_of_rework | roboco/services/notification_delivery.py | api/routes/tasks.py i_am_blocked / escalate / ceo-approval routes; TaskService._alert_auditor_of_rework at QA-fail / rework chokepoints | +| NotificationDeliveryService.acknowledge / list_for_agent / get_for_recipient_and_mark_read | roboco/services/notification_delivery.py | api/routes/notifications.py ACK + list endpoints | +| sweep_expired_notifications | roboco/services/notification_delivery.py | orchestrator periodic loop (orchestrator.py:5780) | +| TelegramInboundEngine.run_cycle | roboco/services/telegram_inbound.py | orchestrator `_telegram_poll_loop` (default off, `telegram_enabled` AND `telegram_inbound_enabled`) | ## Config Flags +- settings.redis_url — Redis URL used by notification_dedup for the SET-NX re-fire guard (derived from ROBOCO_REDIS_HOST/_PORT) +- `ROBOCO_NOTIFICATION_ACK_TTL_HOURS` (default `48`, `ge=0`) — hours until an ack-required notification's `expires_at` is stamped at creation (`_create_notification`, config.py:270); consumed by `NotificationDeliveryService.sweep_expired_notifications`'s re-escalation. `0` disables stamping entirely (`expires_at` stays `NULL`, legacy behavior — never expires). Only ack-required notifications (per `ACK_REQUIRED_BY_TYPE`) get a deadline; informational ones never do regardless of this setting. +- `telegram_enabled` (default off) — V1 master switch; `_notify_telegram` no-ops without it AND stored credentials. +- `telegram_inbound_enabled` (default off, sub-switch on top of `telegram_enabled`) — V2: arms `TelegramInboundEngine.run_cycle` (the poll loop) and makes escalation DMs carry an actionable keyboard; with it off the bot only sends, never listens, and any inline button on an old message is inert. +- `telegram_poll_interval_seconds` (5.0) / `telegram_poll_timeout_seconds` (25, Bot API long-poll `timeout`) / `telegram_max_updates_per_cycle` (50) / `telegram_pending_reply_ttl_seconds` (300) — V2 poll-loop tuning. -Panel-tunable flags defined in `services/settings.py:46` `FEATURE_FLAGS` (stored override → `roboco.config.settings.` at startup; env default when unset): - -| Key | Label | Env counterpart | -|---|---|---| -| `external_pr_enabled` | External-PR review | `ROBOCO_EXTERNAL_PR_ENABLED` | -| `internal_pr_enabled` | Internal-PR safety reviewer | `ROBOCO_INTERNAL_PR_ENABLED` | -| `research_enabled` | Web research (Board + PM) | `ROBOCO_RESEARCH_ENABLED` | -| `strategy_engine_enabled` | Strategy engine | `ROBOCO_STRATEGY_ENGINE_ENABLED` | -| `self_heal_enabled` | Self-healing (detect + notify) | `ROBOCO_SELF_HEAL_ENABLED` | -| `self_heal_originate_enabled` | Self-healing — open fix tasks | `ROBOCO_SELF_HEAL_ORIGINATE_ENABLED` | -| `provisioning_enabled` | Pitch auto-provisioning | `ROBOCO_PROVISIONING_ENABLED` | -| `toolchain_match_enabled` | Agent runtime toolchain matching | `ROBOCO_TOOLCHAIN_MATCH_ENABLED` | -| `conventions_enabled` | Architectural conventions standard | `ROBOCO_CONVENTIONS_ENABLED` | -| `rag_auto_update_enabled` | RAG auto-update | `ROBOCO_RAG_AUTO_UPDATE_ENABLED` | -| `transcript_prune_enabled` | Transcript pruning | `ROBOCO_TRANSCRIPT_PRUNE_ENABLED` | -| `gateway_health_enabled` | Gateway-health recovery | `ROBOCO_GATEWAY_HEALTH_ENABLED` | -| `ci_watch_enabled` | Multi-repo CI-watch | `ROBOCO_CI_WATCH_ENABLED` | -| `dep_update_enabled` | Dependency-update bot | `ROBOCO_DEP_UPDATE_ENABLED` | -| `release_manager_enabled` | Gated release manager | `ROBOCO_RELEASE_MANAGER_ENABLED` | -| `docs_sync_enabled` | Docs-divergence sync (release → docs-update task) | `ROBOCO_DOCS_SYNC_ENABLED` | -| `org_memory_enabled` | Organizational memory loop | `ROBOCO_ORG_MEMORY_ENABLED` | -| `sandbox_db_enabled` | Sandboxed per-agent test DB/Redis/Mongo (engine registry) | `ROBOCO_SANDBOX_DB_ENABLED` | -| `x_engine_enabled` | X (Twitter) engine | `ROBOCO_X_ENGINE_ENABLED` | -| `roadmap_engine_enabled` | Board roadmap engine | `ROBOCO_ROADMAP_ENGINE_ENABLED` | - -Cloud auth (`ROBOCO_CLOUD_AUTH_ENABLED`) and DB network isolation (`ROBOCO_DB_NETWORK_ISOLATED`) are deliberately **not** in `FEATURE_FLAGS` — both are compose/env-coupled (cookie/TLS posture and the `networks:` topology respectively) and unsafe for a runtime toggle to flip mid-session; they stay pure env vars, not panel-tunable settings. - -Other settings read here: `transcript_retention_days` (int, ≥1; read by orchestrator at `runtime/orchestrator.py:5910`). Non-flag config consumed: `settings.redis_url` (`health`, `stream_bus`), `settings.encryption_key` (`crypto`). ## Gotchas +- Two notification create paths with DIFFERENT dedup strength: NotificationService._create_notification runs BOTH the Redis re-fire guard AND the DB purpose-dedup; NotificationDeliveryService._persist_and_deliver (task-handoff helpers) runs ONLY the Redis re-fire guard and explicitly skips DB purpose-dedup. A reworded BLOCKER_ESCALATION from the handoff path within 60s is suppressed by Redis, but beyond 60s a duplicate can be re-created since there is no DB dedup on that path. +- notification_dedup fail-open: a Redis error returns False (never suppress) — correct for not dropping notifications, but a sustained Redis outage re-opens the per-tick re-fire storm the guard was added to stop. +- notification_dedup.all_recipients_recently_notified has a side effect: it SET-NX-marks recipients NOT yet notified, so the FIRST call for a fresh recipient returns False (delivers) but acquires the key; a concurrent second call within 60s for the same recipient then returns True (suppresses). The marking happens even on the call that decides to deliver — so a suppressed 'all already held' verdict requires every recipient to have been marked by a prior call. Partial-fresh mixed-recipient calls deliver and mark the fresh ones. +- NotificationService._create_notification opens its OWN get_db_context and commits (line 568), while NotificationDeliveryService._persist_and_deliver operates in the CALLER's transaction and does NOT commit. Mixing the two in one outer transaction would double-commit / cross-session. +- requires_ack is set from ACK_REQUIRED_BY_TYPE (notification.py L555) rather than the column default True; MENTION/KNOWLEDGE_SHARE/BROADCAST etc. are False. `CreateNotificationParams.requires_ack` (default None) wins over the type default when a caller sets it — today only `send_a2a_notification`'s `requires_ack` kwarg (default False, `A2AService`'s CEO-DM wake path passes True) threads through to it; every other typed `send_*` helper leaves it unset and gets the type-default behavior unchanged. +- DB purpose-dedup query uses NotificationTable.to_agents.overlap(to_agents_uuids) AND ~acked_by.contains(to_agents_uuids) — overlap matches ANY recipient; a notification to [A,B] with A acked but B not is NOT suppressed for a new send to [A,B] because acked_by does not contain [A,B] (contains is element-wise). The dedup is per-(sender,type,task) not per-recipient, so a third recipient C added on resend goes through. +- defer_bus_publish registers after_commit/after_rollback listeners keyed on session.info[_DRAIN_REGISTERED_KEY]; listeners are bound to sync_session and accumulate only once per AsyncSession instance. A session reused across multiple commit cycles will re-register only once (guard), but the pending queue is popped each commit — if a second deliver happens after the first commit in the same session, the listeners are already registered and the new events append and fire on the next commit. +- acknowledge publishes NOTIFICATION_ACKED directly to the bus (NOT deferred via after_commit) — unlike deliver. An ACK that is rolled back after publish could emit a phantom ACK event. The ACK path does not use the transactional outbox. +- list_system_notifications filters pending_ack_only POST-fetch because 'not fully acked' is not SQL-friendly on PostgreSQL array columns. For pending_ack_only=True the SQL `limit` is NOT applied — applying it before the Python filter let a window of newer fully-acked rows mask older unacked ones the operator still needs to act on (correctness bug fixed in 115061f3); the full ack-required set is fetched ordered newest-first, Python-filtered to unacked, then sliced to `limit`. The non-pending branch retains the SQL limit. +- get_notification_count loads ALL notifications for an agent into memory (no SQL count) to compute total/unread/pending_ack — O(n) per call, no pagination. +- `TelegramInboundEngine._PENDING_REPLIES` (a force_reply prompt awaiting the CEO's free-text reply) is a per-process, in-memory dict keyed by `(chat_id, prompt_message_id)` — not durable. An orchestrator restart drops any in-flight prompt; the CEO just taps the button again. TTL-swept both lazily (on the next prompt) and on expiry at consume-time. +- `_authorized_chat` (chat id must equal the stored credentials' chat id) is the ONLY identity check a Telegram update carries — there is no agent/session token — so it stands in for every CEO-gated route's `require_ceo_role`. `_authorized_sender` (added in the same wave that added `_authorized_chat`'s callers) is defense-in-depth on top of it: when the update carries a `from` user, its id must ALSO equal the chat id (the supported deployment is a private 1:1 chat); a present-but-mismatched sender is refused, an absent one keeps chat-id-only behavior. +- The getUpdates offset cursor reuses the existing `system_settings` KV store (`telegram_last_update_id`, validated as a non-negative int) rather than a new table/migration — a restart resumes from the last-committed offset instead of replaying processed updates. +- `_dispatch_approve`'s `_approve_release` handler must pre-check the proposal's terminal state itself before calling `dispatch_approve` — that function fires the ~40min release execute as a background task and returns immediately with nothing to inspect, so a stale Approve on an already-rejected/published proposal would otherwise report a false "dispatched" success while the service's own guard silently no-ops it. -- **`resolve_for_agent` never raises for a normal agent** (`llm.py:124`) — decrypt failures, unreachable LOCAL servers, and missing agents all downgrade to the legacy Anthropic path. A misconfigured provider therefore silently spawns against Anthropic instead of erroring; check orchestrator logs for "falling back to legacy path" / "Self-hosted server unreachable". -- **Mix-mode self-hosted models auto-enable the LOCAL provider** (`llm.py:285`) — `upsert_assignment` flips the LOCAL provider row to `enabled=True` whenever an assignment resolves to LOCAL. A prior observation (47392) flagged that this enabling only happens on upsert, not on a bare GLOBAL assignment path — verify the LOCAL row is enabled before relying on self-hosted routing. -- **`probe_ollama_tags` leaks raw exception text** into the returned error string (`llm.py:92`, observation 47334) — the generic `except Exception` branch puts `str(exc)` in the user-facing message. Minor info-disclosure surface. -- **`SingletonHolder[T]` uses PEP 695 generic syntax** (`base.py:184`) — requires Python 3.13+. CLAUDE.md pins 3.13 so this is fine, but it will syntax-error on 3.12 tooling/linters. -- **`apply_persisted_feature_flags` mutates the live `settings` singleton via `setattr`** (`settings.py:163`) — a toggle takes effect only on next restart (documented), and the in-process `settings` object is shared; concurrent reads during the overlay are not synchronized but the overlay runs once at lifespan before serving. -- **`StreamEventBus` ACKs only when every handler succeeded** (`stream_bus.py`) — a single failing handler leaves the message pending; `_reclaim_loop` re-runs `recover_pending` every 60s so the retry fires at runtime without waiting for a restart. Undecodable messages (poison pills) are now dead-lettered then ACKed immediately and never left pending. A per-(event.id, handler) SET-NX guard (`_run_handler_guarded`) makes replay safe for already-succeeded handlers, but the guard is best-effort (fail-open without Redis). Handlers must still be idempotent. -- **`recover_pending` runs at startup and periodically** (`bootstrap.py` via `init_event_bus`; `_reclaim_loop` every 60s at runtime) — on restart or after a runtime handler failure, pending messages are reprocessed under the idempotency guard. -- **`TranscriptionService._periodic_flush` callbacks are sync `Callable`** (`transcription.py:57`) invoked inside an async loop without `await` — a blocking callback stalls the flush task. The `register_callback` signature is `Callable[[StreamBuffer], None]`, not a coroutine. -- **`TranscriptionService.get_ready_buffers` is declared `AsyncIterator` but `yield`s inside an `async for` over a dict** (`transcription.py:208`) — it works but never removes buffers; callers must `flush_buffer` after extraction or buffers accumulate forever (only `_flush_all` on shutdown clears them). -- **`ProactiveKnowledgeService._find_code_patterns` was removed** (`proactive.py`) — the dead method, its call in `build_context_package`, the `"Found N code patterns"` summary line, and the count were all deleted. `ContextPackage.code_patterns` is retained as an always-empty field for API/schema back-compat (serialized in `to_dict`; docstring marks it deprecated). Do not rely on it being populated. -- **`ProactiveKnowledgeService` singleton lazy-inits `OptimalService`** (`proactive.py:537`) — the first `get_proactive_service()` call triggers `get_optimal_service()` which may do embedding-model setup; call it early or expect latency on first claim. -- **Seeds `system` sentinel is appended literally with `team=None`** (`seeds/initial_data.py:144`) — the postgres `team` enum has no `system` value, so seeding it through the normal path would fail. Don't add `system` to `_AGENT_PRESENTATION` expecting it to flow through `_build_default_agents`. -- **`require_uuid` raises `InvalidIdentifierError` (a `ValueError` subclass), not `NotFoundError`** (`utils/converters.py:38`) — callers in `llm.py` use it on `provider.id` which is a PK and never None in practice, but a None would surface as a 500-class error not a 404. The typed subclass allows callers to handle a bad identifier distinctly from other exceptions; existing `except ValueError` handlers are unaffected. -- **`encrypt_token` rejects empty strings** (`crypto.py:53`) — the provider/project token convention is `""` = clear to NULL, handled at the service layer (`_apply_auth_token_change` checks `clear_auth_token`/`not data.auth_token` before calling encrypt); never pass `""` directly to `encrypt_token`. -- **`check_redis` opens a fresh client each call and closes it** (`health.py:27`) — fine for a health probe, but don't reuse the pattern for hot paths. ## Drift from CLAUDE.md +- CLAUDE.md does not describe the notification_dedup Redis re-fire guard, the transactional-outbox (defer_bus_publish / F107) in notification_delivery, or the DB purpose-dedup in NotificationService — all are real, load-bearing behavior added since the baseline and not reflected in the doc's Services table. +- CLAUDE.md's Services table lists NotificationService as 'Formal notifications' but does not mention NotificationDeliveryService at all, nor that NotificationService owns its own DB context+commit while NotificationDeliveryService runs in the caller's txn. +- CLAUDE.md no longer describes notification ack semantics after the channels/groups/discussion-sessions/messages teardown removed the Communication Model section — for the record: requires_ack is False for TASK_ASSIGNMENT/REVIEW_REQUEST/DOCUMENTATION_REQUEST/BROADCAST/KNOWLEDGE_SHARE/MENTION/A2A_REQUEST (ACK_REQUIRED_BY_TYPE), so most notification types do NOT require ack; and notifications can be sent by 'system' (orchestrator-generated), not only PMs/Board. +- CLAUDE.md mentions `roboco/services/notification.py` in the Services table but not `notification_dedup.py` or `notification_delivery.py`. -- **`services/settings.py:46` `FEATURE_FLAGS`** includes `rag_auto_update_enabled` ("RAG auto-update") and `transcript_prune_enabled` ("Transcript pruning") which are **NOT** listed in the CLAUDE.md "Feature Flags / company-in-a-box" enumeration (that list names web research, strategy engine, pitch provisioning, external/internal PR review, toolchain match, conventions, gateway-health, multi-repo CI-watch, dependency-update bot, gated release manager, organizational memory loop, and the self-heal flags). Two flags exist in code with no CLAUDE.md mention. -- **`services/proactive.py`** (`ProactiveKnowledgeService`) — an entire service that injects RAG context on task-claim / session-start — is **not mentioned anywhere in CLAUDE.md**. The org-memory loop (`ROBOCO_ORG_MEMORY_ENABLED`, `_briefing_for` institutional_memory) is documented, but that is a separate, newer system; `proactive.py` predates and overlaps with it yet remains present and wired (`task.py:2548`, `optimal.py:1257`). -- **`services/transcription.py`** (`TranscriptionService`) — the raw-LLM-stream buffering layer between WebSocket and the extraction pipeline — is **not mentioned in CLAUDE.md**. CLAUDE.md documents WebSocket streams and the extraction target (ExtractedMessages) but not the transcription buffer service. -- **`services/llm.py:316` `derive_mode`** returns `"grok"` for a single GLOBAL GROK assignment; CLAUDE.md's provider table lists `GROK` as a `ModelProvider` and documents the Grok CLI provider, and `apply_mode` supports `"grok"` — consistent. No drift here, noted for completeness. -- **`events/handlers.py:332` `handle_auditor_spawn`** spawns the auditor on `TASK_BLOCKED`/`TASK_CANCELLED`/`TASK_AWAITING_CEO_APPROVAL`; CLAUDE.md says "The Auditor sees all" and is a "silent observer" but does not describe the event-driven one-shot spawn on exceptional lifecycle events. Minor doc gap, not a contradiction. -- Otherwise the slice is consistent with CLAUDE.md: agent count (25 AI + 1 CEO) matches `DEFAULT_AGENTS`; `ModelProvider` enum usage matches; feature-flag overlay-on-restart contract matches; Redis-Streams event bus matches the "publish it to the bus" guidance; toolchain matching matches `ROBOCO_TOOLCHAIN_MATCH_ENABLED`. ## Changes Since Baseline -Baseline: `fd10cc862c2020b3f639cdb686d427b0198a2441`. Range `fd10cc86..HEAD` (3aff6e04) contains only 2 commits (`15effce0` "Chore: 141 Gaps fill-in (#283)", `3aff6e04` "Chore: Close gaps (#285)"), and **neither touches any file in this slice** (`git diff --stat fd10cc86..HEAD -- ` is empty; `git log --oneline` against the scope is empty). +| SHA | Subject | Impact | +|---|---|---| +| 15effce0 | Chore: 141 Gaps fill-in (#283) — added requires_ack from ACK_REQUIRED_BY_TYPE, DB purpose-dedup gated to ack-required types, re-fire guard + notification_dedup.py (new file), transactional-outbox defer_bus_publish in notification_delivery | Major hardening: notifications no longer flood inboxes (Redis re-fire + DB dedup scoped), phantom WebSocket pushes eliminated (deferred bus publish), MENTION/BROADCAST no longer inflate unacked sets (requires_ack=False) | +| 3aff6e04 | Chore: Close gaps (#285) — follow-on gap closure touching notification.py / notification_dedup.py / notification_delivery.py | Refinement of the #283 changes (exact hunks not isolated per-file in this merge commit; consolidated the dedup/outbox behavior above) | -No logic-touching commits to list — IMPACT: none. - -> Post-snapshot updates (since 2026-06-29): five commits touched this slice after the baseline was cut. -> - `e4ed970f` [chore] stream-bus: poison-pill ACK + dead-letter (`DEAD_LETTER_STREAM`, `_dead_letter`), periodic `_reclaim_loop` spawned alongside `_listen_loop`, `_run_handler_guarded` catches `BaseException` for cancelled-handler marker cleanup (3 gaps). -> - `6b441e42` [chore] converters: `InvalidIdentifierError(ValueError)` introduced; `require_uuid` now raises it for both None and unparseable input; `repo_key` git-URL normalizer added; orchestrator reaper now logs the typed error instead of silently swallowing it. -> - `321e68d7` [sweep] proactive: `_find_code_patterns` method, its call, summary line, and count removed; `ContextPackage.code_patterns` field retained (always-empty, back-compat). -> - `536bbb64` Chore/all/logical-gaps-sweep (#286) — merge commit pulling the above into the branch. -> - `d83104e9` (2026-07-17, PR #546, "wave-1 quick wins") fix(llm): provider mode switches preserve per-agent model pins — `_apply_anthropic`/`_apply_grok`/`_apply_ollama`/`_apply_self_hosted` now delete only ROLE/GLOBAL `model_assignments` rows (`scope != AGENT_SLUG`) instead of wiping the whole table, so an AGENT_SLUG pin survives a mode switch; `OLLAMA_ROLE_DEFAULTS` removed from `llm_catalog.py` as dead code (it was never consulted by routing — see `models.md`). +> Post-snapshot updates (since 2026-06-29): 115061f3 fixed list_system_notifications pending_ack_only correctness: SQL limit is now dropped for that branch so newer fully-acked rows can't mask older unacked ones (see Gotcha update above). `61e00832` (PR #492) added `notify_auditor_of_rework()` and `_get_auditor_agent()` to power the reactive auditor dispatch path: HIGH-priority ALERT notifications addressed to the auditor agent are emitted when a task enters `needs_revision` via QA/PR/PM rework chokepoints. **Wave 3** (2026-07-17, PR #547): `CreateNotificationParams` gains `requires_ack: bool | None = None`, consulted in `_create_notification` ahead of the `ACK_REQUIRED_BY_TYPE` default; `send_a2a_notification` gains a `requires_ack: bool = False` kwarg (plus an `str | None` `task_id`, for a conversational DM with no task behind it) that threads through — the only caller passing True is `A2AService._maybe_wake_ceo_recipient` (docs/map/a2a-audit-journal-permissions.md), so its wake row is finally visible to the orchestrator's `_dispatch_a2a_work` `pending_ack_only` poll. +> `cd978d11`+fixes (2026-07-18, wave-13): Telegram sends are HTML-styled — `_esc` (text nodes) / `_esc_attr` (href attributes) escaping discipline, balance-aware `_truncate`, `parse_mode`/`disable_link_preview` on the client; new `notify_ceo_of_queue_item` pushes a styled keyboard DM at each held-draft origination (release/x/video engines + `propose_roadmap`), sharing `telegram_inbound.render_queue_item_text`. +> `3b9fd0e0`+`11915f36` (PR #551, Telegram V2): `3b9fd0e0` adds `telegram_inbound.py` (new file, `TelegramInboundEngine`), extends `telegram_client.py` with `get_updates`/`answer_callback_query`/`edit_message_reply_markup`/`edit_message_text`, adds `actionable=True` to `_notify_telegram` (escalation only) so the DM carries an Approve/Reject/Open keyboard, and wires the orchestrator's `_telegram_poll_loop`. `11915f36` closes a live-reproduced approve-after-reject hole reachable via a stale Telegram button (or the pre-existing HTTP routes for X/video): `ReleaseProposalService.approve()` now refuses CANCELLED (`already_rejected`) and COMPLETED (`already_published`) proposals via a new `_approve_precheck`, `.reject()` refuses COMPLETED by raising a new `TaskAlreadyCompletedError`, and `XPostService`/`VideoPostService.approve()` each add a CANCELLED pre-lock-and-under-lock guard returning `already_rejected`. Also adds `_authorized_sender` (chat-id auth is defense-in-depth'd with a sender-id check) and widens `_resolve_task`'s search limit 10→50 so a genuine id-prefix hit can't be pushed out by newer title/description matches. +> `baa87d58`+`c7605b0d` (2026-07-19, PR #576 + #582, Telegram Mini App V4): new `tg_cockpit.py` — `TgCockpitService.today()` (tg_cockpit.py:59) assembles `needs_you`/`fleet`/`spend`/`velocity`/`ship` in one DB-only round trip backing `GET /api/telegram/today` (`api/routes/telegram.py:89`, `require_ceo_role` + 30/60s rate limit); `TgCockpitService.fleet()` (tg_cockpit.py:110) is shared verbatim by the bot's new `/agents` command. New `telegram_bridge.py` — `BridgeSession` (per-chat, in-memory) lifecycle via `start_secretary`/`start_intake`, a sole-consumer `_consume` task draining `PrompterLiveRegistry.stream`, `_forward_event` turning `turn_end`/`draft`/`batch`/`error` stream events into Telegram messages, and `mark_parked`/`discard_draft` routing a draft's Send-to-Board confirm through `PrompterService.confirm_live_draft(route="board")` + registry `park`; `sweep_idle()` reuses `settings.interactive_idle_reap_seconds` and skips parked sessions. `telegram_inbound.py` gains `BOT_COMMANDS` (a single registry driving both `/help`'s `_HELP_TEXT` and a once-per-process `client.set_my_commands` sync via `TelegramInboundEngine._ensure_commands_menu`, called from `run_cycle()`) plus `/agents` (`_render_agents`, calls `TgCockpitService.fleet()`), `/usage` (`_render_usage`, `UsageService.get_today_summary`), `/blocked` (`_render_blocked`, capped `awaiting_ceo_approval`+`blocked` lists with deep-linked rows), and `/secretary`/`/newtask`/`/end` (dispatch straight into `telegram_bridge.py`). `telegram_client.py` gains `set_my_commands` (`LiveTelegramClient`, best-effort `httpx.HTTPError`-suppressed) + a `NullTelegramClient` no-op. No orchestrator wiring changed — `_telegram_poll_loop`/`_run_telegram_poll_cycle` are byte-for-byte unchanged; the bridge's idle sweep and the commands sync both run *inside* the existing `run_cycle()` tick. New response schemas in `api/schemas/telegram.py`: `TodayTaskItem`/`TodayNeedsYou`/`TodayFleetAgent`/`TodayFleet`/`TodaySpend` (gains `series`/`delta_pct` in the `c7605b0d` follow-up)/`TodayVelocity` (new in `c7605b0d`)/`TodayShip`/`TelegramTodayResponse`. +> `56b6693e` ("security-hygiene-sweep"): root-causes a previously dead-on-arrival sweep — `NotificationDeliveryService.sweep_expired_notifications` already ran a real `expires_at < now()` query, but `NotificationService._create_notification` never WROTE `expires_at`, so the query always matched zero rows and every ack-required notification was effectively immortal. `_create_notification` now computes `requires_ack` up front (same derivation as before) and, when ack-required AND `settings.notification_ack_ttl_hours > 0`, stamps `expires_at = now() + timedelta(hours=notification_ack_ttl_hours)` (default 48h) on the `NotificationTable` row; `0` leaves `expires_at` `NULL` (never expires). Informational notifications never get a deadline regardless of the setting. ## Regression Risks -No files in this slice changed between `fd10cc86` and `HEAD`, so there are no *recent* regressions introduced by the diff. The risks below are **standing** landmines in the current code (not newly introduced), listed because they are the places a future change would plausibly break behavior: - | Title | File:Line | Claim | Severity | |---|---|---|---| -| Handler failure leaves Redis stream message pending → duplicate side effects on reclaim | `events/stream_bus.py` | ACK only when *all* handlers succeed; `_reclaim_loop` now re-runs `recover_pending` every 60s at runtime (not just on restart). Undecodable payloads are dead-lettered + ACKed immediately (poison-pill fix). Non-idempotent notification handlers can still double-fire on reclaim — `_run_handler_guarded` idempotency marker mitigates but requires Redis availability. | high | -| `resolve_for_agent` silently downgrades to Anthropic on any provider error | `services/llm.py:124,193,205` | Decrypt failure / unreachable LOCAL / missing assignment all return the legacy Anthropic route instead of raising — a misconfigured Grok/Ollama/self-hosted fleet spawns against Anthropic with only a log warning. | high | -| Mix-mode self-hosted assignment enables LOCAL only on upsert path | `services/llm.py:285` | `upsert_assignment` flips LOCAL `enabled=True`, but a pre-existing GLOBAL LOCAL assignment whose provider row was disabled will not be re-enabled until an upsert touches it — `resolve_for_agent` then skips it (`enabled` check at `llm.py:134`) and falls back to Anthropic. | medium | -| `probe_ollama_tags` leaks raw exception text into error string | `services/llm.py:86-92` | Generic `except Exception` puts `str(exc)` in the returned error, surfaced to the panel/UI — minor info disclosure. | low | -| `TranscriptionService` sync callbacks can stall the flush task | `services/transcription.py:57,233` | `register_callback` takes a sync `Callable` invoked without `await` inside the async `_periodic_flush`; a blocking callback blocks all buffer flushing. | medium | -| `get_ready_buffers` never removes buffers → unbounded growth | `services/transcription.py:208` | Iterates and yields ready buffers without clearing; only `_flush_all` (shutdown) clears. A long-running orchestrator with many sessions accumulates buffers unless callers `flush_buffer` after extraction. | medium | -| `apply_persisted_feature_flags` mutates shared `settings` singleton unsynchronized | `services/settings.py:163` | Runs once at lifespan (safe), but any future caller that re-invokes it mid-serve could race concurrent flag reads. | low | -| ~~`ProactiveKnowledgeService._find_code_patterns` vestigial~~ | `services/proactive.py` | **FIXED** (321e68d7): method removed; `ContextPackage.code_patterns` retained as always-empty for back-compat. | ~~low~~ | -| `require_uuid` raises untyped `ValueError` not `NotFoundError` | `utils/converters.py:38` | **FIXED** (6b441e42): now raises `InvalidIdentifierError(ValueError)` — typed, backward-compatible. A None/bad identifier still surfaces as a 500-class error (not 404), but callers can now handle it distinctly. | low | +| DB purpose-dedup now gated to ack-required types only — informational duplicates no longer suppressed | roboco/services/notification.py:521 | is_ack_required = ACK_REQUIRED_BY_TYPE.get(params.notification_type, True); the DB dup_q is only run when is_ack_required. REVIEW_REQUEST/DOCUMENTATION_REQUEST/TASK_ASSIGNMENT are ack-required=False, so they skip DB dedup and rely SOLELY on the 60s Redis window. Beyond 60s, a coordinator can re-fire the same REVIEW_REQUEST every tick and each one persists (the original bug the dedup was meant to stop). The Redis guard coalesces within 60s but a tick interval >60s re-opens the flood. Severity medium because the Redis guard covers the common per-tick storm. | medium | +| _persist_and_deliver skips DB purpose-dedup entirely — task-handoff duplicates not DB-deduped | roboco/services/notification_delivery.py:875 | _persist_and_deliver applies only all_recipients_recently_notified (Redis) and then add+flush+deliver with no DB dup_q. notify_pm_of_block / escalate_and_notify / notify_assignee_of_unblock can each be re-triggered (e.g. a retried i_am_blocked, a re-issued escalate) and, past the 60s Redis window, create a second BLOCKER_ESCALATION for the same (sender, type, task) while the first is unacked — exactly the inbox inflation + i_am_idle soft-block the DB dedup was added to prevent on the other path. Two paths for the same notification type with different dedup strength is a real hole. | medium | +| acknowledge publishes NOTIFICATION_ACKED directly, not via the transactional outbox | roboco/services/notification_delivery.py:451 | deliver was migrated to defer_bus_publish (after_commit) to kill phantom pushes, but acknowledge still does `await bus.publish(...)` inside the open transaction before the caller commits. A rollback after a successful ACK publish emits a phantom NOTIFICATION_ACKED for an ACK that didn't persist — the same class of bug F107 fixed for deliver, left unfixed for the ACK path. | medium | +| all_recipients_recently_notified marks recipients as a side effect on the deciding call | roboco/services/notification_dedup.py:78 | The function SET-NX-marks each fresh recipient while computing the verdict, so the call that DECIDES TO DELIVER also acquires keys for the fresh recipients. A subsequent resend within 60s then sees all-held and suppresses — intended — but it means the very first notification in a window consumes the TTL for recipients who genuinely received it, and a legit follow-up to a subset within 60s is suppressed if all of that subset were marked by the prior send. For BROADCAST this can drop a legitimately re-targeted broadcast within the window. | low | +| get_notification_count loads all agent notifications into memory | roboco/services/notification_delivery.py:379 | base_query selects all NotificationTable rows where to_agents contains agent_id with no limit, then counts in Python. For a long-running agent this row count grows unbounded; called via get_delivery_summary on the panel it is an O(n) DB read per dashboard load. Not a correctness regression from the baseline but the slice's new dedup reduces new-row growth, masking the unbounded-scan risk. | low | +| defer_bus_publish listener registration tied to session.info on the AsyncSession — session reuse hazard | roboco/services/notification_delivery.py:116 | _DRAIN_REGISTERED_KEY is set once per AsyncSession and the SQLAlchemy event.listens_for(sync_session, ...) is bound to sync_session. If an AsyncSession is reused for multiple independent transactions (connection-pool recycling), the listener stays registered and fires _schedule_pending_publishes on every subsequent commit even when no new events were deferred — _schedule_pending_publishes pops an empty queue and no-ops, so it is benign, but the listener is never removed and accumulates on the sync_session for the session's lifetime. A long-lived sync_session with many AsyncSession wraps could accumulate listeners. | low | +| `notify_auditor_of_rework` is best-effort and not deduplicated beyond the Redis re-fire guard | roboco/services/notification_delivery.py:937 | Delivery failures are swallowed and logged by the TaskService caller so the needs_revision transition never blocks. ALERT is ack-required, so each unacked rework event persists until the auditor acks it; repeated QA/PR/PM rejects on the same task emit one ALERT per transition. | low | ## Health - -This slice is a mature, mostly-stable support layer: the service-base/error hierarchy and crypto/UUID helpers are well-factored and widely reused; the Redis-Streams event bus is correctly durable (consumer groups, ACK-on-success, pending recovery) with the one real caveat that handler idempotency is the caller's job. Post-snapshot commits improved the bus (poison-pill dead-letter + periodic reclaim loop + `BaseException` marker cleanup), typed the UUID error surface (`InvalidIdentifierError`), and removed the vestigial `_find_code_patterns` call from `ProactiveKnowledgeService`. `TranscriptionService` (sync-callback + unbounded-buffer risks) remains the softest spot. Model routing's fail-safe-quietly design is intentional (a stalled spawn is worse than a wrong provider) but shifts diagnosis to logs. Overall integrity: solid, with `TranscriptionService` the one service worth either finishing or marking clearly as legacy. +This slice is substantially hardened since the baseline: the transactional-outbox for delivery (F107), the Redis re-fire guard, and the ACK_REQUIRED_BY_TYPE-driven requires_ack are all real, well-documented fixes that close prior meltdowns. The main integrity gap is dedup-path fragmentation: NotificationService._create_notification runs two dedup layers (Redis + DB purpose-dedup) while NotificationDeliveryService._persist_and_deliver runs only the Redis layer, so the task-handoff notifications (blocker/escalation/ceo-rejection) are not protected by DB purpose-dedup past the 60s Redis window — a retried i_am_blocked or escalate beyond 60s can re-create an unacked duplicate, the exact inbox-inflation + i_am_idle soft-block the DB dedup was added to prevent. A secondary consistency gap is that acknowledge publishes NOTIFICATION_ACKED directly to the bus instead of through the deferred outbox, leaving the same phantom-event class F107 fixed for deliver. Neither is a crash bug; both are correctness drift between two paths that should behave identically. Code quality is high (terse comments, clear docstrings, explicit race handling), and the slice is well-covered by the orchestrator sweeper integration and route-level callers. ## Purpose The AgentOrchestrator is the runtime brain of RoboCo: it owns the per-agent Docker container lifecycle, the per-tick dispatcher that matches tasks to agents, the stale-claim reaper, the provider rate-limit/overload park-and-probe recovery loop, and the default-off background engines (self-heal, CI-watch, dep-update, release-manager, strategy, external-PR poll, X-engine mentions poll, board roadmap engine, video render loop). It claims tasks on behalf of agents before spawning, injects briefings/manifests/git context at spawn time, provisions a per-spawn sandbox (postgres/redis/mongo via the engine registry) when opted in, captures per-session token usage, and persists durable runtime state (WaitingRecord, respawn_tracker) across restarts. @@ -4941,6 +3462,1014 @@ stateDiagram-v2 ## Health The orchestrator is the most behaviour-critical module in RoboCo and it shows: every race-prone path (respawn-tracker upsert, park-and-probe resume, readopt liveness, broken-gateway kill) carries an explicit docstring naming the live incident it prevents and the failure mode it degrades to, and the durable-state paths mirror the hardened WaitingRecord pattern (best-effort, can only suppress a spawn). Post-536bbb64, four previously medium risks are resolved: the park/probe revival race, the readopt zombie registration, the shutdown respawn-tracker re-burn, and the non-GROK stuck-agent blind spot (now caught by `_maybe_kill_stuck_claude` past 3600s). The residual exposure is the strategy-engine failure alert threshold (a misconfigured assess() could still silently fail for up to _STRATEGY_FAIL_CEO_NOTIFY_THRESHOLD ticks before surfacing) and the restart-window race where a terminal transition landing mid-restore drops a strike count. Net: integrity is high and well-instrumented. +## Purpose +The organizational-memory + playbooks slice: captures cross-agent learnings and curated playbooks, embeds them into the LEARNINGS and PLAYBOOKS pgvector RAG indexes via the OptimalService plugin architecture, and re-injects the top-K most relevant past lessons/playbooks into every agent briefing (the keystone retrieve step). Distillation at task completion runs on the local LLM only; playbook curation (draft/approve/reject/archive) is a status state-machine whose RAG index writes are split from the DB status commit so the corpus never leads the status transaction. + +## Files + +| Path | Role | LOC | +|---|---|---| +| roboco/services/memory_distiller.py | Local-LLM distiller: turns a completed task into one <=120-word Problem/Approach/Gotcha lesson (best-effort, returns None on failure) | 98 | +| roboco/services/playbook.py | PlaybookService: draft + Auditor curation state machine (draft/approve/reject/archive) + post-commit RAG index/de-index orchestration | 256 | +| roboco/services/learning.py | LearningPropagationService: record a learning, index it, notify same-scope non-human agents, scope-filtered retrieval (legacy pre-distiller capture path) | 525 | +| roboco/services/optimal.py | OptimalService: plugin-based RAG hub over pgvector; owns index_playbook/unindex_playbook/record_learning/search/search_learnings and the singleton accessor | 2044 | +| roboco/services/optimal_brain/indexes/base.py | BaseIndexPlugin ABC: chunk/filter/embed/store pipeline, hybrid search, 429-retried ask(); atomic replace_chunks reingest semantics | 1173 | +| roboco/services/optimal_brain/indexes/learnings.py | LearningsIndexPlugin: record/search cross-agent learnings; forces shareable=True on shared retrieval so private reflections never leak into briefings | 251 | +| roboco/services/optimal_brain/indexes/playbooks.py | PlaybooksIndexPlugin: index approved playbooks (title+when-to-use+procedure) and delete_playbook de-index | 97 | +| roboco/services/optimal_brain/indexes/__init__.py | Plugin registry exports (LearningsIndexPlugin, PlaybooksIndexPlugin, etc.) | 35 | +| roboco/services/optimal_brain/vector_store.py | VectorStore: asyncpg pool over chunks_ pgvector tables; add_chunks/delete_by_source/replace_chunks(hybrid_search) | 521 | +| roboco/services/repositories/base.py | BaseRepository: generic CRUD mixin used by IndexedDocumentRepository | 281 | +| roboco/services/repositories/indexed_document.py | IndexedDocumentRepository: upsert/get/delete_by_source for the indexed_documents tracking table (used by playbook de-index) | 128 | +| roboco/services/repositories/query_helpers.py | Generic SQLAlchemy query helpers (pagination/status/team/agent/timestamp filters, slug resolution) — not slice-specific | 264 | + +## Key Symbols + +| Name | Kind | File:Line | Responsibility | +|---|---|---|---| +| LessonInput | dataclass | roboco/services/memory_distiller.py:25 | Completed-task facts (title, ACs, dev/qa notes, commit messages) fed to the distiller | +| _build_prompt | function | roboco/services/memory_distiller.py:40 | Render the fixed Problem/Approach/Gotcha <=120-word distillation prompt | +| _chat | function | roboco/services/memory_distiller.py:62 | One OpenAI-compatible call to the local LLM (glm-5.2:cloud); None on non-success/empty | +| MemoryDistiller.distill | method | roboco/services/memory_distiller.py:87 | Return a <=120-word lesson or None (NONE sentinel, failure, or over-limit truncation) | +| _slugify | function | roboco/services/playbook.py:35 | Derive a unique <=80-char slug from the playbook title | +| PlaybookService.draft | method | roboco/services/playbook.py:45 | Create a DRAFT playbook; savepoint-isolates the insert to convert slug-UNIQUE TOCTOU into a clean ConflictError | +| PlaybookService.approve | method | roboco/services/playbook.py:89 | Auditor: draft->approved + stamps approver/at; flushes status ONLY (caller commits then index_approved) | +| PlaybookService.archive | method | roboco/services/playbook.py:113 | Auditor: approved->archived (retire); flushes status ONLY, caller commits then unindex_playbook | +| PlaybookService.index_approved | method | roboco/services/playbook.py:138 | Post-commit: embed the approved playbook into PLAYBOOKS index; org_memory_enabled-gated, best-effort | +| PlaybookService.reject | method | roboco/services/playbook.py:173 | Auditor: draft->archived with a reason; flushes status ONLY, caller commits then unindex_playbook | +| PlaybookService.unindex_playbook | method | roboco/services/playbook.py:195 | Post-commit: de-index a rejected/archived playbook from PLAYBOOKS; org_memory_enabled-gated, best-effort, idempotent | +| PlaybookService._get_by_slug | method | roboco/services/playbook.py:238 | Fast-path UX pre-check for slug uniqueness (DB constraint is the real guard) | +| LearningScope | enum | roboco/services/learning.py:32 | Visibility scope: PERSONAL/TEAM/CELL/ORG | +| LearningType | enum | roboco/services/learning.py:41 | Learning category: SOLUTION/PATTERN/GOTCHA/INSIGHT/REVIEW_FEEDBACK | +| LearningPropagationService.record_learning | method | roboco/services/learning.py:122 | Index the learning + create same-scope non-human notifications (skips notifications for PERSONAL) | +| LearningPropagationService._index_learning | method | roboco/services/learning.py:189 | Bridge to OptimalService.record_learning with shareable = scope != PERSONAL; passes team=None | +| LearningPropagationService._create_notifications | method | roboco/services/learning.py:205 | Open a DB session, query non-author non-human agents in scope, create formal KNOWLEDGE_SHARE notifications | +| LearningPropagationService.get_learnings_for_agent | method | roboco/services/learning.py:303 | Role-shaped search_learnings + post-filter by scope visibility (personal/team) | +| LearningPropagationService.search_similar_learnings | method | roboco/services/learning.py:484 | Similar-learnings search via OptimalService.search over LEARNINGS index | +| get_learning_service | function | roboco/services/learning.py:521 | Process-wide singleton accessor for LearningPropagationService | +| PLUGIN_REGISTRY | dict | roboco/services/optimal.py:139 | IndexType -> plugin class map (includes LEARNINGS, PLAYBOOKS) | +| OptimalService.initialize | method | roboco/services/optimal.py:183 | Graceful-degradation init of all plugins; starts background auto-index + periodic tasks | +| OptimalService.close | method | roboco/services/optimal.py:563 | Cancel _indexing_task FIRST, then periodic task, then close plugins (prevent writes to closed plugins) | +| OptimalService._get_plugin | method | roboco/services/optimal.py:593 | Typed plugin lookup with a clear RuntimeError when missing/uninitialized | +| OptimalService.index_playbook | method | roboco/services/optimal.py:859 | Embed an approved playbook + write the indexed_documents tracking row | +| OptimalService.unindex_playbook | method | roboco/services/optimal.py:889 | Delete playbook chunks from vector store AND drop tracking row; both steps best-effort/idempotent | +| OptimalService.record_learning | method | roboco/services/optimal.py:1061 | Embed a learning via LearningsIndexPlugin + write tracking row (source learn-{md5(full content)}) | +| OptimalService.search | method | roboco/services/optimal.py:1145 | Embed-once fan-out: concurrent hybrid search across selected indexes; used by similar_memory | +| OptimalService._aggregate_citations | method | roboco/services/optimal.py:1221 | Embed once + concurrent per-index search into an aggregation buffer for RAG query() | +| OptimalService.search_learnings | method | roboco/services/optimal.py:1507 | LEARNINGS-only search with optional category/team filter (shareable_only=True default) | +| get_optimal_service | function | roboco/services/optimal.py:2017 | Lock-guarded singleton: publish instance only after initialize() completes | +| BaseIndexPlugin.ingest | method | roboco/services/optimal_brain/indexes/base.py:351 | Validate -> metadata -> source URI -> chunk/filter/embed/store pipeline | +| BaseIndexPlugin._chunk_filter_embed_store | method | roboco/services/optimal_brain/indexes/base.py:429 | Chunk + quality-filter + embed + atomic replace_chunks/add_chunks; returns stored count | +| BaseIndexPlugin._citations_to_results | method | roboco/services/optimal_brain/indexes/base.py:772 | Apply exact-match metadata filters to citations and cap to top_k SearchResults | +| BaseIndexPlugin.search_with_embedding | method | roboco/services/optimal_brain/indexes/base.py:808 | Pre-computed-embedding hybrid search entry; LearningsIndexPlugin overrides to force shareable | +| BaseIndexPlugin.search | method | roboco/services/optimal_brain/indexes/base.py:852 | Embed-then-search_with_embedding convenience entry | +| BaseIndexPlugin.ask | method | roboco/services/optimal_brain/indexes/base.py:893 | Per-index RAG Q&A with 15s search timeout + 429-retried LLM synthesis | +| LearningsIndexPlugin.search_with_embedding | method | roboco/services/optimal_brain/indexes/learnings.py:44 | Force shareable=True filter unless include_private opt-in; prevents private reflections leaking into briefings | +| LearningsIndexPlugin.search | method | roboco/services/optimal_brain/indexes/learnings.py:75 | Embed-then-search that threads include_private to search_with_embedding | +| LearningsIndexPlugin.record_learning | method | roboco/services/optimal_brain/indexes/learnings.py:124 | Build lrn-{md5(content[:100])} doc_id + enriched content; ingest with category/role/team/shareable metadata | +| LearningsIndexPlugin.search_learnings | method | roboco/services/optimal_brain/indexes/learnings.py:174 | Category/team-filtered search; include_private=not shareable_only to thread the shareable default | +| IndexPlaybookParams | dataclass | roboco/services/optimal_brain/indexes/playbooks.py:18 | Params for indexing an approved playbook (id/title/problem/procedure/tags/team/scope) | +| PlaybooksIndexPlugin.index_playbook | method | roboco/services/optimal_brain/indexes/playbooks.py:55 | Embed title + when-to-use + procedure + tags; metadata status=approved | +| PlaybooksIndexPlugin.delete_playbook | method | roboco/services/optimal_brain/indexes/playbooks.py:73 | Delete a playbook's chunks by source URI (idempotent no-op when none match) | +| PlaybooksIndexPlugin.search_playbooks | method | roboco/services/optimal_brain/indexes/playbooks.py:86 | Optional team-scoped search over approved playbooks | +| VectorStore.replace_chunks | method | roboco/services/optimal_brain/vector_store.py:239 | Atomic single-connection single-tx DELETE+INSERT replacing a source's chunks (closes concurrent reindex duplicate race) | +| VectorStore.delete_by_source | method | roboco/services/optimal_brain/vector_store.py:225 | Delete every chunk row for a source URI (idempotent) | +| VectorStore.hybrid_search | method | roboco/services/optimal_brain/vector_store.py:342 | pgvector + full-text hybrid retrieval returning Citation rows | +| IndexedDocumentRepository.delete_by_source | method | roboco/services/repositories/indexed_document.py:103 | Drop the indexed_documents tracking row by (index_type, source_hash); idempotent bool return | +| IndexedDocumentRepository.upsert_batch | method | roboco/services/repositories/indexed_document.py:22 | Bulk upsert tracking rows keyed by (index_type, source_hash) | + +## Data Flow +CAPTURE (task completion): TaskService._extract_completion_learnings (task.py:2837) is fire-and-forget on completion. With org_memory_enabled it calls _completion_learnings_for (task.py:2798) which runs MemoryDistiller().distill(LessonInput(...)) against the local LLM (memory_distiller.py) to produce ONE <=120-word lesson, else falls back to the legacy raw-notes _collect_completion_learnings. The lesson goes to LearningPropagationService.record_learning (learning.py:122) -> _index_learning -> OptimalService.record_learning (optimal.py:1061) -> LearningsIndexPlugin.record_learning (learnings.py:124), which embeds via the shared qwen3 embedder and stores chunks in the chunks_learnings pgvector table with metadata {category, agent_role, shareable, ...}. _create_notifications (learning.py:205) opens a separate DB session and creates formal KNOWLEDGE_SHARE notifications for same-scope non-author, non-human agents (CEO/prompter/secretary excluded via _HUMAN_ONLY_ROLES). + +PLAYBOOK CURATION: A delivery agent calls the draft_playbook content verb (do_server -> v1/do.py -> content_actions.draft_playbook -> PlaybookService.draft) which writes a DRAFT row with a slug-unique constraint (savepoint TOCTOU guard). The Auditor (gateway verb) or Auditor/CEO (panel route /api/playbooks) calls approve/reject/archive. The status flush and the RAG index write are deliberately split: approve() flushes status ONLY; the caller (api/routes/playbooks.py or content_actions._curate_playbook) commits the DB transaction FIRST, then calls index_approved() which -> OptimalService.index_playbook -> PlaybooksIndexPlugin.index_playbook -> BaseIndexPlugin.ingest -> chunk/embed/store in chunks_playbooks (+ a tracking row in indexed_documents). reject/archive similarly commit then call unindex_playbook -> OptimalService.unindex_playbook -> delete_playbook (vector store) + IndexedDocumentRepository.delete_by_source (tracking row). + +RETRIEVE (keystone briefing): Choreographer._briefing_for (_impl.py:814) is called on give_me_work/claim/done/qa/doc/pr_review/board routes. It calls _institutional_memory (_impl.py:877) which, when org_memory_enabled and a task is in hand, shapes a role-shaped query via shape_memory_query (evidence_builder.py) and calls EvidenceRepo.similar_memory (evidence_repo.py:323). similar_memory runs OptimalService.search over [LEARNINGS, PLAYBOOKS] indexes (embed-once, concurrent hybrid search), filters results by min_score, and returns top-K {kind, summary, source, score} items injected as briefing['institutional_memory']. LearningsIndexPlugin.search_with_embedding forces shareable=True so private reflections never surface. Memory is best-effort: any RAG/embed failure returns [] so the briefing path never breaks. + +## Mermaid +```mermaid +stateDiagram-v2 + direction LR + [*] --> draft: delivery agent draft_playbook + draft --> approved: Auditor/CEO approve (commit then index_approved) + draft --> archived: Auditor reject (commit then unindex_playbook) + approved --> archived: Auditor/CEO archive (commit then unindex_playbook) + approved --> [*]: surfaces in briefings (PLAYBOOKS index) + archived --> [*]: terminal (de-indexed) +``` + +```mermaid +sequenceDiagram + autonumber + participant Route as PanelRoute + participant PS as PlaybookService + participant DB as Postgres + participant OS as OptimalService + participant VS as VectorStore + participant TR as indexed_documents + Route->>PS: approve(id, approver) + PS->>PS: guard status==DRAFT + PS->>DB: status=APPROVED, flush in-tx + Route->>DB: commit() + Route->>PS: index_approved(playbook) + PS->>OS: index_playbook(IndexPlaybookParams) + OS->>VS: ingest chunk+embed+replace_chunks + OS->>TR: upsert tracking row + Note over VS,DB: index never leads the status commit +``` + +```mermaid +erDiagram + playbooks ||--o{ chunks_playbooks : "approved to embedded" + playbooks ||--o| indexed_documents : "tracking row" + learnings ||--o{ chunks_learnings : "shareable to embedded" + learnings ||--o| indexed_documents : "tracking row" + playbooks { + uuid id PK + str slug UK + str status + uuid approved_by + } + chunks_playbooks { + str source + vector embedding + jsonb metadata + } + chunks_learnings { + str source + vector embedding + jsonb metadata + } +``` + +## Logical Tree +``` +org-memory-playbooks + Capture (completion) + MemoryDistiller (local LLM only) + LessonInput -> _build_prompt -> _chat -> distill (<=120w or None) + TaskService._completion_learnings_for [external, task.py] + org_memory_enabled ? distill : legacy raw capture + LearningPropagationService + record_learning -> _index_learning -> OptimalService.record_learning + _create_notifications (KNOWLEDGE_SHARE, non-human, scope-filtered) + Playbook curation state machine (PlaybookService) + draft (slug UNIQUE + savepoint TOCTOU guard) + approve (draft->approved; commit-then-index) + reject (draft->archived; commit-then-unindex) + archive (approved->archived; commit-then-unindex) + index_approved / unindex_playbook (post-commit, org_memory-gated) + RAG hub (OptimalService + plugins) + PLUGIN_REGISTRY: LEARNINGS, PLAYBOOKS, ... + BaseIndexPlugin: ingest / search / ask / replace_chunks + LearningsIndexPlugin: forces shareable=True on shared retrieval + PlaybooksIndexPlugin: index_playbook / delete_playbook + VectorStore: chunks_ pgvector tables; replace_chunks atomic + IndexedDocumentRepository: tracking-row upsert / delete_by_source + Retrieve (keystone briefing) [external callers] + Choreographer._briefing_for -> _institutional_memory + shape_memory_query (role-shaped) + EvidenceRepo.similar_memory -> OptimalService.search([LEARNINGS,PLAYBOOKS]) + -> briefing['institutional_memory'] (top-K, min_score-floored) +``` + +## Dependencies +- Internal: roboco.config.settings (org_memory_enabled, org_memory_top_k, org_memory_min_score, local_llm_*, default_embedding_model, embedding_dimensions, rag_*), roboco.db.tables.PlaybookTable / IndexedDocumentTable, roboco.db.get_db_context, roboco.models.base.PlaybookStatus, roboco.models.optimal.IndexType / SearchResult / SearchOutcome / QueryContext, roboco.models.playbook.PlaybookCreate / Playbook, roboco.services.base.BaseService / ConflictError / NotFoundError, roboco.services.exceptions (RateLimitError, parse_retry_after_header, HTTP_TOO_MANY_REQUESTS, MAX_RATE_LIMIT_RETRIES), roboco.services.optimal_brain.text_chunker (TextChunker, Chunk, Citation, Document), roboco.services.optimal_brain.shared_embedder.get_shared_embedder, roboco.services.gateway.evidence_repo.EvidenceRepo.similar_memory, roboco.services.gateway.evidence_builder.shape_memory_query, roboco.services.gateway.choreographer._impl._briefing_for / _institutional_memory, roboco.services.gateway.content_actions (draft/approve/reject/archive_playbook), roboco.services.task.TaskService._completion_learnings_for / _extract_completion_learnings, roboco.foundation.identity.Role, roboco.api.routes.playbooks (panel route), roboco.mcp.do_server (draft/approve/reject/archive_playbook verbs) +- External: httpx (local LLM chat + RAG synthesis), structlog, sqlalchemy (select, delete, func, IntegrityError, AsyncSession), asyncpg (VectorStore pool + transaction), pgvector (vector column), dataclasses / enum / hashlib / re / asyncio + +## Entry Points + +| Name | File | Trigger | +|---|---|---| +| draft_playbook verb | roboco/services/gateway/content_actions.py | Agent content verb via do_server -> POST /api/v1/do/draft_playbook (delivery roles only) | +| approve/reject/archive_playbook verbs | roboco/services/gateway/content_actions.py | Auditor content verb via do_server -> POST /api/v1/do/{approve,reject,archive}_playbook | +| GET/POST /api/playbooks[/{id}/{approve,reject,archive}] | roboco/api/routes/playbooks.py | Panel review-queue HTTP (Auditor or CEO only) | +| _extract_completion_learnings | roboco/services/task.py | Fire-and-forget on task completion (TaskService complete/ceo_approve path) | +| _briefing_for / _institutional_memory | roboco/services/gateway/choreographer/_impl.py | Every choreographer verb that builds a context_briefing (give_me_work, claim, done, qa, doc, pr_review, board) | +| OptimalService.initialize / get_optimal_service | roboco/services/optimal.py | FastAPI lifespan startup; first RAG caller (lazy singleton) | +| OptimalService.close | roboco/services/optimal.py | FastAPI lifespan shutdown (cancels indexing + periodic tasks, closes plugins) | + +## Config Flags +- ROBOCO_ORG_MEMORY_ENABLED (default off) — gates the whole loop: distill-vs-legacy capture, index_approved/unindex_playbook no-op when off, _institutional_memory returns [] when off +- ROBOCO_ORG_MEMORY_TOP_K (default 3, 1..10) — max institutional-memory items injected into a briefing +- ROBOCO_ORG_MEMORY_MIN_SCORE (default 0.6, 0..1) — cosine-similarity floor; below it nothing is injected +- ROBOCO_LOCAL_LLM_MODEL (default glm-5.2:cloud) + ROBOCO_LOCAL_LLM_BASE_URL — the distiller + RAG synthesis LLM endpoint +- ROBOCO_DEFAULT_EMBEDDING_MODEL (default qwen3-embedding:0.6b) + ROBOCO_EMBEDDING_DIMENSIONS — embedder for LEARNINGS/PLAYBOOKS chunks +- ROBOCO_RAG_CHUNK_STRATEGY / ROBOCO_RAG_CHUNK_SIZE / ROBOCO_RAG_CHUNK_OVERLAP / ROBOCO_RAG_PERSIST_DIR / ROBOCO_RAG_STORE_URL — chunking + store DSN +- ROBOCO_DATABASE_* (VectorStore.store_url derived) — required; missing store_url raises at plugin initialize() + + +## Gotchas +- Index-vs-status ordering is a hard contract: approve()/reject()/archive() flush status ONLY; the caller MUST commit the DB tx BEFORE calling index_approved()/unindex_playbook(). The vector store writes through its own auto-committing pool connection, so indexing before commit would durably land (or drop) a playbook in the corpus even if the status tx rolled back. Both the panel route and content_actions honour this; any new caller must too. +- archive() and reject() previously overwrote approved_by/approved_at — FIXED in 536bbb64: migration 053 added archived_by/archived_at columns; archive() now writes archived_by/archived_at at playbook.py:132-133 and reject() likewise at playbook.py:189-190, leaving approved_by/approved_at intact. +- Learning doc-id / source-URI mismatch: FIXED in 536bbb64. OptimalService.record_learning now derives source = f"roboco://learnings/{doc_id}" from the plugin's returned doc_id (optimal.py:1078) instead of independently computing learn-{md5(full content)}, so the tracking row and chunk rows share the same source URI. +- LearningPropagationService._index_learning always passes team=None, so team-scoped search_learnings(team=...) will never match auto-captured completion learnings (learnings.py:199). +- LearningsIndexPlugin.search_with_embedding forces shareable=True via exact equality on metadata. This relies on the stored metadata value being a JSON bool that round-trips to Python True; a learning indexed with a string 'true' would be filtered out. Currently safe (prepare_metadata sets a Python bool) but brittle if metadata serialization changes. +- BaseIndexPlugin._citations_to_results applies filters as exact equality on every key-value pair. The forced shareable=True filter is therefore exact-match; any NULL/missing shareable metadata (older rows) would be excluded from briefings after the fix. +- replace_chunks on a re-ingest: the embedder-failure case (non-empty chunks list but no usable embeddings returned) is now guarded at vector_store.py:272 — the wipe is skipped and existing rows are preserved (FIXED in 536bbb64). The deliberate-clear case (empty chunks list) still deletes, by design. +- PlaybooksIndexPlugin inherits BaseIndexPlugin.replace_on_reingest=True; re-indexing an already-approved playbook (e.g. approve twice via different paths) atomically replaces chunks. approve() now guards status==DRAFT so a double-approve is blocked before reaching index. +- draft()'s _get_by_slug pre-check is a UX fast-path, NOT the guard: two concurrent same-title drafts both miss it and the loser hits the slug UNIQUE constraint — handled by the savepoint + IntegrityError->ConflictError conversion. Don't rely on the pre-check for uniqueness. +- _HUMAN_ONLY_ROLES (CEO, prompter, secretary) are excluded as learning notification recipients (learning.py:27), resolved from the foundation Role enum at import time. CLAUDE.md states agent learnings exclude human/human-driven roles — code matches. +- OptimalService is a process-wide singleton published only after initialize() completes under a lock; a half-built instance is never observable. But the singleton is event-loop-bound — calling get_optimal_service() from a different loop raises 'bound to a different event loop' (noted at optimal.py:2010). + + +## Drift from CLAUDE.md +- CLAUDE.md says institutional memory is injected 'on claim'. Code injects it on EVERY _briefing_for call that carries a task (give_me_work, i_will_plan, claim, done, qa, doc, pr_review, board, submit_up) — broader than 'on claim' (choreographer/_impl.py:814-875, called from ~30 sites). The _institutional_memory guard is only `org_memory_enabled and task is not None`, not claim-specific. +- CLAUDE.md's verb-surface table lists Auditor verbs as only `triage` (read-only) in the role table, while the prose below it lists Auditor `approve_playbook`/`reject_playbook`/`archive_playbook` curation. The gateway enforces _CURATE_PLAYBOOK_ROLES={'auditor'} (content_actions.py:310) — auditor-only via the verb path — while the panel route allows Auditor OR CEO (_CURATOR_ROLES in api/routes/playbooks.py:21). The Auditor/CEO split is documented for /api/playbooks but the verb path is auditor-only, which is consistent with the prose but not reflected in the verb table row. + + +## Changes Since Baseline + +| SHA | Subject | Impact | +|---|---|---| +| 15effce0 | [feature] org-memory/playbooks curation + retrieval hardening (bundled in 141-Gaps fill-in PR #283) | playbook.py: split status flush from RAG index write — approve() no longer indexes inline; added archive() (approved->archived) + public index_approved()/unindex_playbook(); added status==DRAFT precondition guards on approve/reject; savepoint-isolated draft insert to convert slug-UNIQUE TOCTOU into ConflictError. | +| 15effce0 | [fix] learnings: force shareable=True on shared retrieval | learnings.py: overrode search_with_embedding/search to force shareable=True unless include_private opt-in; threaded include_private=not shareable_only through search_learnings. Prevents private (shareable=False) reflections leaking into cross-agent briefings via OptimalService.search. | +| 15effce0 | [fix] playbook de-index path | optimal.py added OptimalService.unindex_playbook (delete chunks + tracking row, best-effort); playbooks.py added PlaybooksIndexPlugin.delete_playbook; repositories/indexed_document.py added IndexedDocumentRepository.delete_by_source. reject/archive now actually remove a previously-approved playbook from the corpus. | +| 15effce0 | [fix] atomic reindex (F108) | base.py replaced separate delete_by_source + add_chunks with VectorStore.replace_chunks (single connection, single tx) for replace_on_reingest plugins — closes concurrent-reindex duplicate-chunk race; failed insert now reverts the delete. | +| 15effce0 | [fix] OptimalService.close() ordering | optimal.py: close() now cancels the startup _indexing_task FIRST (can be mid-flight writing through plugins) before the periodic task and plugin close — prevents writes against closed plugins. | +| 15effce0 | [chore] glm-5 -> glm-5.2:cloud | memory_distiller.py docstring + IndexConfig.llm_model default bumped from glm-5:cloud to glm-5.2:cloud (matches the fleet LLM bump). No behavior change beyond the model name. | + +> Post-snapshot updates (since 2026-06-29): commit 536bbb64 (Chore/all/logical gaps sweep, PR #286) touched three files in this slice: (1) roboco/services/playbook.py — archive() and reject() now write archived_by/archived_at (new columns, migration 053) instead of overwriting approved_by/approved_at; content_actions._curate_playbook wraps the gating session.commit() in a PendingRollbackError guard (#55) so a poisoned session returns a clean invalid_state and never falls through to index an uncommitted playbook. (2) roboco/services/optimal.py — record_learning reuses the plugin's returned doc_id for the tracking-row source URI, closing the lrn-/learn- mismatch (#182/#183). (3) roboco/services/optimal_brain/vector_store.py — replace_chunks skips the wipe when chunks is non-empty but all lack embeddings (#181), preserving existing rows on embedder failure. + +## Regression Risks + +| Title | File:Line | Claim | Severity | +|---|---|---|---| +| ~~archive() overwrites the original approver attribution~~ **FIXED 536bbb64** | roboco/services/playbook.py:132 | Migration 053 added archived_by/archived_at; archive() (line 132-133) and reject() (line 189-190) now write those columns, leaving approved_by/approved_at intact. | medium | +| approve() index write contract is now caller-owned — a missed call silently skips indexing | roboco/services/playbook.py:109 | Before baseline, approve() called _index_approved inline. Now approve() flushes status ONLY and the caller must commit then call index_approved(). If any caller (current or future) calls approve() without the commit+index_approved pair, the playbook is APPROVED in DB but NEVER embedded — it will not surface in briefings. Today only api/routes/playbooks.py and content_actions._curate_playbook call it (both correct), but the contract is a footgun. | medium | +| unindex_playbook returns early on vector-store failure, leaving tracking row stale | roboco/services/optimal.py:909 | On a vector-store delete exception, unindex_playbook logs + `return`s before dropping the indexed_documents tracking row. If the VS delete partially succeeded (some chunks gone) but raised, the tracking row lingers referencing a partially-deleted source — inconsistent index/tracking state. Best-effort by design, but the divergence is silent. | low | +| ~~Learnings tracking-row source URI never matches the embedded chunk source URI~~ **FIXED 536bbb64** | roboco/services/optimal.py:1078 | record_learning now reuses the plugin's returned doc_id: source = f"roboco://learnings/{doc_id}" — tracking row and chunk rows share the same URI. | low | +| replace_chunks wipes a source when a re-ingest produces zero embedded chunks | roboco/services/optimal_brain/indexes/base.py:475 | **Embedder-failure case FIXED 536bbb64** (vector_store.py:272): when chunks is non-empty but no records have embeddings, replace_chunks now returns early, preserving existing rows. The deliberate-clear case (empty chunks list) still deletes by design. | low | +| Forced shareable=True filter excludes any learning whose metadata lacks a shareable key | roboco/services/optimal_brain/indexes/learnings.py:70 | _citations_to_results applies filters as chunk_meta.get(k) == v. With forced shareable=True, any older learning chunk whose metadata has no 'shareable' key (get returns None) is excluded from briefings. If pre-fix rows exist without the shareable metadata field, they stop surfacing after this change — a silent recall regression for legacy learnings. | low | +| close() awaits a cancelled _indexing_task that may be mid-DB-write | roboco/services/optimal.py:571 | close() now cancels _indexing_task and awaits it (suppressing CancelledError). If the indexing task is mid-flight inside an asyncpg executemany/transaction at shutdown, cancellation can leave a partial chunk insert. Shutdown-only, best-effort, and the new ordering is strictly better than the old close-then-write-to-closed-plugin race it fixes — but the cancellation mid-write is new surface. | low | + +## Health +The slice is coherent and has been further hardened by PR #286 (536bbb64): archive()/reject() provenance loss and the learnings tracking-row/chunk source-URI mismatch are both fixed, and the embedder-failure wipe in replace_chunks is now guarded. The main residual risks are (a) the caller-owned commit-then-index contract on approve/reject/archive — a future caller that forgets index_approved silently produces an un-indexed approved playbook (the poisoned-session guard in content_actions is a step forward but the footgun remains for any new caller); (b) the forced shareable=True filter silently excluding older learning rows that lack the metadata key. The org_memory_enabled gate is consistently applied at every entry (distill, index_approved, unindex_playbook, _institutional_memory), so the whole loop is inert when off. Best-effort semantics are uniformly observed: every RAG/embed failure returns []/None and never blocks completion or the briefing. No critical regressions found. + +## Purpose +The two choreographer mixins that implement the PR-reviewer's two distinct surfaces: the in-path assembled-PR gate (PRGateMixin: claim_gate_review / pr_pass / pr_fail between a PM's submit and merge) and the inbound external/fork PR review (PRReviewerMixin: claim_pr_review / post_pr_review, read-only, posts one change-request and completes). Both are mixed into the composed Choreographer and route through the spec gate + verb runner, returning standardized Envelopes. + +## Files + +| Path | Role | LOC | +|---|---|---| +| /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/gateway/choreographer/pr_gate.py | PRGateMixin — in-path assembled-PR gate verbs (claim_gate_review, pr_pass, pr_fail) and their helpers | 626 | +| /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/gateway/choreographer/pr_review.py | PRReviewerMixin — inbound external/fork PR review verbs (claim_pr_review, post_pr_review) + module-level resolve_task_project_slug | 601 | + +## Key Symbols + +| Name | Kind | File:Line | Responsibility | +|---|---|---|---| +| PRGateMixin | class | roboco/services/gateway/choreographer/pr_gate.py:40 | Mixin: in-path assembled-PR gate verbs (claim_gate_review/pr_pass/pr_fail) + helpers; inherits ChoreographerHelpers only under TYPE_CHECKING | +| PRGateMixin.claim_gate_review | method | roboco/services/gateway/choreographer/pr_gate.py:43 | Reviewer claims an awaiting_pr_review task without transitioning it (status stays awaiting_pr_review); returns assembled PR diff inline as evidence | +| PRGateMixin.pr_pass | method | roboco/services/gateway/choreographer/pr_gate.py:115 | Pass the assembled PR: awaiting_pr_review → awaiting_pm_review; delegates to _gate_decision | +| PRGateMixin.pr_fail | method | roboco/services/gateway/choreographer/pr_gate.py:123 | Fail the assembled PR with structured `findings` (the deprecated `issues=[str]` shim still works this release) → needs_revision; validates + count-guards (nudge 5 / hard-cap 10) via `choreographer/findings.py`, inserts one append-only ledger row per finding BEFORE `_record_gate_verdict_for` so the verdict note carries real ids, delegates to `_gate_decision`. See `docs/map/review-findings.md`. | +| PRGateMixin._gate_preflight | method | roboco/services/gateway/choreographer/pr_gate.py:148 | Ownership + role + spec gate (with self_review_block via actor_slug/original_developer_slug) + free-text soup guard for pr_pass/pr_fail; returns rejection Envelope or (t,agent,role_str,briefing,spec_ctx) | +| PRGateMixin._record_gate_verdict_for | method | roboco/services/gateway/choreographer/pr_gate.py:229 | Author the canonical pr_review verdict note before the transition; on pr_fail also capture the assembled PR head SHA for the unchanged-PR gate; on pr_pass with ci_note, stamp the ci_status field into the verdict with evidence the CI guard ran | +| PRGateMixin._post_gate_review | method | roboco/services/gateway/choreographer/pr_gate.py:245 | Post the gate verdict to the PR itself (best-effort, after the DB transition); resolves reviewer slug | +| PRGateMixin._deliver_pr_fail_to_owner | method | roboco/services/gateway/choreographer/pr_gate.py:253 | a2a the pr_fail change-requests to the owning PM (best-effort) with a Main-PM-root steer to re-delegate not re-submit; closes the blind re-submit loop | +| PRGateMixin._gate_decision | method | roboco/services/gateway/choreographer/pr_gate.py:292 | Shared body for pr_pass/pr_fail: preflight + tracing + pr_pass blocked guards + record verdict + run_intent + None-guard for concurrent transition + post-PR + a2a on fail | +| PRGateMixin._pr_pass_blocked | method | roboco/services/gateway/choreographer/pr_gate.py:373 | Refuse pr_pass on a broken toolchain, block-level convention violation, or non-green CI on the assembled PR's head commit; returns (rejection_envelope, ci_note). Both toolchain and conventions guards inert when their flags are off; CI guard fails open on configuration gaps | +| PRGateMixin._ci_status_guard | method | roboco/services/gateway/choreographer/pr_gate.py:520 | Refuse pr_pass unless CI on the assembled PR's head commit is green. Failing/pending/unscheduled CI blocks with reviewer-aware remediation pointing at pr_fail; configuration gaps, unreachable/nonexistent repos, or real API failures on reachable repos each pass through with distinct classifications (no_ci_configured vs error) via git.get_pr_ci_status | +| PRGateMixin._resolve_ci_status | method | roboco/services/gateway/choreographer/pr_gate.py:480 | Thin wrapper: calls git.get_pr_ci_status, interprets the returned dict (no_ci_configured/pending/failure/error/success), and composes a rejection Envelope if CI must block pr_pass | +| PRGateMixin._record_gate_verdict | method | roboco/services/gateway/choreographer/pr_gate.py:403 | Persist the gate verdict as the canonical pr_review structured note (passed/failed), with issues slot for pr_fail, head_sha stamp for pr_fail, and ci_status evidence on pr_pass; best-effort (ContentValidationError logged not raised) | +| PRGateMixin._capture_pr_head_sha | method | roboco/services/gateway/choreographer/pr_gate.py:468 | Best-effort capture of the assembled PR head SHA at pr_fail time via _project_slug_for + git.get_pr_head_sha; returns None on any failure (fail-open) | +| PRGateMixin._post_gate_review_to_pr | method | roboco/services/gateway/choreographer/pr_gate.py:502 | Post APPROVE/REQUEST_CHANGES on cell→root PRs; always COMMENT on root→master (only CEO merges master); best-effort | +| PRGateMixin._gate_role_or_rejection | method | roboco/services/gateway/choreographer/pr_gate.py:545 | Parse the role enum from role_str or return a not_authorized rejection Envelope | +| PRGateMixin._gate_tracing | method | roboco/services/gateway/choreographer/pr_gate.py:569 | Tracing gate for pr_pass/pr_fail: requires journal:learning entry + substantive pr_reviewer_notes (notes threaded via SimpleNamespace shim) | +| PRGateMixin._re_stamp_pr_fail_head_sha_if_advanced | method | roboco/services/gateway/choreographer/pr_gate.py:255 | Re-capture the PR head SHA AFTER the transition commits and re-stamp the verdict note only if it advanced past the pre-transition capture (#189 fix for stale-SHA false-allow loop) | +| PRGateMixin._gate_review_event_verdict | staticmethod | roboco/services/gateway/choreographer/pr_gate.py:555 | Map gate verb → (review event, verdict label): pr_pass → APPROVE/PASSED, pr_fail → REQUEST_CHANGES/CHANGES REQUESTED — both downgraded to COMMENT on root→master (is_root) | +| PRGateMixin._gate_review_body | staticmethod | roboco/services/gateway/choreographer/pr_gate.py:568 | Render the gate-review comment body posted to the assembled PR (includes CEO-only footer for root→master PRs) | +| PRGateMixin._build_gate_review_evidence | method | roboco/services/gateway/choreographer/pr_gate.py:697 | Inline evidence for claim_gate_review: assembled branch diff + pr_number/pr_url + acceptance_criteria + is_assembled_pr | +| PRGateMixin._gate_diff_parent | method | roboco/services/gateway/choreographer/pr_gate.py:920 | The assembled task's REAL parent branch (via `resolve_parent_branch`, reading the parent TASK's own `branch_name`) or None for a branchless task; threaded as `preferred_parent` into `git.diff` (claim_gate_review evidence) and the conventions guard. Replaces the string-derived `parent_branch_for`, which reused the child branch's own team segment and was wrong across every cross-team hop (e.g. a frontend child of a main_pm root). Fail-open on a lookup error (falls back to the derived-base default), like every other `resolve_parent_branch` call site. | +| PRReviewerMixin | class | roboco/services/gateway/choreographer/pr_review.py:44 | Mixin: inbound external/fork PR review verbs (claim_pr_review/post_pr_review) + helpers; read-only, never checks out contributor code | +| PRReviewerMixin.claim_pr_review | method | roboco/services/gateway/choreographer/pr_review.py:47 | Reviewer claims an external-PR review task (pending→in_progress via task.pr_review_claim, branch-gate exempt); returns contributor diff inline read-only | +| PRReviewerMixin._build_pr_review_content | staticmethod | roboco/services/gateway/choreographer/pr_review.py:124 | Validate summary+findings+event into a PrReviewContent via validate_content, or return an invalid_state Envelope | +| PRReviewerMixin._resolve_post_body | method | roboco/services/gateway/choreographer/pr_review.py:149 | Resolve the GitHub comment body: canonical render when findings given (and stored structured), else free-text body; Envelope on malformed findings | +| PRReviewerMixin._is_hand_formatted_verdict | staticmethod | roboco/services/gateway/choreographer/pr_review.py:163 | True when a free-text body carries verdict/section markdown headers (## summary/issues/verdict/findings) the system would otherwise generate | +| PRReviewerMixin._post_review_side_effects | method | roboco/services/gateway/choreographer/pr_review.py:180 | Post the review to GitHub + send external-pr-reviewed CEO notification (both best-effort, after DB transition) | +| PRReviewerMixin.post_pr_review | method | roboco/services/gateway/choreographer/pr_review.py:211 | Post ONE change-request to the PR and finish the review task (in_progress→completed); content gates pre-side-effect, side-effects post-transition | +| PRReviewerMixin._post_pr_review_preflight | method | roboco/services/gateway/choreographer/pr_review.py:290 | Pre-runner guards for post_pr_review: non-empty body, role, spec gate, tracing gate; returns (agent,role_str,briefing,spec_ctx) or rejection | +| PRReviewerMixin._verdict_consistency_gate | method | roboco/services/gateway/choreographer/pr_review.py:343 | Reject a self-contradicting (event, findings) pair via pr_review_conflict pure invariant; runs before any side effect | +| PRReviewerMixin._post_pr_review_content_gates | method | roboco/services/gateway/choreographer/pr_review.py:376 | Folded content gates: verdict consistency then no-hand-formatted-body guard (only when no findings); returns first rejection or None | +| PRReviewerMixin._resolve_role | method | roboco/services/gateway/choreographer/pr_review.py:442 | Parse role enum or return not_authorized rejection Envelope | +| PRReviewerMixin._runner_failure | method | roboco/services/gateway/choreographer/pr_review.py:466 | Shared rejection envelope for a verb-runner failure | +| PRReviewerMixin._build_pr_review_evidence | method | roboco/services/gateway/choreographer/pr_review.py:488 | Inline evidence for claim_pr_review: PR unified diff via git.get_pr_diff (read-only) + pr_number/pr_url + is_external_pr | +| PRReviewerMixin._pr_review_tracing_gate | method | roboco/services/gateway/choreographer/pr_review.py:501 | Tracing gate for post_pr_review: journal:learning entry + substantive pr_reviewer_notes (body threaded via SimpleNamespace shim) | +| PRReviewerMixin._project_slug_for | method | roboco/services/gateway/choreographer/pr_review.py:545 | Resolve project slug for a task; delegates to module-level resolve_task_project_slug | +| resolve_task_project_slug | function | roboco/services/gateway/choreographer/pr_review.py:555 | Module-level slug resolver shared with _impl.py unchanged-PR gate: project_id → product first distinct project → cell_projects first distinct project | + +## Data Flow +Both mixins are composed into the Choreographer and invoked by the flow MCP server (roboco-flow) when a pr_reviewer agent calls a verb. Inputs: reviewer_agent_id + task_id (+ notes/issues for verdicts, + body/event/findings for post_pr_review). Each verb fetches the task (self.task.get), resolves the agent role (self.task.agent_for), builds a briefing (self._briefing_for), runs the spec gate (spec_module.can_invoke_intent) and claim guards (self._run_claim_guards), then either calls a service claim (self.task.pr_gate_claim / pr_review_claim — claim WITHOUT transition for the gate, pending→in_progress for external) or the verb runner (self._verb_runner().run_intent) for the transition. Gate verdicts are authored as structured pr_review notes (apply_structured_note) BEFORE the transition and posted to the GitHub PR AFTER (self.git.post_pr_review); pr_fail also captures the PR head SHA (self.git.get_pr_head_sha) and a2a's the owning PM (self.a2a.send). Outputs: standardized Envelope (ok with status/next/evidence/context_briefing, or error with remediate). Callers: the flow verb dispatcher + HTTP routes /api/v1/flow/pr_reviewer/*. Callees: TaskService (get/agent_for/pr_gate_claim/pr_review_claim), JournalService (has_learning_for_task), GitService (diff/get_pr_diff/get_pr_head_sha/post_pr_review), NotificationService, A2AService, ProjectService/ProductService (via resolve_task_project_slug), foundation policy (lifecycle.can_invoke_intent, tracing.check_requirements, content.validate_content/markers). + +## Mermaid +```mermaid +stateDiagram-v2 + direction LR + [*] --> awaiting_pr_review: PM submit_up/submit_root + awaiting_pr_review --> claimed_gate: claim_gate_review (no transition) + claimed_gate --> awaiting_pm_review: pr_pass + claimed_gate --> needs_revision: pr_fail (captures head_sha + a2a PM) + needs_revision --> awaiting_pr_review: PM re-submit (unchanged-PR gate hard-blocks if head_sha identical) + awaiting_pm_review --> [*]: PM merge/escalate + + state claim_gate_review_evidence [\"diff inline (read-only)\"] + state pr_pass_blocked [\"toolchain_broken? conventions block? -> refuse pass\"] + state post_verdict [\"record pr_review note + post to PR (COMMENT on root→master) + a2a on fail\"] + + direction TB + [*] --> pending_ext: external/fork PR review task + pending_ext --> in_progress_ext: claim_pr_review (pending→in_progress, branch-exempt) + in_progress_ext --> completed: post_pr_review (in_progress→completed) + state post_pr_review_gates [\"verdict consistency + no-hand-format + tracing\"] + state post_side [\"GitHub review post + CEO notify (best-effort)\"] +``` + +## Logical Tree +``` +pr-gate-review slice +├── PRGateMixin (in-path assembled-PR gate) +│ ├── claim_gate_review — claim without transition + assembled diff evidence +│ ├── pr_pass — awaiting_pr_review → awaiting_pm_review +│ ├── pr_fail — awaiting_pr_review → needs_revision (issues required) +│ └── helpers +│ ├── _gate_decision — shared body (preflight→tracing→blocked→record→run_intent→None-guard→post→a2a) +│ ├── _gate_preflight — ownership/role/spec-gate (self_review_block) + soup guard +│ ├── _gate_tracing — journal:learning + pr_reviewer_notes min chars +│ ├── _pr_pass_blocked — toolchain-broken + conventions block + CI-status guards (returns rejection, ci_note) +│ ├── _ci_status_guard — refuse pr_pass on failing/pending/unscheduled CI; config gaps + unreachable repos pass through with evidence stamp; real API failures stay fail-closed +│ ├── _resolve_ci_status — thin wrapper calling git.get_pr_ci_status, interprets result dict, returns rejection Envelope if CI must block +│ ├── _record_gate_verdict_for / _record_gate_verdict — structured pr_review note (+ issues + head_sha + ci_status) +│ ├── _re_stamp_pr_fail_head_sha_if_advanced — re-capture head SHA post-transition and re-stamp verdict note if advanced (#189) +│ ├── _capture_pr_head_sha — best-effort PR head SHA for unchanged-PR gate +│ ├── _post_gate_review / _post_gate_review_to_pr — PR review post (COMMENT on root→master or MegaTask root-subtask) +│ ├── _gate_review_event_verdict / _gate_review_body — static helpers for post_gate_review_to_pr (extracted in 536bbb64) +│ ├── _deliver_pr_fail_to_owner — a2a change-requests to owning PM (+ Main-PM-root steer) +│ ├── _gate_role_or_rejection — role enum parse +│ └── _build_gate_review_evidence — assembled diff + AC +└── PRReviewerMixin (inbound external/fork PR review) + ├── claim_pr_review — pending→in_progress (branch-exempt) + read-only diff evidence + ├── post_pr_review — in_progress→completed, one change-request + └── helpers + ├── _post_pr_review_preflight — non-empty body/role/spec-gate/tracing + ├── _post_pr_review_content_gates — verdict consistency + no-hand-format + ├── _verdict_consistency_gate — pr_review_conflict pure invariant + ├── _is_hand_formatted_verdict — detect ## headers in free-text body + ├── _resolve_post_body — canonical render vs free-text + ├── _build_pr_review_content — validate_content into PrReviewContent + ├── _post_review_side_effects — GitHub post + CEO notify + ├── _pr_review_tracing_gate — journal:learning + notes min chars + ├── _resolve_role / _runner_failure / _build_pr_review_evidence + ├── _project_slug_for — delegates to module resolver + └── resolve_task_project_slug (module-level, shared with _impl.py) — project_id → product → cell_projects +``` + +## Entry Points + +| Name | File | Trigger | +|---|---|---| +| claim_gate_review | roboco/services/gateway/choreographer/pr_gate.py | pr_reviewer agent calls flow verb claim_gate_review(task_id) via roboco-flow MCP / POST /api/v1/flow/pr_reviewer/claim_gate_review on an awaiting_pr_review assembled-PR task | +| pr_pass | roboco/services/gateway/choreographer/pr_gate.py | pr_reviewer calls pr_pass(task_id, notes) via roboco-flow / HTTP route after claim_gate_review | +| pr_fail | roboco/services/gateway/choreographer/pr_gate.py | pr_reviewer calls pr_fail(task_id, issues=[...]) via roboco-flow / HTTP route after claim_gate_review | +| claim_pr_review | roboco/services/gateway/choreographer/pr_review.py | pr_reviewer calls claim_pr_review(task_id) via roboco-flow / HTTP route on a pending external/fork-PR review task | +| post_pr_review | roboco/services/gateway/choreographer/pr_review.py | pr_reviewer calls post_pr_review(task_id, body, event, findings?) via roboco-flow / HTTP route to post one change-request and complete | + +## Config Flags +- ROBOCO_TOOLCHAIN_MATCH_ENABLED (gates _toolchain_broken_guard in _pr_pass_blocked — inert when off) +- ROBOCO_CONVENTIONS_ENABLED (gates _conventions_guard in _pr_pass_blocked — inert when off) +- ROBOCO_PR_REVIEWER_NOTES_MIN_CHARS / settings.pr_reviewer_notes_min_chars (tracing gate substantive-note threshold for pr_pass/pr_fail/post_pr_review) +- CI-status guard is always armed when the toolchain can reach get_pr_ci_status via git service. Configuration gaps (missing project/git_url/token) and unreachable/nonexistent repos (404 or network error) classify as no_ci_configured and pass through with evidence stamp. Genuine API failures on reachable repos classify as error and stay fail-closed (retryable). A project with no CI configured at all also passes through cleanly (no_ci_configured). The guard never blocks pr_pass on a misconfigured project. + + +## Gotchas +- self_review_block is dormant by design: markers.get_original_developer is never set on assembled coordination tasks (only on dev-leaf tasks at QA/doc claim), and GatewayAgentView carries no slug so actor_slug was previously always None. The fix sets actor_slug=str(reviewer_agent_id) so the gate is wired, but it only fires if the marker were ever set to the reviewer's UUID — currently never. Don't assume the self-review defense is active in production today. +- pr_fail captures the PR head SHA BEFORE the DB transition commits (_record_gate_verdict_for runs before run_intent). If the branch advances between capture and transition the recorded SHA is stale, but the unchanged-PR gate in submit_root fails open on stale/missing SHA — only the exact-unchanged case is hard-blocked. +- _gate_decision guards t is None after run_intent: a concurrent cancel or racing reviewer between the precondition gate and the runner's final action makes run_intent return None; without this guard the post-PR/a2a dereferences would crash. Any future reorder must preserve this check. +- _post_gate_review_to_pr always posts COMMENT (not APPROVE/REQUEST_CHANGES) on a root→master PR because only the CEO merges master. A root→master PR is now identified by `is_root = parent_task_id is None OR is_batch_root_subtask(batch_id, parent_task_id)` — so a MegaTask root-subtask (which has a parent = the umbrella but opens its own root→master PR) also gets COMMENT, not APPROVE. A non-batch cell-PM coordination root keeps batch_id=None so it remains a cell→root PR (APPROVE/REQUEST_CHANGES). Added in f90565ea. +- resolve_task_project_slug cell_projects branch sorts by m.team.value — assumes every cell_map mapping has a non-None team with a .value; a malformed mapping would raise AttributeError (uncaught) and bubble out of the slug resolver (which callers tolerate as best-effort None only if wrapped — _capture_pr_head_sha wraps it, _post_gate_review_to_pr does NOT wrap the slug call). +- _is_hand_formatted_verdict (UPDATED in 536bbb64 #188): previously a plain lower-case substring match that would false-refuse a body quoting a PR's own ## headers (e.g. `> ## Summary`); now uses a regex anchored to line-start (`^[ \t]*## ...`, re.MULTILINE) so a quoted/indented header or a mid-prose mention does not trip the guard. The remaining false-positive window: a reviewer deliberately writing `## Summary` at the start of a line in their free-text body (with findings=[]) — intentionally refused, steering them to the structured-findings path. +- _record_gate_verdict for pr_fail with issues now writes a templated summary ('In-path PR-review gate requested changes - N issue(s) listed below.') instead of the full notes into the structured note's summary field; the full issues text lives in the issues slot. The GitHub PR post and a2a still use the raw notes string. Readers of notes_structured.pr_review.summary no longer get the verbatim issues. +- (Revision-findings ledger, uncommitted branch `feature/findings-ledger`) `PrReviewContent.findings` was previously hardcoded to `[]` on every `_gate_verdict_payload` write — it now carries the real validated `Finding` list on a findings-driven `pr_fail`, the first time this slot is ever non-empty. `claim_gate_review`'s evidence (`_build_gate_review_evidence`) additionally carries `revision_findings` (open) and, on a round ≥2 review, `prior_findings` (the full ledger) so a re-reviewing reviewer checks prior findings instead of re-deriving them. `pr_pass` now bulk-verifies (`addressed→verified`) every `origin=pr_gate` finding in the same transaction via `_stamp_gate_findings_verified_or_rejection` — a stamp failure rejects the pass outright, it is NOT best-effort. Full detail: `docs/map/review-findings.md`. +- claim_gate_review does NOT transition the task (status stays awaiting_pr_review) — this is intentional so pr_pass/pr_fail's source-status still matches. A reviewer who claims but never decides leaves the task assigned but still awaiting_pr_review; the stale-claim reaper path is the recovery. +- PRReviewerMixin.post_pr_review runs content gates BEFORE _resolve_post_body, but _resolve_post_body itself can return an Envelope (malformed findings) which is then handled — the verdict_consistency_gate already ran on the (event, findings) pair, so a malformed-findings Envelope is a distinct later failure. +- resolve_task_project_slug was extracted to module-level specifically so _impl.py's _LegacyChoreographer (which does NOT inherit ChoreographerHelpers) can reach it; the mixin method _project_slug_for is now a one-line delegate. Changing the resolver signature would break both the mixin and the _impl.py unchanged-PR gate. +- Before `_gate_diff_parent` (#444/#454), the gate's evidence diff and the conventions guard derived their base via `parent_branch_for` string surgery on the child's OWN branch name — correct for a same-team hop (cell dev → cell PM) but wrong for a cross-team hop (a frontend child of a main_pm root derives a ref that never existed) and silently fell back to the repo default branch, so the reviewer judged inherited base-branch content as the task's own work. `_gate_diff_parent` is now consulted only when no explicit `base` is given, so the pinned literal-base contract (`base="HEAD~1"`) and every other diff caller (QA/doc/content) are untouched; it is skipped entirely while `conventions_enabled` is off since only the conventions guard consumes it in `_pr_pass_blocked`. + + +## Drift from CLAUDE.md +- CLAUDE.md verb table lists pr_reviewer verbs as 'claim_pr_review, post_pr_review (inbound external/fork PRs), claim_gate_review, pr_pass, pr_fail (in-path assembled-PR gate)' — matches the code exactly. No drift. +- CLAUDE.md says the pr_reviewer 'posts its change-request on the PR itself (no agent comms)'. The code now ALSO a2a's pr_fail change-requests to the owning PM (_deliver_pr_fail_to_owner) — an additive agent-comms side effect NOT reflected in CLAUDE.md's 'no agent comms' claim for the in-path gate. This is intentional (closes the blind re-submit loop) but the doc still says no agent comms. +- CLAUDE.md does not mention the pr_fail head_sha capture / unchanged-PR submit_root gate (the 2026-06-27 pr_fail loop fix) anywhere — it's a live behavioral guarantee absent from the doc. + + +## Changes Since Baseline + +| SHA | Subject | Impact | +|---|---|---| +| 15effce0 | Chore: 141 Gaps fill-in (#283) — pr_gate.py: wire self_review_block for pr_pass/pr_fail | _gate_preflight spec_ctx now passes actor_slug=str(reviewer_agent_id) and original_developer_slug=markers.get_original_developer(t) instead of agent.slug (always None for GatewayAgentView). Self-review defense is now wired, though dormant because the marker is never set on assembled coordination tasks. | +| 15effce0 | Chore: 141 Gaps fill-in (#283) — pr_gate.py: capture pr_fail head_sha into verdict note | New _record_gate_verdict_for + _capture_pr_head_sha stamp the assembled PR head SHA into notes_structured.pr_review.head_sha on pr_fail, feeding the submit_root unchanged-PR hard-block gate. Fail-open on any capture failure. | +| 15effce0 | Chore: 141 Gaps fill-in (#283) — pr_gate.py: a2a pr_fail to owning PM + Main-PM-root steer | New _deliver_pr_fail_to_owner sends the pr_fail change-requests to the assigned PM via a2a (best-effort) with a steer for Main-PM branch-bearing roots to re-delegate not re-submit. Closes the blind re-submit loop (PR #138). | +| 15effce0 | Chore: 141 Gaps fill-in (#283) — pr_gate.py: None-guard after run_intent for concurrent transition | _gate_decision now checks t is None after runner.run_intent and returns a clean invalid_state envelope instead of dereffing None → 500. Covers concurrent cancel / racing reviewer between precondition gate and final action. | +| 15effce0 | Chore: 141 Gaps fill-in (#283) — pr_gate.py: _toolchain_broken_guard now reviewer=True | _pr_pass_blocked passes reviewer=True to _toolchain_broken_guard (signature widened to distinguish reviewer context). pr_pass still refused on broken toolchain; pr_fail unaffected. | +| 15effce0 | Chore: 141 Gaps fill-in (#283) — pr_gate.py: structured verdict note carries issues + head_sha | _record_gate_verdict payload now includes issues list for pr_fail and head_sha; summary for pr_fail-with-issues is a templated sentence instead of the full notes (dedup on the Task Details card). | +| 15effce0 | Chore: 141 Gaps fill-in (#283) — pr_review.py: extract module-level resolve_task_project_slug + cell_projects fallback | _project_slug_for delegates to new module-level resolve_task_project_slug (shared with _impl.py unchanged-PR gate); adds a third fallback branch for ad-hoc per-cell-map root-subtasks (migration 052) so the gate verdict reaches the PR in the mapped repo. | +| 15effce0 | Chore: 141 Gaps fill-in (#283) — pr_review.py: hand-formatted-verdict body guard | New _is_hand_formatted_verdict + _post_pr_review_content_gates refuse a free-text body carrying ## summary/issues/verdict/findings headers when findings is empty, steering the reviewer to the structured-findings path. Observed live: a duplicated self-formatted verdict posted to a contributor's PR. | +| 15effce0 | Chore: 141 Gaps fill-in (#283) — pr_review.py: fold content gates into one helper | post_pr_review now calls _post_pr_review_content_gates (verdict consistency + no-hand-format) instead of only _verdict_consistency_gate; keeps the verb body under the return-count lint ceiling. | + +> Post-snapshot updates (since 2026-06-29): two commits touched this slice. +> - **536bbb64** (Chore/all/logical gaps sweep #286, 2026-06-30): pr_gate.py — (a) `_post_gate_review_to_pr` wraps slug-resolution in try/except so a malformed cell_map mapping can no longer 500 the reviewer after a committed gate transition (#82 FIXED); (b) `_is_hand_formatted_verdict` regex anchored to line-start so quoted/indented PR headers no longer false-refuse post_pr_review (#188 FIXED); (c) `_re_stamp_pr_fail_head_sha_if_advanced` new method — re-captures head SHA after the transition commits and re-stamps the verdict note only if it advanced, closing the stale-SHA false-allow window (#189 FIXED); (d) `claim_gate_review` passes `skip_dev_guards=True` to `_run_claim_guards` so already_active/paused/lane guards never block a pr_reviewer from claiming a gate review (#192 FIXED); (e) two new static helpers extracted from `_post_gate_review_to_pr`: `_gate_review_event_verdict` and `_gate_review_body`. The unchanged-PR guard's fail-open slug/git error now logs a warning so a regression cannot silently disable the loop-stopper (#5/#222). +> - **f90565ea** ([sweep] pr_gate: classify MegaTask root-subtask as root #608, 2026-06-30): `_post_gate_review_to_pr` now uses `is_batch_root_subtask` (imported from `roboco.foundation.policy.batch`) in addition to `parent_task_id is None` to identify root→master PRs. A MegaTask root-subtask (parent=umbrella, batch_id set) opens its own root→master PR but previously got APPROVE/REQUEST_CHANGES instead of COMMENT — fix prevents a single-approval branch-protection rule from allowing a non-CEO merge. +> - **7ff70ab5** (fix(gateway): gate review diffs against the task's real parent branch #444/#454, 2026-07-10): new `_gate_diff_parent` + `preferred_parent` param threaded through `git.diff` / `list_changed_files` / `conventions_check_for_task`, resolving the assembled task's real parent branch from the parent TASK's own `branch_name` instead of string surgery on the child branch name — fixes a live cross-team false-fail (bounced a goals-tab fix three times) where the gate attributed inherited base-branch content to the task under review. +> - (uncommitted, branch `feature/findings-ledger`, 2026-07-11) Revision-findings ledger: `pr_fail` gains structured `findings` (validated + count-guarded + inserted to the ledger before the verdict note), `pr_pass` gains a same-transaction verify-stamp, and gate evidence gains `revision_findings`/`prior_findings`. See `docs/map/review-findings.md`. + +## Regression Risks + +| Title | File:Line | Claim | Severity | +|---|---|---|---| +| CI-status guard reads GitHub check-runs only, not legacy commit-status API | roboco/services/gateway/choreographer/pr_gate.py:520 | _ci_status_guard and get_pr_ci_status read only the check-runs API endpoint. A repo whose only CI signal is the legacy commit-status API would show zero check-runs and be classified as no_ci_configured (passes through). Noted as ponytail-comment in git.py with the upgrade path if a project ever needs it. | low | +| CI-status classification: config gaps and unreachable repos now explicitly no_ci_configured | roboco/services/git.py:_resolve_ci_head_sha | Missing project/git_url/git-token, or unreachable/nonexistent repo (404 on PR head lookup or network error), all classify as no_ci_configured → pr_pass passes through with evidence stamp. Only genuine GitHub API failures on reachable repos classify as error → pr_pass stays fail-closed (retryable). By design: configuration gaps should not block the gate, but real API failures should fail-closed to avoid false-green verdicts. A deliberately misconfigured project's CI is silently not enforced, with clear evidence in the verdict note. | low | +|---|---|---|---| +| self_review_block could fire if a reviewer is also the original developer | roboco/services/gateway/choreographer/pr_gate.py:202 | actor_slug=str(reviewer_agent_id) + original_developer_slug=markers.get_original_developer(t). The comment asserts dormancy because the marker is never set on assembled coordination tasks. If a future change sets the marker on an assembled task (or a reviewer UUID coincides with the recorded dev UUID), pr_pass/pr_fail would be refused as self-review with no remediate path. The defense is correctly wired but unguarded by a test asserting dormancy. | low | +| ~~resolve_task_project_slug cell_projects branch can raise AttributeError on malformed mapping~~ **FIXED 536bbb64 #82** | roboco/services/gateway/choreographer/pr_review.py:594 | ~~sorted(cell_map, key=lambda m: m.team.value) assumes every mapping has a non-None team with .value. _capture_pr_head_sha wraps the slug call in try/except (fail-open), but _post_gate_review_to_pr calls self._project_slug_for(t) WITHOUT a try/except — a malformed cell_map mapping would raise and abort the verdict PR post (best-effort but the exception escapes the helper, caught only by the outer try in _post_gate_review_to_pr's git.post_pr_review call, NOT the slug resolution).~~ _post_gate_review_to_pr now wraps the slug-resolve call in its own try/except (mirrors _capture_pr_head_sha) — a malformed mapping logs and returns, no longer 500s the reviewer after the committed gate transition. The underlying AttributeError possibility in resolve_task_project_slug remains but is contained. | medium | +| ~~_is_hand_formatted_verdict false-positive on summaries quoting PR-added headers~~ **FIXED 536bbb64 #188** | roboco/services/gateway/choreographer/pr_review.py:174 | ~~Substring match on '## summary'/'## issues'/'## verdict'/'## findings' in lowercased body. A reviewer summarizing a PR that itself adds a '## Summary' section (quoting it in the body) with findings=[] would be falsely refused.~~ Regex now anchored to line-start (^[ \t]*## ..., re.MULTILINE) — quoted headers (> ## Summary) and mid-prose mentions no longer trip the guard. | low | +| ~~pr_fail head_sha captured before transition may be stale vs the committed verdict~~ **FIXED 536bbb64 #189** | roboco/services/gateway/choreographer/pr_gate.py:240 | ~~_record_gate_verdict_for awaits _capture_pr_head_sha (GitHub pulls API) then writes the note, all before run_intent commits the transition. If the assembled PR advances between capture and the transition commit, the recorded SHA no longer matches the PR head at the moment of needs_revision.~~ New `_re_stamp_pr_fail_head_sha_if_advanced` re-captures the SHA after run_intent commits and re-stamps the note only if it advanced; no-advance is a single write. Fail-open: a re-capture failure leaves the pre-transition SHA in place. | low | +| Structured pr_review summary no longer contains verbatim issues for pr_fail | roboco/services/gateway/choreographer/pr_gate.py:444 | _record_gate_verdict now writes a templated summary for pr_fail-with-issues instead of the full notes. Any consumer that parsed notes_structured.pr_review.summary for the change-request text (rather than .issues) now gets a generic sentence. The a2a body and GitHub PR post still use raw notes, but briefing/mirror readers of the summary field lose the verbatim issues. | low | +| _deliver_pr_fail_to_owner a2a to assigned PM may target the wrong agent after a reassign | roboco/services/gateway/choreographer/pr_gate.py:267 | a2a.send to_agent=t.assigned_to at the moment pr_fail runs. If the task was reassigned between claim_gate_review and pr_fail, the change-requests go to the new assignee, not the reviewer who claimed it. Best-effort and the assigned PM is the intended recipient, but a just-reassigned PM with no context receives raw review issues. | low | +| ~~claim_gate_review runs _run_claim_guards but the gate task is not a normal claim~~ **FIXED 536bbb64 #192** | roboco/services/gateway/choreographer/pr_gate.py:84 | ~~_run_claim_guards is invoked for claim_gate_review (which does NOT transition). If a claim guard (e.g. already_active / lane barrier) rejects, the reviewer cannot claim the gate review.~~ `_run_claim_guards` is now called with `skip_dev_guards=True` — the already_active, paused, and lane barriers are skipped for claim_gate_review; only the dependency guard is kept. A pr_reviewer with another active task is no longer blocked from claiming a gate review. | low | + +## Health +The slice is well-structured and defensively hardened. Both mixins follow the established choreographer pattern (TYPE_CHECKING-only base, spec gate + tracing gate + verb runner, best-effort side-effects after the DB transition, standardized Envelopes). The 15effce0 changes are coherent: the pr_fail loop is closed at three layers (head_sha capture + submit_root hard-block, a2a to owning PM, Main-PM-root steer), the concurrent-transition None-guard plugs a real crash, and the external-PR hand-format guard addresses an observed live defect. Post-snapshot (536bbb64 + f90565ea) hardening: the slug-resolution AttributeError in _post_gate_review_to_pr is now contained by try/except (#82 FIXED), the hand-format guard is now regex-anchored-to-line-start instead of substring (#188 FIXED), the stale-SHA false-allow window is closed by the post-transition re-stamp (#189 FIXED), the pr_reviewer active-task guard is skipped for claim_gate_review (#192 FIXED), and MegaTask root-subtasks correctly get COMMENT on their root→master PR. The main remaining latent concern is the self_review_block dormancy (correctly wired, no test asserting dormancy — low, no known path to activate). Coverage and tracing parity with QA's pass_review/fail_review is maintained. + +## Purpose +This slice is the prompt-composition pipeline and the role/team/permission taxonomy that feeds it. At agent spawn the orchestrator resolves an agent's role+team from the canonical foundation-derived maps in agents_config.py, then compose_prompt layers (in order) a tool-load directive, the autogenerated lifecycle verb surface, the universal base rules, an optional fable-mode doctrine layer (`fable_mode_enabled`), an optional ponytail build-laziness doctrine layer (bundled with Fable, role-scoped — developers get the full ladder, other roles get the ethos-only cut), the role prompt, the autogenerated per-role verb-signature table, the team prompt, the agent identity, and an optional architectural-conventions ambient block — writing the result to a per-agent .md file the runtime mounts. A separate prompt-injection guard (prompt_guard.py) denies poisoned incoming turns at the interactive input boundary for both Claude-SDK and Grok sessions, mirroring the bash UserPromptSubmit hook. agents_config.py is also the MCP-layer permission taxonomy (HMAC agent tokens, role/team helpers, escalation chain, A2A routing) that gates tool visibility and identity-binding at spawn. + +## Files + +| Path | Role | LOC | +|---|---|---| +| roboco/agents/factories/_base.py | Layered prompt composer: loads/concatenates tool-directive + lifecycle + base + an optional fable-mode doctrine + an optional ponytail build-laziness doctrine + role + autogen-verbs + team + identity + ambient layers; exports PROMPTS_BASE_PATH, role/team/builtin-tool maps, compose_prompt, fable_doctrine_layer, ponytail_doctrine_layer, conventions_ambient_layer, make_slug | 298 | +| roboco/agents/factories/__init__.py | One-line re-export shim pointing to _base | 1 | +| roboco/agents_config.py | 691-line MCP-layer permission/taxonomy module: HMAC agent-token issue/verify, role+team+cell maps derived from foundation, escalation chain, ROLE_PERMISSION_LEVELS, ROLE_SKILLS, A2A routing helpers | 691 | +| roboco/agent_sdk/prompt_guard.py | Reusable Python port of the bash injection guard: _PATTERNS regex list, detect_injection, refusal_message, CLI main (exit 1 on injection) for the grok entrypoint | 93 | +| agents/prompts/base.md | Universal base layer: identity separation, gateway-verb-only action, envelope shapes, missing-key cheatsheet, resume-from-briefing, charter alignment, todo rules, ground rules | 93 | +| agents/prompts/doctrine/fable.md | Vendored Fable-5 behavioral doctrine (from `github.com/rennf93/opus-fable-playbook`, MIT, YAML frontmatter stripped): communication/turn-discipline/autonomy-calibration/honesty/code-discipline/delegation/precedence sections; loaded only when `fable_mode_enabled`, injected right after base.md via `fable_doctrine_layer()` | 47 | +| agents/prompts/doctrine/ponytail.md | Vendored Ponytail build-laziness doctrine for developers (from the ponytail plugin, MIT, Copyright (c) 2026 DietrichGebert, trimmed, YAML frontmatter stripped): the ladder (YAGNI → reuse-in-this-codebase → stdlib → native-platform → installed-dep → one-line → minimal), the rules, the Intensity table (lite/full/ultra), a 5-point RoboCo preamble that makes the ladder yield to placement / coverage gate / design bar / task hygiene / reviewer feedback, and the `ponytail:` comment convention; loaded only when `fable_mode_enabled`, injected right after the Fable doctrine via `ponytail_doctrine_layer()` for `AgentRole.DEVELOPER` only, with a trailing `**Operative intensity: {ponytail_intensity}.**` directive | 88 | +| agents/prompts/doctrine/ponytail-ethos.md | Vendored Ponytail ethos-only doctrine for non-developer roles (same source/attribution, trimmed): the ethos rules and the RoboCo preamble (the 6th point guards free-text field obligations), with the code-mechanics rungs (the ladder) and the Intensity table removed so they can't leak into prose artifacts; loaded by `ponytail_doctrine_layer()` for every role except `DEVELOPER`; no intensity directive (ethos runs a fixed restrained stance) | 40 | +| agents/prompts/roles/developer.md | Developer role prompt: implement-only identity, verb table (give_me_work/i_will_work_on/commit/open_pr/i_am_done/sync_branch/...), workspace path, behind-base -> sync_branch guidance, conventions waiver path, a pointer scoping frontend/ux_ui's `## Design bar` to those teams only | 25176 | +| agents/prompts/roles/qa.md | QA role prompt: review-only identity, claim_review/pass/fail/i_am_blocked verbs, ac_verdicts per criterion, circuit-breaker guidance | 13355 | +| agents/prompts/roles/documenter.md | Documenter role prompt: docs-on-same-branch identity, claim_doc_task/commit/i_documented/i_am_blocked verbs, circuit-breaker guidance | 10739 | +| agents/prompts/roles/cell_pm.md | Cell PM role prompt: coordinator identity, i_will_plan/delegate/complete/submit_up/unblock verbs, AC coverage gate, collision-surface declaration (intends_to_touch/adds_migration/touches_shared/depends_on), behind-base escalation | 35682 | +| agents/prompts/roles/main_pm.md | Main PM role prompt: org-level coordinator, delegate/complete/submit_root/triage_all/unblock/escalate_to_ceo verbs, upstream-handoff precondition, branch-bearing vs branchless root gate | 32363 | +| agents/prompts/roles/pr_reviewer.md | PR Reviewer role prompt: external-PR review (claim_pr_review/post_pr_review) + in-path gate (claim_gate_review/pr_pass/pr_fail), trust gate, conventions strictness | 8129 | +| agents/prompts/roles/board.md | Board role prompt (PO/HoM/Auditor): strategic overseer, triage/escalate_to_ceo, no unblock verb, Auditor silent | 12253 | +| agents/prompts/roles/prompter.md | Intake interviewer role prompt: CEO-only chat, propose_draft/propose_batch, MegaTask batch + per-cell-project-map drafting, root-subtask coordination-level AC guidance | 13825 | +| agents/prompts/roles/secretary.md | Secretary role prompt: CEO chief-of-staff, gated-action confirm protocol, reading-free/prepare-direct/high-impact-bounce discipline | 4338 | +| agents/prompts/teams/backend.md | Backend team layer: Python/FastAPI/Postgres stack, teammates, uv quality commands | 983 | +| agents/prompts/teams/frontend.md | Frontend team layer: TS/Next.js stack, pnpm quality commands, `## Design bar` section (taste-skill-distilled layout/typography/motion/spacing rules + 3 tuning dials + an "AI tells to avoid" list) | 976 | +| agents/prompts/teams/ux_ui.md | UX/UI team layer: design-system focus areas, teammates, `## Design bar` section (same taste-skill basis as frontend.md, plus a design-artifact-to-code handoff bullet), plus a video-mode override line: video-authoring (`source=video`) tasks are FILMS not UI, so the section's web dials (DESIGN_VARIANCE/MOTION_INTENSITY/VISUAL_DENSITY) do not apply — use `motion/README.md`'s cinematography bar and the vendored `motion/skills/` doctrine instead | 1009 | +| agents/prompts/identities/ | 19 per-agent identity files (be-dev-1/2, fe-dev-1/2, ux-dev-1/2, be/fe/ux -qa/-doc/-pm, main-pm, product-owner, head-marketing, auditor): YAML id/name/role/team/cell/reports_to + scope blurb; loaded by slug | 0 | +| agents/prompts/_generated/lifecycle-developer.md | Autogenerated lifecycle verb list for developer (regenerated from lifecycle spec by make lifecycle); lists sync_branch etc. | 1498 | +| agents/prompts/_generated/lifecycle-main_pm.md | Autogenerated lifecycle verbs for main_pm; submit_root now branch-keyed not task_type-keyed | 2034 | +| agents/prompts/_generated/lifecycle-cell_pm.md | Autogenerated lifecycle verbs for cell_pm | 1839 | +| agents/prompts/_generated/lifecycle-qa.md | Autogenerated lifecycle verbs for qa | 818 | +| agents/prompts/_generated/lifecycle-documenter.md | Autogenerated lifecycle verbs for documenter | 736 | +| agents/prompts/_generated/lifecycle-pr_reviewer.md | Autogenerated lifecycle verbs for pr_reviewer | 1071 | +| agents/prompts/_generated/lifecycle-product_owner.md | Autogenerated lifecycle verbs for product_owner | 421 | +| agents/prompts/_generated/lifecycle-head_marketing.md | Autogenerated lifecycle verbs for head_marketing | 422 | +| agents/prompts/_generated/lifecycle-auditor.md | Autogenerated lifecycle verbs for auditor (triage + i_am_idle) | 325 | +| agents/prompts/_generated/lifecycle-prompter.md | Autogenerated lifecycle verbs for prompter (i_am_idle only — driver-based) | 275 | +| agents/prompts/_generated/lifecycle-secretary.md | Autogenerated lifecycle verbs for secretary (i_am_idle only — driver-based) | 276 | +| agents/prompts/_generated/lifecycle-ceo.md | Autogenerated lifecycle verbs for ceo (empty — human, not spawned) | 181 | +| agents/prompts/_generated/lifecycle-system.md | Autogenerated lifecycle verbs for system sentinel (empty) | 184 | +| agents/prompts/_generated/developer.md | Per-role autogenerated verb-signature table (Flow + Content tools) for developer, regenerated by scripts/regenerate_verb_tables.py from Pydantic schemas + role_config | 2384 | +| agents/prompts/_generated/qa.md | Per-role autogenerated verb-signature table for qa (pass_review ac_verdicts carries BeforeValidator) | 2040 | +| agents/prompts/_generated/documenter.md | Per-role autogenerated verb-signature table for documenter | 2083 | +| agents/prompts/_generated/cell_pm.md | Per-role autogenerated verb-signature table for cell_pm; delegate signature includes intends_to_touch/adds_migration/touches_shared/depends_on | 3270 | +| agents/prompts/_generated/main_pm.md | Per-role autogenerated verb-signature table for main_pm; delegate signature includes collision-surface fields | 3316 | +| agents/prompts/_generated/pr_reviewer.md | Per-role autogenerated verb-signature table for pr_reviewer (claim_gate_review/pr_pass/pr_fail + claim_pr_review/post_pr_review) | 1461 | +| agents/prompts/_generated/product_owner.md | Per-role autogenerated verb-signature table for product_owner (triage/escalate_to_ceo + pitch/notify) | 1765 | +| agents/prompts/_generated/head_marketing.md | Per-role autogenerated verb-signature table for head_marketing | 1765 | +| agents/prompts/_generated/auditor.md | Per-role autogenerated verb-signature table for auditor (triage/i_am_idle + note/evidence + approve/reject/archive_playbook) | 1273 | +| agents/prompts/_generated/verbs.md | Aggregate reference doc of all per-role verb shapes (NOT injected at spawn; _base.py loads the per-role file instead); notes driver-based roles omitted | 18302 | + +## Key Symbols + +| Name | Kind | File:Line | Responsibility | +|---|---|---|---| +| _get_prompts_base_path | function | roboco/agents/factories/_base.py:17 | Resolve project_root/agents/prompts/ from this file's location with a cwd-relative fallback | +| PROMPTS_BASE_PATH | constant | roboco/agents/factories/_base.py:37 | Module-level cached prompts base path used by default in compose_prompt | +| _load_layer | function | roboco/agents/factories/_base.py:40 | Read a prompt layer file, return '' if missing (graceful fallback) | +| _ROLE_LAYER_MAP | dict | roboco/agents/factories/_base.py:55 | Maps role string -> roles/*.md filename; board roles all share board.md; prompter/secretary/pr_reviewer have own files | +| _TEAM_LAYER_MAP | dict | roboco/agents/factories/_base.py:77 | Maps team string (backend/frontend/ux_ui) -> teams/*.md filename | +| _role_layer | function | roboco/agents/factories/_base.py:84 | Load the role-specific prompt layer or None if role unknown | +| _team_layer | function | roboco/agents/factories/_base.py:93 | Load the team prompt layer or None if unset/unknown | +| _autogen_verbs_layer | function | roboco/agents/factories/_base.py:104 | Load _generated/.md autogenerated verb-signature table for the role | +| _BUILTIN_TOOLS_COMMON | tuple | roboco/agents/factories/_base.py:127 | Built-in Claude Code tools every role gets: Read,Bash,Grep,Glob,TodoWrite | +| _BUILTIN_TOOLS_AUTHORS | tuple | roboco/agents/factories/_base.py:134 | Authors set (developer/documenter) adds Edit,Write to the common set | +| _ROLE_BUILTIN_TOOLS | dict | roboco/agents/factories/_base.py:136 | Per-role builtin-tool grant map; non-authors get common-only | +| _tool_load_directive_layer | function | roboco/agents/factories/_base.py:149 | Build top-of-prompt 'your tools are ready' block; steers away from ToolSearch and shell-redirect rewrites | +| _lifecycle_layer | function | roboco/agents/factories/_base.py:187 | Load _generated/lifecycle-.md canonical verb-surface fragment (from lifecycle spec, CI-gated) | +| fable_doctrine_layer | function | roboco/agents/factories/_base.py:203 | Return the vendored `doctrine/fable.md` doctrine text, or None when `fable_mode_enabled` is off / the file is missing; only caller is compose_prompt, inserted right after base.md | +| ponytail_doctrine_layer | function | roboco/agents/factories/_base.py | Return the vendored Ponytail build-laziness doctrine, role-scoped and gated on the same `fable_mode_enabled` flag (no separate flag — ponytail is Fable's complementary build-doctrine). Developers → `doctrine/ponytail.md` (full ladder) with a trailing `**Operative intensity: {settings.ponytail_intensity}.**` directive; every other role → `doctrine/ponytail-ethos.md` (ethos-only, no dial). None when the flag is off / the file is missing; only caller is compose_prompt, inserted immediately after the Fable doctrine layer | +| compose_prompt | function | roboco/agents/factories/_base.py:203 | Compose the full system prompt by concatenating tool-directive, lifecycle, base, role, autogen-verbs, team, identity, ambient layers with '---' separators, skipping empty layers | +| _AMBIENT_TOTAL_CAP | constant | roboco/agents/factories/_base.py:256 | 3000-char cap on the concatenated conventions ambient block | +| conventions_ambient_layer | async function | roboco/agents/factories/_base.py:259 | Render per-project architectural-standard ambient block(s), multi-project headed, capped; None when conventions off / no projects | +| make_slug | function | roboco/agents/factories/_base.py:296 | Lowercase + dash slug helper | +| _AUTH_SECRET_ENV | constant | roboco/agents_config.py:42 | Env var name ROBOCO_AGENT_AUTH_SECRET for the HMAC signing key | +| _auth_secret | function | roboco/agents_config.py:45 | Return HMAC secret bytes or None when unset | +| _signing_payload | function | roboco/agents_config.py:51 | Canonical lowercase stripped agent_id:role:team HMAC message | +| issue_agent_token | function | roboco/agents_config.py:61 | Mint hex HMAC-SHA256 token binding agent identity to role+team; returns UNSIGNED sentinel if secret unset | +| verify_agent_token | function | roboco/agents_config.py:79 | Constant-time HMAC verification; fail-closed on unset secret / UNSIGNED | +| issue_panel_token | function | roboco/agents_config.py:94 | Mint the CEO-identity token the panel presents (signed for CEO_AGENT_ID/ceo/empty team) | +| _UUID_TO_SLUG | dict | roboco/agents_config.py:109 | Reverse map UUID->slug from AGENT_UUIDS seeds | +| _resolve_to_slug | function | roboco/agents_config.py:114 | Resolve UUID or slug input to slug | +| AGENT_ROLE_MAP | dict | roboco/agents_config.py:127 | slug->role.value for every non-SYSTEM agent (derived from foundation.AGENTS) | +| AGENT_TEAM_MAP | dict | roboco/agents_config.py:133 | slug->team.value derived from foundation | +| CELL_MEMBERS | dict | roboco/agents_config.py:139 | team.value -> sorted slug list per cell | +| ALL_AGENTS | list | roboco/agents_config.py:146 | All agent slugs | +| BOARD_MEMBERS | list | roboco/agents_config.py:149 | product-owner, head-marketing, auditor | +| ALL_DOCS | list | roboco/agents_config.py:152 | Cross-cell documenter slugs for docs workspace perms | +| TASK_CREATOR_ROLES | frozenset | roboco/agents_config.py:159 | Roles that can call task.create (cell_pm, main_pm, product_owner, head_marketing, ceo) | +| ESCALATION_CHAIN | dict | roboco/agents_config.py:170 | slug -> escalation target slug (dev/qa/doc -> cell PM -> main-pm -> product-owner -> ceo) | +| get_agent_role | function | roboco/agents_config.py:204 | Role string for an agent (UUID or slug); 'unknown' if missing | +| get_agent_team | function | roboco/agents_config.py:210 | Team string for an agent or None | +| get_agent_cell | function | roboco/agents_config.py:216 | Alias of get_agent_team | +| get_cell_members | function | roboco/agents_config.py:221 | Slugs for a cell | +| is_pm | function | roboco/agents_config.py:226 | cell_pm or main_pm predicate | +| is_board_member | function | roboco/agents_config.py:232 | Board membership predicate by slug | +| is_management | function | roboco/agents_config.py:237 | PM/Board/CEO predicate | +| is_ceo | function | roboco/agents_config.py:250 | CEO predicate (full permission bypass) | +| can_send_notifications | function | roboco/agents_config.py:255 | Role in foundation NOTIFY_SENDER_ROLES | +| can_create_tasks | function | roboco/agents_config.py:263 | Role in TASK_CREATOR_ROLES | +| can_assign_tasks | function | roboco/agents_config.py:269 | Same set as can_create_tasks | +| _CANCEL_ROLES | set | roboco/agents_config.py:276 | Roles that may cancel (cell_pm/main_pm/product_owner/head_marketing — NOT ceo/auditor) | +| can_cancel_tasks | function | roboco/agents_config.py:284 | Role in _CANCEL_ROLES | +| get_escalation_target | function | roboco/agents_config.py:290 | Next escalation slug from ESCALATION_CHAIN | +| get_pm_for_team | function | roboco/agents_config.py:295 | Cell PM slug for a team | +| get_pm_for_agent | function | roboco/agents_config.py:305 | Responsible PM: cell PM for members, main-pm for cell PMs, product-owner for main PM | +| _slugs_for_role_set | function | roboco/agents_config.py:355 | Expand a role-set to sorted slugs honoring optional team_scope; excludes system sentinel | +| ROLE_PERMISSION_LEVELS | dict | roboco/agents_config.py:404 | Single source of truth role->permission-level (CEO/BOARD/AUDITOR/MAIN_PM/CELL_PM/CELL_MEMBER) used by PermissionService | +| VALID_NOTIFICATION_TYPES | frozenset | roboco/agents_config.py:425 | Valid NotificationType values | +| VALID_NOTIFICATION_PRIORITIES | frozenset | roboco/agents_config.py:428 | Valid NotificationPriority values | +| ROLE_SKILLS | dict | roboco/agents_config.py:438 | Role -> A2A skill descriptor list for Agent Cards | +| get_agent_skills | function | roboco/agents_config.py:566 | A2A skills for an agent by role | +| _BOARD_ROLES | frozenset | roboco/agents_config.py:584 | Foundation board roles (PO/HoM/Auditor; main_pm intentionally excluded) | +| _MAIN_PM_TARGETS | frozenset | roboco/agents_config.py:585 | Roles a main PM may A2A directly | +| _check_cell_pm_a2a | function | roboco/agents_config.py:590 | A2A permission for cell PM (own cell / other PMs / main-pm allowed; board escalated) | +| _check_cell_member_a2a | function | roboco/agents_config.py:604 | A2A permission for cell members (same-cell allowed; cross-cell via PMs) | +| _check_main_pm_a2a | function | roboco/agents_config.py:624 | A2A permission for main PM (_MAIN_PM_TARGETS allowed) | +| can_a2a_direct | function | roboco/agents_config.py:632 | (allowed, error) for direct A2A from one agent to another; routes CEO via notify, board/main_pm/cell-member via handlers; the CEO branch now consults `_check_ceo_a2a` (below) instead of an unconditional `True` | +| _check_ceo_a2a | function | roboco/agents_config.py:635 | CEO-initiated A2A target check: refuses `to_role in NO_COMMS_ROLES` (auditor/pr_reviewer/prompter/secretary — no `dm`/`read_a2a` on the manifest, so nothing on the other end could read or answer it), else allowed | +| get_a2a_route_hint | function | roboco/agents_config.py:670 | Human-readable routing hint for an A2A message | +| A2A_ALLOWED_PAIRS | constant | roboco/agents_config.py | Statically-derived (via `_compute_a2a_allowed_pairs()`, calling `can_a2a_direct` for every pair) set of legal A2A pairs — sized 88 (`ceo` group 18) after `_check_ceo_a2a` excludes no-comms roles; the panel switchboard's section matrix reads off this same computation | +| _PATTERNS | list | roboco/agent_sdk/prompt_guard.py:28 | Five (regex, reason) injection patterns: ignore-previous, role-override, fake role prefix, control-token mimicry, fake executive-order | +| detect_injection | function | roboco/agent_sdk/prompt_guard.py:63 | Return deny reason if text matches an injection pattern (lowercased), else None | +| refusal_message | function | roboco/agent_sdk/prompt_guard.py:72 | Guidance string shown on denial (mirrors bash hook text) | +| main | function | roboco/agent_sdk/prompt_guard.py:82 | CLI entry: exit 1 if argv[1] is an injection (used by grok entrypoint) | + +## Data Flow +Spawn-time composition (synchronous, per agent): orchestrator._generate_prompt(role/team/agent_id, ambient) calls agents_config.get_agent_role/get_agent_team to resolve the canonical strings from foundation-derived AGENT_ROLE_MAP/AGENT_TEAM_MAP, converts to AgentRole/Team enums, then calls compose_prompt. compose_prompt resolves prompts_path (PROMPTS_BASE_PATH = project_root/agents/prompts) and builds an ordered list: _tool_load_directive_layer(role) (inline), _lifecycle_layer (reads _generated/lifecycle-.md), base.md, _role_layer (roles/.md via _ROLE_LAYER_MAP), _autogen_verbs_layer (_generated/.md), _team_layer (teams/.md via _TEAM_LAYER_MAP, None for board/main-pm), identities/.md, then the optional ambient string. Empty/None layers are dropped; the rest are joined with "\n\n---\n\n". The composed string is written to /app/prompts-generated/-prompt.md (container) or $TMPDIR/roboco-prompts/ (host) and the path returned to the spawn path that mounts it as the agent's system prompt. + +Ambient resolution (async, best-effort): orchestrator._resolve_conventions_ambient gates on settings.conventions_enabled, opens a DB session, resolves in-scope projects (single project_slug for delivery roles, or per-cell projects from a task's product_id for PO/Intake), and calls conventions_ambient_layer -> ConventionsService.render_ambient_block per project (ensuring a read clone), multi-project-headed, capped to 3000 chars. Any exception is caught and returns None so a compose is never blocked by conventions. + +Identity binding at spawn: agents_config.issue_agent_token(agent_id, role, team) HMAC-signs the canonical lowercase agent_id:role:team with ROBOCO_AGENT_AUTH_SECRET and the orchestrator injects the token into the agent env; verify_agent_token (called server-side on X-Agent-Token headers) fail-closes on unset secret or UNSIGNED. The panel gets issue_panel_token() signed for the CEO identity. + +Injection guard (runtime, per turn): IntakeDriver.send_turn calls detect_injection(text) before sending to the model; on a match it emits an error chunk with refusal_message(reason) and returns without forwarding. The grok one-shot entrypoint runs `python -m roboco.agent_sdk.prompt_guard ` and refuses start (exit 1) on a match. The same five patterns run in docker/scripts/user-prompt-hook.sh for non-SDK Claude sessions. + +Callers: orchestrator.py:3238 (compose_prompt), orchestrator.py:3265/3299 (conventions_ambient_layer), intake_driver.py:379-382 (detect_injection/refusal_message). Callees from this slice: roboco.foundation.identity (AGENTS/Role/Team/slugs_for_team), roboco.foundation.policy.communications (NOTIFY_SENDER_ROLES), roboco.seeds.initial_data (AGENT_UUIDS/CEO_AGENT_ID), roboco.services.conventions (get_conventions_service), roboco.config.settings, roboco.models.base (NotificationType/Priority). + +## Mermaid +```mermaid +graph TD + subgraph "Spawn-time prompt composition" + O[orchestrator._generate_prompt] --> AC[agents_config.get_agent_role/team] + AC --> FOUND[foundation.identity.AGENTS] + O --> CP[compose_prompt] + CP --> TLD[_tool_load_directive_layer role] + CP --> LL[_lifecycle_layer _generated/lifecycle-role.md] + CP --> BASE[base.md] + CP --> RL[_role_layer roles/file.md] + CP --> AVL[_autogen_verbs_layer _generated/role.md] + CP --> TL[_team_layer teams/file.md] + CP --> ID[identities/agent_slug.md] + CP --> AMB[ambient string] + TLD --> OUT["/app/prompts-generated/agent_id-prompt.md"] + LL --> OUT + BASE --> OUT + RL --> OUT + AVL --> OUT + TL --> OUT + ID --> OUT + AMB --> OUT + end + subgraph "Ambient (async, best-effort)" + OA[orchestrator._resolve_conventions_ambient] -->|settings.conventions_enabled| CAL[conventions_ambient_layer] + CAL --> CS[ConventionsService.render_ambient_block] + CS --> RC[ensure read clone] + CAL -->|cap 3000| AMB + end + subgraph "Identity binding" + IAT[issue_agent_token] --> HMAC[HMAC-SHA256 agent_id:role:team] + VAT[verify_agent_token] --> HMAC + IPT[issue_panel_token] --> IAT + end + subgraph "Injection guard (per turn)" + IDV[IntakeDriver.send_turn] --> DI[detect_injection] + DI -->|match| RM[refusal_message -> error chunk, return] + DI -->|clean| MODEL[forward to model] + GE[grok entrypoint] -->|CLI main exit 1| DI + BASH[user-prompt-hook.sh] -.same 5 patterns.-> DI + end +``` + +## Logical Tree +``` +prompts-roles-taxonomy slice +├── Prompt composition (roboco/agents/factories/) +│ ├── _base.py +│ │ ├── PROMPTS_BASE_PATH resolver +│ │ ├── _load_layer (file -> str|'') +│ │ ├── Layer maps: _ROLE_LAYER_MAP, _TEAM_LAYER_MAP +│ │ ├── Layer loaders: _role_layer, _team_layer, _autogen_verbs_layer, _lifecycle_layer +│ │ ├── Builtin-tool grant: _BUILTIN_TOOLS_COMMON/AUTHORS, _ROLE_BUILTIN_TOOLS, _tool_load_directive_layer +│ │ ├── compose_prompt (ordered join with '---') +│ │ └── conventions_ambient_layer (async, multi-project, 3000-char cap) + _AMBIENT_TOTAL_CAP +│ └── __init__.py (shim) +├── Permission taxonomy (roboco/agents_config.py) +│ ├── HMAC token layer: _auth_secret, _signing_payload, issue_agent_token, verify_agent_token, issue_panel_token +│ ├── UUID<->slug: _UUID_TO_SLUG, _resolve_to_slug +│ ├── Derived maps: AGENT_ROLE_MAP, AGENT_TEAM_MAP, CELL_MEMBERS, ALL_AGENTS, BOARD_MEMBERS, ALL_DOCS +│ ├── Role sets: TASK_CREATOR_ROLES, _CANCEL_ROLES, ROLE_PERMISSION_LEVELS, _BOARD_ROLES, _MAIN_PM_TARGETS +│ ├── Escalation: ESCALATION_CHAIN, get_escalation_target, get_pm_for_team, get_pm_for_agent +│ ├── Helpers: get_agent_role/team/cell, get_cell_members, is_pm/board_member/management/ceo, can_send_notifications/create/assign/cancel_tasks +│ ├── Notification enums: VALID_NOTIFICATION_TYPES/PRIORITIES +│ ├── A2A skills: ROLE_SKILLS, get_agent_skills +│ └── A2A routing: _check_cell_pm/cell_member/main_pm_a2a, can_a2a_direct, get_a2a_route_hint +├── Injection guard (roboco/agent_sdk/prompt_guard.py) +│ ├── _PATTERNS (5 regexes mirroring user-prompt-hook.sh) +│ ├── detect_injection, refusal_message +│ └── main (CLI for grok entrypoint) +└── Prompt corpus (agents/prompts/) + ├── base.md (universal rules) + ├── roles/ (9 files: developer, qa, documenter, cell_pm, main_pm, pr_reviewer, board, prompter, secretary) + ├── teams/ (3 files: backend, frontend, ux_ui) + ├── identities/ (19 per-agent YAML+blurb files) + └── _generated/ (regenerated artifacts) + ├── lifecycle-.md (x14; from lifecycle spec via make lifecycle; CI-gated no-drift) + ├── .md verb-signature tables (x12; from schemas + role_config via regenerate_verb_tables.py) + └── verbs.md (aggregate reference doc; NOT injected at spawn) +``` + +## Dependencies +- Internal: roboco.foundation.identity (AGENTS, Role, Team, CELL_TEAMS, BOARD_ROLES, slugs_for_team), roboco.foundation.policy.communications (NOTIFY_SENDER_ROLES), roboco.seeds.initial_data (AGENT_UUIDS, CEO_AGENT_ID), roboco.models.base (AgentRole, Team, NotificationType, NotificationPriority), roboco.config.settings (conventions_enabled), roboco.services.conventions (get_conventions_service, ConventionsService.render_ambient_block/resolve_workspace), roboco.db.base (get_session_factory), roboco.db.tables (ProjectTable), roboco.runtime.orchestrator (_generate_prompt, _resolve_conventions_ambient, _resolve_ambient_projects), roboco.agent_sdk.intake_driver (IntakeDriver.send_turn consumer), scripts/regenerate_verb_tables.py (regenerates _generated/.md + verbs.md), scripts/build_lifecycle_artifacts.py (regenerates _generated/lifecycle-.md; make lifecycle), docker/scripts/user-prompt-hook.sh (canonical bash guard mirrored by prompt_guard.py), roboco.api.schemas.v1 (Pydantic verb schemas the autogen tables derive from), roboco.services.gateway.role_config (role->verb config the autogen tables derive from) +- External: pathlib.Path, hmac / hashlib (HMAC-SHA256 tokens), os.environ (ROBOCO_AGENT_AUTH_SECRET), re (injection regexes), sys (prompt_guard CLI), sqlalchemy.ext.asyncio.AsyncSession (ambient layer), typing.Final, argparse-less sys.argv CLI + +## Entry Points + +| Name | File | Trigger | +|---|---|---| +| orchestrator._generate_prompt | roboco/runtime/orchestrator.py | Called per agent spawn to compose + write the system-prompt .md file; calls compose_prompt (line 3238) | +| orchestrator._resolve_conventions_ambient | roboco/runtime/orchestrator.py | Async, called from the spawn path before _generate_prompt to resolve the optional ambient block; calls conventions_ambient_layer (line 3299) | +| IntakeDriver.send_turn | roboco/agent_sdk/intake_driver.py | Per interactive turn (Intake/Secretary Claude-SDK and Grok sessions); calls detect_injection before forwarding to the model (line 379) | +| python -m roboco.agent_sdk.prompt_guard | roboco/agent_sdk/prompt_guard.py | CLI invoked by the grok one-shot entrypoint on ROBOCO_INITIAL_PROMPT; exit 1 denies start | +| make lifecycle | scripts/build_lifecycle_artifacts.py | Developer/CI target regenerating _generated/lifecycle-*.md; CI gates on git diff --exit-code | +| scripts/regenerate_verb_tables.py | scripts/regenerate_verb_tables.py | Developer target regenerating _generated/.md + verbs.md after role_config/schema changes | + +## Config Flags +- ROBOCO_AGENT_AUTH_SECRET (env) — HMAC signing secret for agent/panel tokens; unset => verify_agent_token fail-closes (rejects every token), issue_*_token returns UNSIGNED +- ROBOCO_CONVENTIONS_ENABLED — gates whether conventions_ambient_layer resolves + injects the architectural-standard ambient block; off => compose_prompt omits the ambient layer entirely +- ROBOCO_FABLE_MODE_ENABLED (default off) — gates fable_doctrine_layer AND ponytail_doctrine_layer (bundled — no separate ponytail flag); off => compose_prompt omits both doctrine layers entirely (byte-for-byte unchanged prompt) +- ROBOCO_PONYTAIL_INTENSITY (default full) — string value (lite/full/ultra), NOT a feature flag; selects the operative intensity the developer ponytail doctrine runs at (appended as a `**Operative intensity: ...**` directive for developers only; non-developers run a fixed restrained ethos regardless). `roboco/config.py` `ponytail_intensity`, validated as `Literal["lite","full","ultra"]` at Settings instantiation +- ROBOCO_SDK_URL (env, default http://localhost:9000) — used by the bash user-prompt-hook.sh (sister guard), not prompt_guard.py directly +- ROBOCO_INITIAL_PROMPT (env) — the one-shot prompt the grok entrypoint hands to prompt_guard CLI main +- PROJECT_HOST_PATH (orchestrator) — selects container (/app/prompts-generated) vs host ($TMPDIR/roboco-prompts) output dir for composed prompts + + +## Gotchas +- Layer ORDER matters and is load-bearing: tool-directive FIRST, then lifecycle, then base, then an optional fable-mode doctrine layer (`fable_mode_enabled`, `agents/prompts/doctrine/fable.md`), then an optional ponytail build-laziness doctrine layer (`fable_mode_enabled`, `agents/prompts/doctrine/ponytail.md` for developers / `ponytail-ethos.md` for other roles — bundled with Fable, no separate flag, role-scoped), then role, then autogen-verbs, then team, then identity, then ambient. The lifecycle fragment is intentionally before base so the agent reads its allowed verb surface before any other instruction. Reordering would change model attention priority. +- Empty/missing layers are silently dropped (compose_prompt skips falsy layers). An unknown role yields _role_layer=None AND _autogen_verbs_layer=None AND _lifecycle_layer=None — the agent would still spawn with just tool-directive + base + identity + ambient, missing its entire role+verb surface. The orchestrator guards upstream (raises ValueError on unknown role), but a typo in _ROLE_LAYER_MAP silently degrades to a roleless prompt rather than failing. +- _ROLE_LAYER_MAP maps all three board roles (product_owner/head_marketing/auditor) to the SAME board.md file. The per-role distinction (PO vs HoM vs Auditor) comes only from the identity file + the _generated/.md verb table, not from the role layer. A board role missing its identity file would lose its role-specific scope. +- verbs.md is the aggregate reference doc but is NOT injected at spawn — _base.py loads the per-role _generated/.md file instead. Editing verbs.md has zero prompt effect; it is a documentation/CI artifact only. The per-role files are the load-bearing ones. +- Identity YAML files carry a stale `role:` label (e.g. main-pm.md says `role: pm`, product-owner.md says `role: board`) that does NOT match the AgentRole enum values (main_pm/product_owner). The composition pipeline ignores this field entirely (loads identity by slug only); the real role comes from agents_config.AGENT_ROLE_MAP. Do not trust the identity YAML role label for enforcement. +- Board members (product_owner/head_marketing/auditor) have team=None, so _team_layer returns None for them — they get no team layer. main-pm likewise. Only cell members (dev/qa/doc/cell_pm) get a team layer. +- The autogen verb tables (_generated/.md) and lifecycle fragments are REGENERATED artifacts (make lifecycle / regenerate_verb_tables.py) gated on CI (git diff --exit-code). Hand-editing them is futile and will fail CI; change the source (lifecycle spec / Pydantic schemas / role_config) and regenerate. +- prompt_guard.py mirrors user-prompt-hook.sh patterns but is a separate implementation. The two must be kept in sync manually — there is no shared source. Drift between them means Claude (bash hook) and Grok/SDK (Python guard) apply different deny rules. +- detect_injection lowercases the text and uses loose anchoring (^|[\s>]) so injected content mid-message is caught, but the regexes are intentionally narrow (5 patterns). False negatives are expected by design — this is a classic-jailbreak denylist, not a comprehensive classifier; content that doesn't match still reaches the model. +- conventions_ambient_layer is best-effort and wraps the whole resolution in a try/except in the orchestrator (_resolve_conventions_ambient). A conventions resolution failure degrades silently to no ambient layer — a compose is never blocked by conventions. This means a conventions regression could quietly stop injecting the standard with no error surface. +- _AMBIENT_TOTAL_CAP (3000) truncates the ambient block with a trailing ellipsis. A large multi-project spawn (PO spanning several cells) can have its architectural standard silently truncated mid-block, leaving the agent with a partial standard. +- HMAC token verification fail-closes when ROBOCO_AGENT_AUTH_SECRET is unset — every agent token is rejected. issue_agent_token returns the literal sentinel 'UNSIGNED' which verify_agent_token also rejects. Deploying without the secret bricks all agent API auth (by design). + + +## Drift from CLAUDE.md +- CLAUDE.md Project Overview states '25 AI agents + 1 human CEO', but agents/prompts/base.md line 3 says '22 AI agents + 1 human CEO'. The base prompt agent count is stale relative to CLAUDE.md (memory notes a 20->22 update on 2026-06-16; CLAUDE.md later moved to 25). A spawned agent reads '22' in its system prompt while the org chart it sees has 25. +- CLAUDE.md's verb-surface table lists `i_am_blocked` for the qa and documenter roles. The role prompts qa.md and documenter.md ONLY added the i_am_blocked verb row in commit 15effce0 (this slice's baseline diff) — before that the role prompts omitted it even though the gateway accepted it. The prompts are now aligned, but the documenter.md circuit-breaker section previously explicitly said 'you don't have an i_am_blocked verb', which was false vs the gateway and vs CLAUDE.md. Fixed in 15effce0. +- CLAUDE.md says the lifecycle is defined in roboco/foundation/policy/lifecycle.py with a shim at roboco/enforcement/task_lifecycle.py. _base.py:_lifecycle_layer (line 187-200) docstring says the lifecycle fragment is regenerated from `roboco/lifecycle/spec.py` by `make lifecycle`. The actual source path the regenerator uses is roboco/lifecycle/spec.py (per the docstring), which is not mentioned in CLAUDE.md's lifecycle section — minor doc-path drift, not behavioral. +- CLAUDE.md describes the prompt composition as 'base + role + team + identity prompts' and an ambient 'Architectural Standard' block at spawn. The actual compose_prompt order (line 295-306) is tool-directive + lifecycle + base + fable + ponytail + role + autogen-verbs + team + identity + ambient — i.e. FIVE additional layers (tool-directive, lifecycle, fable, ponytail, autogen-verbs) not named in CLAUDE.md's composition description. CLAUDE.md undersells the actual layer stack. +- Identity files declare `role: pm` / `role: board` (e.g. identities/main-pm.md, identities/product-owner.md) which do not match the canonical AgentRole enum values (main_pm, cell_pm, product_owner, head_marketing, auditor) that CLAUDE.md and agents_config use. The pipeline ignores this label so it is cosmetic, but it is inconsistent with the canonical taxonomy CLAUDE.md documents. + + +## Changes Since Baseline + +| SHA | Subject | Impact | +|---|---|---| +| 15effce0 | Chore: 141 Gaps fill-in (#283) — sole commit touching this slice since fd10cc86 | Prompt-surface alignment with gateway/spec: (1) developer.md + lifecycle-developer.md + verbs.md add the new sync_branch verb and rewrite the behind-base guidance on developer/cell_pm/main_pm to point devs at sync_branch instead of i_am_blocked/escalate_up (cell/root integration branches still escalate). (2) qa.md and documenter.md add the i_am_blocked verb row and rewrite the circuit-breaker section to use i_am_blocked instead of 'you don't have an i_am_blocked verb -> unclaim' — corrects a false prompt claim. (3) cell_pm.md delegate signature + verbs.md add the collision-surface fields intends_to_touch/adds_migration/touches_shared/depends_on and a new 'Collision surface' section instructing the PM to declare them on every code subtask so siblings sequence. (4) pr_reviewer.md adds the in-path gate verbs claim_gate_review/pr_pass/pr_fail + an 'In-path gate review' section. (5) prompter.md adds MegaTask root-subtask coordination-level AC guidance (task_type=planning, coordination-level ACs). (6) lifecycle-main_pm.md submit_root description changes from 'Only for code roots' to 'branch-bearing roots; gate is branch-keyed not task_type-keyed'. (7) note verb schema in all autogen tables gains done/next/where_to_look top-level string params; pass_review ac_verdicts and delegate covers_parent_criteria/intends_to_touch now show BeforeValidator in the signature. | + +> Post-snapshot updates (since 2026-06-29): **536bbb64** (Chore/all/logical gaps sweep #286) — (a) agents_config.py: `_TEAM_SCOPED_ROLES` deduped: was inline-defined, now re-exported as `_comms.TEAM_SCOPED_ROLES` from `foundation.policy.communications` (values unchanged: dev/qa/doc/cell_pm); (b) _generated/cell_pm.md, main_pm.md, qa.md, verbs.md: BeforeValidator repr cleaned from delegate/pass_review signatures — now renders `list[str] | None = None` instead of the memory-address-bearing BeforeValidator literal; (c) lifecycle spec: `PRECONDITION_ROOT_NOT_CODE` added to `submit_root` extra_preconditions, backing the branch-keyed / planning-typed claim the prompt asserts. **aba57359** ([chore] lifecycle artifacts regenerate, foundation-check) — lifecycle-cell_pm.md, lifecycle-developer.md, lifecycle-documenter.md, lifecycle-main_pm.md, lifecycle-qa.md: `unclaim` description expanded with "A PR reviewer who claimed an external/gate review and cannot finish releases the claim here rather than wedging the lane"; lifecycle-cell_pm.md + lifecycle-main_pm.md: `complete` description clarified "The merge runs BEFORE the complete transition" ordering. +> +> **v0.18.0** (2026-07-04): Fable mode adds a 9th conditional compose_prompt layer — `fable_doctrine_layer()` (_base.py:203) injects `agents/prompts/doctrine/fable.md` right after base.md, gated by `fable_mode_enabled` (default off; off = byte-for-byte unchanged prompt). FE/UX-UI design bar: `## Design bar` sections added to `teams/frontend.md` + `teams/ux_ui.md` (taste-skill-distilled dials + rules), plus a scoping pointer in `roles/developer.md` — doc-only, no flag, no compose_prompt change (team/role layers already existed; only their file contents grew). +> +> **v0.19.0** (2026-07-05): Ponytail build-laziness doctrine bundled with Fable — `ponytail_doctrine_layer(prompts_path, role)` in `roboco/agents/factories/_base.py`, gated on the same `fable_mode_enabled` flag (no separate flag — ponytail is Fable's complementary build-doctrine), slotted into compose_prompt immediately after `fable_doctrine_layer`. Role-scoped: developers (`AgentRole.DEVELOPER`) → `agents/prompts/doctrine/ponytail.md` (the full ladder: YAGNI → reuse-in-this-codebase → stdlib → native-platform → installed-dep → one-line → minimal, the rules, the Intensity table, the `ponytail:` comment convention) plus a trailing `**Operative intensity: {settings.ponytail_intensity}.**` directive; every other role → `agents/prompts/doctrine/ponytail-ethos.md` (ethos-only — the code-mechanics rungs and the Intensity table are dropped so they can't leak into prose artifacts like task plans / review notes / docs). Both files vendored from the ponytail plugin (MIT, Copyright (c) 2026 DietrichGebert), trimmed, YAML frontmatter stripped, and carry a 5-point RoboCo preamble that makes the ladder yield to the Architectural Conventions Standard (placement), the 80% coverage gate + QA review + self-verification, the per-team design bar, task hygiene, and reviewer feedback — overlap mitigated by scoping, not deletion. `ROBOCO_PONYTAIL_INTENSITY` (lite/full/ultra, default full; `roboco/config.py` `ponytail_intensity`, a string value — NOT a feature flag) selects the developer's operative intensity; non-developers get no dial. Prompt-only — no hooks, no grok-path changes; a flag-off spawn is byte-for-byte unchanged. +> +> **PR #544** (2026-07-17, `fd621f0d`): The design bar's web dials (`DESIGN_VARIANCE`/`MOTION_INTENSITY`/`VISUAL_DENSITY`) were silently steering a `source=video` authoring task toward "dense product UI → motion 2-3" — wrong for a marketing film. `agents/prompts/teams/ux_ui.md` gains a one-line video-mode override (see Files above) telling the dev the web dials do not apply to a video-authoring task and pointing it at `motion/README.md`'s cinematography bar + the vendored `motion/skills/` doctrine instead. Doc-only within this slice — the runtime half of the same PR (playwright MCP registered for the video-authoring ux-dev spawn, `_is_video_authoring_spawn`) is documented in `docs/map/video-engine.md` and `docs/map/orchestrator.md`. +> +> **"taste-skill-aesthetics"** (deferred half of the Design bar work): two more `Leonxlnx/taste-skill` (MIT) distillations layered onto the core Design bar. A `## Niche aesthetic vocabularies` section — identical body in both `agents/prompts/teams/frontend.md` and `ux_ui.md` — names three opt-in visual systems (industrial brutalist, minimalist editorial, premium agency), each keyed onto the same three dials (a vocabulary changes *what* the dials produce, never whether they apply); picked only when a task brief explicitly calls for one. A `## Image direction` section lives in `ux_ui.md` only (composition variety, palette discipline, anti-slop imagery, iconography, mockup/device-frame conventions, cross-asset set consistency) — `frontend.md` carries a one-line pointer instead of duplicating it. Doc-only, same `tests/unit/agents/test_design_bar_layer.py` guard as the core bar; no compose_prompt/flag change. +> +> **CEO A2A pairs, two sequential fixes.** `fc6d6f64` (2026-07-18, "CEO pairs join the switchboard matrix"): the static pair matrix previously filtered by `is_human_only_role` (a spawn-semantics check) which dropped the CEO before `can_a2a_direct` (which explicitly allows CEO→anyone) ever ran, so `A2A_ALLOWED_PAIRS` carried ZERO CEO pairs; `agents_config.py` now excludes only `prompter`/`secretary`/`system` from that pre-filter (matrix size 70→93). `56b6693e` ("security-hygiene-sweep") then narrows it correctly: `_check_ceo_a2a` (agents_config.py:635, consulted from `can_a2a_direct`'s CEO branch — previously an unconditional `True`) refuses a CEO target in `NO_COMMS_ROLES` (see `docs/map/foundation-policy-misc.md`), shrinking `A2A_ALLOWED_PAIRS` 93→88 (`ceo` group 23→18) with no separate matrix edit — the derivation is fully automatic off `can_a2a_direct`. See `docs/map/panel.md` for the switchboard's `"CEO Direct"` section + collapsible-sections panel change, and `docs/map/a2a-audit-journal-permissions.md` for the conversation-creation-time refusal. + +## Regression Risks + +| Title | File:Line | Claim | Severity | +|---|---|---|---| +| Collision-surface declaration is prompt-only, not gate-enforced | agents/prompts/roles/cell_pm.md:141 | The new 'Collision surface' section tells the cell PM to fill intends_to_touch/adds_migration/touches_shared on every code subtask so the analyzer can sequence colliding siblings, but the section explicitly says 'leave it empty only for a research/design subtask' — there is no documented gate that REFUSES a code delegate with empty intends_to_touch. If a PM omits it on a code subtask, two siblings editing the same file run in parallel and collide (the exact 2026-06-27 out-of-order break this was added to prevent). The protection hinges on agent compliance with prompt prose, not a hard gate. | medium | +| sync_branch guidance contradicting the i_am_done gate for behind-base branches | agents/prompts/roles/developer.md:126 | developer.md now says 'call sync_branch as soon as roboco_git_status shows your branch behind, OR when i_am_done refuses with your branch is N behind'. If the i_am_done gate's behind-base check and sync_branch's rebase disagree on what 'base' means (e.g. base resolved from the recorded branch vs the parent task's head), a dev could sync_branch successfully and still hit the i_am_done behind-base refusal, looping. The prompt assumes both use the same base resolution; a divergence there would trap the dev. No fallback to i_am_blocked is offered anymore (the prompt explicitly forbids it for a plain behind-base condition), removing the previous escape hatch. | medium | +| ~~documenter/qa circuit-breaker now directs to i_am_blocked — verify the verb is actually granted to those roles~~ **VERIFIED OK** | agents/prompts/roles/documenter.md:100 | The circuit-breaker section was rewritten from 'you don't have an i_am_blocked verb -> unclaim' to 'i_am_blocked(task_id, reason=...) to escalate'. This relies on i_am_blocked being genuinely callable by qa and documenter at the gateway. The autogen verbs.md shows i_am_blocked for qa but the documenter section in verbs.md (and _generated/documenter.md) must also list it — if the role_config does not grant i_am_blocked to documenter, the new prompt guidance sends the agent to a verb that will return not_authorized, trapping them on a circuit_open with no documented fallback (the old unclaim-only path was removed). **Verified 2026-07-01: lifecycle spec `i_am_blocked.allowed_roles = frozenset(_DEV_ROLES | _QA_ROLES | _DOC_ROLES)` — documenter IS granted this verb at baseline and post-snapshot; not a live risk.** | high | +| ~~submit_root description changed from code-root to branch-bearing-root semantics~~ **RESOLVED (536bbb64)** | agents/prompts/_generated/lifecycle-main_pm.md:14 | The lifecycle fragment now says submit_root is 'For branch-bearing roots' and 'The gate is branch-keyed, not task_type-keyed — a Main-PM root is planning-typed, never code'. If the gateway's submit_root implementation still keys off task_type=code (the old contract), a planning-typed branch-bearing root would be rejected by the gate while the prompt tells the Main PM to call submit_root on it — a loop. The prompt now asserts a behavior the gateway must match; mismatch breaks Main-PM root submission. **Fixed 2026-06-30: 536bbb64 added `PRECONDITION_ROOT_NOT_CODE` (`_p_root_not_code`: checks `task_type != code`) to `submit_root.extra_preconditions` in the lifecycle spec, and the spec description was updated to match; prompt and gateway are now aligned.** | high | +| note schema advertises done/next/where_to_look the gateway must accept | agents/prompts/_generated/developer.md:25 | All autogen verb tables now show note(...) with done/next/where_to_look top-level params. If the Pydantic note schema (the regenerator source) was updated but the gateway's note handler / DB journal model does not persist these fields, agents will pass them, the schema accepts them, but they are silently dropped — the handoff/quick_context fields the meltdown #1 fix intended to surface top-level would never reach downstream briefings. The prompt advertises params that may be no-ops at the persistence layer. | medium | +| ~~BeforeValidator rendering in verb signatures leaks into the prompt~~ **RESOLVED (536bbb64)** | agents/prompts/_generated/verbs.md:58 | The regenerated verb tables now render `BeforeValidator(func=, json_schema_input_type=PydanticUndefined)` literally into the agent's system prompt for pass_review.ac_verdicts, delegate.covers_parent_criteria and delegate.intends_to_touch. This is a memory-address-bearing repr of an internal Pydantic validator injected into every qa/cell_pm/main_pm prompt. It is noise the model must parse around and the address is non-deterministic across runs, which could in principle perturb caching/reprompt determinism. Not a correctness bug but a prompt-hygiene regression introduced by the regenerator. **Fixed 2026-06-30: 536bbb64 cleaned the regenerated tables; all three fields now render `list[str] | None = None`.** | low | +| cell_pm.md behind-base guidance split between dev sync_branch and cell-branch escalate_up | agents/prompts/roles/cell_pm.md:178 | The rewritten behind-base section tells the PM to direct devs to sync_branch for their leaf but to escalate_up for the cell integration branch. If a PM mis-classifies a behind-base condition (tells a dev to escalate_up instead of sync_branch, or calls escalate_up on a dev's leaf), the dev waits on a platform action that won't come (sync_branch is the dev's own verb). The split is correct but easy to misapply; a misroute strands the dev. | low | + +## Health +The composition pipeline is well-structured and deterministic: a single ordered join over gracefully-degrading layers, with CI-gated autogenerated artifacts (lifecycle + verb tables) that cannot silently drift from the spec, and a clean separation between the prompt corpus (markdown), the taxonomy (agents_config.py, derived from foundation so it cannot drift from the org chart), and the guard (prompt_guard.py, mirroring the bash hook). The main open integrity risks are (a) the prompt-only enforcement of the new collision-surface declaration on delegate — the 2026-06-27 out-of-order break this was added to fix can still recur if a PM omits intends_to_touch on a code subtask; (b) the stale agent-count in base.md (22 vs CLAUDE.md's 25) and the cosmetic but inconsistent `role:` labels in identity YAML. Previously flagged risks (b/c as of 2026-06-29 snapshot) have been closed: documenter/qa i_am_blocked is confirmed granted by the lifecycle spec (_DEV_ROLES|_QA_ROLES|_DOC_ROLES), and submit_root's branch-keyed claim is now backed by PRECONDITION_ROOT_NOT_CODE in the spec gate (536bbb64). BeforeValidator repr in prompt tables also cleaned (536bbb64). + +## Purpose +The gated release manager: a default-off background loop that deterministically assesses RoboCo's own repo (diff-since-tag → conventional-commit classification → semver bump → readiness gaps) and originates ONE held release PROPOSAL task for the CEO; the CEO's panel approve/reject routes call a fail-closed ReleaseExecutor that bumps versions, runs `make quality`, commits `chore(release): X.Y.Z`, waits for green release-commit CI, and `gh release create`s — aborting before commit on a red gate and before publish on red CI. It never auto-merges or auto-deploys; the CEO is the only actor who can trigger a publish. + +**Env-ladder era.** Release ops now target the project's env-ladder **prod rung** (`roboco.models.env_branches.prod_branch`) instead of the raw `projects.default_branch` column — a project with no declared ladder resolves to the same value via the read-time shim, so single-branch projects are unaffected. Before bumping, the executor runs a **full-chain promotion** (`promote_env_chain`) that merges every rung between head and prod, in order, into the prod checkout — the release commits + tags the promoted state, not just prod's own prior tip — and aborts fail-closed (`promotion_failed`) before any bump on a fetch/merge conflict. `release_readiness` diffs `prod..head` (falling back to `last_tag..HEAD` when the prod rung can't be resolved) and cross-checks the last tag against the prod tip (`_tag_drift_gaps`) to flag a hotfix that landed on prod outside the ladder. + +## Files + +| Path | Role | LOC | +|---|---|---| +| roboco/services/release_executor.py | Fail-closed bump→gate→commit/push→CI→publish orchestrator with a Protocol seam (ReleaseOps) over a writable token-authenticated clone (_GitReleaseOps); idempotent on already-published versions; half-landed (publish_failed) retry skips re-bump/re-commit. | 490 | +| roboco/services/release_proposal.py | CEO approve/reject glue over the single held proposal task; approve runs `_approve_precheck` then dispatches the ~40min executor as a background asyncio task (returns 202 immediately); heartbeat-guarded Redis fencing-token mutex; closes proposal on published OR already_published; approve refuses a CANCELLED (`already_rejected`) or COMPLETED (`already_published`) proposal before ever touching the lock; reject records required changes and keeps it held, raising `TaskAlreadyCompletedError` if the proposal already published. | 547 | +| roboco/services/release_readiness.py | Pure conventional-commit classification + semver-derivation primitives + the git/filesystem snapshot gatherer + the assess() report builder; serializes/deserializes the report for JSONB storage on the proposal task. | 572 | +| roboco/services/release_manager_engine.py | Default-off detection loop: per interval, if no proposal is open and the gate is green and changes past threshold, originate ONE PENDING HELD Secretary-owned proposal carrying the readiness report; never publishes. | 239 | + +## Key Symbols + +| Name | Kind | File:Line | Responsibility | +|---|---|---|---| +| ReleaseResult | dataclass | roboco/services/release_executor.py:36 | Frozen outcome of an execute attempt: status (published/gate_failed/ci_failed/commit_failed/publish_failed/already_published/already_in_progress/lock_lost/redis_unavailable), version, files_changed, commit_sha, release_url, detail. | +| ReleaseOps | Protocol | roboco/services/release_executor.py:48 | Side-effecting release steps (is_already_published, release_commit_sha, apply_version_bumps, write_changelog_entry, run_gate, commit_and_push, wait_for_ci, publish_release) injected so the fail-closed ordering is unit-testable. | +| ReleaseExecutor | class | roboco/services/release_executor.py:70 | Orchestrates the fail-closed release pipeline over a ReleaseOps, aborting on any red step and returning a ReleaseResult. | +| ReleaseExecutor.execute | method | roboco/services/release_executor.py:76 | Bump→gate→commit/push→CI→publish; short-circuits on already_published; detects half-landed (publish_failed) retry via release_commit_sha and rejoins CI→publish tail without re-bumping; returns gate_failed/ci_failed/commit_failed/publish_failed ReleaseResult on red steps. | +| _await_proc | function | roboco/services/release_executor.py:208 | Communicate with a subprocess under a deadline; on timeout kill() the child, await proc.wait() to reap it (no zombie), and return non-zero rc so fail-closed branches fire instead of hanging the release loop. | +| _ReleaseContext | dataclass | roboco/services/release_executor.py:259 | Writable-clone coordinates: slug, prod_branch (the env-ladder prod-rung target, resolved via `roboco.models.env_branches.prod_branch`), root Path, git_url, git_prefix, ci_workflow, env_chain (the head→…→just-below-prod rung branches to promote; empty for a degenerate head==prod ladder). | +| _GitReleaseOps.promote_env_chain | method | roboco/services/release_executor.py:364 | Full-chain promotion: fetch origin, then merge (`--no-edit`) each `env_chain` branch into the prod checkout in head-first order before the bump; fail-closed RuntimeError on a fetch or merge-conflict; no-op for an empty chain (degenerate ladder). | +| _GitReleaseOps | class | roboco/services/release_executor.py:238 | Production ReleaseOps on a fresh token-authenticated writable clone: real git/make/gh with per-step subprocess deadlines. | +| _GitReleaseOps._git | method | roboco/services/release_executor.py:249 | Run a git -C command under _GIT_OP_TIMEOUT_SECONDS, returning (rc, stdout). | +| _GitReleaseOps.is_already_published | method | roboco/services/release_executor.py:260 | git ls-remote --tags origin v; true if the tag already exists (idempotency guard). | +| _GitReleaseOps.release_commit_sha | method | roboco/services/release_executor.py:264 | Half-landed detection: if the clone's working version == target version AND a `chore(release): {version}` commit appears in the recent log, return its sha (publish_failed retry → skip re-bump); else None. | +| _GitReleaseOps._current_version | method | roboco/services/release_executor.py:287 | Read the version string out of pyproject.toml (the old value for the bump replace). | +| _GitReleaseOps.apply_version_bumps | method | roboco/services/release_executor.py:292 | Replace old version with new across the bump plan, skipping CHANGELOG.md and bumping uv.lock only in the roboco package block. | +| _GitReleaseOps.write_changelog_entry | method | roboco/services/release_executor.py:315 | Insert the drafted CHANGELOG entry above the first released version heading. | +| _GitReleaseOps.run_gate | method | roboco/services/release_executor.py:320 | Run `make quality` in the clone under _RELEASE_GATE_TIMEOUT_SECONDS; return rc==0. | +| _GitReleaseOps.commit_and_push | method | roboco/services/release_executor.py:336 | git add -A, commit -S chore(release): , rev-parse HEAD, push HEAD:default_branch; raises RuntimeError on add/commit/push failure. | +| _GitReleaseOps.wait_for_ci | method | roboco/services/release_executor.py:359 | Poll GitService.get_latest_ci_conclusion for the slug up to 80×30s (~40min), requiring head_sha match + success conclusion. | +| _GitReleaseOps.publish_release | method | roboco/services/release_executor.py:378 | gh release create v --target default_branch; raises RuntimeError on non-zero rc (caught by execute → publish_failed), returns the release URL. | +| _bump_uv_lock | function | roboco/services/release_executor.py:407 | Bump only the roboco package version block inside uv.lock so a same-versioned dependency is not clobbered. | +| _insert_changelog_entry | function | roboco/services/release_executor.py:415 | Insert the new entry above the first ## [] heading (Keep a Changelog format). | +| _resolve_release_ci_workflow | function | roboco/services/release_executor.py:427 | Return settings.release_ci_workflow or "ci.yml" — decoupled from self_heal_ci_workflow; never returns None or empty, so the release gate always scopes to a named workflow. | +| get_release_executor | function | roboco/services/release_executor.py:441 | Build a ReleaseExecutor over a fresh writable clone: resolve the RoboCo project, decrypt token, inject into URL, _prepare_release_clone; ci_workflow set from _resolve_release_ci_workflow(). | +| _prepare_release_clone | function | roboco/services/release_executor.py:467 | rm -rf and re-clone the release clone at workspaces_root/_release/ on the default branch. | +| _run | function | roboco/services/release_executor.py:483 | Run a subprocess under _CLONE_TIMEOUT_SECONDS via _await_proc (used by _prepare_release_clone). | +| ReleaseLockUnavailable | exception | roboco/services/release_proposal.py:40 | Distinct from "lock is held" — Redis itself is unreachable (infra failure, not a concurrent approve). Both paths are fail-closed but the error surface differs. | +| TaskAlreadyCompletedError | exception | roboco/services/release_proposal.py:49 | Raised by `reject()` when the proposal is already COMPLETED (published) — a stale reject (e.g. a queued Telegram button on a proposal a concurrent approve already shipped) can't cancel a release that already happened. | +| ReleaseProposalService | class | roboco/services/release_proposal.py:77 | Find/approve/reject the single open release proposal; approve dispatches the executor as a background asyncio task (202), with a heartbeat-guarded fencing-token Redis mutex. | +| ReleaseProposalService.open_proposal | method | roboco/services/release_proposal.py:98 | Return the first non-terminal release_manager-source task or None. | +| ReleaseProposalService._approve_precheck | method | roboco/services/release_proposal.py:103 | Resolve the proposal + stored report, or a canned refusal: CANCELLED (the CEO already rejected it) returns an `already_rejected` ReleaseResult, COMPLETED (already published) returns `already_published` — both WITHOUT ever touching the Redis lock/executor. Split out of `approve()` to keep its own return-count bounded as more terminal-state guards are added. | +| ReleaseProposalService.approve | method | roboco/services/release_proposal.py:165 | Runs `_approve_precheck` first (returns its canned refusal on a terminal-state proposal); otherwise acquires the Redis fencing-token mutex (raises ReleaseLockUnavailable on Redis outage → returns redis_unavailable; returns already_in_progress if lock held); runs executor as asyncio.Task guarded by a heartbeat; marks COMPLETED on published OR already_published; returns lock_lost if heartbeat cancels execute on TTL expiry. | +| ReleaseProposalService._finalize_release_lock | method | roboco/services/release_proposal.py:191 | finally-block: cancel heartbeat/execute tasks and compare-and-del the release mutex. | +| ReleaseProposalService._acquire_release_lock | method | roboco/services/release_proposal.py:207 | SET NX EX the release mutex with a fencing-token value; returns token if acquired, None if held (concurrent approve); raises ReleaseLockUnavailable if Redis is unreachable (caller surfaces redis_unavailable, not already_in_progress). | +| ReleaseProposalService._release_release_lock | method | roboco/services/release_proposal.py:230 | Compare-and-del the release mutex via Lua CAS — only deletes if the key still holds our fencing token, so a late first-finally can't delete a usurper's lock. | +| ReleaseProposalService._heartbeat_release_lock | method | roboco/services/release_proposal.py:241 | Compare-and-expire the release mutex (Lua); returns True if we still own it. | +| ReleaseProposalService._heartbeat_loop | method | roboco/services/release_proposal.py:256 | Refreshes lock TTL while execute is running; if the lock is no longer ours (>TTL Redis outage let it expire), sets lock_lost and cancels execute fail-closed. | +| ReleaseProposalService.reject | method | roboco/services/release_proposal.py:414 | Record the CEO's required_changes marker on the proposal; keep it held for revision. Raises TaskAlreadyCompletedError when the proposal is already COMPLETED (published) — a stale reject can't lie about an already-public release's real state. | +| get_release_proposal_service | function | roboco/services/release_proposal.py:442 | Construct a ReleaseProposalService bound to a session. | +| dispatch_approve | function | roboco/services/release_proposal.py:538 | Spawn the ~40min release execute as a background asyncio.Task (registered in _INFLIGHT_APPROVES) so the HTTP route returns 202 immediately; done-callback removes the entry. | +| _run_approve_background | function | roboco/services/release_proposal.py:490 | Run approve() in a background task with a fresh session (the request session closes on the 202 response); commits on success, rolls back and logs on failure. | +| CommitInfo | dataclass | roboco/services/release_readiness.py:89 | One commit since the last release tag: sha, subject, body, pr_number, labels. | +| ClassifiedChange | dataclass | roboco/services/release_readiness.py:100 | A commit annotated with normalized kind, breaking flag, summary, needs_manual_classification. | +| _has_breaking_label | function | roboco/services/release_readiness.py:111 | True if any PR label is in the breaking-label set. | +| _classify_one | function | roboco/services/release_readiness.py:115 | Classify a commit by conventional-commit prefix, then PR-label fallback, then needs_manual_classification. | +| classify_changes | function | roboco/services/release_readiness.py:152 | Map classify_one over a list of commits. | +| derive_bump | function | roboco/services/release_readiness.py:157 | Reduce the change set to a semver bump: breaking→major, feat→minor, else patch. | +| next_version | function | roboco/services/release_readiness.py:166 | Apply a bump to a MAJOR.MINOR.PATCH string (leading v ok). | +| Gap | dataclass | roboco/services/release_readiness.py:176 | One readiness shortfall (category, detail) the CEO must see before approving. | +| ReleaseRepoSnapshot | dataclass | roboco/services/release_readiness.py:185 | Raw read-only release facts: version, last_tag, commits, version-ref files, canonical bump files, changelog, migrations, CI conclusion, agent counts, verb-tables-stale flag. | +| ReleaseReadinessReport | dataclass | roboco/services/release_readiness.py:207 | The deterministic CEO-reviewable proposal: proposed_version, bump_kind, change_summary, drafted_changelog, version_bump_plan, gaps, migration_notes, gate_state. | +| _is_documented | function | roboco/services/release_readiness.py:221 | True if a change's PR number or summary text appears in the CHANGELOG. | +| _draft_changelog | function | roboco/services/release_readiness.py:228 | Build a Keep-a-Changelog ## [version] - date block with Added/Changed/Fixed/Security sections. | +| _changelog_gaps | function | roboco/services/release_readiness.py:246 | Flag feat/fix/security/perf/refactor changes not present in the CHANGELOG. | +| _version_ref_gaps | function | roboco/services/release_readiness.py:259 | Flag files embedding the current version but not in the canonical bump plan. | +| _docs_drift_gaps | function | roboco/services/release_readiness.py:271 | Flag declared-vs-actual agent-count mismatch and stale verb-surface tables. | +| _migration_gaps_and_notes | function | roboco/services/release_readiness.py:291 | Emit migration run-notes for new migrations and a gap if there is >1 alembic head. | +| _gate_state | function | roboco/services/release_readiness.py:312 | Map a CI conclusion to green/unknown/red. | +| assess | function | roboco/services/release_readiness.py:320 | Turn a snapshot into a gap-flagged ReleaseReadinessReport (classify → derive bump → next version → assemble gaps → draft changelog). | +| _run_git | function | roboco/services/release_readiness.py:365 | Synchronous git subprocess helper (capture stdout, no check). | +| _pyproject_version | function | roboco/services/release_readiness.py:375 | Read the version out of pyproject.toml. | +| _last_tag | function | roboco/services/release_readiness.py:381 | git describe --tags --abbrev=0 (most recent tag) or None. | +| _commits_since | function | roboco/services/release_readiness.py:386 | git log ..HEAD with record/field separators; parse into CommitInfo with PR-number extraction. | +| _tracked_files_with_version | function | roboco/services/release_readiness.py:408 | git grep -lF excluding tests/ — files embedding the version string. | +| _canonical_bump_files | function | roboco/services/release_readiness.py:415 | Derive the bump set from the previous chore(release): commit's files (subject-filtered), falling back to the version-ref scan on first release. | +| _new_migrations | function | roboco/services/release_readiness.py:447 | git diff --name-only --diff-filter=A ..HEAD -- alembic/versions/. | +| _migration_head_count | function | roboco/services/release_readiness.py:464 | Parse alembic/versions/*.py revision/down_revision lines; count unreferenced heads. | +| _declared_agent_count | function | roboco/services/release_readiness.py:484 | Regex the 'N AI agents' declaration out of roboco/__init__.py. | +| _actual_agent_count | function | roboco/services/release_readiness.py:493 | Count non-system/non-ceo rows in foundation.identity.AGENTS (best-effort). | +| gather_snapshot | function | roboco/services/release_readiness.py:505 | Build a ReleaseRepoSnapshot from a real checkout (read-only); verb_tables_stale left False (regen would write). | +| _read_changelog | function | roboco/services/release_readiness.py:534 | Read CHANGELOG.md or '' on OSError. | +| report_to_dict | function | roboco/services/release_readiness.py:541 | Serialize a report to a plain dict for JSONB marker storage. | +| report_from_dict | function | roboco/services/release_readiness.py:557 | Rebuild a report from its stored dict (inverse of report_to_dict). | +| ReleaseAssessor | type alias | roboco/services/release_manager_engine.py:58 | Callable[[], Awaitable[ReleaseReadinessReport / None]] — injectable assessor (default production, tests synthetic). | +| _roboco_slug | function | roboco/services/release_manager_engine.py:61 | The registered project that IS RoboCo itself (self_heal_project_slug or 'roboco-api'). | +| _past_threshold | function | roboco/services/release_manager_engine.py:66 | True when commit count >= release_min_commits OR bump is non-patch OR any security change. | +| _proposal_description | function | roboco/services/release_manager_engine.py:75 | Human-readable proposal body: version, bump, change count, gate, drafted CHANGELOG, gaps, migrations. | +| ReleaseManagerEngine | class | roboco/services/release_manager_engine.py:95 | Detect release-readiness and originate ONE CEO-gated held proposal; never publishes. | +| ReleaseManagerEngine.run_cycle | method | roboco/services/release_manager_engine.py:106 | No-op unless enabled; if no proposal open, ready_report, resolve project, originate. | +| ReleaseManagerEngine._ready_report | method | roboco/services/release_manager_engine.py:130 | Assess; return report only when gate is green and past threshold, else None. | +| ReleaseManagerEngine._originate | method | roboco/services/release_manager_engine.py:149 | Create a PENDING HELD Secretary-owned ADMINISTRATIVE task with the report marker; notify CEO. | +| ReleaseManagerEngine._notify_ceo | method | roboco/services/release_manager_engine.py:188 | Best-effort ack-notification to the CEO summarizing the proposal (never blocks origination). | +| ReleaseManagerEngine._production_assess | method | roboco/services/release_manager_engine.py:206 | Real path: ensure read clone, fetch CI conclusion, gather_snapshot, assess; None on any resolution failure. | +| get_release_manager_engine | function | roboco/services/release_manager_engine.py:234 | Build a ReleaseManagerEngine with optional injected assessor. | + +## Data Flow +DETECT loop: the orchestrator spawns `_release_manager_loop` (an asyncio task started in `start()`) which, when `release_manager_enabled`, sleeps `release_manager_interval_seconds` then calls `_run_release_manager_cycle` → opens a DB session → `get_release_manager_engine(db).run_cycle()`. `run_cycle` short-circuits if disabled, if `TaskService.list_open_release_proposals()` already returns one (dedup by `source='release_manager'` + non-terminal status), or if `_ready_report()` returns None. `_ready_report` calls the injected assessor (default `_production_assess`): resolve the RoboCo project by `self_heal_project_slug`, `WorkspaceService.ensure_read_clone` (pinned to the **head** rung's HEAD), `GitService.get_latest_ci_conclusion`, then best-effort fetch the project's **prod** rung into that head-pinned read clone (`_ensure_prod_fetched` — a no-op when prod==head; a fetch failure degrades to the `last_tag..HEAD` baseline rather than aborting) so `origin/` resolves, then `gather_snapshot(read_clone_root, master_ci_conclusion, prod_branch=prod_for_snapshot)` + `assess(snapshot, today)`. `assess` runs `classify_changes` → `derive_bump` → `next_version` → assembles gaps (changelog, version_ref, docs_drift, migration, classification, gate). If green + past threshold, `_originate` creates a PENDING HELD `RELEASE_MANAGER_SOURCE` task owned by `secretary-1` via `TaskService.create(TaskCreateRequest(..., confirmed_by_human=False))`, stores the report dict via `markers.set_release_report`, flushes, and best-effort notifies the CEO. The orchestrator cycle commits the session. + +CEO ACT path: `GET /api/release/proposal` (CEO-only) → `ReleaseProposalService.open_proposal()` → `list_open_release_proposals()[0]`. `POST /proposal/approve` (returns 202 immediately): route calls `dispatch_approve(task_id, session_factory)` which spawns `_run_approve_background` as a background asyncio.Task (the request session closes at the 202 return; the background task opens a fresh session). In `approve(task_id)`: loads the task, verifies `source == RELEASE_MANAGER_SOURCE`, reads `markers.get_release_report`; acquires Redis fencing-token mutex (`SET NX EX 3000`) — raises `ReleaseLockUnavailable` on Redis outage → returns `redis_unavailable`; returns `already_in_progress` if lock is held. Then: `get_release_executor(session)` → `executor.execute(report)` run as `asyncio.Task` while a `_heartbeat_loop` task refreshes the TTL every 60s (cancels execute and returns `lock_lost` if the lock is no longer ours). `get_release_executor` resolves the project + token, clones at the **prod rung** (`roboco.models.env_branches.prod_branch`) — not the raw `default_branch` column — computes `env_chain` via `promotion_chain(project)` (the head→…→just-below-prod rungs to promote; empty for a degenerate head==prod ladder), and injects the token as a per-call `http.extraheader` (never into argv); `ci_workflow` is set from `_resolve_release_ci_workflow()` (not self_heal_ci_workflow). `execute`: `is_already_published` (ls-remote tag); `release_commit_sha` (half-landed check — if prior release commit on branch, skip re-bump and rejoin CI→publish tail); else `_run_fresh_release`: `promote_env_chain` (fetch origin + merge each `env_chain` branch into the prod checkout head-first; a fetch/merge failure aborts fail-closed with `promotion_failed` before any bump) → `apply_version_bumps` (replace old version across plan, uv.lock scoped) + `write_changelog_entry` → `run_gate` (make quality, 1800s) → `commit_and_push` (add -A, commit -S, push HEAD:prod_branch; RuntimeError → `commit_failed`) → `wait_for_ci` (poll GitService 80×30s, scoped to release_ci_workflow) → `publish_release` (REST POST to the GitHub releases API — the orchestrator image ships no `gh` binary; RuntimeError → `publish_failed`). On `published` OR `already_published`, the proposal task is set COMPLETED + flushed, and the publish-success path then hands the release to the best-effort post-publish hooks: `_draft_x_post(report)`, `_draft_video(report)`, and `_draft_docs_update(report)`. Each hook catches `Exception` broadly and logs a warning so a drafting/origination failure never affects the already-succeeded release. `_draft_docs_update` invokes `DocsSyncEngine.originate_docs_update(version=report.proposed_version, changelog=report.drafted_changelog)`; if `ROBOCO_DOCS_SYNC_ENABLED` is on and `roboco-website` is registered, exactly one PENDING Main-PM docs-update task is created for that release tag. On gate/CI/commit/publish failure a ReleaseResult is returned and the proposal stays open. The background task commits the session on success, rolls back on failure. The panel polls `GET /proposal` for the final status. `POST /proposal/reject` → `svc.reject(task_id, required_changes)` writes `markers.set_release_required_changes` and keeps the task held. + +## Mermaid +```mermaid +sequenceDiagram + participant MLoop as _release_manager_loop + participant Eng as ReleaseManagerEngine + participant Ready as release_readiness + participant TS as TaskService + participant CEO as CEO (panel) + participant Route as /api/release routes + participant Prop as ReleaseProposalService + participant Exec as ReleaseExecutor + participant Ops as _GitReleaseOps + + MLoop->>Eng: run_cycle() + Eng->>TS: list_open_release_proposals() + alt none open + Eng->>Ready: _production_assess() (read clone + CI) + Ready-->>Eng: ReleaseReadinessReport + Eng->>Eng: _ready_report (green gate + past threshold) + Eng->>TS: create(PENDING, HELD, secretary-1, source=release_manager) + Eng->>TS: markers.set_release_report(report) + Eng-->>CEO: notify (best-effort) + end + + CEO->>Route: POST /api/release/proposal/approve + Route->>Prop: approve(task_id) + Prop->>Prop: acquire Redis SET NX EX mutex + alt mutex held / Redis down + Prop-->>Route: already_in_progress + else acquired + Prop->>Exec: get_release_executor + execute(report) + Exec->>Ops: is_already_published? + Ops->>Ops: apply_version_bumps + write_changelog + Ops->>Ops: run_gate (make quality) + alt gate red + Exec-->>Prop: gate_failed (proposal stays open) + else gate green + Ops->>Ops: commit -S + push + Ops->>Ops: wait_for_ci (poll) + alt CI red + Exec-->>Prop: ci_failed (proposal stays open) + else CI green + Ops->>Ops: gh release create + Exec-->>Prop: published + Prop->>TS: task.status = COMPLETED + end + end + Prop->>Prop: release mutex (finally) + end +``` + +## Logical Tree +``` +release-manager slice +├── release_manager_engine.py (detect loop, default-off) +│ ├── ReleaseManagerEngine +│ │ ├── run_cycle (gate + dedup + originate) +│ │ ├── _ready_report (assess → green + threshold filter) +│ │ ├── _originate (create HELD proposal task + report marker + CEO notify) +│ │ └── _production_assess (read clone + CI conclusion + gather_snapshot + assess) +│ ├── _past_threshold (commit floor OR non-patch OR security) +│ └── _proposal_description (human-readable proposal body) +├── release_readiness.py (pure readiness + git snapshot) +│ ├── primitives +│ │ ├── classify_changes / _classify_one (conventional-commit + label fallback) +│ │ ├── derive_bump (breaking>feat>patch) +│ │ └── next_version (semver apply) +│ ├── report builder +│ │ ├── assess (snapshot → gaps + drafted changelog + plan) +│ │ ├── _changelog_gaps / _version_ref_gaps / _docs_drift_gaps / _migration_gaps_and_notes / _gate_state +│ │ └── _draft_changelog +│ ├── gather_snapshot (read-only git + filesystem I/O) +│ │ ├── _pyproject_version / _last_tag / _commits_since +│ │ ├── _tracked_files_with_version (excludes tests/) +│ │ ├── _canonical_bump_files (prev chore(release): files, subject-filtered, first-release fallback) +│ │ ├── _new_migrations / _migration_head_count +│ │ └── _declared_agent_count / _actual_agent_count +│ └── report_to_dict / report_from_dict (JSONB marker ser/de) +├── release_proposal.py (CEO approve/reject glue) +│ └── ReleaseProposalService +│ ├── open_proposal +│ ├── approve (Redis mutex → executor → COMPLETED on publish) +│ │ ├── _acquire_release_lock (SET NX EX, fail-closed) +│ │ └── _release_release_lock (DEL, best-effort) +│ └── reject (record required_changes, keep held) +└── release_executor.py (fail-closed publish pipeline) + ├── ReleaseExecutor.execute (bump→gate→commit→CI→publish, abort on red) + ├── ReleaseOps Protocol (test seam) + ├── _GitReleaseOps (production: real git/make/gh with deadlines) + │ ├── is_already_published / apply_version_bumps / write_changelog_entry + │ ├── run_gate / commit_and_push / wait_for_ci / publish_release + │ └── _bump_uv_lock / _insert_changelog_entry helpers + ├── _await_proc (subprocess deadline + kill-on-timeout) + └── get_release_executor / _prepare_release_clone (writable clone bootstrap) +``` + +## Dependencies +- Internal: roboco.config.settings (release_manager_enabled, release_min_commits, release_manager_interval_seconds, self_heal_project_slug, self_heal_ci_workflow, workspaces_root, redis_url), roboco.models.env_branches (head_branch, prod_branch, promotion_chain — the env-ladder resolvers backing the release clone/commit/tag target, the full-chain promotion, and the readiness diff baseline), roboco.services.task.TaskService / TaskCreateRequest / RELEASE_MANAGER_SOURCE / get_task_service, roboco.services.project.ProjectService / get_project_service, roboco.services.workspace.WorkspaceService / get_workspace_service / ensure_read_clone / _inject_token_into_url, roboco.services.git.GitService / get_git_service / get_latest_ci_conclusion, roboco.services.notification.NotificationService.send_ack_notification, roboco.services.base.BaseService, roboco.foundation.identity.AGENTS (secretary-1, system), roboco.foundation.policy.content.markers (get_release_report, set_release_report, set_release_required_changes, get_release_required_changes), roboco.models.base (TaskStatus, TaskType, Team, Complexity, TaskNature, AgentRole), roboco.db.tables.TaskTable, roboco.api.routes.release (CEO-only routes), roboco.api.schemas.release, roboco.runtime.orchestrator._release_manager_loop / _run_release_manager_cycle +- External: asyncio (subprocess, wait_for, sleep), subprocess (sync git in release_readiness), pathlib.Path, re, dataclasses, structlog, redis.asyncio, sqlalchemy.ext.asyncio.AsyncSession, fastapi (routes) + +## Entry Points + +| Name | File | Trigger | +|---|---|---| +| _release_manager_loop | roboco/runtime/orchestrator.py | asyncio task created in Orchestrator.start() (line 1063); sleeps release_manager_interval_seconds then _run_release_manager_cycle → get_release_manager_engine(db).run_cycle() | +| GET /api/release/proposal | roboco/api/routes/release.py | CEO panel fetch of the held proposal (CEO-only, 404 when none); panel polls this to get final status after a 202 approve | +| POST /api/release/proposal/approve | roboco/api/routes/release.py | CEO panel approve → dispatch_approve → background _run_approve_background → ReleaseProposalService.approve → ReleaseExecutor.execute (returns 202 immediately; panel polls GET /proposal for outcome) | +| POST /api/release/proposal/reject | roboco/api/routes/release.py | CEO panel reject-with-changes → ReleaseProposalService.reject (keep held) | + +## Config Flags +- ROBOCO_RELEASE_MANAGER_ENABLED (release_manager_enabled, default False) — master switch; when off the loop returns immediately and no proposal is ever originated +- ROBOCO_RELEASE_MIN_COMMITS (release_min_commits, default 8, min 1) — commit floor for the _past_threshold gate +- ROBOCO_RELEASE_MANAGER_INTERVAL_SECONDS (release_manager_interval_seconds, default 3600, min 60) — sleep between assessment passes +- ROBOCO_SELF_HEAL_PROJECT_SLUG (self_heal_project_slug) — reused as the 'this project IS RoboCo' pointer (default 'roboco-api') +- ROBOCO_RELEASE_CI_WORKFLOW (release_ci_workflow, default "ci.yml") — dedicated workflow name for the release fail-closed CI gate; decoupled from ROBOCO_SELF_HEAL_CI_WORKFLOW (which allows empty-string for single-workflow repos — inheriting that would degrade the release gate to the unreliable all-workflows mode); empty or unset falls back to "ci.yml", never None +- ROBOCO_SELF_HEAL_CI_WORKFLOW (self_heal_ci_workflow) — reused as the read-clone CI conclusion in release_manager_engine._production_assess (NOT the executor's release-commit CI gate — that uses ROBOCO_RELEASE_CI_WORKFLOW) +- ROBOCO_WORKSPACES_ROOT (workspaces_root) — base for the read clone and the _release/ writable clone +- ROBOCO_REDIS_URL (redis_url) — the approve-mutex backing store + + +## Gotchas +- [FIXED 05616607+2759edf7] Redis mutex TTL (3000s = 50min) was shorter than the worst-case execute path — RESOLVED by a heartbeat loop (_heartbeat_loop) that calls _heartbeat_release_lock (compare-and-expire Lua) every 60s to refresh the TTL while execute owns the lock. The TTL is now a crash backstop, not a hard ceiling. If the heartbeat detects lock-loss (an extended Redis outage let the TTL expire), it cancels execute fail-closed (lock_lost result) so a concurrent approve can't rm -rf the in-flight clone. +- [FIXED 05616607] _acquire_release_lock formerly returned None on Redis outage causing approve to return already_in_progress — RESOLVED: now raises ReleaseLockUnavailable so approve returns a distinct redis_unavailable result. The CEO knows to fix Redis rather than waiting on a phantom concurrent approve. Still fail-closed (execute never runs). +- [FIXED 2759edf7] _GitReleaseOps.commit_and_push raised RuntimeError that execute did NOT catch → 500. RESOLVED: execute now wraps commit_and_push in try/except RuntimeError and returns a structured commit_failed ReleaseResult. Similarly publish_release RuntimeError now returns publish_failed instead of propagating. +- [FIXED 2759edf7] _await_proc on timeout called proc.kill() but never awaited proc.wait() — zombie risk. RESOLVED: _await_proc now awaits proc.wait() after kill(), and contextlib.suppress(ProcessLookupError) handles already-exited children. +- [FIXED 0bf6c848] The ~40min synchronous HTTP approve blocked the server and 504'd at any proxy. RESOLVED: POST /proposal/approve now returns 202 immediately and dispatches the execute as a background asyncio.Task (dispatch_approve → _run_approve_background with a fresh session). The panel polls GET /proposal for the final status. +- [FIXED 05616607] approve formerly marked COMPLETED only on status=="published" — a retry that finds the tag already published (prior publish whose route 504'd left proposal non-terminal) left it wedged open. RESOLVED: both "published" and "already_published" now close the proposal. +- _canonical_bump_files first-release fallback now returns _tracked_files_with_version (which excludes tests/). Before the change it returned [], so _version_ref_gaps would flag every version-ref file on first release; now the fallback makes planned==tracked so first-release version_ref gaps are silently empty. Intended, but means the first-release gap report is weaker than subsequent releases. +- _canonical_bump_files uses `git log --grep ^chore(release):` then filters by subject prefix; if a real release commit's subject was ever not exactly `chore(release): X.Y.Z` (e.g. a merge-commit subject), it would be skipped and a stale/false bump set used. +- [FIXED 2759edf7] wait_for_ci formerly inherited self_heal_ci_workflow which allows empty/None, risking an all-workflows-mode conclusion on a multi-workflow repo. RESOLVED: the executor now calls _resolve_release_ci_workflow() which always returns a non-empty named workflow (ROBOCO_RELEASE_CI_WORKFLOW or "ci.yml" fallback), so wait_for_ci always scopes the CI poll to a specific workflow. +- The proposal is created with status=PENDING and confirmed_by_human=False but the release-manager loop NEVER cancels it on a subsequent cycle — list_open_release_proposals dedups by non-terminal status, so a stale PENDING proposal blocks all future proposals until the CEO acts. There is no expiry/reaper for an abandoned proposal. +- apply_version_bumps does a naive str.replace(old, new) on every non-uv.lock, non-CHANGELOG file in the plan. If the current version string appears as a substring of an unrelated value in any bump file (e.g. a comment, a path), it gets clobbered. The bump plan is derived from the previous release commit's files, so this is bounded but not surgical. +- _prepare_release_clone rm -rf's workspaces_root/_release/ with no locking at the filesystem level — the Redis mutex is the only guard, and it is per-proposal-task, not per-clone-path. Two different proposals for the same slug (impossible while dedup holds, but dedup is by source+non-terminal, so a CANCELLED + new PENDING could overlap) would race the rm -rf. +- `_approve_precheck`'s two refusals (already_rejected / already_published) exist because a live-reproduced hole let a stale approve resurrect a proposal the CEO had already rejected or that a concurrent approve had already published — the callback surface that made this reachable is Telegram's inline Approve button, which targets a proposal by id regardless of its current status (a stale, still-clickable button in an old chat message), but the same hole was equally reachable by replaying the HTTP route, so the fix is in the service, not the Telegram layer. + +## Changes Since Baseline + +| SHA | Subject | Impact | +|---|---|---| +| 15effce0 | Chore: 141 Gaps fill-in (#283) — release_executor.py | Added per-subprocess deadlines via _await_proc (git 300s, gate 1800s, publish 300s, clone 600s) with kill-on-timeout returning rc=124 so fail-closed branches fire instead of hanging the release loop. commit_and_push now checks add/commit rc and raises RuntimeError on failure (previously fire-and-forget: a failed commit would still push the pre-bump base as the release). | +| 15effce0 | Chore: 141 Gaps fill-in (#283) — release_proposal.py | Added a Redis SET-NX-EX mutex keyed by proposal id around approve() (F013) — a concurrent approve/double-click returns already_in_progress instead of racing on the rm -rf'd writable clone. Fail-closed on Redis outage (treated as held). Proposal still only COMPLETED on status==published. | +| 15effce0 | Chore: 141 Gaps fill-in (#283) — release_readiness.py | _canonical_bump_files signature changed to take `version`; the git-log grep now filters candidates by subject prefix `chore(release):` (was matching any body line that referenced the type, shadowing the real release commit). First-release fallback changed from returning [] to returning the version-ref scan, so the first release now has a real bump plan and no spurious version_ref gaps. | + +> Post-snapshot updates (since 2026-06-29): +> - 536bbb64 Chore/all/logical gaps sweep (#286) — PR merge carrying the sweep commits below. +> - 2759edf7 [B-REL] release executor: idempotent half-landed retry + commit-scoped CI + decoupled workflow — adds `release_commit_sha` to ReleaseOps/`_GitReleaseOps` for half-landed (publish_failed) retry detection (reuse existing release commit, skip re-bump); execute now catches commit_and_push RuntimeError → `commit_failed` and publish_release RuntimeError → `publish_failed`; `_await_proc` now awaits `proc.wait()` after kill (zombie fix); decoupled release CI gate from self_heal_ci_workflow via new `_resolve_release_ci_workflow()` / `settings.release_ci_workflow` (`ROBOCO_RELEASE_CI_WORKFLOW`, default "ci.yml"). +> - 05616607 [chore] logical-gaps: release-proposal already_published closes proposal + heartbeat-lock-loss cancels execute — adds `ReleaseLockUnavailable` exception (Redis outage → `redis_unavailable` result, not `already_in_progress`); fencing-token compare-and-del (`_RELEASE_LOCK_RELEASE_SCRIPT` Lua) + compare-and-expire heartbeat (`_RELEASE_LOCK_HEARTBEAT_SCRIPT` Lua, `_heartbeat_loop`, `_heartbeat_release_lock`); `lock_lost` result when heartbeat detects TTL expiry and cancels execute fail-closed; `_finalize_release_lock` finally helper; approve now closes proposal on `already_published` in addition to `published`. +> - 0bf6c848 [chore] logical-gaps: release approve async dispatch (202) — adds `dispatch_approve` + `_run_approve_background` + `_INFLIGHT_APPROVES` registry; POST /proposal/approve returns 202 immediately; the ~40min execute runs in a background asyncio.Task with a fresh session; panel polls GET /proposal for final status. +> - b3558d4e [chore] complexity: split 5 C-rank blocks to <=B for the xenon gate — refactored large methods in executor/proposal for xenon compliance; no behavior change. +> - 8621d01d / fe9940de / d80dfb8b (#534, env-branches ladder) — replaces `default_branch`-keyed release targeting: `_ReleaseContext` gains `prod_branch` (renamed from `default_branch`) + `env_chain`; `get_release_executor` resolves the clone/commit/tag target via `roboco.models.env_branches.prod_branch` and computes `env_chain` via `promotion_chain`; `_GitReleaseOps` gains `promote_env_chain` (fetch + head-first merge of `env_chain` into the prod checkout, fail-closed `promotion_failed` on conflict), run as the first step of `_run_fresh_release`, before any version bump; `release_readiness.gather_snapshot` takes an optional `prod_branch` and diffs `prod..head` (falling back to `last_tag..HEAD` when unset) with a new `_tag_drift_gaps` check (last-tag sha vs. prod tip); `release_manager_engine._production_assess` best-effort fetches the prod rung into the head-pinned read clone before gathering the snapshot. A project with no declared ladder is unaffected (the shim resolves prod_branch/head_branch to the same `default_branch` value). +> - `11915f36` (PR #551, Telegram V2 security follow-up) — `_approve_precheck` (new) makes `approve()` refuse a CANCELLED proposal (`already_rejected`) or a COMPLETED one (`already_published`) BEFORE touching the Redis lock/executor; `reject()` now raises the new `TaskAlreadyCompletedError` on a COMPLETED proposal instead of silently cancelling an already-public release. Closes a live-reproduced approve-after-reject hole reachable via a stale Telegram Approve button (or a replayed HTTP call). + +## Regression Risks + +| Title | File:Line | Claim | Severity | +|---|---|---|---| +| [RESOLVED 05616607+2759edf7] Redis mutex TTL shorter than worst-case execute — concurrent approve can race the rm -rf clone | roboco/services/release_proposal.py:52 | RESOLVED: heartbeat loop (_heartbeat_loop) refreshes TTL every 60s via compare-and-expire Lua while execute is running; the 3000s TTL is now a crash backstop. On lock-loss (extended Redis outage let TTL expire) the heartbeat cancels execute fail-closed and approve returns lock_lost — a concurrent approve therefore cannot rm -rf the in-flight clone. | high | +| [RESOLVED 2759edf7] commit_and_push RuntimeError unhandled by execute → 500 instead of structured ReleaseResult | roboco/services/release_executor.py:336 | RESOLVED: execute now wraps commit_and_push in try/except RuntimeError and returns commit_failed; publish_release RuntimeError returns publish_failed. All failure paths now surface as structured ReleaseResult, not 500s. | medium | +| [RESOLVED 05616607] Redis outage fully blocks release approval (fail-closed = treat as held) | roboco/services/release_proposal.py:207 | RESOLVED: _acquire_release_lock raises ReleaseLockUnavailable on any Redis exception; approve returns redis_unavailable (distinguished from already_in_progress so the CEO knows to fix Redis rather than waiting). Still fail-closed — execute never runs without the mutex. | medium | +| _canonical_bump_files first-release fallback silences version_ref gaps | roboco/services/release_readiness.py:444 | On first release (no prior chore(release): commit) the bump plan equals _tracked_files_with_version, so _version_ref_gaps emits zero gaps. The CEO no longer sees 'these files hold the version but are not in the bump plan' on the first release — weaker readiness signal, intentional by design (comment explains). | low | +| [RESOLVED 2759edf7] _await_proc leaves zombie on timeout (kill without wait) | roboco/services/release_executor.py:214 | RESOLVED: _await_proc now awaits proc.wait() after proc.kill(); contextlib.suppress(ProcessLookupError) handles already-exited children. | low | + +## Health +The slice is well-structured: deterministic correctness lives in pure primitives (release_readiness) with a Protocol seam (ReleaseOps) making the fail-closed ordering unit-testable, and the detect→originate→hold→CEO-approve→publish separation is clean and matches CLAUDE.md. Post-snapshot hardening rounds (2759edf7, 05616607, 0bf6c848) resolved all four previously-flagged regression risks: (1) the Redis mutex TTL race is closed by a heartbeat loop that keeps the TTL refreshed and aborts execute fail-closed on lock-loss; (2) commit_and_push/publish_release RuntimeErrors are now caught by execute and returned as structured commit_failed/publish_failed results (no 500); (3) Redis outage now returns redis_unavailable (not already_in_progress) so the CEO knows to fix Redis; (4) the zombie-on-timeout is fixed by awaiting proc.wait(). The approve route is now async-202 with a background dispatcher (dispatch_approve). The half-landed (publish_failed) retry path (release_commit_sha) closes the prior gap where a second CEO approve re-inserted the changelog entry and created a duplicate release commit. The release CI gate is decoupled from self_heal_ci_workflow via a dedicated settings.release_ci_workflow. One low-severity known-by-design item remains: first-release version_ref gap suppression (intentional). release_manager_engine.py and release_readiness.py are unchanged since 15effce0. + ## Purpose This slice is the agent-runtime + LLM-provider seam plus the in-container agent SDK. The provider layer (roboco/llm/providers/) abstracts how agents are spawned/stopped/health-checked/removed across LLM backends (Claude Code default, Grok CLI) behind an AgentProvider ABC + ProviderRegistry, with a Grok auth-token refresh loop keeping the SuperGrok credential live. The agent SDK (roboco/agent_sdk/) is the FastAPI sidecar running inside every agent container handling A2A messaging, tool-budget/loop/verb-circuit breakers, token-usage capture, and the interactive intake/secretary chat drivers (Claude SDK + Grok CLI). The runtime helpers (spawn_manifest, streaming, transcript_retention) build the per-role tool manifest, wire reasoning-stream callbacks, and select old agent transcripts to prune. @@ -5285,950 +4814,2931 @@ runtime-providers This slice is coherent and well-factored: the provider ABC + registry cleanly isolates the Grok backend while the Anthropic/Ollama/LOCAL paths stay on the built-in spawn (additive seam, no destabilization), and the agent_sdk sidecar centralizes budget/loop/verb-circuit/token state that hooks share. The single baseline-to-HEAD commit (15effce0) landed three genuine hardening fixes — the F006 refresh_token-loss guard with direct-write fallback, the JWT-exp decode so a refreshed token isn't forever rejected, and the /usage/sync path-traversal guard — plus the grok directory-mount fix that resolves the inode-pinning hang. The main integrity concerns are operational rather than structural: the grok directory mount widens RO exposure to host grok state, the 6h expires_at default can burn the single-use refresh_token on the rare double-miss, the in-process SDK state is lost on every container restart (by design, but means verb-circuit/budget counters reset), and ClaudeCodeProvider is dead reference code whose 'default' label in CLAUDE.md is misleading. Interactive intake/secretary parity between Claude and Grok is real (shared IntakeDriver, only the SessionFactory differs). No obviously broken logic was introduced; the regression risks are edge-case behavior shifts, not holes. Recommend re-running the grok auth refresh test against a token that omits expires_in to confirm the JWT-exp path, and a /usage/sync test with a symlinked transcript to confirm the new guard fails loud where appropriate. ## Purpose -This slice is the prompt-composition pipeline and the role/team/permission taxonomy that feeds it. At agent spawn the orchestrator resolves an agent's role+team from the canonical foundation-derived maps in agents_config.py, then compose_prompt layers (in order) a tool-load directive, the autogenerated lifecycle verb surface, the universal base rules, an optional fable-mode doctrine layer (`fable_mode_enabled`), an optional ponytail build-laziness doctrine layer (bundled with Fable, role-scoped — developers get the full ladder, other roles get the ethos-only cut), the role prompt, the autogenerated per-role verb-signature table, the team prompt, the agent identity, and an optional architectural-conventions ambient block — writing the result to a per-agent .md file the runtime mounts. A separate prompt-injection guard (prompt_guard.py) denies poisoned incoming turns at the interactive input boundary for both Claude-SDK and Grok sessions, mirroring the bash UserPromptSubmit hook. agents_config.py is also the MCP-layer permission taxonomy (HMAC agent tokens, role/team helpers, escalation chain, A2A routing) that gates tool visibility and identity-binding at spawn. + +The Obsidian vault (V1+V2): a rebuildable, human-readable DB projection of the org's memory (tasks, journal entries, A2A thread digests) as wikilinked markdown, plus a default-off inbox watcher that turns `#roboco`-tagged vault notes into board-review intake drafts. V2 adds three things on top of the V1 projection: materialize-on-create (a task's note exists from the moment it's created, not just at curation/rebuild), a drift janitor (hourly-ticked, daily/weekly-gated: re-projects changed tasks, verifies a random sample, archives old terminal tasks, writes the weekly org-report), and KB ingest (the CEO's own `RoboCo/Notes/` notes become one more RAG corpus the fleet can retrieve). Default-off (`ROBOCO_OBSIDIAN_VAULT_ENABLED`; both compose files arm it `true`). Still structurally different from the other default-off engines: the projection never originates delivery work itself — the ONE writer-side effect that reaches delivery (the intake watcher) rides the existing board-review path, not a held-artifact queue. ## Files -| Path | Role | LOC | +| Path | Role | approx LOC | |---|---|---| -| roboco/agents/factories/_base.py | Layered prompt composer: loads/concatenates tool-directive + lifecycle + base + an optional fable-mode doctrine + an optional ponytail build-laziness doctrine + role + autogen-verbs + team + identity + ambient layers; exports PROMPTS_BASE_PATH, role/team/builtin-tool maps, compose_prompt, fable_doctrine_layer, ponytail_doctrine_layer, conventions_ambient_layer, make_slug | 298 | -| roboco/agents/factories/__init__.py | One-line re-export shim pointing to _base | 1 | -| roboco/agents_config.py | 691-line MCP-layer permission/taxonomy module: HMAC agent-token issue/verify, role+team+cell maps derived from foundation, escalation chain, ROLE_PERMISSION_LEVELS, ROLE_SKILLS, A2A routing helpers | 691 | -| roboco/agent_sdk/prompt_guard.py | Reusable Python port of the bash injection guard: _PATTERNS regex list, detect_injection, refusal_message, CLI main (exit 1 on injection) for the grok entrypoint | 93 | -| agents/prompts/base.md | Universal base layer: identity separation, gateway-verb-only action, envelope shapes, missing-key cheatsheet, resume-from-briefing, charter alignment, todo rules, ground rules | 93 | -| agents/prompts/doctrine/fable.md | Vendored Fable-5 behavioral doctrine (from `github.com/rennf93/opus-fable-playbook`, MIT, YAML frontmatter stripped): communication/turn-discipline/autonomy-calibration/honesty/code-discipline/delegation/precedence sections; loaded only when `fable_mode_enabled`, injected right after base.md via `fable_doctrine_layer()` | 47 | -| agents/prompts/doctrine/ponytail.md | Vendored Ponytail build-laziness doctrine for developers (from the ponytail plugin, MIT, Copyright (c) 2026 DietrichGebert, trimmed, YAML frontmatter stripped): the ladder (YAGNI → reuse-in-this-codebase → stdlib → native-platform → installed-dep → one-line → minimal), the rules, the Intensity table (lite/full/ultra), a 5-point RoboCo preamble that makes the ladder yield to placement / coverage gate / design bar / task hygiene / reviewer feedback, and the `ponytail:` comment convention; loaded only when `fable_mode_enabled`, injected right after the Fable doctrine via `ponytail_doctrine_layer()` for `AgentRole.DEVELOPER` only, with a trailing `**Operative intensity: {ponytail_intensity}.**` directive | 88 | -| agents/prompts/doctrine/ponytail-ethos.md | Vendored Ponytail ethos-only doctrine for non-developer roles (same source/attribution, trimmed): the ethos rules and the RoboCo preamble (the 6th point guards free-text field obligations), with the code-mechanics rungs (the ladder) and the Intensity table removed so they can't leak into prose artifacts; loaded by `ponytail_doctrine_layer()` for every role except `DEVELOPER`; no intensity directive (ethos runs a fixed restrained stance) | 40 | -| agents/prompts/roles/developer.md | Developer role prompt: implement-only identity, verb table (give_me_work/i_will_work_on/commit/open_pr/i_am_done/sync_branch/...), workspace path, behind-base -> sync_branch guidance, conventions waiver path, a pointer scoping frontend/ux_ui's `## Design bar` to those teams only | 25176 | -| agents/prompts/roles/qa.md | QA role prompt: review-only identity, claim_review/pass/fail/i_am_blocked verbs, ac_verdicts per criterion, circuit-breaker guidance | 13355 | -| agents/prompts/roles/documenter.md | Documenter role prompt: docs-on-same-branch identity, claim_doc_task/commit/i_documented/i_am_blocked verbs, circuit-breaker guidance | 10739 | -| agents/prompts/roles/cell_pm.md | Cell PM role prompt: coordinator identity, i_will_plan/delegate/complete/submit_up/unblock verbs, AC coverage gate, collision-surface declaration (intends_to_touch/adds_migration/touches_shared/depends_on), behind-base escalation | 35682 | -| agents/prompts/roles/main_pm.md | Main PM role prompt: org-level coordinator, delegate/complete/submit_root/triage_all/unblock/escalate_to_ceo verbs, upstream-handoff precondition, branch-bearing vs branchless root gate | 32363 | -| agents/prompts/roles/pr_reviewer.md | PR Reviewer role prompt: external-PR review (claim_pr_review/post_pr_review) + in-path gate (claim_gate_review/pr_pass/pr_fail), trust gate, conventions strictness | 8129 | -| agents/prompts/roles/board.md | Board role prompt (PO/HoM/Auditor): strategic overseer, triage/escalate_to_ceo, no unblock verb, Auditor silent | 12253 | -| agents/prompts/roles/prompter.md | Intake interviewer role prompt: CEO-only chat, propose_draft/propose_batch, MegaTask batch + per-cell-project-map drafting, root-subtask coordination-level AC guidance | 13825 | -| agents/prompts/roles/secretary.md | Secretary role prompt: CEO chief-of-staff, gated-action confirm protocol, reading-free/prepare-direct/high-impact-bounce discipline | 4338 | -| agents/prompts/teams/backend.md | Backend team layer: Python/FastAPI/Postgres stack, teammates, uv quality commands | 983 | -| agents/prompts/teams/frontend.md | Frontend team layer: TS/Next.js stack, pnpm quality commands, `## Design bar` section (taste-skill-distilled layout/typography/motion/spacing rules + 3 tuning dials + an "AI tells to avoid" list) | 976 | -| agents/prompts/teams/ux_ui.md | UX/UI team layer: design-system focus areas, teammates, `## Design bar` section (same taste-skill basis as frontend.md, plus a design-artifact-to-code handoff bullet), plus a video-mode override line: video-authoring (`source=video`) tasks are FILMS not UI, so the section's web dials (DESIGN_VARIANCE/MOTION_INTENSITY/VISUAL_DENSITY) do not apply — use `motion/README.md`'s cinematography bar and the vendored `motion/skills/` doctrine instead | 1009 | -| agents/prompts/identities/ | 19 per-agent identity files (be-dev-1/2, fe-dev-1/2, ux-dev-1/2, be/fe/ux -qa/-doc/-pm, main-pm, product-owner, head-marketing, auditor): YAML id/name/role/team/cell/reports_to + scope blurb; loaded by slug | 0 | -| agents/prompts/_generated/lifecycle-developer.md | Autogenerated lifecycle verb list for developer (regenerated from lifecycle spec by make lifecycle); lists sync_branch etc. | 1498 | -| agents/prompts/_generated/lifecycle-main_pm.md | Autogenerated lifecycle verbs for main_pm; submit_root now branch-keyed not task_type-keyed | 2034 | -| agents/prompts/_generated/lifecycle-cell_pm.md | Autogenerated lifecycle verbs for cell_pm | 1839 | -| agents/prompts/_generated/lifecycle-qa.md | Autogenerated lifecycle verbs for qa | 818 | -| agents/prompts/_generated/lifecycle-documenter.md | Autogenerated lifecycle verbs for documenter | 736 | -| agents/prompts/_generated/lifecycle-pr_reviewer.md | Autogenerated lifecycle verbs for pr_reviewer | 1071 | -| agents/prompts/_generated/lifecycle-product_owner.md | Autogenerated lifecycle verbs for product_owner | 421 | -| agents/prompts/_generated/lifecycle-head_marketing.md | Autogenerated lifecycle verbs for head_marketing | 422 | -| agents/prompts/_generated/lifecycle-auditor.md | Autogenerated lifecycle verbs for auditor (triage + i_am_idle) | 325 | -| agents/prompts/_generated/lifecycle-prompter.md | Autogenerated lifecycle verbs for prompter (i_am_idle only — driver-based) | 275 | -| agents/prompts/_generated/lifecycle-secretary.md | Autogenerated lifecycle verbs for secretary (i_am_idle only — driver-based) | 276 | -| agents/prompts/_generated/lifecycle-ceo.md | Autogenerated lifecycle verbs for ceo (empty — human, not spawned) | 181 | -| agents/prompts/_generated/lifecycle-system.md | Autogenerated lifecycle verbs for system sentinel (empty) | 184 | -| agents/prompts/_generated/developer.md | Per-role autogenerated verb-signature table (Flow + Content tools) for developer, regenerated by scripts/regenerate_verb_tables.py from Pydantic schemas + role_config | 2384 | -| agents/prompts/_generated/qa.md | Per-role autogenerated verb-signature table for qa (pass_review ac_verdicts carries BeforeValidator) | 2040 | -| agents/prompts/_generated/documenter.md | Per-role autogenerated verb-signature table for documenter | 2083 | -| agents/prompts/_generated/cell_pm.md | Per-role autogenerated verb-signature table for cell_pm; delegate signature includes intends_to_touch/adds_migration/touches_shared/depends_on | 3270 | -| agents/prompts/_generated/main_pm.md | Per-role autogenerated verb-signature table for main_pm; delegate signature includes collision-surface fields | 3316 | -| agents/prompts/_generated/pr_reviewer.md | Per-role autogenerated verb-signature table for pr_reviewer (claim_gate_review/pr_pass/pr_fail + claim_pr_review/post_pr_review) | 1461 | -| agents/prompts/_generated/product_owner.md | Per-role autogenerated verb-signature table for product_owner (triage/escalate_to_ceo + pitch/notify) | 1765 | -| agents/prompts/_generated/head_marketing.md | Per-role autogenerated verb-signature table for head_marketing | 1765 | -| agents/prompts/_generated/auditor.md | Per-role autogenerated verb-signature table for auditor (triage/i_am_idle + note/evidence + approve/reject/archive_playbook) | 1273 | -| agents/prompts/_generated/verbs.md | Aggregate reference doc of all per-role verb shapes (NOT injected at spawn; _base.py loads the per-role file instead); notes driver-based roles omitted | 18302 | +| `roboco/services/vault_writer.py` | `VaultWriter` — pure, DB-free markdown materializer. `write_task` / `write_journal_entry` / `append_a2a_message` / `write_agent` / `touch_task_frontmatter` / `write_org_report`. Every note carries `aliases: []` so a rename never breaks a `[[id8\|title]]` wikilink; `existing_narrative` reads back an Auditor-authored `## Narrative` section so a rebuild never clobbers it. V2: `write_task` is archive-aware (`TaskNoteData.archive_year` routes it to `RoboCo/Archive//Tasks//` instead of `Tasks//`, removing the stale copy on a move); `find_task_note`/`task_note_status` locate/inspect a note wherever it lives (recursive id8 lookup across both trees) for the janitor's drift check; `write_org_report` renders `RoboCo/Reports/.md`. (Uncommitted, `feature/findings-ledger`) `FindingRow` + `_FINDINGS_CAP=20` + `_findings_section` render a `## Findings` section (one `[F-id8] (severity, round N, status) file:line — expected → actual → fix` line per open/resolved finding, an overflow line past the cap) into every task note — see `docs/map/review-findings.md`. | 534 | +| `roboco/services/vault_assembly.py` | `assemble_task_note_data` — resolves a task's project slug, parent, subtasks, dependencies, and (V2) archive eligibility (`_archive_year`, gated on `vault_archive_days`) via the live `TaskService`/`ProjectService` into a `TaskNoteData`. `reproject_task` (V2) bundles assemble + narrative-preservation + `write_task` into the one code path shared by `rebuild`, the janitor's changed/sample/archival passes, and the create-on-task seam — none of them can drift on how a note gets refreshed. (Uncommitted, `feature/findings-ledger`) `_resolve_findings` fetches the task's ledger via `ReviewFindingsRepository.list_for_task`, fails open (empty tuple) on a missing session or any exception — a findings-fetch failure drops the section, never blocks the note write. | ~125 | +| `roboco/services/vault_intake_engine.py` | `VaultIntakeEngine.run_cycle` — scans the vault's inbox dir for `#roboco`-tagged notes, dedupes via `vault_seen_notes` (path + content-hash), screens the body through `injection_guard.screen_external_text`, extracts a title/description/action-items via a local-model chat call (deterministic fallback: first heading / raw body / checkbox lines), and opens ONE PENDING board-review draft (`source=vault_note`, Product-Owner-assigned, `team=board`) per note. Appends a feedback callout back into the note (best-effort). V2: the frontmatter split + content-hash helpers moved to the shared `foundation/policy/vault_notes.py` (this module now just imports them). | ~350 | +| `roboco/services/vault_janitor.py` **(new, V2)** | `VaultJanitor.run_cycle` — one state-gated sweep: re-project tasks changed since the last sweep (`TaskService.list_updated_since`, capped/paged, per-item isolated), verify a random stale sample (`sample_stale_tasks`), archive old terminal tasks (`list_archive_candidates`), and (weekly) render the org-report. Dueness is tracked in a JSON state file (`RoboCo/_meta/.janitor_state.json`: `last_sweep`, `last_report_week`, `archive_watermark`), not the loop's own cadence — restart-proof. | ~343 | +| `roboco/services/vault_kb_engine.py` **(new, V2)** | `VaultKBEngine.run_cycle` — scans the allowlisted `vault_kb_dirs` (default `RoboCo/Notes`), dedups by content hash, screens every note body through the injection guard as a hard GATE (flagged → quarantined, never indexed), and ingests/deindexes into `IndexType.VAULT_NOTES` via `OptimalService.index_vault_note`/`unindex_vault_note`. Defense-in-depth containment: symlinks and any resolved-path escape from the vault root are skipped, independent of the config-load validator. | ~315 | +| `roboco/foundation/policy/vault_notes.py` **(new, V2)** | Shared pure helpers: `content_hash` (sha256 with every `> [!kind] RoboCo: ...` feedback callout stripped first, so appending one doesn't change what the next scan considers "changed") and `split_frontmatter` (YAML frontmatter + body). Used by both the intake watcher's "drafted" callout and the KB engine's "quarantined" callout — one shared convention instead of two copies drifting. | ~46 | +| `roboco/services/optimal_brain/indexes/vault_notes.py` **(new, V2)** | `VaultNotesIndexPlugin` — `IndexType.VAULT_NOTES` plugin, mirrors `PlaybooksIndexPlugin`'s shape (`index_note`/`delete_note`/`search_notes`, source URI `vault://`). Scope enforced by the KB engine's dir allowlist, not this plugin. | ~70 | +| `roboco/vault.py` | `python -m roboco.vault {rebuild,relocate}` CLI. `rebuild` re-projects every agent/task/journal-entry/A2A-thread from the DB (now archive-aware via `vault_assembly.reproject_task` — an old terminal task projects straight into `Archive//`) and materializes `.obsidian/` + `RoboCo/_meta/` from `roboco/vault_assets/` (never overwrites an existing file). `relocate ` moves the tree; grafts `RoboCo/` into an existing destination vault without touching its own config. | ~237 | +| `roboco/vault_assets/` | Packaged templates copied by `ensure_vault_assets`: `.obsidian/` (Dataview, Kanban, graph-group config — V2 adds `Archive`/`Reports` graph color groups) + `meta/` (dashboard + kanban-board + README, V2 adds `Task Board.base` + `Reports.base` for Obsidian's core Bases plugin, and `Sync to your Mac.md`, the Syncthing/SMB/Obsidian-Sync runbook). Dataview dashboard queries now exclude `Archive/`. | — | +| `roboco/foundation/policy/injection_guard.py` | `screen_external_text` / `detect_injection` — the shared prompt-injection screen-and-neutralize (data path) and hard-deny (interactive-input path) pattern set. V2 reuses it a third time: the KB engine's ingest-time hard gate (quarantine on a hit, vs. the intake watcher's screen-and-still-process posture). | 125 | +| `roboco/services/gateway/content_actions.py` `curate_vault` | Server-side do-action: Auditor-only, re-materializes a task's note with the Auditor's `narrative` filling `## Narrative`. Inert (`invalid_state`) when the flag is off. | — | +| `roboco/mcp/do_server.py` `curate_vault` | Do-tool the Auditor calls exactly once per completed root, POSTing to `/api/v1/do/curate_vault`. | — | +| `roboco/db/tables.py` `VaultSeenNoteTable` | Dedup ledger for the intake watcher: `(note_path, content_hash)` — an unchanged note is never reprocessed; an edited one is eligible again. | — | +| `roboco/services/task.py` `_materialize_vault_note` / `list_updated_since` / `list_archive_candidates` / `sample_stale_tasks` | V2: the create-time seam + the janitor's three query methods. See `docs/map/task-service.md`. | — | + +## Data Flow + +**PROJECTION (always-on when the flag is armed).** `TaskService.create` (V2) calls `_materialize_vault_note` — best-effort, same swallow-and-log posture as every other seam — so a task's note exists from the moment it's created, not just at curation/rebuild. Three more best-effort event seams fire from existing services: `TaskService._emit_status_transition_audit` → `_touch_vault_frontmatter` patches an EXISTING note's status/team/pr fields in place (now effectively always finds one for any task created post-V2, since materialize-on-create ran; a pre-V2 task without a note is still a no-op here — the janitor's changed/sample passes are what backfill it); `JournalService`'s entry-write path → `_materialize_vault_note` writes one immutable file per non-private entry; `A2AService.send` → `_materialize_vault_note` appends to a per-thread digest file, deduped per message id via an in-body marker comment. All import `get_vault_writer()` lazily and catch every exception. + +**CURATION (root-completion hook, orchestrator-driven).** `AgentOrchestrator._dispatch_vault_curation_work` (one of the 18 tick dispatchers, gated on `obsidian_vault_enabled`) reads `TaskService.list_completed_roots_pending_vault_curation` and calls `_maybe_spawn_vault_curation` per candidate: an in-memory one-shot guard (`_board_dispatched`) plus a durable `vault_curation_dispatched` marker (survives a restart) precede a bindingless Auditor spawn. The Auditor writes one narrative paragraph and calls `curate_vault(task_id, narrative)` exactly once; the verb re-resolves the task's parent/subtasks/dependencies fresh via `assemble_task_note_data` and fully re-materializes the note, filling the `## Narrative` section a deterministic write otherwise leaves as `_Pending Auditor curation._`. + +**INTAKE (independently-gated inbox watcher).** `AgentOrchestrator._vault_intake_loop` (both `obsidian_vault_enabled` AND `vault_intake_enabled` required) ticks `VaultIntakeEngine.run_cycle` every `vault_intake_interval_seconds`. Per note under the inbox dir: skip if no `#roboco` tag; skip if already seen (path + content-hash in `vault_seen_notes`); screen the body via `screen_external_text`; extract title/description/action-items via a local-model chat call against the SCREENED text, falling back to deterministic extraction; open ONE PENDING task (`source=vault_note`, Product-Owner-assigned, `team=board`) capped by `vault_intake_max_open_drafts`/`vault_intake_max_per_cycle`; append a feedback callout (best-effort). Never starts delivery directly — the board-review path is the only door. + +**JANITOR (V2, hourly-ticked, day/week-gated).** `AgentOrchestrator._vault_janitor_loop` (gated on `obsidian_vault_enabled` alone) ticks every `JANITOR_LOOP_INTERVAL_SECONDS` (3600, no config knob) and calls `VaultJanitor.run_cycle`. Actual work only happens when the restart-proof state file (`RoboCo/_meta/.janitor_state.json`) says it's due: +- **Sweep** (due when `last_sweep` is >= 24h stale): `_reproject_changed` re-projects every task touched since the last sweep (`TaskService.list_updated_since`, ascending, paged 100 at a time, capped at `_MAX_REPROJECT_PER_CYCLE=200` per tick, one bad item logged-and-skipped rather than wedging the pass) via the shared `reproject_task`; `_verify_sample` pulls a random 20-task sample of tasks last touched before the sweep window (`sample_stale_tasks`) and repairs any whose note is missing or whose frontmatter status disagrees with the DB (via `touch_task_frontmatter`, not a full re-projection); `_archive_pass` moves terminal tasks past `vault_archive_days` into `Archive//` (see below). A capped tick advances `last_sweep`/`archive_watermark` only to the last-processed item's stamp (not "now"), so the very next hourly tick — already due again — picks up the tail with no gap. Logs one `vault_drift_repaired` line: `count` (repaired) / `archived` / `failed`. +- **Weekly report** (due when `last_report_week` != the current ISO week, and `vault_report_enabled` is on): `_run_weekly_report` pulls `MetricsService.get_velocity/get_cycle_time_by_stage/get_bottleneck_distribution/get_rework_metrics` (days=7) + `UsageService.get_summary("7d")`, renders `VaultWriter.write_org_report`, and best-effort notifies the CEO (`NotificationService.send_weekly_report_notification`) — a notification failure never invalidates the already-written note. + +**ARCHIVAL (V2, folded into the janitor sweep).** Policy: a terminal (`completed`/`cancelled`) task whose terminal timestamp (`completed_at` else `updated_at` else `created_at`) is older than `vault_archive_days` (default 30; `0` disables archival outright) moves from `RoboCo/Tasks//` to `RoboCo/Archive//Tasks//`. `TaskService.list_archive_candidates(after, before, ...)` returns terminal tasks whose terminal timestamp falls in `[watermark, cutoff)`, paged/capped identically to the changed-task pass (`_MAX_ARCHIVE_PER_CYCLE=200`). The move itself is free: `VaultWriter.write_task` is archive-aware (`TaskNoteData.archive_year` set by `vault_assembly._archive_year`) — it looks up an existing note across BOTH `Tasks/` and `Archive/` by id8, writes the new copy at the archive-aware target directory, and deletes the stale copy if it moved. Alias-based wikilinks (`[[id8|title]]`) mean nothing pointing at an archived task ever breaks. `rebuild` is archive-aware for free (routes through the same `reproject_task`), and the shipped Dataview dashboard + graph color groups exclude `Archive/`. + +**KB INGEST (V2, independently double-gated).** `AgentOrchestrator._vault_kb_loop` (BOTH `obsidian_vault_enabled` AND `vault_kb_enabled` required) ticks `VaultKBEngine.run_cycle` every `vault_kb_interval_seconds` (default 900). Per allowlisted dir in `vault_kb_dirs` (default `RoboCo/Notes`; config-load validation in `Settings._validate_vault_kb_dirs` rejects an absolute/`..`-carrying entry or one overlapping `vault_intake_dir` or a reserved projection dir): recursively scan `*.md`, skip a note that's a symlink, escapes the resolved vault root, or exceeds 64KB; content-hash-dedup against the currently-tracked `IndexType.VAULT_NOTES` docs (an unchanged note is skipped); screen the frontmatter-stripped body through `screen_external_text` as a hard GATE — a flagged note is quarantined (skipped, warn-logged, a one-line feedback callout appended, and any PRIOR indexed chunks removed if it was previously clean) rather than indexed; a clean note ingests via `OptimalService.index_vault_note` (bounded to `_MAX_INGEST_PER_CYCLE=50` per tick — the tail waits for the next cycle). A deletion pass deindexes any previously-tracked path no longer seen on disk. Consumers: `roboco_kb_search` picks up `VAULT_NOTES` for free once the enum exists; `MentorService`'s default/general domain search list includes it (labeled "Vault Notes"); `EvidenceRepo.similar_memory` includes it in claim-time briefings (kind `vault_note`), same relevance floor as learnings/playbooks; the panel's KB browser has a full type entry (nav/filter/badge/stats). + +**REBUILD/RELOCATE (operator/CLI, not agent-facing).** `python -m roboco.vault rebuild` walks every agent, then every task (via the shared `reproject_task` — archive-aware, narrative-preserving), then every non-private journal entry, then every A2A thread, and materializes the shipped `.obsidian/`/`_meta/` assets if absent. `relocate ` moves `RoboCo/` into a destination, refusing if the destination already has a `RoboCo/` subtree. + +## Config Flags + +- `ROBOCO_OBSIDIAN_VAULT_ENABLED` — master switch; off = `VaultWriter` is never invoked from any seam, `curate_vault` returns `invalid_state`, the janitor/KB loops return immediately, and `python -m roboco.vault` refuses. Config default `false`; both compose files set it `true`. +- `ROBOCO_VAULT_PATH` (default `/data/vault`) — root directory the vault materializes into; bind-mounted in both compose files. +- `ROBOCO_VAULT_INTAKE_ENABLED` — independent switch for `_vault_intake_loop`; inert unless the master switch is ALSO on. Config default `false`; both compose files set it `true`. +- `ROBOCO_VAULT_INTAKE_INTERVAL_SECONDS` / `ROBOCO_VAULT_INTAKE_DIR` / `ROBOCO_VAULT_INTAKE_MAX_PER_CYCLE` / `ROBOCO_VAULT_INTAKE_MAX_OPEN_DRAFTS` — cadence, inbox subfolder, per-cycle origination cap, rolling open-draft cap. +- `ROBOCO_VAULT_ARCHIVE_DAYS` (default `30`, `0` disables) — age past which a terminal task's note archives during the janitor sweep. Checked only under the master switch — no separate enable flag. +- `ROBOCO_VAULT_REPORT_ENABLED` (default `true`) — the janitor's weekly org-report + CEO notification. Config default `true` in both compose files (deterministic, no LLM, cheap to leave on). +- `ROBOCO_VAULT_KB_ENABLED` (default `false`) — master switch for KB ingest; off = `_vault_kb_loop` returns immediately and `IndexType.VAULT_NOTES` stays empty. NAS compose (`docker-compose.yml`) sets it `true`; the public registry compose (`docker-compose.registry.yml`) leaves it `false` (optional engines ship off). +- `ROBOCO_VAULT_KB_DIRS` (default `RoboCo/Notes`, CSV) — vault-relative folders the KB engine scans. Rejected at config load if absolute, `..`-carrying, or overlapping `vault_intake_dir`/`Tasks`/`Journals`/`A2A`/`Agents`/`Archive`/`Reports`/`_meta`/`.obsidian`. +- `ROBOCO_VAULT_KB_INTERVAL_SECONDS` (default `900`, min `60`) — KB-engine scan cadence. + +## Health + +The projection side is zero-risk by construction: every seam is best-effort and DB-free from the writer's perspective, so a filesystem or permission failure degrades to a stale/missing note, never a blocked verb. Materialize-on-create closes the V1 gap where the Dataview board only ever showed curated/rebuilt tasks — a fresh task is visible immediately. The janitor is the freshness backstop for everything best-effort seams can miss: it's restart-proof (dueness lives in a state file, not loop cadence — an orchestrator that restarts more often than daily still sweeps exactly once per elapsed day), self-healing against a corrupt/hand-edited state file (any unparseable value degrades to "no state," never a wedged loop), and every per-item drain (changed-task, sample-verify, archive) isolates failures — one bad row is logged and skipped, never aborts the pass, and re-qualifies on its next change or the next sample draw. Per-cycle caps (200 reprojects, 200 archives) mean a first-enable or long-downtime backlog drains in bounded hourly slices via the resume-marker convention (a capped tick advances the marker only to the last item it actually processed) rather than one unbounded burst. + +The KB-ingest side is the one path with real security stakes — once a vault note is agent-retrievable, unscreened note text is injection into the fleet's retrieval context, not just a drafting risk. It layers defense-in-depth: the config-load validator rejects a dangerous `vault_kb_dirs` entry outright (can't even start with an escaping/overlapping dir); the engine independently re-checks every allowlisted dir resolves under the vault root before scanning it; every individual note is re-checked for symlink-ness and resolved-path escape before it's read (belt-and-suspenders against a dir-level check being bypassed by a per-file symlink); and the injection guard runs as a hard GATE (not the intake watcher's screen-and-still-process posture) — a flagged note is never embedded, only quarantined with a visible callout so the CEO knows why. Content-hash dedup (shared with the intake watcher's ledger convention) makes both re-scans and the quarantine callout's own append idempotent — appending the callout never itself re-triggers reprocessing. + +Rebuild/relocate remain idempotent and additive-safe (`ensure_vault_assets` never overwrites an existing file), so re-running against a CEO-customized vault cannot clobber `.obsidian/`/`_meta/` edits. + +## Related + +- `docs/rag/architecture/obsidian-vault.md` — the agent-facing doc (what the Auditor and vault-intake-originated tasks actually see, plus what changed for KB retrieval) +- `docs/rag/roles/auditor.md` — the `curate_vault` verb +- `docs/map/orchestrator.md` — `_dispatch_vault_curation_work` / `_maybe_spawn_vault_curation` / `_vault_intake_loop` / `_vault_janitor_loop` / `_vault_kb_loop` +- `docs/map/task-service.md` — `_materialize_vault_note` / `list_updated_since` / `list_archive_candidates` / `sample_stale_tasks` +- `docs/map/product-strategy-research-pitch.md` — `XEngine`, the sibling engine sharing `injection_guard.screen_external_text` +- `docs/internal/specs/2026-07-09-obsidian-vault.md` — the original V1 design spec (vault layout, link-stability rationale) +- `docs/internal/specs/2026-07-11-obsidian-vault-v2.md` — the V2 spec (materialize-on-create, janitor, archival, KB ingest, weekly report, Bases, sync doc) + +## Purpose + +The RoboCo video engine: a default-off subsystem that authors bespoke short marketing videos (release announcements, feature spotlights, on-demand CEO briefs) and distributes them to X and TikTok — nothing renders or posts without the flags on, and nothing posts without an explicit per-clip CEO approval. It mirrors the `XEngine` / `ReleaseManagerEngine` held-artifact shape, but splits across the real delivery lifecycle: a normal ASSIGNED UX/UI authoring task ships the composition through the standard commit/PR/QA/doc/review gate, then an orchestrator render loop renders the merged `motion/` source via the credential-free `video-renderer` sidecar and materializes a held `video_post` draft for the CEO. Two task kinds (authoring + held post), one render pass between them. + +## Files + +| Path | Role | approx LOC | +|---|---|---| +| `roboco/services/video_engine.py` | `VideoEngine` — opens the ASSIGNED authoring task (`open_video_task`, balanced across `ux-dev-1`/`ux-dev-2`) and originates the held `video_post` draft once the render succeeds (`_originate_video_post`); release/spotlight/on-demand trigger wiring. | 324 | +| `roboco/services/video_post_service.py` | `VideoPostService` — CEO approve/reject over the held post; the ONLY caller of the X-v2 and TikTok posters; runs the critical section under a heartbeat-renewed Redis mutex, commits each platform's posted-id durably before the next, idempotent on already-`COMPLETED` AND on already-CANCELLED (rejected). | 629 | +| `roboco/services/video_renderer_client.py` | `VideoRenderer` — tars the merged `motion/` dir, POSTs the tarball to the sidecar (`ROBOCO_VIDEO_RENDERER_BASE_URL`), saves the returned MP4s to `video_output_dir` and PUTs each to MinIO (`_save`). `NullVideoRenderer` raises on unconfigured so the render loop fails loud rather than silently no-op'ing a real trigger; `get_video_renderer()` factory. | 188 | +| `roboco/services/minio_client.py` | Singleton `Minio` (minio-py) with an unconfigured guard (`get_client()` returns `None` when `minio_endpoint` empty); `put_object` / `get_object_stream` / `stat_object`, sync, call sites wrapped in `asyncio.to_thread`. | 129 | +| `roboco/services/tiktok_client.py` | `TikTokPoster` — TikTok inbox-upload poster (v2 media, OAuth2 refresh). Fernet-encrypted singleton `tiktok_credentials` row (migration 062); agents never hold creds or egress. | 326 | +| `roboco/services/x_video_client.py` | `XVideoPoster` — X v2 media upload + tweet poster. `NullXVideoPoster` makes the unconfigured leg a graceful no-op. | 266 | +| `roboco/runtime/heartbeat_mutex.py` | `HeartbeatMutex` — Redis mutex with heartbeat-renewed TTL, shared with `ReleaseProposalService`'s release-execute lock shape; backs `VideoPostService.approve`'s long video-upload critical section. | — | +| `roboco/mcp/do_server.py` `propose_video` | Do-tool the UX/UI dev calls exactly once per authoring task to stamp the `video_draft` marker (composition id + per-platform captions + input props); metadata-only, does not render. | — | +| `roboco/services/gateway/content_actions.py` `propose_video` | Server-side action: team-gated (`_caller_team` rejects be-dev/fe-dev), resolves the caller's open video task, `markers.set_video_draft` with the metadata. | — | +| `roboco/services/gateway/content_actions.py` `request_render` | Do-verb (developer/QA): renders the caller's ACTUAL composition to keyframe PNGs via the sidecar's frames mode and stamps the `render_preview` marker — dev renders their own tree (worktree-aware, `head_sha`/`dirty` stamped), QA a read-only branch export (`WorkspaceService.export_branch_motion`). Frames land at the container-shared `{workspaces_root}/{project}/.previews/{task8}/{orientation}/`. | — | +| `roboco/foundation/policy/tracing.py` `RENDER_VERIFIED` | `i_am_done` requirement on `source=video` tasks: no stamped `render_preview` → tracing gap naming `render_preview` (hint: call `request_render`, Read every frame). Mirrored in the possibilities-matrix fast path. | — | +| `alembic/versions/062_tiktok_credentials.py` | Migration 062 — the `tiktok_credentials` singleton row (Fernet-encrypted OAuth2 secrets, all-or-nothing set/clear, mirroring the git-token / `x_credentials` pattern). | 44 | +| `video-renderer/` | The sidecar: `server.js` (HTTP+tarball boundary), `render.js` (`@hyperframes/producer` `createRenderJob` + `executeRenderJob`, system `ffmpeg`, headless Chromium). Credential-free and git-free — reads only what's POSTed. pnpm-managed (`pnpm-lock.yaml`, no npm `package-lock.json`); `@hyperframes/producer` pinned exact at `0.7.36` (`0.7.60` fails every render). | — | +| `docker/video-renderer.Dockerfile` | Sidecar image (`roboco-video-renderer`): Node + Chromium + system `ffmpeg`; installs `@hyperframes/producer`. No RoboCo source, no creds. | — | +| `motion/README.md` `## Design bar` / `## Visual design bar (demo/kit register)` | Authoring craft an assigned UX/UI dev consults before building a composition: color/type/motion/layout dials for the text-card register, plus spacing, beat pacing (`animation-delay`, never `data-start` for beats), `pk-chip`/`pk-pill` semantic-variant discipline, camera+cursor+rhythm, and anti-generic tells for the `kit/` demo register. | 149 (file total) | +| `motion/skills/references/{house-style,video-composition,beat-direction,motion-principles}.md` | Four upstream HyperFrames craft references (palette/lazy-defaults, video-medium scale/density, per-beat rhythm planning, ease/speed/direction variance) vendored verbatim at pinned commit `9d148d28` (Apache-2.0, header in each file) — back every rule in `motion/README.md`'s design-bar sections; re-vendor when bumping `@hyperframes/producer`. | 462 | +| `motion/skills/hyperframes-catalog-index.md` | RoboCo-authored index of the public HyperFrames catalog (109 blocks + 24 components, 133 entries) with per-category kit-mapping triage (maps onto an existing `pk-*` piece / a choreography engine / needs a new kit piece); read on demand when planning a beat, not injected into any agent prompt. | 186 | +| `motion/skills/{hyperframes-core,hyperframes-creative,hyperframes-keyframes}.md` | The vendor's own official HyperFrames agent skills (composition contract, beat planning, seek-safe keyframes across runtimes), vendored verbatim at a pinned upstream commit (Apache-2.0, header + re-vendor note in each file); `motion/README.md` points authoring devs at them before a new register. Primary seek-safe primitive is GSAP tweens on `window.__timelines` — this kit's CSS-animation register is a house pattern, and the clip-window rule is its empirically-derived companion. | 382 | +| `motion/kit/kit.js` `choreographCursor` / `choreographCamera` | Choreography engines: `choreographCursor` reads `data-waypoints="t x y [click]; ..."` off a `.pk-cursor` and generates a multi-leg eased path with fade in/out, an idle-hand sway between legs, and click rings + glyph press dips at flagged waypoints (click lands ~0.2s before the thing it triggers) — replaces the old single-glide `--pk-cursor-x0/y0/x1/y1` cameo. `choreographCamera` reads `data-shots="t x y scale; ..."` off a `.pk-camera` wrapper and eases push-ins/pull-backs, settling at identity by the end. `motion/kit/README.md` documents both. | 156 (diff) | +| `roboco/runtime/orchestrator.py` `_is_video_authoring_spawn` | Fail-closed spawn-time probe (role==developer, team==ux_ui, task.source==video) that registers the `playwright` MCP for the composition author — the one non-QA case, so the dev can preview the authored HTML in a real browser between renders. Gating-only: `agent-ux`'s image already bakes the browser + wrapper entrypoint. | — | + +## Data Flow + +DETECT → AUTHOR: a release publish (`ROBOCO_VIDEO_ON_RELEASE`), a CEO-approved feature-spotlight draft that requests one (`ROBOCO_VIDEO_ON_SPOTLIGHT`), or a CEO on-demand `POST /api/video/request` calls `VideoEngine.open_video_task`, which creates a normal ASSIGNED UX/UI authoring task (`source=video`, `confirmed_by_human=True`, balanced across the two ux-devs) — NOT held, NOT in any dispatcher's skip bucket. The assigned dev authors `motion/compositions//{vertical,square}.html` (HyperFrames render params on ``), reads the vendored `motion/skills/hyperframes-{core,creative,keyframes}.md` skills and consults `kit.js`'s `choreographCursor`/`choreographCamera` engines for camera+cursor craft (a locked-off camera or a popping/freezing cursor is automatic revision; clip windows are for structural layers ONLY — beats ride base-hidden delayed CSS animations, since the renderer's clip scheduler drops any beat driven off its own clip window), previews the live HTML via the `playwright` MCP registered for this spawn (`_is_video_authoring_spawn`), calls the `propose_video` do-tool exactly once (server-side `content_actions.propose_video` is team-gated and stamps `video_draft`), then verifies the ARTIFACT: `request_render` renders the dev's actual working tree to keyframe PNGs the dev must Read (every scene fully visible and legible — the gate that catches an authored duration shorter than its scene list), iterating fix → re-render until the frames prove the brief; `i_am_done` refuses without the stamped `render_preview` marker (`Requirement.RENDER_VERIFIED`). Then `commit` + `open_pr` through the normal PR-review gate. The authoring task rides the standard QA/doc/review lifecycle to `completed`, with QA's `claim_review` evidence carrying a `video_context` block (the dev's preview + an instruction to `request_render` the branch state fresh). + +RENDER: once the authoring task is `completed`, the orchestrator's `_video_render_loop` (bounded retry, `_MAX_VIDEO_RENDER_ATTEMPTS`) resolves the project's read-clone at the merged HEAD, tars the `motion/` dir, and POSTs it to the credential-free `video-renderer` sidecar (`ROBOCO_VIDEO_RENDERER_BASE_URL`). The sidecar untars, runs `@hyperframes/producer`'s `createRenderJob` + `executeRenderJob` per orientation (headless Chrome + system `ffmpeg`, `ROBOCO_VIDEO_RENDER_TIMEOUT_SECONDS` per render), and streams both 9:16 and 1:1 MP4s back. `VideoRenderer` saves them to `ROBOCO_VIDEO_OUTPUT_DIR` (`_save` also PUTs each to MinIO when `minio_endpoint` is set, non-fatal). On success `VideoEngine._originate_video_post` materializes a held `video_post` draft (`source=video_post`, `confirmed_by_human=False`, Secretary-owned, skipped by every dispatcher) carrying `mp4_paths` (`{vertical, square}` absolute paths) + the per-platform captions. + +CEO ACT: `GET /api/video/posts` lists held drafts (including `mp4_paths`); `GET /api/video/posts/{id}/media?cut=vertical|square` streams the MP4 bytes for the preview player (CEO-gated, falls back to `FileResponse` on `S3Error`/unconfigured MinIO). The CEO edits captions and approves/rejects in the panel's `video-post-queue.tsx`. `POST /api/video/posts/{id}/approve` is the ONLY caller of `XVideoPoster` / `TikTokPoster`: it acquires `HeartbeatMutex`, re-reads the committed task state inside the lock, commits `COMPLETED` before releasing (so a concurrent approve can't double-post), commits each platform's posted-id durably before attempting the next (a partial failure never re-posts an already-succeeded platform on retry), and is idempotent (an already-`COMPLETED` draft returns the stored ids without calling a poster). A CANCELLED draft (already rejected) is refused both pre-lock and re-checked under lock, returning `already_rejected` — closes a hole where a stale approve (e.g. a queued Telegram button targeting the draft by id regardless of its current status) could post a draft the CEO had already rejected. `POST /api/video/posts/{id}/reject` cancels the draft with a reason — and, for a non-empty reason, `VideoEngine.reauthor_from_rejection` opens a fresh authoring task (same occasion, brief = the CEO's verbatim feedback + revise-in-place pointer at the existing composition) so the rejection feedback re-enters the delivery flow instead of dying on the cancelled draft; best-effort, never fails the reject. + +## Config Flags + +- `ROBOCO_VIDEO_ENGINE_ENABLED` — master switch; off = no video-authoring task is ever opened and no render/post happens. Panel-toggleable. +- `ROBOCO_VIDEO_ON_RELEASE` / `ROBOCO_VIDEO_ON_SPOTLIGHT` — sub-switches for the two automatic triggers, independent of the master switch and of the CEO's on-demand `POST /video/request`. +- `ROBOCO_VIDEO_RENDER_INTERVAL_SECONDS` / `ROBOCO_VIDEO_RENDER_TIMEOUT_SECONDS` / `ROBOCO_VIDEO_REQUEST_TIMEOUT_SECONDS` / `ROBOCO_VIDEO_OUTPUT_DIR` — render loop cadence, per-render deadline, sidecar HTTP deadline, MP4 output dir (bind-mounted in all three compose files so renders survive container recreation). +- `ROBOCO_VIDEO_RENDERER_BASE_URL` — the sidecar endpoint (default `http://roboco-video-renderer:3001`). +- `ROBOCO_MINIO_*` — MinIO object storage (default-off; `video_renderer_client._save` PUTs each render after the local write; serve route streams via `StreamingResponse` with `FileResponse` fallback). + +## Changes Since Baseline + +- **2026-07-17** (PR #543, `3e801697`): Two renderer root causes fixed — `@hyperframes/producer` was floating (`^0.7.36`, no lockfile), so image builds silently picked up `0.7.60`, which fails EVERY render ("Cannot access 'rt' before initialization"); pinned exact (`0.7.36`, no caret) and committed a lockfile (regenerated as `pnpm-lock.yaml` by the immediate follow-up `a12fefcb`, not the npm `package-lock.json` this PR first wrote — this package is pnpm-managed). Second: the producer's per-clip visibility scheduler runs on a clock that lags ~50% behind the encoded timeline on a long cut, so tail scenes (past roughly the halfway mark) were silently missing from the MP4 regardless of authoring — fixed by treating `class="clip"` + `data-start`/`data-duration` as a structural-layer-only primitive and driving every beat with base-hidden styles + a delayed CSS animation instead (documented in `motion/README.md`'s "Clip windows are for structural layers only" rule). Also added the two choreography engines to `motion/kit/kit.js` (`choreographCursor` / `choreographCamera`, see Files above) plus a "Cinematography & rhythm" section in `motion/README.md` and a craft-bar block in the dev video spawn prompt (`roboco/runtime/orchestrator.py`) so a locked-off camera or a popping/freezing cursor reads as an automatic revision. +- **2026-07-17** (PR #544, `fd621f0d`): The three craft capabilities wired one hop closer to the hands doing video work — vendored the vendor's own official HyperFrames agent skills (`hyperframes-core`/`-creative`/`-keyframes.md`, see Files above; supersedes the external-pointer-only version briefly added by the intervening `1416bd1d`); registered the `playwright` MCP for a ux-dev spawned onto a `source=video` task (`_is_video_authoring_spawn`, fail-closed role/team/task-source probe — gating-only, `agent-ux`'s image already bakes the browser); and added a video-mode override to the `ux_ui` team prompt's design bar ("video-authoring tasks are FILMS, not UI — these dials do not apply") so a video task no longer reads its own "dense product UI → motion 2-3" dial as license to ship a static slideshow. +- **2026-07-17** (Wave 6, PR #550): Authoring craft, not engine code — `motion/README.md` gained `## Visual design bar (demo/kit register)` (spacing/hierarchy, beat pacing, `pk-chip`/`pk-pill` semantic discipline, camera+cursor+rhythm, anti-generic tells for the `kit/` register), four upstream HyperFrames craft references vendored verbatim under `motion/skills/references/` (fixing `hyperframes-creative.md`'s previously-dead `references/` pointers), and a new `motion/skills/hyperframes-catalog-index.md` (133-entry HyperFrames catalog vocabulary index, read-on-demand). No service/verb/schema change; the render/post pipeline documented above is untouched. +- **2026-07-18** (PR #570, "project-branded drafts"): the release-video script/prompt/brief builders in `roboco/services/video_engine.py` (`_fallback_release_script`, `_release_video_prompt`, `_release_video_brief`, `_draft_release_script`) all gained a required `product_name` param — `draft_release_post_video` (the `ReleaseProposalService.approve` publish-success hook, mirroring `XEngine.draft_release_post`) now resolves it via `CompanyGoalsService.resolve_product_name(project)` (the release's own project name → charter `company_goals.company_name` → "RoboCo" literal) instead of hardcoding "RoboCo" into the script/brief text — see `docs/map/product-strategy-research-pitch.md` for the shared resolver. `GET /api/video/posts` responses also gained `project_slug`/`project_name` (`api/schemas/video.py`, via the same `task_project_fields` helper `x.py` uses) so the panel's `video-post-queue.tsx` can render a `ProjectBadge` alongside the source-kind badge. + +## Health + +Default-off, CEO-gated at two independent points (the flags, then per-clip approval). The held-draft shape mirrors the XEngine / ReleaseManagerEngine pattern, so the dispatchers never see it. The render pass is bounded retry with `_MAX_VIDEO_RENDER_ATTEMPTS` and a per-render deadline; `NullVideoRenderer` raises on unconfigured so a misflagged trigger fails loud rather than silently no-op'ing. The approve critical section is heartbeat-mutex protected so a double-click can't double-post and a partial platform failure is recoverable, and (Wave 5, PR #551) `approve` now also refuses a CANCELLED draft outright rather than posting it. TikTok's OAuth2 secrets live Fernet-encrypted in a singleton row (migration 062); agents never hold creds or egress — `VideoPostService.approve` is the only caller of the posters. + +## Related + +- `docs/rag/architecture/video-engine.md` — the user-facing architecture doc +- `docs/rag/architecture/minio-storage.md` — the decoupled-durable render storage +- `docs/map/release-manager.md` — the sibling held-artifact engine whose lock shape `VideoPostService.approve` mirrors +- `docs/map/engines-heal-ciwatch-depupdate.md` — the other default-off originate-and-stop engines + +# models slice + +## Purpose + +The Pydantic/dataclass domain surface of RoboCo — the typed contract the API, services, orchestrator, and agent runtimes all speak. These are **not** the ORM tables (`roboco/db/tables.py` owns persistence); models here are request/response schemas, domain aggregates, runtime DTOs, and enums that get crossed into SQLAlchemy rows by the service layer. Validation (`RobocoBase`: `extra="forbid"`, `validate_assignment`, `use_enum_values`) lives here, and several files (`agents.py`, `runtime.py`, `optimal.py`, `metrics.py`, `llm.py`, `audit.py`, `dashboard.py`, `transcription.py`, `extraction.py`, `permissions.py`) are pure dataclasses/StrEnums with no Pydantic model at all — runtime value types the orchestrator and services pass around. + +## Files + +| Path | Role | approx LOC | +|------|------|------------| +| `__init__.py` | Public re-export surface (`Agent`, `Task`, `Notification`, `Journal`, `CommitRef`, enums, `get_column_config`, …) | 149 | +| `base.py` | All shared StrEnums (`TaskStatus`, `TaskType`, `AgentStatus`, `ModelProvider`, `JournalEntryType`, …) + `RobocoBase`/`TimestampMixin` + `AgentRole`/`Team` aliases to `foundation.identity` | 275 | +| `task.py` | `Task` aggregate + `CommitRef`/`DocRef`/`ProgressUpdate`/`Checkpoint`/`SubTask`/`TaskPlan` + `TaskCreate`/`TaskUpdate`/`TaskCreateRequest` | 502 | +| `a2a.py` | A2A protocol wire models (`AgentCard`, `A2ATask`, `A2AMessage`, parts) + persistent conversation models (`A2AConversation`, `A2AChatMessage`) + state mappers | 592 | +| `agents.py` | Agent **runtime** domain types — per-role phase enums (`DevTaskPhase`, `QATaskPhase`, `CellPMPhase`, …), `AgentConfig`/`AgentState`, `TaskContext`/`ReviewContext`/`DocContext`, `AuditFlag`/`AuditReport` | 405 | +| `journal.py` | `Journal`/`JournalEntry` + 5 factory param dataclasses + `create_*_entry` factories + `JournalStats`/`GrowthMetrics` | 374 | +| `permissions.py` | `PermissionLevel` IntEnum, `ROLE_LEVELS` (built from `agents_config`), `AgentContext`, `COMMUNICATION_MATRIX`, `TASK_PERMISSIONS`, `KB_PERMISSIONS` | 284 | +| `optimal.py` | RAG domain types — `IndexType`, `SearchResult`/`SearchOutcome`/`RAGResponse`, `ErrorPattern`/`Decision`/`Standard`, `MentorResponse`, `CodeReviewResult` | 275 | +| `metrics.py` | `VelocityMetrics`/`BlockerMetrics`/`TeamMetrics`/`AgentMetrics` + v0.10.0 observability: `StageTiming`/`StageBottleneck`/`BottleneckReport`/`AgentReworkRate`/`ReworkReport`/`Scorecard` | 249 | +| `events.py` | `EventType` StrEnum + `Event` dataclass (JSON round-trip) + `NotificationServiceProtocol`/`OrchestratorAccessProtocol`/`EventContext` DI container | 199 | +| `handoff.py` | `DocumenterHandoff` + `CodeSample`/`DocumentationItem`/`ConversationRef` + `HandoffCreate` (RESERVED — see file header) | 202 | +| `project.py` | `Project` + `BranchReason` + `ProjectCreate`/`ProjectUpdate` (CI-watch, dep-update, quality_command fields) | 178 | +| `agent.py` | `Agent` API model + `ModelConfig`/`AgentPermissions`/`AgentMetrics` + `AgentCreate`/`AgentUpdate` | 172 | +| `kanban.py` | `KanbanBoard`/`KanbanColumn`/`KanbanCard`/`KanbanSwimlane` + per-role column configs + `get_column_config` | 159 | +| `llm_catalog.py` | `CatalogEntry` + `MODEL_CATALOG`/`MODEL_CATALOG_BY_NAME`/`provider_type_for_model` + `OLLAMA_DEFAULT_MODEL` (Settings dropdown source of truth) | 103 | +| `message.py` | `ExtractedMessage` + `RawStream` | 137 | +| `runtime.py` | Orchestrator runtime types — `OrchestratorAgentState`, `SpawnGitContext`, `OrchestratorAgentConfig`, `AgentInstance`, `WaitingRecord`, `MODEL_MAP`, `ROLE_MODEL_MAP` | 128 | +| `transcription.py` | `StreamBuffer` (flush heuristic) + `TranscriptionConfig` | 119 | +| `llm.py` | `LLMUsage`/`ToonConfig`/`EncodedBlock`/`ToonMetrics` (token + TOON serialization metrics) | 118 | +| `notification.py` | `Notification` + `NotificationCreate` + `CreateNotificationParams` (gains a per-row `requires_ack: bool \| None` override, wave 3) | 121 | +| `work_session.py` | `WorkSession` + `WorkSessionStatus` + `WorkSessionCreate`/`WorkSessionUpdate` | 117 | +| `pitch.py` | `Pitch` + `PitchStatus` + `PitchCreate` (Board proposal → provisioning) | 73 | +| `extraction.py` | `ExtractionContext`/`ExtractionResult`/`ExtractionConfig` | 68 | +| `product.py` | `Product` + `ProductCellMapping` (cell→Project map; validator enforces cell-only) + create/update DTOs | 69 | +| `playbook.py` | `Playbook` + `PlaybookCreate`/`PlaybookUpdate` (curated procedure; `from_attributes` for ORM load) | 58 | +| `audit.py` | `AuditEventType` StrEnum + `PermissionDenialContext`/`StateTransitionDenialContext` dataclasses | 58 | +| `dashboard.py` | `FlagData`/`ReportData`/`TeamHealthData`/`AuditQueueItem`/`CreateFlagParams`/`DashboardStorage` | 96 | +| `secretary.py` | `DirectiveKind`/`DirectiveStatus` StrEnums + `GATED_KINDS` frozenset | 41 | +| `README.md` | Architecture doc for the models package | ~250 | ## Key Symbols | Name | Kind | File:Line | Responsibility | -|---|---|---|---| -| _get_prompts_base_path | function | roboco/agents/factories/_base.py:17 | Resolve project_root/agents/prompts/ from this file's location with a cwd-relative fallback | -| PROMPTS_BASE_PATH | constant | roboco/agents/factories/_base.py:37 | Module-level cached prompts base path used by default in compose_prompt | -| _load_layer | function | roboco/agents/factories/_base.py:40 | Read a prompt layer file, return '' if missing (graceful fallback) | -| _ROLE_LAYER_MAP | dict | roboco/agents/factories/_base.py:55 | Maps role string -> roles/*.md filename; board roles all share board.md; prompter/secretary/pr_reviewer have own files | -| _TEAM_LAYER_MAP | dict | roboco/agents/factories/_base.py:77 | Maps team string (backend/frontend/ux_ui) -> teams/*.md filename | -| _role_layer | function | roboco/agents/factories/_base.py:84 | Load the role-specific prompt layer or None if role unknown | -| _team_layer | function | roboco/agents/factories/_base.py:93 | Load the team prompt layer or None if unset/unknown | -| _autogen_verbs_layer | function | roboco/agents/factories/_base.py:104 | Load _generated/.md autogenerated verb-signature table for the role | -| _BUILTIN_TOOLS_COMMON | tuple | roboco/agents/factories/_base.py:127 | Built-in Claude Code tools every role gets: Read,Bash,Grep,Glob,TodoWrite | -| _BUILTIN_TOOLS_AUTHORS | tuple | roboco/agents/factories/_base.py:134 | Authors set (developer/documenter) adds Edit,Write to the common set | -| _ROLE_BUILTIN_TOOLS | dict | roboco/agents/factories/_base.py:136 | Per-role builtin-tool grant map; non-authors get common-only | -| _tool_load_directive_layer | function | roboco/agents/factories/_base.py:149 | Build top-of-prompt 'your tools are ready' block; steers away from ToolSearch and shell-redirect rewrites | -| _lifecycle_layer | function | roboco/agents/factories/_base.py:187 | Load _generated/lifecycle-.md canonical verb-surface fragment (from lifecycle spec, CI-gated) | -| fable_doctrine_layer | function | roboco/agents/factories/_base.py:203 | Return the vendored `doctrine/fable.md` doctrine text, or None when `fable_mode_enabled` is off / the file is missing; only caller is compose_prompt, inserted right after base.md | -| ponytail_doctrine_layer | function | roboco/agents/factories/_base.py | Return the vendored Ponytail build-laziness doctrine, role-scoped and gated on the same `fable_mode_enabled` flag (no separate flag — ponytail is Fable's complementary build-doctrine). Developers → `doctrine/ponytail.md` (full ladder) with a trailing `**Operative intensity: {settings.ponytail_intensity}.**` directive; every other role → `doctrine/ponytail-ethos.md` (ethos-only, no dial). None when the flag is off / the file is missing; only caller is compose_prompt, inserted immediately after the Fable doctrine layer | -| compose_prompt | function | roboco/agents/factories/_base.py:203 | Compose the full system prompt by concatenating tool-directive, lifecycle, base, role, autogen-verbs, team, identity, ambient layers with '---' separators, skipping empty layers | -| _AMBIENT_TOTAL_CAP | constant | roboco/agents/factories/_base.py:256 | 3000-char cap on the concatenated conventions ambient block | -| conventions_ambient_layer | async function | roboco/agents/factories/_base.py:259 | Render per-project architectural-standard ambient block(s), multi-project headed, capped; None when conventions off / no projects | -| make_slug | function | roboco/agents/factories/_base.py:296 | Lowercase + dash slug helper | -| _AUTH_SECRET_ENV | constant | roboco/agents_config.py:42 | Env var name ROBOCO_AGENT_AUTH_SECRET for the HMAC signing key | -| _auth_secret | function | roboco/agents_config.py:45 | Return HMAC secret bytes or None when unset | -| _signing_payload | function | roboco/agents_config.py:51 | Canonical lowercase stripped agent_id:role:team HMAC message | -| issue_agent_token | function | roboco/agents_config.py:61 | Mint hex HMAC-SHA256 token binding agent identity to role+team; returns UNSIGNED sentinel if secret unset | -| verify_agent_token | function | roboco/agents_config.py:79 | Constant-time HMAC verification; fail-closed on unset secret / UNSIGNED | -| issue_panel_token | function | roboco/agents_config.py:94 | Mint the CEO-identity token the panel presents (signed for CEO_AGENT_ID/ceo/empty team) | -| _UUID_TO_SLUG | dict | roboco/agents_config.py:109 | Reverse map UUID->slug from AGENT_UUIDS seeds | -| _resolve_to_slug | function | roboco/agents_config.py:114 | Resolve UUID or slug input to slug | -| AGENT_ROLE_MAP | dict | roboco/agents_config.py:127 | slug->role.value for every non-SYSTEM agent (derived from foundation.AGENTS) | -| AGENT_TEAM_MAP | dict | roboco/agents_config.py:133 | slug->team.value derived from foundation | -| CELL_MEMBERS | dict | roboco/agents_config.py:139 | team.value -> sorted slug list per cell | -| ALL_AGENTS | list | roboco/agents_config.py:146 | All agent slugs | -| BOARD_MEMBERS | list | roboco/agents_config.py:149 | product-owner, head-marketing, auditor | -| ALL_DOCS | list | roboco/agents_config.py:152 | Cross-cell documenter slugs for docs workspace perms | -| TASK_CREATOR_ROLES | frozenset | roboco/agents_config.py:159 | Roles that can call task.create (cell_pm, main_pm, product_owner, head_marketing, ceo) | -| ESCALATION_CHAIN | dict | roboco/agents_config.py:170 | slug -> escalation target slug (dev/qa/doc -> cell PM -> main-pm -> product-owner -> ceo) | -| get_agent_role | function | roboco/agents_config.py:204 | Role string for an agent (UUID or slug); 'unknown' if missing | -| get_agent_team | function | roboco/agents_config.py:210 | Team string for an agent or None | -| get_agent_cell | function | roboco/agents_config.py:216 | Alias of get_agent_team | -| get_cell_members | function | roboco/agents_config.py:221 | Slugs for a cell | -| is_pm | function | roboco/agents_config.py:226 | cell_pm or main_pm predicate | -| is_board_member | function | roboco/agents_config.py:232 | Board membership predicate by slug | -| is_management | function | roboco/agents_config.py:237 | PM/Board/CEO predicate | -| is_ceo | function | roboco/agents_config.py:250 | CEO predicate (full permission bypass) | -| can_send_notifications | function | roboco/agents_config.py:255 | Role in foundation NOTIFY_SENDER_ROLES | -| can_create_tasks | function | roboco/agents_config.py:263 | Role in TASK_CREATOR_ROLES | -| can_assign_tasks | function | roboco/agents_config.py:269 | Same set as can_create_tasks | -| _CANCEL_ROLES | set | roboco/agents_config.py:276 | Roles that may cancel (cell_pm/main_pm/product_owner/head_marketing — NOT ceo/auditor) | -| can_cancel_tasks | function | roboco/agents_config.py:284 | Role in _CANCEL_ROLES | -| get_escalation_target | function | roboco/agents_config.py:290 | Next escalation slug from ESCALATION_CHAIN | -| get_pm_for_team | function | roboco/agents_config.py:295 | Cell PM slug for a team | -| get_pm_for_agent | function | roboco/agents_config.py:305 | Responsible PM: cell PM for members, main-pm for cell PMs, product-owner for main PM | -| _slugs_for_role_set | function | roboco/agents_config.py:355 | Expand a role-set to sorted slugs honoring optional team_scope; excludes system sentinel | -| ROLE_PERMISSION_LEVELS | dict | roboco/agents_config.py:404 | Single source of truth role->permission-level (CEO/BOARD/AUDITOR/MAIN_PM/CELL_PM/CELL_MEMBER) used by PermissionService | -| VALID_NOTIFICATION_TYPES | frozenset | roboco/agents_config.py:425 | Valid NotificationType values | -| VALID_NOTIFICATION_PRIORITIES | frozenset | roboco/agents_config.py:428 | Valid NotificationPriority values | -| ROLE_SKILLS | dict | roboco/agents_config.py:438 | Role -> A2A skill descriptor list for Agent Cards | -| get_agent_skills | function | roboco/agents_config.py:566 | A2A skills for an agent by role | -| _BOARD_ROLES | frozenset | roboco/agents_config.py:584 | Foundation board roles (PO/HoM/Auditor; main_pm intentionally excluded) | -| _MAIN_PM_TARGETS | frozenset | roboco/agents_config.py:585 | Roles a main PM may A2A directly | -| _check_cell_pm_a2a | function | roboco/agents_config.py:590 | A2A permission for cell PM (own cell / other PMs / main-pm allowed; board escalated) | -| _check_cell_member_a2a | function | roboco/agents_config.py:604 | A2A permission for cell members (same-cell allowed; cross-cell via PMs) | -| _check_main_pm_a2a | function | roboco/agents_config.py:624 | A2A permission for main PM (_MAIN_PM_TARGETS allowed) | -| can_a2a_direct | function | roboco/agents_config.py:632 | (allowed, error) for direct A2A from one agent to another; routes CEO via notify, board/main_pm/cell-member via handlers | -| get_a2a_route_hint | function | roboco/agents_config.py:670 | Human-readable routing hint for an A2A message | -| _PATTERNS | list | roboco/agent_sdk/prompt_guard.py:28 | Five (regex, reason) injection patterns: ignore-previous, role-override, fake role prefix, control-token mimicry, fake executive-order | -| detect_injection | function | roboco/agent_sdk/prompt_guard.py:63 | Return deny reason if text matches an injection pattern (lowercased), else None | -| refusal_message | function | roboco/agent_sdk/prompt_guard.py:72 | Guidance string shown on denial (mirrors bash hook text) | -| main | function | roboco/agent_sdk/prompt_guard.py:82 | CLI entry: exit 1 if argv[1] is an injection (used by grok entrypoint) | +|------|------|-----------|----------------| +| `Task` | Pydantic model | task.py:132 | Atomic unit of work; carries status, branch, PR, batch surface, ACs, gateway lock, structured notes | +| `TaskStatus` | StrEnum | base.py:31 | 15-state lifecycle (backlog→pending→claimed→…→completed/cancelled) | +| `TaskType` | StrEnum | base.py:63 | code/documentation/research/planning/design/administrative | +| `TaskNature` | StrEnum | base.py:74 | technical/non_technical | +| `CommitRef` | Pydantic model | task.py:31 | Git commit reference (hash/message/timestamp/author) | +| `DocRef` | Pydantic model | task.py:42 | Document reference with version + author trail | +| `TaskPlan` | Pydantic model | task.py:109 | Approach + ordered `SubTask`s + risks/open_questions | +| `TaskCreate` | Pydantic schema | task.py:350 | Request schema; `_exactly_one_target` validator (project_id / product_id / cell_projects) | +| `TaskCreateRequest` | dataclass | task.py:456 | Service-layer create params mirroring `TASK_AT_CREATE` (no silent defaults) | +| `Agent` | Pydantic model | agent.py:79 | API agent model (role, team, model config, permissions, metrics, journal_id) | +| `AgentRole` | alias | base.py:23 | `= identity.Role` — canonical Role enum lives in `foundation/identity` | +| `Team` | alias | base.py:24 | `= identity.Team` — canonical Team enum lives in `foundation/identity` | +| `ModelProvider` | StrEnum | base.py:197 | anthropic/ollama_cloud/openai/local/grok | +| `ModelConfig` | Pydantic model | agent.py:27 | provider + name + fallback + temperature + max_tokens | +| `AgentPermissions` | Pydantic model | agent.py:45 | can_notify | +| `WorkSession` | Pydantic model | work_session.py:25 | Branch/PR/merge tracking for a (project, task, agent) work episode | +| `WorkSessionStatus` | StrEnum | work_session.py:17 | active/completed/abandoned | +| `Project` | Pydantic model | project.py:47 | Git repo config + CI/dep-update/`sandbox_services` opt-ins + `assigned_cell` | +| `BranchReason` | StrEnum | project.py:18 | feature/bug/chore/docs/hotfix (branch-name prefixes) | +| `ExtractedMessage` | Pydantic model | message.py:51 | Stored message with embedding, edit_history, mentions, task_id | +| `MessageType` | StrEnum | base.py:128 | reasoning/dialogue/decision/action/blocker/technical | +| `RawStream` | Pydantic model | message.py:32 | Ephemeral WebSocket chunk payload | +| `Notification` | Pydantic model | notification.py:26 | Formal signal requiring ACK; from/to_agents, acked_by, acked_at | +| `NotificationType` | StrEnum | base.py:139 | task_assignment/priority_change/blocker_escalation/review_request/…/a2a_request | +| `Journal` | Pydantic model | journal.py:75 | Agent's personal journal; entries_by_type, latest_summary | +| `JournalEntry` | Pydantic model | journal.py:25 | Reflection/learning/struggle/decision entry with embedding + `is_private` | +| `JournalEntryType` | StrEnum | base.py:172 | task_reflection/decision_log/learning/struggle/general | +| `Playbook` | Pydantic model | playbook.py:17 | Curated procedure (draft→approved/archived); `from_attributes` for ORM load | +| `PlaybookStatus` | StrEnum | base.py:112 | draft/approved/archived | +| `AuditEventType` | StrEnum | audit.py:13 | permission_denied/unauthorized_access/role_changed/pm_override/… | +| `A2ATask` | Pydantic model | a2a.py:265 | A2A protocol work unit (maps to internal TaskTable) | +| `A2AMessage` | Pydantic model | a2a.py:212 | A2A communication turn with `Part` union (text/file/data/artifact) | +| `AgentCard` | Pydantic model | a2a.py:103 | A2A agent discovery card (published at /.well-known/agent.json) | +| `A2AConversation` | Pydantic model | a2a.py:468 | Persistent agent-pair conversation (canonical agent_a.md), base.md, _role_layer (roles/.md via _ROLE_LAYER_MAP), _autogen_verbs_layer (_generated/.md), _team_layer (teams/.md via _TEAM_LAYER_MAP, None for board/main-pm), identities/.md, then the optional ambient string. Empty/None layers are dropped; the rest are joined with "\n\n---\n\n". The composed string is written to /app/prompts-generated/-prompt.md (container) or $TMPDIR/roboco-prompts/ (host) and the path returned to the spawn path that mounts it as the agent's system prompt. -Ambient resolution (async, best-effort): orchestrator._resolve_conventions_ambient gates on settings.conventions_enabled, opens a DB session, resolves in-scope projects (single project_slug for delivery roles, or per-cell projects from a task's product_id for PO/Intake), and calls conventions_ambient_layer -> ConventionsService.render_ambient_block per project (ensuring a read clone), multi-project-headed, capped to 3000 chars. Any exception is caught and returns None so a compose is never blocked by conventions. - -Identity binding at spawn: agents_config.issue_agent_token(agent_id, role, team) HMAC-signs the canonical lowercase agent_id:role:team with ROBOCO_AGENT_AUTH_SECRET and the orchestrator injects the token into the agent env; verify_agent_token (called server-side on X-Agent-Token headers) fail-closes on unset secret or UNSIGNED. The panel gets issue_panel_token() signed for the CEO identity. - -Injection guard (runtime, per turn): IntakeDriver.send_turn calls detect_injection(text) before sending to the model; on a match it emits an error chunk with refusal_message(reason) and returns without forwarding. The grok one-shot entrypoint runs `python -m roboco.agent_sdk.prompt_guard ` and refuses start (exit 1) on a match. The same five patterns run in docker/scripts/user-prompt-hook.sh for non-SDK Claude sessions. - -Callers: orchestrator.py:3238 (compose_prompt), orchestrator.py:3265/3299 (conventions_ambient_layer), intake_driver.py:379-382 (detect_injection/refusal_message). Callees from this slice: roboco.foundation.identity (AGENTS/Role/Team/slugs_for_team), roboco.foundation.policy.communications (NOTIFY_SENDER_ROLES), roboco.seeds.initial_data (AGENT_UUIDS/CEO_AGENT_ID), roboco.services.conventions (get_conventions_service), roboco.config.settings, roboco.models.base (NotificationType/Priority). +API request bodies → Pydantic `*Create`/`*Update` schemas (validation boundary, `extra="forbid"`) → route handlers → services. Services translate between these models and the SQLAlchemy ORM tables in `roboco/db/tables.py` (e.g. `TaskTable`, `WorkSessionTable`, `ProjectTable`, `NotificationTable`, `JournalTable`, `JournalEntryTable`, `AgentTable`, `PlaybookTable`, `ProductTable`, `PitchTable`): the ORM row is the persistence shape; the Pydantic model is the contract shape. Several ORM tables load back into a model via `model_config = ConfigDict(from_attributes=True)` (`Playbook` at playbook.py:20, `Product`/`ProductCellMapping` via `RobocoBase`). Runtime-only DTOs (`AgentInstance`, `WaitingRecord`, `SpawnGitContext`, `Event`, `ExtractionResult`, `StreamBuffer`, the metrics/observability dataclasses, `AuditFlag`/`AuditReport`) never touch the DB directly — they are orchestrator/service in-process values. Enum parity between model and ORM is enforced by the test gate (`enum` columns in `tables.py` reference the same `StrEnum` classes from `base.py`/`foundation.identity`). `agent.py` `Agent` is the API/persistence model; `agents.py` `AgentConfig` is the runtime analogue used by the agent implementations — the comment at agents.py:7 calls this split out explicitly. Validation lives almost entirely on the Pydantic schemas (`min_length`, `ge`/`le`, `pattern`, `model_validator`); the dataclass models are intentionally validation-light. ## Mermaid + ```mermaid -graph TD - subgraph "Spawn-time prompt composition" - O[orchestrator._generate_prompt] --> AC[agents_config.get_agent_role/team] - AC --> FOUND[foundation.identity.AGENTS] - O --> CP[compose_prompt] - CP --> TLD[_tool_load_directive_layer role] - CP --> LL[_lifecycle_layer _generated/lifecycle-role.md] - CP --> BASE[base.md] - CP --> RL[_role_layer roles/file.md] - CP --> AVL[_autogen_verbs_layer _generated/role.md] - CP --> TL[_team_layer teams/file.md] - CP --> ID[identities/agent_slug.md] - CP --> AMB[ambient string] - TLD --> OUT["/app/prompts-generated/agent_id-prompt.md"] - LL --> OUT - BASE --> OUT - RL --> OUT - AVL --> OUT - TL --> OUT - ID --> OUT - AMB --> OUT - end - subgraph "Ambient (async, best-effort)" - OA[orchestrator._resolve_conventions_ambient] -->|settings.conventions_enabled| CAL[conventions_ambient_layer] - CAL --> CS[ConventionsService.render_ambient_block] - CS --> RC[ensure read clone] - CAL -->|cap 3000| AMB - end - subgraph "Identity binding" - IAT[issue_agent_token] --> HMAC[HMAC-SHA256 agent_id:role:team] - VAT[verify_agent_token] --> HMAC - IPT[issue_panel_token] --> IAT - end - subgraph "Injection guard (per turn)" - IDV[IntakeDriver.send_turn] --> DI[detect_injection] - DI -->|match| RM[refusal_message -> error chunk, return] - DI -->|clean| MODEL[forward to model] - GE[grok entrypoint] -->|CLI main exit 1| DI - BASH[user-prompt-hook.sh] -.same 5 patterns.-> DI - end +erDiagram + Project ||--o{ Task : "project_id" + Product ||--o{ Task : "product_id" + Product ||--o{ ProductCellMapping : cells + Task ||--o{ Task : "parent_task_id" + Task ||--o{ Task : "dependency_ids" + Task ||--o{ CommitRef : commits + Task ||--o{ DocRef : documents + Task ||--o{ ProgressUpdate : progress_updates + Task ||--o{ Checkpoint : checkpoints + Task ||--|| TaskPlan : plan + TaskPlan ||--o{ SubTask : sub_tasks + Task ||--o| WorkSession : "work_session_id" + WorkSession }o--|| Project : "project_id" + WorkSession }o--|| Task : "task_id" + WorkSession }o--|| Agent : "agent_id" + Agent ||--o{ Journal : "journal_id" + Journal ||--o{ JournalEntry : entries + JournalEntry }o--o| Task : task_id + ExtractedMessage }o--|| Agent : "agent_id" + Agent ||--o{ Notification : "from_agent" + Notification }o--o{ Task : "related_task_id" + Task ||--o| Playbook : "source_task_ids" + Pitch ||--o| Product : "provisioned_product_id" + Pitch ||--o{ Project : "provisioned_project_ids" ``` ## Logical Tree + ``` -prompts-roles-taxonomy slice -├── Prompt composition (roboco/agents/factories/) -│ ├── _base.py -│ │ ├── PROMPTS_BASE_PATH resolver -│ │ ├── _load_layer (file -> str|'') -│ │ ├── Layer maps: _ROLE_LAYER_MAP, _TEAM_LAYER_MAP -│ │ ├── Layer loaders: _role_layer, _team_layer, _autogen_verbs_layer, _lifecycle_layer -│ │ ├── Builtin-tool grant: _BUILTIN_TOOLS_COMMON/AUTHORS, _ROLE_BUILTIN_TOOLS, _tool_load_directive_layer -│ │ ├── compose_prompt (ordered join with '---') -│ │ └── conventions_ambient_layer (async, multi-project, 3000-char cap) + _AMBIENT_TOTAL_CAP -│ └── __init__.py (shim) -├── Permission taxonomy (roboco/agents_config.py) -│ ├── HMAC token layer: _auth_secret, _signing_payload, issue_agent_token, verify_agent_token, issue_panel_token -│ ├── UUID<->slug: _UUID_TO_SLUG, _resolve_to_slug -│ ├── Derived maps: AGENT_ROLE_MAP, AGENT_TEAM_MAP, CELL_MEMBERS, ALL_AGENTS, BOARD_MEMBERS, ALL_DOCS -│ ├── Role sets: TASK_CREATOR_ROLES, _CANCEL_ROLES, ROLE_PERMISSION_LEVELS, _BOARD_ROLES, _MAIN_PM_TARGETS -│ ├── Escalation: ESCALATION_CHAIN, get_escalation_target, get_pm_for_team, get_pm_for_agent -│ ├── Helpers: get_agent_role/team/cell, get_cell_members, is_pm/board_member/management/ceo, can_send_notifications/create/assign/cancel_tasks -│ ├── Notification enums: VALID_NOTIFICATION_TYPES/PRIORITIES -│ ├── A2A skills: ROLE_SKILLS, get_agent_skills -│ └── A2A routing: _check_cell_pm/cell_member/main_pm_a2a, can_a2a_direct, get_a2a_route_hint -├── Injection guard (roboco/agent_sdk/prompt_guard.py) -│ ├── _PATTERNS (5 regexes mirroring user-prompt-hook.sh) -│ ├── detect_injection, refusal_message -│ └── main (CLI for grok entrypoint) -└── Prompt corpus (agents/prompts/) - ├── base.md (universal rules) - ├── roles/ (9 files: developer, qa, documenter, cell_pm, main_pm, pr_reviewer, board, prompter, secretary) - ├── teams/ (3 files: backend, frontend, ux_ui) - ├── identities/ (19 per-agent YAML+blurb files) - └── _generated/ (regenerated artifacts) - ├── lifecycle-.md (x14; from lifecycle spec via make lifecycle; CI-gated no-drift) - ├── .md verb-signature tables (x12; from schemas + role_config via regenerate_verb_tables.py) - └── verbs.md (aggregate reference doc; NOT injected at spawn) +models/ +├── tasks +│ ├── task.py Task, TaskCreate, TaskUpdate, TaskCreateRequest, CommitRef, DocRef, ProgressUpdate, Checkpoint, SubTask, TaskPlan +│ ├── kanban.py KanbanBoard, KanbanCard, KanbanColumn, KanbanSwimlane, column configs +│ ├── handoff.py DocumenterHandoff, CodeSample, DocumentationItem, ConversationRef, HandoffCreate +│ └── product.py Product, ProductCellMapping, ProductCreate, ProductUpdate +├── agents +│ ├── agent.py Agent, AgentCreate, AgentUpdate, ModelConfig, AgentPermissions, AgentMetrics +│ ├── agents.py AgentConfig, AgentState, DevTaskPhase/QATaskPhase/CellPMPhase/MainPMPhase/DocTaskPhase/ProductOwnerPhase/HeadMarketingPhase/AuditorPhase, TaskContext/ReviewContext/DocContext, AuditFlag, AuditReport +│ └── permissions.py PermissionLevel, ROLE_LEVELS, AgentContext, COMMUNICATION_MATRIX, TASK_PERMISSIONS, KB_PERMISSIONS +├── comms +│ ├── message.py ExtractedMessage, RawStream +│ ├── notification.py Notification, NotificationCreate, CreateNotificationParams +│ ├── a2a.py AgentCard, A2ATask, A2AMessage, parts, A2AConversation, A2AChatMessage, state mappers +│ ├── extraction.py ExtractionContext, ExtractionResult, ExtractionConfig +│ └── transcription.py StreamBuffer, TranscriptionConfig +├── git / work-session +│ ├── work_session.py WorkSession, WorkSessionStatus, WorkSessionCreate, WorkSessionUpdate +│ └── project.py Project, BranchReason, ProjectCreate, ProjectUpdate +├── journal / audit +│ ├── journal.py Journal, JournalEntry, JournalEntryCreate, factory params, create_*_entry, JournalStats, GrowthMetrics +│ ├── audit.py AuditEventType, PermissionDenialContext, StateTransitionDenialContext +│ └── dashboard.py FlagData, ReportData, TeamHealthData, AuditQueueItem, DashboardStorage +├── llm +│ ├── llm.py LLMUsage, ToonConfig, EncodedBlock, ToonMetrics +│ ├── llm_catalog.py CatalogEntry, MODEL_CATALOG, provider_type_for_model, OLLAMA_DEFAULT_MODEL +│ └── runtime.py OrchestratorAgentState, SpawnGitContext, OrchestratorAgentConfig, AgentInstance, WaitingRecord, MODEL_MAP, ROLE_MODEL_MAP +├── metrics +│ └── metrics.py VelocityMetrics, BlockerMetrics, TeamMetrics, AgentMetrics, StageTiming, StageBottleneck, BottleneckReport, AgentReworkRate, TeamReworkRate, ReworkReport, Scorecard +├── optimal (RAG) +│ └── optimal.py IndexType, SearchResult, SearchOutcome, RAGResponse, QueryContext, ErrorPattern, Decision, Standard, MentorResponse, CodeReviewResult, ValidationResult +├── events +│ └── events.py EventType, Event, NotificationServiceProtocol, OrchestratorAccessProtocol, EventContext +├── company / strategy +│ ├── pitch.py Pitch, PitchStatus, PitchCreate +│ ├── secretary.py DirectiveKind, DirectiveStatus, GATED_KINDS +│ └── playbook.py Playbook, PlaybookCreate, PlaybookUpdate +└── base + └── base.py RobocoBase, TimestampMixin, all shared StrEnums, AgentRole/Team aliases, Annotated ID types ``` ## Dependencies -- Internal: roboco.foundation.identity (AGENTS, Role, Team, CELL_TEAMS, BOARD_ROLES, slugs_for_team), roboco.foundation.policy.communications (NOTIFY_SENDER_ROLES), roboco.seeds.initial_data (AGENT_UUIDS, CEO_AGENT_ID), roboco.models.base (AgentRole, Team, NotificationType, NotificationPriority), roboco.config.settings (conventions_enabled), roboco.services.conventions (get_conventions_service, ConventionsService.render_ambient_block/resolve_workspace), roboco.db.base (get_session_factory), roboco.db.tables (ProjectTable), roboco.runtime.orchestrator (_generate_prompt, _resolve_conventions_ambient, _resolve_ambient_projects), roboco.agent_sdk.intake_driver (IntakeDriver.send_turn consumer), scripts/regenerate_verb_tables.py (regenerates _generated/.md + verbs.md), scripts/build_lifecycle_artifacts.py (regenerates _generated/lifecycle-.md; make lifecycle), docker/scripts/user-prompt-hook.sh (canonical bash guard mirrored by prompt_guard.py), roboco.api.schemas.v1 (Pydantic verb schemas the autogen tables derive from), roboco.services.gateway.role_config (role->verb config the autogen tables derive from) -- External: pathlib.Path, hmac / hashlib (HMAC-SHA256 tokens), os.environ (ROBOCO_AGENT_AUTH_SECRET), re (injection regexes), sys (prompt_guard CLI), sqlalchemy.ext.asyncio.AsyncSession (ambient layer), typing.Final, argparse-less sys.argv CLI + +- **Pydantic** (`BaseModel`, `ConfigDict`, `Field`, `model_validator`, `field_validator`) — every `RobocoBase` subclass. +- **stdlib** `dataclasses`, `enum.StrEnum`, `datetime`, `uuid`, `typing` — the pure-dataclass files. +- **`roboco.foundation.identity`** — `base.py:21` imports `identity` and aliases `AgentRole = identity.Role`, `Team = identity.Team`. `product.py` and `pitch.py` import `CELL_TEAMS`/`Team` directly from `foundation.identity`. +- **`roboco.agents_config`** — `permissions.py:11` imports `ROLE_PERMISSION_LEVELS` to build `ROLE_LEVELS` at import time. +- **`roboco.models.runtime`** — `llm_catalog.py:22` imports `MODEL_MAP` to derive Anthropic catalog entries. +- **`roboco.models.message`** — `extraction.py:12` imports `ExtractedMessage`. +- **`roboco.models.product`** — `task.py:24` imports `ProductCellMapping` (the `cell_projects` field). +- Internal cross-imports are otherwise minimal; `__init__.py` is the single aggregation point. ## Entry Points -| Name | File | Trigger | -|---|---|---| -| orchestrator._generate_prompt | roboco/runtime/orchestrator.py | Called per agent spawn to compose + write the system-prompt .md file; calls compose_prompt (line 3238) | -| orchestrator._resolve_conventions_ambient | roboco/runtime/orchestrator.py | Async, called from the spawn path before _generate_prompt to resolve the optional ambient block; calls conventions_ambient_layer (line 3299) | -| IntakeDriver.send_turn | roboco/agent_sdk/intake_driver.py | Per interactive turn (Intake/Secretary Claude-SDK and Grok sessions); calls detect_injection before forwarding to the model (line 379) | -| python -m roboco.agent_sdk.prompt_guard | roboco/agent_sdk/prompt_guard.py | CLI invoked by the grok one-shot entrypoint on ROBOCO_INITIAL_PROMPT; exit 1 denies start | -| make lifecycle | scripts/build_lifecycle_artifacts.py | Developer/CI target regenerating _generated/lifecycle-*.md; CI gates on git diff --exit-code | -| scripts/regenerate_verb_tables.py | scripts/regenerate_verb_tables.py | Developer target regenerating _generated/.md + verbs.md after role_config/schema changes | +- `from roboco.models import …` — the canonical import surface (`__init__.py` re-exports the API/Pydantic models + enums + `get_column_config`). +- Direct module imports for the dataclass-only files: `from roboco.models.runtime import AgentInstance, WaitingRecord, MODEL_MAP`, `from roboco.models.events import Event, EventType`, `from roboco.models.metrics import BottleneckReport, ReworkReport, Scorecard`, `from roboco.models.optimal import SearchResult, RAGResponse`, `from roboco.models.agents import AgentConfig, DevTaskPhase`, `from roboco.models.llm_catalog import MODEL_CATALOG, provider_type_for_model`. +- `roboco.models.base` — import `RobocoBase`, `TimestampMixin`, and any shared enum when building a new model. ## Config Flags -- ROBOCO_AGENT_AUTH_SECRET (env) — HMAC signing secret for agent/panel tokens; unset => verify_agent_token fail-closes (rejects every token), issue_*_token returns UNSIGNED -- ROBOCO_CONVENTIONS_ENABLED — gates whether conventions_ambient_layer resolves + injects the architectural-standard ambient block; off => compose_prompt omits the ambient layer entirely -- ROBOCO_FABLE_MODE_ENABLED (default off) — gates fable_doctrine_layer AND ponytail_doctrine_layer (bundled — no separate ponytail flag); off => compose_prompt omits both doctrine layers entirely (byte-for-byte unchanged prompt) -- ROBOCO_PONYTAIL_INTENSITY (default full) — string value (lite/full/ultra), NOT a feature flag; selects the operative intensity the developer ponytail doctrine runs at (appended as a `**Operative intensity: ...**` directive for developers only; non-developers run a fixed restrained ethos regardless). `roboco/config.py` `ponytail_intensity`, validated as `Literal["lite","full","ultra"]` at Settings instantiation -- ROBOCO_SDK_URL (env, default http://localhost:9000) — used by the bash user-prompt-hook.sh (sister guard), not prompt_guard.py directly -- ROBOCO_INITIAL_PROMPT (env) — the one-shot prompt the grok entrypoint hands to prompt_guard CLI main -- PROJECT_HOST_PATH (orchestrator) — selects container (/app/prompts-generated) vs host ($TMPDIR/roboco-prompts) output dir for composed prompts +None — pure models, no flags. (The `Project` model *carries* opt-in fields `ci_watch_enabled`, `dep_update_command`, `dep_update_paths`, `sandbox_services` (project.py:142, validated against `VALID_SANDBOX_SERVICES` — sandboxed dev DB/Redis/Mongo, gated by `ROBOCO_SANDBOX_DB_ENABLED` elsewhere) that other layers gate on, and `llm_catalog` carries the "pure Ollama" defaults, but the models package itself reads no env / toggles nothing.) `VALID_SANDBOX_SERVICES` is now derived from the `SANDBOX_ENGINES` registry in `roboco/models/sandbox.py` (was a hardcoded `{"postgres","redis"}` set) — mongo is just another registry entry, no new migration (rides existing 057) and no new feature flag. ## Gotchas -- Layer ORDER matters and is load-bearing: tool-directive FIRST, then lifecycle, then base, then an optional fable-mode doctrine layer (`fable_mode_enabled`, `agents/prompts/doctrine/fable.md`), then an optional ponytail build-laziness doctrine layer (`fable_mode_enabled`, `agents/prompts/doctrine/ponytail.md` for developers / `ponytail-ethos.md` for other roles — bundled with Fable, no separate flag, role-scoped), then role, then autogen-verbs, then team, then identity, then ambient. The lifecycle fragment is intentionally before base so the agent reads its allowed verb surface before any other instruction. Reordering would change model attention priority. -- Empty/missing layers are silently dropped (compose_prompt skips falsy layers). An unknown role yields _role_layer=None AND _autogen_verbs_layer=None AND _lifecycle_layer=None — the agent would still spawn with just tool-directive + base + identity + ambient, missing its entire role+verb surface. The orchestrator guards upstream (raises ValueError on unknown role), but a typo in _ROLE_LAYER_MAP silently degrades to a roleless prompt rather than failing. -- _ROLE_LAYER_MAP maps all three board roles (product_owner/head_marketing/auditor) to the SAME board.md file. The per-role distinction (PO vs HoM vs Auditor) comes only from the identity file + the _generated/.md verb table, not from the role layer. A board role missing its identity file would lose its role-specific scope. -- verbs.md is the aggregate reference doc but is NOT injected at spawn — _base.py loads the per-role _generated/.md file instead. Editing verbs.md has zero prompt effect; it is a documentation/CI artifact only. The per-role files are the load-bearing ones. -- Identity YAML files carry a stale `role:` label (e.g. main-pm.md says `role: pm`, product-owner.md says `role: board`) that does NOT match the AgentRole enum values (main_pm/product_owner). The composition pipeline ignores this field entirely (loads identity by slug only); the real role comes from agents_config.AGENT_ROLE_MAP. Do not trust the identity YAML role label for enforcement. -- Board members (product_owner/head_marketing/auditor) have team=None, so _team_layer returns None for them — they get no team layer. main-pm likewise. Only cell members (dev/qa/doc/cell_pm) get a team layer. -- The autogen verb tables (_generated/.md) and lifecycle fragments are REGENERATED artifacts (make lifecycle / regenerate_verb_tables.py) gated on CI (git diff --exit-code). Hand-editing them is futile and will fail CI; change the source (lifecycle spec / Pydantic schemas / role_config) and regenerate. -- prompt_guard.py mirrors user-prompt-hook.sh patterns but is a separate implementation. The two must be kept in sync manually — there is no shared source. Drift between them means Claude (bash hook) and Grok/SDK (Python guard) apply different deny rules. -- detect_injection lowercases the text and uses loose anchoring (^|[\s>]) so injected content mid-message is caught, but the regexes are intentionally narrow (5 patterns). False negatives are expected by design — this is a classic-jailbreak denylist, not a comprehensive classifier; content that doesn't match still reaches the model. -- conventions_ambient_layer is best-effort and wraps the whole resolution in a try/except in the orchestrator (_resolve_conventions_ambient). A conventions resolution failure degrades silently to no ambient layer — a compose is never blocked by conventions. This means a conventions regression could quietly stop injecting the standard with no error surface. -- _AMBIENT_TOTAL_CAP (3000) truncates the ambient block with a trailing ellipsis. A large multi-project spawn (PO spanning several cells) can have its architectural standard silently truncated mid-block, leaving the agent with a partial standard. -- HMAC token verification fail-closes when ROBOCO_AGENT_AUTH_SECRET is unset — every agent token is rejected. issue_agent_token returns the literal sentinel 'UNSIGNED' which verify_agent_token also rejects. Deploying without the secret bricks all agent API auth (by design). +- `AgentRole` and `Team` are **not defined in `models/base.py`** — they are `identity.Role` / `identity.Team` aliased at base.py:23–24. The comment says "Removed in Phase 4 housekeeping after every consumer is migrated." SQLAlchemy `sa.Enum(AgentRole, name="agentrole")` still works because Python identity is preserved. New code should import from `roboco.foundation.identity` directly. +- `ProductCellMapping` deliberately overrides `use_enum_values=False` (product.py:22) so `team` stays a real `Team` enum for `is`-checks and the validator's `.value` error — the only model that deviates from `RobocoBase`'s `use_enum_values=True`. +- `Task` mutations are not done on the model; `task.py:337` directs callers to `TaskService` (`claim`, `start`, `block`, `complete`, …). Same pattern for `Notification`, `Journal`, `WorkSession` — service-owned state. +- `TaskCreate` has **no silent defaults** for `task_type` / `nature` / `estimated_complexity` (task.py:350 docstring) — mirrors `foundation.policy.task_completeness.TASK_AT_CREATE`. The 2026-05-08 trace of agents omitting `task_type` and deadlocking the lifecycle is the reason. +- `TaskCreate._exactly_one_target` (task.py:405) enforces exactly one of `project_id` / `product_id` / `cell_projects`. Old callers passing `cell_projects` alongside either of the others now fail. +- The "one active WorkSession per task" invariant is **not** in the model — it's a DB partial-unique index (migration 047) + service-layer guard. The model alone won't stop you constructing two active `WorkSession`s. +- `handoff.py` is **RESERVED** (file header, handoff.py:7) — `DocumenterHandoff`/`HandoffStatus` are defined but not wired into the service layer; current flow uses `dev_notes` + `handoff_summary` on `Task`. +- `a2a.py` has two parallel model families: the **wire/protocol** models (`AgentCard`, `A2ATask`, `A2AMessage`, parts — A2A spec) and the **persistent** models (`A2AConversation`, `A2AChatMessage`, `A2AInboxSummary` — DB storage). `A2AMessage` (wire) ≠ `A2AChatMessage` (stored). `task_status_to_a2a_state` / `a2a_state_to_task_status` bridge RoboCo `TaskStatus` ↔ `A2ATaskState` (lossy — many states collapse to `WORKING`). +- `agents.py` `AgentState` (agents.py:81) is a **different** class from the API `Agent.status`/`AgentStatus` enum — runtime vs persistence. Same for `AgentConfig` vs `Agent`. +- `events.py` carries a `TYPE_CHECKING` import of `WaitingRecord` from `roboco.runtime.orchestrator` — a rare upward reference, kept out of runtime by the `Protocol` seam. +- `permissions.py` `ROLE_LEVELS` is built at import time from `agents_config.ROLE_PERMISSION_LEVELS`; entries that don't parse are silently skipped (`except (ValueError, KeyError): pass` at permissions.py:35). +- `llm_catalog.py` Anthropic entries are **derived from `runtime.MODEL_MAP`** — bumping a Claude id there updates the catalog automatically. Ollama Cloud entries are hand-maintained. +- `Playbook` uses `from_attributes=True` (playbook.py:20) to load from the ORM `PlaybookTable`; most other models do not (they're constructed explicitly). ## Drift from CLAUDE.md -- CLAUDE.md Project Overview states '25 AI agents + 1 human CEO', but agents/prompts/base.md line 3 says '22 AI agents + 1 human CEO'. The base prompt agent count is stale relative to CLAUDE.md (memory notes a 20->22 update on 2026-06-16; CLAUDE.md later moved to 25). A spawned agent reads '22' in its system prompt while the org chart it sees has 25. -- CLAUDE.md's verb-surface table lists `i_am_blocked` for the qa and documenter roles. The role prompts qa.md and documenter.md ONLY added the i_am_blocked verb row in commit 15effce0 (this slice's baseline diff) — before that the role prompts omitted it even though the gateway accepted it. The prompts are now aligned, but the documenter.md circuit-breaker section previously explicitly said 'you don't have an i_am_blocked verb', which was false vs the gateway and vs CLAUDE.md. Fixed in 15effce0. -- CLAUDE.md says the lifecycle is defined in roboco/foundation/policy/lifecycle.py with a shim at roboco/enforcement/task_lifecycle.py. _base.py:_lifecycle_layer (line 187-200) docstring says the lifecycle fragment is regenerated from `roboco/lifecycle/spec.py` by `make lifecycle`. The actual source path the regenerator uses is roboco/lifecycle/spec.py (per the docstring), which is not mentioned in CLAUDE.md's lifecycle section — minor doc-path drift, not behavioral. -- CLAUDE.md describes the prompt composition as 'base + role + team + identity prompts' and an ambient 'Architectural Standard' block at spawn. The actual compose_prompt order (line 295-306) is tool-directive + lifecycle + base + fable + ponytail + role + autogen-verbs + team + identity + ambient — i.e. FIVE additional layers (tool-directive, lifecycle, fable, ponytail, autogen-verbs) not named in CLAUDE.md's composition description. CLAUDE.md undersells the actual layer stack. -- Identity files declare `role: pm` / `role: board` (e.g. identities/main-pm.md, identities/product-owner.md) which do not match the canonical AgentRole enum values (main_pm, cell_pm, product_owner, head_marketing, auditor) that CLAUDE.md and agents_config use. The pipeline ignores this label so it is cosmetic, but it is inconsistent with the canonical taxonomy CLAUDE.md documents. +- CLAUDE.md "Agent/Role/Team/ModelProvider in agent.py+base.py" — `Role` and `Team` are no longer *defined* in `base.py`; they are aliases to `roboco/foundation/identity.py` (base.py:21–24). The CLAUDE.md note about `ModelProvider` (`ANTHROPIC`/`GROK`/`LOCAL`/`OLLAMA_CLOUD`/`OPENAI` reserved) matches base.py:197–218 exactly. +- CLAUDE.md lists the `Task` model key fields (`task_type`, `project_id`, `branch_name`, `work_session_id`, `pr_number`, `pr_url`, `docs_complete`, `pr_created`, `commits: list[CommitRef]`) — all present on `Task` (task.py:163–255). CLAUDE.md does **not** mention `cell_projects` (task.py:179) or `batch_id`/`intends_to_touch`/`adds_migration`/`touches_shared` (task.py:225–236), which the MegaTask/sequencing sections elsewhere in CLAUDE.md do cover — so the model is ahead of the "Data Models" prose but consistent with the MegaTask section. +- CLAUDE.md "A task has at most one active WorkSession" — enforced by DB partial-unique index (migration 047) + service layer, not by the `WorkSession` model itself (work_session.py has no such constraint). Consistent with CLAUDE.md's "enforced both at the service layer and by a DB partial-unique index". +- The slice prompt named `AuditEvent` and `A2AEnvelope` as landmarks; the actual symbols are `AuditEventType` (audit.py:13, no `AuditEvent` class) and there is no `A2AEnvelope` in `a2a.py` (the gateway `Envelope` lives in `services/gateway/`, not here). Listed the real landmarks instead. +- Otherwise: `TaskStatus` 15-state enum, `TaskType` 6 values, `NotificationType`/`JournalEntryType` all match CLAUDE.md verbatim. +- v0.17.0 delta: the three new ORM tables backing cloud auth + the X engine (`UserTable`, `XCredentialsTable`, `XSeenMentionTable` — migrations 058/059) have **no Pydantic counterpart in this package** — cloud auth's `UserTable` is consumed directly by `fastapi_users`/`roboco.api.auth.*` and the X engine's two tables are read/written directly off the ORM row by `roboco.services.x_*`. They are documented as ORM `Key Symbols` in `db-migrations.md`, not here, consistent with this file's own Purpose statement ("these are not the ORM tables"). ## Changes Since Baseline -| SHA | Subject | Impact | -|---|---|---| -| 15effce0 | Chore: 141 Gaps fill-in (#283) — sole commit touching this slice since fd10cc86 | Prompt-surface alignment with gateway/spec: (1) developer.md + lifecycle-developer.md + verbs.md add the new sync_branch verb and rewrite the behind-base guidance on developer/cell_pm/main_pm to point devs at sync_branch instead of i_am_blocked/escalate_up (cell/root integration branches still escalate). (2) qa.md and documenter.md add the i_am_blocked verb row and rewrite the circuit-breaker section to use i_am_blocked instead of 'you don't have an i_am_blocked verb -> unclaim' — corrects a false prompt claim. (3) cell_pm.md delegate signature + verbs.md add the collision-surface fields intends_to_touch/adds_migration/touches_shared/depends_on and a new 'Collision surface' section instructing the PM to declare them on every code subtask so siblings sequence. (4) pr_reviewer.md adds the in-path gate verbs claim_gate_review/pr_pass/pr_fail + an 'In-path gate review' section. (5) prompter.md adds MegaTask root-subtask coordination-level AC guidance (task_type=planning, coordination-level ACs). (6) lifecycle-main_pm.md submit_root description changes from 'Only for code roots' to 'branch-bearing roots; gate is branch-keyed not task_type-keyed'. (7) note verb schema in all autogen tables gains done/next/where_to_look top-level string params; pass_review ac_verdicts and delegate covers_parent_criteria/intends_to_touch now show BeforeValidator in the signature. | +Range `fd10cc862c2020b3f639cdb686d427b0198a2441..HEAD`, `git log -- roboco/models/` → 2 commits (both the same MegaTask per-cell project-map feature, PRs #283/#285). `git diff --stat`: -> Post-snapshot updates (since 2026-06-29): **536bbb64** (Chore/all/logical gaps sweep #286) — (a) agents_config.py: `_TEAM_SCOPED_ROLES` deduped: was inline-defined, now re-exported as `_comms.TEAM_SCOPED_ROLES` from `foundation.policy.communications` (values unchanged: dev/qa/doc/cell_pm); (b) _generated/cell_pm.md, main_pm.md, qa.md, verbs.md: BeforeValidator repr cleaned from delegate/pass_review signatures — now renders `list[str] | None = None` instead of the memory-address-bearing BeforeValidator literal; (c) lifecycle spec: `PRECONDITION_ROOT_NOT_CODE` added to `submit_root` extra_preconditions, backing the branch-keyed / planning-typed claim the prompt asserts. **aba57359** ([chore] lifecycle artifacts regenerate, foundation-check) — lifecycle-cell_pm.md, lifecycle-developer.md, lifecycle-documenter.md, lifecycle-main_pm.md, lifecycle-qa.md: `unclaim` description expanded with "A PR reviewer who claimed an external/gate review and cannot finish releases the claim here rather than wedging the lane"; lifecycle-cell_pm.md + lifecycle-main_pm.md: `complete` description clarified "The merge runs BEFORE the complete transition" ordering. -> -> **v0.18.0** (2026-07-04): Fable mode adds a 9th conditional compose_prompt layer — `fable_doctrine_layer()` (_base.py:203) injects `agents/prompts/doctrine/fable.md` right after base.md, gated by `fable_mode_enabled` (default off; off = byte-for-byte unchanged prompt). FE/UX-UI design bar: `## Design bar` sections added to `teams/frontend.md` + `teams/ux_ui.md` (taste-skill-distilled dials + rules), plus a scoping pointer in `roles/developer.md` — doc-only, no flag, no compose_prompt change (team/role layers already existed; only their file contents grew). -> -> **v0.19.0** (2026-07-05): Ponytail build-laziness doctrine bundled with Fable — `ponytail_doctrine_layer(prompts_path, role)` in `roboco/agents/factories/_base.py`, gated on the same `fable_mode_enabled` flag (no separate flag — ponytail is Fable's complementary build-doctrine), slotted into compose_prompt immediately after `fable_doctrine_layer`. Role-scoped: developers (`AgentRole.DEVELOPER`) → `agents/prompts/doctrine/ponytail.md` (the full ladder: YAGNI → reuse-in-this-codebase → stdlib → native-platform → installed-dep → one-line → minimal, the rules, the Intensity table, the `ponytail:` comment convention) plus a trailing `**Operative intensity: {settings.ponytail_intensity}.**` directive; every other role → `agents/prompts/doctrine/ponytail-ethos.md` (ethos-only — the code-mechanics rungs and the Intensity table are dropped so they can't leak into prose artifacts like task plans / review notes / docs). Both files vendored from the ponytail plugin (MIT, Copyright (c) 2026 DietrichGebert), trimmed, YAML frontmatter stripped, and carry a 5-point RoboCo preamble that makes the ladder yield to the Architectural Conventions Standard (placement), the 80% coverage gate + QA review + self-verification, the per-team design bar, task hygiene, and reviewer feedback — overlap mitigated by scoping, not deletion. `ROBOCO_PONYTAIL_INTENSITY` (lite/full/ultra, default full; `roboco/config.py` `ponytail_intensity`, a string value — NOT a feature flag) selects the developer's operative intensity; non-developers get no dial. Prompt-only — no hooks, no grok-path changes; a flag-off spawn is byte-for-byte unchanged. -> -> **PR #544** (2026-07-17, `fd621f0d`): The design bar's web dials (`DESIGN_VARIANCE`/`MOTION_INTENSITY`/`VISUAL_DENSITY`) were silently steering a `source=video` authoring task toward "dense product UI → motion 2-3" — wrong for a marketing film. `agents/prompts/teams/ux_ui.md` gains a one-line video-mode override (see Files above) telling the dev the web dials do not apply to a video-authoring task and pointing it at `motion/README.md`'s cinematography bar + the vendored `motion/skills/` doctrine instead. Doc-only within this slice — the runtime half of the same PR (playwright MCP registered for the video-authoring ux-dev spawn, `_is_video_authoring_spawn`) is documented in `docs/map/video-engine.md` and `docs/map/orchestrator.md`. +``` + roboco/models/llm_catalog.py | 30 +++++++++++++++--------------- + roboco/models/runtime.py | 6 ++++++ + roboco/models/task.py | 36 ++++++++++++++++++++++++++++++------ + 3 files changed, 51 insertions(+), 21 deletions(-) +``` + +Logic-touching commits: + +- **15effce0 / 3aff6e04 — "Chore: 141 Gaps fill-in (#283)" / "Chore: Close gaps (#285)"** (MegaTask per-cell project map + Ollama catalog refresh) + - `task.py`: added `cell_projects: list[ProductCellMapping]` field to `Task` (task.py:171), `TaskCreate` (task.py:390), and `TaskCreateRequest` (task.py:482); imported `ProductCellMapping` from `product.py`; renamed `_project_or_product` → `_exactly_one_target` and widened from 2-way (`project_id`/`product_id`) to 3-way (`+ cell_projects`) with `sum(targets) != 1` rejection. Impact: a MegaTask root-subtask can now target an ad-hoc per-cell project map; old callers passing neither target still fail; callers passing `cell_projects` alongside another target now fail (previously silently passed because `cell_projects` didn't exist). + - `runtime.py`: `SpawnGitContext` gained `task_short_id: str | None = None` (runtime.py:38) — the per-task worktree id the agent must edit in; branchless coordination roots leave it `None`. Impact: additive, backward-compatible. + - `llm_catalog.py`: `glm-5.1:cloud` → `glm-5.2:cloud` in `MODEL_CATALOG` and `OLLAMA_ROLE_DEFAULTS`; role defaults reshuffled (`developer` minimax-m3 → kimi-k2.7-code, `cell_pm`/`main_pm`/`auditor` kimi-k2.6 → kimi-k2.7-code, `product_owner`/`head_marketing`/`ceo` kimi-k2.6 → glm-5.2, `documenter` glm-5.1 → kimi-k2.7-code); `OLLAMA_DEFAULT_MODEL` minimax-m3 → kimi-k2.7-code. Impact: catalog/UI labels + "pure Ollama" mode defaults only — **persisted `model_assignments` DB rows are not touched** (spawn model = persisted DB row, per the standing memory note), so an existing fleet doesn't silently switch models; only new "pure Ollama" provisioning picks the new defaults. + +> Post-snapshot updates (since 2026-06-29): 4 additional commits touched `roboco/models/`. +> - **c71f9b3b** `[chore] logical-gaps: kanban board column coverage + status-class fixes` — `kanban.py`: `DEV_COLUMNS` expanded from 7 to 15 entries (all `TaskStatus` values now covered); `QA_COLUMNS` dropped the `VERIFYING→"In Review"` entry (VERIFYING is the dev's self-check, not a QA state); `PM_COLUMNS` widened to include all gate/revision/paused/cancelled/backlog columns. No model API surface change; internal column configs only. +> - **d8a5bb48** `[chore] logical-gaps: a2a service hierarchy gate + persist skill on message row` — `a2a.py`: `A2AChatMessage` gained `skill: str | None` field (migration 054 adds the DB column). The `send()` verb now persists which capability the A2A is about so the receiver and inbox can surface it. +> - **536bbb64** `[chore] logical-gaps sweep (#286)` — `task.py`: `DocRef` gained `commit_status: str | None` (whether the doc reached the project repo: `committed`/`skipped`/`failed`); this 8-line insertion shifts all subsequent task.py line numbers by +8 vs the baseline annotations above. `playbook.py`: `Playbook` gained `archived_by: UUID | None` and `archived_at: datetime | None` to support the archive curation path. +> - **76ce53e3** `[fix] chat: wire live message delivery end-to-end (MESSAGE_SENT)` — `events.py`: added `EventType.MESSAGE_SENT = "message.sent"`; the bridge forwarded it to `/ws/sessions/{id}` and `/ws/channels/{id}` subscribers. **Now removed** by the comms-subsystem teardown (`docs/internal/specs/2026-07-03-comms-teardown-trace.md`) — the `/ws/sessions`/`/ws/channels` bridge and the tables it served are gone; `EventType.MESSAGE_SENT` itself is left as a dead/inert enum member, not deleted. ## Regression Risks | Title | File:Line | Claim | Severity | -|---|---|---|---| -| Collision-surface declaration is prompt-only, not gate-enforced | agents/prompts/roles/cell_pm.md:141 | The new 'Collision surface' section tells the cell PM to fill intends_to_touch/adds_migration/touches_shared on every code subtask so the analyzer can sequence colliding siblings, but the section explicitly says 'leave it empty only for a research/design subtask' — there is no documented gate that REFUSES a code delegate with empty intends_to_touch. If a PM omits it on a code subtask, two siblings editing the same file run in parallel and collide (the exact 2026-06-27 out-of-order break this was added to prevent). The protection hinges on agent compliance with prompt prose, not a hard gate. | medium | -| sync_branch guidance contradicting the i_am_done gate for behind-base branches | agents/prompts/roles/developer.md:126 | developer.md now says 'call sync_branch as soon as roboco_git_status shows your branch behind, OR when i_am_done refuses with your branch is N behind'. If the i_am_done gate's behind-base check and sync_branch's rebase disagree on what 'base' means (e.g. base resolved from the recorded branch vs the parent task's head), a dev could sync_branch successfully and still hit the i_am_done behind-base refusal, looping. The prompt assumes both use the same base resolution; a divergence there would trap the dev. No fallback to i_am_blocked is offered anymore (the prompt explicitly forbids it for a plain behind-base condition), removing the previous escape hatch. | medium | -| ~~documenter/qa circuit-breaker now directs to i_am_blocked — verify the verb is actually granted to those roles~~ **VERIFIED OK** | agents/prompts/roles/documenter.md:100 | The circuit-breaker section was rewritten from 'you don't have an i_am_blocked verb -> unclaim' to 'i_am_blocked(task_id, reason=...) to escalate'. This relies on i_am_blocked being genuinely callable by qa and documenter at the gateway. The autogen verbs.md shows i_am_blocked for qa but the documenter section in verbs.md (and _generated/documenter.md) must also list it — if the role_config does not grant i_am_blocked to documenter, the new prompt guidance sends the agent to a verb that will return not_authorized, trapping them on a circuit_open with no documented fallback (the old unclaim-only path was removed). **Verified 2026-07-01: lifecycle spec `i_am_blocked.allowed_roles = frozenset(_DEV_ROLES | _QA_ROLES | _DOC_ROLES)` — documenter IS granted this verb at baseline and post-snapshot; not a live risk.** | high | -| ~~submit_root description changed from code-root to branch-bearing-root semantics~~ **RESOLVED (536bbb64)** | agents/prompts/_generated/lifecycle-main_pm.md:14 | The lifecycle fragment now says submit_root is 'For branch-bearing roots' and 'The gate is branch-keyed, not task_type-keyed — a Main-PM root is planning-typed, never code'. If the gateway's submit_root implementation still keys off task_type=code (the old contract), a planning-typed branch-bearing root would be rejected by the gate while the prompt tells the Main PM to call submit_root on it — a loop. The prompt now asserts a behavior the gateway must match; mismatch breaks Main-PM root submission. **Fixed 2026-06-30: 536bbb64 added `PRECONDITION_ROOT_NOT_CODE` (`_p_root_not_code`: checks `task_type != code`) to `submit_root.extra_preconditions` in the lifecycle spec, and the spec description was updated to match; prompt and gateway are now aligned.** | high | -| note schema advertises done/next/where_to_look the gateway must accept | agents/prompts/_generated/developer.md:25 | All autogen verb tables now show note(...) with done/next/where_to_look top-level params. If the Pydantic note schema (the regenerator source) was updated but the gateway's note handler / DB journal model does not persist these fields, agents will pass them, the schema accepts them, but they are silently dropped — the handoff/quick_context fields the meltdown #1 fix intended to surface top-level would never reach downstream briefings. The prompt advertises params that may be no-ops at the persistence layer. | medium | -| ~~BeforeValidator rendering in verb signatures leaks into the prompt~~ **RESOLVED (536bbb64)** | agents/prompts/_generated/verbs.md:58 | The regenerated verb tables now render `BeforeValidator(func=, json_schema_input_type=PydanticUndefined)` literally into the agent's system prompt for pass_review.ac_verdicts, delegate.covers_parent_criteria and delegate.intends_to_touch. This is a memory-address-bearing repr of an internal Pydantic validator injected into every qa/cell_pm/main_pm prompt. It is noise the model must parse around and the address is non-deterministic across runs, which could in principle perturb caching/reprompt determinism. Not a correctness bug but a prompt-hygiene regression introduced by the regenerator. **Fixed 2026-06-30: 536bbb64 cleaned the regenerated tables; all three fields now render `list[str] | None = None`.** | low | -| cell_pm.md behind-base guidance split between dev sync_branch and cell-branch escalate_up | agents/prompts/roles/cell_pm.md:178 | The rewritten behind-base section tells the PM to direct devs to sync_branch for their leaf but to escalate_up for the cell integration branch. If a PM mis-classifies a behind-base condition (tells a dev to escalate_up instead of sync_branch, or calls escalate_up on a dev's leaf), the dev waits on a platform action that won't come (sync_branch is the dev's own verb). The split is correct but easy to misapply; a misroute strands the dev. | low | +|-------|-----------|-------|----------| +| `TaskCreate._exactly_one_target` 3-way validator | task.py:405 | Tests/clients asserting the old 2-way error message ("a task needs either a project_id… or a product_id…") will fail against the new message ("a task needs exactly one target: … or cell_projects …"). Any caller that constructed a `TaskCreate` with both `project_id` and `product_id` was already rejected; callers passing `cell_projects` + another target are newly rejected. | Medium | +| `Task.cell_projects` requires migration 052 | task.py:179 | The `cell_projects` field exists on the model regardless of DB state, but persistence (`TaskTable.cell_projects` relationship + `task_cell_projects` table) needs migration 052. On a DB where 052 hasn't run, `TaskService.create` with a non-empty `cell_projects` will fail at insert. | Medium | +| `SpawnGitContext.task_short_id` consumer parity | runtime.py:38 | The field is additive with a `None` default, but every spawn-path consumer that should route the agent into the per-task worktree must read it; a missed consumer silently falls back to the clone root (the old behavior). | Low | +| Ollama catalog defaults vs persisted assignments | llm_catalog.py:103 | `OLLAMA_DEFAULT_MODEL` changed, but spawn reads persisted `model_assignments` rows — so the default only applies when no row exists. An operator who deletes the `model_assignments` rows (the documented "kill stale fleet model" procedure) will now get kimi-k2.7-code instead of minimax-m3. Verify the tag actually works on the Ollama Cloud plan before relying on this. (`OLLAMA_ROLE_DEFAULTS` was removed 2026-07-17 as dead code — it was never consulted by routing.) | Low | +| `ProductCellMapping` import cycle risk | task.py:24 | `task.py` now imports from `product.py` at module load. `product.py` imports only from `foundation.identity` and `base.py` — no cycle today, but a future `product.py` → `task.py` import would create one. | Low | +| `AgentRole`/`Team` alias removal pending | base.py:21–24 | The aliases to `foundation.identity` are a migration shim ("Removed in Phase 4 housekeeping"). Consumers still importing `AgentRole`/`Team` from `roboco.models.base` will break when the shim is removed. | Low | ## Health -The composition pipeline is well-structured and deterministic: a single ordered join over gracefully-degrading layers, with CI-gated autogenerated artifacts (lifecycle + verb tables) that cannot silently drift from the spec, and a clean separation between the prompt corpus (markdown), the taxonomy (agents_config.py, derived from foundation so it cannot drift from the org chart), and the guard (prompt_guard.py, mirroring the bash hook). The main open integrity risks are (a) the prompt-only enforcement of the new collision-surface declaration on delegate — the 2026-06-27 out-of-order break this was added to fix can still recur if a PM omits intends_to_touch on a code subtask; (b) the stale agent-count in base.md (22 vs CLAUDE.md's 25) and the cosmetic but inconsistent `role:` labels in identity YAML. Previously flagged risks (b/c as of 2026-06-29 snapshot) have been closed: documenter/qa i_am_blocked is confirmed granted by the lifecycle spec (_DEV_ROLES|_QA_ROLES|_DOC_ROLES), and submit_root's branch-keyed claim is now backed by PRECONDITION_ROOT_NOT_CODE in the spec gate (536bbb64). BeforeValidator repr in prompt tables also cleaned (536bbb64). + +The models package is coherent and well-layered: a single `RobocoBase` config drives consistency, enums are centralized in `base.py` (with the `AgentRole`/`Team` alias-to-`foundation.identity` migration clearly commented), and the API/Pydantic vs runtime/dataclass split is explicit (`agent.py` vs `agents.py`, with a docstring calling it out). The recent MegaTask per-cell-map change is small, additive, and validator-guarded; the main follow-on risk is migration 052 parity and test assertions on the renamed validator message — both mechanical. Two long-standing cleanliness items linger: `handoff.py` is a reserved-but-unwired model (file header says so), and several files (`dashboard.py`, `transcription.py`, `extraction.py`) are pure dataclasses that read as service-layer DTOs rather than domain models — harmless but slightly muddies the "models = typed contract surface" framing. Enum parity with the ORM (`tables.py`) is enforced by the test gate. No blocking issues; the package is in good shape. +# db-migrations slice ## Purpose -This slice implements RoboCo's formal-notification backbone: NotificationService is the typed notification factory (blocker, QA-ready, A2A, board-review, ack), NotificationDeliveryService handles delivery (transactional-outbox bus publish), ACK tracking, expiry sweeps, PM/CEO task-handoff notifications, and the best-effort Telegram DM bridge (`_notify_telegram`), and notification_dedup is a bounded Redis SET-NX re-fire guard for loop-prone notification types. Together they turn lifecycle events into both a durable DB record and a real-time push, with multiple dedup layers (Redis re-fire window + DB purpose-dedup) to keep agent inboxes from flooding under coordinator loops. The Telegram side has grown into its own two-way bridge: V1 (outbound-only DMs on escalation/completion) plus V2's `TelegramInboundEngine` (`telegram_inbound.py`) — a poll loop that turns the CEO's Telegram replies/button-taps into the same CEO-gated service calls the HTTP routes make. +The DB layer is async SQLAlchemy 2.0 over PostgreSQL+asyncpg, with pgvector for the in-house RAG engine. Schema evolution is owned by an Alembic chain (001→061) that runs on every boot via `init_db()`; `Base.metadata.create_all` is no longer the source of truth — migration 017 reconciled the drift the other way. The ORM tables live in one fat module `roboco/db/tables.py` (~2.5k lines, 38 tables). ## Files -| Path | Role | LOC | -|---|---|---| -| roboco/services/notification.py | Typed notification factory (blocker/QA/docs/handoff/A2A/board-review/ack) with slug→UUID recipient resolution, DB purpose-dedup + Redis re-fire guard, owns its own DB context and commit | 943 | -| roboco/services/notification_dedup.py | Bounded Redis SET-NX re-fire guard for loop-prone notification types (TASK_ASSIGNMENT/REVIEW_REQUEST/DOCUMENTATION_REQUEST/BROADCAST); 60s TTL, fail-open | 91 | -| roboco/services/notification_delivery.py | Delivery (transactional-outbox deferred bus publish), ACK/read tracking, expiry sweep, PM/CEO task-handoff notifications (notify_pm_of_block, escalate_and_notify, etc.), API-facing list/CRUD, `_notify_telegram` best-effort CEO DM fan-out (V2: `actionable=True` on escalation attaches an Approve/Reject/Open inline keyboard) | 1305 | -| roboco/services/telegram_client.py | Bot API client ABC + `NullTelegramClient` (unconfigured, never egresses) + `LiveTelegramClient`: `send_message` (reply_markup/reply_to_message_id), V2 additions `get_updates` (long-poll), `answer_callback_query`, `edit_message_reply_markup`, `edit_message_text` | 247 | -| roboco/services/telegram_credentials.py | Singleton Fernet-encrypted `bot_token`/`chat_id` CRUD (mirrors `x_credentials.py`); decrypts server-side only, API returns `has_credentials` only | 109 | -| roboco/services/telegram_inbound.py | V2: `TelegramInboundEngine` — getUpdates poll cycle (offset persisted as `telegram_last_update_id` in system_settings), chat-id AND sender-id authorization, `/status` `/queue` `/task` command router, `apv|rej::` callback codec, force_reply reject/approve-notes state machine (in-memory `_PENDING_REPLIES`, TTL), per-kind dispatch to the SAME service methods the CEO-gated HTTP routes call (task/release/xpost/video/roadmap), `via=telegram` audit rows | 858 | - -## Data Flow -Two create-and-deliver paths exist. (A) NotificationService._create_notification (notification.py) opens its OWN get_db_context, resolves sender + recipients to UUIDs via _resolve_agent_uuid, runs the Redis re-fire guard (all_recipients_recently_notified), then DB purpose-dedup (ack-required types only, same sender+type+task+overlapping recipients not yet acked), builds NotificationTable with requires_ack from ACK_REQUIRED_BY_TYPE, flushes, calls NotificationDeliveryService.deliver (which defers NOTIFICATION_SENT bus events to after_commit), and finally commits — the commit triggers the deferred bus drain. (B) NotificationDeliveryService._persist_and_deliver (notification_delivery.py) is used by the task-handoff helpers (notify_pm_of_block, escalate_and_notify, etc.): it runs inside the CALLER's open transaction, applies only the Redis re-fire guard (no DB purpose-dedup), adds+flushes+delivers, and leaves the commit to the caller (api/routes/tasks.py). Sweeper loops in the orchestrator call sweep_expired_notifications periodically. Real-time push: deliver defers per-recipient NOTIFICATION_SENT events; the after_commit listener schedules _drain_pending_publishes which publishes to the StreamEventBus; websocket_bridge forwards to /ws/notifications/{id} sockets. ACKs flow acknowledge → acked_by/read_by mutation + NOTIFICATION_ACKED event. - -## Mermaid -```mermaid -flowchart TD - subgraph Notification - TYPED["send_blocker / send_qa_ready / send_a2a / send_ack ..."] --> CN["_create_notification"] - CN --> RES["_resolve_agent_uuid (sender+recipients)"] - CN --> RD["all_recipients_recently_notified
Redis SET-NX 60s"] - CN --> DD["DB purpose-dedup
ack-required only"] - CN --> NT["NotificationTable (requires_ack=ACK_REQUIRED_BY_TYPE)"] - CN --> DLV["deliver"] - HANDOFF["notify_pm_of_block / escalate_and_notify / notify_ceo_of_escalation"] --> PAD["_persist_and_deliver"] - PAD --> RD - PAD --> NT2["NotificationTable"] - PAD --> DLV - end - - subgraph Delivery - DLV -->|"in-tx"| DA["delivered_at = now"] - DLV --> DEF["defer_bus_publish
per-recipient NOTIFICATION_SENT"] - DEF -.->|"after_commit"| DRN["_drain_pending_publishes"] - DEF -.->|"after_rollback"| DIS["_discard_pending_publishes (no phantom)"] - DRN --> BUS["StreamEventBus"] - BUS --> WS["websocket_bridge → /ws/notifications/{id}"] - ACK["acknowledge"] --> AB["acked_by / read_by / acked_at"] - ACK --> BUS2["NOTIFICATION_ACKED event"] - end - - RD --> REDIS[("Redis SET-NX")] - DD --> PG[("PostgreSQL notifications")] - NT --> PG - DRN --> REDIS -``` - -## Logical Tree -``` -notification -├── notification.py (NotificationService) -│ ├── _resolve_agent_uuid (slug/UUID → UUID; 'system' seed) -│ ├── send_blocker / send_stuck_agent / send_qa_ready / send_docs_ready -│ ├── send_handoff / send_qa_failed / send_board_review_complete -│ ├── send_external_pr_reviewed / send_ack / send_a2a (tristate priority; `requires_ack` kwarg overrides the A2A_REQUEST type default of False — only the A2A CEO-DM wake path sets it True) -│ ├── _notification_type_label / _resolve_recipients -│ └── _create_notification (own DB context, re-fire guard, DB dedup, requires_ack, commit) -├── notification_dedup.py -│ ├── _LOOP_PRONE_TYPES (4 types) -│ ├── _DEDUP_TTL_SECONDS (60) -│ ├── _key (type:from:recipient:task) -│ └── all_recipients_recently_notified (per-recipient SET-NX, fail-open) -└── notification_delivery.py (NotificationDeliveryService) - ├── Transactional outbox (F107) - │ ├── defer_bus_publish (enqueue + register listeners) - │ ├── _schedule_pending_publishes (after_commit → loop.create_task) - │ ├── _drain_pending_publishes (best-effort publish) - │ └── _discard_pending_publishes (after_rollback) - ├── EscalationError / EscalationOutcome / BlockerDetails - ├── deliver (delivered_at in-tx, defer per-recipient events) - ├── get_notification / _notification_is_fully_acked / _log_expired_notification - ├── sweep_expired_notifications (log stale unacked) - ├── get_pending_for_agent / get_unacknowledged_for_agent / get_notification_count - ├── acknowledge / mark_read / bulk_acknowledge / get_ack_status / get_delivery_summary - ├── Task-handoff / audit-bridge notifications - │ ├── notify_pm_of_block / notify_pm_of_docs_complete / notify_pm_of_review_submission - │ ├── notify_assignee_of_unblock / notify_assignee_of_ceo_rejection - │ ├── escalate_and_notify (EscalationError/EscalationOutcome) - │ ├── notify_ceo_of_escalation - │ ├── notify_auditor_of_rework (ALERT to the auditor agent on needs_revision) - │ └── _persist_and_deliver (re-fire guard only, caller commits) - ├── Recipient helpers: _resolve_team_pm / _resolve_pm_for_agent_or_team / _get_agent_by_id/slug / _get_ceo_agent / _get_auditor_agent - ├── API-facing: list_system_notifications / list_for_agent / get_for_recipient_and_mark_read / acknowledge_for_recipient / mark_read_for_recipient - └── _notify_telegram (best-effort CEO DM; actionable=True on escalation attaches build_action_keyboard) -telegram_client.py (TelegramClient ABC / NullTelegramClient / LiveTelegramClient) -├── V1: send_message (reply_markup, reply_to_message_id) -└── V2: get_updates (long-poll) / answer_callback_query / edit_message_reply_markup / edit_message_text -telegram_inbound.py (TelegramInboundEngine, V2) -├── run_cycle (getUpdates offset cursor, dispatch each update, advance+persist offset) -├── _handle_message (chat+sender auth, force_reply pending-consume, command dispatch) -├── _dispatch_command (/status /queue /task /help) -├── _handle_callback (chat+sender auth, parse_callback, needs_reply branch → _prompt_for_reply, else _dispatch_approve) -├── _dispatch_approve / _dispatch_reject (per-kind handler dict: task/release/xpost/video/roadmap) -│ └── each handler calls the SAME service method the CEO-gated HTTP route calls; _mark_audit stamps a via=telegram AuditLogTable row -└── _finish_action (clears the buttoned message's keyboard, stamps the outcome) -``` - -## Dependencies -- Internal: roboco.config.settings (redis_url), roboco.db.tables (NotificationTable, AgentTable, TaskTable), roboco.db.base.get_db_context, roboco.events (Event, EventType, get_event_bus), roboco.foundation.policy.communications.ACK_REQUIRED_BY_TYPE, roboco.models.base (NotificationPriority, NotificationType, AgentRole), roboco.models.notification.CreateNotificationParams, roboco.services.base (BaseService, ConflictError, NotFoundError), roboco.services.permissions.has_privileged_access, roboco.services.repositories.get_agent_slug, roboco.agents_config (get_escalation_target, get_pm_for_agent, get_pm_for_team), roboco.utils.converters (require_uuid, to_python_uuid) -- External: sqlalchemy (select, and_, or_, func, event, joinedload, selectinload, with_for_update, IntegrityError, AsyncSession), redis.asyncio (from_url, set NX EX, aclose), asyncio (create_task, get_running_loop), structlog, datetime (UTC, datetime, timedelta), uuid.UUID, dataclasses -- telegram_inbound.py additionally imports: roboco.services.release_proposal (dispatch_approve, get_release_proposal_service, TaskAlreadyCompletedError), roboco.services.roadmap_service, roboco.services.task.get_task_service, roboco.services.telegram_credentials, roboco.services.tiktok_client/tiktok_credentials, roboco.services.video_post_service (VideoPostService, TaskAlreadyCompletedError, VideoCaptionTooLongError), roboco.services.x_credentials, roboco.services.x_post_service (TaskAlreadyCompletedError, XPostBodyTooLongError), roboco.services.x_video_client, roboco.foundation.policy.content.validators.reject_trivial, roboco.seeds.initial_data.AGENT_UUIDS - -## Entry Points - -| Name | File | Trigger | -|---|---|---| -| NotificationService.send_*_notification | roboco/services/notification.py | TaskService / orchestrator lifecycle transitions (blocker, qa-ready, docs, a2a, board-review) | -| NotificationService.send_ack_notification | roboco/services/notification.py | gateway `notify` content verb (PM/Board only) | -| NotificationDeliveryService.notify_pm_of_block / escalate_and_notify / notify_ceo_of_escalation / notify_auditor_of_rework | roboco/services/notification_delivery.py | api/routes/tasks.py i_am_blocked / escalate / ceo-approval routes; TaskService._alert_auditor_of_rework at QA-fail / rework chokepoints | -| NotificationDeliveryService.acknowledge / list_for_agent / get_for_recipient_and_mark_read | roboco/services/notification_delivery.py | api/routes/notifications.py ACK + list endpoints | -| sweep_expired_notifications | roboco/services/notification_delivery.py | orchestrator periodic loop (orchestrator.py:5780) | -| TelegramInboundEngine.run_cycle | roboco/services/telegram_inbound.py | orchestrator `_telegram_poll_loop` (default off, `telegram_enabled` AND `telegram_inbound_enabled`) | - -## Config Flags -- settings.redis_url — Redis URL used by notification_dedup for the SET-NX re-fire guard (derived from ROBOCO_REDIS_HOST/_PORT) -- `telegram_enabled` (default off) — V1 master switch; `_notify_telegram` no-ops without it AND stored credentials. -- `telegram_inbound_enabled` (default off, sub-switch on top of `telegram_enabled`) — V2: arms `TelegramInboundEngine.run_cycle` (the poll loop) and makes escalation DMs carry an actionable keyboard; with it off the bot only sends, never listens, and any inline button on an old message is inert. -- `telegram_poll_interval_seconds` (5.0) / `telegram_poll_timeout_seconds` (25, Bot API long-poll `timeout`) / `telegram_max_updates_per_cycle` (50) / `telegram_pending_reply_ttl_seconds` (300) — V2 poll-loop tuning. - - -## Gotchas -- Two notification create paths with DIFFERENT dedup strength: NotificationService._create_notification runs BOTH the Redis re-fire guard AND the DB purpose-dedup; NotificationDeliveryService._persist_and_deliver (task-handoff helpers) runs ONLY the Redis re-fire guard and explicitly skips DB purpose-dedup. A reworded BLOCKER_ESCALATION from the handoff path within 60s is suppressed by Redis, but beyond 60s a duplicate can be re-created since there is no DB dedup on that path. -- notification_dedup fail-open: a Redis error returns False (never suppress) — correct for not dropping notifications, but a sustained Redis outage re-opens the per-tick re-fire storm the guard was added to stop. -- notification_dedup.all_recipients_recently_notified has a side effect: it SET-NX-marks recipients NOT yet notified, so the FIRST call for a fresh recipient returns False (delivers) but acquires the key; a concurrent second call within 60s for the same recipient then returns True (suppresses). The marking happens even on the call that decides to deliver — so a suppressed 'all already held' verdict requires every recipient to have been marked by a prior call. Partial-fresh mixed-recipient calls deliver and mark the fresh ones. -- NotificationService._create_notification opens its OWN get_db_context and commits (line 568), while NotificationDeliveryService._persist_and_deliver operates in the CALLER's transaction and does NOT commit. Mixing the two in one outer transaction would double-commit / cross-session. -- requires_ack is set from ACK_REQUIRED_BY_TYPE (notification.py L555) rather than the column default True; MENTION/KNOWLEDGE_SHARE/BROADCAST etc. are False. `CreateNotificationParams.requires_ack` (default None) wins over the type default when a caller sets it — today only `send_a2a_notification`'s `requires_ack` kwarg (default False, `A2AService`'s CEO-DM wake path passes True) threads through to it; every other typed `send_*` helper leaves it unset and gets the type-default behavior unchanged. -- DB purpose-dedup query uses NotificationTable.to_agents.overlap(to_agents_uuids) AND ~acked_by.contains(to_agents_uuids) — overlap matches ANY recipient; a notification to [A,B] with A acked but B not is NOT suppressed for a new send to [A,B] because acked_by does not contain [A,B] (contains is element-wise). The dedup is per-(sender,type,task) not per-recipient, so a third recipient C added on resend goes through. -- defer_bus_publish registers after_commit/after_rollback listeners keyed on session.info[_DRAIN_REGISTERED_KEY]; listeners are bound to sync_session and accumulate only once per AsyncSession instance. A session reused across multiple commit cycles will re-register only once (guard), but the pending queue is popped each commit — if a second deliver happens after the first commit in the same session, the listeners are already registered and the new events append and fire on the next commit. -- acknowledge publishes NOTIFICATION_ACKED directly to the bus (NOT deferred via after_commit) — unlike deliver. An ACK that is rolled back after publish could emit a phantom ACK event. The ACK path does not use the transactional outbox. -- list_system_notifications filters pending_ack_only POST-fetch because 'not fully acked' is not SQL-friendly on PostgreSQL array columns. For pending_ack_only=True the SQL `limit` is NOT applied — applying it before the Python filter let a window of newer fully-acked rows mask older unacked ones the operator still needs to act on (correctness bug fixed in 115061f3); the full ack-required set is fetched ordered newest-first, Python-filtered to unacked, then sliced to `limit`. The non-pending branch retains the SQL limit. -- get_notification_count loads ALL notifications for an agent into memory (no SQL count) to compute total/unread/pending_ack — O(n) per call, no pagination. -- `TelegramInboundEngine._PENDING_REPLIES` (a force_reply prompt awaiting the CEO's free-text reply) is a per-process, in-memory dict keyed by `(chat_id, prompt_message_id)` — not durable. An orchestrator restart drops any in-flight prompt; the CEO just taps the button again. TTL-swept both lazily (on the next prompt) and on expiry at consume-time. -- `_authorized_chat` (chat id must equal the stored credentials' chat id) is the ONLY identity check a Telegram update carries — there is no agent/session token — so it stands in for every CEO-gated route's `require_ceo_role`. `_authorized_sender` (added in the same wave that added `_authorized_chat`'s callers) is defense-in-depth on top of it: when the update carries a `from` user, its id must ALSO equal the chat id (the supported deployment is a private 1:1 chat); a present-but-mismatched sender is refused, an absent one keeps chat-id-only behavior. -- The getUpdates offset cursor reuses the existing `system_settings` KV store (`telegram_last_update_id`, validated as a non-negative int) rather than a new table/migration — a restart resumes from the last-committed offset instead of replaying processed updates. -- `_dispatch_approve`'s `_approve_release` handler must pre-check the proposal's terminal state itself before calling `dispatch_approve` — that function fires the ~40min release execute as a background task and returns immediately with nothing to inspect, so a stale Approve on an already-rejected/published proposal would otherwise report a false "dispatched" success while the service's own guard silently no-ops it. - - -## Drift from CLAUDE.md -- CLAUDE.md does not describe the notification_dedup Redis re-fire guard, the transactional-outbox (defer_bus_publish / F107) in notification_delivery, or the DB purpose-dedup in NotificationService — all are real, load-bearing behavior added since the baseline and not reflected in the doc's Services table. -- CLAUDE.md's Services table lists NotificationService as 'Formal notifications' but does not mention NotificationDeliveryService at all, nor that NotificationService owns its own DB context+commit while NotificationDeliveryService runs in the caller's txn. -- CLAUDE.md no longer describes notification ack semantics after the channels/groups/discussion-sessions/messages teardown removed the Communication Model section — for the record: requires_ack is False for TASK_ASSIGNMENT/REVIEW_REQUEST/DOCUMENTATION_REQUEST/BROADCAST/KNOWLEDGE_SHARE/MENTION/A2A_REQUEST (ACK_REQUIRED_BY_TYPE), so most notification types do NOT require ack; and notifications can be sent by 'system' (orchestrator-generated), not only PMs/Board. -- CLAUDE.md mentions `roboco/services/notification.py` in the Services table but not `notification_dedup.py` or `notification_delivery.py`. - - -## Changes Since Baseline - -| SHA | Subject | Impact | -|---|---|---| -| 15effce0 | Chore: 141 Gaps fill-in (#283) — added requires_ack from ACK_REQUIRED_BY_TYPE, DB purpose-dedup gated to ack-required types, re-fire guard + notification_dedup.py (new file), transactional-outbox defer_bus_publish in notification_delivery | Major hardening: notifications no longer flood inboxes (Redis re-fire + DB dedup scoped), phantom WebSocket pushes eliminated (deferred bus publish), MENTION/BROADCAST no longer inflate unacked sets (requires_ack=False) | -| 3aff6e04 | Chore: Close gaps (#285) — follow-on gap closure touching notification.py / notification_dedup.py / notification_delivery.py | Refinement of the #283 changes (exact hunks not isolated per-file in this merge commit; consolidated the dedup/outbox behavior above) | - -> Post-snapshot updates (since 2026-06-29): 115061f3 fixed list_system_notifications pending_ack_only correctness: SQL limit is now dropped for that branch so newer fully-acked rows can't mask older unacked ones (see Gotcha update above). `61e00832` (PR #492) added `notify_auditor_of_rework()` and `_get_auditor_agent()` to power the reactive auditor dispatch path: HIGH-priority ALERT notifications addressed to the auditor agent are emitted when a task enters `needs_revision` via QA/PR/PM rework chokepoints. **Wave 3** (2026-07-17, PR #547): `CreateNotificationParams` gains `requires_ack: bool | None = None`, consulted in `_create_notification` ahead of the `ACK_REQUIRED_BY_TYPE` default; `send_a2a_notification` gains a `requires_ack: bool = False` kwarg (plus an `str | None` `task_id`, for a conversational DM with no task behind it) that threads through — the only caller passing True is `A2AService._maybe_wake_ceo_recipient` (docs/map/a2a-audit-journal-permissions.md), so its wake row is finally visible to the orchestrator's `_dispatch_a2a_work` `pending_ack_only` poll. -> `3b9fd0e0`+`11915f36` (PR #551, Telegram V2): `3b9fd0e0` adds `telegram_inbound.py` (new file, `TelegramInboundEngine`), extends `telegram_client.py` with `get_updates`/`answer_callback_query`/`edit_message_reply_markup`/`edit_message_text`, adds `actionable=True` to `_notify_telegram` (escalation only) so the DM carries an Approve/Reject/Open keyboard, and wires the orchestrator's `_telegram_poll_loop`. `11915f36` closes a live-reproduced approve-after-reject hole reachable via a stale Telegram button (or the pre-existing HTTP routes for X/video): `ReleaseProposalService.approve()` now refuses CANCELLED (`already_rejected`) and COMPLETED (`already_published`) proposals via a new `_approve_precheck`, `.reject()` refuses COMPLETED by raising a new `TaskAlreadyCompletedError`, and `XPostService`/`VideoPostService.approve()` each add a CANCELLED pre-lock-and-under-lock guard returning `already_rejected`. Also adds `_authorized_sender` (chat-id auth is defense-in-depth'd with a sender-id check) and widens `_resolve_task`'s search limit 10→50 so a genuine id-prefix hit can't be pushed out by newer title/description matches. - -## Regression Risks - -| Title | File:Line | Claim | Severity | -|---|---|---|---| -| DB purpose-dedup now gated to ack-required types only — informational duplicates no longer suppressed | roboco/services/notification.py:521 | is_ack_required = ACK_REQUIRED_BY_TYPE.get(params.notification_type, True); the DB dup_q is only run when is_ack_required. REVIEW_REQUEST/DOCUMENTATION_REQUEST/TASK_ASSIGNMENT are ack-required=False, so they skip DB dedup and rely SOLELY on the 60s Redis window. Beyond 60s, a coordinator can re-fire the same REVIEW_REQUEST every tick and each one persists (the original bug the dedup was meant to stop). The Redis guard coalesces within 60s but a tick interval >60s re-opens the flood. Severity medium because the Redis guard covers the common per-tick storm. | medium | -| _persist_and_deliver skips DB purpose-dedup entirely — task-handoff duplicates not DB-deduped | roboco/services/notification_delivery.py:875 | _persist_and_deliver applies only all_recipients_recently_notified (Redis) and then add+flush+deliver with no DB dup_q. notify_pm_of_block / escalate_and_notify / notify_assignee_of_unblock can each be re-triggered (e.g. a retried i_am_blocked, a re-issued escalate) and, past the 60s Redis window, create a second BLOCKER_ESCALATION for the same (sender, type, task) while the first is unacked — exactly the inbox inflation + i_am_idle soft-block the DB dedup was added to prevent on the other path. Two paths for the same notification type with different dedup strength is a real hole. | medium | -| acknowledge publishes NOTIFICATION_ACKED directly, not via the transactional outbox | roboco/services/notification_delivery.py:451 | deliver was migrated to defer_bus_publish (after_commit) to kill phantom pushes, but acknowledge still does `await bus.publish(...)` inside the open transaction before the caller commits. A rollback after a successful ACK publish emits a phantom NOTIFICATION_ACKED for an ACK that didn't persist — the same class of bug F107 fixed for deliver, left unfixed for the ACK path. | medium | -| all_recipients_recently_notified marks recipients as a side effect on the deciding call | roboco/services/notification_dedup.py:78 | The function SET-NX-marks each fresh recipient while computing the verdict, so the call that DECIDES TO DELIVER also acquires keys for the fresh recipients. A subsequent resend within 60s then sees all-held and suppresses — intended — but it means the very first notification in a window consumes the TTL for recipients who genuinely received it, and a legit follow-up to a subset within 60s is suppressed if all of that subset were marked by the prior send. For BROADCAST this can drop a legitimately re-targeted broadcast within the window. | low | -| get_notification_count loads all agent notifications into memory | roboco/services/notification_delivery.py:379 | base_query selects all NotificationTable rows where to_agents contains agent_id with no limit, then counts in Python. For a long-running agent this row count grows unbounded; called via get_delivery_summary on the panel it is an O(n) DB read per dashboard load. Not a correctness regression from the baseline but the slice's new dedup reduces new-row growth, masking the unbounded-scan risk. | low | -| defer_bus_publish listener registration tied to session.info on the AsyncSession — session reuse hazard | roboco/services/notification_delivery.py:116 | _DRAIN_REGISTERED_KEY is set once per AsyncSession and the SQLAlchemy event.listens_for(sync_session, ...) is bound to sync_session. If an AsyncSession is reused for multiple independent transactions (connection-pool recycling), the listener stays registered and fires _schedule_pending_publishes on every subsequent commit even when no new events were deferred — _schedule_pending_publishes pops an empty queue and no-ops, so it is benign, but the listener is never removed and accumulates on the sync_session for the session's lifetime. A long-lived sync_session with many AsyncSession wraps could accumulate listeners. | low | -| `notify_auditor_of_rework` is best-effort and not deduplicated beyond the Redis re-fire guard | roboco/services/notification_delivery.py:937 | Delivery failures are swallowed and logged by the TaskService caller so the needs_revision transition never blocks. ALERT is ack-required, so each unacked rework event persists until the auditor acks it; repeated QA/PR/PM rejects on the same task emit one ALERT per transition. | low | - -## Health -This slice is substantially hardened since the baseline: the transactional-outbox for delivery (F107), the Redis re-fire guard, and the ACK_REQUIRED_BY_TYPE-driven requires_ack are all real, well-documented fixes that close prior meltdowns. The main integrity gap is dedup-path fragmentation: NotificationService._create_notification runs two dedup layers (Redis + DB purpose-dedup) while NotificationDeliveryService._persist_and_deliver runs only the Redis layer, so the task-handoff notifications (blocker/escalation/ceo-rejection) are not protected by DB purpose-dedup past the 60s Redis window — a retried i_am_blocked or escalate beyond 60s can re-create an unacked duplicate, the exact inbox-inflation + i_am_idle soft-block the DB dedup was added to prevent. A secondary consistency gap is that acknowledge publishes NOTIFICATION_ACKED directly to the bus instead of through the deferred outbox, leaving the same phantom-event class F107 fixed for deliver. Neither is a crash bug; both are correctness drift between two paths that should behave identically. Code quality is high (terse comments, clear docstrings, explicit race handling), and the slice is well-covered by the orchestrator sweeper integration and route-level callers. - -## Purpose -This slice is the agent-to-agent communication, audit forensics, journaling, structured-note persistence, message extraction, and role-based access-control backbone of RoboCo's service layer. A2AService builds Agent Cards and manages persistent slug-keyed A2A conversations/messages plus the legacy A2A-protocol task/notification path. AuditService best-effort persists security/lifecycle events to audit_log and backs the PM-respawn tracing-gap circuit breaker. JournalService owns agent journal CRUD, gateway scope→type entry writes, and fire-and-forget RAG indexing. content_notes is the single chokepoint that validates and persists structured task notes plus their derived TEXT mirrors. ExtractionService classifies raw LLM stream buffers into typed messages. PermissionService enforces notification/task/KB access from agents_config. - -## Files - -| Path | Role | LOC | -|---|---|---| -| roboco/services/a2a.py | A2A protocol + persistent conversation service: Agent Cards, task↔A2A conversion, legacy A2A task notifications, bidirectional response spawning, slug-keyed conversation/message CRUD, gateway send adapter, CEO admin/live-view surface (reply budget + org-wide read), CEO-DM offline-recipient wake | 2090 | -| roboco/services/audit.py | AuditService singleton: best-effort persist denial/lifecycle/agent events to audit_log, resolve actor role + slug→UUID at write time, tracing-gap query for respawn circuit breaker, recent-events query | 462 | -| roboco/services/journal.py | JournalService: journal/entry CRUD, gateway scope-string→JournalEntryType adapter, fire-and-forget RAG indexing (private excluded), tracing-gate existence checks, board review brief, growth analytics | 1029 | -| roboco/services/content_notes.py | Single chokepoint applying structured notes: validate via foundation ContentModel, store in notes_structured, regenerate derived TEXT mirror column (dev_notes/qa_notes/etc.) | 74 | -| roboco/services/extraction.py | ExtractionService + ExtractionPipeline: regex-pattern (and optional LLM/TOON) classification of raw agent LLM buffers into typed ExtractedMessages; transcription pipeline callback fan-out | 508 | -| roboco/services/permissions.py | PermissionService singleton + async helpers: notification scope (all/cell/board-chain), task-action and KB-action RBAC from agents_config; privileged/PM-role DB lookups | 425 | +| Path | Role | +|------|------| +| `roboco/db/__init__.py` | Re-exports `Base`, session helpers, `bootstrap_database`, table classes. | +| `roboco/db/base.py` | `Base` (DeclarativeBase + naming convention), `get_engine`, `get_session_factory`, `get_db` (FastAPI dep), `get_db_context`, `run_migrations`, `init_db`, `_db_has_*` probes. Stamps a pre-Alembic DB at 001 then upgrades head. | +| `roboco/db/tables.py` | All 37 ORM table classes (single module). | +| `roboco/db/seed.py` | `bootstrap_database()` — runs `init_db` then seeds agents. | +| `alembic/env.py` | Async Alembic env; imports `roboco.db.tables` to register metadata, overrides `sqlalchemy.url` from settings, `compare_type` + `compare_server_default` on. | +| `alembic.ini` | Standard config; `script_location=alembic`, `prepend_sys_path=.`, no URL (set in env.py). | +| `alembic/versions/` | 61 migration files 001..061 (two share number 026 — chained, not a collision). | ## Key Symbols | Name | Kind | File:Line | Responsibility | -|---|---|---|---| -| A2AService | class | roboco/services/a2a.py:55 | Service layer for A2A protocol ops and persistent agent conversations; takes an AsyncSession | -| get_service_endpoint | staticmethod | roboco/services/a2a.py:69 | Build outbound callback URL; dials loopback when host binds 0.0.0.0/:: to avoid bandit B104 | -| build_system_agent_card | staticmethod | roboco/services/a2a.py:87 | Return the system-level Agent Card served at /.well-known/agent.json | -| build_agent_card | method | roboco/services/a2a.py:144 | Resolve agent by UUID or slug and return its AgentCard (None if missing) | -| _agent_to_card | method | roboco/services/a2a.py:172 | Map an AgentTable row to an AgentCard with role-keyed skills + bearer security scheme | -| task_to_a2a | method | roboco/services/a2a.py:265 | Canonical RoboCo TaskTable→A2ATask conversion with status mapping and metadata | -| get_task | method | roboco/services/a2a.py:320 | Fetch task by UUID string and return A2ATask or None | -| list_tasks | method | roboco/services/a2a.py:345 | Paginated task listing with has_more detection (fetches page_size+1) | -| _status_value_of | staticmethod | roboco/services/a2a.py:383 | Extract task status as a comparable string (enum .value or str); factored from cancel_task for xenon complexity gate | -| _apply_cancel_note | method | roboco/services/a2a.py:387 | Append actor-attributed cancellation note to task.dev_notes and flush; factored from cancel_task | -| cancel_task | method | roboco/services/a2a.py:408 | Cancel task and cascade to descendants; now takes agent_role (threaded into TaskService.cancel role gate) and actor_slug (recorded in cancellation note); rejects terminal states | -| discover_agents | method | roboco/services/a2a.py:442 | List AgentCards filtered by role/team/skill_tag | -| get_team_from_agent | staticmethod | roboco/services/a2a.py:485 | Map agent slug to Team enum via agents_config.get_agent_team (defaults BACKEND) | -| resolve_target_agent | staticmethod | roboco/services/a2a.py:496 | Resolve target agent slug from metadata (explicit target_agent or skill-based routing) | -| extract_message_text | staticmethod | roboco/services/a2a.py:523 | Split first text part into (title, description, full_text) | -| update_task_with_message | staticmethod | roboco/services/a2a.py:540 | Append A2A-protocol message text to task.dev_notes (legacy A2A thread store, not gateway conversations) | -| resolve_creator_agent | method | roboco/services/a2a.py:563 | Resolve creator AgentTable from from_agent slug or fall back to first main_pm | -| create_a2a_notification | method | roboco/services/a2a.py:619 | Legacy A2A peer-to-peer notification (requires task_id); requires from_agent present and target_agent resolvable (raises ValueError with distinct messages if either missing); enforces hierarchy unconditionally via validate_a2a_access (A2AAccessDeniedError), then parse_priority, delegates to NotificationService.send_a2a_notification | -| update_task_from_message | method | roboco/services/a2a.py:657 | Append response message to existing A2A task and notify/spawn original requester (bidirectional) | -| _lookup_requester_slug | staticmethod | roboco/services/a2a.py:700 | Reverse-lookup agent slug from creator UUID via static AGENT_UUIDS map | -| _publish_a2a_response_event | staticmethod | roboco/services/a2a.py:711 | Publish TASK_ASSIGNED event to event bus to spawn/notify the original requester; swallows errors | -| _notify_original_requester | method | roboco/services/a2a.py:741 | If task.dev_notes carries 'A2A Request' marker, publish response event to requester (skips self-response) | -| _canonical_pair | staticmethod | roboco/services/a2a.py:779 | Return two agent slugs in lexically-sorted order (conversation uniqueness key) | -| get_or_create_conversation | method | roboco/services/a2a.py:784 | Validate A2A access (validate_a2a_access), canonical-order lookup or create A2AConversationTable row | -| get_conversation | method | roboco/services/a2a.py:850 | Fetch conversation by ID only if agent_slug is a participant | -| get_conversation_admin | method | roboco/services/a2a.py:952 | Wave-2 CEO live view: like get_conversation but WITHOUT the participant check — returns any conversation by id for the org-wide read; None only if it truly doesn't exist | -| list_conversations | method | roboco/services/a2a.py:881 | List conversation summaries for an agent with optional status/with_agent/task_id filters; per-conv last-message preview query (N+1) | -| list_conversations_admin | method | roboco/services/a2a.py:1063 | Wave-2 CEO live view: list conversations across every agent pair (no participant filter), most-recent-first; backs GET /chat/admin/conversations | -| list_admin_pairs | method | roboco/services/a2a.py:1101 | Wave-2c switchboard: every agents_config.A2A_ALLOWED_PAIRS entry joined with its representative conversation (most-recently-updated when >1) via one bulk tuple_(agent_a,agent_b).in_() query — never N+1; backs GET /chat/admin/pairs | -| close_conversation | method | roboco/services/a2a.py:962 | Mark conversation CLOSED with optional resolution; participant-only | -| _enforce_ceo_reply_budget | method | roboco/services/a2a.py:1177 | Wave-2 reply-then-wait budget on the CEO's inbox — the one stateful gate the stateless can_a2a_direct matrix can't see. An agent may message the CEO only inside a conversation the CEO itself opened, and only up to the CEO's own message count there (rejects once agent_count >= ceo_count); no-op for CEO-authored sends or non-CEO conversations | -| send_chat_message | method | roboco/services/a2a.py:1225 | Send message in conversation; nil-UUID guard; dedup unread identical (conv,sender,kind,content); calls _enforce_ceo_reply_budget before persisting; bump unread for other side; reads skill from opts and persists it on the message row (nullable) | -| get_messages | method | roboco/services/a2a.py:1337 | Paginated chronological message list for a participant | -| get_messages_admin | method | roboco/services/a2a.py:1375 | Wave-2 CEO live view: like get_messages but WITHOUT the participant check — reads any conversation's transcript; [] only if the conversation truly doesn't exist | -| mark_read | method | roboco/services/a2a.py:1410 | Zero agent's per-side unread counter and bulk UPDATE read_at on inbound unread messages | -| mark_all_read | method | roboco/services/a2a.py:1451 | Agent-keyed bulk mark_read across all conversations with unread for this agent; returns count cleared | -| get_inbox_summary | method | roboco/services/a2a.py:1492 | Aggregate total unread, conversations with unread, pending + unanswered requires_response counts | -| list_pairs | method | roboco/services/a2a.py:1553 | Group conversations into unique agent pairs with rollup counts/unread/last_activity for frontend | -| _conv_to_model | method | roboco/services/a2a.py:1603 | A2AConversationTable→A2AConversation Pydantic model | -| _msg_to_model | method | roboco/services/a2a.py:1621 | A2AMessageTable→A2AChatMessage Pydantic model; now maps skill field (migration 054 adds nullable skill column on a2a_messages) | -| _resolve_slug_from_id | method | roboco/services/a2a.py:1642 | Lookup agent slug from UUID; raise ValueError if missing (gateway send adapter) | -| _get_conversation_for_reply_to_ceo | method | roboco/services/a2a.py:1652 | Wave-2: resolve the conversation for an agent replying to the CEO by direct lookup (bypasses get_or_create_conversation's validate-first gate, which would deny even a legitimate reply) — an existing pair conversation's mere presence proves the CEO opened it, since agents can never create one | -| send | method | roboco/services/a2a.py:1684 | Gateway adapter: resolve both ends to slugs, get_or_create_conversation (or _get_conversation_for_reply_to_ceo when replying to "ceo") + send_chat_message; publishes A2A_MESSAGE_SENT via _publish_a2a_message_sent afterward | -| _publish_a2a_message_sent | staticmethod | roboco/services/a2a.py:1742 | Wave-2: best-effort publish of A2A_MESSAGE_SENT (conversation_id/message_id/task_id/from_agent/to_agent/skill/body_excerpt/timestamp) to the event bus for the operator live view; a bus outage is logged and never rolls back the already-persisted message | -| _maybe_wake_ceo_recipient | method | roboco/services/a2a.py:1983 | CEO-authored send only (`from_slug == "ceo"`); gates on `is_spawnable_agent_slug` + the recipient role carrying `read_a2a` (else an unackable row would be immortal), dedups against an already-pending wake, then calls `send_a2a_notification(..., requires_ack=True)` so the row is visible to the orchestrator's `_dispatch_a2a_work` pending_ack_only poll. Called from `send_chat_message` and `interject_as_ceo`; best-effort, never breaks the send | -| _ack_pending_wake_notifications | method | roboco/services/a2a.py:2059 | Bulk-acknowledges this agent's pending CEO-DM wake notification(s); called from `mark_all_read`/`get_unread_messages` (the gateway's `read_a2a`) once the inbox is actually drained, so the wake row doesn't sit pending forever and permanently block the next dedup check | -| _AuditEvent | dataclass | roboco/services/audit.py:20 | Bundled fields for one audit row write (event_type, agent_id, target, severity, details) | -| _coerce_uuid | function | roboco/services/audit.py:36 | Best-effort coerce str/UUID to UUID; returns None for slugs/invalid | -| AuditService | class | roboco/services/audit.py:48 | SingletonService for audit logging; structured log + best-effort audit_log persistence | -| _persist | method | roboco/services/audit.py:77 | Write an audit row in its own session+commit; never propagate failures (observability must not block) | -| log_task_action_denial | method | roboco/services/audit.py:111 | Log a denied task action; resolves actual actor role from DB at write time over caller-supplied param; preserves non-UUID task_id sentinels (e.g. "N/A") in details["target_id_raw"] rather than silently coercing to NULL target_id | -| log_task_creation_denial | method | roboco/services/audit.py:163 | Log a pre-task-creation denial (no task row exists yet); records attempted payload under target_type="task_creation" with target_id=None — distinct from log_task_action_denial's NULL-target-id so role-escalation attempts are attributable | -| log_task_event | method | roboco/services/audit.py:204 | Log a task-lifecycle event (creation/transition) for TaskService chokepoint | -| log_event | method | roboco/services/audit.py:238 | Free-form generic audit event (e.g. gateway.rejected) for Choreographer forensics | -| log_agent_event | method | roboco/services/audit.py:273 | Log orchestrator agent event (spawned/stopped/stranded); resolves slug→UUID so agent_id is a real FK | -| _resolve_actor_role_from_db | method | roboco/services/audit.py:314 | Read agents.role for actor UUID at write time (DB authoritative over caller-supplied role) | -| _resolve_agent_id_by_slug | method | roboco/services/audit.py:302 | Static AGENT_UUIDS fast-path then DB lookup for slug→UUID; best-effort None on failure | -| has_recent_tracing_gap | method | roboco/services/audit.py:353 | Query audit_log for gateway.rejected/tracing_gap rows since cutoff; backs PM-respawn circuit breaker reset decision | -| get_recent_events | method | roboco/services/audit.py:399 | Fetch recent audit events as dicts (Auditor/CEO queries) with optional type/agent/severity filters | -| get_audit_service | function | roboco/services/audit.py:458 | Lazy singleton accessor for AuditService | -| JournalService | class | roboco/services/journal.py:64 | BaseService for journal/entry CRUD, gateway adapters, RAG indexing, tracing-gate existence checks | -| _get_optimal_service | method | roboco/services/journal.py:78 | Lazy-load OptimalService singleton (avoid circular import) | -| resolve_agent_id | method | roboco/services/journal.py:86 | Resolve UUID-or-slug string to agent UUID via repositories.resolve_agent_uuid | -| get_agent_slug | method | roboco/services/journal.py:102 | Reverse slug lookup via repositories.get_agent_slug | -| get_or_create_journal | method | roboco/services/journal.py:120 | Fetch or create a journal row for an agent (commits on create) | -| create_entry | method | roboco/services/journal.py:215 | Insert entry, bump journal metadata counters, commit; IntegrityError→rollback+None; schedule fire-and-forget RAG index | -| _schedule_rag_index | method | roboco/services/journal.py:324 | asyncio.create_task best-effort RAG index; skips private entries in shared JOURNALS index; strong-ref in _RAG_INDEX_TASKS | -| list_entries | method | roboco/services/journal.py:401 | Filtered/paginated entry listing (excludes private unless include_private) | -| board_review_brief | method | roboco/services/journal.py:455 | PO+HoM DECISION_LOG entries for a task (board handoff for CEO approval/intake redraft) | -| delete_entry | method | roboco/services/journal.py:494 | Delete entry and decrement journal counters (floored at 0) | -| add_task_reflection/add_decision_log/add_learning/add_struggle/add_general_entry | method | roboco/services/journal.py:532 | Convenience builders that get_or_create journal then create_entry via the journal model factories | -| get_growth_metrics | method | roboco/services/journal.py:702 | Compute learning/struggle/decision counts, resolution rate, learning frequency from entries_by_type + content scan | -| search_entries | method | roboco/services/journal.py:757 | Semantic RAG search over an agent's JOURNALS index; re-fetches entries and filters by owning journal | -| _has_entry_of_type | method | roboco/services/journal.py:835 | Existence check: agent has entry of type for task (backs tracing gates) | -| has_decision_for_task/latest_decision_at/has_note_for_task/has_learning_for_task/has_reflect_for_task/has_struggle_for_task | method | roboco/services/journal.py:854 | Per-type tracing-gate existence checks used by the Choreographer | -| has_recent_entry | method | roboco/services/journal.py:912 | Any entry within window (backs auditor i_am_idle session-scoped note obligation) | -| write_struggle/write_decision | method | roboco/services/journal.py:934 | Write-then-gate helpers deriving title from first content line for PM verbs | -| write_entry | method | roboco/services/journal.py:987 | Gateway adapter: scope string→JournalEntryType, get_or_create journal, create_entry | -| get_journal_service | function | roboco/services/journal.py:1027 | Factory: JournalService(db) | -| drain_rag_index_tasks | function | roboco/services/journal.py:51 | Test helper: await all in-flight background RAG index tasks | -| apply_structured_note | function | roboco/services/content_notes.py:57 | Validate payload via foundation ContentModel, store in notes_structured[content_type], regenerate TEXT mirror column; raises before any mutation | -| content_type_for_role | function | roboco/services/content_notes.py:45 | Map agent role to the note section content-type it authors via note(scope='handoff') | -| _MIRROR_COLUMN | module constant | roboco/services/content_notes.py:22 | content_type→derived TEXT mirror column (dev_notes/qa_notes/auditor_notes/doc_notes/pr_reviewer_notes/quick_context) | -| ExtractionService | class | roboco/services/extraction.py:134 | Pattern-based classifier turning raw agent LLM buffers into typed ExtractedMessages | -| extract | method | roboco/services/extraction.py:166 | Segment + classify content; emit ExtractedMessages with confidence + raw_excerpt | -| _segment_content | method | roboco/services/extraction.py:252 | Split on code blocks then double-newlines into paragraph segments | -| _classify_segment | method | roboco/services/extraction.py:280 | Score each MessageType by matched patterns; default REASONING@0.5 if none | -| _call_anthropic_with_retry | method | roboco/services/extraction.py:316 | Anthropic messages.create with MAX_RATE_LIMIT_RETRIES 429 backoff honoring Retry-After | -| extract_with_llm | method | roboco/services/extraction.py:361 | LLM/TOON classification (claude-3-haiku); falls back to pattern extract on any non-RateLimit error | -| ExtractionPipeline | class | roboco/services/extraction.py:459 | Wraps ExtractionService; process_buffer fans results to registered callbacks | -| PermissionService | class | roboco/services/permissions.py:130 | Singleton RBAC enforcement from agents_config: notifications, task actions, KB actions | -| can_send_notifications | method | roboco/services/permissions.py:255 | Whether role may call notify (foundation.NOTIFY_SENDER_ROLES) | -| can_notify | method | roboco/services/permissions.py:259 | Scope rules: all (main_pm/ceo), cell (cell_pm: PMs or same team), board-chain list, else False | -| can_perform_task_action | method | roboco/services/permissions.py:296 | CEO bypasses all; else TASK_PERMISSIONS with VIEW_OWN team restriction + VIEW_ALL fallback | -| can_perform_kb_action | method | roboco/services/permissions.py:333 | KB_PERMISSIONS role action check | -| check_all | method | roboco/services/permissions.py:357 | Comprehensive permission summary dict for an agent context | -| has_privileged_access | function | roboco/services/permissions.py:382 | Async DB check: agent role in PRIVILEGED_ROLES (CEO/Auditor/Main_PM); queries id OR slug | -| is_pm_role | function | roboco/services/permissions.py:410 | Async DB check: agent role in MANAGEMENT_ROLES (CEO/PO/CellPM/MainPM) | -| _get_notification_scope | function | roboco/services/permissions.py:106 | Return scope ('all'/'cell'/role list/[]) for a sender role | -| _get_agents_for_role_team | function | roboco/services/permissions.py:64 | All agent slugs matching a (role, team) pair from the precomputed lookup | +|------|------|-----------|----------------| +| `Base` | class | db/base.py:38 | DeclarativeBase + MetaData naming convention. | +| `get_engine` | fn | db/base.py:46 | Lazy singleton async engine (pool_pre_ping). | +| `get_db` | fn | db/base.py:70 | FastAPI async session dependency. | +| `get_db_context` | fn | db/base.py | Out-of-request async session context. | +| `init_db` | fn | db/base.py:180 | Boot entry: stamp pre-Alembic DB at 001 then `run_migrations`. | +| `run_migrations` | fn | db/base.py:141 | Runs `alembic upgrade head` via `command.upgrade` in a thread. | +| `bootstrap_database` | fn | db/seed.py:282 | `init_db` + seed default agents. | +| `TaskTable` | class | tables.py:157 | Core task entity (largest table, drives lifecycle). | +| `WorkSessionTable` | class | tables.py:798 | Per-claim session; single-active enforced by 047 partial-unique index. | +| `AgentTable` | class | tables.py:95 | Agent identity, role, team, model provider assignment. | +| `ProjectTable` | class | tables.py:475 | Git repo config + CI/watch/dep-update/quality_command/`sandbox_services` (057) cols. | +| `AuditLogTable` | class | tables.py:1940 | Transition journey; `details` JSONB (010); composite query index (045). | +| `AgentSpawnSessionTable` | class | tables.py:2170 | Per-spawn token totals; feeds usage dashboard. | +| `ProjectConventionsCacheTable` | class | tables.py:2442 | Effective conventions map per (project, HEAD sha). | +| `PlaybookTable` | class | tables.py:721 | Curated procedures (draft→approved→indexed). | +| `RespawnTrackerTable` | class | tables.py:1902 | Durable PM-respawn circuit breaker mirror. | +| `TaskCellProjectTable` | class | tables.py:632 | Per-cell project map for a MegaTask root-subtask (052). | +| `WaitingRecordTable` | class | tables.py:1872 | Persisted dispatcher waiting records (restore at start). | +| `IndexedDocumentTable` | class | tables.py:1651 | RAG corpus docs (added to chain by 017). | +| `UserTable` | class | tables.py:2603 | Cloud-auth (FastAPI Users) single seeded CEO login row (058). | +| `XCredentialsTable` | class | tables.py:2650 | Singleton Fernet-encrypted OAuth 1.0a secrets for the X engine (059). | +| `XSeenMentionTable` | class | tables.py:2675 | X mentions-poll dedup ledger, keyed by mention id (059). | +| `XSeenFeatureTable` | class | tables.py:2264 | X feature-spotlight dedup ledger, keyed by feature slug (061). | +| `run_async_migrations` | fn | env.py | Async online migration runner (NullPool). | + +## Migration Chain + +| Num | File | What it adds/changes | +|-----|------|---------------------| +| 001 | 001_initial_schema.py | All initial tables (agents, tasks, work_sessions, notifications, journals, audit_log, a2a_*); also originally created channels/sessions/messages, later dropped by the comms-teardown migration. | +| 002 | 002_persistence_tables.py | Persistence tables + `NotificationType.APPROVAL`. | +| 003 | 003_blocker_resolver_type.py | `tasks.blocker_resolver_type` + `blockerresolvertype` enum. | +| 004 | 004_provider_routing.py | `provider_configs` + `model_assignments`; `modelprovider`/`assignmentscope` enums (create_type=False). | +| 005 | 005_blocker_raised_by.py | `tasks.blocker_raised_by`. | +| 006 | 006_gateway_columns.py | Gateway cols: claimant lock, heartbeat, pre-block snapshot, AC status, qa evidence flag. | +| 007 | 007_gateway_triggers_table.py | `gateway_triggers` (dispatcher decision log). | +| 008 | 008_align_skills.py | No-op (skills alignment done statically). | +| 009 | 009_enum_reconcile.py | Reconcile every postgres enum with ORM StrEnum (lowercase) + new members. | +| 010 | 010_audit_log_details_jsonb.py | `audit_log.details` JSON→JSONB. | +| 011 | 011_drop_quarantined_state.py | Drop `quarantined` from taskstatus enum (phantom state, audit D15). | +| 012 | 012_align_agentrole_foundation.py | Add agentrole/team enum values foundation declares. | +| 013 | 013_drop_role_enum.py | Drop stray `role` enum (smoke run 2). | +| 014 | 014_drop_pm_approvals.py | Drop unused `tasks.pm_approvals`. | +| 015 | 015_drop_task_execution_outputs.py | Drop unused `execution_log`/`outputs`. | +| 016 | 016_add_products_and_task_product_id.py | `products` + `product_projects`; `tasks.product_id` (team enum create_type=False). | +| 017 | 017_reconcile_orm_schema_drift.py | Add ORM tables/columns the chain never had (e.g. `indexed_documents`). | +| 018 | 018_task_project_id_nullable.py | `tasks.project_id` nullable (board/fan-out tasks carry product_id). | +| 019 | 019_seed_default_providers.py | Idempotent seed of default model providers. | +| 020 | 020_backfill_enum_values.py | Backfill ORM enum values the chain never added. | +| 021 | 021_task_board_review_complete.py | `tasks.board_review_complete` (board-review handoff flag). | +| 022 | 022_default_branch_master.py | Flip `projects.default_branch` default `main`→`master`. | +| 023 | 023_prompter_tracking_columns.py | `tasks.source` + `confirmed_by_human` (prompter origin). | +| 024 | 024_add_prompter_tables.py | `prompter_sessions`, `prompter_messages`, `task_drafts`. | +| 025 | 025_agentrole_prompter.py | Add `prompter` to agentrole enum. | +| 026a | 026_completed_dependency_ids.py | `tasks.completed_dependency_ids`. | +| 026b | 026_token_usage_tables.py | `agent_spawn_sessions` + `token_usage_snapshots` (chained off 026a). | +| 027 | 027_system_settings.py | `system_settings` key-value store. | +| 028 | 028_seed_self_hosted_provider.py | Seed Self-Hosted (Ollama LOCAL) provider row. | +| 029 | 029_project_quality_command.py | `projects.quality_command` (fast pre-submit gate). | +| 030 | 030_rag_chunks_content_schema.py | Align RAG chunk tables with vector-store schema. | +| 031 | 031_rag_chunks_fulltext.py | tsvector + GIN index on every chunk table (hybrid retrieval). | +| 032 | 032_company_goals.py | `company_goals` singleton charter. | +| 033 | 033_pitches.py | `pitches` (Board proposals → auto-provision). | +| 034 | 034_agentrole_secretary.py | Add `secretary` to agentrole enum. | +| 035 | 035_secretary_directives.py | `secretary_directives` (command audit + gate queue). | +| 036 | 036_ac_ids_and_parent_refs.py | Per-criterion AC ids + child→parent AC linkage. | +| 037 | 037_agentrole_pr_reviewer.py | Add `pr_reviewer` to agentrole enum. | +| 038 | 038_modelprovider_grok.py | Add `grok` to modelprovider enum. | +| 039 | 039_seed_grok_provider.py | Seed Grok (xAI) provider row. | +| 040 | 040_awaiting_pr_review.py | Add `awaiting_pr_review` to taskstatus enum (PR-review gate). | +| 041 | 041_structured_content_columns.py | `pr_reviewer_notes`, machine-marker split, structured content cols. | +| 042 | 042_worksession_toolchain.py | `work_sessions` toolchain matching cols. | +| 043 | 043_conventions_cache.py | `project_conventions_cache`. | +| 044 | 044_convention_findings.py | `project_convention_findings` (violations feed). | +| 045 | 045_observability_rework.py | `tasks.revision_count` + audit_log composite query index. | +| 046 | 046_batch_intake.py | `tasks.batch_id` + collision descriptors (intends_to_touch, adds_migration, touches_shared). | +| 047 | 047_ws_single_active.py | Partial-unique index: one ACTIVE work_session per task. | +| 048 | 048_ci_watch_project_cols.py | Per-project CI-watch opt-in cols. | +| 049 | 049_dep_update_project_cols.py | Per-project dep-update bot opt-in cols. | +| 050 | 050_playbooks.py | `playbooks` table (curated procedures). | +| 051 | 051_respawn_tracker.py | `respawn_tracker` (durable PM-respawn counter). | +| 052 | 052_task_cell_projects.py | `task_cell_projects` (per-cell project map for MegaTask root-subtask; reuses team enum create_type=False). | +| 053 | 053_playbook_archived_attr.py | `playbooks.archived_by` (UUID) + `playbooks.archived_at` (DateTime) — distinct retirement attribution; keeps `approved_by`/`approved_at` as approval-only provenance. | +| 054 | 054_a2a_message_skill.py | `a2a_messages.skill` (String 100, nullable) — persists the capability a directed A2A message concerns; was silently dropped on send. | +| 055 | 055_spawn_session_turns_tool_calls.py | `agent_spawn_sessions.turns` + `.tool_calls` (BigInteger, DEFAULT 0) — per-stint LLM iterations + tool invocations for the granular per-member performance metrics. | +| 056 | 056_member_perf_daily.py | `member_performance_daily` — one row per (date, member_kind, agent_slug) scorecard rollup (incl. CEO as `member_kind='ceo'`). | +| 057 | 057_project_sandbox_services.py | `projects.sandbox_services` (ARRAY(String), nullable) — per-project opt-in for the sandboxed per-agent-spawn engine provisioner (postgres / redis / mongo via the `SANDBOX_ENGINES` registry). | +| 058 | 058_cloud_auth_users.py | `users` table (FastAPI Users schema) — the single seeded CEO login for cloud auth (`ROBOCO_CLOUD_AUTH_ENABLED`, default off). | +| 059 | 059_x_credentials.py | `x_credentials` (singleton Fernet-encrypted OAuth 1.0a secrets) + `x_seen_mentions` (mentions-poll dedup ledger) — the X (Twitter) engine (`ROBOCO_X_ENGINE_ENABLED`, default off). | +| 060 | 060_drop_messaging.py | Drops the channels/groups/sessions/session_tasks/messages subsystem (comms teardown — A2A is now the sole directed-message channel): `journal_entries.session_id` column, the 5 tables, and 4 enum types (`messagetype`/`sessionstatus`/`sessionscope`/`channeltype`); one-way (`downgrade()` raises `NotImplementedError`). | +| 061 | 061_x_feature_spotlight.py | `x_seen_features` (feature-spotlight dedup ledger, keyed by feature slug) + `company_goals.brand_voice` (Text, CEO-authored brand-voice sample, feeds `_voice_guide`) — X feature-spotlight (`ROBOCO_X_FEATURE_SPOTLIGHT_ENABLED`, default off, sub-switch of `x_engine_enabled`). | +| 062-072 | *(not yet reflected in this table — pre-existing gap, out of scope for this pass)* | Vault V1/V2, revision-findings ledger, sandbox extensions, and other slices landed migrations in this range; see `alembic/versions/` directly until this table is backfilled. | +| 073 | 073_project_environments.py | `projects.environments` (nullable JSONB) — the per-project ordered environment ladder (`list[{name, branch}]`, index 0 = head rung, index -1 = prod rung) that replaces `default_branch` as the source of truth for a project's PR target and release target. Additive: a null value falls back to a degenerate single-branch ladder synthesized from `default_branch` at read time (`roboco/models/env_branches.py`), so existing projects are unaffected until the CEO declares a real ladder. | +| 074 | 074_telegram_credentials.py | `telegram_credentials` (singleton Fernet-encrypted `bot_token_encrypted` + `chat_id_encrypted`, mirrors `x_credentials`) — the Telegram notifications bridge (`ROBOCO_TELEGRAM_ENABLED`, default off). | +| 075 | 075_company_goals_company_name.py | `company_goals.company_name` (Text, `server_default=""`) — CEO-authored product/company name, mirroring `brand_voice`. Feeds `CompanyGoalsService.resolve_product_name` (project name → this field → the "RoboCo" literal fallback), which `XEngine`/`VideoEngine` both call so release posts/videos stop hardcoding "RoboCo". Additive and inert until the CEO sets it in the Business → Goals editor. | +| 076 | 076_project_git_provider.py | `projects.git_provider` (nullable `String(16)`, not a pg enum — validated at the service layer by `roboco.foundation.policy.forge.validate_project_forge`) — Phase 0 of the forge-providers spec (GitHub + Gitea + GitLab). Null = auto-detect from the `git_url` host (github.com → github; anything else is a registration-time rejection unless the operator sets this column explicitly — the GitHub Enterprise / self-hosted escape hatch). Additive: every existing project keeps resolving to GitHub behavior until GitLab/Gitea providers are set. See `docs/map/worksession-git.md`. | ## Data Flow -CONTROL FLOW: (1) A2A — HTTP routes in roboco/api/routes/a2a.py construct A2AService(db) per request for card discovery, task get/list/cancel, conversation CRUD, message send, mark-read, inbox/pairs; the gateway Choreographer/content_actions use A2AService.send (UUID→slug resolved) for directed agent messaging. Legacy A2A-protocol path: create_a2a_notification requires a task_id, requires both from_agent and target_agent to be present/resolvable (raises distinct ValueError if either missing), enforces hierarchy unconditionally via validate_a2a_access (raises A2AAccessDeniedError), parses priority via foundation.policy.communications.parse_priority, then delegates to NotificationService.send_a2a_notification (which now runs the loop-prone 60s Redis re-fire guard). Bidirectional responses: update_task_from_message appends to dev_notes and, if dev_notes contains the 'A2A Request' marker, _notify_original_requester publishes a TASK_ASSIGNED event to the StreamEventBus to spawn the offline requester. CEO live view (wave 2/2c): every send() publishes A2A_MESSAGE_SENT (_publish_a2a_message_sent) which websocket_bridge forwards to /ws/system as an a2a.message frame for the panel's /a2a switchboard; the CEO-only /chat/admin/* routes (_require_ceo) read across all conversations via get_conversation_admin/list_conversations_admin/get_messages_admin/list_admin_pairs (no participant check) and reply_as_ceo chimes in via the normal send() path, which routes a reply-to-CEO through _get_conversation_for_reply_to_ceo and gates every non-CEO send to the CEO through _enforce_ceo_reply_budget. CEO-DM wake: send_chat_message and interject_as_ceo both call _maybe_wake_ceo_recipient after persisting, which — for a CEO-authored send to a read_a2a-capable recipient only — creates an a2a_request NotificationTable row with requires_ack=True (a per-row override on CreateNotificationParams/send_a2a_notification, since A2A_REQUEST's type default is requires_ack=False) so the orchestrator's _dispatch_a2a_work pending_ack_only poll can see and spawn the offline recipient; the pending-row lookup doubles as dedup, and _ack_pending_wake_notifications closes it out once the recipient actually reads via read_a2a. Agent-to-agent DM never wakes — pull-only by design. (2) Audit — TaskService (log_task_event at every transition chokepoint), task routes (log_task_action_denial on 403 for action denials; log_task_creation_denial on 403 for pre-task create denials — distinct target_type="task_creation" with no task_id), the orchestrator (log_agent_event on spawn/stop + has_recent_tracing_gap for the PM-respawn circuit breaker) and the Choreographer (log_event for gateway.rejected) all call get_audit_service(); _persist opens its own session+commit so audit writes never roll back the caller's transaction. (3) Journal — gateway content_actions.note → JournalService.write_entry (scope string→type via foundation SCOPE_TO_TYPE), and write_struggle/write_decision for PM write-then-gate verbs; create_entry commits the row then _schedule_rag_index fires asyncio.create_task (strong-ref in _RAG_INDEX_TASKS) that calls OptimalService.index_journal_entry (skipped for is_private) and record_learning for LEARNING entries; tracing-gate existence checks (has_decision_for_task/latest_decision_at/has_note_for_task/...) feed the Choreographer's gate decisions. (4) content_notes — TaskService._set_structured_note and gateway content_actions handoff path call apply_structured_note(task, content_type, payload); it validates via foundation.policy.content.validate_content BEFORE mutating, reassigns notes_structured (to flag the JSON column dirty), and writes render_markdown() into the derived TEXT mirror column. (5) Extraction — app lifespan builds ExtractionPipeline(ExtractionService()); stream route process_buffer → ExtractionService.extract (regex classify) → callbacks store/broadcast messages; extract_with_llm is the optional Anthropic/TOON path. (6) Permissions — notification/task/KB routes and gateway kb_authz call PermissionService methods synchronously (no DB) from an AgentContext; has_privileged_access/is_pm_role are async DB lookups used by route deps. DATA: inputs are AsyncSession + UUIDs/slugs/payloads; outputs are Pydantic models (A2ATask, A2AConversation, Journal, JournalEntry), audit rows, structured note columns, ExtractedMessage lists, and bool permission decisions. +On boot, `init_db()` probes for application tables and `alembic_version`; if a pre-Alembic DB exists it stamps it at revision 001, then always runs `run_migrations()` → `alembic upgrade head` (in a thread via `asyncio.to_thread`). `env.py` imports `roboco.db.tables` so `Base.metadata` is fully populated, overrides `sqlalchemy.url` from `settings.database_url`, and runs online with an async NullPool engine. `compare_type` + `compare_server_default` are on so autogenerate drift is detectable. `tables.py` classes are the ORM mapping the migrations build; the domain layer reads them through `roboco/models/` dataclasses, not the tables directly. ## Mermaid + ```mermaid -graph TD - subgraph Gateway - CH[Choreographer] -->|send UUID->slug| A2A - CA[content_actions] -->|note scope| JRN - CA -->|handoff| CN - CH -->|log_event gateway.rejected| AUD - CH -->|tracing gates: has_*_for_task| JRN - end - subgraph HTTP - RA[routes/a2a] --> A2A - RT[routes/tasks] --> AUD - RJ[routes/journals] --> JRN - RS[routes/stream] --> EX - end - subgraph Orchestrator - ORC -->|log_agent_event| AUD - ORC -->|has_recent_tracing_gap| AUD - ORC -->|board_review_brief| JRN - end - subgraph Services - A2A[A2AService] -->|create_a2a_notification| NOT[NotificationService] - A2A -->|cancel_task| TS[TaskService] - A2A -->|response event| BUS[StreamEventBus] - JRN[JournalService] -->|fire-forget| OPT[OptimalService RAG] - TS -->|apply_structured_note| CN[content_notes] - CN -->|validate_content| FC[foundation.policy.content] - AUD[AuditService] -->|own session commit| DB[(audit_log)] - JRN -->|commit| DB2[(journals/journal_entries)] - A2A -->|flush/commit| DB3[(a2a_conversations/a2a_messages)] - PERM[PermissionService] -->|can_notify| FND[foundation.NOTIFY_SENDER_ROLES] - end - EX[ExtractionService] -->|messages| CB[stream callbacks] - NOT -->|dedup + re-fire guard| DB4[(notifications)] +graph LR + 001-->002-->003-->004-->005-->006-->007-->008-->009-->010 + 010-->011-->012-->013-->014-->015-->016-->017-->018-->019 + 019-->020-->021-->022-->023-->024-->025-->026a-->026b-->027 + 027-->028-->029-->030-->031-->032-->033-->034-->035-->036 + 036-->037-->038-->039-->040-->041-->042-->043-->044-->045 + 045-->046-->047-->048-->049-->050-->051-->052-->053-->054 + 054-->055-->056-->057-->058-->059-->060-->061 ``` ## Logical Tree + ``` -a2a-audit-journal-permissions -├── A2AService (a2a.py) -│ ├── Agent Card builders: build_system_agent_card, build_agent_card, _agent_to_card, discover_agents -│ ├── Task↔A2A conversion: task_to_a2a, get_task, list_tasks, cancel_task, _status_value_of, _apply_cancel_note -│ ├── Legacy A2A-protocol path: extract_message_text, update_task_with_message, create_a2a_notification, update_task_from_message, _notify_original_requester, _publish_a2a_response_event, resolve_creator_agent, resolve_target_agent -│ ├── Persistent conversations: get_or_create_conversation, get_conversation, list_conversations, close_conversation, _canonical_pair -│ ├── Chat messages: send_chat_message (dedup, _enforce_ceo_reply_budget), get_messages, mark_read, mark_all_read, get_inbox_summary, list_pairs -│ ├── CEO admin/live-view (wave 2/2c): get_conversation_admin, list_conversations_admin, list_admin_pairs, get_messages_admin, _get_conversation_for_reply_to_ceo, _publish_a2a_message_sent -│ ├── CEO-DM wake (wave 3): _maybe_wake_ceo_recipient (send_chat_message + interject_as_ceo), _ack_pending_wake_notifications (mark_all_read + get_unread_messages) -│ ├── Conversions: _conv_to_model, _msg_to_model -│ └── Gateway adapter: send, _resolve_slug_from_id, get_team_from_agent -├── AuditService (audit.py) -│ ├── Persistence: _persist (own session), _coerce_uuid -│ ├── Writers: log_task_action_denial, log_task_creation_denial, log_task_event, log_event, log_agent_event -│ ├── Resolvers: _resolve_actor_role_from_db, _resolve_agent_id_by_slug -│ ├── Queries: has_recent_tracing_gap, get_recent_events -│ └── Singleton: _AuditServiceHolder, get_audit_service -├── JournalService (journal.py) -│ ├── Journal CRUD: get_or_create_journal, get_journal, get_journal_by_agent -│ ├── Entry CRUD: create_entry, get_entry, list_entries, delete_entry -│ ├── RAG indexing: _schedule_rag_index, _RAG_INDEX_TASKS, drain_rag_index_tasks -│ ├── Convenience builders: add_task_reflection, add_decision_log, add_learning, add_struggle, add_general_entry -│ ├── Analytics: get_journal_stats, get_growth_metrics, search_entries -│ ├── Tracing-gate checks: _has_entry_of_type, has_decision_for_task, latest_decision_at, has_note_for_task, has_learning_for_task, has_reflect_for_task, has_struggle_for_task, has_recent_entry -│ ├── Write-then-gate: write_struggle, write_decision -│ ├── Gateway adapter: write_entry (scope→type), _SCOPE_TO_TYPE -│ └── Board: board_review_brief -├── content_notes (content_notes.py) -│ ├── apply_structured_note (validate→persist→mirror) -│ ├── content_type_for_role -│ └── _MIRROR_COLUMN / _ROLE_TO_CONTENT_TYPE maps -├── ExtractionService / ExtractionPipeline (extraction.py) -│ ├── Pattern lists: REASONING/DIALOGUE/DECISION/ACTION/BLOCKER/TECHNICAL -│ ├── extract / _segment_content / _classify_segment / _compile_patterns -│ ├── LLM path: extract_with_llm, _call_anthropic_with_retry (TOON) -│ └── ExtractionPipeline.process_buffer + on_message callbacks -└── PermissionService + helpers (permissions.py) - ├── Notification: can_send_notifications, can_notify, _can_role_send_notifications, _get_notification_scope, _BOARD_NOTIFY_TARGETS - ├── Task/KB: can_perform_task_action, get_task_actions, can_perform_kb_action, get_kb_actions - ├── Utility: get_permission_level, check_all - └── Async DB: has_privileged_access, is_pm_role, PRIVILEGED_ROLES, MANAGEMENT_ROLES +Migration chain 001..059 +├── Initial schema +│ └── 001 initial schema (agents, tasks, work_sessions, notifications, journals, audit_log, a2a_*; also originally channels/sessions/messages, later dropped by the comms-teardown migration) +├── Persistence +│ └── 002 persistence tables + NotificationType.APPROVAL +├── Blocker metadata +│ ├── 003 blocker_resolver_type + blockerresolvertype enum +│ └── 005 blocker_raised_by +├── Provider routing & model assignments +│ ├── 004 provider_configs + model_assignments (modelprovider/assignmentscope enums) +│ ├── 019 seed default model providers +│ ├── 028 seed Self-Hosted (Ollama LOCAL) provider +│ ├── 038 add grok to modelprovider enum +│ └── 039 seed Grok (xAI) provider +├── Gateway +│ ├── 006 gateway columns (claimant lock, heartbeat, pre-block snapshot, AC status, qa evidence) +│ └── 007 gateway_triggers (dispatcher decision log) +├── Enum reconcile / widening +│ ├── 009 reconcile postgres enums with ORM StrEnum +│ ├── 011 drop quarantined from taskstatus enum +│ ├── 012 align agentrole/team enums with foundation +│ ├── 013 drop stray role enum +│ ├── 020 backfill ORM enum values +│ ├── 025 add prompter to agentrole enum +│ ├── 034 add secretary to agentrole enum +│ └── 037 add pr_reviewer to agentrole enum +├── Audit log +│ ├── 010 audit_log.details JSON→JSONB +│ └── 045 tasks.revision_count + audit_log composite query index +├── Cleanup / drops +│ ├── 014 drop unused tasks.pm_approvals +│ ├── 015 drop unused execution_log/outputs +│ └── 008 no-op (skills alignment done statically) +├── Products +│ ├── 016 products + product_projects; tasks.product_id (team enum create_type=False) +│ └── 018 tasks.project_id nullable +├── ORM drift reconcile +│ └── 017 add ORM tables/columns the chain never had (indexed_documents) +├── Board review +│ └── 021 tasks.board_review_complete +├── Project defaults +│ └── 022 flip projects.default_branch default main→master +├── Prompter tracking +│ ├── 023 tasks.source + confirmed_by_human +│ └── 024 prompter_sessions, prompter_messages, task_drafts +├── Dependency / token usage +│ ├── 026a tasks.completed_dependency_ids +│ └── 026b agent_spawn_sessions + token_usage_snapshots (chained off 026a) +├── System settings +│ └── 027 system_settings key-value store +├── Project quality +│ └── 029 projects.quality_command (fast pre-submit gate) +├── RAG +│ ├── 030 align RAG chunk tables with vector-store schema +│ └── 031 tsvector + GIN index on chunk tables (hybrid retrieval) +├── Strategy / provisioning +│ ├── 032 company_goals singleton charter +│ └── 033 pitches (Board proposals → auto-provision) +├── Secretary +│ └── 035 secretary_directives (command audit + gate queue) +├── Acceptance criteria +│ └── 036 per-criterion AC ids + child→parent AC linkage +├── PR review +│ ├── 040 add awaiting_pr_review to taskstatus enum +│ └── 041 pr_reviewer_notes, machine-marker split, structured content cols +├── Worksession toolchain +│ └── 042 work_sessions toolchain matching cols +├── Conventions standard +│ ├── 043 project_conventions_cache +│ └── 044 project_convention_findings (violations feed) +├── MegaTask / batch intake +│ ├── 046 tasks.batch_id + collision descriptors +│ └── 052 task_cell_projects (per-cell project map for MegaTask root-subtask) +├── WorkSession single-active +│ └── 047 partial-unique index: one ACTIVE work_session per task +├── Autonomous maintenance +│ ├── 048 per-project CI-watch opt-in cols +│ └── 049 per-project dep-update bot opt-in cols +├── Organizational memory +│ ├── 050 playbooks table (curated procedures) +│ └── 053 playbooks.archived_by + archived_at (distinct retirement attribution from approval) +├── Orchestrator runtime durability +│ └── 051 respawn_tracker (durable PM-respawn counter) +├── A2A messaging +│ └── 054 a2a_messages.skill (nullable; persists directed-A2A capability context) +├── Per-member performance metrics +│ ├── 055 agent_spawn_sessions.turns + .tool_calls (DEFAULT 0) +│ └── 056 member_performance_daily (per date/member_kind/agent_slug rollup) +├── Sandboxed dev DB/Redis +│ └── 057 projects.sandbox_services (per-project opt-in array) +├── Cloud auth +│ └── 058 users (FastAPI Users; single seeded CEO login) +├── X (Twitter) engine +│ ├── 059 x_credentials (singleton encrypted OAuth 1.0a) + x_seen_mentions (dedup ledger) +│ └── 061 x_seen_features (feature-spotlight dedup ledger) + company_goals.brand_voice +└── Comms teardown + └── 060 drop channels/groups/sessions/session_tasks/messages + journal_entries.session_id (A2A is now the sole directed-message channel; one-way, no downgrade) ``` ## Dependencies -- Internal: roboco.agents_config (ALL_AGENTS, get_agent_skills, get_agent_team), roboco.config.settings (host, port, app_version, anthropic_api_key, pm_decision_window_seconds), roboco.db.tables (A2AConversationTable, A2AMessageTable, AgentTable, TaskTable, JournalTable, JournalEntryTable, AuditLogTable), roboco.db.base.get_session_factory, roboco.enforcement.validate_a2a_access, roboco.events (Event, EventType, get_event_bus), roboco.foundation.policy.communications (parse_priority, NOTIFY_SENDER_ROLES, ACK_REQUIRED_BY_TYPE), roboco.foundation.policy.content (ContentModel, validate_content), roboco.foundation.policy.journaling (SCOPE_TO_TYPE), roboco.foundation.identity (Role, PM_ROLES, is_spawnable_agent_slug), roboco.agents_config.get_agent_role, roboco.services.gateway.role_config.get_role_config (local import — cycles back into this module at module scope), roboco.services.notification_delivery.get_notification_delivery_service, roboco.models (NotificationPriority, NotificationType), roboco.models.a2a, models.audit, models.base, models.journal, models.message, models.extraction, models.optimal, models.permissions, roboco.seeds.initial_data.AGENT_UUIDS, roboco.services.base (SingletonService, BaseService), roboco.services.task.TaskService, roboco.services.notification.NotificationService, roboco.services.optimal.OptimalService / get_optimal_service, roboco.services.repositories (resolve_agent_uuid, get_agent_slug), roboco.services.exceptions (RateLimitError, MAX_RATE_LIMIT_RETRIES), roboco.llm.ToonAdapter, roboco.utils.converters (require_uuid, to_python_uuid) -- External: sqlalchemy (select, update, or_, and_, func, AsyncSession), structlog, anthropic (AsyncAnthropic, RateLimitError), asyncio, ipaddress, re, uuid, dataclasses, datetime +- PostgreSQL 15+ (NULLS NOT DISTINCT) — actually pgvector image on PG 16. +- `pgvector` extension for RAG cosine similarity (`chunks_*` tables, `indexed_documents`). +- `asyncpg` driver; SQLAlchemy 2.0 async. +- Alembic; migrations run on every orchestrator boot. ## Entry Points +- `init_db()` / `run_migrations()` in `roboco/db/base.py` — boot-time `alembic upgrade head`. +- `bootstrap_database()` in `roboco/db/seed.py` — init + seed. +- `alembic upgrade head` (manual, in orchestrator container). +- `conftest` (tests) — ephemeral DB per test; runs migrations or `create_all` depending on PG availability. -| Name | File | Trigger | -|---|---|---| -| HTTP routes /api/a2a/* | roboco/api/routes/a2a.py | Agent card, task, conversation, message, inbox REST endpoints construct A2AService(db) per request | -| HTTP routes /api/a2a/chat/admin/* (CEO-only) | roboco/api/routes/a2a.py | Wave-2/2c: org-wide live view — list_admin_conversations, list_admin_pairs (switchboard), list_admin_chat_messages, reply_as_ceo — all gated by _require_ceo, delegating to A2AService.list_conversations_admin / list_admin_pairs / get_messages_admin / send | -| Gateway content_actions.note / handoff | roboco/services/gateway/content_actions.py | Agent note/dm verbs → JournalService.write_entry + content_type_for_role + apply_structured_note | -| Choreographer send | roboco/services/gateway/choreographer/ | Directed A2A verb → A2AService.send (UUID→slug) | -| Choreographer tracing gates | roboco/services/gateway/choreographer/ | i_will_work_on / delegate / submit gates query JournalService.has_*_for_task and latest_decision_at | -| TaskService transition chokepoint | roboco/services/task.py | Every status transition → AuditService.log_task_event + log_task_action_denial; structured note writes → apply_structured_note | -| Orchestrator spawn/stop + respawn breaker | roboco/runtime/orchestrator.py | Agent spawned/stopped/stranded → AuditService.log_agent_event; PM-respawn strike decision → has_recent_tracing_gap; board review → board_review_brief | -| HTTP routes /api/journals, /api/tasks board-review | roboco/api/routes/journals.py | Journal/entry/list/search/board-review-brief REST via get_journal_service(db) | -| HTTP route /api/stream process_buffer | roboco/api/routes/stream.py | Transcription buffer ready → ExtractionPipeline.process_buffer | -| FastAPI lifespan | roboco/api/app.py | App startup constructs ExtractionPipeline(ExtractionService()) on _AppServices | -| Route deps + notification/KB routes | roboco/api/deps.py | Per-request service bag wires A2A/Journal/Audit/Permission; routes call PermissionService + has_privileged_access/is_pm_role | -| CLI / direct module use | roboco/services/extraction.py | extract_with_llm runnable standalone (Anthropic key) | +## Config Flags +- `ROBOCO_DATABASE_*` (host/port/user/password/name) — `settings.database_url`. +- `ROBOCO_DATABASE_ECHO`, pool size/timeout/recycle. +- No DB-specific feature flag; migrations always run. Feature flags (`ROBOCO_CONVENTIONS_ENABLED`, `ROBOCO_CI_WATCH_ENABLED`, `ROBOCO_DEP_UPDATE_ENABLED`, `ROBOCO_RELEASE_MANAGER_ENABLED`, `ROBOCO_ORG_MEMORY_ENABLED`) gate *use* of tables the migrations already added. ## Gotchas -- a2a.send_chat_message dedup: suppresses an identical (conversation, sender, kind, content) message while a prior one is still unread — protects against respawn re-emits, but a genuinely repeated urgent message is also collapsed until the recipient reads. Keyed on content equality, so rewording defeats it (intended). -- a2a.get_or_create_conversation canonical ordering (_canonical_pair, lexically smaller first) is the uniqueness key; a non-canonical pair lookup will miss the existing row and create a duplicate. validate_a2a_access is enforced BEFORE the canonical swap. -- a2a.create_a2a_notification requires a task_id (raises ValueError without one), requires from_agent present (raises ValueError "requires a 'from_agent' in metadata") and target_agent resolvable (raises ValueError "could not resolve a target agent"), then enforces hierarchy unconditionally via validate_a2a_access (raises A2AAccessDeniedError + route_hint — no longer a bare ValueError indistinguishable from the missing-field errors); it does NOT create a conversation row — it goes through NotificationService, which now runs the 60s Redis loop-prone re-fire guard (all_recipients_recently_notified) that can silently drop a legitimate A2A notification. -- a2a._notify_original_requester only fires when task.dev_notes contains the literal 'A2A Request' marker; the marker is written by the legacy A2A-protocol path (update_task_with_message), NOT by the gateway conversation path, so gateway A2A messages never trigger requester re-spawn via this path. -- a2a.list_conversations runs an N+1 query (last-message preview per conversation); fine at low volume but unbounded by the 50-row limit can cost on heavy agents. -- a2a.mark_read zeroes the conv's per-side counter and bulk-updates read_at on inbound unread messages in the SAME session; if the caller never commits, the read state is lost. -- a2a._enforce_ceo_reply_budget is the only stateful check in an otherwise-stateless access model (can_a2a_direct blocks conversation *creation* unconditionally, not individual sends); it counts messages per conversation on every send, so a very long-running CEO thread pays an extra COUNT query pair per message. -- a2a._get_conversation_for_reply_to_ceo treats conversation existence itself as proof of CEO authorization (agents can never create a CEO conversation) — if that invariant is ever broken elsewhere (e.g. a future seed/migration inserting one directly), an agent could reply into a CEO thread it was never actually invited to. -- The CEO admin/live-view routes (get_conversation_admin, list_conversations_admin, get_messages_admin) intentionally skip the participant check that every non-admin read enforces; they are safe only because the routes themselves are behind _require_ceo — a missing or misapplied _require_ceo on any new admin route would expose every agent's A2A transcript. -- a2a._maybe_wake_ceo_recipient only fires for `from_slug == "ceo"` — agent-to-agent `dm` never wakes an offline recipient, deliberately, so ordinary same-cell chatter can't burn spawns. It also skips a recipient whose role manifest lacks `read_a2a` (auditor, pr_reviewer, prompter, secretary): a wake row that role could never ack would sit pending forever and permanently suppress the dedup pre-check for that recipient going forward. -- a2a._maybe_wake_ceo_recipient's dedup is a pending-notification lookup (`pending_ack_only=True`, `type_filter=A2A_REQUEST`), not a separate dedup table — it relies on `_ack_pending_wake_notifications` actually clearing the row once the recipient reads (`read_a2a`/`get_unread_messages`/`mark_all_read`). A recipient that never reads keeps the wake row pending forever, so a second CEO message to them creates no new wake notification (silently, by design) but also never re-spawns them via this path a second time. -- Before wave 3, `_dispatch_a2a_work`'s `pending_ack_only=True` poll (see docs/map/orchestrator.md) was structurally unable to see ANY a2a_request row, CEO or not, because `A2A_REQUEST`'s `ACK_REQUIRED_BY_TYPE` default is `requires_ack=False`. `send_a2a_notification` gained a `requires_ack` kwarg and `CreateNotificationParams` a per-row `requires_ack` override (`roboco/models/notification.py`) so `_maybe_wake_ceo_recipient` alone can opt its row in; every other `send_a2a_notification` caller (including the legacy `create_a2a_notification` path) still defaults to `requires_ack=False` and remains invisible to that poll. -- audit._persist opens its OWN session and commits independently — audit writes survive caller rollback (good) but mean audit rows can exist for operations that were later rolled back (forensic skew). Failures are logged, never raised. -- audit.log_task_action_denial resolves the actor's role from agents.role at write time, overriding the caller-supplied agent_role param (DB authoritative) — a stale caller param is silently replaced, which can surprise tests asserting the supplied role. -- audit.has_recent_tracing_gap filters details->>'reason' == 'tracing_gap' via JSONB; any row whose details JSON lacks that key or uses a different reason string is invisible to the circuit breaker (it will fall back to strike counting). -- journal.get_or_create_journal COMMITS on create (not flush) — calling it inside an outer unit-of-work will prematurely commit the outer transaction's pending state. -- journal.create_entry commits the entry then schedules RAG indexing fire-and-forget; on IntegrityError it rolls back and returns None (callers must handle None, not raise). The RAG index task holds a strong ref in _RAG_INDEX_TASKS; a RuntimeError when no event loop is running silently skips indexing. -- journal._schedule_rag_index SKIPS index_journal_entry for is_private entries (shared JOURNALS index would leak private reflections), but STILL records a private LEARNING via record_learning with shareable=False — two different sinks with two different privacy rules. -- journal.latest_decision_at backs the pm_decision_window_seconds windowed gate; the window is read by the Choreographer from settings, not enforced here — drift between this query and the choreographer's cutoff can admit or reject a decision based on clock skew. -- content_notes.apply_structured_note raises ContentValidationError BEFORE any mutation (no partial write), but it reassigns task.notes_structured = structured (a new dict) to flag the JSONB column dirty — in-place mutation of the existing dict would NOT mark it dirty and the write would be lost on commit. -- content_notes._MIRROR_COLUMN maps only 6 content types; a content type absent from the map is stored structured-only with NO TEXT mirror, so any legacy reader of the TEXT column sees stale/empty for that type. -- extraction._classify_segment defaults to REASONING@0.5 when no pattern matches — every unclassified utterance becomes 'reasoning', inflating reasoning counts. -- extraction.extract_with_llm hardcodes model='claude-3-haiku-20240307' (stale model id) and only falls back to pattern extract on non-RateLimit errors; a persistent 429 re-raises RateLimitError after MAX_RATE_LIMIT_RETRIES. -- permissions.can_perform_task_action CEO-bypasses ALL actions including team-scoped VIEW_OWN — the CEO operates across teams by design (panel), but any route gating through this helper lets the CEO through. -- permissions.has_privileged_access / is_pm_role query by (id == agent_id) OR (slug == str(agent_id)) because the CEO uses a UUID-style slug; passing a non-UUID slug that happens to collide with a slug column value can match the wrong row. - +- **`sa.Enum(create_type=False)` is silently ignored** — the flag only works on `postgresql.ENUM`, not generic `sa.Enum`. Using `sa.Enum` re-emits CREATE TYPE and fails with "type already exists". 001/016/052 carry the live gotcha comment; 004/016 use the correct `postgresql.ENUM(create_type=False)`. 001 itself uses `sa.Enum(..., create_type=False)` in spots — latent on a clean re-apply. +- **Enum-parity gate can false-green** — `make quality` runs `scripts/verify_postgres_enums.py` only against a migrated DB; an empty `roboco` DB (conftest ephemeral) or `|| echo` masking hides drift. Fixed in 957fb522 but the gate is only as good as the DB it points at. +- **016 latent** — the `team` enum member list under create_type=False is the *original* set, not the later-widened set; inert but misleading. +- **Two files numbered 026** — not a collision: `026_token_usage_tables` chains off `026_completed_dependency_ids`. Renaming is risky (breaks down_revision refs). +- **052 reuses the `team` enum** with `create_type=False` correctly — no new enum added; safe. +- **017 reconciled drift the other way** — added ORM tables the chain had missed; `create_all` is no longer authoritative. ## Drift from CLAUDE.md -- CLAUDE.md Services table lists 'PermissionsService' (plural) as the RBAC service; the actual class is 'PermissionService' (singular) at roboco/services/permissions.py:130. The re-export PM_ROLES and the async helpers has_privileged_access/is_pm_role are not mentioned in CLAUDE.md. -- CLAUDE.md's notification table says notifications are 'sent by PMs/Board only'; permissions.py matches this (_can_role_send_notifications excludes AUDITOR) — no behavioral drift. -- CLAUDE.md describes the A2A/conversation surface only via the gateway Choreographer; it does not document the legacy A2A-protocol path (create_a2a_notification / update_task_with_message / dev_notes 'A2A Request' marker / TASK_ASSIGNED re-spawn) which still lives in A2AService. -- CLAUDE.md says journal 'note' write returns immediately and RAG indexing is fire-and-forget — journal.py:324 _schedule_rag_index matches this exactly (no drift). CLAUDE.md also says 'journal indexing excludes is_private reflections from the shared corpus' — journal.py:343 matches (skips index_journal_entry when is_private). No drift. -- CLAUDE.md does not mention AuditService.has_recent_tracing_gap (the PM-respawn circuit-breaker query) or that audit._persist uses its own session/commit — both are undocumented audit behaviors. +- CLAUDE.md says "52 migrations 001..052" — now stale; chain is 001..059 (59 files). Does not mention the two 026 files (chained, not a conflict). +- CLAUDE.md cites migrations 043/046/047/048/049/050/051 by number in feature sections — all present and consistent. +- No factual drift found in the DB layer description. +## Changes Since Baseline +`git log fd10cc862c2020b3f639cdb686d427b0198a2441..HEAD -- alembic/ roboco/db/`: +- `15effce0` Chore: 141 Gaps fill-in (#283) — adds migration 052 (`task_cell_projects`) + `TaskCellProjectTable`; logic-touching. + +(Only one commit in range touches these paths.) + +> Post-snapshot updates (since 2026-06-29): `536bbb64` (Chore/all/logical gaps sweep #286) — adds migration 053 (`playbooks.archived_by`/`archived_at`), two new columns on `PlaybookTable`; `d8a5bb48` ([chore] a2a hierarchy gate + skill persist) — adds migration 054 (`a2a_messages.skill`), one new column on `A2AMessageTable`, wired through `send_chat_message` and the A2AChatMessage model. +> +> Delta 2026-07-03 (v0.17.0, 5 features): `055_spawn_session_turns_tool_calls` (`agent_spawn_sessions.turns`/`.tool_calls`) + `056_member_perf_daily` (`member_performance_daily`) predate this wave but were never appended to this doc; `057_project_sandbox_services` adds `projects.sandbox_services` (sandboxed dev DB/Redis/Mongo, `ROBOCO_SANDBOX_DB_ENABLED`); `058_cloud_auth_users` adds `users` (`UserTable`, cloud auth, `ROBOCO_CLOUD_AUTH_ENABLED`); `059_x_credentials` adds `x_credentials` (`XCredentialsTable`) + `x_seen_mentions` (`XSeenMentionTable`) (X engine, `ROBOCO_X_ENGINE_ENABLED`). Chain head is now 059. Mongo rides existing 057 (no new migration) — it's just another entry in the `SANDBOX_ENGINES` registry. +> +> Delta 2026-07-04 (v0.18.0): `060_drop_messaging` (the comms-teardown migration — drops `messages`/`session_tasks`/`sessions`/`groups`/`channels` + 4 enum types + `journal_entries.session_id`; A2A is now the sole directed-message channel; one-way, `downgrade()` raises `NotImplementedError`) had already landed on master but was never appended to this doc; `061_x_feature_spotlight` adds `x_seen_features` (`XSeenFeatureTable`) + `company_goals.brand_voice` (X feature-spotlight, `ROBOCO_X_FEATURE_SPOTLIGHT_ENABLED`, sub-switch of `x_engine_enabled`). Chain head is now 061. ORM table count is now 38 (verified via `grep -c '^class .*Table' roboco/db/tables.py`), up from this doc's previously-stated 37 (that figure predates 055-061 and was never recomputed). +> +> Delta 2026-07-18/19: `075_company_goals_company_name` adds `company_goals.company_name` (X/video product-branding fallback) and `076_project_git_provider` adds `projects.git_provider` (forge-providers Phase 0 — GitHub/Gitea/GitLab). Chain head is now 076 (062-072 remain the pre-existing table gap noted above — this delta only closes 073-076). ## Regression Risks | Title | File:Line | Claim | Severity | -|---|---|---|---| -| A2A legacy notification suppressed by new loop-prone re-fire guard | roboco/services/a2a.py:640 | create_a2a_notification delegates to NotificationService.send_a2a_notification. Since 3aff6e04, send_a2a_notification runs all_recipients_recently_notified (60s Redis SET-NX) for loop-prone types before creating the notification. A legitimate A2A peer notification re-sent within 60s (e.g. after a real state change, not a respawn loop) can be silently dropped, so the target agent is never notified/spawned. The A2A path has no awareness of which notification_type it produces being loop-prone, and cannot bypass the guard. | medium | -| PrReviewContent schema change could invalidate stale structured-note payloads | roboco/services/content_notes.py:65 | 15effce0 added optional 'issues' and 'head_sha' fields to PrReviewContent (foundation.policy.content.models). apply_structured_note calls validate_content which routes to PrReviewContent. The new fields have defaults (empty list / None) so existing payloads still validate and existing TEXT mirrors regenerate unchanged (Issues section only appended when issues present). Low risk, but any code that round-trips notes_structured.pr_review and assumed the exact key set may now encounter new keys. | low | -| Journal fire-and-forget RAG index silent on no-event-loop | roboco/services/journal.py:367 | _schedule_rag_index catches RuntimeError (no running event loop) and returns silently — no indexing, no log. Unchanged since baseline, but if a caller (e.g. a sync test or a non-async lifespan hook) writes a journal entry outside a running loop, the entry is persisted with zero RAG indexing and no warning. Combined with the new private-learning dual-sink rule this is easy to mis-verify. | low | -| Audit actor-role override can mismatch test expectations | roboco/services/audit.py:126 | log_task_action_denial resolves actual role from agents.role and overrides the caller-supplied agent_role. If a test seeds an agent with a role but passes a different agent_role param and asserts the persisted details.agent_role, it will see the DB role, not the param. Unchanged since baseline but a known foot-gun for regression tests added in the 15effce0 gap-fill batch. | low | +|-------|-----------|-------|----------| +| sa.Enum create_type silently dropped | alembic/versions/001_initial_schema.py:127,312 | 001 uses `sa.Enum(..., create_type=False)` which is a no-op on the generic Enum; a fresh re-apply on a clean DB can double-emit CREATE TYPE. | High | +| Enum-parity gate false-green | Makefile:540 + scripts/verify_postgres_enums.py | Gate skips on no-migrated-DB; an empty/mismatched `roboco` DB hides postgres-enum drift until a smoke run. | High | +| 016 team enum stale member list | alembic/versions/016_add_products_and_task_product_id.py:38 | `postgresql.ENUM(create_type=False)` member list is frozen at the original set; inert but masks later widening. | Medium | +| Missing pgvector extension blocks RAG | roboco/db/tables.py (chunks_*/indexed_documents) | Migrations assume pgvector installed; on a plain PG the vector columns fail and init_db aborts. | High | +| Two 026 files — rename hazard | alembic/versions/026_*.py | Renaming either 026 file breaks `down_revision` chain; autogenerate may mis-order. | Medium | +| 047 partial-unique index assumes single-active | alembic/versions/047_ws_single_active.py | A duplicate ACTIVE session raises on the partial-unique index; service-layer guard must run first or claim crashes. | Medium | +| 052 reuses team enum — order-dependent | alembic/versions/052_task_cell_projects.py:44 | Depends on `team` enum already existing (from 001/016); a partial chain replay to 052 without 016 would fail. | Low | +| Single-head violation on re-apply | alembic/versions/017_reconcile_orm_schema_drift.py | 017 adds tables/columns that `create_all` had created; on a DB built by `create_all` then stamped, 017 may double-create. | Medium | +| Migration 060 is a one-way removal with no downgrade | alembic/versions/060_drop_messaging.py:57-61 | `downgrade()` raises `NotImplementedError` — recreating channels/groups/sessions/session_tasks/messages + 4 enum types would need the full original schema. Any rollback plan for a bad deploy past 060 must restore from a pre-060 DB backup, not `alembic downgrade`. | Low | + +## Health +The chain is linear and complete (001→076), with `init_db` running `upgrade head` on every boot so deployed schemas stay current. The two structural risks are the `sa.Enum(create_type=False)` no-op in 001 (latent on clean re-applies) and the enum-parity gate's dependence on a populated migrated DB. New migrations consistently use the `postgresql.ENUM(create_type=False)` pattern and `ALTER TYPE ... ADD VALUE IF NOT EXISTS` for enum widening, so recent additions are safe. +# RoboCo Slice Map — `api-core-websocket` + +Slice key: `api-core-websocket` Repo root: `/Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco` Baseline commit: `fd10cc862c2020b3f639cdb686d427b0198a2441` Head: `15effce0` (2026-06-29, "Chore: 141 Gaps fill-in (#283)") + +## Purpose + +The FastAPI application shell, request pipeline, and real-time WebSocket fan-out layer for RoboCo. `app.py` builds the ASGI app, wires ~40 route routers, and runs the async lifespan (DB migrations, feature-flag overlay, transcription/extraction/RAG/learning service init, ordered shutdown). `middleware.py` adds correlation IDs, request logging, and a full exception-handler chain mapping domain/service/HTTP errors to structured JSON. `websocket.py` + `websocket_bridge.py` own the live panel streams (agents, notifications, system) with per-connection bounded send queues and an event-bus bridge. `deps.py` is the dependency-injection spine: agent header auth, role-gate helpers, and Choreographer/ContentActions wiring. `utils/` provides route-layer error factories and get-or-404/ownership helpers. `middleware_docs.py` enforces the docs-path permission matrix. `roboco/security.py` (outside `api/` but wired here) supplies the optional fastapi-guard HTTP security layer: `apply_guard(app)` mounts `SecurityMiddleware` last — outermost — in `create_app`, and `guarded_lifespan(lifespan)` wraps the async lifespan, both gated by `ROBOCO_GUARD_ENABLED` (default off, byte-for-byte unchanged request path while off). + +## Files + +| Path | Role | approx LOC | +|------|------|-----------| +| `roboco/api/app.py` | FastAPI app factory + async lifespan (startup/shutdown ordering) | ~490 | +| `roboco/api/deps.py` | DI: agent header auth, role gates, Choreographer/ContentActions builders, pagination | ~611 | +| `roboco/api/middleware.py` | Correlation-id + request-logging middleware, exception handlers, 422 secret-scrub + UUID remediation | ~469 | +| `roboco/api/middleware_docs.py` | Docs-path access-control matrix (read/write per role/team/slug) | ~330 | +| `roboco/api/websocket.py` | WS routes + ConnectionManager with bounded per-connection send queues, panel-token gate, idle timeout | ~692 | +| `roboco/api/websocket_bridge.py` | Event-bus → WS forwarders (notifications, agents, rate-limit, usage, A2A) | ~252 | +| `roboco/api/utils/__init__.py` | Re-export surface for error factories + resource helpers | ~43 | +| `roboco/api/utils/errors.py` | HTTPException factories + `handle_service_error` + `service_error_handler` decorator | ~214 | +| `roboco/api/utils/resources.py` | `get_or_404`, `get_by_field_or_404`, `require_ownership`/`require_recipient`/`require_membership` | ~180 | +| `roboco/api/__init__.py` | Deliberately does NOT re-export `app` (circular-import guard, documented) | ~14 | +| `roboco/security.py` | fastapi-guard 7.2.1 / guard-core 3.3.0 HTTP security layer: `SecurityMiddleware` + `guard_deco` (`SecurityDecorator`) singleton, gated by `ROBOCO_GUARD_ENABLED` (default off); wired into `create_app` via `apply_guard`/`guarded_lifespan` | ~407 | + +## Key Symbols + +| Name | Kind | File:Line | Responsibility | +|------|------|-----------|----------------| +| `lifespan` | async ctx mgr | app.py:82 | Startup: migrations, flag overlay, transcription/extraction/optimal/learning init; Shutdown: stop orchestrator BEFORE close_db, then close optimal, then DB | +| `create_app` | func | app.py:196 | Build FastAPI, add CORS + custom middleware, mount ~40 routers + ws_router at `/ws` | +| `app` | module attr | app.py:489 | The default ASGI instance (`roboco.api.app:app` entrypoint) | +| `_AppServices` | class | app.py:74 | Holder for transcription/extraction singletons set in lifespan | +| `DbSession` | type alias | deps.py:46 | `Annotated[AsyncSession, Depends(get_db)]` | +| `resolve_agent_id` | func | deps.py:49 | Resolve UUID-or-slug → UUID, 400 on miss | +| `_ServiceHolder` | class | deps.py:74 | Singleton store for PermissionService + orchestrator | +| `set_orchestrator`/`clear_orchestrator`/`get_orchestrator`/`get_orchestrator_or_none` | funcs | deps.py:91-119 | Global orchestrator accessors; `get_orchestrator` 503s when unset, `_or_none` used by shutdown | +| `get_current_agent_id`/`get_current_agent_slug`/`get_optional_agent_id` | funcs | deps.py:125-208 | Header-based agent identity (UUID or slug) | +| `_auth_required` | func | deps.py:211 | Reads `ROBOCO_AGENT_AUTH_REQUIRED` env | +| `_check_agent_auth_token` | func | deps.py:217 | HMAC token enforcement (required in prod, optional-but-verified in dev) | +| `require_panel_token` | func | deps.py:251 | CEO-HMAC gate for live-chat bridges (HTTP analog of WS gate) | +| `_resolve_agent_identity` | func | deps.py:277 | Returns `(agent_id, slug)`, special-casing `system` role | +| `_coerce_agent_role`/`_coerce_agent_team` | funcs | deps.py:298/324 | Parse role/team headers with DB fallback for role | +| `_header_trust_agent_context` | func | deps.py:340 | The original `get_agent_context` body verbatim (header-trust); the OFF-mode path, and also what a valid agent HMAC token delegates to when cloud auth is ON | +| `_slide_session_cookie` | func | deps.py:379 | Re-mints + re-sets the session cookie on every cookie-authenticated request — the sliding 30-day window (only inactivity past `cloud_auth_cookie_max_age` logs out) | +| `_cloud_auth_agent_context` | func | deps.py:390 | Dual-path enforcement when `cloud_auth_enabled`: a valid HMAC token (any role) delegates to `_header_trust_agent_context`; otherwise a non-CEO role claim is rejected outright, and the CEO must present a valid session cookie via `resolve_session_user` | +| `get_agent_context` | func | deps.py:452 | Builds `AgentContext` from X-Agent-* headers + token (+ the `roboco_session` cookie); byte-for-byte header-trust when `cloud_auth_enabled` is False, else delegates to `_cloud_auth_agent_context` | +| `require_pm_or_above`/`require_developer_or_above`/`require_cell_access` | funcs | deps.py:403/428/443 | Coarse role-gate guards (403) | +| `require_ceo_role` | func | deps.py:412 | Single CEO-check: raises 403 unless `role` is CEO; accepts `AgentRole`/`Role`/lowercase string; unifies orchestrator-router + release-handler CEO gates into one source of truth (`536bbb64`) | +| `require_notification_permission`/`require_task_action` | dep factories | deps.py:490/508 | PermissionService-backed dependency factories | +| `get_choreographer`/`get_content_actions` | funcs | deps.py:545/575 | Build Choreographer / ContentActions with all service deps + orchestrator/bus | +| `get_pagination` | func | deps.py:599 | Clamp limit 1-100, offset ≥0 | +| `CorrelationIdMiddleware` | class | middleware.py:54 | X-Correlation-ID in/out + structlog bind | +| `RequestLoggingMiddleware` | class | middleware.py:96 | Log request/response + `X-Response-Time-Ms` header | +| `get_status_code`/`roboco_exception_handler`/`service_exception_handler`/`rate_limit_exception_handler`/`generic_exception_handler`/`http_exception_handler`/`request_validation_handler` | funcs | middleware.py:143-441 | Exception → structured JSONResponse chain | +| `_SERVICE_ERROR_STATUS` | const | middleware.py:191 | Maps service exception types → HTTP status | +| `_uuid_field_remediation` | func | middleware.py:343 | Actionable hint when an agent sends an 8-char task prefix as UUID | +| `_SECRET_FIELD_NAMES`/`_scrub_secrets` | const/func | middleware.py:372/389 | Redact credential fields from 422 log bodies | +| `setup_middleware` | func | middleware.py:444 | Register exception handlers + middleware in order | +| `DOCS_PERMISSIONS` | const | middleware_docs.py:55 | Path-prefix → read/write role matrix | +| `check_docs_access`/`require_docs_access`/`get_allowed_docs_paths` | funcs | middleware_docs.py:222/265/303 | Docs path permission checks | +| `_fast_path_access_decision` | func | middleware_docs.py:203 | CEO/auditor/main_pm short-circuit | +| `ConnectionManager` | class | websocket.py:82 | Tracks all WS subscriptions + per-connection send queues | +| `_ClientConnection` | class | websocket.py:45 | Bounded outbound queue + sender task holder | +| `manager` | singleton | websocket.py:334 | Global ConnectionManager | +| `_require_panel_token` | func | websocket.py:61 | WS upgrade CEO-HMAC gate (returns False to close) | +| `validate_agent_exists` | func | websocket.py:337 | DB existence check for claimed agent_id | +| `_register_sender`/`_run_sender`/`_enqueue_or_send`/`_send_with_timeout` | methods | websocket.py:122/129/244/270 | Non-blocking fan-out plumbing | +| `connect_agent`/`connect_notifications`/`connect_system` | methods | websocket.py:187/213/224 | Accept + register per stream type | +| `disconnect` | method | websocket.py:230 | Remove socket from all sets + cancel sender task | +| `broadcast_to_agent_watchers`/`broadcast_system` | methods | websocket.py:310/332 | Fan-out enqueues | +| `agent_stream`/`notification_stream`/`system_stream` | routes | websocket.py:445/573/625 | WS endpoints under `/ws` | +| `broadcast_agent_chunk`/`broadcast_notification` | funcs | websocket.py:650/665 | Helper broadcasters for external callers | +| `IDLE_TIMEOUT_SECONDS`/`MAX_SEND_QUEUE`/`SEND_TIMEOUT_SECONDS` | consts | websocket.py:35/41/42 | 90s / 256 / 10s tunables | +| `register_websocket_bridge_handlers`/`start_websocket_bridge` | funcs | websocket_bridge.py:169/203 | Subscribe bus handlers | +| `_handle_notification_sent`/`_handle_agent_event`/`_handle_rate_limit_event`/`_handle_usage_event`/`_handle_a2a_message_event` | funcs | websocket_bridge.py | Event-bus → WS forwarders (`_handle_a2a_message_event` fans `A2A_MESSAGE_SENT` to `/ws/system` as an `a2a.message` frame — the CEO's live view of every agent-to-agent chat) | +| `_RATE_LIMIT_WS_TYPES`/`_USAGE_WS_TYPES` | consts | websocket_bridge.py:18/23 | EventType → panel `type` string maps | +| `not_found`/`forbidden`/`unauthorized`/`validation_error`/`conflict`/`service_unavailable` | funcs | utils/errors.py:29-142 | HTTPException factories | +| `handle_service_error`/`service_error_handler` | func/deco | utils/errors.py:150/191 | ServiceError → HTTPException translation | +| `get_or_404`/`get_by_field_or_404` | funcs | utils/resources.py:17/59 | Generic get-or-404 helpers | +| `require_ownership`/`require_recipient`/`require_membership` | funcs | utils/resources.py:96/130/156 | Authorization checks | +| `apply_guard` | func | security.py:378 | Mounts `SecurityMiddleware` on `app` + sets `app.state.guard_decorator`; no-op unless `settings.guard_enabled` | +| `guarded_lifespan` | func | security.py:399 | Wraps `lifespan` with guard's `make_lifespan` (redis/geo/agent init) when armed; passthrough when off | +| `build_security_config` | func | security.py:329 | Assembles the global `SecurityConfig` from settings: passive_mode, fail_secure, enforce_https, WAF calibration fields | +| `security_config` / `guard_deco` | module singletons | security.py:374-375 | Built once at import (pure, no I/O); `guard_deco` is the `SecurityDecorator` route files decorate with `@guard_deco.` | +| `prompt_injection_validator`/`secret_exfil_validator`/`internal_ssrf_validator` | async funcs | security.py:116/128/139 | Custom `@guard_deco.custom_validation` content checks the signature WAF can't cover; each returns a generic 400 (no rule detail leaked) | +| `_WAF_FREETEXT_BODY_FIELDS` | const | security.py:211 | Top-level free-text body-field exclusion set (`excluded_detection_body_fields`) — the WAF calibration; includes free-form container fields (plan/risks/findings/section/payload/...) whose nested prose is stringified and scanned | + +## Data Flow + +**HTTP request**: nginx → ASGI `app` → `CorrelationIdMiddleware` (binds correlation_id + path/method to structlog) → `RequestLoggingMiddleware` (start timer) → route. Route resolves `CurrentAgentContext` via `get_agent_context` (headers + HMAC verify + identity/role/team resolution), plus service deps from `get_choreographer`/`get_content_actions`. When `ROBOCO_CLOUD_AUTH_ENABLED` is off (default) this is byte-for-byte the historical header-trust path (`_header_trust_agent_context`). When on, `_cloud_auth_agent_context` enforces a dual path: a request carrying a valid `X-Agent-Token` HMAC (any role — the agent fleet + the orchestrator's own `system` self-PATCH) is verified then delegated to the same header-trust resolution; a request with no valid token and a non-CEO role claim is rejected outright (closes the LAN header-spoof hole); the CEO alone may instead authenticate via the `roboco_session` cookie (`resolve_session_user`, `roboco.api.auth.session`), which is re-minted on every authenticated request (`_slide_session_cookie`) for a sliding 30-day window. On exception, the handler chain maps: `RequestValidationError` → 422 (scrubbed log + UUID remediation hint), `HTTPException` → standardized error code, `RobocoError` → domain status, `ServiceError` → parallel-hierarchy status, `RateLimitError` → 429 + `Retry-After`, `Exception` → 500. Response gains `X-Correlation-ID` + `X-Response-Time-Ms`. When `ROBOCO_GUARD_ENABLED` is on, `SecurityMiddleware` (mounted last in `create_app`, so outermost) runs before any of this: rate/size/WAF/custom-validator checks either block the request (enforce mode) or only log the detection (`guard_passive_mode`, the calibration posture) ahead of the correlation-id middleware; off by default, the whole path is unchanged. + +**Lifespan startup**: `init_db` (alembic upgrade + create_all fallback) → `apply_persisted_feature_flags` (panel settings overlay, best-effort) → `TranscriptionService.start()` + `ExtractionPipeline` → `get_optimal_service()` (BLOCKS 30-90s for RAG) → `LearningPropagationService.initialize(optimal)`. `app.state.*` holds singletons. **Shutdown**: stop orchestrator (drains bg DB writes) → `close_optimal_service` → `close_db`. The orchestrator-stop-before-DB order is load-bearing. + +**WebSocket**: panel → `wss://.../ws/{kind}/{id}` → `_require_panel_token` (CEO HMAC, except `/ws/system`) → query-param `agent_id`/`viewer_id` UUID parse (+ DB existence check for agent/notifications) → `manager.connect_*` accepts, registers in subscription set, spawns per-connection `_run_sender` task draining a bounded queue. Receive loop `await asyncio.wait_for(receive_text(), IDLE_TIMEOUT_SECONDS)`; "ping"→"pong"; timeout/disconnect → `finally disconnect`. **Broadcasts**: service code calls `manager.broadcast_to_*` → `_enqueue_or_send` per conn → either `conn.queue.put_nowait` (drop+warn on full) or legacy fire-and-forget `_send_with_timeout`. **Event-bus bridge**: `StreamEventBus` publishes `NOTIFICATION_SENT`/`AGENT_*`/`RATE_LIMIT_*`/`USAGE_SNAPSHOT`/`A2A_MESSAGE_SENT` → `websocket_bridge` handlers → `manager.broadcast_*` → panel. `A2A_MESSAGE_SENT` (published on every agent-to-agent DM) fans out to `/ws/system` as an `a2a.message` frame carrying conversation_id/message_id/task_id/from_agent/to_agent/skill + a capped body excerpt — the CEO's live view of every agent-to-agent chat, and the canonical pattern for wiring a new live event to the panel. + +## Mermaid + +```mermaid +sequenceDiagram + participant Panel + participant Nginx + participant ASGI as app (FastAPI) + participant MW as middleware + participant Route as route + deps + participant Chor as Choreographer + participant Bus as StreamEventBus + participant Mgr as ConnectionManager + participant WS as /ws/* client + + Note over ASGI: lifespan startup + ASGI->>ASGI: init_db (alembic) + ASGI->>ASGI: apply_persisted_feature_flags + ASGI->>ASGI: transcription + extraction + optimal(RAG) + learning + + Panel->>Nginx: HTTPS /api/* (X-Agent-Token) + Nginx->>ASGI: forward + ASGI->>MW: CorrelationIdMiddleware (bind cid) + MW->>MW: RequestLoggingMiddleware (start timer) + MW->>Route: dispatch + Route->>Route: get_agent_context (HMAC verify) + Route->>Chor: get_choreographer(db) + Chor-->>Route: Envelope + Route-->>MW: response + MW-->>Nginx: + X-Correlation-ID, X-Response-Time-Ms + Nginx-->>Panel: response + + Panel->>Nginx: wss /ws/system + Nginx->>ASGI: upgrade + ASGI->>Mgr: _require_panel_token -> connect_system + Mgr->>Mgr: _register_sender (queue + task) + Bus-->>ASGI: A2A_MESSAGE_SENT event + ASGI->>Mgr: _handle_a2a_message_event + Mgr->>WS: enqueue -> send_text (a2a.message) + + Note over ASGI: lifespan shutdown + ASGI->>Chor: orchestrator.stop() (drains DB writes) + ASGI->>ASGI: close_optimal_service + ASGI->>ASGI: close_db +``` + +## Logical Tree + +``` +roboco/api/ +├── __init__.py # no re-export of app (circular-import guard) +├── app.py +│ ├── _AppServices # transcription/extraction holders +│ ├── lifespan() # startup + shutdown ordering +│ └── create_app() -> app # ~40 routers + ws_router at /ws +├── deps.py +│ ├── _ServiceHolder # permission_service + orchestrator singletons +│ ├── agent identity (get_current_agent_id / slug / optional / context) +│ ├── _check_agent_auth_token / require_panel_token (HMAC) +│ ├── role gates (require_pm_or_above / developer_or_above / cell_access) +│ ├── permission dep factories (notification, task action) +│ ├── get_choreographer / get_content_actions +│ └── get_pagination +├── middleware.py +│ ├── CorrelationIdMiddleware +│ ├── RequestLoggingMiddleware +│ ├── exception handlers (RequestValidationError, HTTPException, RobocoError, +│ │ ServiceError, RateLimitError, Exception) +│ ├── _scrub_secrets / _uuid_field_remediation +│ └── setup_middleware() +├── middleware_docs.py +│ ├── DOCS_PERMISSIONS matrix +│ ├── check_docs_access / require_docs_access +│ └── get_allowed_docs_paths +├── websocket.py +│ ├── _ClientConnection (queue + sender) +│ ├── _require_panel_token +│ ├── ConnectionManager (agent/notification/system sets + senders) +│ ├── manager singleton +│ ├── validate_agent_exists +│ ├── routes: /agents/{id} /notifications/{id} /system +│ └── broadcast_agent_chunk / broadcast_notification helpers +├── websocket_bridge.py +│ ├── _handle_notification_sent / _handle_agent_event / _handle_a2a_message_event +│ ├── _handle_rate_limit_event / _handle_usage_event +│ └── register_websocket_bridge_handlers / start_websocket_bridge +└── utils/ + ├── __init__.py # re-exports + ├── errors.py # HTTPException factories + service_error_handler + └── resources.py # get_or_404 + ownership/recipient/membership +``` + +## Dependencies + +**Internal**: `roboco.config.settings`; `roboco.db.base` (init_db/close_db/get_db/get_session_factory); `roboco.db.tables.AgentTable`; `roboco.foundation.identity` (BOARD_ROLES/DEV_ROLES/PM_ROLES/Role); `roboco.models` (AgentRole/Team); `roboco.runtime.AgentOrchestrator`; `roboco.agents_config` (CEO_AGENT_ID, verify_agent_token, AGENT_ROLE_MAP/AGENT_TEAM_MAP, ALL_DOCS, _resolve_to_slug); `roboco.events` (Event/EventType/get_event_bus); `roboco.exceptions` (RobocoError tree); `roboco.services.base` (ServiceError tree); `roboco.services.exceptions.RateLimitError`; `roboco.services.{permissions,task,work_session,git,workspace,journal,a2a,product,notification,notification_delivery,audit,settings,extraction,learning,optimal,transcription}`; `roboco.services.gateway.{choreographer,content_actions,evidence_repo}`; `roboco.services.repositories` (resolve_agent_uuid/resolve_agent_identity); `roboco.api.schemas.{optimal.PaginationParams,common.ErrorCode}`; ~40 `roboco.api.routes.*` routers; `roboco.api.routes.v1.*` flow modules. + +**External**: `fastapi` (FastAPI, APIRouter, WebSocket, HTTPException, Depends, Header, status), `starlette.middleware.base.BaseHTTPMiddleware`, `starlette` responses, `sqlalchemy` (select, async session), `structlog`, `pydantic` (via schemas), `asyncio`, `uuid`, `json`, `time`, `contextlib`. (`httpx` was REMOVED from websocket.py in the baseline→head diff — the self-call `validate_channel_access` is gone.) + +## Entry Points + +- `roboco.api.app:app` — the ASGI instance uvicorn/gunicorn serves; `create_app()` called at import. +- `lifespan` — FastAPI async context manager; runs on startup/shutdown. +- `setup_middleware(app)` — called from `create_app` after CORS. +- `register_websocket_bridge_handlers()` / `start_websocket_bridge()` — called by orchestrator/bootstrap after the event bus is up (NOT in `lifespan`; the lifespan does not register bridge handlers). +- WS routes — invoked by the panel via `wss://.../ws/{agents|notifications|system}/...`, mounted at prefix `/ws` in `create_app`. +- `manager` singleton — imported directly by services and the bridge to broadcast. +- DI deps — resolved per-request by FastAPI (`get_agent_context`, `get_choreographer`, `get_content_actions`, `get_pagination`, `require_*` factories). +- `require_panel_token` — HTTP dep on the live intake/secretary chat bridges. + +## Config Flags + +- `ROBOCO_AGENT_AUTH_REQUIRED` — gates HMAC token enforcement (deps.py:211, websocket.py:71, middleware docstring app.py:94). Unset → header-trust/dev mode; set `true`/`1`/`yes` → strict. +- `ROBOCO_AGENT_AUTH_SECRET` — the HMAC secret consumed by `verify_agent_token` (read inside `roboco.agents_config`). +- `ROBOCO_CLOUD_AUTH_ENABLED` (+ `_EMAIL`/`_PASSWORD`/`_SECRET`/`_COOKIE_MAX_AGE`, default off) — `deps.get_agent_context`'s dual-path switch (`_cloud_auth_agent_context` vs byte-for-byte `_header_trust_agent_context`); the login/logout FastAPI Users router is mounted by `roboco.api.auth.routes.mount_cloud_auth` only when true, but `/api/auth/status` is always mounted (public probe for the panel's `proxy.ts`). +- `ROBOCO_DATABASE_*`, `ROBOCO_REDIS_*` — read transitively via `settings` / `init_db`. +- `settings.cors_origins` / `settings.cors_allow_credentials` — CORS middleware config (app.py:218). +- `settings.app_version` / `settings.environment` / `settings.debug` — logged at startup; docs/redoc URLs are unconditional (the `if settings.debug` is commented out, app.py:207-208). +- `settings.host` / `settings.port` — no longer used in websocket.py (the httpx self-call was removed); still referenced elsewhere. +- `ROBOCO_GUARD_ENABLED` / `_PASSIVE_MODE` / `_FAIL_SECURE` / `_TELEMETRY_ENABLED` / `_AGENT_API_KEY` / `_PROJECT_ID` / `_EMERGENCY` / `_EMERGENCY_WHITELIST` — read by `roboco/security.py`, wired into `create_app` via `apply_guard(app)` (app.py:234) + `guarded_lifespan(lifespan)` (app.py:212); `ROBOCO_ENVIRONMENT` additionally drives `enforce_https` (production only). +- Otherwise no direct ROBOCO_* feature flags live in this slice; the lifespan applies persisted flag overlays via `apply_persisted_feature_flags` but does not itself read individual subsystem flags. + +## Gotchas + +- **~~`/ws/system` was the only WS endpoint WITHOUT `_require_panel_token`~~ — RESOLVED (`536bbb64`)**: `/ws/system` now calls `_require_panel_token` before subscribing (websocket.py:621); all `/ws/*` endpoints are consistently gated (five at the time of `536bbb64`; three after the comms-subsystem teardown removed `/ws/channels` + `/ws/sessions`). In strict mode a missing token closes with `WS_1008_POLICY_VIOLATION`; a forged token is rejected even in dev mode. +- **`websocket.py` module docstring (lines 8-13) is STALE** — it claims WS "validates agent_id via query params and verify the agent exists in the database. In production, this should be enhanced with proper token-based authentication." Actual security is now the CEO HMAC panel token via `_require_panel_token`, and `validate_agent_exists` is only called on `/agents`, `/notifications`. The docstring misleads. +- **`get_choreographer` passes `stream_bus=None` when no orchestrator is set** (deps.py:557) — fine, but means the rate-limit park path is inert during the startup window before bootstrap sets the orchestrator. +- **`_check_agent_auth_token` dev mode**: a missing token is allowed; a presented-but-invalid token is rejected. The header-trust warning at app.py:94-102 is the only signal. In dev, any reachable client can act as any role (including `ceo`) by setting headers. +- **`_resolve_agent_identity` `system` role special-case** (deps.py:281) returns `UUID(x_agent_id)` with NO DB lookup. Combined with dev-mode no-auth, a caller can claim `role=system` with an arbitrary UUID and bypass agent resolution entirely. `ROBOCO_CLOUD_AUTH_ENABLED` does not add a DB lookup here either — it only requires that a `system`/any-role claim carry a *verified* HMAC token first (`_cloud_auth_agent_context` rejects any non-CEO role claim without one), so the identity-bypass itself is unchanged, just gated behind a valid signature. +- **`_coerce_agent_role` falls back to the DB role when the header isn't a valid enum** (deps.py:298). The X-Agent-Role header is therefore advisory when malformed — the authoritative role is the agent row's. Good for safety, but means a caller cannot escalate by header alone (the DB role wins). +- **`request_validation_handler` returns the UNSCRUBBED body to the client** (middleware.py:434). Only the server log is scrubbed (`_scrub_secrets`); the 422 response echoes whatever the client sent, including any secret fields. By design (the client sent them), but worth knowing. +- **`_run_sender` self-cancels on send error**: on a hard send error it calls `self.disconnect(ws)` which cancels `conn.sender` — the very task currently running (websocket.py:155). It returns immediately after, so the cancellation lands on an already-returning task; harmless in practice but a subtle self-cancel. +- **Lifespan shutdown order is load-bearing**: orchestrator.stop() MUST run before close_db (app.py:170-186). Reverting this order silently drops final audit-log rows + respawn_tracker upserts + agent-state finalizes. `stop()` is idempotent (bootstrap's finally re-calls it). +- **`ConnectionManager` sets are NOT mutated under a lock** — relies on asyncio single-threadedness. A broadcast iterating a set while `disconnect` mutates it is safe within one event loop, but `_run_sender`'s `disconnect(ws)` is called from a different task than the receive loop's `finally disconnect`, so two tasks can concurrently mutate the same set. `set.discard` is safe but iteration-during-mutation could raise `RuntimeError: Set changed size during iteration` in pathological cases. +- **`broadcast_notification` (websocket.py:665) bypasses the `broadcast_to_*` pattern** and reaches into `manager._enqueue_or_send` directly with a pre-serialized `data` string, while `broadcast_to_*` serialize inside. Inconsistent but works. +- **`roboco/api/__init__.py` deliberately does NOT re-export `app`** — importing `roboco.api.schemas.X` must not transitively load the FastAPI app + routes (circular-import cycle). The entrypoint imports `roboco.api.app:app` directly. Do not "helpfully" re-export here. +- **`docs_url`/`redoc_url` are unconditional** (app.py:207-208) — the `if settings.debug` gating is commented out, so `/docs` and `/redoc` are always served. +- **`apply_persisted_feature_flags` is best-effort** (app.py:115-121) — a DB failure logs a warning and continues with env defaults; startup is never blocked. +- **fastapi-guard is a genuine no-op when off** (`ROBOCO_GUARD_ENABLED` default `false`) — `apply_guard` returns before `add_middleware`, so `create_app`'s request path is byte-for-byte unchanged; the per-route `@guard_deco.*` decorators across ~21 route files are harmless because the decorator only takes effect once `app.state.guard_decorator` is set by `apply_guard` (security.py:388). +- **`excluded_detection_body_fields` is the only reliable WAF-calibration knob on guard 7.2.1** — the per-route `categories`/`enabled_detection_categories` config is bypassed for JSON bodies, and the body scanner excludes TOP-LEVEL keys only, scanning `str(value)` of every non-excluded field (the whole stringified subtree). A free-form container field (e.g. `plan`, `findings`) must therefore be excluded wholesale or its nested prose still trips the WAF. + +## Drift from CLAUDE.md + +- `CLAUDE.md` "WebSocket streams" section lists `/ws/agents/{id}`, `/ws/notifications/{id}`, `/ws/system` — **matches** `websocket.py` routes. No drift: both sides now agree the surviving set is these three. `/ws/channels/{id}` and `/ws/sessions/{id}` are removed from `websocket.py` itself as part of the comms-subsystem teardown (a real route deletion, not a stale doc omission). +- `CLAUDE.md` says `/ws/system` carries "the rate-limit lifecycle (`RATE_LIMIT_HIT` / `RATE_LIMIT_LIFTED`) and live usage (`USAGE_SNAPSHOT`)" — **matches** `websocket_bridge.py:138-166` (`_handle_rate_limit_event` + `_handle_usage_event`). No drift. +- `CLAUDE.md` says "To add a new live event: define an `EventType`, publish it to the bus, add a `_handle_*` forwarder in `websocket_bridge`, and consume it on the panel via the `useWebSocket(...)` hook" — **matches** the `_handle_*` pattern. CLAUDE.md's worked example is now `A2A_MESSAGE_SENT` → `_handle_a2a_message_event` → `/ws/system` (replacing the retired `MESSAGE_SENT`/`useSessionStream` example), consistent with this slice's implementation. No drift. +- `CLAUDE.md` "Startup Sequence" says "FastAPI lifespan indexes documents using Ollama (~30-60s)" — `app.py:137-147` initializes `OptimalService` (RAG) with a 30-90s blocking init and logs "OptimalService (RAG) initialized successfully"; the "indexes documents" framing is approximate but consistent. No material drift. +- `CLAUDE.md` does NOT document the `_require_panel_token` CEO-HMAC WS gate or the HTTP `require_panel_token` dep. The `websocket.py` module docstring (lines 8-13) describes the OLD query-param model, contradicting the actual HMAC-token implementation. The CLAUDE.md "Agent Gateway" section's "Agents do not call the API or per-domain MCP tools directly" is consistent with `/ws/*` being operator-only. **Drift: in-file docstring vs actual code (websocket.py:8-13); CLAUDE.md itself is silent on WS auth, so no CLAUDE.md contradiction.** +- `CLAUDE.md` "Orchestrator runtime-state durability" notes the respawn_tracker DB-durable writes are drained on `stop()` — `app.py:170-186` implements the required ordering (stop before close_db). Consistent. +- `CLAUDE.md` "Feature flags / company-in-a-box" says flags "toggle from the panel's Settings → Feature Flags card ... A toggle persists in the settings store and takes effect on the next backend restart" — `app.py:115-121` applies them in lifespan. Consistent. +- `CLAUDE.md` does not mention the `CorrelationIdMiddleware` / `RequestLoggingMiddleware` / exception-handler chain by name; `middleware.py` is the implementation of the implied "structured error" contract. No contradiction. +- `CLAUDE.md`'s "Feature flags / company-in-a-box" list of env-gated default-off subsystems does not mention `ROBOCO_GUARD_ENABLED` / the fastapi-guard HTTP security layer (`roboco/security.py`, wired here via `apply_guard`/`guarded_lifespan`); the doc is silent rather than contradictory. + +Net: **no direct contradictions with CLAUDE.md**; the one stale security docstring lives in `websocket.py` itself. ## Changes Since Baseline +Only ONE commit in `fd10cc86..HEAD` touched this slice: `15effce0` "Chore: 141 Gaps fill-in (#283)" (2026-06-29). + +Diff stat: `app.py +20`, `deps.py +44`, `middleware.py +49`, `websocket.py +309/-66` (net), `middleware_docs.py` / `websocket_bridge.py` / `utils/*` UNCHANGED. + +> **Post-snapshot update (2026-07-01, chat-subsystem live-delivery work `76ce53e3`):** `websocket_bridge.py` is no longer unchanged — it gained `_handle_message_event` (forwards `EventType.MESSAGE_SENT` to `/ws/sessions/{id}` + `/ws/channels/{id}` as a `message.new` frame) and a `MESSAGE_SENT` subscription in `register_websocket_bridge_handlers`. This is the live transcript-update path that was previously dead (`send_message` never broadcast). **Now removed** by the comms-subsystem teardown (`docs/internal/specs/2026-07-03-comms-teardown-trace.md`): the `/ws/sessions` + `/ws/channels` routes, the `_handle_message_event` forwarder, and its `MESSAGE_SENT` subscription are all gone; `EventType.MESSAGE_SENT` itself is left as a dead/inert enum member, not deleted. The surviving bridge worked example is `_handle_a2a_message_event` (`EventType.A2A_MESSAGE_SENT` → `/ws/system` as an `a2a.message` frame). + +> **Post-snapshot update (2026-07-01, logical-gap sweep `536bbb64`):** `deps.py` gained `require_ceo_role` (deps.py:412) — single source-of-truth CEO-role check shared by the orchestrator router gate and the release handler, replacing two diverged inline comparisons. `websocket.py` gated `/ws/system` with `_require_panel_token` (websocket.py:621), closing the medium regression risk; all `/ws/*` endpoints are now consistently gated (five at the time of this commit; the comms-subsystem teardown since removed `/ws/channels` + `/ws/sessions`, leaving `/ws/agents`, `/ws/notifications`, `/ws/system`). + +> **Local branch (not on master, NOT deployed):** `feature/fastapi-guard-hardening` (6 fastapi-guard commits `896532a3`..`99ee666e`, branched off `ab69851d`, plus 2 unrelated bundled commits) adds `roboco/security.py` and wires it into this slice — `apply_guard(app)` mounts `SecurityMiddleware` last in `create_app` (app.py:234) and `guarded_lifespan(lifespan)` wraps the async lifespan (app.py:212), both gated by `ROBOCO_GUARD_ENABLED` (default off, byte-for-byte unchanged request path when off). Per-route `@guard_deco.*` decorators (rate_limit/max_request_size/content_type_filter/behavior_analysis/block_clouds/honeypot_detection/usage_monitor/suspicious_detection/custom_validation — 9 kinds) are applied across 21 route files outside this slice (api-routes-schemas + v1 flow/do). `build_security_config` also carries a WAF false-positive calibration: `excluded_detection_body_fields` (75 free-text top-level body fields, including container fields like plan/risks/findings/section/payload) plus `enable_penetration_detection=True`, dropping active-mode false positives on RoboCo's own code/SQL/diff/URL-bearing traffic to zero while leaving the three custom validators and the WAF on non-excluded (id/enum/slug/branch) fields fully in force. New tests: `tests/unit/test_security.py` (unit) + `tests/unit/test_security_middleware.py` (integration — mounts the real middleware end-to-end). Both NAS composes (`docker-compose.yml`/`.yaml`) arm the layer passive/log-only (`c496b677`, Phase 5) — see deployment-tooling. + +Logic-touching changes in that commit, scoped to this slice: + +| Change | File:Line | IMPACT | +|--------|-----------|--------| +| Lifespan shutdown now stops orchestrator BEFORE close_db (was: DB closed first, only bootstrap's finally stopped orchestrator) | app.py:170-186 | Final audit-log rows / respawn_tracker upserts / agent-state finalizes no longer silently dropped on shutdown. `get_orchestrator_or_none()` added so lifespan doesn't 503 when no orchestrator is wired (tests/skip_orchestrator). | +| `get_orchestrator_or_none` + `clear_orchestrator` added | deps.py:96-119 | New accessor for shutdown + test teardown; `get_orchestrator` still 503s. | +| `require_panel_token` HTTP dep added (CEO HMAC for live-chat bridges) | deps.py:251-274 | New gate; mirrors WS `_require_panel_token`. Browser EventSource can't set headers, so token-only. | +| `CEO_AGENT_ID` import added to deps.py | deps.py:18 | Required by `require_panel_token` + reused by WS gate. | +| `_SECRET_FIELD_NAMES` + `_scrub_secrets` added; 422 log scrubs credential fields | middleware.py:372-404, 426 | Plaintext GitHub PAT / provider API key / bearer tokens no longer dumped to structlog on a 422. Response body unchanged. | +| WS panel-token gate (`_require_panel_token`) added to channel/agent/session/notifications streams | websocket.py:61, 371/439/503/567 | `/ws/*` now CEO-HMAC-gated in prod; dev allows missing token but rejects forged. **`/ws/system` was NOT gated** (inconsistency). | +| Per-connection bounded send queue + sender task (`_ClientConnection`, `_register_sender`, `_run_sender`, `_enqueue_or_send`, `_send_with_timeout`) | websocket.py:45-156, 244-281 | Replaced `asyncio.gather(*[conn.send_text(data)])` with non-blocking enqueue. One slow client can no longer back-pressure the fan-out; full queue drops+warns. `_run_sender` reaps dead sockets on hard send error. | +| `IDLE_TIMEOUT_SECONDS` (90s) `asyncio.wait_for` on `receive_text` | websocket.py:35, 400/471/534/586/623 | Half-open sockets from dead containers no longer block the receive loop forever; `TimeoutError` → `finally disconnect`. | +| `httpx` self-call `validate_channel_access` REMOVED; `settings` import dropped from websocket.py | websocket.py (was) | The channel WS no longer makes an HTTP round-trip to `/api/permissions/check` on the local server (deadlocks/latency risk gone). Channel access now gated only by panel token. | +| `structlog` logger (`log`) replaced the old logger; `validate_agent_exists` kept for agent/session/notifications | websocket.py:30, 337 | Logging consistent with rest of API. | +| `system_stream` endpoint + `broadcast_system` + `connect_system` (already present pre-baseline) — the commit RETAINED the no-token path for `/ws/system` | websocket.py:208-212, 316-322, 608 | Operator stream stays ungated; rate-limit/usage telemetry reachable without panel token. | + +`middleware_docs.py` and `utils/*` are byte-for-byte unchanged since baseline. `websocket_bridge.py` was unchanged at the snapshot but has since been edited by the chat-subsystem live-delivery work (the `_handle_message_event` forwarder + `MESSAGE_SENT` subscription — see the post-snapshot note above). + +## Regression Risks + +| Title | File:Line | Claim | Severity | +|-------|-----------|-------|----------| +| `/ws/system` ungated while siblings require panel token | websocket.py:608 | ~~The rate-limit/usage operator stream has no `_require_panel_token` call; if nginx does not edge-gate `/ws/system`, any reachable client gets RATE_LIMIT_HIT/LIFTED + USAGE_SNAPSHOT telemetry. The other four streams gate.~~ **RESOLVED (`536bbb64`)**: `/ws/system` now calls `_require_panel_token` (websocket.py:621); all `/ws/*` endpoints are consistently gated (five then; the comms-subsystem teardown since removed `/ws/channels` + `/ws/sessions`, leaving three). | ~~medium~~ resolved | +| `_run_sender` self-cancels its own task on send error | websocket.py:155 | `self.disconnect(ws)` pops `conn.sender` and calls `.cancel()` on the task currently running `_run_sender`. The immediate `return` mitigates, but a cancellation landing on a returning task is a subtle race; under heavy churn could mask a later send or trip a CancelledError in an `except Exception` handler. | low | +| Concurrent set mutation between sender-task `disconnect` and receive-loop `finally disconnect` | websocket.py:155 / 425 | Two tasks (sender + receive loop) can call `disconnect(ws)` on the same socket concurrently. `set.discard` is idempotent, but a `broadcast_to_*` iterating the same set on the loop thread during the sender-task's `disconnect` could raise `Set changed size during iteration`. Single-loop asyncio makes this rare but not impossible. | low | +| `validate_agent_exists` opens its own DB session via `async for db in get_db()` per WS upgrade | websocket.py:347 | Each `/agents`, `/notifications` upgrade grabs a session just to confirm the viewer exists — extra session pressure under many concurrent panel connections; the check is also bypassable in dev (no token). | low | +| Lifespan shutdown hangs if `orchestrator.stop()` blocks | app.py:184 | The new stop-before-close-db order is correct, but a hung `stop()` now blocks `close_optimal_service` + `close_db` (wrapped in try/except, so a hang — not an exception — is the failure mode). Pre-baseline, a hung stop only affected bootstrap's finally. | low | +| 422 response body still echoes unscrubbed secrets | middleware.py:434 | `_scrub_secrets` only scrubs the LOG, not the response. A client sending a `git_token` that fails validation gets it back in the 422 `body` field. By design but a leak surface if logs/responses are captured. | low | +| `_resolve_agent_identity` `system` role bypasses DB lookup | deps.py:281 | `role=system` returns `UUID(x_agent_id)` with no DB check. In dev (no auth) a caller can claim `system` with any UUID and get an `AgentContext` for a non-existent agent. Auth-required mode still needs a valid HMAC, mitigating prod. | low | +| `_enqueue_or_send` legacy fallback creates unbounded fire-and-forget tasks | websocket.py:266 | For an unregistered socket (shouldn't happen since every `connect_*` registers), a `_send_with_timeout` task is spawned per message with no queue cap. Only the legacy path; held in `_pending_sends` for GC safety. | low | +| `broadcast_notification` double-iterates notification connections | websocket.py:665-691 | It reads `manager.notification_connections.get(agent_id)` then calls `manager._enqueue_or_send` per conn, bypassing the `broadcast_to_*` serializer. If a disconnect happens between the snapshot and the enqueue, the enqueue targets a dead socket whose sender was cancelled — `_enqueue_or_send` falls to the legacy fire-and-forget path. Low. | low | +| Stale module docstring misleads future edits | websocket.py:8-13 | Describes query-param agent validation as the security model; actual model is CEO HMAC. A future edit trusting the docstring could weaken the gate. | low | + +## Health + +This slice is the API spine and is in **good structural shape**. The baseline→head refactor (`15effce0`) materially hardened it: the WebSocket fan-out no longer back-pressures on a slow client (bounded per-connection queues + sender tasks), half-open sockets are reaped by an idle timeout, the lifespan shutdown ordering fix stops silent final-write drops, 422 logs no longer leak credentials, and the WS + HTTP panel-token gates close the operator-only invariant. Post-snapshot, `536bbb64` closed the remaining medium risk by gating `/ws/system` (all `/ws/*` endpoints are now consistently gated — five then, three after the comms-subsystem teardown) and added `require_ceo_role` as a single-source CEO check; `76ce53e3` wired the live message-delivery path (`_handle_message_event`, since removed by the teardown). The event-bus bridge is clean and follows the documented `_handle_*` extension pattern. The main remaining integrity concerns are minor: the `websocket.py` module docstring is stale vs the HMAC implementation, and the concurrent-set-mutation window between the sender task's error-path `disconnect` and the receive loop's `finally disconnect` is theoretically present under asyncio single-threadedness. No logic-touching change since baseline looks broken; the regression risks above are edge-case and mostly low severity. The unchanged files (`middleware_docs.py`, `utils/*`) are stable; `websocket_bridge.py` gained `_handle_message_event` (`76ce53e3`, since removed by the comms-subsystem teardown — the surviving forwarder is `_handle_a2a_message_event`). Recommended follow-ups: refresh the `websocket.py` docstring, and consider a lock or snapshot-iteration in `ConnectionManager` broadcast paths. +# Slice: api-routes-schemas + +## Purpose +The FastAPI surface of RoboCo: every HTTP route under `roboco/api/routes/` (the operator/panel `api/*` CRUD + dashboard/orchestrator/a2a/live bridges) and the agent-gateway `api/v1/flow/*` (intent verbs) + `api/v1/do/*` (content tools), with Pydantic request/response schemas under `roboco/api/schemas/`. Routes are thin handlers that resolve services via `Depends` and return typed responses; all agent-gateway verbs funnel through the Choreographer. + +## Files + +| Path | Role | +|------|------| +| roboco/api/routes/health.py | Liveness/readiness (DB + Redis probes). | +| roboco/api/routes/agents.py | List/get agents. | +| roboco/api/routes/notifications.py | Notification list/ack/send. | +| roboco/api/routes/stream.py | Agent stream chunk/complete/extract + permissions. | +| roboco/api/routes/journals.py | Journal entries, search, growth stats. | +| roboco/api/routes/kanban.py | Per-team kanban boards + main-pm/board/stats. | +| roboco/api/routes/cockpit.py | Cockpit summary/signals. | +| roboco/api/routes/company_goals.py | Company goals get/put. | +| roboco/api/routes/settings.py | Settings + feature-flags get/set. | +| roboco/api/routes/dashboard.py | CEO/auditor/kanban/metrics/agents/activity dashboards. | +| roboco/api/routes/tasks.py | Task CRUD + lifecycle transitions (claim/start/verify/qa/complete...). | +| roboco/api/routes/work_session.py | Work-session list/commit/files/PR/merge/complete/abandon. | +| roboco/api/routes/git.py | Per-project git status/log/diff/commit/push/PR/rebase/branch-cleanup sweep. | +| roboco/api/routes/project.py | Project CRUD + workspace/sync/access + conventions. | +| roboco/api/routes/product.py | Product CRUD. | +| roboco/api/routes/optimal.py | RAG: kb/search, rag/query, mentor/ask, learnings, decisions, review. | +| roboco/api/routes/research.py | Web search/fetch. | +| roboco/api/routes/orchestrator.py | CEO-gated spawn/stop/resolve-wait/mark-waiting + status. | +| roboco/api/routes/a2a.py | Agent-to-agent inbox/conversations/tasks + SSE streams. | +| roboco/api/routes/prompter_live.py | Live Intake chat (start/stream/messages/confirm/confirm-batch). | +| roboco/api/routes/secretary.py | Company state + CEO directives confirm/reject. | +| roboco/api/routes/secretary_live.py | Live Secretary chat (start/stream/messages/stop/events). | +| roboco/api/routes/release.py | CEO-only release proposal approve/reject. | +| roboco/api/routes/playbooks.py | Playbook approve/reject/archive (Auditor/CEO). | +| roboco/api/routes/pitch.py | Pitch create/list/approve/reject. | +| roboco/api/routes/provider.py | Provider catalog + ollama/grok/self-hosted key + mode. | +| roboco/api/routes/usage.py | Token usage summary/time-series/by-agent/team/model/role/sessions, cache-efficiency, spawn-waste (per-role unproductive-spawn rate + respawn strikes). | +| roboco/api/routes/system.py | System-wide info. | +| roboco/api/routes/docs.py | Project docs write/read/list/delete. | +| roboco/api/routes/x.py | X (Twitter) engine — CEO-only: list/approve/reject held draft posts + set/status OAuth 1.0a credentials. | +| roboco/api/routes/roadmap.py | Board roadmap engine — CEO-only: list open cycles + per-item approve/reject. | +| roboco/api/routes/telegram.py | Telegram credentials CRUD (CEO-only, write-only) + `webapp_auth_router` — a separate public, pre-auth `POST /webapp-auth` mounted only when `telegram_miniapp_enabled` AND `cloud_auth_enabled` are both armed (`mount_telegram_miniapp_auth`); validates a Mini App's `initData` and mints the cloud-auth session cookie; adds its own unconditional `LoginRateLimiter`. | +| roboco/api/auth/ | Cloud auth (FastAPI Users, default off): `backend.py` (cookie transport + password-fingerprint-bound JWT strategy), `manager.py` (`UserManager` + DI chain), `session.py` (`resolve_session_user`, shared by the HTTP dual-path and the WS panel-token gate), `seed.py` (idempotent single seeded CEO login upsert), `routes.py` (always-public `/auth/status` + conditional login/logout mount), `login_limit.py` (`LoginRateLimiter` — per-IP POST rate limit, path-keyed via a `paths: tuple[str, ...]` set so `/login` and `telegram.py`'s `/webapp-auth` get independent buckets). | +| roboco/api/routes/v1/_role_dep.py | Per-role HMAC guards + `envelope_to_response` helper. | +| roboco/api/routes/v1/do.py | Content verbs `/api/v1/do/*` (commit/note/say/dm/evidence/playbook...). | +| roboco/api/routes/v1/flow_dev.py | Developer flow verbs. | +| roboco/api/routes/v1/flow_qa.py | QA flow verbs (claim/pass/fail_review). | +| roboco/api/routes/v1/flow_doc.py | Documenter flow verbs. | +| roboco/api/routes/v1/flow_cell_pm.py | Cell-PM flow verbs (delegate/submit_up/triage/complete...). | +| roboco/api/routes/v1/flow_main_pm.py | Main-PM flow verbs (submit_root/triage_all/escalate_to_ceo...). | +| roboco/api/routes/v1/flow_board.py | Board (product_owner/head_marketing) triage/escalate_to_ceo. | +| roboco/api/routes/v1/flow_auditor.py | Auditor triage/i_am_idle. | +| roboco/api/routes/v1/flow_pr_reviewer.py | PR-reviewer verbs incl. gate pr_pass/pr_fail. | +| roboco/api/schemas/*.py | Per-domain Pydantic request/response models (one per route file). | +| roboco/api/schemas/v1/flow.py | All flow-verb request bodies + `StrList` coercion validator. | +| roboco/api/schemas/v1/do.py | All do-verb request bodies. | + +## Key Endpoints + +| Method | Path | Handler (file) | Auth/Role | +|--------|------|----------------|-----------| +| GET | /api/health, /api/ready | health.py | none | +| GET | /api/dashboard/ceo | dashboard.py | agent context | +| GET | /api/dashboard/metrics/{cycle-time,bottlenecks,rework,scorecard/*,member/{id},member/ceo,members} | dashboard.py | agent context — `members` (no `{id}`) is the batch scorecard fetch replacing N per-agent calls | +| GET/POST/PATCH/DELETE | /api/tasks, /api/tasks/{id}/{claim,start,verify,submit-qa,pass-qa,fail-qa,complete,cancel,escalate-to-ceo} | tasks.py | agent context + `require_task_action` | +| GET | /api/tasks/summary?q= (list_tasks_summary) | tasks.py | agent context — trimmed list-view rows; server-side title/description/id-prefix search via `TaskService.search_tasks` when `q` is set (wave 1, `d1cf6ecb`) | +| GET/POST | /api/orchestrator/{status,agents/{id},waiting} ; /spawn,/stop,/resolve-wait,/mark-waiting | orchestrator.py | `_require_ceo` (HMAC) | +| POST | /api/a2a/{send,send-stream} ; /chat/conversations ; /tasks/{id}/cancel | a2a.py | `require_any_authenticated_agent` | +| GET/POST | /api/a2a/chat/admin/{conversations,pairs,conversations/{id}/messages,conversations/{id}/reply} | a2a.py | `_require_ceo` (org-wide live view + reply-as-CEO; wave 2 `da563487` / wave 2c `876e19b3`) | +| POST | /api/prompter/live, /live/{id}/{stream,status,messages,stop,confirm,confirm-batch} | prompter_live.py | `require_panel_token` (CEO HMAC) | +| GET | /api/prompter/live/{id}/search-tasks | prompter_live.py | session-aliveness check (no agent identity) — intake's `search_past_tasks` tool (wave 1, `d1cf6ecb`) | +| POST | /api/secretary/live, /live/{id}/{stream,messages,stop,events} ; /api/secretary/{state,directives} | secretary*.py | panel token / agent ctx | +| GET | /api/secretary/tasks?q= (search_tasks) | secretary.py | agent ctx, Secretary or CEO role — resolve a task NAME to id(s) for a directive (wave 1, `d1cf6ecb`) | +| GET/POST | /api/release/proposal, /proposal/approve, /proposal/reject | release.py | `_require_ceo` (agent.role==CEO) | +| GET/POST | /api/playbooks, /{id}/{approve,reject,archive} | playbooks.py | agent context (Auditor/CEO) | +| GET/POST | /api/x/posts, /posts/{id}/{approve,reject}, /credentials | x.py | `require_ceo_role` (agent context) | +| GET/POST | /api/roadmap/cycles, /cycles/{id}/items/{id}/{approve,reject} | roadmap.py | `require_ceo_role` (agent context) | +| GET/POST | /api/telegram/credentials | telegram.py | `require_ceo_role` (agent context) | +| POST | /api/telegram/webapp-auth | telegram.py | public, pre-auth — Telegram `initData` HMAC validation; mounted only when `telegram_miniapp_enabled` AND `cloud_auth_enabled` | +| GET | /api/telegram/today | telegram.py | `require_ceo_role` (agent context) + 30/60s rate limit — Mini App V4's "Today" brief, backed by `TgCockpitService.today()` (one DB round trip, see `docs/map/notification.md`) | +| GET/POST | /api/auth/status (always), /auth/login, /auth/logout (mounted only when `cloud_auth_enabled`) | auth/routes.py | none (status) / FastAPI Users cookie login | +| GET/POST/PUT/DELETE | /api/projects, /{id}/conventions, /workspace, /sync | project.py | agent context | +| POST | /api/git/branches/cleanup | git.py | agent context, PM/CEO role-gated like `/rebase`; rate-limit 5/60 — cursor-resumable stale-branch sweep, `GitBranchCleanupRequest`/`Response` (wave 2, open PR #548) | +| POST | /api/v1/flow/developer/{give_me_work,i_will_work_on,open_pr,i_am_done,unclaim,resume,sync_branch} | flow_dev.py | `require_dev` (role + HMAC) | +| POST | /api/v1/flow/qa/{claim_review,pass_review,fail_review} | flow_qa.py | `require_qa` | +| POST | /api/v1/flow/cell_pm/{delegate,submit_up,complete,triage,unblock,reassign} | flow_cell_pm.py | `require_cell_pm` | +| POST | /api/v1/flow/main_pm/{submit_root,triage_all,escalate_to_ceo,complete} | flow_main_pm.py | `require_main_pm` | +| POST | /api/v1/flow/pr_reviewer/{claim_pr_review,claim_gate_review,pr_pass,pr_fail,post_pr_review} | flow_pr_reviewer.py | `require_pr_reviewer` | +| POST | /api/v1/do/{commit,note,say,dm,notify,evidence,draft_playbook,approve_playbook,...} | do.py | `require_any_authenticated_agent` (HMAC, any role) | +| GET | /ws/{agents,notifications,system}/{id} | websocket.py | WS panel/HMAC token | + +## Key Symbols + +| Name | Kind | File:Line | Responsibility | +|------|------|-----------|----------------| +| `require_any_authenticated_agent` | dep | v1/_role_dep.py | HMAC-verify X-Agent-ID/role/team token; router-level guard on do + a2a. | +| `require_` (require_dev/qa/...) | dep | v1/_role_dep.py | Per-role guard: HMAC + role assertion, applied as router dependency. | +| `envelope_to_response` | fn | v1/_role_dep.py | Convert Choreographer `Envelope` to JSON, set status from `envelope.status`. | +| `_check_agent_auth_token` | fn | api/deps.py:217 | Core HMAC verify; rejects invalid tokens even in dev; required-only in prod. | +| `require_panel_token` | dep | api/deps.py:251 | CEO-signed HMAC gate for live-chat bridges (HTTP analog of WS gate). | +| `CurrentAgentContext` | dep | api/deps.py:376 | Resolves agent from headers + HMAC, injects `AgentContext`. | +| `_require_ceo` | dep | routes/orchestrator.py:37 | Router-level CEO-HMAC guard on orchestrator control routes. | +| `_validated_agent_id` | fn | routes/orchestrator.py:99 | Path-injection guard (rejects empty/`.`/`..`/`/`/`\`/NUL) then normalizes via `_resolve_to_slug` — spawn/stop/status/resolve-wait/mark-waiting accept either a DB UUID or a slug and address the runtime container by the resolved slug; an unknown UUID passes through unchanged. | +| `setup_middleware` | fn | api/middleware.py | Register exception handlers (422 scrub, HTTP, RobocoError, generic). | +| `request_validation_handler` | fn | api/middleware.py:407 | Log 422 body (secrets scrubbed) + uuid remediate hint. | +| `_scrub_secrets` | fn | api/middleware.py:389 | Deep-redact known secret fields from logged 422 bodies. | +| `StrList` | type | schemas/v1/flow.py:20 | `list[str]` with `coerce_str_list` BeforeValidator (XML-nested LLM lists). | +| `Choreographer` | svc | services/gateway/choreographer.py | Composes service intents behind every flow verb. | +| `ContentActions` | svc | services/gateway/ | Composes do-verb content actions (commit/note/say/...). | +| `router` (do) | router | v1/do.py:38 | `/api/v1/do` router, `require_any_authenticated_agent` dep. | +| `router` (flow_dev) | router | v1/flow_dev.py:24 | `/api/v1/flow/developer` router, `require_dev` dep. | + +## Data Flow +Request hits nginx (port 3000) -> FastAPI app (`api/app.py`) registers routers under `/api/*` plus `/api/v1/flow/*` and `/api/v1/do/*`. Middleware chain (CorrelationId -> RequestLogging) attaches a correlation ID and logs; exception handlers intercept 422/HTTP/RobocoError/generic. Router-level `Depends` resolves `DbSession` + agent context (HMAC-verified from `X-Agent-*` headers) and, on agent-gateway routes, the role guard. The thin handler pulls a service via `Depends` (TaskService, Choreographer, ContentActions, GitService, OptimalService, ReleaseProposalService...) and returns a typed Pydantic response; flow/do verbs return the Choreographer `Envelope` via `envelope_to_response`. SSE (`EventSourceResponse`) is used for live-chat streams and a2a send-stream. + +## Mermaid +```mermaid +graph TD + app[FastAPI app.py] + app -->|/api/*| ops[Operator/Panel routes] + app -->|/api/v1/flow/*| flow[Flow routers] + app -->|/api/v1/do/*| do[do router] + app -->|/ws/*| ws[websocket.py] + ops --> tasks[tasks.py -> TaskService] + ops --> dash[dashboard.py -> MetricsService] + ops --> orch[orchestrator.py -> AgentOrchestrator CEO-gate] + ops --> a2a[a2a.py -> A2AService SSE] + ops --> livep[prompter_live.py -> PrompterService SSE panel-token] + ops --> lives[secretary_live.py -> SecretaryService SSE] + ops --> rel[release.py -> ReleaseProposalService CEO-gate] + flow --> fdev[flow_dev -> Choreographer.give_me_work/i_will_work_on/...] + flow --> fqa[flow_qa -> Choreographer.claim_review/pass/fail] + flow --> fpm[flow_cell_pm/main_pm -> Choreographer.delegate/submit_up/submit_root] + flow --> fpr[flow_pr_reviewer -> Choreographer.pr_pass/pr_fail] + do --> doR[do.py -> ContentActions.commit/note/say/dm/evidence] + flow -.->|HMAC role guard| _role_dep[_role_dep.py] + do -.->|HMAC any-role guard| _role_dep + orch -.->|HMAC CEO guard| deps[deps._require_ceo] + livep -.->|panel HMAC| deps2[deps.require_panel_token] + ws --> cm[ConnectionManager -> StreamEventBus] +``` + +## Logical Tree +``` +roboco/api/ +├── routes/ +│ ├── operator-panel (api/*) +│ │ ├── health.py liveness/readiness +│ │ ├── agents.py agent list/get +│ │ ├── notifications.py notification ack/send +│ │ ├── stream.py agent stream chunks/extract +│ │ ├── journals.py journal entries + growth +│ │ ├── kanban.py kanban boards +│ │ ├── cockpit.py cockpit summary/signals +│ │ ├── company_goals.py company goals +│ │ ├── settings.py settings + feature-flags +│ │ ├── dashboard.py CEO/auditor/metrics dashboards +│ │ ├── tasks.py task CRUD + lifecycle +│ │ ├── work_session.py work-session/PR/merge +│ │ ├── git.py per-project git ops +│ │ ├── project.py project CRUD + conventions +│ │ ├── product.py product CRUD +│ │ ├── optimal.py RAG kb/query/mentor +│ │ ├── research.py web search/fetch +│ │ ├── docs.py project docs +│ │ ├── system.py system info +│ │ └── usage.py token usage +│ ├── ceo-gated / live bridges +│ │ ├── orchestrator.py CEO spawn/stop/mark-waiting +│ │ ├── release.py release proposal approve/reject +│ │ ├── playbooks.py playbook curation +│ │ ├── pitch.py pitch approve/reject +│ │ ├── x.py X engine post queue approve/reject + credentials +│ │ ├── roadmap.py board roadmap cycle item approve/reject +│ │ ├── telegram.py credentials CRUD + webapp-auth (Mini App initData → session cookie) +│ │ ├── a2a.py agent-to-agent + SSE +│ │ ├── prompter_live.py live Intake chat +│ │ ├── secretary.py company state + directives +│ │ ├── secretary_live.py live Secretary chat +│ │ └── provider.py provider catalog/keys +│ └── v1/ (agent-gateway) +│ ├── _role_dep.py HMAC role guards + envelope helper +│ ├── do.py /api/v1/do/* content verbs +│ ├── flow_dev.py developer flow verbs +│ ├── flow_qa.py QA flow verbs +│ ├── flow_doc.py documenter flow verbs +│ ├── flow_cell_pm.py cell-PM flow verbs +│ ├── flow_main_pm.py main-PM flow verbs +│ ├── flow_board.py board flow verbs +│ ├── flow_auditor.py auditor flow verbs +│ └── flow_pr_reviewer.py PR-reviewer flow verbs +├── auth/ (cloud auth, default off — ROBOCO_CLOUD_AUTH_ENABLED) +│ ├── backend.py cookie transport + password-fingerprint-bound JWT strategy +│ ├── manager.py UserManager + get_user_db/get_user_manager DI chain +│ ├── session.py resolve_session_user (shared HTTP + WS cookie validation) +│ ├── seed.py ensure_seed_user / ensure_seed_user_startup (single CEO row) +│ ├── routes.py always-public /status + conditional login/logout mount +│ └── login_limit.py LoginRateLimiter (per-IP POST limit; path-keyed, shared with telegram.py webapp-auth) +└── schemas/ + ├── *.py per-domain Pydantic models + └── v1/ + ├── flow.py flow-verb bodies + StrList + └── do.py do-verb bodies +``` + +## Dependencies +- FastAPI + sse-starlette (SSE), pydantic v2. +- `roboco/api/deps.py` — shared deps (DbSession, agent context, HMAC, orchestrator). +- `roboco/api/middleware.py` — exception handlers + correlation/log middleware. +- `roboco/services/*` — TaskService, GitService, OptimalService, AgentOrchestrator, Choreographer, ContentActions, ReleaseProposalService, PrompterService, SecretaryService, MetricsService, XPostService, XCredentialsService, RoadmapService, etc. +- `roboco/api/auth/*` (`auth_backend`, `get_user_manager`, `resolve_session_user`, `mount_cloud_auth`) — cloud auth, consumed by `deps.get_agent_context`'s dual-path and the WS panel-token gate. +- `roboco/foundation/identity.py` (`Role`) + `roboco/agents_config.py` (`verify_agent_token`, `CEO_AGENT_ID`). +- `roboco/api/websocket.py` + `websocket_bridge.py` (WS event forwarding). + +## Entry Points +- `roboco/api/app.py` `create_app()` builds the FastAPI app, mounts all routers under `/api` (prefix) + `/ws` (WS router). +- `roboco/api/routes/v1/_role_dep.py` is imported by every flow router + do + a2a for HMAC/role guards and `envelope_to_response`. +- `roboco/api/routes/orchestrator.py` router constructed with `dependencies=[Depends(_require_ceo)]` (router-wide CEO gate). + +## Config Flags +- Auth-gate mode: `_auth_required()` (env-driven; HMAC mandatory in prod-ish, optional in dev) — `api/deps.py`. +- Feature-flag routes are inert when their backing engine is off: `release.py` (ROBOCO_RELEASE_MANAGER_ENABLED), `prompter_live.py` MegaTask batch, `optimal.py` learnings (ROBOCO_ORG_MEMORY_ENABLED), `research.py` (ROBOCO_RESEARCH_ENABLED), `provider.py` grok/self-hosted (ROBOCO_GROK / self-hosted), CI-watch/dep-update originate elsewhere but surface via orchestrator/tasks. +- `telegram.py`'s `webapp_auth_router` doesn't merely no-op off — the route doesn't exist at all unless `telegram_miniapp_enabled` AND `cloud_auth_enabled` are both true (`mount_telegram_miniapp_auth`, called from `app.py`, mirrors `mount_cloud_auth`'s conditional mount). + +## Gotchas +- `do` + `a2a` routers are token-only (any authenticated role), not role-asserted — any signed agent can call any content verb; service-layer scope is the only gate. +- `request_validation_handler` scrubs secrets from the **log** but the 422 **response body echoes the client's submission unchanged** (comment explicit) — secrets can still leak to the caller if the caller is not the legitimate owner. +- SSE live-chat bridges open one session per query/stream and rely on `require_panel_token` (CEO HMAC injected by nginx); a missing/invalid token in dev mode is tolerated (`_auth_required()` false) — prod must arm it. +- `StrList` BeforeValidator is load-bearing: without it the Claude SDK's XML-nested list input crashes `i_will_plan`/`delegate` with 422 (MegaTask memory Bug 3). +- `orchestrator.py` and `release.py` use two different `_require_ceo` implementations (HMAC header vs `agent.role==CEO` from context) — keep their semantics aligned. +- WS endpoints live on `/ws/*` (separate router in `websocket.py`), not under `/api`; the bridge subscribes to `StreamEventBus` and forwards per resource-id. +- `/api/tasks` PATCH is not a single admin surface: `_pm_editor_scope` (tasks.py:256) routes cell_pm/main_pm to a content-only allowlist (`_PM_LIGHTER_UPDATE_FIELDS`: title/description/acceptance_criteria/priority, zero status changes) enforced by `_enforce_pm_lighter_fields` (tasks.py:278), while CEO/Board/Auditor keep the unrestricted admin bypass; a cell_pm editing a task outside its own team 403s before the field check even runs. +- `GET /api/tasks/summary` and `GET /api/secretary/tasks` both call the same `TaskService.search_tasks` (ILIKE title/description + id-prefix) but through different auth (agent-context view-scope vs Secretary-or-CEO role check) and different response shapes (trimmed `TaskSummaryResponse` vs a hand-built dict list) — don't assume one route's pagination/limit semantics apply to the other. + +## Drift from CLAUDE.md +- CLAUDE.md lists `pr_pass`/`pr_fail` under `pr_reviewer` verbs and the in-path gate; code matches (`flow_pr_reviewer.py` exposes `claim_gate_review`, `pr_pass`, `pr_fail`). No drift found. +- CLAUDE.md says agent comms use `dm`/`read_a2a`/`notify` via do_server (the channel/session `say`/`open_session`/`link_session` surface was removed in the comms-subsystem teardown); code matches. No drift. +- CLAUDE.md lists `sync_branch` as a developer verb; present in `flow_dev.py:125`. No drift. +- CLAUDE.md's verb table omits `flow_pr_reviewer.post_pr_review` (external PR comment) — present in code; additive, not contradictory. +- None material. + +## Changes Since Baseline +`git log fd10cc86..HEAD -- roboco/api/routes/ roboco/api/schemas/`: +- `15effce0` Chore: 141 Gaps fill-in (#283) — broad route/schema hardening pass (the only logic-touching commit in range at the time this section was last refreshed). + +> Post-snapshot: many further commits touch this slice (536bbb64, df87fcf0, a8cb2470, 0ca9d91b, cfde4369, 0f1ed3cc, 1c87a4e4, and the three below) — only the wave-1/2/2c ones relevant to this pass are itemized; a full re-audit of the intervening route/schema history is still owed. +> - `d1cf6ecb` Wave 1 (#295) — adds `GET /api/tasks/summary?q=` search (`TaskService.search_tasks`), `GET /api/prompter/live/{id}/search-tasks` (intake memory), `GET /api/secretary/tasks?q=` (Secretary task-by-name lookup), and the Secretary `edit` directive action. +> - `da563487` Wave 2 (#297) — adds the CEO-only `/api/a2a/chat/admin/{conversations,conversations/{id}/messages,conversations/{id}/reply}` routes (`_require_ceo`) for the A2A live view + reply-as-CEO. +> - `876e19b3` Wave 2c (#298) — adds `/api/a2a/chat/admin/pairs` (the switchboard, same `_require_ceo` gate); tightens `/api/tasks` PATCH so cell/main PM roles get a content-only field allowlist instead of the unrestricted CEO/Board/Auditor admin bypass (`_pm_editor_scope` / `_enforce_pm_lighter_fields`, `roboco/api/routes/tasks.py:256,278`) — closes an over-permission hole where PM identities could edit any-team tasks via the ASSIGN-holding bypass. +> - `637c75dc` (2026-07-17, PR #546, "wave-1 quick wins") fix(api): normalize agent UUID to slug at the orchestrator route boundary — `_validated_agent_id` now also calls `_resolve_to_slug` after its path-injection checks, so a caller-supplied DB UUID (e.g. from the panel) resolves to the canonical slug before spawn/stop/status/resolve-wait/mark-waiting address the runtime, fixing UUID-named containers and registry misses. +> - `496c24d1` (PR #548, "git hygiene", 2026-07-17) adds `POST /api/git/branches/cleanup` (PM/CEO role-gated like `/rebase`, rate-limit 5/60) + `GitBranchCleanupRequest`/`GitBranchCleanupResponse` schemas — cursor-resumable sweep of terminal tasks' remote+local branches, backing a confirm-dialog button on the panel Git page. +> - `82642bea`+`e16fb634`+`8d727785` (2026-07-18, PR #554, Telegram V3 Mini App) adds `POST /api/telegram/webapp-auth` (`webapp_auth_router`, mounted only when `telegram_miniapp_enabled` AND `cloud_auth_enabled` via `mount_telegram_miniapp_auth`) + `TelegramWebAppAuthRequest` schema, exchanging a validated Telegram `initData` payload for the same cloud-auth session cookie `/api/auth/login` mints (binds to the CEO's stored `chat_id`, audits `telegram.webapp.login`); generalizes `LoginRateLimiter` from a single `prefix` to a `paths: tuple[str, ...]` set (`roboco/api/auth/login_limit.py`) so `/webapp-auth` gets its own unconditional per-IP bucket, independent of the guard middleware's `rate_limit` decorator; the fix commit also rejects a far-future `initData.auth_date` (only ±60s clock-skew tolerated) and anchors the panel's `/tg` matcher exclusion. +> - `baa87d58` (2026-07-19, PR #576, Telegram Mini App V4) adds `GET /api/telegram/today` (`require_ceo_role` + 30/60s rate limit) backed by new `TgCockpitService` + the `TelegramTodayResponse`/`TodayNeedsYou`/`TodayFleet`/`TodaySpend`/`TodayVelocity`/`TodayShip` schema family in `api/schemas/telegram.py` — see `docs/map/notification.md` for the service, `docs/map/panel.md` for the cockpit's Today tab. +> - `461a6e1a`+`96401f4c`+`5f32d876` (2026-07-18/19, forge Phases 1-4, #571/#575/#581) — no new HTTP routes (the forge routing is internal to `GitService`), but `roboco/api/schemas/project.py`/`project_fields.py` gain `git_provider` (project CRUD schemas) and the shared `task_project_fields` helper the X/video routes now call — see `docs/map/worksession-git.md` and `docs/map/product-strategy-research-pitch.md`. +> - ("panel-perf-p3-p4") adds `GET /api/dashboard/metrics/members` (batch scorecard fetch) — see `docs/map/metrics-observability.md`. + +## Regression Risks + +| Title | File:Line | Claim | Severity | +|-------|-----------|-------|----------| +| do/a2a any-role token gate | v1/do.py:43, a2a.py:114 | `require_any_authenticated_agent` only verifies HMAC + that the agent exists; it does NOT assert the role matches the verb's intended role family — a QA-signed token could call `do/commit`, or any agent could call the participant-scoped `a2a` routes (send/conversations) for a pair it has no policy access to (only the gateway's `can_a2a_direct`/`validate_a2a_access` matrix, a service-layer check, stops it). Service-layer scope is the sole guard on these paths; a missed service check = privilege escape. **Correction:** the `/chat/admin/*` routes (org-wide live view + reply-as-CEO) are NOT on this gate — they carry their own router-level `_require_ceo` guard, added in wave 2 (`da563487`) and extended to `/chat/admin/pairs` in wave 2c (`876e19b3`); a non-CEO agent 403s before reaching the service layer on those. | High | +| 422 response echoes secrets | middleware.py:407 | `_scrub_secrets` redacts only the **log** body; the JSON response still contains `body` with the caller's original secret fields. A 422 on `git_token`/`api_key` returns the secret back to the client (and to any MITM/log of the response). | High | +| orchestrator CEO gate vs release CEO gate divergence | orchestrator.py:37 vs release.py:32 | Two independent `_require_ceo` implementations: orchestrator uses HMAC header verification, release uses `agent.role == CEO` from `CurrentAgentContext`. If one path's HMAC/context resolution drifts, the two CEO surfaces enforce different identities. | Medium | +| SSE transport errors swallowed | prompter_live.py:122, secretary_live.py:61, a2a.py:195 | `EventSourceResponse` streams run long-lived; a Choreographer/orchestrator raise mid-stream is caught by `contextlib` suppress but can drop the stream silently without a terminal event to the panel. | Medium | +| Cross-repo PR collision via /api/work-sessions/{id}/pr/merge | work_session.py:259 | PR merge by global `pr_number` (no project_id scoping in the route signature) — the same class of cross-repo collision already fixed in `cell_pm_complete` could recur if this endpoint is wired to merge. | Medium | +| Dashboard/metrics endpoints role-gating | dashboard.py:58+ | `/ceo`, `/auditor`, `/scorecard/*` rely on `CurrentAgentContext` but the route-level gating is weak (no explicit `require_pm_or_above`); a non-CEO agent calling `/dashboard/ceo` is filtered only by service-layer logic, not the router. | Medium | +| WS panel-token vs agent-token dual gate | websocket.py / deps.py | `/ws/*` endpoints use a WS-specific `_require_panel_token` for panel streams but agent-id keying for `/ws/agents/{id}`; mismatched HMAC secret rotation between the two could grant panel read of agent streams or vice-versa. | Low-Med | +| flow `i_will_plan` StrList crash recurrence | schemas/v1/flow.py:20 | If a new LLM-authored `list[str]` field is added to a flow schema without `StrList`, the SDK XML-nesting crash reappears (silent 422 loop). Reviewer-only by inspection. | Low-Med | + +## Health +The route layer is thin, consistently organized (one router per domain, one schema file per router), and the agent-gateway HMAC guard is centralized in `_role_dep.py` + `deps.py`. Main risks are the any-role `do`/`a2a` gate (relies on service-layer scope), the 422 response echoing secrets, and the two divergent CEO guards — all addressable without structural change. SSE live-chat streams are the fragile transport path. +# RoboCo Slice Map — `mcp-servers` + +Scope: `roboco/mcp/` (every server file + `schemas/` + `utils.py`). Repo root: `/Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco`. + +## Purpose + +The `roboco/mcp` package is the agent-side MCP gateway: a set of `FastMCP` server processes that run **inside each agent container** and expose the RoboCo intent-verb / content-tool / RAG / docs / git-readonly / intake / secretary / web-research surfaces to the Claude Code or grok-CLI runtime as MCP tools. They are thin bridges — every tool either POSTs to the orchestrator's HTTP gateway (`/api/v1/flow/*`, `/api/v1/do/*`, `/api/git/*`, `/optimal/*`, `/docs/*`, `/research/*`, `/api/secretary/*`, `/api/prompter/live/*`) or, for the flow/do path, additionally forwards rejections to a per-container SDK loopback (`ROBOCO_SDK_URL`) that runs the per-verb circuit breaker. The orchestrator (not the MCP layer) is the authority for role scoping, state transitions, and git-side effects; the MCP layer only shapes calls, classifies rejections, and substitutes `circuit_open` envelopes when the breaker trips. + +## Files + +| Path | Role | approx LOC | +|------|------|------------| +| `roboco/mcp/__init__.py` | Package docstring only — deliberately import-free so `python -m roboco.mcp.` does not pull sibling modules (esp. `optimal_server`'s pgvector/ollama stack, ~6s startup). | 23 | +| `roboco/mcp/utils.py` | Shared HTTP helpers: `_get_agent_headers`, `format_error_response`, `ApiResponse`, `ApiClient` (async httpx wrapper). Used by `optimal_server`, `docs_server`, `search_server`. | 383 | +| `roboco/mcp/schemas/__init__.py` | Pydantic input models. After Phase-4 T9 deletions only `WriteDocInput` remains. | 35 | +| `roboco/mcp/flow_server.py` | `roboco-flow` MCP server — intent verbs (lifecycle). Manifest-scoped registration, role-scoped path `/api/v1/flow//`, per-verb circuit breaker + 404-route synthesis. | 1028 | +| `roboco/mcp/do_server.py` | `roboco-do` MCP server — content tools (commit, note, pitch, propose_roadmap, dm, notify, evidence, progress, playbook curation, pr_update). Manifest-scoped; fixed `/api/v1/do/` path; mirror circuit breaker. | 976 | +| `roboco/mcp/optimal_server.py` | `roboco-optimal` MCP server — RAG / KB / mentor / error / decision / standards / learnings / index-mgmt / proactive-context. Factory `create_optimal_mcp_server(agent_id)`; calls `/optimal/*`. | 1102 | +| `roboco/mcp/docs_server.py` | `roboco-docs` MCP server — docs write/read/list/delete via `/docs/*`. Factory `create_docs_mcp_server(agent_id)`. RAG-based dedup on write. | 251 | +| `roboco/services/docs.py` | `DocsService` — the `/docs/*` route handlers' backing service; refuses `write_doc(doc_type="user_facing")` with a message naming the deployer's docs-site project/URL | 785 | +| `roboco/mcp/git_readonly.py` | `roboco-git-readonly` MCP server — four read-only git views (status/log/diff/branch list) via `/api/git/*`. No breaker, no manifest. | 123 | +| `roboco/mcp/intake_server.py` | `roboco-intake` MCP server — grok intake path only: `propose_draft` / `propose_batch` POST directly to the prompter-live relay. | 215 | +| `roboco/mcp/secretary_server.py` | `roboco-secretary` MCP server — grok secretary path only: `read_company_state` / `read_task` / `submit_directive`, delegating to `agent_sdk.secretary_driver`. | 64 | +| `roboco/mcp/search_server.py` | `roboco-search` MCP server — `web_search` / `web_fetch` via `/research/*` (provider key stays server-side). Factory `create_search_mcp_server(agent_id)`. | 130 | + +(Excluded: `__pycache__/`, `.DS_Store`.) + +## Key Symbols + +| Name | Kind | File:Line | Responsibility | +|------|------|-----------|----------------| +| `mcp` (flow) | `FastMCP` | `flow_server.py:252` | Server instance `roboco-flow`; tools registered onto it at import time. | +| `mcp` (do) | `FastMCP` | `do_server.py:220` | Server instance `roboco-do`. | +| `mcp` (git-readonly) | `FastMCP` | `git_readonly.py:31` | Server instance `roboco-git-readonly`. | +| `mcp` (intake) | `FastMCP` | `intake_server.py:32` | Server instance `roboco-intake`. | +| `mcp` (secretary) | `FastMCP` | `secretary_server.py:30` | Server instance `roboco-secretary`. | +| `StrList` | type alias | `flow_server.py:39` | `Annotated[list[str], BeforeValidator(coerce_str_list)]` — tolerates Claude SDK's nested XML-ish tool-input shapes before MCP validation rejects. | +| `_CIRCUIT_REJECTION_KINDS` | frozenset | `flow_server.py:66`, `do_server.py:50` | The 4 breaker-counted kinds: `tracing_gap`, `invalid_state`, `not_authorized`, `incomplete_input`. | +| `_DICT_ERROR_CODE_MAP` | dict | `flow_server.py:89`, `do_server.py:73` | Exact code→kind map for known RobocoError codes (e.g. `AUTHENTICATION_REQUIRED`→`not_authorized`). Unknown codes fall through to a substring branch for forward-compat. Added in 536bbb64 to fix AUTHENTICATION_REQUIRED mis-routing (#161). | +| `_classify_dict_error_code` | func | `flow_server.py:110`, `do_server.py:94` | Map a dict-shaped `error.code` to a counted breaker kind: consults `_DICT_ERROR_CODE_MAP` first (exact), then substring fallback for unknown codes; NOT_FOUND → None. | +| `_remediate_for_kind` | func | `flow_server.py:130`, `do_server.py:150` | Synthesize a directed recovery hint string for each counted kind (not_found / incomplete_input / not_authorized / invalid_state). Used by `_normalize_exception_envelope`. | +| `_normalize_exception_envelope` | func | `flow_server.py:164`, `do_server.py:180` | Lift a dict-`error` exception-handler body or 422 `detail` list into Envelope wire format (string kind + message + remediate + missing). Returns None when payload is already a valid Envelope. Added in 0d714b6c (#232). | +| `_classify_rejection` | func | `flow_server.py:216`, `do_server.py:114` | Classify all 3 rejection shapes (string kind / dict error / 422 `detail`) → counted kind or None. Guards the `dict in frozenset` `TypeError`. | +| `_build_headers` | func | `flow_server.py:256`, `do_server.py:224` | Per-call headers: `X-Agent-ID`, `X-Agent-Role`, fresh `X-Correlation-ID` (UUID per MCP call). | +| `_post` (flow) | func | `flow_server.py:272` | POST to orchestrator; normalize exception bodies via `_normalize_exception_envelope`; synthesize `invalid_state` on bare 404 missing route; `not_found` on descriptive 404 detail; `transport_error` on non-JSON; forward rejection to breaker. | +| `_post` (do) | func | `do_server.py:238` | Mirror of flow `_post` for content tools. | +| `_verb_from_path` | func | `flow_server.py:384`, `do_server.py:339` | Extract verb name from path for breaker reporting. | +| `_record_and_check_circuit` | func | `flow_server.py:394`, `do_server.py:349` | Forward a rejection to `SDK_URL/verb/attempted`; if SDK says `open`, replace the payload with `circuit_envelope` (dict-copied, original nested as `inner`, task_id/correlation_id lifted to top level). Best-effort (fail-open). | +| `_ROLE_TO_ROUTE_PREFIX` | dict | `flow_server.py:476` | Maps `product_owner`/`head_marketing` → `board` route segment; every other role passes through unchanged. | +| `_role_path` | func | `flow_server.py:483` | Build `/api/v1/flow//` path. | +| `_TOOLS` (flow) | dict | `flow_server.py:879` | Verb name → Python impl map (27 verbs). `pass`/`fail` keys bridge the `pass_review`/`fail_review` IntentSpec names. | +| `_INTENT_TO_PUBLIC` | dict | `flow_server.py:929` | `pass_review`→`pass`, `fail_review`→`fail` — fixes the dogfood gap where QA tools were silently dropped. | +| `_load_manifest_flow_tools` | func | `flow_server.py:935` | Read `/app/tool-manifest.json` `flow_tools`; None if missing/unreadable. | +| `_register_tools` (flow) | func | `flow_server.py:965` | Raise `RuntimeError` if manifest missing and `ROBOCO_ALLOW_FULL_TOOLSET` not set; if set, registers full tool set as dev/test escape hatch. | +| `_REGISTERED_TOOLS` (flow) | var | `flow_server.py:1024` | Import-time registration side effect. | +| `_TOOLS` (do) | dict | `do_server.py:863` | Tool name → impl map (21 content tools, incl. `propose_roadmap`). | +| `_load_manifest_do_tools` | func | `do_server.py:866` | Read manifest `do_tools` list. | +| `_register_tools` (do) | func | `do_server.py:896` | Manifest-scoped registration; raise `RuntimeError` if manifest missing (unless `ROBOCO_ALLOW_FULL_TOOLSET` set). | +| `give_me_work` … `i_am_idle` | verb funcs | `flow_server.py:491–625` | Dev verbs. | +| `claim_review` / `pass_review` / `fail_review` | verb funcs | `flow_server.py:628–653` | QA verbs (registered as `pass`/`fail`). | +| `claim_pr_review` / `post_pr_review` / `claim_gate_review` / `pr_pass` / `pr_fail` | verb funcs | `flow_server.py:656–876` | PR-reviewer verbs (inbound + in-path gate). | +| `i_will_plan` / `delegate` / `submit_up` / `submit_root` | verb funcs | `flow_server.py:759–856` | PM coordination verbs. | +| `note` | verb func | `do_server.py:428` | Journal entry + handoff section writer (top-level `done`/`next` strings — the meltdown-#1 fix). | +| `commit` / `dm` / `notify` / `evidence` | verb funcs | `do_server.py:423–598` | Core content tools. | +| `propose_roadmap` | verb func | `do_server.py:531` | Product Owner (board-roadmap-only, `_PRODUCT_OWNER_DO`): propose a themed roadmap cycle (goal + 3-7 item drafts) exactly once per exploration task. | +| `draft_playbook` / `approve_playbook` / `reject_playbook` / `archive_playbook` | verb funcs | `do_server.py:600–644` | Playbook curation (delivery + Auditor). | +| `progress` / `notify_list` / `notify_get` / `notify_ack` / `pr_update` / `read_messages` | verb funcs | `do_server.py:646–840` | Wave-1 parity content tools. | +| `create_optimal_mcp_server` | factory | `optimal_server.py:1068` | Build `roboco-optimal-{agent_id}` server; registers 8 tool groups. | +| `roboco_kb_search` / `roboco_rag_query` / `roboco_kb_stats` | tools | `optimal_server.py:70–213` | Search / RAG / stats. | +| `roboco_ask_mentor` | tool | `optimal_server.py:371` | Primary conversational RAG tool (65s timeout). | +| `roboco_search_error` / `roboco_record_error_solution` | tools | `optimal_server.py:439–535` | Error-pattern memory. | +| `roboco_check_decision` / `roboco_record_decision` | tools | `optimal_server.py:542–625` | Decision memory (`RecordDecisionInput` pydantic model at L33). | +| `roboco_get_standards` / `roboco_validate_action` / `roboco_review_code` | tools | `optimal_server.py:632–765` | Standards + validation. | +| `roboco_record_learning` / `roboco_search_learnings` | tools | `optimal_server.py:772–875` | Learnings. | +| `roboco_clear_index` / `roboco_reindex_all` / `roboco_index_status` | tools | `optimal_server.py:882–993` | Index admin. | +| `roboco_get_proactive_context` | tool | `optimal_server.py:1000` | Stored-then-fresh proactive context for a task. | +| `normalize_index_types` | func | `optimal_server.py:53` | Map legacy `docs` alias → `documentation` before route's `IndexType(...)` conversion. | +| `create_docs_mcp_server` | factory | `docs_server.py:160` | Build `roboco-docs-{agent_id}` server (write/read/list/delete). | +| `DocsService.write_doc` | method | `services/docs.py:192` | Refuses `doc_type="user_facing"` writes — normal tasks in the deployer's docs-site project (MDX) are the right home instead | +| `_refused_doc_types` | func | `services/docs.py:84` | Builds the refusal-message dict fresh on every call from `settings.docs_site_project_slug`/`docs_site_public_url` (was a fixed `REFUSED_DOC_TYPES` module constant hardcoding "roboco-website"/"docs.roboco.tech"); falls back to generic "your docs-site project" / "ship on your docs site" text when either setting is left blank | +| `WriteDocInput` | pydantic | `schemas/__init__.py:12` | Docs write input (task_id, filename, doc_type, title, content). | +| `roboco_git_status` / `roboco_git_log` / `roboco_git_diff` / `roboco_git_branch_list` | tools | `git_readonly.py:45–119` | Read-only git views via `/api/git/*`. | +| `propose_draft` / `propose_batch` | tools | `intake_server.py:104–215` | Grok intake: POST draft/batch to prompter-live relay; `propose_batch` drops malformed (no string `title` or `name`) entries via `_normalize_batch_drafts` and refuses empty batches. | +| `post_draft` / `post_batch` / `_post_event` | funcs | `intake_server.py:41–102` | Relay POST helpers (never raise; unit-testable with `httpx.MockTransport`). `_post_event` now captures relay response body under `detail` on non-success so the grok agent sees the real reason (0d714b6c). | +| `_draft_title` | func | `intake_server.py:135` | Extract a string title from a batch draft dict, accepting `title` or `name` key; returns None if neither is a string. | +| `_normalize_batch_drafts` | func | `intake_server.py:146` | Filter + normalize MegaTask batch drafts: drops title-less/name-less entries, normalizes name-only drafts onto `title` key. Returns `(well_formed, dropped_count)`. | +| `read_company_state` / `read_task` / `submit_directive` | tools | `secretary_server.py:33–60` | Secretary CEO-authority tools; delegate to `agent_sdk.secretary_driver`. | +| `ApiClient` | class | `utils.py:115` | Async httpx client with agent headers, base URL `settings.internal_api_url`, `get/post/put/patch/delete` + `*_or_error` tuples. | +| `ApiResponse` | class | `utils.py:82` | Response wrapper (`ok`, `status_code`, `json`, `text`, `is_status`). | +| `_get_agent_headers` | func | `utils.py:27` | `X-Agent-ID`/`X-Agent-Role`/`X-Agent-Team`/`X-Agent-Token` (HMAC token from `ROBOCO_AGENT_TOKEN`). | +| `format_error_response` | func | `utils.py:52` | Wraps `roboco.api.schemas.common.error_response`. | +| `create_search_mcp_server` | factory | `search_server.py:83` | `roboco-search-{agent_id}` with `web_search`/`web_fetch`. | + +## Data Flow + +Every server is launched as its own subprocess (`uv run --no-sync python -m roboco.mcp. [agent_id]`) by the orchestrator's `_generate_mcp_config` (for flow/do/git-readonly/optimal/docs/search) or by the grok intake/secretary mains (for intake/secretary). At import time the flow/do servers read `/app/tool-manifest.json` (env `ROBOCO_TOOL_MANIFEST_PATH`) and register only the verbs/tools listed for the role — refusing to start if the manifest is missing, unless `ROBOCO_ALLOW_FULL_TOOLSET` is set (dev/test only; the all-tools fallback was the original bug that let PMs see dev verbs and 404). The optimal/docs/search/intake/secretary/git-readonly servers register their full tool surface unconditionally (role gating is server-side at the route). + +When an agent calls a tool: + +1. **flow/do** — `_post` builds headers (fresh `X-Correlation-ID` per call), POSTs the JSON body to the orchestrator at `/api/v1/flow//` or `/api/v1/do/`. On 404 (a manifest-advertised verb with no matching route) it synthesizes an `invalid_state` Envelope so the agent gets a `remediate` hint instead of a raw `detail` body. On non-JSON body it synthesizes `transport_error`. Otherwise the Envelope is surfaced as-is (success or rejection). +2. **breaker path** — `_classify_rejection` determines whether the envelope is a counted rejection (string kind / dict `error.code` / 422 `detail` list). If counted, `_record_and_check_circuit` POSTs to the local SDK at `ROBOCO_SDK_URL/verb/attempted` (2s timeout). If the SDK returns `open=true`, the original rejection is **replaced** by the SDK's `circuit_envelope` before returning to the agent — stopping retry storms. SDK unreachable → fail-open (return original payload + log). +3. **optimal/docs/search** — `ApiClient` (async httpx) calls the orchestrator's `/optimal/*`, `/docs/*`, `/research/*` routes with `X-Agent-*` headers; shapes the response into a tool-specific dict (status, results, hints). +4. **git-readonly** — `_get` does a synchronous httpx GET to `/api/git/*` with `X-Agent-ID`/`X-Agent-Role`; `raise_for_status` propagates HTTP errors. +5. **intake** — `propose_draft`/`propose_batch` POST directly to `/api/prompter/live/{session}/events` (the prompter-live relay) because grok's `streaming-json` output does not surface tool-call events. Returns a human-readable string (not an Envelope). +6. **secretary** — the three tools delegate to `agent_sdk.secretary_driver._do_*` helpers (shared with the Claude SDK path) and `json.dumps` the result. + +## Mermaid + +```mermaid +graph LR + subgraph AgentContainer["Agent container (per spawn)"] + CC["Claude Code / grok CLI runtime"] + FS["roboco-flow MCP"] + DS["roboco-do MCP"] + GR["roboco-git-readonly MCP"] + OP["roboco-optimal MCP"] + DOC["roboco-docs MCP (conditional)"] + SRCH["roboco-search MCP (conditional)"] + INT["roboco-intake MCP (grok only)"] + SEC["roboco-secretary MCP (grok only)"] + SDK["per-verb SDK loopback :9000"] + end + + CC -->|MCP tool call| FS + CC -->|MCP tool call| DS + CC -->|MCP tool call| GR + CC -->|MCP tool call| OP + CC -->|MCP tool call| DOC + CC -->|MCP tool call| SRCH + CC -->|MCP tool call| INT + CC -->|MCP tool call| SEC + + FS -->|POST /api/v1/flow//| ORC["Orchestrator HTTP gateway"] + DS -->|POST /api/v1/do/| ORC + GR -->|GET /api/git/*| ORC + OP -->|POST /optimal/*| ORC + DOC -->|POST /docs/*| ORC + SRCH -->|POST /research/*| ORC + INT -->|"POST /api/prompter/live/{s}/events"| ORC + SEC -->|POST /api/secretary/*| ORC + + FS -.->|rejection → /verb/attempted| SDK + DS -.->|rejection → /verb/attempted| SDK + SDK -.->|open=true → circuit_envelope| FS + SDK -.->|open=true → circuit_envelope| DS + + ORC -->|Envelope 2xx/4xx| FS + ORC -->|Envelope 2xx/4xx| DS +``` + +## Logical Tree + +``` +roboco/mcp/ +├── __init__.py # import-free package docstring (avoid sibling-load startup tax) +├── utils.py +│ ├── _get_agent_headers() # X-Agent-ID/Role/Team/Token (HMAC) +│ ├── format_error_response() +│ ├── ApiResponse # ok / status_code / json / text / is_status +│ └── ApiClient # async httpx; get/post/put/patch/delete + *_or_error +├── schemas/__init__.py +│ └── WriteDocInput # only survivor of Phase-4 T9 deletions +├── flow_server.py # roboco-flow (intent verbs) +│ ├── StrList # BeforeValidator(coerce_str_list) — SDK XML-ish input +│ ├── _CIRCUIT_REJECTION_KINDS / _DICT_ERROR_CODE_MAP / _classify_dict_error_code / _classify_rejection +│ ├── _remediate_for_kind / _normalize_exception_envelope +│ ├── _build_headers / _post / _verb_from_path / _record_and_check_circuit +│ ├── _ROLE_TO_ROUTE_PREFIX (PO/HM → board) / _role_path +│ ├── dev verbs: give_me_work, i_will_work_on, open_pr, i_am_done, i_am_blocked, unclaim, reassign, resume, sync_branch, i_am_idle +│ ├── QA verbs: claim_review, pass_review(→pass), fail_review(→fail) +│ ├── PR-reviewer verbs: claim_pr_review, post_pr_review, claim_gate_review, pr_pass, pr_fail +│ ├── Doc verbs: claim_doc_task, i_documented +│ ├── PM verbs: triage, triage_all, unblock, complete, escalate_up, i_will_plan, delegate, submit_up, submit_root +│ ├── Board/Main-PM: escalate_to_ceo +│ ├── _TOOLS / _INTENT_TO_PUBLIC +│ └── _load_manifest_flow_tools / _register_tools (fails loud if no manifest; ROBOCO_ALLOW_FULL_TOOLSET escape hatch) +├── do_server.py # roboco-do (content tools) +│ ├── mirror breaker machinery (_CIRCUIT_REJECTION_KINDS, _DICT_ERROR_CODE_MAP, _classify_*, _remediate_for_kind, _normalize_exception_envelope, _record_and_check_circuit) +│ ├── commit, note (handoff done/next), pitch, propose_roadmap, dm, notify, evidence +│ ├── progress, notify_list/get/ack, pr_update, read_messages +│ ├── draft_playbook, approve_playbook, reject_playbook, archive_playbook +│ └── _TOOLS / _load_manifest_do_tools / _register_tools (fails loud; ROBOCO_ALLOW_FULL_TOOLSET escape hatch) +├── optimal_server.py # roboco-optimal (RAG/KB) +│ ├── RecordDecisionInput, normalize_index_types (docs→documentation) +│ ├── _register_search_tools (kb_search, rag_query, kb_stats) +│ ├── _register_indexing_tools (index_code, index_docs) +│ ├── _register_utility_tools (tokens_estimate) +│ ├── _register_mentor_tools (ask_mentor) +│ ├── _register_error_tools (search_error, record_error_solution) +│ ├── _register_decision_tools (check_decision, record_decision) +│ ├── _register_standards_tools (get_standards, validate_action, review_code) +│ ├── _register_learning_tools (record_learning, search_learnings) +│ ├── _register_index_management_tools (clear_index, reindex_all, index_status) +│ ├── _register_proactive_tools (get_proactive_context) +│ └── create_optimal_mcp_server(agent_id) +├── docs_server.py # roboco-docs +│ ├── _handle_write/read/list/delete +│ └── create_docs_mcp_server(agent_id) +├── git_readonly.py # roboco-git-readonly (4 read-only tools) +├── intake_server.py # roboco-intake (grok only) +│ ├── _post_event / post_draft / post_batch +│ ├── _draft_title / _normalize_batch_drafts # title-or-name filter + name→title normalization +│ └── propose_draft / propose_batch (MegaTask) +├── secretary_server.py # roboco-secretary (grok only; delegates to secretary_driver) +└── search_server.py # roboco-search (web research, Board+PM) + ├── _handle_search / _handle_fetch + └── create_search_mcp_server(agent_id) +``` + +## Dependencies + +**Internal (roboco):** +- `roboco.config.settings` — `internal_api_url` (utils), `research_enabled` (orchestrator mount gate). +- `roboco.agents_config` — `get_agent_role`, `get_agent_team` (utils headers). +- `roboco.api.schemas.common.error_response` (utils `format_error_response`). +- `roboco.foundation.policy.content.validators.coerce_str_list` (flow `StrList`). +- `roboco.agent_sdk.secretary_driver` — `_do_read_state` / `_do_read_task` / `_do_submit_directive` (secretary server). +- `roboco.mcp.schemas.WriteDocInput` (docs server). +- `roboco.mcp.utils.ApiClient` / `format_error_response` (optimal, docs, search). + +**External:** +- `mcp.server.fastmcp.FastMCP` — MCP server framework (all servers). +- `pydantic` (`BaseModel`, `Field`, `BeforeValidator`, `Annotated`) — input validation. +- `httpx` — sync (flow/do/git-readonly) + async (utils ApiClient, intake) HTTP. +- `structlog` — logging (flow/do). +- `fastapi.status` — HTTP status constants (optimal). + +**Downstream consumers:** +- `roboco.runtime.orchestrator._generate_mcp_config` — mounts flow/do/git-readonly/optimal (always), docs (docs_roles), search (research_roles + `research_enabled`). +- `roboco.agent_sdk.grok_intake_main` / `grok_secretary_main` — mount intake/secretary for the grok path. +- `roboco.runtime.spawn_manifest` — writes `/app/tool-manifest.json` (the `flow_tools`/`do_tools` lists the flow/do servers read at import). + +## Entry Points + +- `python -m roboco.mcp.flow_server` — `mcp.run()` at `flow_server.py:1028`. Env required: `ROBOCO_AGENT_ID`, `ROBOCO_AGENT_ROLE`; reads `ROBOCO_ORCHESTRATOR_URL`, `ROBOCO_SDK_URL`, `ROBOCO_TOOL_MANIFEST_PATH`. +- `python -m roboco.mcp.do_server` — `mcp.run()` at `do_server.py:954`. Same env as flow. +- `python -m roboco.mcp.git_readonly` — `mcp.run()` at `git_readonly.py:123`. Env: `ROBOCO_AGENT_ID`, `ROBOCO_AGENT_ROLE`, `ROBOCO_ORCHESTRATOR_URL`. +- `python -m roboco.mcp.optimal_server ` — `server.run()` at `optimal_server.py:1102`. Positional `agent_id` arg. +- `python -m roboco.mcp.docs_server ` — `server.run()` at `docs_server.py:251`. +- `python -m roboco.mcp.search_server ` — `server.run()` at `search_server.py:130`. +- `python -m roboco.mcp.intake_server` — `mcp.run()` at `intake_server.py:215`. Env: `ROBOCO_API_URL`, `ROBOCO_PROMPTER_SESSION_ID`. Mounted by `grok_intake_main`, NOT by the orchestrator. +- `python -m roboco.mcp.secretary_server` — `mcp.run()` at `secretary_server.py:64`. Env: `ROBOCO_API_URL`, `ROBOCO_AGENT_ID`, `ROBOCO_AGENT_ROLE`, `ROBOCO_AGENT_TOKEN`. Mounted by `grok_secretary_main`, NOT by the orchestrator. + +Invocation is one subprocess per agent container (the orchestrator writes `roboco-mcp-{agent_id}.json` into `/app/mcp-configs` and the runtime launches each `mcpServers` entry with `uv run --no-sync` pinned to `/app/.venv`). + +## Config Flags + +Env vars read in this slice (all `ROBOCO_*`): + +| Flag / env | Where | Purpose | +|------------|-------|---------| +| `ROBOCO_AGENT_ID` | flow, do, git-readonly (required) | Agent identity for `X-Agent-ID` header + role-path. | +| `ROBOCO_AGENT_ROLE` | flow, do, git-readonly (required) | Role for `X-Agent-Role` + flow route prefix. | +| `ROBOCO_AGENT_TOKEN` | utils `_get_agent_headers` | HMAC agent token injected by orchestrator at spawn; sent as `X-Agent-Token`. | +| `ROBOCO_ORCHESTRATOR_URL` | flow, do, git-readonly | Orchestrator base URL (default `http://roboco-orchestrator:8000`). | +| `ROBOCO_SDK_URL` | flow, do | Per-container SDK loopback for the breaker (default `http://localhost:9000`). | +| `ROBOCO_TOOL_MANIFEST_PATH` | flow, do | Path to the spawn manifest (default `/app/tool-manifest.json`). | +| `ROBOCO_API_URL` | intake, secretary | Orchestrator base URL for the grok-path servers. | +| `ROBOCO_PROMPTER_SESSION_ID` | intake | The live intake session id; without it `propose_draft`/`propose_batch` return a no-op string. | +| `ROBOCO_PROJECT_SLUG` / `ROBOCO_BRANCH` | set by orchestrator into `mcp_env` (consumed indirectly by `/api/git/*`) | Git context. | +| `settings.research_enabled` | orchestrator mount gate for `roboco-search` (not read inside the slice) | Web-research server armed only when true AND role in research_roles. | +| `settings.internal_api_url` | utils `ApiClient.base_url` | Base URL for optimal/docs/search async calls. | +| `ROBOCO_DOCS_SITE_PROJECT_SLUG` (default `"roboco-website"`) | `services/docs.py` `_refused_doc_types` | Deployer-configurable project slug named in the `write_doc(doc_type="user_facing")` refusal message — distinct from `docs_sync_*` (which stays RoboCo-only by design for the docs-divergence sync engine, see `docs/map/engine-docs-sync.md`); defaults to RoboCo's own value so behavior is unchanged until overridden. | +| `ROBOCO_DOCS_SITE_PUBLIC_URL` (default `"docs.roboco.tech"`) | `services/docs.py` `_refused_doc_types` | Deployer-configurable public docs URL, same refusal message. | +| `ROBOCO_ALLOW_FULL_TOOLSET` | flow `_register_tools`, do `_register_tools` | Dev/test escape hatch: when set, a missing manifest registers the full tool set instead of raising `RuntimeError`. Never set in production — the full-toolset path was the original bug this policy replaced. | + +No default-off feature flag is armed *inside* this slice; the only flag-gated server here is `roboco-search` (gated upstream by `ROBOCO_RESEARCH_ENABLED` in the orchestrator mount). + +## Gotchas + +- **Import-free `__init__.py` is load-bearing.** Re-exporting server factories here would force `optimal_server` (pgvector/ollama stack, ~6s) to load on every `python -m roboco.mcp.` and time out the MCP init — symptom: "roboco-flow/do tools never register". +- **flow/do refuse to start without the manifest** unless `ROBOCO_ALLOW_FULL_TOOLSET` is set. A missing `/app/tool-manifest.json` raises `RuntimeError` at import (production path). Previously the fallback registered all verbs, letting PMs call dev verbs at wrong URLs (404 storm). Local test runs without the bind mount can either set `ROBOCO_TOOL_MANIFEST_PATH` to a real file or set `ROBOCO_ALLOW_FULL_TOOLSET` to skip the hard-fail; the latter must never reach production containers. +- **`pass`/`fail` are Python keywords.** The IntentSpec layer uses `pass_review`/`fail_review`; the MCP layer exposes the public names `pass`/`fail`. `_INTENT_TO_PUBLIC` bridges the two. Forgetting this bridge silently drops QA tools from the palette (the dogfood bug that motivated it). +- **Dict-shaped `error` crashes a naive breaker.** `error in frozenset` raises `TypeError: unhashable type: 'dict'` when FastAPI exception handlers return `error` as a dict. `_classify_rejection` uses `isinstance` checks first — never a `dict in frozenset` membership test. +- **404 handling has three cases (updated 536bbb64).** (1) Bare default FastAPI 404 (`{"detail": "Not Found"}`) → missing route, synthesized as `invalid_state` with a wiring-gap remediate. (2) 404 carrying an `error` field → surfaced as-is (proxy re-status edge case). (3) 404 with a *descriptive* `detail` string → surfaced as `not_found` with a re-fetch remediate (#61). Previously only cases 1 and 2 existed and a descriptive 404 was mis-synthesized as `invalid_state`. +- **`StrList` is not just cosmetic.** A bare `list[str]` annotation hard-rejects the Claude SDK's nested `[[["…"]]]` / `[{item: {$text}}]` tool-input shapes at MCP validation *before* the verb body runs — surfacing as a confusing `1 validation error for i_will_planArguments…`. The `BeforeValidator` flattens first. +- **Breaker is fail-open.** SDK unreachable/slow/malformed → return the original rejection. The breaker is a safety net only; it must never break the gateway path. `_SDK_TIMEOUT=2.0` is tight by design. +- **`note(scope='handoff')` top-level `done`/`next` are the load-bearing fields for PM resumption.** Passing an empty `section={}` used to crash the minimax PMs (`done Field required` → `note circuit_open` → tracing gate blocked `delegate`). The MCP signature now has `done`/`next` as discrete string params. Do not pass `section={}`. +- **`propose_batch` filters and refuses empty batches.** Drafts without a string `title` OR `name` are dropped (via `_draft_title`); a `name`-only draft is normalized onto `title` before posting; if all are dropped it returns an error string instead of POSTing (would silently vanish on the panel side). `dropped` count is sent to the relay. Previously only `title` was accepted — `name`-only drafts were silently dropped even if well-formed (536bbb64). +- **intake/secretary are NOT mounted by the orchestrator.** They are mounted by `grok_intake_main`/`grok_secretary_main` for the grok path only. The orchestrator's `_generate_mcp_config` only knows flow/do/git-readonly/optimal/docs/search. +- **`optimal_server` positional `agent_id` is mandatory.** `python -m roboco.mcp.optimal_server` with no arg prints usage and exits 1. Same for docs/search. +- **`normalize_index_types`** maps the legacy `docs` alias to `documentation` before the route's `IndexType(...)` conversion — without it agents passing `index_types=["docs"]` get a 400. +- **git-readonly uses `raise_for_status`.** Unlike flow/do (which surface 4xx Envelopes), git-readonly propagates HTTP errors as exceptions. A non-200 from `/api/git/*` surfaces to the agent as a transport error, not an Envelope. +- **`X-Correlation-ID` is minted per MCP call** in flow/do (not per session). The orchestrator's `CorrelationIdMiddleware` accepts it as the inbound id and binds structlog + audit row to it. + +## Drift from CLAUDE.md + +CLAUDE.md "MCP servers running per agent container" table lists 5 servers (`roboco-flow`, `roboco-do`, `roboco-git-readonly`, `roboco-optimal`, `roboco-docs`). The actual `roboco/mcp/` directory contains **8** server modules: + +- `roboco/mcp/intake_server.py` (roboco-intake) — omitted from the CLAUDE.md table. Mounted by `grok_intake_main` for the grok intake path, not by `_generate_mcp_config`. +- `roboco/mcp/secretary_server.py` (roboco-secretary) — omitted from the CLAUDE.md table. Mounted by `grok_secretary_main` for the grok secretary path. +- `roboco/mcp/search_server.py` (roboco-search) — omitted from the CLAUDE.md table. Mounted by `_generate_mcp_config` (orchestrator.py:2915) only when `settings.research_enabled` AND role in `(cell_pm, main_pm, product_owner, head_marketing)`. + +CLAUDE.md "roboco-optimal" row says the server exposes `roboco_ask_mentor`, `roboco_kb_search` only. The actual `optimal_server.py` registers **18** tools (search/rag/stats/index_code/index_docs/tokens_estimate/ask_mentor/search_error/record_error_solution/check_decision/record_decision/get_standards/validate_action/review_code/record_learning/search_learnings/clear_index/reindex_all/index_status/get_proactive_context). Understatement, not contradiction. + +CLAUDE.md "roboco-do" row lists `commit, note, say, dm, evidence` and (in the Agent Gateway section) `draft_playbook` for delivery roles + `approve_playbook`/`reject_playbook`/`archive_playbook` for the Auditor. The actual `do_server.py` `do_tools` registry also contains `pitch`, `propose_roadmap` (Product Owner only), `progress`, `notify`, `notify_list`, `notify_get`, `notify_ack`, `pr_update`, `read_messages`, `read_a2a` — none mentioned in CLAUDE.md (`do_tools` is 18 entries, not the 5 named). Understatement. + +CLAUDE.md says the `note`/journal write "returns as soon as the entry is persisted; RAG indexing runs fire-and-forget." The MCP `note` tool itself is synchronous w.r.t. the orchestrator (a single POST); the fire-and-forget behavior is server-side, not visible in this slice — consistent, not drift. + +No contradicted claims found in this slice; the drift is omission (3 servers, ~13 do-tools, ~16 optimal tools not listed). + +## Changes Since Baseline + +Baseline: `fd10cc862c2020b3f639cdb686d427b0198a2441`. Commands: + +``` +git log --oneline fd10cc862c..HEAD -- roboco/mcp/ +git diff --stat fd10cc862c..HEAD -- roboco/mcp/ +``` + +Diff stat: `do_server.py +140/-? `, `flow_server.py +172/-?`, `intake_server.py +20/-?` (3 files, +298/-34). + +Only **one** commit touched this slice since baseline: + +- **`15effce0` — "Chore: 141 Gaps fill-in (#283)"** (merged PR, 2026-06-29). IMPACT on this slice: + - **`flow_server.py`**: added `StrList` (`BeforeValidator(coerce_str_list)`) so the Claude SDK's nested XML-ish tool-input shapes flatten before MCP validation; added `_MISSING_ROUTE_STATUS` 404 handling that synthesizes an `invalid_state` Envelope for manifest-registered verbs whose HTTP route is missing; added `_classify_dict_error_code` + `_classify_rejection` so dict-shaped `error` (FastAPI exception handlers) and 422 `detail`-list rejections count toward the per-verb circuit breaker (previously bypassed → unbounded retries); guarded against `TypeError: unhashable type: 'dict'`. + - **`do_server.py`**: mirrored the same breaker machinery (the dogfood gap: `note(scope='decision')` had looped 8× returning `incomplete_input` with no breaker) + the same 404 missing-route synthesis. + - **`intake_server.py`**: docstring-only change — `propose_draft`/`propose_batch` tool descriptions now declare the per-cell `project_id` on `the_work[]` entries (MegaTask multi-cell fan-out). No logic change in intake. + +No other commits in this slice since baseline. + > Post-snapshot updates (since 2026-06-29): -> - **b49337e7** `[chore] route-layer force gate + privileged-field gate + pre-task audit attribution` — audit.py: added `log_task_creation_denial` (target_type="task_creation", no task_id) as distinct from `log_task_action_denial`; `log_task_action_denial` now preserves non-UUID task_id sentinels in `details["target_id_raw"]` instead of silently dropping to NULL. -> - **d8a5bb48** `[chore] a2a service hierarchy gate (typed, unconditional) + persist skill on message row` — a2a.py: `create_a2a_notification` hierarchy gate is now unconditional (raises distinct ValueError if from_agent missing or target unresolvable, then calls `validate_a2a_access` raising typed A2AAccessDeniedError + route_hint instead of bare ValueError); `send_chat_message` reads and persists `skill` from opts on the message row (migration 054 adds nullable skill column on a2a_messages); `_msg_to_model` maps skill field; `send()` docstring updated. -> - **5bec3ec5** `[chore] a2a-routes: authenticate send_message responder + gate cancel task (PM-only)` — a2a.py: `cancel_task` gains `agent_role` (threaded into TaskService.cancel role gate) and `actor_slug` (recorded in cancellation note) params; the route now requires PM/management auth and passes the authenticated slug. -> - **b3558d4e** `[chore] complexity: split 5 C-rank blocks to <=B for xenon gate` — a2a.py: `cancel_task` factored into helpers `_status_value_of` (line 383) and `_apply_cancel_note` (line 387); no behavior change. -> - **da563487** `Wave 2 features: A2A live view (CEO chime-in + reply budget) and prompter memory (#297)` — a2a.py grows by ~250 lines: adds the CEO admin/live-view surface (`get_conversation_admin`, `list_conversations_admin`, `get_messages_admin`, `_enforce_ceo_reply_budget`, `_get_conversation_for_reply_to_ceo`) and the `A2A_MESSAGE_SENT` publish (`_publish_a2a_message_sent`, called from `send`) for the operator's org-wide watch view; `roboco/models/events.py` adds `EventType.A2A_MESSAGE_SENT`; `websocket_bridge.py` adds `_handle_a2a_message_event` forwarding it to `/ws/system` as an `a2a.message` frame. `routes/a2a.py` adds the CEO-gated `/chat/admin/conversations`, `/chat/admin/conversations/{id}/messages`, `/chat/admin/conversations/{id}/reply` routes (`_require_ceo`). -> - **876e19b3** `A2A switchboard (pair cards), Secretary/PM task access + closed over-permission hole, MegaTask conventions fix (#298)` — a2a.py adds `list_admin_pairs` (the switchboard's one-bulk-query pair+conversation join over `agents_config.A2A_ALLOWED_PAIRS`); `routes/a2a.py` adds the CEO-gated `/chat/admin/pairs` route. This commit also tightened `roboco/api/routes/tasks.py` (`_pm_editor_scope` / `_enforce_pm_lighter_fields`, out of this slice) and gave `SecretaryService` its `edit` directive action — see `docs/map/intake-secretary.md`. -> - **Wave 3** (2026-07-17, branch `feature/wave-3-a2a-ceo`, PR #547) — CEO-authored A2A DMs (panel "New DM" composer or `interject_as_ceo`) now wake an offline recipient: `_maybe_wake_ceo_recipient` (new) + `_ack_pending_wake_notifications` (new), gated to `read_a2a`-capable roles, reusing the legacy `a2a_request` NotificationTable row with a new per-row `requires_ack=True` override so it's finally visible to `_dispatch_a2a_work`'s `pending_ack_only` poll (previously structurally dead for every `a2a_request` row — see Gotchas). `send_a2a_notification` gains `requires_ack: bool = False` and an optional (`str | None`) `task_id`; `CreateNotificationParams` gains `requires_ack: bool | None = None`. Agent-to-agent `dm` is unaffected — still pull-only, no wake. Companion panel work (New-DM dialog, CEO direct-thread composer) is in `docs/map/panel.md`; the docs scrub that removed CEO-DM teaching from `docs/rag`/`agents/prompts` landed on the same branch (`ee620cf3`). +> +> - **`536bbb64` — "Chore/all/logical gaps sweep (#286)"** (merged 2026-06-30). IMPACT on this slice: +> - **`flow_server.py` + `do_server.py`**: added `_DICT_ERROR_CODE_MAP` (exact code→kind map, replacing pure-substring classification; closes AUTHENTICATION_REQUIRED mis-routing #161); added descriptive-404 carve-out in `_post` (surfaced as `not_found` instead of `invalid_state` for real resource-not-found 404s, #61); the circuit_open substitution now dict-copies the SDK envelope and nests the original rejection as `inner` (#60); `_register_tools` now accepts `ROBOCO_ALLOW_FULL_TOOLSET` as a dev/test escape hatch instead of always raising `RuntimeError` (#162). +> - **`intake_server.py`**: `propose_batch` now accepts `name` as a fallback for `title` (via new `_draft_title` / `_normalize_batch_drafts` helpers); `name`-only drafts are normalized onto `title` before posting rather than silently dropped (#163). +> - **`docs_server.py`**: `_handle_write` now surfaces a `commit_status == "failed"` outcome in the tool return string, telling the documenter to warn the cell PM when the doc could not be committed to the project repo (#34). +> +> - **`0d714b6c` — "[chore] mcp-servers: normalize exception bodies to Envelope + lift task_id/correlation_id on circuit_open"** (committed 2026-06-30). IMPACT on this slice: +> - **`flow_server.py` + `do_server.py`**: added `_remediate_for_kind` and `_normalize_exception_envelope`; the non-404 JSON path in `_post` now normalizes dict-`error` exception-handler bodies and 422 `detail` lists into the Envelope wire format so agents get `remediate`/`next` instead of raw exception bodies (#232); `_record_and_check_circuit` lifts `task_id`/`correlation_id` from the original rejection onto the circuit_open envelope top level (#359). +> - **`intake_server.py`**: `_post_event` captures relay response body under `detail` on non-success so the grok intake agent sees the real failure reason instead of an opaque `http_422` token (#57). +> +> - **v0.18.0** (2026-07-04): No commit touched `roboco/mcp/` for the X feature-spotlight workstream — `do_server.py`'s `_TOOLS` dict (21 content tools) is unchanged, which is itself the finding: `propose_feature_spotlight` was wired at the role-config + content-actions layers but never registered here. See Regression Risks below. +> - **`a0baf94b`** ("agnosticism-residue", audit item B8): `services/docs.py`'s `write_doc(doc_type="user_facing")` refusal message stops hardcoding "roboco-website"/"docs.roboco.tech" — the module-level `REFUSED_DOC_TYPES` constant becomes `_refused_doc_types()`, a function reading the new `ROBOCO_DOCS_SITE_PROJECT_SLUG`/`ROBOCO_DOCS_SITE_PUBLIC_URL` settings live on every call (both default to RoboCo's own values, so behavior is unchanged for this deployment). This `services/docs.py` file was previously undocumented in this slice's Files table — added above alongside the fix since it's the `docs_server.py` MCP server's sole backing service. + +## Regression Risks + +| Title | File:Line | Claim | Severity | +|-------|-----------|-------|----------| +| Breaker substitution could mask a real, fixable rejection | `flow_server.py:394`, `do_server.py:349` | **Partially mitigated (536bbb64):** the circuit_open envelope is now a dict-copy of the SDK's envelope with the original rejection nested as `inner` (preserving its kind/message/remediate). The agent sees `circuit_open` at the top level but the underlying rejection survives for ops debugging. The core risk remains: if the breaker trips on a mis-counted storm, the agent stops instead of retrying. | medium | +| Dict-error classification substring fallback → mis-routing risk for unknown codes | `flow_server.py:110`, `do_server.py:94` | **Partially mitigated (536bbb64):** `_classify_dict_error_code` now consults `_DICT_ERROR_CODE_MAP` first (exact match for all known RobocoError codes, closing the AUTHENTICATION_REQUIRED mis-routing bug #161). Only codes NOT in the map fall through to the substring branch. A novel code that accidentally contains `DENIED`/`AUTH`/`PERMISSION` but is semantically different would still mis-route. Risk is now confined to future unknown codes only. | low | +| 404 synthesis assumptions | `flow_server.py:272`, `do_server.py:238` | **Partially mitigated (536bbb64 #61):** a third 404 case was added: a 404 with a *descriptive* `detail` string (not the bare FastAPI default `"Not Found"`) is now surfaced as `not_found`, not `invalid_state`. Residual risk: a future route that returns a bare 404 with no `detail`/`error` field for a real resource-not-found would still synthesize `invalid_state`. The two carve-outs (`error` field → as-is; descriptive `detail` → `not_found`) cover the known cases. | low | +| `_register_tools` raises at import if manifest missing — local dev breakage | `flow_server.py:965`, `do_server.py:896` | **Mitigated (536bbb64):** `ROBOCO_ALLOW_FULL_TOOLSET` env var added as a dev/test escape hatch that bypasses the `RuntimeError` and registers the full tool set. Production behaviour is unchanged (ROBOCO_ALLOW_FULL_TOOLSET is not set in the orchestrator manifest). The mitigant must not leak into production containers. | low | +| `propose_batch` well-formed filter could silently drop intended drafts | `intake_server.py:146` | **Partially mitigated (536bbb64 #163):** `_draft_title` now accepts `name` as a fallback for `title`, and `_normalize_batch_drafts` normalizes `name`-only drafts onto `title` before posting. Residual risk: a draft using a different key (e.g. `label`) is still dropped silently; the `dropped` count is sent but the CEO may not notice. | low | +| Breaker SDK timeout (2s) may be too tight under load | `flow_server.py:55`, `do_server.py:37` | `_SDK_TIMEOUT=2.0` for the loopback `/verb/attempted` POST. Under container CPU contention the SDK could exceed 2s and the breaker fails open (returns original payload) — re-introducing the unbounded-retry condition the breaker was added to stop. Fail-open is safe but defeats the protection. | low | +| Circuit-breaker forwarding does not pass `task_id` for content tools that lack one | `do_server.py:379` | `body.get("task_id")` is sent to the SDK. Several do-tools (`read_messages`, `notify_list`) have no `task_id` — the breaker records `task_id=None`, so the per-verb (not per-task) breaker still works, but any future per-task breaker logic would mis-attribute these. | low | +| `propose_feature_spotlight` granted by role_config but not registered in `do_server.py` | `do_server.py:785-888`, `roboco/services/gateway/role_config.py:120-123` | v0.18.0's Head-of-Marketing-only content verb (`ContentActions.propose_feature_spotlight`) has no wrapper function or `_TOOLS` entry here, unlike `propose_roadmap` which has both (`do_server.py:529`, `:789`). `_register_tools()` only registers the intersection of the manifest's granted verbs and `_TOOLS` (`unknown = [verb for verb in allowed if verb not in _TOOLS]` silently drops anything else), so a spawned Head-of-Marketing agent cannot actually call this tool via MCP despite the role-config grant. Found via static read (grep for the verb name across `do_server.py` returns zero hits) — not reproduced against a live spawn. | Medium | ## Health -This slice is mature and internally consistent: the six services have clear separation of concerns (A2A transport/conversation, audit forensics, journal CRUD+RAG, note persistence chokepoint, stream extraction, RBAC), and the gateway/HTTP/orchestrator entry points map cleanly onto them. The code is defensive in the right places — audit._persist is best-effort with its own session, journal RAG indexing is fire-and-forget with a strong-ref guard, content_notes validates before mutating, and A2A conversation dedup prevents respawn re-emit storms. The main integrity concerns are cross-layer, not in-slice: (1) the new 60s Redis loop-prone notification re-fire guard (3aff6e04) sits between A2A's create_a2a_notification and delivery and can silently drop legitimate A2A notifications; (2) the legacy A2A-protocol path (dev_notes 'A2A Request' marker, TASK_ASSIGNED re-spawn) is undocumented in CLAUDE.md and coexists with the gateway conversation path, a known source of future confusion. No in-slice file changed since the fd10cc86 baseline, so there is no direct regression surface; the risks above are all dependency-mediated. Recommend a regression test that an A2A notification fired twice within 60s for genuinely different reasons still delivers. + +The slice is coherent and well-defended: the flow/do servers share a near-identical, heavily-commented breaker/404-synthesis contract (with the duplication acknowledged in comments as a deliberate mirror), the manifest-gated registration is fail-loud and blocks the off-role-verb class, and the `StrList` / dict-error / 404 fixes added in `15effce0` close real observed retry-storm and validation-rejection loops. The main integrity concerns are (a) the duplicated breaker logic across flow/do is a drift hazard — a future change to `_CIRCUIT_REJECTION_KINDS` or classification must be applied in both files or the two servers diverge; (b) CLAUDE.md's server table is stale (3 servers + many tools unlisted), which could mislead a reader into thinking intake/secretary/search are not agent-facing MCP servers; (c) the breaker's fail-open posture is correct but means the protection is only as good as the SDK loopback staying responsive within 2s. No correctness bugs observed; the slice is fit for purpose. +# Choreographer Slice Map ## Purpose -The two choreographer mixins that implement the PR-reviewer's two distinct surfaces: the in-path assembled-PR gate (PRGateMixin: claim_gate_review / pr_pass / pr_fail between a PM's submit and merge) and the inbound external/fork PR review (PRReviewerMixin: claim_pr_review / post_pr_review, read-only, posts one change-request and completes). Both are mixed into the composed Choreographer and route through the spec gate + verb runner, returning standardized Envelopes. +The Choreographer is the server-side composition layer that turns agent intent-verbs (`give_me_work`, `i_will_work_on`, `i_am_done`, `delegate`, `submit_up`, `submit_root`, `complete`, …) into ordered sequences of atomic TaskService / GitService actions. It owns the precondition gates the lifecycle spec does not model (concurrency invariants, tracing/progress gates, free-text soup, conventions, behind-base, unchanged-PR loop-stoppers) and wraps every composed mutation in a SAVEPOINT via `VerbRunner`. Every verb returns a standardized `Envelope` (`ok` / `error` + `next` + `remediate` + `context_briefing`). ## Files -| Path | Role | LOC | -|---|---|---| -| /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/gateway/choreographer/pr_gate.py | PRGateMixin — in-path assembled-PR gate verbs (claim_gate_review, pr_pass, pr_fail) and their helpers | 626 | -| /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/gateway/choreographer/pr_review.py | PRReviewerMixin — inbound external/fork PR review verbs (claim_pr_review, post_pr_review) + module-level resolve_task_project_slug | 601 | +| Path | Role | +|------|------| +| `roboco/services/gateway/choreographer/_impl.py` | The `_LegacyChoreographer` / `Choreographer` class — all verb bodies + guard helpers (~6.9k lines). | +| `roboco/services/gateway/choreographer/_protocol.py` | `ChoreographerHelpers` — TYPE_CHECKING-only stub of helpers role mixins call on `self`, so mypy sees typed signatures (runtime `object`). | +| `roboco/services/gateway/choreographer/_verb_runner.py` | `VerbRunner` — composed-actions runner; wraps `composes` in `session.begin_nested()` SAVEPOINT, runs `pre_side_effects` / `side_effects` outside. | + +## Key Symbols (landmarks only) + +| Name | Kind | File:Line | Responsibility | +|------|------|-----------|----------------| +| `Choreographer` | class | `_impl.py:339` | Composed entry point; deps-injected; exposes verbs + helpers. | +| `ChoreographerDeps` | dataclass | `_impl.py:207` | Dependency injection container (task, work_session, git, a2a, journal, audit, evidence_repo, messaging, product, orchestrator, stream_bus). | +| `_COORDINATOR_ROLES` | ClassVar | `_impl.py:914` | `{main_pm, cell_pm}` — exempt from `already_active`/`paused` claim guards + advisory lock. | +| `give_me_work` | async verb | `_impl.py:766` | Picks next task for agent + builds briefing (institutional memory injected here). | +| `_briefing_for` | async helper | `_impl.py:813` | Builds `context_briefing`. Claim-scoped: `full=True` (context-acquisition verbs only — give_me_work/claims/plan/resume/triage) carries the heavy sections via `_heavy_briefing_sections` (company_goals, recent_team_activity, blockers, task_handoff, institutional_memory); every other verb gets slim signals-only (unread a2a/mentions/notifications + metadata gaps). `include_company_goals=True` is a narrower, cheap-only opt-in (`_resolve_company_goals`) that fetches just the company_goals singleton without the rest of `full`'s heavy sections — used by `board_triage`'s idle branch so the CEO's charter (brand_voice/north_star) still reaches the Product Owner's roadmap-exploration / Head of Marketing's feature-spotlight-exploration one-shot spawns, whose directly-assigned exploration task is never itself a strategic root awaiting PM review (so the `full=True` branch never fires for it). AC coverage stays independent of `full`. | +| `_run_claim_guards` | async helper | `_impl.py:916` | already_active / paused / unmet_dependency (with re-check race narrowing) + `_lane_claim_guard`; `skip_dev_guards=False` param skips dev-only guards for pr_reviewer gate claims (claim_gate_review). | +| `_lane_claim_guard` | async helper | `_impl.py:977` | Out-of-order-start barrier: refuse code leaf behind an earlier open same-assignee sibling. Fail-closed on lookup error. | +| `_claim_plan_start_gate` | async helper | `_impl.py:1179` | spec gate → advisory claim lock (non-PM) → behavioral guards. | +| `_claim_plan_start_run` | async helper | `_impl.py:1251` | `runner.run_intent(verb)` + ensure_work_session + `_touch`. | +| `i_will_work_on` | async verb | `_impl.py:1326` | Dev claim+plan+start path; routes re-entry vs fresh claim. | +| `open_pr` | async verb | `_impl.py:1571` | Pre-flight + `run_intent("open_pr")` (push_branch + create_pr side effects). | +| `i_am_done` | async verb | `_impl.py:1771` | Dev pre-submit; runs `_i_am_done_gate` then `run_intent("i_am_done")`. | +| `_i_am_done_gate` | async helper | `_impl.py:1885` | Ordered gate chain: tracing → submit_qa fields → push → behind_base → quality → toolchain → conventions; then write AC status. | +| `_behind_base_gate` | async helper | `_impl.py:2154` | Refuse submit when branch behind its base (sibling PR merged); fail-open on git error. | +| `_toolchain_broken_guard` | async helper | `_impl.py:1939` | Block delivery gate when agent workspace can't run suite; `reviewer=True` for `pr_pass`. | +| `_conventions_guard` | async helper | `_impl.py:2043` | Run architectural-conventions validator; `block` finding refuses gate. | +| `_pm_task_type_error` | static method | `_impl.py:4752` | Reject a code/non-planning task_type delegated to a PM (cell or main); extracted from `_validate_assignee_task_type` to keep that dispatcher under complexity budget. | +| `i_am_blocked` | async verb | `_impl.py:3086` | Rate-limit parking vs generic block; `run_intent("i_am_blocked")`. | +| `_handle_rate_limited_parking` | async helper | `_impl.py:2983` | Park provider on 429/overload/session-limit. | +| `unclaim` | async verb | `_impl.py:3184` | Release claimed task → pending (optional reassign). | +| `reassign` | async verb | `_impl.py:3343` | Reassign task with `_validate_reassign`. | +| `resume` | async verb | `_impl.py:3422` | Resume paused/blocked task. | +| `sync_branch` | async verb | `_impl.py:3528` | Gate-level rebase verb (new since baseline). | +| `i_am_idle` | async verb | `_impl.py:3678` | Idle signal; auto-pause in_progress tasks; pending-assignment / PM review / auditor guards. | +| `i_will_plan` | async verb | `_impl.py:4071` | PM plan verb; `_pm_sub_tasks_gate` enforces substantive approach/sub_tasks. | +| `delegate` | async verb | `_impl.py:4173` | PM creates subtask; sizing, sibling-dedup, spine-cap, lifecycle guards; `_create_subtask_from_inputs`. | +| `submit_up` | async verb | `_impl.py:5348` | Cell PM opens cell→root PR + enters `awaiting_pr_review`; unchanged-PR guard. | +| `submit_root` | async verb | `_impl.py:6260` | Main PM opens root→master PR + enters gate; umbrella hard-reject + unchanged-PR guard. | +| `_submit_root_unchanged_pr_guard` | async helper | `_impl.py:6149` | Loop-stopper: refuse re-submit when PR head SHA == last `pr_fail` SHA. Fail-open on ambiguity. | +| `_current_pr_head_sha` | async helper | `_impl.py:6196` | Best-effort current PR head SHA via `_project_slug_for` + `git.get_pr_head_sha`. | +| `complete` | async verb | `_impl.py:6599` | Role-dispatch to `cell_pm_complete` / `main_pm_complete`; umbrella-in-progress bypasses spec gate. | +| `main_pm_complete` | async verb | `_impl.py:6496` | Main PM merge + escalate to CEO (never merges master itself). | +| `escalate_to_ceo` | async verb | `_impl.py:6844` | Escalate to `awaiting_ceo_approval`. | +| `VerbRunner.run_intent` | async method | `_verb_runner.py:37` | pre_side_effects → SAVEPOINT(composes) → side_effects; intermediate-None raises INVALID_STATE. | +| `VerbRunner._do_pr_merge` | async handler | `_verb_runner.py:257` | `pr_merge` with `project_id` scoping (cross-repo collision fix) + `resolve_parent_branch`. | +| `ChoreographerHelpers` | stub class | `_protocol.py:31` | TYPE_CHECKING-only typed view of `self` helpers for role mixins. | + +## Data Flow +An MCP `flow/*` call hits the orchestrator → the role-specific gateway verb → `Choreographer.(agent_id, task_id, ...)`. The verb fetches the task (`self.task.get`), builds a briefing (`_briefing_for`), runs the spec gate (`spec.can_invoke_intent(role, verb, t, ctx)`) and any verb-specific preflight guards (free-text soup, claim guards, conventions, behind-base, unchanged-PR). On rejection it emits via `_emit_rejection` with `next`/`remediate`. On allow it calls `VerbRunner.run_intent(verb, t, agent, ctx)`: `pre_side_effects` (e.g. `create_root_pr` for `submit_root`) run OUTSIDE the SAVEPOINT; then `session.begin_nested()` wraps the composed atomic actions (`claim`/`set_plan`/`start`/`submit_qa`/…) re-fetching the task after each; then `side_effects` (push_branch / create_pr / pr_merge) run after the savepoint commits. An intermediate `None` from a composed action raises `INVALID_STATE` (concurrent transition); a trailing `None` flows out as the verb result. The verb wraps the final task in `Envelope.ok(status, next_hint, briefing)` with `.with_introspection(task, role)`. + +## Mermaid +```mermaid +sequenceDiagram + participant Agent + participant Gateway as MCP flow verb + participant Ch as Choreographer + participant Spec as lifecycle spec + participant VR as VerbRunner + participant TS as TaskService + participant Git as GitService + Agent->>Gateway: verb(agent_id, task_id, notes) + Gateway->>Ch: verb(agent_id, task_id, notes) + Ch->>TS: task.get(task_id) + Ch->>Ch: _briefing_for(...) + Ch->>Spec: can_invoke_intent(role, verb, t, ctx) + alt not allowed + Ch-->>Agent: Envelope error (from_decision) + remediate + else allowed + Ch->>Ch: preflight guards (soup / claim / conventions / behind_base / unchanged_pr) + alt guard rejects + Ch-->>Agent: Envelope invalid_state + remediate + else pass + Ch->>VR: run_intent(verb, t, agent, ctx) + VR->>Git: pre_side_effects (create_root_pr) + VR->>TS: session.begin_nested() (SAVEPOINT) + loop composes + VR->>TS: atomic action (claim / set_plan / start / submit_qa / ...) + TS-->>VR: updated task (None on concurrent source-state mismatch) + alt intermediate None + VR-->>Ch: raise INVALID_STATE + end + end + VR->>TS: commit savepoint + VR->>Git: side_effects (push_branch / create_pr / pr_merge) + VR-->>Ch: final task + Ch->>TS: ensure_work_session / _touch + Ch-->>Agent: Envelope ok(status, next, briefing) + end + end +``` + +## Logical Tree +``` +Choreographer (composed class, _impl.py) +├── Deps: task / work_session / git / a2a / journal / audit / evidence_repo / messaging / product / orchestrator / stream_bus +├── Verb bodies +│ ├── Dev: give_me_work, i_will_work_on, open_pr, i_am_done, i_am_blocked, unclaim, resume, sync_branch, i_am_idle +│ ├── PM: i_will_plan, delegate, submit_up, submit_root, complete→{cell_pm_complete, main_pm_complete}, escalate_up, triage, triage_all, unblock, reassign, pm_give_me_work +│ └── Board/CEO: escalate_to_ceo +├── Guard helpers +│ ├── _run_claim_guards → already_active / paused / unmet_dependency / _lane_claim_guard +│ ├── _i_am_done_gate chain → tracing / submit_qa_fields / push / behind_base / quality / toolchain / conventions +│ ├── _submit_up_guard / _submit_up_unchanged_pr_guard / _submit_root_unchanged_pr_guard +│ ├── _guard_free_text / _free_text_soup / _soup_or_decision_env +│ └── _conventions_guard / _toolchain_broken_guard +└── VerbRunner (_verb_runner.py) + ├── pre_side_effects → {create_root_pr} + ├── composes (SAVEPOINT) → {claim, set_plan, start, submit_qa, qa_pass, qa_fail, docs_complete, complete, submit_pm_review, submit_for_review, pr_pass, pr_fail, escalate_to_ceo, block, unblock, resume, pr_review_done} + └── side_effects → {push_branch, create_pr, create_root_pr, pr_merge} +``` + +## Dependencies +- **Internal services**: `TaskService`, `WorkSessionService`, `GitService`, `EvidenceRepo`/`evidence_builder`, `a2a`, `JournalService`, `AuditService`, `ProjectService`/product, orchestrator handle, `StreamEventBus`. +- **Policy (pure)**: `roboco.foundation.policy.lifecycle` (`can_invoke_intent`, `Context`, `Role`, `_INTENT_VERBS`), `foundation.policy.batch` (`is_batch_umbrella`), `foundation.policy.content` markers + `reject_trivial`. +- **Gateway helpers**: `claim_guards` (`already_active_guard`, `paused_tasks_guard`, `unmet_dependency_guard`), `merge_chain.resolve_parent_branch`, `envelope.Envelope`, `remediation` hints. +- **External**: `structlog`, `asyncpg` (via task_service session / SAVEPOINT), `redis` (rate-limit parking), `git`/`gh` CLI (via git_service). + +## Entry Points +- MCP `roboco-flow` server → per-role verb methods (manifest-driven allowlist from `role_config.py`). +- Orchestrator `/api/v1/flow/*` REST endpoints → same `Choreographer` methods. +- Internal cross-verb calls (e.g. `complete` → `cell_pm_complete` / `main_pm_complete`; `_claim_plan_start_run` shared by `i_will_work_on` + `i_will_plan`). + +## Config Flags +- `ROBOCO_TOOLCHAIN_MATCH_ENABLED` — gates `_toolchain_broken_guard` (default-off). +- `ROBOCO_CONVENTIONS_ENABLED` — gates `_conventions_guard` (default-off). +- `ROBOCO_OVERLOAD_BREAK_ENABLED` — rate-limit/overload parking path (default-on). +- `ROBOCO_ORG_MEMORY_ENABLED` — institutional-memory injection in `_briefing_for` (default-off). +- `ROBOCO_GATEWAY_HEALTH_ENABLED` — reaper gateway-health probe (default-on; not directly in choreographer but feeds the parking path). + +## Gotchas +- The SAVEPOINT wraps only DB atomic actions; `pre_side_effects` (e.g. `create_root_pr`) run BEFORE the savepoint and are NOT rolled back if a later composed action raises — they are idempotent by contract. +- `side_effects` (push/create_pr/pr_merge) run AFTER savepoint commit and are idempotent + retryable; a crash between commit and side-effect leaves the PR uncreated (recovered by re-issue / `open_pr` parity). +- `VerbRunner` only raises on an INTERMEDIATE `None` composed action; a trailing `None` is the verb's own "no transition" result and each verb body must handle it (submit_root_finalize, _claim_plan_start_run do; a verb that forgets will None-deref). +- `_lane_claim_guard` fail-closed on lookup error — a DB hiccup rejects the claim (calls `release_dependency_blocked_claim`); acceptable but can briefly bounce a dev. +- `_submit_*_unchanged_pr_guard` FAILS OPEN on every ambiguous case (no recorded sha, no project slug, git error, closed PR) — only exact-unchanged is hard-blocked; a regression in `_current_pr_head_sha` resolver silently re-opens the loop. +- `complete` bypasses the spec gate for an in-progress batch umbrella (`_is_umbrella_in_progress`) and relies on `main_pm_complete`'s own guards — a mis-classified umbrella could skip the AWAITING_PM_REVIEW status constraint. + +## Drift from CLAUDE.md +- CLAUDE.md verb table lists `submit_root` for `main_pm` and `submit_up` for `cell_pm` — matches code (`_impl.py:5348`, `6260`). No drift. +- CLAUDE.md: "PM coordinator concurrency … claim-time concurrency guards skipped for `_COORDINATOR_ROLES`" — matches (`_impl.py:939`, `1234`). No drift. +- CLAUDE.md verb surface omits `sync_branch` from the developer list — code has `sync_branch` at `_impl.py:3528` (added since baseline; memory note `project_sync_branch_tracing_gap.md` flags it). Minor doc drift. +- CLAUDE.md: "PR is created BEFORE QA review" — `i_am_done` gate chain pushes + creates PR context but the actual `create_pr` side-effect runs in `open_pr`/`submit_up`/`submit_root`, not `i_am_done`; consistent with the described flow. No drift. +- CLAUDE.md: "only the CEO merges master; Main PM ready root PR → awaiting_ceo_approval (does NOT merge)" — `main_pm_complete` escalates; `VerbRunner._do_pr_merge` exists for cell-level merges and `create_root_pr` opens but the root merge is CEO-gated. Consistent. No drift. + +## Changes Since Baseline +`git log --oneline fd10cc862c2020b3f639cdb686d427b0198a2441..HEAD -- roboco/services/gateway/choreographer/` → 2 commits touching these files (+814/−89): + +1. `15effce0` — 141 Gaps fill-in (#283): added out-of-order-start guards (`_lane_claim_guard`, `_behind_base_gate`, `sync_branch`), unchanged-PR loop-stoppers (`_submit_root_unchanged_pr_guard`, `_submit_up_unchanged_pr_guard`, `_current_pr_head_sha`), `project_id` scoping on `pr_merge` (cross-repo collision fix), umbrella-in-progress bypass in `complete`, `_submit_root_finalize` None-guard, reviewer flag on `_toolchain_broken_guard`. +2. `3aff6e04` — Close gaps (#285): follow-on touch-ups in the same areas (per-cell project map root-subtask support umbrella handling). + +> Post-snapshot updates (since 2026-06-29): +> - `536bbb64` — logical-gaps sweep: `_run_claim_guards` gains `skip_dev_guards: bool = False` param (pr_reviewer `claim_gate_review` calls skip dev-only already_active/paused/lane guards; dependency guard still runs). `_pm_task_type_error` extracted as new `@staticmethod` on `Choreographer` (`_impl.py:4752`) from `_validate_assignee_task_type` — fixes Main PM omission from the PM-cannot-own-code delegate gate. `_submit_*_unchanged_pr_guard` now logs a `warning` on head_sha resolver failure so fail-open behavior is observable. Matching `skip_dev_guards` stub added to `_protocol.py`. `_impl.py` +78/−0 lines; `_protocol.py` +1 line. +> - `0e7674af` — verb_runner trailing-None side-effect guard + actor_agent_id threading: `run_intent` now skips the `side_effects` loop when the trailing composed action returns `None` (prevents `_do_push_branch(None)` / `_do_pr_merge(None)` AttributeError crash, converting it to a clean caller-handled `None`). `_do_push_branch`, `_do_create_pr`, `_do_create_root_pr`, and `_do_escalate_to_ceo` all forward `actor_agent_id=agent.id` into `git_service` / `task_service`. `main_pm_complete` escalate path in `_impl.py` also forwards `actor_agent_id`. `_verb_runner.py` +52/−8 lines; `_impl.py` +8/−2 lines. + +## Regression Risks + +| Title | File:Line | Claim | Severity | +|-------|-----------|-------|----------| +| Intermediate-None contract depends on every verb body handling trailing None | `_verb_runner.py:89` + `_impl.py:1277,6358` | A verb that forgets the trailing-None guard None-derefs `t.status`; submit_root + claim_plan_start handle it, but any NEW verb using `run_intent` with a possibly-None last action inherits the trap. **Side-effect crash path closed (0e7674af)**: `run_intent` now skips the `side_effects` loop on a trailing `None`, so `_do_push_branch(None)` no longer crashes. Verb bodies still need to handle `None` return for their own error messaging but will not AttributeError. Risk reduced: runner-level crash path fixed; verb-body None-deref in messaging remains a code-discipline risk. | High → Medium | +| `pr_merge` project_id scoping assumes `task.project_id` is non-None | `_verb_runner.py:263` | `project_id=task.project_id` — if a coordination/umbrella task ever reaches `pr_merge` with `project_id=None`, the cross-repo collision guard silently matches nothing or None-keys the scoping; could merge the wrong PR or no-op. | High | +| `_submit_*_unchanged_pr_guard` fails open on resolver regressions | `_impl.py:6175,6240` | Any future break in `_current_pr_head_sha` / `_project_slug_for` / `git.get_pr_head_sha` makes the loop-stopper a no-op, re-opening the 2026-06-27 pr_fail re-submit loop. **Partially mitigated (536bbb64)**: `_current_pr_head_sha` now emits a `structlog.warning` on resolver failure so the fail-open path is observable in logs; the underlying fail-open behavior is intentional and unchanged. | High | +| `_lane_claim_guard` calls `release_dependency_blocked_claim` on lookup error | `_impl.py:988,996` | Fail-closed path releases the claim before returning the error envelope; if the lookup error is transient the dev is bounced + work-session abandoned even though the lane was actually free. | Medium | +| `complete` umbrella-in-progress bypass skips the spec AWAITING_PM_REVIEW status check | `_impl.py:6657` | `_is_umbrella_in_progress` mis-classification (e.g. a non-batch branchless task with matching predicates) lets a non-awaiting_pm_review task reach `main_pm_complete`/`cell_pm_complete`. | Medium | +| `_run_claim_guards` dependency re-check narrows but does not close the race | `_impl.py:965-967` | The re-check returns None (skip release) when fresh read sees deps met, but the window between re-check and the caller's claim is still unlocked; a concurrent terminal transition there is benign (monotonic), but a non-monotonic future status could re-open. | Low | +| `submit_root` runs `create_root_pr` as a pre_side_effect OUTSIDE the savepoint | `_verb_runner.py:67` + `_impl.py:6339` | If a later composed `submit_for_review` raises, the root→master PR is already opened and NOT rolled back; re-issue is idempotent-by-contract but a non-idempotent future pre_side_effect would leak. | Medium | +| `_i_am_done_gate` writes AC criteria status AFTER all gates pass but BEFORE `run_intent` | `_impl.py:1910` | `_write_criteria_status` runs in the gate phase; if `run_intent("i_am_done")` then raises, the AC status rows persist for a task that did not transition — a stale write the next attempt must overwrite. | Low | + +## Health +The slice is structurally sound: the SAVEPOINT boundary, intermediate-None INVALID_STATE guard, and role-exempt coordinator concurrency model are coherent and well-documented. The highest-temperature areas are the new (since baseline) fail-open loop-stoppers and fail-closed lane guard — both correct by design but tightly coupled to resolvers (`_current_pr_head_sha`, `has_earlier_incomplete_code_sibling`) whose regressions silently revert the protection. The 814-line delta is concentrated in guard additions rather than control-flow rewrites, so baseline behavior is largely preserved; the main residual risk is verb-body discipline around the trailing-None contract for any future verb. +# task-service slice + +## Purpose +`TaskService` is the authoritative owner of the task lifecycle: CRUD, hierarchical create (incl. MegaTask umbrella + root-subtasks), claim/locking, every status transition, completion/CEO approval/cancellation, dependency DAG wiring, rework routing, and completion-time learning capture. All status writes funnel through `_validate_and_set_status` + `_emit_status_transition_audit` so the audit journey and the `revision_count` rework counter stay in lockstep with real task state. + +## Files + +| Path | Role | +|------|------| +| `roboco/services/task.py` | Single 8.7k-line service module implementing TaskService + a few internal dataclass containers (`_CompletionSnapshot`, `SoftBlockInput`, `SoftBlockInfo`, `GatewayAgentView`). | ## Key Symbols | Name | Kind | File:Line | Responsibility | -|---|---|---|---| -| PRGateMixin | class | roboco/services/gateway/choreographer/pr_gate.py:40 | Mixin: in-path assembled-PR gate verbs (claim_gate_review/pr_pass/pr_fail) + helpers; inherits ChoreographerHelpers only under TYPE_CHECKING | -| PRGateMixin.claim_gate_review | method | roboco/services/gateway/choreographer/pr_gate.py:43 | Reviewer claims an awaiting_pr_review task without transitioning it (status stays awaiting_pr_review); returns assembled PR diff inline as evidence | -| PRGateMixin.pr_pass | method | roboco/services/gateway/choreographer/pr_gate.py:115 | Pass the assembled PR: awaiting_pr_review → awaiting_pm_review; delegates to _gate_decision | -| PRGateMixin.pr_fail | method | roboco/services/gateway/choreographer/pr_gate.py:123 | Fail the assembled PR with structured `findings` (the deprecated `issues=[str]` shim still works this release) → needs_revision; validates + count-guards (nudge 5 / hard-cap 10) via `choreographer/findings.py`, inserts one append-only ledger row per finding BEFORE `_record_gate_verdict_for` so the verdict note carries real ids, delegates to `_gate_decision`. See `docs/map/review-findings.md`. | -| PRGateMixin._gate_preflight | method | roboco/services/gateway/choreographer/pr_gate.py:148 | Ownership + role + spec gate (with self_review_block via actor_slug/original_developer_slug) + free-text soup guard for pr_pass/pr_fail; returns rejection Envelope or (t,agent,role_str,briefing,spec_ctx) | -| PRGateMixin._record_gate_verdict_for | method | roboco/services/gateway/choreographer/pr_gate.py:229 | Author the canonical pr_review verdict note before the transition; on pr_fail also capture the assembled PR head SHA for the unchanged-PR gate; on pr_pass with ci_note, stamp the ci_status field into the verdict with evidence the CI guard ran | -| PRGateMixin._post_gate_review | method | roboco/services/gateway/choreographer/pr_gate.py:245 | Post the gate verdict to the PR itself (best-effort, after the DB transition); resolves reviewer slug | -| PRGateMixin._deliver_pr_fail_to_owner | method | roboco/services/gateway/choreographer/pr_gate.py:253 | a2a the pr_fail change-requests to the owning PM (best-effort) with a Main-PM-root steer to re-delegate not re-submit; closes the blind re-submit loop | -| PRGateMixin._gate_decision | method | roboco/services/gateway/choreographer/pr_gate.py:292 | Shared body for pr_pass/pr_fail: preflight + tracing + pr_pass blocked guards + record verdict + run_intent + None-guard for concurrent transition + post-PR + a2a on fail | -| PRGateMixin._pr_pass_blocked | method | roboco/services/gateway/choreographer/pr_gate.py:373 | Refuse pr_pass on a broken toolchain, block-level convention violation, or non-green CI on the assembled PR's head commit; returns (rejection_envelope, ci_note). Both toolchain and conventions guards inert when their flags are off; CI guard fails open on configuration gaps | -| PRGateMixin._ci_status_guard | method | roboco/services/gateway/choreographer/pr_gate.py:520 | Refuse pr_pass unless CI on the assembled PR's head commit is green. Failing/pending/unscheduled CI blocks with reviewer-aware remediation pointing at pr_fail; configuration gaps, unreachable/nonexistent repos, or real API failures on reachable repos each pass through with distinct classifications (no_ci_configured vs error) via git.get_pr_ci_status | -| PRGateMixin._resolve_ci_status | method | roboco/services/gateway/choreographer/pr_gate.py:480 | Thin wrapper: calls git.get_pr_ci_status, interprets the returned dict (no_ci_configured/pending/failure/error/success), and composes a rejection Envelope if CI must block pr_pass | -| PRGateMixin._record_gate_verdict | method | roboco/services/gateway/choreographer/pr_gate.py:403 | Persist the gate verdict as the canonical pr_review structured note (passed/failed), with issues slot for pr_fail, head_sha stamp for pr_fail, and ci_status evidence on pr_pass; best-effort (ContentValidationError logged not raised) | -| PRGateMixin._capture_pr_head_sha | method | roboco/services/gateway/choreographer/pr_gate.py:468 | Best-effort capture of the assembled PR head SHA at pr_fail time via _project_slug_for + git.get_pr_head_sha; returns None on any failure (fail-open) | -| PRGateMixin._post_gate_review_to_pr | method | roboco/services/gateway/choreographer/pr_gate.py:502 | Post APPROVE/REQUEST_CHANGES on cell→root PRs; always COMMENT on root→master (only CEO merges master); best-effort | -| PRGateMixin._gate_role_or_rejection | method | roboco/services/gateway/choreographer/pr_gate.py:545 | Parse the role enum from role_str or return a not_authorized rejection Envelope | -| PRGateMixin._gate_tracing | method | roboco/services/gateway/choreographer/pr_gate.py:569 | Tracing gate for pr_pass/pr_fail: requires journal:learning entry + substantive pr_reviewer_notes (notes threaded via SimpleNamespace shim) | -| PRGateMixin._re_stamp_pr_fail_head_sha_if_advanced | method | roboco/services/gateway/choreographer/pr_gate.py:255 | Re-capture the PR head SHA AFTER the transition commits and re-stamp the verdict note only if it advanced past the pre-transition capture (#189 fix for stale-SHA false-allow loop) | -| PRGateMixin._gate_review_event_verdict | staticmethod | roboco/services/gateway/choreographer/pr_gate.py:555 | Map gate verb → (review event, verdict label): pr_pass → APPROVE/PASSED, pr_fail → REQUEST_CHANGES/CHANGES REQUESTED — both downgraded to COMMENT on root→master (is_root) | -| PRGateMixin._gate_review_body | staticmethod | roboco/services/gateway/choreographer/pr_gate.py:568 | Render the gate-review comment body posted to the assembled PR (includes CEO-only footer for root→master PRs) | -| PRGateMixin._build_gate_review_evidence | method | roboco/services/gateway/choreographer/pr_gate.py:697 | Inline evidence for claim_gate_review: assembled branch diff + pr_number/pr_url + acceptance_criteria + is_assembled_pr | -| PRGateMixin._gate_diff_parent | method | roboco/services/gateway/choreographer/pr_gate.py:920 | The assembled task's REAL parent branch (via `resolve_parent_branch`, reading the parent TASK's own `branch_name`) or None for a branchless task; threaded as `preferred_parent` into `git.diff` (claim_gate_review evidence) and the conventions guard. Replaces the string-derived `parent_branch_for`, which reused the child branch's own team segment and was wrong across every cross-team hop (e.g. a frontend child of a main_pm root). Fail-open on a lookup error (falls back to the derived-base default), like every other `resolve_parent_branch` call site. | -| PRReviewerMixin | class | roboco/services/gateway/choreographer/pr_review.py:44 | Mixin: inbound external/fork PR review verbs (claim_pr_review/post_pr_review) + helpers; read-only, never checks out contributor code | -| PRReviewerMixin.claim_pr_review | method | roboco/services/gateway/choreographer/pr_review.py:47 | Reviewer claims an external-PR review task (pending→in_progress via task.pr_review_claim, branch-gate exempt); returns contributor diff inline read-only | -| PRReviewerMixin._build_pr_review_content | staticmethod | roboco/services/gateway/choreographer/pr_review.py:124 | Validate summary+findings+event into a PrReviewContent via validate_content, or return an invalid_state Envelope | -| PRReviewerMixin._resolve_post_body | method | roboco/services/gateway/choreographer/pr_review.py:149 | Resolve the GitHub comment body: canonical render when findings given (and stored structured), else free-text body; Envelope on malformed findings | -| PRReviewerMixin._is_hand_formatted_verdict | staticmethod | roboco/services/gateway/choreographer/pr_review.py:163 | True when a free-text body carries verdict/section markdown headers (## summary/issues/verdict/findings) the system would otherwise generate | -| PRReviewerMixin._post_review_side_effects | method | roboco/services/gateway/choreographer/pr_review.py:180 | Post the review to GitHub + send external-pr-reviewed CEO notification (both best-effort, after DB transition) | -| PRReviewerMixin.post_pr_review | method | roboco/services/gateway/choreographer/pr_review.py:211 | Post ONE change-request to the PR and finish the review task (in_progress→completed); content gates pre-side-effect, side-effects post-transition | -| PRReviewerMixin._post_pr_review_preflight | method | roboco/services/gateway/choreographer/pr_review.py:290 | Pre-runner guards for post_pr_review: non-empty body, role, spec gate, tracing gate; returns (agent,role_str,briefing,spec_ctx) or rejection | -| PRReviewerMixin._verdict_consistency_gate | method | roboco/services/gateway/choreographer/pr_review.py:343 | Reject a self-contradicting (event, findings) pair via pr_review_conflict pure invariant; runs before any side effect | -| PRReviewerMixin._post_pr_review_content_gates | method | roboco/services/gateway/choreographer/pr_review.py:376 | Folded content gates: verdict consistency then no-hand-formatted-body guard (only when no findings); returns first rejection or None | -| PRReviewerMixin._resolve_role | method | roboco/services/gateway/choreographer/pr_review.py:442 | Parse role enum or return not_authorized rejection Envelope | -| PRReviewerMixin._runner_failure | method | roboco/services/gateway/choreographer/pr_review.py:466 | Shared rejection envelope for a verb-runner failure | -| PRReviewerMixin._build_pr_review_evidence | method | roboco/services/gateway/choreographer/pr_review.py:488 | Inline evidence for claim_pr_review: PR unified diff via git.get_pr_diff (read-only) + pr_number/pr_url + is_external_pr | -| PRReviewerMixin._pr_review_tracing_gate | method | roboco/services/gateway/choreographer/pr_review.py:501 | Tracing gate for post_pr_review: journal:learning entry + substantive pr_reviewer_notes (body threaded via SimpleNamespace shim) | -| PRReviewerMixin._project_slug_for | method | roboco/services/gateway/choreographer/pr_review.py:545 | Resolve project slug for a task; delegates to module-level resolve_task_project_slug | -| resolve_task_project_slug | function | roboco/services/gateway/choreographer/pr_review.py:555 | Module-level slug resolver shared with _impl.py unchanged-PR gate: project_id → product first distinct project → cell_projects first distinct project | +|------|------|-----------|----------------| +| `_validate_and_set_status` | method | task.py:548 | Single chokepoint: validate transition + git requirements, set status, poke dispatcher, emit audit. | +| `_emit_status_transition_audit` | method | task.py:652 | Write `task.` audit row in caller session; bump `revision_count` on entry into `needs_revision`. | +| `_alert_auditor_of_rework` | method | task.py:1019 | Best-effort helper that asks `NotificationDeliveryService` to send a HIGH `ALERT` to the auditor when a task enters `needs_revision`. Called from `fail_qa`, `pr_fail`, and `request_changes` immediately after `await self.session.flush()` so the transition row is visible before the alert is dispatched. | +| `create` | method | task.py:864 | New task; depth/batch/AC validation; branchless/umbrella flags; baseline constraints attachment; (V2) vault materialize-on-create. | +| `_attach_baseline_constraints` | method | task.py:971 | Append conventions baseline constraints to task prompt (gated `conventions_enabled`). | +| `_materialize_vault_note` | method | task.py:910 | V2: best-effort vault seam called from `create` — assembles + writes a deterministic task note (narrative placeholder) so a task is visible in the vault from the moment it exists, not just at Auditor curation/rebuild. Gated `obsidian_vault_enabled`; swallows + logs any failure. | +| `list_updated_since` | method | task.py:7101 | V2: tasks touched (`COALESCE(updated_at, created_at)`) since a timestamp, ascending, paged — the vault janitor's changed-task re-projection set. | +| `list_archive_candidates` | method | task.py:7124 | V2: terminal tasks whose terminal timestamp falls in `[after, before)`, ascending, paged — the vault janitor's archival-pass candidate window (watermark-bounded so a sweep never rescans the whole archive). | +| `sample_stale_tasks` | method | task.py:7154 | V2: random sample of tasks last touched before a cutoff — the vault janitor's drift-verification sample. | +| `activate` | method | task.py:1577 | `backlog→pending` (PM only); batch-shape guard. | +| `_ensure_branch_for_task` | method | task.py:1675 | Branch resolution for claim; `""` for branchless/umbrella. | +| `_auto_create_branch` | method | task.py:1833 | Cut hierarchical branch + per-task worktree add (F123). | +| `_remove_task_worktree` | method | task.py:1913 | Low-level worktree removal by task id. | +| `admin_set_status` | method | task.py:2060 | Privileged override (bypass validator); restores pre-block owner; still emits audit. Post-#2176: the blocked→pending/in_progress restore path now attributes the audit row to the admin actor (not the restored owner) and emits a `task.admin_override` row (`forced=False, restore=True`) independent of the `force` flag. | +| `_restore_block_ownership` | method | task.py:8526 | Factored out of `_apply_pre_block_restore` (b3558d4e complexity split): applies snapshotted status/owner restore (branchless in_progress→pending divert), returns `(pre_status, restored_status, restored_owner)`. | +| `_emit_admin_override_audit` | method | task.py:8555 | Factored out of `_apply_pre_block_restore`: writes `task.admin_override` audit row for admin-triggered blocked restores (`forced=False, restore=True`). | +| `claim` | method | task.py:3112 | `FOR UPDATE` lock + `_validate_claim_preconditions` + `_finalize_claim`; calls `_validate_and_set_status(claimed)`. | +| `_validate_claim_preconditions` | method | task.py:2883 | Per-claim validator chain: status, `_claim_blocked_by_sequencing` (dependency + sequence), team, pre-assignment theft, self-review. | +| `_claim_blocked_by_sequence` | method | task.py:2805 | Strict sibling-sequence gate: a PENDING/`needs_revision` task with parent + effective `sequence` (`COALESCE(sequence, 0)`) N is held while any same-parent sibling with a strictly lower effective sequence is non-terminal — assignee-blind, independent of `dependency_ids`. Ties run parallel; cancelled siblings never block. | +| `_claim_blocked_by_dependencies` | method | task.py:2781 | `unmet_dependency` TIMING gate: refuses claim while any `dependency_ids` entry is non-terminal. | +| `is_pending_claim_blocked` | method | task.py:2864 | Read-only wrapper over `_claim_blocked_by_sequencing` (dependency OR sequence) so the orchestrator dispatcher can filter a doomed claim before attempting it (`_pending_claim_blocked` in orchestrator.py). | +| `stamp_wave_sequence` | method | task.py:7452 | Stamps a freshly delegated subtask's `sequence` as `1 + max(sequence of each same-parent dependency target)`, or `0` when independent — so independent siblings tie (parallel under the sequence gate) while colliding/ordered work ascends. Runs POST-wiring (after the collision DAG / cross-cell edges land); PM-authored sequences are never rewritten. | +| `_apply_dependency_lineage` / `_merge_one_dependency` | method | task.py:2308 / 2337 | Claim-time content assist (not a gate): merges each same-repo dependency's landed work into a freshly cut branch when it lies outside the branch's own ancestor chain (`GitService.merge_dependency_lineage`); a real conflict aborts the merge and stamps a `dependency_lineage_conflict` transition note instead of failing the claim. | +| `_finalize_claim` | method | task.py:2925 | Work-session create/inherit, branch cut, proactive-context injection. | +| `_inject_proactive_context` | method | task.py:3154 | Briefing injection at claim (institutional memory when `org_memory_enabled`). | +| `_completion_learnings_for` | method | task.py:2798 | Distill one lesson (ON) vs legacy raw capture (OFF). | +| `_extract_completion_learnings` | method | task.py:2837 | Fire-and-forget learning record + RAG indexing. | +| `start` | method | task.py:3354 | `claimed→in_progress`. | +| `unclaim_for_agent` / `_force_unclaim_to_pending` | method | task.py:3579 / 3507 | Release claim to pool; abandon stale work session. | +| `block` / `soft_block` / `unblock` | method | task.py:3760 / 3823 / 3897 | Snapshot pre-block owner; restore on unblock. | +| `submit_for_qa` | method | task.py:4065 | `verifying→awaiting_qa`; clears claimed_by (passes explicit audit_agent_id). | +| `pass_qa` / `fail_qa` | method | task.py:4112 / 4187 | QA verdict; `fail_qa` routes back to original dev (marker → work-session fallback), then calls `_alert_auditor_of_rework` after flush to emit a best-effort auditor rework ALERT. | +| `_resolve_revision_dev` | method | task.py:4301 | Work-session fallback when `original_developer` marker missing. | +| `docs_complete` | method | task.py:4336 | `awaiting_documentation→awaiting_pm_review` (parallel completion). | +| `request_changes` | method | task.py:9975 | PM merge-review request-changes path; transitions to `needs_revision`, then calls `_alert_auditor_of_rework` after flush to emit a best-effort auditor rework ALERT. | +| `submit_for_pm_review` / `complete` | method | task.py:4690 / 4882 | PM review submit + completion / CEO escalation chain. | +| `_apply_complete_approval_chain` | method | task.py:4811 | Leaf→completed vs root→awaiting_ceo_approval. | +| `_assert_pr_merged_for_complete` | method | task.py:4845 | PR-merged gate before `complete`. | +| `apply_escalation` | method | task.py:4942 | `in_progress→blocked` direct status set + audit emit (bypasses validator by design). | +| `escalate_to_ceo` | method | task.py:5064 | `awaiting_pm_review→awaiting_ceo_approval`; gained `actor_agent_id: UUID | None = None` param (stamped as `audit_agent_id` so the transition row attributes to the specific PM/Board agent, not just the role). | +| `ceo_approve` | method | task.py:5146 | CEO merges then approves; `awaiting_ceo_approval→completed`. | +| `ceo_reject` | method | task.py:5414 | Reject → `needs_revision` (dev) or `pending` (branchless root via admin_set_status); now validates `reason` (`reject_trivial` — previously an uncaught Pydantic error could 500 on empty/trivial input) and inserts one `origin=ceo` Finding onto the revision-findings ledger; the branchless-root path manually bumps `revision_count` + emits `task.ceo_reject` since it skips `_emit_status_transition_audit`. See `docs/map/review-findings.md`. | +| `_delete_task_branch_best_effort` | method | task.py:6726 | Cancel-path cleanup: remote branch delete + `_remove_task_worktree_best_effort(force_branch_delete=True)`; skipped once branch is unset. | +| `_remove_task_worktree_best_effort` | method | task.py:6767 | Shared worktree+local-branch+previews cleanup called by both cancel and terminal paths; force-deletes the local branch ref unless it's an environment-ladder rung (`effective_environments`). | +| `_cleanup_task_previews_best_effort` | method | task.py:6804 | `rmtree` the task's `.previews/{task8}` video-render dir; path-containment-checked against the project workspace dir before deleting. | +| `_remove_task_worktree_on_terminal` | method | task.py:6829 | Best-effort worktree + local-branch (force `-D`, squash-merge is never an ancestor) + previews cleanup on complete/ceo_approve; no-op for branchless. | +| `cancel` | method | task.py:5644 | Cascade-cancel descendants through the validator. | +| `reassign` / `reassign_active_claim` | method | task.py:7657 / 7807 | Reassignment with Board/Main-PM diversion guards. | +| `pr_pass` / `pr_fail` | method | task.py:8100 / 8137 | In-path PR-review gate verdicts; `pr_fail` transitions to `needs_revision`, then calls `_alert_auditor_of_rework` after flush to emit a best-effort auditor rework ALERT. | +| `list_open_docs_sync_tasks` | method | task.py:1580 | Returns open `source=docs_sync` tasks, optionally scoped to one release version via the `docs_sync_release_version` marker. The version predicate is applied in SQL so dedupe/cap checks do not haul every open row into Python. | ## Data Flow -Both mixins are composed into the Choreographer and invoked by the flow MCP server (roboco-flow) when a pr_reviewer agent calls a verb. Inputs: reviewer_agent_id + task_id (+ notes/issues for verdicts, + body/event/findings for post_pr_review). Each verb fetches the task (self.task.get), resolves the agent role (self.task.agent_for), builds a briefing (self._briefing_for), runs the spec gate (spec_module.can_invoke_intent) and claim guards (self._run_claim_guards), then either calls a service claim (self.task.pr_gate_claim / pr_review_claim — claim WITHOUT transition for the gate, pending→in_progress for external) or the verb runner (self._verb_runner().run_intent) for the transition. Gate verdicts are authored as structured pr_review notes (apply_structured_note) BEFORE the transition and posted to the GitHub PR AFTER (self.git.post_pr_review); pr_fail also captures the PR head SHA (self.git.get_pr_head_sha) and a2a's the owning PM (self.a2a.send). Outputs: standardized Envelope (ok with status/next/evidence/context_briefing, or error with remediate). Callers: the flow verb dispatcher + HTTP routes /api/v1/flow/pr_reviewer/*. Callees: TaskService (get/agent_for/pr_gate_claim/pr_review_claim), JournalService (has_learning_for_task), GitService (diff/get_pr_diff/get_pr_head_sha/post_pr_review), NotificationService, A2AService, ProjectService/ProductService (via resolve_task_project_slug), foundation policy (lifecycle.can_invoke_intent, tracing.check_requirements, content.validate_content/markers). +Request → `TaskService` loads `TaskTable` (`get`/`_load_task_or_raise`) → validates role/transition (`validate_task_transition`) + git reqs (`validate_git_requirements`, branchless/umbrella/external-review exempt) → mutates columns → `_emit_status_transition_audit` writes `AuditLogTable` row + bumps `revision_count` in the same session → pokes orchestrator `trigger_dispatch()` → fires fire-and-forget background tasks (RAG indexing, learning distillation, worktree cleanup, work-session close). Terminal states trigger `_unblock_dependents` to revive waiting tasks. ## Mermaid ```mermaid stateDiagram-v2 - direction LR - [*] --> awaiting_pr_review: PM submit_up/submit_root - awaiting_pr_review --> claimed_gate: claim_gate_review (no transition) - claimed_gate --> awaiting_pm_review: pr_pass - claimed_gate --> needs_revision: pr_fail (captures head_sha + a2a PM) - needs_revision --> awaiting_pr_review: PM re-submit (unchanged-PR gate hard-blocks if head_sha identical) - awaiting_pm_review --> [*]: PM merge/escalate + [*] --> backlog: create + backlog --> pending: activate (PM) + pending --> claimed: claim (role-matched) + claimed --> in_progress: start + claimed --> pending: unclaim + in_progress --> blocked: block + blocked --> in_progress: unblock(restore) + in_progress --> verifying: submit_for_verification + verifying --> awaiting_qa: submit_for_qa (PR open) + awaiting_qa --> awaiting_documentation: pass_qa + awaiting_qa --> needs_revision: fail_qa + awaiting_documentation --> awaiting_pm_review: docs_complete + in_progress --> awaiting_pr_review: submit_up/submit_root (PM) + awaiting_pr_review --> awaiting_pm_review: pr_pass + awaiting_pr_review --> needs_revision: pr_fail + awaiting_pm_review --> completed: complete (leaf) + awaiting_pm_review --> awaiting_ceo_approval: escalate_to_ceo (root) + awaiting_ceo_approval --> completed: ceo_approve + awaiting_ceo_approval --> needs_revision: ceo_reject (dev) + awaiting_ceo_approval --> pending: ceo_reject (branchless root) + needs_revision --> claimed: re-claim + completed --> [*] + cancelled --> [*] +``` - state claim_gate_review_evidence [\"diff inline (read-only)\"] - state pr_pass_blocked [\"toolchain_broken? conventions block? -> refuse pass\"] - state post_verdict [\"record pr_review note + post to PR (COMMENT on root→master) + a2a on fail\"] +## Logical Tree +- TaskService + - State core: `_validate_and_set_status`, `_emit_status_transition_audit`, `admin_set_status`, `_restore_block_ownership`, `_emit_admin_override_audit` + - Create/shape: `create`, `_validate_parent_depth`, `_validate_batch_membership`, `activate` + - Branch/worktree: `_ensure_branch_for_task`, `_auto_create_branch`, `_delete_task_branch_best_effort`, `_remove_task_worktree*`, `_cleanup_task_previews_best_effort` + - Claim: `claim`, `_validate_claim_preconditions`, `_claim_blocked_by_sequence`, `_claim_blocked_by_dependencies`, `_finalize_claim`, `_apply_dependency_lineage`, `_inject_proactive_context`, `acquire_*_lock` + - Lifecycle verbs: `start`, `block*`, `unblock`, `pause`, `resume`, `submit_for_qa`, `pass_qa`, `fail_qa`, `docs_complete`, `submit_for_pm_review` + - Completion: `complete`, `_apply_complete_approval_chain`, `ceo_approve`, `ceo_reject`, `cancel` + - Rework routing: `fail_qa`, `_resolve_revision_dev`, `ceo_reject` + - Learning/indexing: `_completion_learnings_for`, `_extract_completion_learnings`, `_trigger_completion_hooks`, `_index_*_background` + - Dependencies/sequencing: `add_dependency`, `wire_sibling_collision_dag`, `wire_cell_task_wave_chain`, `_unblock_dependents` + - Reassign/escalate: `reassign*`, `escalate*`, `_maybe_divert_*` + - PR gate: `pr_gate_claim`, `submit_for_review`, `pr_pass`, `pr_fail` + - Queries: `list_*`, `count_*`, `*_ac_coverage`, `all_subtasks_terminal` - direction TB - [*] --> pending_ext: external/fork PR review task - pending_ext --> in_progress_ext: claim_pr_review (pending→in_progress, branch-exempt) - in_progress_ext --> completed: post_pr_review (in_progress→completed) - state post_pr_review_gates [\"verdict consistency + no-hand-format + tracing\"] - state post_side [\"GitHub review post + CEO notify (best-effort)\"] +## Dependencies +- `roboco.foundation.policy.lifecycle` (transitions, role restrictions, git requirements, `is_branchless_coordination`, `is_batch_umbrella`) +- `roboco.foundation.policy.batch` / `sequencing` (batch predicates, sibling DAG) +- `roboco.services.work_session` (close/abandon), `roboco.services.workspace`, `roboco.services.learning`, `roboco.services.memory_distiller` +- `roboco.services.conventions` (`_attach_baseline_constraints`) +- `roboco.db.tables` (`TaskTable`, `AuditLogTable`, `WorkSessionTable`, `ProjectTable`) +- `roboco.api.deps.get_orchestrator` (lazy; dispatch poke), `roboco.config.settings` +- Markers / `extract_original_developer` helpers + +## Entry Points +- `TaskService.create` / `create_subtask` — task creation (orchestrator intake, batch confirm, gateway delegate). +- `TaskService.claim` — gateway `give_me_work` / `i_will_work_on` / `claim_review` / `claim_doc_task`. +- Lifecycle verbs (`start`, `submit_for_qa`, `pass_qa`/`fail_qa`, `docs_complete`, `submit_for_pm_review`, `complete`, `cancel`, `pr_pass`/`pr_fail`, `escalate_*`, `ceo_approve`/`ceo_reject`) — all gateway flow verbs. +- `admin_set_status` — operator PATCH + orchestrator auto-recover. +- `wire_*` / `add_dependency` — `SequencingService` / `BatchPlacement`. + +## Config Flags +- `ROBOCO_ORG_MEMORY_ENABLED` — `_completion_learnings_for` swaps raw capture for one distilled lesson (task.py:2810). +- `ROBOCO_CONVENTIONS_ENABLED` — `_attach_baseline_constraints` skipped when off (task.py:1000). +- (Indirect, via called services) `ROBOCO_SELF_HEAL_*`, `ROBOCO_CI_WATCH_*`, `ROBOCO_DEP_UPDATE_*`, `ROBOCO_RELEASE_MANAGER_*` gate the `list_open_*`/`list_open_release_proposals` query paths. + +## Gotchas +- `_emit_status_transition_audit` writes the audit row in the CALLER's session — callers that clear `claimed_by` before transitioning MUST pass `audit_agent_id` or the row lands unattributed (task.py:688). +- `apply_escalation` (task.py:4942) sets `task.status` directly and calls `_emit_status_transition_audit` deliberately bypassing the strict validator (blocked is a terminal-ish hold) — only audited privileged-style path besides `admin_set_status`. +- `fail_qa` accepts `claimed`/`in_progress` (QA is mid-review); the `original_developer` marker is unreliable — the work-session fallback (`_resolve_revision_dev`) is load-bearing (task.py:4248). +- Branchless/umbrella/external-review tasks are exempt from the branch gate inside `GitContext` (task.py:597-611); umbrella is also exempt from the `awaiting_pm_review→awaiting_ceo_approval` pr_number gate. +- `complete()` requires PR merged (`_assert_pr_merged_for_complete`) EXCEPT branchless roots; `ceo_approve` separately checks `work_session.pr_status=="merged"` and refuses otherwise. +- Background indexing/learning/cleanup tasks are tracked on `self._background_tasks` and are best-effort — a failure never blocks the transition. +- The sequence gate (`_claim_blocked_by_sequence`) is enforced ONLY in `_validate_claim_preconditions`, i.e. inside `claim` itself — both the gateway claim verbs AND the orchestrator's raw dispatch claim cross it because they both funnel through `TaskService.claim`, unlike the pre-#382 dependency gate which briefly lived only on the gateway side. Any future claim path that bypasses `TaskService.claim` (a raw `admin_set_status`, for instance) does NOT get sequence enforcement. +- `_apply_dependency_lineage` is scoped to SAME-REPO dependencies only (`dep_task.project_id != ctx.project.id` short-circuits) — a cross-repo dependency edge (e.g. a MegaTask root-subtask in another project) has no shared git history to merge and is silently skipped; the dependency TIMING gate still holds the claim regardless of repo. +- `TaskTable.orchestration_markers` is generic `JSON`, not `JSONB`. Any SQL predicate on a marker key must use `.as_string()` (or the JSON dialect's generic comparator), not `.astext`, which is JSONB-only and raises `AttributeError` at compile time. `list_open_docs_sync_tasks(version=...)` at task.py:1596 is the current example; the inline comment records the rationale. +- Both cancel and terminal-completion now force-delete (`-D`) the task's LOCAL branch ref in the assignee's clone alongside the worktree — a completed task's PR was squash-merged (its local ref is never an ancestor of base, so a "safe" `-d` refuses unconditionally) and a cancelled task's work is discarded by decision, so the ref is spent either way. Skipped when the branch name coincides with an environment-ladder rung (`effective_environments`), which outlives any one task. + +## Drift from CLAUDE.md +- CLAUDE.md states ceo_reject "~4779 skips _validate_and_set_status in branchless path". Actual: branchless branch of `ceo_reject` is at task.py:5488 and routes through `admin_set_status` (which DOES emit audit at task.py:2100). The non-branchless branch DOES call `_validate_and_set_status` (task.py:5461). No audit gap — the line reference is stale. +- CLAUDE.md "PR is created BEFORE QA review" — `submit_for_qa` enforces `pr_number` via `validate_git_requirements` (consistent, no drift). +- CLAUDE.md verb table lists `pr_reviewer` `pr_pass`/`pr_fail` — present at task.py:8100/8137 (consistent). + +## Changes Since Baseline +`git log fd10cc86..HEAD -- roboco/services/task.py`: +- `15effce0` Chore: 141 Gaps fill-in (#283) — bulk gap closure; transition audit chokepoint + `revision_count` centralization (task.py:685-706), branchless/umbrella git-context exemptions, fail_qa work-session fallback, ceo_reject branchless routing. +- `3aff6e04` Chore: Close gaps (#285) — follow-on gap close (worktree-on-terminal cleanup F123 Phase C, escalation audit emit, rework routing hardening). + +> Post-snapshot updates (since 2026-06-29): `20f1f9ba` admin_set_status: thread actor_id/actor_role into `_apply_pre_block_restore`; blocked→pending/in_progress restore now attributes the audit row to the admin actor (not the restored owner) and emits a `task.admin_override` row (forced=False, restore=True) independent of the force flag. `b3558d4e` complexity: extract `_restore_block_ownership` (line 8526) + `_emit_admin_override_audit` (line 8555) from `_apply_pre_block_restore` — no behavior change, splits a C-rank block for the xenon gate. `0e7674af` escalate_to_ceo gains `actor_agent_id: UUID | None = None` param stamped as audit_agent_id; push_branch / create_pr / create_root_pr / escalate_to_ceo side-effect handlers in the verb runner now forward actor_agent_id (was dropped, causing wrong workspace or role-only audit attribution). `8f3f4236` (#452) "sequence is the bar" — adds `_claim_blocked_by_sequence` + `_validate_claim_preconditions` wiring, `stamp_wave_sequence` (replacing a raw per-sibling delegation ordinal), and migration 069 (`tasks.parent_task_id` index, the sibling probe's hot path). `f2834cf5` (#466) adds `_apply_dependency_lineage`/`_merge_one_dependency`, called from `_create_branch_in_project` right after a fresh branch cut. `61e00832` (PR #492) added `_alert_auditor_of_rework()` and invoked it from `fail_qa`, `pr_fail`, and `request_changes` after each transition to `needs_revision`, wiring the reactive auditor ALERT path. `f6c75237` (PR #509) restored those `_alert_auditor_of_rework()` calls after they were accidentally deleted by the docs-sync PR: all three call sites now dispatch the alert immediately after `await self.session.flush()` so the `needs_revision` transition row is committed before the auditor notification is created. The same commit also changed the descendant-traversal casts in `_supersede_replacement_landed` and `get_all_descendants`, but it used `cast(UUID, child.id)` with a scoped `# noqa: TC006` and `child.id` with a `# type: ignore[arg-type]`, respectively. `e4b7dd0f` / PR #511 reverted those two cast regressions to the preferred string-literal form `cast('UUID', child.id)` with no lint or type suppression, leaving `DOCS_SYNC_SOURCE` and `list_open_docs_sync_tasks` untouched. +> +> (uncommitted, branch `feature/findings-ledger`, 2026-07-11) Revision-findings ledger: `_audit_events_for` (task.py:997) gains `task.request_changes` (agent_role `cell_pm`/`main_pm`) and `task.ceo_reject` (agent_role `ceo`) branches alongside the existing `task.qa_fail`/`task.pr_fail`; `ceo_reject` gains reason validation + a ledger `Finding` insert (see above); `qa_fail` and `request_changes` drop their raw `dev_notes` appends (the mirror-column data-loss bug) in favor of the ledger + a structured note. Full detail: `docs/map/review-findings.md`. +> +> (open PR #548, branch `feature/wave-2-hygiene-charts`, 2026-07-17) Local branch refs stop leaking: `_delete_task_branch_best_effort`/`_remove_task_worktree_on_terminal` now also force-delete the assignee's local branch ref (via new `WorkspaceService.delete_local_branch`) and rmtree the task's `.previews/{task8}` video-preview dir, both skipped for environment-ladder rungs. See `docs/map/worksession-git.md` for the paired `GitService.cleanup_stale_branches` sweep. + +## Regression Risks + +| Title | File:Line | Claim | Severity | +|-------|-----------|-------|----------| +| `ceo_approve` skips work-session close | task.py:5146 | `ceo_approve` calls `_remove_task_worktree_on_terminal` but NOT `_close_work_session_for_task` (only `complete()` at 4934 does). Approved-via-CEO tasks leave the WorkSession row not marked `completed`/closed → reporting/session-resolution drift. | High | +| `ceo_approve` skips full completion hooks | task.py:5200-5210 | Only fires `_extract_completion_learnings` manually; skips `_trigger_completion_hooks` so code-changes RAG indexing + decision indexing never run for CEO-approved (root) tasks. | Medium | +| `apply_escalation` bypasses validator | task.py:4942 | Sets `task.status` directly then emits audit; a caller passing a wrong target status would skip `validate_task_transition`/git-req checks. Relies on call-site discipline. | Medium | +| `fail_qa` route depends on unreliable marker | task.py:4228-4272 | Fast path reads `original_developer` marker; if absent, falls to `_resolve_revision_dev`. If both miss (no dev work session, e.g. parent-only edit) task is unassigned to pool → PM may grab a dev task (the original 2026-06-27 loop). | High | +| Branchless `ceo_reject` uses `admin_set_status` | task.py:5488 | Bypasses strict validator (intended) but `awaiting_ceo_approval→pending` is not in `VALID_TRANSITIONS`; any future tightening of admin override could wedge coordination-root rejection. | Medium | +| `revision_count` bump is in audit helper only | task.py:702-706 | Any future transition path that sets `task.status` directly WITHOUT calling `_emit_status_transition_audit` (mirroring `apply_escalation`'s pattern) would silently skip the rework counter — metric drift. | Medium | +| `_remove_task_worktree_on_terminal` silent-fail | task.py:5614-5627 | Cleanup failure is logged-warning only; on recurring FS/permission error worktrees leak indefinitely with no operator signal beyond logs. | Low | +| Concurrent mid-verb state change | task.py:548 | `_validate_and_set_status` does not re-fetch the task after validation; a concurrent committer could flip status between load and set, producing an invalid edge that the validator already passed. Mitigated upstream by verb-runner savepoints, not here. | Medium | +| `cancel` cascade swallows role violations | task.py:5679-5690 | Descendants that fail role validation are skipped (warning), so a cancel can leave non-terminal descendants orphaned in `awaiting_ceo_approval` (only CEO may cancel those). | Medium | +| `submit_for_qa` clears `claimed_by` before transition | task.py:568-572, 4065 | Relies on `audit_agent_id` being passed to attribute the row to the dev; if a future caller forgets, the `awaiting_qa` audit row lands `agent_id=NULL`. | Low | + +## Health +`TaskService` is the most load-bearing service and the most hardened: the audit chokepoint, `revision_count` centralization, branchless/umbrella exemptions, and worktree-on-terminal cleanup all landed in the two recent gap-closure commits. The residual risk is concentrated in the two CEO-path asymmetries (`ceo_approve` not closing the work session / not running the full completion hooks) and in `fail_qa`/`ceo_reject` rework routing, which depends on the unreliable `original_developer` marker and a work-session fallback that has no guarantee a developer session exists. +# RoboCo Slice Map — `worksession-git` + +Scope key: `worksession-git` Repo root: `/Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco` Files in scope: +- `roboco/services/work_session.py` +- `roboco/services/git.py` +- `roboco/templates/git/` (`__init__.py`, `branch.py`, `commit.py`, `constants.py`, `pr_internal.py`, `pr_root.py`) +- `roboco/services/forge/` (`__init__.py`, `base.py`, `github.py`, `gitea.py`, `gitlab.py`, `registry.py`, `router.py`, `shaping.py`) +- `roboco/foundation/policy/forge.py` +- `roboco/foundation/policy/pr_labels.py` + +## Purpose + +This slice is the git substrate every delivery agent works on. `GitService` runs all git subprocesses (status/commit/branch/push/rebase), mints branches + commit messages + PR bodies from templates, and drives the REST API for PR create/merge/close — now routed through a **multi-forge layer** (`roboco/services/forge/`) instead of talking to GitHub directly. `WorkSessionService` persists the per-claim row that links an agent to a task's branch/commits/PR and enforces the single-active-per-task invariant. The `roboco/templates/git/` package is the pure rendering layer for branch names, commit messages, and internal/root PR bodies. `roboco/foundation/policy/pr_labels.py` derives the org-structure label vocabulary (`to master`/`to slave`, `root`, `MegaTask`, layer labels) applied at PR-open. Together they are the boundary between the task lifecycle and the actual git history on disk + the configured forge (GitHub, Gitea, or GitLab). + +## Files + +| Path | Role | approx LOC | +|------|------|------------| +| `roboco/services/work_session.py` | WorkSession CRUD, commit/file tracking, PR-lifecycle record, single-active invariant | 685 | +| `roboco/services/git.py` | Git subprocess execution, branch/commit/PR/rebase/merge/sync, GitHub REST API, conventions validator runner | 4596 | +| `roboco/templates/git/__init__.py` | Package re-exports for branch/commit/PR templates | 48 | +| `roboco/templates/git/branch.py` | Hierarchical branch name builder + root-task resolver | 131 | +| `roboco/templates/git/constants.py` | `BRANCH_TYPES`, `COMMIT_TYPES`, `MAX_TASK_DEPTH`, length constants | 52 | +| `roboco/templates/git/commit.py` | `CommitContext` + `build_commit_message` (traceability links) | 114 | +| `roboco/templates/git/pr_internal.py` | Internal (subtask→parent) PR title/body builder | 140 | +| `roboco/templates/git/pr_root.py` | Root (→master, CEO-level) PR title/body builder with task tree | 245 | +| `roboco/services/forge/__init__.py` | Package re-exports (`ForgeRouter`, `GitProvider`, `GitHubProvider`, `GiteaProvider`, `GitLabProvider`, `RepoRef`, `provider_for`, `register_project_forge`) | 32 | +| `roboco/services/forge/base.py` | Pure contract: `RepoRef` dataclass + the `GitProvider` ABC (~20 abstract methods every forge implements) | 208 | +| `roboco/services/forge/github.py` | GitHub.com/GHE REST transport — the pre-existing inline `httpx` logic pulled out unchanged; owns the sole retry policy in the package (`list_ci_runs`) | 432 | +| `roboco/services/forge/gitea.py` | Self-hosted Gitea REST transport, adapting the wire contract back into GitHub shapes | 471 | +| `roboco/services/forge/gitlab.py` | GitLab REST v4 transport — heaviest adaptation (MR `iid`, diff reassembly, pipelines) | 679 | +| `roboco/services/forge/registry.py` | Host↔provider(+scheme) map + `provider_for(project)` resolution, self-healing per-process | 104 | +| `roboco/services/forge/router.py` | `ForgeRouter` — implements `GitProvider` by picking a transport per call from `RepoRef.host` | 195 | +| `roboco/services/forge/shaping.py` | `ShapedResponse` — an `httpx.Response`-compatible stand-in a non-GitHub provider returns when it must synthesize a status the wire call didn't produce (e.g. a shaped 501 for `merge_branch`) | 51 | +| `roboco/foundation/policy/forge.py` | Pure host/provider detection (`extract_host`, `detect_provider`) + registration-time validation (`validate_project_forge`) | 99 | +| `roboco/foundation/policy/pr_labels.py` | `derive_pr_labels` — the org-structure label vocabulary (`to `, `root`, `MegaTask`, layer labels) applied at every PR-open site | ~90 | + +## Key Symbols + +| Name | Kind | File:Line | Responsibility | +|------|------|-----------|----------------| +| `WorkSessionService` | class | work_session.py:29 | Session lifecycle + single-active invariant | +| `WorkSessionService.create` | method | work_session.py:50 | Validate project/task, refuse duplicate, supersede stale ACTIVE, insert row | +| `WorkSessionService.get_active_for_task` | method | work_session.py:184 | Most-recent ACTIVE row (resilient to dup-rows defect) | +| `WorkSessionService.supersede_active_sessions_for_task` | method | work_session.py:247 | ABANDON every other ACTIVE session for a task (single-active) | +| `WorkSessionService.add_commit` | method | work_session.py:363 | Append dedup'd commit SHA to session.commits | +| `WorkSessionService.create_pr` | method | work_session.py:427 | Record pr_number/pr_url/pr_created_at | +| `WorkSessionService.merge_pr` | method | work_session.py:488 | Record merge + COMPLETED; idempotent active-guard (F062) | +| `WorkSessionService.close` | method | work_session.py:618 | Idempotent COMPLETED on task completion | +| `WorkSessionService.abandon` | method | work_session.py:577 | ABANDONED + ended_at (non-active → warning + None) | +| `WorkSessionService.has_unpushed_commits` | method | work_session.py:662 | PR-existence proxy for unpushed commit detection | +| `WorkSessionService.task_team_for_session` | method | work_session.py:147 | Return the task's team (cell) for a given session; used by route layer's PM cell-ownership check on `merge_pr` | +| `get_work_session_service` | factory | work_session.py:682 | Construct service from AsyncSession | +| `GitService` | class | git.py:234 | All git operations + GitHub API | +| `_GIT_EXECUTOR` | module const | git.py:116 | Dedicated ThreadPoolExecutor for git subprocesses (16 workers) | +| `resolve_git_dir` | func | git.py:129 | Resolve `.git` dir for clone OR linked worktree | +| `_remove_stale_git_locks` | func | git.py:156 | Best-effort clear orphaned `.git/**/*.lock` after timeout SIGKILL | +| `_select_ci_head_run` | func | git.py:220 | Pick CI run matching current HEAD (anti-stale-green) | +| `GitService._run_git` | method | git.py:247 | Run git subprocess in dedicated pool, token header, chown-back, lock cleanup | +| `GitService._token_for_project` | method | git.py:344 | Decrypted project PAT (logs loudly on key-rotation failure) | +| `GitService.get_workspace` | method | git.py:387 | Resolve/clone agent workspace (auto_clone aware) | +| `GitService.get_status` | method | git.py:499 | Porcelain status + ahead/behind | +| `GitService._classify_porcelain` | static | git.py:439 | Split porcelain into staged/unstaged/untracked (column-safe) | +| `GitService._parse_git_url` | static | git.py:557 | Extract (owner,repo) from tokened/https/ssh GitHub URL | +| `GitService.create_commit` | method | git.py:640 | Stage + commit with template message + worktree ensure | +| `GitService._worktree_for_task` | static | git.py:747 | Per-task worktree path `{clone_root}/.worktrees/{task_id[:8]}` (F123) | +| `GitService._ensure_worktree_for_commit` | method | git.py:756 | Re-attach a pruned worktree before cwd-dependent op | +| `GitService._assert_on_task_branch` | method | git.py:771 | Recover drifted clone onto task branch (never discards work) | +| `GitService.commit_for_task` | method | git.py:875 | Agent-facing commit verb backing the `commit` content tool | +| `GitService.create_branch` | method | git.py:989 | Build branch name, fetch base, `worktree add` (F123), push -u | +| `GitService.create_branch_for_task` | method | git.py:1188 | Resolve workspace/team, create branch, commit DB | +| `GitService.checkout_branch_for_agent` | method | git.py:1276 | Allowlist-bounded checkout for agent verb | +| `GitService.push_for_task` | method | git.py:1428 | Push the task's recorded branch by name (clone-checkout-independent) | +| `GitService.push_task_branch` | method | git.py:1467 | Gateway branch-keyed push | +| `GitService.create_pull_request` | method | git.py:2132 | Open PR via GitHub API (legacy project-scoped) | +| `GitService.create_pr_for_task` | method | git.py:2639 | Agent-facing open_pr verb | +| `GitService.update_pr_for_task` | method | git.py:2506 | Patch PR title/body; 404→typed GitError | +| `GitService.get_pr_head_sha` | method | git.py:2451 | PR head SHA for pr_fail re-submit loop guard (fail-open) | +| `GitService.get_latest_ci_conclusion` | method | git.py:1929 | Per-project CI signal (unknown never false-green) | +| `GitService.get_pr_ci_status` | method | git.py:2662 | CI status of a PR's current head commit for the in-path pr_pass gate; returns {state, failing_checks?, head_sha} or None on config gaps (fail-open) | +| `GitService._ci_status_prereqs` | method | git.py:2698 | Resolve (owner, repo, auth headers, head_sha) for CI-status lookup or None on any gap | +| `GitService._fetch_check_runs` | method | git.py:2730 | GET check-runs for head_sha; None on any API failure | +| `GitService._classify_check_runs` | method | git.py:2770 | State classification: success/failure/pending from check-run conclusions list | +| `GitService._classify_zero_check_runs` | method | git.py:2790 | State classification when zero check-runs exist: pending_not_scheduled or no_ci_configured (depends on workflow count) | +| `GitService.list_open_prs` | method | git.py:1844 | Normalized open-PR list | +| `GitService.post_pr_review` | method | git.py:2329 | Post reviewer comments via GitHub API | +| `GitService.merge_pull_request` | method | git.py:2914 | GitHub merge API + method fallback + already-merged disambiguation | +| `GitService.merge_pr_for_task` | method | git.py:3046 | Role-gated merge + recorded-PR verification + auto-complete | +| `GitService._assert_merge_role` | method | git.py:2989 | PM/CEO approval-chain role gate | +| `GitService.pr_merge` | method | git.py:3616 | Gateway merge: project_id-scoped, parent row lock, 409 retry, CEO-only default guard | +| `GitService._merge_with_retry` | method | git.py:3555 | Single 409 retry + already-merged disambiguation → MergeConflictError | +| `GitService._lock_parent_task_for_merge` | method | git.py:3503 | SELECT FOR UPDATE on parent task (sibling merge serialization) | +| `GitService._resolve_merger_id` | static | git.py:3530 | merged_by attribution: actor→assigned→created→UUID(0) | +| `GitService.rebase_onto_base` | method | git.py:3733 | Rebase primitive: rebased/superseded/conflicts classification | +| `GitService.rebase_pr_for_task` | method | git.py:3792 | PR-keyed rebase via PR refs (project_id scoped) | +| `GitService.sync_task_branch` | method | git.py:3847 | Task-keyed rebase through dev `sync_branch` verb (pre-PR) | +| `GitService.is_behind_base` | method | git.py:3889 | `(behind, ahead)` counts for i_am_done submit gate | +| `GitService.close_pull_request` | method | git.py:3940 | Close superseded PR + optional comment + branch cleanup (idempotent) | +| `GitService._delete_remote_branch_best_effort` | method | git.py:3608 | Best-effort remote delete; skips main/master/develop + open-dependent-PR branches; returns `bool` (issued vs skipped/failed) | +| `GitService.delete_task_branch` | method | git.py:3671 | Cancel-path remote branch delete; chokepoint for the environment-ladder skip (`effective_environments`) so a task's `branch_name` can never collide-delete a ladder rung; returns `bool` | +| `GitService.close_task_pr_best_effort` | method | git.py:3655 | No-clone-needed cancel-path cleanup: resolves the project token + `git_url`→`RepoRef`, fetches the PR via `self._forge.get_pr`, no-ops unless `state=="open"`, else `update_pr(payload={"state":"closed"})`; every failure path (missing token/project, unparseable URL, `httpx.HTTPError`) returns `False` rather than raising | +| `GitService.cleanup_stale_branches` | method | git.py:3697 | `POST /git/branches/cleanup` backing sweep: terminal (completed/cancelled) tasks' branches, remote (`delete_task_branch`) + local force-delete in the assignee's clone; capped 200/call, cursor-resumable | +| `GitService._stale_branch_window` | method | git.py:3767 | One deterministic `ORDER BY id` window of sweep candidates; ladder rungs excluded from results but still advance the cursor | +| `GitService._live_task_dependents` | method | git.py:3810 | Companion sweep guard: excludes a terminal candidate's branch when a non-terminal task still records that exact `branch_name`, or when a non-terminal task is a direct child of the candidate (the child's future PR base would resolve to the parent's branch via `resolve_parent_branch` even before it has opened a PR — catches what the OPEN-PR-only `_branch_has_open_dependents` can't see) | +| `GitService._cleanup_one_stale_branch` | method | git.py:3841 | Per-branch remote+local delete for one sweep candidate; raises on unexpected failure so the caller's try/except counts it as an error | +| `GitService.pr_target` | method | git.py:4021 | Return PR base branch (project_id scoped) | +| `GitService.create_pr` | method | git.py:3418 | Branch-keyed open PR (gateway path; ensures base on remote) | +| `GitService._record_pr_atomically` | method | git.py:2601 | Atomic pr_number/url write to task | +| `GitService.run_pre_submit_quality_gate` | method | git.py:3208 | `make quality` gate before submit | +| `GitService.conventions_check_for_task` | method | git.py:4368 | Run conventions validator on changed files (fail-closed) | +| `GitService._run_conventions_validator` | method | git.py:4408 | Subprocess `python -m roboco.conventions` with 120s cap | +| `GitService.open_conventions_pr` | method | git.py:4456 | Scaffold `.roboco/conventions.yml` on a branch + open PR | +| `GitService.diff` / `list_changed_files` / `read_file_at_branch` | methods | git.py:4192/4225/4259 | Read-only git queries (gateway + routes) | +| `GitService.commit` | method | git.py:4286 | Gateway content-verb commit (branch-keyed) | +| `get_git_service` | factory | git.py:4594 | Construct GitService from AsyncSession | +| `build_branch_name` | func | templates/git/branch.py:37 | `{type}/{team}/{root}--{sub}--...` up to MAX_TASK_DEPTH | +| `get_root_task_id` | func | templates/git/branch.py:97 | Walk parent chain to root | +| `BranchNameError` | exc | templates/git/branch.py:33 | Bad type / missing task / over-depth | +| `build_commit_message` | func | templates/git/commit.py:63 | Rich commit msg with task/root/agent/session links | +| `CommitContext` | dataclass | templates/git/commit.py:34 | Validated commit-message input | +| `build_pr_body_internal` / `build_pr_title_internal` | funcs | templates/git/pr_internal.py:75/130 | Subtask→parent PR rendering | +| `build_pr_body_root` / `build_pr_title_root` | funcs | templates/git/pr_root.py:167/235 | Root PR rendering with task tree + AC checklist | +| `MAX_TASK_DEPTH` | const | templates/git/constants.py:45 | 4 (umbrella→root→cell→dev) | +| `BRANCH_TYPES` / `COMMIT_TYPES` | consts | templates/git/constants.py:10/21 | Allowed prefixes | +| `RepoRef` | dataclass | forge/base.py:26 | Provider-opaque repo identity (`owner`, `repo`, `host`); GitLab packs the full URL-encoded namespace path into `owner`, leaving `repo` empty | +| `GitProvider` | ABC | forge/base.py:49 | The ~20-method transport contract every forge implements (PR CRUD, review flow, `merge_branch`, CI signal surface, repo/label/branch/release/provisioning) | +| `GitService._forge` | property | git.py:399 | A fresh `ForgeRouter()` per access (cheap, no I/O — built this way, not `__init__`-cached, so tests can `GitService.__new__` bypass the constructor) | +| `ForgeRouter` | class | forge/router.py:37 | Implements `GitProvider` by picking a transport per call from `RepoRef.host` | +| `ForgeRouter._provider_for_ref` | staticmethod | forge/router.py:40 | `None` host → `GitHubProvider()`; a registered `"gitea"`/`"gitlab"` host → that provider constructed with its remembered scheme; unregistered → `GitError` naming the fix | +| `register_project_forge` | func | forge/registry.py:50 | Records a project's host→provider(+scheme) mapping; called from `ProjectService.create`/`update`/`get`/`get_by_slug` — in-memory, per-process, self-healing (a restart forgets it; the next project read re-registers) | +| `provider_for` | func | forge/registry.py:77 | Resolve a `GitProvider` for a project (duck-typed on `.git_provider`/`.git_url`) | +| `GitHubProvider` | class | forge/github.py:71 | GitHub.com/GHE REST transport — the pre-existing inline `httpx` logic, byte-for-byte; owns the sole retry policy in the package (`list_ci_runs`, 3 attempts/0.5s backoff, retryable on 429+5xx) | +| `GiteaProvider` | class | forge/gitea.py:55 | Self-hosted Gitea transport: `token`-scheme auth (Bearer is rejected by classic PATs); duplicate-PR 409→422 reshape on `create_pr`; `merge_pr` POSTs (not PUTs) with the method under a `"Do"` key; commit-status → synthetic `check_runs`/`workflow_runs`; branch names with slashes URL-`quote()`-encoded before hitting Gitea's router (caught live by the e2e suite) | +| `GitLabProvider` | class | forge/gitlab.py:85 | GitLab REST v4 transport — the most semantically divergent: MR `iid`→`number`, `source_branch`/`target_branch`→`head.ref`/`base.ref`; `get_pr_diff` reassembles a unified-diff string from up to 3 pages of per-file JSON diffs (no raw-diff media type); `post_review` routes APPROVE to `/approve` and everything else to a plain note (no request-changes verb exists); `request_reviewers` synthetic-skips (needs numeric user ids RoboCo doesn't store); `create_org_repo` resolves a group path to a numeric namespace id, falling back to the token's personal namespace on 404 | +| `ShapedResponse` | class | forge/shaping.py:17 | `httpx.Response`-compatible stand-in a non-GitHub provider returns to synthesize a status the wire call didn't produce (e.g. `merge_branch`'s shaped 501) | +| `extract_host` / `detect_provider` / `validate_project_forge` | funcs | foundation/policy/forge.py:27/45/62 | Pure host extraction (https/ssh/scp-like URLs), github.com/gitlab.com auto-detection, and registration-time validation (a self-hosted host with no explicit `git_provider` is a registration-time rejection — the GHE/self-hosted escape hatch requires the operator to set the column) | +| `derive_pr_labels` | func | foundation/policy/pr_labels.py | Org-structure label vocabulary: `base_branch` (required kwarg, the PR's real resolved target — never assumed from `is_root_pr`) → `f"to {base_branch}"`, plus `root`/`MegaTask`/layer labels (`main-pm`, `cell/{team}`, `subtask/{team}`) | + +Neither Gitea nor GitLab has GitHub's server-side merges API for the env-sync cascade: both return a shaped 501 from `merge_branch`, landing `GitService.sync_env_branch` on the local-git fallback (`_local_merge_branch`: throwaway clone → merge → push; a conflict aborts leaving the remote untouched, same status vocabulary as the GitHub server-side path) — this fallback lives entirely in `GitService`, not the forge package. Plain git (clone/fetch/push) needed zero forge-specific work: all three forges accept a PAT as the Basic-auth password with username ignored, so the existing `x-access-token:` extraheader works unchanged (verified live against a dockerized Gitea). + +## Data Flow + +A developer claims a task → the orchestrator/choreographer calls `create_branch_for_task` → `build_branch_name` walks the task parent chain (`TaskService.get`) up to `MAX_TASK_DEPTH=4`, joins `--`-separated 8-char UUID prefixes, and yields `{type}/{team}/{root}--{sub}--...`. `create_branch` fetches only the needed refs from origin, runs `git worktree add` under `{clone_root}/.worktrees/{task_id[:8]}` (F123 per-task isolation), force-pushes the branch with `-u`, and stores `branch_name` on the task. A `WorkSession` row is created (`WorkSessionService.create`), first superseding any other agent's stale ACTIVE session on that task. + +The agent commits via the `commit` content verb → `GitService.commit` (or `commit_for_task` route), which ensures the worktree is attached, asserts the workspace is on the task branch (recovering a drifted clone), stages, runs `build_commit_message` (`CommitContext` → header + metadata + links), commits, then best-effort links the SHA to the task + work session (`_link_commit_to_task`). Every `_run_git` call re-chowns the tree to the agent uid and clears orphaned lock files on timeout. + +`open_pr` → `create_pr` resolves the task by branch name, ensures the parent branch exists on origin, POSTs the PR via GitHub REST, and atomically records `pr_number`/`pr_url` on the task (`_record_pr_atomically`) and work session (`create_pr`). PR title/body come from `task.title`/`task.description` for gateway PRs; the rich `build_pr_body_root`/`_internal` templates are used by the older `create_pull_request` path. + +Merge: a cell PM `complete`/`submit_up` → `pr_merge` (gateway) scopes the task lookup by `project_id` (cross-repo PR-number collision guard), takes a `SELECT FOR UPDATE` lock on the parent task, calls `_merge_with_retry` (squash; on 409 re-syncs target + retries once; on 405 disambiguates already-merged vs real conflict → `MergeConflictError`), deletes the PR branch, syncs the local target, and records the merge on the work session (`merge_pr`, idempotent). The CEO-only root→master merge goes through `merge_pr_for_task` (role-gated, recorded-PR verification) → `merge_pull_request`. The CEO-merge never targets the default branch via `pr_merge` (the `target == default_branch` guard refuses it) — `default_branch` here is `_project_default_branch`, which now resolves via `roboco.models.env_branches.head_branch(project)` (the env-ladder head rung) rather than reading `project.default_branch` directly; a project with no declared ladder resolves to the same value via the read-time shim. + +Behind-base recovery: `is_behind_base` feeds the `i_am_done` submit gate; on a non-zero behind, the dev calls `sync_branch` → `sync_task_branch` → `rebase_onto_base` (rebased/superseded/conflicts). On a merge conflict, the choreographer calls `rebase_pr_for_task`, then either re-merges or `close_pull_request`s a superseded PR. `get_pr_head_sha` feeds the `submit_root` re-submit loop guard. + +## Mermaid + +```mermaid +stateDiagram-v2 + [*] --> Active: create() (supersede stale) + Active --> Active: add_commit / add_files_modified + Active --> Active: create_pr (pr_number set) + Active --> Completed: merge_pr (idempotent active-guard) + Active --> Completed: close() (task completion) + Active --> Abandoned: abandon() / supersede_active_sessions_for_task + Completed --> [*]: terminal + Abandoned --> [*]: terminal + note right of Active + single-active per task enforced at + create + DB partial-unique index (mig 047) + end note +``` + +```mermaid +sequenceDiagram + participant G as Choreographer/Gateway + participant GS as GitService + participant GH as GitHub REST API + participant WS as WorkSessionService + participant DB as DB (TaskTable) + + G->>GS: pr_merge(pr_number, target, project_id) + GS->>DB: SELECT task WHERE pr_number AND project_id (scoped) + GS->>DB: SELECT FOR UPDATE parent_task (serialize siblings) + GS->>GH: PUT /pulls/{n}/merge (squash) + alt 409 conflict + GS->>GS: _sync_target_branch (re-pull) + GS->>GH: PUT /pulls/{n}/merge (retry once) + end + alt non-success + GS->>GH: GET /pulls/{n} (already-merged?) + opt already merged + GS-->>G: idempotent success + end + opt real conflict + GS-->>G: raise MergeConflictError + end + end + GS->>GH: DELETE PR branch (best-effort) + GS->>GS: _sync_target_branch_best_effort + GS->>WS: merge_pr(session_id, merger_id) + WS->>WS: guard status==ACTIVE else return unchanged + WS-->>GS: COMPLETED + merged_by + GS-->>G: {"merge_commit_sha": ...} +``` + +## Logical Tree + +``` +roboco/ +├── services/ +│ ├── work_session.py +│ │ └── WorkSessionService (BaseService) +│ │ ├── create / get / get_or_raise / update +│ │ ├── get_active_for_task(_and_agent) +│ │ ├── supersede_active_sessions_for_task # single-active invariant +│ │ ├── list_by_agent / list_by_project / list_active_sessions +│ │ ├── add_commit / add_files_modified +│ │ ├── create_pr / update_pr_status / merge_pr +│ │ ├── complete / abandon / close +│ │ └── files_changed / has_unpushed_commits # gateway backfill +│ └── git.py +│ ├── _GIT_EXECUTOR (ThreadPoolExecutor, 16) +│ ├── resolve_git_dir / _remove_stale_git_locks / _select_ci_head_run +│ └── GitService (BaseService) +│ ├── _run_git (token, timeout, chown, lock-cleanup) +│ ├── _token_for_project / _token_for_workspace / get_workspace +│ ├── status: get_status / get_current_branch / _classify_porcelain / _ahead_behind +│ ├── commit: create_commit / commit_for_task / commit (gateway) / _link_commit_to_task +│ ├── worktree (F123): _worktree_for_task / _ensure_worktree_for_commit / _assert_on_task_branch +│ ├── branch: create_branch / create_branch_for_task / create_branch_from_pr_head / checkout* +│ ├── push/pull/fetch/rebase: push_for_task / push_task_branch / pull / fetch / rebase +│ ├── PR context: _build_root_pr_context / _build_internal_pr_context / _generate_pr_title_body +│ ├── PR list/find: _find_existing_pr / list_open_prs / _fetch_open_prs / _normalize_open_pr +│ ├── PR create: create_pull_request / create_pr_for_task / create_pr (branch-keyed) +│ ├── PR update/review: update_pr_for_task / post_pr_review / _patch_pr_title_body +│ ├── PR read: get_pr_diff / get_pr_head_sha / pr_target +│ ├── CI: get_latest_ci_conclusion / _get_ci_runs_response / _fetch_latest_ci_run +│ ├── merge: merge_pull_request / merge_pr_for_task / pr_merge / _merge_with_retry +│ │ _assert_merge_role / _lock_parent_task_for_merge / _resolve_merger_id +│ │ _pr_is_merged / _auto_complete_on_merge / _first_allowed_merge_method +│ ├── branch cleanup: _delete_remote_branch_best_effort / _delete_pr_branch_best_effort +│ │ delete_task_branch / _branch_has_open_dependents +│ │ cleanup_stale_branches / _stale_branch_window / _cleanup_one_stale_branch (sweep) +│ ├── rebase/sync: rebase_onto_base / rebase_pr_for_task / sync_task_branch / is_behind_base +│ ├── close: close_pull_request +│ ├── quality: run_pre_submit_quality_gate / toolchain_status_for_task / _fast_gate_commands +│ ├── conventions: conventions_check_for_task / _run_conventions_validator / open_conventions_pr +│ ├── read-only: diff / list_changed_files / read_file_at_branch / _ref_exists / _resolve_diff_base +│ └── helpers: _task_for_branch / _project_for_task / _workspace_for_branch / _token_for_branch ... +└── templates/git/ + ├── __init__.py # re-exports + ├── constants.py # BRANCH_TYPES / COMMIT_TYPES / MAX_TASK_DEPTH=4 + ├── branch.py # build_branch_name / get_root_task_id / BranchNameError + ├── commit.py # CommitContext / build_commit_message / CommitMessageError + ├── pr_internal.py # InternalPRContext / build_pr_body_internal / build_pr_title_internal + └── pr_root.py # RootPRContext / SubtaskInfo / build_pr_body_root / build_pr_title_root +``` + +## Dependencies + +Internal: +- `roboco.config.settings` (timeouts, URLs, workspace root, auto_clone) +- `roboco.exceptions` (`GitCommandError`, `GitError`, `GitTimeoutError`, `MergeConflictError`) +- `roboco.foundation.policy.lifecycle` (role/intent parity) +- `roboco.models.base` (`AgentRole`, `TaskStatus`) +- `roboco.models.work_session` (`WorkSessionCreate/Update/Status`) +- `roboco.db.tables` (`ProjectTable`, `TaskTable`, `WorkSessionTable`) +- `roboco.services.base` (`BaseService`, `NotFoundError`, `ConflictError`, `ValidationError`, `UnauthorizedError`, `ServiceError`) +- `roboco.services.project` / `roboco.services.task` / `roboco.services.workspace` (composed for clone/workspace/branch resolution) +- `roboco.services.gateway.quality_gate` (`GateResult`, `run_quality_commands`) +- `roboco.templates.git` (all template builders) +- `roboco.utils.converters.require_uuid`, `roboco.utils.crypto.EncryptionError` +- `roboco.api.schemas.git` (TYPE_CHECKING only — runtime duck-typed) + +External: +- `sqlalchemy` / `sqlalchemy.ext.asyncio` +- `httpx` (GitHub REST API) +- `asyncio`, `subprocess`, `concurrent.futures.ThreadPoolExecutor` +- `dataclasses`, `pathlib`, `uuid`, `base64`, `re`, `json`, `time`, `os`, `sys` + +## Entry Points + +- **HTTP routes** (`roboco/api/routes/git.py`): `get_status`, `log`, `diff`, `commit_for_task`, `push_for_task`, `create_branch_for_task`, `checkout_branch_for_agent`, `create_pr_for_task`, `merge_pr_for_task`, `pull`, `fetch`, `rebase`, `cleanup_stale_branches` (`POST /git/branches/cleanup`, PM/CEO role-gated like `/rebase`, rate-limit 5/60) — all construct via `get_git_service(db)`. +- **HTTP routes** (`roboco/api/routes/tasks.py:253`): `get_git_service` for task-scoped git. +- **Gateway Choreographer** (`roboco/services/gateway/choreographer/`): + - `_verb_runner._do_pr_merge` → `pr_merge` + - `_impl` → `conventions_check_for_task` (i_am_done + pr_pass gates), `is_behind_base` (submit gate), `sync_task_branch` (sync_branch verb), `pr_merge` / `rebase_pr_for_task` / `close_pull_request` (merge-conflict resolution), `get_pr_head_sha` (submit_root re-submit guard) + - `pr_gate.py` → `get_pr_head_sha` + - `qa.py` → `conventions_check_for_task` +- **WorkspaceService** calls `ensure_worktree` / `ensure_worktree_for_resume` (F123). +- **Lifespan/CLI**: none direct; `git_service` is constructed per-request via `deps.py` (`git=GitService(db_session)`). + +## Config Flags + +| Flag / setting | Source | Used for | +|----------------|--------|----------| +| `ROBOCO_GIT_EXECUTOR_WORKERS` (env, default 16) | `os.environ` at git.py:117 | Dedicated git subprocess pool size | +| `settings.git_command_timeout_seconds` | config.py:728 | Default `_run_git` timeout | +| `settings.git_commit_timeout_seconds` | config.py:737 | Staging/commit large changeset timeout | +| `settings.git_network_timeout_seconds` | config.py:748 | fetch/pull/push/ls-remote timeout | +| `settings.workspaces_root` | config.py:603 | Workspace path root (token derivation) | +| `settings.workspace_auto_clone` | config.py:607 | `get_workspace` auto-clone branch | +| `settings.public_base_url` | config.py:827 | Commit/PR template link base (`+ /api`) | +| `settings.internal_api_url` | config.py:57 | Internal PR body link base | +| `settings.github_api_base_url` | config.py:327 | GitHub REST API base (PR/CI/review) | +| `settings.release_ci_workflow` | config.py:597 | Named workflow file (e.g. `ci.yml`) used by the release CI gate in `get_latest_ci_conclusion`; always resolves a named workflow so the gate never degrades to the imprecise all-workflows mode | + +Module-level tunables (not env): `_SLOW_GIT_OP_MS=5000`, `_CI_RUN_WINDOW=20`, `_CI_FETCH_ATTEMPTS=3`, `_CI_FETCH_BACKOFF_SECONDS=0.5`, `_CI_RETRYABLE_STATUS`, `_CONVENTIONS_VALIDATOR_TIMEOUT_SECONDS=120`, `_GH_UNPROCESSABLE=422`, `_HTTP_NOT_FOUND=404`, `_HTTP_CONFLICT=409`. + +## Gotchas + +- **Token transport split**: git-over-HTTPS uses HTTP **Basic** (`x-access-token:...`) via `http.extraheader` (git.py:289); the GitHub REST API uses **Bearer**. Swapping them causes silent credential-prompt failures. +- **`pr_number` is NOT repo-scoped in `tasks.pr_number`** — every gateway merge/close path (`pr_merge`, `close_pull_request`, `pr_target`, `rebase_pr_for_task`) requires `project_id` to scope the task lookup. The route `merge_pr_for_task` instead verifies `data.pr_number == task.pr_number` (recorded PR is source of truth). +- **`get_active_for_task` returns the most-recent ACTIVE row**, not `scalar_one_or_none` — the historical duplicate-ACTIVE defect would otherwise raise `MultipleResultsFound`. The invariant is also enforced at `create` and by a DB partial-unique index (migration 047). +- **`merge_pr` idempotency guard (F062)**: a terminal session is returned unchanged; `complete`/`abandon` on a non-active session return `None` (warning). `close` returns the session unchanged. Behavior differs between these three on terminal state — `merge_pr`/`close` are silent, `complete`/`abandon` warn-and-None. +- **F123 per-task worktrees**: commit/checkout/rebase/conventions MUST run inside `{clone_root}/.worktrees/{task_id[:8]}`, not the clone root (which sits on the default branch). `_worktree_for_task` + `_ensure_worktree_for_commit` are the seam; forgetting them makes checkout fail with "already checked out at ''" or false-passes the conventions validator. +- **`create_branch` runs `reset --hard` on a no-commit branch** in the worktree (git.py:1104) — safe because `unique == 0`, but a branch carrying real work is left as-is. The fresh-claim path only; resume short-circuits before it. +- **`_run_git` re-chowns the tree** after every op (root → agent uid 1000); without it the agent's next commit fails with "unable to append to .git/logs/refs/...". +- **Porcelain parsing** uses `splitlines()` not `strip().split("\n")` — strip eats the leading space on ` D file` and false-stages deletions. +- **`get_current_branch` raises on detached HEAD** instead of returning `""` — the empty string used to leak "(HEAD detached at ...)" into `checkout -b`. +- **`MAX_TASK_DEPTH=4`** (was 3) — MegaTask's umbrella→root→cell→dev needs 4; validator rejects a child whose depth would *reach* MAX_TASK_DEPTH, so 4 permits dev at depth 3. +- **Branch name uses 8-char UUID prefix** (`_SHORT_ID_LEN=8`), not full UUID — full UUIDs produced 140-char branch names. +- **`rebase_onto_base` force-pushes with `--force-with-lease`** only the head branch; never touches base. `superseded` (unique==0) means close-without-merge. +- **`is_behind_base` raises on git failure**; the i_am_done gate fail-opens on the raised error so a flaky fetch can't strand the task. +- **Conventions validator fails closed** (`could_not_run=True` blocks submit) on resolution error / timeout / non-zero exit; branchless + no-changed-files fail open. +- **`_assert_on_task_branch` never discards work** — it does `checkout`, not `reset --hard`, to preserve a resumed agent's unpushed commits. +- **CEO-only master merge**: `pr_merge` refuses `target == default_branch` for agents; only `merge_pr_for_task` (CEO role-gated from `awaiting_ceo_approval`) may merge to master. `default_branch` resolves through the env-ladder head rung (`_project_default_branch` → `head_branch(project)`), not the raw `projects.default_branch` column. +- **`cleanup_stale_branches` cursor is required, not optional**: task rows never change as a side effect of the sweep (unlike, say, a queue that drains), so a repeat call with no `after_cursor` re-scans the identical first 200-row window forever instead of progressing. `_stale_branch_window` still advances the cursor past ladder-rung rows even though they're excluded from `candidates`, so `truncated` can't false-negative when a rung lands inside the window. +- **Local branch delete in the sweep is always `force=True`** (`-D`) regardless of completed vs cancelled — a completed task's PR was squash-merged, so its local ref is never an ancestor of base and a "safe" `-d` would refuse every single candidate. + +## Drift from CLAUDE.md + +- **CLAUDE.md "WorkSessionService" table** claims the service handles "Git session management, PR lifecycle" — accurate. No drift. +- **CLAUDE.md says** `ROBOCO_WORKSPACE_CLONE_TIMEOUT=300` is a WorkspaceService config; not referenced in this slice (lives in `workspace.py`). No drift in scope. +- **CLAUDE.md verb table** lists `sync_branch` for developers and `/rebase` for PM/CEO. The code matches: `sync_task_branch` is the dev path, `rebase_pr_for_task` the PR-keyed path. No drift. +- **CLAUDE.md** states "A task has at most one active WorkSession ... enforced both at the service layer and by a DB partial-unique index (migration 047)." Code matches (`supersede_active_sessions_for_task` + `get_active_for_task` resilient return). No drift. +- **CLAUDE.md** "PR is created BEFORE QA review" — `create_pr`/`create_pr_for_task` only sets `pr_number`; QA pass requires it. Matches. +- **Minor doc vs code**: CLAUDE.md commit-format example is `[{task-id[:8]}] {message}` (single ID), but `build_commit_message` (templates/git/commit.py:80) emits `[{root_short}:{task_short}] {type}({scope}): {desc}` — a richer two-ID header. The doc undersells the actual format; not a bug, but the template header is not the literal `[{task-id[:8]}]` the doc shows. +- **CLAUDE.md** lists `merge_pull_request`-style PM merges; the agent-facing path is now `pr_merge` (gateway) with parent-row locking + 409 retry, which the doc does not describe. Additive (the route `merge_pr_for_task` still exists) — doc is incomplete rather than wrong. + +## Changes Since Baseline + +Baseline: `fd10cc862c2020b3f639cdb686d427b0198a2441` (master tip before the metrics-granularity branch). Diff stat: `git.py +614/-107`-ish, `work_session.py +11`, `branch.py +9/-`, `constants.py +12/-`. Commits touching these files: `15effce0` (#283 "141 Gaps fill-in"), `3aff6e04` (#285 "Close gaps"). + +| Commit | IMPACT (one line) | +|--------|-------------------| +| `15effce0` (141 Gaps fill-in) | Added `resolve_git_dir` + worktree-aware `_remove_stale_git_locks`; added F123 `_worktree_for_task`/`_ensure_worktree_for_commit` and routed commit/checkout/rebase/conventions into the per-task worktree; added `get_pr_head_sha` (pr_fail re-submit guard); added `sync_task_branch` + `is_behind_base` (dev sync_branch verb + i_am_done behind gate); added `rebase_onto_base`/`rebase_pr_for_task` (merge-conflict resolver); added `close_pull_request` (superseded-PR close); added `pr_merge` with `project_id` scoping + parent-row lock + 409 retry + CEO-only default-branch guard + already-merged disambiguation; added `_merge_with_retry`/`_pr_is_merged`/`_resolve_merger_id`/`_lock_parent_task_for_merge`; added conventions validator runner (`conventions_check_for_task`/`_run_conventions_validator`/`open_conventions_pr`); raised `MAX_TASK_DEPTH` 3→4 (MegaTask depth-cap fix); `WorkSessionService.merge_pr` idempotent active-guard (F062). | +| `3aff6e04` (Close gaps) | Same mega-commit (the PR body is identical — #285 is the merge closure of the #283 batch); the in-scope file deltas are the same set of additions. No additional logic change to these files beyond what #283 listed. | + +> Post-snapshot updates (since 2026-06-29): `536bbb64` (Chore/all/logical gaps sweep #286 — closed regression risks #108 and #109 in this slice: `_merge_with_retry` now falls back to a permitted merge method on 405 via `_first_allowed_merge_method` before raising `MergeConflictError`; `close_pull_request` default flipped to `delete_branch=False`, choreographer caller now passes `delete_branch=True` explicitly). `00513399` ([bug] push_branch — `push_branch(branch_name)` now passes `branch=branch_name` to `self.push()` so the gateway `open_pr` path pushes the actual named task branch rather than the clone root's current checkout; fixes the "No commits between" 422 → `i_am_blocked` wedge observed in the F123 per-worktree model). `2759edf7` ([B-REL] release executor — added `_CiRunQuery` dataclass at git.py:241 to bundle per-project CI-fetch inputs; `get_latest_ci_conclusion` and `_fetch_latest_ci_run` now accept an optional `head_sha` so the release CI gate polls a specific release commit's own run rather than branch-latest; `settings.release_ci_workflow` config flag added). `69071030` ([chore] work-session-routes — added `WorkSessionService.task_team_for_session` helper (route layer PM cell-ownership check for `merge_pr`); route layer now stamps `merged_by` from the authenticated caller rather than the request body — `WorkSessionService.merge_pr` signature is unchanged, but `MergePRRequest` schema dropped `merged_by` field). +> +> `496c24d1` (PR #548, "git hygiene", 2026-07-17) Local branch refs stop leaking alongside remote ones: `delete_task_branch` now also skips environment-ladder rungs (previously only the remote-delete's own main/master/develop guard existed) and returns `bool`; new `cleanup_stale_branches` + `_stale_branch_window` + `_cleanup_one_stale_branch` back a PM/CEO-only `POST /git/branches/cleanup` sweep of terminal tasks' remote+local branches, exposed as a confirm-dialog button on the panel Git page. See `docs/map/task-service.md` for the paired per-task reap at cancel/completion and `docs/map/workspace.md` for the new `WorkspaceService.delete_local_branch` primitive both routes share. +> +> **Forge providers — GitHub + Gitea + GitLab (2026-07-18/19, PRs #569/#571/#575/#579/#581).** A new `roboco/services/forge/` package (`base.py`/`github.py`/`gitea.py`/`gitlab.py`/`registry.py`/`router.py`/`shaping.py`) plus `roboco/foundation/policy/forge.py` route every REST call `GitService` makes (PRs/CI/reviews/labels/releases/provisioning) through a provider-agnostic transport. `388bab24` (Phase 0, #569): `projects.git_provider` column (migration 076, nullable, plain string not a pg enum — validated at the service layer, not the DB) + `validate_project_forge`/`detect_provider` (github.com auto-detects, self-hosted needs an explicit column value — the GHE/self-hosted escape hatch). `461a6e1a` (Phase 1, #571): the `GitProvider` ABC + `GitHubProvider` extracted byte-for-byte from `GitService`'s old inline `httpx` calls; `GitService._forge` (git.py:399) becomes the seam every call site routes through. `96401f4c` (Phases 2/2.1/3, #575): `GiteaProvider` + `GitLabProvider` + `ForgeRouter` (per-call transport dispatch off `RepoRef.host`) + the local-git `merge_branch` fallback for forges with no server-side merges API. `5f32d876` (Phase 4, #581): `roboco/services/github_provisioning.py` becomes provider-aware (`ROBOCO_PROVISIONING_PROVIDER`/`ROBOCO_PROVISIONING_HOST` — see `docs/map/product-strategy-research-pitch.md`) so pitch-driven repo creation works on all three forges. `d4cb5797` (#579) + the pre-existing `tests/e2e_smoke/test_gitea_live.py` are the live contract suites (self-seeding against a dockerized `gitea/gitea` / real `gitlab.com`, env-gated) that caught the slash-encoding and http-scheme gaps in the Gitea provider. Panel: the edit-project dialog's "Forge" `` still reads "...GitLab support is planned" even though `gitlab` is a live `SelectItem` one line below and the backend has full GitLab Phase 3 support (`GitLabProvider`, `ForgeRouter`) — cosmetic only, no functional gap, but confusing to an operator reading the tooltip before picking GitLab. | low | +| F123 worktree routing — commit/conventions/rebase run in worktree, merge sync runs in clone root | git.py:3696/3785 | `pr_merge` calls `_sync_target_branch_best_effort(workspace,...)` with the clone-root workspace (from `get_workspace`), not the per-task worktree. If the target branch is checked out in a worktree, the sync's `checkout` of target in the clone root fails ("already checked out"). Best-effort swallows it, but the local target ref may stay stale for the next sibling merge. | medium | +| ~~`_merge_with_retry` retries on 409 only; 405 falls through to already-merged check then `MergeConflictError`~~ | git.py:3597 | **FIXED** (`536bbb64` #108) — `_merge_with_retry` now falls back to a permitted merge method (via `_first_allowed_merge_method`, exclude='squash') on 405 before raising `MergeConflictError`, mirroring the CEO `merge_pull_request` path. A 405 with no permitted fallback or a second 405 still falls through to disambiguation/`MergeConflictError`. | ~~medium~~ resolved | +| `is_behind_base` raises on fetch failure; gate fail-opens | git.py:3922/3938 | A flaky origin fetch makes `is_behind_base` raise; the i_am_done gate catches it and fail-opens, letting a behind branch submit. The merge layer's own behind check is the backstop, but a genuinely-behind branch can reach QA. Documented, but a regression in the "gate is authoritative" expectation. | low | +| `MAX_TASK_DEPTH` 3→4 changes branch-name length + validation | constants.py:45 / branch.py:71 | Any pre-existing task hierarchy at depth 4 that was previously rejected now builds a 4-segment branch name; tasks created under the old cap that stored a shorter branch are unaffected, but new subtasks of a previously-maxed tree now cut branches where they couldn't before — could surface latent assumptions in downstream consumers parsing branch names. | low | +| ~~`close_pull_request` deletes branch on close by default~~ | git.py:4005 | **FIXED** (`536bbb64` #109) — default flipped to `delete_branch=False` (opt-in deletion); the choreographer supersede caller now passes `delete_branch=True` explicitly when it wants deletion, matching the orchestrator supersede path. A superseded PR's branch is preserved by default. | ~~medium~~ resolved | +| Conventions validator fail-closed on resolution error | git.py:4392 | A workspace resolution failure (missing clone, diff error) returns `could_not_run=True`, which the block-gate treats as a hard refuse. A transient workspace/clone issue can now block `i_am_done`/`pr_pass` where previously the gate would have passed. Intentional but a new stranding vector. | low | +| `get_pr_head_sha` fail-open returns None on any error | git.py:2496 | The `submit_root` re-submit loop guard only hard-blocks on an *exact* unchanged head SHA; any GitHub error / closed PR returns None and the guard passes, so a flaky API call lets a weak coordinator re-submit the same failed PR. Documented fail-open, but a regression vs. a strict gate. | low | + +## Health + +Integrity is **good and actively hardened**. The slice carries the scars of multiple live meltdowns (single-active work-session defect, pr_fail re-submit loop, cell_pm merge block<->unblock, MegaTask depth cap, cross-repo PR-number collision) and each is closed with a deterministic guard plus a comment explaining the failure mode. The F123 per-task-worktree routing is consistently threaded through commit/checkout/rebase/conventions, and the merge path has layered defenses (parent-row lock, 409 retry, already-merged disambiguation, CEO-only master guard). Two formerly-medium risks in the merge path have since been closed: `_merge_with_retry` now has the 405 method-fallback that `merge_pull_request` has (`536bbb64`), and `close_pull_request` now defaults to `delete_branch=False` (`536bbb64`). The remaining residual risk is **`pr_merge`'s post-merge target sync** running in the clone root (not the worktree) and best-effort-swallowing a checkout conflict — the local target ref may stay stale for the next sibling merge. Test coverage of the work-session lifecycle is solid; the newer `pr_merge`/`sync_task_branch`/`rebase_onto_base`/`close_pull_request` quartet deserves the most scrutiny on any future change. No outright bugs found; the drift vs CLAUDE.md is documentation undersell (commit header format, gateway merge-path description), not behavioral mismatch. The forge-providers rollout (GitHub/Gitea/GitLab) added real breadth without touching the merge-path defenses above — `GitService`'s callers still reason in GitHub-shaped responses, and `ForgeRouter`/`GiteaProvider`/`GitLabProvider` carry the entire adaptation burden behind that seam; the live-forge e2e suites (`tests/e2e_smoke/test_gitea_live.py`/`test_gitlab_live.py`, both env-gated and self-seeding) are the only coverage that exercises a real forge over the wire rather than a mocked transport, and they already caught two real gaps (slash-encoding, http scheme) the mocked unit tests couldn't have found. +# workspace slice + +## Purpose +WorkspaceService manages the per-agent git clone layout under {workspaces_root}/{project}/{team}/{agent}/, plus the F123 per-task linked worktrees under {clone_root}/.worktrees/{task}/. It clones, refresh-fetches, repairs ownership, installs dev deps, scaffolds the conventions standard on first clone, maintains a project-level read clone for the conventions engine, and runs the read-only dep-upgrade probe. It is the filesystem/git-clone substrate every agent spawn and every git verb eventually lands on. + +## Files + +| Path | Role | LOC | +|---|---|---| +| /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/workspace.py | WorkspaceService + helpers: clone/own/refresh/install deps, per-task worktrees, read clone, dep-upgrade probe | 1757 | + +## Key Symbols + +| Name | Kind | File:Line | Responsibility | +|---|---|---|---| +| _chown_entry | function | roboco/services/workspace.py:84 | Chown one entry to (AGENT_UID, AGID); return True on success or already-correct | +| _make_owner_and_group_rw | function | roboco/services/workspace.py:95 | Best-effort chmod ensuring owner+group rw (+x for dirs) for ACL-inheriting NAS volumes | +| _own_and_grant_rw | function | roboco/services/workspace.py:122 | Chown + grant rw on one entry; return 1 if chown failed | +| _iter_ownable_entries | function | roboco/services/workspace.py:129 | Yield workspace root + every entry, pruning heavy gitignored trees (_PRUNE_DIRS) so os.walk stays fast | +| _ensure_agent_owned | function | roboco/services/workspace.py:145 | Chown + group-write the whole working tree (pruned) so uid-1000 agent can read+write .git and sources | +| _resolve_clone_root | function | roboco/services/workspace.py:184 | Given a worktree path, return its clone root (parent.parent when under .worktrees/); pure path logic | +| _uv_subprocess_env | function | roboco/services/workspace.py:198 | Env for orchestrator-side uv subprocess: pin UV_PYTHON_INSTALL_DIR to /.uv-python so fetched CPython lands on the mount; also pops VIRTUAL_ENV and UV_PROJECT_ENVIRONMENT to drop the image-baked /app/.venv pin (9faf2763) | +| _monotonic | function | roboco/services/workspace.py:223 | Thin wrapper over time.monotonic so tests can patch it without breaking asyncio's own clock | +| _ensure_lock_for | function | roboco/services/workspace.py:244 | Return (lazy-create) the per (project_slug, agent_slug) asyncio.Lock serializing ensure_workspace/concurrent clones | +| _inject_token_into_url | function | roboco/services/workspace.py:254 | Embed a GitHub PAT into an HTTPS git URL for clone/fetch auth; pass-through for SSH and already-tokenized URLs | +| WorkspaceError | class | roboco/services/workspace.py:283 | Exception raised on workspace/clone/worktree failures | +| _lockfile_digest | function | roboco/services/workspace.py:306 | SHA-256 over present lockfiles (uv.lock/pnpm-lock.yaml/package-lock.json/package.json) for idempotent dev-deps install | +| _detect_dep_commands | function | roboco/services/workspace.py:332 | Detect ecosystem + return (label, argv) install commands: uv sync --extra dev (optionally --python X), pnpm/npm | +| WorkspaceService | class | roboco/services/workspace.py:374 | Service: per-agent workspace path math, clone/own/refresh, worktrees, read clone, dep probe, dev-deps install | +| WorkspaceService.get_workspace_path | method | roboco/services/workspace.py:399 | Compute {root}/{project}/{team}/{agent}/ path; raise if team is None | +| WorkspaceService.get_clone_root_path | method | roboco/services/workspace.py:433 | Same as get_workspace_path; named separately to express clone-root vs worktree intent | +| WorkspaceService.get_worktree_path | method | roboco/services/workspace.py:447 | Per-task worktree path {clone_root}/.worktrees/{task_short_id}; raise on empty id | +| WorkspaceService._clone_root_default_branch | staticmethod | roboco/services/workspace.py:486 | Read origin/HEAD to get the default branch name ("main"); returns "" when unresolvable (cfe725da) | +| WorkspaceService._park_clone_root_off_branch | staticmethod | roboco/services/workspace.py:501 | Restore F123 invariant before worktree add: if clone root HEAD is the task branch, move back to default branch or detach so the branch ref is free for the worktree (cfe725da) | +| WorkspaceService._worktree_git | staticmethod | roboco/services/workspace.py:469 | Run git -C capturing output, check optional | +| WorkspaceService._link_shared_venv | staticmethod | roboco/services/workspace.py:534 | Symlink worktree/.venv -> ../../.venv only if clone-root .venv exists; idempotent via lexists guard | +| WorkspaceService.ensure_worktree | method | roboco/services/workspace.py:555 | git worktree add -b (or reuse existing branch); calls _park_clone_root_off_branch first; link venv; chown worktree + clone root | +| WorkspaceService.ensure_worktree_for_resume | method | roboco/services/workspace.py:591 | Re-add a pruned worktree on resume (no -b; branch ref survives); calls _park_clone_root_off_branch first; idempotent; link venv + chown | +| WorkspaceService._fetch_branch_ref | method | roboco/services/workspace.py:613 | Token-aware git fetch origin into clone_root; best-effort (never raises); used by ensure_worktree_self_heal (536bbb64) | +| WorkspaceService.ensure_worktree_self_heal | method | roboco/services/workspace.py:671 | Orchestrator spawn-time chokepoint: re-attaches a per-task worktree after clone vanished (redeploy/disk loss); fetches branch ref from origin when the local ref is absent after a re-clone, then delegates to ensure_worktree (536bbb64) | +| WorkspaceService.remove_worktree | method | roboco/services/workspace.py:733 | Best-effort git worktree remove --force + prune; no-op if gone (cancel/terminal/reaper evict) | +| WorkspaceService.delete_local_branch | method | roboco/services/workspace.py:787 | Best-effort `git branch -d/-D ` in a clone; never raises; skips main/master/develop/empty (mirrors GitService._delete_remote_branch_best_effort); callers run it AFTER remove_worktree (a still-checked-out branch refuses) | +| WorkspaceService.resolve_workspace | method | roboco/services/workspace.py:745 | Look up agent (UUID or slug) -> team+slug -> workspace path; default team BACKEND | +| WorkspaceService._lookup_agent_or_raise | method | roboco/services/workspace.py:787 | Find agent by UUID or slug; raise WorkspaceError if missing | +| WorkspaceService._is_workspace_healthy | staticmethod | roboco/services/workspace.py:806 | True only if .git exists AND has HEAD + objects/ (rejects stub clones) | +| WorkspaceService._prune_broken_refs | staticmethod | roboco/services/workspace.py:816 | Delete .bak ref debris + loose refs whose content is neither sha nor symref before a fetch; best-effort | +| WorkspaceService._fetch_origin_best_effort | staticmethod | roboco/services/workspace.py:854 | Scoped credential-less git fetch of current+default branch with 30s TTL; downgrades expected auth-fail to DEBUG | +| WorkspaceService._resolve_git_token | staticmethod | roboco/services/workspace.py:947 | Decrypt project git token; raise WorkspaceError on decrypt failure or HTTPS-with-no-token | +| WorkspaceService.ensure_workspace | method | roboco/services/workspace.py:969 | Idempotent ensure: healthy short-circuit (own+fetch+install_deps) or rmtree partial then clone+scaffold; per (project,agent) lock | +| WorkspaceService._maybe_scaffold_conventions | method | roboco/services/workspace.py:1101 | Flag-gated once-per-process scaffold of .roboco/conventions.yml on a project's first clone; swallows all failures | +| WorkspaceService.ensure_read_clone | method | roboco/services/workspace.py:1133 | Ensure project-level read clone at {root}/{project}/_meta/conventions, hard-reset to `origin/` (the env-ladder head rung via `roboco.models.env_branches.head_branch`, shimmed from `default_branch` when no ladder is declared); 30s TTL fetch | +| WorkspaceService._read_clone_token | staticmethod | roboco/services/workspace.py:1184 | Decrypt project token for read-clone refresh; return None on failure (public repos ok) | +| WorkspaceService._sync_read_clone | staticmethod | roboco/services/workspace.py:1200 | Token-authed fetch + checkout + reset --hard FETCH_HEAD on the read clone; best-effort | +| WorkspaceService._clone_repo | method | roboco/services/workspace.py:1239 | git clone --branch --no-tags (no --single-branch) + configure identity/fileMode + scrub PAT + leak-check + chown + install_dev_deps; rmtree on any failure | +| WorkspaceService.install_dev_deps | method | roboco/services/workspace.py:1424 | Idempotent dev-deps install via lockfile digest marker; runs detected cmds, chowns results, records toolchain marker | +| WorkspaceService._resolve_toolchain_target | staticmethod | roboco/services/workspace.py:1478 | Return target Python version when toolchain_match_enabled, else None | +| WorkspaceService._record_toolchain | method | roboco/services/workspace.py:1486 | Run pytest --collect-only smoke under target python and write .git/.roboco-toolchain marker JSON | +| WorkspaceService._run_toolchain_smoke | staticmethod | roboco/services/workspace.py:1504 | Return ok/broken/unknown from pytest collect-only under the target interpreter (precision over recall) | +| WorkspaceService.read_toolchain_status | staticmethod | roboco/services/workspace.py:1546 | Read (python, status) from the toolchain marker; (None,None) when absent/unreadable | +| WorkspaceService._dep_install_cache_hit | staticmethod | roboco/services/workspace.py:1562 | True when stored digest equals current lockfile digest (skip install) | +| WorkspaceService._run_dep_install | staticmethod | roboco/services/workspace.py:1575 | Run one install command in a thread; swallow FileNotFoundError/timeout/OSError; return True only on exit 0 | +| WorkspaceService.dry_upgrade_changes_lockfile | method | roboco/services/workspace.py:1634 | Read-only dep-upgrade probe: local --no-hardlinks clone of read clone under lock, run dep_update_command, report dirty lockfile paths; fail-safe False | +| WorkspaceService._clone_local_into | staticmethod | roboco/services/workspace.py:1692 | git clone --local --no-hardlinks of read clone into throwaway dir (independent copy) | +| WorkspaceService._probe_lockfile_on_clone | staticmethod | roboco/services/workspace.py:1716 | Run upgrade via shlex.split (no shell) + git status --porcelain on lock paths; False on non-zero | +| WorkspaceService.workspace_exists | method | roboco/services/workspace.py:1752 | Bool: workspace resolved and .git exists | +| WorkspaceService.list_workspaces | method | roboco/services/workspace.py:1764 | Scan {root}/{project}/*/* for dirs containing .git; return info dicts | +| WorkspaceService._resolve_branch_to_project_slug | method | roboco/services/workspace.py:1797 | Look up task by branch_name -> project slug; raise if no task or project missing | +| WorkspaceService.fetch_branch_for_inspection | method | roboco/services/workspace.py:1823 | Ensure workspace for QA/Doc/PM, git fetch origin with token http.extraheader; re-chown; return workspace path | +| WorkspaceService.delete_workspace | method | roboco/services/workspace.py:1897 | rmtree the resolved workspace; True if deleted, False if absent | +| get_workspace_service | function | roboco/services/workspace.py:1930 | Factory: WorkspaceService(session) | + +## Data Flow +Inputs: an AsyncSession, a project_slug, an agent_id (UUID or slug), optionally a git_url/default_branch/force. The orchestrator, GitService, TaskService, conventions service, dep_update_engine, and gateway content_actions all obtain a WorkspaceService via get_workspace_service(session) (or WorkspaceService(db) directly in the spawn path). Control flow on ensure_workspace: _lookup_agent_or_raise -> get_workspace_path -> acquire per-(project,agent) asyncio.Lock -> if _is_workspace_healthy (.git+HEAD+objects): _ensure_agent_owned (to_thread), prune broken refs, scoped _fetch_origin_best_effort (30s TTL, force override), re-chown, install_dev_deps (digest-cache hit short-circuits), return. Else: rmtree any partial/stub dir, ProjectService.get_by_slug, resolve the clone target branch via `head_branch(project)` (the env-ladder head rung — `roboco.models.env_branches`, shimmed from `default_branch` when no ladder is declared), _resolve_git_token (decrypt PAT; raise on HTTPS-with-no-token), _clone_repo (git clone --branch --no-tags, configure identity/fileMode, scrub PAT from remote URL, _assert_no_pat_leak scanning .git/** for ghp_/github_pat_/x-access-token, chown, install_dev_deps), _maybe_scaffold_conventions (once-per-process, flag-gated). Per-task worktree path: get_clone_root_path + get_worktree_path (.worktrees/{task_short_id}); ensure_worktree runs git worktree add -b (or reuses an existing branch ref), _link_shared_venv (symlink to clone-root .venv only if it exists), chowns both worktree and clone root. GitService.create_branch calls ensure_worktree; commit/rebase paths call ensure_worktree_for_resume via GitService._ensure_worktree_for_commit; the orchestrator's _ensure_worktree_before_spawn calls ensure_worktree_self_heal (post-536bbb64) which first fetches the branch ref from origin if the local ref is absent after a re-clone, then delegates to ensure_worktree; TaskService.complete/cancel call remove_worktree. ensure_read_clone is called by ConventionsService for the project-level read clone at _meta/conventions, hard-reset to origin/default. dry_upgrade_changes_lockfile (dep_update_engine) clones the read clone --local --no-hardlinks into a throwaway under the read-clone lock, runs dep_update_command, and checks git status --porcelain on lockfile paths. Outputs: workspace Path (and side effects: on-disk clone/worktree, .venv symlink, .git/.roboco-dep-install + .git/.roboco-toolchain markers, root-owned refs re-chowned to agent uid). All git/subprocess work runs via asyncio.to_thread; tokens are injected only transiently into argv (never written to .git/config) and scrubbed post-clone. + +## Mermaid +```mermaid +flowchart TD + caller["Callers: orchestrator spawn, GitService, TaskService, ConventionsService, DepUpdateEngine, gateway content_actions"] + factory["get_workspace_service(session)"] + WS["WorkspaceService"] + caller --> factory --> WS + + subgraph ensure["ensure_workspace (per project+agent lock)"] + health{"_is_workspace_healthy?
.git+HEAD+objects"} + own1["_ensure_agent_owned"] + prune["_prune_broken_refs"] + fetch["_fetch_origin_best_effort
scoped, 30s TTL, force override"] + deps["install_dev_deps
digest-cache hit -> skip"] + health -->|yes| own1 --> prune --> fetch --> own2["re-chown after fetch"] --> deps --> ret1["return workspace"] + health -->|no| rm["rmtree partial/stub"] --> proj["ProjectService.get_by_slug"] + proj --> tok["_resolve_git_token (decrypt PAT)"] + tok --> clone["_clone_repo
clone --no-tags --branch
configure+scrub PAT
leak-check + chown + install_dev_deps"] + clone --> scaffold["_maybe_scaffold_conventions
once-per-process, flag-gated"] --> ret2["return workspace"] + end + WS --> ensure + + subgraph wt["Per-task worktrees (F123)"] + gwp["get_worktree_path
{clone_root}/.worktrees/{task}"] + ew["ensure_worktree
worktree add -b branch base"] + ewr["ensure_worktree_for_resume
re-add pruned, no -b"] + rmw["remove_worktree
worktree remove --force + prune"] + link["_link_shared_venv
symlink -> ../../.venv if exists"] + ew --> link --> chown2["_ensure_agent_owned x2"] + ewr --> link + end + GitService --> ew + GitService --> ewr + orchestrator --> ewr + TaskService --> rmw + + subgraph rc["Read clone + dep probe"] + erc["ensure_read_clone
_meta/conventions, hard-reset origin/default, 30s TTL"] + src["_sync_read_clone
token-authed fetch + reset --hard FETCH_HEAD"] + dry["dry_upgrade_changes_lockfile
local --no-hardlinks clone under lock
run dep_update_command -> dirty?"] + erc --> src + dry --> erc + end + ConventionsService --> erc + DepUpdateEngine --> dry ``` ## Logical Tree ``` -pr-gate-review slice -├── PRGateMixin (in-path assembled-PR gate) -│ ├── claim_gate_review — claim without transition + assembled diff evidence -│ ├── pr_pass — awaiting_pr_review → awaiting_pm_review -│ ├── pr_fail — awaiting_pr_review → needs_revision (issues required) -│ └── helpers -│ ├── _gate_decision — shared body (preflight→tracing→blocked→record→run_intent→None-guard→post→a2a) -│ ├── _gate_preflight — ownership/role/spec-gate (self_review_block) + soup guard -│ ├── _gate_tracing — journal:learning + pr_reviewer_notes min chars -│ ├── _pr_pass_blocked — toolchain-broken + conventions block + CI-status guards (returns rejection, ci_note) -│ ├── _ci_status_guard — refuse pr_pass on failing/pending/unscheduled CI; config gaps + unreachable repos pass through with evidence stamp; real API failures stay fail-closed -│ ├── _resolve_ci_status — thin wrapper calling git.get_pr_ci_status, interprets result dict, returns rejection Envelope if CI must block -│ ├── _record_gate_verdict_for / _record_gate_verdict — structured pr_review note (+ issues + head_sha + ci_status) -│ ├── _re_stamp_pr_fail_head_sha_if_advanced — re-capture head SHA post-transition and re-stamp verdict note if advanced (#189) -│ ├── _capture_pr_head_sha — best-effort PR head SHA for unchanged-PR gate -│ ├── _post_gate_review / _post_gate_review_to_pr — PR review post (COMMENT on root→master or MegaTask root-subtask) -│ ├── _gate_review_event_verdict / _gate_review_body — static helpers for post_gate_review_to_pr (extracted in 536bbb64) -│ ├── _deliver_pr_fail_to_owner — a2a change-requests to owning PM (+ Main-PM-root steer) -│ ├── _gate_role_or_rejection — role enum parse -│ └── _build_gate_review_evidence — assembled diff + AC -└── PRReviewerMixin (inbound external/fork PR review) - ├── claim_pr_review — pending→in_progress (branch-exempt) + read-only diff evidence - ├── post_pr_review — in_progress→completed, one change-request - └── helpers - ├── _post_pr_review_preflight — non-empty body/role/spec-gate/tracing - ├── _post_pr_review_content_gates — verdict consistency + no-hand-format - ├── _verdict_consistency_gate — pr_review_conflict pure invariant - ├── _is_hand_formatted_verdict — detect ## headers in free-text body - ├── _resolve_post_body — canonical render vs free-text - ├── _build_pr_review_content — validate_content into PrReviewContent - ├── _post_review_side_effects — GitHub post + CEO notify - ├── _pr_review_tracing_gate — journal:learning + notes min chars - ├── _resolve_role / _runner_failure / _build_pr_review_evidence - ├── _project_slug_for — delegates to module resolver - └── resolve_task_project_slug (module-level, shared with _impl.py) — project_id → product → cell_projects +WorkspaceService slice ++-- Module-level helpers +| +-- _chown_entry / _make_owner_and_group_rw / _own_and_grant_rw +| +-- _iter_ownable_entries (prunes _PRUNE_DIRS) +| +-- _ensure_agent_owned (whole-tree chown+chmod, best-effort) +| +-- _resolve_clone_root (worktree -> clone root path logic) +| +-- _uv_subprocess_env (UV_PYTHON_INSTALL_DIR pin) +| +-- _monotonic (test-patchable clock) +| +-- _ensure_lock_for (per project+agent asyncio.Lock) +| +-- _inject_token_into_url (PAT into HTTPS URL) +| +-- _lockfile_digest / _detect_dep_commands +| +-- markers: _DEP_INSTALL_MARKER, _TOOLCHAIN_MARKER ++-- WorkspaceError ++-- WorkspaceService +| +-- Path math: get_workspace_path / get_clone_root_path / get_worktree_path +| +-- Worktree ops: _clone_root_default_branch / _park_clone_root_off_branch / _worktree_git / _link_shared_venv / ensure_worktree / ensure_worktree_for_resume / _fetch_branch_ref / ensure_worktree_self_heal / remove_worktree / delete_local_branch +| +-- Agent lookup: resolve_workspace / _lookup_agent_or_raise +| +-- Health + refs: _is_workspace_healthy / _prune_broken_refs / _fetch_origin_best_effort +| +-- Token: _resolve_git_token / _read_clone_token +| +-- Clone + ensure: ensure_workspace / _clone_repo / _maybe_scaffold_conventions +| +-- Read clone: ensure_read_clone / _sync_read_clone +| +-- Dev deps + toolchain: install_dev_deps / _resolve_toolchain_target / _record_toolchain / _run_toolchain_smoke / read_toolchain_status / _dep_install_cache_hit / _run_dep_install +| +-- Dep-update probe: dry_upgrade_changes_lockfile / _clone_local_into / _probe_lockfile_on_clone +| +-- Misc: workspace_exists / list_workspaces / _resolve_branch_to_project_slug / fetch_branch_for_inspection / delete_workspace ++-- get_workspace_service (factory) ``` +## Dependencies +- Internal: roboco.config.settings, roboco.db.tables.AgentTable, roboco.db.tables.TaskTable, roboco.logging.get_logger, roboco.models.base.Team, roboco.models.env_branches.head_branch (env-ladder head-rung resolver backing the clone target in ensure_workspace and ensure_read_clone), roboco.services.toolchain.resolve_target_python, roboco.services.project.get_project_service / ProjectService, roboco.services.conventions.get_conventions_service / ConventionsService, roboco.utils.crypto.EncryptionError, roboco.db.base.get_db_context (orchestrator spawn path) +- External: asyncio, contextlib, json, math, os, re, shlex, shutil, subprocess, tempfile, time, pathlib.Path, uuid.UUID, collections.abc.Iterator, sqlalchemy.ext.asyncio.AsyncSession, sqlalchemy.select, hashlib (lazy in _lockfile_digest), stat (lazy in _make_owner_and_group_rw), base64 (lazy in fetch_branch_for_inspection) + ## Entry Points | Name | File | Trigger | |---|---|---| -| claim_gate_review | roboco/services/gateway/choreographer/pr_gate.py | pr_reviewer agent calls flow verb claim_gate_review(task_id) via roboco-flow MCP / POST /api/v1/flow/pr_reviewer/claim_gate_review on an awaiting_pr_review assembled-PR task | -| pr_pass | roboco/services/gateway/choreographer/pr_gate.py | pr_reviewer calls pr_pass(task_id, notes) via roboco-flow / HTTP route after claim_gate_review | -| pr_fail | roboco/services/gateway/choreographer/pr_gate.py | pr_reviewer calls pr_fail(task_id, issues=[...]) via roboco-flow / HTTP route after claim_gate_review | -| claim_pr_review | roboco/services/gateway/choreographer/pr_review.py | pr_reviewer calls claim_pr_review(task_id) via roboco-flow / HTTP route on a pending external/fork-PR review task | -| post_pr_review | roboco/services/gateway/choreographer/pr_review.py | pr_reviewer calls post_pr_review(task_id, body, event, findings?) via roboco-flow / HTTP route to post one change-request and complete | +| ensure_workspace | roboco/services/workspace.py | GitService.create_branch_for_task / push / PR ops; orchestrator spawn ensure; gateway content_actions; called transitively by many verbs | +| ensure_worktree | roboco/services/workspace.py | GitService.create_branch_for_task on fresh claim (worktree add -b ) | +| ensure_worktree_for_resume | roboco/services/workspace.py | GitService._ensure_worktree_for_commit (commit/rebase paths) | +| ensure_worktree_self_heal | roboco/services/workspace.py | orchestrator._ensure_worktree_before_spawn before -w container launch (replaces the former ensure_worktree_for_resume call there; handles vanished clones + missing branch refs) | +| remove_worktree | roboco/services/workspace.py | TaskService terminal/cancel paths + claim-rollback (mid-claim failure) | +| delete_local_branch | roboco/services/workspace.py | TaskService terminal/cancel paths (right after remove_worktree) + GitService.cleanup_stale_branches sweep | +| ensure_read_clone | roboco/services/workspace.py | ConventionsService.scaffold/effective-map reads (project-level conventions metadata) | +| dry_upgrade_changes_lockfile | roboco/services/workspace.py | DepUpdateEngine periodic probe loop | +| fetch_branch_for_inspection | roboco/services/workspace.py | gateway content_actions (QA/Documenter/PM need to read a dev branch) | +| read_toolchain_status | roboco/services/workspace.py | GitService spawn-time toolchain runnability check | +| list_workspaces / workspace_exists / delete_workspace | roboco/services/workspace.py | admin API routes / project service maintenance | ## Config Flags -- ROBOCO_TOOLCHAIN_MATCH_ENABLED (gates _toolchain_broken_guard in _pr_pass_blocked — inert when off) -- ROBOCO_CONVENTIONS_ENABLED (gates _conventions_guard in _pr_pass_blocked — inert when off) -- ROBOCO_PR_REVIEWER_NOTES_MIN_CHARS / settings.pr_reviewer_notes_min_chars (tracing gate substantive-note threshold for pr_pass/pr_fail/post_pr_review) -- CI-status guard is always armed when the toolchain can reach get_pr_ci_status via git service. Configuration gaps (missing project/git_url/token) and unreachable/nonexistent repos (404 or network error) classify as no_ci_configured and pass through with evidence stamp. Genuine API failures on reachable repos classify as error and stay fail-closed (retryable). A project with no CI configured at all also passes through cleanly (no_ci_configured). The guard never blocks pr_pass on a misconfigured project. +- ROBOCO_WORKSPACES_ROOT (settings.workspaces_root; default /data/workspaces) +- ROBOCO_WORKSPACE_AUTO_CLONE (settings.workspace_auto_clone; default true) +- ROBOCO_WORKSPACE_CLONE_TIMEOUT (settings.workspace_clone_timeout; default 300s) - bounds git clone + fetch_branch_for_inspection fetch +- ROBOCO_WORKSPACE_REFRESH_FETCH_TIMEOUT_SECONDS (settings.workspace_refresh_fetch_timeout_seconds; default 60s) - bounds the healthy-clone scoped refresh fetch +- ROBOCO_WORKSPACE_INSTALL_DEV_DEPS (settings.workspace_install_dev_deps; default true) - gates post-clone dev-dep install +- ROBOCO_WORKSPACE_DEP_INSTALL_TIMEOUT_SECONDS (settings.workspace_dep_install_timeout_seconds; default 600s) - bounds uv sync / pnpm install / toolchain smoke / dep-upgrade probe +- ROBOCO_TOOLCHAIN_MATCH_ENABLED (settings.toolchain_match_enabled; default off) - gates provisioning against the target project's declared Python + the runnability smoke marker +- ROBOCO_CONVENTIONS_ENABLED (settings.conventions_enabled; default off) - gates the first-clone conventions scaffold PR +- ROBOCO_AGENT_UID / ROBOCO_AGENT_GID (env; default 1000/1000) - the agent container user the workspace is chowned to ## Gotchas -- self_review_block is dormant by design: markers.get_original_developer is never set on assembled coordination tasks (only on dev-leaf tasks at QA/doc claim), and GatewayAgentView carries no slug so actor_slug was previously always None. The fix sets actor_slug=str(reviewer_agent_id) so the gate is wired, but it only fires if the marker were ever set to the reviewer's UUID — currently never. Don't assume the self-review defense is active in production today. -- pr_fail captures the PR head SHA BEFORE the DB transition commits (_record_gate_verdict_for runs before run_intent). If the branch advances between capture and transition the recorded SHA is stale, but the unchanged-PR gate in submit_root fails open on stale/missing SHA — only the exact-unchanged case is hard-blocked. -- _gate_decision guards t is None after run_intent: a concurrent cancel or racing reviewer between the precondition gate and the runner's final action makes run_intent return None; without this guard the post-PR/a2a dereferences would crash. Any future reorder must preserve this check. -- _post_gate_review_to_pr always posts COMMENT (not APPROVE/REQUEST_CHANGES) on a root→master PR because only the CEO merges master. A root→master PR is now identified by `is_root = parent_task_id is None OR is_batch_root_subtask(batch_id, parent_task_id)` — so a MegaTask root-subtask (which has a parent = the umbrella but opens its own root→master PR) also gets COMMENT, not APPROVE. A non-batch cell-PM coordination root keeps batch_id=None so it remains a cell→root PR (APPROVE/REQUEST_CHANGES). Added in f90565ea. -- resolve_task_project_slug cell_projects branch sorts by m.team.value — assumes every cell_map mapping has a non-None team with a .value; a malformed mapping would raise AttributeError (uncaught) and bubble out of the slug resolver (which callers tolerate as best-effort None only if wrapped — _capture_pr_head_sha wraps it, _post_gate_review_to_pr does NOT wrap the slug call). -- _is_hand_formatted_verdict (UPDATED in 536bbb64 #188): previously a plain lower-case substring match that would false-refuse a body quoting a PR's own ## headers (e.g. `> ## Summary`); now uses a regex anchored to line-start (`^[ \t]*## ...`, re.MULTILINE) so a quoted/indented header or a mid-prose mention does not trip the guard. The remaining false-positive window: a reviewer deliberately writing `## Summary` at the start of a line in their free-text body (with findings=[]) — intentionally refused, steering them to the structured-findings path. -- _record_gate_verdict for pr_fail with issues now writes a templated summary ('In-path PR-review gate requested changes - N issue(s) listed below.') instead of the full notes into the structured note's summary field; the full issues text lives in the issues slot. The GitHub PR post and a2a still use the raw notes string. Readers of notes_structured.pr_review.summary no longer get the verbatim issues. -- (Revision-findings ledger, uncommitted branch `feature/findings-ledger`) `PrReviewContent.findings` was previously hardcoded to `[]` on every `_gate_verdict_payload` write — it now carries the real validated `Finding` list on a findings-driven `pr_fail`, the first time this slot is ever non-empty. `claim_gate_review`'s evidence (`_build_gate_review_evidence`) additionally carries `revision_findings` (open) and, on a round ≥2 review, `prior_findings` (the full ledger) so a re-reviewing reviewer checks prior findings instead of re-deriving them. `pr_pass` now bulk-verifies (`addressed→verified`) every `origin=pr_gate` finding in the same transaction via `_stamp_gate_findings_verified_or_rejection` — a stamp failure rejects the pass outright, it is NOT best-effort. Full detail: `docs/map/review-findings.md`. -- claim_gate_review does NOT transition the task (status stays awaiting_pr_review) — this is intentional so pr_pass/pr_fail's source-status still matches. A reviewer who claims but never decides leaves the task assigned but still awaiting_pr_review; the stale-claim reaper path is the recovery. -- PRReviewerMixin.post_pr_review runs content gates BEFORE _resolve_post_body, but _resolve_post_body itself can return an Envelope (malformed findings) which is then handled — the verdict_consistency_gate already ran on the (event, findings) pair, so a malformed-findings Envelope is a distinct later failure. -- resolve_task_project_slug was extracted to module-level specifically so _impl.py's _LegacyChoreographer (which does NOT inherit ChoreographerHelpers) can reach it; the mixin method _project_slug_for is now a one-line delegate. Changing the resolver signature would break both the mixin and the _impl.py unchanged-PR gate. -- Before `_gate_diff_parent` (#444/#454), the gate's evidence diff and the conventions guard derived their base via `parent_branch_for` string surgery on the child's OWN branch name — correct for a same-team hop (cell dev → cell PM) but wrong for a cross-team hop (a frontend child of a main_pm root derives a ref that never existed) and silently fell back to the repo default branch, so the reviewer judged inherited base-branch content as the task's own work. `_gate_diff_parent` is now consulted only when no explicit `base` is given, so the pinned literal-base contract (`base="HEAD~1"`) and every other diff caller (QA/doc/content) are untouched; it is skipped entirely while `conventions_enabled` is off since only the conventions guard consumes it in `_pr_pass_blocked`. +- The per-(project,agent) asyncio.Lock (_ENSURE_WORKSPACE_LOCKS) is process-local only. Across orchestrator processes (or restarts) two coroutines can still race the .git-exists check; the rmtree-partial-then-clone path assumes single-process serialization. +- _is_workspace_healthy requires .git + HEAD + objects/ — a stub clone from a failed `git clone` (only FETCH_HEAD) is intentionally rejected and re-cloned. A regression that loosens this check re-mounts agents on broken clones. +- _fetch_origin_best_effort is credential-LESS (token was scrubbed from .git/config by _clone_repo). For PRIVATE repos the refresh fetch silently fails (downgraded to DEBUG) and the workspace stays at clone-time refs until the next token-bearing operation (create_branch / fetch_branch_for_inspection). Stale-base risk for private repos. +- _ensure_agent_owned is called TWICE in the healthy path (before and after the fetch) because the root-side fetch writes root-owned pack/refs under .git/objects and .git/refs — skipping the second chown leaves the agent unable to update refs. +- 30s TTL fetch caches (_fetch_cache instance attr, _read_clone_synced module attr) are keyed by str(workspace); a Path that resolves to the same dir via a different route (worktree vs clone root) would not share a cache entry. +- _link_shared_venv only symlinks if clone_root/.venv EXISTS (F-fix 0f7d6929). On the very first claim, install_dev_deps provisions the venv AFTER ensure_worktree already ran — the worktree add path can run before .venv exists, so the symlink is skipped and a later ensure (resume/commit) self-heals it. If no later ensure fires, uv re-syncs a worktree-local venv (the bug the F-fix mitigated but did not fully close — recovery of an already-clobbered worktree venv is out of scope). +- _PRUNE_DIRS excludes .venv/node_modules from the chown walk for speed, but .uv-python is intentionally NOT pruned (so the fetched CPython is chowned). If .uv-python grows huge on a monorepo, the walk slows. +- _clone_repo does NOT pass --single-branch: agents/QA/doc must fetch peer feature branches. A regression adding --single-branch would silently break `checkout origin/feature/...`. +- _assert_no_pat_leak scans .git/** for ghp_/github_pat_/x-access-token bytes and rm-trees the workspace on any hit. Binary pack files are read as bytes; a coincidental byte sequence in a blob is unlikely but theoretically possible — false positives destroy the workspace. +- Both clone-failure except branches (CalledProcessError, TimeoutExpired) rmtree the workspace before raising (F063 bb94e6ba). If rmtree itself fails (busy mount), the half-configured clone with the PAT in .git/config could survive — the leak-check did not run. +- _maybe_scaffold_conventions uses a process-wide _SCAFFOLD_ATTEMPTED set: the scaffold is attempted at most once per project per orchestrator process. A first-clone failure is never retried within the same process lifetime. +- dry_upgrade_changes_lockfile holds the read-clone lock only for the local clone step, then releases it before the upgrade runs. The tiny gap between ensure_read_clone releasing and the probe re-acquiring is safe only because any concurrent _sync_read_clone completes under the lock first — a future change that interleaves could race. +- get_workspace_path raises WorkspaceError if team is None rather than producing a literal 'None' segment; resolve_workspace falls back to Team.BACKEND when agent.team is falsy — agents missing a team silently land under backend/. +- fetch_branch_for_inspection reuses workspace_clone_timeout (300s) for a single-branch fetch, not the shorter refresh timeout — a hung remote blocks the QA/Doc verb for 5 minutes. +- delete_local_branch only detaches the ref; remove_worktree only detaches the worktree. Callers MUST run remove_worktree first — `git branch -d/-D` refuses a branch still checked out elsewhere in the clone (the worktree). Skipping the order silently no-ops the branch delete (check=False swallows the refusal). ## Drift from CLAUDE.md -- CLAUDE.md verb table lists pr_reviewer verbs as 'claim_pr_review, post_pr_review (inbound external/fork PRs), claim_gate_review, pr_pass, pr_fail (in-path assembled-PR gate)' — matches the code exactly. No drift. -- CLAUDE.md says the pr_reviewer 'posts its change-request on the PR itself (no agent comms)'. The code now ALSO a2a's pr_fail change-requests to the owning PM (_deliver_pr_fail_to_owner) — an additive agent-comms side effect NOT reflected in CLAUDE.md's 'no agent comms' claim for the in-path gate. This is intentional (closes the blind re-submit loop) but the doc still says no agent comms. -- CLAUDE.md does not mention the pr_fail head_sha capture / unchanged-PR submit_root gate (the 2026-06-27 pr_fail loop fix) anywhere — it's a live behavioral guarantee absent from the doc. +- CLAUDE.md (Git Credentials / Token flow) says 'HTTPS URLs require tokens - attempting to clone without a token will raise WorkspaceError' — matches _resolve_git_token (line 787). No drift. +- CLAUDE.md (Multi-Agent Workspace Structure) says a Python workspace runs `uv sync --extra dev` (not plain `uv sync`) so the dev extra is present — matches _detect_dep_commands (line 359). No drift. +- CLAUDE.md (Work Sessions / fresh claim) says a fresh claim git-resets the workspace to a clean tree (`git reset --hard`) before checking out the new branch. ACTUAL code: F123 (67107f8a) replaced that reset+checkout with per-task `git worktree add` under {clone_root}/.worktrees/{task}/ — the reset --hard no longer runs on fresh claim. This is a real drift between the doc narrative and the post-F123 code. +- CLAUDE.md (Architectural Conventions Standard) says the read clone is pinned to the default branch's HEAD via WorkspaceService.ensure_read_clone — matches (line 958). No drift. +- CLAUDE.md (Dependency-update bot) says WorkspaceService.dry_upgrade_changes_lockfile runs dep_update_command in a throwaway clone of the READ CLONE and the read clone is never mutated — matches (line 1459). No drift. ## Changes Since Baseline | SHA | Subject | Impact | |---|---|---| -| 15effce0 | Chore: 141 Gaps fill-in (#283) — pr_gate.py: wire self_review_block for pr_pass/pr_fail | _gate_preflight spec_ctx now passes actor_slug=str(reviewer_agent_id) and original_developer_slug=markers.get_original_developer(t) instead of agent.slug (always None for GatewayAgentView). Self-review defense is now wired, though dormant because the marker is never set on assembled coordination tasks. | -| 15effce0 | Chore: 141 Gaps fill-in (#283) — pr_gate.py: capture pr_fail head_sha into verdict note | New _record_gate_verdict_for + _capture_pr_head_sha stamp the assembled PR head SHA into notes_structured.pr_review.head_sha on pr_fail, feeding the submit_root unchanged-PR hard-block gate. Fail-open on any capture failure. | -| 15effce0 | Chore: 141 Gaps fill-in (#283) — pr_gate.py: a2a pr_fail to owning PM + Main-PM-root steer | New _deliver_pr_fail_to_owner sends the pr_fail change-requests to the assigned PM via a2a (best-effort) with a steer for Main-PM branch-bearing roots to re-delegate not re-submit. Closes the blind re-submit loop (PR #138). | -| 15effce0 | Chore: 141 Gaps fill-in (#283) — pr_gate.py: None-guard after run_intent for concurrent transition | _gate_decision now checks t is None after runner.run_intent and returns a clean invalid_state envelope instead of dereffing None → 500. Covers concurrent cancel / racing reviewer between precondition gate and final action. | -| 15effce0 | Chore: 141 Gaps fill-in (#283) — pr_gate.py: _toolchain_broken_guard now reviewer=True | _pr_pass_blocked passes reviewer=True to _toolchain_broken_guard (signature widened to distinguish reviewer context). pr_pass still refused on broken toolchain; pr_fail unaffected. | -| 15effce0 | Chore: 141 Gaps fill-in (#283) — pr_gate.py: structured verdict note carries issues + head_sha | _record_gate_verdict payload now includes issues list for pr_fail and head_sha; summary for pr_fail-with-issues is a templated sentence instead of the full notes (dedup on the Task Details card). | -| 15effce0 | Chore: 141 Gaps fill-in (#283) — pr_review.py: extract module-level resolve_task_project_slug + cell_projects fallback | _project_slug_for delegates to new module-level resolve_task_project_slug (shared with _impl.py unchanged-PR gate); adds a third fallback branch for ad-hoc per-cell-map root-subtasks (migration 052) so the gate verdict reaches the PR in the mapped repo. | -| 15effce0 | Chore: 141 Gaps fill-in (#283) — pr_review.py: hand-formatted-verdict body guard | New _is_hand_formatted_verdict + _post_pr_review_content_gates refuse a free-text body carrying ## summary/issues/verdict/findings headers when findings is empty, steering the reviewer to the structured-findings path. Observed live: a duplicated self-formatted verdict posted to a contributor's PR. | -| 15effce0 | Chore: 141 Gaps fill-in (#283) — pr_review.py: fold content gates into one helper | post_pr_review now calls _post_pr_review_content_gates (verdict consistency + no-hand-format) instead of only _verdict_consistency_gate; keeps the verb body under the return-count lint ceiling. | +| bb94e6ba | [F063] workspace._clone_repo: rmtree half-configured clone on failure | Both clone-failure except branches (CalledProcessError, TimeoutExpired) now shutil.rmtree the workspace before raising WorkspaceError, so a half-configured clone with the PAT still in .git/config cannot survive and be re-mounted by the next ensure_workspace health short-circuit. | +| c3057bb3 | Updated domain | Trivial: git config user.email domain bump in _clone_repo's _configure_git (now {slug}@roboco.tech). No behavior change beyond commit author email. | +| 1a773e45 | [F116] hold the read-clone lock across the dep-probe local clone | dry_upgrade_changes_lockfile split into _clone_local_into (run UNDER the _meta-conventions lock) + _probe_lockfile_on_clone (lock-free on the independent copy), closing a race where a concurrent ensure_read_clone hard-reset could mutate the read clone mid-clone. | +| 3441e371 | [sweep] strip Fxxx audit-ID tokens + trim bloated comments/docstrings | Comment/docstring prose only — no code-line edits in workspace.py. Reduced narrative bulk; no behavioral change. | +| 67107f8a | [F123] per-task git worktrees — coordinator PM roots no longer clobber each other | Major: added get_clone_root_path/get_worktree_path/ensure_worktree/ensure_worktree_for_resume/remove_worktree/_link_shared_venv/_worktree_git + _resolve_clone_root + _uv_subprocess_env worktree-awareness. Replaced the fresh-claim `git reset --hard` + `checkout -b` with `git worktree add` under {clone_root}/.worktrees/{task}/ so a coordinator PM holding multiple in_progress roots no longer clobbers one root's working tree by checking out another's branch. .venv symlinked from worktree to clone root; .uv-python gitignored. | +| 0f7d6929 | [F-fix] gate the worktree .venv symlink on the clone-root venv existing | _link_shared_venv now no-ops when clone_root/.venv does not yet exist (instead of dangling a symlink), so uv no longer errors or silently re-syncs a worktree-local venv in the near-zero gap before install_dev_deps provisions the clone-root venv. A later ensure self-heals the link. | -> Post-snapshot updates (since 2026-06-29): two commits touched this slice. -> - **536bbb64** (Chore/all/logical gaps sweep #286, 2026-06-30): pr_gate.py — (a) `_post_gate_review_to_pr` wraps slug-resolution in try/except so a malformed cell_map mapping can no longer 500 the reviewer after a committed gate transition (#82 FIXED); (b) `_is_hand_formatted_verdict` regex anchored to line-start so quoted/indented PR headers no longer false-refuse post_pr_review (#188 FIXED); (c) `_re_stamp_pr_fail_head_sha_if_advanced` new method — re-captures head SHA after the transition commits and re-stamps the verdict note only if it advanced, closing the stale-SHA false-allow window (#189 FIXED); (d) `claim_gate_review` passes `skip_dev_guards=True` to `_run_claim_guards` so already_active/paused/lane guards never block a pr_reviewer from claiming a gate review (#192 FIXED); (e) two new static helpers extracted from `_post_gate_review_to_pr`: `_gate_review_event_verdict` and `_gate_review_body`. The unchanged-PR guard's fail-open slug/git error now logs a warning so a regression cannot silently disable the loop-stopper (#5/#222). -> - **f90565ea** ([sweep] pr_gate: classify MegaTask root-subtask as root #608, 2026-06-30): `_post_gate_review_to_pr` now uses `is_batch_root_subtask` (imported from `roboco.foundation.policy.batch`) in addition to `parent_task_id is None` to identify root→master PRs. A MegaTask root-subtask (parent=umbrella, batch_id set) opens its own root→master PR but previously got APPROVE/REQUEST_CHANGES instead of COMMENT — fix prevents a single-approval branch-protection rule from allowing a non-CEO merge. -> - **7ff70ab5** (fix(gateway): gate review diffs against the task's real parent branch #444/#454, 2026-07-10): new `_gate_diff_parent` + `preferred_parent` param threaded through `git.diff` / `list_changed_files` / `conventions_check_for_task`, resolving the assembled task's real parent branch from the parent TASK's own `branch_name` instead of string surgery on the child branch name — fixes a live cross-team false-fail (bounced a goals-tab fix three times) where the gate attributed inherited base-branch content to the task under review. -> - (uncommitted, branch `feature/findings-ledger`, 2026-07-11) Revision-findings ledger: `pr_fail` gains structured `findings` (validated + count-guarded + inserted to the ledger before the verdict note), `pr_pass` gains a same-transaction verify-stamp, and gate evidence gains `revision_findings`/`prior_findings`. See `docs/map/review-findings.md`. +> Post-snapshot updates (since 2026-06-29): 5 commits touched workspace.py. (1) 9faf2763 [hotfix] strip VIRTUAL_ENV + UV_PROJECT_ENVIRONMENT from _uv_subprocess_env so workspace uv calls stop warning about the image-baked /app/.venv pin. (2) cfe725da [hotfix] worktree: clone root left on the task branch caused fatal "already checked out" on every worktree add re-dispatch — added _clone_root_default_branch + _park_clone_root_off_branch; ensure_worktree and ensure_worktree_for_resume now call _park_clone_root_off_branch before the add to restore the F123 invariant. (3) 536bbb64 (logical-gap sweep PR#286) added _fetch_branch_ref + ensure_worktree_self_heal: the orchestrator's _ensure_worktree_before_spawn now calls ensure_worktree_self_heal instead of bare ensure_worktree_for_resume so a vanished clone (redeploy/disk loss) that left no local branch ref recovers the pushed commits from origin before re-attaching. (4) 3aff6e04 and 15effce0 (gap-fill PRs #285/#283) contributed earlier worktree + dep-probe plumbing (the _clone_local_into / _probe_lockfile_on_clone split already captured in the baseline). +> +> Further post-snapshot update (#534, env-branches ladder): `ensure_workspace`'s fresh-clone branch and `ensure_read_clone` both resolve their target branch via `roboco.models.env_branches.head_branch(project)` — the env-ladder's head rung — instead of reading `project.default_branch` directly. A project with no declared ladder resolves to the identical `default_branch` value via the read-time shim, so this is behavior-preserving until the CEO declares a real ladder in the panel. +> +> (open PR #548, branch `feature/wave-2-hygiene-charts`, 2026-07-17) Added `delete_local_branch` (line 787) so `TaskService`'s cancel/terminal-completion cleanup and `GitService.cleanup_stale_branches` can reap a spent local branch ref, not just the worktree — previously every task an agent ever claimed leaked a permanent `refs/heads/{branch}` in that agent's clone. ## Regression Risks | Title | File:Line | Claim | Severity | |---|---|---|---| -| CI-status guard reads GitHub check-runs only, not legacy commit-status API | roboco/services/gateway/choreographer/pr_gate.py:520 | _ci_status_guard and get_pr_ci_status read only the check-runs API endpoint. A repo whose only CI signal is the legacy commit-status API would show zero check-runs and be classified as no_ci_configured (passes through). Noted as ponytail-comment in git.py with the upgrade path if a project ever needs it. | low | -| CI-status classification: config gaps and unreachable repos now explicitly no_ci_configured | roboco/services/git.py:_resolve_ci_head_sha | Missing project/git_url/git-token, or unreachable/nonexistent repo (404 on PR head lookup or network error), all classify as no_ci_configured → pr_pass passes through with evidence stamp. Only genuine GitHub API failures on reachable repos classify as error → pr_pass stays fail-closed (retryable). By design: configuration gaps should not block the gate, but real API failures should fail-closed to avoid false-green verdicts. A deliberately misconfigured project's CI is silently not enforced, with clear evidence in the verdict note. | low | -|---|---|---|---| -| self_review_block could fire if a reviewer is also the original developer | roboco/services/gateway/choreographer/pr_gate.py:202 | actor_slug=str(reviewer_agent_id) + original_developer_slug=markers.get_original_developer(t). The comment asserts dormancy because the marker is never set on assembled coordination tasks. If a future change sets the marker on an assembled task (or a reviewer UUID coincides with the recorded dev UUID), pr_pass/pr_fail would be refused as self-review with no remediate path. The defense is correctly wired but unguarded by a test asserting dormancy. | low | -| ~~resolve_task_project_slug cell_projects branch can raise AttributeError on malformed mapping~~ **FIXED 536bbb64 #82** | roboco/services/gateway/choreographer/pr_review.py:594 | ~~sorted(cell_map, key=lambda m: m.team.value) assumes every mapping has a non-None team with .value. _capture_pr_head_sha wraps the slug call in try/except (fail-open), but _post_gate_review_to_pr calls self._project_slug_for(t) WITHOUT a try/except — a malformed cell_map mapping would raise and abort the verdict PR post (best-effort but the exception escapes the helper, caught only by the outer try in _post_gate_review_to_pr's git.post_pr_review call, NOT the slug resolution).~~ _post_gate_review_to_pr now wraps the slug-resolve call in its own try/except (mirrors _capture_pr_head_sha) — a malformed mapping logs and returns, no longer 500s the reviewer after the committed gate transition. The underlying AttributeError possibility in resolve_task_project_slug remains but is contained. | medium | -| ~~_is_hand_formatted_verdict false-positive on summaries quoting PR-added headers~~ **FIXED 536bbb64 #188** | roboco/services/gateway/choreographer/pr_review.py:174 | ~~Substring match on '## summary'/'## issues'/'## verdict'/'## findings' in lowercased body. A reviewer summarizing a PR that itself adds a '## Summary' section (quoting it in the body) with findings=[] would be falsely refused.~~ Regex now anchored to line-start (^[ \t]*## ..., re.MULTILINE) — quoted headers (> ## Summary) and mid-prose mentions no longer trip the guard. | low | -| ~~pr_fail head_sha captured before transition may be stale vs the committed verdict~~ **FIXED 536bbb64 #189** | roboco/services/gateway/choreographer/pr_gate.py:240 | ~~_record_gate_verdict_for awaits _capture_pr_head_sha (GitHub pulls API) then writes the note, all before run_intent commits the transition. If the assembled PR advances between capture and the transition commit, the recorded SHA no longer matches the PR head at the moment of needs_revision.~~ New `_re_stamp_pr_fail_head_sha_if_advanced` re-captures the SHA after run_intent commits and re-stamps the note only if it advanced; no-advance is a single write. Fail-open: a re-capture failure leaves the pre-transition SHA in place. | low | -| Structured pr_review summary no longer contains verbatim issues for pr_fail | roboco/services/gateway/choreographer/pr_gate.py:444 | _record_gate_verdict now writes a templated summary for pr_fail-with-issues instead of the full notes. Any consumer that parsed notes_structured.pr_review.summary for the change-request text (rather than .issues) now gets a generic sentence. The a2a body and GitHub PR post still use raw notes, but briefing/mirror readers of the summary field lose the verbatim issues. | low | -| _deliver_pr_fail_to_owner a2a to assigned PM may target the wrong agent after a reassign | roboco/services/gateway/choreographer/pr_gate.py:267 | a2a.send to_agent=t.assigned_to at the moment pr_fail runs. If the task was reassigned between claim_gate_review and pr_fail, the change-requests go to the new assignee, not the reviewer who claimed it. Best-effort and the assigned PM is the intended recipient, but a just-reassigned PM with no context receives raw review issues. | low | -| ~~claim_gate_review runs _run_claim_guards but the gate task is not a normal claim~~ **FIXED 536bbb64 #192** | roboco/services/gateway/choreographer/pr_gate.py:84 | ~~_run_claim_guards is invoked for claim_gate_review (which does NOT transition). If a claim guard (e.g. already_active / lane barrier) rejects, the reviewer cannot claim the gate review.~~ `_run_claim_guards` is now called with `skip_dev_guards=True` — the already_active, paused, and lane barriers are skipped for claim_gate_review; only the dependency guard is kept. A pr_reviewer with another active task is no longer blocked from claiming a gate review. | low | +| Worktree .venv symlink self-heal depends on a later ensure firing | roboco/services/workspace.py:534 | ensure_worktree (fresh claim) runs _link_shared_venv BEFORE install_dev_deps provisions clone_root/.venv, so the symlink is skipped on the first claim. The shared-venv optimization only self-heals if a later ensure (resume/commit via _ensure_worktree_for_commit) re-runs _link_shared_venv. If the agent commits via a path that does not re-invoke ensure and uv re-syncs a worktree-local .venv first, the lexists guard prevents replacing the real dir and the worktree is stuck with a duplicated venv. The F-fix mitigated the dangling-symlink case but did not close the already-clobbered-venv recovery (explicitly out of scope per commit msg). | medium | +| ensure_worktree reuses an existing branch ref without validating it points at base | roboco/services/workspace.py:570 | When branch_exists is True (re-claim after rollback) ensure_worktree runs `worktree add ` with no -b and no base. If the surviving branch ref was left at an unexpected commit (e.g. a prior partial rebase, or a force-pushed-and-locally-stale ref), the worktree is created at that commit, not at the intended base. The caller (GitService.create_branch_for_task) assumes a fresh branch at base; a stale ref could spawn the agent on the wrong HEAD. | medium | +| ensure_worktree_for_resume silently re-adds a worktree whose branch was force-updated remotely | roboco/services/workspace.py:591 | On resume via GitService._ensure_worktree_for_commit, ensure_worktree_for_resume re-adds the worktree from the surviving local branch ref (no fetch, no base). If the branch was force-pushed remotely while the agent was down and the local ref is stale, the agent resumes on the old commits with no warning. NOTE: the orchestrator spawn path (cfe725da/536bbb64) now calls ensure_worktree_self_heal instead, which fetches the branch ref from origin before re-attaching — the spawn path is resolved. The GitService commit path still uses ensure_worktree_for_resume without a fetch. | medium | +| PAT-leak scan cannot run if .git was wiped by a prior failed rmtree | roboco/services/workspace.py:1391 | _assert_no_pat_leak (line 1352) guards on `git_dir.exists()` and returns early if not. If a catastrophic clone left .git partially absent but the auth URL written elsewhere (e.g. into .git/config before .git/objects was created), the early return means the leak check is skipped. Combined with the F063 rmtree-on-failure this is low risk, but a rmtree that fails silently (ignore_errors=True at line 1382 only triggers on leak detection, not on the failure branches) could leave a tokenized .git/config. | low | +| dry_upgrade probe lock gap could race a future interleaved sync | roboco/services/workspace.py:1673 | The dep-update probe acquires the _meta-conventions lock only around _clone_local_into (line 1676) and releases it before _probe_lockfile_on_clone (line 1679). The commit msg argues this is safe because any concurrent _sync_read_clone completes under the lock first. This holds ONLY because _sync_read_clone is the sole other holder; if a future change adds a third concurrent mutator of the read clone that interleaves between the release and re-acquire (none today), the local clone could read a half-mutated source. Fragile invariant documented only in the commit, not enforced. | low | +| _fetch_origin_best_effort TTL cache not shared between clone root and worktree paths | roboco/services/workspace.py:1038 | _fetch_cache is keyed by str(workspace) on the instance. ensure_workspace is called with the clone-root path, but a worktree-path caller (none currently call ensure_workspace directly with a worktree path, but _resolve_clone_root exists to support worktree-aware uv env) would get a separate cache entry. Not a current bug, but a future worktree-aware ensure_workspace call could double-fetch. | low | +| _ensure_agent_owned walk excludes .venv/node_modules but agent may need to write them | roboco/services/workspace.py:67 | _PRUNE_DIRS skips .venv, node_modules, .next etc. from the chown walk for speed. The agent normally owns these (it created them) and the symlinked worktree .venv points to the clone-root .venv which IS walked (it is not under a pruned name at clone root). But a worktree-local .venv created by uv when the symlink was missing (regression risk #1) would NOT be chowned, leaving the agent unable to write into it. Edge case, low severity. | low | ## Health -The slice is well-structured and defensively hardened. Both mixins follow the established choreographer pattern (TYPE_CHECKING-only base, spec gate + tracing gate + verb runner, best-effort side-effects after the DB transition, standardized Envelopes). The 15effce0 changes are coherent: the pr_fail loop is closed at three layers (head_sha capture + submit_root hard-block, a2a to owning PM, Main-PM-root steer), the concurrent-transition None-guard plugs a real crash, and the external-PR hand-format guard addresses an observed live defect. Post-snapshot (536bbb64 + f90565ea) hardening: the slug-resolution AttributeError in _post_gate_review_to_pr is now contained by try/except (#82 FIXED), the hand-format guard is now regex-anchored-to-line-start instead of substring (#188 FIXED), the stale-SHA false-allow window is closed by the post-transition re-stamp (#189 FIXED), the pr_reviewer active-task guard is skipped for claim_gate_review (#192 FIXED), and MegaTask root-subtasks correctly get COMMENT on their root→master PR. The main remaining latent concern is the self_review_block dormancy (correctly wired, no test asserting dormancy — low, no known path to activate). Coverage and tracing parity with QA's pass_review/fail_review is maintained. +WorkspaceService is a mature, heavily-instrumented slice with strong defensive hygiene: per-(project,agent) asyncio locks, partial-clone detection + rmtree, a real .git+HEAD+objects health check, scoped + TTL-cached refresh fetches, PAT injection that is never persisted to .git/config, a belt-and-suspenders leak scan that destroys the workspace on any hit, idempotent lockfile-digest-gated dev-deps install, and F123 per-task worktrees that eliminated the coordinator-PM clobber. The F063 + F116 + F123 + F-fix wave closed real deploy-blocker races (PAT leak on half-configured clone, read-clone mid-clone race, root-clobber, dangling venv symlink). Residual risk is concentrated in the worktree venv-symlink timing (first-claim skip depends on a later ensure to self-heal), the resume path reusing a possibly-stale local branch ref without a freshness check, and the process-local (not cross-process) ensure-workspace lock. The slice diverges from CLAUDE.md's "fresh claim git reset --hard" narrative — by design, post-F123 — and that doc drift should be reconciled. Overall integrity is high; the regression risks are edge-case rather than core-path. +# RoboCo Slice Map — `support-services` + +Slice key: `support-services` Repo root: `/Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco` Scope: `roboco/services/{agent,health,settings,toolchain,provider,llm,proactive,transcription,base,exceptions}.py`, `roboco/events/`, `roboco/seeds/`, `roboco/utils/` + +## Purpose + +Cross-cutting support layer beneath the delivery services: the service-base/error hierarchy every service inherits, Fernet crypto + UUID converters, the Redis-Streams event bus + workflow trigger handlers, the static seed data that bootstraps agents, infrastructure health probes, the runtime-editable settings/feature-flag store, agent-record lookups, per-project Python interpreter resolution, provider-config CRUD + per-agent model routing, proactive knowledge injection, and raw-stream transcription buffering. None of these own the task lifecycle; they are the plumbing the lifecycle services, orchestrator, and API routes compose on top. + +## Files + +| Path | Role | approx LOC | +|---|---|---| +| `roboco/services/base.py` | `BaseService` (session-bound) + `SingletonService` + `SingletonHolder[T]` + `ServiceError` hierarchy (NotFound/Validation/Conflict/Unauthorized/ServiceUnavailable) | 226 | +| `roboco/services/exceptions.py` | LLM-provider rate-limit exception + `Retry-After` parser + retry constants | 81 | +| `roboco/services/agent.py` | Thin read-side `AgentService` over `AgentTable` (list/get by uuid/slug/raise) | 74 | +| `roboco/services/health.py` | `check_database` / `check_redis` infrastructure probes backing `/health` | 32 | +| `roboco/services/settings.py` | `SettingsService` CRUD over `system_settings` + `FEATURE_FLAGS` registry + startup overlay onto `roboco.config.settings` | 165 | +| `roboco/services/toolchain.py` | Pure resolver: target project's Python interpreter from `pyproject.toml` / `.python-version` | 117 | +| `roboco/services/provider.py` | `ProviderService` CRUD for `provider_configs` rows + Fernet-encrypted token tri-state updates | 229 | +| `roboco/services/llm.py` | `ModelRoutingService`: resolve (provider, model) per agent spawn; assignment CRUD; mode apply (anthropic/grok/ollama/self_hosted/mix); Ollama probe | 599 | +| `roboco/services/proactive.py` | `ProactiveKnowledgeService`: assemble RAG context packages on task-claim / session-start | 542 | +| `roboco/services/transcription.py` | `TranscriptionService`: buffer raw LLM stream chunks into extractable segments | 278 | +| `roboco/events/__init__.py` | Public re-exports for the event system | 41 | +| `roboco/events/bus.py` | Backward-compat shim: `EventBus = StreamEventBus`, `get_event_bus`, `init_event_bus` | 57 | +| `roboco/events/handlers.py` | Workflow trigger handlers (task status → notifications, QA result, blocker, question, auditor spawn) + `register_default_handlers` | 419 | +| `roboco/events/stream_bus.py` | `StreamEventBus`: Redis Streams durable event bus with consumer groups, ACK, pending recovery, periodic reclaim loop, dead-letter for undecodable messages | 604 | +| `roboco/seeds/__init__.py` | Re-exports seed constants | 25 | +| `roboco/seeds/initial_data.py` | Static seed data (agents) derived from `foundation` catalogs | 332 | +| `roboco/utils/__init__.py` | Re-exports crypto + converter helpers | 23 | +| `roboco/utils/converters.py` | `InvalidIdentifierError` + `require_uuid` / `to_python_uuid` / `to_python_uuid_list` + `repo_key` | 99 | +| `roboco/utils/crypto.py` | Fernet `encrypt_token` / `decrypt_token` / `is_encryption_configured` + `EncryptionError` | 111 | +| `roboco/utils/telegram_initdata.py` | Pure Telegram Mini App `initData` validation (no I/O): `WebAppData`-keyed HMAC-SHA256 derivation, `hmac.compare_digest` check, freshness window (`auth_date` within `max_age_seconds`, ±60s clock-skew tolerance, no far-future) | 76 | + +## Key Symbols + +| Name | Kind | File:Line | Responsibility | +|---|---|---|---| +| `BaseService` | class | `services/base.py:116` | Session-bound service base; holds `self.session` + `self.log` (structlog bound to `service_name`) | +| `SingletonService` | class | `services/base.py:154` | Stateless singleton base with logger only | +| `SingletonHolder[T]` | generic class | `services/base.py:184` | PEP-695 generic singleton holder (get/set/clear/is_initialized) | +| `ServiceError` | class | `services/base.py:25` | Base service exception with `message` + `details` | +| `NotFoundError` | class | `services/base.py:34` | 404-bound; carries `resource_type` / `resource_id` | +| `ConflictError` | class | `services/base.py:64` | 409-bound; carries `resource_type` | +| `ValidationError` | class | `services/base.py:51` | 400-bound; carries `field` | +| `RateLimitError` | class | `services/exceptions.py:31` | Raised after all 429 retries exhausted; carries `provider` + `retry_after` | +| `parse_retry_after_header` | func | `services/exceptions.py:65` | Numeric `Retry-After` → float seconds (HTTP-date unsupported) | +| `MAX_RATE_LIMIT_RETRIES` | const | `services/exceptions.py:20` | `= 5` | +| `AgentService` | class | `services/agent.py:19` | Read-only agent queries (list/get_by_uuid/get_by_slug/get_by_uuid_or_slug_or_raise) | +| `check_database` | func | `services/health.py:14` | Opens a DB context, `SELECT 1`, returns `(msg, ok)` | +| `check_redis` | func | `services/health.py:24` | `redis.from_url(settings.redis_url).ping()` then close | +| `SettingsService` | class | `services/settings.py:83` | KV CRUD on `system_settings` (get/get_int/get_bool/set/all); `set` validates + flushes, caller commits | +| `FEATURE_FLAGS` | tuple | `services/settings.py:46` | Panel-tunable flag registry `(key, label)`; maps to `Settings` bool attrs of same name | +| `validate_setting` | func | `services/settings.py:75` | Reject unknown keys + run per-key validator | +| `apply_persisted_feature_flags` | func | `services/settings.py:146` | Startup overlay: stored flag value → `setattr(settings, key, bool)`; returns overridden keys | +| `feature_flag_effective_values` | func | `services/settings.py:131` | Stored override else env default; backs Settings panel card | +| `SettingValidationError` | class | `services/settings.py:22` | Unknown/invalid setting on write | +| `resolve_target_python` | func | `services/toolchain.py:103` | Returns `ResolvedPython(version, source)` or None; honors `.python-version` only if it satisfies `requires-python` | +| `satisfies` | func | `services/toolchain.py:48` | PEP 440 membership test (empty specifier = any) | +| `ResolvedPython` | dataclass | `services/toolchain.py:40` | Frozen `(version, source)` result | +| `ProviderService` | class | `services/provider.py:61` | CRUD for `provider_configs`; tri-state token update; 409 on delete-with-assignments | +| `ProviderCreate` / `ProviderUpdate` | dataclasses | `services/provider.py:32,43` | Service-boundary shapes; `ProviderUpdate.auth_token` tri-state + `clear_auth_token` | +| `get_decrypted_token` | method | `services/provider.py:211` | Decrypt provider token or None; raises `EncryptionError` on bad key | +| `ModelRoutingService` | class | `services/llm.py:119` | Per-agent route resolution + assignment CRUD + mode apply | +| `AgentRoute` | dataclass | `services/llm.py:95` | Frozen resolved route `(provider_id, type, base_url, auth_token, model_name)`; None base_url/token = Anthropic default | +| `resolve_for_agent` | method | `services/llm.py:124` | Precedence ladder agent>role>global; never raises — downgrades to legacy Anthropic path | +| `probe_ollama_tags` | func | `services/llm.py:63` | `{base_url}/api/tags` probe; never raises, returns `([], error)` | +| `upsert_assignment` | method | `services/llm.py:241` | Insert-or-update by `(scope, scope_value)`; routes non-catalog names to LOCAL; auto-enables LOCAL provider | +| `apply_mode` | method | `services/llm.py:418` | Wipe role/global rows (AGENT_SLUG pins preserved) + set GLOBAL for anthropic/grok/ollama/self_hosted; per-agent map for mix | +| `derive_mode` | method | `services/llm.py:314` | Settings UI label from current assignments | +| `set_ollama_api_key` / `set_grok_api_key` | methods | `services/llm.py:340,360` | Encrypt+enable / clear+disable on the seeded provider row | +| `ProactiveKnowledgeService` | class | `services/proactive.py:90` | Builds `ContextPackage` from multiple RAG indexes on claim/session | +| `ContextPackage` | dataclass | `services/proactive.py:27` | Aggregates similar_tasks/learnings/code_patterns/standards/decisions/known_issues + summary | +| `on_task_claimed` / `on_session_started` | methods | `services/proactive.py:116,201` | Best-effort multi-index search; each source wrapped in try/except | +| `get_proactive_service` | func | `services/proactive.py:534` | Singleton holder; lazy-inits with `OptimalService` | +| `TranscriptionService` | class | `services/transcription.py:28` | Per-(agent,session) `StreamBuffer` map; periodic flush task; callback registration | +| `process_chunk` | method | `services/transcription.py:120` | Append chunk, return buffer if ready-for-extraction else None | +| `_periodic_flush` | method | `services/transcription.py:225` | Background loop: sleep `flush_interval_seconds`, yield ready buffers to callbacks | +| `StreamEventBus` | class | `events/stream_bus.py:35` | Redis Streams bus: `xadd` trim, `xreadgroup` block=5000, ACK-on-success, `xclaim` recovery, periodic `_reclaim_loop`, dead-letter for undecodable messages | +| `DEAD_LETTER_STREAM` | class attr | `events/stream_bus.py:51` | `"roboco:stream:dead-letter"` — undecodable messages are parked here before ACK for operator inspection | +| `publish` / `publish_task_event` | methods | `events/stream_bus.py:152,191` | `xadd` to category-grouped stream, returns message id | +| `recover_pending` | method | `events/stream_bus.py:534` | `xpending_range` + `xclaim` idle≥60s messages; called at startup and periodically by `_reclaim_loop` | +| `_reclaim_loop` | method | `events/stream_bus.py:253` | Background task spawned alongside `_listen_loop`; re-runs `recover_pending` every 60s so runtime handler failures are retried without waiting for a restart | +| `_handle_message` | method | `events/stream_bus.py:434` | Decode (poison-pill: dead-letter+ACK on `Event.from_json` failure) → dispatch → ACK iff all handlers succeeded (else stays pending for reclaim) | +| `_dead_letter` | method | `events/stream_bus.py:403` | Best-effort write to `DEAD_LETTER_STREAM`; never blocks ACK on publish failure | +| `get_stream_event_bus` / `init_stream_event_bus` | funcs | `events/stream_bus.py:577,584` | Singleton + connect + optional startup pending recovery | +| `handle_task_status_change` | func | `events/handlers.py:141` | Routes task.* events to PM/QA/Documenter/developer notifications | +| `handle_auditor_spawn` | func | `events/handlers.py:332` | One-shot auditor spawn on blocked/cancelled/awaiting_ceo_approval; failures swallowed | +| `register_default_handlers` | func | `events/handlers.py:372` | Subscribes all default handlers on the bus | +| `set_event_context` / `get_event_context` | funcs | `events/handlers.py:26,37` | DI of `notification_service` + `orchestrator` into module-global `_context` | +| `DEFAULT_AGENTS` | const | `seeds/initial_data.py:156` | Composed from `foundation.AGENTS` + presentation names; system sentinel appended literally (team=None) | +| `AGENT_UUIDS` / `CEO_AGENT_ID` | consts | `seeds/initial_data.py:83,173` | String-keyed UUID map for legacy consumers | +| `encrypt_token` / `decrypt_token` | funcs | `utils/crypto.py:40,67` | Fernet symmetric encrypt/decrypt; `EncryptionError` on empty/bad key | +| `is_encryption_configured` | func | `utils/crypto.py:102` | True iff `settings.encryption_key` yields a valid Fernet | +| `InvalidIdentifierError` | class | `utils/converters.py:11` | `ValueError` subclass raised by `require_uuid` on None or unparseable input; typed so callers can distinguish a bad identifier instead of broad-catching (#25) | +| `require_uuid` | func | `utils/converters.py:21` | Coerce to `UUID`, raise `InvalidIdentifierError` (a `ValueError` subclass) on None or bad input | +| `repo_key` | func | `utils/converters.py:47` | Normalize a git URL to a case/`.git`-suffix/trailing-slash insensitive key for ci_watch/dep_update dedupe (#1267) | +| `to_python_uuid` / `to_python_uuid_list` | funcs | `utils/converters.py:61,81` | None-safe SQLAlchemy UUID coercion | + +## Data Flow + +**Spawn routing.** The orchestrator (`runtime/orchestrator.py:3294`) opens a DB session, calls `get_model_routing_service(db).resolve_for_agent(agent_slug)`. `ModelRoutingService` walks `model_assignments` (AGENT_SLUG → ROLE → GLOBAL), joins the `provider_configs` row, decrypts any token via `ProviderService.get_decrypted_token` (Fernet from `utils/crypto`), probes LOCAL providers via `probe_ollama_tags`, and returns an `AgentRoute`. On any failure (decrypt, unreachable server, missing row) it falls back to `_legacy_route` (role → `MODEL_MAP` short name, Anthropic, mounted `~/.claude`). The orchestrator injects `ANTHROPIC_*`/base_url/auth env into the container only when the route is non-Anthropic. + +**Settings/flags.** At FastAPI lifespan (`api/app.py:117`), `apply_persisted_feature_flags(db)` reads each `FEATURE_FLAGS` key from `system_settings` and `setattr`s the live `roboco.config.settings` singleton so the rest of the app reads panel choices; an unset key keeps the env default. The Settings panel reads effective values via `feature_flag_effective_values` and writes via `SettingsService.set` (validates → upsert → flush; route commits). + +**Events.** `bootstrap.py:92` calls `init_event_bus()` → `init_stream_event_bus` → `connect()` + `recover_pending()` (reclaims idle ≥60s messages from crashed consumers via `xclaim`). `register_default_handlers` subscribes task/session/handoff/QA/blocker/question/auditor handlers. Publishers call `bus.publish(Event)` → `xadd` to `roboco:stream:{category}` (trimmed to 10000). `start_listening` spawns both `_listen_loop` and `_reclaim_loop`; the reclaim loop re-runs `recover_pending` every 60s so a runtime handler failure is retried without waiting for a restart. `_listen_loop` blocks on `xreadgroup` (count=10, block=5000); `_handle_message` first tries `Event.from_json` in its own try/except — an undecodable payload (unknown `EventType`, bad UUID/timestamp, malformed JSON) is dead-lettered to `DEAD_LETTER_STREAM` then ACKed so a poison pill never wedges the stream. Successfully decoded events are dispatched via `asyncio.gather` with a per-(event.id, handler) SET-NX idempotency guard (`_run_handler_guarded`), and the message is ACKed only if every handler succeeded — failed handlers leave the message pending for the reclaim loop. `_run_handler_guarded` catches `BaseException` (including `asyncio.CancelledError`) so a mid-flight cancellation clears the idempotency marker and allows replay. Handlers use the injected `_context` (notification_service + orchestrator). + +**Proactive injection.** `TaskService.claim_task` (`services/task.py:2548`) lazily calls `get_proactive_service()`, which singleton-inits with `OptimalService`. `on_task_claimed` runs five best-effort RAG searches (similar tasks, learnings, standards, decisions, known issues), each in its own try/except, builds a `ContextPackage` + summary. The `code_patterns` field is retained in `ContextPackage` for API/schema back-compat but is always empty — `_find_code_patterns` was removed. `api/routes/optimal.py:1255` exposes `get_context_for_task`/`get_context_for_session` which own DB lookups so routes don't query directly. + +**Transcription.** `api/app.py:124` constructs `TranscriptionService()` at lifespan; `ExtractionService` (`services/extraction.py`) composes it. `process_chunk` appends to the per-(agent,connection) `StreamBuffer`; when `is_ready_for_extraction` (min chars / idle threshold / max chars) it returns the buffer for extraction. A `_periodic_flush` background task wakes every `flush_interval_seconds` and pushes ready buffers to registered callbacks. + +**Seeds.** `db/seed.py` iterates `DEFAULT_AGENTS` to populate the DB on bootstrap; `foundation/_validate_lifecycle.py:237` validates against `DEFAULT_AGENTS`. It derives from `foundation.AGENTS` so adding an agent is a single foundation edit. + +**Toolchain.** `WorkspaceService` (`services/workspace.py:1308`) calls `resolve_target_python(workspace)` to pick the interpreter for `uv --python` when provisioning a target project's clone. + +**Health.** `api/routes/health.py:41` calls `check_database()` + `check_redis()` for `/health`. + +## Mermaid + +```mermaid +graph TD + subgraph "Model routing (spawn time)" + ORC["orchestrator.resolve_model_route"] --> MRS["ModelRoutingService.resolve_for_agent"] + MRS -->|"agent>role>global"| FIND["_find_assignment"] + FIND --> ROW["model_assignments row"] + ROW --> PROV["ProviderConfigTable"] + PROV -->|"token?"| PS["ProviderService.get_decrypted_token"] + PS --> CRYPTO["utils/crypto Fernet"] + PROV -->|"LOCAL?"| PROBE["probe_ollama_tags /api/tags"] + PROBE -->|unreachable| LEG["_legacy_route ANTHROPIC"] + PS -->|decrypt fail| LEG + MRS -->|ok| ROUTE["AgentRoute"] + LEG --> ROUTE + end +``` + +```mermaid +sequenceDiagram + participant B as bootstrap.py + participant S as StreamEventBus + participant R as Redis Streams + participant H as Handlers + participant N as NotificationService + B->>S: init_event_bus(consumer_name) + S->>R: connect + xpending/xclaim (recover_pending) + S->>S: register_default_handlers + loop listen + S->>R: xreadgroup(block=5000, count=10) + R-->>S: messages + S->>H: _dispatch_event (gather) + H->>N: send_*_notification + H-->>S: ok/exception + alt all succeeded + S->>R: xack + else any failed + S->>S: leave pending (reclaim later) + end + end +``` + +```mermaid +graph LR + subgraph "Settings overlay" + PANEL["Settings panel"] -->|POST| API["api/routes/settings"] + API --> SS["SettingsService.set"] + SS --> VALID["validate_setting"] + SS --> DB[("system_settings")] + LIFESPAN["api/app.py lifespan"] --> APPLY["apply_persisted_feature_flags"] + APPLY --> DB + APPLY -->|"setattr(key,bool)"| CFG["roboco.config.settings singleton"] + CFG --> CONSUMERS["all flag-gated code"] + end +``` + +## Logical Tree + +``` +support-services +├── services/ +│ ├── base.py # BaseService / SingletonService / SingletonHolder[T] / ServiceError* +│ ├── exceptions.py # RateLimitError + Retry-After parser +│ ├── agent.py # AgentService (read-only) +│ ├── health.py # check_database / check_redis +│ ├── settings.py # SettingsService + FEATURE_FLAGS + startup overlay +│ ├── toolchain.py # resolve_target_python (pure) +│ ├── provider.py # ProviderService (provider_configs CRUD + Fernet tokens) +│ ├── llm.py # ModelRoutingService (spawn routing + modes + Ollama probe) +│ ├── proactive.py # ProactiveKnowledgeService (RAG context packages) +│ └── transcription.py # TranscriptionService (stream buffering) +├── events/ +│ ├── __init__.py # public re-exports +│ ├── bus.py # EventBus = StreamEventBus (compat shim) +│ ├── handlers.py # workflow trigger handlers + register_default_handlers +│ └── stream_bus.py # Redis Streams durable bus (xadd/xreadgroup/xack/xclaim) +├── seeds/ +│ ├── __init__.py +│ └── initial_data.py # DEFAULT_AGENTS from foundation +└── utils/ + ├── __init__.py + ├── converters.py # UUID coercion + └── crypto.py # Fernet encrypt/decrypt +``` + +## Dependencies + +**Internal (downstream):** +- `roboco.config.settings` (encryption key, redis_url, feature-flag defaults) — `crypto`, `stream_bus`, `health`, `settings` +- `roboco.db.tables` (`AgentTable`, `ProviderConfigTable`, `ModelAssignmentTable`, `SystemSettingTable`, `TaskTable`) — `agent`, `provider`, `llm`, `settings`, `proactive` +- `roboco.db.base.get_db_context` — `health`, `proactive` +- `roboco.models.base` (`AgentRole`, `Team`, `ModelProvider`, `AssignmentScope`) — `agent`, `provider`, `llm` +- `roboco.models.events` (`Event`, `EventType`, `EventContext`, protocols) — `events/*` +- `roboco.models.llm_catalog` (`MODEL_CATALOG_BY_NAME`, `OLLAMA_DEFAULT_MODEL`) — `llm` +- `roboco.models.runtime` (`MODEL_MAP`, `ROLE_MODEL_MAP`) — `llm` +- `roboco.models.optimal` (`IndexType`, `QueryContext`, `SearchResult`) — `proactive` +- `roboco.models.message.RawStream`, `roboco.models.transcription.*` — `transcription` +- `roboco.agents_config.get_agent_role` — `llm` +- `roboco.foundation.identity` — `seeds` +- `roboco.services.optimal.get_optimal_service` — `proactive` (lazy) +- `roboco.logging.get_logger` — `crypto` + +**External:** +- `sqlalchemy` / `sqlalchemy.ext.asyncio` — ORM sessions +- `redis.asyncio` — streams bus, health probe +- `cryptography.fernet` — token encryption +- `httpx` — Ollama probe +- `structlog` — logging everywhere +- `packaging` (`Version`, `SpecifierSet`) — `toolchain` +- `tomllib` — `toolchain` pyproject parse + +## Entry Points + +| Symbol | Invoked from | Trigger | +|---|---|---| +| `apply_persisted_feature_flags` | `api/app.py:117` | FastAPI lifespan (after DB ready) | +| `get_settings_service` | `api/routes/settings.py`, `runtime/orchestrator.py:5910` | `GET/POST /api/settings`; orchestrator transcript-retention read | +| `get_model_routing_service().resolve_for_agent` | `runtime/orchestrator.py:3301` | Each agent spawn | +| `ModelRoutingService.*` assignment/mode ops | `api/routes/provider.py` | `GET/POST/DELETE /api/providers/*` | +| `ProviderService.*` | `api/routes/provider.py` | provider CRUD routes | +| `get_agent_service` | `api/routes/agents.py`, `services/task.py`, `services/pitch.py` | `/api/agents`; task/pitch main-pm lookup | +| `check_database` / `check_redis` | `api/routes/health.py:41` | `GET /health` | +| `resolve_target_python` | `services/workspace.py:1308` | Workspace provisioning (clone/claim) | +| `init_event_bus` / `register_default_handlers` / `set_event_context` | `bootstrap.py:92-96` | App bootstrap | +| `bus.publish` / `publish_task_event` | services, orchestrator, websocket_bridge | Status transitions, live events | +| `get_proactive_service` | `services/task.py:2548`, `api/routes/optimal.py:1257` | claim_task, `/api/optimal/context` | +| `TranscriptionService` | `api/app.py:124`, `services/extraction.py` | Lifespan construct; extraction pipeline | +| `DEFAULT_AGENTS` | `db/seed.py`, `foundation/_validate_lifecycle.py` | DB bootstrap + lifecycle validation | +| `encrypt_token` / `decrypt_token` | `provider`, `llm` (via ProviderService), ProjectService | Token persist / read | + +## Config Flags + +Panel-tunable flags defined in `services/settings.py:46` `FEATURE_FLAGS` (stored override → `roboco.config.settings.` at startup; env default when unset): + +| Key | Label | Env counterpart | +|---|---|---| +| `external_pr_enabled` | External-PR review | `ROBOCO_EXTERNAL_PR_ENABLED` | +| `internal_pr_enabled` | Internal-PR safety reviewer | `ROBOCO_INTERNAL_PR_ENABLED` | +| `research_enabled` | Web research (Board + PM) | `ROBOCO_RESEARCH_ENABLED` | +| `strategy_engine_enabled` | Strategy engine | `ROBOCO_STRATEGY_ENGINE_ENABLED` | +| `self_heal_enabled` | Self-healing (detect + notify) | `ROBOCO_SELF_HEAL_ENABLED` | +| `self_heal_originate_enabled` | Self-healing — open fix tasks | `ROBOCO_SELF_HEAL_ORIGINATE_ENABLED` | +| `provisioning_enabled` | Pitch auto-provisioning | `ROBOCO_PROVISIONING_ENABLED` | +| `toolchain_match_enabled` | Agent runtime toolchain matching | `ROBOCO_TOOLCHAIN_MATCH_ENABLED` | +| `conventions_enabled` | Architectural conventions standard | `ROBOCO_CONVENTIONS_ENABLED` | +| `rag_auto_update_enabled` | RAG auto-update | `ROBOCO_RAG_AUTO_UPDATE_ENABLED` | +| `transcript_prune_enabled` | Transcript pruning | `ROBOCO_TRANSCRIPT_PRUNE_ENABLED` | +| `gateway_health_enabled` | Gateway-health recovery | `ROBOCO_GATEWAY_HEALTH_ENABLED` | +| `ci_watch_enabled` | Multi-repo CI-watch | `ROBOCO_CI_WATCH_ENABLED` | +| `dep_update_enabled` | Dependency-update bot | `ROBOCO_DEP_UPDATE_ENABLED` | +| `release_manager_enabled` | Gated release manager | `ROBOCO_RELEASE_MANAGER_ENABLED` | +| `docs_sync_enabled` | Docs-divergence sync (release → docs-update task) | `ROBOCO_DOCS_SYNC_ENABLED` | +| `org_memory_enabled` | Organizational memory loop | `ROBOCO_ORG_MEMORY_ENABLED` | +| `sandbox_db_enabled` | Sandboxed per-agent test DB/Redis/Mongo (engine registry) | `ROBOCO_SANDBOX_DB_ENABLED` | +| `x_engine_enabled` | X (Twitter) engine | `ROBOCO_X_ENGINE_ENABLED` | +| `roadmap_engine_enabled` | Board roadmap engine | `ROBOCO_ROADMAP_ENGINE_ENABLED` | + +Cloud auth (`ROBOCO_CLOUD_AUTH_ENABLED`) and DB network isolation (`ROBOCO_DB_NETWORK_ISOLATED`) are deliberately **not** in `FEATURE_FLAGS` — both are compose/env-coupled (cookie/TLS posture and the `networks:` topology respectively) and unsafe for a runtime toggle to flip mid-session; they stay pure env vars, not panel-tunable settings. `ROBOCO_TELEGRAM_MINIAPP_ENABLED` (Telegram Mini App sign-in) joins them for the same reason — security/TLS-coupled, and `Settings` fails loud at startup if it's armed without `cloud_auth_enabled`; its sibling `telegram_initdata_max_age_seconds` (default 600) is likewise env-only. + +Other settings read here: `transcript_retention_days` (int, ≥1; read by orchestrator at `runtime/orchestrator.py:5910`). Non-flag config consumed: `settings.redis_url` (`health`, `stream_bus`), `settings.encryption_key` (`crypto`). + +## Gotchas + +- **`resolve_for_agent` never raises for a normal agent** (`llm.py:124`) — decrypt failures, unreachable LOCAL servers, and missing agents all downgrade to the legacy Anthropic path. A misconfigured provider therefore silently spawns against Anthropic instead of erroring; check orchestrator logs for "falling back to legacy path" / "Self-hosted server unreachable". +- **Mix-mode self-hosted models auto-enable the LOCAL provider** (`llm.py:285`) — `upsert_assignment` flips the LOCAL provider row to `enabled=True` whenever an assignment resolves to LOCAL. A prior observation (47392) flagged that this enabling only happens on upsert, not on a bare GLOBAL assignment path — verify the LOCAL row is enabled before relying on self-hosted routing. +- **`probe_ollama_tags` leaks raw exception text** into the returned error string (`llm.py:92`, observation 47334) — the generic `except Exception` branch puts `str(exc)` in the user-facing message. Minor info-disclosure surface. +- **`SingletonHolder[T]` uses PEP 695 generic syntax** (`base.py:184`) — requires Python 3.13+. CLAUDE.md pins 3.13 so this is fine, but it will syntax-error on 3.12 tooling/linters. +- **`apply_persisted_feature_flags` mutates the live `settings` singleton via `setattr`** (`settings.py:163`) — a toggle takes effect only on next restart (documented), and the in-process `settings` object is shared; concurrent reads during the overlay are not synchronized but the overlay runs once at lifespan before serving. +- **`StreamEventBus` ACKs only when every handler succeeded** (`stream_bus.py`) — a single failing handler leaves the message pending; `_reclaim_loop` re-runs `recover_pending` every 60s so the retry fires at runtime without waiting for a restart. Undecodable messages (poison pills) are now dead-lettered then ACKed immediately and never left pending. A per-(event.id, handler) SET-NX guard (`_run_handler_guarded`) makes replay safe for already-succeeded handlers, but the guard is best-effort (fail-open without Redis). Handlers must still be idempotent. +- **`recover_pending` runs at startup and periodically** (`bootstrap.py` via `init_event_bus`; `_reclaim_loop` every 60s at runtime) — on restart or after a runtime handler failure, pending messages are reprocessed under the idempotency guard. +- **`TranscriptionService._periodic_flush` callbacks are sync `Callable`** (`transcription.py:57`) invoked inside an async loop without `await` — a blocking callback stalls the flush task. The `register_callback` signature is `Callable[[StreamBuffer], None]`, not a coroutine. +- **`TranscriptionService.get_ready_buffers` is declared `AsyncIterator` but `yield`s inside an `async for` over a dict** (`transcription.py:208`) — it works but never removes buffers; callers must `flush_buffer` after extraction or buffers accumulate forever (only `_flush_all` on shutdown clears them). +- **`ProactiveKnowledgeService._find_code_patterns` was removed** (`proactive.py`) — the dead method, its call in `build_context_package`, the `"Found N code patterns"` summary line, and the count were all deleted. `ContextPackage.code_patterns` is retained as an always-empty field for API/schema back-compat (serialized in `to_dict`; docstring marks it deprecated). Do not rely on it being populated. +- **`ProactiveKnowledgeService` singleton lazy-inits `OptimalService`** (`proactive.py:537`) — the first `get_proactive_service()` call triggers `get_optimal_service()` which may do embedding-model setup; call it early or expect latency on first claim. +- **Seeds `system` sentinel is appended literally with `team=None`** (`seeds/initial_data.py:144`) — the postgres `team` enum has no `system` value, so seeding it through the normal path would fail. Don't add `system` to `_AGENT_PRESENTATION` expecting it to flow through `_build_default_agents`. +- **`require_uuid` raises `InvalidIdentifierError` (a `ValueError` subclass), not `NotFoundError`** (`utils/converters.py:38`) — callers in `llm.py` use it on `provider.id` which is a PK and never None in practice, but a None would surface as a 500-class error not a 404. The typed subclass allows callers to handle a bad identifier distinctly from other exceptions; existing `except ValueError` handlers are unaffected. +- **`encrypt_token` rejects empty strings** (`crypto.py:53`) — the provider/project token convention is `""` = clear to NULL, handled at the service layer (`_apply_auth_token_change` checks `clear_auth_token`/`not data.auth_token` before calling encrypt); never pass `""` directly to `encrypt_token`. +- **`check_redis` opens a fresh client each call and closes it** (`health.py:27`) — fine for a health probe, but don't reuse the pattern for hot paths. + +## Drift from CLAUDE.md + +- **`services/settings.py:46` `FEATURE_FLAGS`** includes `rag_auto_update_enabled` ("RAG auto-update") and `transcript_prune_enabled` ("Transcript pruning") which are **NOT** listed in the CLAUDE.md "Feature Flags / company-in-a-box" enumeration (that list names web research, strategy engine, pitch provisioning, external/internal PR review, toolchain match, conventions, gateway-health, multi-repo CI-watch, dependency-update bot, gated release manager, organizational memory loop, and the self-heal flags). Two flags exist in code with no CLAUDE.md mention. +- **`services/proactive.py`** (`ProactiveKnowledgeService`) — an entire service that injects RAG context on task-claim / session-start — is **not mentioned anywhere in CLAUDE.md**. The org-memory loop (`ROBOCO_ORG_MEMORY_ENABLED`, `_briefing_for` institutional_memory) is documented, but that is a separate, newer system; `proactive.py` predates and overlaps with it yet remains present and wired (`task.py:2548`, `optimal.py:1257`). +- **`services/transcription.py`** (`TranscriptionService`) — the raw-LLM-stream buffering layer between WebSocket and the extraction pipeline — is **not mentioned in CLAUDE.md**. CLAUDE.md documents WebSocket streams and the extraction target (ExtractedMessages) but not the transcription buffer service. +- **`services/llm.py:316` `derive_mode`** returns `"grok"` for a single GLOBAL GROK assignment; CLAUDE.md's provider table lists `GROK` as a `ModelProvider` and documents the Grok CLI provider, and `apply_mode` supports `"grok"` — consistent. No drift here, noted for completeness. +- **`events/handlers.py:332` `handle_auditor_spawn`** spawns the auditor on `TASK_BLOCKED`/`TASK_CANCELLED`/`TASK_AWAITING_CEO_APPROVAL`; CLAUDE.md says "The Auditor sees all" and is a "silent observer" but does not describe the event-driven one-shot spawn on exceptional lifecycle events. Minor doc gap, not a contradiction. +- Otherwise the slice is consistent with CLAUDE.md: agent count (25 AI + 1 CEO) matches `DEFAULT_AGENTS`; `ModelProvider` enum usage matches; feature-flag overlay-on-restart contract matches; Redis-Streams event bus matches the "publish it to the bus" guidance; toolchain matching matches `ROBOCO_TOOLCHAIN_MATCH_ENABLED`. + +## Changes Since Baseline + +Baseline: `fd10cc862c2020b3f639cdb686d427b0198a2441`. Range `fd10cc86..HEAD` (3aff6e04) contains only 2 commits (`15effce0` "Chore: 141 Gaps fill-in (#283)", `3aff6e04` "Chore: Close gaps (#285)"), and **neither touches any file in this slice** (`git diff --stat fd10cc86..HEAD -- ` is empty; `git log --oneline` against the scope is empty). + +No logic-touching commits to list — IMPACT: none. + +> Post-snapshot updates (since 2026-06-29): five commits touched this slice after the baseline was cut. +> - `e4ed970f` [chore] stream-bus: poison-pill ACK + dead-letter (`DEAD_LETTER_STREAM`, `_dead_letter`), periodic `_reclaim_loop` spawned alongside `_listen_loop`, `_run_handler_guarded` catches `BaseException` for cancelled-handler marker cleanup (3 gaps). +> - `6b441e42` [chore] converters: `InvalidIdentifierError(ValueError)` introduced; `require_uuid` now raises it for both None and unparseable input; `repo_key` git-URL normalizer added; orchestrator reaper now logs the typed error instead of silently swallowing it. +> - `321e68d7` [sweep] proactive: `_find_code_patterns` method, its call, summary line, and count removed; `ContextPackage.code_patterns` field retained (always-empty, back-compat). +> - `536bbb64` Chore/all/logical-gaps-sweep (#286) — merge commit pulling the above into the branch. +> - `d83104e9` (2026-07-17, PR #546, "wave-1 quick wins") fix(llm): provider mode switches preserve per-agent model pins — `_apply_anthropic`/`_apply_grok`/`_apply_ollama`/`_apply_self_hosted` now delete only ROLE/GLOBAL `model_assignments` rows (`scope != AGENT_SLUG`) instead of wiping the whole table, so an AGENT_SLUG pin survives a mode switch; `OLLAMA_ROLE_DEFAULTS` removed from `llm_catalog.py` as dead code (it was never consulted by routing — see `models.md`). +> - `82642bea` (2026-07-18, PR #554, Telegram V3 Mini App) adds `roboco/utils/telegram_initdata.py` (new file, pure `validate_init_data`) — no other file in this slice's scope touched by the PR. + +## Regression Risks + +No files in this slice changed between `fd10cc86` and `HEAD`, so there are no *recent* regressions introduced by the diff. The risks below are **standing** landmines in the current code (not newly introduced), listed because they are the places a future change would plausibly break behavior: + +| Title | File:Line | Claim | Severity | +|---|---|---|---| +| Handler failure leaves Redis stream message pending → duplicate side effects on reclaim | `events/stream_bus.py` | ACK only when *all* handlers succeed; `_reclaim_loop` now re-runs `recover_pending` every 60s at runtime (not just on restart). Undecodable payloads are dead-lettered + ACKed immediately (poison-pill fix). Non-idempotent notification handlers can still double-fire on reclaim — `_run_handler_guarded` idempotency marker mitigates but requires Redis availability. | high | +| `resolve_for_agent` silently downgrades to Anthropic on any provider error | `services/llm.py:124,193,205` | Decrypt failure / unreachable LOCAL / missing assignment all return the legacy Anthropic route instead of raising — a misconfigured Grok/Ollama/self-hosted fleet spawns against Anthropic with only a log warning. | high | +| Mix-mode self-hosted assignment enables LOCAL only on upsert path | `services/llm.py:285` | `upsert_assignment` flips LOCAL `enabled=True`, but a pre-existing GLOBAL LOCAL assignment whose provider row was disabled will not be re-enabled until an upsert touches it — `resolve_for_agent` then skips it (`enabled` check at `llm.py:134`) and falls back to Anthropic. | medium | +| `probe_ollama_tags` leaks raw exception text into error string | `services/llm.py:86-92` | Generic `except Exception` puts `str(exc)` in the returned error, surfaced to the panel/UI — minor info disclosure. | low | +| `TranscriptionService` sync callbacks can stall the flush task | `services/transcription.py:57,233` | `register_callback` takes a sync `Callable` invoked without `await` inside the async `_periodic_flush`; a blocking callback blocks all buffer flushing. | medium | +| `get_ready_buffers` never removes buffers → unbounded growth | `services/transcription.py:208` | Iterates and yields ready buffers without clearing; only `_flush_all` (shutdown) clears. A long-running orchestrator with many sessions accumulates buffers unless callers `flush_buffer` after extraction. | medium | +| `apply_persisted_feature_flags` mutates shared `settings` singleton unsynchronized | `services/settings.py:163` | Runs once at lifespan (safe), but any future caller that re-invokes it mid-serve could race concurrent flag reads. | low | +| ~~`ProactiveKnowledgeService._find_code_patterns` vestigial~~ | `services/proactive.py` | **FIXED** (321e68d7): method removed; `ContextPackage.code_patterns` retained as always-empty for back-compat. | ~~low~~ | +| `require_uuid` raises untyped `ValueError` not `NotFoundError` | `utils/converters.py:38` | **FIXED** (6b441e42): now raises `InvalidIdentifierError(ValueError)` — typed, backward-compatible. A None/bad identifier still surfaces as a 500-class error (not 404), but callers can now handle it distinctly. | low | + +## Health + +This slice is a mature, mostly-stable support layer: the service-base/error hierarchy and crypto/UUID helpers are well-factored and widely reused; the Redis-Streams event bus is correctly durable (consumer groups, ACK-on-success, pending recovery) with the one real caveat that handler idempotency is the caller's job. Post-snapshot commits improved the bus (poison-pill dead-letter + periodic reclaim loop + `BaseException` marker cleanup), typed the UUID error surface (`InvalidIdentifierError`), and removed the vestigial `_find_code_patterns` call from `ProactiveKnowledgeService`. `TranscriptionService` (sync-callback + unbounded-buffer risks) remains the softest spot. Model routing's fail-safe-quietly design is intentional (a stalled spawn is worse than a wrong provider) but shifts diagnosis to logs. Overall integrity: solid, with `TranscriptionService` the one service worth either finishing or marking clearly as legacy. # RoboCo Map — `review-findings` slice ## Purpose @@ -6421,7 +7931,7 @@ The metrics & observability slice is the read-only measurement layer of RoboCo: | Path | Role | approx LOC | |---|---|---| -| `roboco/services/metrics.py` | `MetricsService` — velocity, blockers, team/agent metrics, health, cycle-time/bottleneck/rework/scorecard observability | 905 | +| `roboco/services/metrics.py` | `MetricsService` — velocity, blockers, team/agent metrics, health, cycle-time/bottleneck/rework/scorecard observability | 1521 | | `roboco/services/dashboard.py` | `DashboardService` — auditor flags/reports (in-memory singleton), CEO overview, audit queue, agent status, recent activity | 457 | | `roboco/services/cockpit.py` | `CockpitService` — read-only CEO "is the business winning?" summary (goals+delivery+spend+signals) | 97 | | `roboco/services/usage.py` | `UsageService` — token usage summary, time-series, by-agent/team/model, projection, cache efficiency, today summary, recent sessions | 478 | @@ -6451,6 +7961,11 @@ The metrics & observability slice is the read-only measurement layer of RoboCo: | `MetricsService.get_task_metrics` | method | metrics.py:~881 | Per-task `qa_fails`/`pr_fails`/`pm_rejects`/`ceo_rejects` (one aggregated audit_log query) + `findings_open`/`findings_total` (a second query against `ReviewFindingsRepository.list_for_task`, counted in Python from the fetched rows). | | `MetricsService._rework_cost` | method | metrics.py:720 | Sum of `estimated_cost_usd` over spawn sessions of reworked tasks | | `MetricsService.get_scorecard` | method | metrics.py:795 | Fused per-agent or per-cell scorecard (completed, cycle, rework, tokens, cost) | +| `MetricsService.get_member_scorecard` | method | metrics.py:1178 | Single-agent `MemberScorecard` (FPY, effort-throughput, utilization) — 3 DB queries, now delegates the assembly logic to `_assemble_member_scorecard` | +| `MetricsService._assemble_member_scorecard` | method | metrics.py:1128 | Shared builder extracted from the old `get_member_scorecard` body so the single-agent and batch paths derive identically (no duplicated logic) | +| `MetricsService.get_all_member_scorecards` | method | metrics.py:1301 | Batch replacement for N×`get_member_scorecard` calls — the whole non-CEO/non-SYSTEM roster (optionally team-filtered) in a **fixed 3 queries total**, not 3-per-agent: (1) agent list, (2) one `GROUP BY agent_slug` rollup via `_rollup_sums_by_agent`, (3) one `GROUP BY task_id` live-overlay query via `_live_inflight_overlay_by_agent`. Closes the "~20 agents = ~40 extra queries per panel Members-tab poll" N+1 the scorecards-tab previously incurred. | +| `MetricsService._rollup_sums_by_agent` | method | metrics.py:1200 | `SELECT agent_slug, SUM(...) ... GROUP BY agent_slug` over `MemberPerformanceDailyTable` for the whole agent-id list in one query, replacing a per-agent `_rollup_sums` call | +| `MetricsService._live_inflight_overlay_by_agent` | method | metrics.py:1232 | One `TaskTable` lookup for all non-terminal tasks assigned to the agent-id list, then one grouped `AgentSpawnSessionTable` query for their open sessions, aggregated back into a per-agent dict in Python | | `MetricsService._tokens_cost_for` | method | metrics.py:768 | Sum tokens+cost from spawn sessions for an agent slug or team | | `MetricsService._avg_cycle_hours` | method | metrics.py:869 | Avg started→completed hours for completed tasks | | `_as_hours` | func | metrics.py:47 | Coerce SQL epoch aggregate to rounded float (avoids Decimal→JSON string crash) | @@ -6586,7 +8101,7 @@ metrics-observability ## Entry Points - **HTTP routes** (`roboco/api/routes/usage.py`): `/api/usage/{summary,time-series,by-agent,by-team,by-model,projection,cache-efficiency,sessions}` → `get_usage_service`. -- **HTTP routes** (`roboco/api/routes/dashboard.py`): `/api/dashboard/auditor*`, `/api/dashboard/ceo*`, `/api/dashboard/kanban/*`, `/api/dashboard/agents/status`, `/api/dashboard/activity/recent`, `/api/dashboard/metrics/{velocity,blockers,team/{team},communication,health,agent/{id},cycle-time,bottlenecks,rework,scorecard/agent/{id},scorecard/team/{team}}` → `get_dashboard_service` + `get_metrics_service`. Auditor/CEO routes gated by `_require_auditor_or_ceo`. +- **HTTP routes** (`roboco/api/routes/dashboard.py`): `/api/dashboard/auditor*`, `/api/dashboard/ceo*`, `/api/dashboard/kanban/*`, `/api/dashboard/agents/status`, `/api/dashboard/activity/recent`, `/api/dashboard/metrics/{velocity,blockers,team/{team},communication,health,agent/{id},cycle-time,bottlenecks,rework,scorecard/agent/{id},scorecard/team/{team}}` → `get_dashboard_service` + `get_metrics_service`. Auditor/CEO routes gated by `_require_auditor_or_ceo`. `GET /api/dashboard/metrics/member/{agent_id}` (dashboard.py:617, `get_member_scorecard`) and `GET /api/dashboard/metrics/member/ceo` (dashboard.py:606, declared first — route-order matters, a literal path segment must win over the `{agent_id}` path param) back the panel Members tab's per-row card. `GET /api/dashboard/metrics/members` (dashboard.py:634, `get_all_member_scorecards`, optional `team`/`days` query params) is the new batch fetch replacing N per-row calls — see `MetricsService.get_all_member_scorecards` above. - **HTTP routes** (`roboco/api/routes/cockpit.py`): `/api/cockpit/{summary,signals}` → `get_cockpit_service`. - **Orchestrator loop tick**: `runtime/orchestrator.py` token sweep (~line 5270) → `calculate_cost` + `publish_usage_snapshot`; runs per dispatch tick on active agents. - **Self-heal / CI-watch loop ticks**: `services/self_heal_engine.py` and `services/ci_watch_engine.py` construct their telemetry source and call `fetch()` each cycle; armed by their respective ROBOCO_* flags. @@ -6638,6 +8153,8 @@ No flags live *inside* this slice's files, but the slice's behavior is gated/par > Post-snapshot updates (since 2026-06-29): commit **536bbb64** ("Chore/all/logical gaps sweep #286") touched `billing/pricing.py`, `billing/__init__.py`, `services/metrics.py`, and `services/usage.py`. Changes: (1) `CostResult` dataclass + `calculate_cost_result` function added to `pricing.py`; `calculate_cost` refactored to a thin wrapper; `__init__.py` now re-exports all three. (2) `_blocked_since_map` helper added to `MetricsService` — reads `task.blocked` audit row as authoritative "blocked since" (#67); `get_blocker_metrics` uses it with `updated_at` fallback. (3) `get_summary` docstring corrected — no longer falsely claims rollup reads (#66). (4) `metrics._blocked_since_map` extracted as a xenon complexity refactor (no behavior change beyond the audit-row fix). > > (uncommitted, branch `feature/findings-ledger`, 2026-07-11) Revision-findings ledger: `services/metrics.py` and `models/metrics.py` are touched for the first time since baseline — `_rework_by_agent`/`get_task_metrics` widen from 2 to 4 named rework events (`pm_rejects`/`ceo_rejects` join `qa_fails`/`pr_fails`) and `get_task_metrics` gains `findings_open`/`findings_total`; `AgentReworkRate`/`TaskMetrics` (models/metrics.py) gain the matching fields, all defaulted for back-compat. See `docs/map/review-findings.md`. +> +> **"panel-perf-p3-p4"** (2026-07-19): closes the "scorecards N+1" gap — the panel Members tab previously fired one `GET /dashboard/metrics/member/{id}` (3 DB queries) per agent on every poll (~20 agents ≈ 40 extra queries per poll). `MetricsService.get_all_member_scorecards` (metrics.py:1301) returns the whole roster's `MemberScorecard` list in a **fixed 3 queries total**: agent list, one `GROUP BY agent_slug` rollup (`_rollup_sums_by_agent`), one `GROUP BY task_id` live-overlay (`_live_inflight_overlay_by_agent`) — both new grouped-query helpers replacing what used to be N separate `_rollup_sums`/overlay calls. The single-agent `get_member_scorecard` path is behaviorally untouched, just refactored to share `_assemble_member_scorecard` with the new batch path so the FPY/effort-throughput/utilization derivation logic can't drift between the two. New route `GET /api/dashboard/metrics/members` (dashboard.py:634, optional `team`/`days`); panel's `scorecards-tab.tsx` now calls `useAllMemberScorecards()` once instead of `useMemberScorecard` per row (`panel/src/hooks/use-observability.ts` + `lib/api/observability.ts`, both new client-side pairings) — see `docs/map/panel.md`. ## Regression Risks @@ -6658,7 +8175,6 @@ Because the slice is unchanged since baseline, there are no *recent* changes wit ## Health The slice is internally coherent and well-documented at the method level; the observability reconstruction (cycle-time/bottleneck/rework) is correctly designed around the audit-log event-naming contract and `revision_count` chokepoint, and the provider-aware pricing is sound. The main integrity concerns are coupling, not correctness: cycle-time and rework-cost are tightly bound to upstream audit-log event naming and `agent_spawn_sessions.task_id` population, so any drift there silently degrades panels without an in-slice guard. The in-memory `DashboardService` store is the clearest remaining local hygiene debt (the `get_summary` docstring mismatch and blocked-hours heuristic were resolved in 536bbb64). Commit 536bbb64 also adds `CostResult.unpriced` attribution — the mitigation for the silent-$0 risk — though orchestrator callers haven't switched to `calculate_cost_result` yet. The standing landmines above (especially cycle-time SQL naming convention and rework-cost task_id join) warrant upstream-contract tests. - # Slice Map — conventions-service-validator ## Purpose @@ -6939,272 +8455,6 @@ No other logic-touching commits in the range. ## Health Integrity is solid and actively improving. The validator is pure, layered cleanly (scan → effective_map → runner → per-check-family), and obeys its fail-loud contract (exit 3 / `ValidatorCouldNotRun`) consistently; the service keeps the read path resilient (degraded → last-good uncached, missing → auto-derived, concurrent-cache → savepoint with UNIQUE-only swallow). The 15effce0 + 536bbb64 delta fixed five real defects (concurrent cache-poison F042; bypassable docs path-traversal on read/delete; IntegrityError over-swallow in `_cache_put`; degraded-row stickiness hiding in-place repairs; `stream`/`stream_scalars` thin-route false-negatives) and added a typo-language-scope warn signal and a repo-commit outcome surface on `doc_ref.commit_status`, with no guard dropped. Remaining risks are low-severity under-enforcement paths (dialect family lookup doesn't cover future `jsx`/`mts`, backfill write-side-effect on `get_map`, same-filename RAG dedup still collapses on content match, broad `except Exception` in `_commit_doc_to_repo` still catches) — all consistent with the "precision over recall" stance. The standard is default-off and gated. No blockers. - -## Purpose -The CEO-facing intake and chief-of-staff slice. PrompterService turns a confirmed live-intake structured draft (or a MegaTask batch of drafts) into real Task rows, routing ownership/team and sequencing collision-free waves. PrompterLiveRegistry is the in-process bridge that relays a live chat between a spawned prompter/secretary container and the panel (SSE stream + turn delivery + park/idle lifecycle). SecretaryService reads company state and executes or gates the CEO's directives (relay/announce/charter/pitch/task-control), recording every directive auditably. - -## Files - -| Path | Role | LOC | -|---|---|---| -| roboco/services/prompter.py | PrompterService: create tasks from confirmed intake drafts (single + MegaTask batch), route owning team, sequence drafts into waves; plus pure description/readiness helpers, the wave-1/2 prompter-memory history-digest builders, and compact task-search row rendering | 1313 | -| roboco/services/prompter_live.py | PrompterLiveRegistry: process-wide singleton bridging live intake/secretary chat between panel (SSE) and spawned container (HTTP turn), with open/close/park/idle-reap lifecycle | 234 | -| roboco/services/secretary.py | SecretaryService: read company state + submit/confirm/reject gated CEO directives (relay/announce/charter/pitch/task-control incl. wave-1 full-content `edit` + claim-aware reassignment), persisted in secretary_directives | 418 | - -## Key Symbols - -| Name | Kind | File:Line | Responsibility | -|---|---|---|---| -| ReadinessTag | dataclass | roboco/services/prompter.py:71 | Parsed contents of an assistant turn's trailing roboco-meta JSON block (covered, ready, scale) | -| BatchPlacement | dataclass | roboco/services/prompter.py:80 | Where a draft sits in a MegaTask batch (parent_task_id, batch_id, sequence, team_override) | -| PrompterService | class | roboco/services/prompter.py:97 | Create tasks from confirmed intake drafts; pure draft/description helpers + DB-backed create | -| PrompterService._session | property | roboco/services/prompter.py:108 | Return the AsyncSession or raise ServiceError if constructed without one | -| PrompterService._assignee_is_board | method | roboco/services/prompter.py:118 | True if agent_id is a board/advisory role (PO / HoM / Auditor) | -| PrompterService._validate_draft_target | staticmethod | roboco/services/prompter.py:125 | A draft targets exactly one of project/product/per-cell-map, or none for an umbrella | -| PrompterService._resolve_owning_team | method | roboco/services/prompter.py:163 | Route owning team: team_override wins; if no product: multi-cell map (≥2 cells) -> MAIN_PM else lead cell; if product: board assignee -> BOARD else MAIN_PM (product/board routing checked BEFORE multi-cell force) | -| PrompterService._validate_and_coerce_draft | method | roboco/services/prompter.py:196 | Validate title+AC, coerce list fields (acceptance_criteria/what_this_builds/notes/the_work[].items) to list[str] in place | -| PrompterService._resolve_draft_assignee | method | roboco/services/prompter.py:243 | Explicit confirm-button assignment wins; else fall back to draft.assigned_to UUID | -| PrompterService._coerce_pm_code_to_planning | method | roboco/services/prompter.py:256 | Coerce code->planning when owner is a coordination PM role; two layers: team-based (main_pm_cannot_own_code) then assignee-based (pm_cannot_own_code); issue-resolution carve-out never applies for new intake tasks | -| PrompterService.create_task_from_draft | method | roboco/services/prompter.py:293 | Operate on a _copy_draft copy (caller never mutated), compose description, validate target, coerce enums, route team, coerce PM+code->planning via _coerce_pm_code_to_planning, persist via TaskService.create | -| PrompterService.confirm_live_draft | method | roboco/services/prompter.py:368 | Confirm a live-intake single draft -> create at PENDING assigned to product-owner (board) or main-pm route; return task id | -| PrompterService._sequence_drafts | method | roboco/services/prompter.py:419 | Build DraftSurface list and run SequencingService.analyze into waves; SequencingError -> ValidationError 400 | -| PrompterService.preview_batch | method | roboco/services/prompter.py:459 | Compute MegaTask waves+warnings WITHOUT creating (panel pre-confirm preview) | -| PrompterService._validate_batch_scope | staticmethod | roboco/services/prompter.py:473 | Each draft targets scoped repos via cell map or top-level project_id; union across drafts spans >=2 distinct projects | -| PrompterService.confirm_live_batch | method | roboco/services/prompter.py:524 | Create MegaTask umbrella + N sequenced root-subtasks, wire dependency edges; return umbrella_id/root_ids/waves/warnings | -| PrompterService.update_live_draft | method | roboco/services/prompter.py:628 | Apply a board-informed re-draft to an existing task in place; route via approve_and_start or re-board (clear board_review_complete) | -| PrompterService._resolve_uuid_field | staticmethod | roboco/services/prompter.py:676 | Parse draft_data[key] as UUID; None if absent, ValidationError if malformed | -| PrompterService._lead_cell_team | staticmethod | roboco/services/prompter.py:690 | Owner of a single-cell task: first valid Team in the_work, else default | -| PrompterService._coerce_draft_enums | staticmethod | roboco/services/prompter.py:704 | Coerce team/task_type/nature/complexity to valid enums; default on invalid/missing so confirm never hard-fails | -| PrompterService._coerce_priority | staticmethod | roboco/services/prompter.py:734 | Coerce priority (word or number) to int 0-3, default 2 | -| parse_readiness | function | roboco/services/prompter.py:786 | Split assistant reply into (clean_text, ReadinessTag) from trailing roboco-meta JSON fence | -| _as_work_entry | function | roboco/services/prompter.py:818 | Normalize a the_work entry (bare string -> {team:str}) so .get works on all entries | -| _cell_teams | function | roboco/services/prompter.py:835 | Distinct cell team values present in the_work, in order | -| _draft_cell_map | function | roboco/services/prompter.py:846 | Per-cell (team, project_id) map from the_work entries; de-duped by team; the multi-cell MegaTask root-subtask seam | -| derive_scale | function | roboco/services/prompter.py:882 | 'multi' when >1 cell participates, else 'single' | -| _clean_list | function | roboco/services/prompter.py:947 | coerce_str_list wrapper: trimmed non-empty string items, extracting dict-wrapped text | -| _copy_draft | function | roboco/services/prompter.py:956 | Shallow copy of draft dict with the_work unit dicts also copied, so _validate_and_coerce_draft cannot mutate the caller's dict | -| _text | function | roboco/services/prompter.py:896 | Trimmed string from a possibly-missing scalar | -| _bullets | function | roboco/services/prompter.py:901 | Render a markdown bullet list | -| _cell_label | function | roboco/services/prompter.py:906 | Display label for a team value | -| _render_work_entry | function | roboco/services/prompter.py:911 | Render one cell's slice: bold heading + summary + deliverable bullets | -| _render_the_work | function | roboco/services/prompter.py:926 | Render The Work section, prepending a board-led lead line when multi-cell | -| _section | function | roboco/services/prompter.py:938 | Append a markdown section when its body is non-empty | -| format_board_briefing | function | roboco/services/prompter.py:944 | Render board review entries into a markdown briefing to seed a re-draft intake session | -| compose_redraft_message | function | roboco/services/prompter.py:969 | Seed message for a re-draft session: current draft + board feedback | -| compose_description | function | roboco/services/prompter.py:985 | Deterministically build the markdown description from structured fields; fall back to model description if too sparse (<20 chars) | -| _compose_umbrella_draft | function | roboco/services/prompter.py:1015 | Build the branchless umbrella draft from batch + wave plan; task_type=planning | -| get_prompter_service | function | roboco/services/prompter.py:1059 | Factory: construct PrompterService with optional db session | -| LiveIntakeSession | dataclass | roboco/services/prompter_live.py:39 | One live chat: session_id, agent_id, asyncio queue, closed flag, parked task_id, last_activity timestamp | -| PrompterLiveRegistry | class | roboco/services/prompter_live.py:58 | Tracks live intake/secretary sessions; bridges panel<->container via push/stream/deliver; lifecycle open/close/park | -| PrompterLiveRegistry.open | method | roboco/services/prompter_live.py:68 | Register a live session; idempotent (returns existing un-closed session instead of orphaning its SSE queue) | -| PrompterLiveRegistry.get | method | roboco/services/prompter_live.py:88 | Return the session or None | -| PrompterLiveRegistry.is_alive | method | roboco/services/prompter_live.py:91 | True when a live un-closed session exists (panel reload reconnect decision) | -| PrompterLiveRegistry.close | method | roboco/services/prompter_live.py:101 | End a session: pop, mark closed, push _CLOSE sentinel to unblock the SSE stream | -| PrompterLiveRegistry.close_by_agent | method | roboco/services/prompter_live.py:110 | Close every live session bound to agent_id (forced kill); optional final error event; returns closed ids | -| PrompterLiveRegistry.park | method | roboco/services/prompter_live.py:129 | Mark session parked awaiting board review of task_id (keeps it alive for in-context re-draft) | -| PrompterLiveRegistry.find_by_task | method | roboco/services/prompter_live.py:148 | Return the live un-closed session parked for task_id, if any | -| PrompterLiveRegistry.push | method | roboco/services/prompter_live.py:157 | Queue one agent event for SSE; bump last_activity; False if no/gone session | -| PrompterLiveRegistry.idle_session_ids | method | roboco/services/prompter_live.py:166 | Return (session_id, agent_id) idle past threshold; excludes closed and board-parked sessions | -| PrompterLiveRegistry.stream | method | roboco/services/prompter_live.py:185 | Async generator yielding queued events until _CLOSE sentinel | -| PrompterLiveRegistry.deliver | method | roboco/services/prompter_live.py:198 | POST the human's text to the container's /turn receiver; bump last_activity; debug-log transient failures | -| get_live_registry | function | roboco/services/prompter_live.py:230 | Process-wide singleton accessor (lazily instantiates PrompterLiveRegistry) | -| SecretaryService | class | roboco/services/secretary.py:54 | Read company state + execute/queue CEO directives; BaseService subclass bound to a session | -| SecretaryService.read_company_state | method | roboco/services/secretary.py:63 | Aggregate goals + task counts + proposed pitches + pending directives for the CEO dashboard | -| SecretaryService.read_task | method | roboco/services/secretary.py:79 | Read a single task's id/title/status/team/assignee/description or NotFoundError | -| SecretaryService.get_directive | method | roboco/services/secretary.py:96 | Fetch a directive row by id or None | -| SecretaryService.list_directives | method | roboco/services/secretary.py:104 | List directives ordered by requested_at desc, optional status filter | -| SecretaryService.submit_directive | method | roboco/services/secretary.py:115 | Validate payload; persist row; if gated -> notify CEO pending + return; else run immediately | -| SecretaryService.confirm_directive | method | roboco/services/secretary.py:134 | CEO confirms a pending directive: set decided_by, run it | -| SecretaryService.reject_directive | method | roboco/services/secretary.py:142 | CEO rejects a pending directive: REJECTED + decided_by/at + result reason | -| SecretaryService.to_dict | staticmethod | roboco/services/secretary.py:153 | Serialize a directive row to a dict for API response | -| SecretaryService._pending_or_raise | method | roboco/services/secretary.py:171 | Fetch directive; NotFoundError if missing, ConflictError if not PENDING | -| SecretaryService._validate_payload | staticmethod | roboco/services/secretary.py:182 | Require the per-kind payload keys from _REQUIRED_PAYLOAD else ValidationError | -| SecretaryService._run | method | roboco/services/secretary.py:188 | Execute a directive: set result, EXECUTED on success, FAILED+error message on caught domain errors | -| SecretaryService._execute | method | roboco/services/secretary.py:246 | Dispatch by kind: relay/announce deliver via notification, update_charter upsert, approve_pitch approve+provision, else _control_task | -| SecretaryService._EDITABLE_TASK_FIELDS | ClassVar[frozenset] | roboco/services/secretary.py:278 | Wave-1: the full content-field allowlist the Secretary may edit on CEO confirmation (title/description/acceptance_criteria/priority/team/estimated_complexity/nature/assigned_to) — status is deliberately excluded (its own audited start/cancel/override path); git fields (branch/PR) are never editable | -| SecretaryService._control_task | method | roboco/services/secretary.py:302 | Task control action dispatch: **edit** (wave-1, full content surface via _edit_task), start (approve_and_start), cancel, or override status with CEO as actor | -| SecretaryService._edit_task | method | roboco/services/secretary.py:325 | Wave-1: apply the Secretary's edit — allowlisted content fields go through TaskService.update after enum coercion (team/estimated_complexity/nature); assigned_to is popped out and routed through claim-aware reassignment (_reassign_task) instead of a plain field set | -| SecretaryService._reassign_task | method | roboco/services/secretary.py:362 | Route an edit's reassignment through claim-aware paths: reassign_active_claim (reseeds heartbeat) when the task is claimed/in_progress, else the general reassign (review-state handoffs, or explicit unassign) — never a naive setattr on assigned_to | -| SecretaryService._resolve_assignee | method | roboco/services/secretary.py:384 | Resolve an edit's assigned_to to a UUID: accepts None (unassign), a UUID string, or an agent slug (same convention as the CEO chat's REST PATCH path) | -| SecretaryService._notify_ceo_pending | method | roboco/services/secretary.py:403 | Send an ack notification to the CEO that a gated directive awaits confirmation | -| get_secretary_service | function | roboco/services/secretary.py:416 | Factory: construct SecretaryService bound to a session | -| build_history_digest | function | roboco/services/prompter.py:1221 | Wave-1/2 prompter memory: render a chronological digest of recent tasks (top `limit`, reversed to oldest-first for a timeline read) into markdown bullet lines; empty input -> "" | -| project_history_digest | function | roboco/services/prompter.py:1236 | One project's rendered history digest via `TaskService.list_recent_for_project`; None if the project has no tasks | -| history_digest_layer | function | roboco/services/prompter.py:1253 | Ambient task-history-digest block for the in-scope project(s), one sub-block per project (headed by slug when >1 — the MegaTask case); None when no in-scope project has any tasks (no empty-header noise) | -| compact_task_rows | function | roboco/services/prompter.py:1286 | Render TaskTable rows into the compact id/title/status/team/priority dicts returned by the intake `search_past_tasks` HTTP route | -| TaskService.list_recent_for_project | method | roboco/services/task.py:6361 | Recent tasks for a project ordered by coalesce(completed_at, updated_at, created_at) desc — backs the prompter's per-project history digest so a just-touched task surfaces ahead of an old completed one | -| TaskService.search_tasks | method | roboco/services/task.py:6384 | Case-insensitive ILIKE search over title/description + id-prefix match; backs the panel's task search bar (GET /tasks/summary?q=), the Secretary's task-by-name lookup (GET /secretary/tasks?q=), and the intake `search_past_tasks` tool | -| search_past_tasks (route) | route | roboco/api/routes/prompter_live.py:376 | GET /live/{session}/search-tasks: session-aliveness-gated (mirrors /events' trust boundary — the intake container has no agent identity) bounded compact search calling TaskService.search_tasks + compact_task_rows | -| query_past_tasks / format_search_results | function | roboco/mcp/intake_server.py:108,145 | Shared HTTP-call + bounding + rendering logic for `search_past_tasks`, module-level so both the grok MCP tool and the Claude SDK in-process tool call the exact same implementation | -| search_past_tasks (grok MCP tool) | mcp tool | roboco/mcp/intake_server.py:161 | Grok-CLI intake's "have we done something like this before?" tool; reads ROBOCO_PROMPTER_SESSION_ID, delegates to query_past_tasks + format_search_results | -| _search_past_tasks (Claude SDK in-process tool) | tool | roboco/agent_sdk/intake_driver.py:512 | Claude SDK driver's in-process parity tool for the same feature — imports query_past_tasks/format_search_results from intake_server.py directly (one implementation, both runtimes) | - -## Data Flow -Intake: the orchestrator spawns a prompter container and calls PrompterLiveRegistry.open(session_id, INTAKE_AGENT_ID); the panel SSE endpoint calls stream() and the message endpoint calls deliver() -> POST http://roboco-agent-{agent_id}:{SDK_PORT}/turn. The container driver POSTs normalized StreamChunks to the relay push() endpoint. When the agent emits a roboco-meta fence, parse_readiness extracts ReadinessTag (covered/ready/scale) used by the orchestrator to decide proposal readiness. The CEO confirms via panel: confirm_live_draft (single) or confirm_live_batch (MegaTask) -> PrompterService.create_task_from_draft -> TaskService.create (DB). For a batch, _sequence_drafts (pure SequencingService.analyze) computes waves/edges, the umbrella is created branchless via _compose_umbrella_draft, N root-subtasks are created with BatchPlacement(parent=umbrella, batch_id, sequence=wave_index), then TaskService.add_dependency wires each edge (b depends on a). preview_batch returns the same waves without creating. update_live_draft applies board feedback to an existing task (update + approve_and_start or re-board). On board review, registry.park(session_id, task_id) keeps the chat alive; find_by_task recovers it for the re-draft injection. Idle sweep: orchestrator calls idle_session_ids(threshold) and close()s abandoned chats; close_by_agent fires on forced kill. Secretary: routes /api/secretary/* call read_company_state/read_task/submit_directive/confirm_directive/reject_directive -> SecretaryService -> TaskService/CompanyGoalsService/PitchService/NotificationService (relay/announce deliver via notification to the target agent(s)); gated kinds (UPDATE_CHARTER/CONTROL_TASK/APPROVE_PITCH/ANNOUNCE) persist PENDING + notify CEO, then confirm_directive runs _execute with the CEO as actor; RELAY_MESSAGE runs immediately. A CONTROL_TASK directive's payload["action"] fans out inside _control_task: "edit" (wave-1) is the new full-content path — _edit_task applies the allowlisted fields via TaskService.update after enum coercion, and pops assigned_to for claim-aware reassignment (_reassign_task) rather than a plain field set; "start"/"cancel"/"override" are the pre-existing status-only actions. The CEO refers to tasks by NAME in the Secretary chat, so GET /api/secretary/tasks?q= (TaskService.search_tasks) resolves a name to a concrete id before a directive targets it. Prompter memory (wave 1/2): at intake spawn, the orchestrator's `_resolve_history_digest_ambient` calls `history_digest_layer`, which fans out `project_history_digest` per in-scope project — each pulling `TaskService.list_recent_for_project` and rendering it via `build_history_digest` — into one ambient "Recent tasks" block injected into the spawn prompt; mid-conversation, the intake agent's `search_past_tasks` tool (routed through `query_past_tasks`/`format_search_results` in roboco/mcp/intake_server.py, shared byte-for-byte with the Claude SDK's in-process `_search_past_tasks` tool in roboco/agent_sdk/intake_driver.py) hits GET /live/{session}/search-tasks -> TaskService.search_tasks -> compact_task_rows. - -## Mermaid -```mermaid -sequenceDiagram - participant CEO - participant Panel - participant Orchestrator - participant Reg as PrompterLiveRegistry - participant Container as prompter container - participant PS as PrompterService - participant TS as TaskService - CEO->>Orchestrator: start intake chat - Orchestrator->>Reg: open(session_id, INTAKE_AGENT_ID) - Panel->>Reg: stream(session_id) (SSE) - CEO->>Panel: type message - Panel->>Reg: deliver(session_id, text) - Reg->>Container: POST /turn - Container->>Reg: push(session_id, StreamChunk) - Reg->>Panel: yield event - CEO->>Panel: confirm draft (board/main_pm) - Panel->>PS: confirm_live_draft(draft, agent_id, route) - PS->>PS: create_task_from_draft - PS->>TS: create(TaskCreateRequest, confirmed_by_human=True) - TS-->>Panel: task_id - alt MegaTask - Panel->>PS: confirm_live_batch(title, drafts, project_ids, route) - PS->>PS: _sequence_drafts -> waves + edges - PS->>TS: create umbrella (branchless) - loop each draft - PS->>TS: create root-subtask (BatchPlacement) - end - loop each edge (a,b) - PS->>TS: add_dependency(b, a) - end - end - Orchestrator->>Reg: park(session_id, task_id) (board review) - Orchestrator->>Reg: idle_session_ids(threshold) -> close() abandoned -``` - -```mermaid -stateDiagram-v2 - direction LR - [*] --> Pending: submit_directive (gated) - Pending --> Executed: confirm_directive (_run ok) - Pending --> Rejected: reject_directive - Pending --> Failed: _run raised domain error - [*] --> Executed: submit_directive (RELAY_MESSAGE, direct) - Executed --> [*] - Rejected --> [*] - Failed --> [*] -``` - -## Logical Tree -``` -intake-secretary - PrompterService (roboco/services/prompter.py) - Draft validation & coercion - _validate_and_coerce_draft - _validate_draft_target (project/product/cell-map/umbrella) - _coerce_draft_enums (team/task_type/nature/complexity) - _coerce_priority - _resolve_uuid_field - _resolve_draft_assignee - Team routing - _resolve_owning_team - _assignee_is_board - _lead_cell_team - Task creation - create_task_from_draft (single + placement) - confirm_live_draft (board / main_pm route) - update_live_draft (re-draft in place) - MegaTask batch - _sequence_drafts -> SequencingService.analyze - preview_batch (no-create preview) - _validate_batch_scope (>=2 distinct projects, in-scope) - confirm_live_batch (umbrella + N root-subtasks + edges) - _compose_umbrella_draft - Pure helpers - parse_readiness, compose_description, format_board_briefing, compose_redraft_message - _as_work_entry, _cell_teams, _draft_cell_map, derive_scale, _clean_list, _text, _bullets, _cell_label, _render_work_entry, _render_the_work, _section - Prompter memory (wave 1/2): build_history_digest, project_history_digest, history_digest_layer, compact_task_rows - Dataclasses: ReadinessTag, BatchPlacement - PrompterLiveRegistry (roboco/services/prompter_live.py) - LiveIntakeSession dataclass (queue, closed, task_id, last_activity) - Lifecycle: open, get, is_alive, close, close_by_agent, park, find_by_task - Agent->panel: push, stream, idle_session_ids - Panel->agent: deliver (POST /turn) - Singleton: _RegistryHolder, get_live_registry - SecretaryService (roboco/services/secretary.py) - Reads: read_company_state, read_task - Directives: get_directive, list_directives, submit_directive, confirm_directive, reject_directive, to_dict - Internals: _pending_or_raise, _validate_payload, _run, _execute, _control_task, _notify_ceo_pending - Task edit (wave-1): _EDITABLE_TASK_FIELDS, _edit_task, _reassign_task (claim-aware), _resolve_assignee (uuid-or-slug) -``` - -## Dependencies -- Internal: roboco.services.task.get_task_service / TaskService, roboco.services.sequencing.SequencingService, roboco.services.company_goals.get_company_goals_service, roboco.services.pitch.get_pitch_service, roboco.services.notification.NotificationService, roboco.services.base.BaseService/NotFoundError/ConflictError/ValidationError/ServiceError, roboco.db.tables.AgentTable/TaskTable/SecretaryDirectiveTable, roboco.foundation.identity.CELL_TEAMS/AGENTS/role_for_uuid_or_none, roboco.foundation.policy.batch.is_batch_umbrella/main_pm_cannot_own_code/pm_cannot_own_code, roboco.foundation.policy.content.validators.coerce_str_list, roboco.foundation.policy.sequencing.models.DraftSurface/SequencePlan/SequencingError, roboco.foundation.policy.lifecycle (TaskStatus source), roboco.models.base (AgentRole/Complexity/TaskNature/TaskStatus/TaskType/Team), roboco.models.product.ProductCellMapping, roboco.models.task.TaskCreateRequest, roboco.models.secretary (DirectiveKind/DirectiveStatus/GATED_KINDS), roboco.seeds.initial_data.AGENT_UUIDS, roboco.utils.converters.require_uuid -- External: sqlalchemy (select, AsyncSession), structlog, httpx, asyncio, contextlib, json, re, dataclasses, uuid, datetime - -## Entry Points - -| Name | File | Trigger | -|---|---|---| -| POST /api/prompter/live/{session}/confirm-draft (confirm_live_draft) | roboco/api/routes/prompter_live.py | panel confirm button -> PrompterService.confirm_live_draft | -| POST /api/prompter/live/{session}/confirm-batch (confirm_live_batch) | roboco/api/routes/prompter_live.py | panel MegaTask confirm -> PrompterService.confirm_live_batch | -| GET /api/prompter/live/preview-batch (preview_batch) | roboco/api/routes/prompter_live.py | panel pre-confirm preview -> PrompterService.preview_batch | -| POST /api/prompter/live/{session}/redraft (update_live_draft) | roboco/api/routes/prompter_live.py | panel re-draft confirm -> PrompterService.update_live_draft | -| relay push/stream/deliver/is_alive endpoints | roboco/api/routes/prompter_live.py + secretary_live.py | panel SSE + message POST over PrompterLiveRegistry | -| orchestrator live-intake spawn/reap/idle hooks | roboco/runtime/orchestrator.py | _spawn_intake_container / _spawn_secretary_container / idle-reap sweep / board-review park / close_by_agent on kill | -| GET /api/prompter/live/{session}/search-tasks (search_past_tasks) | roboco/api/routes/prompter_live.py | Intake agent's `search_past_tasks` tool -> TaskService.search_tasks + compact_task_rows; session-aliveness-gated | -| POST /api/secretary/state, /task, /directive, /directive/{id}/confirm\|reject | roboco/api/routes/secretary.py | Secretary panel surface -> SecretaryService reads + directive lifecycle | -| GET /api/secretary/tasks?q= (search_tasks) | roboco/api/routes/secretary.py | Secretary or CEO resolves a task NAME to concrete id(s) -> TaskService.search_tasks, for targeting a `control_task` directive | - -## Config Flags -- ROBOCO_WORKSPACE_AUTO_CLONE / ROBOCO_WORKSPACE_CLONE_TIMEOUT (intake multi-repo clone scope: _clone_intake_scope, indirectly via orchestrator) -- ROBOCO_SELF_HEAL_ORIGINATE_ENABLED etc. do NOT gate this slice -- No direct ROBOCO_* flag in these three files; intake is a core capability (not feature-flagged), secretary is always-on; MegaTask is additive core, not gated - - -## Gotchas -- PrompterLiveRegistry.open is deliberately idempotent: a second open for an un-closed session returns the existing one instead of swapping the queue, because stream() captures the queue once and a fresh queue would strand the browser SSE on the old one while events push to the new one. -- Registry is a process-wide singleton held on _RegistryHolder (not a `global`); orchestrator is single-process and holds container state in memory — the relay is in-process only, not cross-process. -- deliver() logs transient POST failures at DEBUG (not ERROR) because the opening-message delivery retries until the container receiver is up; callers surface real failure (the /messages route 404s, _deliver_when_ready warns once after N tries). -- park() keeps a session alive (opposite of close) so board feedback can be injected in-context for an in-place re-draft; idle_session_ids explicitly excludes task_id-set (parked) sessions from idle reaping. -- TaskService is imported lazily inside create_task_from_draft / confirm_live_batch / update_live_draft to avoid circular imports. -- PM + code is structurally impossible: intake coerces code->planning via `_coerce_pm_code_to_planning`, which has two layers — team-based (main_pm_cannot_own_code) and assignee-based (pm_cannot_own_code for any PM assignee on a cell team). The umbrella is task_type=planning. TaskService.create is the backstop for non-intake HTTP paths. -- AGENTS['ceo'].uuid is captured at import time as _CEO_ID in secretary.py — CEO identity is a fixed seed uuid, not a DB lookup. -- _draft_cell_map de-dupes by team (first mapping wins) because task_cell_projects is unique per (task, team); a second the_work entry for the same cell is silently dropped. -- _compose_umbrella_draft produces a draft with NO project_id/product_id (branchless); _validate_draft_target's umbrella branch hard-rejects any target on it. -- Secretary _run catches ConflictError/NotFoundError/ValidationError/ValueError/KeyError -> FAILED with `error: {exc}` in result; any other exception propagates (no rollback of the flush). -- GATED_KINDS = {UPDATE_CHARTER, CONTROL_TASK, APPROVE_PITCH, ANNOUNCE}; only RELAY_MESSAGE runs immediately on submit_directive — ANNOUNCE is gated (needs CEO confirm), despite being a 'post a message' shape. -- compose_description falls back to the raw model description if the composed body is < _MIN_DESCRIPTION_LEN (20) chars — so a too-sparse structured draft still clears the schema minimum. -- SecretaryService._edit_task never touches status — _EDITABLE_TASK_FIELDS deliberately excludes it (status rides the separate audited start/cancel/override actions), so an "edit" directive that also needs a status change requires a second CONTROL_TASK directive. -- SecretaryService._reassign_task branches on the task's CURRENT status at call time: claimed/in_progress goes through reassign_active_claim (reseeds the heartbeat so the new assignee isn't immediately stale to the reaper), everything else falls through to the general reassign — a caller relying on one code path for both is testing the wrong branch depending on task state. -- history_digest_layer / build_history_digest return None / "" respectively on no data — a brand-new project or a board-level (no-project) spawn injects nothing into the ambient prompt (no empty "Recent tasks" header noise), which also means there is no explicit signal in the prompt that the digest was even attempted. -- search_past_tasks (both the grok MCP tool and the Claude SDK in-process tool) reads ROBOCO_PROMPTER_SESSION_ID from the environment and calls the session-scoped HTTP route — a tool call with no live session (or a session the registry has already closed) returns a plain string error, not an exception, so a stale intake container can call it silently forever without a hard failure surfacing. - - -## Changes Since Baseline - -| SHA | Subject | Impact | -|---|---|---| -| 15effce0 | feat(megatask): per-cell project map root-subtasks (multi-project, multi-cell) + main_pm+code impossibility + re-draft/batch hardening | Only commit touching this slice since baseline (prompter.py +228/-55; prompter_live.py and secretary.py unchanged). Adds the ad-hoc per-cell project map as a third draft target shape: _draft_cell_map, _MULTI_CELL_MIN, has_cell_projects param on _validate_draft_target, cell_projects on TaskCreateRequest, _validate_batch_scope counting per-cell pids. Adds main_pm_cannot_own_code coercion (code->planning) and switches umbrella task_type CODE->PLANNING. Extracts _validate_and_coerce_draft (coerces list fields via coerce_str_list) and _resolve_draft_assignee. Adds _as_work_entry to tolerate bare-string the_work entries. Changes _clean_list to use coerce_str_list (extracts dict-wrapped text instead of str(dict)). | - -> Post-snapshot updates (since 2026-06-29): 536bbb64 (Chore/all/logical gaps sweep, PR#286, 2026-06-30) touched prompter.py only (prompter_live.py and secretary.py still unchanged). Key changes: (1) fixes Risk #1 — 1-cell map branch now conditioned on `resolved_project_id is None and resolved_product_id is None` so a top-level target is no longer silently dropped; (2) fixes Risk #2 — `_draft_cell_map` now raises `ValidationError` on a malformed project_id instead of silently continuing; (3) fixes Risk #4 — `create_task_from_draft` calls `_copy_draft` first so `_validate_and_coerce_draft` never mutates the caller's dict; (4) fixes Risk #5 — product/board routing is now checked BEFORE the multi-cell map force (multi-cell is inside the `if resolved_product_id is None:` branch); (5) extracts code->planning coercion into `_coerce_pm_code_to_planning`, extending it to cover PM assignees on any team (via the new `pm_cannot_own_code` helper imported from `roboco.foundation.policy.batch`); (6) adds `_copy_draft` module-level function. LOC grew from ~1066 to 1142. -> -> `d1cf6ecb` Wave 1: PR-gate turn cut, task search, trace timestamps, Secretary edits + e2e scenarios 2–3 (#295) — secretary.py gains the full `edit` action (`_EDITABLE_TASK_FIELDS`, `_edit_task`, `_reassign_task`, `_resolve_assignee`) on `_control_task`; prompter.py gains the prompter-memory digest builders (`build_history_digest`, `project_history_digest`, `history_digest_layer`, `compact_task_rows`) plus the `TaskService.list_recent_for_project` / `search_tasks` backing queries; adds the `GET /live/{session}/search-tasks` route and the `search_past_tasks` MCP tool + Claude-SDK in-process parity tool. First commit to touch secretary.py since baseline. -> -> `da563487` Wave 2 features: A2A live view (CEO chime-in + reply budget) and prompter memory (#297) / `876e19b3` A2A switchboard + Secretary/PM task access + closed over-permission hole (#298) — no further changes to prompter.py/prompter_live.py/secretary.py beyond wave 1 above; these two commits' Secretary/PM-access work landed in `roboco/api/routes/tasks.py` (`_pm_editor_scope` / `_enforce_pm_lighter_fields`, closing the PM-role unrestricted-admin hole — out of this slice, see `docs/map/api-routes-schemas.md`) and their A2A work is entirely in `docs/map/a2a-audit-journal-permissions.md`. - -## Regression Risks - -| Title | File:Line | Claim | Severity | -|---|---|---|---| -| ~~1-cell map silently drops product_id and top-level project_id~~ **RESOLVED 536bbb64** | roboco/services/prompter.py:346 | ~~When _draft_cell_map returns exactly 1 entry, create_task_from_draft overwrites resolved_project_id with cell_map[0][1] and forces resolved_product_id=None.~~ Fixed: the 1-cell branch is now guarded by `resolved_project_id is None and resolved_product_id is None`; a top-level target is preserved over a redundant 1-cell map. | ~~medium~~ fixed | -| ~~Invalid project_id in a multi-cell map silently collapses the shape~~ **RESOLVED 536bbb64** | roboco/services/prompter.py:926 | ~~_draft_cell_map skips any the_work entry whose project_id fails UUID(str(pid)) (try/except continues).~~ Fixed: _draft_cell_map now raises `ValidationError` (clean 400) for a present-but-malformed project_id instead of silently continuing; the human is prompted to re-enter it. | ~~medium~~ fixed | -| Umbrella target gate is a behavior tightening that could reject previously-tolerated drafts | roboco/services/prompter.py:139 | The rewritten _validate_draft_target now hard-rejects an umbrella (is_batch_umbrella) that carries ANY target (project/product/cell-map). Before this commit an umbrella with only a project_id (no product_id) would not raise. Internal _compose_umbrella_draft never sets a project_id so the happy path is safe, but any external caller that builds a BatchPlacement(is_umbrella=True) draft with a stray project_id now gets a ValidationError instead of silent acceptance. | low | -| _clean_list semantics changed: dict-wrapped items now extracted instead of str(dict) | roboco/services/prompter.py:887 | _clean_list now delegates to coerce_str_list, which extracts text from dict-wrapped items (e.g. {'$text': ...}) instead of rendering `str(dict)`. This changes the rendered description text for any draft whose list fields contain dict-wrapped items. If coerce_str_list returns an unexpected shape for a non-string non-dict item (e.g. a list-of-lists), the rendered bullets / intended_to_touch / batch-scope counting could differ from the prior behavior. | low | -| ~~_validate_and_coerce_draft mutates the caller's draft dict in place~~ **RESOLVED 536bbb64** | roboco/services/prompter.py:207 | ~~_validate_and_coerce_draft overwrites draft_data fields in place; create_task_from_draft and confirm_live_batch callers were guarded by dict() copies but update_live_draft was not.~~ Fixed: create_task_from_draft now calls `_copy_draft(draft_data)` first (deep-copies the_work unit dicts too); the remaining concern for update_live_draft (no _validate_and_coerce_draft call) is unchanged. | ~~medium~~ partially fixed | -| ~~Multi-cell map team routing precedes product/board routing~~ **RESOLVED 536bbb64** | roboco/services/prompter.py:163 | ~~_resolve_owning_team checked multi-cell before product/board.~~ Fixed: product/board routing is now checked first (`if resolved_product_id is None:` gates the multi-cell path); a product draft with a ≥2-cell the_work map stays on the board-review path as required. The representation limit (product + cell-map not simultaneously expressible) is intentional, not a bug. | ~~low~~ fixed | - -## Health -The slice is coherent and well-defended. prompter_live.py remains unchanged since baseline and reads as a clean, focused singleton with correct lifecycle semantics (idempotent open, sentinel-based stream close, park-vs-close distinction). prompter.py and secretary.py both changed in wave 1 (`d1cf6ecb`): prompter.py gained the per-cell MegaTask map shape, the main_pm+code->planning coercion that closes the 2026-06-27 meltdown class, AND the prompter-memory digest builders (history digest + compact task search) — its validation is stricter and coercion is robust against LLM-emitted shapes (bare-string the_work, dict-wrapped list items, word-valued priority). secretary.py gained a genuinely new capability (the full-content `edit` directive action with claim-aware reassignment), its first change since baseline; the split between the allowlisted content fields and the status-only start/cancel/override actions is clean and the reassignment logic correctly branches on claim state. The main integrity concerns are two pre-existing silent-collapse paths in the cell-map handling: a 1-cell map silently drops product_id/top-level project_id, and a malformed project_id in a multi-cell map silently collapses the shape to single-cell — neither raises, so an LLM producing a slightly-off draft will create a mis-shaped task instead of a clean 400. The update_live_draft path skips the new _validate_and_coerce_draft guard, so re-drafts are not protected against empty-after-coercion AC. No drift from CLAUDE.md was found; the MegaTask umbrella is branchless/planning, ANNOUNCE is gated, and single-task intake is preserved. Overall the slice is healthy but the silent-collapse edges warrant a hardening pass to convert them into ValidationErrors. - # Slice Map — product-strategy-research-pitch ## Purpose @@ -7218,16 +8468,16 @@ The product / strategy / research / pitch slice covers the "company layer" above | `roboco/services/project.py` | CRUD + git-token encryption + cell access control for Projects (git repos) | 604 | | `roboco/services/product.py` | Product CRUD + per-cell `project_for` routing resolver + idempotent cell-map replace | 152 | | `roboco/services/kanban.py` | Role-specific kanban board views (dev/qa/documenter/pm/main-pm/board) from task data | 587 | -| `roboco/services/company_goals.py` | CRUD for the singleton company charter (north star + objectives + constraints + policy) | 83 | +| `roboco/services/company_goals.py` | CRUD for the singleton company charter (north star + objectives + constraints + policy + brand_voice + company_name); `resolve_product_name` is the shared product-name fallback chain `XEngine`/`VideoEngine` both call | 110 | | `roboco/services/strategy_engine.py` | Dormant "engine 2": assesses company state vs goals, notify-only to CEO | 111 | | `roboco/services/research.py` | Pluggable web-search/fetch — provider adapters (Tavily/Brave/Exa/Null) + clamping service | 431 | | `roboco/services/research_quota.py` | Per-agent UTC-daily Redis quota counter for research calls (fail-open) | 78 | | `roboco/services/pitch.py` | Board pitch CRUD + CEO approve → provision repos/Projects(+Product) + seed Main-PM task | 274 | -| `roboco/services/github_provisioning.py` | The only service that CREATES GitHub repos (POST `/orgs/{org}/repos`) | 174 | +| `roboco/services/github_provisioning.py` | The only service that CREATES repos for pitch provisioning — now provider-aware (GitHub/Gitea/GitLab, Phase 4 forge parity), despite the GitHub-flavored name (kept for backward compatibility) | 232 | | `roboco/services/roadmap_engine.py` | Dormant weekly engine: originates ONE held roadmap-exploration task for the Product Owner (default off) | 111 | | `roboco/services/roadmap_service.py` | CEO's per-item approve/reject glue over a held roadmap cycle; approve materializes a BACKLOG task | 211 | | `roboco/api/routes/roadmap.py` | CEO-only routes: list open cycles, approve/reject one item | 124 | -| `roboco/services/x_engine.py` | Dormant "engine 4": drafts X (Twitter) release posts (event hook), mention replies (poll), and — new — feature-spotlight explorations (dormant interval, spawns Head of Marketing), ALL held for CEO approval (default off) | 463 | +| `roboco/services/x_engine.py` | Dormant "engine 4": drafts X (Twitter) release posts (event hook), mention replies (poll), and feature-spotlight explorations (dormant interval, spawns Head of Marketing), ALL held for CEO approval (default off); prompt builders take a `product_name` param resolved via `CompanyGoalsService.resolve_product_name` instead of hardcoding "RoboCo" | 871 | | `roboco/services/x_post_service.py` | CEO's approve/reject over a held X draft; approve posts via a Redis single-flight lock, idempotent on already-posted AND on already-rejected (CANCELLED) | 298 | | `roboco/services/x_client.py` | OAuth 1.0a HMAC-SHA1 X API client (`LiveXClient`) + `NullXClient` (no creds, never egresses) + `build_x_client` factory | 318 | | `roboco/services/x_credentials.py` | Singleton Fernet-encrypted OAuth 1.0a credential CRUD; decrypts server-side only | 140 | @@ -7284,12 +8534,13 @@ The product / strategy / research / pitch slice covers the "company layer" above | `PitchService._register_topology` | method | pitch.py:201 | Multi-cell → Product (reuse existing by slug + refresh cell map); single-cell → seed project only | | `PitchService._seed_main_pm_task` | method | pitch.py:234 | Creates PENDING Main-PM CODE task (`source="pitch"`, `confirmed_by_human=True`) | | `PitchService._proposed_or_raise` | method | pitch.py:152 | 404 if missing, 409 if not `proposed` (no re-deciding) | -| `GitHubProvisioningService` | class | github_provisioning.py:45 | Create private repos in configured org | -| `GitHubProvisioningService.enabled` | prop | github_provisioning.py:67 | True only when master switch + token + org all set | -| `GitHubProvisioningService.create_repo` | method | github_provisioning.py:81 | POST `/orgs/{org}/repos` with `auto_init=true`; handles GitHub 422 "already exists" idempotently via `_fetch_existing_repo` (#83/#84) | -| `GitHubProvisioningService._fetch_existing_repo` | method | github_provisioning.py:140 | GET `org/name` and reconstruct `ProvisionedRepo` — called on 422 to reuse an orphaned repo from a rolled-back prior approval | -| `_GITHUB_REPO_EXISTS_STATUS` | constant | github_provisioning.py:42 | `422` — GitHub's "name already exists" status sentinel | -| `ProvisionedRepo` / `ProvisioningError` / `ProvisioningDisabledError` | dataclass/exc | github_provisioning.py:32 / 23 / 27 | Result + error types | +| `GitHubProvisioningService` | class | github_provisioning.py:81 | Create private repos for pitch provisioning — Phase 4 forge parity: provider-dispatched via `_build_provider`, not GitHub-only despite the class name | +| `GitHubProvisioningService.enabled` | prop | github_provisioning.py:123 | True only when master switch + token + org all set; ALSO requires `ROBOCO_PROVISIONING_HOST` when the provider is gitlab/gitea (self-hosted needs a host, github.com doesn't) | +| `GitHubProvisioningService.create_repo` | method | github_provisioning.py:142 | Provider-dispatched repo creation with `auto_init=true`; handles the "already exists" response idempotently across all three forges via `_fetch_existing_repo`/`_is_already_exists` (#83/#84) | +| `GitHubProvisioningService._fetch_existing_repo` | method | github_provisioning.py:203 | GET the existing repo and reconstruct `ProvisionedRepo` — called on an "already exists" response to reuse an orphaned repo from a rolled-back prior approval | +| `_build_provider` | func | github_provisioning.py:68 | Picks the concrete provider (`GitHubProvider`/`GiteaProvider`/`GitLabProvider` — a `Union`, not the `GitProvider` ABC, since provisioning needs `client=`/`timeout=` kwargs the ABC doesn't declare) by `ROBOCO_PROVISIONING_PROVIDER` | +| `_is_already_exists` | func | github_provisioning.py:61 | Matches GitHub's 422, Gitea's 409/422, and GitLab's reshaped 422 "already exists"/"has already been taken" by status+phrase | +| `ProvisionedRepo` / `ProvisioningError` / `ProvisioningDisabledError` | dataclass/exc | github_provisioning.py | Result + error types | | `RoadmapEngine` | class | roadmap_engine.py:49 | Dormant "engine 3": mirrors the release-manager "detect → originate a CEO-gated artifact → hold" shape, but the artifact is a cycle the PO *authors*, not a report the engine assembles | | `RoadmapEngine.run_cycle` | method | roadmap_engine.py:54 | No-op unless `roadmap_engine_enabled`, a cycle is already open (`list_open_roadmap_cycles`), or the RoboCo project isn't resolvable; else opens ONE held PENDING exploration task assigned to the Product Owner | | `RoadmapService` | class | roadmap_service.py:50 | List / approve / reject items within the open roadmap cycle(s) | @@ -7299,19 +8550,21 @@ The product / strategy / research / pitch slice covers the "company layer" above | `RoadmapService._maybe_complete_cycle` | staticmethod | roadmap_service.py:202 | Completes the exploration task once every item on it is terminal (approved/rejected) | | `RoadmapItemResult` | dataclass | roadmap_service.py:37 | Outcome of one approve/reject call (status/item_id/materialized_task_id/detail) | | `get_roadmap_engine` / `get_roadmap_service` | factory | roadmap_engine.py:109 / roadmap_service.py:209 | Session-bound constructors | -| `XEngine` | class | x_engine.py:150 | Dormant "engine 4": mirrors the release-manager "detect → originate a CEO-gated artifact → hold" shape across THREE responsibilities — release posts, mention replies, feature spotlights | -| `XEngine._voice_guide` | method | x_engine.py:173 | Baseline house-voice constant (`_HOM_VOICE`) plus the CEO's `company_goals.brand_voice` sample when set — feeds release/reply prompts AND is the mechanism the HoM identity file points to for its own drafting | -| `XEngine.draft_release_post` | method | x_engine.py:192 | Event-driven hook (called from `ReleaseProposalService.approve`'s publish-success branch); local-model-drafted, deduped per version, capped by `x_max_open_posts` | -| `XEngine.run_cycle` | method | x_engine.py:255 | Periodic mentions poll; no-op unless `x_engine_enabled` AND `x_replies_enabled`; filters bot-like/low-engagement mentions, dedupes by mention id (`XSeenMentionTable`); each mention's text is run through `screen_external_text` before the local-model reply prompt sees it | +| `XEngine` | class | x_engine.py:230 | Dormant "engine 4": mirrors the release-manager "detect → originate a CEO-gated artifact → hold" shape across THREE responsibilities — release posts, mention replies, feature spotlights | +| `XEngine._voice_guide` | method | x_engine.py:259 | `_voice_guide(product_name)`: baseline house-voice constant (`_HOM_VOICE`) plus the CEO's `company_goals.brand_voice` sample when set — feeds release/reply prompts AND is the mechanism the HoM identity file points to for its own drafting; `product_name` is resolved once per call site via `CompanyGoalsService.resolve_product_name(project)` (project's own name → charter `company_name` → "RoboCo" literal), not hardcoded | +| `XEngine.draft_release_post` | method | x_engine.py:279 | Event-driven hook (called from `ReleaseProposalService.approve`'s publish-success branch); local-model-drafted, deduped per version, capped by `x_max_open_posts`; resolves and threads `product_name` from the release's own project | +| `XEngine.run_cycle` | method | x_engine.py:353 | Periodic mentions poll; no-op unless `x_engine_enabled` AND `x_replies_enabled`; filters bot-like/low-engagement mentions, dedupes by mention id (`XSeenMentionTable`); each mention's text is run through `screen_external_text` before the local-model reply prompt sees it; resolves `product_name` once per cycle and threads it through `_originate_reply`/`_draft_reply_body`/`_reply_prompt` | | `screen_external_text` | function | foundation/policy/injection_guard.py:95 | Shared screen-and-neutralize guard for unattended attacker-writable text feeds (X mentions, vault inbox notes): wraps the text in an untrusted-content envelope and flags any matched injection-pattern LINE inline — nothing is removed, so the CEO-facing draft still shows what the source really said | -| `XEngine.open_feature_spotlight_exploration` | method | x_engine.py:337 | No-ops unless `x_engine_enabled` AND `x_feature_spotlight_enabled`, no creds, a cycle already open, the open-post cap reached, or project unresolvable; else opens ONE held PENDING exploration task for the Head of Marketing (`source=x_feature_exploration`) carrying a `x_seen_features` marker snapshot | -| `XEngine.materialize_feature_spotlight` | method | x_engine.py:433 | Called from the `propose_feature_spotlight` do-tool: marks the feature slug seen (`XSeenFeatureTable`), creates the held draft (`source=x_feature`, identical shape to a release/reply draft), completes the exploration task | +| `XEngine.open_feature_spotlight_exploration` | method | x_engine.py:487 | No-ops unless `x_engine_enabled` AND `x_feature_spotlight_enabled`, no creds, a cycle already open, the open-post cap reached, or project unresolvable; else opens ONE held PENDING exploration task for the Head of Marketing (`source=x_feature_exploration`) carrying a `x_seen_features` marker snapshot; the description is built by `_feature_exploration_description(product_name)`, no longer the fixed `_FEATURE_EXPLORATION_DESCRIPTION` string | +| `XEngine.materialize_feature_spotlight` | method | x_engine.py:841 | Called from the `propose_feature_spotlight` do-tool: marks the feature slug seen (`XSeenFeatureTable`), creates the held draft (`source=x_feature`, identical shape to a release/reply draft), completes the exploration task | | `XPostService.approve` | method | x_post_service.py:92 | The ONLY caller of `x_client.post_tweet`; Redis single-flight lock, re-reads task under lock, idempotent on an already-posted draft (`already_posted`); a CANCELLED draft is refused both pre-lock and re-checked under lock (`already_rejected`) — a stale approve (e.g. a queued Telegram button) can't resurrect a draft the CEO already rejected | | `XPostService.reject` | method | x_post_service.py:251 | Records the CEO's reason; cancels the held draft | | `XClient` / `NullXClient` / `LiveXClient` | ABC/class | x_client.py:150 / 166 / 186 | `NullXClient.configured` is False (no creds) — drafting still runs (content nobody can post is a no-op upstream), just never originates; `LiveXClient` signs OAuth 1.0a HMAC-SHA1 | | `build_x_client` | factory | x_client.py:306 | Returns `LiveXClient` when credentials decrypt, else `NullXClient` | | `XCredentialsService.set_credentials` / `.get_decrypted` | method | x_credentials.py:61 / 116 | All-or-nothing Fernet-encrypted singleton credential set/clear; decrypts server-side only, never exposed to agents | -| `get_x_engine` | factory | x_engine.py:461 | Session-bound constructor (optional injected `XClient` for tests) | +| `get_x_engine` | factory | x_engine.py:869 | Session-bound constructor (optional injected `XClient` for tests) | +| `CompanyGoalsService.resolve_product_name` | method | company_goals.py:79 | The shared product-name fallback chain: `project.name` if set, else the charter's `company_name`, else the "RoboCo" literal — single source so `XEngine`/`VideoEngine` can't drift apart on branding | +| `task_project_fields` | func | api/schemas/project_fields.py:19 | `(project_slug, project_name)` or `(None, None)` for a task response — `sa_inspect(task).unloaded` guard before touching `task.project` (a freshly-created task can have an unloaded relationship); shared by the X and video queue response builders so a multi-project CEO can tell drafts apart via the panel's `ProjectBadge` | ## Data Flow @@ -7526,6 +8779,8 @@ product-strategy-research-pitch | `ROBOCO_GITHUB_API_BASE_URL` | `https://api.github.com` | config.py:327 | Override for GitHub Enterprise | | `ROBOCO_PROVISIONING_TIMEOUT_SECONDS` | `30.0` | config.py:331 | Outbound GitHub provisioning timeout | | `ROBOCO_PROVISIONING_REPO_PRIVATE` | `True` | config.py:336 | Whether provisioned repos are private | +| `ROBOCO_PROVISIONING_PROVIDER` | `github` | config.py:570 | Phase 4 forge parity: `github`/`gitlab`/`gitea` selects the concrete provisioning target via `_build_provider` | +| `ROBOCO_PROVISIONING_HOST` | `""` | config.py:579 | Self-hosted forge host (e.g. `gitlab.example.com`); `.enabled` additionally requires this when `provisioning_provider` is gitlab/gitea (ignored for github/gitlab.com) | | `ROBOCO_STRATEGY_ENGINE_ENABLED` | `False` | config.py:348 | Master switch — loop never starts when off | | `ROBOCO_STRATEGY_ENGINE_INTERVAL_SECONDS` | `1800` | config.py:354 | Seconds between strategy assessment passes | | `ROBOCO_STRATEGY_STRANDED_BLOCKED_MINUTES` | `120` | config.py:360 | Blocked-task threshold for "stranded" observation | @@ -7574,6 +8829,9 @@ product-strategy-research-pitch > - `b3558d4e` ([chore] complexity: split 5 C-rank blocks to <=B, 2026-06-30): `kanban.py` `get_main_pm_board_flat` — refactored if/elif routing to a dict-dispatch (`status_col` + `team_col` maps) for xenon complexity gate; no functional change. > - **v0.18.0** (2026-07-04): the X feature-spotlight content in this slice (`XEngine` feature-spotlight methods, `_x_feature_spotlight_loop`/`_dispatch_feature_spotlight_exploration`, migration 061, `x_feature_spotlight_enabled`) was authored directly into this file's Files/Key Symbols/Data Flow/Mermaid/Logical Tree/Entry Points sections at implementation time rather than landing as a dated delta — noted here for changelog continuity; the body text above is current as of this date. (Config Flags is unchanged — the X-engine flags live in deployment-tooling.md's comprehensive list, not here.) > - `11915f36` (PR #551, Telegram V2 security follow-up, 2026-07-17): `x_post_service.py` — `XPostService.approve`/`_approve_locked` add a CANCELLED-task guard (pre-lock and re-checked under lock) returning a new `already_rejected` status, closing a live-reproduced approve-after-reject hole reachable via a stale Telegram Approve button (or a replayed HTTP call). +> - `7e01c0ce` (PR #570, "project-branded drafts + project badges", 2026-07-18): migration 075 adds `company_goals.company_name`; `CompanyGoalsService.resolve_product_name` (company_goals.py:79) is the new single fallback chain (project name → charter `company_name` → "RoboCo") consumed by both `XEngine._voice_guide`/`draft_release_post` and `VideoEngine` (see `docs/map/video-engine.md`) so their prompt builders stop hardcoding "RoboCo". New `roboco/api/schemas/project_fields.py`'s `task_project_fields` helper adds `project_slug`/`project_name` to the X and video post-queue API responses (`api/routes/x.py`, `api/routes/video.py`); the panel renders them via a shared `ProjectBadge` — see `docs/map/panel.md`. +> - `461a6e1a`+`96401f4c`+`5f32d876` (Phases 1/2-3/4, 2026-07-18/19, #571/#575/#581) — Phase 4 makes `GitHubProvisioningService` provider-aware: `_build_provider` (github_provisioning.py:68) dispatches to `GitHubProvider`/`GiteaProvider`/`GitLabProvider` by `ROBOCO_PROVISIONING_PROVIDER`, `.enabled` additionally requires `ROBOCO_PROVISIONING_HOST` for gitlab/gitea, and `_is_already_exists` (github_provisioning.py:61) matches the "already exists" idempotency signal across all three forges' differing status codes/phrasing. The forge transport package itself (`GitProvider`/`ForgeRouter`/provider implementations) is documented in `docs/map/worksession-git.md` — this slice only covers the provisioning consumer. +> - `a0baf94b` ("agnosticism-residue", agnosticism audit items B6/B8): `x_engine.py`'s remaining hardcoded `"RoboCo"` literals (the reply-prompt builder and the feature-spotlight exploration description — `draft_release_post`/`_voice_guide` were already fixed by `7e01c0ce` above) are threaded out: `_reply_prompt` gains a `product_name` param, `_FEATURE_EXPLORATION_DESCRIPTION` (a module constant) becomes `_feature_exploration_description(product_name)` (a function), and `run_cycle`/`open_feature_spotlight_exploration` each resolve `product_name` once via `resolve_product_name` and thread it through. ## Regression Risks @@ -7593,471 +8851,6 @@ No commit since `fd10cc86` modified any file in this slice, so there are **no re ## Health This slice is internally coherent and consistent with CLAUDE.md: every documented flag, default, and behavior matches the code, and the two slices-of-flow (CEO-driven pitch origination into the normal lifecycle; dormant notify-only strategy watcher) are cleanly separated and default-safe. The services follow a uniform `BaseService` + session-bound factory pattern, provider/research quotas fail open where cost-control (not security) is the goal, and the provisioning path is inert without token+org. The pitch approval path's external-side-effect non-atomicity remains (GitHub repo creation cannot roll back with the DB transaction), but re-approval is now idempotent end-to-end: `create_repo` handles GitHub 422 "already exists" by fetching the existing repo, and Project/Product rows are reused by slug, so a CEO re-approving after a partial failure recovers cleanly. The remaining open risk is `_seed_main_pm_task` failing after repos are already created (missing `main-pm` agent row). Post-snapshot three commits updated this slice's files, resolving four standing risks (kanban subtask counts, flat-board dropped cards, `project.update` None-field skip, and the pitch re-approval collision). - -## Purpose -Three default-off background "engine" services that watch CI / dependencies and originate a single PENDING fix task into the normal delivery lifecycle, then stop. SelfHealEngine watches RoboCo's OWN repo CI and (behind a second opt-in) opens a CEO-held fix task; CiWatchEngine fans that out to every opted-in project; DepUpdateEngine probes whether a dependency upgrade would change lockfiles and opens an "update dependencies" task. All three are detect+originate only — none ever start, approve, merge, or deploy; they flush writes and the orchestrator loop owns the commit. - -## Files - -| Path | Role | LOC | -|---|---|---| -| /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/self_heal_engine.py | Single-repo self-heal: detect a regression in RoboCo's own CI via telemetry, notify CEO, optionally open a HELD PENDING fix task | 310 | -| /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/ci_watch_engine.py | Multi-repo CI-watch: for each opted-in project whose CI is red, open one READY-to-start PENDING fix task (deduped per git_url) and notify the cell PM | 190 | -| /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/dep_update_engine.py | Dependency-update bot: probe each opted-in project's lockfile for changes and open one READY-to-start PENDING update task per repo | 138 | -| /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/runtime/orchestrator.py | Owns the three background loops (_self_heal_loop, _ci_watch_loop, _dep_update_loop) that construct the engines, call run_cycle, and commit the session | 0 | -| /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/task.py | Provides SELF_HEAL/CI_WATCH/DEP_UPDATE source tags, list_open_*_tasks dedupe queries, extract_self_heal_fingerprint, and the give_me_work self-heal hold filter | 0 | -| /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/telemetry/source.py | TelemetrySource protocol + GitHubCITelemetrySource (single repo) + MultiProjectCITelemetrySource (fan-out) feeding breach samples to the engines | 0 | - -## Key Symbols - -| Name | Kind | File:Line | Responsibility | -|---|---|---|---| -| RegressionObservation | dataclass | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/self_heal_engine.py:54 | Frozen record of one detected regression: fingerprint, signal/repo names, summary/detail/raw_ref | -| _fingerprint | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/self_heal_engine.py:65 | Stable 16-char sha256 prefix of the signal name — the dedupe key for open self-heal fix tasks | -| _NOTIFY_DEDUPE_KEY_PREFIX | constant | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/self_heal_engine.py:70 | Module-level Redis key prefix for per-fingerprint CEO-notify dedupe ("self_heal:notified:") | -| SelfHealEngine | class | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/self_heal_engine.py:73 | Detect regressions in RoboCo's own repo, notify CEO, optionally originate a HELD fix task | -| SelfHealEngine.assess | method | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/self_heal_engine.py:84 | Read telemetry samples, return RegressionObservations for breaches; pure, no side effects | -| SelfHealEngine.run_cycle | method | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/self_heal_engine.py:103 | Gate on self_heal_enabled, assess, notify CEO per obs (deduped per fingerprint via Redis), optionally originate; returns observations; flushes, caller commits | -| SelfHealEngine._open_self_heal_task_ids_by_fp | method | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/self_heal_engine.py:145 | Map each open self-heal task's fingerprint to its task id; best-effort (returns {} on DB error) — used to link CEO alert to fix task and corroborate notify dedupe | -| SelfHealEngine._already_notified | method | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/self_heal_engine.py:166 | Fail-open Redis check: True when the fingerprint was CEO-notified this episode (a Redis outage returns False so the notify fires anyway) | -| SelfHealEngine._mark_notified | method | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/self_heal_engine.py:184 | Record that this fingerprint was CEO-notified; sets a Redis key with self_heal_notify_dedupe_seconds TTL; best-effort (failure swallowed) | -| SelfHealEngine._dedupe_key | staticmethod | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/self_heal_engine.py:207 | Build the Redis key _NOTIFY_DEDUPE_KEY_PREFIX + fingerprint | -| SelfHealEngine._originate | method | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/self_heal_engine.py:210 | Open one PENDING HELD (confirmed_by_human=False) fix task per NEW regression, bounded by per-cycle/rolling caps + fingerprint dedupe | -| get_self_heal_engine | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/self_heal_engine.py:306 | Factory: construct SelfHealEngine bound to a session with optional test source | -| _cell_pm_slug_for | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/ci_watch_engine.py:45 | Resolve the cell-PM agent slug owning a team (e.g. Team.BACKEND -> 'be-pm'), or None | -| CiWatchEngine | class | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/ci_watch_engine.py:53 | Open a fix task per opted-in project whose CI is red; never merges | -| CiWatchEngine.run_cycle | method | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/ci_watch_engine.py:62 | Gate on ci_watch_enabled, fetch breaches for the watch set, originate fix tasks; returns opened tasks; flushes, caller commits | -| CiWatchEngine._originate | method | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/ci_watch_engine.py:78 | Open one ci_watch fix task per NEW red repo bounded by caps; notify the cell PM best-effort per opened task | -| CiWatchEngine._notify_cell_pm | method | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/ci_watch_engine.py:108 | Best-effort ack notification to the red project's cell PM; failure never rolls back origination | -| CiWatchEngine._should_open | method | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/ci_watch_engine.py:136 | True when project resolves and has no open ci_watch task for its git_url (monorepo dedupe) | -| CiWatchEngine._open_fix_task | method | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/ci_watch_engine.py:160 | Create the PENDING READY-to-start (confirmed_by_human=True) Main-PM coordination root fix task | -| get_ci_watch_engine | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/ci_watch_engine.py:198 | Factory: construct CiWatchEngine bound to a session with optional test source | -| DepUpdateEngine | class | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/dep_update_engine.py:39 | Open an update-dependencies task per opted-in project with lockfile changes available | -| DepUpdateEngine.run_cycle | method | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/dep_update_engine.py:48 | Gate on dep_update_enabled, probe each project, open tasks bounded by caps; returns opened tasks; flushes, caller commits | -| DepUpdateEngine._eligible | method | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/dep_update_engine.py:81 | Cheap checks (command set, id present, per-git_url dedupe) then expensive read-only lockfile probe; returns eligibility bool | -| DepUpdateEngine._open_task | method | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/dep_update_engine.py:96 | Create the PENDING READY-to-start (confirmed_by_human=True) Main-PM coordination root dep-update task | -| get_dep_update_engine | function | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/dep_update_engine.py:133 | Factory: construct DepUpdateEngine bound to a session with optional test workspace probe | - -## Data Flow -Each engine is constructed per-cycle by its orchestrator loop inside a `get_db_context()` session. SelfHealEngine pulls breach samples from a TelemetrySource (GitHubCITelemetrySource for RoboCo's own repo); CiWatchEngine's MultiProjectCITelemetrySource.fetch(projects) takes the watch set the orchestrator loaded; DepUpdateEngine does NOT use telemetry — it calls WorkspaceService.dry_upgrade_changes_lockfile(project) (read-only probe in a throwaway clone). On a breach, each engine calls TaskService.list_open_*_tasks (optionally scoped by git_url for ci_watch/dep_update) to dedupe, checks per-cycle + rolling open-task caps against settings, resolves the target project (self_heal resolves by slug via ProjectService.get_by_slug; ci_watch/dep_update already have the project row), then calls TaskService.create(TaskCreateRequest) with the matching source tag (SELF_HEAL_SOURCE / CI_WATCH_SOURCE / DEP_UPDATE_SOURCE), team=MAIN_PM, assigned_to the main-pm agent UUID, status=PENDING. The self-heal task is HELD (confirmed_by_human=False) and carries a fingerprint via markers.set_self_heal_fingerprint so later cycles see it as already-open; ci_watch and dep_update tasks are READY (confirmed_by_human=True). Self-heal always notifies the CEO via NotificationService.send_ack_notification; ci_watch notifies the red project's cell PM best-effort; dep_update notifies no one. Each engine only flushes; the orchestrator loop commits. The created tasks then ride the normal delivery lifecycle — give_me_work offers ci_watch/dep_update tasks immediately, but excludes source=self_heal + confirmed_by_human=False tasks until the CEO's approve_and_start flips the flag (task.py line ~7304). - -## Mermaid -```mermaid -graph TD - subgraph Orchestrator loops - SHL[_self_heal_loop] --> SH[SelfHealEngine.run_cycle] - CWL[_ci_watch_loop] --> CWR[_run_ci_watch_cycle] --> CW[CiWatchEngine.run_cycle] - DUL[_dep_update_loop] --> DUR[_run_dep_update_cycle] --> DU[DepUpdateEngine.run_cycle] - end - TS[TelemetrySource / MultiProjectCITelemetrySource] -->|breach samples| SH - TS -->|breach samples| CW - WS[WorkspaceService.dry_upgrade_changes_lockfile] -->|lockfile changed?| DU - SH -->|notify| CEO[CEO ack notif] - CW -->|notify best-effort| CELLPM[Cell PM ack notif] - SH --> TSVC[TaskService.create source=self_heal HELD] - CW --> TSVC2[TaskService.create source=ci_watch READY] - DU --> TSVC3[TaskService.create source=dep_update READY] - TSVC --> DB[(TaskTable)] - TSVC2 --> DB - TSVC3 --> DB - DB -->|list_open_*_tasks dedupe + caps| SH - DB -->|list_open_*_tasks dedupe + caps| CW - DB -->|list_open_*_tasks dedupe + caps| DU - DB -->|give_me_work hold filter| GMW[give_me_work: self_heal+unconfirmed excluded] - GMW --> CEO_APPROVE[CEO approve_and_start flips confirmed_by_human] - CEO_APPROVE --> NORMAL[Normal delivery lifecycle: dev->QA->PR review->CEO merge] - CW_TASK[ci_watch/dep_update PENDING task] --> NORMAL - settings_self_heal[self_heal_enabled + _originate_enabled] -.gate.-> SH - settings_cw[ci_watch_enabled] -.gate.-> CW - settings_du[dep_update_enabled] -.gate.-> DU -``` - -## Logical Tree -``` -engines-heal-ciwatch-depupdate - SelfHealEngine (roboco/services/self_heal_engine.py) - RegressionObservation (frozen dataclass: fingerprint, signal_name, repo_hint, summary, detail, raw_ref) - _fingerprint(signal_name) -> 16-char sha256 prefix - _NOTIFY_DEDUPE_KEY_PREFIX = "self_heal:notified:" - __init__(session, source=None) -> binds TelemetrySource - assess() -> [RegressionObservation] for breaches (pure) - run_cycle() -> gates on self_heal_enabled; assess; notify CEO deduped per fingerprint via Redis; optionally _originate - _open_self_heal_task_ids_by_fp() -> {fingerprint: task_id} for open self-heal tasks; best-effort - _already_notified(fingerprint) -> bool; fail-open Redis check - _mark_notified(fingerprint) -> set Redis key with notify_dedupe_seconds TTL; best-effort - _dedupe_key(fingerprint) -> Redis key string - _originate(observations) -> dedupe by fingerprint + caps; create HELD PENDING task; set_self_heal_fingerprint - get_self_heal_engine(session, source=None) - CiWatchEngine (roboco/services/ci_watch_engine.py) - _cell_pm_slug_for(team) -> cell PM slug - __init__(session, source=None) -> binds MultiProjectCITelemetrySource - run_cycle(projects) -> gates on ci_watch_enabled; fetch breaches; _originate - _originate(breaches, by_slug) -> per-repo dedupe + caps; _open_fix_task + _notify_cell_pm - _notify_cell_pm(project, sample) -> best-effort ack to cell PM - _should_open(task_svc, project) -> project resolves + no open task for git_url - _open_fix_task(task_svc, project, sample) -> create READY PENDING Main-PM root - get_ci_watch_engine(session, source=None) - DepUpdateEngine (roboco/services/dep_update_engine.py) - __init__(session, workspace=None) -> binds WorkspaceService - run_cycle(projects) -> gates on dep_update_enabled; per-project probe; _open_task - _eligible(task_svc, project) -> command set + id + git_url dedupe + dry_upgrade_changes_lockfile - _open_task(task_svc, project) -> create READY PENDING Main-PM root - get_dep_update_engine(session, workspace=None) - Orchestrator loops (roboco/runtime/orchestrator.py) - _self_heal_loop -> get_self_heal_engine(db).run_cycle() + db.commit() - _ci_watch_loop -> _run_ci_watch_cycle -> _load_ci_watch_set (one per (repo, effective workflow)) + get_ci_watch_engine(db).run_cycle(watch_set) + db.commit() - _dep_update_loop -> _run_dep_update_cycle -> _load_dep_update_set + get_dep_update_engine(db).run_cycle(projects) + db.commit() -``` - -## Dependencies -- Internal: roboco.config.settings, roboco.foundation.identity (AGENTS, Role), roboco.foundation.policy.content.markers (set_self_heal_fingerprint), roboco.models.base (Complexity, TaskNature, TaskStatus, TaskType, Team), roboco.services.base.BaseService, roboco.services.notification.NotificationService, roboco.services.project.get_project_service, roboco.services.task (TaskService, TaskCreateRequest, SELF_HEAL_SOURCE, CI_WATCH_SOURCE, DEP_UPDATE_SOURCE, extract_self_heal_fingerprint, get_task_service), roboco.services.telemetry (get_ci_telemetry_source), roboco.services.telemetry.source (get_multi_ci_telemetry_source), roboco.services.workspace.get_workspace_service (dry_upgrade_changes_lockfile), roboco.runtime.orchestrator (the three loops), roboco.db.get_db_context -- External: sqlalchemy.ext.asyncio.AsyncSession, asyncio, hashlib, dataclasses, typing - -## Entry Points - -| Name | File | Trigger | -|---|---|---| -| _self_heal_loop | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/runtime/orchestrator.py | asyncio.create_task at orchestrator start() (line 1012); sleeps self_heal_interval_seconds, opens a DB session, calls SelfHealEngine.run_cycle, commits; early-returns when self_heal_enabled is False | -| _ci_watch_loop | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/runtime/orchestrator.py | asyncio.create_task at orchestrator start() (line 1013); sleeps ci_watch_interval_seconds, runs _run_ci_watch_cycle (loads watch set, runs CiWatchEngine.run_cycle, commits); early-returns when ci_watch_enabled is False | -| _dep_update_loop | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/runtime/orchestrator.py | asyncio.create_task at orchestrator start() (line 1014); sleeps dep_update_interval_seconds, runs _run_dep_update_cycle (loads eligible projects, runs DepUpdateEngine.run_cycle, commits); early-returns when dep_update_enabled is False | - -## Config Flags -- ROBOCO_SELF_HEAL_ENABLED (self_heal_enabled) — master switch for the self-heal loop -- ROBOCO_SELF_HEAL_ORIGINATE_ENABLED (self_heal_originate_enabled) — second opt-in: actually open a fix task (notify-only otherwise) -- ROBOCO_SELF_HEAL_PROJECT_SLUG (self_heal_project_slug) — the single repo self-heal targets -- ROBOCO_SELF_HEAL_CI_WORKFLOW (self_heal_ci_workflow) — CI workflow name for the self-heal telemetry source -- ROBOCO_SELF_HEAL_INTERVAL_SECONDS (self_heal_interval_seconds) — loop period -- ROBOCO_SELF_HEAL_MAX_OPEN_TASKS (self_heal_max_open_tasks) — rolling open-task cap -- ROBOCO_SELF_HEAL_MAX_PER_CYCLE (self_heal_max_per_cycle) — per-cycle origination cap -- ROBOCO_SELF_HEAL_NOTIFY_DEDUPE_SECONDS (self_heal_notify_dedupe_seconds, default 7200) — per-fingerprint CEO-notify dedupe window; a regression that stays red notifies once per episode, not every tick; the key expires after this window so a recurrence notifies again; fail-open (Redis outage still lets the notify through) -- ROBOCO_CI_WATCH_ENABLED (ci_watch_enabled) — master switch for multi-repo CI-watch -- ROBOCO_CI_WATCH_DEFAULT_WORKFLOW (ci_watch_default_workflow) — fallback workflow when a project sets none -- ROBOCO_CI_WATCH_INTERVAL_SECONDS (ci_watch_interval_seconds) -- ROBOCO_CI_WATCH_MAX_OPEN_TASKS (ci_watch_max_open_tasks) -- ROBOCO_CI_WATCH_MAX_PER_CYCLE (ci_watch_max_per_cycle) -- ROBOCO_DEP_UPDATE_ENABLED (dep_update_enabled) — master switch for the dep-update bot -- ROBOCO_DEP_UPDATE_INTERVAL_SECONDS (dep_update_interval_seconds, default 604800 = weekly) -- ROBOCO_DEP_UPDATE_MAX_OPEN_TASKS (dep_update_max_open_tasks) -- ROBOCO_DEP_UPDATE_MAX_PER_CYCLE (dep_update_max_per_cycle) -- per-project projects.ci_watch_enabled / ci_watch_workflow / dep_update_command / assigned_cell (DB columns) - - -## Gotchas -- SelfHealEngine.run_cycle dedupes CEO notifications per fingerprint via Redis (_already_notified / _mark_notified): a regression that stays red across cycles pings the CEO once per episode, not every tick. The check fails open — a Redis outage returns False so the notify still fires (never a swallowed regression). The dedupe key TTL is self_heal_notify_dedupe_seconds (default 7200s), so a regression that clears and recurs within the window is not re-notified (expected: a cleared regression lifts the red signal and a new episode resets the key). The notification layer's purpose-dedup is now a belt-and-suspenders rather than the sole guard. -- SelfHealEngine._originate dedupes by the fingerprint carried in orchestration_markers (extract_self_heal_fingerprint); ci_watch and dep_update instead dedupe by git_url via list_open_*_tasks(git_url=...). The two mechanisms are independent — a self-heal task and a ci_watch task for the same repo are NOT deduped against each other (different source tags). -- Self-heal tasks are created HELD (confirmed_by_human=False) and excluded from give_me_work until the CEO approves; ci_watch and dep_update tasks are created READY (confirmed_by_human=True) and dispatch immediately. A wrong flag here would either strand a fix or auto-dispatch a held one. -- The engines only flush; the orchestrator loop commits. An exception between flush and commit (or a crashed loop iteration logged but swallowed at orchestrator.py 'cycle failed') loses the opened task rows — but they were already flushed into the session that is rolled back on the next get_db_context exit. -- CiWatchEngine._should_open now dedupes per (git_url, effective workflow): a same-workflow monorepo (several cell-projects on one repo) still collapses to one fix task, but two RED workflows of the same repo each get their own fix task (#44, fixed in 536bbb64). The effective workflow is ci_watch_workflow falling back to ci_watch_default_workflow; an empty-string ci_watch_workflow is treated as NULL via SQL NULLIF (d34bc1a7) so it correctly collapses to the default rather than opening a spurious second task. -- DepUpdateEngine._eligible orders cheap checks (command set, id, git_url dedupe) before the expensive dry_upgrade_changes_lockfile probe — but the per-cycle and rolling caps in run_cycle are checked BEFORE _eligible, so a project that fails eligibility still consumed a loop slot but did not consume a cap slot. -- _fingerprint hashes only signal_name (which 'already encodes the repo') — if two distinct regressions share a signal_name on the same repo they collide and the second is deduped away. -- Cell PM notification (ci_watch) is best-effort and catches Exception broadly; a notification failure logs a warning but never rolls back the already-flushed task, so a fix task can exist with no PM ping. -- _cell_pm_slug_for iterates _foundation.AGENTS.values() looking for role==CELL_PM and team==team; if the org config has no cell PM for an assigned_cell, the notification is silently skipped (pm_slug None -> return). -- SelfHealEngine.assess is pure but run_cycle constructs NotificationService() with no session — relies on NotificationService resolving its own session; if it ever needs the engine's session the wiring would break. - - -## Drift from CLAUDE.md -- CLAUDE.md says ci_watch 'reuses the exact hardened per-project GitService.get_latest_ci_conclusion' — the engines themselves do not call GitService; they consume breaches via MultiProjectCITelemetrySource.fetch(projects) in roboco/services/telemetry/source.py. The GitService call is inside the telemetry source, not in ci_watch_engine.py. Minor framing drift, not a code bug. -- CLAUDE.md says dep-update 'Detection is read-only ... WorkspaceService.dry_upgrade_changes_lockfile runs the project's dep_update_command in a throwaway clone of the read clone'. The engine calls self._workspace.dry_upgrade_changes_lockfile(project) but the engine file itself does not reference a 'read clone'; that detail lives in WorkspaceService. Accurate at the system level, not visible in this slice. -- CLAUDE.md states self-heal 'terminates at awaiting_ceo_approval'. The engine itself only creates a PENDING confirmed_by_human=False task; the awaiting_ceo_approval terminal is reached later by the normal lifecycle, not by any code in self_heal_engine.py. Consistent but the engine does not enforce the terminal state itself. - - -## Changes Since Baseline - -| SHA | Subject | Impact | -|---|---|---| -| 15effce0 | Chore: 141 Gaps fill-in (#283) | Single commit touching all three engine files (self_heal +45/-, ci_watch 20 lines tweaked, dep_update 14 lines). Docstring/comment tightening and minor structural cleanup across the three engines — no behavior change to the originate/dedupe/cap logic. Diffstat: 46 insertions, 33 deletions across the three files. | - -> Post-snapshot updates (since 2026-06-29): -> - **536bbb64** (Chore/all/logical gaps sweep, #286) — two behavior changes to engine files: (1) self_heal_engine.py: added per-fingerprint Redis CEO-notify dedupe (_NOTIFY_DEDUPE_KEY_PREFIX constant + _open_self_heal_task_ids_by_fp / _already_notified / _mark_notified / _dedupe_key methods); run_cycle now skips a CEO ping when the fingerprint was already notified this episode; also links the alert to the open fix task via task_id. LOC grew 227→310. (2) ci_watch_engine.py: _should_open now dedupes per (git_url, effective workflow) instead of just git_url — two red workflows of one monorepo each get their own fix task. -> - **d34bc1a7** ([chore] ci-watch/dep-update dedupe: normalize git_url + treat empty-string workflow as default, #148 #1267) — touched task.py and orchestrator.py (NOT the engine files directly): list_open_ci_watch_tasks and list_open_dep_update_tasks now normalize git_url via repo_key SQL mirror (regexp_replace/rtrim/lower) so URL accidentals (.git suffix, trailing slash, case) don't defeat the one-open-task-per-repo invariant; ci_watch workflow dedupe wraps with NULLIF so an empty-string ci_watch_workflow collapses to the default instead of opening a duplicate task every red cycle. - -## Regression Risks - -| Title | File:Line | Claim | Severity | -|---|---|---|---| -| ~~Self-heal CEO notification spam — no engine-level dedupe~~ **RESOLVED 536bbb64** | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/self_heal_engine.py:103 | ~~run_cycle notifies the CEO for EVERY observation EVERY cycle while a regression stays red.~~ Fixed in 536bbb64 (logical-gaps sweep): run_cycle now dedupes per fingerprint via Redis (_already_notified / _mark_notified with self_heal_notify_dedupe_seconds TTL, default 7200s). A persistent red regression pings the CEO once per episode; the check fails open (Redis outage = notify fires anyway). | medium | -| ~~ci_watch per-(repo,workflow) collapse vs per-git_url dedupe under-counts multi-workflow monorepos~~ **RESOLVED 536bbb64** | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/ci_watch_engine.py:137 | ~~_should_open dedupes by git_url only~~ Fixed in 536bbb64: _should_open now dedupes per (git_url, effective workflow), so two red workflows of one monorepo each get their own fix task. The d34bc1a7 companion normalizes git_url with repo_key in the DB query and adds NULLIF for empty-string workflows so the SQL matches Python truthiness collapse. | medium | -| Cap-check ordering in dep_update lets a non-eligible project consume a loop slot but not a cap slot | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/dep_update_engine.py:60 | run_cycle checks per-cycle/rolling caps BEFORE _eligible; a project that fails eligibility (no lockfile change) does not increment open_count, so caps are only consumed by real originations. Correct, but means the expensive dry_upgrade_changes_lockfile probe runs on every eligible project each cycle regardless of how many tasks already opened this cycle until the cap is hit — minor wasted probe cost, not a correctness bug. | low | -| ci_watch cell-PM notify swallows all exceptions after task creation | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/ci_watch_engine.py:129 | _notify_cell_pm catches Exception broadly and only logs a warning. A task is already flushed before the notify, so a notification failure leaves an orphan fix task with no PM ping. Best-effort by design, but the broad except could mask a persistent notification-service outage as a series of warnings. | low | -| Self-heal fingerprint collision on shared signal_name | /Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco/roboco/services/self_heal_engine.py:65 | _fingerprint hashes only signal_name. Two distinct regressions on the same repo with the same signal_name collide; the second is deduped away and never gets a fix task. Unlikely in practice but a latent correctness gap. | low | - -## Health -All three engines are small, single-purpose, and follow a deliberately conservative pattern: gate on a default-off flag, read-only detect, bounded+deduped originate of one PENDING task, flush-only (caller commits), never start/approve/merge/deploy. The safety invariants (self_heal HELD behind CEO approve; ci_watch/dep_update READY but ride normal gates; per-git_url dedupe for monorepos; per-cycle + rolling caps) are intact and consistent with CLAUDE.md. Two medium risks present at baseline are now resolved: CEO notify spam (536bbb64 added Redis per-fingerprint dedupe) and multi-workflow monorepo under-count (536bbb64 changed _should_open to dedupe per (git_url, workflow); d34bc1a7 hardened the SQL to normalize git_url accidentals and treat empty-string workflow as NULL). Remaining standing risks are low-severity: fingerprint collision on shared signal_name, dep_update cap-check ordering (non-eligibles consume a loop slot not a cap slot), and ci_watch cell-PM notify swallowing all exceptions. Health is good. - -## See also -- `docs/map/engine-docs-sync.md` — a sibling originate-only engine that opens a docs-update task on release publish (release-triggered, no background loop). - -## Purpose -The gated release manager: a default-off background loop that deterministically assesses RoboCo's own repo (diff-since-tag → conventional-commit classification → semver bump → readiness gaps) and originates ONE held release PROPOSAL task for the CEO; the CEO's panel approve/reject routes call a fail-closed ReleaseExecutor that bumps versions, runs `make quality`, commits `chore(release): X.Y.Z`, waits for green release-commit CI, and `gh release create`s — aborting before commit on a red gate and before publish on red CI. It never auto-merges or auto-deploys; the CEO is the only actor who can trigger a publish. - -**Env-ladder era.** Release ops now target the project's env-ladder **prod rung** (`roboco.models.env_branches.prod_branch`) instead of the raw `projects.default_branch` column — a project with no declared ladder resolves to the same value via the read-time shim, so single-branch projects are unaffected. Before bumping, the executor runs a **full-chain promotion** (`promote_env_chain`) that merges every rung between head and prod, in order, into the prod checkout — the release commits + tags the promoted state, not just prod's own prior tip — and aborts fail-closed (`promotion_failed`) before any bump on a fetch/merge conflict. `release_readiness` diffs `prod..head` (falling back to `last_tag..HEAD` when the prod rung can't be resolved) and cross-checks the last tag against the prod tip (`_tag_drift_gaps`) to flag a hotfix that landed on prod outside the ladder. - -## Files - -| Path | Role | LOC | -|---|---|---| -| roboco/services/release_executor.py | Fail-closed bump→gate→commit/push→CI→publish orchestrator with a Protocol seam (ReleaseOps) over a writable token-authenticated clone (_GitReleaseOps); idempotent on already-published versions; half-landed (publish_failed) retry skips re-bump/re-commit. | 490 | -| roboco/services/release_proposal.py | CEO approve/reject glue over the single held proposal task; approve runs `_approve_precheck` then dispatches the ~40min executor as a background asyncio task (returns 202 immediately); heartbeat-guarded Redis fencing-token mutex; closes proposal on published OR already_published; approve refuses a CANCELLED (`already_rejected`) or COMPLETED (`already_published`) proposal before ever touching the lock; reject records required changes and keeps it held, raising `TaskAlreadyCompletedError` if the proposal already published. | 547 | -| roboco/services/release_readiness.py | Pure conventional-commit classification + semver-derivation primitives + the git/filesystem snapshot gatherer + the assess() report builder; serializes/deserializes the report for JSONB storage on the proposal task. | 572 | -| roboco/services/release_manager_engine.py | Default-off detection loop: per interval, if no proposal is open and the gate is green and changes past threshold, originate ONE PENDING HELD Secretary-owned proposal carrying the readiness report; never publishes. | 239 | - -## Key Symbols - -| Name | Kind | File:Line | Responsibility | -|---|---|---|---| -| ReleaseResult | dataclass | roboco/services/release_executor.py:36 | Frozen outcome of an execute attempt: status (published/gate_failed/ci_failed/commit_failed/publish_failed/already_published/already_in_progress/lock_lost/redis_unavailable), version, files_changed, commit_sha, release_url, detail. | -| ReleaseOps | Protocol | roboco/services/release_executor.py:48 | Side-effecting release steps (is_already_published, release_commit_sha, apply_version_bumps, write_changelog_entry, run_gate, commit_and_push, wait_for_ci, publish_release) injected so the fail-closed ordering is unit-testable. | -| ReleaseExecutor | class | roboco/services/release_executor.py:70 | Orchestrates the fail-closed release pipeline over a ReleaseOps, aborting on any red step and returning a ReleaseResult. | -| ReleaseExecutor.execute | method | roboco/services/release_executor.py:76 | Bump→gate→commit/push→CI→publish; short-circuits on already_published; detects half-landed (publish_failed) retry via release_commit_sha and rejoins CI→publish tail without re-bumping; returns gate_failed/ci_failed/commit_failed/publish_failed ReleaseResult on red steps. | -| _await_proc | function | roboco/services/release_executor.py:208 | Communicate with a subprocess under a deadline; on timeout kill() the child, await proc.wait() to reap it (no zombie), and return non-zero rc so fail-closed branches fire instead of hanging the release loop. | -| _ReleaseContext | dataclass | roboco/services/release_executor.py:259 | Writable-clone coordinates: slug, prod_branch (the env-ladder prod-rung target, resolved via `roboco.models.env_branches.prod_branch`), root Path, git_url, git_prefix, ci_workflow, env_chain (the head→…→just-below-prod rung branches to promote; empty for a degenerate head==prod ladder). | -| _GitReleaseOps.promote_env_chain | method | roboco/services/release_executor.py:364 | Full-chain promotion: fetch origin, then merge (`--no-edit`) each `env_chain` branch into the prod checkout in head-first order before the bump; fail-closed RuntimeError on a fetch or merge-conflict; no-op for an empty chain (degenerate ladder). | -| _GitReleaseOps | class | roboco/services/release_executor.py:238 | Production ReleaseOps on a fresh token-authenticated writable clone: real git/make/gh with per-step subprocess deadlines. | -| _GitReleaseOps._git | method | roboco/services/release_executor.py:249 | Run a git -C command under _GIT_OP_TIMEOUT_SECONDS, returning (rc, stdout). | -| _GitReleaseOps.is_already_published | method | roboco/services/release_executor.py:260 | git ls-remote --tags origin v; true if the tag already exists (idempotency guard). | -| _GitReleaseOps.release_commit_sha | method | roboco/services/release_executor.py:264 | Half-landed detection: if the clone's working version == target version AND a `chore(release): {version}` commit appears in the recent log, return its sha (publish_failed retry → skip re-bump); else None. | -| _GitReleaseOps._current_version | method | roboco/services/release_executor.py:287 | Read the version string out of pyproject.toml (the old value for the bump replace). | -| _GitReleaseOps.apply_version_bumps | method | roboco/services/release_executor.py:292 | Replace old version with new across the bump plan, skipping CHANGELOG.md and bumping uv.lock only in the roboco package block. | -| _GitReleaseOps.write_changelog_entry | method | roboco/services/release_executor.py:315 | Insert the drafted CHANGELOG entry above the first released version heading. | -| _GitReleaseOps.run_gate | method | roboco/services/release_executor.py:320 | Run `make quality` in the clone under _RELEASE_GATE_TIMEOUT_SECONDS; return rc==0. | -| _GitReleaseOps.commit_and_push | method | roboco/services/release_executor.py:336 | git add -A, commit -S chore(release): , rev-parse HEAD, push HEAD:default_branch; raises RuntimeError on add/commit/push failure. | -| _GitReleaseOps.wait_for_ci | method | roboco/services/release_executor.py:359 | Poll GitService.get_latest_ci_conclusion for the slug up to 80×30s (~40min), requiring head_sha match + success conclusion. | -| _GitReleaseOps.publish_release | method | roboco/services/release_executor.py:378 | gh release create v --target default_branch; raises RuntimeError on non-zero rc (caught by execute → publish_failed), returns the release URL. | -| _bump_uv_lock | function | roboco/services/release_executor.py:407 | Bump only the roboco package version block inside uv.lock so a same-versioned dependency is not clobbered. | -| _insert_changelog_entry | function | roboco/services/release_executor.py:415 | Insert the new entry above the first ## [] heading (Keep a Changelog format). | -| _resolve_release_ci_workflow | function | roboco/services/release_executor.py:427 | Return settings.release_ci_workflow or "ci.yml" — decoupled from self_heal_ci_workflow; never returns None or empty, so the release gate always scopes to a named workflow. | -| get_release_executor | function | roboco/services/release_executor.py:441 | Build a ReleaseExecutor over a fresh writable clone: resolve the RoboCo project, decrypt token, inject into URL, _prepare_release_clone; ci_workflow set from _resolve_release_ci_workflow(). | -| _prepare_release_clone | function | roboco/services/release_executor.py:467 | rm -rf and re-clone the release clone at workspaces_root/_release/ on the default branch. | -| _run | function | roboco/services/release_executor.py:483 | Run a subprocess under _CLONE_TIMEOUT_SECONDS via _await_proc (used by _prepare_release_clone). | -| ReleaseLockUnavailable | exception | roboco/services/release_proposal.py:40 | Distinct from "lock is held" — Redis itself is unreachable (infra failure, not a concurrent approve). Both paths are fail-closed but the error surface differs. | -| TaskAlreadyCompletedError | exception | roboco/services/release_proposal.py:49 | Raised by `reject()` when the proposal is already COMPLETED (published) — a stale reject (e.g. a queued Telegram button on a proposal a concurrent approve already shipped) can't cancel a release that already happened. | -| ReleaseProposalService | class | roboco/services/release_proposal.py:77 | Find/approve/reject the single open release proposal; approve dispatches the executor as a background asyncio task (202), with a heartbeat-guarded fencing-token Redis mutex. | -| ReleaseProposalService.open_proposal | method | roboco/services/release_proposal.py:98 | Return the first non-terminal release_manager-source task or None. | -| ReleaseProposalService._approve_precheck | method | roboco/services/release_proposal.py:103 | Resolve the proposal + stored report, or a canned refusal: CANCELLED (the CEO already rejected it) returns an `already_rejected` ReleaseResult, COMPLETED (already published) returns `already_published` — both WITHOUT ever touching the Redis lock/executor. Split out of `approve()` to keep its own return-count bounded as more terminal-state guards are added. | -| ReleaseProposalService.approve | method | roboco/services/release_proposal.py:165 | Runs `_approve_precheck` first (returns its canned refusal on a terminal-state proposal); otherwise acquires the Redis fencing-token mutex (raises ReleaseLockUnavailable on Redis outage → returns redis_unavailable; returns already_in_progress if lock held); runs executor as asyncio.Task guarded by a heartbeat; marks COMPLETED on published OR already_published; returns lock_lost if heartbeat cancels execute on TTL expiry. | -| ReleaseProposalService._finalize_release_lock | method | roboco/services/release_proposal.py:191 | finally-block: cancel heartbeat/execute tasks and compare-and-del the release mutex. | -| ReleaseProposalService._acquire_release_lock | method | roboco/services/release_proposal.py:207 | SET NX EX the release mutex with a fencing-token value; returns token if acquired, None if held (concurrent approve); raises ReleaseLockUnavailable if Redis is unreachable (caller surfaces redis_unavailable, not already_in_progress). | -| ReleaseProposalService._release_release_lock | method | roboco/services/release_proposal.py:230 | Compare-and-del the release mutex via Lua CAS — only deletes if the key still holds our fencing token, so a late first-finally can't delete a usurper's lock. | -| ReleaseProposalService._heartbeat_release_lock | method | roboco/services/release_proposal.py:241 | Compare-and-expire the release mutex (Lua); returns True if we still own it. | -| ReleaseProposalService._heartbeat_loop | method | roboco/services/release_proposal.py:256 | Refreshes lock TTL while execute is running; if the lock is no longer ours (>TTL Redis outage let it expire), sets lock_lost and cancels execute fail-closed. | -| ReleaseProposalService.reject | method | roboco/services/release_proposal.py:414 | Record the CEO's required_changes marker on the proposal; keep it held for revision. Raises TaskAlreadyCompletedError when the proposal is already COMPLETED (published) — a stale reject can't lie about an already-public release's real state. | -| get_release_proposal_service | function | roboco/services/release_proposal.py:442 | Construct a ReleaseProposalService bound to a session. | -| dispatch_approve | function | roboco/services/release_proposal.py:538 | Spawn the ~40min release execute as a background asyncio.Task (registered in _INFLIGHT_APPROVES) so the HTTP route returns 202 immediately; done-callback removes the entry. | -| _run_approve_background | function | roboco/services/release_proposal.py:490 | Run approve() in a background task with a fresh session (the request session closes on the 202 response); commits on success, rolls back and logs on failure. | -| CommitInfo | dataclass | roboco/services/release_readiness.py:89 | One commit since the last release tag: sha, subject, body, pr_number, labels. | -| ClassifiedChange | dataclass | roboco/services/release_readiness.py:100 | A commit annotated with normalized kind, breaking flag, summary, needs_manual_classification. | -| _has_breaking_label | function | roboco/services/release_readiness.py:111 | True if any PR label is in the breaking-label set. | -| _classify_one | function | roboco/services/release_readiness.py:115 | Classify a commit by conventional-commit prefix, then PR-label fallback, then needs_manual_classification. | -| classify_changes | function | roboco/services/release_readiness.py:152 | Map classify_one over a list of commits. | -| derive_bump | function | roboco/services/release_readiness.py:157 | Reduce the change set to a semver bump: breaking→major, feat→minor, else patch. | -| next_version | function | roboco/services/release_readiness.py:166 | Apply a bump to a MAJOR.MINOR.PATCH string (leading v ok). | -| Gap | dataclass | roboco/services/release_readiness.py:176 | One readiness shortfall (category, detail) the CEO must see before approving. | -| ReleaseRepoSnapshot | dataclass | roboco/services/release_readiness.py:185 | Raw read-only release facts: version, last_tag, commits, version-ref files, canonical bump files, changelog, migrations, CI conclusion, agent counts, verb-tables-stale flag. | -| ReleaseReadinessReport | dataclass | roboco/services/release_readiness.py:207 | The deterministic CEO-reviewable proposal: proposed_version, bump_kind, change_summary, drafted_changelog, version_bump_plan, gaps, migration_notes, gate_state. | -| _is_documented | function | roboco/services/release_readiness.py:221 | True if a change's PR number or summary text appears in the CHANGELOG. | -| _draft_changelog | function | roboco/services/release_readiness.py:228 | Build a Keep-a-Changelog ## [version] - date block with Added/Changed/Fixed/Security sections. | -| _changelog_gaps | function | roboco/services/release_readiness.py:246 | Flag feat/fix/security/perf/refactor changes not present in the CHANGELOG. | -| _version_ref_gaps | function | roboco/services/release_readiness.py:259 | Flag files embedding the current version but not in the canonical bump plan. | -| _docs_drift_gaps | function | roboco/services/release_readiness.py:271 | Flag declared-vs-actual agent-count mismatch and stale verb-surface tables. | -| _migration_gaps_and_notes | function | roboco/services/release_readiness.py:291 | Emit migration run-notes for new migrations and a gap if there is >1 alembic head. | -| _gate_state | function | roboco/services/release_readiness.py:312 | Map a CI conclusion to green/unknown/red. | -| assess | function | roboco/services/release_readiness.py:320 | Turn a snapshot into a gap-flagged ReleaseReadinessReport (classify → derive bump → next version → assemble gaps → draft changelog). | -| _run_git | function | roboco/services/release_readiness.py:365 | Synchronous git subprocess helper (capture stdout, no check). | -| _pyproject_version | function | roboco/services/release_readiness.py:375 | Read the version out of pyproject.toml. | -| _last_tag | function | roboco/services/release_readiness.py:381 | git describe --tags --abbrev=0 (most recent tag) or None. | -| _commits_since | function | roboco/services/release_readiness.py:386 | git log ..HEAD with record/field separators; parse into CommitInfo with PR-number extraction. | -| _tracked_files_with_version | function | roboco/services/release_readiness.py:408 | git grep -lF excluding tests/ — files embedding the version string. | -| _canonical_bump_files | function | roboco/services/release_readiness.py:415 | Derive the bump set from the previous chore(release): commit's files (subject-filtered), falling back to the version-ref scan on first release. | -| _new_migrations | function | roboco/services/release_readiness.py:447 | git diff --name-only --diff-filter=A ..HEAD -- alembic/versions/. | -| _migration_head_count | function | roboco/services/release_readiness.py:464 | Parse alembic/versions/*.py revision/down_revision lines; count unreferenced heads. | -| _declared_agent_count | function | roboco/services/release_readiness.py:484 | Regex the 'N AI agents' declaration out of roboco/__init__.py. | -| _actual_agent_count | function | roboco/services/release_readiness.py:493 | Count non-system/non-ceo rows in foundation.identity.AGENTS (best-effort). | -| gather_snapshot | function | roboco/services/release_readiness.py:505 | Build a ReleaseRepoSnapshot from a real checkout (read-only); verb_tables_stale left False (regen would write). | -| _read_changelog | function | roboco/services/release_readiness.py:534 | Read CHANGELOG.md or '' on OSError. | -| report_to_dict | function | roboco/services/release_readiness.py:541 | Serialize a report to a plain dict for JSONB marker storage. | -| report_from_dict | function | roboco/services/release_readiness.py:557 | Rebuild a report from its stored dict (inverse of report_to_dict). | -| ReleaseAssessor | type alias | roboco/services/release_manager_engine.py:58 | Callable[[], Awaitable[ReleaseReadinessReport / None]] — injectable assessor (default production, tests synthetic). | -| _roboco_slug | function | roboco/services/release_manager_engine.py:61 | The registered project that IS RoboCo itself (self_heal_project_slug or 'roboco-api'). | -| _past_threshold | function | roboco/services/release_manager_engine.py:66 | True when commit count >= release_min_commits OR bump is non-patch OR any security change. | -| _proposal_description | function | roboco/services/release_manager_engine.py:75 | Human-readable proposal body: version, bump, change count, gate, drafted CHANGELOG, gaps, migrations. | -| ReleaseManagerEngine | class | roboco/services/release_manager_engine.py:95 | Detect release-readiness and originate ONE CEO-gated held proposal; never publishes. | -| ReleaseManagerEngine.run_cycle | method | roboco/services/release_manager_engine.py:106 | No-op unless enabled; if no proposal open, ready_report, resolve project, originate. | -| ReleaseManagerEngine._ready_report | method | roboco/services/release_manager_engine.py:130 | Assess; return report only when gate is green and past threshold, else None. | -| ReleaseManagerEngine._originate | method | roboco/services/release_manager_engine.py:149 | Create a PENDING HELD Secretary-owned ADMINISTRATIVE task with the report marker; notify CEO. | -| ReleaseManagerEngine._notify_ceo | method | roboco/services/release_manager_engine.py:188 | Best-effort ack-notification to the CEO summarizing the proposal (never blocks origination). | -| ReleaseManagerEngine._production_assess | method | roboco/services/release_manager_engine.py:206 | Real path: ensure read clone, fetch CI conclusion, gather_snapshot, assess; None on any resolution failure. | -| get_release_manager_engine | function | roboco/services/release_manager_engine.py:234 | Build a ReleaseManagerEngine with optional injected assessor. | - -## Data Flow -DETECT loop: the orchestrator spawns `_release_manager_loop` (an asyncio task started in `start()`) which, when `release_manager_enabled`, sleeps `release_manager_interval_seconds` then calls `_run_release_manager_cycle` → opens a DB session → `get_release_manager_engine(db).run_cycle()`. `run_cycle` short-circuits if disabled, if `TaskService.list_open_release_proposals()` already returns one (dedup by `source='release_manager'` + non-terminal status), or if `_ready_report()` returns None. `_ready_report` calls the injected assessor (default `_production_assess`): resolve the RoboCo project by `self_heal_project_slug`, `WorkspaceService.ensure_read_clone` (pinned to the **head** rung's HEAD), `GitService.get_latest_ci_conclusion`, then best-effort fetch the project's **prod** rung into that head-pinned read clone (`_ensure_prod_fetched` — a no-op when prod==head; a fetch failure degrades to the `last_tag..HEAD` baseline rather than aborting) so `origin/` resolves, then `gather_snapshot(read_clone_root, master_ci_conclusion, prod_branch=prod_for_snapshot)` + `assess(snapshot, today)`. `assess` runs `classify_changes` → `derive_bump` → `next_version` → assembles gaps (changelog, version_ref, docs_drift, migration, classification, gate). If green + past threshold, `_originate` creates a PENDING HELD `RELEASE_MANAGER_SOURCE` task owned by `secretary-1` via `TaskService.create(TaskCreateRequest(..., confirmed_by_human=False))`, stores the report dict via `markers.set_release_report`, flushes, and best-effort notifies the CEO. The orchestrator cycle commits the session. - -CEO ACT path: `GET /api/release/proposal` (CEO-only) → `ReleaseProposalService.open_proposal()` → `list_open_release_proposals()[0]`. `POST /proposal/approve` (returns 202 immediately): route calls `dispatch_approve(task_id, session_factory)` which spawns `_run_approve_background` as a background asyncio.Task (the request session closes at the 202 return; the background task opens a fresh session). In `approve(task_id)`: loads the task, verifies `source == RELEASE_MANAGER_SOURCE`, reads `markers.get_release_report`; acquires Redis fencing-token mutex (`SET NX EX 3000`) — raises `ReleaseLockUnavailable` on Redis outage → returns `redis_unavailable`; returns `already_in_progress` if lock is held. Then: `get_release_executor(session)` → `executor.execute(report)` run as `asyncio.Task` while a `_heartbeat_loop` task refreshes the TTL every 60s (cancels execute and returns `lock_lost` if the lock is no longer ours). `get_release_executor` resolves the project + token, clones at the **prod rung** (`roboco.models.env_branches.prod_branch`) — not the raw `default_branch` column — computes `env_chain` via `promotion_chain(project)` (the head→…→just-below-prod rungs to promote; empty for a degenerate head==prod ladder), and injects the token as a per-call `http.extraheader` (never into argv); `ci_workflow` is set from `_resolve_release_ci_workflow()` (not self_heal_ci_workflow). `execute`: `is_already_published` (ls-remote tag); `release_commit_sha` (half-landed check — if prior release commit on branch, skip re-bump and rejoin CI→publish tail); else `_run_fresh_release`: `promote_env_chain` (fetch origin + merge each `env_chain` branch into the prod checkout head-first; a fetch/merge failure aborts fail-closed with `promotion_failed` before any bump) → `apply_version_bumps` (replace old version across plan, uv.lock scoped) + `write_changelog_entry` → `run_gate` (make quality, 1800s) → `commit_and_push` (add -A, commit -S, push HEAD:prod_branch; RuntimeError → `commit_failed`) → `wait_for_ci` (poll GitService 80×30s, scoped to release_ci_workflow) → `publish_release` (REST POST to the GitHub releases API — the orchestrator image ships no `gh` binary; RuntimeError → `publish_failed`). On `published` OR `already_published`, the proposal task is set COMPLETED + flushed, and the publish-success path then hands the release to the best-effort post-publish hooks: `_draft_x_post(report)`, `_draft_video(report)`, and `_draft_docs_update(report)`. Each hook catches `Exception` broadly and logs a warning so a drafting/origination failure never affects the already-succeeded release. `_draft_docs_update` invokes `DocsSyncEngine.originate_docs_update(version=report.proposed_version, changelog=report.drafted_changelog)`; if `ROBOCO_DOCS_SYNC_ENABLED` is on and `roboco-website` is registered, exactly one PENDING Main-PM docs-update task is created for that release tag. On gate/CI/commit/publish failure a ReleaseResult is returned and the proposal stays open. The background task commits the session on success, rolls back on failure. The panel polls `GET /proposal` for the final status. `POST /proposal/reject` → `svc.reject(task_id, required_changes)` writes `markers.set_release_required_changes` and keeps the task held. - -## Mermaid -```mermaid -sequenceDiagram - participant MLoop as _release_manager_loop - participant Eng as ReleaseManagerEngine - participant Ready as release_readiness - participant TS as TaskService - participant CEO as CEO (panel) - participant Route as /api/release routes - participant Prop as ReleaseProposalService - participant Exec as ReleaseExecutor - participant Ops as _GitReleaseOps - - MLoop->>Eng: run_cycle() - Eng->>TS: list_open_release_proposals() - alt none open - Eng->>Ready: _production_assess() (read clone + CI) - Ready-->>Eng: ReleaseReadinessReport - Eng->>Eng: _ready_report (green gate + past threshold) - Eng->>TS: create(PENDING, HELD, secretary-1, source=release_manager) - Eng->>TS: markers.set_release_report(report) - Eng-->>CEO: notify (best-effort) - end - - CEO->>Route: POST /api/release/proposal/approve - Route->>Prop: approve(task_id) - Prop->>Prop: acquire Redis SET NX EX mutex - alt mutex held / Redis down - Prop-->>Route: already_in_progress - else acquired - Prop->>Exec: get_release_executor + execute(report) - Exec->>Ops: is_already_published? - Ops->>Ops: apply_version_bumps + write_changelog - Ops->>Ops: run_gate (make quality) - alt gate red - Exec-->>Prop: gate_failed (proposal stays open) - else gate green - Ops->>Ops: commit -S + push - Ops->>Ops: wait_for_ci (poll) - alt CI red - Exec-->>Prop: ci_failed (proposal stays open) - else CI green - Ops->>Ops: gh release create - Exec-->>Prop: published - Prop->>TS: task.status = COMPLETED - end - end - Prop->>Prop: release mutex (finally) - end -``` - -## Logical Tree -``` -release-manager slice -├── release_manager_engine.py (detect loop, default-off) -│ ├── ReleaseManagerEngine -│ │ ├── run_cycle (gate + dedup + originate) -│ │ ├── _ready_report (assess → green + threshold filter) -│ │ ├── _originate (create HELD proposal task + report marker + CEO notify) -│ │ └── _production_assess (read clone + CI conclusion + gather_snapshot + assess) -│ ├── _past_threshold (commit floor OR non-patch OR security) -│ └── _proposal_description (human-readable proposal body) -├── release_readiness.py (pure readiness + git snapshot) -│ ├── primitives -│ │ ├── classify_changes / _classify_one (conventional-commit + label fallback) -│ │ ├── derive_bump (breaking>feat>patch) -│ │ └── next_version (semver apply) -│ ├── report builder -│ │ ├── assess (snapshot → gaps + drafted changelog + plan) -│ │ ├── _changelog_gaps / _version_ref_gaps / _docs_drift_gaps / _migration_gaps_and_notes / _gate_state -│ │ └── _draft_changelog -│ ├── gather_snapshot (read-only git + filesystem I/O) -│ │ ├── _pyproject_version / _last_tag / _commits_since -│ │ ├── _tracked_files_with_version (excludes tests/) -│ │ ├── _canonical_bump_files (prev chore(release): files, subject-filtered, first-release fallback) -│ │ ├── _new_migrations / _migration_head_count -│ │ └── _declared_agent_count / _actual_agent_count -│ └── report_to_dict / report_from_dict (JSONB marker ser/de) -├── release_proposal.py (CEO approve/reject glue) -│ └── ReleaseProposalService -│ ├── open_proposal -│ ├── approve (Redis mutex → executor → COMPLETED on publish) -│ │ ├── _acquire_release_lock (SET NX EX, fail-closed) -│ │ └── _release_release_lock (DEL, best-effort) -│ └── reject (record required_changes, keep held) -└── release_executor.py (fail-closed publish pipeline) - ├── ReleaseExecutor.execute (bump→gate→commit→CI→publish, abort on red) - ├── ReleaseOps Protocol (test seam) - ├── _GitReleaseOps (production: real git/make/gh with deadlines) - │ ├── is_already_published / apply_version_bumps / write_changelog_entry - │ ├── run_gate / commit_and_push / wait_for_ci / publish_release - │ └── _bump_uv_lock / _insert_changelog_entry helpers - ├── _await_proc (subprocess deadline + kill-on-timeout) - └── get_release_executor / _prepare_release_clone (writable clone bootstrap) -``` - -## Dependencies -- Internal: roboco.config.settings (release_manager_enabled, release_min_commits, release_manager_interval_seconds, self_heal_project_slug, self_heal_ci_workflow, workspaces_root, redis_url), roboco.models.env_branches (head_branch, prod_branch, promotion_chain — the env-ladder resolvers backing the release clone/commit/tag target, the full-chain promotion, and the readiness diff baseline), roboco.services.task.TaskService / TaskCreateRequest / RELEASE_MANAGER_SOURCE / get_task_service, roboco.services.project.ProjectService / get_project_service, roboco.services.workspace.WorkspaceService / get_workspace_service / ensure_read_clone / _inject_token_into_url, roboco.services.git.GitService / get_git_service / get_latest_ci_conclusion, roboco.services.notification.NotificationService.send_ack_notification, roboco.services.base.BaseService, roboco.foundation.identity.AGENTS (secretary-1, system), roboco.foundation.policy.content.markers (get_release_report, set_release_report, set_release_required_changes, get_release_required_changes), roboco.models.base (TaskStatus, TaskType, Team, Complexity, TaskNature, AgentRole), roboco.db.tables.TaskTable, roboco.api.routes.release (CEO-only routes), roboco.api.schemas.release, roboco.runtime.orchestrator._release_manager_loop / _run_release_manager_cycle -- External: asyncio (subprocess, wait_for, sleep), subprocess (sync git in release_readiness), pathlib.Path, re, dataclasses, structlog, redis.asyncio, sqlalchemy.ext.asyncio.AsyncSession, fastapi (routes) - -## Entry Points - -| Name | File | Trigger | -|---|---|---| -| _release_manager_loop | roboco/runtime/orchestrator.py | asyncio task created in Orchestrator.start() (line 1063); sleeps release_manager_interval_seconds then _run_release_manager_cycle → get_release_manager_engine(db).run_cycle() | -| GET /api/release/proposal | roboco/api/routes/release.py | CEO panel fetch of the held proposal (CEO-only, 404 when none); panel polls this to get final status after a 202 approve | -| POST /api/release/proposal/approve | roboco/api/routes/release.py | CEO panel approve → dispatch_approve → background _run_approve_background → ReleaseProposalService.approve → ReleaseExecutor.execute (returns 202 immediately; panel polls GET /proposal for outcome) | -| POST /api/release/proposal/reject | roboco/api/routes/release.py | CEO panel reject-with-changes → ReleaseProposalService.reject (keep held) | - -## Config Flags -- ROBOCO_RELEASE_MANAGER_ENABLED (release_manager_enabled, default False) — master switch; when off the loop returns immediately and no proposal is ever originated -- ROBOCO_RELEASE_MIN_COMMITS (release_min_commits, default 8, min 1) — commit floor for the _past_threshold gate -- ROBOCO_RELEASE_MANAGER_INTERVAL_SECONDS (release_manager_interval_seconds, default 3600, min 60) — sleep between assessment passes -- ROBOCO_SELF_HEAL_PROJECT_SLUG (self_heal_project_slug) — reused as the 'this project IS RoboCo' pointer (default 'roboco-api') -- ROBOCO_RELEASE_CI_WORKFLOW (release_ci_workflow, default "ci.yml") — dedicated workflow name for the release fail-closed CI gate; decoupled from ROBOCO_SELF_HEAL_CI_WORKFLOW (which allows empty-string for single-workflow repos — inheriting that would degrade the release gate to the unreliable all-workflows mode); empty or unset falls back to "ci.yml", never None -- ROBOCO_SELF_HEAL_CI_WORKFLOW (self_heal_ci_workflow) — reused as the read-clone CI conclusion in release_manager_engine._production_assess (NOT the executor's release-commit CI gate — that uses ROBOCO_RELEASE_CI_WORKFLOW) -- ROBOCO_WORKSPACES_ROOT (workspaces_root) — base for the read clone and the _release/ writable clone -- ROBOCO_REDIS_URL (redis_url) — the approve-mutex backing store - - -## Gotchas -- [FIXED 05616607+2759edf7] Redis mutex TTL (3000s = 50min) was shorter than the worst-case execute path — RESOLVED by a heartbeat loop (_heartbeat_loop) that calls _heartbeat_release_lock (compare-and-expire Lua) every 60s to refresh the TTL while execute owns the lock. The TTL is now a crash backstop, not a hard ceiling. If the heartbeat detects lock-loss (an extended Redis outage let the TTL expire), it cancels execute fail-closed (lock_lost result) so a concurrent approve can't rm -rf the in-flight clone. -- [FIXED 05616607] _acquire_release_lock formerly returned None on Redis outage causing approve to return already_in_progress — RESOLVED: now raises ReleaseLockUnavailable so approve returns a distinct redis_unavailable result. The CEO knows to fix Redis rather than waiting on a phantom concurrent approve. Still fail-closed (execute never runs). -- [FIXED 2759edf7] _GitReleaseOps.commit_and_push raised RuntimeError that execute did NOT catch → 500. RESOLVED: execute now wraps commit_and_push in try/except RuntimeError and returns a structured commit_failed ReleaseResult. Similarly publish_release RuntimeError now returns publish_failed instead of propagating. -- [FIXED 2759edf7] _await_proc on timeout called proc.kill() but never awaited proc.wait() — zombie risk. RESOLVED: _await_proc now awaits proc.wait() after kill(), and contextlib.suppress(ProcessLookupError) handles already-exited children. -- [FIXED 0bf6c848] The ~40min synchronous HTTP approve blocked the server and 504'd at any proxy. RESOLVED: POST /proposal/approve now returns 202 immediately and dispatches the execute as a background asyncio.Task (dispatch_approve → _run_approve_background with a fresh session). The panel polls GET /proposal for the final status. -- [FIXED 05616607] approve formerly marked COMPLETED only on status=="published" — a retry that finds the tag already published (prior publish whose route 504'd left proposal non-terminal) left it wedged open. RESOLVED: both "published" and "already_published" now close the proposal. -- _canonical_bump_files first-release fallback now returns _tracked_files_with_version (which excludes tests/). Before the change it returned [], so _version_ref_gaps would flag every version-ref file on first release; now the fallback makes planned==tracked so first-release version_ref gaps are silently empty. Intended, but means the first-release gap report is weaker than subsequent releases. -- _canonical_bump_files uses `git log --grep ^chore(release):` then filters by subject prefix; if a real release commit's subject was ever not exactly `chore(release): X.Y.Z` (e.g. a merge-commit subject), it would be skipped and a stale/false bump set used. -- [FIXED 2759edf7] wait_for_ci formerly inherited self_heal_ci_workflow which allows empty/None, risking an all-workflows-mode conclusion on a multi-workflow repo. RESOLVED: the executor now calls _resolve_release_ci_workflow() which always returns a non-empty named workflow (ROBOCO_RELEASE_CI_WORKFLOW or "ci.yml" fallback), so wait_for_ci always scopes the CI poll to a specific workflow. -- The proposal is created with status=PENDING and confirmed_by_human=False but the release-manager loop NEVER cancels it on a subsequent cycle — list_open_release_proposals dedups by non-terminal status, so a stale PENDING proposal blocks all future proposals until the CEO acts. There is no expiry/reaper for an abandoned proposal. -- apply_version_bumps does a naive str.replace(old, new) on every non-uv.lock, non-CHANGELOG file in the plan. If the current version string appears as a substring of an unrelated value in any bump file (e.g. a comment, a path), it gets clobbered. The bump plan is derived from the previous release commit's files, so this is bounded but not surgical. -- _prepare_release_clone rm -rf's workspaces_root/_release/ with no locking at the filesystem level — the Redis mutex is the only guard, and it is per-proposal-task, not per-clone-path. Two different proposals for the same slug (impossible while dedup holds, but dedup is by source+non-terminal, so a CANCELLED + new PENDING could overlap) would race the rm -rf. -- `_approve_precheck`'s two refusals (already_rejected / already_published) exist because a live-reproduced hole let a stale approve resurrect a proposal the CEO had already rejected or that a concurrent approve had already published — the callback surface that made this reachable is Telegram's inline Approve button, which targets a proposal by id regardless of its current status (a stale, still-clickable button in an old chat message), but the same hole was equally reachable by replaying the HTTP route, so the fix is in the service, not the Telegram layer. - -## Changes Since Baseline - -| SHA | Subject | Impact | -|---|---|---| -| 15effce0 | Chore: 141 Gaps fill-in (#283) — release_executor.py | Added per-subprocess deadlines via _await_proc (git 300s, gate 1800s, publish 300s, clone 600s) with kill-on-timeout returning rc=124 so fail-closed branches fire instead of hanging the release loop. commit_and_push now checks add/commit rc and raises RuntimeError on failure (previously fire-and-forget: a failed commit would still push the pre-bump base as the release). | -| 15effce0 | Chore: 141 Gaps fill-in (#283) — release_proposal.py | Added a Redis SET-NX-EX mutex keyed by proposal id around approve() (F013) — a concurrent approve/double-click returns already_in_progress instead of racing on the rm -rf'd writable clone. Fail-closed on Redis outage (treated as held). Proposal still only COMPLETED on status==published. | -| 15effce0 | Chore: 141 Gaps fill-in (#283) — release_readiness.py | _canonical_bump_files signature changed to take `version`; the git-log grep now filters candidates by subject prefix `chore(release):` (was matching any body line that referenced the type, shadowing the real release commit). First-release fallback changed from returning [] to returning the version-ref scan, so the first release now has a real bump plan and no spurious version_ref gaps. | - -> Post-snapshot updates (since 2026-06-29): -> - 536bbb64 Chore/all/logical gaps sweep (#286) — PR merge carrying the sweep commits below. -> - 2759edf7 [B-REL] release executor: idempotent half-landed retry + commit-scoped CI + decoupled workflow — adds `release_commit_sha` to ReleaseOps/`_GitReleaseOps` for half-landed (publish_failed) retry detection (reuse existing release commit, skip re-bump); execute now catches commit_and_push RuntimeError → `commit_failed` and publish_release RuntimeError → `publish_failed`; `_await_proc` now awaits `proc.wait()` after kill (zombie fix); decoupled release CI gate from self_heal_ci_workflow via new `_resolve_release_ci_workflow()` / `settings.release_ci_workflow` (`ROBOCO_RELEASE_CI_WORKFLOW`, default "ci.yml"). -> - 05616607 [chore] logical-gaps: release-proposal already_published closes proposal + heartbeat-lock-loss cancels execute — adds `ReleaseLockUnavailable` exception (Redis outage → `redis_unavailable` result, not `already_in_progress`); fencing-token compare-and-del (`_RELEASE_LOCK_RELEASE_SCRIPT` Lua) + compare-and-expire heartbeat (`_RELEASE_LOCK_HEARTBEAT_SCRIPT` Lua, `_heartbeat_loop`, `_heartbeat_release_lock`); `lock_lost` result when heartbeat detects TTL expiry and cancels execute fail-closed; `_finalize_release_lock` finally helper; approve now closes proposal on `already_published` in addition to `published`. -> - 0bf6c848 [chore] logical-gaps: release approve async dispatch (202) — adds `dispatch_approve` + `_run_approve_background` + `_INFLIGHT_APPROVES` registry; POST /proposal/approve returns 202 immediately; the ~40min execute runs in a background asyncio.Task with a fresh session; panel polls GET /proposal for final status. -> - b3558d4e [chore] complexity: split 5 C-rank blocks to <=B for the xenon gate — refactored large methods in executor/proposal for xenon compliance; no behavior change. -> - 8621d01d / fe9940de / d80dfb8b (#534, env-branches ladder) — replaces `default_branch`-keyed release targeting: `_ReleaseContext` gains `prod_branch` (renamed from `default_branch`) + `env_chain`; `get_release_executor` resolves the clone/commit/tag target via `roboco.models.env_branches.prod_branch` and computes `env_chain` via `promotion_chain`; `_GitReleaseOps` gains `promote_env_chain` (fetch + head-first merge of `env_chain` into the prod checkout, fail-closed `promotion_failed` on conflict), run as the first step of `_run_fresh_release`, before any version bump; `release_readiness.gather_snapshot` takes an optional `prod_branch` and diffs `prod..head` (falling back to `last_tag..HEAD` when unset) with a new `_tag_drift_gaps` check (last-tag sha vs. prod tip); `release_manager_engine._production_assess` best-effort fetches the prod rung into the head-pinned read clone before gathering the snapshot. A project with no declared ladder is unaffected (the shim resolves prod_branch/head_branch to the same `default_branch` value). -> - `11915f36` (PR #551, Telegram V2 security follow-up) — `_approve_precheck` (new) makes `approve()` refuse a CANCELLED proposal (`already_rejected`) or a COMPLETED one (`already_published`) BEFORE touching the Redis lock/executor; `reject()` now raises the new `TaskAlreadyCompletedError` on a COMPLETED proposal instead of silently cancelling an already-public release. Closes a live-reproduced approve-after-reject hole reachable via a stale Telegram Approve button (or a replayed HTTP call). - -## Regression Risks - -| Title | File:Line | Claim | Severity | -|---|---|---|---| -| [RESOLVED 05616607+2759edf7] Redis mutex TTL shorter than worst-case execute — concurrent approve can race the rm -rf clone | roboco/services/release_proposal.py:52 | RESOLVED: heartbeat loop (_heartbeat_loop) refreshes TTL every 60s via compare-and-expire Lua while execute is running; the 3000s TTL is now a crash backstop. On lock-loss (extended Redis outage let TTL expire) the heartbeat cancels execute fail-closed and approve returns lock_lost — a concurrent approve therefore cannot rm -rf the in-flight clone. | high | -| [RESOLVED 2759edf7] commit_and_push RuntimeError unhandled by execute → 500 instead of structured ReleaseResult | roboco/services/release_executor.py:336 | RESOLVED: execute now wraps commit_and_push in try/except RuntimeError and returns commit_failed; publish_release RuntimeError returns publish_failed. All failure paths now surface as structured ReleaseResult, not 500s. | medium | -| [RESOLVED 05616607] Redis outage fully blocks release approval (fail-closed = treat as held) | roboco/services/release_proposal.py:207 | RESOLVED: _acquire_release_lock raises ReleaseLockUnavailable on any Redis exception; approve returns redis_unavailable (distinguished from already_in_progress so the CEO knows to fix Redis rather than waiting). Still fail-closed — execute never runs without the mutex. | medium | -| _canonical_bump_files first-release fallback silences version_ref gaps | roboco/services/release_readiness.py:444 | On first release (no prior chore(release): commit) the bump plan equals _tracked_files_with_version, so _version_ref_gaps emits zero gaps. The CEO no longer sees 'these files hold the version but are not in the bump plan' on the first release — weaker readiness signal, intentional by design (comment explains). | low | -| [RESOLVED 2759edf7] _await_proc leaves zombie on timeout (kill without wait) | roboco/services/release_executor.py:214 | RESOLVED: _await_proc now awaits proc.wait() after proc.kill(); contextlib.suppress(ProcessLookupError) handles already-exited children. | low | - -## Health -The slice is well-structured: deterministic correctness lives in pure primitives (release_readiness) with a Protocol seam (ReleaseOps) making the fail-closed ordering unit-testable, and the detect→originate→hold→CEO-approve→publish separation is clean and matches CLAUDE.md. Post-snapshot hardening rounds (2759edf7, 05616607, 0bf6c848) resolved all four previously-flagged regression risks: (1) the Redis mutex TTL race is closed by a heartbeat loop that keeps the TTL refreshed and aborts execute fail-closed on lock-loss; (2) commit_and_push/publish_release RuntimeErrors are now caught by execute and returned as structured commit_failed/publish_failed results (no 500); (3) Redis outage now returns redis_unavailable (not already_in_progress) so the CEO knows to fix Redis; (4) the zombie-on-timeout is fixed by awaiting proc.wait(). The approve route is now async-202 with a background dispatcher (dispatch_approve). The half-landed (publish_failed) retry path (release_commit_sha) closes the prior gap where a second CEO approve re-inserted the changelog entry and created a duplicate release commit. The release CI gate is decoupled from self_heal_ci_workflow via a dedicated settings.release_ci_workflow. One low-severity known-by-design item remains: first-release version_ref gap suppression (intentional). release_manager_engine.py and release_readiness.py are unchanged since 15effce0. - # RoboCo Slice Map — `engine-docs-sync` Slice key: `engine-docs-sync`. Repo root: `/Users/renzof/Documents/GitHub/ZZZ/roboco-master/roboco`. Scope: `roboco/services/docs_sync_engine.py`, the docs-sync touch points in `roboco/services/release_proposal.py` and `roboco/services/task.py`, and the release-version marker in `roboco/foundation/policy/content/markers.py`. @@ -8254,391 +9047,16 @@ The flag is registered in `roboco/services/settings.py`'s `FEATURE_FLAGS` tuple, The engine is intentionally small and conservative: default-off, no background loop, bounded, deduped, and flush-only. It follows the same safety model as the other autonomy engines (never start/approve/merge/deploy) while staying out of the orchestrator's periodic loops entirely. The main dependency on operator action is registering `roboco-website` as a project before enabling the flag. Health is good. -## Purpose - -The RoboCo video engine: a default-off subsystem that authors bespoke short marketing videos (release announcements, feature spotlights, on-demand CEO briefs) and distributes them to X and TikTok — nothing renders or posts without the flags on, and nothing posts without an explicit per-clip CEO approval. It mirrors the `XEngine` / `ReleaseManagerEngine` held-artifact shape, but splits across the real delivery lifecycle: a normal ASSIGNED UX/UI authoring task ships the composition through the standard commit/PR/QA/doc/review gate, then an orchestrator render loop renders the merged `motion/` source via the credential-free `video-renderer` sidecar and materializes a held `video_post` draft for the CEO. Two task kinds (authoring + held post), one render pass between them. - -## Files - -| Path | Role | approx LOC | -|---|---|---| -| `roboco/services/video_engine.py` | `VideoEngine` — opens the ASSIGNED authoring task (`open_video_task`, balanced across `ux-dev-1`/`ux-dev-2`) and originates the held `video_post` draft once the render succeeds (`_originate_video_post`); release/spotlight/on-demand trigger wiring. | 324 | -| `roboco/services/video_post_service.py` | `VideoPostService` — CEO approve/reject over the held post; the ONLY caller of the X-v2 and TikTok posters; runs the critical section under a heartbeat-renewed Redis mutex, commits each platform's posted-id durably before the next, idempotent on already-`COMPLETED` AND on already-CANCELLED (rejected). | 629 | -| `roboco/services/video_renderer_client.py` | `VideoRenderer` — tars the merged `motion/` dir, POSTs the tarball to the sidecar (`ROBOCO_VIDEO_RENDERER_BASE_URL`), saves the returned MP4s to `video_output_dir` and PUTs each to MinIO (`_save`). `NullVideoRenderer` raises on unconfigured so the render loop fails loud rather than silently no-op'ing a real trigger; `get_video_renderer()` factory. | 188 | -| `roboco/services/minio_client.py` | Singleton `Minio` (minio-py) with an unconfigured guard (`get_client()` returns `None` when `minio_endpoint` empty); `put_object` / `get_object_stream` / `stat_object`, sync, call sites wrapped in `asyncio.to_thread`. | 129 | -| `roboco/services/tiktok_client.py` | `TikTokPoster` — TikTok inbox-upload poster (v2 media, OAuth2 refresh). Fernet-encrypted singleton `tiktok_credentials` row (migration 062); agents never hold creds or egress. | 326 | -| `roboco/services/x_video_client.py` | `XVideoPoster` — X v2 media upload + tweet poster. `NullXVideoPoster` makes the unconfigured leg a graceful no-op. | 266 | -| `roboco/runtime/heartbeat_mutex.py` | `HeartbeatMutex` — Redis mutex with heartbeat-renewed TTL, shared with `ReleaseProposalService`'s release-execute lock shape; backs `VideoPostService.approve`'s long video-upload critical section. | — | -| `roboco/mcp/do_server.py` `propose_video` | Do-tool the UX/UI dev calls exactly once per authoring task to stamp the `video_draft` marker (composition id + per-platform captions + input props); metadata-only, does not render. | — | -| `roboco/services/gateway/content_actions.py` `propose_video` | Server-side action: team-gated (`_caller_team` rejects be-dev/fe-dev), resolves the caller's open video task, `markers.set_video_draft` with the metadata. | — | -| `roboco/services/gateway/content_actions.py` `request_render` | Do-verb (developer/QA): renders the caller's ACTUAL composition to keyframe PNGs via the sidecar's frames mode and stamps the `render_preview` marker — dev renders their own tree (worktree-aware, `head_sha`/`dirty` stamped), QA a read-only branch export (`WorkspaceService.export_branch_motion`). Frames land at the container-shared `{workspaces_root}/{project}/.previews/{task8}/{orientation}/`. | — | -| `roboco/foundation/policy/tracing.py` `RENDER_VERIFIED` | `i_am_done` requirement on `source=video` tasks: no stamped `render_preview` → tracing gap naming `render_preview` (hint: call `request_render`, Read every frame). Mirrored in the possibilities-matrix fast path. | — | -| `alembic/versions/062_tiktok_credentials.py` | Migration 062 — the `tiktok_credentials` singleton row (Fernet-encrypted OAuth2 secrets, all-or-nothing set/clear, mirroring the git-token / `x_credentials` pattern). | 44 | -| `video-renderer/` | The sidecar: `server.js` (HTTP+tarball boundary), `render.js` (`@hyperframes/producer` `createRenderJob` + `executeRenderJob`, system `ffmpeg`, headless Chromium). Credential-free and git-free — reads only what's POSTed. pnpm-managed (`pnpm-lock.yaml`, no npm `package-lock.json`); `@hyperframes/producer` pinned exact at `0.7.36` (`0.7.60` fails every render). | — | -| `docker/video-renderer.Dockerfile` | Sidecar image (`roboco-video-renderer`): Node + Chromium + system `ffmpeg`; installs `@hyperframes/producer`. No RoboCo source, no creds. | — | -| `motion/README.md` `## Design bar` / `## Visual design bar (demo/kit register)` | Authoring craft an assigned UX/UI dev consults before building a composition: color/type/motion/layout dials for the text-card register, plus spacing, beat pacing (`animation-delay`, never `data-start` for beats), `pk-chip`/`pk-pill` semantic-variant discipline, camera+cursor+rhythm, and anti-generic tells for the `kit/` demo register. | 149 (file total) | -| `motion/skills/references/{house-style,video-composition,beat-direction,motion-principles}.md` | Four upstream HyperFrames craft references (palette/lazy-defaults, video-medium scale/density, per-beat rhythm planning, ease/speed/direction variance) vendored verbatim at pinned commit `9d148d28` (Apache-2.0, header in each file) — back every rule in `motion/README.md`'s design-bar sections; re-vendor when bumping `@hyperframes/producer`. | 462 | -| `motion/skills/hyperframes-catalog-index.md` | RoboCo-authored index of the public HyperFrames catalog (109 blocks + 24 components, 133 entries) with per-category kit-mapping triage (maps onto an existing `pk-*` piece / a choreography engine / needs a new kit piece); read on demand when planning a beat, not injected into any agent prompt. | 186 | -| `motion/skills/{hyperframes-core,hyperframes-creative,hyperframes-keyframes}.md` | The vendor's own official HyperFrames agent skills (composition contract, beat planning, seek-safe keyframes across runtimes), vendored verbatim at a pinned upstream commit (Apache-2.0, header + re-vendor note in each file); `motion/README.md` points authoring devs at them before a new register. Primary seek-safe primitive is GSAP tweens on `window.__timelines` — this kit's CSS-animation register is a house pattern, and the clip-window rule is its empirically-derived companion. | 382 | -| `motion/kit/kit.js` `choreographCursor` / `choreographCamera` | Choreography engines: `choreographCursor` reads `data-waypoints="t x y [click]; ..."` off a `.pk-cursor` and generates a multi-leg eased path with fade in/out, an idle-hand sway between legs, and click rings + glyph press dips at flagged waypoints (click lands ~0.2s before the thing it triggers) — replaces the old single-glide `--pk-cursor-x0/y0/x1/y1` cameo. `choreographCamera` reads `data-shots="t x y scale; ..."` off a `.pk-camera` wrapper and eases push-ins/pull-backs, settling at identity by the end. `motion/kit/README.md` documents both. | 156 (diff) | -| `roboco/runtime/orchestrator.py` `_is_video_authoring_spawn` | Fail-closed spawn-time probe (role==developer, team==ux_ui, task.source==video) that registers the `playwright` MCP for the composition author — the one non-QA case, so the dev can preview the authored HTML in a real browser between renders. Gating-only: `agent-ux`'s image already bakes the browser + wrapper entrypoint. | — | - -## Data Flow - -DETECT → AUTHOR: a release publish (`ROBOCO_VIDEO_ON_RELEASE`), a CEO-approved feature-spotlight draft that requests one (`ROBOCO_VIDEO_ON_SPOTLIGHT`), or a CEO on-demand `POST /api/video/request` calls `VideoEngine.open_video_task`, which creates a normal ASSIGNED UX/UI authoring task (`source=video`, `confirmed_by_human=True`, balanced across the two ux-devs) — NOT held, NOT in any dispatcher's skip bucket. The assigned dev authors `motion/compositions//{vertical,square}.html` (HyperFrames render params on ``), reads the vendored `motion/skills/hyperframes-{core,creative,keyframes}.md` skills and consults `kit.js`'s `choreographCursor`/`choreographCamera` engines for camera+cursor craft (a locked-off camera or a popping/freezing cursor is automatic revision; clip windows are for structural layers ONLY — beats ride base-hidden delayed CSS animations, since the renderer's clip scheduler drops any beat driven off its own clip window), previews the live HTML via the `playwright` MCP registered for this spawn (`_is_video_authoring_spawn`), calls the `propose_video` do-tool exactly once (server-side `content_actions.propose_video` is team-gated and stamps `video_draft`), then verifies the ARTIFACT: `request_render` renders the dev's actual working tree to keyframe PNGs the dev must Read (every scene fully visible and legible — the gate that catches an authored duration shorter than its scene list), iterating fix → re-render until the frames prove the brief; `i_am_done` refuses without the stamped `render_preview` marker (`Requirement.RENDER_VERIFIED`). Then `commit` + `open_pr` through the normal PR-review gate. The authoring task rides the standard QA/doc/review lifecycle to `completed`, with QA's `claim_review` evidence carrying a `video_context` block (the dev's preview + an instruction to `request_render` the branch state fresh). - -RENDER: once the authoring task is `completed`, the orchestrator's `_video_render_loop` (bounded retry, `_MAX_VIDEO_RENDER_ATTEMPTS`) resolves the project's read-clone at the merged HEAD, tars the `motion/` dir, and POSTs it to the credential-free `video-renderer` sidecar (`ROBOCO_VIDEO_RENDERER_BASE_URL`). The sidecar untars, runs `@hyperframes/producer`'s `createRenderJob` + `executeRenderJob` per orientation (headless Chrome + system `ffmpeg`, `ROBOCO_VIDEO_RENDER_TIMEOUT_SECONDS` per render), and streams both 9:16 and 1:1 MP4s back. `VideoRenderer` saves them to `ROBOCO_VIDEO_OUTPUT_DIR` (`_save` also PUTs each to MinIO when `minio_endpoint` is set, non-fatal). On success `VideoEngine._originate_video_post` materializes a held `video_post` draft (`source=video_post`, `confirmed_by_human=False`, Secretary-owned, skipped by every dispatcher) carrying `mp4_paths` (`{vertical, square}` absolute paths) + the per-platform captions. - -CEO ACT: `GET /api/video/posts` lists held drafts (including `mp4_paths`); `GET /api/video/posts/{id}/media?cut=vertical|square` streams the MP4 bytes for the preview player (CEO-gated, falls back to `FileResponse` on `S3Error`/unconfigured MinIO). The CEO edits captions and approves/rejects in the panel's `video-post-queue.tsx`. `POST /api/video/posts/{id}/approve` is the ONLY caller of `XVideoPoster` / `TikTokPoster`: it acquires `HeartbeatMutex`, re-reads the committed task state inside the lock, commits `COMPLETED` before releasing (so a concurrent approve can't double-post), commits each platform's posted-id durably before attempting the next (a partial failure never re-posts an already-succeeded platform on retry), and is idempotent (an already-`COMPLETED` draft returns the stored ids without calling a poster). A CANCELLED draft (already rejected) is refused both pre-lock and re-checked under lock, returning `already_rejected` — closes a hole where a stale approve (e.g. a queued Telegram button targeting the draft by id regardless of its current status) could post a draft the CEO had already rejected. `POST /api/video/posts/{id}/reject` cancels the draft with a reason — and, for a non-empty reason, `VideoEngine.reauthor_from_rejection` opens a fresh authoring task (same occasion, brief = the CEO's verbatim feedback + revise-in-place pointer at the existing composition) so the rejection feedback re-enters the delivery flow instead of dying on the cancelled draft; best-effort, never fails the reject. - -## Config Flags - -- `ROBOCO_VIDEO_ENGINE_ENABLED` — master switch; off = no video-authoring task is ever opened and no render/post happens. Panel-toggleable. -- `ROBOCO_VIDEO_ON_RELEASE` / `ROBOCO_VIDEO_ON_SPOTLIGHT` — sub-switches for the two automatic triggers, independent of the master switch and of the CEO's on-demand `POST /video/request`. -- `ROBOCO_VIDEO_RENDER_INTERVAL_SECONDS` / `ROBOCO_VIDEO_RENDER_TIMEOUT_SECONDS` / `ROBOCO_VIDEO_REQUEST_TIMEOUT_SECONDS` / `ROBOCO_VIDEO_OUTPUT_DIR` — render loop cadence, per-render deadline, sidecar HTTP deadline, MP4 output dir (bind-mounted in all three compose files so renders survive container recreation). -- `ROBOCO_VIDEO_RENDERER_BASE_URL` — the sidecar endpoint (default `http://roboco-video-renderer:3001`). -- `ROBOCO_MINIO_*` — MinIO object storage (default-off; `video_renderer_client._save` PUTs each render after the local write; serve route streams via `StreamingResponse` with `FileResponse` fallback). - -## Changes Since Baseline - -- **2026-07-17** (PR #543, `3e801697`): Two renderer root causes fixed — `@hyperframes/producer` was floating (`^0.7.36`, no lockfile), so image builds silently picked up `0.7.60`, which fails EVERY render ("Cannot access 'rt' before initialization"); pinned exact (`0.7.36`, no caret) and committed a lockfile (regenerated as `pnpm-lock.yaml` by the immediate follow-up `a12fefcb`, not the npm `package-lock.json` this PR first wrote — this package is pnpm-managed). Second: the producer's per-clip visibility scheduler runs on a clock that lags ~50% behind the encoded timeline on a long cut, so tail scenes (past roughly the halfway mark) were silently missing from the MP4 regardless of authoring — fixed by treating `class="clip"` + `data-start`/`data-duration` as a structural-layer-only primitive and driving every beat with base-hidden styles + a delayed CSS animation instead (documented in `motion/README.md`'s "Clip windows are for structural layers only" rule). Also added the two choreography engines to `motion/kit/kit.js` (`choreographCursor` / `choreographCamera`, see Files above) plus a "Cinematography & rhythm" section in `motion/README.md` and a craft-bar block in the dev video spawn prompt (`roboco/runtime/orchestrator.py`) so a locked-off camera or a popping/freezing cursor reads as an automatic revision. -- **2026-07-17** (PR #544, `fd621f0d`): The three craft capabilities wired one hop closer to the hands doing video work — vendored the vendor's own official HyperFrames agent skills (`hyperframes-core`/`-creative`/`-keyframes.md`, see Files above; supersedes the external-pointer-only version briefly added by the intervening `1416bd1d`); registered the `playwright` MCP for a ux-dev spawned onto a `source=video` task (`_is_video_authoring_spawn`, fail-closed role/team/task-source probe — gating-only, `agent-ux`'s image already bakes the browser); and added a video-mode override to the `ux_ui` team prompt's design bar ("video-authoring tasks are FILMS, not UI — these dials do not apply") so a video task no longer reads its own "dense product UI → motion 2-3" dial as license to ship a static slideshow. -- **2026-07-17** (Wave 6, PR #550): Authoring craft, not engine code — `motion/README.md` gained `## Visual design bar (demo/kit register)` (spacing/hierarchy, beat pacing, `pk-chip`/`pk-pill` semantic discipline, camera+cursor+rhythm, anti-generic tells for the `kit/` register), four upstream HyperFrames craft references vendored verbatim under `motion/skills/references/` (fixing `hyperframes-creative.md`'s previously-dead `references/` pointers), and a new `motion/skills/hyperframes-catalog-index.md` (133-entry HyperFrames catalog vocabulary index, read-on-demand). No service/verb/schema change; the render/post pipeline documented above is untouched. - -## Health - -Default-off, CEO-gated at two independent points (the flags, then per-clip approval). The held-draft shape mirrors the XEngine / ReleaseManagerEngine pattern, so the dispatchers never see it. The render pass is bounded retry with `_MAX_VIDEO_RENDER_ATTEMPTS` and a per-render deadline; `NullVideoRenderer` raises on unconfigured so a misflagged trigger fails loud rather than silently no-op'ing. The approve critical section is heartbeat-mutex protected so a double-click can't double-post and a partial platform failure is recoverable, and (Wave 5, PR #551) `approve` now also refuses a CANCELLED draft outright rather than posting it. TikTok's OAuth2 secrets live Fernet-encrypted in a singleton row (migration 062); agents never hold creds or egress — `VideoPostService.approve` is the only caller of the posters. - -## Related - -- `docs/rag/architecture/video-engine.md` — the user-facing architecture doc -- `docs/rag/architecture/minio-storage.md` — the decoupled-durable render storage -- `docs/map/release-manager.md` — the sibling held-artifact engine whose lock shape `VideoPostService.approve` mirrors -- `docs/map/engines-heal-ciwatch-depupdate.md` — the other default-off originate-and-stop engines - -## Purpose -The organizational-memory + playbooks slice: captures cross-agent learnings and curated playbooks, embeds them into the LEARNINGS and PLAYBOOKS pgvector RAG indexes via the OptimalService plugin architecture, and re-injects the top-K most relevant past lessons/playbooks into every agent briefing (the keystone retrieve step). Distillation at task completion runs on the local LLM only; playbook curation (draft/approve/reject/archive) is a status state-machine whose RAG index writes are split from the DB status commit so the corpus never leads the status transaction. - -## Files - -| Path | Role | LOC | -|---|---|---| -| roboco/services/memory_distiller.py | Local-LLM distiller: turns a completed task into one <=120-word Problem/Approach/Gotcha lesson (best-effort, returns None on failure) | 98 | -| roboco/services/playbook.py | PlaybookService: draft + Auditor curation state machine (draft/approve/reject/archive) + post-commit RAG index/de-index orchestration | 256 | -| roboco/services/learning.py | LearningPropagationService: record a learning, index it, notify same-scope non-human agents, scope-filtered retrieval (legacy pre-distiller capture path) | 525 | -| roboco/services/optimal.py | OptimalService: plugin-based RAG hub over pgvector; owns index_playbook/unindex_playbook/record_learning/search/search_learnings and the singleton accessor | 2044 | -| roboco/services/optimal_brain/indexes/base.py | BaseIndexPlugin ABC: chunk/filter/embed/store pipeline, hybrid search, 429-retried ask(); atomic replace_chunks reingest semantics | 1173 | -| roboco/services/optimal_brain/indexes/learnings.py | LearningsIndexPlugin: record/search cross-agent learnings; forces shareable=True on shared retrieval so private reflections never leak into briefings | 251 | -| roboco/services/optimal_brain/indexes/playbooks.py | PlaybooksIndexPlugin: index approved playbooks (title+when-to-use+procedure) and delete_playbook de-index | 97 | -| roboco/services/optimal_brain/indexes/__init__.py | Plugin registry exports (LearningsIndexPlugin, PlaybooksIndexPlugin, etc.) | 35 | -| roboco/services/optimal_brain/vector_store.py | VectorStore: asyncpg pool over chunks_ pgvector tables; add_chunks/delete_by_source/replace_chunks(hybrid_search) | 521 | -| roboco/services/repositories/base.py | BaseRepository: generic CRUD mixin used by IndexedDocumentRepository | 281 | -| roboco/services/repositories/indexed_document.py | IndexedDocumentRepository: upsert/get/delete_by_source for the indexed_documents tracking table (used by playbook de-index) | 128 | -| roboco/services/repositories/query_helpers.py | Generic SQLAlchemy query helpers (pagination/status/team/agent/timestamp filters, slug resolution) — not slice-specific | 264 | - -## Key Symbols - -| Name | Kind | File:Line | Responsibility | -|---|---|---|---| -| LessonInput | dataclass | roboco/services/memory_distiller.py:25 | Completed-task facts (title, ACs, dev/qa notes, commit messages) fed to the distiller | -| _build_prompt | function | roboco/services/memory_distiller.py:40 | Render the fixed Problem/Approach/Gotcha <=120-word distillation prompt | -| _chat | function | roboco/services/memory_distiller.py:62 | One OpenAI-compatible call to the local LLM (glm-5.2:cloud); None on non-success/empty | -| MemoryDistiller.distill | method | roboco/services/memory_distiller.py:87 | Return a <=120-word lesson or None (NONE sentinel, failure, or over-limit truncation) | -| _slugify | function | roboco/services/playbook.py:35 | Derive a unique <=80-char slug from the playbook title | -| PlaybookService.draft | method | roboco/services/playbook.py:45 | Create a DRAFT playbook; savepoint-isolates the insert to convert slug-UNIQUE TOCTOU into a clean ConflictError | -| PlaybookService.approve | method | roboco/services/playbook.py:89 | Auditor: draft->approved + stamps approver/at; flushes status ONLY (caller commits then index_approved) | -| PlaybookService.archive | method | roboco/services/playbook.py:113 | Auditor: approved->archived (retire); flushes status ONLY, caller commits then unindex_playbook | -| PlaybookService.index_approved | method | roboco/services/playbook.py:138 | Post-commit: embed the approved playbook into PLAYBOOKS index; org_memory_enabled-gated, best-effort | -| PlaybookService.reject | method | roboco/services/playbook.py:173 | Auditor: draft->archived with a reason; flushes status ONLY, caller commits then unindex_playbook | -| PlaybookService.unindex_playbook | method | roboco/services/playbook.py:195 | Post-commit: de-index a rejected/archived playbook from PLAYBOOKS; org_memory_enabled-gated, best-effort, idempotent | -| PlaybookService._get_by_slug | method | roboco/services/playbook.py:238 | Fast-path UX pre-check for slug uniqueness (DB constraint is the real guard) | -| LearningScope | enum | roboco/services/learning.py:32 | Visibility scope: PERSONAL/TEAM/CELL/ORG | -| LearningType | enum | roboco/services/learning.py:41 | Learning category: SOLUTION/PATTERN/GOTCHA/INSIGHT/REVIEW_FEEDBACK | -| LearningPropagationService.record_learning | method | roboco/services/learning.py:122 | Index the learning + create same-scope non-human notifications (skips notifications for PERSONAL) | -| LearningPropagationService._index_learning | method | roboco/services/learning.py:189 | Bridge to OptimalService.record_learning with shareable = scope != PERSONAL; passes team=None | -| LearningPropagationService._create_notifications | method | roboco/services/learning.py:205 | Open a DB session, query non-author non-human agents in scope, create formal KNOWLEDGE_SHARE notifications | -| LearningPropagationService.get_learnings_for_agent | method | roboco/services/learning.py:303 | Role-shaped search_learnings + post-filter by scope visibility (personal/team) | -| LearningPropagationService.search_similar_learnings | method | roboco/services/learning.py:484 | Similar-learnings search via OptimalService.search over LEARNINGS index | -| get_learning_service | function | roboco/services/learning.py:521 | Process-wide singleton accessor for LearningPropagationService | -| PLUGIN_REGISTRY | dict | roboco/services/optimal.py:139 | IndexType -> plugin class map (includes LEARNINGS, PLAYBOOKS) | -| OptimalService.initialize | method | roboco/services/optimal.py:183 | Graceful-degradation init of all plugins; starts background auto-index + periodic tasks | -| OptimalService.close | method | roboco/services/optimal.py:563 | Cancel _indexing_task FIRST, then periodic task, then close plugins (prevent writes to closed plugins) | -| OptimalService._get_plugin | method | roboco/services/optimal.py:593 | Typed plugin lookup with a clear RuntimeError when missing/uninitialized | -| OptimalService.index_playbook | method | roboco/services/optimal.py:859 | Embed an approved playbook + write the indexed_documents tracking row | -| OptimalService.unindex_playbook | method | roboco/services/optimal.py:889 | Delete playbook chunks from vector store AND drop tracking row; both steps best-effort/idempotent | -| OptimalService.record_learning | method | roboco/services/optimal.py:1061 | Embed a learning via LearningsIndexPlugin + write tracking row (source learn-{md5(full content)}) | -| OptimalService.search | method | roboco/services/optimal.py:1145 | Embed-once fan-out: concurrent hybrid search across selected indexes; used by similar_memory | -| OptimalService._aggregate_citations | method | roboco/services/optimal.py:1221 | Embed once + concurrent per-index search into an aggregation buffer for RAG query() | -| OptimalService.search_learnings | method | roboco/services/optimal.py:1507 | LEARNINGS-only search with optional category/team filter (shareable_only=True default) | -| get_optimal_service | function | roboco/services/optimal.py:2017 | Lock-guarded singleton: publish instance only after initialize() completes | -| BaseIndexPlugin.ingest | method | roboco/services/optimal_brain/indexes/base.py:351 | Validate -> metadata -> source URI -> chunk/filter/embed/store pipeline | -| BaseIndexPlugin._chunk_filter_embed_store | method | roboco/services/optimal_brain/indexes/base.py:429 | Chunk + quality-filter + embed + atomic replace_chunks/add_chunks; returns stored count | -| BaseIndexPlugin._citations_to_results | method | roboco/services/optimal_brain/indexes/base.py:772 | Apply exact-match metadata filters to citations and cap to top_k SearchResults | -| BaseIndexPlugin.search_with_embedding | method | roboco/services/optimal_brain/indexes/base.py:808 | Pre-computed-embedding hybrid search entry; LearningsIndexPlugin overrides to force shareable | -| BaseIndexPlugin.search | method | roboco/services/optimal_brain/indexes/base.py:852 | Embed-then-search_with_embedding convenience entry | -| BaseIndexPlugin.ask | method | roboco/services/optimal_brain/indexes/base.py:893 | Per-index RAG Q&A with 15s search timeout + 429-retried LLM synthesis | -| LearningsIndexPlugin.search_with_embedding | method | roboco/services/optimal_brain/indexes/learnings.py:44 | Force shareable=True filter unless include_private opt-in; prevents private reflections leaking into briefings | -| LearningsIndexPlugin.search | method | roboco/services/optimal_brain/indexes/learnings.py:75 | Embed-then-search that threads include_private to search_with_embedding | -| LearningsIndexPlugin.record_learning | method | roboco/services/optimal_brain/indexes/learnings.py:124 | Build lrn-{md5(content[:100])} doc_id + enriched content; ingest with category/role/team/shareable metadata | -| LearningsIndexPlugin.search_learnings | method | roboco/services/optimal_brain/indexes/learnings.py:174 | Category/team-filtered search; include_private=not shareable_only to thread the shareable default | -| IndexPlaybookParams | dataclass | roboco/services/optimal_brain/indexes/playbooks.py:18 | Params for indexing an approved playbook (id/title/problem/procedure/tags/team/scope) | -| PlaybooksIndexPlugin.index_playbook | method | roboco/services/optimal_brain/indexes/playbooks.py:55 | Embed title + when-to-use + procedure + tags; metadata status=approved | -| PlaybooksIndexPlugin.delete_playbook | method | roboco/services/optimal_brain/indexes/playbooks.py:73 | Delete a playbook's chunks by source URI (idempotent no-op when none match) | -| PlaybooksIndexPlugin.search_playbooks | method | roboco/services/optimal_brain/indexes/playbooks.py:86 | Optional team-scoped search over approved playbooks | -| VectorStore.replace_chunks | method | roboco/services/optimal_brain/vector_store.py:239 | Atomic single-connection single-tx DELETE+INSERT replacing a source's chunks (closes concurrent reindex duplicate race) | -| VectorStore.delete_by_source | method | roboco/services/optimal_brain/vector_store.py:225 | Delete every chunk row for a source URI (idempotent) | -| VectorStore.hybrid_search | method | roboco/services/optimal_brain/vector_store.py:342 | pgvector + full-text hybrid retrieval returning Citation rows | -| IndexedDocumentRepository.delete_by_source | method | roboco/services/repositories/indexed_document.py:103 | Drop the indexed_documents tracking row by (index_type, source_hash); idempotent bool return | -| IndexedDocumentRepository.upsert_batch | method | roboco/services/repositories/indexed_document.py:22 | Bulk upsert tracking rows keyed by (index_type, source_hash) | - -## Data Flow -CAPTURE (task completion): TaskService._extract_completion_learnings (task.py:2837) is fire-and-forget on completion. With org_memory_enabled it calls _completion_learnings_for (task.py:2798) which runs MemoryDistiller().distill(LessonInput(...)) against the local LLM (memory_distiller.py) to produce ONE <=120-word lesson, else falls back to the legacy raw-notes _collect_completion_learnings. The lesson goes to LearningPropagationService.record_learning (learning.py:122) -> _index_learning -> OptimalService.record_learning (optimal.py:1061) -> LearningsIndexPlugin.record_learning (learnings.py:124), which embeds via the shared qwen3 embedder and stores chunks in the chunks_learnings pgvector table with metadata {category, agent_role, shareable, ...}. _create_notifications (learning.py:205) opens a separate DB session and creates formal KNOWLEDGE_SHARE notifications for same-scope non-author, non-human agents (CEO/prompter/secretary excluded via _HUMAN_ONLY_ROLES). - -PLAYBOOK CURATION: A delivery agent calls the draft_playbook content verb (do_server -> v1/do.py -> content_actions.draft_playbook -> PlaybookService.draft) which writes a DRAFT row with a slug-unique constraint (savepoint TOCTOU guard). The Auditor (gateway verb) or Auditor/CEO (panel route /api/playbooks) calls approve/reject/archive. The status flush and the RAG index write are deliberately split: approve() flushes status ONLY; the caller (api/routes/playbooks.py or content_actions._curate_playbook) commits the DB transaction FIRST, then calls index_approved() which -> OptimalService.index_playbook -> PlaybooksIndexPlugin.index_playbook -> BaseIndexPlugin.ingest -> chunk/embed/store in chunks_playbooks (+ a tracking row in indexed_documents). reject/archive similarly commit then call unindex_playbook -> OptimalService.unindex_playbook -> delete_playbook (vector store) + IndexedDocumentRepository.delete_by_source (tracking row). - -RETRIEVE (keystone briefing): Choreographer._briefing_for (_impl.py:814) is called on give_me_work/claim/done/qa/doc/pr_review/board routes. It calls _institutional_memory (_impl.py:877) which, when org_memory_enabled and a task is in hand, shapes a role-shaped query via shape_memory_query (evidence_builder.py) and calls EvidenceRepo.similar_memory (evidence_repo.py:323). similar_memory runs OptimalService.search over [LEARNINGS, PLAYBOOKS] indexes (embed-once, concurrent hybrid search), filters results by min_score, and returns top-K {kind, summary, source, score} items injected as briefing['institutional_memory']. LearningsIndexPlugin.search_with_embedding forces shareable=True so private reflections never surface. Memory is best-effort: any RAG/embed failure returns [] so the briefing path never breaks. - -## Mermaid -```mermaid -stateDiagram-v2 - direction LR - [*] --> draft: delivery agent draft_playbook - draft --> approved: Auditor/CEO approve (commit then index_approved) - draft --> archived: Auditor reject (commit then unindex_playbook) - approved --> archived: Auditor/CEO archive (commit then unindex_playbook) - approved --> [*]: surfaces in briefings (PLAYBOOKS index) - archived --> [*]: terminal (de-indexed) -``` - -```mermaid -sequenceDiagram - autonumber - participant Route as PanelRoute - participant PS as PlaybookService - participant DB as Postgres - participant OS as OptimalService - participant VS as VectorStore - participant TR as indexed_documents - Route->>PS: approve(id, approver) - PS->>PS: guard status==DRAFT - PS->>DB: status=APPROVED, flush in-tx - Route->>DB: commit() - Route->>PS: index_approved(playbook) - PS->>OS: index_playbook(IndexPlaybookParams) - OS->>VS: ingest chunk+embed+replace_chunks - OS->>TR: upsert tracking row - Note over VS,DB: index never leads the status commit -``` - -```mermaid -erDiagram - playbooks ||--o{ chunks_playbooks : "approved to embedded" - playbooks ||--o| indexed_documents : "tracking row" - learnings ||--o{ chunks_learnings : "shareable to embedded" - learnings ||--o| indexed_documents : "tracking row" - playbooks { - uuid id PK - str slug UK - str status - uuid approved_by - } - chunks_playbooks { - str source - vector embedding - jsonb metadata - } - chunks_learnings { - str source - vector embedding - jsonb metadata - } -``` - -## Logical Tree -``` -org-memory-playbooks - Capture (completion) - MemoryDistiller (local LLM only) - LessonInput -> _build_prompt -> _chat -> distill (<=120w or None) - TaskService._completion_learnings_for [external, task.py] - org_memory_enabled ? distill : legacy raw capture - LearningPropagationService - record_learning -> _index_learning -> OptimalService.record_learning - _create_notifications (KNOWLEDGE_SHARE, non-human, scope-filtered) - Playbook curation state machine (PlaybookService) - draft (slug UNIQUE + savepoint TOCTOU guard) - approve (draft->approved; commit-then-index) - reject (draft->archived; commit-then-unindex) - archive (approved->archived; commit-then-unindex) - index_approved / unindex_playbook (post-commit, org_memory-gated) - RAG hub (OptimalService + plugins) - PLUGIN_REGISTRY: LEARNINGS, PLAYBOOKS, ... - BaseIndexPlugin: ingest / search / ask / replace_chunks - LearningsIndexPlugin: forces shareable=True on shared retrieval - PlaybooksIndexPlugin: index_playbook / delete_playbook - VectorStore: chunks_ pgvector tables; replace_chunks atomic - IndexedDocumentRepository: tracking-row upsert / delete_by_source - Retrieve (keystone briefing) [external callers] - Choreographer._briefing_for -> _institutional_memory - shape_memory_query (role-shaped) - EvidenceRepo.similar_memory -> OptimalService.search([LEARNINGS,PLAYBOOKS]) - -> briefing['institutional_memory'] (top-K, min_score-floored) -``` - -## Dependencies -- Internal: roboco.config.settings (org_memory_enabled, org_memory_top_k, org_memory_min_score, local_llm_*, default_embedding_model, embedding_dimensions, rag_*), roboco.db.tables.PlaybookTable / IndexedDocumentTable, roboco.db.get_db_context, roboco.models.base.PlaybookStatus, roboco.models.optimal.IndexType / SearchResult / SearchOutcome / QueryContext, roboco.models.playbook.PlaybookCreate / Playbook, roboco.services.base.BaseService / ConflictError / NotFoundError, roboco.services.exceptions (RateLimitError, parse_retry_after_header, HTTP_TOO_MANY_REQUESTS, MAX_RATE_LIMIT_RETRIES), roboco.services.optimal_brain.text_chunker (TextChunker, Chunk, Citation, Document), roboco.services.optimal_brain.shared_embedder.get_shared_embedder, roboco.services.gateway.evidence_repo.EvidenceRepo.similar_memory, roboco.services.gateway.evidence_builder.shape_memory_query, roboco.services.gateway.choreographer._impl._briefing_for / _institutional_memory, roboco.services.gateway.content_actions (draft/approve/reject/archive_playbook), roboco.services.task.TaskService._completion_learnings_for / _extract_completion_learnings, roboco.foundation.identity.Role, roboco.api.routes.playbooks (panel route), roboco.mcp.do_server (draft/approve/reject/archive_playbook verbs) -- External: httpx (local LLM chat + RAG synthesis), structlog, sqlalchemy (select, delete, func, IntegrityError, AsyncSession), asyncpg (VectorStore pool + transaction), pgvector (vector column), dataclasses / enum / hashlib / re / asyncio - -## Entry Points - -| Name | File | Trigger | -|---|---|---| -| draft_playbook verb | roboco/services/gateway/content_actions.py | Agent content verb via do_server -> POST /api/v1/do/draft_playbook (delivery roles only) | -| approve/reject/archive_playbook verbs | roboco/services/gateway/content_actions.py | Auditor content verb via do_server -> POST /api/v1/do/{approve,reject,archive}_playbook | -| GET/POST /api/playbooks[/{id}/{approve,reject,archive}] | roboco/api/routes/playbooks.py | Panel review-queue HTTP (Auditor or CEO only) | -| _extract_completion_learnings | roboco/services/task.py | Fire-and-forget on task completion (TaskService complete/ceo_approve path) | -| _briefing_for / _institutional_memory | roboco/services/gateway/choreographer/_impl.py | Every choreographer verb that builds a context_briefing (give_me_work, claim, done, qa, doc, pr_review, board) | -| OptimalService.initialize / get_optimal_service | roboco/services/optimal.py | FastAPI lifespan startup; first RAG caller (lazy singleton) | -| OptimalService.close | roboco/services/optimal.py | FastAPI lifespan shutdown (cancels indexing + periodic tasks, closes plugins) | - -## Config Flags -- ROBOCO_ORG_MEMORY_ENABLED (default off) — gates the whole loop: distill-vs-legacy capture, index_approved/unindex_playbook no-op when off, _institutional_memory returns [] when off -- ROBOCO_ORG_MEMORY_TOP_K (default 3, 1..10) — max institutional-memory items injected into a briefing -- ROBOCO_ORG_MEMORY_MIN_SCORE (default 0.6, 0..1) — cosine-similarity floor; below it nothing is injected -- ROBOCO_LOCAL_LLM_MODEL (default glm-5.2:cloud) + ROBOCO_LOCAL_LLM_BASE_URL — the distiller + RAG synthesis LLM endpoint -- ROBOCO_DEFAULT_EMBEDDING_MODEL (default qwen3-embedding:0.6b) + ROBOCO_EMBEDDING_DIMENSIONS — embedder for LEARNINGS/PLAYBOOKS chunks -- ROBOCO_RAG_CHUNK_STRATEGY / ROBOCO_RAG_CHUNK_SIZE / ROBOCO_RAG_CHUNK_OVERLAP / ROBOCO_RAG_PERSIST_DIR / ROBOCO_RAG_STORE_URL — chunking + store DSN -- ROBOCO_DATABASE_* (VectorStore.store_url derived) — required; missing store_url raises at plugin initialize() - - -## Gotchas -- Index-vs-status ordering is a hard contract: approve()/reject()/archive() flush status ONLY; the caller MUST commit the DB tx BEFORE calling index_approved()/unindex_playbook(). The vector store writes through its own auto-committing pool connection, so indexing before commit would durably land (or drop) a playbook in the corpus even if the status tx rolled back. Both the panel route and content_actions honour this; any new caller must too. -- archive() and reject() previously overwrote approved_by/approved_at — FIXED in 536bbb64: migration 053 added archived_by/archived_at columns; archive() now writes archived_by/archived_at at playbook.py:132-133 and reject() likewise at playbook.py:189-190, leaving approved_by/approved_at intact. -- Learning doc-id / source-URI mismatch: FIXED in 536bbb64. OptimalService.record_learning now derives source = f"roboco://learnings/{doc_id}" from the plugin's returned doc_id (optimal.py:1078) instead of independently computing learn-{md5(full content)}, so the tracking row and chunk rows share the same source URI. -- LearningPropagationService._index_learning always passes team=None, so team-scoped search_learnings(team=...) will never match auto-captured completion learnings (learnings.py:199). -- LearningsIndexPlugin.search_with_embedding forces shareable=True via exact equality on metadata. This relies on the stored metadata value being a JSON bool that round-trips to Python True; a learning indexed with a string 'true' would be filtered out. Currently safe (prepare_metadata sets a Python bool) but brittle if metadata serialization changes. -- BaseIndexPlugin._citations_to_results applies filters as exact equality on every key-value pair. The forced shareable=True filter is therefore exact-match; any NULL/missing shareable metadata (older rows) would be excluded from briefings after the fix. -- replace_chunks on a re-ingest: the embedder-failure case (non-empty chunks list but no usable embeddings returned) is now guarded at vector_store.py:272 — the wipe is skipped and existing rows are preserved (FIXED in 536bbb64). The deliberate-clear case (empty chunks list) still deletes, by design. -- PlaybooksIndexPlugin inherits BaseIndexPlugin.replace_on_reingest=True; re-indexing an already-approved playbook (e.g. approve twice via different paths) atomically replaces chunks. approve() now guards status==DRAFT so a double-approve is blocked before reaching index. -- draft()'s _get_by_slug pre-check is a UX fast-path, NOT the guard: two concurrent same-title drafts both miss it and the loser hits the slug UNIQUE constraint — handled by the savepoint + IntegrityError->ConflictError conversion. Don't rely on the pre-check for uniqueness. -- _HUMAN_ONLY_ROLES (CEO, prompter, secretary) are excluded as learning notification recipients (learning.py:27), resolved from the foundation Role enum at import time. CLAUDE.md states agent learnings exclude human/human-driven roles — code matches. -- OptimalService is a process-wide singleton published only after initialize() completes under a lock; a half-built instance is never observable. But the singleton is event-loop-bound — calling get_optimal_service() from a different loop raises 'bound to a different event loop' (noted at optimal.py:2010). - - -## Drift from CLAUDE.md -- CLAUDE.md says institutional memory is injected 'on claim'. Code injects it on EVERY _briefing_for call that carries a task (give_me_work, i_will_plan, claim, done, qa, doc, pr_review, board, submit_up) — broader than 'on claim' (choreographer/_impl.py:814-875, called from ~30 sites). The _institutional_memory guard is only `org_memory_enabled and task is not None`, not claim-specific. -- CLAUDE.md's verb-surface table lists Auditor verbs as only `triage` (read-only) in the role table, while the prose below it lists Auditor `approve_playbook`/`reject_playbook`/`archive_playbook` curation. The gateway enforces _CURATE_PLAYBOOK_ROLES={'auditor'} (content_actions.py:310) — auditor-only via the verb path — while the panel route allows Auditor OR CEO (_CURATOR_ROLES in api/routes/playbooks.py:21). The Auditor/CEO split is documented for /api/playbooks but the verb path is auditor-only, which is consistent with the prose but not reflected in the verb table row. - - -## Changes Since Baseline - -| SHA | Subject | Impact | -|---|---|---| -| 15effce0 | [feature] org-memory/playbooks curation + retrieval hardening (bundled in 141-Gaps fill-in PR #283) | playbook.py: split status flush from RAG index write — approve() no longer indexes inline; added archive() (approved->archived) + public index_approved()/unindex_playbook(); added status==DRAFT precondition guards on approve/reject; savepoint-isolated draft insert to convert slug-UNIQUE TOCTOU into ConflictError. | -| 15effce0 | [fix] learnings: force shareable=True on shared retrieval | learnings.py: overrode search_with_embedding/search to force shareable=True unless include_private opt-in; threaded include_private=not shareable_only through search_learnings. Prevents private (shareable=False) reflections leaking into cross-agent briefings via OptimalService.search. | -| 15effce0 | [fix] playbook de-index path | optimal.py added OptimalService.unindex_playbook (delete chunks + tracking row, best-effort); playbooks.py added PlaybooksIndexPlugin.delete_playbook; repositories/indexed_document.py added IndexedDocumentRepository.delete_by_source. reject/archive now actually remove a previously-approved playbook from the corpus. | -| 15effce0 | [fix] atomic reindex (F108) | base.py replaced separate delete_by_source + add_chunks with VectorStore.replace_chunks (single connection, single tx) for replace_on_reingest plugins — closes concurrent-reindex duplicate-chunk race; failed insert now reverts the delete. | -| 15effce0 | [fix] OptimalService.close() ordering | optimal.py: close() now cancels the startup _indexing_task FIRST (can be mid-flight writing through plugins) before the periodic task and plugin close — prevents writes against closed plugins. | -| 15effce0 | [chore] glm-5 -> glm-5.2:cloud | memory_distiller.py docstring + IndexConfig.llm_model default bumped from glm-5:cloud to glm-5.2:cloud (matches the fleet LLM bump). No behavior change beyond the model name. | - -> Post-snapshot updates (since 2026-06-29): commit 536bbb64 (Chore/all/logical gaps sweep, PR #286) touched three files in this slice: (1) roboco/services/playbook.py — archive() and reject() now write archived_by/archived_at (new columns, migration 053) instead of overwriting approved_by/approved_at; content_actions._curate_playbook wraps the gating session.commit() in a PendingRollbackError guard (#55) so a poisoned session returns a clean invalid_state and never falls through to index an uncommitted playbook. (2) roboco/services/optimal.py — record_learning reuses the plugin's returned doc_id for the tracking-row source URI, closing the lrn-/learn- mismatch (#182/#183). (3) roboco/services/optimal_brain/vector_store.py — replace_chunks skips the wipe when chunks is non-empty but all lack embeddings (#181), preserving existing rows on embedder failure. - -## Regression Risks - -| Title | File:Line | Claim | Severity | -|---|---|---|---| -| ~~archive() overwrites the original approver attribution~~ **FIXED 536bbb64** | roboco/services/playbook.py:132 | Migration 053 added archived_by/archived_at; archive() (line 132-133) and reject() (line 189-190) now write those columns, leaving approved_by/approved_at intact. | medium | -| approve() index write contract is now caller-owned — a missed call silently skips indexing | roboco/services/playbook.py:109 | Before baseline, approve() called _index_approved inline. Now approve() flushes status ONLY and the caller must commit then call index_approved(). If any caller (current or future) calls approve() without the commit+index_approved pair, the playbook is APPROVED in DB but NEVER embedded — it will not surface in briefings. Today only api/routes/playbooks.py and content_actions._curate_playbook call it (both correct), but the contract is a footgun. | medium | -| unindex_playbook returns early on vector-store failure, leaving tracking row stale | roboco/services/optimal.py:909 | On a vector-store delete exception, unindex_playbook logs + `return`s before dropping the indexed_documents tracking row. If the VS delete partially succeeded (some chunks gone) but raised, the tracking row lingers referencing a partially-deleted source — inconsistent index/tracking state. Best-effort by design, but the divergence is silent. | low | -| ~~Learnings tracking-row source URI never matches the embedded chunk source URI~~ **FIXED 536bbb64** | roboco/services/optimal.py:1078 | record_learning now reuses the plugin's returned doc_id: source = f"roboco://learnings/{doc_id}" — tracking row and chunk rows share the same URI. | low | -| replace_chunks wipes a source when a re-ingest produces zero embedded chunks | roboco/services/optimal_brain/indexes/base.py:475 | **Embedder-failure case FIXED 536bbb64** (vector_store.py:272): when chunks is non-empty but no records have embeddings, replace_chunks now returns early, preserving existing rows. The deliberate-clear case (empty chunks list) still deletes by design. | low | -| Forced shareable=True filter excludes any learning whose metadata lacks a shareable key | roboco/services/optimal_brain/indexes/learnings.py:70 | _citations_to_results applies filters as chunk_meta.get(k) == v. With forced shareable=True, any older learning chunk whose metadata has no 'shareable' key (get returns None) is excluded from briefings. If pre-fix rows exist without the shareable metadata field, they stop surfacing after this change — a silent recall regression for legacy learnings. | low | -| close() awaits a cancelled _indexing_task that may be mid-DB-write | roboco/services/optimal.py:571 | close() now cancels _indexing_task and awaits it (suppressing CancelledError). If the indexing task is mid-flight inside an asyncpg executemany/transaction at shutdown, cancellation can leave a partial chunk insert. Shutdown-only, best-effort, and the new ordering is strictly better than the old close-then-write-to-closed-plugin race it fixes — but the cancellation mid-write is new surface. | low | - -## Health -The slice is coherent and has been further hardened by PR #286 (536bbb64): archive()/reject() provenance loss and the learnings tracking-row/chunk source-URI mismatch are both fixed, and the embedder-failure wipe in replace_chunks is now guarded. The main residual risks are (a) the caller-owned commit-then-index contract on approve/reject/archive — a future caller that forgets index_approved silently produces an un-indexed approved playbook (the poisoned-session guard in content_actions is a step forward but the footgun remains for any new caller); (b) the forced shareable=True filter silently excluding older learning rows that lack the metadata key. The org_memory_enabled gate is consistently applied at every entry (distill, index_approved, unindex_playbook, _institutional_memory), so the whole loop is inert when off. Best-effort semantics are uniformly observed: every RAG/embed failure returns []/None and never blocks completion or the briefing. No critical regressions found. - -## Purpose - -The Obsidian vault (V1+V2): a rebuildable, human-readable DB projection of the org's memory (tasks, journal entries, A2A thread digests) as wikilinked markdown, plus a default-off inbox watcher that turns `#roboco`-tagged vault notes into board-review intake drafts. V2 adds three things on top of the V1 projection: materialize-on-create (a task's note exists from the moment it's created, not just at curation/rebuild), a drift janitor (hourly-ticked, daily/weekly-gated: re-projects changed tasks, verifies a random sample, archives old terminal tasks, writes the weekly org-report), and KB ingest (the CEO's own `RoboCo/Notes/` notes become one more RAG corpus the fleet can retrieve). Default-off (`ROBOCO_OBSIDIAN_VAULT_ENABLED`; both compose files arm it `true`). Still structurally different from the other default-off engines: the projection never originates delivery work itself — the ONE writer-side effect that reaches delivery (the intake watcher) rides the existing board-review path, not a held-artifact queue. - -## Files - -| Path | Role | approx LOC | -|---|---|---| -| `roboco/services/vault_writer.py` | `VaultWriter` — pure, DB-free markdown materializer. `write_task` / `write_journal_entry` / `append_a2a_message` / `write_agent` / `touch_task_frontmatter` / `write_org_report`. Every note carries `aliases: []` so a rename never breaks a `[[id8\|title]]` wikilink; `existing_narrative` reads back an Auditor-authored `## Narrative` section so a rebuild never clobbers it. V2: `write_task` is archive-aware (`TaskNoteData.archive_year` routes it to `RoboCo/Archive//Tasks//` instead of `Tasks//`, removing the stale copy on a move); `find_task_note`/`task_note_status` locate/inspect a note wherever it lives (recursive id8 lookup across both trees) for the janitor's drift check; `write_org_report` renders `RoboCo/Reports/.md`. (Uncommitted, `feature/findings-ledger`) `FindingRow` + `_FINDINGS_CAP=20` + `_findings_section` render a `## Findings` section (one `[F-id8] (severity, round N, status) file:line — expected → actual → fix` line per open/resolved finding, an overflow line past the cap) into every task note — see `docs/map/review-findings.md`. | 534 | -| `roboco/services/vault_assembly.py` | `assemble_task_note_data` — resolves a task's project slug, parent, subtasks, dependencies, and (V2) archive eligibility (`_archive_year`, gated on `vault_archive_days`) via the live `TaskService`/`ProjectService` into a `TaskNoteData`. `reproject_task` (V2) bundles assemble + narrative-preservation + `write_task` into the one code path shared by `rebuild`, the janitor's changed/sample/archival passes, and the create-on-task seam — none of them can drift on how a note gets refreshed. (Uncommitted, `feature/findings-ledger`) `_resolve_findings` fetches the task's ledger via `ReviewFindingsRepository.list_for_task`, fails open (empty tuple) on a missing session or any exception — a findings-fetch failure drops the section, never blocks the note write. | ~125 | -| `roboco/services/vault_intake_engine.py` | `VaultIntakeEngine.run_cycle` — scans the vault's inbox dir for `#roboco`-tagged notes, dedupes via `vault_seen_notes` (path + content-hash), screens the body through `injection_guard.screen_external_text`, extracts a title/description/action-items via a local-model chat call (deterministic fallback: first heading / raw body / checkbox lines), and opens ONE PENDING board-review draft (`source=vault_note`, Product-Owner-assigned, `team=board`) per note. Appends a feedback callout back into the note (best-effort). V2: the frontmatter split + content-hash helpers moved to the shared `foundation/policy/vault_notes.py` (this module now just imports them). | ~350 | -| `roboco/services/vault_janitor.py` **(new, V2)** | `VaultJanitor.run_cycle` — one state-gated sweep: re-project tasks changed since the last sweep (`TaskService.list_updated_since`, capped/paged, per-item isolated), verify a random stale sample (`sample_stale_tasks`), archive old terminal tasks (`list_archive_candidates`), and (weekly) render the org-report. Dueness is tracked in a JSON state file (`RoboCo/_meta/.janitor_state.json`: `last_sweep`, `last_report_week`, `archive_watermark`), not the loop's own cadence — restart-proof. | ~343 | -| `roboco/services/vault_kb_engine.py` **(new, V2)** | `VaultKBEngine.run_cycle` — scans the allowlisted `vault_kb_dirs` (default `RoboCo/Notes`), dedups by content hash, screens every note body through the injection guard as a hard GATE (flagged → quarantined, never indexed), and ingests/deindexes into `IndexType.VAULT_NOTES` via `OptimalService.index_vault_note`/`unindex_vault_note`. Defense-in-depth containment: symlinks and any resolved-path escape from the vault root are skipped, independent of the config-load validator. | ~315 | -| `roboco/foundation/policy/vault_notes.py` **(new, V2)** | Shared pure helpers: `content_hash` (sha256 with every `> [!kind] RoboCo: ...` feedback callout stripped first, so appending one doesn't change what the next scan considers "changed") and `split_frontmatter` (YAML frontmatter + body). Used by both the intake watcher's "drafted" callout and the KB engine's "quarantined" callout — one shared convention instead of two copies drifting. | ~46 | -| `roboco/services/optimal_brain/indexes/vault_notes.py` **(new, V2)** | `VaultNotesIndexPlugin` — `IndexType.VAULT_NOTES` plugin, mirrors `PlaybooksIndexPlugin`'s shape (`index_note`/`delete_note`/`search_notes`, source URI `vault://`). Scope enforced by the KB engine's dir allowlist, not this plugin. | ~70 | -| `roboco/vault.py` | `python -m roboco.vault {rebuild,relocate}` CLI. `rebuild` re-projects every agent/task/journal-entry/A2A-thread from the DB (now archive-aware via `vault_assembly.reproject_task` — an old terminal task projects straight into `Archive//`) and materializes `.obsidian/` + `RoboCo/_meta/` from `roboco/vault_assets/` (never overwrites an existing file). `relocate ` moves the tree; grafts `RoboCo/` into an existing destination vault without touching its own config. | ~237 | -| `roboco/vault_assets/` | Packaged templates copied by `ensure_vault_assets`: `.obsidian/` (Dataview, Kanban, graph-group config — V2 adds `Archive`/`Reports` graph color groups) + `meta/` (dashboard + kanban-board + README, V2 adds `Task Board.base` + `Reports.base` for Obsidian's core Bases plugin, and `Sync to your Mac.md`, the Syncthing/SMB/Obsidian-Sync runbook). Dataview dashboard queries now exclude `Archive/`. | — | -| `roboco/foundation/policy/injection_guard.py` | `screen_external_text` / `detect_injection` — the shared prompt-injection screen-and-neutralize (data path) and hard-deny (interactive-input path) pattern set. V2 reuses it a third time: the KB engine's ingest-time hard gate (quarantine on a hit, vs. the intake watcher's screen-and-still-process posture). | 125 | -| `roboco/services/gateway/content_actions.py` `curate_vault` | Server-side do-action: Auditor-only, re-materializes a task's note with the Auditor's `narrative` filling `## Narrative`. Inert (`invalid_state`) when the flag is off. | — | -| `roboco/mcp/do_server.py` `curate_vault` | Do-tool the Auditor calls exactly once per completed root, POSTing to `/api/v1/do/curate_vault`. | — | -| `roboco/db/tables.py` `VaultSeenNoteTable` | Dedup ledger for the intake watcher: `(note_path, content_hash)` — an unchanged note is never reprocessed; an edited one is eligible again. | — | -| `roboco/services/task.py` `_materialize_vault_note` / `list_updated_since` / `list_archive_candidates` / `sample_stale_tasks` | V2: the create-time seam + the janitor's three query methods. See `docs/map/task-service.md`. | — | - -## Data Flow - -**PROJECTION (always-on when the flag is armed).** `TaskService.create` (V2) calls `_materialize_vault_note` — best-effort, same swallow-and-log posture as every other seam — so a task's note exists from the moment it's created, not just at curation/rebuild. Three more best-effort event seams fire from existing services: `TaskService._emit_status_transition_audit` → `_touch_vault_frontmatter` patches an EXISTING note's status/team/pr fields in place (now effectively always finds one for any task created post-V2, since materialize-on-create ran; a pre-V2 task without a note is still a no-op here — the janitor's changed/sample passes are what backfill it); `JournalService`'s entry-write path → `_materialize_vault_note` writes one immutable file per non-private entry; `A2AService.send` → `_materialize_vault_note` appends to a per-thread digest file, deduped per message id via an in-body marker comment. All import `get_vault_writer()` lazily and catch every exception. - -**CURATION (root-completion hook, orchestrator-driven).** `AgentOrchestrator._dispatch_vault_curation_work` (one of the 18 tick dispatchers, gated on `obsidian_vault_enabled`) reads `TaskService.list_completed_roots_pending_vault_curation` and calls `_maybe_spawn_vault_curation` per candidate: an in-memory one-shot guard (`_board_dispatched`) plus a durable `vault_curation_dispatched` marker (survives a restart) precede a bindingless Auditor spawn. The Auditor writes one narrative paragraph and calls `curate_vault(task_id, narrative)` exactly once; the verb re-resolves the task's parent/subtasks/dependencies fresh via `assemble_task_note_data` and fully re-materializes the note, filling the `## Narrative` section a deterministic write otherwise leaves as `_Pending Auditor curation._`. - -**INTAKE (independently-gated inbox watcher).** `AgentOrchestrator._vault_intake_loop` (both `obsidian_vault_enabled` AND `vault_intake_enabled` required) ticks `VaultIntakeEngine.run_cycle` every `vault_intake_interval_seconds`. Per note under the inbox dir: skip if no `#roboco` tag; skip if already seen (path + content-hash in `vault_seen_notes`); screen the body via `screen_external_text`; extract title/description/action-items via a local-model chat call against the SCREENED text, falling back to deterministic extraction; open ONE PENDING task (`source=vault_note`, Product-Owner-assigned, `team=board`) capped by `vault_intake_max_open_drafts`/`vault_intake_max_per_cycle`; append a feedback callout (best-effort). Never starts delivery directly — the board-review path is the only door. - -**JANITOR (V2, hourly-ticked, day/week-gated).** `AgentOrchestrator._vault_janitor_loop` (gated on `obsidian_vault_enabled` alone) ticks every `JANITOR_LOOP_INTERVAL_SECONDS` (3600, no config knob) and calls `VaultJanitor.run_cycle`. Actual work only happens when the restart-proof state file (`RoboCo/_meta/.janitor_state.json`) says it's due: -- **Sweep** (due when `last_sweep` is >= 24h stale): `_reproject_changed` re-projects every task touched since the last sweep (`TaskService.list_updated_since`, ascending, paged 100 at a time, capped at `_MAX_REPROJECT_PER_CYCLE=200` per tick, one bad item logged-and-skipped rather than wedging the pass) via the shared `reproject_task`; `_verify_sample` pulls a random 20-task sample of tasks last touched before the sweep window (`sample_stale_tasks`) and repairs any whose note is missing or whose frontmatter status disagrees with the DB (via `touch_task_frontmatter`, not a full re-projection); `_archive_pass` moves terminal tasks past `vault_archive_days` into `Archive//` (see below). A capped tick advances `last_sweep`/`archive_watermark` only to the last-processed item's stamp (not "now"), so the very next hourly tick — already due again — picks up the tail with no gap. Logs one `vault_drift_repaired` line: `count` (repaired) / `archived` / `failed`. -- **Weekly report** (due when `last_report_week` != the current ISO week, and `vault_report_enabled` is on): `_run_weekly_report` pulls `MetricsService.get_velocity/get_cycle_time_by_stage/get_bottleneck_distribution/get_rework_metrics` (days=7) + `UsageService.get_summary("7d")`, renders `VaultWriter.write_org_report`, and best-effort notifies the CEO (`NotificationService.send_weekly_report_notification`) — a notification failure never invalidates the already-written note. - -**ARCHIVAL (V2, folded into the janitor sweep).** Policy: a terminal (`completed`/`cancelled`) task whose terminal timestamp (`completed_at` else `updated_at` else `created_at`) is older than `vault_archive_days` (default 30; `0` disables archival outright) moves from `RoboCo/Tasks//` to `RoboCo/Archive//Tasks//`. `TaskService.list_archive_candidates(after, before, ...)` returns terminal tasks whose terminal timestamp falls in `[watermark, cutoff)`, paged/capped identically to the changed-task pass (`_MAX_ARCHIVE_PER_CYCLE=200`). The move itself is free: `VaultWriter.write_task` is archive-aware (`TaskNoteData.archive_year` set by `vault_assembly._archive_year`) — it looks up an existing note across BOTH `Tasks/` and `Archive/` by id8, writes the new copy at the archive-aware target directory, and deletes the stale copy if it moved. Alias-based wikilinks (`[[id8|title]]`) mean nothing pointing at an archived task ever breaks. `rebuild` is archive-aware for free (routes through the same `reproject_task`), and the shipped Dataview dashboard + graph color groups exclude `Archive/`. - -**KB INGEST (V2, independently double-gated).** `AgentOrchestrator._vault_kb_loop` (BOTH `obsidian_vault_enabled` AND `vault_kb_enabled` required) ticks `VaultKBEngine.run_cycle` every `vault_kb_interval_seconds` (default 900). Per allowlisted dir in `vault_kb_dirs` (default `RoboCo/Notes`; config-load validation in `Settings._validate_vault_kb_dirs` rejects an absolute/`..`-carrying entry or one overlapping `vault_intake_dir` or a reserved projection dir): recursively scan `*.md`, skip a note that's a symlink, escapes the resolved vault root, or exceeds 64KB; content-hash-dedup against the currently-tracked `IndexType.VAULT_NOTES` docs (an unchanged note is skipped); screen the frontmatter-stripped body through `screen_external_text` as a hard GATE — a flagged note is quarantined (skipped, warn-logged, a one-line feedback callout appended, and any PRIOR indexed chunks removed if it was previously clean) rather than indexed; a clean note ingests via `OptimalService.index_vault_note` (bounded to `_MAX_INGEST_PER_CYCLE=50` per tick — the tail waits for the next cycle). A deletion pass deindexes any previously-tracked path no longer seen on disk. Consumers: `roboco_kb_search` picks up `VAULT_NOTES` for free once the enum exists; `MentorService`'s default/general domain search list includes it (labeled "Vault Notes"); `EvidenceRepo.similar_memory` includes it in claim-time briefings (kind `vault_note`), same relevance floor as learnings/playbooks; the panel's KB browser has a full type entry (nav/filter/badge/stats). - -**REBUILD/RELOCATE (operator/CLI, not agent-facing).** `python -m roboco.vault rebuild` walks every agent, then every task (via the shared `reproject_task` — archive-aware, narrative-preserving), then every non-private journal entry, then every A2A thread, and materializes the shipped `.obsidian/`/`_meta/` assets if absent. `relocate ` moves `RoboCo/` into a destination, refusing if the destination already has a `RoboCo/` subtree. - -## Config Flags - -- `ROBOCO_OBSIDIAN_VAULT_ENABLED` — master switch; off = `VaultWriter` is never invoked from any seam, `curate_vault` returns `invalid_state`, the janitor/KB loops return immediately, and `python -m roboco.vault` refuses. Config default `false`; both compose files set it `true`. -- `ROBOCO_VAULT_PATH` (default `/data/vault`) — root directory the vault materializes into; bind-mounted in both compose files. -- `ROBOCO_VAULT_INTAKE_ENABLED` — independent switch for `_vault_intake_loop`; inert unless the master switch is ALSO on. Config default `false`; both compose files set it `true`. -- `ROBOCO_VAULT_INTAKE_INTERVAL_SECONDS` / `ROBOCO_VAULT_INTAKE_DIR` / `ROBOCO_VAULT_INTAKE_MAX_PER_CYCLE` / `ROBOCO_VAULT_INTAKE_MAX_OPEN_DRAFTS` — cadence, inbox subfolder, per-cycle origination cap, rolling open-draft cap. -- `ROBOCO_VAULT_ARCHIVE_DAYS` (default `30`, `0` disables) — age past which a terminal task's note archives during the janitor sweep. Checked only under the master switch — no separate enable flag. -- `ROBOCO_VAULT_REPORT_ENABLED` (default `true`) — the janitor's weekly org-report + CEO notification. Config default `true` in both compose files (deterministic, no LLM, cheap to leave on). -- `ROBOCO_VAULT_KB_ENABLED` (default `false`) — master switch for KB ingest; off = `_vault_kb_loop` returns immediately and `IndexType.VAULT_NOTES` stays empty. NAS compose (`docker-compose.yml`) sets it `true`; the public registry compose (`docker-compose.registry.yml`) leaves it `false` (optional engines ship off). -- `ROBOCO_VAULT_KB_DIRS` (default `RoboCo/Notes`, CSV) — vault-relative folders the KB engine scans. Rejected at config load if absolute, `..`-carrying, or overlapping `vault_intake_dir`/`Tasks`/`Journals`/`A2A`/`Agents`/`Archive`/`Reports`/`_meta`/`.obsidian`. -- `ROBOCO_VAULT_KB_INTERVAL_SECONDS` (default `900`, min `60`) — KB-engine scan cadence. - -## Health - -The projection side is zero-risk by construction: every seam is best-effort and DB-free from the writer's perspective, so a filesystem or permission failure degrades to a stale/missing note, never a blocked verb. Materialize-on-create closes the V1 gap where the Dataview board only ever showed curated/rebuilt tasks — a fresh task is visible immediately. The janitor is the freshness backstop for everything best-effort seams can miss: it's restart-proof (dueness lives in a state file, not loop cadence — an orchestrator that restarts more often than daily still sweeps exactly once per elapsed day), self-healing against a corrupt/hand-edited state file (any unparseable value degrades to "no state," never a wedged loop), and every per-item drain (changed-task, sample-verify, archive) isolates failures — one bad row is logged and skipped, never aborts the pass, and re-qualifies on its next change or the next sample draw. Per-cycle caps (200 reprojects, 200 archives) mean a first-enable or long-downtime backlog drains in bounded hourly slices via the resume-marker convention (a capped tick advances the marker only to the last item it actually processed) rather than one unbounded burst. - -The KB-ingest side is the one path with real security stakes — once a vault note is agent-retrievable, unscreened note text is injection into the fleet's retrieval context, not just a drafting risk. It layers defense-in-depth: the config-load validator rejects a dangerous `vault_kb_dirs` entry outright (can't even start with an escaping/overlapping dir); the engine independently re-checks every allowlisted dir resolves under the vault root before scanning it; every individual note is re-checked for symlink-ness and resolved-path escape before it's read (belt-and-suspenders against a dir-level check being bypassed by a per-file symlink); and the injection guard runs as a hard GATE (not the intake watcher's screen-and-still-process posture) — a flagged note is never embedded, only quarantined with a visible callout so the CEO knows why. Content-hash dedup (shared with the intake watcher's ledger convention) makes both re-scans and the quarantine callout's own append idempotent — appending the callout never itself re-triggers reprocessing. - -Rebuild/relocate remain idempotent and additive-safe (`ensure_vault_assets` never overwrites an existing file), so re-running against a CEO-customized vault cannot clobber `.obsidian/`/`_meta/` edits. - -## Related - -- `docs/rag/architecture/obsidian-vault.md` — the agent-facing doc (what the Auditor and vault-intake-originated tasks actually see, plus what changed for KB retrieval) -- `docs/rag/roles/auditor.md` — the `curate_vault` verb -- `docs/map/orchestrator.md` — `_dispatch_vault_curation_work` / `_maybe_spawn_vault_curation` / `_vault_intake_loop` / `_vault_janitor_loop` / `_vault_kb_loop` -- `docs/map/task-service.md` — `_materialize_vault_note` / `list_updated_since` / `list_archive_candidates` / `sample_stale_tasks` -- `docs/map/product-strategy-research-pitch.md` — `XEngine`, the sibling engine sharing `injection_guard.screen_external_text` -- `docs/internal/specs/2026-07-09-obsidian-vault.md` — the original V1 design spec (vault layout, link-stability rationale) -- `docs/internal/specs/2026-07-11-obsidian-vault-v2.md` — the V2 spec (materialize-on-create, janitor, archival, KB ingest, weekly report, Bases, sync doc) - # Panel — RoboCo Control Panel Map ## Purpose -The Next.js 16 control panel (`panel/`, package `roboco-panel` v0.14.0) is the single operator UI for the human CEO: it drives intake/prompter chats, task/kanban management, agent observability, metrics, release approval, playbook curation, and feature-flag arming. It is served internally on port 3000 behind the nginx reverse proxy and talks to the orchestrator exclusively over relative `/api` + `/ws` URLs. +The Next.js 16 control panel (`panel/`, package `roboco-panel` v0.25.0) is the single operator UI for the human CEO: it drives intake/prompter chats, task/kanban management, agent observability, metrics, release approval, playbook curation, and feature-flag arming. It is served internally on port 3000 behind the nginx reverse proxy and talks to the orchestrator exclusively over relative `/api` + `/ws` URLs. ## Files / Structure | Path | Role | |---|---| -| `panel/package.json` | Deps: Next 16.1.1, React 19.2, TanStack Query 5.90, Radix UI, Tailwind 4, zustand 5, recharts 3, dnd-kit 6/10, axios, react-hook-form, zod 4 | +| `panel/package.json` | Deps: Next 16.1.7 (held family pin), React 19.2, TanStack Query 5.90, Radix UI, Tailwind 4, zustand 5, recharts 3, dnd-kit 6/10, `@tanstack/react-virtual` 3.14.6 (kanban column windowing), axios, react-hook-form, zod 4 | | `panel/src/app/layout.tsx` | Root layout (providers, theme, fonts) | | `panel/src/app/(dashboard)/layout.tsx` | Dashboard shell: sidebar + header + connection status | | `panel/src/app/(dashboard)/overview/page.tsx` | Overview = `` | @@ -8653,13 +9071,17 @@ The Next.js 16 control panel (`panel/`, package `roboco-panel` v0.14.0) is the s | `panel/src/app/(dashboard)/workstation/page.tsx` | Workstation: Products + Projects merged as URL-param tabs (`?tab=products\|projects`, Products first); `products/page.tsx` and `projects/page.tsx` are now server-component redirects to it | | `panel/src/app/(dashboard)/{business,git,knowledge-base,auditor,work-sessions,notifications}/page.tsx` | Per-domain pages | | `panel/src/app/(auth)/login/page.tsx` | Cloud-auth login form (email/password → `useLogin` → `/auth/login`); only reachable/relevant once `proxy.ts` starts gating the `(dashboard)` group | -| `panel/src/proxy.ts` | Next 16's rename of `middleware.ts`: probes `/auth/status` (docker-internal orchestrator URL, fails open to "off" on any error/timeout) and redirects to `/login` when cloud auth is on and no session cookie is present | -| `panel/src/components/dashboard/` | Overview cards: command-center, key-metrics, release-proposal, playbook-review-queue, ceo-approval-queue, pr-review-queue, usage-overview, team-health, active-blockers, auditor-alerts, strategy-signals, quick-actions-card, quick-actions-registry, recent-activity, `x-post-queue.tsx`, `roadmap-review-queue.tsx` | -| `panel/src/components/metrics/` | delivery-tab, usage-time-series-chart, agent/team-usage-chart, model-usage-donut, sessions-table | -| `panel/src/components/kanban/{core,shared,views}/` | core: kanban-board/column/card + bypass-preconditions; views: dev/qa/pm/pr-review kanban | +| `panel/src/app/(tg)/layout.tsx` + `(tg)/tg/page.tsx` | Telegram Mini App cockpit at `/tg`: slim shell (`id="tg-shell"` theme-scoping hook, `--tg-viewport-stable-height` sizing, `next/font/local` Share Tech Mono loaded as `--font-share-tech`, `next/script` loads the Telegram WebApp bridge `afterInteractive`) + bootstrap page that resolves `window.Telegram.WebApp`, POSTs its `initData` to `/telegram/webapp-auth` unconditionally, then renders the tabbed cockpit (or an "Open from Telegram" wall for ANY non-dev-mock bridge with empty `initData` — closes a prod-only bug where a plain-browser visit still gets the WebApp bridge object but empty `initData`, 422ing into "Couldn't sign in") | +| `panel/src/proxy.ts` | Next 16's rename of `middleware.ts`: probes `/auth/status` (docker-internal orchestrator URL, fails open to "off" on any error/timeout) and redirects to `/login` when cloud auth is on and no session cookie is present; matcher excludes `tg(?:/|$)` — the Mini App authenticates via Telegram `initData`, not the password cookie, so it must never be redirected to `/login` | +| `panel/src/components/dashboard/` | Overview cards: command-center, key-metrics, release-proposal, playbook-review-queue, ceo-approval-queue, pr-review-queue, usage-overview, team-health, active-blockers, auditor-alerts, strategy-signals, quick-actions-card, quick-actions-registry, recent-activity, `x-post-queue.tsx`, `video-post-queue.tsx`, `roadmap-review-queue.tsx`, `project-badge.tsx` (shared project/repo badge — renders nothing when neither slug nor name is set; used by both the X and video post-queue rows so a multi-project CEO can tell drafts apart) | +| `panel/src/components/metrics/` | delivery-tab, usage-time-series-chart, agent/team-usage-chart, model-usage-donut, sessions-table, `scorecards-tab.tsx` (calls `useAllMemberScorecards()` once instead of per-row `useMemberScorecard` — see `docs/map/metrics-observability.md`) | +| `panel/src/components/kanban/{core,shared,views}/` | core: `kanban-board.tsx` (`tasksByStatus` grouping memoized, `handleAction` stabilized via `useCallback` + a `tasksRef`), `kanban-card.tsx` (wrapped in `memo`), `kanban-column.tsx` (windowed via `@tanstack/react-virtual`'s `useVirtualizer`, also `memo`-wrapped — the column itself stays the dnd-kit droppable target so windowing doesn't break drag targeting) + bypass-preconditions; views: dev/qa/pm/pr-review kanban | | `panel/src/components/prompter/` | intake-form, chat-messages, chat-composer, draft-proposal-card, batch-review-card, success-card, board-review-sent-card | -| `panel/src/components/a2a/` | a2a-view.tsx (`A2AView` — the Conversations tab's full body, a pure lift of the old standalone `/a2a` page; owns the `?dm=` quick-action latch, see Gotchas), a2a-switchboard (org-chart pair cards, 45s pulse fade) + a2a-switchboard-utils (pairKey/grouping/pulse), a2a-pair-card, a2a-conversation-list (classic fallback), a2a-transcript, a2a-reply-composer (CEO chime-in on a watched conversation), a2a-new-dm-dialog (CEO opens a fresh 1:1, or preselects a validated `?dm=` deep-link target), a2a-direct-composer (CEO's own thread, no task link required), a2a-utils | -| `panel/src/components/tasks/` + `tasks/task-detail/` | task-table, create/edit-task-dialog, task-filters, acceptance-criteria-editor, dependency-selector, task-detail tabs (overview/plan/progress/commits/sessions/notes/dependencies/**findings**) | +| `panel/src/components/a2a/` | a2a-view.tsx (`A2AView` — the Conversations tab's full body, a pure lift of the old standalone `/a2a` page; owns the `?dm=` quick-action latch, see Gotchas), a2a-switchboard (org-chart pair cards, 45s pulse fade) + a2a-switchboard-utils (pairKey/grouping/pulse), a2a-pair-card, a2a-conversation-list (classic fallback), a2a-transcript, a2a-reply-composer (CEO chime-in on a watched conversation), a2a-new-dm-dialog (CEO opens a fresh 1:1, or preselects a validated `?dm=` deep-link target; exports `EXCLUDE_NON_DM_ROLES` so `tg-chat-tab.tsx`'s compose picker shares the same non-DM-capable-role exclusion instead of drifting out of sync), a2a-direct-composer (CEO's own thread, no task link required), a2a-utils | +| `panel/src/components/tasks/` + `tasks/task-detail/` | task-table (row/card extraction into `memo`-wrapped `TaskTableRow`/`TaskTableCard`, `toggleExpand` stabilized via `useCallback` — pagination-based, not virtualized), create/edit-task-dialog, task-filters, acceptance-criteria-editor, dependency-selector, task-detail tabs (overview/plan/progress/commits/sessions/notes/dependencies/**findings**), `mobile-task-board.tsx` (read-only, grouped-by-status phone board for the `/tg` cockpit; gained `tasks`/`onTaskPress` props for demo-mode + the task detail sheet) | +| `panel/src/components/tg/` | The `/tg` cockpit's tabs — now **5-tab** bottom nav (`today` default + `approvals`/`inbox`/`board`/`chat`): `tg-today-tab.tsx` (the default-opening "Today" brief — spend hero with 7-day sparkline, quick-action ring, needs-you banner, live fleet avatars, velocity bars), `tg-approvals-tab.tsx` (native card stack over `approvals/` — 7-file subdirectory: `use-approval-queue.ts`, `primary-action.tsx`, `reject-form.tsx`, `release-detail.tsx`, `x-post-detail.tsx`, `video-post-detail.tsx`, `roadmap-item-detail.tsx`), `tg-inbox-tab.tsx` (notifications + ack, `TgAvatar` sender tokens), `tg-board-tab.tsx` (wraps `mobile-task-board.tsx` + the new `tg-task-sheet.tsx` detail sheet), `tg-chat-tab.tsx` (WS-live via `/ws/system`, 10s poll fallback only while the socket is down), `ui.tsx` (shared visual-language primitives: `TgCircleAction`/`TgAvatar`/`TgSection`/`TgRow`/`TgRowIcon`/`TgStat`), `charts.tsx` (hand-rolled inline-SVG `Sparkline`/`DayBars`, no charting lib in the Mini App bundle), `tg-icons.tsx` (9-icon hand-drawn duotone set for hero surfaces, utility chrome stays lucide-react), `motion.tsx` (`useCountUp` numeral count-up hook + `TgSheet` bottom-sheet dialog wired to Telegram's native `BackButton`), `tg-task-sheet.tsx` (read-only task-detail bottom sheet: status/bounced-chip/ACs/up-to-5-open-findings/PR link) | +| `panel/src/lib/telegram/webapp.ts` | Typed wrapper over the global `window.Telegram.WebApp` (`ready`/`expand`/`initData`, `TelegramThemeParams`/`MainButton`/`BackButton`/`HapticFeedback` interfaces); `waitForTelegramWebApp` polls (100ms, 1.5s timeout) for the CDN script since it loads `afterInteractive`; `createDevMockWebApp`/`isDevMockWebApp` back the `/tg?demo=1` dev-browser fallback; null-safe `haptics` const (no-ops outside Telegram) | +| `panel/src/lib/telegram/{hooks.tsx,theme.ts,demo.ts,demo-data.ts}` | `hooks.tsx`: `TgWebAppProvider`/`useTgWebApp` context + `useMainButton`/`useBackButton` (declarative wrappers over Telegram's native buttons — no consumer touches `window.Telegram` directly). `theme.ts`: `applyTelegramTheme` maps `themeParams` → shadcn CSS vars, hex-only trust boundary, scoped to `#tg-shell` only (the desktop dashboard is untouched); `startTelegramThemeSync` re-applies on the bridge's `themeChanged` event. `demo.ts`/`demo-data.ts`: `isTgDemoMode()` (dev-only, `?demo=1`, dead-code-eliminated in prod) gates dynamically-imported `DEMO_TASKS`/`DEMO_NOTIFICATIONS`/`DEMO_RELEASE`/`DEMO_X_POSTS`/`DEMO_VIDEO_POSTS`/`DEMO_ROADMAP`/`DEMO_TODAY` fixtures | | `panel/src/components/settings/` | feature-flags-card, ai-routing-card, transcript-retention-card, self-hosted-section, `x-credentials-card.tsx` (write-only OAuth 1.0a secrets, mounted in `settings/page.tsx`) | | `panel/src/components/conventions/conventions-tab.tsx` | Per-project architecture map + health (in edit-project dialog) | | `panel/src/components/projects/`, `products/`, `agents/`, `business/`, `auditor/`, `knowledge-base/`, `git/`, `journals/`, `work-sessions/`, `notifications/`, `rate-limit/`, `layout/`, `ui/` | Per-domain component groups (`projects/` and `products/` each export a `*-view.tsx` consumed by `workstation/page.tsx`, plus a `*-card-grid.tsx` reusing the sibling table's exported badge renderers; `agents/` similarly exports `agents-fleet-view.tsx` (`AgentsFleetView`) consumed by `agents/page.tsx`'s Fleet tab, plus `agent-card.tsx`'s DM quick-action button; `journals/` similarly exports `journals-view.tsx` (`JournalsView`) consumed by `agents/page.tsx`'s Journals tab); `ui/` = Radix-based primitives (dialog, table, tabs, select, switch, required-notes-dialog, sonner toaster, markdown) | @@ -8693,12 +9115,13 @@ The Next.js 16 control panel (`panel/`, package `roboco-panel` v0.14.0) is the s | Agents — Conversations (switchboard + reply + New DM) | `app/(dashboard)/agents/page.tsx` (`?tab=conversations`) + `components/a2a/a2a-view.tsx` + `components/a2a/*` | CEO watches every agent-to-agent conversation live: default org-chart switchboard (pair cards grouped by cell/PM-chain/board, pulsing on fresh `a2a.message` frames) or the classic conversation list; drill-in shows the transcript + a reply composer that lets the CEO chime into a watched thread as itself (task-linked conversations only). "New DM" opens a fresh CEO-owned 1:1 with any DM-capable agent (no task link needed) — either from the dialog's own picker or preselected from a validated `?dm=` deep link off the Fleet tab; the recipient is woken via the a2a_request dispatch path if offline, and the CEO's own threads render with `A2ADirectComposer` instead of the reply composer. `/a2a` redirects here | | Agents — Journals | `app/(dashboard)/agents/page.tsx` (`?tab=journals`) + `components/journals/journals-view.tsx` + `components/journals/*` | Per-agent reflection log lifted into the Agents hub's third tab: agent picker (left, local search state seeded from the last saved query) + a type/task-filterable entry list (right, `JournalView`); each task-linked entry carries a Task badge + copy-UUID button. Entry detail still lives at `/journals/{entryId}`, its back-links now pointing at `/agents?tab=journals`. `/journals` redirects here | | Workstation | `app/(dashboard)/workstation/page.tsx` + `components/{products,projects}/*-view.tsx` | Products + Projects as one sidebar entry, tab-switched via `?tab=`; each surface has a Cards\|Table view toggle (default Cards, persisted per-surface via `ui-store`'s `productsView`/`projectsView`) with client-side name/cell(-count) sorting in card view; Projects' q/cell/inactive filters are local `useState`, not URL params (scroll-bounce prevention) | -| Project Settings / Conventions | `components/projects/edit-project-dialog.tsx` + `components/conventions/conventions-tab.tsx` | Per-project `.roboco/conventions.yml` map + health; Save / Restore via PR | +| Project Settings / Conventions | `components/projects/edit-project-dialog.tsx` + `components/conventions/conventions-tab.tsx` | Per-project `.roboco/conventions.yml` map + health; Save / Restore via PR. Also carries the "Forge" `` (Auto-detect/GitHub/Gitea/GitLab, `project.git_provider`) — see `docs/map/worksession-git.md` | | Usage Dashboard | `components/dashboard/usage-overview-panel.tsx` + `hooks/use-usage.ts` | Token/cost totals; live WS snapshot with HTTP-polling fallback | | Git | `app/(dashboard)/git/page.tsx` | Repository / Work Sessions tabs (business-page tab idiom, `?tab=`); `GitBrowser` (status/branches/log/diff + actions incl. confirm-gated "Clean Up Stale Branches") and `WorkSessionsView` (active sessions, search/status filter kept LOCAL not in URL params); old `/work-sessions` route now redirects to `/git?tab=sessions` | | Kanban | `components/kanban/{core,views}/*` | dnd-kit drag board; dev/qa/pm/pr-review views; drag routes through admin status-override with bypass-precondition prompt | | Task Detail | `components/tasks/task-detail/*` | Tabbed: overview, plan, progress, commits, sessions, notes, dependencies, **findings**, AC, action dialogs | | AI Providers | `app/(dashboard)/settings/ai-providers/page.tsx` + `components/settings/ai-routing-card.tsx` | Per-slug/role/global model routing | -| Telegram Mini App | `app/(tg)/tg/page.tsx` + `components/tg/*` | The CEO's phone cockpit, outside the `(dashboard)` shell: 4 tabs (Approvals — the held-artifact queues restacked; Inbox — notifications + ack; Board — `mobile-task-board.tsx` read-only grouped-by-status; Chat — A2A conversation list/compose/thread, polled not WS). Bootstraps via Telegram `initData` → `/telegram/webapp-auth`, requires both `telegram_miniapp_enabled` and `cloud_auth_enabled` armed server-side | +| Telegram Mini App | `app/(tg)/tg/page.tsx` + `components/tg/*` | The CEO's phone cockpit, outside the `(dashboard)` shell: 5 tabs — Today (default: `GET /telegram/today` spend/fleet/velocity/needs-you brief + a 4-action ops ring), Approvals (native card stack over the 4 held-artifact queues, Telegram `MainButton`/`BackButton`-driven), Inbox (notifications + ack), Board (`mobile-task-board.tsx` read-only grouped-by-status + a `tg-task-sheet.tsx` detail sheet), Chat (A2A conversation list/compose/thread, now WS-live via `/ws/system`). Bootstraps via Telegram `initData` → `/telegram/webapp-auth`, requires both `telegram_miniapp_enabled` and `cloud_auth_enabled` armed server-side; `themeParams` scoped to `#tg-shell` only; a dev-only `?demo=1` mock bridge + fixtures make the whole surface workable in a plain browser | ## Key Symbols @@ -89,7 +90,7 @@ The Next.js 16 control panel (`panel/`, package `roboco-panel` v0.14.0) is the s | `usePrompter` | hook | `hooks/use-prompter.ts` | Intake state machine: SSE refs, draft/batch extraction, turn lifecycle | | `useRateLimitWebsocket` | hook | `hooks/use-rate-limit-websocket.ts` | Single `/ws/system` subscriber; dispatches RATE_LIMIT_* + USAGE_SNAPSHOT; clears usage on disconnect | | `useA2ALiveStream` | hook | `hooks/use-websocket.ts` | Second `/ws/system` consumer (same shared connection): filters `a2a.message` frames, exposes `lastMessage`/`a2aMessages`/`isConnected` for the Conversations tab's invalidate-on-frame + switchboard pulses | -| `useA2AAdminPairs` / `useA2AConversations` / `useA2AMessages` | hooks | `hooks/use-a2a-live.ts` | TanStack Query wrappers over `a2aApi.listAdminPairs/listAdminConversations/listAdminMessages`; 30s `staleTime`, invalidated by `a2a.message` frames; `useA2AMessages` takes an optional `{ refetchInterval }` (default off — the desktop Conversations tab relies on WS invalidation) that `tg-chat-tab.tsx` sets to ~10s since the `/tg` cockpit has no WS wiring | +| `useA2AAdminPairs` / `useA2AConversations` / `useA2AMessages` | hooks | `hooks/use-a2a-live.ts` | TanStack Query wrappers over `a2aApi.listAdminPairs/listAdminConversations/listAdminMessages`; 30s `staleTime`, invalidated by `a2a.message` frames; `useA2AMessages` takes an optional `{ refetchInterval }` (default off — the desktop Conversations tab relies on WS invalidation) that `tg-chat-tab.tsx` now uses only as a fallback (`THREAD_POLL_MS=10_000`) while `/ws/system` is down, since it also gained its own `useA2ALiveStream()` WS invalidation in the V4 premium-cockpit pass | | `useReplyAsCeo` | hook | `hooks/use-a2a-live.ts` | Mutation wrapping `a2aApi.replyAsCeo`; invalidates the conversation list + the watched transcript's messages on success | | `A2AView` | comp | `components/a2a/a2a-view.tsx` | Conversations tab body — pure lift of the old standalone `/a2a` page (switchboard/list, transcript, reply/direct composer, New DM); owns the `?dm=` quick-action latch (`dmParam`/`prevDmParam` render-phase state, re-arms once `dm` is stripped) and the `?conversation=` selection, both now targeting `/agents` instead of `/a2a` | | `AgentsFleetView` | comp | `components/agents/agents-fleet-view.tsx` | Fleet tab body — pure lift of the old standalone `/agents` page (roster grids, orchestrator status, spawn/stop controls); `AgentCard`'s DM button pushes the `?dm=` deep link the Conversations tab's latch consumes | @@ -164,8 +165,9 @@ panel/ (Next.js 16, package roboco-panel v0.14.0) │ ├── layout.tsx (root layout: providers, theme, fonts) │ ├── (auth)/login/page.tsx (cloud-auth login form; gated by proxy.ts) │ ├── (tg)/ -│ │ ├── layout.tsx (slim shell, no sidebar/header; loads Telegram WebApp bridge script) -│ │ └── tg/page.tsx (bootstrap: initData → /telegram/webapp-auth, then 4-tab cockpit) +│ │ ├── layout.tsx (slim shell, #tg-shell theme scoping, Share Tech Mono font, loads Telegram WebApp bridge script) +│ │ ├── fonts/ShareTechMono-Regular.woff2 (vendored display monospace, next/font/local) +│ │ └── tg/page.tsx (bootstrap: initData → /telegram/webapp-auth, then 5-tab cockpit incl. deep-link intent routing) │ └── (dashboard)/ │ ├── layout.tsx (dashboard shell: sidebar + header + connection status) │ ├── overview/page.tsx (→ ) @@ -189,7 +191,8 @@ panel/ (Next.js 16, package roboco-panel v0.14.0) │ ├── prompter/ (intake-form, chat-messages, chat-composer, draft-proposal-card, batch-review-card, success-card, board-review-sent-card) │ ├── a2a/ (a2a-view.tsx = Conversations tab body, pure lift of the old /a2a page; a2a-switchboard + a2a-switchboard-utils, a2a-pair-card, a2a-conversation-list, a2a-transcript, a2a-reply-composer, a2a-new-dm-dialog, a2a-direct-composer, a2a-utils) │ ├── tasks/ + tasks/task-detail/ (task-table, create/edit-task-dialog, task-filters, acceptance-criteria-editor, dependency-selector; detail tabs: overview/plan/progress/commits/sessions/notes/dependencies/findings; mobile-task-board.tsx for /tg) -│ ├── tg/ (tg-tab-bar, tg-approvals-tab, tg-inbox-tab, tg-board-tab, tg-chat-tab — the /tg cockpit's own tabs) +│ ├── tg/ (tg-today-tab, tg-approvals-tab, tg-inbox-tab, tg-board-tab, tg-chat-tab, tg-tab-bar — the /tg cockpit's 5 tabs; ui.tsx, charts.tsx, tg-icons.tsx, motion.tsx, tg-task-sheet.tsx shared primitives) +│ │ └── approvals/ (use-approval-queue, primary-action, reject-form, release-detail, x-post-detail, video-post-detail, roadmap-item-detail — the native Approvals card stack) │ ├── settings/ (feature-flags-card, ai-routing-card, transcript-retention-card, self-hosted-section, x-credentials-card) │ ├── conventions/conventions-tab.tsx (per-project architecture map + health) │ ├── projects/projects-view.tsx + project-card-grid.tsx, products/products-view.tsx + product-card-grid.tsx (Workstation tab panes; card grids reuse the sibling table's exported badge renderers) @@ -203,7 +206,7 @@ panel/ (Next.js 16, package roboco-panel v0.14.0) ├── src/lib/ │ ├── api/*.ts (per-domain axios clients; client.ts shared instance; release, playbooks, prompter-live, tasks, settings, usage, cockpit, a2a, auth, x, roadmap, …) │ ├── websocket/connection.ts (WebSocketConnection + getWebSocketUrl) -│ ├── telegram/webapp.ts (window.Telegram.WebApp wrapper + waitForTelegramWebApp poll) +│ ├── telegram/ (webapp.ts wrapper + waitForTelegramWebApp poll; hooks.tsx TgWebAppProvider/useMainButton/useBackButton; theme.ts themeParams→#tg-shell CSS vars; demo.ts/demo-data.ts the ?demo=1 fixtures) │ ├── stores/ (scroll-restoration-store only; ui-store is sole-canonical in src/store/) │ └── {constants,utils,agent-definitions,agent-utils,repo-url,mock-data}.ts ├── src/proxy.ts (Next 16 rename of middleware.ts: gates (dashboard) behind cloud auth) @@ -312,6 +315,12 @@ Deliberately **not** on this card (compose/env-coupled, unsafe for a runtime tog > - `97306fe7`+`151d6e0f` (2026-07-18, PR #557, customizable Quick Actions) — the hardcoded `QuickActionsBar` (deleted) is replaced by `QuickActionsCard` (`components/dashboard/quick-actions-card.tsx`), backed by a new 23-action `QUICK_ACTIONS_REGISTRY` (`components/dashboard/quick-actions-registry.ts`, id/label/icon/href/tip per entry, tab-parameterized deep links e.g. `/git?tab=sessions`, `/metrics?tab=delivery`); `DEFAULT_QUICK_ACTION_IDS` absorbs every legacy bar destination (secretary/journals/auditor) so the swap drops nothing from a fresh install's default view. Selection persists as `quickActionIds` on `useUIStore` (new key in `partialize`, contract-tested); `resolveQuickActions`/`isKnownQuickActionId` drop stale persisted ids instead of crashing. A pencil-icon `QuickActionsCustomizeDialog` (inline in `quick-actions-card.tsx`) lets the CEO show/hide + reorder (up/down arrows) + reset to defaults; the card shows an empty-state message when every action is hidden. `command-center.tsx` and its test swap `QuickActionsBar`/`QuickActionsBarStub` references for `QuickActionsCard`/`QuickActionsCardStub`. > - `431cb2ae`+`70e53bc1` (2026-07-18, branch `feature/wave-9-agents-hub`, PR #558) — Agents hub: `/agents` gains Fleet + Conversations tabs (`?tab=`, Fleet default). The entire former `/a2a` page body moves into `components/a2a/a2a-view.tsx` (`A2AView`, a pure lift — its own `?conversation=` param keeps working, every writer now targets `/agents` preserving the rest of the query string) and a new `components/agents/agents-fleet-view.tsx` (`AgentsFleetView`) holds the old `/agents` roster body; `a2a/page.tsx` becomes a redirect shim → `/agents?tab=conversations`; the sidebar's standalone A2A nav entry is removed (folded into "Agents"). `AgentCard` gains a DM quick-action button (hidden for `EXCLUDE_NON_DM_ROLES`) pushing `/agents?tab=conversations&dm=`; `A2AViewContent` latches the `?dm=` param open once per distinct value and strips it from the URL. Follow-up fix (`70e53bc1`): the deep-linked target was being trusted blindly — `A2ANewDmDialog` now only preselects it once validated against the live roster + `EXCLUDE_NON_DM_ROLES`, and the latch re-arms once `dm` clears so a repeated identical deep link still fires. > - `cc57b9f1` (2026-07-18, branch `feature/wave-11-journals-tab`) — Journals joins the Agents hub as its third tab: `agents/page.tsx`'s `TabDef.value` gains `"journals"`, rendering `` in a third `TabsContent`. The entire former `/journals` page body moves into `components/journals/journals-view.tsx` (`JournalsView`, mirroring the `A2AView` lift — `agent`/`type`/`task` params keep working, every writer now targets `/agents` preserving the rest of the query string); `journals/page.tsx` becomes a redirect shim → `/agents?tab=journals`; `journals/[entryId]/page.tsx`'s three back-links retarget to `/agents?tab=journals` (the detail route itself is unchanged). `sidebar.tsx`'s standalone Journals nav entry is removed (its `BookOpen` icon import dropped) and the "Agents" tip text now reads "...A2A conversations, and journals"; `quick-actions-registry.ts`'s `journals` entry's `href` retargets `/journals` → `/agents?tab=journals` (mirroring the `a2a` entry's wave-9 retarget). +> - `7e01c0ce` (2026-07-18, PR #570, "project-branded drafts + project badges") — the X and video post-queue API responses now carry `project_slug`/`project_name` (backend `api/schemas/project_fields.py` unloaded-guard helper); both `x-post-queue.tsx` and `video-post-queue.tsx` render the new shared `ProjectBadge` (`project-badge.tsx`) next to the existing source-kind badge, so a multi-project CEO can tell drafts apart. Also lands `CompanyGoalsService.resolve_product_name` (project name → charter `company_goals.company_name` → "RoboCo" fallback) so release posts/videos stop hardcoding "RoboCo"; `business/goals-tab.tsx` gains the company-name input backing it. +> - `29b375f1` (2026-07-18, "session links target the owning task") — fixes a dead-link bug: `/work-sessions/` was never a real route. `work-session-card.tsx`'s "View Details" link is deleted outright; `work-session-table.tsx`'s 4 link sites (desktop-table row, table icon-button, mobile-card branch link, mobile icon-button) all retarget `href={`/tasks/${session.task_id}`}` instead, with tooltip copy reworded to "Open the owning task — the session's branch, commits, and PR live on its detail tabs". +> - `fc6d6f64` (2026-07-18, "CEO pairs join the switchboard matrix; sections collapsible") — two independent A2A switchboard fixes. (1) The switchboard's static pair matrix previously filtered by `is_human_only_role` (a spawn-semantics check) which dropped the CEO before `can_a2a_direct` (which explicitly allows CEO→anyone) ever ran, so the matrix carried zero CEO pairs — backend `agents_config.py` now excludes only `prompter`/`secretary`/`system` (matrix size 70→93; a later fix narrows it further, see `docs/map/prompts-roles-taxonomy.md`), and `a2a-switchboard-utils.ts` gains a `"ceo"` entry at the FRONT of `SECTION_ORDER` labeled `"CEO Direct"`. (2) Every switchboard section header becomes a collapse toggle: `a2a-switchboard.tsx` wraps each section in a Radix `Collapsible` (session-local `collapsed` state per `groupKey`, default open), with the `HelpTip` moved onto the inner label span rather than the `CollapsibleTrigger asChild` button — the same asChild-clobbers-`data-state` trap noted elsewhere in this doc's Gotchas (Switch/TabsTrigger). +> - **Telegram Mini App V4** (2026-07-19, PR #576 + #582 + `a072b980` fix, "Today brief, native approvals, live data, bot tier, chat bridges" + "premium Mini App cockpit") — the cockpit gets a 5th default-opening tab and a native-app visual overhaul. `TgTabBar` gains `"today"` as the first tab. New `TgTodayTab` renders off ONE `GET /telegram/today` round trip (TanStack Query, 45s `refetchInterval` + WS-invalidate on any `USAGE_SNAPSHOT` frame): `SpendHero` (cost figure, signed delta-vs-yesterday chip, a 7-day `Sparkline`), a `TgCircleAction` quick-action ring, a `NeedsYouBanner`, a live `TgAvatar` "Fleet" block, and a "Shipped this week" `DayBars` block. New shared primitives in `ui.tsx` (`TgCircleAction`/`TgAvatar`/`TgSection`/`TgRow`/`TgRowIcon`/`TgStat`, all new). `TgApprovalsTab` becomes a genuine native card stack over the new `approvals/` subdirectory, where `PrimaryAction` drives Telegram's `MainButton` via a new `useMainButton` hook and the tab itself wires `useBackButton` for focused-card back nav — both null-safely fall back to a visible button outside Telegram. `useApprovalQueue` normalizes all 4 held-draft sources into one list and surfaces `anyFailed` so a dead queue source is never silently rendered as "queue is clear". `TgChatTab` drops polling for the desktop A2A idiom: `useA2ALiveStream()` invalidates on every `a2a.message` frame with a 10s poll fallback only while `/ws/system` is down. New Telegram-native theme adoption: `webapp.ts` gains theme/button/haptics types + `createDevMockWebApp`/`isDevMockWebApp` (the `/tg?demo=1` dev-browser fallback, `NODE_ENV==="development"`-gated); new `hooks.tsx` + `theme.ts` apply `themeParams` → shadcn CSS vars **scoped to `#tg-shell`** (the desktop dashboard is untouched); `(tg)/layout.tsx` gains the `id="tg-shell"` scoping hook + Share Tech Mono font loading; `globals.css` ships a constant dark-amber `#tg-shell` skin independent of dashboard light/dark, that Telegram's `themeParams` inline-override on top of. `a072b980` (same-day fix) closes a prod-only bug: a plain-browser `/tg` visit gets the WebApp bridge with empty `initData` (the CDN script still loads outside Telegram), which was POSTed and 422'd into "Couldn't sign in" — `tg/page.tsx` now shows the "Open from Telegram" wall for ANY non-dev-mock bridge with empty `initData`. Follow-up polish (`c7605b0d` #582): `charts.tsx` (`Sparkline`/`DayBars`, theme-driven via `currentColor`); `tg-inbox-tab.tsx`/`tg-chat-tab.tsx` adopt `TgAvatar` tokens; `TgRowIcon` gains a `tone` prop so `tg-approvals-tab.tsx`'s `KIND_META` color-codes each queue kind; backend `/telegram/today` gains `spend.series`+`delta_pct`+`velocity` — see `docs/map/notification.md` for the backend side. +> - **"feature/tg-miniapp-v5"** (2026-07-19) — a further visual/interaction polish pass on the V4 cockpit, additive-only (the one relocation: Today's standalone "Approve" button becomes a 4-item ops ring — Approvals is still reachable via the tab bar). New `tg-icons.tsx` (175 lines) — 9 SVG duotone components (`IconToday`/`IconSeal`/`IconInbox`/`IconBoard`/`IconChat`/`IconShip`/`IconAckAll`/`IconSweep`/`IconFleet`), scoped to hero surfaces only (utility chrome stays lucide-react); wired into `tg-tab-bar.tsx` (the 5 tab icons) and `tg-today-tab.tsx` (the new ops ring). New `motion.tsx` (105 lines) — `useCountUp(target, durationMs=650)` (rAF ease-out-cubic, instant under `prefers-reduced-motion`) and `TgSheet` (bottom-sheet dialog wired to Telegram's native `BackButton`, backdrop-tap-to-close, haptic on open); used by the new `tg-task-sheet.tsx` and by `tg-today-tab.tsx`'s Fleet/Sweep sheets. New `tg-task-sheet.tsx` (151 lines) — `TgTaskSheet`: read-only task-detail bottom sheet (status badge, "bounced ×N" chip, ACs, up to 5 open revision findings via `useTaskFindings` with an overflow line, "Open PR #N" link; skips the findings fetch in demo mode); rendered from `tg-board-tab.tsx` alongside `MobileTaskBoard` via a new `onTaskPress` prop. New vendored `ShareTechMono-Regular.woff2` under `app/(tg)/fonts/`, loaded via `next/font/local` as `--font-share-tech` on `#tg-shell`, applied only to the display voice (labels/numerals/wordmark) — never body text. `tg-today-tab.tsx` (largest diff) replaces the old single "Approve" action with a 4-item ops ring (Ship/Ack-all/Sweep/Fleet): `SpendHero`'s numeral gains count-up, a new `FleetSheet` shows the full roster (opened by tapping the truncated 3-agent preview), `runAckAll` bulk-acks via the existing notifications API, `runSweep` calls the existing `gitApi.cleanupBranches` per has-token project (no new backend surface), plus a blinking-cursor "ROBOCO_" wordmark header. `demo-data.ts` gains `DEMO_TASKS`/`DEMO_NOTIFICATIONS` fixtures. +> - **"panel-perf-p3-p4"** (2026-07-19) — kanban + task-table + scorecards perf pass, paired with the backend N+1 fix in `docs/map/metrics-observability.md`. `kanban-board.tsx`: `tasksByStatus` grouping wrapped in `useMemo`, `handleAction` stabilized via `useCallback` + a `tasksRef` so child memoization actually holds. `kanban-card.tsx`: wrapped in `React.memo`. `kanban-column.tsx`: genuine windowing via the new `@tanstack/react-virtual` dependency — `useVirtualizer` (132px estimated card height, overscan 6) replaces a plain `.map()`, only visible virtual rows mount, absolute-positioned + `measureElement`'d for real-height correction; the column itself stays the dnd-kit droppable target so windowing doesn't break drag targeting. `task-table.tsx` (673 changed lines, no virtualization): row/card JSX extracted into top-level `memo`-wrapped `TaskTableRow`/`TaskTableCard`, fed stable props (`toggleExpand` via `useCallback`, lookup maps passed through) — pagination (pre-existing) is the actual bound on rendered rows, not windowing. `scorecards-tab.tsx`: `MemberRow` no longer self-fetches — `ScorecardsTabContent` calls the new `useAllMemberScorecards()` once and passes each row its slice via a `useMemo`'d Map; a table-level "Failed to load member scorecards" banner replaces the old per-row failure cell. ## Regression Risks diff --git a/docs/map/product-strategy-research-pitch.md b/docs/map/product-strategy-research-pitch.md index f819ddce..d32c7599 100644 --- a/docs/map/product-strategy-research-pitch.md +++ b/docs/map/product-strategy-research-pitch.md @@ -11,16 +11,16 @@ The product / strategy / research / pitch slice covers the "company layer" above | `roboco/services/project.py` | CRUD + git-token encryption + cell access control for Projects (git repos) | 604 | | `roboco/services/product.py` | Product CRUD + per-cell `project_for` routing resolver + idempotent cell-map replace | 152 | | `roboco/services/kanban.py` | Role-specific kanban board views (dev/qa/documenter/pm/main-pm/board) from task data | 587 | -| `roboco/services/company_goals.py` | CRUD for the singleton company charter (north star + objectives + constraints + policy) | 83 | +| `roboco/services/company_goals.py` | CRUD for the singleton company charter (north star + objectives + constraints + policy + brand_voice + company_name); `resolve_product_name` is the shared product-name fallback chain `XEngine`/`VideoEngine` both call | 110 | | `roboco/services/strategy_engine.py` | Dormant "engine 2": assesses company state vs goals, notify-only to CEO | 111 | | `roboco/services/research.py` | Pluggable web-search/fetch — provider adapters (Tavily/Brave/Exa/Null) + clamping service | 431 | | `roboco/services/research_quota.py` | Per-agent UTC-daily Redis quota counter for research calls (fail-open) | 78 | | `roboco/services/pitch.py` | Board pitch CRUD + CEO approve → provision repos/Projects(+Product) + seed Main-PM task | 274 | -| `roboco/services/github_provisioning.py` | The only service that CREATES GitHub repos (POST `/orgs/{org}/repos`) | 174 | +| `roboco/services/github_provisioning.py` | The only service that CREATES repos for pitch provisioning — now provider-aware (GitHub/Gitea/GitLab, Phase 4 forge parity), despite the GitHub-flavored name (kept for backward compatibility) | 232 | | `roboco/services/roadmap_engine.py` | Dormant weekly engine: originates ONE held roadmap-exploration task for the Product Owner (default off) | 111 | | `roboco/services/roadmap_service.py` | CEO's per-item approve/reject glue over a held roadmap cycle; approve materializes a BACKLOG task | 211 | | `roboco/api/routes/roadmap.py` | CEO-only routes: list open cycles, approve/reject one item | 124 | -| `roboco/services/x_engine.py` | Dormant "engine 4": drafts X (Twitter) release posts (event hook), mention replies (poll), and — new — feature-spotlight explorations (dormant interval, spawns Head of Marketing), ALL held for CEO approval (default off) | 463 | +| `roboco/services/x_engine.py` | Dormant "engine 4": drafts X (Twitter) release posts (event hook), mention replies (poll), and feature-spotlight explorations (dormant interval, spawns Head of Marketing), ALL held for CEO approval (default off); prompt builders take a `product_name` param resolved via `CompanyGoalsService.resolve_product_name` instead of hardcoding "RoboCo" | 871 | | `roboco/services/x_post_service.py` | CEO's approve/reject over a held X draft; approve posts via a Redis single-flight lock, idempotent on already-posted AND on already-rejected (CANCELLED) | 298 | | `roboco/services/x_client.py` | OAuth 1.0a HMAC-SHA1 X API client (`LiveXClient`) + `NullXClient` (no creds, never egresses) + `build_x_client` factory | 318 | | `roboco/services/x_credentials.py` | Singleton Fernet-encrypted OAuth 1.0a credential CRUD; decrypts server-side only | 140 | @@ -77,12 +77,13 @@ The product / strategy / research / pitch slice covers the "company layer" above | `PitchService._register_topology` | method | pitch.py:201 | Multi-cell → Product (reuse existing by slug + refresh cell map); single-cell → seed project only | | `PitchService._seed_main_pm_task` | method | pitch.py:234 | Creates PENDING Main-PM CODE task (`source="pitch"`, `confirmed_by_human=True`) | | `PitchService._proposed_or_raise` | method | pitch.py:152 | 404 if missing, 409 if not `proposed` (no re-deciding) | -| `GitHubProvisioningService` | class | github_provisioning.py:45 | Create private repos in configured org | -| `GitHubProvisioningService.enabled` | prop | github_provisioning.py:67 | True only when master switch + token + org all set | -| `GitHubProvisioningService.create_repo` | method | github_provisioning.py:81 | POST `/orgs/{org}/repos` with `auto_init=true`; handles GitHub 422 "already exists" idempotently via `_fetch_existing_repo` (#83/#84) | -| `GitHubProvisioningService._fetch_existing_repo` | method | github_provisioning.py:140 | GET `org/name` and reconstruct `ProvisionedRepo` — called on 422 to reuse an orphaned repo from a rolled-back prior approval | -| `_GITHUB_REPO_EXISTS_STATUS` | constant | github_provisioning.py:42 | `422` — GitHub's "name already exists" status sentinel | -| `ProvisionedRepo` / `ProvisioningError` / `ProvisioningDisabledError` | dataclass/exc | github_provisioning.py:32 / 23 / 27 | Result + error types | +| `GitHubProvisioningService` | class | github_provisioning.py:81 | Create private repos for pitch provisioning — Phase 4 forge parity: provider-dispatched via `_build_provider`, not GitHub-only despite the class name | +| `GitHubProvisioningService.enabled` | prop | github_provisioning.py:123 | True only when master switch + token + org all set; ALSO requires `ROBOCO_PROVISIONING_HOST` when the provider is gitlab/gitea (self-hosted needs a host, github.com doesn't) | +| `GitHubProvisioningService.create_repo` | method | github_provisioning.py:142 | Provider-dispatched repo creation with `auto_init=true`; handles the "already exists" response idempotently across all three forges via `_fetch_existing_repo`/`_is_already_exists` (#83/#84) | +| `GitHubProvisioningService._fetch_existing_repo` | method | github_provisioning.py:203 | GET the existing repo and reconstruct `ProvisionedRepo` — called on an "already exists" response to reuse an orphaned repo from a rolled-back prior approval | +| `_build_provider` | func | github_provisioning.py:68 | Picks the concrete provider (`GitHubProvider`/`GiteaProvider`/`GitLabProvider` — a `Union`, not the `GitProvider` ABC, since provisioning needs `client=`/`timeout=` kwargs the ABC doesn't declare) by `ROBOCO_PROVISIONING_PROVIDER` | +| `_is_already_exists` | func | github_provisioning.py:61 | Matches GitHub's 422, Gitea's 409/422, and GitLab's reshaped 422 "already exists"/"has already been taken" by status+phrase | +| `ProvisionedRepo` / `ProvisioningError` / `ProvisioningDisabledError` | dataclass/exc | github_provisioning.py | Result + error types | | `RoadmapEngine` | class | roadmap_engine.py:49 | Dormant "engine 3": mirrors the release-manager "detect → originate a CEO-gated artifact → hold" shape, but the artifact is a cycle the PO *authors*, not a report the engine assembles | | `RoadmapEngine.run_cycle` | method | roadmap_engine.py:54 | No-op unless `roadmap_engine_enabled`, a cycle is already open (`list_open_roadmap_cycles`), or the RoboCo project isn't resolvable; else opens ONE held PENDING exploration task assigned to the Product Owner | | `RoadmapService` | class | roadmap_service.py:50 | List / approve / reject items within the open roadmap cycle(s) | @@ -92,19 +93,21 @@ The product / strategy / research / pitch slice covers the "company layer" above | `RoadmapService._maybe_complete_cycle` | staticmethod | roadmap_service.py:202 | Completes the exploration task once every item on it is terminal (approved/rejected) | | `RoadmapItemResult` | dataclass | roadmap_service.py:37 | Outcome of one approve/reject call (status/item_id/materialized_task_id/detail) | | `get_roadmap_engine` / `get_roadmap_service` | factory | roadmap_engine.py:109 / roadmap_service.py:209 | Session-bound constructors | -| `XEngine` | class | x_engine.py:150 | Dormant "engine 4": mirrors the release-manager "detect → originate a CEO-gated artifact → hold" shape across THREE responsibilities — release posts, mention replies, feature spotlights | -| `XEngine._voice_guide` | method | x_engine.py:173 | Baseline house-voice constant (`_HOM_VOICE`) plus the CEO's `company_goals.brand_voice` sample when set — feeds release/reply prompts AND is the mechanism the HoM identity file points to for its own drafting | -| `XEngine.draft_release_post` | method | x_engine.py:192 | Event-driven hook (called from `ReleaseProposalService.approve`'s publish-success branch); local-model-drafted, deduped per version, capped by `x_max_open_posts` | -| `XEngine.run_cycle` | method | x_engine.py:255 | Periodic mentions poll; no-op unless `x_engine_enabled` AND `x_replies_enabled`; filters bot-like/low-engagement mentions, dedupes by mention id (`XSeenMentionTable`); each mention's text is run through `screen_external_text` before the local-model reply prompt sees it | +| `XEngine` | class | x_engine.py:230 | Dormant "engine 4": mirrors the release-manager "detect → originate a CEO-gated artifact → hold" shape across THREE responsibilities — release posts, mention replies, feature spotlights | +| `XEngine._voice_guide` | method | x_engine.py:259 | `_voice_guide(product_name)`: baseline house-voice constant (`_HOM_VOICE`) plus the CEO's `company_goals.brand_voice` sample when set — feeds release/reply prompts AND is the mechanism the HoM identity file points to for its own drafting; `product_name` is resolved once per call site via `CompanyGoalsService.resolve_product_name(project)` (project's own name → charter `company_name` → "RoboCo" literal), not hardcoded | +| `XEngine.draft_release_post` | method | x_engine.py:279 | Event-driven hook (called from `ReleaseProposalService.approve`'s publish-success branch); local-model-drafted, deduped per version, capped by `x_max_open_posts`; resolves and threads `product_name` from the release's own project | +| `XEngine.run_cycle` | method | x_engine.py:353 | Periodic mentions poll; no-op unless `x_engine_enabled` AND `x_replies_enabled`; filters bot-like/low-engagement mentions, dedupes by mention id (`XSeenMentionTable`); each mention's text is run through `screen_external_text` before the local-model reply prompt sees it; resolves `product_name` once per cycle and threads it through `_originate_reply`/`_draft_reply_body`/`_reply_prompt` | | `screen_external_text` | function | foundation/policy/injection_guard.py:95 | Shared screen-and-neutralize guard for unattended attacker-writable text feeds (X mentions, vault inbox notes): wraps the text in an untrusted-content envelope and flags any matched injection-pattern LINE inline — nothing is removed, so the CEO-facing draft still shows what the source really said | -| `XEngine.open_feature_spotlight_exploration` | method | x_engine.py:337 | No-ops unless `x_engine_enabled` AND `x_feature_spotlight_enabled`, no creds, a cycle already open, the open-post cap reached, or project unresolvable; else opens ONE held PENDING exploration task for the Head of Marketing (`source=x_feature_exploration`) carrying a `x_seen_features` marker snapshot | -| `XEngine.materialize_feature_spotlight` | method | x_engine.py:433 | Called from the `propose_feature_spotlight` do-tool: marks the feature slug seen (`XSeenFeatureTable`), creates the held draft (`source=x_feature`, identical shape to a release/reply draft), completes the exploration task | +| `XEngine.open_feature_spotlight_exploration` | method | x_engine.py:487 | No-ops unless `x_engine_enabled` AND `x_feature_spotlight_enabled`, no creds, a cycle already open, the open-post cap reached, or project unresolvable; else opens ONE held PENDING exploration task for the Head of Marketing (`source=x_feature_exploration`) carrying a `x_seen_features` marker snapshot; the description is built by `_feature_exploration_description(product_name)`, no longer the fixed `_FEATURE_EXPLORATION_DESCRIPTION` string | +| `XEngine.materialize_feature_spotlight` | method | x_engine.py:841 | Called from the `propose_feature_spotlight` do-tool: marks the feature slug seen (`XSeenFeatureTable`), creates the held draft (`source=x_feature`, identical shape to a release/reply draft), completes the exploration task | | `XPostService.approve` | method | x_post_service.py:92 | The ONLY caller of `x_client.post_tweet`; Redis single-flight lock, re-reads task under lock, idempotent on an already-posted draft (`already_posted`); a CANCELLED draft is refused both pre-lock and re-checked under lock (`already_rejected`) — a stale approve (e.g. a queued Telegram button) can't resurrect a draft the CEO already rejected | | `XPostService.reject` | method | x_post_service.py:251 | Records the CEO's reason; cancels the held draft | | `XClient` / `NullXClient` / `LiveXClient` | ABC/class | x_client.py:150 / 166 / 186 | `NullXClient.configured` is False (no creds) — drafting still runs (content nobody can post is a no-op upstream), just never originates; `LiveXClient` signs OAuth 1.0a HMAC-SHA1 | | `build_x_client` | factory | x_client.py:306 | Returns `LiveXClient` when credentials decrypt, else `NullXClient` | | `XCredentialsService.set_credentials` / `.get_decrypted` | method | x_credentials.py:61 / 116 | All-or-nothing Fernet-encrypted singleton credential set/clear; decrypts server-side only, never exposed to agents | -| `get_x_engine` | factory | x_engine.py:461 | Session-bound constructor (optional injected `XClient` for tests) | +| `get_x_engine` | factory | x_engine.py:869 | Session-bound constructor (optional injected `XClient` for tests) | +| `CompanyGoalsService.resolve_product_name` | method | company_goals.py:79 | The shared product-name fallback chain: `project.name` if set, else the charter's `company_name`, else the "RoboCo" literal — single source so `XEngine`/`VideoEngine` can't drift apart on branding | +| `task_project_fields` | func | api/schemas/project_fields.py:19 | `(project_slug, project_name)` or `(None, None)` for a task response — `sa_inspect(task).unloaded` guard before touching `task.project` (a freshly-created task can have an unloaded relationship); shared by the X and video queue response builders so a multi-project CEO can tell drafts apart via the panel's `ProjectBadge` | ## Data Flow @@ -319,6 +322,8 @@ product-strategy-research-pitch | `ROBOCO_GITHUB_API_BASE_URL` | `https://api.github.com` | config.py:327 | Override for GitHub Enterprise | | `ROBOCO_PROVISIONING_TIMEOUT_SECONDS` | `30.0` | config.py:331 | Outbound GitHub provisioning timeout | | `ROBOCO_PROVISIONING_REPO_PRIVATE` | `True` | config.py:336 | Whether provisioned repos are private | +| `ROBOCO_PROVISIONING_PROVIDER` | `github` | config.py:570 | Phase 4 forge parity: `github`/`gitlab`/`gitea` selects the concrete provisioning target via `_build_provider` | +| `ROBOCO_PROVISIONING_HOST` | `""` | config.py:579 | Self-hosted forge host (e.g. `gitlab.example.com`); `.enabled` additionally requires this when `provisioning_provider` is gitlab/gitea (ignored for github/gitlab.com) | | `ROBOCO_STRATEGY_ENGINE_ENABLED` | `False` | config.py:348 | Master switch — loop never starts when off | | `ROBOCO_STRATEGY_ENGINE_INTERVAL_SECONDS` | `1800` | config.py:354 | Seconds between strategy assessment passes | | `ROBOCO_STRATEGY_STRANDED_BLOCKED_MINUTES` | `120` | config.py:360 | Blocked-task threshold for "stranded" observation | @@ -367,6 +372,9 @@ product-strategy-research-pitch > - `b3558d4e` ([chore] complexity: split 5 C-rank blocks to <=B, 2026-06-30): `kanban.py` `get_main_pm_board_flat` — refactored if/elif routing to a dict-dispatch (`status_col` + `team_col` maps) for xenon complexity gate; no functional change. > - **v0.18.0** (2026-07-04): the X feature-spotlight content in this slice (`XEngine` feature-spotlight methods, `_x_feature_spotlight_loop`/`_dispatch_feature_spotlight_exploration`, migration 061, `x_feature_spotlight_enabled`) was authored directly into this file's Files/Key Symbols/Data Flow/Mermaid/Logical Tree/Entry Points sections at implementation time rather than landing as a dated delta — noted here for changelog continuity; the body text above is current as of this date. (Config Flags is unchanged — the X-engine flags live in deployment-tooling.md's comprehensive list, not here.) > - `11915f36` (PR #551, Telegram V2 security follow-up, 2026-07-17): `x_post_service.py` — `XPostService.approve`/`_approve_locked` add a CANCELLED-task guard (pre-lock and re-checked under lock) returning a new `already_rejected` status, closing a live-reproduced approve-after-reject hole reachable via a stale Telegram Approve button (or a replayed HTTP call). +> - `7e01c0ce` (PR #570, "project-branded drafts + project badges", 2026-07-18): migration 075 adds `company_goals.company_name`; `CompanyGoalsService.resolve_product_name` (company_goals.py:79) is the new single fallback chain (project name → charter `company_name` → "RoboCo") consumed by both `XEngine._voice_guide`/`draft_release_post` and `VideoEngine` (see `docs/map/video-engine.md`) so their prompt builders stop hardcoding "RoboCo". New `roboco/api/schemas/project_fields.py`'s `task_project_fields` helper adds `project_slug`/`project_name` to the X and video post-queue API responses (`api/routes/x.py`, `api/routes/video.py`); the panel renders them via a shared `ProjectBadge` — see `docs/map/panel.md`. +> - `461a6e1a`+`96401f4c`+`5f32d876` (Phases 1/2-3/4, 2026-07-18/19, #571/#575/#581) — Phase 4 makes `GitHubProvisioningService` provider-aware: `_build_provider` (github_provisioning.py:68) dispatches to `GitHubProvider`/`GiteaProvider`/`GitLabProvider` by `ROBOCO_PROVISIONING_PROVIDER`, `.enabled` additionally requires `ROBOCO_PROVISIONING_HOST` for gitlab/gitea, and `_is_already_exists` (github_provisioning.py:61) matches the "already exists" idempotency signal across all three forges' differing status codes/phrasing. The forge transport package itself (`GitProvider`/`ForgeRouter`/provider implementations) is documented in `docs/map/worksession-git.md` — this slice only covers the provisioning consumer. +> - `a0baf94b` ("agnosticism-residue", agnosticism audit items B6/B8): `x_engine.py`'s remaining hardcoded `"RoboCo"` literals (the reply-prompt builder and the feature-spotlight exploration description — `draft_release_post`/`_voice_guide` were already fixed by `7e01c0ce` above) are threaded out: `_reply_prompt` gains a `product_name` param, `_FEATURE_EXPLORATION_DESCRIPTION` (a module constant) becomes `_feature_exploration_description(product_name)` (a function), and `run_cycle`/`open_feature_spotlight_exploration` each resolve `product_name` once via `resolve_product_name` and thread it through. ## Regression Risks diff --git a/docs/map/prompts-roles-taxonomy.md b/docs/map/prompts-roles-taxonomy.md index 5ff832a7..ffe7db15 100644 --- a/docs/map/prompts-roles-taxonomy.md +++ b/docs/map/prompts-roles-taxonomy.md @@ -116,8 +116,10 @@ This slice is the prompt-composition pipeline and the role/team/permission taxon | _check_cell_pm_a2a | function | roboco/agents_config.py:590 | A2A permission for cell PM (own cell / other PMs / main-pm allowed; board escalated) | | _check_cell_member_a2a | function | roboco/agents_config.py:604 | A2A permission for cell members (same-cell allowed; cross-cell via PMs) | | _check_main_pm_a2a | function | roboco/agents_config.py:624 | A2A permission for main PM (_MAIN_PM_TARGETS allowed) | -| can_a2a_direct | function | roboco/agents_config.py:632 | (allowed, error) for direct A2A from one agent to another; routes CEO via notify, board/main_pm/cell-member via handlers | +| can_a2a_direct | function | roboco/agents_config.py:632 | (allowed, error) for direct A2A from one agent to another; routes CEO via notify, board/main_pm/cell-member via handlers; the CEO branch now consults `_check_ceo_a2a` (below) instead of an unconditional `True` | +| _check_ceo_a2a | function | roboco/agents_config.py:635 | CEO-initiated A2A target check: refuses `to_role in NO_COMMS_ROLES` (auditor/pr_reviewer/prompter/secretary — no `dm`/`read_a2a` on the manifest, so nothing on the other end could read or answer it), else allowed | | get_a2a_route_hint | function | roboco/agents_config.py:670 | Human-readable routing hint for an A2A message | +| A2A_ALLOWED_PAIRS | constant | roboco/agents_config.py | Statically-derived (via `_compute_a2a_allowed_pairs()`, calling `can_a2a_direct` for every pair) set of legal A2A pairs — sized 88 (`ceo` group 18) after `_check_ceo_a2a` excludes no-comms roles; the panel switchboard's section matrix reads off this same computation | | _PATTERNS | list | roboco/agent_sdk/prompt_guard.py:28 | Five (regex, reason) injection patterns: ignore-previous, role-override, fake role prefix, control-token mimicry, fake executive-order | | detect_injection | function | roboco/agent_sdk/prompt_guard.py:63 | Return deny reason if text matches an injection pattern (lowercased), else None | | refusal_message | function | roboco/agent_sdk/prompt_guard.py:72 | Guidance string shown on denial (mirrors bash hook text) | @@ -277,6 +279,10 @@ prompts-roles-taxonomy slice > **v0.19.0** (2026-07-05): Ponytail build-laziness doctrine bundled with Fable — `ponytail_doctrine_layer(prompts_path, role)` in `roboco/agents/factories/_base.py`, gated on the same `fable_mode_enabled` flag (no separate flag — ponytail is Fable's complementary build-doctrine), slotted into compose_prompt immediately after `fable_doctrine_layer`. Role-scoped: developers (`AgentRole.DEVELOPER`) → `agents/prompts/doctrine/ponytail.md` (the full ladder: YAGNI → reuse-in-this-codebase → stdlib → native-platform → installed-dep → one-line → minimal, the rules, the Intensity table, the `ponytail:` comment convention) plus a trailing `**Operative intensity: {settings.ponytail_intensity}.**` directive; every other role → `agents/prompts/doctrine/ponytail-ethos.md` (ethos-only — the code-mechanics rungs and the Intensity table are dropped so they can't leak into prose artifacts like task plans / review notes / docs). Both files vendored from the ponytail plugin (MIT, Copyright (c) 2026 DietrichGebert), trimmed, YAML frontmatter stripped, and carry a 5-point RoboCo preamble that makes the ladder yield to the Architectural Conventions Standard (placement), the 80% coverage gate + QA review + self-verification, the per-team design bar, task hygiene, and reviewer feedback — overlap mitigated by scoping, not deletion. `ROBOCO_PONYTAIL_INTENSITY` (lite/full/ultra, default full; `roboco/config.py` `ponytail_intensity`, a string value — NOT a feature flag) selects the developer's operative intensity; non-developers get no dial. Prompt-only — no hooks, no grok-path changes; a flag-off spawn is byte-for-byte unchanged. > > **PR #544** (2026-07-17, `fd621f0d`): The design bar's web dials (`DESIGN_VARIANCE`/`MOTION_INTENSITY`/`VISUAL_DENSITY`) were silently steering a `source=video` authoring task toward "dense product UI → motion 2-3" — wrong for a marketing film. `agents/prompts/teams/ux_ui.md` gains a one-line video-mode override (see Files above) telling the dev the web dials do not apply to a video-authoring task and pointing it at `motion/README.md`'s cinematography bar + the vendored `motion/skills/` doctrine instead. Doc-only within this slice — the runtime half of the same PR (playwright MCP registered for the video-authoring ux-dev spawn, `_is_video_authoring_spawn`) is documented in `docs/map/video-engine.md` and `docs/map/orchestrator.md`. +> +> **"taste-skill-aesthetics"** (deferred half of the Design bar work): two more `Leonxlnx/taste-skill` (MIT) distillations layered onto the core Design bar. A `## Niche aesthetic vocabularies` section — identical body in both `agents/prompts/teams/frontend.md` and `ux_ui.md` — names three opt-in visual systems (industrial brutalist, minimalist editorial, premium agency), each keyed onto the same three dials (a vocabulary changes *what* the dials produce, never whether they apply); picked only when a task brief explicitly calls for one. A `## Image direction` section lives in `ux_ui.md` only (composition variety, palette discipline, anti-slop imagery, iconography, mockup/device-frame conventions, cross-asset set consistency) — `frontend.md` carries a one-line pointer instead of duplicating it. Doc-only, same `tests/unit/agents/test_design_bar_layer.py` guard as the core bar; no compose_prompt/flag change. +> +> **CEO A2A pairs, two sequential fixes.** `fc6d6f64` (2026-07-18, "CEO pairs join the switchboard matrix"): the static pair matrix previously filtered by `is_human_only_role` (a spawn-semantics check) which dropped the CEO before `can_a2a_direct` (which explicitly allows CEO→anyone) ever ran, so `A2A_ALLOWED_PAIRS` carried ZERO CEO pairs; `agents_config.py` now excludes only `prompter`/`secretary`/`system` from that pre-filter (matrix size 70→93). `56b6693e` ("security-hygiene-sweep") then narrows it correctly: `_check_ceo_a2a` (agents_config.py:635, consulted from `can_a2a_direct`'s CEO branch — previously an unconditional `True`) refuses a CEO target in `NO_COMMS_ROLES` (see `docs/map/foundation-policy-misc.md`), shrinking `A2A_ALLOWED_PAIRS` 93→88 (`ceo` group 23→18) with no separate matrix edit — the derivation is fully automatic off `can_a2a_direct`. See `docs/map/panel.md` for the switchboard's `"CEO Direct"` section + collapsible-sections panel change, and `docs/map/a2a-audit-journal-permissions.md` for the conversation-creation-time refusal. ## Regression Risks diff --git a/docs/map/tests.md b/docs/map/tests.md index 96cf5133..da33cb11 100644 --- a/docs/map/tests.md +++ b/docs/map/tests.md @@ -23,14 +23,18 @@ The pytest test suite for RoboCo: 571 test_*.py files across tests/foundation, t | tests/unit/ | ~430 unit tests mirroring roboco/: agents, api (+routes/+schemas), billing, config, conventions, db, enforcement, events, foundation/policy, gateway (105), llm, mcp_servers, migrations, models, runtime (64), scripts, services (120), templates, utils | | | tests/unit/gateway/ | 105 Choreographer/verb-runner unit tests — the largest single cluster: every intent verb's guards, envelopes, evidence, claim locks, lane barrier, pr gate, conventions gate, content actions | | | tests/unit/runtime/ | 64 orchestrator unit tests: spawn/manifest/cwd/worktree, reaper, respawn persistence, rate-limit/overload sweeps, ci_watch/dep_update/release/self_heal loops, no_spawn_human_roles, per_dev_lane_queue, readopt_running_agents | | -| tests/unit/services/ | 120 service unit tests: task, git (+worktree), workspace, work_session, release_executor/readiness/manager, sequencing, conventions, playbook, notification, rate_limit_tracker, optimal_brain/ (10) | | +| tests/unit/services/ | 120+ service unit tests: task, git (+worktree), workspace, work_session, release_executor/readiness/manager, sequencing, conventions, playbook, notification, rate_limit_tracker, optimal_brain/ (10), forge/ (`test_gitea_provider.py`, `test_gitlab_provider.py`, `test_router.py` — mocked-transport coverage of the forge package, distinct from the live contract suites below) | | | tests/e2e_smoke/ | Scripted-agent smoke tests against the live in-process orchestrator; separate from the default pytest collection (run by the PR/NAS smoke gate) | | | tests/e2e_smoke/harness.py | E2E harness: E2EStack app + orchestrator client + per-test agent manifests; used by the e2e_smoke tier | ~520 | +| tests/e2e_smoke/test_gitea_live.py | Live-Gitea contract suite for `GiteaProvider` — fully self-seeding (creates its own uniquely-named repo, pushes real commits) against a real Gitea instance; skipped unless `ROBOCO_GITEA_E2E_URL`/`ROBOCO_GITEA_E2E_TOKEN` are both set; exercises PR open → duplicate-409→422 reshape → list/filter → diff → comment review → commit-status CI reshape → squash merge → branch delete → release, plus the git-CLI Basic-auth extraheader claim | 250 | +| tests/e2e_smoke/test_gitlab_live.py | Live-GitLab contract suite for `GitLabProvider` — mirrors `test_gitea_live.py`, self-seeding against a real GitLab instance (gitlab.com works, project deleted afterward best-effort); skipped unless `ROBOCO_GITLAB_E2E_URL`/`ROBOCO_GITLAB_E2E_TOKEN` are both set; exercises MR open → duplicate reshape → GitHub-shape adaptation → diff reassembly → note review → commit-status CI reshape → squash merge → branch delete → release → the oauth2 Basic-auth git-CLI claim | 265 | ## E2E smoke harness The `tests/e2e_smoke/` tier runs scripted agents against a real in-process FastAPI app and orchestrator. It is **not** collected by the default `uv run pytest` invocation; it is exercised separately by the PR gate / NAS smoke run. The harness in `tests/e2e_smoke/harness.py` builds an `E2EStack`, mounts the orchestrator client, and gives each test a per-agent manifest. +**Live-forge contract suites** (`test_gitea_live.py` / `test_gitlab_live.py`) are a different animal from the scripted-agent harness above — no `E2EStack`, no orchestrator, no manifests. Each drives its `GitProvider` implementation directly against a REAL forge instance (a dockerized `gitea/gitea` or real `gitlab.com`), fully self-seeding (creates its own repo/project, pushes real commits via subprocess `git`, tears down after). Both are `pytest.mark.skipif`'d off unless their pair of `ROBOCO_{GITEA,GITLAB}_E2E_{URL,TOKEN}` env vars is set, so a normal CI/local run never attempts network I/O against a real forge. These are what caught the slash-encoding (branch names with `/` need `quote()`-encoding before hitting Gitea's router) and the http-vs-https scheme gaps that the mocked-transport unit tests in `tests/unit/services/forge/` could not have found, since those mock the HTTP layer entirely. + Recent harness hardening for the auditor-revival slice (PR #498, task `8323cd50`): - `tests/conftest.py` now catches a missing `pgvector` extension when creating the ephemeral test database and continues with a warning. The core schema does not require it and the e2e smoke suite does not exercise RAG, so lightweight Postgres sandboxes can run the suite without the pgvector package. @@ -240,6 +244,7 @@ tests/ > - **1f129199** auditor-trigger e2e smoke test: added `tests/e2e_smoke/test_auditor_triggers.py` exercising scheduled sweep and reactive QA-fail alert paths end-to-end, plus harness mount of `/api/notifications` so `_dispatch_audit_work` can poll ALERT rows. > - **babffe0a** fix(e2e_smoke): repair auditor-trigger smoke tests and harden harness (#498): fixed `tests/e2e_smoke/test_auditor_triggers.py` so scheduled/reactive auditor-trigger tests reach their spawn assertions, hardened `tests/conftest.py` to tolerate missing pgvector, and cleared leaked `ROBOCO_AGENT_TOKEN` in `tests/e2e_smoke/harness.py` before scripted agents load `flow_server`. See the E2E smoke harness section above for the exact patterns. > - **f081a574** (PR #502, 2026-07-13): Follow-up e2e lifecycle smoke fix. Restored the `ROBOCO_AGENT_TOKEN` pop in `tests/e2e_smoke/harness.py:ScriptedAgent._module` after it was accidentally removed, and clarified the `/api/notifications` mount comment so it no longer implies the router was newly added. The orchestrator `__new__` pre-init from `89b68786` means `_fresh_orchestrator` no longer needs to manually set `_instances`. +> - **Forge providers, 2026-07-18/19 (#569/#571/#575/#579/#581)**: adds `tests/unit/services/forge/` (`test_gitea_provider.py` 297 lines/17 tests, `test_gitlab_provider.py` 713 lines/33 tests, `test_router.py` 90 lines/9 tests — all mocked-transport) + `tests/unit/foundation/policy/test_forge.py` (132 lines/23 tests, `extract_host`/`detect_provider`/`validate_project_forge`) + the live-forge contract suites `test_gitea_live.py`/`test_gitlab_live.py` (see the "Live-forge contract suites" section above). See `docs/map/worksession-git.md` for the forge package itself. ## Regression Risks diff --git a/docs/map/video-engine.md b/docs/map/video-engine.md index 50a3c5aa..04a20bc7 100644 --- a/docs/map/video-engine.md +++ b/docs/map/video-engine.md @@ -48,6 +48,7 @@ CEO ACT: `GET /api/video/posts` lists held drafts (including `mp4_paths`); `GET - **2026-07-17** (PR #543, `3e801697`): Two renderer root causes fixed — `@hyperframes/producer` was floating (`^0.7.36`, no lockfile), so image builds silently picked up `0.7.60`, which fails EVERY render ("Cannot access 'rt' before initialization"); pinned exact (`0.7.36`, no caret) and committed a lockfile (regenerated as `pnpm-lock.yaml` by the immediate follow-up `a12fefcb`, not the npm `package-lock.json` this PR first wrote — this package is pnpm-managed). Second: the producer's per-clip visibility scheduler runs on a clock that lags ~50% behind the encoded timeline on a long cut, so tail scenes (past roughly the halfway mark) were silently missing from the MP4 regardless of authoring — fixed by treating `class="clip"` + `data-start`/`data-duration` as a structural-layer-only primitive and driving every beat with base-hidden styles + a delayed CSS animation instead (documented in `motion/README.md`'s "Clip windows are for structural layers only" rule). Also added the two choreography engines to `motion/kit/kit.js` (`choreographCursor` / `choreographCamera`, see Files above) plus a "Cinematography & rhythm" section in `motion/README.md` and a craft-bar block in the dev video spawn prompt (`roboco/runtime/orchestrator.py`) so a locked-off camera or a popping/freezing cursor reads as an automatic revision. - **2026-07-17** (PR #544, `fd621f0d`): The three craft capabilities wired one hop closer to the hands doing video work — vendored the vendor's own official HyperFrames agent skills (`hyperframes-core`/`-creative`/`-keyframes.md`, see Files above; supersedes the external-pointer-only version briefly added by the intervening `1416bd1d`); registered the `playwright` MCP for a ux-dev spawned onto a `source=video` task (`_is_video_authoring_spawn`, fail-closed role/team/task-source probe — gating-only, `agent-ux`'s image already bakes the browser); and added a video-mode override to the `ux_ui` team prompt's design bar ("video-authoring tasks are FILMS, not UI — these dials do not apply") so a video task no longer reads its own "dense product UI → motion 2-3" dial as license to ship a static slideshow. - **2026-07-17** (Wave 6, PR #550): Authoring craft, not engine code — `motion/README.md` gained `## Visual design bar (demo/kit register)` (spacing/hierarchy, beat pacing, `pk-chip`/`pk-pill` semantic discipline, camera+cursor+rhythm, anti-generic tells for the `kit/` register), four upstream HyperFrames craft references vendored verbatim under `motion/skills/references/` (fixing `hyperframes-creative.md`'s previously-dead `references/` pointers), and a new `motion/skills/hyperframes-catalog-index.md` (133-entry HyperFrames catalog vocabulary index, read-on-demand). No service/verb/schema change; the render/post pipeline documented above is untouched. +- **2026-07-18** (PR #570, "project-branded drafts"): the release-video script/prompt/brief builders in `roboco/services/video_engine.py` (`_fallback_release_script`, `_release_video_prompt`, `_release_video_brief`, `_draft_release_script`) all gained a required `product_name` param — `draft_release_post_video` (the `ReleaseProposalService.approve` publish-success hook, mirroring `XEngine.draft_release_post`) now resolves it via `CompanyGoalsService.resolve_product_name(project)` (the release's own project name → charter `company_goals.company_name` → "RoboCo" literal) instead of hardcoding "RoboCo" into the script/brief text — see `docs/map/product-strategy-research-pitch.md` for the shared resolver. `GET /api/video/posts` responses also gained `project_slug`/`project_name` (`api/schemas/video.py`, via the same `task_project_fields` helper `x.py` uses) so the panel's `video-post-queue.tsx` can render a `ProjectBadge` alongside the source-kind badge. ## Health diff --git a/docs/map/worksession-git.md b/docs/map/worksession-git.md index 8c94552e..f530fe2c 100644 --- a/docs/map/worksession-git.md +++ b/docs/map/worksession-git.md @@ -4,10 +4,13 @@ Scope key: `worksession-git` Repo root: `/Users/renzof/Documents/GitHub/ZZZ/robo - `roboco/services/work_session.py` - `roboco/services/git.py` - `roboco/templates/git/` (`__init__.py`, `branch.py`, `commit.py`, `constants.py`, `pr_internal.py`, `pr_root.py`) +- `roboco/services/forge/` (`__init__.py`, `base.py`, `github.py`, `gitea.py`, `gitlab.py`, `registry.py`, `router.py`, `shaping.py`) +- `roboco/foundation/policy/forge.py` +- `roboco/foundation/policy/pr_labels.py` ## Purpose -This slice is the git substrate every delivery agent works on. `GitService` runs all git subprocesses (status/commit/branch/push/rebase), mints branches + commit messages + PR bodies from templates, and drives the GitHub REST API for PR create/merge/close. `WorkSessionService` persists the per-claim row that links an agent to a task's branch/commits/PR and enforces the single-active-per-task invariant. The `roboco/templates/git/` package is the pure rendering layer for branch names, commit messages, and internal/root PR bodies. Together they are the boundary between the task lifecycle and the actual git history on disk + GitHub. +This slice is the git substrate every delivery agent works on. `GitService` runs all git subprocesses (status/commit/branch/push/rebase), mints branches + commit messages + PR bodies from templates, and drives the REST API for PR create/merge/close — now routed through a **multi-forge layer** (`roboco/services/forge/`) instead of talking to GitHub directly. `WorkSessionService` persists the per-claim row that links an agent to a task's branch/commits/PR and enforces the single-active-per-task invariant. The `roboco/templates/git/` package is the pure rendering layer for branch names, commit messages, and internal/root PR bodies. `roboco/foundation/policy/pr_labels.py` derives the org-structure label vocabulary (`to master`/`to slave`, `root`, `MegaTask`, layer labels) applied at PR-open. Together they are the boundary between the task lifecycle and the actual git history on disk + the configured forge (GitHub, Gitea, or GitLab). ## Files @@ -21,6 +24,16 @@ This slice is the git substrate every delivery agent works on. `GitService` runs | `roboco/templates/git/commit.py` | `CommitContext` + `build_commit_message` (traceability links) | 114 | | `roboco/templates/git/pr_internal.py` | Internal (subtask→parent) PR title/body builder | 140 | | `roboco/templates/git/pr_root.py` | Root (→master, CEO-level) PR title/body builder with task tree | 245 | +| `roboco/services/forge/__init__.py` | Package re-exports (`ForgeRouter`, `GitProvider`, `GitHubProvider`, `GiteaProvider`, `GitLabProvider`, `RepoRef`, `provider_for`, `register_project_forge`) | 32 | +| `roboco/services/forge/base.py` | Pure contract: `RepoRef` dataclass + the `GitProvider` ABC (~20 abstract methods every forge implements) | 208 | +| `roboco/services/forge/github.py` | GitHub.com/GHE REST transport — the pre-existing inline `httpx` logic pulled out unchanged; owns the sole retry policy in the package (`list_ci_runs`) | 432 | +| `roboco/services/forge/gitea.py` | Self-hosted Gitea REST transport, adapting the wire contract back into GitHub shapes | 471 | +| `roboco/services/forge/gitlab.py` | GitLab REST v4 transport — heaviest adaptation (MR `iid`, diff reassembly, pipelines) | 679 | +| `roboco/services/forge/registry.py` | Host↔provider(+scheme) map + `provider_for(project)` resolution, self-healing per-process | 104 | +| `roboco/services/forge/router.py` | `ForgeRouter` — implements `GitProvider` by picking a transport per call from `RepoRef.host` | 195 | +| `roboco/services/forge/shaping.py` | `ShapedResponse` — an `httpx.Response`-compatible stand-in a non-GitHub provider returns when it must synthesize a status the wire call didn't produce (e.g. a shaped 501 for `merge_branch`) | 51 | +| `roboco/foundation/policy/forge.py` | Pure host/provider detection (`extract_host`, `detect_provider`) + registration-time validation (`validate_project_forge`) | 99 | +| `roboco/foundation/policy/pr_labels.py` | `derive_pr_labels` — the org-structure label vocabulary (`to `, `root`, `MegaTask`, layer labels) applied at every PR-open site | ~90 | ## Key Symbols @@ -85,9 +98,11 @@ This slice is the git substrate every delivery agent works on. `GitService` runs | `GitService.close_pull_request` | method | git.py:3940 | Close superseded PR + optional comment + branch cleanup (idempotent) | | `GitService._delete_remote_branch_best_effort` | method | git.py:3608 | Best-effort remote delete; skips main/master/develop + open-dependent-PR branches; returns `bool` (issued vs skipped/failed) | | `GitService.delete_task_branch` | method | git.py:3671 | Cancel-path remote branch delete; chokepoint for the environment-ladder skip (`effective_environments`) so a task's `branch_name` can never collide-delete a ladder rung; returns `bool` | -| `GitService.cleanup_stale_branches` | method | git.py:3711 | `POST /git/branches/cleanup` backing sweep: terminal (completed/cancelled) tasks' branches, remote (`delete_task_branch`) + local force-delete in the assignee's clone; capped 200/call, cursor-resumable | -| `GitService._stale_branch_window` | method | git.py:3777 | One deterministic `ORDER BY id` window of sweep candidates; ladder rungs excluded from results but still advance the cursor | -| `GitService._cleanup_one_stale_branch` | method | git.py:3810 | Per-branch remote+local delete for one sweep candidate; raises on unexpected failure so the caller's try/except counts it as an error | +| `GitService.close_task_pr_best_effort` | method | git.py:3655 | No-clone-needed cancel-path cleanup: resolves the project token + `git_url`→`RepoRef`, fetches the PR via `self._forge.get_pr`, no-ops unless `state=="open"`, else `update_pr(payload={"state":"closed"})`; every failure path (missing token/project, unparseable URL, `httpx.HTTPError`) returns `False` rather than raising | +| `GitService.cleanup_stale_branches` | method | git.py:3697 | `POST /git/branches/cleanup` backing sweep: terminal (completed/cancelled) tasks' branches, remote (`delete_task_branch`) + local force-delete in the assignee's clone; capped 200/call, cursor-resumable | +| `GitService._stale_branch_window` | method | git.py:3767 | One deterministic `ORDER BY id` window of sweep candidates; ladder rungs excluded from results but still advance the cursor | +| `GitService._live_task_dependents` | method | git.py:3810 | Companion sweep guard: excludes a terminal candidate's branch when a non-terminal task still records that exact `branch_name`, or when a non-terminal task is a direct child of the candidate (the child's future PR base would resolve to the parent's branch via `resolve_parent_branch` even before it has opened a PR — catches what the OPEN-PR-only `_branch_has_open_dependents` can't see) | +| `GitService._cleanup_one_stale_branch` | method | git.py:3841 | Per-branch remote+local delete for one sweep candidate; raises on unexpected failure so the caller's try/except counts it as an error | | `GitService.pr_target` | method | git.py:4021 | Return PR base branch (project_id scoped) | | `GitService.create_pr` | method | git.py:3418 | Branch-keyed open PR (gateway path; ensures base on remote) | | `GitService._record_pr_atomically` | method | git.py:2601 | Atomic pr_number/url write to task | @@ -107,6 +122,21 @@ This slice is the git substrate every delivery agent works on. `GitService` runs | `build_pr_body_root` / `build_pr_title_root` | funcs | templates/git/pr_root.py:167/235 | Root PR rendering with task tree + AC checklist | | `MAX_TASK_DEPTH` | const | templates/git/constants.py:45 | 4 (umbrella→root→cell→dev) | | `BRANCH_TYPES` / `COMMIT_TYPES` | consts | templates/git/constants.py:10/21 | Allowed prefixes | +| `RepoRef` | dataclass | forge/base.py:26 | Provider-opaque repo identity (`owner`, `repo`, `host`); GitLab packs the full URL-encoded namespace path into `owner`, leaving `repo` empty | +| `GitProvider` | ABC | forge/base.py:49 | The ~20-method transport contract every forge implements (PR CRUD, review flow, `merge_branch`, CI signal surface, repo/label/branch/release/provisioning) | +| `GitService._forge` | property | git.py:399 | A fresh `ForgeRouter()` per access (cheap, no I/O — built this way, not `__init__`-cached, so tests can `GitService.__new__` bypass the constructor) | +| `ForgeRouter` | class | forge/router.py:37 | Implements `GitProvider` by picking a transport per call from `RepoRef.host` | +| `ForgeRouter._provider_for_ref` | staticmethod | forge/router.py:40 | `None` host → `GitHubProvider()`; a registered `"gitea"`/`"gitlab"` host → that provider constructed with its remembered scheme; unregistered → `GitError` naming the fix | +| `register_project_forge` | func | forge/registry.py:50 | Records a project's host→provider(+scheme) mapping; called from `ProjectService.create`/`update`/`get`/`get_by_slug` — in-memory, per-process, self-healing (a restart forgets it; the next project read re-registers) | +| `provider_for` | func | forge/registry.py:77 | Resolve a `GitProvider` for a project (duck-typed on `.git_provider`/`.git_url`) | +| `GitHubProvider` | class | forge/github.py:71 | GitHub.com/GHE REST transport — the pre-existing inline `httpx` logic, byte-for-byte; owns the sole retry policy in the package (`list_ci_runs`, 3 attempts/0.5s backoff, retryable on 429+5xx) | +| `GiteaProvider` | class | forge/gitea.py:55 | Self-hosted Gitea transport: `token`-scheme auth (Bearer is rejected by classic PATs); duplicate-PR 409→422 reshape on `create_pr`; `merge_pr` POSTs (not PUTs) with the method under a `"Do"` key; commit-status → synthetic `check_runs`/`workflow_runs`; branch names with slashes URL-`quote()`-encoded before hitting Gitea's router (caught live by the e2e suite) | +| `GitLabProvider` | class | forge/gitlab.py:85 | GitLab REST v4 transport — the most semantically divergent: MR `iid`→`number`, `source_branch`/`target_branch`→`head.ref`/`base.ref`; `get_pr_diff` reassembles a unified-diff string from up to 3 pages of per-file JSON diffs (no raw-diff media type); `post_review` routes APPROVE to `/approve` and everything else to a plain note (no request-changes verb exists); `request_reviewers` synthetic-skips (needs numeric user ids RoboCo doesn't store); `create_org_repo` resolves a group path to a numeric namespace id, falling back to the token's personal namespace on 404 | +| `ShapedResponse` | class | forge/shaping.py:17 | `httpx.Response`-compatible stand-in a non-GitHub provider returns to synthesize a status the wire call didn't produce (e.g. `merge_branch`'s shaped 501) | +| `extract_host` / `detect_provider` / `validate_project_forge` | funcs | foundation/policy/forge.py:27/45/62 | Pure host extraction (https/ssh/scp-like URLs), github.com/gitlab.com auto-detection, and registration-time validation (a self-hosted host with no explicit `git_provider` is a registration-time rejection — the GHE/self-hosted escape hatch requires the operator to set the column) | +| `derive_pr_labels` | func | foundation/policy/pr_labels.py | Org-structure label vocabulary: `base_branch` (required kwarg, the PR's real resolved target — never assumed from `is_root_pr`) → `f"to {base_branch}"`, plus `root`/`MegaTask`/layer labels (`main-pm`, `cell/{team}`, `subtask/{team}`) | + +Neither Gitea nor GitLab has GitHub's server-side merges API for the env-sync cascade: both return a shaped 501 from `merge_branch`, landing `GitService.sync_env_branch` on the local-git fallback (`_local_merge_branch`: throwaway clone → merge → push; a conflict aborts leaving the remote untouched, same status vocabulary as the GitHub server-side path) — this fallback lives entirely in `GitService`, not the forge package. Plain git (clone/fetch/push) needed zero forge-specific work: all three forges accept a PAT as the Basic-auth password with username ignored, so the existing `x-access-token:` extraheader works unchanged (verified live against a dockerized Gitea). ## Data Flow @@ -317,7 +347,11 @@ Baseline: `fd10cc862c2020b3f639cdb686d427b0198a2441` (master tip before the metr > Post-snapshot updates (since 2026-06-29): `536bbb64` (Chore/all/logical gaps sweep #286 — closed regression risks #108 and #109 in this slice: `_merge_with_retry` now falls back to a permitted merge method on 405 via `_first_allowed_merge_method` before raising `MergeConflictError`; `close_pull_request` default flipped to `delete_branch=False`, choreographer caller now passes `delete_branch=True` explicitly). `00513399` ([bug] push_branch — `push_branch(branch_name)` now passes `branch=branch_name` to `self.push()` so the gateway `open_pr` path pushes the actual named task branch rather than the clone root's current checkout; fixes the "No commits between" 422 → `i_am_blocked` wedge observed in the F123 per-worktree model). `2759edf7` ([B-REL] release executor — added `_CiRunQuery` dataclass at git.py:241 to bundle per-project CI-fetch inputs; `get_latest_ci_conclusion` and `_fetch_latest_ci_run` now accept an optional `head_sha` so the release CI gate polls a specific release commit's own run rather than branch-latest; `settings.release_ci_workflow` config flag added). `69071030` ([chore] work-session-routes — added `WorkSessionService.task_team_for_session` helper (route layer PM cell-ownership check for `merge_pr`); route layer now stamps `merged_by` from the authenticated caller rather than the request body — `WorkSessionService.merge_pr` signature is unchanged, but `MergePRRequest` schema dropped `merged_by` field). > -> (open PR #548, branch `feature/wave-2-hygiene-charts`, 2026-07-17) Local branch refs stop leaking alongside remote ones: `delete_task_branch` now also skips environment-ladder rungs (previously only the remote-delete's own main/master/develop guard existed) and returns `bool`; new `cleanup_stale_branches` + `_stale_branch_window` + `_cleanup_one_stale_branch` back a PM/CEO-only `POST /git/branches/cleanup` sweep of terminal tasks' remote+local branches, exposed as a confirm-dialog button on the panel Git page. See `docs/map/task-service.md` for the paired per-task reap at cancel/completion and `docs/map/workspace.md` for the new `WorkspaceService.delete_local_branch` primitive both routes share. +> `496c24d1` (PR #548, "git hygiene", 2026-07-17) Local branch refs stop leaking alongside remote ones: `delete_task_branch` now also skips environment-ladder rungs (previously only the remote-delete's own main/master/develop guard existed) and returns `bool`; new `cleanup_stale_branches` + `_stale_branch_window` + `_cleanup_one_stale_branch` back a PM/CEO-only `POST /git/branches/cleanup` sweep of terminal tasks' remote+local branches, exposed as a confirm-dialog button on the panel Git page. See `docs/map/task-service.md` for the paired per-task reap at cancel/completion and `docs/map/workspace.md` for the new `WorkspaceService.delete_local_branch` primitive both routes share. +> +> **Forge providers — GitHub + Gitea + GitLab (2026-07-18/19, PRs #569/#571/#575/#579/#581).** A new `roboco/services/forge/` package (`base.py`/`github.py`/`gitea.py`/`gitlab.py`/`registry.py`/`router.py`/`shaping.py`) plus `roboco/foundation/policy/forge.py` route every REST call `GitService` makes (PRs/CI/reviews/labels/releases/provisioning) through a provider-agnostic transport. `388bab24` (Phase 0, #569): `projects.git_provider` column (migration 076, nullable, plain string not a pg enum — validated at the service layer, not the DB) + `validate_project_forge`/`detect_provider` (github.com auto-detects, self-hosted needs an explicit column value — the GHE/self-hosted escape hatch). `461a6e1a` (Phase 1, #571): the `GitProvider` ABC + `GitHubProvider` extracted byte-for-byte from `GitService`'s old inline `httpx` calls; `GitService._forge` (git.py:399) becomes the seam every call site routes through. `96401f4c` (Phases 2/2.1/3, #575): `GiteaProvider` + `GitLabProvider` + `ForgeRouter` (per-call transport dispatch off `RepoRef.host`) + the local-git `merge_branch` fallback for forges with no server-side merges API. `5f32d876` (Phase 4, #581): `roboco/services/github_provisioning.py` becomes provider-aware (`ROBOCO_PROVISIONING_PROVIDER`/`ROBOCO_PROVISIONING_HOST` — see `docs/map/product-strategy-research-pitch.md`) so pitch-driven repo creation works on all three forges. `d4cb5797` (#579) + the pre-existing `tests/e2e_smoke/test_gitea_live.py` are the live contract suites (self-seeding against a dockerized `gitea/gitea` / real `gitlab.com`, env-gated) that caught the slash-encoding and http-scheme gaps in the Gitea provider. Panel: the edit-project dialog's "Forge" `` still reads "...GitLab support is planned" even though `gitlab` is a live `SelectItem` one line below and the backend has full GitLab Phase 3 support (`GitLabProvider`, `ForgeRouter`) — cosmetic only, no functional gap, but confusing to an operator reading the tooltip before picking GitLab. | low | | F123 worktree routing — commit/conventions/rebase run in worktree, merge sync runs in clone root | git.py:3696/3785 | `pr_merge` calls `_sync_target_branch_best_effort(workspace,...)` with the clone-root workspace (from `get_workspace`), not the per-task worktree. If the target branch is checked out in a worktree, the sync's `checkout` of target in the clone root fails ("already checked out"). Best-effort swallows it, but the local target ref may stay stale for the next sibling merge. | medium | | ~~`_merge_with_retry` retries on 409 only; 405 falls through to already-merged check then `MergeConflictError`~~ | git.py:3597 | **FIXED** (`536bbb64` #108) — `_merge_with_retry` now falls back to a permitted merge method (via `_first_allowed_merge_method`, exclude='squash') on 405 before raising `MergeConflictError`, mirroring the CEO `merge_pull_request` path. A 405 with no permitted fallback or a second 405 still falls through to disambiguation/`MergeConflictError`. | ~~medium~~ resolved | | `is_behind_base` raises on fetch failure; gate fail-opens | git.py:3922/3938 | A flaky origin fetch makes `is_behind_base` raise; the i_am_done gate catches it and fail-opens, letting a behind branch submit. The merge layer's own behind check is the backstop, but a genuinely-behind branch can reach QA. Documented, but a regression in the "gate is authoritative" expectation. | low | @@ -337,4 +372,4 @@ Baseline: `fd10cc862c2020b3f639cdb686d427b0198a2441` (master tip before the metr ## Health -Integrity is **good and actively hardened**. The slice carries the scars of multiple live meltdowns (single-active work-session defect, pr_fail re-submit loop, cell_pm merge block<->unblock, MegaTask depth cap, cross-repo PR-number collision) and each is closed with a deterministic guard plus a comment explaining the failure mode. The F123 per-task-worktree routing is consistently threaded through commit/checkout/rebase/conventions, and the merge path has layered defenses (parent-row lock, 409 retry, already-merged disambiguation, CEO-only master guard). Two formerly-medium risks in the merge path have since been closed: `_merge_with_retry` now has the 405 method-fallback that `merge_pull_request` has (`536bbb64`), and `close_pull_request` now defaults to `delete_branch=False` (`536bbb64`). The remaining residual risk is **`pr_merge`'s post-merge target sync** running in the clone root (not the worktree) and best-effort-swallowing a checkout conflict — the local target ref may stay stale for the next sibling merge. Test coverage of the work-session lifecycle is solid; the newer `pr_merge`/`sync_task_branch`/`rebase_onto_base`/`close_pull_request` quartet deserves the most scrutiny on any future change. No outright bugs found; the drift vs CLAUDE.md is documentation undersell (commit header format, gateway merge-path description), not behavioral mismatch. \ No newline at end of file +Integrity is **good and actively hardened**. The slice carries the scars of multiple live meltdowns (single-active work-session defect, pr_fail re-submit loop, cell_pm merge block<->unblock, MegaTask depth cap, cross-repo PR-number collision) and each is closed with a deterministic guard plus a comment explaining the failure mode. The F123 per-task-worktree routing is consistently threaded through commit/checkout/rebase/conventions, and the merge path has layered defenses (parent-row lock, 409 retry, already-merged disambiguation, CEO-only master guard). Two formerly-medium risks in the merge path have since been closed: `_merge_with_retry` now has the 405 method-fallback that `merge_pull_request` has (`536bbb64`), and `close_pull_request` now defaults to `delete_branch=False` (`536bbb64`). The remaining residual risk is **`pr_merge`'s post-merge target sync** running in the clone root (not the worktree) and best-effort-swallowing a checkout conflict — the local target ref may stay stale for the next sibling merge. Test coverage of the work-session lifecycle is solid; the newer `pr_merge`/`sync_task_branch`/`rebase_onto_base`/`close_pull_request` quartet deserves the most scrutiny on any future change. No outright bugs found; the drift vs CLAUDE.md is documentation undersell (commit header format, gateway merge-path description), not behavioral mismatch. The forge-providers rollout (GitHub/Gitea/GitLab) added real breadth without touching the merge-path defenses above — `GitService`'s callers still reason in GitHub-shaped responses, and `ForgeRouter`/`GiteaProvider`/`GitLabProvider` carry the entire adaptation burden behind that seam; the live-forge e2e suites (`tests/e2e_smoke/test_gitea_live.py`/`test_gitlab_live.py`, both env-gated and self-seeding) are the only coverage that exercises a real forge over the wire rather than a mocked transport, and they already caught two real gaps (slash-encoding, http scheme) the mocked unit tests couldn't have found. \ No newline at end of file