mirror of https://github.com/rennf93/roboco.git synced 2026-08-03 07:23:24 +02:00

Files

T

df87fcf059 Chore/logical gaps element sweep fixes (#287 )

* [sweep] lifecycle: 6 confirmed gaps fixed (cancel-ceo-gate, claim_pr_review gate, needs_team_match, valid_next_verbs narrowing, pr_reviewer unclaim, complete side_effect ordering)

* [chore] logical-gaps: route-layer force gate + privileged-field gate + pre-task audit attribution

tasks.py (5 gaps):
- _HATCH_OVERRIDE_STATES expanded to 7: a privileged PATCH INTO a gate
  state (completed/cancelled/awaiting_{qa,documentation,pr_review,
  pm_review,ceo_approval}) now requires explicit force — the panel hatch
  is no longer a quiet click that drops a task into/out of a human gate.
- _RESURRECT_SOURCE_STATES: a privileged PATCH OUT of a terminal status
  (completed/cancelled) resurrects finished work and likewise requires
  force, audited as an override.
- _PRIVILEGED_UPDATE_FIELDS gate: a bare task owner (UPDATE_OWN, no
  ASSIGN) cannot self-reassign / re-team / re-parent / re-depend /
  re-block / rewrite-plan / re-project its task — those structural fields
  are PM-gated; the REST surface must not bypass the verb-layer's
  reassign/delegate/triage gate. A 403 names the touched fields + the
  verb to use instead.
- pre-task create denial: a role that cannot create tasks is now logged
  via log_task_creation_denial (distinct task_creation target_type +
  attempted payload) instead of a 'N/A' task_id that coerced to NULL and
  left the role-escalation attempt unattributable.

audit.py:
- split log_task_action_denial (5-param, under PLR0913) from
  log_task_creation_denial (4-param) — the create path has no task_id;
  the non-UUID sentinel (N/A) is preserved in details[target_id_raw]
  rather than dropped to a NULL target_id indistinguishable from any
  other NULL-target denial.

tests:
- test_tasks_routes.py: parametrized admin-override gate (force
  required for gate + terminal states, force succeeds).
- test_tasks_route_privileged_fields.py: dev owner 403 on
  assigned_to/team/parent_task_id, 200 on dev-facing description.
- test_audit.py: pre-task attribution via log_task_creation_denial +
  non-UUID sentinel preservation.

* [chore] logical-gaps: kanban board column coverage + status-class fixes (6 gaps)

models/kanban.py:
- DEV_COLUMNS: cover all 15 lifecycle statuses (was 7; dropped BACKLOG,
  PAUSED, VERIFYING, NEEDS_REVISION, AWAITING_PR_REVIEW, AWAITING_PM_REVIEW,
  AWAITING_CEO_APPROVAL, CANCELLED). A dev whose task bounced to
  needs_revision or sits in a gate used to see their own task vanish.
- PM_COLUMNS: add the gate/revision/paused/cancelled/backlog columns so the
  cell PM sees the QA->docs->PR-review->PM-review->CEO chain on its board.
- QA_COLUMNS: drop the 'In Review'->VERIFYING mapping. VERIFYING is the dev's
  self-verification (task still with the dev, not with QA); it misrepresented
  dev mid-verification as active QA work.

services/kanban.py:
- _build_flat_board: add an 'Other' fallback column for any task whose status
  matches no configured column, so total_cards == sum(card_count) and no card
  is built-then-silently-dropped (the vanished-card leak).
- get_qa_board: drop VERIFYING from qa_statuses (consistent with the column
  change).
- get_documenter_board: scope to task_type=documentation so a dev IN_PROGRESS
  code task sharing the cell team no longer appears under 'Gathering'.
- get_main_pm_board_flat: widen the status filter to include PENDING/CLAIMED/
  COMPLETED and route those to the incoming/distributed/done columns, which
  were structurally always empty under the in-flight-only filter.

tests/integration/test_kanban_service.py: parametrized coverage of every
dropped dev status, PM gate/revision states, QA excludes VERIFYING,
documenter excludes dev code tasks, flat Main PM incoming/distributed/done
populated, and the 'Other' fallback invariant.

* [chore] logical-gaps: lifecycle-enforcement validators + status-class fixes (5 gaps)

enforcement/task_lifecycle.py:
- drop the spurious VERIFYING->awaiting_documentation legacy edge. The
  canonical exit is submit_qa -> awaiting_qa -> (qa_pass) ->
  awaiting_documentation; the direct edge bypassed the entire QA review hop
  (ungated — no role gate existed for it).
- is_waiting_state: add awaiting_pr_review. The PR-review gate parks the PM on
  the reviewer; it is a waiting state. The hard-coded set was never updated
  when AWAITING_PR_REVIEW was added to the enum, so the gate status was
  miscategorized as active.

foundation/_validate_lifecycle.py:
- _check_status_enum_coverage: replace the tautology (STATUS_GRAPH keys every
  Status by construction) with a real bidirectional check — every non-terminal
  Status is the source of a transition (catches orphan states), and every
  source/target referenced is a real Status member (catches stray-string
  targets).
- _check_terminal_exits: split the {COMPLETED, CANCELLED} reachability into a
  COMPLETED-path requirement + a cancel-exit requirement. The cancel fan-out
  made the old check structurally trivial — a status whose sole exit was cancel
  passed with no real forward completion path.
- _check_status_enum_parity (new, registered): cross-check spec.Status against
  models.base.TaskStatus at import so the ORM column type and the lifecycle
  map cannot drift (TaskType had this guard; Status did not).

tests: verifying->awaiting_documentation rejected, self-fail preserved,
awaiting_pr_review is waiting, mutually-disjoint classification invariant,
status enum parity, stray-string-target / orphan-source / cancel-only-exit
validator rejections.

* [chore] logical-gaps: stream-bus poison-pill ACK + dead-letter, periodic reclaim, cancelled-handler marker cleanup (3 gaps)

stream_bus.py:
- _handle_message isolates Event.from_json in its own try/except; an
  undecodable payload (unknown EventType, bad UUID/timestamp, malformed
  JSON) is dead-lettered then ACKed instead of falling through to the
  broad except that only logged — a poison pill stayed pending forever
  and re-failed on every reclaim. (gap: stream-bus-malformed-event-poison-pill)
- _reclaim_loop spawned alongside _listen_loop in start_listening (cancelled
  in disconnect). XREADGROUP '>' delivers only NEW messages, so a runtime
  handler failure left its message pending and unretried until a restart;
  the loop re-runs recover_pending every 60s so the idempotency-guarded
  replay actually fires. (gap: stream-bus-no-runtime-reclaim-loop)
- _run_handler_guarded marker cleanup catches BaseException so a handler
  cancelled mid-flight (asyncio.CancelledError is BaseException-derived
  since 3.8) clears its SET-NX marker; otherwise the guard suppressed the
  very redelivery that would complete the work. (gap: stream-bus-cancelled-
  handler-keeps-idempotency-marker)

TDD: 4 red->green tests in tests/unit/events/test_bus.py.

* [chore] logical-gaps: verb_runner trailing-None side-effect guard + actor_agent_id threading (3 gaps)

_verb_runner.py:
- run_intent skips the side_effects loop when a TRAILING composed action
  returned None (its source-status check failed under a concurrent
  transition). Previously the loop ran unconditionally on the None task
  and _do_push_branch(None)/_do_pr_merge(None) crashed with a
  NoneType AttributeError, turning the clean INVALID_STATE the
  entry/intermediate guards give into a 500/respawn loop. The trailing
  None now flows to the caller's `if task is None` handler. Latent today
  (no shipped intent has both a None-capable compose and trailing
  side_effects) but the runner is generic. (gap: runner-side-effects-fire-
  on-trailing-none-task)
- _do_push_branch / _do_create_pr / _do_create_root_pr forward
  actor_agent_id=agent.id into git_service (push_branch / create_pr),
  matching _do_pr_merge. Without it, a verb on a task whose assigned_to
  was cleared before the side effect falls through to created_by and
  pushes from / opens a PR against the wrong workspace.
  (gap: side-effect-handlers-drop-actor-agent-id)
- _do_escalate_to_ceo forwards actor_agent_id=agent.id so the
  awaiting_ceo_approval audit row attributes to the specific PM/Board
  agent. (gap: do-escalate-to-ceo-drops-actor-agent-id)

task.py: escalate_to_ceo gains actor_agent_id param, passed as
audit_agent_id to _validate_and_set_status and recorded as
escalated_by_agent_id in the event payload + log. escalate_to_ceo_for_agent
forwards agent.agent_id.

_impl.py: the main_pm complete->escalate path forwards
actor_agent_id=main_pm_agent_id.

TDD: 5 red->green tests (synthetic trailing-None intent, actor forwarding
for push_branch/create_pr/create_root_pr/escalate_to_ceo) + real-DB audit
test asserting the awaiting_ceo_approval row carries the actor UUID.
Updated 3 board escalate_to_ceo tests to assert the forwarded actor.

* [B-REL] release executor: idempotent half-landed retry + commit-scoped CI + decoupled workflow

Three confirmed gaps in the release fail-closed pipeline (#87/#318/#402):

#87 publish_failed retry duplicates changelog: execute() only short-circuits
on an existing tag. A publish_failed outcome (commit pushed + CI green, no
tag) left no tag, so a retry re-ran apply_version_bumps + write_changelog_entry
(re-inserting the entry above the already-present heading -> duplicate) and
commit_and_push (a second chore(release) commit). Add ReleaseOps
.release_commit_sha(version) detecting a prior release commit on the branch
(clone already at the target version); when present, skip the bump/changelog/
gate/commit pipeline and rejoin the shared CI -> publish tail on the existing
commit. No second commit, no duplicate entry.

#318 wait_for_ci polls branch-latest, not the release commit: a later push to
master during the ~40min wait made the latest run's head_sha != the release
sha forever, exhausting _CI_MAX_POLLS -> false ci_failed on a release whose
own CI was green. Thread head_sha through get_latest_ci_conclusion /
_fetch_latest_ci_run (GitHub actions/runs?head_sha=) so the gate polls the
release commit's own run; a concurrent push can no longer mask it.

#402 release CI gate reuses self_heal_ci_workflow: that setting documents an
empty-string mode for single-workflow repos which, inherited here, degraded
the fail-closed gate to the all-workflows mode git.py itself flags as
unreliable. Add release_ci_workflow (default ci.yml) and _resolve_release_
ci_workflow(); the release gate always resolves a NAMED workflow, never None.

Refactor: bundle the CI-fetch per-project inputs into a _CiRunQuery dataclass
so _fetch_latest_ci_run stays under the arg-count gate; unify the half-landed
path into execute's shared tail (drops a separate _publish_existing, one
return path). TDD red->green; ruff/mypy clean.

* [chore] logical-gaps: a2a service hierarchy gate (typed, unconditional) + persist skill on message row (3 gaps)

create_a2a_notification gated A2A hierarchy only when both ends resolved
(`if from_agent and target_agent:`), so an unattributed (from_agent falsy)
or unresolvable-target request slipped past the hierarchy matrix and
dispatched with from_agent='unknown' / to_agent='' — and a denial came back
as a bare ValueError indistinguishable from the missing-task_id ValueError.
Require both ends present, then validate via the shared typed
validate_a2a_access path (A2AAccessDeniedError + route_hint) so the legacy
notification surface enforces the same who-may-talk-to-whom invariant as the
conversation path.

send() accepts skill= and the gateway callers (qa/doc/pr_gate) pass it
expecting the receiver to learn which capability the message is about, but
send_chat_message never read it from options — silently dropped. Persist a
nullable skill column (migration 054) on a2a_messages, wire it through
send_chat_message + _msg_to_model + the A2AChatMessage model, and fix the
send() docstring (it claimed 'recorded in message metadata').

TDD: 4 red→green (skill recorded on message + surfaces in inbox; permission
denied raises typed A2AAccessDeniedError with route_hint; self-A2A raises
typed; missing from_agent raises instead of silent dispatch). 103 a2a
integration tests green; ruff/mypy clean; migration 054 verified
upgrade/downgrade on throwaway PG.

* [chore] logical-gaps: release-proposal already_published closes proposal + heartbeat-lock-loss cancels execute (2 gaps)

approve() closed the proposal only on status=='published'. A retry that finds
the tag already shipped returns 'already_published' (is_already_published),
so if a prior publish's route commit failed / HTTP 504'd, the proposal stayed
non-terminal forever — every retry returned already_published and never
closed it; only a manual cancel unstuck it. Close on both published and
already_published: the release shipped either way.

_heartbeat_loop returned silently when the lock was no longer owned (a >TTL
Redis outage let the mutex expire mid-execute), leaving executor.execute
running UNGUARDED — a concurrent approve (once Redis returns) could then
acquire the lock and _prepare_release_clone rm -rf the in-flight shared
release clone while the first execute was still mid-run_gate, re-opening the
very rm -rf-clone race the mutex+heartbeat exist to prevent. Run execute as a
task; on lock-loss the heartbeat sets a flag and cancels it, and approve()
turns the CancelledError into a structured 'lock_lost' result (an external
cancellation of approve itself still propagates — distinguished by the flag).

TDD: 2 red→green (already_published → COMPLETED not wedged; heartbeat lock-loss
→ lock_lost + execute cancelled, proposal not completed). 8 concurrency tests
green; ruff/mypy clean.

* [chore] logical-gaps: release approve async dispatch (202) — kill the 40min synchronous HTTP 504

The approve route ran the whole fail-closed execute inline: clone(600s) +
gate(1800s) + CI poll(2400s) + publish(300s) ≈ up to 85min worst case. nginx
(the single :3000 entry point, ~60s read timeout) 504'd long before it
finished, so the CEO's approve always appeared to fail even when the release
succeeded server-side — the structured ReleaseResult was unreachable over the
wire. dispatch_approve spawns the execute in a background task with a fresh
session (built from the request session's engine) and the route returns 202
'accepted' immediately; _INFLIGHT_APPROVES tracks the dispatched task for
observability (self-cleans via done-callback; the Redis mutex still refuses a
double-execute on a second click). The panel already polls GET /proposal every
30s, so it observes the final status (COMPLETED on published/already_published,
else the proposal stays open for retry); the card's approve toast now treats
'accepted' as an info 'dispatched, running in the background' instead of the
old 'Release halted' warning.

TDD: 2 route tests red→green (approve returns 202 'accepted' + the proposal
transitions to COMPLETED / stays PENDING once the background faked execute
completes; the dispatched task is awaited while the executor patch is live).
83 release tests green; ruff/mypy clean; panel typecheck+lint+format+test
green.

* [chore] mcp-servers: normalize exception bodies to Envelope + lift task_id/correlation_id on circuit_open (#232 #359 #57)

flow_server/do_server: the non-404 JSON path returned exception-handler bodies
raw (dict `error` from roboco/generic/http exception handlers, or a 422
`detail` list) — neither is the Envelope wire format the agent is prompted to
trust (string error kind + message + remediate + missing), so on any
service/validation failure the agent got no remediate and flailed until the
breaker tripped. _normalize_exception_envelope lifts the body into a real
Envelope (code -> counted string kind via _classify_dict_error_code, NOT_FOUND
-> not_found, message lifted, remediate synthesized, missing=[]; 422 -> incomplete_input with the validation detail preserved). The synthesized
envelope still flows through the breaker so a 500/422 storm trips it.

_record_and_check_circuit: the circuit_open substitution dropped task_id /
correlation_id from the top level (the SDK's envelope omits them); lift them
from the original rejection so the agent's envelope contract and ops audit-join
of the trip event still work, not just nested in inner.

intake_server._post_event: capture the relay response body under `detail` on
non-success so the grok intake agent gets the real reason (e.g. 'session not in
MegaTask scope' on a 422) instead of an opaque http_422 token with no
remediation.

TDD red->green; ruff + mypy clean; 157 mcp/SDK-breaker tests pass.

* [chore] a2a-routes: authenticate send_message responder + gate cancel task (PM-only) (#116 #423)

send_message took the responder identity from a client-supplied
metadata.from_agent, so any caller could spoof anyone (e.g.
from_agent='ceo') in the task's notes and in the spawn/notification
routed back to the original requester. Stamp the authenticated caller's
slug as the responder instead (CurrentAgentContext).

cancel_task was ungated: no auth dependency and no role check, so any
agent (or any caller) could cancel a task the lifecycle rule reserves to
PM roles (Any -> cancelled: PM roles only) — and the cascade-cancel of
all non-terminal descendants ran with a hardcoded cell_pm role and no
recorded actor. Add require_any_authenticated_agent + a PM-or-above gate,
and thread the authenticated role (into the cascade role gate) and slug
(into the cancellation note) into A2AService.cancel_task.

Tests: send_message ignores a spoofed from_agent and records the
authenticated slug; cancel rejects a developer (403) and a missing auth
header; a PM cancel threads role + slug into the service; the pre-existing
cancel success/already-terminal/not-found tests now run under a PM context
(the success test's body was missing the A2A 'name' field and false-passed
on a 422 — now genuine).

* [chore] work-session-routes: ownership check on mutating routes + stamp merge_pr merged_by from auth (#158 #271)

Every mutating work-session route keyed off session_id alone after the
role gate, so any developer could commit into / abandon / complete a
peer's active session (breaking the single-active-WorkSession invariant
and stranding that task) and any PM could merge any cell's PR — the REST
surface bypassed the verb layer's active-claimant gate entirely. Add a
shared _assert_ownership guard: dev ops require session.agent_id to be
the caller; PM merge_pr requires a cell PM to own the session's task cell
(main PM / CEO / board coordinate every cell), 404 for a missing session.

merge_pr took merged_by from the request body, so any PM could record a
PR merge under another agent's id, corrupting the merge audit trail the
completion/CEO-approval chain and metrics rely on. Drop the body param
and stamp the authenticated caller's agent_id as merged_by (the
MergePRRequest schema is gone with it).

Tests: a second dev's token hitting a peer's /commits and /abandon -> 403
(session left active); a foreign-cell PM -> 403, same-cell PM -> 200; a
spoofed body merged_by is ignored and the persisted row records the PM.

* [chore] ci-watch/dep-update dedupe: normalize git_url + treat empty-string workflow as default (#148 #1267)

The per-repo open-task dedupe filtered ProjectTable.git_url == git_url
(exact), while the orchestrator collapses its poll set by repo_key
(lower / strip trailing '/' / drop '.git'). Two projects whose git_url
differs only by those accidentals (a monorepo's cell-projects, or a
re-registered canonical project) defeated the one-open-task-per-repo
invariant and opened duplicate fix / dep-update tasks. Extract
roboco.utils.converters.repo_key as the single source and match the
dedupe query on its SQL mirror (regexp_replace(rtrim(lower(...)))).

The ci_watch (git_url, workflow) dedupe used func.coalesce(ci_watch_workflow,
default), but SQL COALESCE only substitutes for NULL — a project saved with
ci_watch_workflow='' (reachable via panel/API) yielded coalesce('', default)
= '' != default, so the DB diverged from the engine/orchestrator (which
collapse '' to the default via Python truthiness) and opened a duplicate
fix task every red cycle. Wrap with func.nullif(..., '') so an empty string
collapses to the default too.

Tests: a ''-workflow + NULL-workflow project on one repo dedupe to one task;
git_url accidentals (.git suffix / trailing slash) dedupe across both
ci_watch and dep_update. The orchestrator _repo_key now delegates to repo_key.

* [chore] admin_set_status: attribute the blocked-restore to the admin actor + emit override row (#2176)

admin_set_status taking a BLOCKED task to pending/in_progress with a
pre-block snapshot returned early via _apply_pre_block_restore, which
emitted its audit row with agent_role=None and audit_agent_id=restored_owner
(the pre-block dev) — the admin actor_id/actor_role were dropped entirely.
Because this branch runs with force=false (pending/in_progress aren't hatch
destinations), the distinguishing task.admin_override row (written only on
the non-restore path, gated by force) was never written, so an operator
could silently re-own a blocked task with no trace of who triggered it.

Thread actor_id/actor_role into _apply_pre_block_restore (admin_set_status
passes them with admin_override=True) so the transition audit row attributes
the re-owning to the admin, and emit a task.admin_override row (forced=False,
restore=True) on this branch independent of the force flag. The in-band
unblock(restore=True) path passes no actor and keeps the legacy attribution
(restored owner) with no override row.

Test: admin PATCH status=pending on a BLOCKED task with a snapshot attributes
every audit row to the admin (not the restored dev) and emits the override
row.

* [chore] converters: typed InvalidIdentifierError from require_uuid + log the orchestrator drop (#25)

require_uuid raised a bare ValueError('UUID value cannot be None'), so a
malformed/None identifier propagated as an opaque error callers either let
500 or broad-catch-and-silently-swallow — the orchestrator reaper call site
wrapped it in a bare except-Exception return with NO log, dropping a bad
task_id_str invisibly. Introduce InvalidIdentifierError(ValueError) and
raise it from require_uuid for both None and unparseable input; it stays a
ValueError subclass so existing except-ValueError / except-Exception callers
are unaffected, but typed so a caller can handle a bad identifier distinctly.
The reaper now catches the typed error, logs at warning, and no-ops — the
drop is visible instead of swallowed.

Tests: None and an unparseable string both raise InvalidIdentifierError; it
subclasses ValueError (back-comat).

* [sweep] notification_delivery: list_system_notifications over-fetch-then-slice for pending_ack_only

The SQL limit was applied before the post-fetch 'not fully acked' Python
filter. A window of newer fully-acked ack-required rows filled the limit
and masked older unacked notifications the operator still needs to act on
(the pending-ACK queue silently under-reported; a CEO-approval notification
could be hidden by newer already-acked noise). pending_ack_only now drops
the SQL limit, filters in Python, then slices to limit; the non-pending
branch keeps the SQL limit unchanged.

* [sweep] proactive: drop vestigial code-patterns surface from context package

Code indexing was removed, so _find_code_patterns always returned [] yet
build_context_package still called it, ContextPackage.code_patterns stayed
a live field, _build_summary advertised 'Found N code patterns', and
_count_items counted it — a permanently-empty slot the system claimed to
populate. The dead method, its call, the summary line, and the count
reference are removed. The code_patterns field itself is retained
(always-empty, serialized in to_dict and the optimal route response) for
API/schema back-compat, marked deprecated in its docstring.

* [sweep] migration 052: integration-test the task_cell_projects unique constraint

The UNIQUE(task_id, team) 'one project per cell per task' invariant was
only exercised through SimpleNamespace stubs that never touch a DB
session, so the real Postgres constraint was unverified. If it were
mis-declared or dropped, two same-team rows could coexist and
_resolve_subtask_project would non-deterministically return one, cutting
a subtask's branch/PR against the wrong repo. Adds an integration test
that inserts two same-(task_id, team) rows and asserts IntegrityError on
uq_task_cell_projects_task_team, plus a positive different-teams case.

* [sweep] pr_gate: classify MegaTask root-subtask as root so its root->master PR gets COMMENT (#608)

_post_gate_review_to_pr identified a root->master PR by absence of a
parent_task_id. A MegaTask root-subtask opens its own root->master PR into
the project's master (submit_root, parent='master') but carries
parent_task_id=umbrella, so is_root was False and the gate posted APPROVE
(pr_pass) / REQUEST_CHANGES (pr_fail) instead of COMMENT. The APPROVE could
satisfy a single-approval master branch-protection rule and let a non-CEO
merge via the GitHub UI before the CEO, against the documented invariant
that only the CEO acts on master. is_root now also covers
is_batch_root_subtask (batch_id set + parented); a non-batch cell-PM
coordination root keeps batch_id=None so it stays a cell->root PR
(APPROVE/REQUEST_CHANGES). Extends the _task test helper with a batch_id
kwarg.

* [sweep] enforcement: complete the status-class partition + coverage invariant (#247)

is_waiting_state already covered awaiting_pr_review (the primary fix), but
the doc's coverage invariant was missing: backlog and pending fell through
ALL three predicates (terminal/active/waiting), so a future enum addition
could silently land in no category. is_waiting_state now also covers
pending (waiting for a claim) and backlog (waiting on PM activation), so
is_terminal_state / is_active_state / is_waiting_state partition the whole
Status enum. Adds test_status_classification_covers_every_enum_member
asserting every Status member is classified by exactly one predicate, so
an enum addition that drifts the partition fails the build.

* [chore] test-suite: unblock the quality gate (mypy + 2 behavior fixes)

12 mypy errors across 5 test files: drop banned type:ignore comments
(lifecycle_spec monkeypatch uses cast(Any, ...); the ignores were unused),
wrap SQLAlchemy-typed ids with cast(UUID, ...) for AgentContext / WorkSession
args (AgentTable.id is Mapped[sqla UUID], not uuid.UUID), annotate **kw: Any,
and cast(Any, svc) for a method-assignment mock.

test_cancel_descendants_cascades_for_authorized_pm: the child was parked in
awaiting_ceo_approval, which the spec gates to CEO-only cancel
(lifecycle.py:378-389) — a cell_pm cascade correctly refuses it (the #103
refuse path). Use a PM-cancelable in_progress child so the positive-cascade
assertion holds; the refuse case is already covered by its sibling test.

test_a2a_message_auth: /message/send now resolves the authenticated
responder slug via get_agent_context (a DB lookup, #116). This is a DB-free
unit test of the token gate + route body, so stub get_agent_context in the
fixture — the gate (require_any_authenticated_agent) still runs real and
401s on a missing/forged token before that dependency resolves.

* [chore] complexity: split 5 C-rank blocks to <=B for the xenon gate

No behavior change; each C-rank function factored into a helper so the
complexity gate (xenon --max-absolute B) holds.

- lifecycle.can_invoke_action: extract the team-match check into
  _check_team_match.
- a2a.cancel_task: extract _status_value_of + _apply_cancel_note.
- task._apply_pre_block_restore: extract _restore_block_ownership (status/
  owner restore + snapshot clear) and _emit_admin_override_audit (#2176).
- release_proposal.approve: extract _finalize_release_lock (heartbeat/
  execute cancel + mutex release) out of the finally.
- kanban.get_main_pm_board_flat: dict-dispatch the column routing instead
  of a 7-branch if/elif ladder (status wins over team; in-flight + no cell
  team falls through to Coordination, #196).

* [chore] lifecycle artifacts: regenerate to match the spec (foundation-check)

The rendered artifacts (docs/rag/lifecycle, panel/lib/lifecycle.json, the
_generated role-prompt fragments) had drifted from the spec — the prior
sweep commits (cancel-CEO gate, claim_pr_review preconditions, pr_reviewer
unclaim, complete merge-first ordering) changed spec data without
regenerating, and the foundation-check render+diff stage never ran because
mypy failed earlier in the gate. make foundation-check now passes.

* [fix] chat: wire live message delivery end-to-end (MESSAGE_SENT)

send_message persisted messages but never broadcast them, there was no
MESSAGE_SENT event type or bridge forwarder, and the panel session view
had no websocket subscription — the live chat path was dead end-to-end.

- add EventType.MESSAGE_SENT and publish it best-effort on every persisted
  send (a bus outage logs, never rolls back the durable row)
- bridge _handle_message_event forwards to /ws/sessions/{id} and
  /ws/channels/{id}; subscribe it in register_websocket_bridge_handlers
- panel useSessionStream subscribes the session view; the page invalidates
  the transcript + session-detail queries on each message.new so the held
  (staleTime Infinity) views refresh live without the manual Refresh

* [fix] chat: return session task_links in one read; drop panel N+1

GET /sessions/{id} ran a bare select and session_to_response omitted
task_links, so it always returned them empty — the panel worked around it
with a triple-fetch (get session, get-tasks-for-session which re-fetched
the same endpoint, then a task GET per link), and the links never showed.

- add get_session_with_links(_or_raise) that eager-loads task_links -> task
- add session_to_response_with_links; GET /sessions/{id} uses both
- panel useSession now relies on the single populated response; remove the
  dead getTasksForSession + per-task fetch and the unused tasksApi import

* [fix] chat: validate reply_to against the effective session; guard closed-session composer

Posting to a closed session transparently redirects the message to the
group's active session (intended for agents holding stale refs), but
reply_to was validated against the requested session, not the one the
message lands in — letting a cross-session reply slip through — and the
panel silently posted there too, so the message vanished from the view.

- validate reply_to against session.id (the effective, possibly-redirected
  session), not req.session_id
- panel: render a "session is closed" notice instead of the composer for a
  non-active session; if a send still lands elsewhere (stale status), toast
  that it went to the active session rather than letting it appear to vanish

* [fix] chat: close session/group/message read IDOR; fix doubled 404s

get_session and the messages-list took an agent id but never used it, and
get_group took none at all — any authenticated agent could read any private
channel's group, session, and message transcripts. Three NotFoundError sites
also passed a full sentence as resource_type, yielding "... not found not found".

- add require_group_read_access / require_session_read_access (channel
  member / silent observer / privileged, mirroring list_group_sessions_for_agent)
  and get_session_with_links_for_agent; enforce on GET /sessions/{id},
  GET /sessions/{id}/tasks, GET /messages, GET /groups/{id} (-> 403 on deny)
- fix the three doubled-404 sites to the NotFoundError(resource_type, resource_id) form

Also folds two gate fixes for the prior chat commits: cast session.id to UUID
for the reply_to validation, and ruff import/format touch-ups.

Note: POST /messages intentionally still skips the channel write-ACL on the
HTTP (human-CEO/panel) path — the CEO is not in writers for 8/11 channels, so
enforcing it there would block the panel; the gateway/agent path enforces it.

* [fix] secretary: harden live chat — stuck spinner, mid-reply clobber, reload

The Secretary live chat had three live-behaviour bugs: a dropped SSE
connection left a permanent "thinking…" spinner (openStream set no
transport-error handler, so the no-data error Event was swallowed by the
JSON-parse guard and streaming never reset); sending mid-reply wiped the
accumulation buffer and pushed a user message without guarding the in-flight
turn, abandoning/duplicating the reply; and the chat lived only in React
state, so a reload wiped it.

- route the dual-purpose `error` listener: server-sent JSON → handleEvent,
  transport error (no data) → reset streaming, surface a notice, close stream
- guard send while streaming (streamingRef); disable the composer Send/Enter
  while a reply is in flight
- persist sessionId + messages to localStorage (TTL'd) and, on mount, restore
  + re-attach the stream once the backend confirms the session is still alive
  (mirrors the intake/prompter durability)

* [chore] groups: extract group-read helper to keep module rank A

The get_group IDOR access-check added try/except branches that tipped the
module to xenon rank B. Extract the service-error→HTTP mapping into a small
helper so get_group stays lean and the module is rank A again (behaviour
unchanged; covered by the groups route tests).

* [fix] chat: correct panel session-task mutation endpoints

linkTask/unlinkTask posted to /add-task and /remove-task (with a body), but
the backend exposes POST /sessions/{id}/tasks and DELETE
/sessions/{id}/tasks/{task_id} (path param) — so every call 404'd. updateTaskLink
targeted /update-task, a route that does not exist at all. Point linkTask and
unlinkTask at the real routes and drop the phantom updateTaskLink. All three were
unused, so no behaviour changes today — this removes a latent 404 trap.

* [docs] chat: document live message delivery (MESSAGE_SENT / message.new)

Document the live transcript-update path the chat-subsystem fixes wired:
- docs/api/websockets.md: add the message.new event-types row (carried on
  /ws/sessions + /ws/channels from EventType.MESSAGE_SENT) and note the
  forwarder sets type:"message.new"
- docs/panel/communications-and-journals.md: the session transcript updates
  live; a closed session is read-only (composer disabled)
- CLAUDE.md: name message.new on the per-resource streams and make
  MESSAGE_SENT the worked example of the add-a-live-event recipe

The internal roboco_map slices (gitignored) were updated in place to match.

* [docs] reconcile published docs with code since v0.13.0

Drift caught by the doc-reconciliation pass (all verified against HEAD):
- CLAUDE.md + rag: pr_reviewer gained the unclaim verb (16b71be8)
- rag permissions/task-states/task-tools: awaiting_ceo_approval -> cancelled is
  CEO-only, not PM+CEO (16b71be8 cancel-ceo-gate; lifecycle.py:373-382)
- deploy/env-reference: ROBOCO_APP_VERSION default 0.9.0 -> 0.14.0 (config.py:31);
  add ROBOCO_RELEASE_CI_WORKFLOW row (2759edf7, config.py:454)
- deploy/data-and-migrations: 44->54 revisions, head 054_a2a_message_skill
- optional/autonomous-maintenance: CI-watch dedupe is per (repo, workflow) (d34bc1a7)

---------

Co-authored-by: Renn F <rennf93@users.noreply.github.com>

2026-07-01 01:11:34 +02:00

7.0 KiB

Raw Blame History

Data & migrations

RoboCo keeps all of its state in PostgreSQL. There is no separate document store, no schema you have to hand-build, and — in normal operation — no migration command you have to remember. This page gives you the operator's-eye view: the handful of entities worth understanding, the one hard database requirement (pgvector), and how the schema keeps itself up to date.

The entities you'll actually see

The full data model spans about thirty Pydantic models in roboco/models/ with matching SQLAlchemy tables in roboco/db/tables.py. You don't need most of them. These are the ones that show up across the panel and explain how work flows:

Entity	What it is
Task	The atomic unit of work. Carries its acceptance criteria, status (the 15-state lifecycle), priority, branch name, PR number/URL, ownership (`created_by` / `assigned_to` / `team`), and its place in the tree (`parent_task_id`, `dependency_ids`, `blocker_ids`). A task points at exactly one of a `project_id` or a `product_id` — a single-repo task uses the project, a Board/fan-out coordination task uses the product. A validator enforces "one or the other, never both."
Project	A git repository configuration: `git_url`, default branch, protected branches, the Fernet-encrypted GitHub PAT (`git_token_encrypted` — the API only ever returns `has_git_token`, never the token), the per-project CI/gate commands, and the assigned cell. See Register your first project.
WorkSession	The link between an agent, a task, and a branch. One is created each time an agent claims a task. It tracks the base/target branches, the commits made, the files modified, the PR and merge outcome, and (when toolchain matching is on) the resolved Python version and toolchain status.
Agent	One row per member of the workforce — role, team, status — plus the human CEO. See Org & roles.
Session / Channel / Message	The communication substrate. A `Channel` is a room; a `Session` is a live conversation inside it; a `Message` is one extracted line of an agent's stream.
Notification	A formal signal that requires acknowledgment (sent by PMs and the Board), distinct from the constant message stream.
Journal	Each agent's personal log of reflections and learnings, with `JournalEntry` rows underneath.

Two newer entities round out the company-in-a-box features: Product (with ProductProject, the per-cell repo-routing map for fan-out work) and Pitch (a Board proposal that the CEO approves and that can auto-provision a repo). You'll meet these on the Business page.

pgvector is required

RoboCo's in-house RAG engine stores embeddings in PostgreSQL using the pgvector extension. The orchestrator enables it for you at startup (CREATE EXTENSION IF NOT EXISTS vector), so the bundled postgres image — which ships pgvector — works out of the box.

!!! warning "If pgvector isn't installed" On a bring-your-own PostgreSQL where the vector extension isn't available, the orchestrator logs a warning and RAG features silently disable — agents lose the knowledge base and mentor lookups, but the rest of the system runs. If you point RoboCo at an external Postgres, make sure pgvector is installed there.

Operationally, PostgreSQL and Redis are the only two stateful services to back up. Everything the company knows lives in Postgres; Redis holds the event bus and short-lived coordination state.

!!! danger "The encryption key is not in the database" Each project's GitHub PAT is stored encrypted with ROBOCO_ENCRYPTION_KEY. A database backup without that key is useless for the tokens — you'll be able to restore every project except its credentials. Keep the key safe and separate. See the environment reference.

The stack migrates itself on startup

You almost never run a migration command by hand. When the orchestrator boots, init_db() enables pgvector and then run_migrations() runs alembic upgrade head in a worker thread:

flowchart LR
    A[Orchestrator starts] --> B[init_db: enable pgvector]
    B --> C[run_migrations: alembic upgrade head]
    C --> D[create_all gap-fill on existing DB]
    D --> E[dispose pool to refresh enum cache]
    E --> F[API ready]

The practical consequence: a normal docker compose up / restart already applies any new migrations. A fresh database is built entirely by the migration chain from base; an existing one runs only the pending steps (a pre-Alembic database is auto-stamped at the initial revision first, so it isn't re-built).

!!! tip "The manual command is belt-and-suspenders" The documented manual step — after pulling a change that adds a migration —

```bash
docker compose exec orchestrator alembic upgrade head
```

is a safety net, not a routine requirement, since the orchestrator runs the same command on boot. In host-dev mode (no container) the equivalent is `uv run alembic upgrade head`, or `make migrate`.

A fresh DB is built by migrations, not `create_all`

This is the one gotcha that bites operators who try to reset state by hand. Several migrations embed seed data — most importantly the LLM provider rows (Anthropic, Ollama, self-hosted, Grok). The schema must be built by running the migration chain, never by a bare SQLAlchemy create_all.

!!! warning "Don't reset a database with create_all" A create_all-only database has empty provider_configs, so the Settings → Providers endpoints return 404. The fix is always the same: let the migrations run (restart the orchestrator, or run alembic upgrade head). The orchestrator's own startup uses create_all only as a gap-fill on an already-migrated database, never as the builder.

Contributor notes

If you write or review custom migrations:

Revision-id length is capped at 32 characters. PostgreSQL stores alembic_version.version_num as VARCHAR(32). A longer id breaks a live upgrade even though the test suite (which renders offline / uses create_all) never catches it. The chain currently has 54 revisions (55 files — revision 026 is split into two consecutive steps); the head is 054_a2a_message_skill, and the longest id in the tree is 015_drop_task_execution_outputs at 31 characters, deliberately just under the limit.
The chain is linear. Every revision has a single down_revision; there are no branches or merges to reconcile.
Models, tables, and migrations must stay in sync. A Pydantic model in roboco/models/, its ORM table in roboco/db/tables.py, and the migration that creates the column are three layers that move together.

→ Bootstrap & seeds — what make db-init puts into a fresh database. → Environment reference — every ROBOCO_DATABASE_* knob and the encryption key.

7.0 KiB Raw Blame History