Commit Graph
5 Commits
Author SHA1 Message Date
536bbb64f3 Chore/all/logical gaps sweep (#286)
* release-manager: fencing-token mutex + executor/readiness hardening

Closes the release-mutex TTL race (#17, HIGH) and the remaining
release-manager gaps (#88, #89, #201, #202):

- #17: the release mutex is now acquired with a uuid4 fencing token and
  released via Lua compare-and-del; a background asyncio heartbeat
  compare-and-expires the TTL ~every 60s while the execute owns the lock,
  so a live execute no longer expires and a crashed one auto-releases
  <=3000s. A second approve after TTL expiry cannot usurp and rm -rf the
  in-flight clone — the fenced first-finally keeps its lock.
- #89: a Redis outage during acquire stays fail-closed (the execute never
  runs) but now returns a distinct redis_unavailable result + log so the
  CEO sees the cause instead of a false already_in_progress.
- #88: commit_and_push RuntimeError is wrapped into a structured
  ReleaseResult(commit_failed) instead of a 500.
- #201: first-release fallback still emits untracked version-ref files as
  gaps (no longer silenced by the first-release branch).
- #202: _await_proc awaits proc.wait() after kill() so a timeout cannot
  leak a zombie.

TDD: tests/unit/services/test_release_proposal_concurrency.py extends
_FakeRedis with eval/get/expire and pins the fencing/heartbeat/usurper
invariants + the redis_unavailable result.

* PM/code-task creation guard + main_pm coverage + issue carve-out

Closes the creation-time role x task_type gap (the user's explicit example)
and the main_pm delegate hole:

- New pure helper `pm_cannot_own_code(role, task_type, is_issue_resolution)`
  in foundation/policy/batch.py — single source of truth. Both PM roles
  (cell_pm + main_pm) coordinate; a `code` task assigned/claimed by a PM is
  a structural mismatch, EXCEPT a PM taking a code task in needs_revision to
  resolve review/QA issues directly (the carve-out).
- Creation-time guard: TaskService.create calls the helper (closes the
  create-with-cell-PM-assignee hole the team-based check misses).
- Delegate path + spec claim gate consult the same helper.
  `_validate_assignee_task_type` / `_task_type_hint_for` now key on the
  Role (CELL_PM OR MAIN_PM), not the cell-PM slug set — closes the
  delegate-to-main-pm-as-code hole.
- identity.role_for_uuid_or_none is None-tolerant (treats None as "not a
  PM" and proceeds) so a malformed/missing assignee cannot crash the guard.
- prompter.create_task_from_draft reuses the guard at draft-create.

TDD: test_batch.py (helper matrix + carve-out), test_main_pm_code_guard.py
(main_pm coverage), test_delegate_assignee_task_type.py (delegate parity),
test_lifecycle_spec.py (claim gate: rejects PM claiming code from pending,
allows from needs_revision + PM claiming planning + dev claiming code).

* task-service: completion hooks + escalation/cancel/audit hardening

Closes the task-service cluster (#21/#98, #99, #100, #101, #103, #216; #102
verified already-covered, #217 verified already-guarded):

- #21/#98: ceo_approve now closes the work session + triggers completion
  hooks before worktree removal (no-op when work_session_id is None), so a
  CEO-approved task lands the same close-path as PM-completed.
- #99: apply_escalation routes through the transition validator with an
  enumerated escalation exemption (_ESCALATABLE_TO_BLOCKED) instead of an
  arbitrary source->BLOCKED write; BACKLOG is refused.
- #100: branchless ceo_reject awaiting_ceo_approval->pending gets a real
  spec edge (ceo_reject_to_pool ActionSpec + _STATUS_TRANSITIONS entry) so
  future admin-override tightening can't wedge the path.
- #101: revision_count bump is documented as the single chokepoint, with
  the pre-block RESTORE path undoing it when restoring a snapshotted
  needs_revision (same cycle resuming, not a new rejection).
- #103: cancel cascade surfaces non-terminal orphans instead of swallowing
  the role violation.
- #216: _remove_task_worktree_on_terminal escalates recurring FS/permission
  failure (audit/notify after N) instead of silent-failing forever.
- #102: pinned in test_verb_runner_midverb_invalid_state.py (committed with
  the choreographer cluster) — verb-runner savepoints already surface a
  concurrent mid-verb state change as INVALID_STATE.
- #217: submit_for_qa claimed_by guard verified intact.

TDD: test_task.py, test_worktree_cleanup_on_complete.py,
test_escalation_board_guard.py (#99), test_task_service_* integration,
test_lifecycle_spec.py.

* choreographer: gate-claim guards + pr-gate hardening + fail-open logging

Closes the choreographer cluster (#5/#222, #29, #30, #82, #188, #189,
#192; #157/#187 verified already-fixed/pinned; #102 pin lives here):

- #5/#222: the unchanged-PR guard's fail-open head_sha lookup now logs
  (warning) on a slug-resolver/git-helper error so a regression cannot
  silently turn the pr_fail re-submit loop-stopper into a no-op. Stays
  fail-open (never wedges the PM).
- #29: pinned (REFUTED-with-pin) — _lane_claim_guard already returns the
  error envelope without releasing the claim on a transient lookup error.
- #30: pinned (REFUTED-with-negative-pin) — a non-batch branchless main_pm
  root cannot bypass the complete spec gate (is_batch_umbrella requires
  batch_id set).
- #82: _post_gate_review_to_pr wraps the slug-resolution call in try/except
  (mirrors _capture_pr_head_sha) so a malformed cell_map AttributeError no
  longer 500s the reviewer after a committed gate transition.
- #188: _is_hand_formatted_verdict anchors the header regex to line-start,
  so a quoted (> ## Summary) or inline (mid-prose) header mention no longer
  false-refuses a hand-formatted verdict.
- #189: pr_fail re-captures the PR head SHA after the transition commits and
  re-stamps the verdict note only when it advanced (closes the stale-SHA
  false-allow loop-hole); no-advance stays a single note write.
- #192: claim_gate_review skips the dev claim guards (already_active/paused/
  lane) via a new skip_dev_guards param — a pr_reviewer inspecting an
  assembled PR does not start work, so the single-active-task / code-lane
  invariants do not apply; the dependency guard is kept, and QA's
  claim_review parity is preserved.
- #157/#187: verified in tree — pr_review-only handoff is intentionally
  prior-work-worth-resuming; self_review_block wiring (reviewer != dev)
  holds on assembled tasks with 4 existing pin tests.

TDD: test_choreographer_*, test_pr_gate_posts_review (#82),
test_pr_review_hand_format_guard (#188), test_submit_root_unchanged_pr_guard
(#189), test_claim_gate_review_guards (#192), test_verb_runner_midverb
_invalid_state (#102 pin).

* playbook curate: guard the gating commit against a poisoned session (#55)

The explicit `session.commit()` that gates the RAG index (commit-before-index
so an uncommitted playbook cannot land in the corpus) raised PendingRollbackError
when a prior mid-verb failure had rolled the caller's session back — 500ing the
whole curation verb instead of returning a clean envelope, and (worse) risking a
fall-through to index an uncommitted playbook. Wrap the commit: on
PendingRollbackError, log + return invalid_state with a re-fetch/retry remediate
and skip the index. The happy path still commits exactly once then indexes.

TDD: test_playbook_verbs.py — poisoned-session returns a clean invalid_state and
does NOT index; clean-session still commits once + indexes (pins no fail-closed
inversion / no double-commit).

* [chore] gateway: atomic activate merge — preserve probe_failures across re-park (#156)

activate() was a blind SET that reset probe_failures to 0, so a probe-failure
increment that just landed (or was in flight) could be wiped by a concurrent
re-park — resetting the give-up / CEO-notify count mid-episode. Route activate
through a server-side Lua merge (roboco:activate_rate_limit) that refreshes the
episode metadata (kind / activated_at / retry_after / affected_agents) while
carrying over the previous probe_failures count. Indivisible w.r.t. the
increment/reset scripts (Redis single-threads an EVAL).

#56 (notify to prompter/secretary refused) verified SAFE — the pin tests
(test_notify_rejects_prompter_recipient / _secretary_recipient /
_allows_ceo_recipient) already cover the only human notify target invariant;
no legitimate send is dropped, no code change.

* [chore] foundation/policy: spec gates + QA retry-key pin (Cluster F)

#50 sync_branch composes=() so the spec gate accepted a terminal/paused/
blocked task and the handler rebased a dead/parked branch — add a
PRECONDITION_SYNC_BRANCH_STATE (claimed/in_progress/verifying/needs_revision
only), rejection_kind=invalid_state. TDD: 28 spec tests.

#148 submit_root's prose asserts 'a Main-PM root is planning-typed, never
code' but only the creation path (main_pm_cannot_own_code) backed it — add
PRECONDITION_ROOT_NOT_CODE on the submit_root IntentSpec (defense in depth),
scoped to submit_root only so the shared submit_for_review action keeps
cell_pm+code submit_up parity. Graceful on Mock/None task_type so the
choreographer Mock-task tests don't crash. TDD: 2 spec tests.

#150 VERB_RETRY_LIMITS is keyed by the MCP-exposed names (pass/fail), not
the IntentSpec-internal pass_review/fail_review — already correct; add a
pin test so a one-sided rename can't silently drop the QA-handoff cap.

#142 main_pm_cannot_own_code/pm_cannot_own_code already normalize casing
(.lower()) — no-op, pin test test_main_pm_cannot_own_code_is_case_insensitive
already in tree.

* [chore] worksession-git: 405 merge-method fallback + non-destructive close (Cluster W)

#108 _merge_with_retry hardcoded 'squash' and raised MergeConflictError on a
405 with no method fallback — wedging the PM on an open, mergeable PR whose
repo merely had the squash button off. Add a 405 fallback to a permitted
method (via _first_allowed_merge_method, exclude='squash'), mirroring the CEO
merge_pull_request path. A 405 with no permitted fallback (or a second 405)
still falls through to the already-merged disambiguation / MergeConflictError.
TDD: 2 new tests (fallback-success, no-permitted-method-raises).

#109 close_pull_request defaulted delete_branch=True, so the choreographer
supersede path deleted a superseded PR's branch while the orchestrator
supersede path explicitly preserved it — the two disagreed, and the
destructive default ran on the 'close the dead PR' path where the branch may
still be referenced / useful for audit. Flip the default to False (opt-in
deletion) and make the choreographer caller explicit (parity with the
orchestrator). TDD: 1 new test (default preserves branch); existing
deletion-when-requested test now passes delete_branch=True explicitly.

Dispositions verified against current code (no silent drops):
- #27 REFUTED/FIXED-UNDEPLOYED: work_session.merge_pr resolves by session_id
  (no global pr_number lookup); the real cross-repo collision fix
  (project_id scoping on pr_merge/close_pull_request/rebase_pr_for_task/
  pr_target) is already in tree + tested (test_pr_merge_scopes_task_lookup_
  by_project_id, test_close_pull_request_scopes_task_lookup_by_project_id,
  test_git_pr_target_scoping). Verify-only.
- #106 REFUTED: a guard exists (rev-list --count {base_ref}..{branch} == 0)
  before reset --hard + base_ref falls back to default_branch; tests lock
  the safety (test_create_branch_never_repoints_branch_with_real_work,
  test_create_branch_does_not_reset_or_checkout_shared_clone).
- #218 BY-DESIGN: the merge_pr idempotent guard intentionally preserves the
  audit trail (docstring + test_merge_pr_idempotent_on_already_completed_
  preserves_audit_trail); a COMPLETED session always carries attribution
  (COMPLETED only via merge_pr), so the NULL-COMPLETED case is unreachable.
- #104 BY-DESIGN: agents never merge to the repo default branch in RoboCo's
  model (root→master is CEO-only); the guard is a correct CEO-only rail,
  locked by test_pr_merge_into_default_branch_is_ceo_only.

* [chore] llm: surface disabled-provider downgrade + scrub probe log (#20/#3/#211)

#20/#3 resolve_for_agent silently fell through to the legacy Anthropic path
when a configured provider was disabled — indistinguishable from 'no
assignment', so the operator got no signal that spawns bypassed the
provider. Surface the bypass with a warning (graceful degradation stays the
default — a stalled spawn is worse than a routing miss) and add an opt-in
ROBOCO_ROUTING_STRICT (default-off) that fail-closes instead. Wired into the
panel Feature Flags card. TDD: 3 unit tests (warn-on-disabled, strict-raises,
no-assignment-stays-silent).

#211 probe_ollama_tags logged str(exc) raw on the generic-exception branch —
structured log could carry connection internals / stack traces. Log the
exception class name only. TDD: existing generic-branch test strengthened to
assert the log kwargs don't leak the raw text.

* [chore] support/stream/optimal/playbook/comms hardening (Cluster S)

Logical-gaps sweep, Cluster S (TDD, red→green per item):

#64 notification_delivery.acknowledge published the NOTIFICATION_ACKED bus
event directly (bypassing the outbox) — a rollback left a phantom ACK. Route
it through defer_bus_publish (after_commit), mirroring deliver.

#76 playbook.archive()/reject() stamped the archiver into approved_by/
approved_at, overwriting approval provenance (and fabricating approval for a
rejected draft). Add archived_by/archived_at (migration 053 + table + model)
and write those on archive/reject, leaving approval attribution intact.

#181 vector_store.replace_chunks wiped existing index rows even when every
chunk lacked an embedding (embedder failure). Skip the wipe when chunks is
non-empty but records is empty — preserve good rows for nothing.

#182/#183 optimal.record_learning recomputed a learn-{md5(full_content)}
tracking source that never matched the URI the plugin embedded chunks under
(roboco://learnings/{doc_id}, doc_id=lrn-{hash100}). Use the plugin's
returned doc_id so de-index/lookup-by-source finds the chunk rows.

#96/#97 transcription periodic flush only peeked ready buffers (unbounded
map growth) and ran sync callbacks on the event loop (a slow callback
blocked the flush task). Flush (remove) each ready buffer after notifying,
and offload each callback to a thread.

#212 _TEAM_SCOPED_ROLES was duplicated across communications/agents_config/
seeds. Single-source it in foundation.policy.communications; consumers
reference that object (identity-tested).

#19 stream_bus._dispatch_event re-ran already-succeeded handlers on a
recover_pending replay (duplicate side effects). Add a per-(event.id,
handler) SET-NX idempotency guard: skip on a hit, clear the key on handler
failure so a replay re-runs it, fail-open when redis is unavailable.

Dispositions (no code change): #77 approve() index-write pair asserted
BY-DESIGN; #62/#63 notification DB-dedup verified pinned; #184/#185 REFUTED;
#214 REFUTED; #215 BY-DESIGN.

* [chore] db/migrations: graph-integrity guard + conftest unreachable-DB warning (Cluster D)

Logical-gaps sweep, Cluster D (TDD + real alembic upgrade head verification):

#16/#37 add tests/unit/test_migration_graph_integrity.py — a static guard that
the alembic migration graph has exactly one head, every down_revision resolves,
every revision is reachable from a root, and no revision id is duplicated. The
suite builds its DB via Base.metadata.create_all (not alembic upgrade head), so
a forked head / dangling down_revision / duplicate id would otherwise ship
silently and break a real deploy mid-stream.

Caught a real bug in the process: migration 053's revision id
"053_playbook_archived_attribution" (33 chars) exceeded alembic's
alembic_version.version_num VARCHAR(32) — a fresh `alembic upgrade head` raised
"value too long for type character varying(32)" at the 053 stamp. Renamed to
"053_playbook_archived_attr" (26 chars). Verified end-to-end on a scratch PG:
upgrade head stamps 053, downgrade -1 returns to 052. (The pre-existing
test_every_migration_revision_id_fits_the_alembic_version_column guard is now
green too; it had been red on the 33-char id.)

#90 conftest silently pytest.skip'd every DB test when Postgres was unreachable
— a non-Docker box reported a green run of all-skips. Extract the warning into
_warn_if_pg_unavailable and fire it at import so the operator sees the DB is
down (the per-test skip path is unchanged). Test: warns when unavailable, silent
when reachable (verified under -W error::UserWarning).

Dispositions (verified against real code + a fresh alembic upgrade head, no code
change): #6 REFUTED — sa.Enum(create_type=False) at 001:119/304 does NOT break a
fresh upgrade head (001→052 applied cleanly on a scratch DB); #8 REFUTED — the
upgrade passed 030/031 (RAG chunk tables) without pgvector installed; pgvector is
a runtime concern handled by roboco/db/base.py, not a migration prerequisite;
#40 REFUTED — the `|| echo` mask was already removed and partial-schema drift
reports exit 1 (only by-design unreachable/unmigrated skips remain); #204 REFUTED
— the property walk seed IS pinned (random.Random(20260504), line 97); #205/#206
REFUTED — the smoke-trace fixture IS wired via
test_lifecycle_smoke_replay.py (8 passed); no shell smoke scripts exist in the
tree to wire; #137 BY-DESIGN — pyproject version 0.14.0 is an operational note,
no code gate.

* [chore] panel: admin-override force flag + kanban subtask_count + ws cleanup + ui-store dedupe (Cluster P)

#13: kanban admin-override into a hatch state (completed / awaiting_qa /
awaiting_pm_review) now requires an explicit force=true from the panel and
emits a dedicated task.admin_override audit row server-side; non-hatch
overrides need no force. Backend gate in tasks route + admin_set_status;
panel kanban-board sends force for hatch targets; TaskUpdate carries force.

#198: kanban service threads the real subtask_count (one grouped query) into
dev + priority-swimlane + main-pm-flat boards instead of a hardcoded 0.

#79: useWebSocket cleanup clears messages/lastMessage/state on unmount or
endpoint change so a dep-change (navigating to another stream) can't leak the
prior subscription's stale snapshot as live.

#186: disambiguate the duplicate ui-store modules -- the session/scroll store
in lib/stores renamed to useScrollRestorationStore / scroll-restoration-store
(barrel + 2 consumers updated); the sidebar/theme useUIStore in @/store is now
the sole useUIStore.

#12: verified already in-tree (release-proposal-card surfaces non-404 errors
with retry; getProposal maps only 404->null). #80 by-design (handleTransportError
already resets isSending on a no-payload SSE drop). #81 docs (streamUrl docstring
records that live-intake SSE auth is session-id-based bearer-style).

Backend: ruff+mypy clean, 278 tests green. Panel: lint+typecheck clean, 159 tests.

* orchestrator: park/reaper/readopt/a2a hardening + self-heal/ci-watch dedupe (Cluster O)

Closes the orchestrator-side logical gaps from the sweep:

- #75 a2a human-only drop surfaced: _dispatch_a2a_work logs the skip
  ("a2a request targets a human-only role; left as a notification for the
  human (not spawned)") instead of silently dropping the target — the
  CEO/secretary/prompter still see the notification; only the spawn is
  suppressed. (orchestrator.py)
- #72 readopt liveness: _readopt_running_agents requires a non-stale live
  claim (via _agent_holds_live_claim) and skips a zombie container so a
  reaped-but-restart-readopted agent isn't double-counted as active.
- #74 shutdown drain: stop() calls _flush_respawn_tracker so the durable
  respawn counter write-throughs aren't lost on a clean stop.
- #71 resolve_wait active-guard + deferred liveness: a rate_limit_lifted
  WaitingRecord is only confirmed-live after a _confirm_resume_liveness
  probe (deferred deletion _resume_confirm_delay=30.0), and an
  already-active agent short-circuits the repark. Scoped to
  rate_limit_lifted records (the only ones at risk of a false lift).
- #73 stuck-Claude kill: _maybe_kill_stuck_claude + _claude_stuck_kill_ttl
  (config.claude_stuck_kill_seconds) — a live container whose heartbeat is
  stale past the grace AND whose gateway probe is broken is killed+evicted,
  not protected forever by the reaper's live-skip.
- #230 verified FIXED-UNDEPLOYED: _gateway_broken_past_grace already
  requires N consecutive false-broken probes (not one flaky streak); no
  change, test added to pin the N-consecutive invariant.
- #43 self-heal per-observation dedupe: a fingerprint collapses repeat
  CEO notifications for the same CI regression.
- #44 ci_watch dedupe by (git_url, workflow): a monorepo's multiple
  workflows each get their own fix task (was collapsed by git_url alone).
- #49 identity.role_for_slug_or_none None-hardening: a stale/malformed
  slug resolves to None and the human-only skip falls through to the safe
  "not spawnable" path instead of crashing.
- #193 strategy engine: notify the CEO on a persistent assess failure
  instead of failing silently in the background loop.

TDD: test_no_spawn_human_roles (a2a skip surfaced), test_orchestrator_
shutdown_drain (#74), test_provider_overload_break (#71), test_readopt_
running_agents (#72), test_resolve_wait_repark (#71), test_stale_claim_
reaper (#73/#230), test_strategy_engine_loop (#193, new),
test_self_heal_engine (#43), test_ci_watch_engine (#44), test_identity
(#49). All red->green.

* chore: make-quality green — xenon complexity refactors + mypy test fixes + lifecycle regen

No behavior changes. Brings the tree to a fully green `make quality` (the
base branch never passed the xenon B-rank gate on several blocks; the
lifecycle artifacts had drifted from the committed ceo_reject_to_pool edge).

Xenon B-rank refactors (extract a helper; preserve semantics exactly):
- api/routes/tasks.py: _apply_forced_status_override + _StatusOverride
  dataclass bundle (update_task override block).
- services/task.py: _enforce_no_pm_code_on_create (create guards) +
  _escalation_diverts_to_pool (collapses the two board/advisory +
  main_pm+code divert branches into one predicate).
- services/prompter.py: _coerce_pm_code_to_planning (create_task_from_draft).
- services/notification.py: _duplicate_unacked_exists (_create_notification
  purpose-based dedup query + ACK_REQUIRED_BY_TYPE gate).
- services/sequencing.py: _same_assignee_lane_edges (the undeclared-surface
  same-assignee lane fallback at the tail of dev_task_collision_edges).
- gateway/choreographer/_impl.py: _pm_task_type_error static helper
  (_validate_assignee_task_type compound PM guard).
- gateway/choreographer/pr_gate.py: _gate_review_event_verdict +
  _gate_review_body static helpers (_post_gate_review_to_pr).

mypy test fixes (no type:ignore — banned; use typing.cast with quoted
strings per TC006):
- test_task_update_completeness: TaskUpdate(acceptance_criteria=None).
- test_bus: cast("Redis", _FakeRedis()); Redis import under TYPE_CHECKING.
- test_pr_merge_concurrency: capture AsyncMocks into locals before asserting.
- test_notification_delivery_phantom: cast("UUID", to_agents[0]).

Lifecycle artifact regen (owed from Cluster T #100 — the
awaiting_ceo_approval -> pending `ceo_reject_to_pool` edge was added to the
spec in 3d633084 without regenerating the derived artifacts the
foundation-check gate diffs against): docs/rag/lifecycle/intent-verbs.md,
docs/rag/lifecycle/status-transitions.md, panel/lib/lifecycle.json.

services/kanban.py: ruff format only (collapses the _load_subtask_counts
signature that drifted unformatted from Cluster P).

* [chore] logical-gaps sweep — Cluster I (intake/product/pitch)

#57/#58 prompter: preserve a top-level product_id with a 1-cell map
(prompter.py create_task_from_draft — top-level target wins over a
redundant 1-cell map instead of dropping product_id); reject — not
silently skip — a malformed project_id in the_work cell entries
(prompter.py _draft_cell_map raises ValidationError).

#59/#159 prompter: create_task_from_draft now operates on a copy
(_copy_draft) so _validate_and_coerce_draft / _clean_list never mutate
the caller's draft dict.

#160 prompter: _resolve_owning_team consults product/board routing
before forcing MAIN_PM on a multi-cell map (product root stays Board,
product+assignee-is-board stays Board).

#83/#84 github_provisioning: create_repo is idempotent by GitHub name
— a 422 "name already exists" (orphaned repo from a rolled-back prior
approval) is fetched and reused instead of erroring; pitch re-approval
now reuses the orphaned repo end-to-end.

#196 kanban: flat main-PM board has a "coordination" column for
non-cell teams (MAIN_PM/Board) instead of dropping their cards.

#197 project update: an explicit null in the PATCH body now clears the
stored field, distinct from an absent field (leave unchanged).
ProjectService.update drops exclude_none so explicit-None applies; the
PATCH route uses ProjectUpdate.model_validate(data.model_dump(
exclude_unset=True)) to preserve the request's unset-tracking (the old
field-by-field construction marked every field set and defeated the
distinction — nulling NOT-NULL git_url).

TDD: prompter 47, github_provisioning+pitch 16, kanban+project 67,
project routes 37 — all green; ruff + mypy clean.

* [chore] logical-gaps sweep — Cluster M (mcp-servers)

#60 flow_server/do_server: the circuit-breaker substitution no longer
erases the fixable rejection — the original envelope (kind/message/
remediate) is nested as inner on a copy of the SDK's circuit_open
envelope (the SDK dict is not mutated in place). The agent still sees
WHY the verb failed, not just that the breaker tripped.

#61 flow_server/do_server: a 404 carrying a *descriptive* detail
(not FastAPI's bare default {"detail":"Not Found"}) is now a
real resource not_found, surfaced as not_found so the agent
re-fetches state — instead of a misleading "server-side wiring gap"
invalid_state. The bare default and unparseable 404s still synthesize
the wiring-gap envelope; a 404 with a real Envelope (error field)
is still surfaced as-is.

#161 flow_server/do_server: dict error.code classification now uses
an exact-code map (authoritative for the codes the handlers emit) with
a substring fallback for unknown codes. Fixes the real regression:
AUTHENTICATION_REQUIRED carries no AUTHORIZED/DENIED/PERMISSION
substring, so the old substring-only rule dropped it to invalid_state
instead of not_authorized — an auth storm attributed as a state storm.
The fallback also adds AUTH so future AUTH-prefixed codes classify.

#162 flow_server/do_server: _register_tools gains a
ROBOCO_ALLOW_FULL_TOOLSET env override (default-off) so a missing
manifest falls back to the full tool set instead of raising — a
dev/test escape hatch. Production fail-loud behaviour is unchanged.

#163 intake_server: propose_batch accepts name as well as
title (intake drafts in the wild have used both), normalizing a
name-only draft onto a copy as title (caller's dict never mutated),
and reports the dropped count + reason in the return instead of
silently vanishing malformed drafts. The empty-batch hint now names
name as an alternative.

TDD: 123 mcp_servers tests green (14 new + 2 updated); ruff + mypy clean.

* [chore] Cluster N — conventions/docs logical-gaps sweep

#33: _create_new_doc/_update_existing_doc now resolve via
_resolve_contained_path (the RAG-returned update path was not containment-
checked — an escaping source could write/overwrite outside the docs dir).
#34: _commit_doc_to_repo returns committed/skipped/failed instead of
swallowing all exceptions; surfaced on DocRef.commit_status, the write
response, and the docs MCP guidance so a failed repo commit is fail-loud.
#35: write_doc only updates the similar doc when its filename matches — a
different filename creates a new file instead of collapsing onto the
similar doc's path (the dedup-overwrite defect codified by the old tests).
#129: a custom rule scoped to a language the validator never reports (a
typo) is surfaced as a warn finding on .roboco/conventions.yml via the
runner's once-per-run validation; #32 (tsx->typescript dialect) stays
BY-DESIGN.
#130: _cache_put only swallows a UNIQUE violation (23505) as a concurrent
duplicate; a non-unique IntegrityError (FK/NOT NULL/check) is log-errored
and re-raised instead of being silently misattributed.
#132: health re-reads the live file status (a cached degraded row hid an
in-place repair at a stale head key); get_map skips cached degraded rows
and stops caching degraded so a repaired file re-derives. #134 BY-DESIGN.
#133: _DB_METHODS gains stream/stream_scalars (SQLAlchemy 2.0 streaming
constructs are data access too — a route calling them is not thin).
#199: regenerate_verb_tables._annot_str strips Annotated[...] metadata
(BeforeValidator) before rendering; regenerated verbs.md + per-role
prompts so the BeforeValidator(func=...) repr (with a memory address) no
longer leaks into agent-facing prompt text.

TDD: 206 conventions/docs tests green (incl. 5 new files / appended
cases); ruff + mypy clean.

* [chore] Cluster 16 — cross-cutting hygiene logical-gaps sweep

Disposition + fix the 10 cross-cutting-hygiene gaps, TDD. make quality green
(ruff, mypy 944 files, pytest, xenon, vulture, foundation-check, enum-parity).

FIX:
- #24 /ws/system now gated by _require_panel_token (matches every sibling
  /ws/* stream); rejects a missing token in strict mode and a forged token
  even in dev. (roboco/api/websocket.py)
- #25 two drifted _require_ceo implementations (orchestrator router vs release
  handler) unified on a single require_ceo_role helper in deps — same 403,
  same role set, accepts Role/AgentRole/"ceo". (roboco/api/deps.py,
  routes/orchestrator.py, routes/release.py)
- #11 a spawn session for a delivery role (developer/qa/documenter) with no
  task_id now logs an unattributed-usage warning via is_unattributed_delivery_spawn.
  (roboco/runtime/orchestrator.py)
- #65 pricing returns a structured CostResult(cost_usd, unpriced, is_anthropic)
  so an unpriced Anthropic model (real spend we'd undercount) is flagged
  instead of silently $0; calculate_cost stays a thin float wrapper.
  (roboco/billing/pricing.py, billing/__init__.py)
- #67 blocker-metrics "blocked since" reads the task.blocked audit transition
  (indexed on target_id/event_type/timestamp), not updated_at — which
  over-counted when a blocked task was touched for a non-blocking reason.
  Falls back to updated_at/created_at only with no audit row.
  (roboco/services/metrics.py)
- #94 grok refresh_if_stale uses double-checked locking (_refresh_lock +
  _recheck_or_refresh) so two concurrent callers don't both POST the
  single-use refresh grant and burn the credential. (grok_auth.py)

DOCS (fix the doc, behavior already correct/pinned by tests):
- #66 get_summary docstring corrected — it sums raw agent_spawn_sessions rows
  (sub-day precise); daily_usage_rollups/get_today_summary can diverge for
  "today" until the sweeper catches up. (roboco/services/usage.py)
- #68 DashboardStorage is a documented in-memory stub; added a test pinning
  that auditor flags are lost on storage reset (persisting = a migration +
  service refactor, out of scope as a half-implementation).
  (tests/integration/test_dashboard_service.py)

BY-DESIGN (no code change, with file:line evidence):
- #28 dashboard reads are open to the authenticated operator (dashboard.py:36
  documents this); mutating auditor routes already gate via
  _require_auditor_or_ceo. Role-gating reads would break the panel (no
  X-Agent-ID on dashboard reads) and CEO-token-gating the router would block
  the Auditor (auditor token != CEO token). nginx is the prod boundary.

REFUTED (narrowing would reintroduce a documented hang):
- #93 the ~/.grok directory mount (vs a single auth.json file) is load-bearing
  — a single-file bind mount pins the inode so the atomic tmp.replace refresh
  doesn't propagate to running containers (they hang at grok's login prompt).
  Already documented in grok.py:161 and locked by
  test_intake_grok_mounts_subscription_auth_when_present.

Incidental gate-greening (mypy errors a stale .mypy_cache had hidden in
earlier-cluster test files; xenon refactors for the new B-threshold):
- tests/unit/test_regenerate_verb_tables.py: type the dynamic-module loader.
- tests/unit/services/test_prompter.py: annotate the draft dict as dict[str,Any].
- tests/unit/services/test_conventions_cache_put.py: _FakeOrig is a real Exception
  (IntegrityError's orig arg requires BaseException).
- metrics._blocked_since_map extracted from get_blocker_metrics (complexity).
- intake_server._normalize_batch_drafts extracted from propose_batch (complexity).

* Docs update

* [bug] spawn: self-heal vanished clone + branch ref before worktree ensure (be-dev-1 fatal loop)

A vanished clone_root (disk loss / /data/workspaces wipe / manual cleanup)
fatal-looped the resume path: _ensure_worktree_before_spawn ran
`git -C <missing>` and released the claim, but the reaper-style release
preserves assigned_to + branch_name so the next dispatch is a RESUME
(create_branch never re-runs to re-clone) and the same missing clone failed
every ~30s.

- workspace.py: ensure_worktree_self_heal re-attaches a present worktree +
  symlinks the shared .venv; on a missing local branch ref it fetches from
  origin (create_branch pushes at claim time, so pushed work survives) and
  re-creates the ref, falling back to -b origin/HEAD only when the branch
  was never pushed. _fetch_branch_ref is the token-aware fetch helper.
- orchestrator.py: _ensure_worktree_before_spawn health-checks the clone
  and re-clones via ensure_workspace BEFORE the worktree self-heal. Fatal
  git-state (WorkspaceError) still releases the claim + aborts; transient
  failures abort without releasing (a fresh claim wouldn't help and
  re-cloning is destructive).

TDD: 21 new + 61 related worktree/git/cancel/cleanup tests green; ruff +
mypy clean.

---------

Co-authored-by: Renn F <rennf93@users.noreply.github.com>
2026-06-30 08:08:35 +02:00
889f3689e7 MegaTask (#248)
* feat(batch): batch_id + collision descriptor columns

Sequenced batch intake ("Mega task") foundation: tasks.batch_id (indexed)
groups a batch of top-level tasks created together; intends_to_touch (text[]),
adds_migration and touches_shared (bool, NOT NULL default false) are the
per-task collision surface the SequencingService will read to wire dependency
waves. Mirrored on the Task model + TaskCreateRequest and wired through
TaskService.create. Migration 046 (real upgrade->downgrade->upgrade verified
vs a throwaway pgvector PG); a non-batch task declares no surface (defaults).

Task 1 of the 0.11.0 sequenced-batch-intake plan.

* feat(batch): flag + draft collision descriptors

Default-off ROBOCO_BATCH_INTAKE_ENABLED (config + FEATURE_FLAGS + panel card);
the propose_draft tool doc + the TS DraftProposal gain the per-task collision
surface intends_to_touch / adds_migration / touches_shared. The draft is a loose
dict so the descriptors ride it through the relay intact (test asserts the
forwarded payload); the analyzer (Task 3) reads them to wire dependency waves.

Task 2 of the 0.11.0 sequenced-batch-intake plan.

* feat(batch): deterministic collision-sequencing analyzer

SequencingService.analyze turns a batch's per-task collision surfaces into a
dependency DAG + execution waves — correctness in CODE, not agent judgment.
Rules in order: file overlap serializes (more-important first), migrations form
a serial chain (no concurrent Alembic heads), touches_shared runs last, cell
contention warns (never serializes); then dedupe, existence + cycle check, and
Kahn topological layering. Pure (no DB/services); SequencingError on a cycle or
out-of-range edge.

Golden test reproduces the CEO's hand-sequenced 4 waves of the 11-item
guard-core-app batch (the effort that deadlocked the Main PM): S6 alone last,
the R1/R3/R4 migration chain, R2/R3/S8 serialized on the shared threat service,
S1/S2/S7 in one parallel wave.

Task 3 of the 0.11.0 sequenced-batch-intake plan.

* chore(batch): brand the user-facing surfaces "MegaTask"

The user-facing name is MegaTask: the feature-flag label is "MegaTask intake",
the panel flag-card and the config description lead with MegaTask. Internal
names stay technical (batch_intake_enabled, batch_id, SequencingService).

* chore(batch): drop the feature flag — MegaTask is a core intake scope

MegaTask is additive and opt-in by its own nature (the Prompter proposes a
batch only when the CEO asks for several tasks; single-task intake is
unchanged), so there is no risk surface a flag protects — 'don't create a
MegaTask' is the off switch. Remove batch_intake_enabled from config, the
FEATURE_FLAGS registry, the panel flag card, and its tests. MegaTask will be
a third scope option in the Intake modal (single-cell / multi-project /
MegaTask), not a toggle.

* feat(batch): MegaTask identity predicate + orchestrator branchless recognition

The single source of truth for the umbrella's exemptions: pure
is_batch_umbrella / is_batch_root_subtask / is_branchless_coordination
(foundation/policy/batch.py) — an umbrella has a batch_id and is top-level; a
root-subtask shares the batch_id but is parented. The orchestrator's
_is_coordination_task now consults is_branchless_coordination, so a MegaTask
umbrella is recognized as doing no git of its own (git-exempt at spawn-readiness
/ stuck-detection) exactly like a product fan-out root. Non-batch behavior is
identical (the predicate reduces to the old no-project+product check; the
orchestrator coordination suite stays green), and the umbrella branch is inert
until the create path exists.

First slice of the MegaTask umbrella enforcement (branchless guard).

* feat(batch): branchless umbrella guard across the git-exemption sites

A MegaTask umbrella does no git of its own — every git-exemption site in
TaskService now consults the shared is_branchless_coordination predicate
instead of an inline product-only check, so the umbrella's exemptions
cannot drift between sites:

- the claimed->in_progress branch gate (GitContext.is_coordination) lets
  an unbranched umbrella reach in_progress and delegate;
- _ensure_branch_for_task short-circuits an umbrella to "" instead of the
  misconfigured raise (the claim path ignores the return, treating it as
  branchless);
- CEO-reject routing sends a rejected umbrella to the Main PM in PENDING
  (needs_revision is developer-claim-only and would deadlock it).

Covers both shapes via the predicate (product fan-out root OR umbrella);
a batch root-subtask keeps its own branch/PR. Adds orchestrator
recognition tests for the umbrella plus claim/branch/reject integration
tests.

* feat(batch): umbrella assembles no PR; completes branchless

submit_root now hard-rejects a MegaTask umbrella up front (a preflight
that also folds in the unknown-role refusal to stay within the
return-count budget): the umbrella spans many projects with no single
master, so each root-subtask opens and is reviewed on its own PR — the
umbrella never enters the in-path review gate. The Main PM completes it
directly once every root-subtask is terminal.

Umbrella completion needs no new code: it is branchless (no branch_name),
so _main_pm_complete_guard already accepts it from in_progress, checks
all_subtasks_terminal, and main_pm_complete walks it to awaiting_pm_review
and escalates to the CEO with no PR creation — exactly the product
fan-out root path. Adds the submit_root-reject and umbrella-completion
gateway tests; pins batch_id=None on the normal-root submit_root test
(a MagicMock auto-attr would otherwise read as an umbrella).

* feat(batch): MegaTask create path — umbrella + sequenced root-subtasks

PrompterService.confirm_live_batch turns N confirmed drafts into a real
MegaTask: it builds each draft's collision surface, runs the pure
SequencingService to get conflict-free waves, creates the branchless
umbrella (batch_id, no project/product), then one root-subtask per draft
(own project, parent=umbrella, sequence=wave index, descriptors), and
wires the analyzer's edges through add_dependency so the existing
dependency-gate runs the waves in order. The route picks the start path
like a single confirm: 'board' holds the root-subtasks in BACKLOG for the
batch review; 'main_pm' creates them PENDING so wave 0 dispatches at once.

create_task_from_draft gains a BatchPlacement (parent/batch/sequence/
team_override) and forwards the collision descriptors; the exactly-one-
target rule (here and the TaskService.create invariant) is relaxed for an
umbrella, which legitimately targets neither. New route
POST /live/{session}/confirm-batch + BatchConfirmRequest mirror the single
confirm. Adds the structural-invariant + board-hold + empty-batch tests.

* feat(batch): release MegaTask root-subtasks on CEO approval; board awareness

The board route holds a MegaTask's root-subtasks in BACKLOG so the work
waits for the batch review. approve_and_start (CEO gate #1, board->Main PM)
now releases them via _activate_batch_root_subtasks: each held child flips
BACKLOG -> PENDING + team=main_pm so the dependency-gate dispatches wave 0.
No-op for a non-umbrella; idempotent (children past BACKLOG untouched).

The Product Owner and Head of Marketing identity prompts gain a MegaTask
section so they review the whole batch + wave plan and adjust scope before
sign-off (they review drafts; the umbrella is their unit). Also extracts
the create() target invariant into _require_target_or_umbrella to keep the
method under the complexity gate after the umbrella exemption. Adds the
umbrella-approval activation test.

* feat(batch): multi-project intake scope for MegaTask

A MegaTask spans several possibly-unrelated repos, so the intake chat can
now be scoped to an explicit project list (not just one project or one
product). StartLiveRequest gains project_ids; /live/start threads it
through start/spawn_intake_session -> _spawn_intake_container ->
_clone_intake_scope. The multi-repo clone machinery already existed for
products; _intake_scope_slugs now also resolves an explicit project_ids
set (split into _slugs_for_project_ids / _slugs_for_product), cloning each
repo with the first as the primary cwd and the siblings readable. Scope
validation is now 'exactly one of project_slug / product_id / project_ids'
via the shared _require_one_intake_scope. Adds scope-resolution, spawn,
and route tests for the MegaTask path.

* feat(batch): propose_batch intake tool (MegaTask multi-draft hand-off)

The intake agent can now hand the panel a whole MegaTask in one tool call.
Both intake paths gain propose_batch alongside propose_draft:
- Claude (intake_driver): a propose_batch tool registered on the in-SDK
  MCP server + allowlisted; the driver intercepts the ToolUseBlock and
  emits ONE StreamChunk(kind="batch") carrying {drafts:[...], title}.
- grok (intake_server): a propose_batch tool that POSTs a "batch" relay
  event via the shared _post_event helper (post_draft/post_batch).

A batch carries N drafts, each the propose_draft shape PLUS its own
project_id (a MegaTask spans unrelated repos) and collision surface so the
analyzer sequences the waves. The prompter prompt documents the MegaTask
scope + when to call propose_batch. Adds Claude-normalize and grok-relay
tests for the batch path.

* feat(batch): MegaTask intake panel — third scope, batch review, waves

The panel now drives a MegaTask end to end. The intake modal gains a
third scope, 'MegaTask', beside Single cell and Board-led: a multi-project
checklist (a MegaTask spans several possibly-unrelated repos), validated
to at least two. start() sends project_ids; use-prompter accumulates the
agent's single propose_batch hand-off as a 'batch' SSE event into a
BatchProposal and lands in a new batch_preview state.

A new BatchReviewCard lists every proposed task with its target project +
collision-surface badges (migration / shared) and offers one start path
for the whole batch — Board review & Start or Approve & Start — wired to
confirmBatch → POST /confirm-batch. The success card shows the sequenced
result: N tasks in M waves (+ any advisory notes). prompter.ts gains the
DraftScale 'megatask' + the BatchConfirm payload/result types; the SSE
client allows the 'batch' kind. Panel typecheck + lint + 113 tests green.

* docs(batch): MegaTask across changelog, CLAUDE.md, site, and RAG

The four documentation obligations for the MegaTask feature:
- CHANGELOG: an Unreleased entry covering the umbrella model, sequencing,
  multi-project intake, propose_batch, and the create/approval path.
- CLAUDE.md: a MegaTask section (identity predicate, umbrella/root-subtask
  hierarchy, sequencing rules, intake + create path, board activation).
- Published site: a user-facing company/megatask.md (scopes, waves, the
  umbrella, the two start buttons) + nav entry; a pointer added to the
  intake chapter of the Tour.
- RAG corpus: workflows/megatask.md so the Main PM (and any agent) can
  retrieve the umbrella's branchless / no-PR / completion rules at runtime.

The runtime concurrent-migration guard is intentionally NOT added: the
analyzer already chains migration-adders into dependencies and the
dependency-gate serializes them, so a separate guard would be dead code.

* feat(batch): batch_id guardrail + wave preview + batch_id on TaskResponse

Guardrail (CEO): a batch_id is denied on any task that is not a well-formed
MegaTask member. is_valid_batch_shape permits batch_id only on an umbrella
(no parent → must target neither project nor product) or a root-subtask
(has a parent → exactly one target); TaskService.create enforces it AND
verifies a root-subtask's parent is the batch umbrella (same batch_id,
top-level). This closes a latent hole: is_batch_umbrella is true for a
batch_id + no-parent task even with a project, so a stray batch_id could
have spoofed the branchless branch-gate / no-PR exemption. (The public
task API never exposed batch_id for write; this guards the service layer.)

Wave preview: PrompterService.preview_batch + POST .../preview-batch
compute a MegaTask's waves from the proposed drafts WITHOUT creating
anything, so the panel can show the sequencing before confirm. Extracted
_sequence_drafts as the single source shared by preview and confirm, so
the previewed waves are exactly the ones wired.

TaskResponse now carries batch_id so the panel can badge the umbrella.

* feat(batch): MegaTask review — project editor, wave preview, persistence, badge

Closes the panel gaps in the MegaTask review experience:
- Per-task project editor: each proposed task gets an inline project
  Select (updateBatchDraftProject), so a task the agent put in the wrong
  or no repo can be fixed before launch — not only by re-chatting. Launch
  stays blocked until every task has a project.
- Wave preview: on a batch proposal the panel fetches POST .../preview-batch
  (no task created) and shows the conflict-free wave plan, so the human
  reviews the sequencing before confirming.
- Refresh durability: the MegaTask review (batch + waves + projectIds) is
  persisted, so a browser reload mid-review restores it like a single draft.
- MegaTask badge: TaskResponse exposes batch_id, the panel Task type
  carries it, and the task table badges the umbrella row 'MegaTask'.

Panel typecheck + lint + 113 tests green.

* test(batch): stub task carries batch_id for task_to_response

task_to_response now serializes batch_id (TaskResponse field), so the
_stub_task SimpleNamespace fixture must provide it — without it the reader
hit AttributeError, failing the 8 task-schema serialization/enrichment
tests. Test-only; the real TaskTable carries the column (migration 046).

* fix(batch): close MegaTask audit gaps — completion crash, analyzer cycle, guardrails

An adversarial multi-agent audit of the feature surfaced 20 verified gaps;
this closes the backend ones.

HIGH:
- Umbrella completion crashed. escalate_to_ceo hard-required a pr_number,
  which a branchless umbrella never has, so main_pm_complete dereferenced
  None. Both pr_number gates now waive a MegaTask umbrella (escalate_to_ceo
  + the awaiting_pm_review->awaiting_ceo_approval lifecycle gate via a new
  GitContext.is_umbrella), and main_pm_complete guards a None return. The
  completion test had mocked escalate_to_ceo, hiding it — now a real
  service test covers the waiver.
- The collision analyzer could fabricate a cycle (a touches_shared +
  adds_migration draft overlapping another migration draft) and raise
  SequencingError — a bare ValueError that escaped as an opaque 500. The
  migration chain is now shared-last-aware (never contradicts rule 3), and
  _sequence_drafts translates SequencingError to a clean 400.

MEDIUM:
- Collisions are now project-scoped: two repos can't collide on a
  coincidental path or serialize independent migrations (DraftSurface
  carries project_id; rules 1/2/3 respect it).
- The batch_id guardrail ran only at create. update() + the PATCH
  null-clear path now re-assert is_valid_batch_shape, so a mutation can't
  break a member's shape and spoof the branchless exemption.
- A draft missing title/acceptance_criteria now raises ValidationError
  (was a bare KeyError -> 500).
- confirm_live_batch re-asserts every draft targets a scoped project and
  the batch spans >=2 distinct projects (project_ids added to the request).
- Route-level tests for confirm-batch / preview-batch.

LOW: strict multi-repo clone (fail loud on any unresolvable project);
malformed/empty propose_batch surfaces an error chunk (Claude) / refuses
to POST (grok) instead of silently acking; dropped malformed drafts are
counted and surfaced; stale grok intake docstrings updated.

* fix(batch): MegaTask panel + doc audit gaps

Frontend half of the audit fixes:
- The confirm payload now carries project_ids (the schema requires it), and
  the panel re-checks every task targets one of the scoped repos before
  launching, naming the offending task.
- The Review-MegaTask project picker is filtered to the scoped repos and
  the per-task validity (border + launch gate) keys off scoped membership,
  so a task can only be (re)pointed at an in-scope project — also fixing the
  case where the agent emitted a non-UUID / unknown project.
- Dropped malformed drafts are surfaced as a chat error so the human knows
  the batch shrank instead of silently confirming fewer tasks.
- Doc wording: a wave releases on the previous wave's terminal state
  (normally a merge; a cancellation releases it too), not strictly 'merged'.

* test(batch): lock the CEO's EXACT 4-wave hand-sequencing as the golden bar

The golden test asserted the constraints (S6 last, the migration chain, the
shared-threats serialization, S1/S2/S7 parallel) but not the full wave
partition. The bar for MegaTask is 'reproduce my exact waves or it's not
done', so assert the exact 4-wave partition the analyzer produces for the
guard-core-app batch:
  wave 1: R1 R2 S1 S2 S3 S5 S7  ·  wave 2: R3  ·  wave 3: R4 S8  ·  wave 4: S6
Confirmed unchanged by the audit's analyzer fixes (no migration is shared;
single project).

* fix(batch): tolerate a stub task in assert_batch_shape_intact

The batch-shape re-validation read task.batch_id directly, but update()'s
partial-caller contract is exercised with a SimpleNamespace stub that has no
batch_id column → AttributeError. Use getattr(..., None) for batch_id and the
shape fields so the guard no-ops on any task lacking the column (a stub, or a
non-batch task) while still enforcing on a real batch member.

* fix(orchestrator): authenticate internal API self-calls with the system identity

The dispatcher httpx clients were built without an agent identity, so the
orchestrator's self-PATCHes to /api/tasks/{id} (auto-block, auto-resume,
auto-recover, SLA annotation) were rejected 401 "Missing X-Agent-ID" and
silently no-op'd. The auto-resume that lifts a PM's paused parent could never
write, so paused/blocked parents stayed wedged and stranded their dependents
(the fe-pm/be-pm respawn churn seen in prod).

Header propagation was inconsistent across the separate AsyncClient call-sites:
only the main dispatch client carried the system identity; the readiness and
sweep clients did not. Hoist the identity into a shared _SYSTEM_API_HEADERS
constant and apply it to every API-facing dispatcher client. The system role
holds TaskAction.ASSIGN, so it is authorized for the audited admin_set_status
path those write routes use. The external provider-recovery probe client is
intentionally left untouched.

---------

Co-authored-by: Renn F <rennf93@users.noreply.github.com>
2026-06-24 01:15:57 +02:00
a6b67a6a58 Feat: board redraft loop (#139)
* feat(board): expose board review brief + guard approve-and-start

Slice 1 of the board-informed intake re-draft loop (backend foundation):

- JournalService.board_review_brief(task_id): the PO + Head of Marketing
  DECISION_LOG entries for a task, oldest-first, each tagged with author —
  the board's review as structured data.
- GET /api/tasks/{task_id}/board-review (PM-or-above) backing the CEO's
  approval/redraft surface, so the real board analysis is readable instead
  of a placeholder; BoardReviewEntry response schema.
- Guard: approve_and_start now refuses a board task whose review is not
  complete (service invariant + precise BOARD_REVIEW_INCOMPLETE at the route).
  Previously only the UI hid the button; the backend let an early/rogue call
  hand the task to Main PM mid-review.

Tests: brief filtering/ordering + endpoint (200/404) + the two guard paths.

* feat(panel): show real board review at the approve gate + live refresh

Slice 1 frontend of the board-informed intake re-draft loop:

- tasksApi.getBoardReview + useBoardReview hook consume GET
  /tasks/{id}/board-review.
- The Approve & Start dialog now renders the actual Product Owner + Head of
  Marketing notes (markdown) instead of a static placeholder, so the CEO reads
  the board's analysis before approving.
- C2: useTask polls (4s) while a task is still on the board with an
  incomplete review, so the Approve & Start button appears as soon as the
  board finishes — there is no per-task websocket. Polling stops once
  board_review_complete flips.

* feat(intake): board-informed re-draft loop (backend, cold path)

Slice 2 of the re-draft loop:

- update_live_draft: apply a board-informed re-draft to the EXISTING task in
  place (title/description/acceptance_criteria) — never a duplicate — then route
  it: 'main_pm' hands it to the Main PM via approve_and_start; 'board' clears
  board_review_complete for another review round.
- confirm route branches on task_id → update_live_draft vs confirm_live_draft;
  LiveConfirmRequest.task_id added (scope taken from the task, not required).
- POST /live/re-interview/{task_id} (PM-or-above): spawns a fresh intake session
  seeded with the current draft + the board brief (compose_redraft_message),
  scoped to the task's product/project. The cold path + Slice-3 fallback.
- format_board_briefing / compose_redraft_message helpers.

Tests: pure helpers + update_live_draft (main_pm hand-off, re-board reset,
missing-task).

* feat(panel): board-informed re-draft entry + prompter re-draft guidance

Slice 2 panel of the re-draft loop:

- 'Re-draft with board feedback' button on a board-reviewed task detail →
  /prompter?redraft=<taskId>.
- usePrompter.startRedraft(taskId): calls POST /prompter/live/re-interview/{id},
  scopes the chat to the task, and streams the re-draft; redraftTaskId is
  threaded (persisted across reload) so confirm carries task_id and updates the
  existing task in place rather than creating a duplicate.
- prompterLiveApi.reInterview; ConfirmPayload.task_id.
- Prompter role prompt: a 'Re-drafting after board review' section so the agent
  revises the included draft from the board brief instead of starting over.

Panel verified by CI (no local node_modules).

* feat(intake): keep-alive re-draft — park the intake agent during board review

Slice 3 of the re-draft loop (in-context fidelity; cold path is the fallback):

- Registry: LiveIntakeSession.task_id + park(session_id, task_id) (keep alive
  instead of reaping) + find_by_task() for board-completion injection.
- Confirm: the board route (first pass) PARKS the intake agent instead of
  reaping, so it keeps the whole interview in context.
- Orchestrator: on board-review completion, inject the synthesized board brief
  into the parked session (_inject_board_brief_into_parked_intake) so the
  resident prompter re-drafts in-context. No-op when nothing is parked (the
  container died / a new intake replaced it) — the cold /re-interview path
  covers that. No reaper change needed (an idle parked session spends no tokens
  and the budget sweep is the only agent-stopping sweep).
- Panel: confirm(board) keeps the chat alive (parked, redraftTaskId set) with a
  notice; the injected revised draft arrives over the existing stream to approve.

Tests: registry park/find_by_task/closed-ignored. Container delivery + the full
panel parked flow need live (container-runtime) verification.

---------

Co-authored-by: Renn F <rennf93@users.noreply.github.com>
2026-06-14 00:40:18 +02:00
b9082a5c70 Fix: agent idle deadlock and lifecycle hardening (#96)
* fix(panel): cap dialog height and pin footer so actions stay reachable

Shared DialogContent now caps at max-h-[85vh] with overflow-y-auto, and the
footer is sticky to the bottom. Long content (e.g. a pasted change-request
note) no longer pushes the submit/cancel buttons past the viewport — the body
scrolls while the actions stay visible. No-op on dialogs that already fit.

* feat(notifications): suppress duplicate same-purpose notifications at send

A notification is not created when an unacknowledged one with the same purpose
— same sender, same type, same task, overlapping recipients — already exists.
Body text is not compared, so rewording cannot defeat it; a different type,
task, sender, or an already-acked recipient all still send through. Stops
agents that loop re-issuing the same signal from piling up unread that
soft-blocks the recipient's idle path.

* fix(gateway): stop board/PM lifecycle verbs from 500-crashing

Two unguarded crashes that wedged the org in respawn/escalate loops:

- escalate_to_ceo dereferenced None.status when the verb runner declined the
  escalation (task not in awaiting_pm_review — e.g. a board agent escalating a
  blocked task). It now returns a clean invalid_state. The message/remediate
  build moved to a helper so the function stays within the complexity gate.

- The coordination-root git ops (pr_target, pr_merge, PR update, branch-token
  resolve) called UUID(str(task.project_id)) directly, which raised on a
  coordination/integration task (project_id is None — 'badly formed hexadecimal
  UUID string'). They now resolve through _project_for_task, which falls back to
  the product's repo for project-less roots.

* refactor(intake): split out _block_to_chunk per-block classifier

Extract the per-block classification from _blocks_to_chunks so each function
stays within the xenon cyclomatic-complexity gate (was rank C). Behaviour is
unchanged — verified by the existing intake_driver tests.

* feat(gateway): make the i_am_idle unread soft-block satisfiable

The soft-block on unread A2A / @mentions had no clearing path, so once those
briefing fields populated an agent could never idle — a whole-org deadlock.
Keep the guard (it is correct) and add the missing clear paths:

- New read_messages content verb (schema -> route -> handler ->
  a2a.mark_all_read -> MCP tool -> role do_tools): bulk-zeroes the caller's
  unread A2A and stamps read_at. The idle hint now points to it.
- list_unread_mentions returns UNACKED MENTION-type notifications (each @mention
  already raises one via messaging._notify_mentions) instead of raw,
  unconditional mentions, so they clear via the existing notify_ack. No schema
  migration needed.

The soft-block is now satisfiable: A2A via read_messages, mentions and
notifications via notify_ack.

* fix(tests): repair notification-dedup db.scalar mocks + prompter agent seeding

The notification send-dedup added a db.scalar() purpose-lookup to
_create_notification; the two hand-rolled _FakeDb test stubs (test_notification,
test_a2a_priority_tristate) had no scalar() method → AttributeError. Add
scalar() returning None (no duplicate) so creation proceeds.

Separately, the prompter '& Start' route tests assign the draft to a fixed
product-owner / main-pm AGENT_UUID but only seeded system + CEO, so the
assigned_to FK failed in isolation (and main-pm flaked in the full suite). Seed
both via idempotent merge() in _seed_project_and_ceo.

* fix(git): gitignore .pnpm-store + flag GH001 push rejection as permanent

A dev once committed the ~115 MB pnpm store → GitHub GH001 (>100 MB) pre-receive
reject → open_pr retry-loop. Two root fixes:
- Add .pnpm-store/ to .gitignore — an ignored dir can't be staged by any git add.
- push() restates a GH001 / file-size rejection as an unmistakable PERMANENT
  error pointing at i_am_blocked, so the agent stops blind-retrying a push that
  can never succeed (it otherwise mis-reads the raw output as a transient timeout).
  The per-verb retry cap (open_pr: 5) already bounded the burn; this ends it.

* fix(gateway): accept a PM decision note as satisfying the complete/submit_up reflect gate

A cell/main PM that wrote a fresh decision but no separate reflect note bounced
on the reflect tracing-gate indefinitely (re-confirmed live: cell PMs looped on
cell_pm_complete -> journal:reflect until reaped, burning tokens — worse because
each respawn resets the per-verb retry cap). For a PM closing/submitting a task
the decision note already documents the close; the separate reflect is the
redundant artifact weak-model PMs forget. Accept a fresh decision as satisfying
reflect for complete + submit_up — the gate still requires a decision +
substantive notes, so the close stays documented.

NOTE (enforcement tradeoff, flagged for CEO review): this intentionally relaxes
the PM complete/submit_up gate. It does NOT touch the developer i_am_done gate.

* feat(gateway): refuse i_am_idle when a PM still owns a task awaiting its review

A cell/main PM once tried to 'send work back' by DMing the developer and going
idle — but a DM changes no task state, so the task stayed awaiting_pm_review and
the orchestrator just re-dispatched the PM in a loop. i_am_idle now refuses (like
the pending-assignment guard) when a PM owns an awaiting_pm_review task, with a
clear remediation: complete() to finish, or reassign()/delegate() to route it
back. PM-only; devs/QA/doc unaffected. Pairs with the reflect-gate relaxation so
the PM can actually complete instead of looping.

* feat(gateway): push a prior-work handoff digest into task-scoped briefings

A freshly spawned or respawned agent previously started cold on every
lifecycle hand-off: the prior worker's PR, commits, acceptance status and
journal highlights lived in task evidence but were pull-on-demand, so each
new role agent re-explored the codebase from scratch — wasted tokens and
fragile context loss across respawns.

build_task_handoff() composes a compact, DB-only digest (no git diff) and
_briefing_for() now attaches it to context_briefing whenever the caller
already holds the task row. The digest is built only from a passed-in task,
so there are zero extra fetches: every resumption entry point (give_me_work
and pm_give_me_work, i_will_work_on, i_will_plan, triage/triage_all,
i_am_done, submit_up, escalate_up, complete) threads the loaded task, while
id-only correction/rejection paths cleanly omit it.

Every field is type-guarded so a partial row never leaks a non-serialisable
value into the envelope.

* docs(prompts): tell agents to resume from the briefing handoff before re-exploring

The base prompt described the success envelope but never told agents to act
on context_briefing, so a respawned or hand-off agent would re-scan the
whole repo and re-derive the plan even when the briefing already carried the
prior worker's PR, commits, acceptance status and journal highlights.

Adds a 'Resume from your briefing' section that walks each task_handoff
field and instructs the agent to continue from it — and to read the unread
A2A / mention / notification lists, which are messages addressed to them.

Pairs with the gateway change that now pushes task_handoff into every
task-scoped briefing.

* feat(tasks): remember cleared dependencies so the unblock briefing can surface them

When an upstream dependency completed, _unblock_dependents removed its id from
the dependent's dependency_ids to let it be claimed — destroying the only
record of which upstream task had just landed. The revived dependent then
re-discovered that work from cold.

Adds tasks.completed_dependency_ids (Alembic 026, uuid[] default '{}'):
_unblock_dependents now appends the cleared id there instead of only dropping
it, and the briefing handoff digest surfaces it so the agent picking the task
back up knows its blocker cleared because that upstream work shipped. The base
prompt documents the field.

Migration round-trip verified against postgres (upgrade adds the column,
downgrade drops it).

* docs(prompts): instruct PMs to split oversized tasks into per-concern subtasks

A subtask carrying a long acceptance list or spanning multiple layers/files
drove repeated QA failures and a PM revision loop — QA can't pass a partial,
and the dev keeps re-touching unrelated parts. Nothing in the PM prompts told
them to decompose by size/concern.

cell_pm gets a 'Sizing' rule: one subtask = one focused concern with ~2-4
criteria and its own dev->QA pass; decompose anything larger before
delegating, sequencing with dependencies. main_pm gets a matching reminder to
scope each cell's slice to that cell's layer rather than handing a cell a
cross-layer monolith that just pushes the problem down a level.

* fix(gateway): mirror the task= kwarg on ChoreographerHelpers helper signatures

The handoff-digest change added a keyword-only task= parameter to
_briefing_for and _build_tracing_gap in _impl, but the ChoreographerHelpers
base that the role mixins inherit still declared the old signatures, so the
composed Choreographer had two incompatible base definitions (mypy [misc]).
Sync the base declarations to match.

* fix(tasks): keep the owner on a substitute-out so the task isn't orphaned

build_substitute_update unconditionally nulled assigned_to, so any
substitute that routes to PENDING (max_retries, low_context, out_of_scope_*)
— the path a verb hitting repeated 500s or its retry limit takes — left the
task pending AND unassigned. The dispatcher only respawns a pending task when
it has an owner, so the task went dormant: no agent ever picked it back up.

Keep the task with its current owner instead. A substitute-out is almost
always a transient stall, so the task re-dispatches to the SAME agent, which
resumes from the briefing handoff. Only the task_complete -> PM-review handoff
changes owner (unchanged).

* feat(a2a): suppress duplicate unread A2A messages at send

A respawned or retrying agent could re-emit the same DM, stacking identical
copies on the recipient's inbox and re-bumping the unread count — noise that
the recipient then has to clear. The notification path already dedups; A2A did
not.

send_chat_message now suppresses a send when an identical message from the
same sender is still unread in the conversation, keyed on (conversation,
sender, message_kind, content). Genuinely different messages are never
collapsed (verified: distinct content still produces distinct rows), so this
avoids the earlier per-pair over-suppression. No migration.

* fix(panel): default the notifications view to Unread, not All

Landing on the All tab buried new notifications under everything already
seen — the most-reported annoyance. The Unread tab is the actionable view, so
make it the default; the All/Pending tabs are one click away.

* fix(panel): show clone progress during intake prep instead of a frozen pill

The first clone of a repo can take a few minutes, during which the intake
form showed only a static 'Preparing the agent…' button — indistinguishable
from a hang. Add a progress region while preparing: an elapsed timer, a
saturating progress bar (approaches but never reaches 100% until the agent
actually answers), and staged copy (spinning up → cloning → first-clone-takes-
a-while → reading the codebase) so the wait reads as work, not a freeze.

* feat(docs): index workspace-authored docs that never reached the RAG store

Docs written through roboco_docs_write land at /app/docs on the orchestrator
and index fine. But a documenter can also write docs with Edit/Write directly
in its own clone (README, CHANGELOG, workspace markdown); those resolve to a
/app/docs path that doesn't exist on the orchestrator, so the indexer reads
nothing and the docs never become searchable — a cross-container miss with no
shared mount to bridge it.

On docs completion, capture each listed doc's committed content out of the
branch (new GitService.read_file_at_branch, via git show) and write it
server-side under /app/docs before indexing, so workspace-authored docs reach
RAG too. Docs already present server-side are skipped; absolute paths and
unreadable/uncommitted files are passed over best-effort.

* feat(prompter): survive a browser reload by reconnecting to the live intake chat

The intake chat lived entirely in React state, so a page reload wiped it and
dropped the human back to the scope form — even though the agent container
outlives the page. Now the chat persists a small TTL'd slice (session id,
messages, scope, draft) to localStorage and, on mount, reconnects: it asks the
new GET /live/{id}/status whether the session is still running and, if so,
restores the history and reopens the SSE stream; if dead or expired it clears
and shows the form. A full reload doesn't run React effect cleanup, so the
navigate-away reap never fires on refresh and the session stays up.

Backend adds the status endpoint + PrompterLiveRegistry.is_alive; localStorage
is cleared on confirm, start-another, and SPA navigate-away.

* chore: remove internal session-bookkeeping refs from code comments (part 1)

Strip leaked task/finding numbers, Wave/Phase/cluster/audit labels from
docstrings and comments across services, foundation policy, runtime, mcp,
api schemas, and agent_sdk — they mean nothing to a repo reader and expose
process internals. Wording preserved; only the labels dropped. Done by hand,
one comment at a time (no scripted rewrite). _impl.py follows separately.

* chore: remove internal session-bookkeeping refs from code comments (part 2)

Finishes the manual scrub: the choreographer _impl.py docstrings/comments plus
the remaining dogfood-run ('smoke-N') labels across runtime, mcp, foundation,
api schemas, services, and agent factories. Reworded to describe the bug or
behaviour in plain words; every label dropped. The repo source is now free of
task/finding numbers, Wave/Phase/cluster/audit/smoke labels. By hand, one
comment at a time.

---------

Co-authored-by: Renn F <rennf93@users.noreply.github.com>
2026-06-10 12:04:24 +02:00
9f8834155a Feature: prompter gold upgrade (#84)
* feat(prompter): make the assistant a RoboCo insider and fully wire launch

The Prompter's intelligence lived in two thin static prompts, so it asked
generic checklist questions and produced a flat task. The launch path was
also only half-wired: the panel called the generic task-create endpoint with
no project, bypassing the Prompter's own confirm flow.

Interview brain
- Rewrite the chat system prompt with RoboCo's org model, the task-spec
  standard, a dimensions playbook, and a reflect-back, 1-2-questions-per-turn,
  auto-stop discipline.
- Inject the live projects/products list each turn so the assistant grounds
  questions in real surfaces and resolves the target itself.
- Replace the brittle phrase-match readiness with a parsed roboco-meta control
  block (parse_readiness); the block is stripped from the visible reply and the
  turn now returns draft_ready + scale.

Structured GOLD draft
- Add first-class draft fields (objective, what_this_builds, the_work, notes)
  carried in the existing draft_data JSONB — no migration.
- Compose the GOLD markdown description deterministically from those fields
  (compose_description); the model never hand-formats the body.

Adaptive routing + wired launch
- Confirm now runs through the Prompter confirm endpoint with the human's
  project/product choice and edited structured draft.
- Single-cell targets a project and the cell team; a multi-cell feature targets
  a product and becomes a Main-PM coordination root that fans out.

Frontend
- Turn-envelope draft_ready (drop the duplicated phrase-match), structured
  draft card, confirm dialog with a project/product picker and a per-cell
  The Work editor, and the corrected priority labels (0 highest .. 3 lowest).

* fix(prompter): commit session writes so they survive across requests

Session create returned 201 but the row was never durably committed, so the
immediately-following /messages call could not find it and 404'd. The prompter
routes were the only write surface that never called db.commit() — every other
write route (tasks, a2a, groups, docs, product) commits explicitly rather than
rely on the request-teardown auto-commit, which is sensitive to middleware and
teardown ordering under the production server.

- Commit explicitly in all four prompter write routes (create session, send
  message, get/generate draft, confirm).
- Fix _get_session's NotFoundError: it passed a full sentence as resource_type,
  producing the doubled "... not found not found" message; now uses the
  (resource_type, resource_id) signature.
- Panel: when a message hits a session the server no longer has, start a fresh
  session and retry once instead of dead-ending on a stale id.

Add a regression test that gives each request its own non-committing session —
the real cross-request boundary the shared-session integration tests never
crossed. It reproduces the production 404 without the route commit and passes
with it.

* refactor(prompter): drop the "GOLD" jargon for plain wording

"GOLD" was informal shorthand for "a good/well-formed spec" that should never
have been baked into the LLM prompts, comments, and docstrings as if it were a
defined term. Replace it everywhere with plain language ("a well-formed task",
"a complete task spec", "the markdown description", "structured spec fields").
No behaviour change.

* feat(intake): add the intake interviewer agent role (static definition)

Phase 1 of the intake-agent feature: a new first-class `prompter` role — the
intake interviewer the CEO chats with to draft a task. This commit defines the
role across every foundation layer (no runtime yet); spawning + the live
session come next.

- identity: Role.PROMPTER, RoleLevel.INTAKE (lowest authority), an AGENTS row
  (intake-1) on the board team, ROLE_LEVEL entry. Deliberately NOT in
  BOARD_ROLES — it interviews, it does not review.
- lifecycle: gets i_am_idle like every agent (its only verb); no
  delivery-lifecycle intents.
- journaling: ReadTier.OWN — isolated, reads only its own journal.
- role_config: human-only manifest — note + evidence only, no say/dm/notify/
  channels; allows_subagent=True (research), allows_write=False.
- agents_config derives it automatically and correctly excludes it from
  TASK_CREATOR_ROLES (it drafts, it never creates tasks).
- seed presentation ("Intake"); regenerated lifecycle artifacts.
- role system prompt: read the code first, single-CEO awareness, propose
  rather than interrogate — written against the failures we saw.
- docs: roster count 19 -> 20, org charts, verb-surface table, usage roster.

All foundation drift checks pass; role/manifest/permission tests green.

* feat(intake): migrate agentrole enum to add 'prompter'

ALTER TYPE agentrole ADD VALUE IF NOT EXISTS 'prompter' so the intake agent
row seeds/spawns against a migrated production DB. Forward-only (postgres
can't drop enum values), guarded for offline mode — matches migration 012.

* style(intake): ruff format the role additions

* fix(intake): unguard the agentrole migration so it renders offline

The enum-migration-parity test renders 'alembic upgrade head --sql' (offline)
and greps for ALTER TYPE ... ADD VALUE. The is_offline_mode() guard skipped
emitting it, so the parity check couldn't see 'prompter'. Drop the guard —
PG16 permits ADD VALUE in a transaction, same as migration 020's backfill.

* feat(intake): the live-session driver (Claude Agent SDK loop)

Phase 2 begins. The intake agent isn't a one-shot `claude -p`; it's a live
Claude Code session the human chats with. This driver is the container's loop:
open one long-lived claude-agent-sdk ClaudeSDKClient, then per human message
run a turn (query + receive_response) and stream its events out, keeping
conversation context in-process — verified against the real SDK (v0.2.94).

- StreamChunk + normalize(): map SDK messages (StreamEvent text deltas,
  AssistantMessage text/thinking/tool_use blocks, ResultMessage→session_id) to
  panel-facing chunks. Duck-typed, so it works on real SDK objects and on test
  fakes alike — SDK-free, fully unit-tested.
- IntakeDriver.run(): the loop, with injected session/source/sink seams; a turn
  failure surfaces as an error chunk without killing the session.
- SdkIntakeSession + build_intake_options: the only SDK-coupled code (lazy
  import; needs the live claude binary, so excluded from coverage).
- Add claude-agent-sdk dependency + mypy ignore-missing-stubs.

Relay, panel SSE, and the persistent on-demand spawn are the next steps.

* Updated uv.lock

* feat(intake): the panel<->agent live bridge (registry, routes, entrypoint, image)

Wires the live intake chat end to end (Phase 2 integration layer):

- prompter_live.py: the orchestrator-side per-session registry — open/close,
  push (agent->panel), stream (SSE drain), deliver (panel->container). In-process
  (the orchestrator is single-process). 7 unit tests.
- routes/prompter_live.py: GET /live/{id}/stream (SSE), POST /live/{id}/messages
  (deliver), POST /live/{id}/events (relay in); registered under /api/prompter.
  5 integration tests.
- agent_sdk/intake_main.py: the container entrypoint — a POST /turn receiver
  (the driver's MessageSource) + a relay-poster EventSink + the ClaudeSDKClient
  session, run concurrently. 4 unit tests on the wiring helpers.
- docker/agent-prompter.Dockerfile: FROM base, ENTRYPOINT = the driver (not the
  one-shot `claude` the other agents use).

Remaining for Phase 2: the orchestrator persistent-spawn path (scope->workspace
clone, CMD = driver, registry.open on spawn, reap-on-confirm) — the deploy-side
piece, best finalized against a buildable image.

* feat(intake): orchestrator persistent spawn + start/stop for the live chat

Add the task-free spawn path for the intake (prompter) agent: one fixed
intake-1 container running the Agent-SDK driver (image ENTRYPOINT, not
claude -p), one live session at a time.

- spawn_intake_session clones the scope's repo(s) via WorkspaceService
  (project -> one; product -> each distinct project, primary first),
  composes the intake-1 prompt, resolves the model, and builds docker run
  via _build_intake_run_cmd: no settings/hook mount (driver owns 9000),
  no MCP config, no -w; registers the live relay and best-effort delivers
  the opening message once the receiver is up.
- reap_intake_session closes the relay and stops the container.
- Routes: POST /live/start (project XOR product) and POST /live/{id}/stop.
- ROLE_MODEL_MAP[prompter]=opus; intake-1 -> roboco-agent-prompter image map.
- Replace the budget-sweep try/except/continue with _fetch_budget_status,
  which logs the swallow at debug instead of silently dropping it.

25 new tests; docker + the clone are mocked. End-to-end container spawn is
pending a built image and the stack.

* feat(intake): wire /prompter to the live agent — scope form + SSE chat

Replace the Ollama chat loop on /prompter with the spawned-agent flow.

- IntakeForm: pick scope (project XOR product) + opening message + Start
  before the chat; the agent clones that scope and reads the real code.
- use-prompter rewritten as the live brain (lib/api/prompter-live.ts): Start
  spawns via POST /live/start, then an EventSource on /live/{id}/stream
  streams the agent working — token deltas fill the assistant bubble,
  tool_use/thinking drive a live activity line, a draft event renders the
  existing DraftProposalCard. Messages go via POST /live/{id}/messages.
- Chat UX unchanged (Keep Chatting / Review & Confirm / ConfirmDialog reused);
  reap-on-confirm and reap-on-leave call POST /live/{id}/stop.
- Drop the dead Ollama prompterApi client; trim prompter.ts to shared types.

Frontend gate green (tsc --noEmit, lint, build). The draft event + the
/live/{id}/confirm endpoint are the Phase 4 backend seam.

* feat(intake): confirm draft -> backlog task + agent draft emission

Complete the live intake vertical: the agent proposes a structured draft and
Review & Confirm turns it into a task.

- Draft emission: the prompter prompt instructs the agent to emit a fenced
  roboco-draft JSON block when the spec is ready; the driver parses it into a
  'draft' event over the existing relay -> the panel's DraftProposalCard. The
  panel strips the raw block from the chat bubble.
- Fix a double-text bug: with include_partial_messages the reply arrives as
  both StreamEvent deltas and the final AssistantMessage; the driver now takes
  text from deltas only and the AssistantMessage for thinking/tool_use/draft.
- POST /live/{id}/confirm -> confirm_live_draft, reusing a draft->task core
  extracted from confirm_draft; reaps the session on success.
- Both prompter confirm paths create at BACKLOG, not pending: backlog is the
  holding area a draft waits in until it's reviewed and promoted to pending
  (TaskService.activate). The legacy Ollama confirm was creating at pending,
  skipping that gate — fixed.
- Remove the dead 'context' bootstrap param from the Ollama session-create
  chain (schema + route + method + tests), superseded by the live scope form.
- No suppressions: replace every type:ignore/noqa across the intake surface
  with a real fix (ORM .id -> UUID(str(x)); fakes -> monkeypatch.setattr;
  lazy imports -> pyproject per-file ignore; union-attr -> recipients[0]).

Full make quality green; frontend tsc + lint green.

* build(intake): add the agent-prompter image builder to compose

The orchestrator references roboco-agent-prompter (AGENT_IMAGES + the
_ensure_agent_image dockerfile map) and docker/agent-prompter.Dockerfile
exists, but docker-compose.yml built every other agent image up front and
left this one out — so the image wasn't pre-built for a stack bring-up.

Mirror the other specialized agent-*-image builders: build from
docker/agent-prompter.Dockerfile, tag roboco-agent-prompter, depend on
agent-base-image.

* Created docker-compose.yaml for the NAS

* fix(intake): non-blocking /live/start so spawn never times out

The start POST awaited the whole spawn — workspace clone + first-time image
build + docker run — which blew past the panel's 60s HTTP timeout ('Request
timed out. The server may be busy.') and triggered a duplicate send. Found on
the 2026-06-09 NAS smoke.

- start_intake_session opens the live relay synchronously, then spawns the
  container in the background (_spawn_intake_container_guarded). The route
  returns the session id immediately; the panel opens the SSE stream right away.
- A background spawn failure is pushed onto the relay as an 'error' event and
  closes the session, so the panel shows it instead of hanging.
- spawn_intake_session stays as the synchronous variant for direct callers/tests.
- Panel shows a 'Preparing the agent…' indicator until the first event arrives.

18 intake-spawn tests green; tsc + lint green. E2E re-validates on next smoke.

* fix(intake): propose_draft MCP tool + lock the agent down

Smoke 2026-06-09 exposed two compounding problems: the agent never reliably
emitted the draft (it narrated the spec instead of typing the magic fence), and
it had inherited the CEO's entire Claude Code env — Write/Edit/Bash + Gmail/
Notion/Calendar/Drive MCP — because bypassPermissions ignored the allowlist and
the mounted ~/.claude leaked the host MCP config.

- propose_draft: build_intake_options now registers an in-process SDK MCP tool
  (create_sdk_mcp_server + @tool). The agent calls it to submit the draft; the
  driver turns that ToolUseBlock into a 'draft' event (_is_propose_draft /
  _draft_from_tool_input, tolerant of nested/flat/JSON-string input). The fenced
  roboco-draft block stays as a fallback.
- Lockdown: strict_mcp_config=True + setting_sources=[] (ignore host MCP +
  settings); permission_mode 'dontAsk' + a can_use_tool gate enforcing a hard
  allowlist (Read/Grep/Glob/Task + propose_draft) replaces bypassPermissions.
- Prompt: call propose_draft (not a fence); the draft's downstream chain is
  backlog -> Board (PO + HoM) -> CEO approve -> Main PM, and the agent's job ends
  at the draft (it never routes or hands off).

SDK API verified against the installed claude-agent-sdk. Driver detection unit-
tested; the SDK-construction is validated on the next NAS smoke (incl. that
setting_sources=[] doesn't break the mounted-~/.claude auth).

* fix(intake): panel UX cluster from the smoke (#3/#4/#6/#12)

- #3 message boundaries: a tool call now ends the current text bubble, so the
  agent's words before and after a tool render as separate messages instead of
  one merged wall (the 'two waves merged into one bubble' the CEO saw).
- #4 activity indicator: promoted from tiny grey text to a prominent primary-
  tinted pill so 'watch it work' is actually visible.
- #12 End chat: a header button (any chat state) reaps the agent and resets to
  the form, reusing startAnother (which already stops the session). Backend
  POST /live/{id}/stop already existed.
- #6 log noise: the opening-message delivery retry logs at debug, not error —
  those failures are expected until the container receiver is up.
- Also fix a latent test gap from the #1 commit: the live-route test's fake
  orchestrator now exposes start_intake_session (the route's non-blocking entry).

Frontend tsc + lint green; live-route + prompter_live tests green.

* fix(intake): render markdown in the chat bubbles (#8)

The agent emits rich markdown (### headers, **bold**, tables, lists) but the
bubble rendered raw text, so it was illegible (CEO-flagged on the smoke). Render
assistant content with react-markdown + remark-gfm (GFM tables) in a prose
container. Adds react-markdown + remark-gfm to the panel.

* feat(intake): #14 — two start routes (Board review vs straight to Main PM)

Per the CEO spec, the draft confirm now starts the task at PENDING with an
explicit assignment instead of parking it at backlog:

- route="board" (Board review & Start): assigned to the Product Owner, so the
  orchestrator dispatches the full Board review (PO + Head of Marketing) before
  the Main PM picks it up.
- route="main_pm" (Approve & Start): assigned straight to the Main PM, who
  delegates to the cells (Board review skipped).

create_task_from_draft gains status + assigned_to params (default BACKLOG, so the
legacy confirm_draft is unchanged); confirm_live_draft + the /live/{id}/confirm
request carry the route. Service tests cover both routes.

* feat(intake): #14 draft-card buttons — Board review vs Approve & Start

Three buttons on the draft card now (CEO spec): Keep chatting / Board review &
Start / Approve & Start. The two action buttons confirm directly with their
route — launchTask(route) sends route to POST /live/{id}/confirm, which starts
the task at pending assigned to the Board (PO+HoM) or straight to the Main PM.

Supersedes the ConfirmDialog review step (scope is chosen up front in the form),
so it's removed from the page flow. The ConfirmDialog component + its sub-editors
are now unused — flagged for a follow-up cleanup, left in place to avoid churn.

tsc + lint green.

* fix(intake): keep the live SSE stream bound to its relay session

The orchestrator opened the relay session twice per live chat — once on the
request path (before the start call returns) and again inside the background
container spawn. The SSE stream binds to the session's queue the moment the
panel connects, so the second open swapped in a fresh queue and stranded the
stream: the agent replied normally, but its events went to the new queue while
the panel kept reading the old one, so the chat looked frozen on "Preparing…".

The second open was always redundant (the relay is opened by the caller before
the spawn). Remove it, and make open() idempotent so a live session is never
replaced out from under a stream that is already connected to it.

* fix(intake): draft-card launch buttons silently did nothing

The launch path required a `description` field, but the prompter draft schema
intentionally has none — it sends `objective` + the structured spec and the
backend composes the description (compose_description). `editableDraft.description`
was therefore undefined, so `description.trim()` inside launch validation threw a
TypeError that propagated out of the button's onClick. Clicking "Board review &
Start" / "Approve & Start" did nothing, with no feedback — the wall blocking the
whole confirm → task → reap flow.

- Map a proposed draft's description from `objective` as a fallback.
- Make launch validation null-safe.
- Replace the silent early-return with a toast that names what's missing, so a
  blocked launch is never a dead, feedback-less button again.

* fix(intake): steer the agent to ask inline, not via AskUserQuestion

The intake's job is to ask clarifying questions, so it reached for the
AskUserQuestion tool — which isn't wired to the live chat panel and isn't in its
allowlist. The bare deny left it to stumble ("let me clarify… — no worries, let
me just lay it out") and waste a visible turn.

- Prompt: spell out that it asks by writing in the chat (the human reads every
  message live) and that no question/prompt tool is available to it.
- Gate: give AskUserQuestion a specific deny message that nudges it to ask inline,
  so even a reflex attempt degrades gracefully.

Also refresh the now-stale "what happens after propose_draft" section: the draft
card has three choices (Keep chatting / Board review & Start / Approve & Start)
and produces a pending task — not the old two-button "backlog" description.

* feat(intake): copy buttons on agent messages and the draft card

The CEO asked for a way to save the agent's plan/spec elsewhere "just in case" —
a cheap manual backstop until refresh-durability lands.

- New CopyButton: async Clipboard API when available, plus a legacy
  textarea+execCommand fallback. The fallback is load-bearing — the panel is
  served over plain http on a LAN IP, where navigator.clipboard is absent
  (clipboard needs a secure context), so the modern API alone would never copy.
- Copy button under each assistant message (copies its text).
- Copy button on the draft card (copies the full spec as markdown: title,
  objective, what-this-builds, the-work per cell, notes, success criteria).

* feat(intake): unbuffer logs + log each turn so the container isn't a black box

Debugging the intake smoke was painful for two reasons: (a) the orchestrator
block-buffered stdout, so `docker logs` lagged minutes behind reality, and (b)
the intake container logged only "session opened" then went silent for the whole
conversation (the chat streams to the relay, not stdout).

- Set PYTHONUNBUFFERED=1 on the orchestrator and agent-base images so structured
  logs reach `docker logs` in real time instead of in large delayed chunks.
- Log each intake turn: "turn received" (with char count) and "turn streamed"
  (chunk count + whether a draft was emitted), so the container logs show the
  conversation's shape at a glance.

* chore(intake): remove the dead ConfirmDialog draft editor

The three-button draft card (Keep chatting / Board review & Start / Approve &
Start) replaced the old review-modal confirm flow, leaving ConfirmDialog and its
sub-editors (StringListEditor, TheWorkEditor) referenced by nothing but the
barrel export. Remove the three files and the export — typecheck + lint confirm
no remaining references.

* fix(intake): coerce bad draft enums on confirm instead of hard-failing

The intake agent is an LLM and will emit off-enum values — e.g. task_type="feature",
which is not a valid TaskType (code/documentation/research/planning/design/
administrative). `_coerce_draft_enums` called `TaskType(value)` directly, which
raised, and the confirm 400'd with "Draft has invalid or missing required fields:
'feature' is not a valid TaskType". That forced the agent to discover the valid
values and self-correct in-chat — unacceptable: clicking "Approve & Start" must
never blow up on a cosmetic enum guess.

Coerce each enum to a sane default on invalid/missing (task_type→code,
nature→technical, complexity→medium); team falls back to the first valid cell in
the_work, then backend. `_lead_cell_team` now skips invalid cell names too. The
confirm/launch action no longer hard-fails on an enum the model got wrong.

* fix(intake): draft card no longer renders above the user's latest message

attachDraft fell back to "the last assistant message anywhere" when the current
turn had no streamed text yet (propose_draft called first). That last message was
often the PREVIOUS turn's — sitting above the user's "Yes, propose it" — so the
draft card rendered above the user's message. Attach only to the current turn's
streaming message; otherwise append a fresh assistant message so the card always
lands at the bottom of the thread.

* test(intake): guard draft enum coercion + invalid-cell skipping

Regression tests for the confirm-time enum coercion: an off-enum task_type
("feature") / nature / complexity coerce to code/technical/medium instead of
raising, and _lead_cell_team skips invalid cell names. Locks in that a bad enum
guess from the agent can never 400 the launch again.

* fix(intake): stop the agent fumbling through Claude Code meta-tools

In smoke it reflexively probed CC built-ins before reaching propose_draft —
plan mode + ExitPlanMode (it announced a written plan and waited instead of
emitting the draft), ToolSearch, Write — each correctly denied by the lockdown
but stumbly, and it only proposed after explicit CEO nudges.

- Gate: ExitPlanMode now gets a specific deny nudge ("you don't use plan mode;
  call propose_draft"), and the generic deny names the actual toolset instead
  of a bare "not available", so any probe degrades into guidance.
- Prompt: forbid plan mode/ExitPlanMode/ToolSearch explicitly and spell out
  "you do not plan and wait — call propose_draft directly when the spec is
  ready," plus an anti-pattern bullet.

* feat(intake): make the container logs transparent mid-turn

`docker logs` on the intake container was a black box: only turn start/end, while
the agent read the codebase and spawned 20+ subagents invisibly (the conversation
streams to the relay, not stdout), and the benign 3x ~/.claude.json warning was
the only thing visible.

- Driver logs each tool call mid-turn ("Intake tool use" with the tool name) and
  the draft emission, plus a tools count in the turn-streamed summary. Text deltas
  stay unlogged (they'd spam). Now the logs show the turn's real shape.
- Pre-create ~/.claude.json ({}) at container boot so the CLI's "config not found"
  warning (printed 3x, self-healed anyway) stops drowning the real logs.

* fix(intake): render markdown in user messages + scope copy to code blocks

Two display fixes from the smoke:
- User messages collapsed newlines (plain {content} in a div) and rendered no
  markdown — a "1.\n2.\n3." answer showed as one run-on line. Render user AND
  assistant bubbles through a shared GFM markdown body that inherits the bubble's
  text color, so lists / newlines / styling render correctly on both.
- Copy was blanketed on every assistant message; scope it to KEY parts — a copy
  button on fenced code blocks (the draft card keeps its own). Removed the
  per-message button.

* fix(intake): prevent duplicate tasks from a double-click on launch

Clicking a draft launch button twice fired two confirms and created duplicate
tasks. Add a synchronous re-entry guard (a ref — no stale-closure window) at the
top of launchTask so a second click returns immediately, and disable + spin the
draft-card buttons while a launch is in flight so it's visually clear it's working.

* docs(how-to): lead task creation with the Task Assistant flow

Rewrite "1 · It starts with you" to walk the Prompter/Task Assistant path —
scope form, the agent reading the codebase, its grounded analysis, the draft
card, and the created task — then flow into the Board review. Replaces the old
manual task-definition form shots.

Image placeholder: images/prompter_draft_card.png (the 3-button card) is
referenced but not yet captured — TODO comment marks it for the next smoke run.
A second comment flags an optional re-capture of prompter_run_2 after the
markdown-rendering fix.

* fix(intake): restore assistant message text contrast

The markdown refactor dropped `dark:prose-invert` and made text inherit the
bubble's color, but the assistant bubble had no explicit text color — so its text
rendered near-invisible (dark-on-dark on bg-muted). Give the assistant bubble an
explicit text-foreground; the user bubble already carries text-primary-foreground,
and [&_*]:!text-inherit now resolves to a readable color on both.

* fix(intake): coerce draft priority too — confirm 500'd on priority="high"

The enum-coercion fix covered task_type/nature/complexity/team, but priority is a
non-enum int field handled by `int(draft_data.get("priority", 2))`, and the agent
guesses a word ("high") as readily as a number — so int("high") raised ValueError
and the confirm 500'd. Same class of bug, one field missed.

Add _coerce_priority: map words (urgent/high/medium/low → 0/1/2/3), clamp numbers
to 0-3, default to 2 (medium) on anything else. The launch can no longer crash on
any field the LLM guessed. + regression test.

* fix(intake): draft card shows distinct cells, not one badge per work item

the_work has one entry per work item, so a cell with several items rendered its
badge repeatedly ("Board-led across Backend Backend Backend Frontend Frontend
…"). De-dupe to distinct teams so the card reads "Board-led across Backend
Frontend" — and the "Cell:" vs "Board-led across" label keys off distinct count.

* docs(how-to): hero the teaser gif + resolve the Prompter/Task Assistant thread

- Move the 12s teaser gif to the top as the hero — it was buried between the
  "prefer video" link and the first screenshot.
- Name the connection: the Task Assistant IS the Prompter, so section 1 (using
  the tool) and the rest (RoboCo building it) read as one story — you use the
  tool the company built for itself, then watch the build.
- Re-anchor the section 1 → Board transition to follow the Prompter's own
  journey, instead of implying section 1's example task is the one reviewed next.

* Included images for how-to.md

* docs(how-to): align agent count to 20 (matches README + CLAUDE.md)

The how-to said "18 agents" with UX/UI at one dev and no Intake — stale against
the authoritative count. Bump 18→20 (prose + spelled-out eighteen→twenty), give
UX/UI 2 devs, and add the Intake line to the org tree (Intake leads section 1, so
it belongs in the tree). README + CLAUDE.md already say 20.

* ci(release): publish all RoboCo images to GHCR + Docker Hub

The release published only the orchestrator to GHCR. Build and push the full set
the stack needs — agent-base, the 8 agent images, orchestrator, and panel — to
BOTH ghcr.io/rennf93/* and docker.io/renzof93/*, at :<version> and :latest, so
consumers can pull instead of compose-building.

- agent-base builds first (the agent images build FROM roboco-agent-base, a local
  tag), then the rest; push only after every build succeeds.
- Image names mirror the docker-compose `image:` values 1:1.
- Free disk on the runner first (11 images is space-heavy).
- Needs a DOCKERHUB_TOKEN repo secret for the Docker Hub login.
- SECURITY.md updated to reference both registries.

* ci(release): use short SHA as the image tag on manual dispatch

A workflow_dispatch runs against a branch, and the branch name (e.g.
feature/prompter-gold-upgrade) was used verbatim as the image tag — but "/" is
illegal in a Docker tag, so the first build failed instantly with "invalid
reference format". Releases still tag from the release tag; manual dispatch now
always uses the short SHA, which is a valid tag.

---------

Co-authored-by: Renn F <rennf93@users.noreply.github.com>
2026-06-09 17:08:34 +02:00