568 Commits
Author SHA1 Message Date
889f3689e7 MegaTask (#248)
* feat(batch): batch_id + collision descriptor columns

Sequenced batch intake ("Mega task") foundation: tasks.batch_id (indexed)
groups a batch of top-level tasks created together; intends_to_touch (text[]),
adds_migration and touches_shared (bool, NOT NULL default false) are the
per-task collision surface the SequencingService will read to wire dependency
waves. Mirrored on the Task model + TaskCreateRequest and wired through
TaskService.create. Migration 046 (real upgrade->downgrade->upgrade verified
vs a throwaway pgvector PG); a non-batch task declares no surface (defaults).

Task 1 of the 0.11.0 sequenced-batch-intake plan.

* feat(batch): flag + draft collision descriptors

Default-off ROBOCO_BATCH_INTAKE_ENABLED (config + FEATURE_FLAGS + panel card);
the propose_draft tool doc + the TS DraftProposal gain the per-task collision
surface intends_to_touch / adds_migration / touches_shared. The draft is a loose
dict so the descriptors ride it through the relay intact (test asserts the
forwarded payload); the analyzer (Task 3) reads them to wire dependency waves.

Task 2 of the 0.11.0 sequenced-batch-intake plan.

* feat(batch): deterministic collision-sequencing analyzer

SequencingService.analyze turns a batch's per-task collision surfaces into a
dependency DAG + execution waves — correctness in CODE, not agent judgment.
Rules in order: file overlap serializes (more-important first), migrations form
a serial chain (no concurrent Alembic heads), touches_shared runs last, cell
contention warns (never serializes); then dedupe, existence + cycle check, and
Kahn topological layering. Pure (no DB/services); SequencingError on a cycle or
out-of-range edge.

Golden test reproduces the CEO's hand-sequenced 4 waves of the 11-item
guard-core-app batch (the effort that deadlocked the Main PM): S6 alone last,
the R1/R3/R4 migration chain, R2/R3/S8 serialized on the shared threat service,
S1/S2/S7 in one parallel wave.

Task 3 of the 0.11.0 sequenced-batch-intake plan.

* chore(batch): brand the user-facing surfaces "MegaTask"

The user-facing name is MegaTask: the feature-flag label is "MegaTask intake",
the panel flag-card and the config description lead with MegaTask. Internal
names stay technical (batch_intake_enabled, batch_id, SequencingService).

* chore(batch): drop the feature flag — MegaTask is a core intake scope

MegaTask is additive and opt-in by its own nature (the Prompter proposes a
batch only when the CEO asks for several tasks; single-task intake is
unchanged), so there is no risk surface a flag protects — 'don't create a
MegaTask' is the off switch. Remove batch_intake_enabled from config, the
FEATURE_FLAGS registry, the panel flag card, and its tests. MegaTask will be
a third scope option in the Intake modal (single-cell / multi-project /
MegaTask), not a toggle.

* feat(batch): MegaTask identity predicate + orchestrator branchless recognition

The single source of truth for the umbrella's exemptions: pure
is_batch_umbrella / is_batch_root_subtask / is_branchless_coordination
(foundation/policy/batch.py) — an umbrella has a batch_id and is top-level; a
root-subtask shares the batch_id but is parented. The orchestrator's
_is_coordination_task now consults is_branchless_coordination, so a MegaTask
umbrella is recognized as doing no git of its own (git-exempt at spawn-readiness
/ stuck-detection) exactly like a product fan-out root. Non-batch behavior is
identical (the predicate reduces to the old no-project+product check; the
orchestrator coordination suite stays green), and the umbrella branch is inert
until the create path exists.

First slice of the MegaTask umbrella enforcement (branchless guard).

* feat(batch): branchless umbrella guard across the git-exemption sites

A MegaTask umbrella does no git of its own — every git-exemption site in
TaskService now consults the shared is_branchless_coordination predicate
instead of an inline product-only check, so the umbrella's exemptions
cannot drift between sites:

- the claimed->in_progress branch gate (GitContext.is_coordination) lets
  an unbranched umbrella reach in_progress and delegate;
- _ensure_branch_for_task short-circuits an umbrella to "" instead of the
  misconfigured raise (the claim path ignores the return, treating it as
  branchless);
- CEO-reject routing sends a rejected umbrella to the Main PM in PENDING
  (needs_revision is developer-claim-only and would deadlock it).

Covers both shapes via the predicate (product fan-out root OR umbrella);
a batch root-subtask keeps its own branch/PR. Adds orchestrator
recognition tests for the umbrella plus claim/branch/reject integration
tests.

* feat(batch): umbrella assembles no PR; completes branchless

submit_root now hard-rejects a MegaTask umbrella up front (a preflight
that also folds in the unknown-role refusal to stay within the
return-count budget): the umbrella spans many projects with no single
master, so each root-subtask opens and is reviewed on its own PR — the
umbrella never enters the in-path review gate. The Main PM completes it
directly once every root-subtask is terminal.

Umbrella completion needs no new code: it is branchless (no branch_name),
so _main_pm_complete_guard already accepts it from in_progress, checks
all_subtasks_terminal, and main_pm_complete walks it to awaiting_pm_review
and escalates to the CEO with no PR creation — exactly the product
fan-out root path. Adds the submit_root-reject and umbrella-completion
gateway tests; pins batch_id=None on the normal-root submit_root test
(a MagicMock auto-attr would otherwise read as an umbrella).

* feat(batch): MegaTask create path — umbrella + sequenced root-subtasks

PrompterService.confirm_live_batch turns N confirmed drafts into a real
MegaTask: it builds each draft's collision surface, runs the pure
SequencingService to get conflict-free waves, creates the branchless
umbrella (batch_id, no project/product), then one root-subtask per draft
(own project, parent=umbrella, sequence=wave index, descriptors), and
wires the analyzer's edges through add_dependency so the existing
dependency-gate runs the waves in order. The route picks the start path
like a single confirm: 'board' holds the root-subtasks in BACKLOG for the
batch review; 'main_pm' creates them PENDING so wave 0 dispatches at once.

create_task_from_draft gains a BatchPlacement (parent/batch/sequence/
team_override) and forwards the collision descriptors; the exactly-one-
target rule (here and the TaskService.create invariant) is relaxed for an
umbrella, which legitimately targets neither. New route
POST /live/{session}/confirm-batch + BatchConfirmRequest mirror the single
confirm. Adds the structural-invariant + board-hold + empty-batch tests.

* feat(batch): release MegaTask root-subtasks on CEO approval; board awareness

The board route holds a MegaTask's root-subtasks in BACKLOG so the work
waits for the batch review. approve_and_start (CEO gate #1, board->Main PM)
now releases them via _activate_batch_root_subtasks: each held child flips
BACKLOG -> PENDING + team=main_pm so the dependency-gate dispatches wave 0.
No-op for a non-umbrella; idempotent (children past BACKLOG untouched).

The Product Owner and Head of Marketing identity prompts gain a MegaTask
section so they review the whole batch + wave plan and adjust scope before
sign-off (they review drafts; the umbrella is their unit). Also extracts
the create() target invariant into _require_target_or_umbrella to keep the
method under the complexity gate after the umbrella exemption. Adds the
umbrella-approval activation test.

* feat(batch): multi-project intake scope for MegaTask

A MegaTask spans several possibly-unrelated repos, so the intake chat can
now be scoped to an explicit project list (not just one project or one
product). StartLiveRequest gains project_ids; /live/start threads it
through start/spawn_intake_session -> _spawn_intake_container ->
_clone_intake_scope. The multi-repo clone machinery already existed for
products; _intake_scope_slugs now also resolves an explicit project_ids
set (split into _slugs_for_project_ids / _slugs_for_product), cloning each
repo with the first as the primary cwd and the siblings readable. Scope
validation is now 'exactly one of project_slug / product_id / project_ids'
via the shared _require_one_intake_scope. Adds scope-resolution, spawn,
and route tests for the MegaTask path.

* feat(batch): propose_batch intake tool (MegaTask multi-draft hand-off)

The intake agent can now hand the panel a whole MegaTask in one tool call.
Both intake paths gain propose_batch alongside propose_draft:
- Claude (intake_driver): a propose_batch tool registered on the in-SDK
  MCP server + allowlisted; the driver intercepts the ToolUseBlock and
  emits ONE StreamChunk(kind="batch") carrying {drafts:[...], title}.
- grok (intake_server): a propose_batch tool that POSTs a "batch" relay
  event via the shared _post_event helper (post_draft/post_batch).

A batch carries N drafts, each the propose_draft shape PLUS its own
project_id (a MegaTask spans unrelated repos) and collision surface so the
analyzer sequences the waves. The prompter prompt documents the MegaTask
scope + when to call propose_batch. Adds Claude-normalize and grok-relay
tests for the batch path.

* feat(batch): MegaTask intake panel — third scope, batch review, waves

The panel now drives a MegaTask end to end. The intake modal gains a
third scope, 'MegaTask', beside Single cell and Board-led: a multi-project
checklist (a MegaTask spans several possibly-unrelated repos), validated
to at least two. start() sends project_ids; use-prompter accumulates the
agent's single propose_batch hand-off as a 'batch' SSE event into a
BatchProposal and lands in a new batch_preview state.

A new BatchReviewCard lists every proposed task with its target project +
collision-surface badges (migration / shared) and offers one start path
for the whole batch — Board review & Start or Approve & Start — wired to
confirmBatch → POST /confirm-batch. The success card shows the sequenced
result: N tasks in M waves (+ any advisory notes). prompter.ts gains the
DraftScale 'megatask' + the BatchConfirm payload/result types; the SSE
client allows the 'batch' kind. Panel typecheck + lint + 113 tests green.

* docs(batch): MegaTask across changelog, CLAUDE.md, site, and RAG

The four documentation obligations for the MegaTask feature:
- CHANGELOG: an Unreleased entry covering the umbrella model, sequencing,
  multi-project intake, propose_batch, and the create/approval path.
- CLAUDE.md: a MegaTask section (identity predicate, umbrella/root-subtask
  hierarchy, sequencing rules, intake + create path, board activation).
- Published site: a user-facing company/megatask.md (scopes, waves, the
  umbrella, the two start buttons) + nav entry; a pointer added to the
  intake chapter of the Tour.
- RAG corpus: workflows/megatask.md so the Main PM (and any agent) can
  retrieve the umbrella's branchless / no-PR / completion rules at runtime.

The runtime concurrent-migration guard is intentionally NOT added: the
analyzer already chains migration-adders into dependencies and the
dependency-gate serializes them, so a separate guard would be dead code.

* feat(batch): batch_id guardrail + wave preview + batch_id on TaskResponse

Guardrail (CEO): a batch_id is denied on any task that is not a well-formed
MegaTask member. is_valid_batch_shape permits batch_id only on an umbrella
(no parent → must target neither project nor product) or a root-subtask
(has a parent → exactly one target); TaskService.create enforces it AND
verifies a root-subtask's parent is the batch umbrella (same batch_id,
top-level). This closes a latent hole: is_batch_umbrella is true for a
batch_id + no-parent task even with a project, so a stray batch_id could
have spoofed the branchless branch-gate / no-PR exemption. (The public
task API never exposed batch_id for write; this guards the service layer.)

Wave preview: PrompterService.preview_batch + POST .../preview-batch
compute a MegaTask's waves from the proposed drafts WITHOUT creating
anything, so the panel can show the sequencing before confirm. Extracted
_sequence_drafts as the single source shared by preview and confirm, so
the previewed waves are exactly the ones wired.

TaskResponse now carries batch_id so the panel can badge the umbrella.

* feat(batch): MegaTask review — project editor, wave preview, persistence, badge

Closes the panel gaps in the MegaTask review experience:
- Per-task project editor: each proposed task gets an inline project
  Select (updateBatchDraftProject), so a task the agent put in the wrong
  or no repo can be fixed before launch — not only by re-chatting. Launch
  stays blocked until every task has a project.
- Wave preview: on a batch proposal the panel fetches POST .../preview-batch
  (no task created) and shows the conflict-free wave plan, so the human
  reviews the sequencing before confirming.
- Refresh durability: the MegaTask review (batch + waves + projectIds) is
  persisted, so a browser reload mid-review restores it like a single draft.
- MegaTask badge: TaskResponse exposes batch_id, the panel Task type
  carries it, and the task table badges the umbrella row 'MegaTask'.

Panel typecheck + lint + 113 tests green.

* test(batch): stub task carries batch_id for task_to_response

task_to_response now serializes batch_id (TaskResponse field), so the
_stub_task SimpleNamespace fixture must provide it — without it the reader
hit AttributeError, failing the 8 task-schema serialization/enrichment
tests. Test-only; the real TaskTable carries the column (migration 046).

* fix(batch): close MegaTask audit gaps — completion crash, analyzer cycle, guardrails

An adversarial multi-agent audit of the feature surfaced 20 verified gaps;
this closes the backend ones.

HIGH:
- Umbrella completion crashed. escalate_to_ceo hard-required a pr_number,
  which a branchless umbrella never has, so main_pm_complete dereferenced
  None. Both pr_number gates now waive a MegaTask umbrella (escalate_to_ceo
  + the awaiting_pm_review->awaiting_ceo_approval lifecycle gate via a new
  GitContext.is_umbrella), and main_pm_complete guards a None return. The
  completion test had mocked escalate_to_ceo, hiding it — now a real
  service test covers the waiver.
- The collision analyzer could fabricate a cycle (a touches_shared +
  adds_migration draft overlapping another migration draft) and raise
  SequencingError — a bare ValueError that escaped as an opaque 500. The
  migration chain is now shared-last-aware (never contradicts rule 3), and
  _sequence_drafts translates SequencingError to a clean 400.

MEDIUM:
- Collisions are now project-scoped: two repos can't collide on a
  coincidental path or serialize independent migrations (DraftSurface
  carries project_id; rules 1/2/3 respect it).
- The batch_id guardrail ran only at create. update() + the PATCH
  null-clear path now re-assert is_valid_batch_shape, so a mutation can't
  break a member's shape and spoof the branchless exemption.
- A draft missing title/acceptance_criteria now raises ValidationError
  (was a bare KeyError -> 500).
- confirm_live_batch re-asserts every draft targets a scoped project and
  the batch spans >=2 distinct projects (project_ids added to the request).
- Route-level tests for confirm-batch / preview-batch.

LOW: strict multi-repo clone (fail loud on any unresolvable project);
malformed/empty propose_batch surfaces an error chunk (Claude) / refuses
to POST (grok) instead of silently acking; dropped malformed drafts are
counted and surfaced; stale grok intake docstrings updated.

* fix(batch): MegaTask panel + doc audit gaps

Frontend half of the audit fixes:
- The confirm payload now carries project_ids (the schema requires it), and
  the panel re-checks every task targets one of the scoped repos before
  launching, naming the offending task.
- The Review-MegaTask project picker is filtered to the scoped repos and
  the per-task validity (border + launch gate) keys off scoped membership,
  so a task can only be (re)pointed at an in-scope project — also fixing the
  case where the agent emitted a non-UUID / unknown project.
- Dropped malformed drafts are surfaced as a chat error so the human knows
  the batch shrank instead of silently confirming fewer tasks.
- Doc wording: a wave releases on the previous wave's terminal state
  (normally a merge; a cancellation releases it too), not strictly 'merged'.

* test(batch): lock the CEO's EXACT 4-wave hand-sequencing as the golden bar

The golden test asserted the constraints (S6 last, the migration chain, the
shared-threats serialization, S1/S2/S7 parallel) but not the full wave
partition. The bar for MegaTask is 'reproduce my exact waves or it's not
done', so assert the exact 4-wave partition the analyzer produces for the
guard-core-app batch:
  wave 1: R1 R2 S1 S2 S3 S5 S7  ·  wave 2: R3  ·  wave 3: R4 S8  ·  wave 4: S6
Confirmed unchanged by the audit's analyzer fixes (no migration is shared;
single project).

* fix(batch): tolerate a stub task in assert_batch_shape_intact

The batch-shape re-validation read task.batch_id directly, but update()'s
partial-caller contract is exercised with a SimpleNamespace stub that has no
batch_id column → AttributeError. Use getattr(..., None) for batch_id and the
shape fields so the guard no-ops on any task lacking the column (a stub, or a
non-batch task) while still enforcing on a real batch member.

* fix(orchestrator): authenticate internal API self-calls with the system identity

The dispatcher httpx clients were built without an agent identity, so the
orchestrator's self-PATCHes to /api/tasks/{id} (auto-block, auto-resume,
auto-recover, SLA annotation) were rejected 401 "Missing X-Agent-ID" and
silently no-op'd. The auto-resume that lifts a PM's paused parent could never
write, so paused/blocked parents stayed wedged and stranded their dependents
(the fe-pm/be-pm respawn churn seen in prod).

Header propagation was inconsistent across the separate AsyncClient call-sites:
only the main dispatch client carried the system identity; the readiness and
sweep clients did not. Hoist the identity into a shared _SYSTEM_API_HEADERS
constant and apply it to every API-facing dispatcher client. The system role
holds TaskAction.ASSIGN, so it is authorized for the audited admin_set_status
path those write routes use. The external provider-recovery probe client is
intentionally left untouched.

---------

Co-authored-by: Renn F <rennf93@users.noreply.github.com>
2026-06-24 01:15:57 +02:00
c09cf80b40 Feature/observability gateway health (#247)
* feat(observability): revision_count + audit_log query index (migration 045)

Adds tasks.revision_count (the O(1) rework counter — forward-only, existing
rows default 0) and the composite index audit_log(target_id, event_type,
timestamp) that powers the cycle-time and rework reconstruction queries.
Verified the real upgrade/downgrade/upgrade chain on a throwaway pgvector PG.
First task of the 0.10.0 observability dashboards.

* feat(observability): count reworks + attribute qa_fail/pr_fail to the rejector

Every transition into needs_revision increments tasks.revision_count at the
single audit chokepoint (exactly once per bounce, across all paths incl. pr_fail
and ceo_reject), so the rework rate is an O(1) read. A QA or PR-review bounce
also emits a named task.qa_fail / task.pr_fail audit event carrying the
rejector's agent_id, so the per-agent rework scorecard charges the rejection to
the reviewer who made it, not the developer who owns the task.

* feat(observability): cycle-time, bottleneck, rework, and scorecard metrics

MetricsService gains four read methods on the audit_log + tasks data: per-stage
cycle time reconstructed from the transition journey (excluding the named
qa_fail/pr_fail events), bottleneck distribution (cumulative dwell + live parked
counts), rework rate (overall/by-team/by-agent with rejector attribution + cost
via spawn-session task_id), and a fused per-agent/per-cell scorecard. Dataclass
models with to_dict(). Verified against a real Postgres journey.

* feat(observability): cycle-time/bottleneck/rework/scorecard read endpoints

Thin read-only routes on the dashboard router delegating to MetricsService:
/metrics/cycle-time, /metrics/bottlenecks, /metrics/rework, and
/metrics/scorecard/{agent,team}. 404 when an agent scorecard target is absent.
5 route tests (200 + shape + the agent-404 case).

* feat(panel): Delivery observability tab (cycle-time, bottlenecks, rework, scorecards)

A third Metrics tab built on the observability endpoints: a per-stage
cycle-time bar chart, a bottleneck panel (worst stage + cumulative dwell +
live parked counts), a rework panel (rate + by-team + by-agent attribution +
cost), and per-cell scorecards. Reuses Recharts + Card/Badge/Skeleton and the
React-Query hook pattern; observabilityApi mirrors usageApi with mock-mode
fallbacks. tsc + eslint clean; 113 panel tests pass.

* docs(observability): changelog + CLAUDE.md for the delivery dashboards

* feat(gateway-health): recover a broken-but-alive agent instead of protecting it

The verb-heartbeat cannot tell a quiet-healthy agent from one whose MCP gateway
is broken (a corrupted /app/.venv firing no verb) yet whose container is up — the
reaper's live-skip would shield it forever. The reaper now probes the gateway
out-of-band (docker exec: does the gateway venv import its deps?) and, once it
has been broken past gateway_health_grace_seconds (tolerating a transient probe
miss), kills + evicts the container so it falls through to release + respawn.
Probe-inconclusive or healthy spares the container. Gated by
gateway_health_enabled (default-on reliability fix; in the panel Feature Flags).
Defers the optional agent-side self-check + full registry re-adoption — the
reaper's docker-liveness fallback already recovers a broken-after-restart agent.

* docs(gateway-health): changelog + CLAUDE.md for broken-but-alive recovery

* docs(observability): user-facing docs for the Delivery dashboards + gateway-health

Documents the new Metrics -> Delivery tab (cycle-time, bottlenecks, rework with
rejector attribution, cell scorecards) in the panel guide and the operations
health-and-metrics guide, and adds the gateway-health env vars + an agent-gateway
recovery note. Published MkDocs site only; settings.md's default-off flag table
intentionally omits the default-on gateway-health flag (same as overload-break).

* chore(release): cut 0.10.0 (changelog section + version refs)

* fix(gateway): exempt PM coordinators from single-task claim guards

A Main/Cell PM plans and delegates many root tasks in parallel; the work
then runs in the delegated cells, not in the PM's own hands. But the
claim-time concurrency guards meant for developers — already_active and
paused (the latter firing after i_am_idle auto-pauses the PM's own
umbrella) — were applied to the PM too, so once it held one root it could
never plan a second: it thrashed between its claimed roots and respawned
forever, burning tokens for zero progress.

_run_claim_guards now skips already_active/paused for the coordinator PM
roles (_COORDINATOR_ROLES = {main_pm, cell_pm}); only unmet_dependency — a
real upstream sequence constraint, which parks the root back to pending —
still gates a PM. paused_tasks_guard also excludes the target task itself,
so a PM re-entering its own paused umbrella never self-blocks.

Tests: a coordinator plans a second root with one in_progress + one paused
sibling (full path + claimed-recovery path), the paused target exclusion,
and the developer guards still fire. Repurposed the pre-fix test that
asserted the now-removed PM block.

* fix(metrics): coerce SQL avg/extract hours aggregates to float (panel toFixed crash)

EXTRACT(epoch ...) returns numeric on PostgreSQL 14+, which asyncpg surfaces
as a Decimal; a Decimal serializes to a quoted JSON string, so the panel's
avg_cycle_hours.toFixed(1) (and the other hours fields) threw 'toFixed is not
a function' and blanked the Delivery tab.

A single _as_hours helper now rounds every SQL-averaged hours field to a real
float — avg_cycle_hours on the new scorecards plus the pre-existing
avg_completion_hours / avg_blocked_hours / longest_blocked_hours. Token and
cost fields were already float()-cast and are unaffected.

Regression test asserts _as_hours coerces Decimal -> float and preserves the
None/zero behavior.

* feat(panel): edit a task's sequence from the details page

A task's sequence (order within siblings, lower runs first) was display-only
with no way to change it from the UI, and TaskUpdate didn't carry the field
so PATCH couldn't set it either. The details page's Dependencies tab now has
an inline sequence editor mirroring the parent / dependency editors, and
PATCH /tasks/{id} accepts a sequence field (owner or privileged role) through
the existing generic update path.

* fix(mypy): green the full make-quality type gate

make quality runs 'mypy roboco/ tests/', which the per-module checks on the
0.10.0 branch never exercised. Two issues surfaced:

- The coordinator-exemption change added role_str to
  Choreographer._run_claim_guards but not to the ChoreographerHelpers
  protocol base, so the composed Choreographer had incompatible base-class
  signatures. Sync the protocol signature.

- The gateway-health / stale-reaper tests stubbed methods by direct
  assignment (orch._m = AsyncMock()) and typed their duck-typed task doubles
  as object, tripping method-assign / assignment / attr-defined. Switch to
  monkeypatch.setattr (keeping a local mock ref for the assertions) and type
  the doubles as Any — no type: ignore.

Full mypy roboco/ tests/ clean (785 files); the 21 runtime tests pass.

* fix(metrics): static cycle-time SQL — clear bandit B608 (CI gate)

The cycle-time query interpolated an optional team clause into the text() SQL
via an f-string, which bandit flags as B608 (hardcoded SQL) and turned the
merge gate red. The team value was always a bound parameter, so it was a false
positive — but the f-string is the trigger. Rebuilt as one static query with
(CAST(:team AS text) IS NULL OR a.details->>'team' = :team) and an always-bound
team param (CAST, not ::text — SQLAlchemy's :param parser collides with
PostgreSQL's :: cast operator, which broke the query as a stray param).

Full make quality green vs a real pgvector PG (all 21 gate steps).

---------

Co-authored-by: Renn F <rennf93@users.noreply.github.com>
2026-06-23 07:26:41 +02:00
Renn F 07008baaf5 fix(api): serialize pr_reviewer_notes/doc_notes/notes_structured in the task response
task_to_response (the builder behind the task list + detail endpoints the panel
reads) set dev_notes/qa_notes/quick_context but dropped pr_reviewer_notes,
doc_notes, and notes_structured, and TaskResponse didn't declare
notes_structured at all — so the PR-reviewer notes, documenter notes, and the
structured PR-review verdict were always blank in the panel regardless of what
agents persisted. The structured-content write-path + obligation gates were
fine; the data just wasn't serialized. Now returns every note section + the
structured source of truth. Regression test asserts all three round-trip.
2026-06-23 02:34:55 +02:00
Renn F ebc3b1ea18 fix(docs): push the documenter's doc commit so it lands in the PR
The documenter writes and commits docs onto the task branch in its own
workspace clone, but i_documented had no push step — so the commit stayed
local and the PM merged the already-open PR without the docs, which then
vanished on merge. i_documented now pushes the task branch before handoff,
mirroring the developer's _ensure_branch_pushed; a push failure holds the
task in awaiting_documentation for a retry instead of silently dropping the
docs. Extract _finalize_documented to keep the verb under the return-count
ceiling.
2026-06-23 00:39:08 +02:00
a8e5fa6467 [ed9c5b59] fix(tests): resolve 3 mypy arg-type errors in test_conventions_resolve using typing.cast (#244) (#245) (#246)
Use cast("ProjectTable", project) at the 3 _resolve() call sites where
SimpleNamespace was passed as a ProjectTable argument, satisfying mypy
without any # type: ignore suppressions. ProjectTable import is kept
under TYPE_CHECKING since the quoted cast form requires no runtime
symbol; cast is imported from typing for the runtime call.

Co-authored-by: Backend Developer 1 <be-dev-1@agents.roboco.dev>
2026-06-23 00:29:07 +02:00
Renn F fe029fe34b fix(self-heal): dispatch fix tasks autonomously instead of stranding them
A self-heal fix task was opened confirmed_by_human=false and held out of
dispatch until "Approve & Start" — but that button only renders for a
board-reviewed Intake task (pending + board_review_complete + team != main_pm),
never for a self-heal task (team=main_pm, no board review). So there was no way
to start it: it sat in pending forever and the Main PM never picked it up.

Self-heal is RoboCo healing itself, not an Intake draft — it shouldn't need a
manual Approve & Start. Origination now opens the fix task confirmed + assigned
to the Main PM agent, the PM dispatcher's self-heal hold is dropped, and the
now-dead approve_and_start special-case is removed. The fix still ships through
the normal gates (dev -> QA -> PR review -> the CEO's merge); the loop never
starts, merges, or deploys.
2026-06-22 23:37:07 +02:00
Renn F 90946c3abc fix(conventions): keep the read clone current on a private repo
The read-clone refresh reused the orchestrator's token-less best-effort fetch,
but _clone_repo scrubs the token from the remote URL — so on a PRIVATE repo the
refresh fetch failed silently and the clone stayed frozen at clone-time, never
seeing commits merged afterwards. The panel then showed "auto-derived defaults"
for a project whose .roboco/conventions.yml was already on the default branch.

Replace the refresh with a token-authenticated fetch + hard-reset to the default
branch (the token is injected transiently into the fetch argv, mirroring the
clone; the read clone is orchestrator-side and never mounted into an agent
container). A public repo with no token still refreshes unauthenticated.
2026-06-22 22:59:18 +02:00
Renn F 17ec52d1b7 feat(conventions): generalize defaults, backfill old projects, adopt the standard in-repo
Harden the architectural-conventions standard so it works out-of-the-box on
any project and resolves for projects that predate it, and make RoboCo pass
its own gate.

General defaults (apply to every project, not just one with a tuned file):
- The auto-scan excludes test and documentation trees (tests/, docs/) — those
  legitimately define fixtures and aren't enforced code.
- Helper placement seeds at warn, not block: `helper` matches any top-level
  function, too blunt a signal to hard-block a route file's small private glue.
  Misplaced model/route/component stay block; the body-level thin_routes check
  remains the real fat-handler guard.
- thin_routes no longer counts transaction-lifecycle calls (commit/flush/
  refresh) as data access — an explicit `db.commit()` after delegating to a
  service is a valid pattern.
- no_lint_suppressions exempts a small allowlist of structurally-unavoidable
  framework codes (ruff TC001-TC003, pydantic prop-decorator); bare or other
  suppressions still flag.
- CLAUDE.md rule-lifting skips bare common-word tokens that would match
  everywhere (e.g. "commit"), keeping only specific identifiers.
- The ambient prompt block lists only constrained modules and truncates at a
  line boundary with a "+N more" pointer instead of cutting mid-line.

Backfill: the standard previously read the committed file + repo scan from
project.workspace_path, a field only a manual API call set — so an older
project (or one whose workspace was cleared) showed an empty "missing" map no
matter what was pushed. The service now ensures a dedicated, default-branch
read clone on demand (WorkspaceService.ensure_read_clone) and resolves from
it, persisting the resolved path + real HEAD. The panel tab, the spawn-time
ambient block, and the per-task constraints all resolve the committed standard
with no manual setup.

Adopt in-repo: relocate the inline request/response models from the system and
*_live route modules into roboco/api/schemas/ so the codebase passes its own
placement gate, and ship a canonical .roboco/conventions.yml. no_models_in_routes
and modular_cohesion are now clean and enforced at block.

Docs updated across the user guide, the agent-facing RAG standard, the
developer and pr_reviewer role prompts, CLAUDE.md, and the changelog. New unit
tests cover the scan exclusions, helper-warn, the suppression allowlist, the
commit exemption, and the resolve/backfill path; the conventions + project
integration suites pass against Postgres.
2026-06-22 18:15:19 +02:00
Renn F 3a3cd69e1e feat(docs): documenter output is committed to the project repo, not just the knowledge store
Documenter docs only ever reached DOCS_BASE_PATH=/app/docs (a host-mounted, RAG-indexed knowledge store) and were never committed to the project's git repository — so the documenter's deliverable never landed in the repo. write_doc now also writes the doc into the agent's workspace clone under docs/<type>/<file> and commits it onto the task branch via GitService.commit, so it rides the existing PR into the repository on merge. Best-effort: a documenter without a cloned workspace or task branch still succeeds (logged, never fatal); the /app/docs knowledge store + RAG indexing are unchanged. Adds tests that the doc is written into the workspace and committed onto the task branch, and that it no-ops cleanly without a branch.
2026-06-22 15:50:46 +02:00
Renn F 2bbd1c2e70 test(conventions): baseline asserts the universal hygiene block rule, not a placement rule
The baseline-constraint and conventions-map integration tests asserted 'no models in routers' — the old UNIVERSAL block default. Now that placement rules are scan-derived (only seeded where the target module exists), an empty-scan test project carries no_models_in_routers no longer; the universal block rule is no_lint_suppressions. Updated the five assertions accordingly; behaviour (baseline attaches, isn't suppressed, is idempotent) is unchanged.
2026-06-22 15:19:50 +02:00
Renn F 4a8c508f91 feat(conventions): modularity checks + scan-derived, language-aware rules
The standard was architectural LINTING (placement + hygiene) — things ruff/eslint already do — and it forced backend rules onto frontend projects. This makes it enforce MODULARIZATION, the separation-of-concerns a senior demands that linters are blind to:

- modular_cohesion: a file that mixes architectural concerns (a model defined in a router, a schema in a component) is a monolith — split it. One concern per file.
- thin_routes (Python): a route handler that runs its own DB access instead of delegating to a service.
- thin_components (TypeScript/React): a component that fetches data in its body instead of using a hook.
- god_class: a class past a method-count threshold (single responsibility).

The checks inspect a definition's BODY and a file's COMPOSITION via tree-sitter, precision-over-recall (fire only on a confident structural signal). Rules are now scan-derived and language-aware: hygiene seeds universally, placement only for modules that exist, and modularity per stack — so a frontend project carries no_models_in_components + thin_components, never a backend no_models_in_routers. BUILTIN_RULES is reduced to language-agnostic hygiene.
2026-06-22 14:26:25 +02:00
Renn F fdfdc23b72 fix(orchestrator): give agent gate containers the test-DB env so the suite runs DB-backed
An agent's make quality runs against no Postgres, so the conftest skips every integration test and coverage collapses far below the 80% threshold — a self-hosted PM read 71% on a suite that is ~96% with a DB and chased it as a code regression. _append_gate_env now injects ROBOCO_TEST_DB_* (host/port/user/password/admin-db) from the orchestrator's own DB settings into each spawn; agents share the Docker network so the host resolves, and the conftest creates throwaway test databases isolated from the live one. The app runtime reads ROBOCO_DATABASE_*, never ROBOCO_TEST_DB_*, so this only feeds the test harness. Gated on toolchain_match_enabled, the faithful-gate flag.
2026-06-22 13:12:04 +02:00
Renn F 0a890c401e fix(gateway): surface an 'unknown' toolchain status at the gate instead of passing silently
The toolchain guard fails open on a recorded 'unknown' status (precision over recall — never strand a task on an inconclusive smoke). But an 'unknown' means provisioning ran yet the smoke could not confirm the suite is collectable under the interpreter, so the gate was proceeding blind with no trace — a silent hollow pass. It now emits a 'toolchain.unverified_gate_pass' warning with the agent and task ids when proceeding past 'unknown', while still not blocking; a missing marker (None) stays silent so the warning carries signal.
2026-06-22 13:06:55 +02:00
Renn F 02f43f5dfe fix(tests): make crypto tests hermetic with a self-supplied Fernet key
test_crypto's round-trip tests called the real encrypt/decrypt path, which needs settings.encryption_key configured — so they silently depended on ROBOCO_ENCRYPTION_KEY being present in the environment. In agent gate containers it is not, so four tests failed there and the agent mis-read it as a code regression. An autouse fixture now monkeypatches a valid generated Fernet key, so the tests pass without any ambient secret and agents never need the production key injected to gate.
2026-06-22 13:04:26 +02:00
Renn F c49dcebae4 fix(toolchain): raise requires-python floor to 3.13 so the resolver matches the runtime
RoboCo's code imports tomllib (3.11+) and the stack runs on 3.13, but requires-python declared >=3.10. The toolchain resolver picks the lowest satisfying version, so it provisioned agent workspaces of the self-hosted roboco-api project at Python 3.10 — an interpreter the suite cannot even be collected under, leaving the workspace .venv unusable and the gate running in an ad-hoc fallback env. Raising the floor to >=3.13 makes resolve_target_python return 3.13, matching the agent image. Re-locks to drop the now-unreachable 3.10-3.12 backports; a guard test pins the repo's own resolution to 3.13.
2026-06-22 13:02:59 +02:00
16789c1ca7 Feature/architectural conventions standard (#243)
* feat(conventions): standard schema models + effective-map merge

* feat(conventions): tree-sitter Python classifier + placement checks

* feat(conventions): TS classifier, hygiene/custom checks, runner + CLI

* feat(conventions): ROBOCO_CONVENTIONS_ENABLED flag + cache table + migration

* feat(conventions): repo auto-scan + scaffold draft renderer

* feat(conventions): ConventionsService (cache/baseline/ambient/scaffold/restore)

* feat(conventions): auto-scaffold on project registration (flag-gated)

* feat(conventions): TaskDescription.constraints + auto-baseline attach

* feat(conventions): ambient architecture-map injection at spawn

* test(conventions): subprocess CLI smoke for the agent-image entrypoint

* feat(conventions): block i_am_done on block-level convention violations

* feat(conventions): block pr_pass on unresolved convention violations

* feat(conventions): surface convention findings into QA evidence

* docs(prompts): convention awareness for PO/Intake/Dev/QA/PR-reviewer

* feat(conventions): panel Conventions tab + flag toggle + parity

* test(conventions): end-to-end block, fix, and waiver through the gate

* refactor(conventions): extract pr_pass guards to keep pr_gate under the gate

* style(conventions): format the baseline-constraints attach in task.create

* test(conventions): type-annotate test helpers for the full mypy gate

* build(conventions): ignore types-PyYAML in deptry (mypy-only type stub)

* docs(conventions): document the standard in CLAUDE.md + PM prompt awareness

* fix(conventions): baseline constraints are non-suppressible (dedup-append)

* feat(conventions): scaffold on first workspace clone (threaded workspace)

* feat(conventions): multi-project ambient map for PO/Intake (per-product)

* feat(conventions): persist findings + violations-feed route (migration 044)

* feat(conventions): panel violations feed in the Conventions tab

* test(conventions): intake-spawn mock accepts the ambient layer kwarg

* fix(docker): ollama-init best-effort pull, gate startup on cached models present

A degraded/slow ollama registry made the model manifest re-check fail under
set -e, so ollama-init exited 1 and blocked the orchestrator's
service_completed_successfully gate — taking the whole stack down even though
both models were already cached. Pulls are now best-effort; success is gated on
the models being present, so a flaky registry can't down a cached deployment.

* refactor(content): drop dead TaskDescription.with_baseline_constraints

The structured baseline-merge helper had zero production callers. Project-task baseline constraints are attached by the wired string backstop (TaskService._attach_baseline_constraints), and a real task is free-form prose that cannot form a valid TaskDescription (requires a non-trivial objective + non-empty the_work), so the helper was unreachable from any live path — a leftover from the structured-merge -> string-append design pivot. Removing it leaves a single enforcement path. The constraints field itself stays: it is a member of the well-formed-spec schema (Objective / What This Builds / The Work / Notes / Constraints / Acceptance Criteria), rendered by render_markdown and unit-tested.

---------

Co-authored-by: Renn F <rennf93@users.noreply.github.com>
2026-06-22 12:37:46 +02:00
01e10ad693 Feat/provider overload break (#242)
* feat(conventions): standard schema models + effective-map merge

* feat(orchestrator): park provider on persistent server overload (529/500)

A 429 rate limit already parks a provider — queue its spawns, probe until it
recovers — but a persistent 529/500/503 overload had no such break: the run
died and the orchestrator crash-retried straight back into the overload,
burning tokens in a respawn loop.

Generalize the park to provider-unavailability. On a non-graceful Anthropic
agent exit, match the API's overload markers (overloaded_error /
internal_server_error / "API Error: 5xx") against the dead container's own
output and park the provider with kind="overloaded"; the existing spawn gate
already queues any parked provider, and the probe-resume loop revives the task
when it recovers. Grok keeps its exit-75 path; both now route through one
_park_provider_unavailable helper. Markers are kept specific so an agent that
merely writes about HTTP 500/529 can't trip the break.

Fix the recovery probe to require a 2xx: it treated any non-429 as recovered,
so a probe that itself got a 529 would have resumed agents straight back into
the overload — wrong for the new path and for a 429 that lifts into a 5xx.

Gated by ROBOCO_OVERLOAD_BREAK_ENABLED (default on; off => crash-retry).

---------

Co-authored-by: Renn F <rennf93@users.noreply.github.com>
2026-06-22 03:49:45 +02:00
Renn F 7d50405ccb fix(toolchain): type-annotate test helpers for the full mypy gate
make quality runs mypy over tests/ too; the toolchain test fixtures
(monkeypatch params, the fake-subprocess factories) were missing annotations.
No behavior change.
2026-06-22 01:59:41 +02:00
Renn F 78c6981089 test(toolchain): end-to-end interpreter-match + loud-fail guard
Ties the pieces at the logic level: a target whose .python-version (3.13)
conflicts with requires-python (>=3.14) provisions against 3.14, a collection
error records 'broken', and the gate guard then refuses the pass. Flag-off
provisions exactly as today and never blocks.
2026-06-22 01:54:50 +02:00
Renn F 0ea9f48187 feat(toolchain): block QA/dev/PR gates when the suite cannot be executed
Adds _toolchain_broken_guard: when toolchain matching is on and the acting
agent's workspace recorded a 'broken' status (the project's suite cannot be
collected under the provisioned interpreter), i_am_done, pass_review, and
pr_pass refuse with an i_am_blocked remediation instead of letting the role
"pass" on a source read. fail_review / pr_fail stay available. git gains
toolchain_status_for_task (resolves the actor's workspace, reads the marker,
fail-open None). Inert when the flag is off; a missing/unknown status never
strands a task.
2026-06-22 01:49:44 +02:00
Renn F 036416878d feat(toolchain): provision against the target interpreter + runnability smoke
When toolchain matching is enabled, install_dev_deps resolves the target's
Python and provisions the venv against it (uv sync --extra dev --python <v>,
uv auto-downloads), then runs a runnability smoke (uv run pytest --collect-only)
and records {python, status} in a .git/.roboco-toolchain marker the gates read.
Precision over recall: only a pytest collection error (the interpreter-mismatch
signature) reports 'broken'; pytest-absent / timeout report 'unknown' so a gate
never blocks on an inconclusive smoke. Flag off => provisioning is unchanged.
2026-06-22 01:40:06 +02:00
Renn F 6df083c594 feat(toolchain): flag + WorkSession toolchain columns + migration
ROBOCO_TOOLCHAIN_MATCH_ENABLED (default-off) gates the whole subsystem.
Adds work_sessions.toolchain_python / toolchain_status (nullable VARCHAR(20))
recording the interpreter the workspace was provisioned with and whether the
target's suite can be executed (ok | broken | unknown). Migration 042 verified
with a real upgrade head + downgrade -1 + re-upgrade on a throwaway Postgres.
2026-06-22 01:31:45 +02:00
Renn F 9274cd4584 feat(toolchain): resolve target interpreter from requires-python
Pure resolver (services/toolchain.py) that derives the Python version an agent
should provision a target workspace with. Defends against uv's resolution
order — a .python-version file overrides requires-python during interpreter
selection — by honoring the pin only when it satisfies requires-python, else
resolving a concrete version from requires-python for the caller to pass via
--python. This is the root cause behind the live guard-core-app failure (pin
3.13 vs packages needing 3.14). Promotes packaging to a direct dependency.
2026-06-22 01:25:47 +02:00
Renn F 8cf697816f feat(content): obligate role note sections like journals
Completes the note(scope='handoff') write-path (WIP 23e6ee57): every role
with a dedicated note section is now obligated to populate it, the same way
journals are obligated.

Obligations (foundation.policy.tracing):
- DEV_NOTES / PR_REVIEWER_NOTES / QUICK_CONTEXT_MIN_CHARS requirements +
  checkers, wired onto i_am_done (dev_notes), delegate (quick_context), and
  pr_pass / pr_fail / post_pr_review (pr_reviewer_notes).
- Fixes a latent bug: the docs-notes checker read dev_notes instead of
  doc_notes (the documenter's section); the i_documented shim now feeds
  doc_notes to match.

Auditor: a session-scoped note obligation on i_am_idle — the auditor owns no
delivery task and has no delivery verb, so it must have recorded an
observation within the window before going idle (JournalService.has_recent_entry).

Write-then-gate: persisted sections (dev_notes / quick_context) are
pre-written by the agent's note(scope='handoff') before the gated verb;
argument-borne sections (doc_notes / pr_reviewer_notes) are checked through a
SimpleNamespace shim, the same pattern qa_notes already uses.

Config: dev/pr_reviewer/quick_context min-chars (40/40/30), panel-tunable.
Plus per-gap remediation hints and full coverage (write-path routing,
ownership, validation->remediation, each obligation, the doc_notes fix).

Full make-quality green: 9777 passed, 95.6% coverage.
2026-06-21 20:31:42 +02:00
Renn F 2aa00ca124 test(content): assert transition notes via markers, not quick_context
Three integration tests still asserted the old quick_context soup format for
completion_notes / escalation_notes — update them to read the new
orchestration_markers transition_note (the source moved in the prior commit).
2026-06-21 08:14:55 +02:00
Renn F 3a67fa4a0f fix(content): anti-soup guard on pitch slug + restore pitch test intent
Sweep found pitch slug was the one content-tool free-text field still outside
the anti-soup guard (a product could be slugged 'wip'/'asdf'). Add it to the
guard (min 2, so real short slugs like 'ui' pass). Also fix two pitch tests
that were false-passing on a 1-char title (the title guard rejected first):
give them substantive fields so they actually exercise the role gate
(not_authorized) and the non-cell-target rejection (invalid_state).
2026-06-21 08:08:28 +02:00
Renn F 33a3ea3d4a fix(content): move lifecycle-transition notes off quick_context into markers
The structure-everything sweep found four more writers packing key:value soup
into quick_context (the human ResumptionNote field), same anti-pattern as the
already-fixed approve_and_start_notes:
  - _record_completion_notes  -> completion_notes:<text>
  - escalate_to_ceo           -> escalation_notes:<text>
  - ceo_approve               -> ceo_approval_notes:<text>
  - ceo_reject                -> ceo_rejection:<reason>

Route them through a unified orchestration_markers['transition_notes'] dict
(keyed by event) via markers.set_transition_note, so quick_context carries only
the structured ResumptionNote and the panel never shows raw <event>:<text> soup.
Adds the typed accessor + a roundtrip test.
2026-06-21 08:04:15 +02:00
Renn F 8463808616 fix(lifecycle): let a PM recover its rejected coordination task from needs_revision
The in-path PR-review gate created a deadlock: when an assembled cell→root /
root→master PR fails the gate (pr_fail) — or qa_fail / ceo_reject fires — the
PM-owned coordination task lands in needs_revision, which was developer-claim-
only. So the task had no actor and no exit but cancel, and the cell PM escalated
in a loop (8KB of [ESCALATED] dev_notes on one task). Pre-gate, the PM simply
re-delegated from in_progress; the gate routed the failure through the PM's own
task instead.

Add NEEDS_REVISION to the CELL_PM / MAIN_PM claim rules so the PM re-claims via
i_will_plan, revises the plan, and re-delegates the fixes — pr_fail/qa_fail
already reassign the failed task to its owning PM and the revision dispatcher
re-spawns it; the claim rule was the only missing piece.

Scope is by give_me_work routing (offers only the caller's own assigned tasks),
the same mechanism that scopes a developer's leaf-revision — NOT a gateway-only
ownership gate, which would violate the spec=gateway parity invariant and can't
use task_type anyway (main-PM coordination roots can be code-typed). Regenerates
panel/lib/lifecycle.json.
2026-06-21 06:09:59 +02:00
Renn F 3ff6967b9d fix(content): keep coordination notes off dev_notes / quick_context
Two agent-authored fields were leaking non-developer content into the
human note columns the panel renders:

- apply_escalation appended '[ESCALATED] From X to Y\nReason: ...' to
  dev_notes (the developer's space). On a re-escalation loop a stuck cell
  PM grew one task's dev_notes to ~8KB across 5 escalations. It now writes
  a structured orchestration_markers['escalation'] record; the target
  still learns the reason from the escalate notification.
- approve_and_start string-packed 'approve_and_start_notes:<text>' into
  quick_context (raw key:value soup). It now writes
  orchestration_markers['approve_and_start_notes'], leaving quick_context
  for the human ResumptionNote only.

Adds typed marker accessors (get/set_escalation, get/set_approve_and_start_notes)
and refactors _record_pr_review under the complexity bound by extracting
_compose_review_body. Documents update_task_with_message as the legacy
A2A-protocol log (dev_notes is intentional there, not pollution).
2026-06-21 05:36:50 +02:00
Renn F 4c85cc6dfa feat(content): anti-soup guard on the flow verbs' free-text
Extends structured-content enforcement from the content tools to every
flow verb that carries agent free-text, closing the last hole where a
dev/PM could pass word soup: i_am_blocked(reason), i_am_done(notes),
submit_up/submit_root/complete(notes), escalate_up/escalate_to_ceo(reason),
pass_review(notes), fail_review/pr_fail(issues), pr_pass(notes),
i_documented(notes), delegate(title/description). Plans (i_will_plan /
i_will_work_on) keep their existing >=150-char approach + sub_task gates
and are skipped here so recovery re-entry with thin values still works.

Shared helpers on the choreographer: _free_text_soup (bare envelope, list
aware) and _soup_or_decision_env (folds the soup check into a verb's
existing spec-gate return so no verb gains a return or tips the xenon
bound). reject_trivial now also catches all-filler multi-token strings.

base.md documents the broadened rule for agents.
2026-06-21 05:11:20 +02:00
Renn F 96ca53eff2 feat(content): extend anti-soup guard to pitch title, open_session topic, pr_update, note narratives
The universal guard covered say/dm/note-text/progress/notify/pitch
problem+solution. Close the remaining content-tool fields the agent
authors: pitch title, open_session topic, pr_update title/body (when
supplied), and the decision/reflect narrative sub-fields of note
(rationale/context/what_done/...). Narrative fields are only checked
when the agent fills them — an omitted field keeps its tolerant
'(not provided)' placeholder so a thin note still records and never
trips the do-server circuit breaker. Fold pr_update's no-fields + soup
checks into one helper to stay under the return-count bound.
2026-06-21 04:46:28 +02:00
Renn F b1f1a91066 feat(content): reject all-filler multi-token strings in reject_trivial
The single non-empty/non-placeholder gate caught a lone banned token
(wip) and below-floor strings, but multi-token soup made entirely of
placeholders (wip wip, tbd / na, todo todo todo) slipped through both
checks. Add an all-tokens-filler test that strips edge punctuation per
token and rejects when every meaningful token is banned — without
flagging real prose that merely contains a filler word (none of the
tests failed). Strengthens every content model + gateway anti-soup
guard that composes reject_trivial.
2026-06-21 04:42:26 +02:00
Renn F db74f546a3 feat(content): universal anti-soup guard on every agent free-text field 2026-06-21 04:20:07 +02:00
Renn F b086dc2c41 feat(content): QA pass/fail stores a structured QaNote 2026-06-21 03:54:33 +02:00
Renn F 11c1d9eee7 feat(content): structured PR-review findings + generated GitHub comment 2026-06-21 03:47:17 +02:00
Renn F b97465a631 test(content): assert pr_fail issues land in pr_reviewer_notes (gate path) 2026-06-21 03:23:07 +02:00
Renn F c47c886508 test(content): mypy-clean structured-model construction + lint fixes 2026-06-21 03:07:58 +02:00
Renn F 2e490a06d7 feat(content): PR reviewer writes its own slot, stops clobbering QA/Dev 2026-06-21 02:56:53 +02:00
Renn F 0fba4eed01 refactor(content): move orchestration markers off quick_context to typed jsonb 2026-06-21 02:48:07 +02:00
Renn F 7e74112fcc feat(content): structured-note persistence + derived TEXT mirror chokepoint 2026-06-21 02:00:21 +02:00
Renn F 31c2d5adc6 feat(content): add doc_notes column for the documenter's own slot 2026-06-21 01:58:43 +02:00
Renn F a6e09f6d75 feat(content): task structured-note fields + orchestration-marker accessors 2026-06-21 01:55:36 +02:00
Renn F dd03c218d1 feat(content): migration for structured note columns + quick_context split 2026-06-21 01:51:48 +02:00
Renn F f0368af757 feat(content): structured content schema models + renderers 2026-06-21 01:44:07 +02:00
Renn F 7662804f1c fix(orchestrator): spawn the owning PM for a gate-failed assembled PR
The PR-review gate's pr_fail sends a cell->root / root->master PR back to
needs_revision, still owned by the cell/main PM. The dispatch side
(_dispatch_revision_coordination_roots) re-spawns the owning PM for any
PM-owned needs_revision task, but the readiness gate
(_readiness_check_role_for_status) only waived the dev/doc role restriction for
coordination tasks — and a gate-failed assembled PR has a project + branch, so
it is NOT coordination. Result: dispatch routed the cell PM, readiness refused
it ("state=needs_revision requires role in {developer,documenter} but agent
be-pm is cell_pm"), and the task deadlocked. Pass owner_is_pm (derived from the
task's assignee, like the dispatch side) so the readiness waiver also fires for
a PM-owned revision, not just coordination roots. QA stays excluded.
2026-06-21 00:04:24 +02:00
Renn F 33ed0209c8 fix(reaper): don't reap a live container the instance registry forgot
The stale-claim reaper skips a task whose assignee holds an ACTIVE instance,
but that check reads the in-memory _instances registry — lost on an
orchestrator restart while the agent's container keeps running. The reaper then
released a task out from under a live agent it had merely forgotten (registry
amnesia). On a registry MISS, fall back to asking Docker directly
(_inspect_container_state); a still-running container is spared. A known
instance (active or stopped) stays authoritative, and an uninitialised registry
(unit-test harness) keeps the prior behaviour.
2026-06-20 22:51:03 +02:00
Renn F 6f3ba2f42e fix(bash-guard): block package-env mutations targeting /app (all providers)
An agent that ran `uv sync` / `pip install` against /app rebuilt the
orchestrator's MCP-gateway venv (/app/.venv) — breaking its own gateway tools
(every roboco-flow/-do/-git verb), stranding it, and getting its task reaped.
Deny package-mutation verbs (uv sync/lock/add/remove, uv pip, pip install)
whose target resolves to /app (cd /app, --project/--directory /app, /app/.venv,
UV_PROJECT_ENVIRONMENT=/app). Placed outside the ROBOCO_GUARD_SKIP_GIT gate so
it fires for every provider — the Claude PreToolUse hook and the grok exfil
hook alike. Reads of /app and workspace installs are untouched.
2026-06-20 22:51:02 +02:00
Renn F 2d8a6c5d0f fix(self-heal): assign the fix task to the Main PM agent, not just the team
Origination created the task with team=main_pm but no assignee, so after
the CEO's Approve-&-Start it fell to the unassigned-team routing, which
the orchestrator picks up slowly or never. Assign the Main PM agent up
front (the seeded foundation uuid) so the dispatcher routes it straight
to that agent via the assigned-PM path once confirmed. The
confirmed_by_human hold is unaffected — the dispatcher's self-heal skip
sits before the assigned/unassigned split, so the task stays inert until
the CEO approves it (now covered by a test).
2026-06-20 19:10:21 +02:00
Renn F 65683394d4 fix(self-heal): make the CI regression signal deterministic
The loop read the latest completed Actions run with per_page=1 and an
empty default workflow scope, so the conclusion flickered: on RoboCo's
8-workflow repo a green run from an unrelated workflow (or a green run on
an older commit) masked a red ci.yml run, and a single transient GitHub
error silently skipped the whole cycle — so self-heal sometimes fired on
a real regression and sometimes did not.

- Default self_heal_ci_workflow to "ci.yml" so the signal is scoped to
  the gate workflow, not "latest run across all workflows".
- Fetch a window of recent completed runs and resolve the conclusion
  against the branch's current HEAD (newest commit's latest attempt), so
  a stale/unrelated green run can't mask the HEAD failure and a green
  re-run supersedes the original failure.
- Retry transient network / 429 / 5xx errors within the cycle instead of
  treating one blip as "all green".
2026-06-20 19:10:21 +02:00
5fe1e6df58 feat: in-path PR-review gate — per-cell + main reviewers (#229)
* feat(lifecycle): add the in-path PR-review gate status + reviewer verbs

Insert awaiting_pr_review between the assembled-PR submit and the PM merge,
giving the merge level the rejection capability it structurally lacks — today
only qa_fail and ceo_reject ever reach needs_revision, so a PM review is a
merge button with no teeth.

- New Status awaiting_pr_review + submit_for_review / pr_pass / pr_fail actions
  (pr_pass -> awaiting_pm_review, pr_fail -> needs_revision, mirroring the QA gate).
- Reviewer verbs claim_gate_review / pr_pass / pr_fail, and a main-PM submit_root
  verb (the root analogue of the cell PM's submit_up; opens the root->master PR).
- Extend the self-review-symmetry validator to the new sign-off actions.
- Mirror the value into the ORM TaskStatus enum + the A2A state map, and add the
  postgres taskstatus enum value (migration 040, forward-only like 037).
- Regenerate the per-role verb tables; add gate spec tests.

Spec surface only; the gateway methods + dispatch are wired in follow-ups, so the
verbs are advertised but dormant (flow_server tolerates unregistered verbs).

* feat(identity): add the three cell PR-review-gate reviewers

The in-path gate needs a reviewer per cell so each cell's assembled cell->root
PR is reviewed by a stack-specialized agent, while pr-reviewer-1 serves the
root->master gate (and keeps doing inbound external PRs).

- be/fe/ux-pr-reviewer: PR_REVIEWER role, team-scoped (so dispatch routes each
  cell's gate to its own reviewer); seeded identities + ROLE_TEAM_RULES + names.
  AI agent count 22 -> 25.
- They reuse the existing roboco-agent-pr-reviewer image (AGENT_IMAGES maps the
  three slugs to it, as be-dev-1/-2 share one image) — no new image.
- Tracing table: pr_pass/pr_fail require a learning entry (parity with
  post_pr_review), submit_root mirrors submit_up, claim_gate_review is waived
  (its tracing applies on pr_pass/pr_fail) — completes the verb surface added
  in the prior commit.
- Update the roster-pinning identity tests.

* feat(gateway): wire the in-path PR-review gate end to end

Make the assembled-PR review gate operational across the choreographer, the
TaskService transitions, and the v1 flow surface.

- TaskService: submit_for_review (in_progress→awaiting_pr_review), pr_gate_claim
  (no-transition reviewer claim), pr_pass (→awaiting_pm_review), pr_fail
  (→needs_revision); mirror qa_pass/qa_fail (clear claim, actor-mismatch warn,
  issues appended for the PM's revision). VerbRunner gains the matching atomic
  handlers + a create_root_pr side effect.
- Repoint submit_up to compose submit_for_review (cell→root PR enters the gate),
  and add a main-PM submit_root verb (opens the root→master PR, enters the gate).
- Split main_pm_complete: a code root must pass the gate first (requires
  awaiting_pm_review; rejects an in_progress code root toward submit_root and no
  longer reopens the PR), while a branchless coordination root still walks
  straight through, ungated.
- PRGateMixin (claim_gate_review / pr_pass / pr_fail) composed onto the
  Choreographer; flow_server forwarders + v1 routes (pr_reviewer + main_pm) +
  request schemas.
- Tests: gate spec + the updated submit_up / main_pm_complete expectations + new
  real-DB integration tests driving submit_for_review→pr_gate_claim→pr_pass and
  pr_fail through the real enforcement layer.

* feat(orchestrator): dispatch the in-path PR-review gate

Make the gate live in the dispatch loop.

- _dispatch_pr_gate_work: route awaiting_pr_review tasks to reviewers by level —
  a cell→root task to its cell reviewer (be/fe/ux-pr-reviewer), the root→master
  task to pr-reviewer-1. The reviewer self-claims via claim_gate_review (no
  pre-claim, mirroring the external-PR dispatcher); registered in
  _dispatch_all_work. _select_agent_for_cell learns the pr_reviewer role.
- _build_pr_gate_prompt: anchors the reviewer to the parent objective + full
  acceptance criteria + the FE<->BE contract, then pr_pass / pr_fail.
- _readiness_check_role_for_status: awaiting_pr_review -> pr_reviewer.
- Fail routing: pr_fail reassigns the failed assembled task to its PM
  (_revision_pm_for_task: cell PM for a cell team, Main PM for the root), and the
  revision dispatcher is generalized from coordination-roots-only to any
  PM-owned needs_revision task so the gate-failed task is re-coordinated instead
  of deadlocking.

* docs: document the in-path PR-review gate + the cell reviewers (22→25)

Reflect the shipped gate across the canonical + RAG docs.

- CLAUDE.md: agent count 22→25, the cell reviewers in the org chart, an
  awaiting_pr_review state + the gate transitions + a gate note in the lifecycle
  section, and submit_root / claim_gate_review / pr_pass / pr_fail in the verb
  surface table.
- docs/rag/architecture: org-structure (count, cell-reviewer roster, cells
  table), agent-uuids (be/fe/ux-pr-reviewer rows), agent-model (role + team
  rows).
- docs/rag/roles/pr-reviewer: the in-path gate section + the gate verbs.
- Wrap reviewer.id with UUID(str(...)) in the gate DB tests for mypy.

* docs: finish the gate doc sweep across README + RAG + generated artifacts

Catch the remaining surfaces beyond the canonical docs.

- README + how-to: agent count 22→25, the 6-agent cells (+ PR Reviewer), the
  main reviewer's root→master gate role.
- RAG: permissions + tool-permissions + task-tools list the gate verbs
  (claim_gate_review / pr_pass / pr_fail) for pr_reviewer; regenerate the
  lifecycle artifacts (intent-verbs, status-transitions, the per-role
  lifecycle-*.md prompts, panel lifecycle.json) from the spec via
  build_lifecycle_artifacts.py so they carry the new status + verbs.

* fix(migration): shorten the 040 revision id to fit alembic_version VARCHAR(32)

The revision id '040_taskstatus_awaiting_pr_review' is 33 chars; alembic's
alembic_version.version_num column is VARCHAR(32), so recording the migration on
a real 'alembic upgrade head' failed with 'value too long for type character
varying(32)' (surfaced on the NAS deploy). The test suite missed it: the test DB
is built via Base.metadata.create_all and the parity test only renders SQL
offline, so nothing actually applied the migration chain.

- Rename to '040_awaiting_pr_review' (22 chars).
- Add a guard test asserting every revision id fits the VARCHAR(32) column.
- Verified by applying the full chain 001->040 against real Postgres: it now
  reaches head and records '040_awaiting_pr_review' without truncation.

* fix(migration): land the actual 040 revision-id shortening + guard test

The prior commit captured only the file rename (git add aborted on the deleted
old path), leaving the long revision id and missing the guard test. This commit
carries the real content: revision id '040_awaiting_pr_review' (22 chars) and the
revision-id length guard. Re-verified against real Postgres — the full chain
reaches head and records the short id without truncation.

* fix(product): flush cell deletes before inserts when re-mapping projects

Editing a product's cell->project map (PATCH /api/products/{id}) 409'd with
'duplicate key value violates unique constraint uq_product_projects_product_team'
whenever a team already had a mapping. _replace_cells clears the old rows and
appends the new ones, but within a single flush SQLAlchemy orders INSERTs before
DELETEs for the same table, so the new (product_id, team) rows collided with the
not-yet-deleted old ones. Flush the deletes first.

Pre-existing bug (unrelated to the PR-review gate); surfaced on the NAS. New
real-Postgres regression test re-maps all three cells to different projects —
it fails with the unique violation without the fix and passes with it. The
existing update test only changed WHICH team was mapped, so it never collided.

* fix(gateway): let main_pm submit_root past the shared submit-up guard

submit_root reused the cell PM's _submit_up_ownership_guard, which
hardcoded agent.role != cell_pm and rejected the Main PM with
"submit_up is reserved for cell_pm". A branch-bearing code root could
then never close: submit_root bounced to complete, while complete
required awaiting_pm_review (reachable only via submit_root) and bounced
back — a circular rejection.

Both callers already run the spec gate (can_invoke_intent), which
enforces submit_up→cell_pm and submit_root→main_pm, so the guard's role
re-check was redundant for submit_up and wrong for submit_root. Broaden
it to accept either PM role as a defense-in-depth non-PM reject.

Adds the first choreographer-level submit_root test (the gap that let
this ship).

* fix(gateway): proactively steer both PMs to their bubble-up verb

The submit_root deadlock had a sibling steering gap: nothing told a PM
which verb opens the gate. The delegate next-hint said only 'i_am_idle
when done', and complete's in_progress rejection named submit_root for
the Main PM but left the Cell PM with a bare 'not ready for completion'
— no submit_up pointer, the same guess-the-verb trap.

- delegate hint now names the role-correct verb (root → submit_root,
  cell parent → submit_up) proactively, before any rejection.
- cell_pm_complete's in_progress rejection now steers to submit_up,
  mirroring the Main PM's submit_root gate hint.

Tests cover both the cell-PM steer and the role-aware delegate hint.

* docs: correct who-merges-which-PR across the gate docs + complete description

Audit of the gate docs found the merge actors mis-stated in several
places — the exact ambiguity that risks 'the reviewer/PM merges the root
PR' confusion:

- complete IntentSpec description said 'Main PM merges root PR' — false;
  main_pm_complete escalates and the CEO merges root→master. Corrected
  (propagated to intent-verbs.md, lifecycle.json, generated role prompts
  via build_lifecycle_artifacts.py).
- task-tools.md: submit_up target was awaiting_pm_review (should be
  awaiting_pr_review); Main PM flow had no submit_root — added it.
- README.md: lifecycle diagram now shows the awaiting_pr_review gate.
- cell-pm.md / main-pm.md: dropped the stale 'submit_up hands work to the
  Main PM who merges your cell branch' model — the cell PM merges its own
  gated cell→root PR; the Main PM owns the root + submit_root; the CEO
  merges master. Added submit_root to the main-pm manifest.
- git-commits.md, pr-creation.md, tool-permissions.md, git-tools.md:
  stopped attributing root→master PR opening to complete (it's submit_root).

No behavior change; verb wiring + state machine verified gap-free this
session (the pr_fail→needs_revision→PM respawn loop closes correctly).

* fix(orchestrator): stop closure respawn waiting the reaper window

A PM that finished its subtasks and idled left its parent 'paused' with a
fresh last_heartbeat_at. _is_recently_paused gated closure respawn on
_claim_heartbeat_ttl — the REAPER window (stale_claim_reap_seconds: 600s
default, 1800s on the NAS) — so the parent sat untouched for up to 10-30
minutes before its PM was respawned to close it. The whole chain stalled
behind it.

The race that guard actually protects against (i_am_idle auto-pauses, then
the agent is marked IDLE + its container tears down) is seconds, and the
live-session case is already covered by _is_agent_active. Introduce a
dedicated short debounce (pm_closure_recently_paused_seconds, default 45s)
and gate closure on that instead.

The existing test fixture masked this by setting _claim_heartbeat_ttl to
claim_stale_seconds (180s), not the production reaper value. Fixture now
mirrors production; adds a regression test that a parent paused past the
debounce but within the reaper window respawns immediately.

* feat(gate): post the in-path review verdict on the assembled PR

The in-path gate previously left no trace on the PR it gated — pr_pass /
pr_fail were pure status transitions. Now each verdict is posted as a
GitHub review on the assembled PR itself (server-side, bot account), so
the decision is visible on the very PR the PM merges.

- pr_pass → APPROVE, pr_fail → REQUEST_CHANGES on a cell→root PR.
- The root→master PR ALWAYS gets a plain COMMENT, never APPROVE/REQUEST_
  CHANGES: only the CEO acts on master, so the gate must never leave an
  approval that could satisfy branch protection (letting someone else
  merge) nor a blocking review that could impede the CEO's merge.
- Best-effort and AFTER the DB transition — a GitHub failure is logged,
  never rolls back the gate decision. Reuses git.post_pr_review's existing
  self-review→COMMENT downgrade for the org's own PRs.

Adds _project_slug_for to the ChoreographerHelpers protocol (mypy) and a
unit suite covering event selection, the master-bound COMMENT rule, the
no-PR skip, and failure-swallowing. Docs updated (pr-reviewer, task-tools).

---------

Co-authored-by: Renn F <rennf93@users.noreply.github.com>
2026-06-20 09:27:29 +02:00