Commit Graph
5 Commits
Author SHA1 Message Date
889f3689e7 MegaTask (#248)
* feat(batch): batch_id + collision descriptor columns

Sequenced batch intake ("Mega task") foundation: tasks.batch_id (indexed)
groups a batch of top-level tasks created together; intends_to_touch (text[]),
adds_migration and touches_shared (bool, NOT NULL default false) are the
per-task collision surface the SequencingService will read to wire dependency
waves. Mirrored on the Task model + TaskCreateRequest and wired through
TaskService.create. Migration 046 (real upgrade->downgrade->upgrade verified
vs a throwaway pgvector PG); a non-batch task declares no surface (defaults).

Task 1 of the 0.11.0 sequenced-batch-intake plan.

* feat(batch): flag + draft collision descriptors

Default-off ROBOCO_BATCH_INTAKE_ENABLED (config + FEATURE_FLAGS + panel card);
the propose_draft tool doc + the TS DraftProposal gain the per-task collision
surface intends_to_touch / adds_migration / touches_shared. The draft is a loose
dict so the descriptors ride it through the relay intact (test asserts the
forwarded payload); the analyzer (Task 3) reads them to wire dependency waves.

Task 2 of the 0.11.0 sequenced-batch-intake plan.

* feat(batch): deterministic collision-sequencing analyzer

SequencingService.analyze turns a batch's per-task collision surfaces into a
dependency DAG + execution waves — correctness in CODE, not agent judgment.
Rules in order: file overlap serializes (more-important first), migrations form
a serial chain (no concurrent Alembic heads), touches_shared runs last, cell
contention warns (never serializes); then dedupe, existence + cycle check, and
Kahn topological layering. Pure (no DB/services); SequencingError on a cycle or
out-of-range edge.

Golden test reproduces the CEO's hand-sequenced 4 waves of the 11-item
guard-core-app batch (the effort that deadlocked the Main PM): S6 alone last,
the R1/R3/R4 migration chain, R2/R3/S8 serialized on the shared threat service,
S1/S2/S7 in one parallel wave.

Task 3 of the 0.11.0 sequenced-batch-intake plan.

* chore(batch): brand the user-facing surfaces "MegaTask"

The user-facing name is MegaTask: the feature-flag label is "MegaTask intake",
the panel flag-card and the config description lead with MegaTask. Internal
names stay technical (batch_intake_enabled, batch_id, SequencingService).

* chore(batch): drop the feature flag — MegaTask is a core intake scope

MegaTask is additive and opt-in by its own nature (the Prompter proposes a
batch only when the CEO asks for several tasks; single-task intake is
unchanged), so there is no risk surface a flag protects — 'don't create a
MegaTask' is the off switch. Remove batch_intake_enabled from config, the
FEATURE_FLAGS registry, the panel flag card, and its tests. MegaTask will be
a third scope option in the Intake modal (single-cell / multi-project /
MegaTask), not a toggle.

* feat(batch): MegaTask identity predicate + orchestrator branchless recognition

The single source of truth for the umbrella's exemptions: pure
is_batch_umbrella / is_batch_root_subtask / is_branchless_coordination
(foundation/policy/batch.py) — an umbrella has a batch_id and is top-level; a
root-subtask shares the batch_id but is parented. The orchestrator's
_is_coordination_task now consults is_branchless_coordination, so a MegaTask
umbrella is recognized as doing no git of its own (git-exempt at spawn-readiness
/ stuck-detection) exactly like a product fan-out root. Non-batch behavior is
identical (the predicate reduces to the old no-project+product check; the
orchestrator coordination suite stays green), and the umbrella branch is inert
until the create path exists.

First slice of the MegaTask umbrella enforcement (branchless guard).

* feat(batch): branchless umbrella guard across the git-exemption sites

A MegaTask umbrella does no git of its own — every git-exemption site in
TaskService now consults the shared is_branchless_coordination predicate
instead of an inline product-only check, so the umbrella's exemptions
cannot drift between sites:

- the claimed->in_progress branch gate (GitContext.is_coordination) lets
  an unbranched umbrella reach in_progress and delegate;
- _ensure_branch_for_task short-circuits an umbrella to "" instead of the
  misconfigured raise (the claim path ignores the return, treating it as
  branchless);
- CEO-reject routing sends a rejected umbrella to the Main PM in PENDING
  (needs_revision is developer-claim-only and would deadlock it).

Covers both shapes via the predicate (product fan-out root OR umbrella);
a batch root-subtask keeps its own branch/PR. Adds orchestrator
recognition tests for the umbrella plus claim/branch/reject integration
tests.

* feat(batch): umbrella assembles no PR; completes branchless

submit_root now hard-rejects a MegaTask umbrella up front (a preflight
that also folds in the unknown-role refusal to stay within the
return-count budget): the umbrella spans many projects with no single
master, so each root-subtask opens and is reviewed on its own PR — the
umbrella never enters the in-path review gate. The Main PM completes it
directly once every root-subtask is terminal.

Umbrella completion needs no new code: it is branchless (no branch_name),
so _main_pm_complete_guard already accepts it from in_progress, checks
all_subtasks_terminal, and main_pm_complete walks it to awaiting_pm_review
and escalates to the CEO with no PR creation — exactly the product
fan-out root path. Adds the submit_root-reject and umbrella-completion
gateway tests; pins batch_id=None on the normal-root submit_root test
(a MagicMock auto-attr would otherwise read as an umbrella).

* feat(batch): MegaTask create path — umbrella + sequenced root-subtasks

PrompterService.confirm_live_batch turns N confirmed drafts into a real
MegaTask: it builds each draft's collision surface, runs the pure
SequencingService to get conflict-free waves, creates the branchless
umbrella (batch_id, no project/product), then one root-subtask per draft
(own project, parent=umbrella, sequence=wave index, descriptors), and
wires the analyzer's edges through add_dependency so the existing
dependency-gate runs the waves in order. The route picks the start path
like a single confirm: 'board' holds the root-subtasks in BACKLOG for the
batch review; 'main_pm' creates them PENDING so wave 0 dispatches at once.

create_task_from_draft gains a BatchPlacement (parent/batch/sequence/
team_override) and forwards the collision descriptors; the exactly-one-
target rule (here and the TaskService.create invariant) is relaxed for an
umbrella, which legitimately targets neither. New route
POST /live/{session}/confirm-batch + BatchConfirmRequest mirror the single
confirm. Adds the structural-invariant + board-hold + empty-batch tests.

* feat(batch): release MegaTask root-subtasks on CEO approval; board awareness

The board route holds a MegaTask's root-subtasks in BACKLOG so the work
waits for the batch review. approve_and_start (CEO gate #1, board->Main PM)
now releases them via _activate_batch_root_subtasks: each held child flips
BACKLOG -> PENDING + team=main_pm so the dependency-gate dispatches wave 0.
No-op for a non-umbrella; idempotent (children past BACKLOG untouched).

The Product Owner and Head of Marketing identity prompts gain a MegaTask
section so they review the whole batch + wave plan and adjust scope before
sign-off (they review drafts; the umbrella is their unit). Also extracts
the create() target invariant into _require_target_or_umbrella to keep the
method under the complexity gate after the umbrella exemption. Adds the
umbrella-approval activation test.

* feat(batch): multi-project intake scope for MegaTask

A MegaTask spans several possibly-unrelated repos, so the intake chat can
now be scoped to an explicit project list (not just one project or one
product). StartLiveRequest gains project_ids; /live/start threads it
through start/spawn_intake_session -> _spawn_intake_container ->
_clone_intake_scope. The multi-repo clone machinery already existed for
products; _intake_scope_slugs now also resolves an explicit project_ids
set (split into _slugs_for_project_ids / _slugs_for_product), cloning each
repo with the first as the primary cwd and the siblings readable. Scope
validation is now 'exactly one of project_slug / product_id / project_ids'
via the shared _require_one_intake_scope. Adds scope-resolution, spawn,
and route tests for the MegaTask path.

* feat(batch): propose_batch intake tool (MegaTask multi-draft hand-off)

The intake agent can now hand the panel a whole MegaTask in one tool call.
Both intake paths gain propose_batch alongside propose_draft:
- Claude (intake_driver): a propose_batch tool registered on the in-SDK
  MCP server + allowlisted; the driver intercepts the ToolUseBlock and
  emits ONE StreamChunk(kind="batch") carrying {drafts:[...], title}.
- grok (intake_server): a propose_batch tool that POSTs a "batch" relay
  event via the shared _post_event helper (post_draft/post_batch).

A batch carries N drafts, each the propose_draft shape PLUS its own
project_id (a MegaTask spans unrelated repos) and collision surface so the
analyzer sequences the waves. The prompter prompt documents the MegaTask
scope + when to call propose_batch. Adds Claude-normalize and grok-relay
tests for the batch path.

* feat(batch): MegaTask intake panel — third scope, batch review, waves

The panel now drives a MegaTask end to end. The intake modal gains a
third scope, 'MegaTask', beside Single cell and Board-led: a multi-project
checklist (a MegaTask spans several possibly-unrelated repos), validated
to at least two. start() sends project_ids; use-prompter accumulates the
agent's single propose_batch hand-off as a 'batch' SSE event into a
BatchProposal and lands in a new batch_preview state.

A new BatchReviewCard lists every proposed task with its target project +
collision-surface badges (migration / shared) and offers one start path
for the whole batch — Board review & Start or Approve & Start — wired to
confirmBatch → POST /confirm-batch. The success card shows the sequenced
result: N tasks in M waves (+ any advisory notes). prompter.ts gains the
DraftScale 'megatask' + the BatchConfirm payload/result types; the SSE
client allows the 'batch' kind. Panel typecheck + lint + 113 tests green.

* docs(batch): MegaTask across changelog, CLAUDE.md, site, and RAG

The four documentation obligations for the MegaTask feature:
- CHANGELOG: an Unreleased entry covering the umbrella model, sequencing,
  multi-project intake, propose_batch, and the create/approval path.
- CLAUDE.md: a MegaTask section (identity predicate, umbrella/root-subtask
  hierarchy, sequencing rules, intake + create path, board activation).
- Published site: a user-facing company/megatask.md (scopes, waves, the
  umbrella, the two start buttons) + nav entry; a pointer added to the
  intake chapter of the Tour.
- RAG corpus: workflows/megatask.md so the Main PM (and any agent) can
  retrieve the umbrella's branchless / no-PR / completion rules at runtime.

The runtime concurrent-migration guard is intentionally NOT added: the
analyzer already chains migration-adders into dependencies and the
dependency-gate serializes them, so a separate guard would be dead code.

* feat(batch): batch_id guardrail + wave preview + batch_id on TaskResponse

Guardrail (CEO): a batch_id is denied on any task that is not a well-formed
MegaTask member. is_valid_batch_shape permits batch_id only on an umbrella
(no parent → must target neither project nor product) or a root-subtask
(has a parent → exactly one target); TaskService.create enforces it AND
verifies a root-subtask's parent is the batch umbrella (same batch_id,
top-level). This closes a latent hole: is_batch_umbrella is true for a
batch_id + no-parent task even with a project, so a stray batch_id could
have spoofed the branchless branch-gate / no-PR exemption. (The public
task API never exposed batch_id for write; this guards the service layer.)

Wave preview: PrompterService.preview_batch + POST .../preview-batch
compute a MegaTask's waves from the proposed drafts WITHOUT creating
anything, so the panel can show the sequencing before confirm. Extracted
_sequence_drafts as the single source shared by preview and confirm, so
the previewed waves are exactly the ones wired.

TaskResponse now carries batch_id so the panel can badge the umbrella.

* feat(batch): MegaTask review — project editor, wave preview, persistence, badge

Closes the panel gaps in the MegaTask review experience:
- Per-task project editor: each proposed task gets an inline project
  Select (updateBatchDraftProject), so a task the agent put in the wrong
  or no repo can be fixed before launch — not only by re-chatting. Launch
  stays blocked until every task has a project.
- Wave preview: on a batch proposal the panel fetches POST .../preview-batch
  (no task created) and shows the conflict-free wave plan, so the human
  reviews the sequencing before confirming.
- Refresh durability: the MegaTask review (batch + waves + projectIds) is
  persisted, so a browser reload mid-review restores it like a single draft.
- MegaTask badge: TaskResponse exposes batch_id, the panel Task type
  carries it, and the task table badges the umbrella row 'MegaTask'.

Panel typecheck + lint + 113 tests green.

* test(batch): stub task carries batch_id for task_to_response

task_to_response now serializes batch_id (TaskResponse field), so the
_stub_task SimpleNamespace fixture must provide it — without it the reader
hit AttributeError, failing the 8 task-schema serialization/enrichment
tests. Test-only; the real TaskTable carries the column (migration 046).

* fix(batch): close MegaTask audit gaps — completion crash, analyzer cycle, guardrails

An adversarial multi-agent audit of the feature surfaced 20 verified gaps;
this closes the backend ones.

HIGH:
- Umbrella completion crashed. escalate_to_ceo hard-required a pr_number,
  which a branchless umbrella never has, so main_pm_complete dereferenced
  None. Both pr_number gates now waive a MegaTask umbrella (escalate_to_ceo
  + the awaiting_pm_review->awaiting_ceo_approval lifecycle gate via a new
  GitContext.is_umbrella), and main_pm_complete guards a None return. The
  completion test had mocked escalate_to_ceo, hiding it — now a real
  service test covers the waiver.
- The collision analyzer could fabricate a cycle (a touches_shared +
  adds_migration draft overlapping another migration draft) and raise
  SequencingError — a bare ValueError that escaped as an opaque 500. The
  migration chain is now shared-last-aware (never contradicts rule 3), and
  _sequence_drafts translates SequencingError to a clean 400.

MEDIUM:
- Collisions are now project-scoped: two repos can't collide on a
  coincidental path or serialize independent migrations (DraftSurface
  carries project_id; rules 1/2/3 respect it).
- The batch_id guardrail ran only at create. update() + the PATCH
  null-clear path now re-assert is_valid_batch_shape, so a mutation can't
  break a member's shape and spoof the branchless exemption.
- A draft missing title/acceptance_criteria now raises ValidationError
  (was a bare KeyError -> 500).
- confirm_live_batch re-asserts every draft targets a scoped project and
  the batch spans >=2 distinct projects (project_ids added to the request).
- Route-level tests for confirm-batch / preview-batch.

LOW: strict multi-repo clone (fail loud on any unresolvable project);
malformed/empty propose_batch surfaces an error chunk (Claude) / refuses
to POST (grok) instead of silently acking; dropped malformed drafts are
counted and surfaced; stale grok intake docstrings updated.

* fix(batch): MegaTask panel + doc audit gaps

Frontend half of the audit fixes:
- The confirm payload now carries project_ids (the schema requires it), and
  the panel re-checks every task targets one of the scoped repos before
  launching, naming the offending task.
- The Review-MegaTask project picker is filtered to the scoped repos and
  the per-task validity (border + launch gate) keys off scoped membership,
  so a task can only be (re)pointed at an in-scope project — also fixing the
  case where the agent emitted a non-UUID / unknown project.
- Dropped malformed drafts are surfaced as a chat error so the human knows
  the batch shrank instead of silently confirming fewer tasks.
- Doc wording: a wave releases on the previous wave's terminal state
  (normally a merge; a cancellation releases it too), not strictly 'merged'.

* test(batch): lock the CEO's EXACT 4-wave hand-sequencing as the golden bar

The golden test asserted the constraints (S6 last, the migration chain, the
shared-threats serialization, S1/S2/S7 parallel) but not the full wave
partition. The bar for MegaTask is 'reproduce my exact waves or it's not
done', so assert the exact 4-wave partition the analyzer produces for the
guard-core-app batch:
  wave 1: R1 R2 S1 S2 S3 S5 S7  ·  wave 2: R3  ·  wave 3: R4 S8  ·  wave 4: S6
Confirmed unchanged by the audit's analyzer fixes (no migration is shared;
single project).

* fix(batch): tolerate a stub task in assert_batch_shape_intact

The batch-shape re-validation read task.batch_id directly, but update()'s
partial-caller contract is exercised with a SimpleNamespace stub that has no
batch_id column → AttributeError. Use getattr(..., None) for batch_id and the
shape fields so the guard no-ops on any task lacking the column (a stub, or a
non-batch task) while still enforcing on a real batch member.

* fix(orchestrator): authenticate internal API self-calls with the system identity

The dispatcher httpx clients were built without an agent identity, so the
orchestrator's self-PATCHes to /api/tasks/{id} (auto-block, auto-resume,
auto-recover, SLA annotation) were rejected 401 "Missing X-Agent-ID" and
silently no-op'd. The auto-resume that lifts a PM's paused parent could never
write, so paused/blocked parents stayed wedged and stranded their dependents
(the fe-pm/be-pm respawn churn seen in prod).

Header propagation was inconsistent across the separate AsyncClient call-sites:
only the main dispatch client carried the system identity; the readiness and
sweep clients did not. Hoist the identity into a shared _SYSTEM_API_HEADERS
constant and apply it to every API-facing dispatcher client. The system role
holds TaskAction.ASSIGN, so it is authorized for the audited admin_set_status
path those write routes use. The external provider-recovery probe client is
intentionally left untouched.

---------

Co-authored-by: Renn F <rennf93@users.noreply.github.com>
2026-06-24 01:15:57 +02:00
Renn F 77c94b74da docs(prompts): announce the note-section obligations to agents
The obligations added in 8cf69781 (dev_notes@i_am_done, quick_context@delegate,
pr_reviewer_notes@pr verbs, auditor@i_am_idle) were only discoverable at
runtime via the gate's remediate field. Surface them upfront so agents satisfy
them on the first call instead of looping into a tracing_gap:

- base.md: the gap-key reference gains rows for dev_notes>=min /
  quick_context>=min / pr_reviewer_notes>=min (parity with the journal rows).
- developer.md: a note(scope='handoff') step before i_am_done, and dev_notes
  added to the i_am_done precondition list.
- cell_pm.md / main_pm.md: fill quick_context (done+next) before the first
  delegate (it persists across the whole queue).
- auditor.md: must record an observation before i_am_idle.

The pre-write cases (dev_notes, quick_context, auditor) carry the real loop
risk; pr_reviewer / doc notes are satisfied by the verb's own argument, so the
base.md row alone suffices for those.
2026-06-21 20:45:29 +02:00
62bda0c497 Gateway/full (#9)
* chore(gateway): scaffold gateway package and test layout

* feat(config): add gateway feature flags, coordination thresholds, commit-validator settings

* feat(gateway): add standardized response envelope with ok/error variants

* feat(gateway): add remediation hint catalog for tracing-gap and invalid-state errors

* feat(gateway): add per-role flow/do tool catalog with developer, qa, doc, pm, board configs

* feat(db): add gateway columns — active_claimant_id, heartbeat, pre_block snapshot, acceptance_criteria_status, qa_evidence_inspected

* feat(db): create gateway_triggers table for dispatcher decision logging

* feat(db): align canonical skill set; substitute qa_review -> code_review across agent seeds

* fix(db/008): make skill alignment in-place + idempotent; preserve column and existing custom skills

* feat(gateway): add claimant_lock for single-active-agent invariant with heartbeat staleness

* feat(gateway): add trigger_filter with stale-cleanup, claimant-queue, and cooldown rules

* feat(gateway): add tracing_gate with plan, progress, journal, acceptance_criteria, qa requirements

* refactor(gateway): drop per-file ruff ignores; refactor tracing_gate with dispatch table + GateContext

* feat(gateway): add merge_chain to resolve PR target by branch hierarchy depth

* feat(gateway): add commit_validator with min-length, banned-words, and conventional-shape hints

* feat(gateway): add evidence_builder for verb-response evidence and capped context_briefing

* feat(gateway): add Choreographer skeleton with per-phase verb signatures and DI protocols

* feat(runtime): add spawn_manifest builder for per-role pre-loaded tool registration

Introduces SpawnInputs dataclass + build_for_role(inputs) + write_manifest()
in roboco/runtime/spawn_manifest.py; reads role_config for allowed verbs/tools,
emits JSON manifest that SDK shim reads at container startup to eliminate ToolSearch.

* feat(runtime): wire gateway pre-spawn check (trigger_filter + claimant_lock) into orchestrator behind ROBOCO_GATEWAY_ENABLED flag

- Add GatewayTriggerTable SQLAlchemy ORM model to roboco/db/tables.py
  (matches existing table from migration 007_gateway_triggers_table)
- Add module-level gateway_pre_spawn_check() + helpers to orchestrator.py
  (gated: returns ("spawn", "gateway disabled") immediately when flag is False)
- Wire gateway check into _safe_spawn() — the single dispatcher choke-point
  for all agent spawns; QUEUE or DROP outcome logs and returns None (no spawn)
- ROBOCO_GATEWAY_ENABLED defaults to False; legacy behaviour is unchanged

* feat(agent_sdk): load tool-manifest.json at startup behind ROBOCO_GATEWAY_ENABLED flag (no agent-visible change yet)

Adds load_tool_manifest() to the SDK server that reads env at call-time
so gateway-enabled agents can obtain their pre-registered tool list at
startup; returns None when the flag is off, leaving the legacy briefing
path completely unchanged.

* fix(optimal_brain): skip indexing when source ID is None to eliminate roboco://journals/None spam

- Add `build_doc_source(kind, id_)` module-level helper in indexes/base.py that
  returns None when id_ is None instead of producing a "roboco://journals/None" URI
- Update abstract `build_source_uri` return type to `str | None` so subclasses
  can legitimately signal a missing ID
- Short-circuit `ingest()` and `_prepare_docs_for_batch()` in BaseIndexPlugin
  when `build_source_uri` returns None (debug log, no push to vector store)
- Fix JournalsIndexPlugin.build_source_uri: `kwargs.get("entry_id")` returns the
  kwarg value even when it is None, so fall back to doc_id before calling
  build_doc_source
- Fix ConversationsIndexPlugin.build_source_uri: return None when session_id is
  None rather than producing "roboco://conversations/None-unknown"
- Add 9 unit tests with a piragi-free conftest that stubs sys.modules

* fix(agent_sdk): inject X-Agent-ID header on notification-poller requests

Both `_check_pending_a2a` and `_auto_ack_a2a_notifications` in
`roboco/mcp/a2a_server.py` were calling the main API without identity
headers, causing orchestrator `Missing X-Agent-ID header` warnings on
`GET /api/v1/notifications/pending-a2a` and the ack-a2a POST.

Add module-level `AGENT_ROLE` constant (mirrors the existing `AGENT_ID`
pattern) and pass `{"X-Agent-ID": AGENT_ID, "X-Agent-Role": AGENT_ROLE}`
on both requests.

* fix(git): use ROBOCO_PUBLIC_BASE_URL for commit-trailer Links instead of hardcoded localhost

* fix(test_runner): call uv run pytest/ruff directly; add make to orchestrator Dockerfile as backstop

FileNotFoundError was propagating as a raw 500 when a project had `make test`
configured but make was not installed in the orchestrator container.

Two fixes:
1. Catch FileNotFoundError in _run_command and re-raise as ValidationError (400)
   with a clear message telling the operator to reconfigure the project command
   (e.g. replace 'make test' with 'uv run pytest').
2. Add `make` to the orchestrator Dockerfile runner-stage apt-get so projects
   that legitimately use make targets continue to work without reconfiguration.

* fix(api/git): resolve project by slug or UUID in git_log endpoint

Add _resolve_project_slug() helper to git routes that tries UUID
lookup first and falls back to slug, matching the pattern already
used in project routes. Apply to all four read-only git endpoints:
status, log, branches, diff.

* fix(a2a): auto-create conversation when conversation_id absent; reject empty IDs in URL builder

* fix(agent_sdk): default subagent model to parent agent's model from spawn manifest, not hardcoded haiku

Inject CLAUDE_CODE_SUBAGENT_MODEL env var into every agent container at
spawn time.  Claude Code ≥2.1.x reads this variable to override the
default Task (Agent) subagent model, which otherwise hard-codes
claude-haiku-4-5-20251001.  When the parent runs on a non-Anthropic
provider (e.g. Ollama Cloud / minimax-m2.7:cloud) that Anthropic model
is unreachable, so subagent dispatch fails.

The value follows the same provider-aware translation already used for
the --model CLI flag: Anthropic short names go through MODEL_MAP, and
non-Anthropic identifiers are passed verbatim.  To avoid calling the
class by name inside a @staticmethod, the shared translation logic is
extracted to the module-level _resolve_agent_cli_model() helper;
_resolve_cli_model() now delegates to it.

Verified: CLAUDE_CODE_SUBAGENT_MODEL is present and honoured in the
Claude Code 2.1.123 binary (grep confirmed the env-var lookup pattern
`if(process.env.CLAUDE_CODE_SUBAGENT_MODEL) return KK(…)`).

* chore(makefile): add quality and quality-fast targets composing every PR gate

* chore(quality): add import-linter dependency and gateway boundary contract

* test(property): scaffold tracing-completeness assertion (filled in Phase 4)

* Format test file to pass ruff check

* fix(gateway): drop Protocol scaffolding from choreographer skeleton; del-statements on unused stub args; clear vulture whitelist

* linting

* feat(gateway): Phase 1 dev cutover — ChoreographerDeps + give_me_work

Add ChoreographerDeps frozen dataclass (7 deps: task, work_session, git,
a2a, journal, audit, evidence_repo), refactor Choreographer.__init__ to
accept the bundle, implement give_me_work + _briefing_for via
evidence_builder.build_context_briefing, and add property accessors for
all deps. All Phase 2-4 stubs gain del-statements and still raise
NotImplementedError. 3 tests added and passing; mypy/ruff/vulture clean.

* feat(gateway): implement i_will_work_on handling pending, claimed, and needs_revision recovery

* feat(gateway): implement i_have_committed with plan-required precondition

Replaces the NotImplementedError stub with the real implementation: looks up
the agent's active task, enforces plan presence before recording, calls
task.add_progress, and returns a structured Envelope. Adds 3 unit tests
(records progress, no active task → invalid_state, no plan → tracing_gap).

* feat(gateway): implement i_am_done with smart catch-up and skill resolution

* feat(gateway): implement i_am_blocked (struggle + escalate) and i_am_idle (with unread soft-block)

* feat(gateway): add ContentActions for commit, note, say, dm, evidence with auto-inject and validation

* feat(api/v2): add /api/v2/flow/dev/* endpoints delegating to Choreographer

Six intent-verb endpoints (give_me_work, i_will_work_on, i_have_committed,
i_am_done, i_am_blocked, i_am_idle) under /api/v2/flow/dev/, each a thin
handler that delegates to Choreographer. Includes Pydantic request schemas,
EvidenceRepo Phase 1 stub (all methods return []), get_choreographer FastAPI
dep wired with all 7 service deps, and 8 unit tests (all passing).

* feat(api/v2): add /api/v2/do/* endpoints for commit, note, say, dm, evidence

* feat(mcp): add roboco-flow MCP server for intent verbs (Phase 1: dev verbs implemented)

* feat(mcp): add roboco-do MCP server for smart-wrapped content tools

* feat(runtime): mount per-agent tool-manifest.json on developer-container spawn; gateway flag enabled for devs only

* docs(prompts): rewrite developer role prompt for gateway-only verbs (~15 lines vs 49)

* chore(mcp): confirm dev manifest excludes legacy task/journal/notify/a2a tools (Phase 1 cutover; servers retired in Phase 4)

* feat(gateway): implement claim_review with inline evidence (kills #15) and qa_evidence_inspected tracking

* feat(gateway): implement pass_review with qa_notes/learning/evidence tracing gates

* feat(gateway): implement fail_review with issue list, tracing gates, and dev A2A handoff

* feat(api/v2): add /api/v2/flow/qa/* endpoints (claim_review, pass, fail, give_me_work, i_am_idle)

* feat(mcp): add QA verbs (claim_review, pass, fail) to roboco-flow MCP server

* docs(prompts): rewrite QA role prompt for gateway verbs; explicitly warn against grep-the-commit anti-pattern

* feat(runtime): enable gateway flag for QA-role spawns (Phase 2 cutover)

* feat(gateway): implement claim_doc_task and i_documented with file-list and notes-min-chars gates

* feat(gateway): implement triage (cell PM) and triage_all (main PM) with priority order

* feat(gateway): implement unblock with pre_block_state restoration (kills #23)

* feat(gateway): implement cell_pm_complete with auto-merge to parent branch (kills #22 for cell scope)

* feat(gateway): implement main_pm_complete (open master PR + escalate to CEO)

* feat(gateway): add complete() dispatcher routing to cell_pm_complete or main_pm_complete by role

* feat(gateway): implement escalate_up routing by role.escalation_target

* feat(api/v2): add /api/v2/flow/{documenter,cell_pm,main_pm}/* endpoints

* feat(mcp): add Doc + PM verbs to roboco-flow MCP server (claim_doc_task, i_documented, triage, triage_all, unblock, complete, escalate_up)

* docs(prompts): rewrite Doc, Cell PM, and Main PM role prompts for gateway verbs

* feat(runtime): enable gateway flag for Doc, Cell PM, and Main PM roles (Phase 3 cutover)

* test(integration): full pending->awaiting_ceo_approval test through dev/QA/doc/cell-PM/main-PM gateway path

* chore(tests): rename unused args to _args in flow_server tests

Cleared RUF059 lint blocker for Phase 3 closeout. The destructured args was only consumed in URL-asserting tests; one variant only checks kwargs["json"], so its args is now _args.

* chore: untrack docs/superpowers/ + add to .gitignore

Plans + spec were inadvertently swept into commits 5d41a4b and de0c5b5 by subagent 'git add -A' calls. Removed from index and gitignored going forward; files remain on disk for ongoing reference. They still exist in history of those two commits — invoke a follow-up filter-repo if a full purge is desired.

* feat(gateway): implement Board escalate_to_ceo with role allow-list

Allows main_pm, product_owner, and head_marketing to escalate tasks to
CEO. Enforces awaiting_pm_review state and journal:decision tracing gate.
Closes Phase 4 Task 1.

* feat(gateway): implement board_triage prioritizing strategic root tasks

Adds Choreographer.board_triage and TaskService.list_strategic_for_board.
PO and Head Marketing get curated lists of strategic-nature root tasks
in awaiting_pm_review. Closes Phase 4 Task 2.

* feat(gateway): implement auditor_triage surfacing long-running blocked-task anomalies

Adds Choreographer.auditor_triage and TaskService.list_long_running_blocked.
The Auditor surfaces tasks blocked >30min as anomalies for reflect-note
observation. Closes Phase 4 Task 3.

* chore(tests): add return + arg type annotations to gateway tests

All gateway test functions now have -> None and parameter annotations. Cleared 63 mypy [no-untyped-def] errors that pre-existed since Phase 1. Mypy now clean across tests/unit/gateway/.

* feat(api/v2): add /api/v2/flow/{board,auditor}/* endpoints

Board: triage, escalate_to_ceo, i_am_idle.
Auditor: triage, i_am_idle (read-only role).
Adds EscalateToCeoRequest schema with reason min_length validation.
Closes Phase 4 Task 4.

* feat(mcp): add Board + Auditor verbs to roboco-flow MCP server

Adds escalate_to_ceo MCP tool used by Board (PO + Head Marketing) and Main PM. Updates the implemented set in _validate_role_compatibility. Auditor uses the existing triage tool with role-routing in URL.

Closes Phase 4 Task 5.

* docs(prompts): rewrite Board (PO, Head-Marketing, Auditor) prompts for gateway verbs

All 3 board identity files + roles/board.md now use the slim, gateway-aware shape (no ToolSearch directive, no state-tool table). Auditor is explicit about its read-only scope. Closes Phase 4 Task 6.

* feat(runtime): enable gateway manifest for ALL roles (Phase 4 cutover)

Adds product_owner, head_marketing, auditor to GATEWAY_ENABLED_ROLES. Every spawned agent now gets a gateway manifest mounted at /app/tool-manifest.json. The legacy briefing path is dead. Closes Phase 4 Task 8.

* test(property): implement tracing-completeness assertion across smoke-test batch

Replaces Phase 0 stub. Asserts the 6 tracing-contract requirements on every
completed task: audit_log agent_id non-null per state-transition row,
DEVELOPER:TASK_REFLECTION journal entry, QA:LEARNING journal entry,
CELL_PM/MAIN_PM:DECISION_LOG journal entry, acceptance_criteria_status
covering every criterion with a referencing_artifact_id, and
qa_evidence_inspected = true.

Uses an in-memory ephemeral Postgres test DB (`roboco_test_<pid>_<rand>`)
provisioned per pytest session, not SQLite — the production schema relies
on Postgres-only types (UUID, ARRAY) the SQLite dialect cannot compile.
Tests requesting db_session/smoke_test_batch are auto-skipped when no
Postgres is reachable on localhost:5432; ROBOCO_TEST_DB_HOST/PORT/USER
override the endpoint.

Schema is built via Base.metadata.create_all + manual ALTER for the
acceptance_criteria_status / qa_evidence_inspected columns, NOT via
`alembic upgrade head`. This sidesteps two pre-existing layer-drift items
that block any fresh migration run today:

  1. Migration 001 declares the agentrole Postgres enum with lowercase
     values (qa, developer, ...) but the SQLAlchemy ORM binds
     Enum(AgentRole) to the StrEnum's uppercase NAMES — production DBs
     mask this by being bootstrapped via create_all and stamped at 001.
  2. Migration 008 runs UPDATE agents SET skills WHERE id over an
     agents.skills column that no migration in this chain ever creates.

Documented in conftest.py so a future migrations cleanup can find them.
Also notes that acceptance_criteria_status/qa_evidence_inspected are in
the DB schema (per migration 006) but are NOT mapped on the ORM TaskTable
nor on the Pydantic Task model — services that read them via
`task.qa_evidence_inspected` rely on those values being set on raw rows.
The property test uses raw SQL to read the columns directly, matching the
DB-level contract.

Closes Phase 4 Task 11.

Side change: pyproject.toml — adds asyncpg.* to the existing
[[tool.mypy.overrides]] ignore_missing_imports list (asyncpg ships no
py.typed marker), matching the convention used for redis, anthropic,
piragi, etc.

Test count: 1; backend: Postgres (localhost test DB).

* style(mcp/flow_server): single-line _post call after format pass

* fix(db): map 7 gateway columns from migration 006 to TaskTable + Task model

active_claimant_id, last_heartbeat_at, pre_block_state, pre_block_assignee, pre_block_metadata, acceptance_criteria_status, qa_evidence_inspected: present in DB since migration 006 but absent from the ORM mapping. Gateway code (tracing_gate, choreographer, claimant_lock) reads these via task.<attr>; without the mapping, runtime would AttributeError. Closes PHASE4-BUG-A.

* fix(db): repair alembic chain — neutralize 008, add 009 enum reconcile, ORM uses values_callable

Three coordinated changes that close PHASE4-BUG-B:

1. roboco/db/tables.py — introduce _str_enum() helper that wraps Enum() with values_callable=lambda obj: [m.value for m in obj]. Apply to all 23 StrEnum-typed mapped columns. ORM now serializes by .value (lowercase) to match alembic 001's declared enum values; default Enum() was using .name (uppercase) which never matched.

2. alembic/versions/008_align_skills.py — replace with documented no-op. The original migration referenced agents.skills, a column that has never existed in any migration (the agents table has capabilities, not skills). The substitution intent (qa_review -> code_review) was already satisfied statically in roboco/agents_config.py.

3. alembic/versions/009_enum_reconcile.py — new migration that:
   - Adds missing enum values: agentrole.system, team.fullstack, taskstatus.quarantined.
   - Detects uppercase drift from a Base.metadata.create_all bootstrap and rebuilds agentrole/team/taskstatus enums with lowercase members + USING lower(col::text)::enum on every column referenced. No-op if already lowercase.

Tests stay green: 281 passed.

* feat(services): backfill 36 gateway-shaped methods for Choreographer

The gateway Choreographer was wired to call methods that the underlying
services did not expose. This adds them as thin wrappers + queries (most
alias canonical methods; a handful are gateway-specific variants).

TaskService — 26 methods: aliases (submit_verification, submit_qa,
list_blocked_for_team, list_blocked_all_teams,
list_awaiting_pm_review_for_team, list_assigned_for_agent), agent
queries (agent_for, qa_agent_for_team, documenter_for_team,
cell_pm_for_team, get_active_task_for_agent, list_paused_for_agent),
triage queries (list_awaiting_main_pm_all, all_subtasks_terminal),
state setters (set_plan, mark_evidence_inspected, mark_agent_idle),
QA/Doc claim variants (qa_claim, doc_claim, qa_pass, qa_fail),
PM completion (cell_pm_complete with merge_commit), unblock with
state restore (unblock_with_restore), and escalation
(escalate, escalate_up_to_role). Also adds GatewayAgentView
dataclass that unifies DB and config-derived agent attributes.

JournalService — 4 methods: existence checks (has_decision_for_task,
has_learning_for_task, has_reflect_for_task) + write_struggle.

GitService — 4 methods: branch-keyed entry points (create_pr,
pr_merge, pr_target, diff) plus push_branch helper. Each derives
project + workspace from the task that owns the branch / PR.

WorkSessionService — 2 methods: files_changed + has_unpushed_commits.
PR existence is the proxy for pushed (no per-commit push column).

Choreographer: switched git.push(branch_name) call to push_branch()
to dispatch to the new gateway-shaped helper.

* test(services): unit tests for 36 gateway-backfill methods

Adds happy-path + edge tests for every method added in the prior
backfill commit. Total 61 new tests across:
  - tests/unit/services/test_task.py (36)
  - tests/unit/services/test_journal.py (8)
  - tests/unit/services/test_git.py (10)
  - tests/unit/services/test_work_session.py (7)

Each test mocks at the session boundary (no DB) and stubs adjacent
service methods via a dynamic _bind helper to avoid mypy
[method-assign] noise without resorting to type:ignore comments.

* test(gateway): switch dev catch-up assertion to push_branch

The Choreographer's catch-up sequence was renamed from git.push(branch)
to git.push_branch(branch) when GitService got a gateway-shaped helper
in the prior commit. This updates the existing assertion to match.

* feat(mcp): add roboco-git-readonly server with status/log/diff/branches

Slim FastMCP server exposing the four read-only git tools every role
needs (status, log, diff, branch_list) by forwarding to /api/v1/git/*
on the orchestrator. Replaces the read-only half of the legacy
roboco-git server; write operations now go through gateway verbs in
roboco-flow / roboco-do.

The endpoint shapes mirror the panel-facing API (project_slug,
include_remote, staged/file_path) so the same backend handlers serve
both human and agent traffic.

* refactor(mcp): delete legacy task/journal/notify/a2a/message/project servers

Phase 4 cutover: agents now reach every state-changing surface through
the gateway (roboco-flow intent verbs + roboco-do content tools), with
roboco-git-readonly + roboco-optimal + roboco-docs covering reads. The
seven legacy MCP servers + their handler trees are dead code from the
agent side, so they're removed:

  roboco/mcp/task_server.py            (1020 LOC)
  roboco/mcp/journal_server.py         (512 LOC)
  roboco/mcp/notify_server.py          (440 LOC)
  roboco/mcp/a2a_server.py             (790 LOC)
  roboco/mcp/message_server.py         (682 LOC)
  roboco/mcp/project_server.py         (667 LOC)
  roboco/mcp/tasks/  (handlers+utils)  (~4300 LOC)
  roboco/mcp/test/   (in-container runner; replaced by gateway evidence
                      + manual smoke)
  roboco/mcp/git/    (full server; read-only half migrates to the new
                      slim roboco-git-readonly module, write half is
                      owned by gateway verbs)

Orchestrator updates:
  - _generate_mcp_config registers only roboco-flow, roboco-do,
    roboco-git-readonly, roboco-optimal, and (for docs roles) roboco-docs.
    No more per-role legacy fan-out.
  - base_allow flips to mcp__roboco-flow__*, mcp__roboco-do__*,
    mcp__roboco-optimal__*, mcp__roboco-git-readonly__*. Role-specific
    allow lists are reduced to file IO scoping, since gateway verbs
    enforce role policy server-side.
  - TRACEABILITY_TRIGGER_TOOLS rewritten in terms of the gateway servers
    (mcp__roboco-flow__* / mcp__roboco-do__*) instead of the now-deleted
    per-tool list.

Test fix: tests/unit/services/test_a2a.py imported _handle_send_chat_message
from the deleted a2a_server. The four MCP-layer URL-builder tests (empty
conversation_id guard) are dropped — the equivalent boundary now lives
in /api/v2/do/* which has its own integration coverage. The two
service-layer nil-UUID guard tests are kept; they exercise A2AService
directly and remain meaningful (the panel still uses the v1 chat surface,
where a buggy caller could pass the nil UUID).

Net: ~9600 LOC removed from roboco/mcp/. quality-fast green:
338 tests pass, mypy clean, ruff clean. No /api/v1/* router changes —
those endpoints stay live for the panel UI which still uses every
lifecycle action; agents have no prompts that name them so the path is
dead code from the agent side.

* docs(claude.md): replace legacy MCP listing with gateway/verb-surface section

Phase 4 cutover: agents go through roboco-flow + roboco-do (gateway), not the deleted task/journal/notify/a2a/message/project servers. Document the verb surface per role + the Envelope response shape so future Claude Code sessions land in the correct mental model. Closes Phase 4 Task 13.

---------

Co-authored-by: Renn F <rennf93@users.noreply.github.com>
2026-05-02 03:11:49 +02:00
Renn F fc55068f2b feat: add Kubernetes manifests and A2A protocol support
Phase 2 - A2A Protocol:
- Add A2A models (AgentCard, Task, Message)
- Add A2A service layer
- Add A2A routes with SSE streaming
- Add agent discovery endpoints

Phase 3 - Kubernetes:
- Add deploy/ directory with Kustomize structure
- PostgreSQL StatefulSet with pgvector
- Redis Deployment with persistence
- API and Orchestrator Deployments
- RBAC for orchestrator to manage Jobs
- ArgoCD Application manifest
- Development and production overlays
- K8s Jobs API support in orchestrator
2025-12-28 19:52:01 +01:00
Renn F 8c3bf9e22c Huge refactoring but stuff is working again; minus some issues here and there. 2025-12-26 18:01:53 +01:00