mirror of
https://github.com/rennf93/roboco.git
synced 2026-08-03 07:23:24 +02:00
e7d7311636b5ceafc241b9bf87d197f2046767a3
7
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
4fb0059556 |
fix: journals/learnings never reached the RAG corpus + git-readonly slug 404s (#339)
* fix(rag): per-index chunk floors — journals and learnings were never indexed The global 200-char garbage floor (sized for code/doc chunks) discarded every templated journal note and most distilled org-memory lessons, silently: ingest returned success with zero chunks, so agent journals and learnings were never retrievable via RAG. IndexConfig now carries a per-type min_chunk_length (journals 40, learnings 80, others unchanged). * fix(mcp): git-readonly tools default project_slug from the container env Agents 404ed /api/git/status with 'Project not found: roboco' — the tools made the LLM supply the slug and six doc examples taught a slug that matches no registered project. The tools now fall back to the ROBOCO_PROJECT_SLUG the orchestrator already injects, and the stale examples are corrected. * feat(rag): startup backfill re-ingests zero-chunk journals and learnings Before the per-index chunk-floor fix, ingest() returned success with chunk_count=0 for undersized content: every historical journal entry and distilled learning below the (then-global) 200-char floor was durably recorded in journal_entries but silently never got a chunks_journals / chunks_learnings row, and no exception meant the existing dead-letter (rag_index_failures) never saw it either. Extends the startup reconcile (roboco/api/app.py _reconcile_rag_indexes) with a new pass: backfill_unindexed_journals (roboco/services/ rag_index_failures.py) queries journal_entries for rows missing from each vector table and re-ingests them through the same live code paths (_reindex_journal_entry / record_learning). Journals and learnings are backfilled independently since a LEARNING entry can clear the (lower) JOURNALS floor while still failing the (higher) LEARNINGS floor — a learning's doc_source is a content hash, not the entry id, so presence there is checked by hashing each candidate the same way LearningsIndexPlugin.record_learning does and batch-querying chunks_learnings for those exact sources. Bounded to 200 rows per pass per boot (converges over restarts on a larger backlog) and best-effort per row (one failure never aborts the pass). Rows still under the current floor are excluded by a length filter in the SELECT so they are never retried forever, and private entries are excluded from the JOURNALS pass exactly like the live indexing path. * test(rag): scope backfill assertions to their own rows --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com> |
||
|
|
536bbb64f3 |
Chore/all/logical gaps sweep (#286)
* release-manager: fencing-token mutex + executor/readiness hardening
Closes the release-mutex TTL race (#17, HIGH) and the remaining
release-manager gaps (#88, #89, #201, #202):
- #17: the release mutex is now acquired with a uuid4 fencing token and
released via Lua compare-and-del; a background asyncio heartbeat
compare-and-expires the TTL ~every 60s while the execute owns the lock,
so a live execute no longer expires and a crashed one auto-releases
<=3000s. A second approve after TTL expiry cannot usurp and rm -rf the
in-flight clone — the fenced first-finally keeps its lock.
- #89: a Redis outage during acquire stays fail-closed (the execute never
runs) but now returns a distinct redis_unavailable result + log so the
CEO sees the cause instead of a false already_in_progress.
- #88: commit_and_push RuntimeError is wrapped into a structured
ReleaseResult(commit_failed) instead of a 500.
- #201: first-release fallback still emits untracked version-ref files as
gaps (no longer silenced by the first-release branch).
- #202: _await_proc awaits proc.wait() after kill() so a timeout cannot
leak a zombie.
TDD: tests/unit/services/test_release_proposal_concurrency.py extends
_FakeRedis with eval/get/expire and pins the fencing/heartbeat/usurper
invariants + the redis_unavailable result.
* PM/code-task creation guard + main_pm coverage + issue carve-out
Closes the creation-time role x task_type gap (the user's explicit example)
and the main_pm delegate hole:
- New pure helper `pm_cannot_own_code(role, task_type, is_issue_resolution)`
in foundation/policy/batch.py — single source of truth. Both PM roles
(cell_pm + main_pm) coordinate; a `code` task assigned/claimed by a PM is
a structural mismatch, EXCEPT a PM taking a code task in needs_revision to
resolve review/QA issues directly (the carve-out).
- Creation-time guard: TaskService.create calls the helper (closes the
create-with-cell-PM-assignee hole the team-based check misses).
- Delegate path + spec claim gate consult the same helper.
`_validate_assignee_task_type` / `_task_type_hint_for` now key on the
Role (CELL_PM OR MAIN_PM), not the cell-PM slug set — closes the
delegate-to-main-pm-as-code hole.
- identity.role_for_uuid_or_none is None-tolerant (treats None as "not a
PM" and proceeds) so a malformed/missing assignee cannot crash the guard.
- prompter.create_task_from_draft reuses the guard at draft-create.
TDD: test_batch.py (helper matrix + carve-out), test_main_pm_code_guard.py
(main_pm coverage), test_delegate_assignee_task_type.py (delegate parity),
test_lifecycle_spec.py (claim gate: rejects PM claiming code from pending,
allows from needs_revision + PM claiming planning + dev claiming code).
* task-service: completion hooks + escalation/cancel/audit hardening
Closes the task-service cluster (#21/#98, #99, #100, #101, #103, #216; #102
verified already-covered, #217 verified already-guarded):
- #21/#98: ceo_approve now closes the work session + triggers completion
hooks before worktree removal (no-op when work_session_id is None), so a
CEO-approved task lands the same close-path as PM-completed.
- #99: apply_escalation routes through the transition validator with an
enumerated escalation exemption (_ESCALATABLE_TO_BLOCKED) instead of an
arbitrary source->BLOCKED write; BACKLOG is refused.
- #100: branchless ceo_reject awaiting_ceo_approval->pending gets a real
spec edge (ceo_reject_to_pool ActionSpec + _STATUS_TRANSITIONS entry) so
future admin-override tightening can't wedge the path.
- #101: revision_count bump is documented as the single chokepoint, with
the pre-block RESTORE path undoing it when restoring a snapshotted
needs_revision (same cycle resuming, not a new rejection).
- #103: cancel cascade surfaces non-terminal orphans instead of swallowing
the role violation.
- #216: _remove_task_worktree_on_terminal escalates recurring FS/permission
failure (audit/notify after N) instead of silent-failing forever.
- #102: pinned in test_verb_runner_midverb_invalid_state.py (committed with
the choreographer cluster) — verb-runner savepoints already surface a
concurrent mid-verb state change as INVALID_STATE.
- #217: submit_for_qa claimed_by guard verified intact.
TDD: test_task.py, test_worktree_cleanup_on_complete.py,
test_escalation_board_guard.py (#99), test_task_service_* integration,
test_lifecycle_spec.py.
* choreographer: gate-claim guards + pr-gate hardening + fail-open logging
Closes the choreographer cluster (#5/#222, #29, #30, #82, #188, #189,
#192; #157/#187 verified already-fixed/pinned; #102 pin lives here):
- #5/#222: the unchanged-PR guard's fail-open head_sha lookup now logs
(warning) on a slug-resolver/git-helper error so a regression cannot
silently turn the pr_fail re-submit loop-stopper into a no-op. Stays
fail-open (never wedges the PM).
- #29: pinned (REFUTED-with-pin) — _lane_claim_guard already returns the
error envelope without releasing the claim on a transient lookup error.
- #30: pinned (REFUTED-with-negative-pin) — a non-batch branchless main_pm
root cannot bypass the complete spec gate (is_batch_umbrella requires
batch_id set).
- #82: _post_gate_review_to_pr wraps the slug-resolution call in try/except
(mirrors _capture_pr_head_sha) so a malformed cell_map AttributeError no
longer 500s the reviewer after a committed gate transition.
- #188: _is_hand_formatted_verdict anchors the header regex to line-start,
so a quoted (> ## Summary) or inline (mid-prose) header mention no longer
false-refuses a hand-formatted verdict.
- #189: pr_fail re-captures the PR head SHA after the transition commits and
re-stamps the verdict note only when it advanced (closes the stale-SHA
false-allow loop-hole); no-advance stays a single note write.
- #192: claim_gate_review skips the dev claim guards (already_active/paused/
lane) via a new skip_dev_guards param — a pr_reviewer inspecting an
assembled PR does not start work, so the single-active-task / code-lane
invariants do not apply; the dependency guard is kept, and QA's
claim_review parity is preserved.
- #157/#187: verified in tree — pr_review-only handoff is intentionally
prior-work-worth-resuming; self_review_block wiring (reviewer != dev)
holds on assembled tasks with 4 existing pin tests.
TDD: test_choreographer_*, test_pr_gate_posts_review (#82),
test_pr_review_hand_format_guard (#188), test_submit_root_unchanged_pr_guard
(#189), test_claim_gate_review_guards (#192), test_verb_runner_midverb
_invalid_state (#102 pin).
* playbook curate: guard the gating commit against a poisoned session (#55)
The explicit `session.commit()` that gates the RAG index (commit-before-index
so an uncommitted playbook cannot land in the corpus) raised PendingRollbackError
when a prior mid-verb failure had rolled the caller's session back — 500ing the
whole curation verb instead of returning a clean envelope, and (worse) risking a
fall-through to index an uncommitted playbook. Wrap the commit: on
PendingRollbackError, log + return invalid_state with a re-fetch/retry remediate
and skip the index. The happy path still commits exactly once then indexes.
TDD: test_playbook_verbs.py — poisoned-session returns a clean invalid_state and
does NOT index; clean-session still commits once + indexes (pins no fail-closed
inversion / no double-commit).
* [chore] gateway: atomic activate merge — preserve probe_failures across re-park (#156)
activate() was a blind SET that reset probe_failures to 0, so a probe-failure
increment that just landed (or was in flight) could be wiped by a concurrent
re-park — resetting the give-up / CEO-notify count mid-episode. Route activate
through a server-side Lua merge (roboco:activate_rate_limit) that refreshes the
episode metadata (kind / activated_at / retry_after / affected_agents) while
carrying over the previous probe_failures count. Indivisible w.r.t. the
increment/reset scripts (Redis single-threads an EVAL).
#56 (notify to prompter/secretary refused) verified SAFE — the pin tests
(test_notify_rejects_prompter_recipient / _secretary_recipient /
_allows_ceo_recipient) already cover the only human notify target invariant;
no legitimate send is dropped, no code change.
* [chore] foundation/policy: spec gates + QA retry-key pin (Cluster F)
#50 sync_branch composes=() so the spec gate accepted a terminal/paused/
blocked task and the handler rebased a dead/parked branch — add a
PRECONDITION_SYNC_BRANCH_STATE (claimed/in_progress/verifying/needs_revision
only), rejection_kind=invalid_state. TDD: 28 spec tests.
#148 submit_root's prose asserts 'a Main-PM root is planning-typed, never
code' but only the creation path (main_pm_cannot_own_code) backed it — add
PRECONDITION_ROOT_NOT_CODE on the submit_root IntentSpec (defense in depth),
scoped to submit_root only so the shared submit_for_review action keeps
cell_pm+code submit_up parity. Graceful on Mock/None task_type so the
choreographer Mock-task tests don't crash. TDD: 2 spec tests.
#150 VERB_RETRY_LIMITS is keyed by the MCP-exposed names (pass/fail), not
the IntentSpec-internal pass_review/fail_review — already correct; add a
pin test so a one-sided rename can't silently drop the QA-handoff cap.
#142 main_pm_cannot_own_code/pm_cannot_own_code already normalize casing
(.lower()) — no-op, pin test test_main_pm_cannot_own_code_is_case_insensitive
already in tree.
* [chore] worksession-git: 405 merge-method fallback + non-destructive close (Cluster W)
#108 _merge_with_retry hardcoded 'squash' and raised MergeConflictError on a
405 with no method fallback — wedging the PM on an open, mergeable PR whose
repo merely had the squash button off. Add a 405 fallback to a permitted
method (via _first_allowed_merge_method, exclude='squash'), mirroring the CEO
merge_pull_request path. A 405 with no permitted fallback (or a second 405)
still falls through to the already-merged disambiguation / MergeConflictError.
TDD: 2 new tests (fallback-success, no-permitted-method-raises).
#109 close_pull_request defaulted delete_branch=True, so the choreographer
supersede path deleted a superseded PR's branch while the orchestrator
supersede path explicitly preserved it — the two disagreed, and the
destructive default ran on the 'close the dead PR' path where the branch may
still be referenced / useful for audit. Flip the default to False (opt-in
deletion) and make the choreographer caller explicit (parity with the
orchestrator). TDD: 1 new test (default preserves branch); existing
deletion-when-requested test now passes delete_branch=True explicitly.
Dispositions verified against current code (no silent drops):
- #27 REFUTED/FIXED-UNDEPLOYED: work_session.merge_pr resolves by session_id
(no global pr_number lookup); the real cross-repo collision fix
(project_id scoping on pr_merge/close_pull_request/rebase_pr_for_task/
pr_target) is already in tree + tested (test_pr_merge_scopes_task_lookup_
by_project_id, test_close_pull_request_scopes_task_lookup_by_project_id,
test_git_pr_target_scoping). Verify-only.
- #106 REFUTED: a guard exists (rev-list --count {base_ref}..{branch} == 0)
before reset --hard + base_ref falls back to default_branch; tests lock
the safety (test_create_branch_never_repoints_branch_with_real_work,
test_create_branch_does_not_reset_or_checkout_shared_clone).
- #218 BY-DESIGN: the merge_pr idempotent guard intentionally preserves the
audit trail (docstring + test_merge_pr_idempotent_on_already_completed_
preserves_audit_trail); a COMPLETED session always carries attribution
(COMPLETED only via merge_pr), so the NULL-COMPLETED case is unreachable.
- #104 BY-DESIGN: agents never merge to the repo default branch in RoboCo's
model (root→master is CEO-only); the guard is a correct CEO-only rail,
locked by test_pr_merge_into_default_branch_is_ceo_only.
* [chore] llm: surface disabled-provider downgrade + scrub probe log (#20/#3/#211)
#20/#3 resolve_for_agent silently fell through to the legacy Anthropic path
when a configured provider was disabled — indistinguishable from 'no
assignment', so the operator got no signal that spawns bypassed the
provider. Surface the bypass with a warning (graceful degradation stays the
default — a stalled spawn is worse than a routing miss) and add an opt-in
ROBOCO_ROUTING_STRICT (default-off) that fail-closes instead. Wired into the
panel Feature Flags card. TDD: 3 unit tests (warn-on-disabled, strict-raises,
no-assignment-stays-silent).
#211 probe_ollama_tags logged str(exc) raw on the generic-exception branch —
structured log could carry connection internals / stack traces. Log the
exception class name only. TDD: existing generic-branch test strengthened to
assert the log kwargs don't leak the raw text.
* [chore] support/stream/optimal/playbook/comms hardening (Cluster S)
Logical-gaps sweep, Cluster S (TDD, red→green per item):
#64 notification_delivery.acknowledge published the NOTIFICATION_ACKED bus
event directly (bypassing the outbox) — a rollback left a phantom ACK. Route
it through defer_bus_publish (after_commit), mirroring deliver.
#76 playbook.archive()/reject() stamped the archiver into approved_by/
approved_at, overwriting approval provenance (and fabricating approval for a
rejected draft). Add archived_by/archived_at (migration 053 + table + model)
and write those on archive/reject, leaving approval attribution intact.
#181 vector_store.replace_chunks wiped existing index rows even when every
chunk lacked an embedding (embedder failure). Skip the wipe when chunks is
non-empty but records is empty — preserve good rows for nothing.
#182/#183 optimal.record_learning recomputed a learn-{md5(full_content)}
tracking source that never matched the URI the plugin embedded chunks under
(roboco://learnings/{doc_id}, doc_id=lrn-{hash100}). Use the plugin's
returned doc_id so de-index/lookup-by-source finds the chunk rows.
#96/#97 transcription periodic flush only peeked ready buffers (unbounded
map growth) and ran sync callbacks on the event loop (a slow callback
blocked the flush task). Flush (remove) each ready buffer after notifying,
and offload each callback to a thread.
#212 _TEAM_SCOPED_ROLES was duplicated across communications/agents_config/
seeds. Single-source it in foundation.policy.communications; consumers
reference that object (identity-tested).
#19 stream_bus._dispatch_event re-ran already-succeeded handlers on a
recover_pending replay (duplicate side effects). Add a per-(event.id,
handler) SET-NX idempotency guard: skip on a hit, clear the key on handler
failure so a replay re-runs it, fail-open when redis is unavailable.
Dispositions (no code change): #77 approve() index-write pair asserted
BY-DESIGN; #62/#63 notification DB-dedup verified pinned; #184/#185 REFUTED;
#214 REFUTED; #215 BY-DESIGN.
* [chore] db/migrations: graph-integrity guard + conftest unreachable-DB warning (Cluster D)
Logical-gaps sweep, Cluster D (TDD + real alembic upgrade head verification):
#16/#37 add tests/unit/test_migration_graph_integrity.py — a static guard that
the alembic migration graph has exactly one head, every down_revision resolves,
every revision is reachable from a root, and no revision id is duplicated. The
suite builds its DB via Base.metadata.create_all (not alembic upgrade head), so
a forked head / dangling down_revision / duplicate id would otherwise ship
silently and break a real deploy mid-stream.
Caught a real bug in the process: migration 053's revision id
"053_playbook_archived_attribution" (33 chars) exceeded alembic's
alembic_version.version_num VARCHAR(32) — a fresh `alembic upgrade head` raised
"value too long for type character varying(32)" at the 053 stamp. Renamed to
"053_playbook_archived_attr" (26 chars). Verified end-to-end on a scratch PG:
upgrade head stamps 053, downgrade -1 returns to 052. (The pre-existing
test_every_migration_revision_id_fits_the_alembic_version_column guard is now
green too; it had been red on the 33-char id.)
#90 conftest silently pytest.skip'd every DB test when Postgres was unreachable
— a non-Docker box reported a green run of all-skips. Extract the warning into
_warn_if_pg_unavailable and fire it at import so the operator sees the DB is
down (the per-test skip path is unchanged). Test: warns when unavailable, silent
when reachable (verified under -W error::UserWarning).
Dispositions (verified against real code + a fresh alembic upgrade head, no code
change): #6 REFUTED — sa.Enum(create_type=False) at 001:119/304 does NOT break a
fresh upgrade head (001→052 applied cleanly on a scratch DB); #8 REFUTED — the
upgrade passed 030/031 (RAG chunk tables) without pgvector installed; pgvector is
a runtime concern handled by roboco/db/base.py, not a migration prerequisite;
#40 REFUTED — the `|| echo` mask was already removed and partial-schema drift
reports exit 1 (only by-design unreachable/unmigrated skips remain); #204 REFUTED
— the property walk seed IS pinned (random.Random(20260504), line 97); #205/#206
REFUTED — the smoke-trace fixture IS wired via
test_lifecycle_smoke_replay.py (8 passed); no shell smoke scripts exist in the
tree to wire; #137 BY-DESIGN — pyproject version 0.14.0 is an operational note,
no code gate.
* [chore] panel: admin-override force flag + kanban subtask_count + ws cleanup + ui-store dedupe (Cluster P)
#13: kanban admin-override into a hatch state (completed / awaiting_qa /
awaiting_pm_review) now requires an explicit force=true from the panel and
emits a dedicated task.admin_override audit row server-side; non-hatch
overrides need no force. Backend gate in tasks route + admin_set_status;
panel kanban-board sends force for hatch targets; TaskUpdate carries force.
#198: kanban service threads the real subtask_count (one grouped query) into
dev + priority-swimlane + main-pm-flat boards instead of a hardcoded 0.
#79: useWebSocket cleanup clears messages/lastMessage/state on unmount or
endpoint change so a dep-change (navigating to another stream) can't leak the
prior subscription's stale snapshot as live.
#186: disambiguate the duplicate ui-store modules -- the session/scroll store
in lib/stores renamed to useScrollRestorationStore / scroll-restoration-store
(barrel + 2 consumers updated); the sidebar/theme useUIStore in @/store is now
the sole useUIStore.
#12: verified already in-tree (release-proposal-card surfaces non-404 errors
with retry; getProposal maps only 404->null). #80 by-design (handleTransportError
already resets isSending on a no-payload SSE drop). #81 docs (streamUrl docstring
records that live-intake SSE auth is session-id-based bearer-style).
Backend: ruff+mypy clean, 278 tests green. Panel: lint+typecheck clean, 159 tests.
* orchestrator: park/reaper/readopt/a2a hardening + self-heal/ci-watch dedupe (Cluster O)
Closes the orchestrator-side logical gaps from the sweep:
- #75 a2a human-only drop surfaced: _dispatch_a2a_work logs the skip
("a2a request targets a human-only role; left as a notification for the
human (not spawned)") instead of silently dropping the target — the
CEO/secretary/prompter still see the notification; only the spawn is
suppressed. (orchestrator.py)
- #72 readopt liveness: _readopt_running_agents requires a non-stale live
claim (via _agent_holds_live_claim) and skips a zombie container so a
reaped-but-restart-readopted agent isn't double-counted as active.
- #74 shutdown drain: stop() calls _flush_respawn_tracker so the durable
respawn counter write-throughs aren't lost on a clean stop.
- #71 resolve_wait active-guard + deferred liveness: a rate_limit_lifted
WaitingRecord is only confirmed-live after a _confirm_resume_liveness
probe (deferred deletion _resume_confirm_delay=30.0), and an
already-active agent short-circuits the repark. Scoped to
rate_limit_lifted records (the only ones at risk of a false lift).
- #73 stuck-Claude kill: _maybe_kill_stuck_claude + _claude_stuck_kill_ttl
(config.claude_stuck_kill_seconds) — a live container whose heartbeat is
stale past the grace AND whose gateway probe is broken is killed+evicted,
not protected forever by the reaper's live-skip.
- #230 verified FIXED-UNDEPLOYED: _gateway_broken_past_grace already
requires N consecutive false-broken probes (not one flaky streak); no
change, test added to pin the N-consecutive invariant.
- #43 self-heal per-observation dedupe: a fingerprint collapses repeat
CEO notifications for the same CI regression.
- #44 ci_watch dedupe by (git_url, workflow): a monorepo's multiple
workflows each get their own fix task (was collapsed by git_url alone).
- #49 identity.role_for_slug_or_none None-hardening: a stale/malformed
slug resolves to None and the human-only skip falls through to the safe
"not spawnable" path instead of crashing.
- #193 strategy engine: notify the CEO on a persistent assess failure
instead of failing silently in the background loop.
TDD: test_no_spawn_human_roles (a2a skip surfaced), test_orchestrator_
shutdown_drain (#74), test_provider_overload_break (#71), test_readopt_
running_agents (#72), test_resolve_wait_repark (#71), test_stale_claim_
reaper (#73/#230), test_strategy_engine_loop (#193, new),
test_self_heal_engine (#43), test_ci_watch_engine (#44), test_identity
(#49). All red->green.
* chore: make-quality green — xenon complexity refactors + mypy test fixes + lifecycle regen
No behavior changes. Brings the tree to a fully green `make quality` (the
base branch never passed the xenon B-rank gate on several blocks; the
lifecycle artifacts had drifted from the committed ceo_reject_to_pool edge).
Xenon B-rank refactors (extract a helper; preserve semantics exactly):
- api/routes/tasks.py: _apply_forced_status_override + _StatusOverride
dataclass bundle (update_task override block).
- services/task.py: _enforce_no_pm_code_on_create (create guards) +
_escalation_diverts_to_pool (collapses the two board/advisory +
main_pm+code divert branches into one predicate).
- services/prompter.py: _coerce_pm_code_to_planning (create_task_from_draft).
- services/notification.py: _duplicate_unacked_exists (_create_notification
purpose-based dedup query + ACK_REQUIRED_BY_TYPE gate).
- services/sequencing.py: _same_assignee_lane_edges (the undeclared-surface
same-assignee lane fallback at the tail of dev_task_collision_edges).
- gateway/choreographer/_impl.py: _pm_task_type_error static helper
(_validate_assignee_task_type compound PM guard).
- gateway/choreographer/pr_gate.py: _gate_review_event_verdict +
_gate_review_body static helpers (_post_gate_review_to_pr).
mypy test fixes (no type:ignore — banned; use typing.cast with quoted
strings per TC006):
- test_task_update_completeness: TaskUpdate(acceptance_criteria=None).
- test_bus: cast("Redis", _FakeRedis()); Redis import under TYPE_CHECKING.
- test_pr_merge_concurrency: capture AsyncMocks into locals before asserting.
- test_notification_delivery_phantom: cast("UUID", to_agents[0]).
Lifecycle artifact regen (owed from Cluster T #100 — the
awaiting_ceo_approval -> pending `ceo_reject_to_pool` edge was added to the
spec in
|
||
|
|
5fe1e6df58 |
feat: in-path PR-review gate — per-cell + main reviewers (#229)
* feat(lifecycle): add the in-path PR-review gate status + reviewer verbs
Insert awaiting_pr_review between the assembled-PR submit and the PM merge,
giving the merge level the rejection capability it structurally lacks — today
only qa_fail and ceo_reject ever reach needs_revision, so a PM review is a
merge button with no teeth.
- New Status awaiting_pr_review + submit_for_review / pr_pass / pr_fail actions
(pr_pass -> awaiting_pm_review, pr_fail -> needs_revision, mirroring the QA gate).
- Reviewer verbs claim_gate_review / pr_pass / pr_fail, and a main-PM submit_root
verb (the root analogue of the cell PM's submit_up; opens the root->master PR).
- Extend the self-review-symmetry validator to the new sign-off actions.
- Mirror the value into the ORM TaskStatus enum + the A2A state map, and add the
postgres taskstatus enum value (migration 040, forward-only like 037).
- Regenerate the per-role verb tables; add gate spec tests.
Spec surface only; the gateway methods + dispatch are wired in follow-ups, so the
verbs are advertised but dormant (flow_server tolerates unregistered verbs).
* feat(identity): add the three cell PR-review-gate reviewers
The in-path gate needs a reviewer per cell so each cell's assembled cell->root
PR is reviewed by a stack-specialized agent, while pr-reviewer-1 serves the
root->master gate (and keeps doing inbound external PRs).
- be/fe/ux-pr-reviewer: PR_REVIEWER role, team-scoped (so dispatch routes each
cell's gate to its own reviewer); seeded identities + ROLE_TEAM_RULES + names.
AI agent count 22 -> 25.
- They reuse the existing roboco-agent-pr-reviewer image (AGENT_IMAGES maps the
three slugs to it, as be-dev-1/-2 share one image) — no new image.
- Tracing table: pr_pass/pr_fail require a learning entry (parity with
post_pr_review), submit_root mirrors submit_up, claim_gate_review is waived
(its tracing applies on pr_pass/pr_fail) — completes the verb surface added
in the prior commit.
- Update the roster-pinning identity tests.
* feat(gateway): wire the in-path PR-review gate end to end
Make the assembled-PR review gate operational across the choreographer, the
TaskService transitions, and the v1 flow surface.
- TaskService: submit_for_review (in_progress→awaiting_pr_review), pr_gate_claim
(no-transition reviewer claim), pr_pass (→awaiting_pm_review), pr_fail
(→needs_revision); mirror qa_pass/qa_fail (clear claim, actor-mismatch warn,
issues appended for the PM's revision). VerbRunner gains the matching atomic
handlers + a create_root_pr side effect.
- Repoint submit_up to compose submit_for_review (cell→root PR enters the gate),
and add a main-PM submit_root verb (opens the root→master PR, enters the gate).
- Split main_pm_complete: a code root must pass the gate first (requires
awaiting_pm_review; rejects an in_progress code root toward submit_root and no
longer reopens the PR), while a branchless coordination root still walks
straight through, ungated.
- PRGateMixin (claim_gate_review / pr_pass / pr_fail) composed onto the
Choreographer; flow_server forwarders + v1 routes (pr_reviewer + main_pm) +
request schemas.
- Tests: gate spec + the updated submit_up / main_pm_complete expectations + new
real-DB integration tests driving submit_for_review→pr_gate_claim→pr_pass and
pr_fail through the real enforcement layer.
* feat(orchestrator): dispatch the in-path PR-review gate
Make the gate live in the dispatch loop.
- _dispatch_pr_gate_work: route awaiting_pr_review tasks to reviewers by level —
a cell→root task to its cell reviewer (be/fe/ux-pr-reviewer), the root→master
task to pr-reviewer-1. The reviewer self-claims via claim_gate_review (no
pre-claim, mirroring the external-PR dispatcher); registered in
_dispatch_all_work. _select_agent_for_cell learns the pr_reviewer role.
- _build_pr_gate_prompt: anchors the reviewer to the parent objective + full
acceptance criteria + the FE<->BE contract, then pr_pass / pr_fail.
- _readiness_check_role_for_status: awaiting_pr_review -> pr_reviewer.
- Fail routing: pr_fail reassigns the failed assembled task to its PM
(_revision_pm_for_task: cell PM for a cell team, Main PM for the root), and the
revision dispatcher is generalized from coordination-roots-only to any
PM-owned needs_revision task so the gate-failed task is re-coordinated instead
of deadlocking.
* docs: document the in-path PR-review gate + the cell reviewers (22→25)
Reflect the shipped gate across the canonical + RAG docs.
- CLAUDE.md: agent count 22→25, the cell reviewers in the org chart, an
awaiting_pr_review state + the gate transitions + a gate note in the lifecycle
section, and submit_root / claim_gate_review / pr_pass / pr_fail in the verb
surface table.
- docs/rag/architecture: org-structure (count, cell-reviewer roster, cells
table), agent-uuids (be/fe/ux-pr-reviewer rows), agent-model (role + team
rows).
- docs/rag/roles/pr-reviewer: the in-path gate section + the gate verbs.
- Wrap reviewer.id with UUID(str(...)) in the gate DB tests for mypy.
* docs: finish the gate doc sweep across README + RAG + generated artifacts
Catch the remaining surfaces beyond the canonical docs.
- README + how-to: agent count 22→25, the 6-agent cells (+ PR Reviewer), the
main reviewer's root→master gate role.
- RAG: permissions + tool-permissions + task-tools list the gate verbs
(claim_gate_review / pr_pass / pr_fail) for pr_reviewer; regenerate the
lifecycle artifacts (intent-verbs, status-transitions, the per-role
lifecycle-*.md prompts, panel lifecycle.json) from the spec via
build_lifecycle_artifacts.py so they carry the new status + verbs.
* fix(migration): shorten the 040 revision id to fit alembic_version VARCHAR(32)
The revision id '040_taskstatus_awaiting_pr_review' is 33 chars; alembic's
alembic_version.version_num column is VARCHAR(32), so recording the migration on
a real 'alembic upgrade head' failed with 'value too long for type character
varying(32)' (surfaced on the NAS deploy). The test suite missed it: the test DB
is built via Base.metadata.create_all and the parity test only renders SQL
offline, so nothing actually applied the migration chain.
- Rename to '040_awaiting_pr_review' (22 chars).
- Add a guard test asserting every revision id fits the VARCHAR(32) column.
- Verified by applying the full chain 001->040 against real Postgres: it now
reaches head and records '040_awaiting_pr_review' without truncation.
* fix(migration): land the actual 040 revision-id shortening + guard test
The prior commit captured only the file rename (git add aborted on the deleted
old path), leaving the long revision id and missing the guard test. This commit
carries the real content: revision id '040_awaiting_pr_review' (22 chars) and the
revision-id length guard. Re-verified against real Postgres — the full chain
reaches head and records the short id without truncation.
* fix(product): flush cell deletes before inserts when re-mapping projects
Editing a product's cell->project map (PATCH /api/products/{id}) 409'd with
'duplicate key value violates unique constraint uq_product_projects_product_team'
whenever a team already had a mapping. _replace_cells clears the old rows and
appends the new ones, but within a single flush SQLAlchemy orders INSERTs before
DELETEs for the same table, so the new (product_id, team) rows collided with the
not-yet-deleted old ones. Flush the deletes first.
Pre-existing bug (unrelated to the PR-review gate); surfaced on the NAS. New
real-Postgres regression test re-maps all three cells to different projects —
it fails with the unique violation without the fix and passes with it. The
existing update test only changed WHICH team was mapped, so it never collided.
* fix(gateway): let main_pm submit_root past the shared submit-up guard
submit_root reused the cell PM's _submit_up_ownership_guard, which
hardcoded agent.role != cell_pm and rejected the Main PM with
"submit_up is reserved for cell_pm". A branch-bearing code root could
then never close: submit_root bounced to complete, while complete
required awaiting_pm_review (reachable only via submit_root) and bounced
back — a circular rejection.
Both callers already run the spec gate (can_invoke_intent), which
enforces submit_up→cell_pm and submit_root→main_pm, so the guard's role
re-check was redundant for submit_up and wrong for submit_root. Broaden
it to accept either PM role as a defense-in-depth non-PM reject.
Adds the first choreographer-level submit_root test (the gap that let
this ship).
* fix(gateway): proactively steer both PMs to their bubble-up verb
The submit_root deadlock had a sibling steering gap: nothing told a PM
which verb opens the gate. The delegate next-hint said only 'i_am_idle
when done', and complete's in_progress rejection named submit_root for
the Main PM but left the Cell PM with a bare 'not ready for completion'
— no submit_up pointer, the same guess-the-verb trap.
- delegate hint now names the role-correct verb (root → submit_root,
cell parent → submit_up) proactively, before any rejection.
- cell_pm_complete's in_progress rejection now steers to submit_up,
mirroring the Main PM's submit_root gate hint.
Tests cover both the cell-PM steer and the role-aware delegate hint.
* docs: correct who-merges-which-PR across the gate docs + complete description
Audit of the gate docs found the merge actors mis-stated in several
places — the exact ambiguity that risks 'the reviewer/PM merges the root
PR' confusion:
- complete IntentSpec description said 'Main PM merges root PR' — false;
main_pm_complete escalates and the CEO merges root→master. Corrected
(propagated to intent-verbs.md, lifecycle.json, generated role prompts
via build_lifecycle_artifacts.py).
- task-tools.md: submit_up target was awaiting_pm_review (should be
awaiting_pr_review); Main PM flow had no submit_root — added it.
- README.md: lifecycle diagram now shows the awaiting_pr_review gate.
- cell-pm.md / main-pm.md: dropped the stale 'submit_up hands work to the
Main PM who merges your cell branch' model — the cell PM merges its own
gated cell→root PR; the Main PM owns the root + submit_root; the CEO
merges master. Added submit_root to the main-pm manifest.
- git-commits.md, pr-creation.md, tool-permissions.md, git-tools.md:
stopped attributing root→master PR opening to complete (it's submit_root).
No behavior change; verb wiring + state machine verified gap-free this
session (the pr_fail→needs_revision→PM respawn loop closes correctly).
* fix(orchestrator): stop closure respawn waiting the reaper window
A PM that finished its subtasks and idled left its parent 'paused' with a
fresh last_heartbeat_at. _is_recently_paused gated closure respawn on
_claim_heartbeat_ttl — the REAPER window (stale_claim_reap_seconds: 600s
default, 1800s on the NAS) — so the parent sat untouched for up to 10-30
minutes before its PM was respawned to close it. The whole chain stalled
behind it.
The race that guard actually protects against (i_am_idle auto-pauses, then
the agent is marked IDLE + its container tears down) is seconds, and the
live-session case is already covered by _is_agent_active. Introduce a
dedicated short debounce (pm_closure_recently_paused_seconds, default 45s)
and gate closure on that instead.
The existing test fixture masked this by setting _claim_heartbeat_ttl to
claim_stale_seconds (180s), not the production reaper value. Fixture now
mirrors production; adds a regression test that a parent paused past the
debounce but within the reaper window respawns immediately.
* feat(gate): post the in-path review verdict on the assembled PR
The in-path gate previously left no trace on the PR it gated — pr_pass /
pr_fail were pure status transitions. Now each verdict is posted as a
GitHub review on the assembled PR itself (server-side, bot account), so
the decision is visible on the very PR the PM merges.
- pr_pass → APPROVE, pr_fail → REQUEST_CHANGES on a cell→root PR.
- The root→master PR ALWAYS gets a plain COMMENT, never APPROVE/REQUEST_
CHANGES: only the CEO acts on master, so the gate must never leave an
approval that could satisfy branch protection (letting someone else
merge) nor a blocking review that could impede the CEO's merge.
- Best-effort and AFTER the DB transition — a GitHub failure is logged,
never rolls back the gate decision. Reuses git.post_pr_review's existing
self-review→COMMENT downgrade for the org's own PRs.
Adds _project_slug_for to the ChoreographerHelpers protocol (mypy) and a
unit suite covering event selection, the master-bound COMMENT rule, the
no-PR skip, and failure-swallowing. Docs updated (pr-reviewer, task-tools).
---------
Co-authored-by: Renn F <rennf93@users.noreply.github.com>
|
||
|
Renn F
|
f48106cbb6 |
docs: reflow hard-wrapped prose to one line per paragraph
Markdown and editors soft-wrap on their own, so the manual ~75-char line breaks across the docs added nothing but noise. Join wrapped prose, list items, and paragraphs into single lines across 67 docs — README, CLAUDE.md, deployment, usage, the RAG knowledge base, and the agent role prompts. Whitespace-only: code fences, tables, and blockquote alerts are byte-identical and the change is token-verified (no content altered). Applied with a deterministic reflow tool (committed separately). Also lands two doc edits that were awaiting commit: the measured under-load resource numbers in usage.md and the pr_reviewer additions to the org-structure RAG doc. |
||
|
Renn F
|
73e1e96851 | Many fixes and cleanups | ||
|
Renn F
|
f1c5b7958c |
Add Project & Workspace MCP System with role-based permissions
- Add roboco_project_* tools (list, get, create, update) with CEO bypass - Add roboco_workspace_* tools (ensure, status, list) for workspace management - Add project_slug and requires_git fields to TaskCreateInput schema - Validate project exists and cell matches when creating git-enabled tasks - Register project MCP server in orchestrator with proper permissions Workspace permissions by role: - Developer: Write to own workspace only - QA: Read-only access to all cell workspaces - Documenter: Write to all cell workspaces (add docs to dev branches) - Cell PM: Write to own workspace, project_update for own cell - Main PM: Full project access (create, update all, workspace_list all) - CEO: Full bypass on all permission checks Also includes: - Git templates for commits, branches, PRs (separation of concerns) - Updated blueprints with project/workspace tools documentation - Updated RAG docs with project tools reference |
||
|
Renn F
|
1d173a5203 | NOW RAG is actually usable... might switch to gemma3:4b from glm-4.6 cloud for expenses reasons but we'll see |