mirror of
https://github.com/rennf93/roboco.git
synced 2026-08-03 07:23:24 +02:00
Leak fixes, gate green again, uv/CI hardening, e2e lifecycle smoke harness (#294)
* test: align phase1 smoke mock with the armed team-match gate The8e5f84c4sweep fixed 13 test files' inconsistent-team mocks but ran only the gateway/foundation/runtime subsets; the full gate caught this integration mock whose parent task carried an auto-generated MagicMock team and died on not_authorized before the incomplete_input assertion. * fix(runtime): attribute every agent.spawned audit to its dispatcher A rogue spawner could not be identified live (2026-07-02): agent.spawned rows carry container/model but not which dispatch loop launched them. spawn_agent now takes spawned_by, stamps it into the spawned/spawn_failed audit details, every call site passes its loop name, and an AST sweep test holds future callers to it. * fix(api): admin-complete refuses when the task's PR is still open PATCH status=completed on a task with an OPEN PR stranded its commits unmerged (bit the CEO twice live 2026-07-02). The override now refuses with the PR number/URL and the consequence before the generic hatch text; force:true stays the deliberate, audited escape. * fix(panel): awaiting_ceo_approval offers the working ceo-approve path The header's only approve action was Approve & Merge (POST /approve-and-merge, no notes) which 400s NO_PR on a branchless MegaTask umbrella — the CEO's approve button just failed. Primary action is now Approve & Complete via the CeoApproveDialog (POST /ceo-approve, notes >=20 chars, proven live); Approve & Merge stays for PR-bearing tasks. * test: stop leaking self-heal + rate-limit state into live Redis Two test files wrote real keys into a developer's localhost Redis: self-heal originate tests left self_heal:notified:* (2h TTL) and the i_am_blocked rate-limited tests left a NO-TTL 'anthropic rate-limited' tracker blob — order/state-dependent poison for anything reading the real tracker, and the prime suspect class for the one-off test_self_heal_engine full-run failure (not reproduced in 5x dir runs, adversarial orders, and a green full gate). Both files now point the computed redis_url at an unreachable port; the engines' fail-open paths keep every assertion intact. Leaked keys scrubbed live. * docs: changelog + map delta for the leak-fix batch; mypy-clean attribution test The attribution test's direct method assignments tripped the full gate's mypy (method-assign) — switched to the house monkeypatch idiom, no suppressions. * fix(gate): clear the ten xenon C-ranks; isolate all tests from live Redis Master CI has been red at the phase1 smoke test, so neither CI nor a local full gate had reached the xenon step since the team-match sweep — whose inline 'agent_team=str(agent.team) if ...' kwarg pushed nine verb bodies from B(10) to C(11-12) unseen. A shared actor_context_fields() (_protocol.py) computes (actor_slug, agent_team) once per verb, restoring all nine to B with zero behavior change; the new admin-complete override helper extraction does the same for routes/tasks.py. tests/conftest.py gains an autouse fixture pointing the computed redis_url at an unreachable port for every test — the root fix for the three families caught writing live-Redis keys (self-heal dedupe, rate-limit tracker, notification purpose-dedupe); no test uses a real Redis, and every production path is fail-open by design. * refactor(runtime): delete the never-wired dispatch-time spawn cooldown _safe_spawn / gateway_pre_spawn_check / trigger_filter had no caller in the repo's entire history (87ef42bfonly flipped the flag). Its five rules are superseded: provider parking runs inside spawn_agent, claim freshness is the guards+reaper, runaway respawns are the progress-aware breaker + notification cooldown; the per-task cooldown rule would queue-stall every normal stage handoff if wired today. gateway_triggers table kept inert. Ratified by the CEO over wiring it. * build: serialize uv — gate recipes never implicitly sync the venv Every uv run re-syncs implicitly, so a background make quality plus any foreground uv run raced two writers on one .venv and tore site-packages apart (the recurring rich/pip/bandit ImportError corruption; bit twice today, four times on 2026-07-02's first session). UV_NO_SYNC=1 is now exported Makefile-wide and quality/quality-fast/gate depend on one explicit up-front sync step. * fix(git): PR/merge/branch REST calls honor github_api_base_url Fifteen sites hardcoded https://api.github.com while the CI-run and open-PR-list calls already read settings.github_api_base_url — a GHE or test override silently applied to half the surface. One _api_base() helper keeps them uniform; default behavior unchanged. * ci: split the monolith — backend CI, Panel CI, E2E Smoke ci.yml keeps its file name and the backend quality job only (self-heal / ci-watch / release-readiness default to the ci.yml workflow); the panel job moves to panel-ci.yml scoped to panel/**, and the new scripted-agent lifecycle smoke gets e2e-smoke.yml + a make e2e-smoke target (env-gated out of the default pytest run). Trade: a panel-only red now lands on Panel CI, which the ci.yml-pinned watch engines don't see. * feat(tests): e2e lifecycle smoke harness — scripted agents, real gates tests/e2e_smoke stands up the real API (flow/do routers + middleware on uvicorn) over the ephemeral test Postgres, a local bare origin standing in for GitHub, and a fake GitHub REST layer whose merges are real git merges. A deterministic driver reloads the real MCP flow/do modules per agent and walks claim (real clone + worktree) -> tracing-gap -> note -> plan gate -> commit -> PR -> the full i_am_done ladder -> QA verdicts -> documenter -> awaiting_pm_review in ~5s. Runs via make e2e-smoke + its own CI workflow; skipped (env-gated) in the default suite. The freeze-lift condition's first half: scenario 1 green. --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com>
This commit is contained in:
@@ -550,3 +550,31 @@ Nine live-run fixes, all merged to `master` the same day (commits `81f448bb`, `0
|
||||
9. **Team-match enforcement armed** (`fe9e5589`, `8e5f84c4`) — `roboco/foundation/policy/lifecycle.py`: `resume`/`unblock`/`activate` flip `needs_team_match=True`; new `_ORG_WIDE_ROLES` exemption (main_pm, CEO, PO, head_marketing, auditor, pr_reviewer) in `_check_team_match(…, role)`. `8e5f84c4` threads `agent_team` through all 27 gateway `Context` construction sites (`choreographer/_impl.py`, `doc.py`, `pr_gate.py`, `pr_review.py`, `qa.py`) so the gate actually receives the team — it had sat in its permissive fallback since shipping.
|
||||
|
||||
Slices touched: worksession-git (2, 3), choreographer/gateway-support (9), orchestrator (6, 7, 8), foundation-lifecycle (7, 9), mcp-servers (1), api-routes-schemas (4), panel (5). `docs/map/_complete_map.md` is the pre-delta concatenation — not regenerated.
|
||||
|
||||
---
|
||||
## Delta 2026-07-02 (evening) — leak-fix batch (branch `fix/leak-fix-batch`)
|
||||
|
||||
The remaining live-run leak fixes from the S6/fb836f80 postmortems, TDD'd per item:
|
||||
|
||||
1. **Spawner attribution** — `roboco/runtime/orchestrator.py` `spawn_agent(spawned_by=)` → `_launch_spawn(spawned_by=)` stamps the dispatching loop's name into `agent.spawned` / `agent.spawn_failed` audit details (`"unspecified"` when absent). All ~29 call sites pass their loop name (orchestrator dispatchers by method name; `bootstrap`, `api.orchestrator.spawn`, `event.auditor_spawn`); `OrchestratorAccessProtocol.spawn_agent` (roboco/models/events.py) gains the kwarg. A whole-package AST sweep test (`tests/unit/runtime/test_spawn_attribution.py`) fails any future caller omitting it. `pyproject.toml` adds PLR0913 to the orchestrator's per-file ignores (spawn contract > 5 params, bundle refused). NOTE found in passing: `_safe_spawn` + `gateway_pre_spawn_check` (orchestrator ~L741/L1932, the trigger_filter dispatch-time cooldown) have NEVER had a caller — flagged to CEO, untouched under the freeze.
|
||||
2. **Admin-complete merge-or-refuse** — `roboco/api/routes/tasks.py` `_apply_forced_status_override`: `status=completed` without `force` on a task whose work session records `pr_status == "open"` now refuses naming PR number/URL + the stranding consequence (checked before the generic hatch text). New `TaskService.open_pr_ref` (`roboco/services/task.py`, beside `_assert_pr_merged_for_complete`) is the lookup.
|
||||
3. **Panel CEO-approve** — `panel/src/components/tasks/task-detail/task-header.tsx` `AWAITING_CEO_APPROVAL`: primary action now `ceo-approve` (CeoApproveDialog → `POST /ceo-approve`, notes ≥ 20); `approve-and-merge` only when `task.pr_number` (umbrellas 400'd `NO_PR`).
|
||||
4. **Live-Redis test leakage killed** — `tests/unit/services/test_self_heal_originate_db.py` wrote `self_heal:notified:*` (2h TTL) and `tests/unit/gateway/test_i_am_blocked_rate_limited.py` wrote a NO-TTL `roboco:rate_limit:anthropic:state` "rate_limited" blob into live localhost Redis on every run (verified live, keys scrubbed). Both now patch `cfg.redis_host/redis_port` (redis_url is computed) to an unreachable port; engines' fail-open paths keep assertions intact. The one-off `test_self_heal_engine` full-run failure was NOT reproduced (adversarial orders, 5× dir loops, green full gate) — this leakage class + the 2026-07-02 corrupted-venv day remain the suspects.
|
||||
5. **Phase1 smoke mock team** — `tests/integration/test_foundation_phase1_smoke.py` parent mock gains `team="backend"`; the armed team-match gate (delta above) rejected the auto-generated MagicMock team before the asserted `incomplete_input` (the sweep fixed 13 files; this integration file was outside the gateway/foundation/runtime subsets it ran).
|
||||
|
||||
6. **Gate green again: ten xenon C-ranks + global test-Redis isolation** — master CI red at the smoke test meant no gate (CI or local) had reached xenon since the team-match sweep; its inline `agent_team=…` kwarg had pushed 9 verb bodies to C(11-12) unseen (`_impl.py` i_am_done/resume/unclaim/submit_up/submit_root/complete/escalate_up/escalate_to_ceo, `qa.py` fail_review). New shared `actor_context_fields()` (`choreographer/_protocol.py`) computes `(actor_slug, agent_team)` once per verb; the admin-complete open-PR check is extracted to `_refuse_unforced_complete_with_open_pr` (routes/tasks.py). `tests/conftest.py` gains the autouse `_no_live_redis` fixture (root fix for the live-Redis leak class; notif_dedup purpose-dedupe keys were a third, self-expiring family).
|
||||
|
||||
7. **Dead spawn-cooldown path deleted (CEO-ratified)** — `_safe_spawn` + `gateway_pre_spawn_check` (orchestrator) + `roboco/services/gateway/trigger_filter.py` + both test files + the dead `spawn_cooldown_seconds`/`role_spawn_rate_per_minute` settings. Never called in repo history; superseded by provider parking (in `spawn_agent`), claim guards/reaper, the respawn circuit breaker, and the notification-spawn cooldown. Rule 4 (per-task cooldown) would have queue-stalled every stage handoff if wired. `GatewayTriggerTable` kept inert (drop = future migration decision).
|
||||
|
||||
Slices touched: orchestrator (1, 7), api-routes-schemas + taskservice (2), panel (3), choreographer/gateway-support (6, 7), tests (4, 5, 6, 7). `docs/map/_complete_map.md` still not regenerated.
|
||||
|
||||
---
|
||||
## Delta 2026-07-02 (late) — CEO-ratified follow-ups (branch `fix/leak-fix-batch`)
|
||||
|
||||
1. **Dead spawn-cooldown path deleted** (see item 7 above; commit `78fa0f6f`).
|
||||
2. **uv serialization in the Makefile** (`cf5043fe`) — `export UV_NO_SYNC := 1` + `sync` prerequisite on quality/quality-fast/gate: gate recipes never implicitly re-sync the venv (the recurring rich/pip/bandit corruption came from two uv writers racing one `.venv`).
|
||||
3. **`_api_base()` in git.py** (`13b9c5d4`) — 15 hardcoded `https://api.github.com` PR/merge/branch sites now honor `settings.github_api_base_url` like the CI/open-PR sites already did (GHE/test override fix; enables the smoke harness's fake GitHub).
|
||||
4. **CI split** (`db6b3088`) — ci.yml (backend `quality` only; FILE name kept — self-heal/ci-watch/release default to it), `panel-ci.yml` (panel job, `panel/**` paths), `e2e-smoke.yml` (new job). Panel-only reds now land on Panel CI, unwatched by the ci.yml-pinned engines.
|
||||
5. **e2e smoke harness** — `tests/e2e_smoke/{conftest,harness,test_dev_lifecycle}.py` + `make e2e-smoke` (env-gated `ROBOCO_E2E_SMOKE=1`, skipped in the default suite). Harness: real routers/middleware on uvicorn over the ephemeral test DB (`settings.database_*` patched + `_DbHolder` reset), bare origin at `<tmp>/github.com/e2e-smoke/proj.git` (satisfies `_parse_git_url`, clones tokenless), fake GitHub REST router doing real squash-merges via an admin clone, `ScriptedAgent` reloading the real MCP modules per role. Scenario 1 (leaf dev arc → awaiting_pm_review) GREEN in ~5s. Learned seams scripted: post-claim tracing gap keeps the claim; note scopes are decision/learning/note/reflect/struggle (no 'progress'); i_am_done demands during-work+handoff+reflect+per-AC artifacts; pass_review demands learning note + ac_verdicts; A2A resolves roles from the STATIC agents_config registry (seed canonical slugs: be-dev-1/be-qa/be-doc/be-pm/main-pm).
|
||||
|
||||
Slices touched: worksession-git (3), orchestrator (1), deployment-tooling (2, 4), tests (5).
|
||||
|
||||
Reference in New Issue
Block a user