Files
roboco/tests/unit/runtime/test_auditor_spawn_trigger.py
1c87a4e4e4 Leak fixes, gate green again, uv/CI hardening, e2e lifecycle smoke harness (#294)
* test: align phase1 smoke mock with the armed team-match gate

The 8e5f84c4 sweep fixed 13 test files' inconsistent-team mocks but ran
only the gateway/foundation/runtime subsets; the full gate caught this
integration mock whose parent task carried an auto-generated MagicMock
team and died on not_authorized before the incomplete_input assertion.

* fix(runtime): attribute every agent.spawned audit to its dispatcher

A rogue spawner could not be identified live (2026-07-02): agent.spawned
rows carry container/model but not which dispatch loop launched them.
spawn_agent now takes spawned_by, stamps it into the spawned/spawn_failed
audit details, every call site passes its loop name, and an AST sweep
test holds future callers to it.

* fix(api): admin-complete refuses when the task's PR is still open

PATCH status=completed on a task with an OPEN PR stranded its commits
unmerged (bit the CEO twice live 2026-07-02). The override now refuses
with the PR number/URL and the consequence before the generic hatch
text; force:true stays the deliberate, audited escape.

* fix(panel): awaiting_ceo_approval offers the working ceo-approve path

The header's only approve action was Approve & Merge (POST
/approve-and-merge, no notes) which 400s NO_PR on a branchless MegaTask
umbrella — the CEO's approve button just failed. Primary action is now
Approve & Complete via the CeoApproveDialog (POST /ceo-approve, notes
>=20 chars, proven live); Approve & Merge stays for PR-bearing tasks.

* test: stop leaking self-heal + rate-limit state into live Redis

Two test files wrote real keys into a developer's localhost Redis:
self-heal originate tests left self_heal:notified:* (2h TTL) and the
i_am_blocked rate-limited tests left a NO-TTL 'anthropic rate-limited'
tracker blob — order/state-dependent poison for anything reading the
real tracker, and the prime suspect class for the one-off
test_self_heal_engine full-run failure (not reproduced in 5x dir runs,
adversarial orders, and a green full gate). Both files now point the
computed redis_url at an unreachable port; the engines' fail-open paths
keep every assertion intact. Leaked keys scrubbed live.

* docs: changelog + map delta for the leak-fix batch; mypy-clean attribution test

The attribution test's direct method assignments tripped the full gate's
mypy (method-assign) — switched to the house monkeypatch idiom, no
suppressions.

* fix(gate): clear the ten xenon C-ranks; isolate all tests from live Redis

Master CI has been red at the phase1 smoke test, so neither CI nor a
local full gate had reached the xenon step since the team-match sweep —
whose inline 'agent_team=str(agent.team) if ...' kwarg pushed nine verb
bodies from B(10) to C(11-12) unseen. A shared actor_context_fields()
(_protocol.py) computes (actor_slug, agent_team) once per verb, restoring
all nine to B with zero behavior change; the new admin-complete override
helper extraction does the same for routes/tasks.py.

tests/conftest.py gains an autouse fixture pointing the computed
redis_url at an unreachable port for every test — the root fix for the
three families caught writing live-Redis keys (self-heal dedupe,
rate-limit tracker, notification purpose-dedupe); no test uses a real
Redis, and every production path is fail-open by design.

* refactor(runtime): delete the never-wired dispatch-time spawn cooldown

_safe_spawn / gateway_pre_spawn_check / trigger_filter had no caller in
the repo's entire history (87ef42bf only flipped the flag). Its five
rules are superseded: provider parking runs inside spawn_agent, claim
freshness is the guards+reaper, runaway respawns are the progress-aware
breaker + notification cooldown; the per-task cooldown rule would
queue-stall every normal stage handoff if wired today. gateway_triggers
table kept inert. Ratified by the CEO over wiring it.

* build: serialize uv — gate recipes never implicitly sync the venv

Every uv run re-syncs implicitly, so a background make quality plus any
foreground uv run raced two writers on one .venv and tore site-packages
apart (the recurring rich/pip/bandit ImportError corruption; bit twice
today, four times on 2026-07-02's first session). UV_NO_SYNC=1 is now
exported Makefile-wide and quality/quality-fast/gate depend on one
explicit up-front sync step.

* fix(git): PR/merge/branch REST calls honor github_api_base_url

Fifteen sites hardcoded https://api.github.com while the CI-run and
open-PR-list calls already read settings.github_api_base_url — a GHE or
test override silently applied to half the surface. One _api_base()
helper keeps them uniform; default behavior unchanged.

* ci: split the monolith — backend CI, Panel CI, E2E Smoke

ci.yml keeps its file name and the backend quality job only (self-heal /
ci-watch / release-readiness default to the ci.yml workflow); the panel
job moves to panel-ci.yml scoped to panel/**, and the new scripted-agent
lifecycle smoke gets e2e-smoke.yml + a make e2e-smoke target (env-gated
out of the default pytest run). Trade: a panel-only red now lands on
Panel CI, which the ci.yml-pinned watch engines don't see.

* feat(tests): e2e lifecycle smoke harness — scripted agents, real gates

tests/e2e_smoke stands up the real API (flow/do routers + middleware on
uvicorn) over the ephemeral test Postgres, a local bare origin standing
in for GitHub, and a fake GitHub REST layer whose merges are real git
merges. A deterministic driver reloads the real MCP flow/do modules per
agent and walks claim (real clone + worktree) -> tracing-gap -> note ->
plan gate -> commit -> PR -> the full i_am_done ladder -> QA verdicts ->
documenter -> awaiting_pm_review in ~5s. Runs via make e2e-smoke + its
own CI workflow; skipped (env-gated) in the default suite. The
freeze-lift condition's first half: scenario 1 green.

---------

Co-authored-by: Renn F <rennf93@users.noreply.github.com>
2026-07-02 18:28:07 +02:00

183 lines
6.0 KiB
Python

"""Wave C6 (2026-05-12): auditor spawns on escalation / block / cancel events.
The auditor's role is "silent observer" — read every channel, write
reflect notes on patterns worth surfacing. Smoke run 3 never spawned
auditor because no spawn trigger was registered for the events the
auditor cares about.
Handler lives in roboco/events/handlers.py and is registered by
register_default_handlers(). The orchestrator is injected via
set_event_context(orchestrator=...) so the handler can call
_context.orchestrator.spawn_agent("auditor").
"""
from __future__ import annotations
from unittest.mock import AsyncMock, MagicMock
from uuid import uuid4
import pytest
from roboco.events.handlers import (
handle_auditor_spawn,
register_default_handlers,
set_event_context,
)
from roboco.models.events import Event, EventContext, EventType
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _make_event(event_type: EventType, task_id: str | None = None) -> Event:
data: dict = {}
if task_id:
data["task_id"] = task_id
return Event(type=event_type, data=data)
def _make_context(spawn_mock: AsyncMock) -> EventContext:
orchestrator = MagicMock()
orchestrator.spawn_agent = spawn_mock
ctx = EventContext(orchestrator=orchestrator)
return ctx
# ---------------------------------------------------------------------------
# handle_auditor_spawn — unit tests (call the function directly)
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_auditor_spawns_on_task_blocked() -> None:
"""Emitting a task.blocked event triggers an auditor spawn."""
spawn = AsyncMock()
ctx = _make_context(spawn)
set_event_context(orchestrator=ctx.orchestrator)
task_id = str(uuid4())
event = _make_event(EventType.TASK_BLOCKED, task_id=task_id)
await handle_auditor_spawn(event)
spawn.assert_awaited_once_with(agent_id="auditor", spawned_by="event.auditor_spawn")
@pytest.mark.asyncio
async def test_auditor_spawns_on_task_cancelled() -> None:
"""task.cancelled triggers auditor spawn."""
spawn = AsyncMock()
ctx = _make_context(spawn)
set_event_context(orchestrator=ctx.orchestrator)
task_id = str(uuid4())
event = _make_event(EventType.TASK_CANCELLED, task_id=task_id)
await handle_auditor_spawn(event)
spawn.assert_awaited_once_with(agent_id="auditor", spawned_by="event.auditor_spawn")
@pytest.mark.asyncio
async def test_auditor_spawns_on_task_escalated_to_ceo() -> None:
"""task.awaiting_ceo_approval triggers auditor spawn."""
spawn = AsyncMock()
ctx = _make_context(spawn)
set_event_context(orchestrator=ctx.orchestrator)
task_id = str(uuid4())
event = _make_event(EventType.TASK_AWAITING_CEO_APPROVAL, task_id=task_id)
await handle_auditor_spawn(event)
spawn.assert_awaited_once_with(agent_id="auditor", spawned_by="event.auditor_spawn")
@pytest.mark.asyncio
async def test_auditor_does_not_spawn_when_no_orchestrator() -> None:
"""No orchestrator in context = silently skip, no crash."""
set_event_context(orchestrator=None)
event = _make_event(EventType.TASK_BLOCKED, task_id=str(uuid4()))
# Should not raise; nothing to assert except no exception
await handle_auditor_spawn(event)
@pytest.mark.asyncio
async def test_auditor_spawn_failure_does_not_propagate() -> None:
"""spawn_agent raising an exception must NOT bubble out of the handler.
Auditor spawn failure must not block the underlying event's processing.
"""
spawn = AsyncMock(side_effect=RuntimeError("container start failed"))
ctx = _make_context(spawn)
set_event_context(orchestrator=ctx.orchestrator)
event = _make_event(EventType.TASK_BLOCKED, task_id=str(uuid4()))
# Must not raise
await handle_auditor_spawn(event)
spawn.assert_awaited_once_with(agent_id="auditor", spawned_by="event.auditor_spawn")
# ---------------------------------------------------------------------------
# register_default_handlers — integration: verify subscriptions are wired
# ---------------------------------------------------------------------------
def test_auditor_handler_registered_for_blocked() -> None:
"""register_default_handlers subscribes handle_auditor_spawn to TASK_BLOCKED."""
bus = MagicMock()
bus.subscribe = MagicMock()
register_default_handlers(bus)
calls = [(c.args[0], c.args[1]) for c in bus.subscribe.call_args_list]
assert (EventType.TASK_BLOCKED, handle_auditor_spawn) in calls
def test_auditor_handler_registered_for_cancelled() -> None:
"""register_default_handlers subscribes handle_auditor_spawn to TASK_CANCELLED."""
bus = MagicMock()
bus.subscribe = MagicMock()
register_default_handlers(bus)
calls = [(c.args[0], c.args[1]) for c in bus.subscribe.call_args_list]
assert (EventType.TASK_CANCELLED, handle_auditor_spawn) in calls
def test_auditor_handler_registered_for_ceo_escalation() -> None:
"""handle_auditor_spawn is subscribed to TASK_AWAITING_CEO_APPROVAL."""
bus = MagicMock()
bus.subscribe = MagicMock()
register_default_handlers(bus)
calls = [(c.args[0], c.args[1]) for c in bus.subscribe.call_args_list]
assert (EventType.TASK_AWAITING_CEO_APPROVAL, handle_auditor_spawn) in calls
def test_auditor_handler_not_registered_for_routine_events() -> None:
"""Routine events (task.claimed, task.started) do NOT trigger auditor.
Auditor's mandate is exceptional events; routine progress doesn't
need observation.
"""
bus = MagicMock()
bus.subscribe = MagicMock()
register_default_handlers(bus)
calls = [(c.args[0], c.args[1]) for c in bus.subscribe.call_args_list]
routine_events = [
EventType.TASK_CLAIMED,
EventType.TASK_STARTED,
EventType.TASK_CREATED,
]
for routine_event in routine_events:
assert (routine_event, handle_auditor_spawn) not in calls, (
f"auditor must not subscribe to routine event {routine_event}"
)