mirror of
https://github.com/rennf93/roboco.git
synced 2026-08-03 07:23:24 +02:00
* test: align phase1 smoke mock with the armed team-match gate The8e5f84c4sweep fixed 13 test files' inconsistent-team mocks but ran only the gateway/foundation/runtime subsets; the full gate caught this integration mock whose parent task carried an auto-generated MagicMock team and died on not_authorized before the incomplete_input assertion. * fix(runtime): attribute every agent.spawned audit to its dispatcher A rogue spawner could not be identified live (2026-07-02): agent.spawned rows carry container/model but not which dispatch loop launched them. spawn_agent now takes spawned_by, stamps it into the spawned/spawn_failed audit details, every call site passes its loop name, and an AST sweep test holds future callers to it. * fix(api): admin-complete refuses when the task's PR is still open PATCH status=completed on a task with an OPEN PR stranded its commits unmerged (bit the CEO twice live 2026-07-02). The override now refuses with the PR number/URL and the consequence before the generic hatch text; force:true stays the deliberate, audited escape. * fix(panel): awaiting_ceo_approval offers the working ceo-approve path The header's only approve action was Approve & Merge (POST /approve-and-merge, no notes) which 400s NO_PR on a branchless MegaTask umbrella — the CEO's approve button just failed. Primary action is now Approve & Complete via the CeoApproveDialog (POST /ceo-approve, notes >=20 chars, proven live); Approve & Merge stays for PR-bearing tasks. * test: stop leaking self-heal + rate-limit state into live Redis Two test files wrote real keys into a developer's localhost Redis: self-heal originate tests left self_heal:notified:* (2h TTL) and the i_am_blocked rate-limited tests left a NO-TTL 'anthropic rate-limited' tracker blob — order/state-dependent poison for anything reading the real tracker, and the prime suspect class for the one-off test_self_heal_engine full-run failure (not reproduced in 5x dir runs, adversarial orders, and a green full gate). Both files now point the computed redis_url at an unreachable port; the engines' fail-open paths keep every assertion intact. Leaked keys scrubbed live. * docs: changelog + map delta for the leak-fix batch; mypy-clean attribution test The attribution test's direct method assignments tripped the full gate's mypy (method-assign) — switched to the house monkeypatch idiom, no suppressions. * fix(gate): clear the ten xenon C-ranks; isolate all tests from live Redis Master CI has been red at the phase1 smoke test, so neither CI nor a local full gate had reached the xenon step since the team-match sweep — whose inline 'agent_team=str(agent.team) if ...' kwarg pushed nine verb bodies from B(10) to C(11-12) unseen. A shared actor_context_fields() (_protocol.py) computes (actor_slug, agent_team) once per verb, restoring all nine to B with zero behavior change; the new admin-complete override helper extraction does the same for routes/tasks.py. tests/conftest.py gains an autouse fixture pointing the computed redis_url at an unreachable port for every test — the root fix for the three families caught writing live-Redis keys (self-heal dedupe, rate-limit tracker, notification purpose-dedupe); no test uses a real Redis, and every production path is fail-open by design. * refactor(runtime): delete the never-wired dispatch-time spawn cooldown _safe_spawn / gateway_pre_spawn_check / trigger_filter had no caller in the repo's entire history (87ef42bfonly flipped the flag). Its five rules are superseded: provider parking runs inside spawn_agent, claim freshness is the guards+reaper, runaway respawns are the progress-aware breaker + notification cooldown; the per-task cooldown rule would queue-stall every normal stage handoff if wired today. gateway_triggers table kept inert. Ratified by the CEO over wiring it. * build: serialize uv — gate recipes never implicitly sync the venv Every uv run re-syncs implicitly, so a background make quality plus any foreground uv run raced two writers on one .venv and tore site-packages apart (the recurring rich/pip/bandit ImportError corruption; bit twice today, four times on 2026-07-02's first session). UV_NO_SYNC=1 is now exported Makefile-wide and quality/quality-fast/gate depend on one explicit up-front sync step. * fix(git): PR/merge/branch REST calls honor github_api_base_url Fifteen sites hardcoded https://api.github.com while the CI-run and open-PR-list calls already read settings.github_api_base_url — a GHE or test override silently applied to half the surface. One _api_base() helper keeps them uniform; default behavior unchanged. * ci: split the monolith — backend CI, Panel CI, E2E Smoke ci.yml keeps its file name and the backend quality job only (self-heal / ci-watch / release-readiness default to the ci.yml workflow); the panel job moves to panel-ci.yml scoped to panel/**, and the new scripted-agent lifecycle smoke gets e2e-smoke.yml + a make e2e-smoke target (env-gated out of the default pytest run). Trade: a panel-only red now lands on Panel CI, which the ci.yml-pinned watch engines don't see. * feat(tests): e2e lifecycle smoke harness — scripted agents, real gates tests/e2e_smoke stands up the real API (flow/do routers + middleware on uvicorn) over the ephemeral test Postgres, a local bare origin standing in for GitHub, and a fake GitHub REST layer whose merges are real git merges. A deterministic driver reloads the real MCP flow/do modules per agent and walks claim (real clone + worktree) -> tracing-gap -> note -> plan gate -> commit -> PR -> the full i_am_done ladder -> QA verdicts -> documenter -> awaiting_pm_review in ~5s. Runs via make e2e-smoke + its own CI workflow; skipped (env-gated) in the default suite. The freeze-lift condition's first half: scenario 1 green. --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com>
135 lines
4.7 KiB
Python
135 lines
4.7 KiB
Python
"""Spawner attribution: every ``agent.spawned`` audit row names its dispatcher.
|
|
|
|
During the 2026-07-02 live run a rogue spawner could not be identified from the
|
|
audit log — ``agent.spawned`` rows carry the container/model but not WHICH
|
|
dispatch loop launched them. ``spawn_agent`` now takes ``spawned_by`` and
|
|
stamps it into the ``agent.spawned`` / ``agent.spawn_failed`` details, and an
|
|
AST sweep holds every call site to passing it.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import ast
|
|
import asyncio
|
|
from pathlib import Path
|
|
from typing import Any
|
|
from unittest.mock import AsyncMock
|
|
|
|
import pytest
|
|
from roboco.models.runtime import AgentInstance
|
|
from roboco.runtime.orchestrator import AgentConfig, AgentOrchestrator, AgentState
|
|
|
|
REPO_ROOT = Path(__file__).resolve().parents[3]
|
|
|
|
|
|
def _make_orchestrator(
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
captured: list[dict[str, Any]],
|
|
container_result: Any,
|
|
) -> AgentOrchestrator:
|
|
orch = AgentOrchestrator.__new__(AgentOrchestrator)
|
|
orch._instances = {}
|
|
orch._lock = asyncio.Lock()
|
|
orch._bg_tasks = set()
|
|
orch._running = True
|
|
monkeypatch.setattr(orch, "_fire_audit", lambda **kw: captured.append(kw))
|
|
monkeypatch.setattr(orch, "_record_spawn_session", AsyncMock(return_value=None))
|
|
monkeypatch.setattr(orch, "_spawn_container", container_result)
|
|
return orch
|
|
|
|
|
|
def _config_and_instance() -> tuple[AgentConfig, AgentInstance]:
|
|
config = AgentConfig(
|
|
agent_id="be-dev-1",
|
|
blueprint_path=Path(),
|
|
model="opus",
|
|
provider_type="anthropic",
|
|
)
|
|
instance = AgentInstance(
|
|
agent_id="be-dev-1", state=AgentState.STARTING, config=config
|
|
)
|
|
return config, instance
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_launch_spawn_audit_carries_spawned_by(
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
) -> None:
|
|
"""``agent.spawned`` details must name the dispatcher that launched it."""
|
|
captured: list[dict[str, Any]] = []
|
|
orch = _make_orchestrator(
|
|
monkeypatch, captured, AsyncMock(return_value="c0ffee" * 11)
|
|
)
|
|
config, instance = _config_and_instance()
|
|
|
|
await orch._launch_spawn(
|
|
"task-1", config, instance, None, None, spawned_by="_dispatch_qa_work"
|
|
)
|
|
|
|
spawned = [c for c in captured if c["event_type"] == "agent.spawned"]
|
|
assert len(spawned) == 1
|
|
assert spawned[0]["details"]["spawned_by"] == "_dispatch_qa_work"
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_spawn_failed_audit_carries_spawned_by(
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
) -> None:
|
|
"""A failed launch must attribute the spawner too — a rogue dispatcher
|
|
that keeps crashing containers is exactly the live-debug case."""
|
|
captured: list[dict[str, Any]] = []
|
|
orch = _make_orchestrator(
|
|
monkeypatch, captured, AsyncMock(side_effect=RuntimeError("boom"))
|
|
)
|
|
config, instance = _config_and_instance()
|
|
|
|
with pytest.raises(RuntimeError):
|
|
await orch._launch_spawn(
|
|
"task-1", config, instance, None, None, spawned_by="_spawn_pending_dev"
|
|
)
|
|
|
|
failed = [c for c in captured if c["event_type"] == "agent.spawn_failed"]
|
|
assert len(failed) == 1
|
|
assert failed[0]["details"]["spawned_by"] == "_spawn_pending_dev"
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_launch_spawn_without_attribution_stamps_unspecified(
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
) -> None:
|
|
"""The field is always present so audit queries never KeyError."""
|
|
captured: list[dict[str, Any]] = []
|
|
orch = _make_orchestrator(
|
|
monkeypatch, captured, AsyncMock(return_value="c0ffee" * 11)
|
|
)
|
|
config, instance = _config_and_instance()
|
|
|
|
await orch._launch_spawn("task-1", config, instance, None, None)
|
|
|
|
spawned = [c for c in captured if c["event_type"] == "agent.spawned"]
|
|
assert spawned[0]["details"]["spawned_by"] == "unspecified"
|
|
|
|
|
|
def _spawn_agent_calls_missing_spawned_by(path: Path) -> list[str]:
|
|
"""Return ``file:line`` for spawn_agent() calls without a spawned_by kwarg."""
|
|
tree = ast.parse(path.read_text())
|
|
missing: list[str] = []
|
|
for node in ast.walk(tree):
|
|
if not isinstance(node, ast.Call):
|
|
continue
|
|
func = node.func
|
|
if not (isinstance(func, ast.Attribute) and func.attr == "spawn_agent"):
|
|
continue
|
|
if not any(kw.arg == "spawned_by" for kw in node.keywords):
|
|
missing.append(f"{path.name}:{node.lineno}")
|
|
return missing
|
|
|
|
|
|
def test_every_spawn_agent_call_site_passes_spawned_by() -> None:
|
|
"""Sweep-guard over the whole package: a dispatcher added without
|
|
attribution fails here, not in a 3am live-debug session."""
|
|
missing: list[str] = []
|
|
for path in sorted((REPO_ROOT / "roboco").rglob("*.py")):
|
|
missing.extend(_spawn_agent_calls_missing_spawned_by(path))
|
|
assert not missing, f"spawn_agent() calls missing spawned_by=: {missing}"
|