[90c9474c] Auditor revival: scheduled audit trigger and reactive alert producers (#499)

* [927e64d5] Backend slice: auditor scheduled trigger and reactive alert producers (#496) * [1f2cdb4b] Reactive alert producers at QA-fail and rework (#492) * [1f2cdb4b] feat(services): add auditor-targeted rework alert producers at QA-fail and rework chokepoints * [1f2cdb4b] test(services): fix mypy typing in auditor alert producer unit tests * [1f2cdb4b] docs(backend): document reactive auditor rework alert producers in map and role docs --------- Co-authored-by: Backend Developer 2 <be-dev-2@roboco.tech> Co-authored-by: Backend Documenter <be-doc@roboco.tech> * [5173415f] Scheduled audit trigger, config, and sweep prompt (#493) * [5173415f] Add scheduled audit trigger, interval config, sweep prompt, and focused tests * [5173415f] Allow ROBOCO_AUDIT_INTERVAL_SECONDS=0 to disable scheduled sweeps * [5173415f] docs(audit): document scheduled auditor sweeps and ROBOCO_AUDIT_INTERVAL_SECONDS --------- Co-authored-by: Backend Developer 1 <be-dev-1@roboco.tech> Co-authored-by: Backend Documenter <be-doc@roboco.tech> * [a26c18b9] E2E smoke test for auditor triggers (#495) * [a26c18b9] Add e2e smoke test for auditor scheduled and reactive triggers * [a26c18b9] docs(tests): add e2e smoke test catalog and changelog entry for auditor triggers --------- Co-authored-by: Backend Developer 2 <be-dev-2@roboco.tech> Co-authored-by: Backend Documenter <be-doc@roboco.tech> * [3bc47cdc] Fix _fresh_orchestrator state for auditor trigger e2e tests (#497) * [3bc47cdc] fix(tests): initialize orchestrator state in _fresh_orchestrator helper * [3bc47cdc] docs(changelog): add _fresh_orchestrator test harness fix entry --------- Co-authored-by: Backend Developer 1 <be-dev-1@roboco.tech> Co-authored-by: Backend Documenter <be-doc@roboco.tech> * [8323cd50] Fix e2e smoke regression on assembled cell PR #496 (#498) * [8323cd50] fix(e2e_smoke): repair auditor-trigger smoke tests and harden harness * [8323cd50] docs(tests): document e2e smoke harness hardening for PR #498 --------- Co-authored-by: Backend Developer 1 <be-dev-1@roboco.tech> Co-authored-by: Backend Documenter <be-doc@roboco.tech> --------- Co-authored-by: Backend Developer 2 <be-dev-2@roboco.tech> Co-authored-by: Backend Documenter <be-doc@roboco.tech> Co-authored-by: Backend Developer 1 <be-dev-1@roboco.tech> * [37e6d999] Backend: repair failing CI checks on auditor revival PR #499 (#503) * [6e79bada] Triage and fix Python quality gate and Analyze (python) failures (#501) * [6e79bada] fix(task): replace type ignore with forward-reference cast for SQLAlchemy Mapped UUID in get_all_descendants * [6e79bada] fix(notification_delivery): add generic type arguments to dict return types in get_ack_status and get_delivery_summary * [6e79bada] docs(changelog): add Python quality gate type-hygiene fixes to Unreleased --------- Co-authored-by: Backend Developer 1 <be-dev-1@roboco.tech> Co-authored-by: Backend Documenter <be-doc@roboco.tech> * [50e7e104] Triage Analyze (javascript-typescript) failure on backend-only diff (#500) * [50e7e104] Split CodeQL workflow so JS/TS analyzer only runs on panel changes * [50e7e104] docs(backend): document split CodeQL workflow triggers and branch protection notes --------- Co-authored-by: Backend Developer 2 <be-dev-2@roboco.tech> Co-authored-by: Backend Documenter <be-doc@roboco.tech> * [203c426b] Triage and fix e2e lifecycle smoke (scripted agents) failure (#502) * [203c426b] fix(orchestrator): pre-initialize _instances in __new__ so __init__-bypass tests survive _dispatch_audit_work; allow audit_interval_seconds=0; mount /api/notifications in e2e harness * [203c426b] fix(e2e_smoke): restore ROBOCO_AGENT_TOKEN isolation and clarify /api/notifications mount comment * [203c426b] docs(map): document orchestrator __new__ pre-init and e2e harness token isolation for auditor-revival smoke fix --------- Co-authored-by: Backend Developer 1 <be-dev-1@roboco.tech> Co-authored-by: Backend Documenter <be-doc@roboco.tech> * [48cb05c2] Fix remaining e2e lifecycle smoke (scripted agents) failure on auditor-revival PR #503 (#504) * [48cb05c2] Harden AgentOrchestrator __new__ pre-init for auditor dispatch state * [48cb05c2] Document auditor-dispatch pre-init rationale in AgentOrchestrator __new__ * [48cb05c2] docs(orchestrator): extend __new__ pre-init docs for auditor-dispatch state --------- Co-authored-by: Backend Developer 1 <be-dev-1@roboco.tech> Co-authored-by: Backend Documenter <be-doc@roboco.tech> --------- Co-authored-by: Backend Developer 1 <be-dev-1@roboco.tech> Co-authored-by: Backend Documenter <be-doc@roboco.tech> Co-authored-by: Backend Developer 2 <be-dev-2@roboco.tech> * [90c9474c] intake: ambient workspace note + dedupe scope clones by git_url Two intake follow-ups folded into 90c9474c's spec Notes: (a) _resolve_intake_ambient now prepends a workspace note so the intake agent knows its cwd holds clones of every project in the scope (the primary at cwd, siblings alongside under /data/workspaces) and drafts against the real trees via Grep/Glob/Read, not from memory. (b) _clone_intake_scope dedupes slugs by git_url before cloning. A multi-project scope can list several projects pointing at one repo (a monorepo's cell-projects share a git_url); cloning each produced redundant identical workspaces. Mirrors CI-watch's per-git_url dedupe: keep the first slug per non-empty git_url; a project with no/empty git_url is never collapsed onto another so distinct local repos still clone. The dedupe is a pure static helper (_dedupe_slugs_by_git_url) with unit coverage. --------- Co-authored-by: Backend Developer 2 <be-dev-2@roboco.tech> Co-authored-by: Backend Documenter <be-doc@roboco.tech> Co-authored-by: Backend Developer 1 <be-dev-1@roboco.tech> Co-authored-by: Renn F <rennf93@users.noreply.github.com>
2026-08-03 07:23:24 +02:00 · 2026-07-13 15:41:00 +02:00
parent 69271f9e98
commit a3524da5f8
23 changed files with 1296 additions and 34 deletions
@@ -322,6 +322,7 @@ def _make_admin_clone(root: Path, origin: Path) -> Path:
 def _build_app(gh: _FakeGitHub) -> FastAPI:
    from roboco.api.middleware import setup_middleware
    from roboco.api.routes.health import router as health_router
+    from roboco.api.routes.notifications import router as notifications_router
    from roboco.api.routes.orchestrator import router as orchestrator_router
    from roboco.api.routes.settings import router as settings_router
    from roboco.api.routes.tasks import router as tasks_router
@@ -344,6 +345,10 @@ def _build_app(gh: _FakeGitHub) -> FastAPI:
    # The REST task surface — scenario 3 drives the real CEO
    # approve-and-merge endpoint (the human gate) through it.
    app.include_router(tasks_router, prefix="/api/tasks")
+    # The notifications router was already mounted here before the auditor
+    # revival diff; it remains so reactive audit dispatch (``_dispatch_audit_work``)
+    # can poll real ALERT rows end-to-end.
+    app.include_router(notifications_router, prefix="/api/notifications")
    # Cloud-auth gate coverage smoke exercises the real _require_ceo and
    # require_panel_token dep paths on these routers.
    app.include_router(orchestrator_router, prefix="/api/orchestrator")
@@ -472,6 +477,12 @@ class ScriptedAgent:
        os.environ["ROBOCO_AGENT_ROLE"] = self.role
        os.environ["ROBOCO_ORCHESTRATOR_URL"] = self.stack.base_url
        os.environ["ROBOCO_TOOL_MANIFEST_PATH"] = str(self._manifest_path)
+        # The host agent environment may carry a real ROBOCO_AGENT_TOKEN issued
+        # for the test runner's identity. flow_server reads it before each call
+        # and forwards it in X-Agent-Token; the token won't match the ephemeral
+        # test agent IDs and causes 401s. Drop it so tests run in the same
+        # unsigned-token mode as CI.
+        os.environ.pop("ROBOCO_AGENT_TOKEN", None)
        module = importlib.import_module(name)
        if getattr(module, "AGENT_ID", None) != str(self.agent_id):
            module = importlib.reload(module)
@@ -0,0 +1,230 @@
+"""e2e smoke test for auditor triggers.
+
+Exercises both the scheduled audit trigger path and the reactive alert producer
+path end-to-end, verifying they result in auditor-targeted work.
+
+- Scheduled path: an in-process orchestrator instance polls the real e2e API,
+  sees recent delivery activity, and calls ``spawn_agent(agent_id="auditor")``
+  with the scheduled sweep prompt.
+- Reactive path: a real QA-fail POST creates an ``ALERT`` notification addressed
+  to the auditor in the DB; the orchestrator's audit dispatcher fetches that
+  alert and calls ``spawn_agent(agent_id="auditor")`` with the quality-alert
+  prompt.
+
+No real auditor container is spawned — ``spawn_agent`` is stubbed so the test
+asserts on the dispatch decision, not the LLM runtime.
+"""
+
+from __future__ import annotations
+
+import asyncio
+from http import HTTPStatus
+from typing import TYPE_CHECKING, Any
+from unittest.mock import AsyncMock
+
+import httpx
+from roboco.config import settings
+from roboco.models import NotificationType
+from roboco.models.base import TaskStatus
+from roboco.runtime.orchestrator import _SYSTEM_API_HEADERS, AgentOrchestrator
+from tests.e2e_smoke.arcs import seed_company, seed_project, seed_task
+
+if TYPE_CHECKING:
+    from uuid import UUID
+
+    import pytest
+    from sqlalchemy.ext.asyncio import AsyncSession
+    from tests.e2e_smoke.harness import E2EStack
+
+
+def _agent_headers(agent_id: Any, role: str) -> dict[str, str]:
+    return {"X-Agent-ID": str(agent_id), "X-Agent-Role": role}
+
+
+def _seed_auditor_agent(stack: E2EStack) -> UUID:
+    """Seed the canonical auditor agent at its fixed foundation UUID.
+
+    ``_resolve_agent_slug`` maps this UUID to ``"auditor"`` so the orchestrator
+    recognises auditor-targeted notifications and spawns the right role.
+    """
+    from roboco.db.tables import AgentTable
+    from roboco.foundation import identity as _foundation
+    from roboco.models import AgentRole, AgentStatus
+
+    async def _run(session: AsyncSession) -> UUID:
+        auditor_id = _foundation.AGENTS["auditor"].uuid
+        session.add(
+            AgentTable(
+                id=auditor_id,
+                name="auditor",
+                slug="auditor",
+                role=AgentRole.AUDITOR,
+                team=None,
+                status=AgentStatus.ACTIVE,
+                model_config={},
+                system_prompt="auditor",
+                capabilities=[],
+                permissions={},
+                metrics={},
+            )
+        )
+        await session.flush()
+        return auditor_id
+
+    auditor_id: UUID = stack.run_db(_run)
+    return auditor_id
+
+
+def _notifications_for_task(
+    stack: E2EStack, task_id: Any, notification_type: Any
+) -> list[dict[str, Any]]:
+    from roboco.db.tables import NotificationTable
+    from sqlalchemy import select
+
+    async def _run(session: AsyncSession) -> list[dict[str, Any]]:
+        rows = (
+            (
+                await session.execute(
+                    select(NotificationTable).where(
+                        NotificationTable.related_task_id == task_id,
+                        NotificationTable.type == notification_type,
+                    )
+                )
+            )
+            .scalars()
+            .all()
+        )
+        return [
+            {
+                "type": str(r.type),
+                "related_task_id": r.related_task_id,
+                "subject": r.subject,
+                "priority": str(r.priority),
+                "to_agents": list(r.to_agents),
+            }
+            for r in rows
+        ]
+
+    rows: list[dict[str, Any]] = stack.run_db(_run)
+    return rows
+
+
+def _fresh_orchestrator(stack: E2EStack, monkeypatch: pytest.MonkeyPatch) -> Any:
+    """Return a bare orchestrator whose internal API points at the e2e app."""
+    # internal_api_url is a computed property; patch its input api_url instead.
+    monkeypatch.setattr(settings, "api_url", stack.base_url)
+    orch: Any = AgentOrchestrator.__new__(AgentOrchestrator)
+    # __new__ bypasses __init__, so the instance attributes that
+    # _is_agent_active and _dispatch_audit_work read must be initialized here.
+    orch._instances = {}
+    orch._last_audit_spawn_at = None
+    orch.spawn_agent = AsyncMock()
+    return orch
+
+
+def test_scheduled_audit_trigger_spawns_auditor(
+    e2e_stack: E2EStack, monkeypatch: pytest.MonkeyPatch
+) -> None:
+    """The scheduled sweep spawns the auditor when delivery activity is recent."""
+    stack = e2e_stack
+    company = seed_company(stack)
+    project_id, _project_slug = seed_project(stack, company)
+    auditor_id = _seed_auditor_agent(stack)
+
+    # Any active delivery task counts as "recent activity" for the sweep gate.
+    task_id = seed_task(
+        stack,
+        title="Scheduled audit smoke task",
+        description="An in-progress task so the scheduled sweep has work to audit.",
+        acceptance_criteria=["the scheduled path sees recent activity"],
+        project_id=project_id,
+        created_by=company.cell_pm_id,
+        assigned_to=company.dev_id,
+        claimed_by=company.dev_id,
+        active_claimant_id=company.dev_id,
+        status=TaskStatus.IN_PROGRESS,
+        branch_name="feature/backend/e2e-scheduled-audit",
+    )
+
+    orch = _fresh_orchestrator(stack, monkeypatch)
+    monkeypatch.setattr(settings, "audit_interval_seconds", 60)
+
+    async def _dispatch() -> None:
+        async with httpx.AsyncClient(
+            timeout=5.0, headers=_SYSTEM_API_HEADERS
+        ) as client:
+            await orch._dispatch_audit_work(client)
+
+    asyncio.run(_dispatch())
+
+    orch.spawn_agent.assert_awaited_once()
+    call = orch.spawn_agent.await_args
+    assert call is not None
+    assert call.kwargs["agent_id"] == "auditor"
+    assert call.kwargs["spawned_by"] == "_dispatch_audit_work"
+    prompt = call.kwargs["initial_prompt"]
+    assert "SCHEDULED AUDIT SWEEP" in prompt
+
+    # No reactive alert should have been created for this path.
+    assert _notifications_for_task(stack, task_id, NotificationType.ALERT) == []
+    assert auditor_id is not None  # auditor was seeded and resolved
+
+
+def test_reactive_alert_producer_spawns_auditor(
+    e2e_stack: E2EStack, monkeypatch: pytest.MonkeyPatch
+) -> None:
+    """QA-fail emits an auditor-targeted ALERT; the dispatcher spawns the auditor."""
+    stack = e2e_stack
+    company = seed_company(stack)
+    project_id, _project_slug = seed_project(stack, company)
+    auditor_id = _seed_auditor_agent(stack)
+
+    task_id = seed_task(
+        stack,
+        title="Reactive alert smoke task",
+        description="A task awaiting QA so fail_qa can emit an auditor alert.",
+        acceptance_criteria=["the reactive path emits an auditor alert"],
+        project_id=project_id,
+        created_by=company.cell_pm_id,
+        assigned_to=company.dev_id,
+        claimed_by=company.dev_id,
+        active_claimant_id=company.dev_id,
+        status=TaskStatus.AWAITING_QA,
+        branch_name="feature/backend/e2e-reactive-alert",
+    )
+
+    resp = httpx.post(
+        f"{stack.base_url}/api/tasks/{task_id}/fail-qa",
+        json={"notes": "missing edge-case coverage"},
+        headers=_agent_headers(company.qa_id, "qa"),
+        timeout=30,
+    )
+    assert resp.status_code == HTTPStatus.OK, (
+        f"fail-qa: {resp.status_code} {resp.text[:1500]}"
+    )
+
+    alerts = _notifications_for_task(stack, task_id, NotificationType.ALERT)
+    assert len(alerts) == 1, alerts
+    alert = alerts[0]
+    assert "rework alert" in alert["subject"].lower(), alert
+    assert auditor_id in alert["to_agents"], alert
+
+    orch = _fresh_orchestrator(stack, monkeypatch)
+    monkeypatch.setattr(settings, "audit_interval_seconds", 60)
+
+    async def _dispatch() -> None:
+        async with httpx.AsyncClient(
+            timeout=5.0, headers=_SYSTEM_API_HEADERS
+        ) as client:
+            await orch._dispatch_audit_work(client)
+
+    asyncio.run(_dispatch())
+
+    orch.spawn_agent.assert_awaited_once()
+    call = orch.spawn_agent.await_args
+    assert call is not None
+    assert call.kwargs["agent_id"] == "auditor"
+    assert call.kwargs["spawned_by"] == "_dispatch_audit_work"
+    prompt = call.kwargs["initial_prompt"]
+    assert "QUALITY ALERT" in prompt
+    assert "missing edge-case coverage" in prompt