Files
roboco/tests/unit/api/test_orchestrator_manual_spawn.py
T
312ec990dd fix: prod triage 2026-07-08 — MCP auth residue, gateway envelopes, verb-loop cap, A2A interjection, manual spawn UX (#334)
* fix(auth): pass agent UUID to CLI-arg MCP servers (optimal/docs/search)

The container token is HMAC-signed over the agent UUID (#314), but the
optimal/docs/search MCP servers received the slug as their CLI arg and
sent X-Agent-ID=<slug>, so every research/RAG/docs call 401ed with
signature mismatch under enforced auth. Pass the already-computed
agent_uuid in the three args lists instead.

* fix(gateway): include remediate in gateway.rejected audit details

Conventions-gate rejections carry the offending file:line listing only
in the envelope's remediate field, which the audit row dropped -- ops
logs showed just the violation count with no way to see what blocked.

* fix(gateway): return envelope on do/commit git failure

A GitError from the commit verb propagated to the generic middleware
handler, so agents got a raw error blob with no remediate/next. Catch
it and return an error envelope; 'no changes added to commit' with an
explicit files list now names the mismatch and the omit-files fallback.

* fix(agent-sdk): absolute rejection cap breaks slow-drip verb loops

The verb circuit breaker only counted rejections inside a 60s sliding
window, so an agent retrying i_am_done every 3-4 minutes looped for 30+
minutes without tripping it. Add a session-scoped cumulative per-(verb,
task) cap at 3x the windowed limit that trips regardless of pacing.

* feat(a2a): CEO chime-in interjects into the viewed conversation

Previously reply_as_ceo re-homed the message into a canonical CEO<->target
conversation with no panel surface, so a chime-in reported success but was
invisible and only opportunistically delivered. interject_as_ceo now inserts
the message into the conversation being viewed (from_agent=ceo, directed via
an @target content prefix), bumps that conversation's counters with the
unread ping keyed to the addressed participant, and both participants see it
in transcript and read_a2a.

* feat(panel): manual spawn carries task + message, surfaces refusals

The agent detail page spawned with no request body (task/message impossible),
the spawn button could double-fire (2.5ms double-POST seen live), and refusal
reasons never reached the UI: readiness refusals were generic 500s and the
already-running no-op looked like success. Detail page now uses
SpawnAgentDialog, a synchronous ref guard blocks re-entry, AgentReadinessError
maps to 409 with its reason shown, already_running is signalled and toasted,
and a task_id builds a task-aware prompt instructing the claim (task_id alone
never did), with the CEO's message appended as a note.

* test(panel): align a2a page test with the interjection footer copy

The chime-in rebuild changed the composer footer; the page-level test
asserting the old copy was outside the rebuild's scoped vitest run.

* fix(api): commit the request DB session before the response is sent

FastAPI unwinds yield-dependencies after the response bytes go out, so
get_db's post-yield commit raced the client's next request -- a verb
could return ok while its claim/status write was still uncommitted (the
e2e ok-without-effect flake family), and a failed commit was silently
lost behind an already-sent 200. DbCommitMiddleware (innermost, pure
ASGI) commits the session stashed by get_db_committed before forwarding
http.response.start; commit failure now surfaces as a 5xx. get_db is
untouched for its direct non-request callers.

* fix(db): invalidate, not rollback, the session on request cancellation

With the commit moved into the send path, the flow-verb timeout can
cancel mid-commit; rolling back then issues another command over an
asyncpg connection stranded mid-wire-protocol, and the poisoned
connection segfaults uvloop/asyncpg when a later checkout recycles it
(3/3 identical CI faulthandler dumps). On CancelledError discard the
connection via session.invalidate() -- SQLAlchemy's documented handling
for a timeout during commit -- and keep rollback for plain exceptions.

---------

Co-authored-by: Renn F <rennf93@users.noreply.github.com>
2026-07-08 10:41:02 +02:00

278 lines
9.2 KiB
Python

"""Manual (panel) spawn: task-aware prompt helper + already-running signaling.
Covers the CEO-facing spawn-refusal / double-fire triage: a task-aware
initial prompt built server-side for a manual spawn (mirroring
``_build_pr_review_prompt``'s tone), an ``AgentReadinessError`` refusal
mapped to 409 (not an opaque 500) so the panel can show the real reason, and
an ``already_running`` marker so a no-op spawn (agent already active) is
distinguishable from a genuine new spawn.
"""
from __future__ import annotations
from datetime import UTC, datetime
from http import HTTPStatus
from types import SimpleNamespace
from typing import TYPE_CHECKING, cast
from unittest.mock import AsyncMock, MagicMock
from uuid import uuid4
import pytest
import pytest_asyncio
import roboco.api.routes.orchestrator as orch_route
from fastapi import FastAPI
from httpx import ASGITransport, AsyncClient
from roboco.api.deps import _ServiceHolder, set_orchestrator
from roboco.api.routes.orchestrator import (
_build_manual_spawn_prompt,
_resolve_manual_spawn_prompt,
)
from roboco.api.routes.orchestrator import (
router as orch_router,
)
from roboco.runtime.orchestrator import AgentReadinessError, AgentState
if TYPE_CHECKING:
from collections.abc import AsyncIterator
from roboco.db.tables import TaskTable
_HDR = {"X-Agent-ID": str(uuid4()), "X-Agent-Role": "ceo"}
def _fake_task(status_value: str = "pending") -> TaskTable:
# SimpleNamespace duck-types TaskTable's 3 fields the helper reads
# (id/title/status.value) without a real ORM row.
return cast(
"TaskTable",
SimpleNamespace(
id="task-123",
title="Fix the thing",
status=SimpleNamespace(value=status_value),
),
)
class _FakeDbCtx:
async def __aenter__(self) -> str:
return "fake-db"
async def __aexit__(self, *exc: object) -> bool:
return False
class _FakeTaskService:
def __init__(self, task: object | None = None, error: Exception | None = None):
self._task = task
self._error = error
async def get(self, _task_id: object) -> object | None:
if self._error:
raise self._error
return self._task
# ---------------------------------------------------------------------------
# _build_manual_spawn_prompt — pure formatting
# ---------------------------------------------------------------------------
def test_build_manual_spawn_prompt_includes_task_fields() -> None:
prompt = _build_manual_spawn_prompt(_fake_task("awaiting_qa"), None)
assert "TASK ID: task-123" in prompt
assert "TITLE: Fix the thing" in prompt
assert "STATUS: awaiting_qa" in prompt
assert "claim verb" in prompt.lower()
assert "CEO NOTE" not in prompt
def test_build_manual_spawn_prompt_appends_ceo_note() -> None:
prompt = _build_manual_spawn_prompt(_fake_task(), "Please prioritize this.")
assert "== CEO NOTE ==" in prompt
assert "Please prioritize this." in prompt
# CEO note comes after the task framing, not instead of it.
assert prompt.index("TASK ID") < prompt.index("CEO NOTE")
# ---------------------------------------------------------------------------
# _resolve_manual_spawn_prompt — best-effort enrichment
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_resolve_prompt_no_task_id_returns_message_unchanged() -> None:
result = await _resolve_manual_spawn_prompt(None, "hello")
assert result == "hello"
@pytest.mark.asyncio
async def test_resolve_prompt_enriches_when_task_found(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.setattr(orch_route, "get_db_context", _FakeDbCtx)
monkeypatch.setattr(
orch_route,
"get_task_service",
lambda _db: _FakeTaskService(task=_fake_task("verifying")),
)
result = await _resolve_manual_spawn_prompt(str(uuid4()), "Ship it")
assert result is not None
assert "STATUS: verifying" in result
assert "Ship it" in result
@pytest.mark.asyncio
async def test_resolve_prompt_falls_back_when_task_not_found(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.setattr(orch_route, "get_db_context", _FakeDbCtx)
monkeypatch.setattr(
orch_route, "get_task_service", lambda _db: _FakeTaskService(task=None)
)
result = await _resolve_manual_spawn_prompt(str(uuid4()), "hello")
assert result == "hello"
@pytest.mark.asyncio
async def test_resolve_prompt_falls_back_on_bad_task_id() -> None:
# Not a valid UUID — must not raise, must fall back unchanged.
result = await _resolve_manual_spawn_prompt("not-a-uuid", "hello")
assert result == "hello"
@pytest.mark.asyncio
async def test_resolve_prompt_falls_back_on_db_error(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.setattr(orch_route, "get_db_context", _FakeDbCtx)
monkeypatch.setattr(
orch_route,
"get_task_service",
lambda _db: _FakeTaskService(error=RuntimeError("db down")),
)
result = await _resolve_manual_spawn_prompt(str(uuid4()), "hello")
assert result == "hello"
@pytest.mark.asyncio
async def test_resolve_prompt_no_message_no_task_returns_none() -> None:
result = await _resolve_manual_spawn_prompt(None, None)
assert result is None
# ---------------------------------------------------------------------------
# Route: AgentReadinessError -> 409, already_running signaling
# ---------------------------------------------------------------------------
@pytest_asyncio.fixture
async def orch_client() -> AsyncIterator[tuple[AsyncClient, MagicMock]]:
app = FastAPI()
app.include_router(orch_router, prefix="/api/orchestrator")
orchestrator = MagicMock()
set_orchestrator(orchestrator)
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
yield client, orchestrator
_ServiceHolder.orchestrator = None
app.dependency_overrides.clear()
@pytest.mark.asyncio
async def test_spawn_readiness_refusal_maps_to_409(
orch_client: tuple[AsyncClient, MagicMock],
) -> None:
client, orch = orch_client
orch.get_instance = MagicMock(return_value=None)
orch.spawn_agent = AsyncMock(
side_effect=AgentReadinessError(
"spawn refused for fe-dev-2 (task=t1): state=awaiting_qa requires "
"role in {'qa'} but agent fe-dev-2 is 'developer'"
)
)
response = await client.post(
"/api/orchestrator/agents/fe-dev-2/spawn",
json={"agent_id": "fe-dev-2", "task_id": "t1"},
headers=_HDR,
)
assert response.status_code == HTTPStatus.CONFLICT
assert "requires role in" in response.json()["detail"]
@pytest.mark.asyncio
async def test_spawn_new_agent_not_flagged_already_running(
orch_client: tuple[AsyncClient, MagicMock],
) -> None:
client, orch = orch_client
orch.get_instance = MagicMock(return_value=None)
instance = SimpleNamespace(
id=uuid4(),
agent_id="be-dev-1",
state=AgentState.STARTING,
current_task_id=None,
error_count=0,
started_at=datetime.now(UTC),
)
orch.spawn_agent = AsyncMock(return_value=instance)
response = await client.post(
"/api/orchestrator/agents/be-dev-1/spawn", headers=_HDR
)
assert response.status_code == HTTPStatus.CREATED
assert response.json()["already_running"] is False
@pytest.mark.asyncio
async def test_spawn_already_running_agent_is_flagged(
orch_client: tuple[AsyncClient, MagicMock],
) -> None:
client, orch = orch_client
shared_id = uuid4()
existing = SimpleNamespace(
id=shared_id,
agent_id="ux-pm",
state=AgentState.STARTING,
current_task_id=None,
error_count=0,
started_at=datetime.now(UTC),
)
orch.get_instance = MagicMock(return_value=existing)
# spawn_agent's own no-op contract: hands back the SAME instance.
orch.spawn_agent = AsyncMock(return_value=existing)
response = await client.post("/api/orchestrator/agents/ux-pm/spawn", headers=_HDR)
assert response.status_code == HTTPStatus.CREATED
body = response.json()
assert body["already_running"] is True
assert body["state"] == "starting"
@pytest.mark.asyncio
async def test_spawn_offline_agent_not_flagged_already_running(
orch_client: tuple[AsyncClient, MagicMock],
) -> None:
"""A pre-existing OFFLINE instance is not "running" — a fresh spawn on top
of it must not be reported as a no-op."""
client, orch = orch_client
offline = SimpleNamespace(
id=uuid4(),
agent_id="be-dev-1",
state=AgentState.OFFLINE,
current_task_id=None,
error_count=0,
started_at=datetime.now(UTC),
)
orch.get_instance = MagicMock(return_value=offline)
new_instance = SimpleNamespace(
id=uuid4(),
agent_id="be-dev-1",
state=AgentState.STARTING,
current_task_id=None,
error_count=0,
started_at=datetime.now(UTC),
)
orch.spawn_agent = AsyncMock(return_value=new_instance)
response = await client.post(
"/api/orchestrator/agents/be-dev-1/spawn", headers=_HDR
)
assert response.status_code == HTTPStatus.CREATED
assert response.json()["already_running"] is False