Files
roboco/tests/unit/runtime/test_per_dev_lane_queue.py
T
cfde4369b1 Token optimization levers — claim-scoped briefing, payload caps, role-scoped optimal, notification-spawn cooldown (#292)
* feat(gateway): claim-scoped context briefing — heavy sections only on context-acquisition verbs

* feat(gateway): cap unbounded LLM-facing payloads — embedded diffs, notification bodies, handoff journal content, north star

* feat(mcp): role-scope the optimal server's tool groups; index management becomes dev/test-only

* feat(mcp): cap per-result content on kb/error/learning search, mentor sources, rag citations

* refactor(gateway): extract heavy-briefing sections + clip helper to keep xenon ranks

* feat(orchestrator): cross-tick cooldown for notification-triggered spawns

* feat(usage,orchestrator): scope spawn-waste to anthropic sessions; cap agent Bash output via settings env

* docs: claim-scoped briefing, payload caps, optimal role-scoping, notification-spawn cooldown

* test(mcp): type the mixed-item cap fixture explicitly

* fix(orchestrator): lazy-init the notification-spawn cooldown store

* fix(lifecycle): admin-override claim reconciliation + PM request_changes verb (S6 postmortem B3+B4)

B3 — admin_set_status now reconciles claim ownership when leaving BLOCKED:
review/queue targets clear claimed_by/claimed_at/active_claimant_id and
consume the pre-block snapshot (a stale escalation claim was stranding the
next claimant: give_me_work handed the task out while note() bounced
not_authorized — the live b8fe0494 wedge). The pending/in_progress restore
path also syncs active_claimant_id, and a REST PATCH unassign releases the
claim with it.

B4 — new PM verb request_changes: awaiting_pm_review -> needs_revision with
concrete issues. The PM previously had no reject at merge review (only
complete/escalate), so an AC/scope violation looped i_am_blocked->escalate
4x live. Full vertical: lifecycle transition + ActionSpec + IntentSpec,
TaskService.request_changes (routes like a QA fail — original dev for a
leaf, revision PM for assembled; issues appended to dev_notes), verb-runner
compose, choreographer verb (spec gate + non-empty issues + soup check +
a2a delivery of the reject reason), HTTP routes on both PM flows, MCP tool,
journal:decision tracing, PM prompts, regenerated lifecycle artifacts.

* fix(panel): stop scorecard fetches for fallback-roster placeholder ids

useAgents() serves the static AGENT_ROSTER (ids "1".."22") while agent
definitions load; the Scorecards tab fetched a member scorecard per row
immediately, firing 22 guaranteed-422 requests per refetch cycle. Through
the browser's per-origin connection limit those queued every metrics-page
query behind them (~10s of skeletons on every tab). Gate the fetch on a
real member id (agent UUID or the "ceo" alias).

* Upgraded uv.lock

* fix(sequencing): declared deps become real edges + full loop-breaker coverage + assembled-branch freshness (S6 postmortem B1/B2/B6 + breaker)

B1a — code delegations REQUIRE a collision surface: new TASK_AT_DELEGATE
completeness spec (conditional FieldRequirement, when=('task_type','code'))
enforced at the gateway delegate gate. A no-surface code sibling is
'parallel to everything' by analyzer design, which is how two devs ran the
CEO's explicitly-ordered work out of order (f3e1afc5: seq#1 started before
seq#0, zero dependency edges). PM prompts updated; REST/manual creation
(TASK_AT_CREATE) unchanged.

B1b — the CEO's declared 'Depends on' lists become real edges: DraftSurface
gains declared_depends_on; SequencingService.analyze unions declared edges
(validated: self/out-of-range rejected) with the derived collision rules,
cycle-checked by the existing toposort. confirm_live_batch/preview_batch
map each draft's depends_on through (string indices coerced); intake tool
doc + prompter role prompt instruct verbatim copying. The live S6 root got
1 of its 3 declared in-batch edges and started alongside still-running R3.

Breaker coverage — the progress-aware respawn circuit breaker
(_pm_respawn_should_gate: strike counting, status-advance reset,
tracing-gap budget, DB durability, one-shot CEO notification) was consulted
by only 3 spawn paths; the doc/QA/dev/PR-review/PR-gate/revision/board
paths spawned unguarded at fixed cadence (the 26-respawn fe-doc loop,
~$7.20). Now consulted at every task-keyed spawn site (14 total).

B2 — assembled-branch freshness: submit_up/submit_root auto-sync the
assembled branch when it has fallen behind its base (children are terminal
at submit time, so the rebase is safe; master is never written). A rebase
conflict is a hard reject naming the files instead of a blind re-review —
kills the needs_revision↔awaiting_pr_review ping-pong of re-submitting a
stale head. Leaf i_am_done already had the behind-base gate; claim-time
fetch-fresh cut already existed.

B6 — documenter revision-pass loop: the awaiting_documentation bail
rejections (i_am_blocked/unclaim) now name the actual exit (i_documented
re-affirm) and the documenter prompt gets an explicit revision-pass rule.

* fix(orchestration): assembly-integrity gate + dispatcher heartbeat (incidents #11, #1)

Assembly integrity — submit_up/submit_root refuse when a completed child's
commits are not patch-present in the assembled branch (git cherry —
rebase-safe; branch pruned after merge or any git error fails open). Live
incident #11: a completed revert subtask's merge was lost from the cell
branch and the review gate re-flagged the exact violation the revert fixed,
spawning another revision cycle.

Dispatcher heartbeat — a dispatcher.alive audit row every 5 minutes from
the dispatch loop. The 2026-07-01 outage was 4h25m of fleet-wide silence
with no way to distinguish 'loop dead' from 'no work'; the loop's stdout
died with the container while audit_log survives. CHANGELOG for tonight's
full sweep included.

* style: ruff format for the orchestration sweep

* refactor(gateway): fold the assembled-submit guards + trim complexity under the xenon gate

_assembled_submit_guards combines the #11 integrity check and B2 freshen for
submit_up/submit_root; lifecycle's invalid-source remediate and git's
per-child cherry probe extracted into helpers. Test harnesses built via
__new__ stub the respawn tracker (the breaker now runs on their paths).

---------

Co-authored-by: Renn F <rennf93@users.noreply.github.com>
2026-07-02 05:46:01 +02:00

203 lines
6.8 KiB
Python

"""Per-dev sequenced queues: a dev works its own code queue one task at a time.
A PM delegates a full per-dev queue of `code` subtasks up front. The dispatch
barrier holds a dev's higher-sequence code leaf until its own lower-sequence
code siblings under the same parent are terminal, so the dev works its queue in
order — while the OTHER dev's lane runs concurrently (two-dev parallelism).
Keyed on the assignee (not the team like the merge barrier) and gates only
`code`. Loop-free (not dispatched, not rejected); best-effort on lookup failure.
"""
from __future__ import annotations
from typing import Any, cast
from unittest.mock import AsyncMock, MagicMock, patch
from uuid import uuid4
import pytest
from roboco.models.base import TaskStatus
from roboco.runtime.orchestrator import AgentOrchestrator
def _new_orchestrator() -> AgentOrchestrator:
orch = AgentOrchestrator.__new__(AgentOrchestrator)
cast("Any", orch)._pm_respawn_tracker = {}
cast("Any", orch)._schedule_respawn_persist = lambda *_a, **_k: None
return orch
def _sibling(
seq: int,
owner: str,
status: TaskStatus,
*,
task_type: str = "code",
) -> MagicMock:
return MagicMock(
id=uuid4(),
sequence=seq,
assigned_to=owner,
status=status,
task_type=task_type,
)
def _patch_siblings(siblings: list[MagicMock]) -> Any:
"""Patch the orchestrator's direct-DB sibling lookup to return ``siblings``."""
svc = MagicMock()
svc.get_subtasks = AsyncMock(return_value=siblings)
class _CM:
async def __aenter__(self) -> MagicMock:
return MagicMock()
async def __aexit__(self, *_a: Any) -> bool:
return False
factory = MagicMock(return_value=_CM())
return (
patch("roboco.db.base.get_session_factory", return_value=factory),
patch("roboco.services.task.get_task_service", return_value=svc),
)
def _task(seq: int, owner: str, *, task_type: str = "code") -> dict[str, Any]:
return {
"id": str(uuid4()),
"parent_task_id": str(uuid4()),
"sequence": seq,
"assigned_to": owner,
"task_type": task_type,
}
@pytest.mark.asyncio
async def test_blocks_when_same_dev_has_earlier_active_code_sibling() -> None:
orch = _new_orchestrator()
task = _task(1, "be-dev-1")
siblings = [_sibling(0, "be-dev-1", TaskStatus.IN_PROGRESS)]
p1, p2 = _patch_siblings(siblings)
with p1, p2:
assert await orch._blocked_by_earlier_lane_sibling(task) is True
@pytest.mark.asyncio
async def test_not_blocked_when_earlier_same_dev_sibling_terminal() -> None:
orch = _new_orchestrator()
task = _task(1, "be-dev-1")
siblings = [
_sibling(0, "be-dev-1", TaskStatus.COMPLETED),
_sibling(0, "be-dev-1", TaskStatus.CANCELLED),
]
p1, p2 = _patch_siblings(siblings)
with p1, p2:
assert await orch._blocked_by_earlier_lane_sibling(task) is False
@pytest.mark.asyncio
async def test_other_devs_earlier_sibling_does_not_block() -> None:
"""The whole point of two-dev parallelism: be-dev-2's in-flight wave-0 leaf
must NOT hold be-dev-1's own wave-0 leaf. Lanes are independent."""
orch = _new_orchestrator()
task = _task(0, "be-dev-1")
siblings = [_sibling(0, "be-dev-2", TaskStatus.IN_PROGRESS)]
p1, p2 = _patch_siblings(siblings)
with p1, p2:
assert await orch._blocked_by_earlier_lane_sibling(task) is False
@pytest.mark.asyncio
async def test_earlier_non_code_sibling_does_not_block() -> None:
"""Only the code queue is gated this way; a planning/doc sibling is irrelevant."""
orch = _new_orchestrator()
task = _task(1, "be-dev-1")
siblings = [_sibling(0, "be-dev-1", TaskStatus.IN_PROGRESS, task_type="planning")]
p1, p2 = _patch_siblings(siblings)
with p1, p2:
assert await orch._blocked_by_earlier_lane_sibling(task) is False
@pytest.mark.asyncio
async def test_higher_sequence_same_dev_sibling_does_not_block() -> None:
orch = _new_orchestrator()
task = _task(0, "be-dev-1")
siblings = [_sibling(1, "be-dev-1", TaskStatus.IN_PROGRESS)]
p1, p2 = _patch_siblings(siblings)
with p1, p2:
assert await orch._blocked_by_earlier_lane_sibling(task) is False
@pytest.mark.asyncio
async def test_non_code_task_is_never_gated_without_db() -> None:
orch = _new_orchestrator()
# A planning/doc task short-circuits before any lookup.
task = _task(1, "fe-pm", task_type="planning")
assert await orch._blocked_by_earlier_lane_sibling(task) is False
@pytest.mark.asyncio
async def test_no_parent_or_owner_short_circuits_without_db() -> None:
orch = _new_orchestrator()
assert (
await orch._blocked_by_earlier_lane_sibling(
{"id": str(uuid4()), "sequence": 0, "task_type": "code"}
)
is False
)
@pytest.mark.asyncio
async def test_db_failure_falls_through_to_dispatch() -> None:
orch = _new_orchestrator()
task = _task(1, "be-dev-1")
boom = patch(
"roboco.db.base.get_session_factory", side_effect=RuntimeError("db down")
)
with boom:
assert await orch._blocked_by_earlier_lane_sibling(task) is False
@pytest.mark.asyncio
async def test_spawn_pending_dev_holds_gated_lane_before_validating(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""_spawn_pending_dev must short-circuit a gated lane before validating or
spawning — the dev's earlier queue item is still live."""
orch = _new_orchestrator()
task = _task(1, "be-dev-1")
spawn = AsyncMock()
validate = AsyncMock()
monkeypatch.setattr(orch, "_is_agent_active", MagicMock(return_value=False))
monkeypatch.setattr(
orch, "_blocked_by_earlier_lane_sibling", AsyncMock(return_value=True)
)
monkeypatch.setattr(orch, "_validate_task_for_spawn", validate)
monkeypatch.setattr(orch, "spawn_agent", spawn)
await orch._spawn_pending_dev(cast("Any", MagicMock()), task, "be-dev-1")
spawn.assert_not_awaited()
validate.assert_not_awaited()
@pytest.mark.asyncio
async def test_spawn_pending_dev_proceeds_when_lane_clear(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""When the lane is clear (no earlier sibling), the dev is spawned normally."""
orch = _new_orchestrator()
task = _task(0, "be-dev-1")
spawn = AsyncMock()
monkeypatch.setattr(orch, "_is_agent_active", MagicMock(return_value=False))
monkeypatch.setattr(
orch, "_blocked_by_earlier_lane_sibling", AsyncMock(return_value=False)
)
monkeypatch.setattr(orch, "_validate_task_for_spawn", AsyncMock(return_value=None))
monkeypatch.setattr(orch, "spawn_agent", spawn)
monkeypatch.setattr(orch, "_get_prompt_for_agent", MagicMock(return_value="prompt"))
monkeypatch.setattr(orch, "_task_git_context", MagicMock(return_value={}))
await orch._spawn_pending_dev(cast("Any", MagicMock()), task, "be-dev-1")
spawn.assert_awaited_once()