Files
roboco/tests/unit/gateway/test_delegate_incomplete_input.py
T
e9ca7d4036 Delegation detail-fidelity + PM-loop hardening (#541)
* feat(gateway): delegation detail-fidelity — details survive hand-off, both directions

Details thinned out at every delegation hop: a PM child task mapped to no
parent criterion was legal (coverage only surfaced at submit_up, after the
whole wave ran — a 12-subtask docs tree grew through 8 review rounds that
way, one child titled 'docs page and route wrapper' shipping only the
page), and QA could pass work on a gestalt read (a 4-scene video brief
shipped 3 scenes past every gate because the features existed only in
prose). Three chokepoint gates:

- delegate (down): every child must declare covers_parent_criteria
  resolving against the parent's real acceptance criteria — no mapping or
  an unresolvable ref rejects naming every offending child and the valid
  criteria; the success envelope carries parent_ac_coverage
  {covered, uncovered} so a wave-planning PM sees remaining gaps in the
  same turn. Full coverage stays enforced at submit_up (waves stay legal).
- pass_review (up): mandatory criteria_verified — one {criterion,
  evidence} entry per task AC, matched by the findings ledger's
  id-or-exact-text matcher, evidence soup-checked and capped; rejects
  naming the unverified criteria; entries render deterministically into
  qa_notes as '[AC] <criterion> — verified: <evidence>' lines. The old
  count-only ac_verdicts gate is superseded (arg kept for back-compat).
- video briefs (structured detail at origination): an enumerable feature
  list (release highlights, or input_props.highlights carried onto a
  reject re-author) becomes its own scene acceptance criterion, bounded to
  the AC caps; a re-author without highlights carries the
  feedback-addressed criterion instead.

Extracted findings.py's criterion matcher into shared unmatched_criteria /
uncovered_acceptance_criteria instead of duplicating it; criteria_verified
joins the WAF free-text exclusion set like findings/issues.

* fix(gateway): break the block/unblock wedge — four hardening fixes from the live PM loop

A cell task looped fe-pm/main-pm block/unblock for hours (10 cycles, 43
spawns): a transient GitHub API error resolving CI became an unwaivable
blocker finding whose own fix text said no code change was required, the
submit freshness guard then demanded a commit no finding called for,
escalate_up auto-blocked, and main-pm's correct recovery plan 422'd on
the approach length cap, degrading it to a bare unblock. Four fixes:

- pr_pass CI-unresolvable refusal is now explicitly transient-worded:
  retry pr_pass shortly, do NOT pr_fail over a CI-status lookup error —
  a platform blip is not a code finding
- submit freshness guard grants ONE unchanged-head resubmission per
  head sha when the findings ledger has zero open rows (all addressed
  without code changes) — stamped via the resubmit_unchanged_head
  marker so the same head can never loop a second time
- unblock carries a flip breaker: block_flip_count marker, and at the
  third flip a one-shot CEO notification flags the task as structurally
  wedged (unblock itself still succeeds — the breaker signals, it does
  not wedge recovery)
- i_will_plan's approach cap truncates at 800 chars instead of
  rejecting — an over-detailed plan must never cost the PM its turn

---------

Co-authored-by: Renn F <rennf93@users.noreply.github.com>
2026-07-17 01:52:33 +02:00

307 lines
11 KiB
Python

"""Gateway delegate must return Envelope.incomplete_input when fields missing.
Pre-migration: services/gateway/choreographer/_impl.py:1852 used
``acceptance_criteria=inputs.acceptance_criteria or []`` which let ``[]``
through and was caught later (or, before that fix, silently substituted
by services/task.py:5061). Now the gateway rejects at the boundary
with ``Envelope.incomplete_input``, so the agent receives a structured
field-by-field guide (the spec §5.2.1 interrogation pattern).
"""
from __future__ import annotations
from datetime import UTC, datetime
from typing import Any
from unittest.mock import AsyncMock, MagicMock
from uuid import uuid4
import pytest
from roboco.services.gateway.choreographer import (
Choreographer,
ChoreographerDeps,
DelegateInputs,
)
def _make_deps(**overrides: Any) -> ChoreographerDeps:
base: dict[str, Any] = {
"task": AsyncMock(),
"work_session": AsyncMock(),
"git": AsyncMock(),
"a2a": AsyncMock(),
"journal": AsyncMock(),
"audit": AsyncMock(),
"evidence_repo": AsyncMock(),
}
base.update(overrides)
repo = base["evidence_repo"]
for method in (
"list_unread_a2a",
"list_unread_mentions",
"list_pending_notifications",
"task_metadata_gaps",
"recent_team_activity",
"blockers_in_lane",
"journal_highlights_for_task",
):
getattr(repo, method).return_value = []
# C8: default-fresh journal:decision so PM-decision gate passes.
# Tests that exercise the gate boundary stub their own value.
# The check matches MagicMock and AsyncMock (the two default sentinel
# types pytest's unittest.mock leaves on un-stubbed return_values).
_ldef = base["journal"].latest_decision_at.return_value
if type(_ldef).__name__ in ("MagicMock", "AsyncMock"):
base["journal"].latest_decision_at.return_value = datetime.now(UTC)
return ChoreographerDeps(**base)
def _parent_in_progress(pm_id: Any) -> MagicMock:
return MagicMock(
id=uuid4(),
project_id=uuid4(),
status="in_progress",
assigned_to=pm_id,
team="backend",
priority=2,
# delegate obligates the PM's quick_context resumption section.
quick_context="Decomposition planned; cells implement their slice next.",
# These tests are about task_completeness, not AC coverage.
acceptance_criteria=[],
)
@pytest.mark.asyncio
async def test_delegate_returns_incomplete_input_when_acceptance_criteria_empty() -> (
None
):
"""Empty acceptance_criteria triggers the incomplete_input envelope."""
pm_id = uuid4()
parent = _parent_in_progress(pm_id)
task_svc = AsyncMock()
task_svc.get.return_value = parent
task_svc.agent_for.return_value = MagicMock(role="cell_pm", team="backend")
task_svc.get_subtasks.return_value = []
deps = _make_deps(task=task_svc)
c = Choreographer(deps)
env = await c.delegate(
pm_id,
parent.id,
DelegateInputs(
title="Implement endpoint",
description="Add /v1/foo endpoint with passing tests please",
assigned_to="be-dev-1",
team="backend",
task_type="code",
nature="technical",
acceptance_criteria=[], # under-filled — must trigger interrogation
),
)
body = env.as_dict()
assert body["error"] == "incomplete_input", body
assert "acceptance_criteria" in body["missing"]
assert "acceptance_criteria" in body["field_hints"]
assert "verifiable" in body["field_hints"]["acceptance_criteria"].lower()
task_svc.create_subtask.assert_not_awaited()
@pytest.mark.asyncio
async def test_delegate_returns_incomplete_input_when_acceptance_criteria_none() -> (
None
):
"""None acceptance_criteria (missing field) triggers the same envelope."""
pm_id = uuid4()
parent = _parent_in_progress(pm_id)
task_svc = AsyncMock()
task_svc.get.return_value = parent
task_svc.agent_for.return_value = MagicMock(role="cell_pm", team="backend")
task_svc.get_subtasks.return_value = []
deps = _make_deps(task=task_svc)
c = Choreographer(deps)
env = await c.delegate(
pm_id,
parent.id,
DelegateInputs(
title="Implement endpoint",
description="Add /v1/foo endpoint with passing tests please",
assigned_to="be-dev-1",
team="backend",
task_type="code",
nature="technical",
acceptance_criteria=None,
),
)
body = env.as_dict()
assert body["error"] == "incomplete_input", body
assert "acceptance_criteria" in body["missing"]
task_svc.create_subtask.assert_not_awaited()
@pytest.mark.asyncio
async def test_delegate_rejects_silent_fallback_phrase() -> None:
"""The denylist catches the legacy 'completed and reviewed by assignee' string."""
pm_id = uuid4()
parent = _parent_in_progress(pm_id)
task_svc = AsyncMock()
task_svc.get.return_value = parent
task_svc.agent_for.return_value = MagicMock(role="cell_pm", team="backend")
task_svc.get_subtasks.return_value = []
deps = _make_deps(task=task_svc)
c = Choreographer(deps)
env = await c.delegate(
pm_id,
parent.id,
DelegateInputs(
title="Implement endpoint",
description="Add /v1/foo endpoint with passing tests please",
assigned_to="be-dev-1",
team="backend",
task_type="code",
nature="technical",
acceptance_criteria=["completed and reviewed by assignee"],
),
)
body = env.as_dict()
assert body["error"] == "incomplete_input", body
assert "acceptance_criteria" in body["missing"]
# The denylist hint should mention the placeholder/legacy fallback.
hint = body["field_hints"]["acceptance_criteria"].lower()
assert "placeholder" in hint or "legacy" in hint or "rejected" in hint
task_svc.create_subtask.assert_not_awaited()
@pytest.mark.asyncio
async def test_delegate_returns_incomplete_input_when_nature_missing() -> None:
"""Missing nature is also caught by task_completeness check."""
pm_id = uuid4()
parent = _parent_in_progress(pm_id)
task_svc = AsyncMock()
task_svc.get.return_value = parent
task_svc.agent_for.return_value = MagicMock(role="cell_pm", team="backend")
task_svc.get_subtasks.return_value = []
deps = _make_deps(task=task_svc)
c = Choreographer(deps)
env = await c.delegate(
pm_id,
parent.id,
DelegateInputs(
title="Implement endpoint",
description="Add /v1/foo endpoint with passing tests please",
assigned_to="be-dev-1",
team="backend",
task_type="code",
nature=None, # not declared
acceptance_criteria=["GET /v1/foo returns 200 with body"],
),
)
body = env.as_dict()
assert body["error"] == "incomplete_input", body
assert "nature" in body["missing"]
task_svc.create_subtask.assert_not_awaited()
@pytest.mark.asyncio
async def test_delegate_passes_when_payload_complete() -> None:
"""Fully-populated payload still creates the subtask (no regression)."""
pm_id = uuid4()
parent = _parent_in_progress(pm_id)
new_task = MagicMock(id=uuid4())
task_svc = AsyncMock()
task_svc.get.return_value = parent
task_svc.agent_for.return_value = MagicMock(role="cell_pm", team="backend")
task_svc.get_subtasks.return_value = []
task_svc.create_subtask.return_value = new_task
deps = _make_deps(task=task_svc)
c = Choreographer(deps)
env = await c.delegate(
pm_id,
parent.id,
DelegateInputs(
title="Implement endpoint",
description="Add /v1/foo endpoint with passing tests please",
assigned_to="be-dev-1",
team="backend",
task_type="code",
nature="technical",
acceptance_criteria=["GET /v1/foo returns 200 with body"],
intends_to_touch=["backend/api/routers/foo.py"],
),
)
assert env.error is None, env.as_dict()
task_svc.create_subtask.assert_awaited_once()
# Verify nature threaded through to TaskCreateRequest.
req = task_svc.create_subtask.call_args.args[0]
assert str(req.nature) == "technical" or req.nature.value == "technical"
@pytest.mark.asyncio
async def test_delegate_code_without_collision_surface_is_incomplete() -> None:
"""A code delegation with NO intends_to_touch is rejected at the boundary.
Live break (f3e1afc5, 2026-07-02): two code siblings delegated to two devs
with no surface — the collision analyzer treats a no-surface sibling as
parallel to everything, so the declared sequence was decorative and seq#1
started before seq#0 on a base missing its prerequisite.
"""
pm_id = uuid4()
parent = _parent_in_progress(pm_id)
task_svc = AsyncMock()
task_svc.get.return_value = parent
task_svc.agent_for.return_value = MagicMock(role="cell_pm", team="backend")
task_svc.get_subtasks.return_value = []
deps = _make_deps(task=task_svc)
c = Choreographer(deps)
env = await c.delegate(
pm_id,
parent.id,
DelegateInputs(
title="Implement endpoint",
description="Add /v1/foo endpoint with passing tests please",
assigned_to="be-dev-1",
team="backend",
task_type="code",
nature="technical",
acceptance_criteria=["GET /v1/foo returns 200 with body"],
),
)
body = env.as_dict()
assert body["error"] == "incomplete_input"
assert "intends_to_touch" in body.get("missing", [])
task_svc.create_subtask.assert_not_awaited()
@pytest.mark.asyncio
async def test_delegate_non_code_needs_no_collision_surface() -> None:
"""Research/design/documentation delegations stay surface-free."""
pm_id = uuid4()
parent = _parent_in_progress(pm_id)
new_task = MagicMock(id=uuid4())
task_svc = AsyncMock()
task_svc.get.return_value = parent
task_svc.agent_for.return_value = MagicMock(role="cell_pm", team="backend")
task_svc.get_subtasks.return_value = []
task_svc.create_subtask.return_value = new_task
deps = _make_deps(task=task_svc)
c = Choreographer(deps)
env = await c.delegate(
pm_id,
parent.id,
DelegateInputs(
title="Research retry semantics",
description="Survey retry/backoff libraries and summarize tradeoffs",
assigned_to="be-dev-1",
team="backend",
task_type="research",
nature="technical",
acceptance_criteria=["Summary doc lists at least 3 options"],
),
)
assert env.error is None, env.as_dict()
task_svc.create_subtask.assert_awaited_once()