Delegation detail-fidelity + PM-loop hardening (#541)

* feat(gateway): delegation detail-fidelity — details survive hand-off, both directions

Details thinned out at every delegation hop: a PM child task mapped to no
parent criterion was legal (coverage only surfaced at submit_up, after the
whole wave ran — a 12-subtask docs tree grew through 8 review rounds that
way, one child titled 'docs page and route wrapper' shipping only the
page), and QA could pass work on a gestalt read (a 4-scene video brief
shipped 3 scenes past every gate because the features existed only in
prose). Three chokepoint gates:

- delegate (down): every child must declare covers_parent_criteria
  resolving against the parent's real acceptance criteria — no mapping or
  an unresolvable ref rejects naming every offending child and the valid
  criteria; the success envelope carries parent_ac_coverage
  {covered, uncovered} so a wave-planning PM sees remaining gaps in the
  same turn. Full coverage stays enforced at submit_up (waves stay legal).
- pass_review (up): mandatory criteria_verified — one {criterion,
  evidence} entry per task AC, matched by the findings ledger's
  id-or-exact-text matcher, evidence soup-checked and capped; rejects
  naming the unverified criteria; entries render deterministically into
  qa_notes as '[AC] <criterion> — verified: <evidence>' lines. The old
  count-only ac_verdicts gate is superseded (arg kept for back-compat).
- video briefs (structured detail at origination): an enumerable feature
  list (release highlights, or input_props.highlights carried onto a
  reject re-author) becomes its own scene acceptance criterion, bounded to
  the AC caps; a re-author without highlights carries the
  feedback-addressed criterion instead.

Extracted findings.py's criterion matcher into shared unmatched_criteria /
uncovered_acceptance_criteria instead of duplicating it; criteria_verified
joins the WAF free-text exclusion set like findings/issues.

* fix(gateway): break the block/unblock wedge — four hardening fixes from the live PM loop

A cell task looped fe-pm/main-pm block/unblock for hours (10 cycles, 43
spawns): a transient GitHub API error resolving CI became an unwaivable
blocker finding whose own fix text said no code change was required, the
submit freshness guard then demanded a commit no finding called for,
escalate_up auto-blocked, and main-pm's correct recovery plan 422'd on
the approach length cap, degrading it to a bare unblock. Four fixes:

- pr_pass CI-unresolvable refusal is now explicitly transient-worded:
  retry pr_pass shortly, do NOT pr_fail over a CI-status lookup error —
  a platform blip is not a code finding
- submit freshness guard grants ONE unchanged-head resubmission per
  head sha when the findings ledger has zero open rows (all addressed
  without code changes) — stamped via the resubmit_unchanged_head
  marker so the same head can never loop a second time
- unblock carries a flip breaker: block_flip_count marker, and at the
  third flip a one-shot CEO notification flags the task as structurally
  wedged (unblock itself still succeeds — the breaker signals, it does
  not wedge recovery)
- i_will_plan's approach cap truncates at 800 chars instead of
  rejecting — an over-detailed plan must never cost the PM its turn

---------

Co-authored-by: Renn F <rennf93@users.noreply.github.com>
This commit is contained in:
Renzo F
2026-07-17 01:52:33 +02:00
committed by GitHub
co-authored by Renn F
parent 028b11871b
commit e9ca7d4036
34 changed files with 1823 additions and 140 deletions
+2
View File
@@ -418,6 +418,8 @@ Agent backends are pluggable. `roboco/llm/providers/` defines an `AgentProvider`
**Reviewer/PM collision map (always-on).** The collision surface authored at delegate time (`intends_to_touch` / `adds_migration` / `touches_shared`) used to be consumed once by `SequencingService` to wire dependency edges and never shown to a reviewer again. `build_collision_context` (`roboco/services/gateway/choreographer/collision.py`, pure — no DB/IO) now surfaces it for a task under review: same-parent siblings that would collide — overlapping declared file globs, or both adding a migration (the Alembic-head collision needs no file overlap) — rendered with the overlapping globs and, where the caller hands real touched files, a declared-vs-actual drift flag. Capped at 10 siblings / 5 globs. The same builder feeds QA's `claim_review` evidence, the PR-gate's `claim_gate_review` evidence, the PM's `i_will_plan` planning briefing (no drift there — no work yet), and the panel's `GET /api/tasks/{id}/collision-map` route backing a Collision tab on the task detail page.
**Delegation detail-fidelity (always-on, 2026-07-16).** Details no longer thin out at hand-off in either direction. DOWN: `delegate` refuses any child that doesn't declare `covers_parent_criteria` mapping onto the parent's real acceptance criteria (matched by id or exact text; an unresolvable ref is rejected naming the valid criteria, never silently dropped — previously the mapping was optional and coverage surfaced only at `submit_up`'s roll-up gate, after the whole wave had already run); the success envelope carries `parent_ac_coverage` `{covered, uncovered}` so a wave-planning PM sees remaining gaps in the same turn, while multi-wave planning stays legal. UP: `pass_review` requires `criteria_verified` — one `{criterion, evidence}` entry per task acceptance criterion (the findings ledger's id-or-exact-text matcher, soup-checked and length-capped evidence), rejecting with the unverified criteria named; entries render deterministically into `qa_notes` as `[AC] <criterion> — verified: <evidence>` lines, so a gestalt "looks good" pass is structurally impossible. Video briefs stopped being prose-only: an enumerable feature list (release `highlights`, or `input_props.highlights` carried onto a reject re-author) becomes its own acceptance criterion ("Every brief-named feature appears as its own fully readable scene: …", bounded to the AC caps; a re-author without highlights carries "every point in the CEO rejection feedback is visibly addressed"), so the dropped-scene class — a four-feature brief shipping three scenes past every gate — is caught by the QA per-AC stamp instead of the CEO's eyeball.
**PR labeler (always-on).** `derive_pr_labels` (`roboco/foundation/policy/pr_labels.py`, pure) derives the org-structure label vocabulary every fleet PR now carries: `to master` vs `to slave` (today, an assembled root→master PR is the only `to master` case — real slave-branch targeting lands with the slave/master fleet wiring), `root` for an assembled root PR, `MegaTask` for a batch-carrying task, and a layer label (`main-pm` for a Main-PM coordination root, `cell/{team}` for a cell-assembled PR, else `subtask/{team}` for a leaf dev PR). Applied best-effort at all three PR-opening sites in `GitService` so a human triaging the PR queue sees which tree and which org layer a PR belongs to at a glance.
**Feature flags / company-in-a-box.** Env-gated, default-off subsystems toggle from the panel's Settings → Feature Flags card (`panel/src/components/settings/feature-flags-card.tsx`) instead of hand-editing env: web research (`ROBOCO_RESEARCH_ENABLED`), the strategy engine (`ROBOCO_STRATEGY_ENGINE_ENABLED`), pitch provisioning (`ROBOCO_PROVISIONING_*`), external / internal PR review, the agent-runtime toolchain match (`ROBOCO_TOOLCHAIN_MATCH_ENABLED`), the architectural-conventions standard (`ROBOCO_CONVENTIONS_ENABLED`), gateway-health recovery (`ROBOCO_GATEWAY_HEALTH_ENABLED`), multi-repo CI-watch (`ROBOCO_CI_WATCH_ENABLED`), the dependency-update bot (`ROBOCO_DEP_UPDATE_ENABLED`), the gated release manager (`ROBOCO_RELEASE_MANAGER_ENABLED`), the organizational memory loop (`ROBOCO_ORG_MEMORY_ENABLED`), the sandboxed dev DB/Redis (`ROBOCO_SANDBOX_DB_ENABLED`), the RoboCo X account (`ROBOCO_X_ENGINE_ENABLED`), the RoboCo video engine (`ROBOCO_VIDEO_ENGINE_ENABLED`), the board roadmap engine (`ROBOCO_ROADMAP_ENGINE_ENABLED`), Fable-mode (`ROBOCO_FABLE_MODE_ENABLED`), the vault weekly report + KB ingest (`ROBOCO_VAULT_REPORT_ENABLED` / `ROBOCO_VAULT_KB_ENABLED`), the env-sync cascade (`ROBOCO_ENV_SYNC_ENABLED`), the Telegram notifications bridge (`ROBOCO_TELEGRAM_ENABLED`), the possibilities matrix (`ROBOCO_POSSIBILITIES_MATRIX_ENABLED`), the docs-divergence sync (`ROBOCO_DOCS_SYNC_ENABLED`), and the self-heal flags above. Cloud auth (`ROBOCO_CLOUD_AUTH_ENABLED`) is deliberately NOT on this card — like `ROBOCO_DB_NETWORK_ISOLATED`, it's a compose/env-coupled flag a runtime toggle can't safely flip mid-session. A toggle persists in the settings store and takes effect on the next backend restart; an unset flag falls back to its environment / config default.
+1 -1
View File
@@ -12,7 +12,7 @@
| `give_me_work` | `give_me_work()` |
| `i_am_blocked` | `i_am_blocked(task_id: UUID, reason: str, blocker_type: str | None = None, what_needed: str | None = None)` |
| `i_am_idle` | `i_am_idle()` |
| `pass_review` | `pass_review(task_id: UUID, notes: str, ac_verdicts: list[str] | None = None)` |
| `pass_review` | `pass_review(task_id: UUID, notes: str, ac_verdicts: list[str] | None = None, criteria_verified: list[str | Any] | None = None)` |
| `resume` | `resume(task_id: UUID)` |
| `unclaim` | `unclaim(task_id: UUID)` |
+1 -1
View File
@@ -57,7 +57,7 @@ real tools live in their agent_sdk drivers, not role_config.
| `give_me_work` | `give_me_work()` |
| `i_am_blocked` | `i_am_blocked(task_id: UUID, reason: str, blocker_type: str | None = None, what_needed: str | None = None)` |
| `i_am_idle` | `i_am_idle()` |
| `pass_review` | `pass_review(task_id: UUID, notes: str, ac_verdicts: list[str] | None = None)` |
| `pass_review` | `pass_review(task_id: UUID, notes: str, ac_verdicts: list[str] | None = None, criteria_verified: list[str | Any] | None = None)` |
| `resume` | `resume(task_id: UUID)` |
| `unclaim` | `unclaim(task_id: UUID)` |
+1 -1
View File
@@ -75,7 +75,7 @@ async def qa_pass(
choreographer: _ChoreographerDep,
) -> dict:
env = await choreographer.pass_review(
x_agent_id, body.task_id, body.notes, body.ac_verdicts
x_agent_id, body.task_id, body.notes, body.ac_verdicts, body.criteria_verified
)
return envelope_to_response(env, request)
+29 -6
View File
@@ -26,6 +26,10 @@ StrList = Annotated[list[str], BeforeValidator(coerce_str_list)]
_AC_MAX_ITEMS = 7
_AC_MAX_ITEM_CHARS = 200
# approach's truncate-not-reject ceiling — must match _PM_APPROACH_MIN_LEN's
# sibling ceiling in choreographer._impl (the gate re-derives its own copy).
_APPROACH_MAX_CHARS = 800
class SubTaskCreate(BaseModel):
"""A PM sub_task — a delegate target AND a progress-checklist item.
@@ -201,9 +205,19 @@ class PassReviewRequest(BaseModel):
ac_verdicts: StrList | None = Field(
default=None,
description=(
"One verification entry per acceptance criterion (in criterion "
"order) stating how QA verified it. Every criterion must be "
"covered before a pass is allowed."
"Legacy free-text per-criterion verdicts — still folded into the "
"persisted notes but no longer gates the pass (see "
"criteria_verified)."
),
)
criteria_verified: list[dict[str, Any]] | None = Field(
default=None,
description=(
"Mandatory when the task has acceptance criteria — one "
"{criterion, evidence} entry per criterion. criterion must match "
"an AC by id or exact text; evidence must be concrete (file:line, "
"screenshot ref, rendered-frame path, test name). Every criterion "
"must be covered or the pass is rejected."
),
)
@@ -348,9 +362,11 @@ class IWillPlanRequest(BaseModel):
# min_length must match choreographer._impl._PM_APPROACH_MIN_LEN. Raised
# 20→150: a 20-char approach was a one-liner; the approach +
# sub_tasks are also the progress checklist, so they must be substantive.
# max_length caps the bloat defect (approach that ran to thousands of
# chars restating the description). Must match the gate ceiling.
approach: str = Field(..., min_length=150, max_length=800)
# Overflow past 800 chars is truncated, not rejected — a 422 here used to
# throw away an otherwise-good plan and degrade the PM to a bare unblock
# (live incident: an over-length approach 422'd mid-flow). min_length
# stays a hard reject: a thin plan IS a defect.
approach: str = Field(..., min_length=150)
sub_tasks: list[SubTaskCreate] = Field(
default_factory=list,
description="List of {title, description} — server assigns id + order",
@@ -359,6 +375,13 @@ class IWillPlanRequest(BaseModel):
risks: list[RiskCreate] = Field(default_factory=list)
open_questions: list[OpenQuestionCreate] = Field(default_factory=list)
@field_validator("approach", mode="before")
@classmethod
def _truncate_approach(cls, v: object) -> object:
if isinstance(v, str) and len(v) > _APPROACH_MAX_CHARS:
return v[: _APPROACH_MAX_CHARS - 3] + "..."
return v
class DelegateRequest(BaseModel):
"""HTTP body for cell_pm + main_pm `delegate` verbs.
@@ -460,3 +460,62 @@ def get_docs_sync_release_version(task: HasMarkers) -> str | None:
def set_docs_sync_release_version(task: HasMarkers, version: str) -> None:
set_marker(task, DOCS_SYNC_RELEASE_VERSION, version)
# --- resubmit-unchanged-head exemption -------------------------------------
# submit_root/submit_up hard-refuse a re-submit whose assembled PR head is
# unchanged since the last pr_fail (the 2026-06-27 loop-stopper). That's a
# structural deadlock when the rejection round needed no code change (e.g. a
# transient CI-lookup error) — every ledger finding gets addressed/waived but
# the head sha never moves. This records the head sha that was granted ONE
# resubmit exemption, so a second attempt at the SAME head still refuses (see
# choreographer._unchanged_pr_guard).
RESUBMIT_UNCHANGED_HEAD = "resubmit_unchanged_head"
def get_resubmit_unchanged_head(task: HasMarkers) -> str | None:
val = get_marker(task, RESUBMIT_UNCHANGED_HEAD)
return str(val) if val else None
def set_resubmit_unchanged_head(task: HasMarkers, head_sha: str) -> None:
set_marker(task, RESUBMIT_UNCHANGED_HEAD, head_sha)
# --- block/unblock flip breaker --------------------------------------------
# Counts successful `unblock` calls on a task so a repeating block/unblock
# cycle (e.g. escalate_up auto-blocks, a PM unblocks, repeat — live incident:
# 10 flips, 43 spawns with no forward progress) can alert the CEO instead of
# burning spawns forever. Payload: {"count": int, "notified": bool} — one key
# so the counter and the notified-once flag stay atomic with each other.
BLOCK_FLIP_COUNT = "block_flip_count"
def get_block_flip_count(task: HasMarkers) -> int:
val = get_marker(task, BLOCK_FLIP_COUNT)
count = val.get("count") if isinstance(val, dict) else None
return int(count) if isinstance(count, int) else 0
def is_block_flip_notified(task: HasMarkers) -> bool:
val = get_marker(task, BLOCK_FLIP_COUNT)
return bool(val.get("notified")) if isinstance(val, dict) else False
def bump_block_flip_count(task: HasMarkers) -> int:
"""Increment the flip counter; returns the new count."""
count = get_block_flip_count(task) + 1
set_marker(
task,
BLOCK_FLIP_COUNT,
{"count": count, "notified": is_block_flip_notified(task)},
)
return count
def mark_block_flip_notified(task: HasMarkers) -> None:
set_marker(
task, BLOCK_FLIP_COUNT, {"count": get_block_flip_count(task), "notified": True}
)
+25 -9
View File
@@ -702,17 +702,29 @@ def claim_review(task_id: str) -> dict[str, Any]:
def pass_review(
task_id: str, notes: str, ac_verdicts: StrList | None = None
task_id: str,
notes: str,
ac_verdicts: StrList | None = None,
criteria_verified: list[dict[str, Any]] | None = None,
) -> dict[str, Any]:
"""QA: accept the work. notes >= 80 chars; journal:learning required.
ac_verdicts: one entry per acceptance criterion (in criterion order) stating
how you verified it. Every criterion must be covered — a pass is rejected
until all are. If any criterion does not hold, call fail_review instead.
criteria_verified: MANDATORY when the task has acceptance criteria — one
{criterion, evidence} entry per criterion. criterion must match an AC by
id or exact text; evidence must be concrete (file:line, screenshot ref,
rendered-frame path, test name), not a gestalt "looks good". Every
criterion must be covered, or the pass is rejected naming exactly which
are missing/unmatched. If any criterion does not hold, call fail_review
instead of passing a partial.
ac_verdicts: legacy free-text, one entry per acceptance criterion — still
folded into the persisted notes but no longer gates the pass.
"""
payload: dict[str, Any] = {"task_id": task_id, "notes": notes}
if ac_verdicts is not None:
payload["ac_verdicts"] = ac_verdicts
if criteria_verified is not None:
payload["criteria_verified"] = criteria_verified
return _post(_role_path("pass"), payload)
@@ -965,11 +977,15 @@ def delegate(
nature: One of "technical" | "non_technical".
acceptance_criteria: Non-empty list of verifiable outcome strings.
estimated_complexity: One of "low" | "medium" | "high". Default "medium".
covers_parent_criteria: The parent task's acceptance-criterion ids this
subtask is responsible for. Declare these so the gateway can verify
EVERY parent criterion is claimed by a subtask and satisfied before
the parent rolls up — split the parent's criteria across subtasks so
their union covers all of them.
covers_parent_criteria: The parent task's acceptance-criterion ids (or
exact text) this subtask is responsible for. REQUIRED — non-empty
and resolvable — whenever the parent carries acceptance criteria;
delegate rejects a child that maps to none of them or to a ref
that matches no real criterion, naming the parent's actual
criteria in the rejection. A single delegate need not cover every
criterion (a wave may leave some for a later delegate — see the
response's evidence.parent_ac_coverage), but every child that IS
delegated must map to something real.
intends_to_touch: Collision surface — file paths/globs this subtask
will modify. REQUIRED for task_type="code": the sibling collision
DAG can only sequence what is declared.
+1
View File
@@ -231,6 +231,7 @@ _WAF_FREETEXT_BODY_FIELDS: set[str] = {
"consequences",
"content",
"context",
"criteria_verified",
"decision",
"description",
"details",
+210 -51
View File
@@ -102,6 +102,11 @@ _PM_SUBTASK_DESC_MAX_LEN = 600
# tasks, not one giant plan.
_PM_SUBTASKS_MAX = 7
# unblock's flip breaker: alert the CEO once a task has flip-flopped
# block/unblock this many times (live incident: 10 flips, 43 spawns with no
# forward progress before anyone noticed).
_BLOCK_FLIP_NOTIFY_THRESHOLD = 3
def _thin_subtask_hint(sub_tasks: list[Any]) -> str | None:
"""Return a hint if any PM sub_task is title-only / thin / over-long.
@@ -5206,8 +5211,9 @@ class Choreographer:
)
# Spec gate passed. Run delegate-specific guards the spec doesn't
# model: tracing (journal:decision), chain validation, enum
# coercion + assignee-vs-task_type, and parent-ownership/subtask-cap.
guard = await self._delegate_extra_guards(
# coercion + assignee-vs-task_type, parent-ownership/subtask-cap, and
# (last) decomposition-coverage.
guard = await self._delegate_post_spec_guards(
pm_agent_id, parent_task_id, parent, role_str, inputs
)
if guard is not None:
@@ -5221,6 +5227,75 @@ class Choreographer:
pm_agent_id, parent, inputs, briefing, role_str
)
async def _delegate_post_spec_guards(
self,
pm_agent_id: UUID,
parent_task_id: UUID,
parent: Any,
role_str: str,
inputs: DelegateInputs,
) -> Envelope | None:
"""``_delegate_extra_guards``, then the decomposition-coverage gate.
Split out only to keep ``delegate()``'s own return count under the
complexity budget. The coverage gate deliberately runs LAST, after
every cap/chain/tracing/static rejection, so those guards keep their
own specific message (existing tests assert on it) and the coverage
gate only ever fires on an otherwise-valid delegate.
"""
guard = await self._delegate_extra_guards(
pm_agent_id, parent_task_id, parent, role_str, inputs
)
if guard is not None:
return guard
return self._delegate_ac_coverage_guard(parent, inputs)
def _delegate_ac_coverage_guard(
self, parent: Any, inputs: DelegateInputs
) -> Envelope | None:
"""Reject a child that doesn't map to the parent's own criteria.
A parent with real acceptance criteria (every task has some) requires
every child to declare which ones it advances via
``covers_parent_criteria`` a child mapped to nothing is either scope
creep or a sign the parent's ACs are stale, and either way the PM must
decide now, not discover the gap late at submit_up's roll-up gate
(``_parent_acs_covered_envelope``) after the whole wave already ran.
Inert when the parent has no acceptance criteria at all (nothing to
map against) mirrors the safe-by-construction posture of
``uncovered_parent_acceptance_criteria`` et al. Does NOT require full
coverage in one call: a wave may deliberately leave criteria for a
later delegate (see the success envelope's ``parent_ac_coverage``
evidence for that signal).
"""
ac_texts = parent.acceptance_criteria or []
if not ac_texts:
return None
refs = inputs.covers_parent_criteria or []
bad = self.task.unknown_ac_refs(parent, refs) if refs else []
if refs and not bad:
return None
listing = "; ".join(ac_texts)
if not refs:
message = (
f"'{inputs.title}' declares no covers_parent_criteria, but the "
"parent has acceptance criteria to decompose."
)
else:
message = (
f"'{inputs.title}' covers_parent_criteria has unresolvable "
f"ref(s): {'; '.join(bad)}"
)
return Envelope.invalid_state(
message=message,
remediate=(
"Map this subtask to the parent criteria it advances via "
"covers_parent_criteria (by id or exact text), or fix the "
f"parent's criteria first. Parent criteria: {listing}"
),
context_briefing={},
)
# Gate Set B subtask cap (pre-gateway implicit, made explicit here).
# Soft warn at 8, hard block at 13. Cap enforced by ``_subtask_cap_guard``.
_SUBTASK_HARD_CAP: int = 12
@@ -5977,9 +6052,29 @@ class Choreographer:
status="created",
task_id=str(new_task.id),
next=spec_module._INTENT_VERBS["delegate"].next_hint(new_task),
evidence=await self._delegate_ac_coverage_evidence(parent, parent_task_id),
context_briefing=briefing,
).with_introspection(task=new_task, role=role_str)
async def _delegate_ac_coverage_evidence(
self, parent: Any, parent_task_id: UUID
) -> dict[str, Any]:
"""``evidence.parent_ac_coverage`` for a successful delegate.
Reuses ``uncovered_parent_acceptance_criteria`` the exact primitive
submit_up's roll-up gate checks — so the PM sees, in the same turn,
which parent criteria the wave-so-far still leaves uncovered instead
of discovering it later at assembly. Wave-based planning stays legal:
this never blocks, it only reports. Empty (``{}``) when the parent
carries no acceptance criteria at all.
"""
ac_texts = parent.acceptance_criteria or []
if not ac_texts:
return {}
uncovered = await self.task.uncovered_parent_acceptance_criteria(parent_task_id)
covered = [t for t in ac_texts if t not in uncovered]
return {"parent_ac_coverage": {"covered": covered, "uncovered": uncovered}}
@staticmethod
def _team_value(value: Any) -> str:
return value.value if hasattr(value, "value") else str(value)
@@ -6792,6 +6887,7 @@ class Choreographer:
)
t = await self.task.unblock_with_restore(pm_agent_id, task_id, restore=restore)
await self._maybe_notify_block_flip(task_id, t)
next_msg = (
"task restored to its pre-block state — original assignee will resume"
if restore
@@ -6804,6 +6900,39 @@ class Choreographer:
context_briefing=await self._briefing_for(pm_agent_id, task_id),
).with_introspection(task=t, role=role)
async def _maybe_notify_block_flip(self, task_id: UUID, t: Any) -> None:
"""Bump the flip counter; alert the CEO once past the threshold.
A resolver that keeps unblocking a task that keeps re-blocking
(fe-pm escalate_up auto-blocks / main_pm unblocks, live incident: 10
flips, 43 spawns) is a structural wedge, not automatic recovery
working as intended. Signal it once at the 3rd flip; the unblock
itself always still succeeds regardless.
"""
flip_count = markers.bump_block_flip_count(t)
if (
flip_count >= _BLOCK_FLIP_NOTIFY_THRESHOLD
and not markers.is_block_flip_notified(t)
):
markers.mark_block_flip_notified(t)
await self._notify_ceo_block_flip(task_id, flip_count)
async def _notify_ceo_block_flip(self, task_id: UUID, flip_count: int) -> None:
"""Best-effort CEO alert for a repeating block/unblock cycle; never
raises the breaker signals, it does not wedge unblock itself."""
from roboco.services.notification import NotificationService
try:
await NotificationService().send_block_flip_notification(
task_id=str(task_id), flip_count=flip_count
)
except Exception:
logger.warning(
"failed to send CEO block-flip notification",
task_id=str(task_id),
flip_count=flip_count,
)
async def _own_review_hint(self, pm_agent_id: UUID, exclude_task_id: UUID) -> str:
"""Remediate suffix naming the PM's OWN task ready to complete.
@@ -7248,25 +7377,40 @@ class Choreographer:
)
return None
async def _submit_root_unchanged_pr_guard(
self, t: Any, briefing: dict[str, Any]
async def _unchanged_pr_guard(
self,
t: Any,
briefing: dict[str, Any],
*,
verb: str,
pr_scope: str,
delegate_target: str,
subtask_noun: str,
work_noun: str,
) -> Envelope | None:
"""Refuse to re-submit a root PR whose diff is unchanged since the last fail.
"""Shared unchanged-PR loop-stopper for submit_root (root) / submit_up (cell).
The hard loop-stopper for the 2026-06-27 ``pr_fail`` re-submit loop. A
prior ``pr_fail`` stamped the assembled PR's head SHA into
``notes_structured.pr_review.head_sha`` (see ``_capture_pr_head_sha`` /
``_record_gate_verdict``). On the next ``submit_root`` this looks up the
PR's CURRENT head SHA and compares. Equal ⇒ no new cell work landed on
the root branch since the fail the diff the reviewer would see is
byte-identical to the one just rejected refuse, so no model can loop
itself back into ``awaiting_pr_review``. Different the branch advanced
allow. Returns ``None`` (proceed) on every ambiguous case so the gate
FAILS OPEN: no prior ``pr_fail`` verdict, no recorded ``head_sha`` (e.g.
a verdict written before this field existed), no ``pr_number``, no
``_record_gate_verdict``). This looks up the PR's CURRENT head SHA and
compares. Equal no new work has landed on the branch since the fail
the diff the reviewer would see is byte-identical to the one just
rejected. Different the branch advanced allow.
A same-head resubmit isn't always a loop: when the rejection round's
findings needed no code change (e.g. a transient CI-lookup error
ledgered as a finding, then waived/addressed), a flat refuse deadlocks
the task forever the head sha can never move because there is
nothing to fix. So once every ledger finding is resolved
(``_open_finding_ids`` empty), ONE resubmission is exempted per head
sha (``markers.resubmit_unchanged_head``); a second attempt at the SAME
head still refuses one-shot, not a standing bypass.
Returns ``None`` (proceed) on every ambiguous case so the gate FAILS
OPEN: no prior ``pr_fail`` verdict, no recorded ``head_sha`` (e.g. a
verdict written before this field existed), no ``pr_number``, no
resolvable project, or a git/closed-PR lookup that returns ``None``.
Only the exact-unchanged case is hard-blocked; the rest fall through to
the reviewer, who can still ``pr_fail`` if the diff is bad.
"""
pr_review = (getattr(t, "notes_structured", None) or {}).get("pr_review") or {}
if pr_review.get("verdict") != "failed":
@@ -7277,24 +7421,55 @@ class Choreographer:
current = await self._current_pr_head_sha(t)
if current is None or current != recorded:
return None
exemption_note = ""
if not await self._open_finding_ids(t.id):
if markers.get_resubmit_unchanged_head(t) == current:
exemption_note = (
" (the one-shot exemption for this head was already used)"
)
else:
markers.set_resubmit_unchanged_head(t, current)
logger.info(
"unchanged-PR resubmit exemption granted — all findings"
" resolved without a code change",
task_id=str(t.id),
verb=verb,
head=current[:7],
)
return None
return Envelope.invalid_state(
message=(
"the assembled root PR is unchanged since the last pr_fail"
f" (head {current[:7]}). No new cell work has landed on the"
" root branch, so re-submitting would re-open the exact diff"
" the reviewer just rejected and loop straight back to"
" awaiting_pr_review."
f"the assembled {pr_scope} PR is unchanged since the last"
f" pr_fail (head {current[:7]}). No new {work_noun} has"
f" landed on the {pr_scope} branch, so re-submitting would"
" re-open the exact diff the reviewer just rejected and loop"
f" straight back to awaiting_pr_review.{exemption_note}"
),
remediate=(
"re-delegate the fixes to the owning cell PM(s) via"
" delegate(...) and wait for the cell subtasks to complete"
" and the root branch to be re-assembled. Do NOT call"
" submit_root again until new cell work has advanced the"
" root branch HEAD."
f"re-delegate the fixes to the {delegate_target} via"
f" delegate(...) and wait for the {subtask_noun} to complete"
f" and the {pr_scope} branch to be re-assembled. Do NOT call"
f" {verb} again until new {work_noun} has advanced the"
f" {pr_scope} branch HEAD."
),
context_briefing=briefing,
)
async def _submit_root_unchanged_pr_guard(
self, t: Any, briefing: dict[str, Any]
) -> Envelope | None:
"""Refuse to re-submit a root PR whose diff is unchanged since the last
fail. See ``_unchanged_pr_guard`` for the shared logic."""
return await self._unchanged_pr_guard(
t,
briefing,
verb="submit_root",
pr_scope="root",
delegate_target="owning cell PM(s)",
subtask_noun="cell subtasks",
work_noun="cell work",
)
async def _current_pr_head_sha(self, t: Any) -> str | None:
"""Best-effort current head SHA of the task's assembled PR (fail-open).
@@ -7338,33 +7513,17 @@ class Choreographer:
Refuses re-submit when the cell PR's current head SHA equals the SHA
``pr_fail`` recorded in ``notes_structured.pr_review.head_sha`` (no new
dev work landed byte-identical diff). Ambiguous cases FAIL OPEN via
``_current_pr_head_sha``; only the exact-unchanged case is hard-blocked.
dev work landed byte-identical diff). See ``_unchanged_pr_guard`` for
the shared logic (including the findings-resolved one-shot exemption).
"""
pr_review = (getattr(t, "notes_structured", None) or {}).get("pr_review") or {}
if pr_review.get("verdict") != "failed":
return None
recorded = pr_review.get("head_sha")
if not recorded:
return None
current = await self._current_pr_head_sha(t)
if current is None or current != recorded:
return None
return Envelope.invalid_state(
message=(
"the assembled cell PR is unchanged since the last pr_fail"
f" (head {current[:7]}). No new dev work has landed on the"
" cell branch, so re-submitting would re-open the exact diff"
" the reviewer just rejected and loop straight back to"
" awaiting_pr_review."
),
remediate=(
"re-delegate the fixes to the owning developer(s) via"
" delegate(...) and wait for the dev subtasks to complete and"
" the cell branch to be re-assembled. Do NOT call submit_up"
" again until new dev work has advanced the cell branch HEAD."
),
context_briefing=briefing,
return await self._unchanged_pr_guard(
t,
briefing,
verb="submit_up",
pr_scope="cell",
delegate_target="owning developer(s)",
subtask_noun="dev subtasks",
work_noun="dev work",
)
async def submit_root(
@@ -124,13 +124,42 @@ def findings_count_hint(findings: Sequence[Any]) -> str | None:
)
def unknown_finding_criteria(task: Any, findings: list[Finding]) -> list[str]:
"""Findings whose ``criterion`` matches neither an AC id nor AC text.
def unmatched_criteria(task: Any, criteria: list[str]) -> list[str]:
"""``criteria`` entries matching neither an AC id nor AC text on ``task``.
Mirrors ``TaskService.unknown_ac_refs`` — a criterion may be supplied by
The shared "by id or exact text" match — a criterion may be supplied by
its stable id (``acceptance_criteria_ids``, migration 036) or its exact
text, since both representations already circulate (``declare_coverage``
accepts either). Findings with no ``criterion`` are unconstrained.
accepts either). Reused by ``unknown_finding_criteria`` (a finding's
``criterion``) and QA's ``criteria_verified`` per-AC gate (qa.py) so the
match semantics can't drift between the two call sites.
"""
valid_ids = set(getattr(task, "acceptance_criteria_ids", None) or [])
valid_texts = set(getattr(task, "acceptance_criteria", None) or [])
return [c for c in criteria if c not in valid_ids and c not in valid_texts]
def uncovered_acceptance_criteria(task: Any, criteria: list[str]) -> list[str]:
"""Task ACs (by exact text) not named by any of ``criteria`` (id or text).
The reverse of ``unmatched_criteria``: walks the task's own criteria
instead of the caller-supplied ones, so a per-AC verification gate can
name exactly which acceptance criterion still lacks a matching entry.
"""
ac_ids = list(getattr(task, "acceptance_criteria_ids", None) or [])
ac_texts = list(getattr(task, "acceptance_criteria", None) or [])
provided = set(criteria)
uncovered = []
for idx, text in enumerate(ac_texts):
ac_id = ac_ids[idx] if idx < len(ac_ids) else None
if text in provided or (ac_id is not None and ac_id in provided):
continue
uncovered.append(text)
return uncovered
def unknown_finding_criteria(task: Any, findings: list[Finding]) -> list[str]:
"""Findings whose ``criterion`` matches neither an AC id nor AC text.
Short-circuits to ``[]`` (never touching ``task.acceptance_criteria*``)
when no finding supplies a criterion — the common case — so a task
@@ -139,9 +168,7 @@ def unknown_finding_criteria(task: Any, findings: list[Finding]) -> list[str]:
criteria = [f.criterion for f in findings if f.criterion]
if not criteria:
return []
valid_ids = set(getattr(task, "acceptance_criteria_ids", None) or [])
valid_texts = set(getattr(task, "acceptance_criteria", None) or [])
return [c for c in criteria if c not in valid_ids and c not in valid_texts]
return unmatched_criteria(task, criteria)
def criterion_mismatch_rejection(task: Any, unknown: list[str]) -> Envelope:
@@ -690,17 +690,18 @@ class PRGateMixin(_Base):
A reviewer must not PASS an assembled PR whose suite can't run in the
workspace, that carries unresolved architectural-convention
violations, or whose CI is red/pending/unscheduled/unresolvable;
pr_fail stays available for all three. Returns ``(rejection, None)``
to block, or ``(None, ci_note)`` to proceed — ``ci_note`` is a
non-None evidence stamp only when the CI guard passed through a
project with no CI configured at all. The toolchain/conventions
guards are inert when their flag is off; the CI guard fails open on
an unresolvable gate-level slug/PR number (``None`` from
``_resolve_ci_status``) and also passes through — with an evidence
stamp — when ``get_pr_ci_status`` itself classifies a missing
project/git_url/token or an unreachable/nonexistent repo as
``no_ci_configured``.
violations, or whose CI is red/pending/unscheduled/unresolvable; only
red CI is a code defect pr_fail should act on — an unresolvable CI
lookup is a platform blip to retry, never a finding. Returns
``(rejection, None)`` to block, or ``(None, ci_note)`` to proceed —
``ci_note`` is a non-None evidence stamp only when the CI guard
passed through a project with no CI configured at all. The
toolchain/conventions guards are inert when their flag is off; the
CI guard fails open on an unresolvable gate-level slug/PR number
(``None`` from ``_resolve_ci_status``) and also passes through —
with an evidence stamp — when ``get_pr_ci_status`` itself classifies
a missing project/git_url/token or an unreachable/nonexistent repo
as ``no_ci_configured``.
"""
from roboco.config import settings as _settings
@@ -806,12 +807,13 @@ class PRGateMixin(_Base):
"""Refuse pr_pass unless CI on the assembled PR's head commit is green.
Failing, pending, unscheduled, or unresolvable-via-API CI states all
block with a reviewer-aware remediation pointing at ``pr_fail`` (a
reviewer has no ``i_am_blocked``) pending/unscheduled/error are
framed as retryable (wait and call pr_pass again), never as a defect
to route back to the dev. A project with no CI configured at all
passes through cleanly, returning an evidence note so the caller can
stamp the verdict with why the guard did not block.
block, but only ``failure`` remediates via ``pr_fail`` (a reviewer
has no ``i_am_blocked``); pending/unscheduled/error are framed as
retryable (wait and call pr_pass again), never as a defect to route
back to the dev — a GitHub API lookup error is a platform blip, not
a finding. A project with no CI configured at all passes through
cleanly, returning an evidence note so the caller can stamp the
verdict with why the guard did not block.
"""
status = await self._resolve_ci_status(task_id, t)
if status is None:
@@ -871,11 +873,16 @@ class PRGateMixin(_Base):
"green — do NOT pr_pass before any check has run",
)
# state == "error" (or an unrecognized value) — a genuine GitHub API
# failure resolving the signal; never treat this as green.
# lookup failure, never treat this as a property of the PR itself.
# Live incident: a reviewer converted this transient blip into an
# unwaivable blocker finding whose own fix text said "no code change
# required" — a platform hiccup is not a diff defect.
return (
"could not determine CI status for the assembled PR (GitHub API error)",
"retry pr_pass shortly once the CI status can be resolved; "
"if it persists, pr_fail(issues=[...]) to unwedge the PR",
"GitHub API error resolving CI status for the assembled PR "
"this is a transient lookup failure, not a property of the PR",
"wait a few minutes and retry pr_pass — do NOT pr_fail over a "
"CI-status lookup error; a platform blip is not a code finding, "
"findings are for defects in the diff",
)
def _record_gate_verdict(
+187 -22
View File
@@ -58,6 +58,11 @@ from roboco.services.gateway.evidence_builder import build_evidence_for_task
logger = structlog.get_logger()
# Cap on one criteria_verified entry's `evidence` — mirrors Finding.fix's cap
# (roboco.foundation.policy.content.models._FINDING_FIX_CAP): a pointer
# (file:line, screenshot ref, test name), not a transcript.
_CRITERION_EVIDENCE_CAP = 500
if TYPE_CHECKING:
from uuid import UUID
@@ -427,13 +432,13 @@ class QAMixin(_Base):
def _qa_ac_coverage_check(
cls, task: Any, ac_verdicts: list[str] | None
) -> Envelope | None:
"""Per-acceptance-criterion verification gate for pass_review.
"""Legacy count-only per-AC gate — superseded by ``criteria_verified``.
QA may not pass a task until it has recorded a verification for EVERY
acceptance criterion. A single gestalt "looks good" approval is how a
silently-unbuilt criterion slips through; requiring one verdict per
criterion forces QA to check each individually. If a criterion does not
hold, the QA fails the review instead of passing a partial.
No longer wired into ``pass_review`` (a count of arbitrary strings
never verified they actually named the right criterion — the live gap
``_validate_criteria_verified`` closes). Kept for ``ac_verdicts``'
existing callers/tests; ``ac_verdicts`` itself still folds into the
persisted notes when supplied.
"""
criteria = list(getattr(task, "acceptance_criteria", None) or [])
if not criteria:
@@ -545,36 +550,181 @@ class QAMixin(_Base):
verb=verb,
)
@staticmethod
def _parse_criterion_entry(entry: Any, idx: int) -> tuple[str, str] | Envelope:
"""Validate one ``criteria_verified`` entry into a (criterion, evidence)
pair, or an ``Envelope`` rejection on any structural problem."""
criterion = entry.get("criterion") if isinstance(entry, dict) else None
evidence = entry.get("evidence") if isinstance(entry, dict) else None
if not isinstance(criterion, str) or not criterion.strip():
return Envelope.invalid_state(
message=f"criteria_verified[{idx}] is missing a `criterion` string",
remediate=(
"each entry needs {criterion, evidence} naming one "
"acceptance criterion"
),
)
if not isinstance(evidence, str) or not evidence.strip():
return Envelope.invalid_state(
message=(
f"criteria_verified[{idx}] ({criterion!r}) is missing `evidence`"
),
remediate=(
"state concrete evidence: file:line, screenshot ref, "
"rendered-frame path, test name"
),
)
if len(evidence) > _CRITERION_EVIDENCE_CAP:
return Envelope.invalid_state(
message=(
f"criteria_verified[{idx}] evidence exceeds "
f"{_CRITERION_EVIDENCE_CAP} chars"
),
remediate="keep evidence concise — a pointer, not a transcript",
)
return criterion.strip(), evidence.strip()
@classmethod
def _parse_criteria_verified_entries(
cls, criteria_verified: list[dict[str, Any]]
) -> tuple[list[tuple[str, str]], Envelope | None]:
"""Shape + soup validation for every ``criteria_verified`` entry.
Pure parsing — AC matching/coverage is the caller's job. Split out
of ``_validate_criteria_verified`` to keep its return count under
the complexity bound.
"""
pairs: list[tuple[str, str]] = []
for idx, entry in enumerate(criteria_verified):
parsed = cls._parse_criterion_entry(entry, idx)
if isinstance(parsed, Envelope):
return [], parsed
pairs.append(parsed)
soup = cls._free_text_soup(
checks=(("criteria_verified.evidence", [e for _, e in pairs], 8),)
)
if soup is not None:
return [], soup
return pairs, None
@classmethod
def _validate_criteria_verified(
cls, t: Any, criteria_verified: list[dict[str, Any]] | None
) -> tuple[list[tuple[str, str]], Envelope | None]:
"""Mandatory per-AC verification gate for pass_review.
Returns ``(pairs, rejection)`` — ``pairs`` is ``[]`` and ``rejection``
non-None on any failure: none supplied (lists every AC verbatim),
a malformed or soupy entry, an entry naming a criterion absent from
the task (names the valid criteria), or a task AC left uncovered
(names the gap). No task ACs imposes no requirement (mirrors the
legacy ``_qa_ac_coverage_check``). A gestalt "looks good" is no
longer enough — every criterion needs its own matched, evidenced
entry, or QA must call ``fail_review`` instead of passing a partial.
"""
criteria = list(getattr(t, "acceptance_criteria", None) or [])
if not criteria:
return [], None
if not criteria_verified:
return [], Envelope.invalid_state(
message=(
"pass_review needs criteria_verified naming every "
f"acceptance criterion; none supplied. Unverified: {criteria!r}"
),
remediate=(
"re-run the review and call pass_review with "
"criteria_verified=[{criterion, evidence}, ...] — stamp "
"EACH criterion with concrete evidence (file:line, "
"screenshot ref, rendered-frame path, test name)"
),
)
pairs, bad = cls._parse_criteria_verified_entries(criteria_verified)
if bad is not None:
return [], bad
provided = [c for c, _ in pairs]
if unknown := findings_lib.unmatched_criteria(t, provided):
return [], Envelope.invalid_state(
message=(
f"criteria_verified names criteria not on this task: {unknown!r}"
),
remediate=(
"each entry's criterion must match one of the task's "
f"acceptance criteria (by id or exact text): {criteria!r}"
),
)
if uncovered := findings_lib.uncovered_acceptance_criteria(t, provided):
return [], Envelope.invalid_state(
message=(
"criteria_verified is missing these acceptance criteria: "
f"{uncovered!r}"
),
remediate=(
"stamp every criterion with concrete evidence, or call "
"fail_review with the specific gap if one does not hold"
),
)
return pairs, None
@staticmethod
def _render_criteria_verified(pairs: list[tuple[str, str]]) -> list[str]:
"""One '[AC] <criterion> — verified: <evidence>' line per entry.
Style-matched to the findings ledger's '[F-<id8>] ...' bracket-tag
rendering (``findings_lib.render_finding_line``).
"""
return [
f"[AC] {criterion} — verified: {evidence}" for criterion, evidence in pairs
]
@classmethod
def _merge_criteria_verified_into_notes(
cls, notes: str, pairs: list[tuple[str, str]]
) -> str:
"""Fold the per-AC verification lines into the persisted QA notes.
Mirrors ``_merge_ac_verdicts_into_notes`` — keeps the per-criterion
verification in the audit trail (qa_notes) so PM/CEO see exactly how
QA verified each acceptance criterion.
"""
lines = cls._render_criteria_verified(pairs)
if not lines:
return notes
return f"{notes}\n\n" + "\n".join(lines)
async def _qa_pass_final_gates(
self,
qa_agent_id: UUID,
task_id: UUID,
t: Any,
role_str: str,
ac_verdicts: list[str] | None,
) -> Envelope | None:
"""AC-coverage + toolchain-runnability gates for pass_review.
criteria_verified: list[dict[str, Any]] | None,
) -> tuple[Envelope | None, list[tuple[str, str]]]:
"""Per-AC verification + toolchain-runnability gates for pass_review.
Returns the first rejection (already emitted), else None. QA must not
PASS on a workspace that cannot run the suite — that is a source-read
Returns ``(rejection, pairs)`` — the first emitted rejection (else
None) and the validated ``criteria_verified`` (criterion, evidence)
pairs for the caller to render into notes. QA must not PASS on a
workspace that cannot run the suite — that is a source-read
"verification"; fail_review is unaffected.
"""
ac_rejection = self._qa_ac_coverage_check(t, ac_verdicts)
if ac_rejection is not None:
return await self._emit_rejection(
ac_rejection.with_introspection(task=t, role=role_str),
pairs, bad = self._validate_criteria_verified(t, criteria_verified)
if bad is not None:
rejection = await self._emit_rejection(
bad.with_introspection(task=t, role=role_str),
agent_id=qa_agent_id,
task_id=task_id,
verb="pass_review",
)
return rejection, []
if toolchain := await self._toolchain_broken_guard(qa_agent_id, t):
return await self._emit_rejection(
rejection = await self._emit_rejection(
toolchain.with_introspection(task=t, role=role_str),
agent_id=qa_agent_id,
task_id=task_id,
verb="pass_review",
)
return None
return rejection, []
return None, pairs
async def pass_review(
self,
@@ -582,6 +732,7 @@ class QAMixin(_Base):
task_id: UUID,
notes: str,
ac_verdicts: list[str] | None = None,
criteria_verified: list[dict[str, Any]] | None = None,
) -> Envelope:
"""QA passes the task; transitions awaiting_qa → awaiting_documentation.
@@ -596,6 +747,16 @@ class QAMixin(_Base):
The composed atomic ``qa_pass`` is then dispatched through
``VerbRunner.run_intent``, after which the verb body reassigns
the documenter for handoff.
``criteria_verified`` ({criterion, evidence} entries) is the
mandatory per-AC verification gate (``_validate_criteria_verified``):
every one of the task's acceptance criteria must be named by exactly
one entry — matched by AC id or exact text, the same match
``fail_review``'s findings ledger uses for its own ``criterion``
field — carrying substantive evidence, or the pass is refused. A
gestalt "looks good" is no longer enough; QA must walk each
criterion. ``ac_verdicts`` (legacy, count-only) still folds into the
persisted notes when supplied but no longer gates the pass.
"""
rejection, t = await self._verify_qa_owner(qa_agent_id, task_id, "pass_review")
if rejection is not None:
@@ -621,18 +782,22 @@ class QAMixin(_Base):
soup_checks=(("notes", notes, 8),),
):
return gate_rejection
if rej := await self._qa_pass_final_gates(
qa_agent_id, task_id, t, role_str, ac_verdicts
):
return rej
final_rejection, criteria_pairs = await self._qa_pass_final_gates(
qa_agent_id, task_id, t, role_str, criteria_verified
)
if final_rejection is not None:
return final_rejection
briefing = await self._briefing_for(qa_agent_id, task_id)
merged_notes = self._merge_criteria_verified_into_notes(
self._merge_ac_verdicts_into_notes(notes, ac_verdicts), criteria_pairs
)
spec_ctx = spec_module.Context(
actor_id=qa_agent_id,
actor_slug=getattr(agent, "slug", None) if agent is not None else None,
agent_team=str(agent.team) if agent is not None and agent.team else None,
original_developer_slug=_extract_original_developer(t),
notes=self._merge_ac_verdicts_into_notes(notes, ac_verdicts),
notes=merged_notes,
)
self._store_qa_note(t, notes, ac_verdicts, passed=True)
runner = self._verb_runner()
+34
View File
@@ -125,6 +125,40 @@ class NotificationService:
)
)
async def send_block_flip_notification(
self,
task_id: str,
flip_count: int,
to_agent: str = "ceo",
) -> None:
"""Alert an overseer that a task keeps flip-flopping block/unblock.
Raised from the choreographer's ``unblock`` once the per-task flip
counter crosses the threshold — a resolver keeps unblocking a task
that keeps re-blocking, which usually means a structural wedge (e.g.
an escalate_up/unblock cycle with no forward progress) rather than a
one-off block.
"""
logger.info(
"Sending block-flip notification", task_id=task_id, flip_count=flip_count
)
body = (
f"Task {task_id} has been blocked and unblocked {flip_count} times. "
"This flip-flop usually means a structural wedge rather than a "
"one-off block — please investigate."
)
await self._create_notification(
CreateNotificationParams(
notification_type=NotificationType.BLOCKER_ESCALATION,
priority=NotificationPriority.HIGH,
from_agent="system",
to_agents=[to_agent],
subject=f"Task {task_id} flip-flopping block/unblock ({flip_count}x)",
body=body,
related_task_id=task_id,
)
)
async def send_qa_ready_notification(
self,
task_id: str,
+70 -1
View File
@@ -54,6 +54,18 @@ _AUTHORING_ACCEPTANCE_CRITERIA = [
]
_POST_ACCEPTANCE_CRITERIA = ["CEO approves or rejects the draft"]
# Mirrors task_completeness._AC_MAX_ITEMS / _AC_MAX_ITEM_CHARS (mig 068) —
# duplicated locally per this file's no-cross-service-internals idiom (see
# vault_intake_engine.py / api/schemas/v1/flow.py for the same pattern).
_AC_MAX_ITEMS = 7
_AC_MAX_ITEM_CHARS = 200
# reauthor_from_rejection's fallback AC when the original brief named no
# enumerable features — the rejection reason is already verbatim in the brief.
_REAUTHOR_FEEDBACK_CRITERION = (
"Every point in the CEO rejection feedback is visibly addressed in the rendered cut"
)
_CHAT_TIMEOUT_SECONDS = 60.0
# The whole CHANGELOG section for a release, not one bullet — capped so a
@@ -146,6 +158,47 @@ def _release_video_brief(version: str, changelog: str, highlights: list[str]) ->
return "\n\n".join(parts)
def _scene_criterion(features: list[str]) -> str | None:
"""Turn an enumerable brief feature list into its own gate-checkable AC.
The live failure this closes: a brief named N features, the task's ACs
stayed generic, a dev shipped fewer scenes, and every gate passed because
"N features" existed only in prose. None for an empty/absent list. The
joined list is bounded to the AC per-item char cap truncated with an
"… (+N more)" tail rather than overrunning it.
"""
trimmed = [f.strip() for f in features if f and f.strip()]
if not trimmed:
return None
prefix = "Every brief-named feature appears as its own fully readable scene: "
budget = _AC_MAX_ITEM_CHARS - len(prefix)
kept = list(trimmed)
while kept:
omitted = len(trimmed) - len(kept)
tail = f"… (+{omitted} more)" if omitted else ""
body = "; ".join(kept) + tail
if len(body) <= budget:
return prefix + body
kept.pop()
return prefix + "" # pathological: even one (huge) feature name overruns
def _authoring_criteria(
suggested_input_props: dict[str, Any] | None,
fallback_acceptance_criterion: str | None,
) -> list[str]:
"""The base authoring ACs plus one derived/fallback criterion: the scene
criterion when ``suggested_input_props["highlights"]`` is a real list,
else ``fallback_acceptance_criterion`` (when given), else nothing."""
criteria = list(_AUTHORING_ACCEPTANCE_CRITERIA)
features = (suggested_input_props or {}).get("highlights")
criterion = _scene_criterion(features if isinstance(features, list) else [])
criterion = criterion or fallback_acceptance_criterion
if criterion is not None and len(criteria) < _AC_MAX_ITEMS:
criteria.append(criterion)
return criteria
def _reauthor_brief(reason: str, draft: dict[str, Any]) -> str:
"""The revision brief for a CEO-rejected cut: the verbatim rejection
feedback, a revise-in-place pointer at the existing composition (when
@@ -270,6 +323,7 @@ class VideoEngine(BaseService):
brief: str,
suggested_input_props: dict[str, Any] | None = None,
project_id: UUID | None = None,
fallback_acceptance_criterion: str | None = None,
) -> TaskTable | None:
"""Originate ONE UX/UI authoring task for a bespoke video, or None.
@@ -292,6 +346,14 @@ class VideoEngine(BaseService):
"highlights": [...]}`` from the release caller) is seeded onto the
marker as-is so the dev copies real structured data into
``propose_video``'s ``input_props`` instead of hand-typing facts.
When ``suggested_input_props`` carries a ``highlights`` list, it
becomes its OWN acceptance criterion (``_scene_criterion``) a
prose-only feature list used to pass gates even when a dev shipped
fewer scenes than named. Absent highlights,
``fallback_acceptance_criterion`` (when supplied) is appended instead
``reauthor_from_rejection`` uses this for its
feedback-must-be-addressed criterion.
"""
if not settings.video_engine_enabled:
return None
@@ -316,6 +378,9 @@ class VideoEngine(BaseService):
assignee = self._select_ux_dev(open_tasks)
enriched_brief = await self._enrich_brief(brief)
acceptance_criteria = _authoring_criteria(
suggested_input_props, fallback_acceptance_criterion
)
# Savepoint-isolate the insert: a DBAPI error here (FK, deadlock,
# dropped connection) must roll back ONLY this insert, never poison the
# shared session — whose next commit is the caller's release-publish
@@ -326,7 +391,7 @@ class VideoEngine(BaseService):
TaskCreateRequest(
title=f"Video: {occasion}",
description=enriched_brief,
acceptance_criteria=list(_AUTHORING_ACCEPTANCE_CRITERIA),
acceptance_criteria=acceptance_criteria,
team=Team.UX_UI,
assigned_to=assignee,
created_by=_foundation.AGENTS["system"].uuid,
@@ -533,6 +598,10 @@ class VideoEngine(BaseService):
draft.get("input_props") or draft.get("suggested_input_props")
),
project_id=project_id,
# Original had highlights -> the scene criterion regenerates
# from them; no highlights -> this fallback names the reason
# (already verbatim in the brief) as the checkable outcome.
fallback_acceptance_criterion=_REAUTHOR_FEEDBACK_CRITERION,
)
except Exception as exc:
self.log.warning(
+7
View File
@@ -516,6 +516,13 @@ def qa_arc(stack: E2EStack, company: Company, task_id: Any) -> None:
ac_verdicts=[
f"{c} — verified against the PR diff on the origin." for c in criteria
],
criteria_verified=[
{
"criterion": c,
"evidence": "verified against the PR diff on the origin",
}
for c in criteria
],
),
"qa pass_review",
)
@@ -517,6 +517,10 @@ async def test_full_chain_through_doc_handoff(
task.id,
notes=qa_notes,
ac_verdicts=[f"verified: {crit}" for crit in task.acceptance_criteria],
criteria_verified=[
{"criterion": crit, "evidence": f"verified against the PR diff: {crit}"}
for crit in task.acceptance_criteria
],
)
assert env.error is None, f"pass_review failed: {env.message}"
assert env.status == "awaiting_documentation"
@@ -513,6 +513,10 @@ async def test_qa_pass_path(
task.id,
notes=_QA_PASS_NOTES,
ac_verdicts=[f"verified: {crit}" for crit in task.acceptance_criteria],
criteria_verified=[
{"criterion": crit, "evidence": f"verified against the PR diff: {crit}"}
for crit in task.acceptance_criteria
],
)
assert env.error is None, f"pass_review failed: {env.message}"
assert env.status == Status.AWAITING_DOCUMENTATION.value
@@ -105,6 +105,7 @@ class _MockChoreographer:
_task_id: object,
_notes: object,
_ac_verdicts: object = None,
_criteria_verified: object = None,
) -> Envelope:
self._state["task_status"] = "awaiting_documentation"
return Envelope.ok(
+26 -3
View File
@@ -8,6 +8,7 @@ from uuid import uuid4
import pytest
from pydantic import ValidationError
from roboco.api.schemas.v1.flow import (
_APPROACH_MAX_CHARS,
DelegateRequest,
IWillPlanRequest,
IWillWorkOnRequest,
@@ -165,13 +166,35 @@ def test_i_will_plan_request_rejects_overlong_plan() -> None:
assert "plan" in str(exc.value)
def test_i_will_plan_request_rejects_overlong_approach() -> None:
"""approach >800 chars is rejected — no ceiling was the bloat bug."""
def test_i_will_plan_request_truncates_overlong_approach() -> None:
"""approach >800 chars is truncated to 800 (797 + "..."), never rejected —
a hard 422 here used to throw away an otherwise-good plan and degrade the
PM to a bare unblock (live incident)."""
req = IWillPlanRequest(
task_id=uuid4(),
plan="plan",
approach="a" * (_APPROACH_MAX_CHARS + 1),
)
assert len(req.approach) == _APPROACH_MAX_CHARS
assert req.approach.endswith("...")
assert req.approach == "a" * (_APPROACH_MAX_CHARS - 3) + "..."
def test_i_will_plan_request_approach_exactly_800_untruncated() -> None:
"""Exactly the ceiling is the boundary — passes through byte-for-byte."""
approach = "a" * _APPROACH_MAX_CHARS
req = IWillPlanRequest(task_id=uuid4(), plan="plan", approach=approach)
assert req.approach == approach
assert not req.approach.endswith("...")
def test_i_will_plan_request_rejects_thin_approach() -> None:
"""approach <150 chars is still a hard reject — a thin plan IS a defect."""
with pytest.raises(ValidationError) as exc:
IWillPlanRequest(
task_id=uuid4(),
plan="plan",
approach="a" * 801,
approach="a" * 149,
)
assert "approach" in str(exc.value)
@@ -6,6 +6,9 @@ from types import SimpleNamespace
from roboco.foundation.policy.content import markers as m
# Named constant — ruff PLR2004 forbids magic-value comparisons.
_TWO = 2
def _task(om: dict | None = None) -> SimpleNamespace:
return SimpleNamespace(orchestration_markers=om)
@@ -130,3 +133,25 @@ def test_docs_sync_release_version_roundtrip() -> None:
assert m.get_docs_sync_release_version(t) is None
m.set_docs_sync_release_version(t, "0.23.0")
assert m.get_docs_sync_release_version(t) == "0.23.0"
def test_resubmit_unchanged_head_roundtrip() -> None:
t = _task()
assert m.get_resubmit_unchanged_head(t) is None
m.set_resubmit_unchanged_head(t, "aaaa1111bbbb2222")
assert m.get_resubmit_unchanged_head(t) == "aaaa1111bbbb2222"
def test_block_flip_count_bump_and_notify() -> None:
t = _task()
assert m.get_block_flip_count(t) == 0
assert m.is_block_flip_notified(t) is False
assert m.bump_block_flip_count(t) == 1
assert m.bump_block_flip_count(t) == _TWO
assert m.get_block_flip_count(t) == _TWO
# notified stays False across bumps until explicitly marked.
assert m.is_block_flip_notified(t) is False
m.mark_block_flip_notified(t)
assert m.is_block_flip_notified(t) is True
# Marking notified must not reset the counter.
assert m.get_block_flip_count(t) == _TWO
@@ -137,6 +137,7 @@ async def test_delegate_allows_when_parent_in_progress_and_owned() -> None:
assigned_to=pm_id,
team="backend",
quick_context="Decomposition planned; cells implement their slice next.",
acceptance_criteria=[],
)
new_task = MagicMock(id=uuid4())
task_svc = AsyncMock()
@@ -194,6 +195,7 @@ async def test_delegate_allows_when_subtask_cap_within_soft_zone() -> None:
assigned_to=pm_id,
team="backend",
quick_context="Decomposition planned; cells implement their slice next.",
acceptance_criteria=[],
)
many = [MagicMock(id=uuid4()) for _ in range(10)]
new_task = MagicMock(id=uuid4())
@@ -223,6 +225,7 @@ async def test_delegate_allows_at_zero_subtasks() -> None:
assigned_to=pm_id,
team="backend",
quick_context="Decomposition planned; cells implement their slice next.",
acceptance_criteria=[],
)
new_task = MagicMock(id=uuid4())
task_svc = AsyncMock()
@@ -313,6 +316,7 @@ async def test_delegate_past_max_depth_returns_invalid_state_not_500() -> None:
assigned_to=pm_id,
team="backend",
quick_context="Decomposition planned; cells implement their slice next.",
acceptance_criteria=[],
)
depth_msg = (
"Task hierarchy would exceed MAX_TASK_DEPTH=4. Create this work as a "
@@ -675,6 +675,8 @@ async def test_delegate_main_pm_to_cell_pm_creates_subtask() -> None:
status="in_progress",
assigned_to=main_pm_id,
quick_context="Decomposition planned; cells implement their slice next.",
# This test is about role/chain wiring, not AC coverage.
acceptance_criteria=[],
)
new_task = MagicMock(id=uuid4())
task_svc = AsyncMock()
@@ -719,6 +721,8 @@ async def test_delegate_cell_pm_to_team_dev_creates_subtask() -> None:
assigned_to=cell_pm_id,
team="backend",
quick_context="Decomposition planned; cells implement their slice next.",
# This test is about role/chain wiring, not AC coverage.
acceptance_criteria=[],
)
new_task = MagicMock(id=uuid4())
task_svc = AsyncMock()
@@ -1185,6 +1189,9 @@ async def test_delegate_main_pm_to_cell_pm_accepts_planning_subtask() -> None:
status="in_progress",
assigned_to=main_pm_id,
quick_context="Decomposition planned; cells implement their slice next.",
# This test is about the code-vs-planning task_type guard, not AC
# coverage.
acceptance_criteria=[],
)
new_task = MagicMock(id=uuid4())
task_svc = AsyncMock()
@@ -320,6 +320,94 @@ async def test_pass_review_succeeds_and_transitions() -> None:
a2a_svc.send.assert_awaited_once()
@pytest.mark.asyncio
async def test_pass_review_rejects_without_criteria_verified_when_acs_present() -> None:
"""A task with real acceptance criteria demands criteria_verified — a
gestalt "looks good" notes string alone is no longer enough."""
qa_id = uuid4()
task_id = uuid4()
t = _qa_owned_task(
task_id, qa_id, acceptance_criteria=["returns 200", "includes timestamp"]
)
task_svc = AsyncMock()
task_svc.get.return_value = t
task_svc.agent_for.return_value = _qa_agent_mock(qa_id)
journal_svc = AsyncMock()
journal_svc.has_learning_for_task.return_value = True
deps = _make_deps(task=task_svc, journal=journal_svc)
c = Choreographer(deps)
notes = "x" * 100
env = await c.pass_review(qa_id, task_id, notes=notes)
body = env.as_dict()
assert body["error"] == "invalid_state", body
assert "returns 200" in body["message"]
assert "includes timestamp" in body["message"]
@pytest.mark.asyncio
async def test_pass_review_renders_criteria_verified_into_notes() -> None:
"""Happy path: every AC matched + evidenced renders '[AC] ...' lines into
the persisted qa_notes and the transition still fires."""
qa_id = uuid4()
task_id = uuid4()
t = _qa_owned_task(
task_id, qa_id, acceptance_criteria=["returns 200", "includes timestamp"]
)
after = MagicMock(
id=task_id,
status="awaiting_documentation",
assigned_to=qa_id,
team="backend",
pr_url="https://x/pr/8",
qa_evidence_inspected=True,
)
task_svc = AsyncMock()
task_svc.get.return_value = t
task_svc.agent_for.return_value = _qa_agent_mock(qa_id)
task_svc.qa_pass.return_value = after
task_svc.documenter_for_team.return_value = MagicMock(id=uuid4())
task_svc.session = MagicMock()
task_svc.session.begin_nested = MagicMock(
return_value=MagicMock(
__aenter__=AsyncMock(return_value=None),
__aexit__=AsyncMock(return_value=False),
)
)
_stub_empty_ledger(task_svc.session)
journal_svc = AsyncMock()
journal_svc.has_learning_for_task.return_value = True
a2a_svc = AsyncMock()
deps = _make_deps(task=task_svc, journal=journal_svc, a2a=a2a_svc)
c = Choreographer(deps)
notes = (
"Reviewed PR carefully. Rendered every scene and checked each frame "
"against the brief before approving."
)
env = await c.pass_review(
qa_id,
task_id,
notes=notes,
criteria_verified=[
{"criterion": "returns 200", "evidence": "test_healthz asserts 200"},
{
"criterion": "includes timestamp",
"evidence": "frame diff shows ts field at README.md line 12",
},
],
)
assert env.error is None, env.as_dict()
assert env.status == "awaiting_documentation"
task_svc.qa_pass.assert_awaited_once()
persisted_notes = task_svc.qa_pass.call_args.args[2]
assert "[AC] returns 200 — verified: test_healthz asserts 200" in persisted_notes
assert (
"[AC] includes timestamp — verified: frame diff shows ts field at "
"README.md line 12" in persisted_notes
)
@pytest.mark.asyncio
async def test_pass_review_not_assigned_returns_not_authorized() -> None:
qa_id = uuid4()
@@ -0,0 +1,213 @@
"""Decomposition-coverage gate: delegate rejects a child that doesn't map to
the parent's acceptance criteria, and a successful delegate reports the
parent's remaining coverage gaps in evidence.
Live failure this closes (see CLAUDE.md "delegate" section): unmapped
children were only ever caught late, at submit_up's roll-up gate — after a
whole wave of subtasks had already run. Moving the mapping check to delegate
time surfaces "child covers nothing" / "ref doesn't match a real criterion"
before any subtask is created.
"""
from __future__ import annotations
from datetime import UTC, datetime
from typing import Any
from unittest.mock import AsyncMock, MagicMock
from uuid import uuid4
import pytest
from roboco.services.gateway.choreographer import (
Choreographer,
ChoreographerDeps,
DelegateInputs,
)
def _make_deps(**overrides: Any) -> ChoreographerDeps:
base: dict[str, Any] = {
"task": AsyncMock(),
"work_session": AsyncMock(),
"git": AsyncMock(),
"a2a": AsyncMock(),
"journal": AsyncMock(),
"audit": AsyncMock(),
"evidence_repo": AsyncMock(),
}
base.update(overrides)
repo = base["evidence_repo"]
for method in (
"list_unread_a2a",
"list_unread_mentions",
"list_pending_notifications",
"task_metadata_gaps",
"recent_team_activity",
"blockers_in_lane",
"journal_highlights_for_task",
):
getattr(repo, method).return_value = []
_ldef = base["journal"].latest_decision_at.return_value
if type(_ldef).__name__ in ("MagicMock", "AsyncMock"):
base["journal"].latest_decision_at.return_value = datetime.now(UTC)
return ChoreographerDeps(**base)
def _parent_with_criteria(pm_id: Any) -> MagicMock:
return MagicMock(
id=uuid4(),
project_id=uuid4(),
status="in_progress",
assigned_to=pm_id,
team="backend",
quick_context="Decomposition planned; cells implement their slice next.",
acceptance_criteria=["Criterion A", "Criterion B"],
acceptance_criteria_ids=["id-a", "id-b"],
)
def _inputs(**kw: Any) -> DelegateInputs:
base: dict[str, Any] = {
"title": "Implement endpoint",
"description": "Add /v1/foo endpoint with tests",
"assigned_to": "be-dev-1",
"team": "backend",
"task_type": "code",
"nature": "technical",
"acceptance_criteria": ["GET /v1/foo returns 200 with body"],
"intends_to_touch": ["backend/api/routers/foo.py"],
}
base.update(kw)
return DelegateInputs(**base)
@pytest.mark.asyncio
async def test_delegate_rejects_child_with_no_mapping() -> None:
"""A parent with real ACs rejects a child that maps to none of them."""
pm_id = uuid4()
parent = _parent_with_criteria(pm_id)
task_svc = AsyncMock()
task_svc.get.return_value = parent
task_svc.agent_for.return_value = MagicMock(role="cell_pm", team="backend")
task_svc.get_subtasks.return_value = []
deps = _make_deps(task=task_svc)
c = Choreographer(deps)
env = await c.delegate(pm_id, parent.id, _inputs(title="Orphan slice"))
body = env.as_dict()
assert body["error"] == "invalid_state", body
assert "Orphan slice" in body["message"]
assert "no covers_parent_criteria" in body["message"]
assert "Criterion A" in body["remediate"] and "Criterion B" in body["remediate"]
task_svc.create_subtask.assert_not_awaited()
task_svc.unknown_ac_refs.assert_not_called()
@pytest.mark.asyncio
async def test_delegate_rejects_unresolvable_ref_lists_valid_criteria() -> None:
"""A ref matching neither a criterion id nor its exact text is rejected,
with the parent's real criteria named so the PM can pick a valid one."""
pm_id = uuid4()
parent = _parent_with_criteria(pm_id)
task_svc = AsyncMock()
task_svc.get.return_value = parent
task_svc.agent_for.return_value = MagicMock(role="cell_pm", team="backend")
task_svc.get_subtasks.return_value = []
task_svc.unknown_ac_refs = MagicMock(return_value=["bogus-ref"])
deps = _make_deps(task=task_svc)
c = Choreographer(deps)
env = await c.delegate(
pm_id,
parent.id,
_inputs(title="Endpoint slice", covers_parent_criteria=["bogus-ref"]),
)
body = env.as_dict()
assert body["error"] == "invalid_state", body
assert "Endpoint slice" in body["message"]
assert "bogus-ref" in body["message"]
assert "Criterion A" in body["remediate"] and "Criterion B" in body["remediate"]
task_svc.create_subtask.assert_not_awaited()
task_svc.unknown_ac_refs.assert_called_once_with(parent, ["bogus-ref"])
@pytest.mark.asyncio
async def test_delegate_rejects_multiple_unresolvable_refs_in_one_envelope() -> None:
"""Every unresolvable ref is named in the one rejection, not just the first."""
pm_id = uuid4()
parent = _parent_with_criteria(pm_id)
task_svc = AsyncMock()
task_svc.get.return_value = parent
task_svc.agent_for.return_value = MagicMock(role="cell_pm", team="backend")
task_svc.get_subtasks.return_value = []
task_svc.unknown_ac_refs = MagicMock(return_value=["bogus-one", "bogus-two"])
deps = _make_deps(task=task_svc)
c = Choreographer(deps)
env = await c.delegate(
pm_id,
parent.id,
_inputs(covers_parent_criteria=["bogus-one", "bogus-two"]),
)
body = env.as_dict()
assert body["error"] == "invalid_state", body
assert "bogus-one" in body["message"]
assert "bogus-two" in body["message"]
task_svc.create_subtask.assert_not_awaited()
@pytest.mark.asyncio
async def test_delegate_success_evidence_carries_covered_and_uncovered() -> None:
"""A resolvable mapping creates the subtask and reports the parent's
coverage split in evidence, using the same primitive submit_up checks."""
pm_id = uuid4()
parent = _parent_with_criteria(pm_id)
new_task = MagicMock(id=uuid4())
task_svc = AsyncMock()
task_svc.get.return_value = parent
task_svc.agent_for.return_value = MagicMock(role="cell_pm", team="backend")
task_svc.get_subtasks.return_value = []
task_svc.unknown_ac_refs = MagicMock(return_value=[])
task_svc.create_subtask.return_value = new_task
task_svc.uncovered_parent_acceptance_criteria.return_value = ["Criterion B"]
deps = _make_deps(task=task_svc)
c = Choreographer(deps)
env = await c.delegate(pm_id, parent.id, _inputs(covers_parent_criteria=["id-a"]))
body = env.as_dict()
assert body["error"] is None, body
assert body["status"] == "created"
assert body["evidence"]["parent_ac_coverage"] == {
"covered": ["Criterion A"],
"uncovered": ["Criterion B"],
}
@pytest.mark.asyncio
async def test_delegate_wave_leaving_acs_uncovered_still_succeeds() -> None:
"""No full-coverage hard gate at delegate: a wave may leave criteria for a
later delegate call the child is still created, gaps just get listed."""
pm_id = uuid4()
parent = _parent_with_criteria(pm_id)
parent.acceptance_criteria = ["Criterion A", "Criterion B", "Criterion C"]
parent.acceptance_criteria_ids = ["id-a", "id-b", "id-c"]
new_task = MagicMock(id=uuid4())
task_svc = AsyncMock()
task_svc.get.return_value = parent
task_svc.agent_for.return_value = MagicMock(role="cell_pm", team="backend")
task_svc.get_subtasks.return_value = []
task_svc.unknown_ac_refs = MagicMock(return_value=[])
task_svc.create_subtask.return_value = new_task
task_svc.uncovered_parent_acceptance_criteria.return_value = [
"Criterion B",
"Criterion C",
]
deps = _make_deps(task=task_svc)
c = Choreographer(deps)
env = await c.delegate(pm_id, parent.id, _inputs(covers_parent_criteria=["id-a"]))
body = env.as_dict()
assert body["error"] is None, body
coverage = body["evidence"]["parent_ac_coverage"]
assert coverage["covered"] == ["Criterion A"]
assert coverage["uncovered"] == ["Criterion B", "Criterion C"]
task_svc.create_subtask.assert_awaited_once()
@@ -65,6 +65,8 @@ def _parent_in_progress(pm_id: Any) -> MagicMock:
priority=2,
# delegate obligates the PM's quick_context resumption section.
quick_context="Decomposition planned; cells implement their slice next.",
# These tests are about task_completeness, not AC coverage.
acceptance_criteria=[],
)
@@ -63,6 +63,8 @@ def _parent(pm_id: object) -> MagicMock:
team="backend",
# delegate obligates the PM's quick_context resumption section.
quick_context="Decomposition planned; cells implement their slice next.",
# These tests are about the parent-lock ordering, not AC coverage.
acceptance_criteria=[],
)
@@ -51,6 +51,8 @@ def _parent(pm_id: Any, product_id: Any = None, project_id: Any = None) -> Magic
team="backend",
# delegate obligates the PM's quick_context resumption section.
quick_context="Decomposition planned; cells implement their slice next.",
# These tests are about project routing, not AC coverage.
acceptance_criteria=[],
)
@@ -4,12 +4,14 @@ Before this guard, ``pr_pass`` had no CI-status check at all — a reviewer coul
pass an assembled PR whose CI was red, still running, or not yet scheduled.
``_ci_status_guard`` (wired into ``_pr_pass_blocked`` alongside the existing
toolchain/conventions guards) reads ``GitService.get_pr_ci_status`` and blocks
on failure/pending/pending_not_scheduled/error with reviewer-aware remediation
(``pr_fail``, never ``i_am_blocked`` a reviewer has no such verb). A project
with no CI configured at all passes through cleanly, stamping the verdict note
with why the guard did not block. ``pr_fail`` is unaffected by CI state
entirely, and the separate inbound ``PRReviewerMixin`` surface
(``claim_pr_review`` / ``post_pr_review``) never consults CI status at all.
on failure/pending/pending_not_scheduled/error; only ``failure`` remediates
via ``pr_fail`` (never ``i_am_blocked`` a reviewer has no such verb) the
``error`` state is a transient GitHub API lookup failure and remediates via
retry only, never ``pr_fail``. A project with no CI configured at all passes
through cleanly, stamping the verdict note with why the guard did not block.
``pr_fail`` is unaffected by CI state entirely, and the separate inbound
``PRReviewerMixin`` surface (``claim_pr_review`` / ``post_pr_review``) never
consults CI status at all.
"""
from __future__ import annotations
@@ -162,6 +164,7 @@ async def test_pr_pass_blocked_on_github_api_error() -> None:
assert env.error == "invalid_state"
assert "GitHub API error" in (env.message or "")
assert "retry" in (env.remediate or "").lower()
assert "do NOT pr_fail" in (env.remediate or "")
@pytest.mark.asyncio
@@ -0,0 +1,175 @@
"""pass_review requires a matched, evidenced verification per acceptance
criterion not just a count of arbitrary strings.
Live failure this closes: QA passed a rendered video shipping 3 of the
brief's 4 named scenes because nothing forced the reviewer to walk each
acceptance criterion individually. Mirrors the test idiom in
test_qa_ac_coverage.py, one level stricter: criteria_verified entries must
each match a real AC (by id or exact text) and carry substantive evidence.
"""
from __future__ import annotations
from types import SimpleNamespace
from roboco.services.gateway.choreographer import Choreographer
_EVIDENCE_CAP = 500
def _task(criteria: list[str], ids: list[str] | None = None) -> SimpleNamespace:
return SimpleNamespace(
acceptance_criteria=criteria,
acceptance_criteria_ids=ids or [],
)
def test_no_criteria_imposes_no_requirement() -> None:
pairs, rej = Choreographer._validate_criteria_verified(_task([]), None)
assert pairs == []
assert rej is None
def test_none_supplied_lists_every_criterion_verbatim() -> None:
criteria = [
"scene 1 renders",
"scene 2 renders",
"scene 3 renders",
"scene 4 renders",
]
t = _task(criteria)
pairs, rej = Choreographer._validate_criteria_verified(t, None)
assert pairs == []
assert rej is not None
body = rej.as_dict()
assert body["error"] == "invalid_state", body
for crit in criteria:
assert crit in body["message"]
def test_empty_list_is_treated_as_none_supplied() -> None:
t = _task(["a"])
pairs, rej = Choreographer._validate_criteria_verified(t, [])
assert pairs == []
assert rej is not None
def test_partial_coverage_names_the_missing_criterion() -> None:
t = _task(["a", "b", "c"])
pairs, rej = Choreographer._validate_criteria_verified(
t,
[
{"criterion": "a", "evidence": "frame 1 shows a rendered"},
{"criterion": "b", "evidence": "frame 2 shows b rendered"},
],
)
assert pairs == []
assert rej is not None
assert "c" in rej.as_dict()["message"]
def test_unmatched_criterion_is_rejected_naming_valid_ones() -> None:
t = _task(["a", "b"])
pairs, rej = Choreographer._validate_criteria_verified(
t,
[
{"criterion": "a", "evidence": "frame 1 shows a rendered"},
{"criterion": "not-a-real-ac", "evidence": "frame 2 shows something"},
],
)
assert pairs == []
assert rej is not None
body = rej.as_dict()
assert "not-a-real-ac" in body["message"]
assert "a" in body["remediate"] and "b" in body["remediate"]
def test_missing_criterion_key_is_rejected() -> None:
t = _task(["a"])
pairs, rej = Choreographer._validate_criteria_verified(
t, [{"evidence": "frame 1 shows a rendered"}]
)
assert pairs == []
assert rej is not None
def test_blank_evidence_is_rejected() -> None:
t = _task(["a"])
pairs, rej = Choreographer._validate_criteria_verified(
t, [{"criterion": "a", "evidence": " "}]
)
assert pairs == []
assert rej is not None
def test_soup_evidence_is_rejected() -> None:
t = _task(["a"])
pairs, rej = Choreographer._validate_criteria_verified(
t, [{"criterion": "a", "evidence": "wip"}]
)
assert pairs == []
assert rej is not None
def test_overlong_evidence_is_rejected() -> None:
t = _task(["a"])
pairs, rej = Choreographer._validate_criteria_verified(
t, [{"criterion": "a", "evidence": "x" * (_EVIDENCE_CAP + 100)}]
)
assert pairs == []
assert rej is not None
assert str(_EVIDENCE_CAP) in rej.as_dict()["message"]
def test_full_coverage_by_exact_text_passes() -> None:
t = _task(["a", "b"])
pairs, rej = Choreographer._validate_criteria_verified(
t,
[
{"criterion": "a", "evidence": "frame 1 shows a rendered fully"},
{"criterion": "b", "evidence": "frame 2 shows b rendered fully"},
],
)
assert rej is None
assert pairs == [
("a", "frame 1 shows a rendered fully"),
("b", "frame 2 shows b rendered fully"),
]
def test_full_coverage_by_ac_id_passes() -> None:
t = _task(["scene renders"], ids=["AC-1"])
pairs, rej = Choreographer._validate_criteria_verified(
t, [{"criterion": "AC-1", "evidence": "rendered-frame path: out/frame3.png"}]
)
assert rej is None
assert pairs == [("AC-1", "rendered-frame path: out/frame3.png")]
def test_extra_entries_beyond_the_ac_set_are_allowed() -> None:
t = _task(["a"])
pairs, rej = Choreographer._validate_criteria_verified(
t,
[{"criterion": "a", "evidence": "frame 1 shows a rendered fully"}],
)
assert rej is None
assert len(pairs) == 1
def test_render_criteria_verified_matches_style() -> None:
lines = Choreographer._render_criteria_verified(
[("scene 1 renders", "frame 12 shows scene 1 fully")]
)
assert lines == ["[AC] scene 1 renders — verified: frame 12 shows scene 1 fully"]
def test_merge_criteria_verified_into_notes() -> None:
merged = Choreographer._merge_criteria_verified_into_notes(
"base review", [("a", "evidence a"), ("b", "evidence b")]
)
assert "[AC] a — verified: evidence a" in merged
assert "[AC] b — verified: evidence b" in merged
def test_merge_with_no_pairs_returns_notes_unchanged() -> None:
assert Choreographer._merge_criteria_verified_into_notes("base", []) == "base"
@@ -0,0 +1,233 @@
"""The one-shot resubmit exemption on the unchanged-PR loop-stopper.
Live wedge: a cell task looped block/unblock for hours because the freshness
guard (``_unchanged_pr_guard``, shared by ``submit_root``/``submit_up``)
demands a new commit after ANY ``pr_fail`` a structural deadlock when the
rejection round's findings require no code change (e.g. a transient
CI-lookup error ledgered as a finding, then waived/addressed). Once every
ledger finding is resolved (``_open_finding_ids`` empty), ONE resubmission is
exempted per head sha (``markers.resubmit_unchanged_head``); a second attempt
at the SAME head still refuses. Covers both call sites: ``submit_root``
(root) and ``submit_up`` (cell).
Note on test style: the upstream ``FINDINGS_ADDRESSED`` tracing gate
(``_check_submit_up_gates``, shared by both verbs) already refuses the whole
verb with ``tracing_gap`` whenever findings are open by the time
``_unchanged_pr_guard`` runs, open findings are already impossible through the
public ``submit_root``/``submit_up`` entrypoints. The "findings still open"
and "ambiguous case short-circuits before the findings check" scenarios are
therefore exercised by calling the guard method directly (defense-in-depth on
the guard's own logic); the exemption-grant and one-shot-per-head scenarios
drive the full verb end-to-end to prove the real wiring.
"""
from __future__ import annotations
from datetime import UTC, datetime
from typing import Any
from unittest.mock import AsyncMock, MagicMock
from uuid import uuid4
import pytest
from roboco.foundation.policy.content import markers
from roboco.services.gateway.choreographer import Choreographer, ChoreographerDeps
SHA_OLD = "aaaa1111bbbb2222cccc3333dddd4444eeee5555"
SHA_NEW = "9999888877776666555544443333222211110000"
def _make_deps(**overrides: Any) -> ChoreographerDeps:
base: dict[str, Any] = {
"task": AsyncMock(),
"work_session": AsyncMock(),
"git": AsyncMock(),
"a2a": AsyncMock(),
"journal": AsyncMock(),
"audit": AsyncMock(),
"evidence_repo": AsyncMock(),
}
base.update(overrides)
base["journal"].has_decision_for_task.return_value = True
base["journal"].latest_decision_at.return_value = datetime.now(UTC)
base["journal"].has_reflect_for_task.return_value = True
return ChoreographerDeps(**base)
def _unchanged_notes(head_sha: str = SHA_OLD) -> dict[str, Any]:
return {"pr_review": {"verdict": "failed", "head_sha": head_sha, "summary": "..."}}
async def _call_guard(
c: Choreographer, kind: str, t: Any, briefing: dict[str, Any]
) -> Any:
if kind == "root":
return await c._submit_root_unchanged_pr_guard(t, briefing)
return await c._submit_up_unchanged_pr_guard(t, briefing)
# ---------------------------------------------------------------------------
# Direct guard tests — isolate ``_unchanged_pr_guard`` from the upstream
# FINDINGS_ADDRESSED gate, which already forbids open findings from ever
# reaching here through the real submit_root/submit_up flow.
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
@pytest.mark.parametrize("kind", ["root", "cell"])
async def test_refused_when_findings_still_open(kind: str) -> None:
"""Unchanged head + open findings -> refused (behavior unchanged)."""
c = Choreographer(_make_deps())
cc: Any = c
cc._current_pr_head_sha = AsyncMock(return_value=SHA_OLD)
cc._open_finding_ids = AsyncMock(return_value=("abcd1234",))
t = MagicMock(
id=uuid4(), notes_structured=_unchanged_notes(), orchestration_markers=None
)
env = await _call_guard(c, kind, t, {})
assert env is not None
assert env.error == "invalid_state", env.as_dict()
assert "unchanged" in (env.message or "").lower()
assert markers.get_resubmit_unchanged_head(t) is None
@pytest.mark.asyncio
@pytest.mark.parametrize("kind", ["root", "cell"])
async def test_allowed_when_head_advanced_before_findings_are_even_checked(
kind: str,
) -> None:
"""A different current head SHA -> allowed; the ambiguity check short-
circuits before the findings/exemption logic ever runs."""
c = Choreographer(_make_deps())
cc: Any = c
cc._current_pr_head_sha = AsyncMock(return_value=SHA_NEW)
open_findings_spy = AsyncMock(return_value=("abcd1234",))
cc._open_finding_ids = open_findings_spy
t = MagicMock(
id=uuid4(), notes_structured=_unchanged_notes(), orchestration_markers=None
)
env = await _call_guard(c, kind, t, {})
assert env is None
open_findings_spy.assert_not_awaited()
# ---------------------------------------------------------------------------
# End-to-end tests — drive the real submit_root/submit_up verb to prove the
# exemption's marker stamp actually lets the assembled PR proceed into the
# gate. Zero open findings satisfies both the upstream FINDINGS_ADDRESSED gate
# and the new exemption check, mirroring test_submit_root_unchanged_pr_guard.py
# / test_submit_up_unchanged_pr_guard.py's fixture shape.
# ---------------------------------------------------------------------------
def _resubmit(
kind: str,
*,
notes_structured: dict[str, Any] | None,
pr_number: int | None = 139,
) -> tuple[Choreographer, Any, Any]:
pm_id = uuid4()
task_id = uuid4()
if kind == "root":
role, team, parent = "main_pm", "main_pm", None
branch = "feature/main_pm/c80e19ff"
else:
role, team, parent = "cell_pm", "backend", uuid4()
branch = "feature/backend/cell-task"
in_prog = MagicMock(
id=task_id,
status="in_progress",
assigned_to=pm_id,
pr_number=pr_number,
branch_name=branch,
parent_task_id=parent,
batch_id=None,
team=team,
notes_structured=notes_structured,
orchestration_markers=None,
)
gated = MagicMock(**{**in_prog.__dict__, "status": "awaiting_pr_review"})
task_svc = AsyncMock()
task_svc.get.return_value = in_prog
task_svc.submit_for_review.return_value = gated
task_svc.all_subtasks_terminal.return_value = True
task_svc.uncovered_parent_acceptance_criteria.return_value = []
task_svc.agent_for.return_value = MagicMock(role=role, team=team)
task_svc.session.begin_nested = MagicMock(
return_value=MagicMock(__aenter__=AsyncMock(), __aexit__=AsyncMock())
)
c = Choreographer(_make_deps(task=task_svc, git=AsyncMock()))
cc: Any = c
cc._project_slug_for = AsyncMock(return_value="proj-slug")
# Zero open findings throughout — satisfies the upstream FINDINGS_ADDRESSED
# gate so the flow reaches the unchanged-PR guard under test.
cc._open_finding_ids = AsyncMock(return_value=())
return c, pm_id, task_id
async def _call_submit(
c: Choreographer, kind: str, pm_id: Any, task_id: Any, notes: str
) -> Any:
if kind == "root":
return await c.submit_root(pm_id, task_id, notes=notes)
return await c.submit_up(pm_id, task_id, notes=notes)
@pytest.mark.asyncio
@pytest.mark.parametrize("kind", ["root", "cell"])
async def test_exemption_granted_when_no_open_findings_and_no_prior_marker(
kind: str,
) -> None:
"""Unchanged head + zero open findings + no marker -> allowed once, marker
stamped with the current head."""
c, pm_id, task_id = _resubmit(kind, notes_structured=_unchanged_notes())
c.git.get_pr_head_sha = AsyncMock(return_value=SHA_OLD)
env = await _call_submit(
c, kind, pm_id, task_id, "re-submitting; CI blip only, no code change needed"
)
assert env.error is None, env.as_dict()
assert env.status == "awaiting_pr_review"
c.task.submit_for_review.assert_awaited_once()
t = await c.task.get(task_id)
assert markers.get_resubmit_unchanged_head(t) == SHA_OLD
@pytest.mark.asyncio
@pytest.mark.parametrize("kind", ["root", "cell"])
async def test_second_attempt_at_same_head_is_refused(kind: str) -> None:
"""The exemption is one-shot: a second resubmit at the SAME head, still
with no open findings, refuses the marker already recorded this head."""
c, pm_id, task_id = _resubmit(kind, notes_structured=_unchanged_notes())
c.git.get_pr_head_sha = AsyncMock(return_value=SHA_OLD)
first = await _call_submit(c, kind, pm_id, task_id, "first resubmit; CI blip only")
assert first.error is None, first.as_dict()
second = await _call_submit(
c, kind, pm_id, task_id, "second resubmit; still the same unchanged head"
)
assert second.error == "invalid_state", second.as_dict()
assert "unchanged" in (second.message or "").lower()
assert "already used" in (second.message or "").lower()
@pytest.mark.asyncio
@pytest.mark.parametrize("kind", ["root", "cell"])
async def test_allowed_when_head_advanced(kind: str) -> None:
"""A different current head SHA -> allowed (existing fail-open behavior),
end to end through the real verb."""
c, pm_id, task_id = _resubmit(kind, notes_structured=_unchanged_notes())
c.git.get_pr_head_sha = AsyncMock(return_value=SHA_NEW)
env = await _call_submit(
c, kind, pm_id, task_id, "resubmitting after the real fix landed"
)
assert env.error is None, env.as_dict()
assert env.status == "awaiting_pr_review"
c.task.submit_for_review.assert_awaited_once()
@@ -105,8 +105,15 @@ def _resubmit_root(
@pytest.mark.asyncio
async def test_submit_root_refuses_unchanged_pr_after_pr_fail() -> None:
"""The loop-stopper: prior pr_fail stamped head SHA X, the PR head is still
X (no new cell work on the root branch) refuse, do not open the gate."""
"""The loop-stopper still holds past the one-shot exemption: prior
pr_fail stamped head SHA X, the PR head is still X (no new cell work on
the root branch). The findings ledger here fail-opens to "nothing open"
(mock session, no real query) the same signal ``_check_submit_up_gates``
upstream already reads for FINDINGS_ADDRESSED so the first resubmit at
this head is the one-shot exemption (see
test_resubmit_unchanged_head_exemption.py for full exemption coverage);
a second resubmit at the SAME head refuses, so the loop still can't run
forever."""
c, main_pm_id, root_task_id = _resubmit_root(
notes_structured={
"pr_review": {"verdict": "failed", "head_sha": SHA_OLD, "summary": "..."}
@@ -114,9 +121,14 @@ async def test_submit_root_refuses_unchanged_pr_after_pr_fail() -> None:
)
c.git.get_pr_head_sha = AsyncMock(return_value=SHA_OLD)
env = await c.submit_root(
first = await c.submit_root(
main_pm_id, root_task_id, notes="re-submitting the root after the fix"
)
assert first.error is None, first.as_dict()
env = await c.submit_root(
main_pm_id, root_task_id, notes="re-submitting again; still unchanged"
)
assert env.error is not None, env.as_dict()
assert env.error == "invalid_state"
@@ -124,8 +136,9 @@ async def test_submit_root_refuses_unchanged_pr_after_pr_fail() -> None:
remediate = env.remediate or ""
assert "re-delegate" in remediate
assert "submit_root" in remediate
# The PR was NOT re-opened / re-pushed — the runner never ran.
c.task.submit_for_review.assert_not_awaited()
# The second attempt's PR was NOT re-opened / re-pushed — the runner ran
# only for the first (exempted) call.
c.task.submit_for_review.assert_awaited_once()
@pytest.mark.asyncio
@@ -80,9 +80,15 @@ def _resubmit_cell(
@pytest.mark.asyncio
async def test_submit_up_refuses_unchanged_pr_after_pr_fail() -> None:
"""The loop-stopper: prior pr_fail stamped head SHA X, the cell PR head is
still X (no new dev work on the cell branch) refuse, do not re-open the
gate."""
"""The loop-stopper still holds past the one-shot exemption: prior
pr_fail stamped head SHA X, the cell PR head is still X (no new dev work
on the cell branch). The findings ledger here fail-opens to "nothing
open" (mock session, no real query) — the same signal
``_check_submit_up_gates`` upstream already reads for FINDINGS_ADDRESSED
so the first resubmit at this head is the one-shot exemption (see
test_resubmit_unchanged_head_exemption.py for full exemption coverage);
a second resubmit at the SAME head refuses, so the loop still can't run
forever."""
c, cell_pm_id, cell_task_id = _resubmit_cell(
notes_structured={
"pr_review": {"verdict": "failed", "head_sha": SHA_OLD, "summary": "..."}
@@ -90,17 +96,23 @@ async def test_submit_up_refuses_unchanged_pr_after_pr_fail() -> None:
)
c.git.get_pr_head_sha = AsyncMock(return_value=SHA_OLD)
env = await c.submit_up(
first = await c.submit_up(
cell_pm_id, cell_task_id, notes="re-submitting the cell after the fix"
)
assert first.error is None, first.as_dict()
env = await c.submit_up(
cell_pm_id, cell_task_id, notes="re-submitting again; still unchanged"
)
assert env.error is not None, env.as_dict()
assert env.error == "invalid_state", env.as_dict()
assert "unchanged" in (env.message or "").lower()
remediate = env.remediate or ""
assert "submit_up" in remediate
# The cell PR was NOT re-opened / re-pushed — the runner never ran.
c.task.submit_for_review.assert_not_awaited()
# The second attempt's PR was NOT re-opened / re-pushed — the runner ran
# only for the first (exempted) call.
c.task.submit_for_review.assert_awaited_once()
@pytest.mark.asyncio
@@ -0,0 +1,151 @@
"""The block/unblock flip-flop breaker.
Live wedge: fe-pm's escalate_up auto-blocks a task and main_pm's unblock
resolves it, repeat 10 flips, 43 spawns, no forward progress, no cycle
breaker. ``unblock`` now stamps a per-task flip counter
(``markers.block_flip_count``) and, at exactly the 3rd flip, best-effort
alerts the CEO once the unblock itself always still succeeds.
"""
from __future__ import annotations
from datetime import UTC, datetime
from typing import Any
from unittest.mock import AsyncMock, MagicMock
from uuid import uuid4
import pytest
from roboco.foundation.policy.content import markers
from roboco.services.gateway.choreographer import Choreographer, ChoreographerDeps
from roboco.services.notification import NotificationService
# Named constants — ruff PLR2004 forbids magic-value comparisons.
_TWO_FLIPS = 2
_THREE_FLIPS = 3
_FOUR_FLIPS = 4
def _make_deps(**overrides: Any) -> ChoreographerDeps:
base: dict[str, Any] = {
"task": AsyncMock(),
"work_session": AsyncMock(),
"git": AsyncMock(),
"a2a": AsyncMock(),
"journal": AsyncMock(),
"audit": AsyncMock(),
"evidence_repo": AsyncMock(),
}
base.update(overrides)
base["journal"].has_decision_for_task.return_value = True
base["journal"].latest_decision_at.return_value = datetime.now(UTC)
return ChoreographerDeps(**base)
def _flip_setup() -> tuple[Choreographer, Any, Any, Any]:
"""A blocked task whose ``unblock_with_restore`` returns the SAME mock
object each call, so the flip-counter marker persists across repeated
unblock() calls the way it would on one real ORM row across requests.
"""
pm_id = uuid4()
task_id = uuid4()
t = MagicMock(
id=task_id,
status="blocked",
pre_block_state="in_progress",
pre_block_assignee=uuid4(),
pre_block_metadata={},
dependency_ids=[],
orchestration_markers=None,
)
task_svc = AsyncMock()
task_svc.get.return_value = t
task_svc.unblock_with_restore.return_value = t
task_svc.unmet_dependency_ids.return_value = []
c = Choreographer(_make_deps(task=task_svc))
return c, pm_id, task_id, t
async def _unblock_once(c: Choreographer, pm_id: Any, task_id: Any, t: Any) -> Any:
"""Re-block before each call — a fresh flip in the cycle."""
t.status = "blocked"
return await c.unblock(pm_id, task_id, "resolved upstream; restoring")
@pytest.mark.asyncio
async def test_first_and_second_unblock_do_not_notify() -> None:
c, pm_id, task_id, t = _flip_setup()
cc: Any = c
notify = AsyncMock()
cc._notify_ceo_block_flip = notify
for _ in range(2):
env = await _unblock_once(c, pm_id, task_id, t)
assert env.error is None, env.as_dict()
notify.assert_not_awaited()
assert markers.get_block_flip_count(t) == _TWO_FLIPS
@pytest.mark.asyncio
async def test_third_unblock_notifies_ceo_once() -> None:
c, pm_id, task_id, t = _flip_setup()
cc: Any = c
notify = AsyncMock()
cc._notify_ceo_block_flip = notify
for _ in range(3):
env = await _unblock_once(c, pm_id, task_id, t)
assert env.error is None, env.as_dict()
notify.assert_awaited_once_with(task_id, _THREE_FLIPS)
assert markers.is_block_flip_notified(t) is True
@pytest.mark.asyncio
async def test_fourth_unblock_does_not_renotify() -> None:
c, pm_id, task_id, t = _flip_setup()
cc: Any = c
notify = AsyncMock()
cc._notify_ceo_block_flip = notify
for _ in range(4):
env = await _unblock_once(c, pm_id, task_id, t)
assert env.error is None, env.as_dict()
notify.assert_awaited_once()
assert markers.get_block_flip_count(t) == _FOUR_FLIPS
@pytest.mark.asyncio
async def test_notification_failure_does_not_fail_unblock(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""The real ``_notify_ceo_block_flip`` swallows a notify-service failure —
unblock still succeeds on the 3rd flip."""
c, pm_id, task_id, t = _flip_setup()
monkeypatch.setattr(
NotificationService,
"send_block_flip_notification",
AsyncMock(side_effect=RuntimeError("notification service down")),
)
env = None
for _ in range(3):
env = await _unblock_once(c, pm_id, task_id, t)
assert env.error is None, env.as_dict()
assert env is not None
assert env.error is None
assert markers.is_block_flip_notified(t) is True
@pytest.mark.asyncio
async def test_counter_persists_via_marker_across_calls() -> None:
c, pm_id, task_id, t = _flip_setup()
cc: Any = c
cc._notify_ceo_block_flip = AsyncMock()
await _unblock_once(c, pm_id, task_id, t)
assert markers.get_block_flip_count(t) == 1
await _unblock_once(c, pm_id, task_id, t)
assert markers.get_block_flip_count(t) == _TWO_FLIPS
+152
View File
@@ -36,6 +36,9 @@ ONE = 1
TWO = 2
THREE = 3
FOUR = 4
FIVE = 5
SEVEN = 7
_AC_ITEM_CHAR_CAP = 200 # mirrors task_completeness._AC_MAX_ITEM_CHARS
async def _seed(session: AsyncSession) -> None:
@@ -896,3 +899,152 @@ async def test_reauthor_from_rejection_missing_draft_returns_none(
result = await engine.reauthor_from_rejection(task, "some reason")
assert result is None
# --------------------------------------------------------------------------- #
# scene criterion — a brief-named feature list becomes its own gate-checkable
# AC (delegation detail-fidelity: prose feature counts used to pass gates
# even when a dev shipped fewer scenes than the brief named)
# --------------------------------------------------------------------------- #
@pytest.mark.asyncio
async def test_draft_release_video_carries_scene_criterion_for_highlights(
db_session: AsyncSession, monkeypatch: pytest.MonkeyPatch
) -> None:
await _seed(db_session)
_enable(monkeypatch, video_on_release=True)
_mock_local_model(monkeypatch, "shipped!")
engine = video_engine_module.VideoEngine(db_session)
task = await engine.draft_release_video(version="1.0.0", changelog=_CHANGELOG)
assert task is not None
assert len(task.acceptance_criteria) == FIVE
scene_ac = task.acceptance_criteria[FOUR]
assert scene_ac.startswith("Every brief-named feature appears as its own")
assert "a huge new release" in scene_ac
@pytest.mark.asyncio
async def test_open_video_task_no_highlights_keeps_ac_count_unchanged(
db_session: AsyncSession, monkeypatch: pytest.MonkeyPatch
) -> None:
await _seed(db_session)
_enable(monkeypatch)
engine = video_engine_module.VideoEngine(db_session)
task = await engine.open_video_task(
occasion="spotlight no-highlights",
script="s",
platforms=["x"],
brief="b",
suggested_input_props={"version": "1.0.0"}, # no "highlights" key
)
assert task is not None
assert len(task.acceptance_criteria) == FOUR # no scene criterion appended
@pytest.mark.asyncio
async def test_reauthor_from_rejection_with_highlights_regenerates_scene_criterion(
db_session: AsyncSession, monkeypatch: pytest.MonkeyPatch
) -> None:
await _seed(db_session)
_enable(monkeypatch)
engine = video_engine_module.VideoEngine(db_session)
source_task = await engine.open_video_task(
occasion="release v3.0.0",
script="s",
platforms=["x"],
brief="b",
suggested_input_props={
"version": "3.0.0",
"highlights": ["Feature A", "Feature B"],
},
)
assert source_task is not None
draft = markers.get_video_draft(source_task) or {}
# Simulate propose_video: the dev's real input_props carries the same
# highlights forward onto the marker.
markers.set_video_draft(
source_task,
{
**draft,
"composition_id": "ReleaseIntro",
"input_props": {
"version": "3.0.0",
"highlights": ["Feature A", "Feature B"],
},
},
)
source_task.status = TS.COMPLETED
await db_session.flush()
post_task = await engine._originate_video_post(
source_task=source_task,
mp4_paths={"vertical": "a.mp4", "square": "b.mp4"},
captions={"x": "cap"},
platforms=["x"],
)
post_task.status = TS.CANCELLED
await db_session.flush()
revision = await engine.reauthor_from_rejection(post_task, "Logo too small")
assert revision is not None
assert len(revision.acceptance_criteria) == FIVE
scene_ac = revision.acceptance_criteria[FOUR]
assert "Feature A" in scene_ac
assert "Feature B" in scene_ac
@pytest.mark.asyncio
async def test_reauthor_from_rejection_without_highlights_adds_feedback_criterion(
db_session: AsyncSession, monkeypatch: pytest.MonkeyPatch
) -> None:
await _seed(db_session)
_enable(monkeypatch)
engine = video_engine_module.VideoEngine(db_session)
source_task = await engine.open_video_task(
occasion="release v4.0.0", script="s", platforms=["x"], brief="b"
)
assert source_task is not None
draft = markers.get_video_draft(source_task) or {}
markers.set_video_draft(source_task, {**draft, "composition_id": "Intro"})
source_task.status = TS.COMPLETED
await db_session.flush()
post_task = await engine._originate_video_post(
source_task=source_task,
mp4_paths={"vertical": "a.mp4", "square": "b.mp4"},
captions={"x": "cap"},
platforms=["x"],
)
post_task.status = TS.CANCELLED
await db_session.flush()
revision = await engine.reauthor_from_rejection(post_task, "Logo is cut off")
assert revision is not None
assert len(revision.acceptance_criteria) == FIVE
assert revision.acceptance_criteria[FOUR] == (
"Every point in the CEO rejection feedback is visibly addressed in "
"the rendered cut"
)
@pytest.mark.asyncio
async def test_open_video_task_scene_criterion_truncates_pathological_feature_list(
db_session: AsyncSession, monkeypatch: pytest.MonkeyPatch
) -> None:
await _seed(db_session)
_enable(monkeypatch)
engine = video_engine_module.VideoEngine(db_session)
features = [f"Feature number {i}" for i in range(50)]
task = await engine.open_video_task(
occasion="release pathological",
script="s",
platforms=["x"],
brief="b",
suggested_input_props={"highlights": features},
)
assert task is not None
assert len(task.acceptance_criteria) <= SEVEN
scene_ac = task.acceptance_criteria[-1]
assert len(scene_ac) <= _AC_ITEM_CHAR_CAP
assert "more)" in scene_ac # truncated, not silently dropped