mirror of
https://github.com/rennf93/roboco.git
synced 2026-08-03 07:23:24 +02:00
Delegation detail-fidelity + PM-loop hardening (#541)
* feat(gateway): delegation detail-fidelity — details survive hand-off, both directions
Details thinned out at every delegation hop: a PM child task mapped to no
parent criterion was legal (coverage only surfaced at submit_up, after the
whole wave ran — a 12-subtask docs tree grew through 8 review rounds that
way, one child titled 'docs page and route wrapper' shipping only the
page), and QA could pass work on a gestalt read (a 4-scene video brief
shipped 3 scenes past every gate because the features existed only in
prose). Three chokepoint gates:
- delegate (down): every child must declare covers_parent_criteria
resolving against the parent's real acceptance criteria — no mapping or
an unresolvable ref rejects naming every offending child and the valid
criteria; the success envelope carries parent_ac_coverage
{covered, uncovered} so a wave-planning PM sees remaining gaps in the
same turn. Full coverage stays enforced at submit_up (waves stay legal).
- pass_review (up): mandatory criteria_verified — one {criterion,
evidence} entry per task AC, matched by the findings ledger's
id-or-exact-text matcher, evidence soup-checked and capped; rejects
naming the unverified criteria; entries render deterministically into
qa_notes as '[AC] <criterion> — verified: <evidence>' lines. The old
count-only ac_verdicts gate is superseded (arg kept for back-compat).
- video briefs (structured detail at origination): an enumerable feature
list (release highlights, or input_props.highlights carried onto a
reject re-author) becomes its own scene acceptance criterion, bounded to
the AC caps; a re-author without highlights carries the
feedback-addressed criterion instead.
Extracted findings.py's criterion matcher into shared unmatched_criteria /
uncovered_acceptance_criteria instead of duplicating it; criteria_verified
joins the WAF free-text exclusion set like findings/issues.
* fix(gateway): break the block/unblock wedge — four hardening fixes from the live PM loop
A cell task looped fe-pm/main-pm block/unblock for hours (10 cycles, 43
spawns): a transient GitHub API error resolving CI became an unwaivable
blocker finding whose own fix text said no code change was required, the
submit freshness guard then demanded a commit no finding called for,
escalate_up auto-blocked, and main-pm's correct recovery plan 422'd on
the approach length cap, degrading it to a bare unblock. Four fixes:
- pr_pass CI-unresolvable refusal is now explicitly transient-worded:
retry pr_pass shortly, do NOT pr_fail over a CI-status lookup error —
a platform blip is not a code finding
- submit freshness guard grants ONE unchanged-head resubmission per
head sha when the findings ledger has zero open rows (all addressed
without code changes) — stamped via the resubmit_unchanged_head
marker so the same head can never loop a second time
- unblock carries a flip breaker: block_flip_count marker, and at the
third flip a one-shot CEO notification flags the task as structurally
wedged (unblock itself still succeeds — the breaker signals, it does
not wedge recovery)
- i_will_plan's approach cap truncates at 800 chars instead of
rejecting — an over-detailed plan must never cost the PM its turn
---------
Co-authored-by: Renn F <rennf93@users.noreply.github.com>
This commit is contained in:
@@ -418,6 +418,8 @@ Agent backends are pluggable. `roboco/llm/providers/` defines an `AgentProvider`
|
|||||||
|
|
||||||
**Reviewer/PM collision map (always-on).** The collision surface authored at delegate time (`intends_to_touch` / `adds_migration` / `touches_shared`) used to be consumed once by `SequencingService` to wire dependency edges and never shown to a reviewer again. `build_collision_context` (`roboco/services/gateway/choreographer/collision.py`, pure — no DB/IO) now surfaces it for a task under review: same-parent siblings that would collide — overlapping declared file globs, or both adding a migration (the Alembic-head collision needs no file overlap) — rendered with the overlapping globs and, where the caller hands real touched files, a declared-vs-actual drift flag. Capped at 10 siblings / 5 globs. The same builder feeds QA's `claim_review` evidence, the PR-gate's `claim_gate_review` evidence, the PM's `i_will_plan` planning briefing (no drift there — no work yet), and the panel's `GET /api/tasks/{id}/collision-map` route backing a Collision tab on the task detail page.
|
**Reviewer/PM collision map (always-on).** The collision surface authored at delegate time (`intends_to_touch` / `adds_migration` / `touches_shared`) used to be consumed once by `SequencingService` to wire dependency edges and never shown to a reviewer again. `build_collision_context` (`roboco/services/gateway/choreographer/collision.py`, pure — no DB/IO) now surfaces it for a task under review: same-parent siblings that would collide — overlapping declared file globs, or both adding a migration (the Alembic-head collision needs no file overlap) — rendered with the overlapping globs and, where the caller hands real touched files, a declared-vs-actual drift flag. Capped at 10 siblings / 5 globs. The same builder feeds QA's `claim_review` evidence, the PR-gate's `claim_gate_review` evidence, the PM's `i_will_plan` planning briefing (no drift there — no work yet), and the panel's `GET /api/tasks/{id}/collision-map` route backing a Collision tab on the task detail page.
|
||||||
|
|
||||||
|
**Delegation detail-fidelity (always-on, 2026-07-16).** Details no longer thin out at hand-off in either direction. DOWN: `delegate` refuses any child that doesn't declare `covers_parent_criteria` mapping onto the parent's real acceptance criteria (matched by id or exact text; an unresolvable ref is rejected naming the valid criteria, never silently dropped — previously the mapping was optional and coverage surfaced only at `submit_up`'s roll-up gate, after the whole wave had already run); the success envelope carries `parent_ac_coverage` `{covered, uncovered}` so a wave-planning PM sees remaining gaps in the same turn, while multi-wave planning stays legal. UP: `pass_review` requires `criteria_verified` — one `{criterion, evidence}` entry per task acceptance criterion (the findings ledger's id-or-exact-text matcher, soup-checked and length-capped evidence), rejecting with the unverified criteria named; entries render deterministically into `qa_notes` as `[AC] <criterion> — verified: <evidence>` lines, so a gestalt "looks good" pass is structurally impossible. Video briefs stopped being prose-only: an enumerable feature list (release `highlights`, or `input_props.highlights` carried onto a reject re-author) becomes its own acceptance criterion ("Every brief-named feature appears as its own fully readable scene: …", bounded to the AC caps; a re-author without highlights carries "every point in the CEO rejection feedback is visibly addressed"), so the dropped-scene class — a four-feature brief shipping three scenes past every gate — is caught by the QA per-AC stamp instead of the CEO's eyeball.
|
||||||
|
|
||||||
**PR labeler (always-on).** `derive_pr_labels` (`roboco/foundation/policy/pr_labels.py`, pure) derives the org-structure label vocabulary every fleet PR now carries: `to master` vs `to slave` (today, an assembled root→master PR is the only `to master` case — real slave-branch targeting lands with the slave/master fleet wiring), `root` for an assembled root PR, `MegaTask` for a batch-carrying task, and a layer label (`main-pm` for a Main-PM coordination root, `cell/{team}` for a cell-assembled PR, else `subtask/{team}` for a leaf dev PR). Applied best-effort at all three PR-opening sites in `GitService` so a human triaging the PR queue sees which tree and which org layer a PR belongs to at a glance.
|
**PR labeler (always-on).** `derive_pr_labels` (`roboco/foundation/policy/pr_labels.py`, pure) derives the org-structure label vocabulary every fleet PR now carries: `to master` vs `to slave` (today, an assembled root→master PR is the only `to master` case — real slave-branch targeting lands with the slave/master fleet wiring), `root` for an assembled root PR, `MegaTask` for a batch-carrying task, and a layer label (`main-pm` for a Main-PM coordination root, `cell/{team}` for a cell-assembled PR, else `subtask/{team}` for a leaf dev PR). Applied best-effort at all three PR-opening sites in `GitService` so a human triaging the PR queue sees which tree and which org layer a PR belongs to at a glance.
|
||||||
|
|
||||||
**Feature flags / company-in-a-box.** Env-gated, default-off subsystems toggle from the panel's Settings → Feature Flags card (`panel/src/components/settings/feature-flags-card.tsx`) instead of hand-editing env: web research (`ROBOCO_RESEARCH_ENABLED`), the strategy engine (`ROBOCO_STRATEGY_ENGINE_ENABLED`), pitch provisioning (`ROBOCO_PROVISIONING_*`), external / internal PR review, the agent-runtime toolchain match (`ROBOCO_TOOLCHAIN_MATCH_ENABLED`), the architectural-conventions standard (`ROBOCO_CONVENTIONS_ENABLED`), gateway-health recovery (`ROBOCO_GATEWAY_HEALTH_ENABLED`), multi-repo CI-watch (`ROBOCO_CI_WATCH_ENABLED`), the dependency-update bot (`ROBOCO_DEP_UPDATE_ENABLED`), the gated release manager (`ROBOCO_RELEASE_MANAGER_ENABLED`), the organizational memory loop (`ROBOCO_ORG_MEMORY_ENABLED`), the sandboxed dev DB/Redis (`ROBOCO_SANDBOX_DB_ENABLED`), the RoboCo X account (`ROBOCO_X_ENGINE_ENABLED`), the RoboCo video engine (`ROBOCO_VIDEO_ENGINE_ENABLED`), the board roadmap engine (`ROBOCO_ROADMAP_ENGINE_ENABLED`), Fable-mode (`ROBOCO_FABLE_MODE_ENABLED`), the vault weekly report + KB ingest (`ROBOCO_VAULT_REPORT_ENABLED` / `ROBOCO_VAULT_KB_ENABLED`), the env-sync cascade (`ROBOCO_ENV_SYNC_ENABLED`), the Telegram notifications bridge (`ROBOCO_TELEGRAM_ENABLED`), the possibilities matrix (`ROBOCO_POSSIBILITIES_MATRIX_ENABLED`), the docs-divergence sync (`ROBOCO_DOCS_SYNC_ENABLED`), and the self-heal flags above. Cloud auth (`ROBOCO_CLOUD_AUTH_ENABLED`) is deliberately NOT on this card — like `ROBOCO_DB_NETWORK_ISOLATED`, it's a compose/env-coupled flag a runtime toggle can't safely flip mid-session. A toggle persists in the settings store and takes effect on the next backend restart; an unset flag falls back to its environment / config default.
|
**Feature flags / company-in-a-box.** Env-gated, default-off subsystems toggle from the panel's Settings → Feature Flags card (`panel/src/components/settings/feature-flags-card.tsx`) instead of hand-editing env: web research (`ROBOCO_RESEARCH_ENABLED`), the strategy engine (`ROBOCO_STRATEGY_ENGINE_ENABLED`), pitch provisioning (`ROBOCO_PROVISIONING_*`), external / internal PR review, the agent-runtime toolchain match (`ROBOCO_TOOLCHAIN_MATCH_ENABLED`), the architectural-conventions standard (`ROBOCO_CONVENTIONS_ENABLED`), gateway-health recovery (`ROBOCO_GATEWAY_HEALTH_ENABLED`), multi-repo CI-watch (`ROBOCO_CI_WATCH_ENABLED`), the dependency-update bot (`ROBOCO_DEP_UPDATE_ENABLED`), the gated release manager (`ROBOCO_RELEASE_MANAGER_ENABLED`), the organizational memory loop (`ROBOCO_ORG_MEMORY_ENABLED`), the sandboxed dev DB/Redis (`ROBOCO_SANDBOX_DB_ENABLED`), the RoboCo X account (`ROBOCO_X_ENGINE_ENABLED`), the RoboCo video engine (`ROBOCO_VIDEO_ENGINE_ENABLED`), the board roadmap engine (`ROBOCO_ROADMAP_ENGINE_ENABLED`), Fable-mode (`ROBOCO_FABLE_MODE_ENABLED`), the vault weekly report + KB ingest (`ROBOCO_VAULT_REPORT_ENABLED` / `ROBOCO_VAULT_KB_ENABLED`), the env-sync cascade (`ROBOCO_ENV_SYNC_ENABLED`), the Telegram notifications bridge (`ROBOCO_TELEGRAM_ENABLED`), the possibilities matrix (`ROBOCO_POSSIBILITIES_MATRIX_ENABLED`), the docs-divergence sync (`ROBOCO_DOCS_SYNC_ENABLED`), and the self-heal flags above. Cloud auth (`ROBOCO_CLOUD_AUTH_ENABLED`) is deliberately NOT on this card — like `ROBOCO_DB_NETWORK_ISOLATED`, it's a compose/env-coupled flag a runtime toggle can't safely flip mid-session. A toggle persists in the settings store and takes effect on the next backend restart; an unset flag falls back to its environment / config default.
|
||||||
|
|||||||
@@ -12,7 +12,7 @@
|
|||||||
| `give_me_work` | `give_me_work()` |
|
| `give_me_work` | `give_me_work()` |
|
||||||
| `i_am_blocked` | `i_am_blocked(task_id: UUID, reason: str, blocker_type: str | None = None, what_needed: str | None = None)` |
|
| `i_am_blocked` | `i_am_blocked(task_id: UUID, reason: str, blocker_type: str | None = None, what_needed: str | None = None)` |
|
||||||
| `i_am_idle` | `i_am_idle()` |
|
| `i_am_idle` | `i_am_idle()` |
|
||||||
| `pass_review` | `pass_review(task_id: UUID, notes: str, ac_verdicts: list[str] | None = None)` |
|
| `pass_review` | `pass_review(task_id: UUID, notes: str, ac_verdicts: list[str] | None = None, criteria_verified: list[str | Any] | None = None)` |
|
||||||
| `resume` | `resume(task_id: UUID)` |
|
| `resume` | `resume(task_id: UUID)` |
|
||||||
| `unclaim` | `unclaim(task_id: UUID)` |
|
| `unclaim` | `unclaim(task_id: UUID)` |
|
||||||
|
|
||||||
|
|||||||
@@ -57,7 +57,7 @@ real tools live in their agent_sdk drivers, not role_config.
|
|||||||
| `give_me_work` | `give_me_work()` |
|
| `give_me_work` | `give_me_work()` |
|
||||||
| `i_am_blocked` | `i_am_blocked(task_id: UUID, reason: str, blocker_type: str | None = None, what_needed: str | None = None)` |
|
| `i_am_blocked` | `i_am_blocked(task_id: UUID, reason: str, blocker_type: str | None = None, what_needed: str | None = None)` |
|
||||||
| `i_am_idle` | `i_am_idle()` |
|
| `i_am_idle` | `i_am_idle()` |
|
||||||
| `pass_review` | `pass_review(task_id: UUID, notes: str, ac_verdicts: list[str] | None = None)` |
|
| `pass_review` | `pass_review(task_id: UUID, notes: str, ac_verdicts: list[str] | None = None, criteria_verified: list[str | Any] | None = None)` |
|
||||||
| `resume` | `resume(task_id: UUID)` |
|
| `resume` | `resume(task_id: UUID)` |
|
||||||
| `unclaim` | `unclaim(task_id: UUID)` |
|
| `unclaim` | `unclaim(task_id: UUID)` |
|
||||||
|
|
||||||
|
|||||||
@@ -75,7 +75,7 @@ async def qa_pass(
|
|||||||
choreographer: _ChoreographerDep,
|
choreographer: _ChoreographerDep,
|
||||||
) -> dict:
|
) -> dict:
|
||||||
env = await choreographer.pass_review(
|
env = await choreographer.pass_review(
|
||||||
x_agent_id, body.task_id, body.notes, body.ac_verdicts
|
x_agent_id, body.task_id, body.notes, body.ac_verdicts, body.criteria_verified
|
||||||
)
|
)
|
||||||
return envelope_to_response(env, request)
|
return envelope_to_response(env, request)
|
||||||
|
|
||||||
|
|||||||
@@ -26,6 +26,10 @@ StrList = Annotated[list[str], BeforeValidator(coerce_str_list)]
|
|||||||
_AC_MAX_ITEMS = 7
|
_AC_MAX_ITEMS = 7
|
||||||
_AC_MAX_ITEM_CHARS = 200
|
_AC_MAX_ITEM_CHARS = 200
|
||||||
|
|
||||||
|
# approach's truncate-not-reject ceiling — must match _PM_APPROACH_MIN_LEN's
|
||||||
|
# sibling ceiling in choreographer._impl (the gate re-derives its own copy).
|
||||||
|
_APPROACH_MAX_CHARS = 800
|
||||||
|
|
||||||
|
|
||||||
class SubTaskCreate(BaseModel):
|
class SubTaskCreate(BaseModel):
|
||||||
"""A PM sub_task — a delegate target AND a progress-checklist item.
|
"""A PM sub_task — a delegate target AND a progress-checklist item.
|
||||||
@@ -201,9 +205,19 @@ class PassReviewRequest(BaseModel):
|
|||||||
ac_verdicts: StrList | None = Field(
|
ac_verdicts: StrList | None = Field(
|
||||||
default=None,
|
default=None,
|
||||||
description=(
|
description=(
|
||||||
"One verification entry per acceptance criterion (in criterion "
|
"Legacy free-text per-criterion verdicts — still folded into the "
|
||||||
"order) stating how QA verified it. Every criterion must be "
|
"persisted notes but no longer gates the pass (see "
|
||||||
"covered before a pass is allowed."
|
"criteria_verified)."
|
||||||
|
),
|
||||||
|
)
|
||||||
|
criteria_verified: list[dict[str, Any]] | None = Field(
|
||||||
|
default=None,
|
||||||
|
description=(
|
||||||
|
"Mandatory when the task has acceptance criteria — one "
|
||||||
|
"{criterion, evidence} entry per criterion. criterion must match "
|
||||||
|
"an AC by id or exact text; evidence must be concrete (file:line, "
|
||||||
|
"screenshot ref, rendered-frame path, test name). Every criterion "
|
||||||
|
"must be covered or the pass is rejected."
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -348,9 +362,11 @@ class IWillPlanRequest(BaseModel):
|
|||||||
# min_length must match choreographer._impl._PM_APPROACH_MIN_LEN. Raised
|
# min_length must match choreographer._impl._PM_APPROACH_MIN_LEN. Raised
|
||||||
# 20→150: a 20-char approach was a one-liner; the approach +
|
# 20→150: a 20-char approach was a one-liner; the approach +
|
||||||
# sub_tasks are also the progress checklist, so they must be substantive.
|
# sub_tasks are also the progress checklist, so they must be substantive.
|
||||||
# max_length caps the bloat defect (approach that ran to thousands of
|
# Overflow past 800 chars is truncated, not rejected — a 422 here used to
|
||||||
# chars restating the description). Must match the gate ceiling.
|
# throw away an otherwise-good plan and degrade the PM to a bare unblock
|
||||||
approach: str = Field(..., min_length=150, max_length=800)
|
# (live incident: an over-length approach 422'd mid-flow). min_length
|
||||||
|
# stays a hard reject: a thin plan IS a defect.
|
||||||
|
approach: str = Field(..., min_length=150)
|
||||||
sub_tasks: list[SubTaskCreate] = Field(
|
sub_tasks: list[SubTaskCreate] = Field(
|
||||||
default_factory=list,
|
default_factory=list,
|
||||||
description="List of {title, description} — server assigns id + order",
|
description="List of {title, description} — server assigns id + order",
|
||||||
@@ -359,6 +375,13 @@ class IWillPlanRequest(BaseModel):
|
|||||||
risks: list[RiskCreate] = Field(default_factory=list)
|
risks: list[RiskCreate] = Field(default_factory=list)
|
||||||
open_questions: list[OpenQuestionCreate] = Field(default_factory=list)
|
open_questions: list[OpenQuestionCreate] = Field(default_factory=list)
|
||||||
|
|
||||||
|
@field_validator("approach", mode="before")
|
||||||
|
@classmethod
|
||||||
|
def _truncate_approach(cls, v: object) -> object:
|
||||||
|
if isinstance(v, str) and len(v) > _APPROACH_MAX_CHARS:
|
||||||
|
return v[: _APPROACH_MAX_CHARS - 3] + "..."
|
||||||
|
return v
|
||||||
|
|
||||||
|
|
||||||
class DelegateRequest(BaseModel):
|
class DelegateRequest(BaseModel):
|
||||||
"""HTTP body for cell_pm + main_pm `delegate` verbs.
|
"""HTTP body for cell_pm + main_pm `delegate` verbs.
|
||||||
|
|||||||
@@ -460,3 +460,62 @@ def get_docs_sync_release_version(task: HasMarkers) -> str | None:
|
|||||||
|
|
||||||
def set_docs_sync_release_version(task: HasMarkers, version: str) -> None:
|
def set_docs_sync_release_version(task: HasMarkers, version: str) -> None:
|
||||||
set_marker(task, DOCS_SYNC_RELEASE_VERSION, version)
|
set_marker(task, DOCS_SYNC_RELEASE_VERSION, version)
|
||||||
|
|
||||||
|
|
||||||
|
# --- resubmit-unchanged-head exemption -------------------------------------
|
||||||
|
# submit_root/submit_up hard-refuse a re-submit whose assembled PR head is
|
||||||
|
# unchanged since the last pr_fail (the 2026-06-27 loop-stopper). That's a
|
||||||
|
# structural deadlock when the rejection round needed no code change (e.g. a
|
||||||
|
# transient CI-lookup error) — every ledger finding gets addressed/waived but
|
||||||
|
# the head sha never moves. This records the head sha that was granted ONE
|
||||||
|
# resubmit exemption, so a second attempt at the SAME head still refuses (see
|
||||||
|
# choreographer._unchanged_pr_guard).
|
||||||
|
|
||||||
|
RESUBMIT_UNCHANGED_HEAD = "resubmit_unchanged_head"
|
||||||
|
|
||||||
|
|
||||||
|
def get_resubmit_unchanged_head(task: HasMarkers) -> str | None:
|
||||||
|
val = get_marker(task, RESUBMIT_UNCHANGED_HEAD)
|
||||||
|
return str(val) if val else None
|
||||||
|
|
||||||
|
|
||||||
|
def set_resubmit_unchanged_head(task: HasMarkers, head_sha: str) -> None:
|
||||||
|
set_marker(task, RESUBMIT_UNCHANGED_HEAD, head_sha)
|
||||||
|
|
||||||
|
|
||||||
|
# --- block/unblock flip breaker --------------------------------------------
|
||||||
|
# Counts successful `unblock` calls on a task so a repeating block/unblock
|
||||||
|
# cycle (e.g. escalate_up auto-blocks, a PM unblocks, repeat — live incident:
|
||||||
|
# 10 flips, 43 spawns with no forward progress) can alert the CEO instead of
|
||||||
|
# burning spawns forever. Payload: {"count": int, "notified": bool} — one key
|
||||||
|
# so the counter and the notified-once flag stay atomic with each other.
|
||||||
|
|
||||||
|
BLOCK_FLIP_COUNT = "block_flip_count"
|
||||||
|
|
||||||
|
|
||||||
|
def get_block_flip_count(task: HasMarkers) -> int:
|
||||||
|
val = get_marker(task, BLOCK_FLIP_COUNT)
|
||||||
|
count = val.get("count") if isinstance(val, dict) else None
|
||||||
|
return int(count) if isinstance(count, int) else 0
|
||||||
|
|
||||||
|
|
||||||
|
def is_block_flip_notified(task: HasMarkers) -> bool:
|
||||||
|
val = get_marker(task, BLOCK_FLIP_COUNT)
|
||||||
|
return bool(val.get("notified")) if isinstance(val, dict) else False
|
||||||
|
|
||||||
|
|
||||||
|
def bump_block_flip_count(task: HasMarkers) -> int:
|
||||||
|
"""Increment the flip counter; returns the new count."""
|
||||||
|
count = get_block_flip_count(task) + 1
|
||||||
|
set_marker(
|
||||||
|
task,
|
||||||
|
BLOCK_FLIP_COUNT,
|
||||||
|
{"count": count, "notified": is_block_flip_notified(task)},
|
||||||
|
)
|
||||||
|
return count
|
||||||
|
|
||||||
|
|
||||||
|
def mark_block_flip_notified(task: HasMarkers) -> None:
|
||||||
|
set_marker(
|
||||||
|
task, BLOCK_FLIP_COUNT, {"count": get_block_flip_count(task), "notified": True}
|
||||||
|
)
|
||||||
|
|||||||
@@ -702,17 +702,29 @@ def claim_review(task_id: str) -> dict[str, Any]:
|
|||||||
|
|
||||||
|
|
||||||
def pass_review(
|
def pass_review(
|
||||||
task_id: str, notes: str, ac_verdicts: StrList | None = None
|
task_id: str,
|
||||||
|
notes: str,
|
||||||
|
ac_verdicts: StrList | None = None,
|
||||||
|
criteria_verified: list[dict[str, Any]] | None = None,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
"""QA: accept the work. notes >= 80 chars; journal:learning required.
|
"""QA: accept the work. notes >= 80 chars; journal:learning required.
|
||||||
|
|
||||||
ac_verdicts: one entry per acceptance criterion (in criterion order) stating
|
criteria_verified: MANDATORY when the task has acceptance criteria — one
|
||||||
how you verified it. Every criterion must be covered — a pass is rejected
|
{criterion, evidence} entry per criterion. criterion must match an AC by
|
||||||
until all are. If any criterion does not hold, call fail_review instead.
|
id or exact text; evidence must be concrete (file:line, screenshot ref,
|
||||||
|
rendered-frame path, test name), not a gestalt "looks good". Every
|
||||||
|
criterion must be covered, or the pass is rejected naming exactly which
|
||||||
|
are missing/unmatched. If any criterion does not hold, call fail_review
|
||||||
|
instead of passing a partial.
|
||||||
|
|
||||||
|
ac_verdicts: legacy free-text, one entry per acceptance criterion — still
|
||||||
|
folded into the persisted notes but no longer gates the pass.
|
||||||
"""
|
"""
|
||||||
payload: dict[str, Any] = {"task_id": task_id, "notes": notes}
|
payload: dict[str, Any] = {"task_id": task_id, "notes": notes}
|
||||||
if ac_verdicts is not None:
|
if ac_verdicts is not None:
|
||||||
payload["ac_verdicts"] = ac_verdicts
|
payload["ac_verdicts"] = ac_verdicts
|
||||||
|
if criteria_verified is not None:
|
||||||
|
payload["criteria_verified"] = criteria_verified
|
||||||
return _post(_role_path("pass"), payload)
|
return _post(_role_path("pass"), payload)
|
||||||
|
|
||||||
|
|
||||||
@@ -965,11 +977,15 @@ def delegate(
|
|||||||
nature: One of "technical" | "non_technical".
|
nature: One of "technical" | "non_technical".
|
||||||
acceptance_criteria: Non-empty list of verifiable outcome strings.
|
acceptance_criteria: Non-empty list of verifiable outcome strings.
|
||||||
estimated_complexity: One of "low" | "medium" | "high". Default "medium".
|
estimated_complexity: One of "low" | "medium" | "high". Default "medium".
|
||||||
covers_parent_criteria: The parent task's acceptance-criterion ids this
|
covers_parent_criteria: The parent task's acceptance-criterion ids (or
|
||||||
subtask is responsible for. Declare these so the gateway can verify
|
exact text) this subtask is responsible for. REQUIRED — non-empty
|
||||||
EVERY parent criterion is claimed by a subtask and satisfied before
|
and resolvable — whenever the parent carries acceptance criteria;
|
||||||
the parent rolls up — split the parent's criteria across subtasks so
|
delegate rejects a child that maps to none of them or to a ref
|
||||||
their union covers all of them.
|
that matches no real criterion, naming the parent's actual
|
||||||
|
criteria in the rejection. A single delegate need not cover every
|
||||||
|
criterion (a wave may leave some for a later delegate — see the
|
||||||
|
response's evidence.parent_ac_coverage), but every child that IS
|
||||||
|
delegated must map to something real.
|
||||||
intends_to_touch: Collision surface — file paths/globs this subtask
|
intends_to_touch: Collision surface — file paths/globs this subtask
|
||||||
will modify. REQUIRED for task_type="code": the sibling collision
|
will modify. REQUIRED for task_type="code": the sibling collision
|
||||||
DAG can only sequence what is declared.
|
DAG can only sequence what is declared.
|
||||||
|
|||||||
@@ -231,6 +231,7 @@ _WAF_FREETEXT_BODY_FIELDS: set[str] = {
|
|||||||
"consequences",
|
"consequences",
|
||||||
"content",
|
"content",
|
||||||
"context",
|
"context",
|
||||||
|
"criteria_verified",
|
||||||
"decision",
|
"decision",
|
||||||
"description",
|
"description",
|
||||||
"details",
|
"details",
|
||||||
|
|||||||
@@ -102,6 +102,11 @@ _PM_SUBTASK_DESC_MAX_LEN = 600
|
|||||||
# tasks, not one giant plan.
|
# tasks, not one giant plan.
|
||||||
_PM_SUBTASKS_MAX = 7
|
_PM_SUBTASKS_MAX = 7
|
||||||
|
|
||||||
|
# unblock's flip breaker: alert the CEO once a task has flip-flopped
|
||||||
|
# block/unblock this many times (live incident: 10 flips, 43 spawns with no
|
||||||
|
# forward progress before anyone noticed).
|
||||||
|
_BLOCK_FLIP_NOTIFY_THRESHOLD = 3
|
||||||
|
|
||||||
|
|
||||||
def _thin_subtask_hint(sub_tasks: list[Any]) -> str | None:
|
def _thin_subtask_hint(sub_tasks: list[Any]) -> str | None:
|
||||||
"""Return a hint if any PM sub_task is title-only / thin / over-long.
|
"""Return a hint if any PM sub_task is title-only / thin / over-long.
|
||||||
@@ -5206,8 +5211,9 @@ class Choreographer:
|
|||||||
)
|
)
|
||||||
# Spec gate passed. Run delegate-specific guards the spec doesn't
|
# Spec gate passed. Run delegate-specific guards the spec doesn't
|
||||||
# model: tracing (journal:decision), chain validation, enum
|
# model: tracing (journal:decision), chain validation, enum
|
||||||
# coercion + assignee-vs-task_type, and parent-ownership/subtask-cap.
|
# coercion + assignee-vs-task_type, parent-ownership/subtask-cap, and
|
||||||
guard = await self._delegate_extra_guards(
|
# (last) decomposition-coverage.
|
||||||
|
guard = await self._delegate_post_spec_guards(
|
||||||
pm_agent_id, parent_task_id, parent, role_str, inputs
|
pm_agent_id, parent_task_id, parent, role_str, inputs
|
||||||
)
|
)
|
||||||
if guard is not None:
|
if guard is not None:
|
||||||
@@ -5221,6 +5227,75 @@ class Choreographer:
|
|||||||
pm_agent_id, parent, inputs, briefing, role_str
|
pm_agent_id, parent, inputs, briefing, role_str
|
||||||
)
|
)
|
||||||
|
|
||||||
|
async def _delegate_post_spec_guards(
|
||||||
|
self,
|
||||||
|
pm_agent_id: UUID,
|
||||||
|
parent_task_id: UUID,
|
||||||
|
parent: Any,
|
||||||
|
role_str: str,
|
||||||
|
inputs: DelegateInputs,
|
||||||
|
) -> Envelope | None:
|
||||||
|
"""``_delegate_extra_guards``, then the decomposition-coverage gate.
|
||||||
|
|
||||||
|
Split out only to keep ``delegate()``'s own return count under the
|
||||||
|
complexity budget. The coverage gate deliberately runs LAST, after
|
||||||
|
every cap/chain/tracing/static rejection, so those guards keep their
|
||||||
|
own specific message (existing tests assert on it) and the coverage
|
||||||
|
gate only ever fires on an otherwise-valid delegate.
|
||||||
|
"""
|
||||||
|
guard = await self._delegate_extra_guards(
|
||||||
|
pm_agent_id, parent_task_id, parent, role_str, inputs
|
||||||
|
)
|
||||||
|
if guard is not None:
|
||||||
|
return guard
|
||||||
|
return self._delegate_ac_coverage_guard(parent, inputs)
|
||||||
|
|
||||||
|
def _delegate_ac_coverage_guard(
|
||||||
|
self, parent: Any, inputs: DelegateInputs
|
||||||
|
) -> Envelope | None:
|
||||||
|
"""Reject a child that doesn't map to the parent's own criteria.
|
||||||
|
|
||||||
|
A parent with real acceptance criteria (every task has some) requires
|
||||||
|
every child to declare which ones it advances via
|
||||||
|
``covers_parent_criteria`` — a child mapped to nothing is either scope
|
||||||
|
creep or a sign the parent's ACs are stale, and either way the PM must
|
||||||
|
decide now, not discover the gap late at submit_up's roll-up gate
|
||||||
|
(``_parent_acs_covered_envelope``) after the whole wave already ran.
|
||||||
|
Inert when the parent has no acceptance criteria at all (nothing to
|
||||||
|
map against) — mirrors the safe-by-construction posture of
|
||||||
|
``uncovered_parent_acceptance_criteria`` et al. Does NOT require full
|
||||||
|
coverage in one call: a wave may deliberately leave criteria for a
|
||||||
|
later delegate (see the success envelope's ``parent_ac_coverage``
|
||||||
|
evidence for that signal).
|
||||||
|
"""
|
||||||
|
ac_texts = parent.acceptance_criteria or []
|
||||||
|
if not ac_texts:
|
||||||
|
return None
|
||||||
|
refs = inputs.covers_parent_criteria or []
|
||||||
|
bad = self.task.unknown_ac_refs(parent, refs) if refs else []
|
||||||
|
if refs and not bad:
|
||||||
|
return None
|
||||||
|
listing = "; ".join(ac_texts)
|
||||||
|
if not refs:
|
||||||
|
message = (
|
||||||
|
f"'{inputs.title}' declares no covers_parent_criteria, but the "
|
||||||
|
"parent has acceptance criteria to decompose."
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
message = (
|
||||||
|
f"'{inputs.title}' covers_parent_criteria has unresolvable "
|
||||||
|
f"ref(s): {'; '.join(bad)}"
|
||||||
|
)
|
||||||
|
return Envelope.invalid_state(
|
||||||
|
message=message,
|
||||||
|
remediate=(
|
||||||
|
"Map this subtask to the parent criteria it advances via "
|
||||||
|
"covers_parent_criteria (by id or exact text), or fix the "
|
||||||
|
f"parent's criteria first. Parent criteria: {listing}"
|
||||||
|
),
|
||||||
|
context_briefing={},
|
||||||
|
)
|
||||||
|
|
||||||
# Gate Set B subtask cap (pre-gateway implicit, made explicit here).
|
# Gate Set B subtask cap (pre-gateway implicit, made explicit here).
|
||||||
# Soft warn at 8, hard block at 13. Cap enforced by ``_subtask_cap_guard``.
|
# Soft warn at 8, hard block at 13. Cap enforced by ``_subtask_cap_guard``.
|
||||||
_SUBTASK_HARD_CAP: int = 12
|
_SUBTASK_HARD_CAP: int = 12
|
||||||
@@ -5977,9 +6052,29 @@ class Choreographer:
|
|||||||
status="created",
|
status="created",
|
||||||
task_id=str(new_task.id),
|
task_id=str(new_task.id),
|
||||||
next=spec_module._INTENT_VERBS["delegate"].next_hint(new_task),
|
next=spec_module._INTENT_VERBS["delegate"].next_hint(new_task),
|
||||||
|
evidence=await self._delegate_ac_coverage_evidence(parent, parent_task_id),
|
||||||
context_briefing=briefing,
|
context_briefing=briefing,
|
||||||
).with_introspection(task=new_task, role=role_str)
|
).with_introspection(task=new_task, role=role_str)
|
||||||
|
|
||||||
|
async def _delegate_ac_coverage_evidence(
|
||||||
|
self, parent: Any, parent_task_id: UUID
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""``evidence.parent_ac_coverage`` for a successful delegate.
|
||||||
|
|
||||||
|
Reuses ``uncovered_parent_acceptance_criteria`` — the exact primitive
|
||||||
|
submit_up's roll-up gate checks — so the PM sees, in the same turn,
|
||||||
|
which parent criteria the wave-so-far still leaves uncovered instead
|
||||||
|
of discovering it later at assembly. Wave-based planning stays legal:
|
||||||
|
this never blocks, it only reports. Empty (``{}``) when the parent
|
||||||
|
carries no acceptance criteria at all.
|
||||||
|
"""
|
||||||
|
ac_texts = parent.acceptance_criteria or []
|
||||||
|
if not ac_texts:
|
||||||
|
return {}
|
||||||
|
uncovered = await self.task.uncovered_parent_acceptance_criteria(parent_task_id)
|
||||||
|
covered = [t for t in ac_texts if t not in uncovered]
|
||||||
|
return {"parent_ac_coverage": {"covered": covered, "uncovered": uncovered}}
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _team_value(value: Any) -> str:
|
def _team_value(value: Any) -> str:
|
||||||
return value.value if hasattr(value, "value") else str(value)
|
return value.value if hasattr(value, "value") else str(value)
|
||||||
@@ -6792,6 +6887,7 @@ class Choreographer:
|
|||||||
)
|
)
|
||||||
|
|
||||||
t = await self.task.unblock_with_restore(pm_agent_id, task_id, restore=restore)
|
t = await self.task.unblock_with_restore(pm_agent_id, task_id, restore=restore)
|
||||||
|
await self._maybe_notify_block_flip(task_id, t)
|
||||||
next_msg = (
|
next_msg = (
|
||||||
"task restored to its pre-block state — original assignee will resume"
|
"task restored to its pre-block state — original assignee will resume"
|
||||||
if restore
|
if restore
|
||||||
@@ -6804,6 +6900,39 @@ class Choreographer:
|
|||||||
context_briefing=await self._briefing_for(pm_agent_id, task_id),
|
context_briefing=await self._briefing_for(pm_agent_id, task_id),
|
||||||
).with_introspection(task=t, role=role)
|
).with_introspection(task=t, role=role)
|
||||||
|
|
||||||
|
async def _maybe_notify_block_flip(self, task_id: UUID, t: Any) -> None:
|
||||||
|
"""Bump the flip counter; alert the CEO once past the threshold.
|
||||||
|
|
||||||
|
A resolver that keeps unblocking a task that keeps re-blocking
|
||||||
|
(fe-pm escalate_up auto-blocks / main_pm unblocks, live incident: 10
|
||||||
|
flips, 43 spawns) is a structural wedge, not automatic recovery
|
||||||
|
working as intended. Signal it once at the 3rd flip; the unblock
|
||||||
|
itself always still succeeds regardless.
|
||||||
|
"""
|
||||||
|
flip_count = markers.bump_block_flip_count(t)
|
||||||
|
if (
|
||||||
|
flip_count >= _BLOCK_FLIP_NOTIFY_THRESHOLD
|
||||||
|
and not markers.is_block_flip_notified(t)
|
||||||
|
):
|
||||||
|
markers.mark_block_flip_notified(t)
|
||||||
|
await self._notify_ceo_block_flip(task_id, flip_count)
|
||||||
|
|
||||||
|
async def _notify_ceo_block_flip(self, task_id: UUID, flip_count: int) -> None:
|
||||||
|
"""Best-effort CEO alert for a repeating block/unblock cycle; never
|
||||||
|
raises — the breaker signals, it does not wedge unblock itself."""
|
||||||
|
from roboco.services.notification import NotificationService
|
||||||
|
|
||||||
|
try:
|
||||||
|
await NotificationService().send_block_flip_notification(
|
||||||
|
task_id=str(task_id), flip_count=flip_count
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
logger.warning(
|
||||||
|
"failed to send CEO block-flip notification",
|
||||||
|
task_id=str(task_id),
|
||||||
|
flip_count=flip_count,
|
||||||
|
)
|
||||||
|
|
||||||
async def _own_review_hint(self, pm_agent_id: UUID, exclude_task_id: UUID) -> str:
|
async def _own_review_hint(self, pm_agent_id: UUID, exclude_task_id: UUID) -> str:
|
||||||
"""Remediate suffix naming the PM's OWN task ready to complete.
|
"""Remediate suffix naming the PM's OWN task ready to complete.
|
||||||
|
|
||||||
@@ -7248,25 +7377,40 @@ class Choreographer:
|
|||||||
)
|
)
|
||||||
return None
|
return None
|
||||||
|
|
||||||
async def _submit_root_unchanged_pr_guard(
|
async def _unchanged_pr_guard(
|
||||||
self, t: Any, briefing: dict[str, Any]
|
self,
|
||||||
|
t: Any,
|
||||||
|
briefing: dict[str, Any],
|
||||||
|
*,
|
||||||
|
verb: str,
|
||||||
|
pr_scope: str,
|
||||||
|
delegate_target: str,
|
||||||
|
subtask_noun: str,
|
||||||
|
work_noun: str,
|
||||||
) -> Envelope | None:
|
) -> Envelope | None:
|
||||||
"""Refuse to re-submit a root PR whose diff is unchanged since the last fail.
|
"""Shared unchanged-PR loop-stopper for submit_root (root) / submit_up (cell).
|
||||||
|
|
||||||
The hard loop-stopper for the 2026-06-27 ``pr_fail`` re-submit loop. A
|
The hard loop-stopper for the 2026-06-27 ``pr_fail`` re-submit loop. A
|
||||||
prior ``pr_fail`` stamped the assembled PR's head SHA into
|
prior ``pr_fail`` stamped the assembled PR's head SHA into
|
||||||
``notes_structured.pr_review.head_sha`` (see ``_capture_pr_head_sha`` /
|
``notes_structured.pr_review.head_sha`` (see ``_capture_pr_head_sha`` /
|
||||||
``_record_gate_verdict``). On the next ``submit_root`` this looks up the
|
``_record_gate_verdict``). This looks up the PR's CURRENT head SHA and
|
||||||
PR's CURRENT head SHA and compares. Equal ⇒ no new cell work landed on
|
compares. Equal ⇒ no new work has landed on the branch since the fail ⇒
|
||||||
the root branch since the fail ⇒ the diff the reviewer would see is
|
the diff the reviewer would see is byte-identical to the one just
|
||||||
byte-identical to the one just rejected ⇒ refuse, so no model can loop
|
rejected. Different ⇒ the branch advanced ⇒ allow.
|
||||||
itself back into ``awaiting_pr_review``. Different ⇒ the branch advanced
|
|
||||||
⇒ allow. Returns ``None`` (proceed) on every ambiguous case so the gate
|
A same-head resubmit isn't always a loop: when the rejection round's
|
||||||
FAILS OPEN: no prior ``pr_fail`` verdict, no recorded ``head_sha`` (e.g.
|
findings needed no code change (e.g. a transient CI-lookup error
|
||||||
a verdict written before this field existed), no ``pr_number``, no
|
ledgered as a finding, then waived/addressed), a flat refuse deadlocks
|
||||||
|
the task forever — the head sha can never move because there is
|
||||||
|
nothing to fix. So once every ledger finding is resolved
|
||||||
|
(``_open_finding_ids`` empty), ONE resubmission is exempted per head
|
||||||
|
sha (``markers.resubmit_unchanged_head``); a second attempt at the SAME
|
||||||
|
head still refuses — one-shot, not a standing bypass.
|
||||||
|
|
||||||
|
Returns ``None`` (proceed) on every ambiguous case so the gate FAILS
|
||||||
|
OPEN: no prior ``pr_fail`` verdict, no recorded ``head_sha`` (e.g. a
|
||||||
|
verdict written before this field existed), no ``pr_number``, no
|
||||||
resolvable project, or a git/closed-PR lookup that returns ``None``.
|
resolvable project, or a git/closed-PR lookup that returns ``None``.
|
||||||
Only the exact-unchanged case is hard-blocked; the rest fall through to
|
|
||||||
the reviewer, who can still ``pr_fail`` if the diff is bad.
|
|
||||||
"""
|
"""
|
||||||
pr_review = (getattr(t, "notes_structured", None) or {}).get("pr_review") or {}
|
pr_review = (getattr(t, "notes_structured", None) or {}).get("pr_review") or {}
|
||||||
if pr_review.get("verdict") != "failed":
|
if pr_review.get("verdict") != "failed":
|
||||||
@@ -7277,24 +7421,55 @@ class Choreographer:
|
|||||||
current = await self._current_pr_head_sha(t)
|
current = await self._current_pr_head_sha(t)
|
||||||
if current is None or current != recorded:
|
if current is None or current != recorded:
|
||||||
return None
|
return None
|
||||||
|
exemption_note = ""
|
||||||
|
if not await self._open_finding_ids(t.id):
|
||||||
|
if markers.get_resubmit_unchanged_head(t) == current:
|
||||||
|
exemption_note = (
|
||||||
|
" (the one-shot exemption for this head was already used)"
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
markers.set_resubmit_unchanged_head(t, current)
|
||||||
|
logger.info(
|
||||||
|
"unchanged-PR resubmit exemption granted — all findings"
|
||||||
|
" resolved without a code change",
|
||||||
|
task_id=str(t.id),
|
||||||
|
verb=verb,
|
||||||
|
head=current[:7],
|
||||||
|
)
|
||||||
|
return None
|
||||||
return Envelope.invalid_state(
|
return Envelope.invalid_state(
|
||||||
message=(
|
message=(
|
||||||
"the assembled root PR is unchanged since the last pr_fail"
|
f"the assembled {pr_scope} PR is unchanged since the last"
|
||||||
f" (head {current[:7]}). No new cell work has landed on the"
|
f" pr_fail (head {current[:7]}). No new {work_noun} has"
|
||||||
" root branch, so re-submitting would re-open the exact diff"
|
f" landed on the {pr_scope} branch, so re-submitting would"
|
||||||
" the reviewer just rejected and loop straight back to"
|
" re-open the exact diff the reviewer just rejected and loop"
|
||||||
" awaiting_pr_review."
|
f" straight back to awaiting_pr_review.{exemption_note}"
|
||||||
),
|
),
|
||||||
remediate=(
|
remediate=(
|
||||||
"re-delegate the fixes to the owning cell PM(s) via"
|
f"re-delegate the fixes to the {delegate_target} via"
|
||||||
" delegate(...) and wait for the cell subtasks to complete"
|
f" delegate(...) and wait for the {subtask_noun} to complete"
|
||||||
" and the root branch to be re-assembled. Do NOT call"
|
f" and the {pr_scope} branch to be re-assembled. Do NOT call"
|
||||||
" submit_root again until new cell work has advanced the"
|
f" {verb} again until new {work_noun} has advanced the"
|
||||||
" root branch HEAD."
|
f" {pr_scope} branch HEAD."
|
||||||
),
|
),
|
||||||
context_briefing=briefing,
|
context_briefing=briefing,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
async def _submit_root_unchanged_pr_guard(
|
||||||
|
self, t: Any, briefing: dict[str, Any]
|
||||||
|
) -> Envelope | None:
|
||||||
|
"""Refuse to re-submit a root PR whose diff is unchanged since the last
|
||||||
|
fail. See ``_unchanged_pr_guard`` for the shared logic."""
|
||||||
|
return await self._unchanged_pr_guard(
|
||||||
|
t,
|
||||||
|
briefing,
|
||||||
|
verb="submit_root",
|
||||||
|
pr_scope="root",
|
||||||
|
delegate_target="owning cell PM(s)",
|
||||||
|
subtask_noun="cell subtasks",
|
||||||
|
work_noun="cell work",
|
||||||
|
)
|
||||||
|
|
||||||
async def _current_pr_head_sha(self, t: Any) -> str | None:
|
async def _current_pr_head_sha(self, t: Any) -> str | None:
|
||||||
"""Best-effort current head SHA of the task's assembled PR (fail-open).
|
"""Best-effort current head SHA of the task's assembled PR (fail-open).
|
||||||
|
|
||||||
@@ -7338,33 +7513,17 @@ class Choreographer:
|
|||||||
|
|
||||||
Refuses re-submit when the cell PR's current head SHA equals the SHA
|
Refuses re-submit when the cell PR's current head SHA equals the SHA
|
||||||
``pr_fail`` recorded in ``notes_structured.pr_review.head_sha`` (no new
|
``pr_fail`` recorded in ``notes_structured.pr_review.head_sha`` (no new
|
||||||
dev work landed ⇒ byte-identical diff). Ambiguous cases FAIL OPEN via
|
dev work landed ⇒ byte-identical diff). See ``_unchanged_pr_guard`` for
|
||||||
``_current_pr_head_sha``; only the exact-unchanged case is hard-blocked.
|
the shared logic (including the findings-resolved one-shot exemption).
|
||||||
"""
|
"""
|
||||||
pr_review = (getattr(t, "notes_structured", None) or {}).get("pr_review") or {}
|
return await self._unchanged_pr_guard(
|
||||||
if pr_review.get("verdict") != "failed":
|
t,
|
||||||
return None
|
briefing,
|
||||||
recorded = pr_review.get("head_sha")
|
verb="submit_up",
|
||||||
if not recorded:
|
pr_scope="cell",
|
||||||
return None
|
delegate_target="owning developer(s)",
|
||||||
current = await self._current_pr_head_sha(t)
|
subtask_noun="dev subtasks",
|
||||||
if current is None or current != recorded:
|
work_noun="dev work",
|
||||||
return None
|
|
||||||
return Envelope.invalid_state(
|
|
||||||
message=(
|
|
||||||
"the assembled cell PR is unchanged since the last pr_fail"
|
|
||||||
f" (head {current[:7]}). No new dev work has landed on the"
|
|
||||||
" cell branch, so re-submitting would re-open the exact diff"
|
|
||||||
" the reviewer just rejected and loop straight back to"
|
|
||||||
" awaiting_pr_review."
|
|
||||||
),
|
|
||||||
remediate=(
|
|
||||||
"re-delegate the fixes to the owning developer(s) via"
|
|
||||||
" delegate(...) and wait for the dev subtasks to complete and"
|
|
||||||
" the cell branch to be re-assembled. Do NOT call submit_up"
|
|
||||||
" again until new dev work has advanced the cell branch HEAD."
|
|
||||||
),
|
|
||||||
context_briefing=briefing,
|
|
||||||
)
|
)
|
||||||
|
|
||||||
async def submit_root(
|
async def submit_root(
|
||||||
|
|||||||
@@ -124,13 +124,42 @@ def findings_count_hint(findings: Sequence[Any]) -> str | None:
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def unknown_finding_criteria(task: Any, findings: list[Finding]) -> list[str]:
|
def unmatched_criteria(task: Any, criteria: list[str]) -> list[str]:
|
||||||
"""Findings whose ``criterion`` matches neither an AC id nor AC text.
|
"""``criteria`` entries matching neither an AC id nor AC text on ``task``.
|
||||||
|
|
||||||
Mirrors ``TaskService.unknown_ac_refs`` — a criterion may be supplied by
|
The shared "by id or exact text" match — a criterion may be supplied by
|
||||||
its stable id (``acceptance_criteria_ids``, migration 036) or its exact
|
its stable id (``acceptance_criteria_ids``, migration 036) or its exact
|
||||||
text, since both representations already circulate (``declare_coverage``
|
text, since both representations already circulate (``declare_coverage``
|
||||||
accepts either). Findings with no ``criterion`` are unconstrained.
|
accepts either). Reused by ``unknown_finding_criteria`` (a finding's
|
||||||
|
``criterion``) and QA's ``criteria_verified`` per-AC gate (qa.py) so the
|
||||||
|
match semantics can't drift between the two call sites.
|
||||||
|
"""
|
||||||
|
valid_ids = set(getattr(task, "acceptance_criteria_ids", None) or [])
|
||||||
|
valid_texts = set(getattr(task, "acceptance_criteria", None) or [])
|
||||||
|
return [c for c in criteria if c not in valid_ids and c not in valid_texts]
|
||||||
|
|
||||||
|
|
||||||
|
def uncovered_acceptance_criteria(task: Any, criteria: list[str]) -> list[str]:
|
||||||
|
"""Task ACs (by exact text) not named by any of ``criteria`` (id or text).
|
||||||
|
|
||||||
|
The reverse of ``unmatched_criteria``: walks the task's own criteria
|
||||||
|
instead of the caller-supplied ones, so a per-AC verification gate can
|
||||||
|
name exactly which acceptance criterion still lacks a matching entry.
|
||||||
|
"""
|
||||||
|
ac_ids = list(getattr(task, "acceptance_criteria_ids", None) or [])
|
||||||
|
ac_texts = list(getattr(task, "acceptance_criteria", None) or [])
|
||||||
|
provided = set(criteria)
|
||||||
|
uncovered = []
|
||||||
|
for idx, text in enumerate(ac_texts):
|
||||||
|
ac_id = ac_ids[idx] if idx < len(ac_ids) else None
|
||||||
|
if text in provided or (ac_id is not None and ac_id in provided):
|
||||||
|
continue
|
||||||
|
uncovered.append(text)
|
||||||
|
return uncovered
|
||||||
|
|
||||||
|
|
||||||
|
def unknown_finding_criteria(task: Any, findings: list[Finding]) -> list[str]:
|
||||||
|
"""Findings whose ``criterion`` matches neither an AC id nor AC text.
|
||||||
|
|
||||||
Short-circuits to ``[]`` (never touching ``task.acceptance_criteria*``)
|
Short-circuits to ``[]`` (never touching ``task.acceptance_criteria*``)
|
||||||
when no finding supplies a criterion — the common case — so a task
|
when no finding supplies a criterion — the common case — so a task
|
||||||
@@ -139,9 +168,7 @@ def unknown_finding_criteria(task: Any, findings: list[Finding]) -> list[str]:
|
|||||||
criteria = [f.criterion for f in findings if f.criterion]
|
criteria = [f.criterion for f in findings if f.criterion]
|
||||||
if not criteria:
|
if not criteria:
|
||||||
return []
|
return []
|
||||||
valid_ids = set(getattr(task, "acceptance_criteria_ids", None) or [])
|
return unmatched_criteria(task, criteria)
|
||||||
valid_texts = set(getattr(task, "acceptance_criteria", None) or [])
|
|
||||||
return [c for c in criteria if c not in valid_ids and c not in valid_texts]
|
|
||||||
|
|
||||||
|
|
||||||
def criterion_mismatch_rejection(task: Any, unknown: list[str]) -> Envelope:
|
def criterion_mismatch_rejection(task: Any, unknown: list[str]) -> Envelope:
|
||||||
|
|||||||
@@ -690,17 +690,18 @@ class PRGateMixin(_Base):
|
|||||||
|
|
||||||
A reviewer must not PASS an assembled PR whose suite can't run in the
|
A reviewer must not PASS an assembled PR whose suite can't run in the
|
||||||
workspace, that carries unresolved architectural-convention
|
workspace, that carries unresolved architectural-convention
|
||||||
violations, or whose CI is red/pending/unscheduled/unresolvable;
|
violations, or whose CI is red/pending/unscheduled/unresolvable; only
|
||||||
pr_fail stays available for all three. Returns ``(rejection, None)``
|
red CI is a code defect pr_fail should act on — an unresolvable CI
|
||||||
to block, or ``(None, ci_note)`` to proceed — ``ci_note`` is a
|
lookup is a platform blip to retry, never a finding. Returns
|
||||||
non-None evidence stamp only when the CI guard passed through a
|
``(rejection, None)`` to block, or ``(None, ci_note)`` to proceed —
|
||||||
project with no CI configured at all. The toolchain/conventions
|
``ci_note`` is a non-None evidence stamp only when the CI guard
|
||||||
guards are inert when their flag is off; the CI guard fails open on
|
passed through a project with no CI configured at all. The
|
||||||
an unresolvable gate-level slug/PR number (``None`` from
|
toolchain/conventions guards are inert when their flag is off; the
|
||||||
``_resolve_ci_status``) and also passes through — with an evidence
|
CI guard fails open on an unresolvable gate-level slug/PR number
|
||||||
stamp — when ``get_pr_ci_status`` itself classifies a missing
|
(``None`` from ``_resolve_ci_status``) and also passes through —
|
||||||
project/git_url/token or an unreachable/nonexistent repo as
|
with an evidence stamp — when ``get_pr_ci_status`` itself classifies
|
||||||
``no_ci_configured``.
|
a missing project/git_url/token or an unreachable/nonexistent repo
|
||||||
|
as ``no_ci_configured``.
|
||||||
"""
|
"""
|
||||||
from roboco.config import settings as _settings
|
from roboco.config import settings as _settings
|
||||||
|
|
||||||
@@ -806,12 +807,13 @@ class PRGateMixin(_Base):
|
|||||||
"""Refuse pr_pass unless CI on the assembled PR's head commit is green.
|
"""Refuse pr_pass unless CI on the assembled PR's head commit is green.
|
||||||
|
|
||||||
Failing, pending, unscheduled, or unresolvable-via-API CI states all
|
Failing, pending, unscheduled, or unresolvable-via-API CI states all
|
||||||
block with a reviewer-aware remediation pointing at ``pr_fail`` (a
|
block, but only ``failure`` remediates via ``pr_fail`` (a reviewer
|
||||||
reviewer has no ``i_am_blocked``) — pending/unscheduled/error are
|
has no ``i_am_blocked``); pending/unscheduled/error are framed as
|
||||||
framed as retryable (wait and call pr_pass again), never as a defect
|
retryable (wait and call pr_pass again), never as a defect to route
|
||||||
to route back to the dev. A project with no CI configured at all
|
back to the dev — a GitHub API lookup error is a platform blip, not
|
||||||
passes through cleanly, returning an evidence note so the caller can
|
a finding. A project with no CI configured at all passes through
|
||||||
stamp the verdict with why the guard did not block.
|
cleanly, returning an evidence note so the caller can stamp the
|
||||||
|
verdict with why the guard did not block.
|
||||||
"""
|
"""
|
||||||
status = await self._resolve_ci_status(task_id, t)
|
status = await self._resolve_ci_status(task_id, t)
|
||||||
if status is None:
|
if status is None:
|
||||||
@@ -871,11 +873,16 @@ class PRGateMixin(_Base):
|
|||||||
"green — do NOT pr_pass before any check has run",
|
"green — do NOT pr_pass before any check has run",
|
||||||
)
|
)
|
||||||
# state == "error" (or an unrecognized value) — a genuine GitHub API
|
# state == "error" (or an unrecognized value) — a genuine GitHub API
|
||||||
# failure resolving the signal; never treat this as green.
|
# lookup failure, never treat this as a property of the PR itself.
|
||||||
|
# Live incident: a reviewer converted this transient blip into an
|
||||||
|
# unwaivable blocker finding whose own fix text said "no code change
|
||||||
|
# required" — a platform hiccup is not a diff defect.
|
||||||
return (
|
return (
|
||||||
"could not determine CI status for the assembled PR (GitHub API error)",
|
"GitHub API error resolving CI status for the assembled PR — "
|
||||||
"retry pr_pass shortly once the CI status can be resolved; "
|
"this is a transient lookup failure, not a property of the PR",
|
||||||
"if it persists, pr_fail(issues=[...]) to unwedge the PR",
|
"wait a few minutes and retry pr_pass — do NOT pr_fail over a "
|
||||||
|
"CI-status lookup error; a platform blip is not a code finding, "
|
||||||
|
"findings are for defects in the diff",
|
||||||
)
|
)
|
||||||
|
|
||||||
def _record_gate_verdict(
|
def _record_gate_verdict(
|
||||||
|
|||||||
@@ -58,6 +58,11 @@ from roboco.services.gateway.evidence_builder import build_evidence_for_task
|
|||||||
|
|
||||||
logger = structlog.get_logger()
|
logger = structlog.get_logger()
|
||||||
|
|
||||||
|
# Cap on one criteria_verified entry's `evidence` — mirrors Finding.fix's cap
|
||||||
|
# (roboco.foundation.policy.content.models._FINDING_FIX_CAP): a pointer
|
||||||
|
# (file:line, screenshot ref, test name), not a transcript.
|
||||||
|
_CRITERION_EVIDENCE_CAP = 500
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
from uuid import UUID
|
from uuid import UUID
|
||||||
|
|
||||||
@@ -427,13 +432,13 @@ class QAMixin(_Base):
|
|||||||
def _qa_ac_coverage_check(
|
def _qa_ac_coverage_check(
|
||||||
cls, task: Any, ac_verdicts: list[str] | None
|
cls, task: Any, ac_verdicts: list[str] | None
|
||||||
) -> Envelope | None:
|
) -> Envelope | None:
|
||||||
"""Per-acceptance-criterion verification gate for pass_review.
|
"""Legacy count-only per-AC gate — superseded by ``criteria_verified``.
|
||||||
|
|
||||||
QA may not pass a task until it has recorded a verification for EVERY
|
No longer wired into ``pass_review`` (a count of arbitrary strings
|
||||||
acceptance criterion. A single gestalt "looks good" approval is how a
|
never verified they actually named the right criterion — the live gap
|
||||||
silently-unbuilt criterion slips through; requiring one verdict per
|
``_validate_criteria_verified`` closes). Kept for ``ac_verdicts``'
|
||||||
criterion forces QA to check each individually. If a criterion does not
|
existing callers/tests; ``ac_verdicts`` itself still folds into the
|
||||||
hold, the QA fails the review instead of passing a partial.
|
persisted notes when supplied.
|
||||||
"""
|
"""
|
||||||
criteria = list(getattr(task, "acceptance_criteria", None) or [])
|
criteria = list(getattr(task, "acceptance_criteria", None) or [])
|
||||||
if not criteria:
|
if not criteria:
|
||||||
@@ -545,36 +550,181 @@ class QAMixin(_Base):
|
|||||||
verb=verb,
|
verb=verb,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _parse_criterion_entry(entry: Any, idx: int) -> tuple[str, str] | Envelope:
|
||||||
|
"""Validate one ``criteria_verified`` entry into a (criterion, evidence)
|
||||||
|
pair, or an ``Envelope`` rejection on any structural problem."""
|
||||||
|
criterion = entry.get("criterion") if isinstance(entry, dict) else None
|
||||||
|
evidence = entry.get("evidence") if isinstance(entry, dict) else None
|
||||||
|
if not isinstance(criterion, str) or not criterion.strip():
|
||||||
|
return Envelope.invalid_state(
|
||||||
|
message=f"criteria_verified[{idx}] is missing a `criterion` string",
|
||||||
|
remediate=(
|
||||||
|
"each entry needs {criterion, evidence} naming one "
|
||||||
|
"acceptance criterion"
|
||||||
|
),
|
||||||
|
)
|
||||||
|
if not isinstance(evidence, str) or not evidence.strip():
|
||||||
|
return Envelope.invalid_state(
|
||||||
|
message=(
|
||||||
|
f"criteria_verified[{idx}] ({criterion!r}) is missing `evidence`"
|
||||||
|
),
|
||||||
|
remediate=(
|
||||||
|
"state concrete evidence: file:line, screenshot ref, "
|
||||||
|
"rendered-frame path, test name"
|
||||||
|
),
|
||||||
|
)
|
||||||
|
if len(evidence) > _CRITERION_EVIDENCE_CAP:
|
||||||
|
return Envelope.invalid_state(
|
||||||
|
message=(
|
||||||
|
f"criteria_verified[{idx}] evidence exceeds "
|
||||||
|
f"{_CRITERION_EVIDENCE_CAP} chars"
|
||||||
|
),
|
||||||
|
remediate="keep evidence concise — a pointer, not a transcript",
|
||||||
|
)
|
||||||
|
return criterion.strip(), evidence.strip()
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _parse_criteria_verified_entries(
|
||||||
|
cls, criteria_verified: list[dict[str, Any]]
|
||||||
|
) -> tuple[list[tuple[str, str]], Envelope | None]:
|
||||||
|
"""Shape + soup validation for every ``criteria_verified`` entry.
|
||||||
|
|
||||||
|
Pure parsing — AC matching/coverage is the caller's job. Split out
|
||||||
|
of ``_validate_criteria_verified`` to keep its return count under
|
||||||
|
the complexity bound.
|
||||||
|
"""
|
||||||
|
pairs: list[tuple[str, str]] = []
|
||||||
|
for idx, entry in enumerate(criteria_verified):
|
||||||
|
parsed = cls._parse_criterion_entry(entry, idx)
|
||||||
|
if isinstance(parsed, Envelope):
|
||||||
|
return [], parsed
|
||||||
|
pairs.append(parsed)
|
||||||
|
soup = cls._free_text_soup(
|
||||||
|
checks=(("criteria_verified.evidence", [e for _, e in pairs], 8),)
|
||||||
|
)
|
||||||
|
if soup is not None:
|
||||||
|
return [], soup
|
||||||
|
return pairs, None
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _validate_criteria_verified(
|
||||||
|
cls, t: Any, criteria_verified: list[dict[str, Any]] | None
|
||||||
|
) -> tuple[list[tuple[str, str]], Envelope | None]:
|
||||||
|
"""Mandatory per-AC verification gate for pass_review.
|
||||||
|
|
||||||
|
Returns ``(pairs, rejection)`` — ``pairs`` is ``[]`` and ``rejection``
|
||||||
|
non-None on any failure: none supplied (lists every AC verbatim),
|
||||||
|
a malformed or soupy entry, an entry naming a criterion absent from
|
||||||
|
the task (names the valid criteria), or a task AC left uncovered
|
||||||
|
(names the gap). No task ACs imposes no requirement (mirrors the
|
||||||
|
legacy ``_qa_ac_coverage_check``). A gestalt "looks good" is no
|
||||||
|
longer enough — every criterion needs its own matched, evidenced
|
||||||
|
entry, or QA must call ``fail_review`` instead of passing a partial.
|
||||||
|
"""
|
||||||
|
criteria = list(getattr(t, "acceptance_criteria", None) or [])
|
||||||
|
if not criteria:
|
||||||
|
return [], None
|
||||||
|
if not criteria_verified:
|
||||||
|
return [], Envelope.invalid_state(
|
||||||
|
message=(
|
||||||
|
"pass_review needs criteria_verified naming every "
|
||||||
|
f"acceptance criterion; none supplied. Unverified: {criteria!r}"
|
||||||
|
),
|
||||||
|
remediate=(
|
||||||
|
"re-run the review and call pass_review with "
|
||||||
|
"criteria_verified=[{criterion, evidence}, ...] — stamp "
|
||||||
|
"EACH criterion with concrete evidence (file:line, "
|
||||||
|
"screenshot ref, rendered-frame path, test name)"
|
||||||
|
),
|
||||||
|
)
|
||||||
|
pairs, bad = cls._parse_criteria_verified_entries(criteria_verified)
|
||||||
|
if bad is not None:
|
||||||
|
return [], bad
|
||||||
|
provided = [c for c, _ in pairs]
|
||||||
|
if unknown := findings_lib.unmatched_criteria(t, provided):
|
||||||
|
return [], Envelope.invalid_state(
|
||||||
|
message=(
|
||||||
|
f"criteria_verified names criteria not on this task: {unknown!r}"
|
||||||
|
),
|
||||||
|
remediate=(
|
||||||
|
"each entry's criterion must match one of the task's "
|
||||||
|
f"acceptance criteria (by id or exact text): {criteria!r}"
|
||||||
|
),
|
||||||
|
)
|
||||||
|
if uncovered := findings_lib.uncovered_acceptance_criteria(t, provided):
|
||||||
|
return [], Envelope.invalid_state(
|
||||||
|
message=(
|
||||||
|
"criteria_verified is missing these acceptance criteria: "
|
||||||
|
f"{uncovered!r}"
|
||||||
|
),
|
||||||
|
remediate=(
|
||||||
|
"stamp every criterion with concrete evidence, or call "
|
||||||
|
"fail_review with the specific gap if one does not hold"
|
||||||
|
),
|
||||||
|
)
|
||||||
|
return pairs, None
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _render_criteria_verified(pairs: list[tuple[str, str]]) -> list[str]:
|
||||||
|
"""One '[AC] <criterion> — verified: <evidence>' line per entry.
|
||||||
|
|
||||||
|
Style-matched to the findings ledger's '[F-<id8>] ...' bracket-tag
|
||||||
|
rendering (``findings_lib.render_finding_line``).
|
||||||
|
"""
|
||||||
|
return [
|
||||||
|
f"[AC] {criterion} — verified: {evidence}" for criterion, evidence in pairs
|
||||||
|
]
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _merge_criteria_verified_into_notes(
|
||||||
|
cls, notes: str, pairs: list[tuple[str, str]]
|
||||||
|
) -> str:
|
||||||
|
"""Fold the per-AC verification lines into the persisted QA notes.
|
||||||
|
|
||||||
|
Mirrors ``_merge_ac_verdicts_into_notes`` — keeps the per-criterion
|
||||||
|
verification in the audit trail (qa_notes) so PM/CEO see exactly how
|
||||||
|
QA verified each acceptance criterion.
|
||||||
|
"""
|
||||||
|
lines = cls._render_criteria_verified(pairs)
|
||||||
|
if not lines:
|
||||||
|
return notes
|
||||||
|
return f"{notes}\n\n" + "\n".join(lines)
|
||||||
|
|
||||||
async def _qa_pass_final_gates(
|
async def _qa_pass_final_gates(
|
||||||
self,
|
self,
|
||||||
qa_agent_id: UUID,
|
qa_agent_id: UUID,
|
||||||
task_id: UUID,
|
task_id: UUID,
|
||||||
t: Any,
|
t: Any,
|
||||||
role_str: str,
|
role_str: str,
|
||||||
ac_verdicts: list[str] | None,
|
criteria_verified: list[dict[str, Any]] | None,
|
||||||
) -> Envelope | None:
|
) -> tuple[Envelope | None, list[tuple[str, str]]]:
|
||||||
"""AC-coverage + toolchain-runnability gates for pass_review.
|
"""Per-AC verification + toolchain-runnability gates for pass_review.
|
||||||
|
|
||||||
Returns the first rejection (already emitted), else None. QA must not
|
Returns ``(rejection, pairs)`` — the first emitted rejection (else
|
||||||
PASS on a workspace that cannot run the suite — that is a source-read
|
None) and the validated ``criteria_verified`` (criterion, evidence)
|
||||||
|
pairs for the caller to render into notes. QA must not PASS on a
|
||||||
|
workspace that cannot run the suite — that is a source-read
|
||||||
"verification"; fail_review is unaffected.
|
"verification"; fail_review is unaffected.
|
||||||
"""
|
"""
|
||||||
ac_rejection = self._qa_ac_coverage_check(t, ac_verdicts)
|
pairs, bad = self._validate_criteria_verified(t, criteria_verified)
|
||||||
if ac_rejection is not None:
|
if bad is not None:
|
||||||
return await self._emit_rejection(
|
rejection = await self._emit_rejection(
|
||||||
ac_rejection.with_introspection(task=t, role=role_str),
|
bad.with_introspection(task=t, role=role_str),
|
||||||
agent_id=qa_agent_id,
|
agent_id=qa_agent_id,
|
||||||
task_id=task_id,
|
task_id=task_id,
|
||||||
verb="pass_review",
|
verb="pass_review",
|
||||||
)
|
)
|
||||||
|
return rejection, []
|
||||||
if toolchain := await self._toolchain_broken_guard(qa_agent_id, t):
|
if toolchain := await self._toolchain_broken_guard(qa_agent_id, t):
|
||||||
return await self._emit_rejection(
|
rejection = await self._emit_rejection(
|
||||||
toolchain.with_introspection(task=t, role=role_str),
|
toolchain.with_introspection(task=t, role=role_str),
|
||||||
agent_id=qa_agent_id,
|
agent_id=qa_agent_id,
|
||||||
task_id=task_id,
|
task_id=task_id,
|
||||||
verb="pass_review",
|
verb="pass_review",
|
||||||
)
|
)
|
||||||
return None
|
return rejection, []
|
||||||
|
return None, pairs
|
||||||
|
|
||||||
async def pass_review(
|
async def pass_review(
|
||||||
self,
|
self,
|
||||||
@@ -582,6 +732,7 @@ class QAMixin(_Base):
|
|||||||
task_id: UUID,
|
task_id: UUID,
|
||||||
notes: str,
|
notes: str,
|
||||||
ac_verdicts: list[str] | None = None,
|
ac_verdicts: list[str] | None = None,
|
||||||
|
criteria_verified: list[dict[str, Any]] | None = None,
|
||||||
) -> Envelope:
|
) -> Envelope:
|
||||||
"""QA passes the task; transitions awaiting_qa → awaiting_documentation.
|
"""QA passes the task; transitions awaiting_qa → awaiting_documentation.
|
||||||
|
|
||||||
@@ -596,6 +747,16 @@ class QAMixin(_Base):
|
|||||||
The composed atomic ``qa_pass`` is then dispatched through
|
The composed atomic ``qa_pass`` is then dispatched through
|
||||||
``VerbRunner.run_intent``, after which the verb body reassigns
|
``VerbRunner.run_intent``, after which the verb body reassigns
|
||||||
the documenter for handoff.
|
the documenter for handoff.
|
||||||
|
|
||||||
|
``criteria_verified`` ({criterion, evidence} entries) is the
|
||||||
|
mandatory per-AC verification gate (``_validate_criteria_verified``):
|
||||||
|
every one of the task's acceptance criteria must be named by exactly
|
||||||
|
one entry — matched by AC id or exact text, the same match
|
||||||
|
``fail_review``'s findings ledger uses for its own ``criterion``
|
||||||
|
field — carrying substantive evidence, or the pass is refused. A
|
||||||
|
gestalt "looks good" is no longer enough; QA must walk each
|
||||||
|
criterion. ``ac_verdicts`` (legacy, count-only) still folds into the
|
||||||
|
persisted notes when supplied but no longer gates the pass.
|
||||||
"""
|
"""
|
||||||
rejection, t = await self._verify_qa_owner(qa_agent_id, task_id, "pass_review")
|
rejection, t = await self._verify_qa_owner(qa_agent_id, task_id, "pass_review")
|
||||||
if rejection is not None:
|
if rejection is not None:
|
||||||
@@ -621,18 +782,22 @@ class QAMixin(_Base):
|
|||||||
soup_checks=(("notes", notes, 8),),
|
soup_checks=(("notes", notes, 8),),
|
||||||
):
|
):
|
||||||
return gate_rejection
|
return gate_rejection
|
||||||
if rej := await self._qa_pass_final_gates(
|
final_rejection, criteria_pairs = await self._qa_pass_final_gates(
|
||||||
qa_agent_id, task_id, t, role_str, ac_verdicts
|
qa_agent_id, task_id, t, role_str, criteria_verified
|
||||||
):
|
)
|
||||||
return rej
|
if final_rejection is not None:
|
||||||
|
return final_rejection
|
||||||
|
|
||||||
briefing = await self._briefing_for(qa_agent_id, task_id)
|
briefing = await self._briefing_for(qa_agent_id, task_id)
|
||||||
|
merged_notes = self._merge_criteria_verified_into_notes(
|
||||||
|
self._merge_ac_verdicts_into_notes(notes, ac_verdicts), criteria_pairs
|
||||||
|
)
|
||||||
spec_ctx = spec_module.Context(
|
spec_ctx = spec_module.Context(
|
||||||
actor_id=qa_agent_id,
|
actor_id=qa_agent_id,
|
||||||
actor_slug=getattr(agent, "slug", None) if agent is not None else None,
|
actor_slug=getattr(agent, "slug", None) if agent is not None else None,
|
||||||
agent_team=str(agent.team) if agent is not None and agent.team else None,
|
agent_team=str(agent.team) if agent is not None and agent.team else None,
|
||||||
original_developer_slug=_extract_original_developer(t),
|
original_developer_slug=_extract_original_developer(t),
|
||||||
notes=self._merge_ac_verdicts_into_notes(notes, ac_verdicts),
|
notes=merged_notes,
|
||||||
)
|
)
|
||||||
self._store_qa_note(t, notes, ac_verdicts, passed=True)
|
self._store_qa_note(t, notes, ac_verdicts, passed=True)
|
||||||
runner = self._verb_runner()
|
runner = self._verb_runner()
|
||||||
|
|||||||
@@ -125,6 +125,40 @@ class NotificationService:
|
|||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
|
async def send_block_flip_notification(
|
||||||
|
self,
|
||||||
|
task_id: str,
|
||||||
|
flip_count: int,
|
||||||
|
to_agent: str = "ceo",
|
||||||
|
) -> None:
|
||||||
|
"""Alert an overseer that a task keeps flip-flopping block/unblock.
|
||||||
|
|
||||||
|
Raised from the choreographer's ``unblock`` once the per-task flip
|
||||||
|
counter crosses the threshold — a resolver keeps unblocking a task
|
||||||
|
that keeps re-blocking, which usually means a structural wedge (e.g.
|
||||||
|
an escalate_up/unblock cycle with no forward progress) rather than a
|
||||||
|
one-off block.
|
||||||
|
"""
|
||||||
|
logger.info(
|
||||||
|
"Sending block-flip notification", task_id=task_id, flip_count=flip_count
|
||||||
|
)
|
||||||
|
body = (
|
||||||
|
f"Task {task_id} has been blocked and unblocked {flip_count} times. "
|
||||||
|
"This flip-flop usually means a structural wedge rather than a "
|
||||||
|
"one-off block — please investigate."
|
||||||
|
)
|
||||||
|
await self._create_notification(
|
||||||
|
CreateNotificationParams(
|
||||||
|
notification_type=NotificationType.BLOCKER_ESCALATION,
|
||||||
|
priority=NotificationPriority.HIGH,
|
||||||
|
from_agent="system",
|
||||||
|
to_agents=[to_agent],
|
||||||
|
subject=f"Task {task_id} flip-flopping block/unblock ({flip_count}x)",
|
||||||
|
body=body,
|
||||||
|
related_task_id=task_id,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
async def send_qa_ready_notification(
|
async def send_qa_ready_notification(
|
||||||
self,
|
self,
|
||||||
task_id: str,
|
task_id: str,
|
||||||
|
|||||||
@@ -54,6 +54,18 @@ _AUTHORING_ACCEPTANCE_CRITERIA = [
|
|||||||
]
|
]
|
||||||
_POST_ACCEPTANCE_CRITERIA = ["CEO approves or rejects the draft"]
|
_POST_ACCEPTANCE_CRITERIA = ["CEO approves or rejects the draft"]
|
||||||
|
|
||||||
|
# Mirrors task_completeness._AC_MAX_ITEMS / _AC_MAX_ITEM_CHARS (mig 068) —
|
||||||
|
# duplicated locally per this file's no-cross-service-internals idiom (see
|
||||||
|
# vault_intake_engine.py / api/schemas/v1/flow.py for the same pattern).
|
||||||
|
_AC_MAX_ITEMS = 7
|
||||||
|
_AC_MAX_ITEM_CHARS = 200
|
||||||
|
|
||||||
|
# reauthor_from_rejection's fallback AC when the original brief named no
|
||||||
|
# enumerable features — the rejection reason is already verbatim in the brief.
|
||||||
|
_REAUTHOR_FEEDBACK_CRITERION = (
|
||||||
|
"Every point in the CEO rejection feedback is visibly addressed in the rendered cut"
|
||||||
|
)
|
||||||
|
|
||||||
_CHAT_TIMEOUT_SECONDS = 60.0
|
_CHAT_TIMEOUT_SECONDS = 60.0
|
||||||
|
|
||||||
# The whole CHANGELOG section for a release, not one bullet — capped so a
|
# The whole CHANGELOG section for a release, not one bullet — capped so a
|
||||||
@@ -146,6 +158,47 @@ def _release_video_brief(version: str, changelog: str, highlights: list[str]) ->
|
|||||||
return "\n\n".join(parts)
|
return "\n\n".join(parts)
|
||||||
|
|
||||||
|
|
||||||
|
def _scene_criterion(features: list[str]) -> str | None:
|
||||||
|
"""Turn an enumerable brief feature list into its own gate-checkable AC.
|
||||||
|
|
||||||
|
The live failure this closes: a brief named N features, the task's ACs
|
||||||
|
stayed generic, a dev shipped fewer scenes, and every gate passed because
|
||||||
|
"N features" existed only in prose. None for an empty/absent list. The
|
||||||
|
joined list is bounded to the AC per-item char cap — truncated with an
|
||||||
|
"… (+N more)" tail rather than overrunning it.
|
||||||
|
"""
|
||||||
|
trimmed = [f.strip() for f in features if f and f.strip()]
|
||||||
|
if not trimmed:
|
||||||
|
return None
|
||||||
|
prefix = "Every brief-named feature appears as its own fully readable scene: "
|
||||||
|
budget = _AC_MAX_ITEM_CHARS - len(prefix)
|
||||||
|
kept = list(trimmed)
|
||||||
|
while kept:
|
||||||
|
omitted = len(trimmed) - len(kept)
|
||||||
|
tail = f"… (+{omitted} more)" if omitted else ""
|
||||||
|
body = "; ".join(kept) + tail
|
||||||
|
if len(body) <= budget:
|
||||||
|
return prefix + body
|
||||||
|
kept.pop()
|
||||||
|
return prefix + "…" # pathological: even one (huge) feature name overruns
|
||||||
|
|
||||||
|
|
||||||
|
def _authoring_criteria(
|
||||||
|
suggested_input_props: dict[str, Any] | None,
|
||||||
|
fallback_acceptance_criterion: str | None,
|
||||||
|
) -> list[str]:
|
||||||
|
"""The base authoring ACs plus one derived/fallback criterion: the scene
|
||||||
|
criterion when ``suggested_input_props["highlights"]`` is a real list,
|
||||||
|
else ``fallback_acceptance_criterion`` (when given), else nothing."""
|
||||||
|
criteria = list(_AUTHORING_ACCEPTANCE_CRITERIA)
|
||||||
|
features = (suggested_input_props or {}).get("highlights")
|
||||||
|
criterion = _scene_criterion(features if isinstance(features, list) else [])
|
||||||
|
criterion = criterion or fallback_acceptance_criterion
|
||||||
|
if criterion is not None and len(criteria) < _AC_MAX_ITEMS:
|
||||||
|
criteria.append(criterion)
|
||||||
|
return criteria
|
||||||
|
|
||||||
|
|
||||||
def _reauthor_brief(reason: str, draft: dict[str, Any]) -> str:
|
def _reauthor_brief(reason: str, draft: dict[str, Any]) -> str:
|
||||||
"""The revision brief for a CEO-rejected cut: the verbatim rejection
|
"""The revision brief for a CEO-rejected cut: the verbatim rejection
|
||||||
feedback, a revise-in-place pointer at the existing composition (when
|
feedback, a revise-in-place pointer at the existing composition (when
|
||||||
@@ -270,6 +323,7 @@ class VideoEngine(BaseService):
|
|||||||
brief: str,
|
brief: str,
|
||||||
suggested_input_props: dict[str, Any] | None = None,
|
suggested_input_props: dict[str, Any] | None = None,
|
||||||
project_id: UUID | None = None,
|
project_id: UUID | None = None,
|
||||||
|
fallback_acceptance_criterion: str | None = None,
|
||||||
) -> TaskTable | None:
|
) -> TaskTable | None:
|
||||||
"""Originate ONE UX/UI authoring task for a bespoke video, or None.
|
"""Originate ONE UX/UI authoring task for a bespoke video, or None.
|
||||||
|
|
||||||
@@ -292,6 +346,14 @@ class VideoEngine(BaseService):
|
|||||||
"highlights": [...]}`` from the release caller) is seeded onto the
|
"highlights": [...]}`` from the release caller) is seeded onto the
|
||||||
marker as-is so the dev copies real structured data into
|
marker as-is so the dev copies real structured data into
|
||||||
``propose_video``'s ``input_props`` instead of hand-typing facts.
|
``propose_video``'s ``input_props`` instead of hand-typing facts.
|
||||||
|
|
||||||
|
When ``suggested_input_props`` carries a ``highlights`` list, it
|
||||||
|
becomes its OWN acceptance criterion (``_scene_criterion``) — a
|
||||||
|
prose-only feature list used to pass gates even when a dev shipped
|
||||||
|
fewer scenes than named. Absent highlights,
|
||||||
|
``fallback_acceptance_criterion`` (when supplied) is appended instead
|
||||||
|
— ``reauthor_from_rejection`` uses this for its
|
||||||
|
feedback-must-be-addressed criterion.
|
||||||
"""
|
"""
|
||||||
if not settings.video_engine_enabled:
|
if not settings.video_engine_enabled:
|
||||||
return None
|
return None
|
||||||
@@ -316,6 +378,9 @@ class VideoEngine(BaseService):
|
|||||||
|
|
||||||
assignee = self._select_ux_dev(open_tasks)
|
assignee = self._select_ux_dev(open_tasks)
|
||||||
enriched_brief = await self._enrich_brief(brief)
|
enriched_brief = await self._enrich_brief(brief)
|
||||||
|
acceptance_criteria = _authoring_criteria(
|
||||||
|
suggested_input_props, fallback_acceptance_criterion
|
||||||
|
)
|
||||||
# Savepoint-isolate the insert: a DBAPI error here (FK, deadlock,
|
# Savepoint-isolate the insert: a DBAPI error here (FK, deadlock,
|
||||||
# dropped connection) must roll back ONLY this insert, never poison the
|
# dropped connection) must roll back ONLY this insert, never poison the
|
||||||
# shared session — whose next commit is the caller's release-publish
|
# shared session — whose next commit is the caller's release-publish
|
||||||
@@ -326,7 +391,7 @@ class VideoEngine(BaseService):
|
|||||||
TaskCreateRequest(
|
TaskCreateRequest(
|
||||||
title=f"Video: {occasion}",
|
title=f"Video: {occasion}",
|
||||||
description=enriched_brief,
|
description=enriched_brief,
|
||||||
acceptance_criteria=list(_AUTHORING_ACCEPTANCE_CRITERIA),
|
acceptance_criteria=acceptance_criteria,
|
||||||
team=Team.UX_UI,
|
team=Team.UX_UI,
|
||||||
assigned_to=assignee,
|
assigned_to=assignee,
|
||||||
created_by=_foundation.AGENTS["system"].uuid,
|
created_by=_foundation.AGENTS["system"].uuid,
|
||||||
@@ -533,6 +598,10 @@ class VideoEngine(BaseService):
|
|||||||
draft.get("input_props") or draft.get("suggested_input_props")
|
draft.get("input_props") or draft.get("suggested_input_props")
|
||||||
),
|
),
|
||||||
project_id=project_id,
|
project_id=project_id,
|
||||||
|
# Original had highlights -> the scene criterion regenerates
|
||||||
|
# from them; no highlights -> this fallback names the reason
|
||||||
|
# (already verbatim in the brief) as the checkable outcome.
|
||||||
|
fallback_acceptance_criterion=_REAUTHOR_FEEDBACK_CRITERION,
|
||||||
)
|
)
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
self.log.warning(
|
self.log.warning(
|
||||||
|
|||||||
@@ -516,6 +516,13 @@ def qa_arc(stack: E2EStack, company: Company, task_id: Any) -> None:
|
|||||||
ac_verdicts=[
|
ac_verdicts=[
|
||||||
f"{c} — verified against the PR diff on the origin." for c in criteria
|
f"{c} — verified against the PR diff on the origin." for c in criteria
|
||||||
],
|
],
|
||||||
|
criteria_verified=[
|
||||||
|
{
|
||||||
|
"criterion": c,
|
||||||
|
"evidence": "verified against the PR diff on the origin",
|
||||||
|
}
|
||||||
|
for c in criteria
|
||||||
|
],
|
||||||
),
|
),
|
||||||
"qa pass_review",
|
"qa pass_review",
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -517,6 +517,10 @@ async def test_full_chain_through_doc_handoff(
|
|||||||
task.id,
|
task.id,
|
||||||
notes=qa_notes,
|
notes=qa_notes,
|
||||||
ac_verdicts=[f"verified: {crit}" for crit in task.acceptance_criteria],
|
ac_verdicts=[f"verified: {crit}" for crit in task.acceptance_criteria],
|
||||||
|
criteria_verified=[
|
||||||
|
{"criterion": crit, "evidence": f"verified against the PR diff: {crit}"}
|
||||||
|
for crit in task.acceptance_criteria
|
||||||
|
],
|
||||||
)
|
)
|
||||||
assert env.error is None, f"pass_review failed: {env.message}"
|
assert env.error is None, f"pass_review failed: {env.message}"
|
||||||
assert env.status == "awaiting_documentation"
|
assert env.status == "awaiting_documentation"
|
||||||
|
|||||||
@@ -513,6 +513,10 @@ async def test_qa_pass_path(
|
|||||||
task.id,
|
task.id,
|
||||||
notes=_QA_PASS_NOTES,
|
notes=_QA_PASS_NOTES,
|
||||||
ac_verdicts=[f"verified: {crit}" for crit in task.acceptance_criteria],
|
ac_verdicts=[f"verified: {crit}" for crit in task.acceptance_criteria],
|
||||||
|
criteria_verified=[
|
||||||
|
{"criterion": crit, "evidence": f"verified against the PR diff: {crit}"}
|
||||||
|
for crit in task.acceptance_criteria
|
||||||
|
],
|
||||||
)
|
)
|
||||||
assert env.error is None, f"pass_review failed: {env.message}"
|
assert env.error is None, f"pass_review failed: {env.message}"
|
||||||
assert env.status == Status.AWAITING_DOCUMENTATION.value
|
assert env.status == Status.AWAITING_DOCUMENTATION.value
|
||||||
|
|||||||
@@ -105,6 +105,7 @@ class _MockChoreographer:
|
|||||||
_task_id: object,
|
_task_id: object,
|
||||||
_notes: object,
|
_notes: object,
|
||||||
_ac_verdicts: object = None,
|
_ac_verdicts: object = None,
|
||||||
|
_criteria_verified: object = None,
|
||||||
) -> Envelope:
|
) -> Envelope:
|
||||||
self._state["task_status"] = "awaiting_documentation"
|
self._state["task_status"] = "awaiting_documentation"
|
||||||
return Envelope.ok(
|
return Envelope.ok(
|
||||||
|
|||||||
@@ -8,6 +8,7 @@ from uuid import uuid4
|
|||||||
import pytest
|
import pytest
|
||||||
from pydantic import ValidationError
|
from pydantic import ValidationError
|
||||||
from roboco.api.schemas.v1.flow import (
|
from roboco.api.schemas.v1.flow import (
|
||||||
|
_APPROACH_MAX_CHARS,
|
||||||
DelegateRequest,
|
DelegateRequest,
|
||||||
IWillPlanRequest,
|
IWillPlanRequest,
|
||||||
IWillWorkOnRequest,
|
IWillWorkOnRequest,
|
||||||
@@ -165,13 +166,35 @@ def test_i_will_plan_request_rejects_overlong_plan() -> None:
|
|||||||
assert "plan" in str(exc.value)
|
assert "plan" in str(exc.value)
|
||||||
|
|
||||||
|
|
||||||
def test_i_will_plan_request_rejects_overlong_approach() -> None:
|
def test_i_will_plan_request_truncates_overlong_approach() -> None:
|
||||||
"""approach >800 chars is rejected — no ceiling was the bloat bug."""
|
"""approach >800 chars is truncated to 800 (797 + "..."), never rejected —
|
||||||
|
a hard 422 here used to throw away an otherwise-good plan and degrade the
|
||||||
|
PM to a bare unblock (live incident)."""
|
||||||
|
req = IWillPlanRequest(
|
||||||
|
task_id=uuid4(),
|
||||||
|
plan="plan",
|
||||||
|
approach="a" * (_APPROACH_MAX_CHARS + 1),
|
||||||
|
)
|
||||||
|
assert len(req.approach) == _APPROACH_MAX_CHARS
|
||||||
|
assert req.approach.endswith("...")
|
||||||
|
assert req.approach == "a" * (_APPROACH_MAX_CHARS - 3) + "..."
|
||||||
|
|
||||||
|
|
||||||
|
def test_i_will_plan_request_approach_exactly_800_untruncated() -> None:
|
||||||
|
"""Exactly the ceiling is the boundary — passes through byte-for-byte."""
|
||||||
|
approach = "a" * _APPROACH_MAX_CHARS
|
||||||
|
req = IWillPlanRequest(task_id=uuid4(), plan="plan", approach=approach)
|
||||||
|
assert req.approach == approach
|
||||||
|
assert not req.approach.endswith("...")
|
||||||
|
|
||||||
|
|
||||||
|
def test_i_will_plan_request_rejects_thin_approach() -> None:
|
||||||
|
"""approach <150 chars is still a hard reject — a thin plan IS a defect."""
|
||||||
with pytest.raises(ValidationError) as exc:
|
with pytest.raises(ValidationError) as exc:
|
||||||
IWillPlanRequest(
|
IWillPlanRequest(
|
||||||
task_id=uuid4(),
|
task_id=uuid4(),
|
||||||
plan="plan",
|
plan="plan",
|
||||||
approach="a" * 801,
|
approach="a" * 149,
|
||||||
)
|
)
|
||||||
assert "approach" in str(exc.value)
|
assert "approach" in str(exc.value)
|
||||||
|
|
||||||
|
|||||||
@@ -6,6 +6,9 @@ from types import SimpleNamespace
|
|||||||
|
|
||||||
from roboco.foundation.policy.content import markers as m
|
from roboco.foundation.policy.content import markers as m
|
||||||
|
|
||||||
|
# Named constant — ruff PLR2004 forbids magic-value comparisons.
|
||||||
|
_TWO = 2
|
||||||
|
|
||||||
|
|
||||||
def _task(om: dict | None = None) -> SimpleNamespace:
|
def _task(om: dict | None = None) -> SimpleNamespace:
|
||||||
return SimpleNamespace(orchestration_markers=om)
|
return SimpleNamespace(orchestration_markers=om)
|
||||||
@@ -130,3 +133,25 @@ def test_docs_sync_release_version_roundtrip() -> None:
|
|||||||
assert m.get_docs_sync_release_version(t) is None
|
assert m.get_docs_sync_release_version(t) is None
|
||||||
m.set_docs_sync_release_version(t, "0.23.0")
|
m.set_docs_sync_release_version(t, "0.23.0")
|
||||||
assert m.get_docs_sync_release_version(t) == "0.23.0"
|
assert m.get_docs_sync_release_version(t) == "0.23.0"
|
||||||
|
|
||||||
|
|
||||||
|
def test_resubmit_unchanged_head_roundtrip() -> None:
|
||||||
|
t = _task()
|
||||||
|
assert m.get_resubmit_unchanged_head(t) is None
|
||||||
|
m.set_resubmit_unchanged_head(t, "aaaa1111bbbb2222")
|
||||||
|
assert m.get_resubmit_unchanged_head(t) == "aaaa1111bbbb2222"
|
||||||
|
|
||||||
|
|
||||||
|
def test_block_flip_count_bump_and_notify() -> None:
|
||||||
|
t = _task()
|
||||||
|
assert m.get_block_flip_count(t) == 0
|
||||||
|
assert m.is_block_flip_notified(t) is False
|
||||||
|
assert m.bump_block_flip_count(t) == 1
|
||||||
|
assert m.bump_block_flip_count(t) == _TWO
|
||||||
|
assert m.get_block_flip_count(t) == _TWO
|
||||||
|
# notified stays False across bumps until explicitly marked.
|
||||||
|
assert m.is_block_flip_notified(t) is False
|
||||||
|
m.mark_block_flip_notified(t)
|
||||||
|
assert m.is_block_flip_notified(t) is True
|
||||||
|
# Marking notified must not reset the counter.
|
||||||
|
assert m.get_block_flip_count(t) == _TWO
|
||||||
|
|||||||
@@ -137,6 +137,7 @@ async def test_delegate_allows_when_parent_in_progress_and_owned() -> None:
|
|||||||
assigned_to=pm_id,
|
assigned_to=pm_id,
|
||||||
team="backend",
|
team="backend",
|
||||||
quick_context="Decomposition planned; cells implement their slice next.",
|
quick_context="Decomposition planned; cells implement their slice next.",
|
||||||
|
acceptance_criteria=[],
|
||||||
)
|
)
|
||||||
new_task = MagicMock(id=uuid4())
|
new_task = MagicMock(id=uuid4())
|
||||||
task_svc = AsyncMock()
|
task_svc = AsyncMock()
|
||||||
@@ -194,6 +195,7 @@ async def test_delegate_allows_when_subtask_cap_within_soft_zone() -> None:
|
|||||||
assigned_to=pm_id,
|
assigned_to=pm_id,
|
||||||
team="backend",
|
team="backend",
|
||||||
quick_context="Decomposition planned; cells implement their slice next.",
|
quick_context="Decomposition planned; cells implement their slice next.",
|
||||||
|
acceptance_criteria=[],
|
||||||
)
|
)
|
||||||
many = [MagicMock(id=uuid4()) for _ in range(10)]
|
many = [MagicMock(id=uuid4()) for _ in range(10)]
|
||||||
new_task = MagicMock(id=uuid4())
|
new_task = MagicMock(id=uuid4())
|
||||||
@@ -223,6 +225,7 @@ async def test_delegate_allows_at_zero_subtasks() -> None:
|
|||||||
assigned_to=pm_id,
|
assigned_to=pm_id,
|
||||||
team="backend",
|
team="backend",
|
||||||
quick_context="Decomposition planned; cells implement their slice next.",
|
quick_context="Decomposition planned; cells implement their slice next.",
|
||||||
|
acceptance_criteria=[],
|
||||||
)
|
)
|
||||||
new_task = MagicMock(id=uuid4())
|
new_task = MagicMock(id=uuid4())
|
||||||
task_svc = AsyncMock()
|
task_svc = AsyncMock()
|
||||||
@@ -313,6 +316,7 @@ async def test_delegate_past_max_depth_returns_invalid_state_not_500() -> None:
|
|||||||
assigned_to=pm_id,
|
assigned_to=pm_id,
|
||||||
team="backend",
|
team="backend",
|
||||||
quick_context="Decomposition planned; cells implement their slice next.",
|
quick_context="Decomposition planned; cells implement their slice next.",
|
||||||
|
acceptance_criteria=[],
|
||||||
)
|
)
|
||||||
depth_msg = (
|
depth_msg = (
|
||||||
"Task hierarchy would exceed MAX_TASK_DEPTH=4. Create this work as a "
|
"Task hierarchy would exceed MAX_TASK_DEPTH=4. Create this work as a "
|
||||||
|
|||||||
@@ -675,6 +675,8 @@ async def test_delegate_main_pm_to_cell_pm_creates_subtask() -> None:
|
|||||||
status="in_progress",
|
status="in_progress",
|
||||||
assigned_to=main_pm_id,
|
assigned_to=main_pm_id,
|
||||||
quick_context="Decomposition planned; cells implement their slice next.",
|
quick_context="Decomposition planned; cells implement their slice next.",
|
||||||
|
# This test is about role/chain wiring, not AC coverage.
|
||||||
|
acceptance_criteria=[],
|
||||||
)
|
)
|
||||||
new_task = MagicMock(id=uuid4())
|
new_task = MagicMock(id=uuid4())
|
||||||
task_svc = AsyncMock()
|
task_svc = AsyncMock()
|
||||||
@@ -719,6 +721,8 @@ async def test_delegate_cell_pm_to_team_dev_creates_subtask() -> None:
|
|||||||
assigned_to=cell_pm_id,
|
assigned_to=cell_pm_id,
|
||||||
team="backend",
|
team="backend",
|
||||||
quick_context="Decomposition planned; cells implement their slice next.",
|
quick_context="Decomposition planned; cells implement their slice next.",
|
||||||
|
# This test is about role/chain wiring, not AC coverage.
|
||||||
|
acceptance_criteria=[],
|
||||||
)
|
)
|
||||||
new_task = MagicMock(id=uuid4())
|
new_task = MagicMock(id=uuid4())
|
||||||
task_svc = AsyncMock()
|
task_svc = AsyncMock()
|
||||||
@@ -1185,6 +1189,9 @@ async def test_delegate_main_pm_to_cell_pm_accepts_planning_subtask() -> None:
|
|||||||
status="in_progress",
|
status="in_progress",
|
||||||
assigned_to=main_pm_id,
|
assigned_to=main_pm_id,
|
||||||
quick_context="Decomposition planned; cells implement their slice next.",
|
quick_context="Decomposition planned; cells implement their slice next.",
|
||||||
|
# This test is about the code-vs-planning task_type guard, not AC
|
||||||
|
# coverage.
|
||||||
|
acceptance_criteria=[],
|
||||||
)
|
)
|
||||||
new_task = MagicMock(id=uuid4())
|
new_task = MagicMock(id=uuid4())
|
||||||
task_svc = AsyncMock()
|
task_svc = AsyncMock()
|
||||||
|
|||||||
@@ -320,6 +320,94 @@ async def test_pass_review_succeeds_and_transitions() -> None:
|
|||||||
a2a_svc.send.assert_awaited_once()
|
a2a_svc.send.assert_awaited_once()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_pass_review_rejects_without_criteria_verified_when_acs_present() -> None:
|
||||||
|
"""A task with real acceptance criteria demands criteria_verified — a
|
||||||
|
gestalt "looks good" notes string alone is no longer enough."""
|
||||||
|
qa_id = uuid4()
|
||||||
|
task_id = uuid4()
|
||||||
|
t = _qa_owned_task(
|
||||||
|
task_id, qa_id, acceptance_criteria=["returns 200", "includes timestamp"]
|
||||||
|
)
|
||||||
|
task_svc = AsyncMock()
|
||||||
|
task_svc.get.return_value = t
|
||||||
|
task_svc.agent_for.return_value = _qa_agent_mock(qa_id)
|
||||||
|
journal_svc = AsyncMock()
|
||||||
|
journal_svc.has_learning_for_task.return_value = True
|
||||||
|
deps = _make_deps(task=task_svc, journal=journal_svc)
|
||||||
|
c = Choreographer(deps)
|
||||||
|
|
||||||
|
notes = "x" * 100
|
||||||
|
env = await c.pass_review(qa_id, task_id, notes=notes)
|
||||||
|
body = env.as_dict()
|
||||||
|
assert body["error"] == "invalid_state", body
|
||||||
|
assert "returns 200" in body["message"]
|
||||||
|
assert "includes timestamp" in body["message"]
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_pass_review_renders_criteria_verified_into_notes() -> None:
|
||||||
|
"""Happy path: every AC matched + evidenced renders '[AC] ...' lines into
|
||||||
|
the persisted qa_notes and the transition still fires."""
|
||||||
|
qa_id = uuid4()
|
||||||
|
task_id = uuid4()
|
||||||
|
t = _qa_owned_task(
|
||||||
|
task_id, qa_id, acceptance_criteria=["returns 200", "includes timestamp"]
|
||||||
|
)
|
||||||
|
after = MagicMock(
|
||||||
|
id=task_id,
|
||||||
|
status="awaiting_documentation",
|
||||||
|
assigned_to=qa_id,
|
||||||
|
team="backend",
|
||||||
|
pr_url="https://x/pr/8",
|
||||||
|
qa_evidence_inspected=True,
|
||||||
|
)
|
||||||
|
task_svc = AsyncMock()
|
||||||
|
task_svc.get.return_value = t
|
||||||
|
task_svc.agent_for.return_value = _qa_agent_mock(qa_id)
|
||||||
|
task_svc.qa_pass.return_value = after
|
||||||
|
task_svc.documenter_for_team.return_value = MagicMock(id=uuid4())
|
||||||
|
task_svc.session = MagicMock()
|
||||||
|
task_svc.session.begin_nested = MagicMock(
|
||||||
|
return_value=MagicMock(
|
||||||
|
__aenter__=AsyncMock(return_value=None),
|
||||||
|
__aexit__=AsyncMock(return_value=False),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
_stub_empty_ledger(task_svc.session)
|
||||||
|
journal_svc = AsyncMock()
|
||||||
|
journal_svc.has_learning_for_task.return_value = True
|
||||||
|
a2a_svc = AsyncMock()
|
||||||
|
deps = _make_deps(task=task_svc, journal=journal_svc, a2a=a2a_svc)
|
||||||
|
c = Choreographer(deps)
|
||||||
|
|
||||||
|
notes = (
|
||||||
|
"Reviewed PR carefully. Rendered every scene and checked each frame "
|
||||||
|
"against the brief before approving."
|
||||||
|
)
|
||||||
|
env = await c.pass_review(
|
||||||
|
qa_id,
|
||||||
|
task_id,
|
||||||
|
notes=notes,
|
||||||
|
criteria_verified=[
|
||||||
|
{"criterion": "returns 200", "evidence": "test_healthz asserts 200"},
|
||||||
|
{
|
||||||
|
"criterion": "includes timestamp",
|
||||||
|
"evidence": "frame diff shows ts field at README.md line 12",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
)
|
||||||
|
assert env.error is None, env.as_dict()
|
||||||
|
assert env.status == "awaiting_documentation"
|
||||||
|
task_svc.qa_pass.assert_awaited_once()
|
||||||
|
persisted_notes = task_svc.qa_pass.call_args.args[2]
|
||||||
|
assert "[AC] returns 200 — verified: test_healthz asserts 200" in persisted_notes
|
||||||
|
assert (
|
||||||
|
"[AC] includes timestamp — verified: frame diff shows ts field at "
|
||||||
|
"README.md line 12" in persisted_notes
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_pass_review_not_assigned_returns_not_authorized() -> None:
|
async def test_pass_review_not_assigned_returns_not_authorized() -> None:
|
||||||
qa_id = uuid4()
|
qa_id = uuid4()
|
||||||
|
|||||||
@@ -0,0 +1,213 @@
|
|||||||
|
"""Decomposition-coverage gate: delegate rejects a child that doesn't map to
|
||||||
|
the parent's acceptance criteria, and a successful delegate reports the
|
||||||
|
parent's remaining coverage gaps in evidence.
|
||||||
|
|
||||||
|
Live failure this closes (see CLAUDE.md "delegate" section): unmapped
|
||||||
|
children were only ever caught late, at submit_up's roll-up gate — after a
|
||||||
|
whole wave of subtasks had already run. Moving the mapping check to delegate
|
||||||
|
time surfaces "child covers nothing" / "ref doesn't match a real criterion"
|
||||||
|
before any subtask is created.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
from typing import Any
|
||||||
|
from unittest.mock import AsyncMock, MagicMock
|
||||||
|
from uuid import uuid4
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from roboco.services.gateway.choreographer import (
|
||||||
|
Choreographer,
|
||||||
|
ChoreographerDeps,
|
||||||
|
DelegateInputs,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _make_deps(**overrides: Any) -> ChoreographerDeps:
|
||||||
|
base: dict[str, Any] = {
|
||||||
|
"task": AsyncMock(),
|
||||||
|
"work_session": AsyncMock(),
|
||||||
|
"git": AsyncMock(),
|
||||||
|
"a2a": AsyncMock(),
|
||||||
|
"journal": AsyncMock(),
|
||||||
|
"audit": AsyncMock(),
|
||||||
|
"evidence_repo": AsyncMock(),
|
||||||
|
}
|
||||||
|
base.update(overrides)
|
||||||
|
repo = base["evidence_repo"]
|
||||||
|
for method in (
|
||||||
|
"list_unread_a2a",
|
||||||
|
"list_unread_mentions",
|
||||||
|
"list_pending_notifications",
|
||||||
|
"task_metadata_gaps",
|
||||||
|
"recent_team_activity",
|
||||||
|
"blockers_in_lane",
|
||||||
|
"journal_highlights_for_task",
|
||||||
|
):
|
||||||
|
getattr(repo, method).return_value = []
|
||||||
|
_ldef = base["journal"].latest_decision_at.return_value
|
||||||
|
if type(_ldef).__name__ in ("MagicMock", "AsyncMock"):
|
||||||
|
base["journal"].latest_decision_at.return_value = datetime.now(UTC)
|
||||||
|
return ChoreographerDeps(**base)
|
||||||
|
|
||||||
|
|
||||||
|
def _parent_with_criteria(pm_id: Any) -> MagicMock:
|
||||||
|
return MagicMock(
|
||||||
|
id=uuid4(),
|
||||||
|
project_id=uuid4(),
|
||||||
|
status="in_progress",
|
||||||
|
assigned_to=pm_id,
|
||||||
|
team="backend",
|
||||||
|
quick_context="Decomposition planned; cells implement their slice next.",
|
||||||
|
acceptance_criteria=["Criterion A", "Criterion B"],
|
||||||
|
acceptance_criteria_ids=["id-a", "id-b"],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _inputs(**kw: Any) -> DelegateInputs:
|
||||||
|
base: dict[str, Any] = {
|
||||||
|
"title": "Implement endpoint",
|
||||||
|
"description": "Add /v1/foo endpoint with tests",
|
||||||
|
"assigned_to": "be-dev-1",
|
||||||
|
"team": "backend",
|
||||||
|
"task_type": "code",
|
||||||
|
"nature": "technical",
|
||||||
|
"acceptance_criteria": ["GET /v1/foo returns 200 with body"],
|
||||||
|
"intends_to_touch": ["backend/api/routers/foo.py"],
|
||||||
|
}
|
||||||
|
base.update(kw)
|
||||||
|
return DelegateInputs(**base)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_delegate_rejects_child_with_no_mapping() -> None:
|
||||||
|
"""A parent with real ACs rejects a child that maps to none of them."""
|
||||||
|
pm_id = uuid4()
|
||||||
|
parent = _parent_with_criteria(pm_id)
|
||||||
|
task_svc = AsyncMock()
|
||||||
|
task_svc.get.return_value = parent
|
||||||
|
task_svc.agent_for.return_value = MagicMock(role="cell_pm", team="backend")
|
||||||
|
task_svc.get_subtasks.return_value = []
|
||||||
|
deps = _make_deps(task=task_svc)
|
||||||
|
c = Choreographer(deps)
|
||||||
|
|
||||||
|
env = await c.delegate(pm_id, parent.id, _inputs(title="Orphan slice"))
|
||||||
|
body = env.as_dict()
|
||||||
|
assert body["error"] == "invalid_state", body
|
||||||
|
assert "Orphan slice" in body["message"]
|
||||||
|
assert "no covers_parent_criteria" in body["message"]
|
||||||
|
assert "Criterion A" in body["remediate"] and "Criterion B" in body["remediate"]
|
||||||
|
task_svc.create_subtask.assert_not_awaited()
|
||||||
|
task_svc.unknown_ac_refs.assert_not_called()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_delegate_rejects_unresolvable_ref_lists_valid_criteria() -> None:
|
||||||
|
"""A ref matching neither a criterion id nor its exact text is rejected,
|
||||||
|
with the parent's real criteria named so the PM can pick a valid one."""
|
||||||
|
pm_id = uuid4()
|
||||||
|
parent = _parent_with_criteria(pm_id)
|
||||||
|
task_svc = AsyncMock()
|
||||||
|
task_svc.get.return_value = parent
|
||||||
|
task_svc.agent_for.return_value = MagicMock(role="cell_pm", team="backend")
|
||||||
|
task_svc.get_subtasks.return_value = []
|
||||||
|
task_svc.unknown_ac_refs = MagicMock(return_value=["bogus-ref"])
|
||||||
|
deps = _make_deps(task=task_svc)
|
||||||
|
c = Choreographer(deps)
|
||||||
|
|
||||||
|
env = await c.delegate(
|
||||||
|
pm_id,
|
||||||
|
parent.id,
|
||||||
|
_inputs(title="Endpoint slice", covers_parent_criteria=["bogus-ref"]),
|
||||||
|
)
|
||||||
|
body = env.as_dict()
|
||||||
|
assert body["error"] == "invalid_state", body
|
||||||
|
assert "Endpoint slice" in body["message"]
|
||||||
|
assert "bogus-ref" in body["message"]
|
||||||
|
assert "Criterion A" in body["remediate"] and "Criterion B" in body["remediate"]
|
||||||
|
task_svc.create_subtask.assert_not_awaited()
|
||||||
|
task_svc.unknown_ac_refs.assert_called_once_with(parent, ["bogus-ref"])
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_delegate_rejects_multiple_unresolvable_refs_in_one_envelope() -> None:
|
||||||
|
"""Every unresolvable ref is named in the one rejection, not just the first."""
|
||||||
|
pm_id = uuid4()
|
||||||
|
parent = _parent_with_criteria(pm_id)
|
||||||
|
task_svc = AsyncMock()
|
||||||
|
task_svc.get.return_value = parent
|
||||||
|
task_svc.agent_for.return_value = MagicMock(role="cell_pm", team="backend")
|
||||||
|
task_svc.get_subtasks.return_value = []
|
||||||
|
task_svc.unknown_ac_refs = MagicMock(return_value=["bogus-one", "bogus-two"])
|
||||||
|
deps = _make_deps(task=task_svc)
|
||||||
|
c = Choreographer(deps)
|
||||||
|
|
||||||
|
env = await c.delegate(
|
||||||
|
pm_id,
|
||||||
|
parent.id,
|
||||||
|
_inputs(covers_parent_criteria=["bogus-one", "bogus-two"]),
|
||||||
|
)
|
||||||
|
body = env.as_dict()
|
||||||
|
assert body["error"] == "invalid_state", body
|
||||||
|
assert "bogus-one" in body["message"]
|
||||||
|
assert "bogus-two" in body["message"]
|
||||||
|
task_svc.create_subtask.assert_not_awaited()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_delegate_success_evidence_carries_covered_and_uncovered() -> None:
|
||||||
|
"""A resolvable mapping creates the subtask and reports the parent's
|
||||||
|
coverage split in evidence, using the same primitive submit_up checks."""
|
||||||
|
pm_id = uuid4()
|
||||||
|
parent = _parent_with_criteria(pm_id)
|
||||||
|
new_task = MagicMock(id=uuid4())
|
||||||
|
task_svc = AsyncMock()
|
||||||
|
task_svc.get.return_value = parent
|
||||||
|
task_svc.agent_for.return_value = MagicMock(role="cell_pm", team="backend")
|
||||||
|
task_svc.get_subtasks.return_value = []
|
||||||
|
task_svc.unknown_ac_refs = MagicMock(return_value=[])
|
||||||
|
task_svc.create_subtask.return_value = new_task
|
||||||
|
task_svc.uncovered_parent_acceptance_criteria.return_value = ["Criterion B"]
|
||||||
|
deps = _make_deps(task=task_svc)
|
||||||
|
c = Choreographer(deps)
|
||||||
|
|
||||||
|
env = await c.delegate(pm_id, parent.id, _inputs(covers_parent_criteria=["id-a"]))
|
||||||
|
body = env.as_dict()
|
||||||
|
assert body["error"] is None, body
|
||||||
|
assert body["status"] == "created"
|
||||||
|
assert body["evidence"]["parent_ac_coverage"] == {
|
||||||
|
"covered": ["Criterion A"],
|
||||||
|
"uncovered": ["Criterion B"],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_delegate_wave_leaving_acs_uncovered_still_succeeds() -> None:
|
||||||
|
"""No full-coverage hard gate at delegate: a wave may leave criteria for a
|
||||||
|
later delegate call — the child is still created, gaps just get listed."""
|
||||||
|
pm_id = uuid4()
|
||||||
|
parent = _parent_with_criteria(pm_id)
|
||||||
|
parent.acceptance_criteria = ["Criterion A", "Criterion B", "Criterion C"]
|
||||||
|
parent.acceptance_criteria_ids = ["id-a", "id-b", "id-c"]
|
||||||
|
new_task = MagicMock(id=uuid4())
|
||||||
|
task_svc = AsyncMock()
|
||||||
|
task_svc.get.return_value = parent
|
||||||
|
task_svc.agent_for.return_value = MagicMock(role="cell_pm", team="backend")
|
||||||
|
task_svc.get_subtasks.return_value = []
|
||||||
|
task_svc.unknown_ac_refs = MagicMock(return_value=[])
|
||||||
|
task_svc.create_subtask.return_value = new_task
|
||||||
|
task_svc.uncovered_parent_acceptance_criteria.return_value = [
|
||||||
|
"Criterion B",
|
||||||
|
"Criterion C",
|
||||||
|
]
|
||||||
|
deps = _make_deps(task=task_svc)
|
||||||
|
c = Choreographer(deps)
|
||||||
|
|
||||||
|
env = await c.delegate(pm_id, parent.id, _inputs(covers_parent_criteria=["id-a"]))
|
||||||
|
body = env.as_dict()
|
||||||
|
assert body["error"] is None, body
|
||||||
|
coverage = body["evidence"]["parent_ac_coverage"]
|
||||||
|
assert coverage["covered"] == ["Criterion A"]
|
||||||
|
assert coverage["uncovered"] == ["Criterion B", "Criterion C"]
|
||||||
|
task_svc.create_subtask.assert_awaited_once()
|
||||||
@@ -65,6 +65,8 @@ def _parent_in_progress(pm_id: Any) -> MagicMock:
|
|||||||
priority=2,
|
priority=2,
|
||||||
# delegate obligates the PM's quick_context resumption section.
|
# delegate obligates the PM's quick_context resumption section.
|
||||||
quick_context="Decomposition planned; cells implement their slice next.",
|
quick_context="Decomposition planned; cells implement their slice next.",
|
||||||
|
# These tests are about task_completeness, not AC coverage.
|
||||||
|
acceptance_criteria=[],
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -63,6 +63,8 @@ def _parent(pm_id: object) -> MagicMock:
|
|||||||
team="backend",
|
team="backend",
|
||||||
# delegate obligates the PM's quick_context resumption section.
|
# delegate obligates the PM's quick_context resumption section.
|
||||||
quick_context="Decomposition planned; cells implement their slice next.",
|
quick_context="Decomposition planned; cells implement their slice next.",
|
||||||
|
# These tests are about the parent-lock ordering, not AC coverage.
|
||||||
|
acceptance_criteria=[],
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -51,6 +51,8 @@ def _parent(pm_id: Any, product_id: Any = None, project_id: Any = None) -> Magic
|
|||||||
team="backend",
|
team="backend",
|
||||||
# delegate obligates the PM's quick_context resumption section.
|
# delegate obligates the PM's quick_context resumption section.
|
||||||
quick_context="Decomposition planned; cells implement their slice next.",
|
quick_context="Decomposition planned; cells implement their slice next.",
|
||||||
|
# These tests are about project routing, not AC coverage.
|
||||||
|
acceptance_criteria=[],
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -4,12 +4,14 @@ Before this guard, ``pr_pass`` had no CI-status check at all — a reviewer coul
|
|||||||
pass an assembled PR whose CI was red, still running, or not yet scheduled.
|
pass an assembled PR whose CI was red, still running, or not yet scheduled.
|
||||||
``_ci_status_guard`` (wired into ``_pr_pass_blocked`` alongside the existing
|
``_ci_status_guard`` (wired into ``_pr_pass_blocked`` alongside the existing
|
||||||
toolchain/conventions guards) reads ``GitService.get_pr_ci_status`` and blocks
|
toolchain/conventions guards) reads ``GitService.get_pr_ci_status`` and blocks
|
||||||
on failure/pending/pending_not_scheduled/error with reviewer-aware remediation
|
on failure/pending/pending_not_scheduled/error; only ``failure`` remediates
|
||||||
(``pr_fail``, never ``i_am_blocked`` — a reviewer has no such verb). A project
|
via ``pr_fail`` (never ``i_am_blocked`` — a reviewer has no such verb) — the
|
||||||
with no CI configured at all passes through cleanly, stamping the verdict note
|
``error`` state is a transient GitHub API lookup failure and remediates via
|
||||||
with why the guard did not block. ``pr_fail`` is unaffected by CI state
|
retry only, never ``pr_fail``. A project with no CI configured at all passes
|
||||||
entirely, and the separate inbound ``PRReviewerMixin`` surface
|
through cleanly, stamping the verdict note with why the guard did not block.
|
||||||
(``claim_pr_review`` / ``post_pr_review``) never consults CI status at all.
|
``pr_fail`` is unaffected by CI state entirely, and the separate inbound
|
||||||
|
``PRReviewerMixin`` surface (``claim_pr_review`` / ``post_pr_review``) never
|
||||||
|
consults CI status at all.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
@@ -162,6 +164,7 @@ async def test_pr_pass_blocked_on_github_api_error() -> None:
|
|||||||
assert env.error == "invalid_state"
|
assert env.error == "invalid_state"
|
||||||
assert "GitHub API error" in (env.message or "")
|
assert "GitHub API error" in (env.message or "")
|
||||||
assert "retry" in (env.remediate or "").lower()
|
assert "retry" in (env.remediate or "").lower()
|
||||||
|
assert "do NOT pr_fail" in (env.remediate or "")
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
|
|||||||
@@ -0,0 +1,175 @@
|
|||||||
|
"""pass_review requires a matched, evidenced verification per acceptance
|
||||||
|
criterion — not just a count of arbitrary strings.
|
||||||
|
|
||||||
|
Live failure this closes: QA passed a rendered video shipping 3 of the
|
||||||
|
brief's 4 named scenes because nothing forced the reviewer to walk each
|
||||||
|
acceptance criterion individually. Mirrors the test idiom in
|
||||||
|
test_qa_ac_coverage.py, one level stricter: criteria_verified entries must
|
||||||
|
each match a real AC (by id or exact text) and carry substantive evidence.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from types import SimpleNamespace
|
||||||
|
|
||||||
|
from roboco.services.gateway.choreographer import Choreographer
|
||||||
|
|
||||||
|
_EVIDENCE_CAP = 500
|
||||||
|
|
||||||
|
|
||||||
|
def _task(criteria: list[str], ids: list[str] | None = None) -> SimpleNamespace:
|
||||||
|
return SimpleNamespace(
|
||||||
|
acceptance_criteria=criteria,
|
||||||
|
acceptance_criteria_ids=ids or [],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_no_criteria_imposes_no_requirement() -> None:
|
||||||
|
pairs, rej = Choreographer._validate_criteria_verified(_task([]), None)
|
||||||
|
assert pairs == []
|
||||||
|
assert rej is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_none_supplied_lists_every_criterion_verbatim() -> None:
|
||||||
|
criteria = [
|
||||||
|
"scene 1 renders",
|
||||||
|
"scene 2 renders",
|
||||||
|
"scene 3 renders",
|
||||||
|
"scene 4 renders",
|
||||||
|
]
|
||||||
|
t = _task(criteria)
|
||||||
|
pairs, rej = Choreographer._validate_criteria_verified(t, None)
|
||||||
|
assert pairs == []
|
||||||
|
assert rej is not None
|
||||||
|
body = rej.as_dict()
|
||||||
|
assert body["error"] == "invalid_state", body
|
||||||
|
for crit in criteria:
|
||||||
|
assert crit in body["message"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_empty_list_is_treated_as_none_supplied() -> None:
|
||||||
|
t = _task(["a"])
|
||||||
|
pairs, rej = Choreographer._validate_criteria_verified(t, [])
|
||||||
|
assert pairs == []
|
||||||
|
assert rej is not None
|
||||||
|
|
||||||
|
|
||||||
|
def test_partial_coverage_names_the_missing_criterion() -> None:
|
||||||
|
t = _task(["a", "b", "c"])
|
||||||
|
pairs, rej = Choreographer._validate_criteria_verified(
|
||||||
|
t,
|
||||||
|
[
|
||||||
|
{"criterion": "a", "evidence": "frame 1 shows a rendered"},
|
||||||
|
{"criterion": "b", "evidence": "frame 2 shows b rendered"},
|
||||||
|
],
|
||||||
|
)
|
||||||
|
assert pairs == []
|
||||||
|
assert rej is not None
|
||||||
|
assert "c" in rej.as_dict()["message"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_unmatched_criterion_is_rejected_naming_valid_ones() -> None:
|
||||||
|
t = _task(["a", "b"])
|
||||||
|
pairs, rej = Choreographer._validate_criteria_verified(
|
||||||
|
t,
|
||||||
|
[
|
||||||
|
{"criterion": "a", "evidence": "frame 1 shows a rendered"},
|
||||||
|
{"criterion": "not-a-real-ac", "evidence": "frame 2 shows something"},
|
||||||
|
],
|
||||||
|
)
|
||||||
|
assert pairs == []
|
||||||
|
assert rej is not None
|
||||||
|
body = rej.as_dict()
|
||||||
|
assert "not-a-real-ac" in body["message"]
|
||||||
|
assert "a" in body["remediate"] and "b" in body["remediate"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_missing_criterion_key_is_rejected() -> None:
|
||||||
|
t = _task(["a"])
|
||||||
|
pairs, rej = Choreographer._validate_criteria_verified(
|
||||||
|
t, [{"evidence": "frame 1 shows a rendered"}]
|
||||||
|
)
|
||||||
|
assert pairs == []
|
||||||
|
assert rej is not None
|
||||||
|
|
||||||
|
|
||||||
|
def test_blank_evidence_is_rejected() -> None:
|
||||||
|
t = _task(["a"])
|
||||||
|
pairs, rej = Choreographer._validate_criteria_verified(
|
||||||
|
t, [{"criterion": "a", "evidence": " "}]
|
||||||
|
)
|
||||||
|
assert pairs == []
|
||||||
|
assert rej is not None
|
||||||
|
|
||||||
|
|
||||||
|
def test_soup_evidence_is_rejected() -> None:
|
||||||
|
t = _task(["a"])
|
||||||
|
pairs, rej = Choreographer._validate_criteria_verified(
|
||||||
|
t, [{"criterion": "a", "evidence": "wip"}]
|
||||||
|
)
|
||||||
|
assert pairs == []
|
||||||
|
assert rej is not None
|
||||||
|
|
||||||
|
|
||||||
|
def test_overlong_evidence_is_rejected() -> None:
|
||||||
|
t = _task(["a"])
|
||||||
|
pairs, rej = Choreographer._validate_criteria_verified(
|
||||||
|
t, [{"criterion": "a", "evidence": "x" * (_EVIDENCE_CAP + 100)}]
|
||||||
|
)
|
||||||
|
assert pairs == []
|
||||||
|
assert rej is not None
|
||||||
|
assert str(_EVIDENCE_CAP) in rej.as_dict()["message"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_full_coverage_by_exact_text_passes() -> None:
|
||||||
|
t = _task(["a", "b"])
|
||||||
|
pairs, rej = Choreographer._validate_criteria_verified(
|
||||||
|
t,
|
||||||
|
[
|
||||||
|
{"criterion": "a", "evidence": "frame 1 shows a rendered fully"},
|
||||||
|
{"criterion": "b", "evidence": "frame 2 shows b rendered fully"},
|
||||||
|
],
|
||||||
|
)
|
||||||
|
assert rej is None
|
||||||
|
assert pairs == [
|
||||||
|
("a", "frame 1 shows a rendered fully"),
|
||||||
|
("b", "frame 2 shows b rendered fully"),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def test_full_coverage_by_ac_id_passes() -> None:
|
||||||
|
t = _task(["scene renders"], ids=["AC-1"])
|
||||||
|
pairs, rej = Choreographer._validate_criteria_verified(
|
||||||
|
t, [{"criterion": "AC-1", "evidence": "rendered-frame path: out/frame3.png"}]
|
||||||
|
)
|
||||||
|
assert rej is None
|
||||||
|
assert pairs == [("AC-1", "rendered-frame path: out/frame3.png")]
|
||||||
|
|
||||||
|
|
||||||
|
def test_extra_entries_beyond_the_ac_set_are_allowed() -> None:
|
||||||
|
t = _task(["a"])
|
||||||
|
pairs, rej = Choreographer._validate_criteria_verified(
|
||||||
|
t,
|
||||||
|
[{"criterion": "a", "evidence": "frame 1 shows a rendered fully"}],
|
||||||
|
)
|
||||||
|
assert rej is None
|
||||||
|
assert len(pairs) == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_render_criteria_verified_matches_style() -> None:
|
||||||
|
lines = Choreographer._render_criteria_verified(
|
||||||
|
[("scene 1 renders", "frame 12 shows scene 1 fully")]
|
||||||
|
)
|
||||||
|
assert lines == ["[AC] scene 1 renders — verified: frame 12 shows scene 1 fully"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_merge_criteria_verified_into_notes() -> None:
|
||||||
|
merged = Choreographer._merge_criteria_verified_into_notes(
|
||||||
|
"base review", [("a", "evidence a"), ("b", "evidence b")]
|
||||||
|
)
|
||||||
|
assert "[AC] a — verified: evidence a" in merged
|
||||||
|
assert "[AC] b — verified: evidence b" in merged
|
||||||
|
|
||||||
|
|
||||||
|
def test_merge_with_no_pairs_returns_notes_unchanged() -> None:
|
||||||
|
assert Choreographer._merge_criteria_verified_into_notes("base", []) == "base"
|
||||||
@@ -0,0 +1,233 @@
|
|||||||
|
"""The one-shot resubmit exemption on the unchanged-PR loop-stopper.
|
||||||
|
|
||||||
|
Live wedge: a cell task looped block/unblock for hours because the freshness
|
||||||
|
guard (``_unchanged_pr_guard``, shared by ``submit_root``/``submit_up``)
|
||||||
|
demands a new commit after ANY ``pr_fail`` — a structural deadlock when the
|
||||||
|
rejection round's findings require no code change (e.g. a transient
|
||||||
|
CI-lookup error ledgered as a finding, then waived/addressed). Once every
|
||||||
|
ledger finding is resolved (``_open_finding_ids`` empty), ONE resubmission is
|
||||||
|
exempted per head sha (``markers.resubmit_unchanged_head``); a second attempt
|
||||||
|
at the SAME head still refuses. Covers both call sites: ``submit_root``
|
||||||
|
(root) and ``submit_up`` (cell).
|
||||||
|
|
||||||
|
Note on test style: the upstream ``FINDINGS_ADDRESSED`` tracing gate
|
||||||
|
(``_check_submit_up_gates``, shared by both verbs) already refuses the whole
|
||||||
|
verb with ``tracing_gap`` whenever findings are open — by the time
|
||||||
|
``_unchanged_pr_guard`` runs, open findings are already impossible through the
|
||||||
|
public ``submit_root``/``submit_up`` entrypoints. The "findings still open"
|
||||||
|
and "ambiguous case short-circuits before the findings check" scenarios are
|
||||||
|
therefore exercised by calling the guard method directly (defense-in-depth on
|
||||||
|
the guard's own logic); the exemption-grant and one-shot-per-head scenarios
|
||||||
|
drive the full verb end-to-end to prove the real wiring.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
from typing import Any
|
||||||
|
from unittest.mock import AsyncMock, MagicMock
|
||||||
|
from uuid import uuid4
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from roboco.foundation.policy.content import markers
|
||||||
|
from roboco.services.gateway.choreographer import Choreographer, ChoreographerDeps
|
||||||
|
|
||||||
|
SHA_OLD = "aaaa1111bbbb2222cccc3333dddd4444eeee5555"
|
||||||
|
SHA_NEW = "9999888877776666555544443333222211110000"
|
||||||
|
|
||||||
|
|
||||||
|
def _make_deps(**overrides: Any) -> ChoreographerDeps:
|
||||||
|
base: dict[str, Any] = {
|
||||||
|
"task": AsyncMock(),
|
||||||
|
"work_session": AsyncMock(),
|
||||||
|
"git": AsyncMock(),
|
||||||
|
"a2a": AsyncMock(),
|
||||||
|
"journal": AsyncMock(),
|
||||||
|
"audit": AsyncMock(),
|
||||||
|
"evidence_repo": AsyncMock(),
|
||||||
|
}
|
||||||
|
base.update(overrides)
|
||||||
|
base["journal"].has_decision_for_task.return_value = True
|
||||||
|
base["journal"].latest_decision_at.return_value = datetime.now(UTC)
|
||||||
|
base["journal"].has_reflect_for_task.return_value = True
|
||||||
|
return ChoreographerDeps(**base)
|
||||||
|
|
||||||
|
|
||||||
|
def _unchanged_notes(head_sha: str = SHA_OLD) -> dict[str, Any]:
|
||||||
|
return {"pr_review": {"verdict": "failed", "head_sha": head_sha, "summary": "..."}}
|
||||||
|
|
||||||
|
|
||||||
|
async def _call_guard(
|
||||||
|
c: Choreographer, kind: str, t: Any, briefing: dict[str, Any]
|
||||||
|
) -> Any:
|
||||||
|
if kind == "root":
|
||||||
|
return await c._submit_root_unchanged_pr_guard(t, briefing)
|
||||||
|
return await c._submit_up_unchanged_pr_guard(t, briefing)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Direct guard tests — isolate ``_unchanged_pr_guard`` from the upstream
|
||||||
|
# FINDINGS_ADDRESSED gate, which already forbids open findings from ever
|
||||||
|
# reaching here through the real submit_root/submit_up flow.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
@pytest.mark.parametrize("kind", ["root", "cell"])
|
||||||
|
async def test_refused_when_findings_still_open(kind: str) -> None:
|
||||||
|
"""Unchanged head + open findings -> refused (behavior unchanged)."""
|
||||||
|
c = Choreographer(_make_deps())
|
||||||
|
cc: Any = c
|
||||||
|
cc._current_pr_head_sha = AsyncMock(return_value=SHA_OLD)
|
||||||
|
cc._open_finding_ids = AsyncMock(return_value=("abcd1234",))
|
||||||
|
t = MagicMock(
|
||||||
|
id=uuid4(), notes_structured=_unchanged_notes(), orchestration_markers=None
|
||||||
|
)
|
||||||
|
|
||||||
|
env = await _call_guard(c, kind, t, {})
|
||||||
|
|
||||||
|
assert env is not None
|
||||||
|
assert env.error == "invalid_state", env.as_dict()
|
||||||
|
assert "unchanged" in (env.message or "").lower()
|
||||||
|
assert markers.get_resubmit_unchanged_head(t) is None
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
@pytest.mark.parametrize("kind", ["root", "cell"])
|
||||||
|
async def test_allowed_when_head_advanced_before_findings_are_even_checked(
|
||||||
|
kind: str,
|
||||||
|
) -> None:
|
||||||
|
"""A different current head SHA -> allowed; the ambiguity check short-
|
||||||
|
circuits before the findings/exemption logic ever runs."""
|
||||||
|
c = Choreographer(_make_deps())
|
||||||
|
cc: Any = c
|
||||||
|
cc._current_pr_head_sha = AsyncMock(return_value=SHA_NEW)
|
||||||
|
open_findings_spy = AsyncMock(return_value=("abcd1234",))
|
||||||
|
cc._open_finding_ids = open_findings_spy
|
||||||
|
t = MagicMock(
|
||||||
|
id=uuid4(), notes_structured=_unchanged_notes(), orchestration_markers=None
|
||||||
|
)
|
||||||
|
|
||||||
|
env = await _call_guard(c, kind, t, {})
|
||||||
|
|
||||||
|
assert env is None
|
||||||
|
open_findings_spy.assert_not_awaited()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# End-to-end tests — drive the real submit_root/submit_up verb to prove the
|
||||||
|
# exemption's marker stamp actually lets the assembled PR proceed into the
|
||||||
|
# gate. Zero open findings satisfies both the upstream FINDINGS_ADDRESSED gate
|
||||||
|
# and the new exemption check, mirroring test_submit_root_unchanged_pr_guard.py
|
||||||
|
# / test_submit_up_unchanged_pr_guard.py's fixture shape.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def _resubmit(
|
||||||
|
kind: str,
|
||||||
|
*,
|
||||||
|
notes_structured: dict[str, Any] | None,
|
||||||
|
pr_number: int | None = 139,
|
||||||
|
) -> tuple[Choreographer, Any, Any]:
|
||||||
|
pm_id = uuid4()
|
||||||
|
task_id = uuid4()
|
||||||
|
if kind == "root":
|
||||||
|
role, team, parent = "main_pm", "main_pm", None
|
||||||
|
branch = "feature/main_pm/c80e19ff"
|
||||||
|
else:
|
||||||
|
role, team, parent = "cell_pm", "backend", uuid4()
|
||||||
|
branch = "feature/backend/cell-task"
|
||||||
|
in_prog = MagicMock(
|
||||||
|
id=task_id,
|
||||||
|
status="in_progress",
|
||||||
|
assigned_to=pm_id,
|
||||||
|
pr_number=pr_number,
|
||||||
|
branch_name=branch,
|
||||||
|
parent_task_id=parent,
|
||||||
|
batch_id=None,
|
||||||
|
team=team,
|
||||||
|
notes_structured=notes_structured,
|
||||||
|
orchestration_markers=None,
|
||||||
|
)
|
||||||
|
gated = MagicMock(**{**in_prog.__dict__, "status": "awaiting_pr_review"})
|
||||||
|
task_svc = AsyncMock()
|
||||||
|
task_svc.get.return_value = in_prog
|
||||||
|
task_svc.submit_for_review.return_value = gated
|
||||||
|
task_svc.all_subtasks_terminal.return_value = True
|
||||||
|
task_svc.uncovered_parent_acceptance_criteria.return_value = []
|
||||||
|
task_svc.agent_for.return_value = MagicMock(role=role, team=team)
|
||||||
|
task_svc.session.begin_nested = MagicMock(
|
||||||
|
return_value=MagicMock(__aenter__=AsyncMock(), __aexit__=AsyncMock())
|
||||||
|
)
|
||||||
|
c = Choreographer(_make_deps(task=task_svc, git=AsyncMock()))
|
||||||
|
cc: Any = c
|
||||||
|
cc._project_slug_for = AsyncMock(return_value="proj-slug")
|
||||||
|
# Zero open findings throughout — satisfies the upstream FINDINGS_ADDRESSED
|
||||||
|
# gate so the flow reaches the unchanged-PR guard under test.
|
||||||
|
cc._open_finding_ids = AsyncMock(return_value=())
|
||||||
|
return c, pm_id, task_id
|
||||||
|
|
||||||
|
|
||||||
|
async def _call_submit(
|
||||||
|
c: Choreographer, kind: str, pm_id: Any, task_id: Any, notes: str
|
||||||
|
) -> Any:
|
||||||
|
if kind == "root":
|
||||||
|
return await c.submit_root(pm_id, task_id, notes=notes)
|
||||||
|
return await c.submit_up(pm_id, task_id, notes=notes)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
@pytest.mark.parametrize("kind", ["root", "cell"])
|
||||||
|
async def test_exemption_granted_when_no_open_findings_and_no_prior_marker(
|
||||||
|
kind: str,
|
||||||
|
) -> None:
|
||||||
|
"""Unchanged head + zero open findings + no marker -> allowed once, marker
|
||||||
|
stamped with the current head."""
|
||||||
|
c, pm_id, task_id = _resubmit(kind, notes_structured=_unchanged_notes())
|
||||||
|
c.git.get_pr_head_sha = AsyncMock(return_value=SHA_OLD)
|
||||||
|
|
||||||
|
env = await _call_submit(
|
||||||
|
c, kind, pm_id, task_id, "re-submitting; CI blip only, no code change needed"
|
||||||
|
)
|
||||||
|
|
||||||
|
assert env.error is None, env.as_dict()
|
||||||
|
assert env.status == "awaiting_pr_review"
|
||||||
|
c.task.submit_for_review.assert_awaited_once()
|
||||||
|
t = await c.task.get(task_id)
|
||||||
|
assert markers.get_resubmit_unchanged_head(t) == SHA_OLD
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
@pytest.mark.parametrize("kind", ["root", "cell"])
|
||||||
|
async def test_second_attempt_at_same_head_is_refused(kind: str) -> None:
|
||||||
|
"""The exemption is one-shot: a second resubmit at the SAME head, still
|
||||||
|
with no open findings, refuses — the marker already recorded this head."""
|
||||||
|
c, pm_id, task_id = _resubmit(kind, notes_structured=_unchanged_notes())
|
||||||
|
c.git.get_pr_head_sha = AsyncMock(return_value=SHA_OLD)
|
||||||
|
|
||||||
|
first = await _call_submit(c, kind, pm_id, task_id, "first resubmit; CI blip only")
|
||||||
|
assert first.error is None, first.as_dict()
|
||||||
|
|
||||||
|
second = await _call_submit(
|
||||||
|
c, kind, pm_id, task_id, "second resubmit; still the same unchanged head"
|
||||||
|
)
|
||||||
|
assert second.error == "invalid_state", second.as_dict()
|
||||||
|
assert "unchanged" in (second.message or "").lower()
|
||||||
|
assert "already used" in (second.message or "").lower()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
@pytest.mark.parametrize("kind", ["root", "cell"])
|
||||||
|
async def test_allowed_when_head_advanced(kind: str) -> None:
|
||||||
|
"""A different current head SHA -> allowed (existing fail-open behavior),
|
||||||
|
end to end through the real verb."""
|
||||||
|
c, pm_id, task_id = _resubmit(kind, notes_structured=_unchanged_notes())
|
||||||
|
c.git.get_pr_head_sha = AsyncMock(return_value=SHA_NEW)
|
||||||
|
|
||||||
|
env = await _call_submit(
|
||||||
|
c, kind, pm_id, task_id, "resubmitting after the real fix landed"
|
||||||
|
)
|
||||||
|
|
||||||
|
assert env.error is None, env.as_dict()
|
||||||
|
assert env.status == "awaiting_pr_review"
|
||||||
|
c.task.submit_for_review.assert_awaited_once()
|
||||||
@@ -105,8 +105,15 @@ def _resubmit_root(
|
|||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_submit_root_refuses_unchanged_pr_after_pr_fail() -> None:
|
async def test_submit_root_refuses_unchanged_pr_after_pr_fail() -> None:
|
||||||
"""The loop-stopper: prior pr_fail stamped head SHA X, the PR head is still
|
"""The loop-stopper still holds past the one-shot exemption: prior
|
||||||
X (no new cell work on the root branch) → refuse, do not open the gate."""
|
pr_fail stamped head SHA X, the PR head is still X (no new cell work on
|
||||||
|
the root branch). The findings ledger here fail-opens to "nothing open"
|
||||||
|
(mock session, no real query) — the same signal ``_check_submit_up_gates``
|
||||||
|
upstream already reads for FINDINGS_ADDRESSED — so the first resubmit at
|
||||||
|
this head is the one-shot exemption (see
|
||||||
|
test_resubmit_unchanged_head_exemption.py for full exemption coverage);
|
||||||
|
a second resubmit at the SAME head refuses, so the loop still can't run
|
||||||
|
forever."""
|
||||||
c, main_pm_id, root_task_id = _resubmit_root(
|
c, main_pm_id, root_task_id = _resubmit_root(
|
||||||
notes_structured={
|
notes_structured={
|
||||||
"pr_review": {"verdict": "failed", "head_sha": SHA_OLD, "summary": "..."}
|
"pr_review": {"verdict": "failed", "head_sha": SHA_OLD, "summary": "..."}
|
||||||
@@ -114,9 +121,14 @@ async def test_submit_root_refuses_unchanged_pr_after_pr_fail() -> None:
|
|||||||
)
|
)
|
||||||
c.git.get_pr_head_sha = AsyncMock(return_value=SHA_OLD)
|
c.git.get_pr_head_sha = AsyncMock(return_value=SHA_OLD)
|
||||||
|
|
||||||
env = await c.submit_root(
|
first = await c.submit_root(
|
||||||
main_pm_id, root_task_id, notes="re-submitting the root after the fix"
|
main_pm_id, root_task_id, notes="re-submitting the root after the fix"
|
||||||
)
|
)
|
||||||
|
assert first.error is None, first.as_dict()
|
||||||
|
|
||||||
|
env = await c.submit_root(
|
||||||
|
main_pm_id, root_task_id, notes="re-submitting again; still unchanged"
|
||||||
|
)
|
||||||
|
|
||||||
assert env.error is not None, env.as_dict()
|
assert env.error is not None, env.as_dict()
|
||||||
assert env.error == "invalid_state"
|
assert env.error == "invalid_state"
|
||||||
@@ -124,8 +136,9 @@ async def test_submit_root_refuses_unchanged_pr_after_pr_fail() -> None:
|
|||||||
remediate = env.remediate or ""
|
remediate = env.remediate or ""
|
||||||
assert "re-delegate" in remediate
|
assert "re-delegate" in remediate
|
||||||
assert "submit_root" in remediate
|
assert "submit_root" in remediate
|
||||||
# The PR was NOT re-opened / re-pushed — the runner never ran.
|
# The second attempt's PR was NOT re-opened / re-pushed — the runner ran
|
||||||
c.task.submit_for_review.assert_not_awaited()
|
# only for the first (exempted) call.
|
||||||
|
c.task.submit_for_review.assert_awaited_once()
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
|
|||||||
@@ -80,9 +80,15 @@ def _resubmit_cell(
|
|||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_submit_up_refuses_unchanged_pr_after_pr_fail() -> None:
|
async def test_submit_up_refuses_unchanged_pr_after_pr_fail() -> None:
|
||||||
"""The loop-stopper: prior pr_fail stamped head SHA X, the cell PR head is
|
"""The loop-stopper still holds past the one-shot exemption: prior
|
||||||
still X (no new dev work on the cell branch) → refuse, do not re-open the
|
pr_fail stamped head SHA X, the cell PR head is still X (no new dev work
|
||||||
gate."""
|
on the cell branch). The findings ledger here fail-opens to "nothing
|
||||||
|
open" (mock session, no real query) — the same signal
|
||||||
|
``_check_submit_up_gates`` upstream already reads for FINDINGS_ADDRESSED
|
||||||
|
— so the first resubmit at this head is the one-shot exemption (see
|
||||||
|
test_resubmit_unchanged_head_exemption.py for full exemption coverage);
|
||||||
|
a second resubmit at the SAME head refuses, so the loop still can't run
|
||||||
|
forever."""
|
||||||
c, cell_pm_id, cell_task_id = _resubmit_cell(
|
c, cell_pm_id, cell_task_id = _resubmit_cell(
|
||||||
notes_structured={
|
notes_structured={
|
||||||
"pr_review": {"verdict": "failed", "head_sha": SHA_OLD, "summary": "..."}
|
"pr_review": {"verdict": "failed", "head_sha": SHA_OLD, "summary": "..."}
|
||||||
@@ -90,17 +96,23 @@ async def test_submit_up_refuses_unchanged_pr_after_pr_fail() -> None:
|
|||||||
)
|
)
|
||||||
c.git.get_pr_head_sha = AsyncMock(return_value=SHA_OLD)
|
c.git.get_pr_head_sha = AsyncMock(return_value=SHA_OLD)
|
||||||
|
|
||||||
env = await c.submit_up(
|
first = await c.submit_up(
|
||||||
cell_pm_id, cell_task_id, notes="re-submitting the cell after the fix"
|
cell_pm_id, cell_task_id, notes="re-submitting the cell after the fix"
|
||||||
)
|
)
|
||||||
|
assert first.error is None, first.as_dict()
|
||||||
|
|
||||||
|
env = await c.submit_up(
|
||||||
|
cell_pm_id, cell_task_id, notes="re-submitting again; still unchanged"
|
||||||
|
)
|
||||||
|
|
||||||
assert env.error is not None, env.as_dict()
|
assert env.error is not None, env.as_dict()
|
||||||
assert env.error == "invalid_state", env.as_dict()
|
assert env.error == "invalid_state", env.as_dict()
|
||||||
assert "unchanged" in (env.message or "").lower()
|
assert "unchanged" in (env.message or "").lower()
|
||||||
remediate = env.remediate or ""
|
remediate = env.remediate or ""
|
||||||
assert "submit_up" in remediate
|
assert "submit_up" in remediate
|
||||||
# The cell PR was NOT re-opened / re-pushed — the runner never ran.
|
# The second attempt's PR was NOT re-opened / re-pushed — the runner ran
|
||||||
c.task.submit_for_review.assert_not_awaited()
|
# only for the first (exempted) call.
|
||||||
|
c.task.submit_for_review.assert_awaited_once()
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
|
|||||||
@@ -0,0 +1,151 @@
|
|||||||
|
"""The block/unblock flip-flop breaker.
|
||||||
|
|
||||||
|
Live wedge: fe-pm's escalate_up auto-blocks a task and main_pm's unblock
|
||||||
|
resolves it, repeat — 10 flips, 43 spawns, no forward progress, no cycle
|
||||||
|
breaker. ``unblock`` now stamps a per-task flip counter
|
||||||
|
(``markers.block_flip_count``) and, at exactly the 3rd flip, best-effort
|
||||||
|
alerts the CEO once — the unblock itself always still succeeds.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
from typing import Any
|
||||||
|
from unittest.mock import AsyncMock, MagicMock
|
||||||
|
from uuid import uuid4
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from roboco.foundation.policy.content import markers
|
||||||
|
from roboco.services.gateway.choreographer import Choreographer, ChoreographerDeps
|
||||||
|
from roboco.services.notification import NotificationService
|
||||||
|
|
||||||
|
# Named constants — ruff PLR2004 forbids magic-value comparisons.
|
||||||
|
_TWO_FLIPS = 2
|
||||||
|
_THREE_FLIPS = 3
|
||||||
|
_FOUR_FLIPS = 4
|
||||||
|
|
||||||
|
|
||||||
|
def _make_deps(**overrides: Any) -> ChoreographerDeps:
|
||||||
|
base: dict[str, Any] = {
|
||||||
|
"task": AsyncMock(),
|
||||||
|
"work_session": AsyncMock(),
|
||||||
|
"git": AsyncMock(),
|
||||||
|
"a2a": AsyncMock(),
|
||||||
|
"journal": AsyncMock(),
|
||||||
|
"audit": AsyncMock(),
|
||||||
|
"evidence_repo": AsyncMock(),
|
||||||
|
}
|
||||||
|
base.update(overrides)
|
||||||
|
base["journal"].has_decision_for_task.return_value = True
|
||||||
|
base["journal"].latest_decision_at.return_value = datetime.now(UTC)
|
||||||
|
return ChoreographerDeps(**base)
|
||||||
|
|
||||||
|
|
||||||
|
def _flip_setup() -> tuple[Choreographer, Any, Any, Any]:
|
||||||
|
"""A blocked task whose ``unblock_with_restore`` returns the SAME mock
|
||||||
|
object each call, so the flip-counter marker persists across repeated
|
||||||
|
unblock() calls the way it would on one real ORM row across requests.
|
||||||
|
"""
|
||||||
|
pm_id = uuid4()
|
||||||
|
task_id = uuid4()
|
||||||
|
t = MagicMock(
|
||||||
|
id=task_id,
|
||||||
|
status="blocked",
|
||||||
|
pre_block_state="in_progress",
|
||||||
|
pre_block_assignee=uuid4(),
|
||||||
|
pre_block_metadata={},
|
||||||
|
dependency_ids=[],
|
||||||
|
orchestration_markers=None,
|
||||||
|
)
|
||||||
|
task_svc = AsyncMock()
|
||||||
|
task_svc.get.return_value = t
|
||||||
|
task_svc.unblock_with_restore.return_value = t
|
||||||
|
task_svc.unmet_dependency_ids.return_value = []
|
||||||
|
c = Choreographer(_make_deps(task=task_svc))
|
||||||
|
return c, pm_id, task_id, t
|
||||||
|
|
||||||
|
|
||||||
|
async def _unblock_once(c: Choreographer, pm_id: Any, task_id: Any, t: Any) -> Any:
|
||||||
|
"""Re-block before each call — a fresh flip in the cycle."""
|
||||||
|
t.status = "blocked"
|
||||||
|
return await c.unblock(pm_id, task_id, "resolved upstream; restoring")
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_first_and_second_unblock_do_not_notify() -> None:
|
||||||
|
c, pm_id, task_id, t = _flip_setup()
|
||||||
|
cc: Any = c
|
||||||
|
notify = AsyncMock()
|
||||||
|
cc._notify_ceo_block_flip = notify
|
||||||
|
|
||||||
|
for _ in range(2):
|
||||||
|
env = await _unblock_once(c, pm_id, task_id, t)
|
||||||
|
assert env.error is None, env.as_dict()
|
||||||
|
|
||||||
|
notify.assert_not_awaited()
|
||||||
|
assert markers.get_block_flip_count(t) == _TWO_FLIPS
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_third_unblock_notifies_ceo_once() -> None:
|
||||||
|
c, pm_id, task_id, t = _flip_setup()
|
||||||
|
cc: Any = c
|
||||||
|
notify = AsyncMock()
|
||||||
|
cc._notify_ceo_block_flip = notify
|
||||||
|
|
||||||
|
for _ in range(3):
|
||||||
|
env = await _unblock_once(c, pm_id, task_id, t)
|
||||||
|
assert env.error is None, env.as_dict()
|
||||||
|
|
||||||
|
notify.assert_awaited_once_with(task_id, _THREE_FLIPS)
|
||||||
|
assert markers.is_block_flip_notified(t) is True
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_fourth_unblock_does_not_renotify() -> None:
|
||||||
|
c, pm_id, task_id, t = _flip_setup()
|
||||||
|
cc: Any = c
|
||||||
|
notify = AsyncMock()
|
||||||
|
cc._notify_ceo_block_flip = notify
|
||||||
|
|
||||||
|
for _ in range(4):
|
||||||
|
env = await _unblock_once(c, pm_id, task_id, t)
|
||||||
|
assert env.error is None, env.as_dict()
|
||||||
|
|
||||||
|
notify.assert_awaited_once()
|
||||||
|
assert markers.get_block_flip_count(t) == _FOUR_FLIPS
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_notification_failure_does_not_fail_unblock(
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
) -> None:
|
||||||
|
"""The real ``_notify_ceo_block_flip`` swallows a notify-service failure —
|
||||||
|
unblock still succeeds on the 3rd flip."""
|
||||||
|
c, pm_id, task_id, t = _flip_setup()
|
||||||
|
monkeypatch.setattr(
|
||||||
|
NotificationService,
|
||||||
|
"send_block_flip_notification",
|
||||||
|
AsyncMock(side_effect=RuntimeError("notification service down")),
|
||||||
|
)
|
||||||
|
|
||||||
|
env = None
|
||||||
|
for _ in range(3):
|
||||||
|
env = await _unblock_once(c, pm_id, task_id, t)
|
||||||
|
assert env.error is None, env.as_dict()
|
||||||
|
|
||||||
|
assert env is not None
|
||||||
|
assert env.error is None
|
||||||
|
assert markers.is_block_flip_notified(t) is True
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_counter_persists_via_marker_across_calls() -> None:
|
||||||
|
c, pm_id, task_id, t = _flip_setup()
|
||||||
|
cc: Any = c
|
||||||
|
cc._notify_ceo_block_flip = AsyncMock()
|
||||||
|
|
||||||
|
await _unblock_once(c, pm_id, task_id, t)
|
||||||
|
assert markers.get_block_flip_count(t) == 1
|
||||||
|
await _unblock_once(c, pm_id, task_id, t)
|
||||||
|
assert markers.get_block_flip_count(t) == _TWO_FLIPS
|
||||||
@@ -36,6 +36,9 @@ ONE = 1
|
|||||||
TWO = 2
|
TWO = 2
|
||||||
THREE = 3
|
THREE = 3
|
||||||
FOUR = 4
|
FOUR = 4
|
||||||
|
FIVE = 5
|
||||||
|
SEVEN = 7
|
||||||
|
_AC_ITEM_CHAR_CAP = 200 # mirrors task_completeness._AC_MAX_ITEM_CHARS
|
||||||
|
|
||||||
|
|
||||||
async def _seed(session: AsyncSession) -> None:
|
async def _seed(session: AsyncSession) -> None:
|
||||||
@@ -896,3 +899,152 @@ async def test_reauthor_from_rejection_missing_draft_returns_none(
|
|||||||
|
|
||||||
result = await engine.reauthor_from_rejection(task, "some reason")
|
result = await engine.reauthor_from_rejection(task, "some reason")
|
||||||
assert result is None
|
assert result is None
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# scene criterion — a brief-named feature list becomes its own gate-checkable
|
||||||
|
# AC (delegation detail-fidelity: prose feature counts used to pass gates
|
||||||
|
# even when a dev shipped fewer scenes than the brief named)
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_draft_release_video_carries_scene_criterion_for_highlights(
|
||||||
|
db_session: AsyncSession, monkeypatch: pytest.MonkeyPatch
|
||||||
|
) -> None:
|
||||||
|
await _seed(db_session)
|
||||||
|
_enable(monkeypatch, video_on_release=True)
|
||||||
|
_mock_local_model(monkeypatch, "shipped!")
|
||||||
|
engine = video_engine_module.VideoEngine(db_session)
|
||||||
|
task = await engine.draft_release_video(version="1.0.0", changelog=_CHANGELOG)
|
||||||
|
assert task is not None
|
||||||
|
assert len(task.acceptance_criteria) == FIVE
|
||||||
|
scene_ac = task.acceptance_criteria[FOUR]
|
||||||
|
assert scene_ac.startswith("Every brief-named feature appears as its own")
|
||||||
|
assert "a huge new release" in scene_ac
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_open_video_task_no_highlights_keeps_ac_count_unchanged(
|
||||||
|
db_session: AsyncSession, monkeypatch: pytest.MonkeyPatch
|
||||||
|
) -> None:
|
||||||
|
await _seed(db_session)
|
||||||
|
_enable(monkeypatch)
|
||||||
|
engine = video_engine_module.VideoEngine(db_session)
|
||||||
|
task = await engine.open_video_task(
|
||||||
|
occasion="spotlight no-highlights",
|
||||||
|
script="s",
|
||||||
|
platforms=["x"],
|
||||||
|
brief="b",
|
||||||
|
suggested_input_props={"version": "1.0.0"}, # no "highlights" key
|
||||||
|
)
|
||||||
|
assert task is not None
|
||||||
|
assert len(task.acceptance_criteria) == FOUR # no scene criterion appended
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_reauthor_from_rejection_with_highlights_regenerates_scene_criterion(
|
||||||
|
db_session: AsyncSession, monkeypatch: pytest.MonkeyPatch
|
||||||
|
) -> None:
|
||||||
|
await _seed(db_session)
|
||||||
|
_enable(monkeypatch)
|
||||||
|
engine = video_engine_module.VideoEngine(db_session)
|
||||||
|
source_task = await engine.open_video_task(
|
||||||
|
occasion="release v3.0.0",
|
||||||
|
script="s",
|
||||||
|
platforms=["x"],
|
||||||
|
brief="b",
|
||||||
|
suggested_input_props={
|
||||||
|
"version": "3.0.0",
|
||||||
|
"highlights": ["Feature A", "Feature B"],
|
||||||
|
},
|
||||||
|
)
|
||||||
|
assert source_task is not None
|
||||||
|
draft = markers.get_video_draft(source_task) or {}
|
||||||
|
# Simulate propose_video: the dev's real input_props carries the same
|
||||||
|
# highlights forward onto the marker.
|
||||||
|
markers.set_video_draft(
|
||||||
|
source_task,
|
||||||
|
{
|
||||||
|
**draft,
|
||||||
|
"composition_id": "ReleaseIntro",
|
||||||
|
"input_props": {
|
||||||
|
"version": "3.0.0",
|
||||||
|
"highlights": ["Feature A", "Feature B"],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
source_task.status = TS.COMPLETED
|
||||||
|
await db_session.flush()
|
||||||
|
|
||||||
|
post_task = await engine._originate_video_post(
|
||||||
|
source_task=source_task,
|
||||||
|
mp4_paths={"vertical": "a.mp4", "square": "b.mp4"},
|
||||||
|
captions={"x": "cap"},
|
||||||
|
platforms=["x"],
|
||||||
|
)
|
||||||
|
post_task.status = TS.CANCELLED
|
||||||
|
await db_session.flush()
|
||||||
|
|
||||||
|
revision = await engine.reauthor_from_rejection(post_task, "Logo too small")
|
||||||
|
assert revision is not None
|
||||||
|
assert len(revision.acceptance_criteria) == FIVE
|
||||||
|
scene_ac = revision.acceptance_criteria[FOUR]
|
||||||
|
assert "Feature A" in scene_ac
|
||||||
|
assert "Feature B" in scene_ac
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_reauthor_from_rejection_without_highlights_adds_feedback_criterion(
|
||||||
|
db_session: AsyncSession, monkeypatch: pytest.MonkeyPatch
|
||||||
|
) -> None:
|
||||||
|
await _seed(db_session)
|
||||||
|
_enable(monkeypatch)
|
||||||
|
engine = video_engine_module.VideoEngine(db_session)
|
||||||
|
source_task = await engine.open_video_task(
|
||||||
|
occasion="release v4.0.0", script="s", platforms=["x"], brief="b"
|
||||||
|
)
|
||||||
|
assert source_task is not None
|
||||||
|
draft = markers.get_video_draft(source_task) or {}
|
||||||
|
markers.set_video_draft(source_task, {**draft, "composition_id": "Intro"})
|
||||||
|
source_task.status = TS.COMPLETED
|
||||||
|
await db_session.flush()
|
||||||
|
|
||||||
|
post_task = await engine._originate_video_post(
|
||||||
|
source_task=source_task,
|
||||||
|
mp4_paths={"vertical": "a.mp4", "square": "b.mp4"},
|
||||||
|
captions={"x": "cap"},
|
||||||
|
platforms=["x"],
|
||||||
|
)
|
||||||
|
post_task.status = TS.CANCELLED
|
||||||
|
await db_session.flush()
|
||||||
|
|
||||||
|
revision = await engine.reauthor_from_rejection(post_task, "Logo is cut off")
|
||||||
|
assert revision is not None
|
||||||
|
assert len(revision.acceptance_criteria) == FIVE
|
||||||
|
assert revision.acceptance_criteria[FOUR] == (
|
||||||
|
"Every point in the CEO rejection feedback is visibly addressed in "
|
||||||
|
"the rendered cut"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_open_video_task_scene_criterion_truncates_pathological_feature_list(
|
||||||
|
db_session: AsyncSession, monkeypatch: pytest.MonkeyPatch
|
||||||
|
) -> None:
|
||||||
|
await _seed(db_session)
|
||||||
|
_enable(monkeypatch)
|
||||||
|
engine = video_engine_module.VideoEngine(db_session)
|
||||||
|
features = [f"Feature number {i}" for i in range(50)]
|
||||||
|
task = await engine.open_video_task(
|
||||||
|
occasion="release pathological",
|
||||||
|
script="s",
|
||||||
|
platforms=["x"],
|
||||||
|
brief="b",
|
||||||
|
suggested_input_props={"highlights": features},
|
||||||
|
)
|
||||||
|
assert task is not None
|
||||||
|
assert len(task.acceptance_criteria) <= SEVEN
|
||||||
|
scene_ac = task.acceptance_criteria[-1]
|
||||||
|
assert len(scene_ac) <= _AC_ITEM_CHAR_CAP
|
||||||
|
assert "more)" in scene_ac # truncated, not silently dropped
|
||||||
|
|||||||
Reference in New Issue
Block a user