Files
roboco/tests/unit/gateway/test_video_context_in_qa_evidence.py
aa15dc40cc Feature/video artifact verification (#537)
* fix(release): CI wait polls the prod rung; escape the header tooltip apostrophe

get_latest_ci_conclusion defaults to the ladder's head rung, so
wait_for_ci searched slave for a release commit that lives on master
and timed out after 40 minutes with the run already green. The wait
now passes the prod branch explicitly. Also fixes the
react/no-unescaped-entities error that turned master's Panel CI red.

* fix(panel,video): dead dialog triggers behind tooltips; dotted composition ids render

HelpTip nested inside a Dialog/AlertDialog trigger puts the trigger's
click handler on the Tooltip root, which renders no DOM — the agents
Spawn item and the KB Reindex-All / Delete-index confirms were dead.
Tooltips now wrap the triggers. The video renderer accepts interior
single dots in composition ids (release-0.25.0) with '..' still
unrepresentable, and propose_video refuses an unrenderable id at
authoring time.

* fix(dispatch): restart-safe PM review turns

A leaf task in awaiting_pm_review had no periodic pickup: the closure
dispatcher bailed on childless tasks and skipped PR-bearing review
tasks as already-promoted, assuming the submit-time PM session was
still alive — an assumption every restart breaks. Proven live on the
docs-sync leaf after the 0.25.0 redeploy, which also dependency-blocked
its sibling dev task. Childless awaiting_pm_review tasks now flow to
the PM's review turn, and the merge turn respawns its PM when none is
active.

* feat(video): verify the rendered artifact, not the source

The 14s release-0.25.0 cut shipped with only one of four scenes visibly
registering: the dev authored DOM, the smoke asserted DOM, QA read code —
nobody consumed the rendered MP4 before the CEO did. Close that loop, and
the reject loop behind it:

- sidecar frames mode: POST /render with frames=1..32 renders the cut,
  ffprobes the REAL duration, extracts midpoint-sampled keyframe PNGs
  (timestamps in filenames), streams a tar.gz back with X-Video-Duration
- request_render do-verb (developer/QA, request_sandbox's shape): renders
  the caller's ACTUAL composition — dev's own worktree (head_sha/dirty
  provenance), QA a read-only git-archive export of the assembled branch —
  extracts frames to the container-shared .previews/ path, stamps the
  render_preview marker, returns the paths as envelope evidence
- gate: i_am_done on a source=video task refuses without a stamped
  render_preview (Requirement.RENDER_VERIFIED; canonical source string
  moved to foundation as markers.VIDEO_TASK_SOURCE; mirrored in the
  possibilities-matrix fast path so it cannot bypass the check)
- QA claim_review evidence carries video_context (composition id, the
  dev's preview, a re-render instruction) so review checks output
- dev spawn prompt block + a 4th authoring AC order Read-every-frame
  verification before submitting
- reject -> re-author: a CEO reject with a reason opens a fresh authoring
  task carrying the verbatim feedback + a revise-in-place pointer at the
  existing composition (best-effort, never fails the reject) — rejection
  feedback no longer dies on the cancelled draft

E2E: rendered the committed release-0.25.0 composition through the new
frames mode locally — the returned keyframes show exactly the reported
failure (blank frame at 5.8s, only 'Env ladder' by 12.8s), the check the
fleet was missing.

---------

Co-authored-by: Renn F <rennf93@users.noreply.github.com>
2026-07-16 19:49:26 +02:00

68 lines
2.3 KiB
Python

"""QA claim_review evidence carries a video-artifact context for
video-source tasks — pointing QA at the rendered artifact, not just source."""
from __future__ import annotations
from unittest.mock import MagicMock
from roboco.foundation.policy.content import markers
from roboco.services.gateway.choreographer import Choreographer
from roboco.services.gateway.evidence_builder import build_evidence_for_task
def _stub_task() -> MagicMock:
task = MagicMock()
task.pr_number = None
task.pr_url = None
task.commits = []
task.dev_notes = None
task.acceptance_criteria_status = []
task.source = "code"
task.orchestration_markers = None
return task
def test_video_context_none_for_non_video_task() -> None:
assert Choreographer._qa_video_context(_stub_task()) is None
def test_video_context_present_for_video_task_with_render_preview() -> None:
task = _stub_task()
task.source = markers.VIDEO_TASK_SOURCE
task.orchestration_markers = {
markers.VIDEO_DRAFT: {"composition_id": "intro-v1"},
markers.RENDER_PREVIEW: {"frames": ["a.png", "b.png"]},
}
ctx = Choreographer._qa_video_context(task)
assert ctx is not None
assert ctx["composition_id"] == "intro-v1"
assert ctx["render_preview"] == {"frames": ["a.png", "b.png"]}
assert "request_render" in ctx["note"]
def test_video_context_render_preview_none_without_marker() -> None:
task = _stub_task()
task.source = markers.VIDEO_TASK_SOURCE
task.orchestration_markers = {markers.VIDEO_DRAFT: {"composition_id": "intro-v1"}}
ctx = Choreographer._qa_video_context(task)
assert ctx is not None
assert ctx["composition_id"] == "intro-v1"
assert ctx["render_preview"] is None
def test_evidence_payload_includes_video_context() -> None:
video_context = {"composition_id": "x", "render_preview": None, "note": "n"}
ev = build_evidence_for_task(
_stub_task(),
journal_highlights=[],
files_changed=[],
video_context=video_context,
)
assert ev.as_dict()["video_context"] == video_context
def test_evidence_payload_video_context_default_absent() -> None:
ev = build_evidence_for_task(_stub_task(), journal_highlights=[], files_changed=[])
assert ev.video_context is None
assert "video_context" not in ev.as_dict()