Feature/video artifact verification (#537)

* fix(release): CI wait polls the prod rung; escape the header tooltip apostrophe

get_latest_ci_conclusion defaults to the ladder's head rung, so
wait_for_ci searched slave for a release commit that lives on master
and timed out after 40 minutes with the run already green. The wait
now passes the prod branch explicitly. Also fixes the
react/no-unescaped-entities error that turned master's Panel CI red.

* fix(panel,video): dead dialog triggers behind tooltips; dotted composition ids render

HelpTip nested inside a Dialog/AlertDialog trigger puts the trigger's
click handler on the Tooltip root, which renders no DOM — the agents
Spawn item and the KB Reindex-All / Delete-index confirms were dead.
Tooltips now wrap the triggers. The video renderer accepts interior
single dots in composition ids (release-0.25.0) with '..' still
unrepresentable, and propose_video refuses an unrenderable id at
authoring time.

* fix(dispatch): restart-safe PM review turns

A leaf task in awaiting_pm_review had no periodic pickup: the closure
dispatcher bailed on childless tasks and skipped PR-bearing review
tasks as already-promoted, assuming the submit-time PM session was
still alive — an assumption every restart breaks. Proven live on the
docs-sync leaf after the 0.25.0 redeploy, which also dependency-blocked
its sibling dev task. Childless awaiting_pm_review tasks now flow to
the PM's review turn, and the merge turn respawns its PM when none is
active.

* feat(video): verify the rendered artifact, not the source

The 14s release-0.25.0 cut shipped with only one of four scenes visibly
registering: the dev authored DOM, the smoke asserted DOM, QA read code —
nobody consumed the rendered MP4 before the CEO did. Close that loop, and
the reject loop behind it:

- sidecar frames mode: POST /render with frames=1..32 renders the cut,
  ffprobes the REAL duration, extracts midpoint-sampled keyframe PNGs
  (timestamps in filenames), streams a tar.gz back with X-Video-Duration
- request_render do-verb (developer/QA, request_sandbox's shape): renders
  the caller's ACTUAL composition — dev's own worktree (head_sha/dirty
  provenance), QA a read-only git-archive export of the assembled branch —
  extracts frames to the container-shared .previews/ path, stamps the
  render_preview marker, returns the paths as envelope evidence
- gate: i_am_done on a source=video task refuses without a stamped
  render_preview (Requirement.RENDER_VERIFIED; canonical source string
  moved to foundation as markers.VIDEO_TASK_SOURCE; mirrored in the
  possibilities-matrix fast path so it cannot bypass the check)
- QA claim_review evidence carries video_context (composition id, the
  dev's preview, a re-render instruction) so review checks output
- dev spawn prompt block + a 4th authoring AC order Read-every-frame
  verification before submitting
- reject -> re-author: a CEO reject with a reason opens a fresh authoring
  task carrying the verbatim feedback + a revise-in-place pointer at the
  existing composition (best-effort, never fails the reject) — rejection
  feedback no longer dies on the cancelled draft

E2E: rendered the committed release-0.25.0 composition through the new
frames mode locally — the returned keyframes show exactly the reported
failure (blank frame at 5.8s, only 'Env ladder' by 12.8s), the check the
fleet was missing.

---------

Co-authored-by: Renn F <rennf93@users.noreply.github.com>
This commit is contained in:
Renzo F
2026-07-16 19:49:26 +02:00
committed by GitHub
co-authored by Renn F
parent 797847e379
commit aa15dc40cc
37 changed files with 2146 additions and 54 deletions
+1
View File
@@ -35,6 +35,7 @@ def test_requirement_enum_has_canonical_values() -> None:
"pr_reviewer_notes>=min",
"quick_context>=min",
"findings_addressed",
"render_verified",
}
actual = {r.value for r in tracing.Requirement}
assert actual == expected, f"Requirement drift: {actual ^ expected}"
@@ -0,0 +1,58 @@
"""The RENDER_VERIFIED tracing requirement — i_am_done's video-artifact gate.
Pure unit tests against ``foundation.policy.tracing`` (no DB, no
choreographer): the checker itself, its registration in
``VERB_REQUIREMENTS["i_am_done"]``, and the "non-video task is untouched"
contract.
"""
from __future__ import annotations
from types import SimpleNamespace
from roboco.foundation.policy import tracing as tr
from roboco.foundation.policy.content import markers
def test_render_verified_is_required_by_i_am_done() -> None:
assert tr.Requirement.RENDER_VERIFIED in tr.VERB_REQUIREMENTS["i_am_done"]
def test_render_verified_not_required_by_submit_up_or_submit_root() -> None:
assert tr.Requirement.RENDER_VERIFIED not in tr.VERB_REQUIREMENTS["submit_up"]
assert tr.Requirement.RENDER_VERIFIED not in tr.VERB_REQUIREMENTS["submit_root"]
def test_non_video_task_passes_regardless_of_marker() -> None:
task = SimpleNamespace(source="code", orchestration_markers=None)
assert tr._check_render_verified(task, tr.GateContext()) == []
def test_video_task_without_preview_fails() -> None:
task = SimpleNamespace(source=markers.VIDEO_TASK_SOURCE, orchestration_markers=None)
assert tr._check_render_verified(task, tr.GateContext()) == ["render_preview"]
def test_video_task_with_preview_passes() -> None:
task = SimpleNamespace(
source=markers.VIDEO_TASK_SOURCE,
orchestration_markers={markers.RENDER_PREVIEW: {"frames": ["a.png"]}},
)
assert tr._check_render_verified(task, tr.GateContext()) == []
def test_i_am_done_requirements_include_the_pre_existing_set_too() -> None:
"""Adding RENDER_VERIFIED must not have dropped any prior requirement."""
required = tr.VERB_REQUIREMENTS["i_am_done"]
for expected in (
tr.Requirement.COMMITS_AT_LEAST_ONE,
tr.Requirement.PR_OPEN,
tr.Requirement.PROGRESS_AT_LEAST_ONE,
tr.Requirement.SELF_VERIFIED,
tr.Requirement.JOURNAL_REFLECT,
tr.Requirement.JOURNAL_DURING_WORK_AT_LEAST_ONE,
tr.Requirement.ACCEPTANCE_CRITERIA_ADDRESSED,
tr.Requirement.DEV_NOTES_MIN_CHARS,
tr.Requirement.FINDINGS_ADDRESSED,
):
assert expected in required
@@ -16,6 +16,7 @@ from unittest.mock import AsyncMock, MagicMock
from uuid import uuid4
import pytest
from roboco.foundation.policy.content import markers
from roboco.services.gateway.choreographer import Choreographer, ChoreographerDeps
from roboco.services.gateway.envelope import Envelope
@@ -61,6 +62,8 @@ def _t(
t.pr_number = 12345 if pr_created else None
t.acceptance_criteria = list(criteria)
t.acceptance_criteria_status = ac_status
t.source = "code"
t.orchestration_markers = None
return t
@@ -334,6 +337,44 @@ async def test_fast_path_open_findings_blocks(monkeypatch: pytest.MonkeyPatch) -
c.task.submit_qa.assert_not_awaited()
@pytest.mark.asyncio
async def test_fast_path_video_task_without_render_preview_rejects(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""A video-authoring task with no request_render preview must not be
able to fast-path around looking at the rendered artifact."""
c = Choreographer(_deps())
stubs = _stub_fast_path(c, monkeypatch)
tracing_gap = AsyncMock(
return_value=Envelope.tracing_gap(missing=["render_preview"], remediate="x")
)
monkeypatch.setattr(c, "_build_tracing_gap", tracing_gap)
ctx = _ctx()
ctx.task = _t()
ctx.task.source = markers.VIDEO_TASK_SOURCE
await c._i_am_done_fast_path(ctx)
stubs.reject.assert_awaited_once()
c.task.submit_qa.assert_not_awaited()
tracing_gap.assert_awaited_once_with(
ctx.agent_id, ctx.task_id, ["render_preview"], task=ctx.task
)
@pytest.mark.asyncio
async def test_fast_path_video_task_with_render_preview_passes(
monkeypatch: pytest.MonkeyPatch,
) -> None:
c = Choreographer(_deps())
stubs = _stub_fast_path(c, monkeypatch)
ctx = _ctx()
ctx.task = _t()
ctx.task.source = markers.VIDEO_TASK_SOURCE
ctx.task.orchestration_markers = {markers.RENDER_PREVIEW: {"frames": ["a.png"]}}
await c._i_am_done_fast_path(ctx)
stubs.ok.assert_awaited_once()
c.task.submit_qa.assert_awaited_once()
@pytest.mark.asyncio
async def test_fast_path_conventions_block_rejects(
monkeypatch: pytest.MonkeyPatch,
@@ -0,0 +1,485 @@
"""ContentActions.request_render — render a video composition to preview
frames so an agent verifies the RENDERED artifact, not just its source.
Guard matrix (flag off / no active task / non-video source / renderer
unconfigured / missing composition_id / bad composition_id-orientation-
frame_count / missing composition dir on disk / role gate), the dev and QA
success paths (frame extraction + marker payload shape), and a
VideoRendererError surfacing as a retryable rejection.
"""
from __future__ import annotations
import io
import tarfile
from pathlib import Path
from unittest.mock import AsyncMock, MagicMock
from uuid import uuid4
import pytest
from roboco.config import settings
from roboco.foundation.policy.content import markers
from roboco.services.gateway.content_actions import ContentActions, ContentActionsDeps
from roboco.services.video_renderer_client import VideoRendererError
def _make_actions(
*,
task_obj: MagicMock | None,
role: str = "developer",
team: str | None = "ux_ui",
workspace: MagicMock | None = None,
) -> tuple[ContentActions, MagicMock]:
task = AsyncMock()
task.get_active_task_for_agent.return_value = task_obj
task.session = MagicMock()
task.session.flush = AsyncMock()
task.heartbeat = AsyncMock()
agent = MagicMock()
agent.role = role
agent.team = team
task.agent_for = AsyncMock(return_value=agent)
deps = ContentActionsDeps(
task=task,
git=MagicMock(),
a2a=MagicMock(),
journal=MagicMock(),
workspace=workspace or MagicMock(),
notifications=MagicMock(),
)
return ContentActions(deps), task
def _task(
*,
project_id: object | None = uuid4(),
source: str = markers.VIDEO_TASK_SOURCE,
branch_name: str | None = "feature/ux_ui/ABCD1234",
draft: dict[str, object] | None = None,
) -> MagicMock:
t = MagicMock()
t.id = uuid4()
t.project_id = project_id
t.status = "in_progress"
t.source = source
t.branch_name = branch_name
t.orchestration_markers = {"video_draft": draft} if draft else None
return t
def _stub_project(
monkeypatch: pytest.MonkeyPatch, *, slug: str = "demo-project"
) -> MagicMock:
project = MagicMock(slug=slug, git_url="https://example.invalid/demo.git")
project_service = MagicMock()
project_service.get = AsyncMock(return_value=project)
monkeypatch.setattr(
"roboco.services.project.get_project_service", lambda _s: project_service
)
return project
def _stub_renderer(
monkeypatch: pytest.MonkeyPatch,
*,
frames_tar_gz: bytes = b"",
duration: float = 2.5,
error: Exception | None = None,
) -> MagicMock:
renderer = MagicMock()
if error is not None:
renderer.render_frames = AsyncMock(side_effect=error)
else:
renderer.render_frames = AsyncMock(return_value=(frames_tar_gz, duration))
monkeypatch.setattr(
"roboco.services.video_renderer_client.get_video_renderer",
lambda: renderer,
)
return renderer
def _make_frames_tar(names: list[str]) -> bytes:
"""An in-memory tar.gz of a few frame files, matching what render_frames
returns on success — the render loop extracts this straight to disk."""
buf = io.BytesIO()
with tarfile.open(fileobj=buf, mode="w:gz") as tar:
for name in names:
data = f"fake-png-bytes-{name}".encode()
info = tarfile.TarInfo(name=name)
info.size = len(data)
tar.addfile(info, io.BytesIO(data))
return buf.getvalue()
def _arm(
monkeypatch: pytest.MonkeyPatch, *, renderer_url: str = "http://sidecar:3001"
) -> None:
monkeypatch.setattr(settings, "video_engine_enabled", True)
monkeypatch.setattr(settings, "video_renderer_base_url", renderer_url)
# --------------------------------------------------------------------------- #
# guard matrix
# --------------------------------------------------------------------------- #
@pytest.mark.asyncio
async def test_flag_off_refuses_before_task_lookup(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.setattr(settings, "video_engine_enabled", False)
actions, task_svc = _make_actions(task_obj=None)
env = await actions.request_render(agent_id=uuid4())
assert env.error == "invalid_state"
task_svc.get_active_task_for_agent.assert_not_awaited()
@pytest.mark.asyncio
async def test_no_active_task_refused(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr(settings, "video_engine_enabled", True)
actions, _task_svc = _make_actions(task_obj=None)
env = await actions.request_render(agent_id=uuid4())
assert env.error == "invalid_state"
@pytest.mark.asyncio
async def test_non_video_source_refused(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr(settings, "video_engine_enabled", True)
t = _task(source="chore")
actions, _task_svc = _make_actions(task_obj=t)
env = await actions.request_render(agent_id=uuid4())
assert env.error == "invalid_state"
assert "video-authoring" in (env.message or "") + (env.remediate or "")
@pytest.mark.asyncio
async def test_renderer_unconfigured_refused(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr(settings, "video_engine_enabled", True)
monkeypatch.setattr(settings, "video_renderer_base_url", "")
t = _task()
actions, _task_svc = _make_actions(task_obj=t)
env = await actions.request_render(agent_id=uuid4())
assert env.error == "invalid_state"
assert "ROBOCO_VIDEO_RENDERER_BASE_URL" in (env.remediate or "")
@pytest.mark.asyncio
async def test_missing_composition_id_is_incomplete_input(
monkeypatch: pytest.MonkeyPatch,
) -> None:
_arm(monkeypatch)
t = _task(draft=None)
actions, _task_svc = _make_actions(task_obj=t)
env = await actions.request_render(agent_id=uuid4())
assert env.error == "incomplete_input"
assert "composition_id" in (env.missing or [])
@pytest.mark.asyncio
async def test_bad_composition_id_regex_rejected(
monkeypatch: pytest.MonkeyPatch,
) -> None:
_arm(monkeypatch)
t = _task()
actions, _task_svc = _make_actions(task_obj=t)
env = await actions.request_render(agent_id=uuid4(), composition_id="bad id!")
assert env.error == "invalid_state"
assert "not renderable" in (env.message or "")
@pytest.mark.asyncio
async def test_bad_orientation_rejected(monkeypatch: pytest.MonkeyPatch) -> None:
_arm(monkeypatch)
t = _task()
actions, _task_svc = _make_actions(task_obj=t)
env = await actions.request_render(
agent_id=uuid4(), composition_id="release-v1", orientation="landscape"
)
assert env.error == "invalid_state"
@pytest.mark.asyncio
async def test_frame_count_out_of_range_rejected(
monkeypatch: pytest.MonkeyPatch,
) -> None:
_arm(monkeypatch)
t = _task()
actions, _task_svc = _make_actions(task_obj=t)
too_low = await actions.request_render(
agent_id=uuid4(), composition_id="release-v1", frame_count=0
)
too_high = await actions.request_render(
agent_id=uuid4(), composition_id="release-v1", frame_count=33
)
assert too_low.error == "invalid_state"
assert too_high.error == "invalid_state"
@pytest.mark.asyncio
async def test_missing_composition_dir_on_disk_rejected(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_arm(monkeypatch)
monkeypatch.setattr(settings, "workspaces_root", str(tmp_path / "workspaces"))
_stub_project(monkeypatch)
clone_root = tmp_path / "clone"
clone_root.mkdir(parents=True) # no motion/ tree at all
workspace = MagicMock()
workspace.get_clone_root_path.return_value = clone_root
workspace.get_worktree_path.return_value = clone_root / ".worktrees" / "x"
t = _task(draft={"composition_id": "release-v1"})
actions, _task_svc = _make_actions(task_obj=t, workspace=workspace)
env = await actions.request_render(agent_id=uuid4())
assert env.error == "invalid_state"
assert "motion/compositions/release-v1" in (env.message or "")
@pytest.mark.asyncio
async def test_other_role_not_authorized(monkeypatch: pytest.MonkeyPatch) -> None:
_arm(monkeypatch)
_stub_project(monkeypatch)
t = _task(draft={"composition_id": "release-v1"})
actions, _task_svc = _make_actions(task_obj=t, role="documenter", team="ux_ui")
env = await actions.request_render(agent_id=uuid4())
assert env.error == "not_authorized"
# --------------------------------------------------------------------------- #
# dev happy path — own working tree
# --------------------------------------------------------------------------- #
@pytest.mark.asyncio
async def test_dev_happy_path_extracts_frames_and_stamps_marker(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_arm(monkeypatch)
monkeypatch.setattr(settings, "workspaces_root", str(tmp_path / "workspaces"))
_stub_project(monkeypatch, slug="demo-project")
clone_root = tmp_path / "clone"
(clone_root / "motion" / "compositions" / "release-v1").mkdir(parents=True)
workspace = MagicMock()
workspace.get_clone_root_path.return_value = clone_root
workspace.get_worktree_path.return_value = clone_root / ".worktrees" / "deadbeef"
t = _task(draft={"composition_id": "release-v1"})
actions, task_svc = _make_actions(
task_obj=t, role="developer", team="ux_ui", workspace=workspace
)
expected_frame_count = 2
expected_duration = 3.2
default_frame_count = 8
tar_bytes = _make_frames_tar(["frame-0.png", "frame-1.png"])
_stub_renderer(monkeypatch, frames_tar_gz=tar_bytes, duration=expected_duration)
env = await actions.request_render(agent_id=uuid4())
assert env.error is None
assert env.status == t.status
assert env.task_id == str(t.id)
assert env.evidence is not None
frames = env.evidence["frames"]
assert len(frames) == expected_frame_count
for p in frames:
assert Path(p).is_file()
assert env.evidence["duration_seconds"] == expected_duration
assert env.evidence["source"] == "workspace"
assert env.evidence["dirty"] is False
assert env.evidence["rendered_by"]
assert "note" in env.evidence
assert "frames[]" in (env.next or "")
payload = markers.get_render_preview(t)
assert payload is not None
assert payload["composition_id"] == "release-v1"
assert payload["orientation"] == "vertical"
assert payload["frame_count"] == default_frame_count
assert payload["duration_seconds"] == expected_duration
assert payload["source"] == "workspace"
assert payload["dirty"] is False
assert payload["frames"] == frames
assert "at" in payload
task_svc.session.flush.assert_awaited()
task_svc.heartbeat.assert_awaited_once()
@pytest.mark.asyncio
async def test_explicit_composition_id_backfills_video_draft(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""A dev who never called propose_video must still leave
video_draft.composition_id stamped — the post-completion render loop
keys on it and skips silently when absent (proven live)."""
_arm(monkeypatch)
monkeypatch.setattr(settings, "workspaces_root", str(tmp_path / "workspaces"))
_stub_project(monkeypatch, slug="demo-project")
clone_root = tmp_path / "clone"
(clone_root / "motion" / "compositions" / "release-v1").mkdir(parents=True)
workspace = MagicMock()
workspace.get_clone_root_path.return_value = clone_root
workspace.get_worktree_path.return_value = clone_root / ".worktrees" / "deadbeef"
t = _task(draft=None)
actions, _ = _make_actions(
task_obj=t, role="developer", team="ux_ui", workspace=workspace
)
_stub_renderer(
monkeypatch, frames_tar_gz=_make_frames_tar(["frame-0.png"]), duration=1.0
)
env = await actions.request_render(agent_id=uuid4(), composition_id="release-v1")
assert env.error is None
draft = markers.get_video_draft(t)
assert draft is not None
assert draft["composition_id"] == "release-v1"
@pytest.mark.asyncio
async def test_existing_video_draft_composition_id_not_overwritten(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_arm(monkeypatch)
monkeypatch.setattr(settings, "workspaces_root", str(tmp_path / "workspaces"))
_stub_project(monkeypatch, slug="demo-project")
clone_root = tmp_path / "clone"
for comp in ("release-v1", "release-v2"):
(clone_root / "motion" / "compositions" / comp).mkdir(parents=True)
workspace = MagicMock()
workspace.get_clone_root_path.return_value = clone_root
workspace.get_worktree_path.return_value = clone_root / ".worktrees" / "deadbeef"
t = _task(draft={"composition_id": "release-v1", "occasion": "r1"})
actions, _ = _make_actions(
task_obj=t, role="developer", team="ux_ui", workspace=workspace
)
_stub_renderer(
monkeypatch, frames_tar_gz=_make_frames_tar(["frame-0.png"]), duration=1.0
)
env = await actions.request_render(agent_id=uuid4(), composition_id="release-v2")
assert env.error is None
draft = markers.get_video_draft(t)
assert draft is not None
assert draft["composition_id"] == "release-v1"
assert draft["occasion"] == "r1"
# --------------------------------------------------------------------------- #
# QA happy path — read-only branch export, never a working tree
# --------------------------------------------------------------------------- #
@pytest.mark.asyncio
async def test_qa_happy_path_uses_branch_export(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_arm(monkeypatch)
monkeypatch.setattr(settings, "workspaces_root", str(tmp_path / "workspaces"))
_stub_project(monkeypatch, slug="demo-project")
scratch = tmp_path / "scratch"
(scratch / "motion" / "compositions" / "release-v1").mkdir(parents=True)
read_clone = tmp_path / "readclone"
read_clone.mkdir()
workspace = MagicMock()
workspace.export_branch_motion = AsyncMock(return_value=scratch)
workspace.ensure_read_clone = AsyncMock(return_value=read_clone)
t = _task(
draft={"composition_id": "release-v1"},
branch_name="feature/ux_ui/ABCD1234",
)
actions, task_svc = _make_actions(
task_obj=t, role="qa", team=None, workspace=workspace
)
tar_bytes = _make_frames_tar(["frame-0.png"])
_stub_renderer(monkeypatch, frames_tar_gz=tar_bytes, duration=1.0)
env = await actions.request_render(agent_id=uuid4())
assert env.error is None
assert env.evidence is not None
assert env.evidence["source"] == "branch"
assert env.evidence["dirty"] is False
assert len(env.evidence["frames"]) == 1
workspace.export_branch_motion.assert_awaited_once()
called_project, called_branch = workspace.export_branch_motion.call_args.args
assert called_project.slug == "demo-project"
assert called_branch == "feature/ux_ui/ABCD1234"
payload = markers.get_render_preview(t)
assert payload is not None
assert payload["source"] == "branch"
task_svc.heartbeat.assert_awaited_once()
@pytest.mark.asyncio
async def test_qa_without_branch_name_rejected(
monkeypatch: pytest.MonkeyPatch,
) -> None:
_arm(monkeypatch)
_stub_project(monkeypatch)
t = _task(draft={"composition_id": "release-v1"}, branch_name=None)
actions, _task_svc = _make_actions(task_obj=t, role="qa", team=None)
env = await actions.request_render(agent_id=uuid4())
assert env.error == "invalid_state"
# --------------------------------------------------------------------------- #
# renderer failure
# --------------------------------------------------------------------------- #
@pytest.mark.asyncio
async def test_video_renderer_error_is_retryable_invalid_state(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
_arm(monkeypatch)
monkeypatch.setattr(settings, "workspaces_root", str(tmp_path / "workspaces"))
_stub_project(monkeypatch, slug="demo-project")
clone_root = tmp_path / "clone"
(clone_root / "motion" / "compositions" / "release-v1").mkdir(parents=True)
workspace = MagicMock()
workspace.get_clone_root_path.return_value = clone_root
workspace.get_worktree_path.return_value = clone_root / ".worktrees" / "x"
t = _task(draft={"composition_id": "release-v1"})
actions, _task_svc = _make_actions(
task_obj=t, role="developer", team="ux_ui", workspace=workspace
)
_stub_renderer(monkeypatch, error=VideoRendererError("sidecar unreachable"))
env = await actions.request_render(agent_id=uuid4())
assert env.error == "invalid_state"
assert "retry" in (env.remediate or "").lower()
assert markers.get_render_preview(t) is None
@@ -64,6 +64,7 @@ def _build_choreographer() -> Choreographer:
"commits>=1",
"pr_open",
"self_verified",
"render_preview",
],
)
def test_hint_registered_for_previously_unhinted_token(token: str) -> None:
@@ -93,6 +94,13 @@ def test_during_work_hint_warns_reflect_doesnt_count() -> None:
), f"during_work hint must warn that scope='reflect' doesn't satisfy this: {hint!r}"
def test_render_preview_hint_mentions_request_render() -> None:
"""The render_preview hint must name the exact next call (request_render)."""
hint = Choreographer._hint_for_missing_key("render_preview", uuid4())
assert hint is not None
assert "request_render" in hint
# ---------------------------------------------------------------------------
# Multi-hint remediate — numbered list, not semicolon-joined
# ---------------------------------------------------------------------------
@@ -0,0 +1,67 @@
"""QA claim_review evidence carries a video-artifact context for
video-source tasks pointing QA at the rendered artifact, not just source."""
from __future__ import annotations
from unittest.mock import MagicMock
from roboco.foundation.policy.content import markers
from roboco.services.gateway.choreographer import Choreographer
from roboco.services.gateway.evidence_builder import build_evidence_for_task
def _stub_task() -> MagicMock:
task = MagicMock()
task.pr_number = None
task.pr_url = None
task.commits = []
task.dev_notes = None
task.acceptance_criteria_status = []
task.source = "code"
task.orchestration_markers = None
return task
def test_video_context_none_for_non_video_task() -> None:
assert Choreographer._qa_video_context(_stub_task()) is None
def test_video_context_present_for_video_task_with_render_preview() -> None:
task = _stub_task()
task.source = markers.VIDEO_TASK_SOURCE
task.orchestration_markers = {
markers.VIDEO_DRAFT: {"composition_id": "intro-v1"},
markers.RENDER_PREVIEW: {"frames": ["a.png", "b.png"]},
}
ctx = Choreographer._qa_video_context(task)
assert ctx is not None
assert ctx["composition_id"] == "intro-v1"
assert ctx["render_preview"] == {"frames": ["a.png", "b.png"]}
assert "request_render" in ctx["note"]
def test_video_context_render_preview_none_without_marker() -> None:
task = _stub_task()
task.source = markers.VIDEO_TASK_SOURCE
task.orchestration_markers = {markers.VIDEO_DRAFT: {"composition_id": "intro-v1"}}
ctx = Choreographer._qa_video_context(task)
assert ctx is not None
assert ctx["composition_id"] == "intro-v1"
assert ctx["render_preview"] is None
def test_evidence_payload_includes_video_context() -> None:
video_context = {"composition_id": "x", "render_preview": None, "note": "n"}
ev = build_evidence_for_task(
_stub_task(),
journal_highlights=[],
files_changed=[],
video_context=video_context,
)
assert ev.as_dict()["video_context"] == video_context
def test_evidence_payload_video_context_default_absent() -> None:
ev = build_evidence_for_task(_stub_task(), journal_highlights=[], files_changed=[])
assert ev.video_context is None
assert "video_context" not in ev.as_dict()
@@ -0,0 +1,55 @@
"""Video-authoring dev prompt block: the request_render/propose_video
verification instructions appear only for ``source=VIDEO_SOURCE`` tasks,
never for anything else mirrors test_possibilities_matrix_prompt.py's
bare-instance ``_build_dev_prompt`` idiom.
"""
from __future__ import annotations
from typing import Any
from uuid import uuid4
import pytest
from roboco.runtime.orchestrator import AgentOrchestrator
from roboco.services.task import VIDEO_SOURCE
def _orch() -> AgentOrchestrator:
orch = object.__new__(AgentOrchestrator)
orch._instances = {}
return orch
def _task(**over: Any) -> dict[str, Any]:
base: dict[str, Any] = {
"id": str(uuid4()),
"title": "Video: release v1.0.0",
"status": "claimed",
"plan": None,
}
base.update(over)
return base
@pytest.mark.asyncio
async def test_video_source_prompt_includes_request_render_instruction() -> None:
prompt = await _orch()._build_dev_prompt(_task(source=VIDEO_SOURCE))
assert "request_render" in prompt
assert "propose_video" in prompt
assert "render preview" in prompt
@pytest.mark.asyncio
async def test_non_video_source_prompt_omits_request_render_instruction() -> None:
prompt = await _orch()._build_dev_prompt(_task(source="code"))
assert "request_render" not in prompt
@pytest.mark.asyncio
async def test_missing_source_prompt_omits_request_render_instruction() -> None:
prompt = await _orch()._build_dev_prompt(_task())
assert "request_render" not in prompt
if __name__ == "__main__":
pytest.main([__file__, "-q"])
+80 -2
View File
@@ -35,6 +35,7 @@ SLUG = "roboco"
ONE = 1
TWO = 2
THREE = 3
FOUR = 4
async def _seed(session: AsyncSession) -> None:
@@ -138,9 +139,11 @@ async def test_open_video_task_creates_assigned_authoring_task(
assert task.estimated_complexity == Complexity.LOW
assert task.acceptance_criteria # non-empty
# Third AC line: composition follows the design bar / demo-kit register.
assert len(task.acceptance_criteria) == THREE
assert len(task.acceptance_criteria) == FOUR
assert "motion/README.md" in task.acceptance_criteria[2]
assert "panel-demo" in task.acceptance_criteria[2]
# Fourth AC line: request_render preview frames must be verified.
assert "request_render" in task.acceptance_criteria[3]
project = await db_session.get(ProjectTable, task.project_id)
assert project is not None
assert project.slug == SLUG
@@ -154,6 +157,7 @@ async def test_open_video_task_creates_assigned_authoring_task(
assert "motion/README.md" in draft["brief"]
assert "motion/kit/README.md" in draft["brief"]
assert "compositions/panel-demo/" in draft["brief"]
assert "request_render" in draft["brief"] # verify-the-render pointer
assert "Brand voice" not in draft["brief"] # unset -> omitted
assert draft["suggested_input_props"] == {} # none supplied
assert task.description == draft["brief"]
@@ -715,9 +719,10 @@ async def test_open_video_task_acceptance_criteria_has_design_bar_line(
occasion="release v9.9.9", script="s", platforms=["x"], brief="b"
)
assert task is not None
assert len(task.acceptance_criteria) == THREE
assert len(task.acceptance_criteria) == FOUR
assert "motion/README.md" in task.acceptance_criteria[2]
assert "panel-demo" in task.acceptance_criteria[2]
assert "request_render" in task.acceptance_criteria[3]
@pytest.mark.asyncio
@@ -818,3 +823,76 @@ async def test_rerender_none_for_missing_task(db_session: AsyncSession) -> None:
assert result is None
open_tasks = await get_task_service(db_session).list_open_video_posts()
assert open_tasks == []
# --------------------------------------------------------------------------- #
# reauthor_from_rejection — CEO reject feedback loop
# --------------------------------------------------------------------------- #
@pytest.mark.asyncio
async def test_reauthor_from_rejection_opens_revision_with_reason_and_pointer(
db_session: AsyncSession, monkeypatch: pytest.MonkeyPatch
) -> None:
await _seed(db_session)
_enable(monkeypatch)
engine = video_engine_module.VideoEngine(db_session)
source_task = await engine.open_video_task(
occasion="release v1.0.0",
script="Here's what shipped",
platforms=["x", "tiktok"],
brief="Announce the release",
)
assert source_task is not None
draft = markers.get_video_draft(source_task) or {}
markers.set_video_draft(source_task, {**draft, "composition_id": "ReleaseIntro"})
# Mirrors production: the render loop only fires once the authoring task
# has gone through the full delivery lifecycle to COMPLETED — an open
# source task with the same occasion would otherwise dedupe the revision
# reauthor is about to open right back against itself.
source_task.status = TS.COMPLETED
await db_session.flush()
post_task = await engine._originate_video_post(
source_task=source_task,
mp4_paths={"vertical": "a.mp4", "square": "b.mp4"},
captions={"x": "cap", "tiktok": "cap2"},
platforms=["x", "tiktok"],
)
# Mirrors VideoPostService.reject: the draft is already CANCELLED by the
# time reauthor_from_rejection runs, so open_video_task's own-occasion
# dedup (which only scans OPEN drafts) doesn't block against it.
post_task.status = TS.CANCELLED
await db_session.flush()
revision = await engine.reauthor_from_rejection(
post_task, "Logo is cut off in the second scene"
)
assert revision is not None
assert revision.id != post_task.id
assert revision.source == VIDEO_SOURCE
revision_draft = markers.get_video_draft(revision)
assert revision_draft is not None
assert revision_draft["occasion"] == "release v1.0.0" # SAME occasion
assert revision_draft["platforms"] == ["x", "tiktok"]
assert "Logo is cut off in the second scene" in revision_draft["brief"]
assert "motion/compositions/ReleaseIntro/" in revision_draft["brief"]
assert "do not start a new composition" in revision_draft["brief"]
@pytest.mark.asyncio
async def test_reauthor_from_rejection_missing_draft_returns_none(
db_session: AsyncSession, monkeypatch: pytest.MonkeyPatch
) -> None:
await _seed(db_session)
_enable(monkeypatch)
engine = video_engine_module.VideoEngine(db_session)
task = await engine.open_video_task(
occasion="no-draft", script="s", platforms=["x"], brief="b"
)
assert task is not None
task.orchestration_markers = {} # strip the video_draft marker
await db_session.flush()
result = await engine.reauthor_from_rejection(task, "some reason")
assert result is None
@@ -509,6 +509,80 @@ async def test_reject_records_reason_and_cancels(db_session: AsyncSession) -> No
assert markers.get_video_reject_reason(updated) == "Doesn't match the release"
@pytest.mark.asyncio
async def test_reject_with_reason_calls_reauthor_with_cancelled_task(
db_session: AsyncSession,
) -> None:
"""A non-blank reject reason routes into VideoEngine.reauthor_from_rejection,
called with the just-cancelled task and the verbatim reason."""
task = await _seed_video_post(db_session)
fake_engine = MagicMock()
fake_engine.reauthor_from_rejection = AsyncMock(return_value=None)
with (
_LOCKED[0],
_LOCKED[1],
patch(
"roboco.services.video_engine.get_video_engine",
return_value=fake_engine,
),
):
updated = await _svc(
db_session, x_poster=_StubXPoster(), tiktok_poster=_StubTikTokPoster()
).reject(_id(task), "Doesn't match the release")
assert updated is not None
assert updated.status == TS.CANCELLED
fake_engine.reauthor_from_rejection.assert_awaited_once()
called_task, called_reason = fake_engine.reauthor_from_rejection.await_args.args
assert called_task.id == task.id
assert called_reason == "Doesn't match the release"
@pytest.mark.asyncio
async def test_reject_succeeds_even_when_reauthor_raises(
db_session: AsyncSession,
) -> None:
"""A reauthor failure must never fail or roll back the reject — the
cancel already committed before this best-effort seam runs."""
task = await _seed_video_post(db_session)
fake_engine = MagicMock()
fake_engine.reauthor_from_rejection = AsyncMock(side_effect=RuntimeError("boom"))
with (
_LOCKED[0],
_LOCKED[1],
patch(
"roboco.services.video_engine.get_video_engine",
return_value=fake_engine,
),
):
updated = await _svc(
db_session, x_poster=_StubXPoster(), tiktok_poster=_StubTikTokPoster()
).reject(_id(task), "Doesn't match the release")
assert updated is not None
assert updated.status == TS.CANCELLED
assert markers.get_video_reject_reason(updated) == "Doesn't match the release"
@pytest.mark.asyncio
async def test_reject_blank_reason_skips_reauthor(db_session: AsyncSession) -> None:
task = await _seed_video_post(db_session)
fake_engine = MagicMock()
fake_engine.reauthor_from_rejection = AsyncMock()
with (
_LOCKED[0],
_LOCKED[1],
patch(
"roboco.services.video_engine.get_video_engine",
return_value=fake_engine,
),
):
updated = await _svc(
db_session, x_poster=_StubXPoster(), tiktok_poster=_StubTikTokPoster()
).reject(_id(task), " ")
assert updated is not None
assert updated.status == TS.CANCELLED
fake_engine.reauthor_from_rejection.assert_not_awaited()
@pytest.mark.asyncio
async def test_reject_takes_the_same_lock_approve_holds(
db_session: AsyncSession,
@@ -152,6 +152,130 @@ async def test_null_renderer_raises_without_network_call(tmp_path: Path) -> None
)
@pytest.mark.asyncio
async def test_render_frames_posts_frames_field_and_parses_duration(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
source = _make_source(tmp_path)
monkeypatch.setattr(cfg, "video_request_timeout_seconds", 5.0)
monkeypatch.setattr(cfg, "video_render_timeout_seconds", 30.0)
captured: dict[str, bytes] = {}
expected_duration = 12.5
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = request.content
return httpx.Response(
200,
content=b"fake-frames-tar-gz",
headers={"X-Video-Duration": str(expected_duration)},
)
transport = httpx.MockTransport(handler)
http_client = httpx.AsyncClient(transport=transport)
renderer = VideoRenderer(base_url="http://fake-video-renderer", client=http_client)
tar_bytes, duration = await renderer.render_frames(
str(source),
composition_id="Intro",
input_props={"title": "hello"},
orientation="vertical",
frame_count=8,
)
await http_client.aclose()
assert tar_bytes == b"fake-frames-tar-gz"
assert duration == expected_duration
body = captured["body"]
assert isinstance(body, bytes)
assert b'name="frames"' in body
assert b"8" in body
@pytest.mark.asyncio
async def test_render_frames_missing_duration_header_returns_zero(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
source = _make_source(tmp_path)
monkeypatch.setattr(cfg, "video_request_timeout_seconds", 5.0)
monkeypatch.setattr(cfg, "video_render_timeout_seconds", 30.0)
def handler(_request: httpx.Request) -> httpx.Response:
return httpx.Response(200, content=b"fake-frames-tar-gz")
transport = httpx.MockTransport(handler)
http_client = httpx.AsyncClient(transport=transport)
renderer = VideoRenderer(base_url="http://fake-video-renderer", client=http_client)
tar_bytes, duration = await renderer.render_frames(
str(source),
composition_id="Intro",
input_props={},
orientation="square",
frame_count=4,
)
await http_client.aclose()
assert tar_bytes == b"fake-frames-tar-gz"
assert duration == 0.0
@pytest.mark.asyncio
async def test_render_frames_non_success_response_raises_clear_error(
tmp_path: Path,
) -> None:
source = _make_source(tmp_path)
def handler(_request: httpx.Request) -> httpx.Response:
return httpx.Response(400, text="frames out of bounds")
transport = httpx.MockTransport(handler)
http_client = httpx.AsyncClient(transport=transport)
renderer = VideoRenderer(base_url="http://fake-video-renderer", client=http_client)
with pytest.raises(VideoRendererError, match="400"):
await renderer.render_frames(
str(source),
composition_id="Intro",
input_props={},
orientation="square",
frame_count=4,
)
await http_client.aclose()
@pytest.mark.asyncio
async def test_render_frames_unconfigured_renderer_raises_without_network_call(
tmp_path: Path,
) -> None:
source = _make_source(tmp_path)
renderer = VideoRenderer(base_url="")
with pytest.raises(VideoRendererError, match="not configured"):
await renderer.render_frames(
str(source),
composition_id="Intro",
input_props={},
orientation="vertical",
frame_count=4,
)
@pytest.mark.asyncio
async def test_render_frames_null_renderer_raises_without_network_call(
tmp_path: Path,
) -> None:
source = _make_source(tmp_path)
renderer = NullVideoRenderer()
with pytest.raises(VideoRendererError, match="not configured"):
await renderer.render_frames(
str(source),
composition_id="Intro",
input_props={},
orientation="vertical",
frame_count=4,
)
def test_get_video_renderer_returns_null_when_unset(
monkeypatch: pytest.MonkeyPatch,
) -> None: