roboco/docs/map/video-engine.md

## Purpose

The RoboCo video engine: a default-off subsystem that authors bespoke short marketing videos (release announcements, feature spotlights, on-demand CEO briefs) and distributes them to X and TikTok — nothing renders or posts without the flags on, and nothing posts without an explicit per-clip CEO approval. It mirrors the `XEngine` / `ReleaseManagerEngine` held-artifact shape, but splits across the real delivery lifecycle: a normal ASSIGNED UX/UI authoring task ships the composition through the standard commit/PR/QA/doc/review gate, then an orchestrator render loop renders the merged `motion/` source via the credential-free `video-renderer` sidecar and materializes a held `video_post` draft for the CEO. Two task kinds (authoring + held post), one render pass between them.

## Files

| Path | Role | approx LOC |
|---|---|---|
| `roboco/services/video_engine.py` | `VideoEngine` — opens the ASSIGNED authoring task (`open_video_task`, balanced across `ux-dev-1`/`ux-dev-2`) and originates the held `video_post` draft once the render succeeds (`_originate_video_post`); release/spotlight/on-demand trigger wiring. | 324 |
| `roboco/services/video_post_service.py` | `VideoPostService` — CEO approve/reject over the held post; the ONLY caller of the X-v2 and TikTok posters; runs the critical section under a heartbeat-renewed Redis mutex, commits each platform's posted-id durably before the next, idempotent on already-`COMPLETED` AND on already-CANCELLED (rejected). | 629 |
| `roboco/services/video_renderer_client.py` | `VideoRenderer` — tars the merged `motion/` dir, POSTs the tarball to the sidecar (`ROBOCO_VIDEO_RENDERER_BASE_URL`), saves the returned MP4s to `video_output_dir` and PUTs each to MinIO (`_save`). `NullVideoRenderer` raises on unconfigured so the render loop fails loud rather than silently no-op'ing a real trigger; `get_video_renderer()` factory. | 188 |
| `roboco/services/minio_client.py` | Singleton `Minio` (minio-py) with an unconfigured guard (`get_client()` returns `None` when `minio_endpoint` empty); `put_object` / `get_object_stream` / `stat_object`, sync, call sites wrapped in `asyncio.to_thread`. | 129 |
| `roboco/services/tiktok_client.py` | `TikTokPoster` — TikTok inbox-upload poster (v2 media, OAuth2 refresh). Fernet-encrypted singleton `tiktok_credentials` row (migration 062); agents never hold creds or egress. | 326 |
| `roboco/services/x_video_client.py` | `XVideoPoster` — X v2 media upload + tweet poster. `NullXVideoPoster` makes the unconfigured leg a graceful no-op. | 266 |
| `roboco/runtime/heartbeat_mutex.py` | `HeartbeatMutex` — Redis mutex with heartbeat-renewed TTL, shared with `ReleaseProposalService`'s release-execute lock shape; backs `VideoPostService.approve`'s long video-upload critical section. | — |
| `roboco/mcp/do_server.py` `propose_video` | Do-tool the UX/UI dev calls exactly once per authoring task to stamp the `video_draft` marker (composition id + per-platform captions + input props); metadata-only, does not render. | — |
| `roboco/services/gateway/content_actions.py` `propose_video` | Server-side action: team-gated (`_caller_team` rejects be-dev/fe-dev), resolves the caller's open video task, `markers.set_video_draft` with the metadata. | — |
| `roboco/services/gateway/content_actions.py` `request_render` | Do-verb (developer/QA): renders the caller's ACTUAL composition to keyframe PNGs via the sidecar's frames mode and stamps the `render_preview` marker — dev renders their own tree (worktree-aware, `head_sha`/`dirty` stamped), QA a read-only branch export (`WorkspaceService.export_branch_motion`). Frames land at the container-shared `{workspaces_root}/{project}/.previews/{task8}/{orientation}/`. | — |
| `roboco/foundation/policy/tracing.py` `RENDER_VERIFIED` | `i_am_done` requirement on `source=video` tasks: no stamped `render_preview` → tracing gap naming `render_preview` (hint: call `request_render`, Read every frame). Mirrored in the possibilities-matrix fast path. | — |
| `roboco/api/routes/video.py` `GET /video/preview-frames/{task_id}` / `.../{orientation}/{filename}` | CEO-gated routes serving the dev's already-rendered `request_render` preview frames — a `source=video` authoring task reaches `awaiting_ceo_approval` with no MP4 yet (rendering only happens post-completion), so the CEO otherwise has nothing to review. Frame listing is parsed from the self-describing `.previews/{task8}/{orientation}/frame-<idx>-of-<n>-at-<t>s.png` filenames (`_FRAME_NAME_RE`) rather than the `render_preview` marker (which only holds the LAST call's single orientation); the per-frame route streams a PNG behind the same `_resolve_preview_path` confinement guard the composition-HTML proxy already used. | — |
| `alembic/versions/062_tiktok_credentials.py` | Migration 062 — the `tiktok_credentials` singleton row (Fernet-encrypted OAuth2 secrets, all-or-nothing set/clear, mirroring the git-token / `x_credentials` pattern). | 44 |
| `video-renderer/` | The sidecar: `server.js` (HTTP+tarball boundary), `render.js` (`@hyperframes/producer` `createRenderJob` + `executeRenderJob`, system `ffmpeg`, headless Chromium). Credential-free and git-free — reads only what's POSTed. pnpm-managed (`pnpm-lock.yaml`, no npm `package-lock.json`); `@hyperframes/producer` pinned exact at `0.7.36` (`0.7.60` fails every render). | — |
| `docker/video-renderer.Dockerfile` | Sidecar image (`roboco-video-renderer`): Node + Chromium + system `ffmpeg`; installs `@hyperframes/producer`. No RoboCo source, no creds. | — |
| `motion/README.md` `## Design bar` / `## Visual design bar (demo/kit register)` | Authoring craft an assigned UX/UI dev consults before building a composition: color/type/motion/layout dials for the text-card register, plus spacing, beat pacing (`animation-delay`, never `data-start` for beats), `pk-chip`/`pk-pill` semantic-variant discipline, camera+cursor+rhythm, and anti-generic tells for the `kit/` demo register. | 149 (file total) |
| `motion/skills/references/{house-style,video-composition,beat-direction,motion-principles}.md` | Four upstream HyperFrames craft references (palette/lazy-defaults, video-medium scale/density, per-beat rhythm planning, ease/speed/direction variance) vendored verbatim at pinned commit `9d148d28` (Apache-2.0, header in each file) — back every rule in `motion/README.md`'s design-bar sections; re-vendor when bumping `@hyperframes/producer`. | 462 |
| `motion/skills/hyperframes-catalog-index.md` | RoboCo-authored index of the public HyperFrames catalog (109 blocks + 24 components, 133 entries) with per-category kit-mapping triage (maps onto an existing `pk-*` piece / a choreography engine / needs a new kit piece); read on demand when planning a beat, not injected into any agent prompt. | 186 |
| `motion/skills/{hyperframes-core,hyperframes-creative,hyperframes-keyframes}.md` | The vendor's own official HyperFrames agent skills (composition contract, beat planning, seek-safe keyframes across runtimes), vendored verbatim at a pinned upstream commit (Apache-2.0, header + re-vendor note in each file); `motion/README.md` points authoring devs at them before a new register. Primary seek-safe primitive is GSAP tweens on `window.__timelines` — this kit's CSS-animation register is a house pattern, and the clip-window rule is its empirically-derived companion. | 382 |
| `motion/kit/kit.js` `choreographCursor` / `choreographCamera` | Choreography engines: `choreographCursor` reads `data-waypoints="t x y [click]; ..."` off a `.pk-cursor` and generates a multi-leg eased path with fade in/out, an idle-hand sway between legs, and click rings + glyph press dips at flagged waypoints (click lands ~0.2s before the thing it triggers) — replaces the old single-glide `--pk-cursor-x0/y0/x1/y1` cameo. `choreographCamera` reads `data-shots="t x y scale; ..."` off a `.pk-camera` wrapper and eases push-ins/pull-backs, settling at identity by the end. `motion/kit/README.md` documents both. | 156 (diff) |
| `roboco/runtime/orchestrator.py` `_is_video_authoring_spawn` | Fail-closed spawn-time probe (role==developer, team==ux_ui, task.source==video) that registers the `playwright` MCP for the composition author — the one non-QA case, so the dev can preview the authored HTML in a real browser between renders. Gating-only: `agent-ux`'s image already bakes the browser + wrapper entrypoint. | — |

## Data Flow

DETECT → AUTHOR: a release publish (`ROBOCO_VIDEO_ON_RELEASE`), a CEO-approved feature-spotlight draft that requests one (`ROBOCO_VIDEO_ON_SPOTLIGHT`), or a CEO on-demand `POST /api/video/request` calls `VideoEngine.open_video_task`, which creates a normal ASSIGNED UX/UI authoring task (`source=video`, `confirmed_by_human=True`, balanced across the two ux-devs) — NOT held, NOT in any dispatcher's skip bucket. The assigned dev authors `motion/compositions/<id>/{vertical,square}.html` (HyperFrames render params on `<html>`), reads the vendored `motion/skills/hyperframes-{core,creative,keyframes}.md` skills and consults `kit.js`'s `choreographCursor`/`choreographCamera` engines for camera+cursor craft (a locked-off camera or a popping/freezing cursor is automatic revision; clip windows are for structural layers ONLY — beats ride base-hidden delayed CSS animations, since the renderer's clip scheduler drops any beat driven off its own clip window), previews the live HTML via the `playwright` MCP registered for this spawn (`_is_video_authoring_spawn`), calls the `propose_video` do-tool exactly once (server-side `content_actions.propose_video` is team-gated and stamps `video_draft`), then verifies the ARTIFACT: `request_render` renders the dev's actual working tree to keyframe PNGs the dev must Read (every scene fully visible and legible — the gate that catches an authored duration shorter than its scene list), iterating fix → re-render until the frames prove the brief; `i_am_done` refuses without the stamped `render_preview` marker (`Requirement.RENDER_VERIFIED`). Then `commit` + `open_pr` through the normal PR-review gate. The authoring task rides the standard QA/doc/review lifecycle to `completed`, with QA's `claim_review` evidence carrying a `video_context` block (the dev's preview + an instruction to `request_render` the branch state fresh).

RENDER: once the authoring task is `completed`, the orchestrator's `_video_render_loop` (bounded retry, `_MAX_VIDEO_RENDER_ATTEMPTS`) resolves the project's read-clone at the merged HEAD, tars the `motion/` dir, and POSTs it to the credential-free `video-renderer` sidecar (`ROBOCO_VIDEO_RENDERER_BASE_URL`). The sidecar untars, runs `@hyperframes/producer`'s `createRenderJob` + `executeRenderJob` per orientation (headless Chrome + system `ffmpeg`, `ROBOCO_VIDEO_RENDER_TIMEOUT_SECONDS` per render), and streams both 9:16 and 1:1 MP4s back. `VideoRenderer` saves them to `ROBOCO_VIDEO_OUTPUT_DIR` (`_save` also PUTs each to MinIO when `minio_endpoint` is set, non-fatal). On success `VideoEngine._originate_video_post` materializes a held `video_post` draft (`source=video_post`, `confirmed_by_human=False`, Secretary-owned, skipped by every dispatcher) carrying `mp4_paths` (`{vertical, square}` absolute paths) + the per-platform captions.

CEO ACT: `GET /api/video/posts` lists held drafts (including `mp4_paths`); `GET /api/video/posts/{id}/media?cut=vertical|square` streams the MP4 bytes for the preview player (CEO-gated, falls back to `FileResponse` on `S3Error`/unconfigured MinIO). The CEO edits captions and approves/rejects in the panel's `video-post-queue.tsx`. `POST /api/video/posts/{id}/approve` is the ONLY caller of `XVideoPoster` / `TikTokPoster`: it acquires `HeartbeatMutex`, re-reads the committed task state inside the lock, commits `COMPLETED` before releasing (so a concurrent approve can't double-post), commits each platform's posted-id durably before attempting the next (a partial failure never re-posts an already-succeeded platform on retry), and is idempotent (an already-`COMPLETED` draft returns the stored ids without calling a poster). A CANCELLED draft (already rejected) is refused both pre-lock and re-checked under lock, returning `already_rejected` — closes a hole where a stale approve (e.g. a queued Telegram button targeting the draft by id regardless of its current status) could post a draft the CEO had already rejected. `POST /api/video/posts/{id}/reject` cancels the draft with a reason — and, for a non-empty reason, `VideoEngine.reauthor_from_rejection` opens a fresh authoring task (same occasion, brief = the CEO's verbatim feedback + revise-in-place pointer at the existing composition) so the rejection feedback re-enters the delivery flow instead of dying on the cancelled draft; best-effort, never fails the reject.

## Config Flags

- `ROBOCO_VIDEO_ENGINE_ENABLED` — master switch; off = no video-authoring task is ever opened and no render/post happens. Panel-toggleable.
- `ROBOCO_VIDEO_ON_RELEASE` / `ROBOCO_VIDEO_ON_SPOTLIGHT` — sub-switches for the two automatic triggers, independent of the master switch and of the CEO's on-demand `POST /video/request`.
- `ROBOCO_VIDEO_RENDER_INTERVAL_SECONDS` / `ROBOCO_VIDEO_RENDER_TIMEOUT_SECONDS` / `ROBOCO_VIDEO_REQUEST_TIMEOUT_SECONDS` / `ROBOCO_VIDEO_OUTPUT_DIR` — render loop cadence, per-render deadline, sidecar HTTP deadline, MP4 output dir (bind-mounted in all three compose files so renders survive container recreation).
- `ROBOCO_VIDEO_RENDERER_BASE_URL` — the sidecar endpoint (default `http://roboco-video-renderer:3001`).
- `ROBOCO_MINIO_*` — MinIO object storage (default-off; `video_renderer_client._save` PUTs each render after the local write; serve route streams via `StreamingResponse` with `FileResponse` fallback).

## Changes Since Baseline

- **2026-07-20** (#608, `a5d8c6bd`, "CEO can preview a video authoring task's frames before approving"): two CEO-gated routes serve the `request_render` preview frames (see Files above) so the CEO has something concrete to review at `awaiting_ceo_approval` before any MP4 exists. The task-detail Overview gains a Video preview card — a 9:16/1:1 toggle + prev/next/scrubber frame stepper with composition id, duration, and a dirty badge — shown for a video task with preview frames or awaiting CEO approval.
- **2026-07-17** (PR #543, `3e801697`): Two renderer root causes fixed — `@hyperframes/producer` was floating (`^0.7.36`, no lockfile), so image builds silently picked up `0.7.60`, which fails EVERY render ("Cannot access 'rt' before initialization"); pinned exact (`0.7.36`, no caret) and committed a lockfile (regenerated as `pnpm-lock.yaml` by the immediate follow-up `a12fefcb`, not the npm `package-lock.json` this PR first wrote — this package is pnpm-managed). Second: the producer's per-clip visibility scheduler runs on a clock that lags ~50% behind the encoded timeline on a long cut, so tail scenes (past roughly the halfway mark) were silently missing from the MP4 regardless of authoring — fixed by treating `class="clip"` + `data-start`/`data-duration` as a structural-layer-only primitive and driving every beat with base-hidden styles + a delayed CSS animation instead (documented in `motion/README.md`'s "Clip windows are for structural layers only" rule). Also added the two choreography engines to `motion/kit/kit.js` (`choreographCursor` / `choreographCamera`, see Files above) plus a "Cinematography & rhythm" section in `motion/README.md` and a craft-bar block in the dev video spawn prompt (`roboco/runtime/orchestrator.py`) so a locked-off camera or a popping/freezing cursor reads as an automatic revision.
- **2026-07-17** (PR #544, `fd621f0d`): The three craft capabilities wired one hop closer to the hands doing video work — vendored the vendor's own official HyperFrames agent skills (`hyperframes-core`/`-creative`/`-keyframes.md`, see Files above; supersedes the external-pointer-only version briefly added by the intervening `1416bd1d`); registered the `playwright` MCP for a ux-dev spawned onto a `source=video` task (`_is_video_authoring_spawn`, fail-closed role/team/task-source probe — gating-only, `agent-ux`'s image already bakes the browser); and added a video-mode override to the `ux_ui` team prompt's design bar ("video-authoring tasks are FILMS, not UI — these dials do not apply") so a video task no longer reads its own "dense product UI → motion 2-3" dial as license to ship a static slideshow.
- **2026-07-17** (Wave 6, PR #550): Authoring craft, not engine code — `motion/README.md` gained `## Visual design bar (demo/kit register)` (spacing/hierarchy, beat pacing, `pk-chip`/`pk-pill` semantic discipline, camera+cursor+rhythm, anti-generic tells for the `kit/` register), four upstream HyperFrames craft references vendored verbatim under `motion/skills/references/` (fixing `hyperframes-creative.md`'s previously-dead `references/` pointers), and a new `motion/skills/hyperframes-catalog-index.md` (133-entry HyperFrames catalog vocabulary index, read-on-demand). No service/verb/schema change; the render/post pipeline documented above is untouched.
- **2026-07-18** (PR #570, "project-branded drafts"): the release-video script/prompt/brief builders in `roboco/services/video_engine.py` (`_fallback_release_script`, `_release_video_prompt`, `_release_video_brief`, `_draft_release_script`) all gained a required `product_name` param — `draft_release_post_video` (the `ReleaseProposalService.approve` publish-success hook, mirroring `XEngine.draft_release_post`) now resolves it via `CompanyGoalsService.resolve_product_name(project)` (the release's own project name → charter `company_goals.company_name` → "RoboCo" literal) instead of hardcoding "RoboCo" into the script/brief text — see `docs/map/product-strategy-research-pitch.md` for the shared resolver. `GET /api/video/posts` responses also gained `project_slug`/`project_name` (`api/schemas/video.py`, via the same `task_project_fields` helper `x.py` uses) so the panel's `video-post-queue.tsx` can render a `ProjectBadge` alongside the source-kind badge.

## Health

Default-off, CEO-gated at two independent points (the flags, then per-clip approval). The held-draft shape mirrors the XEngine / ReleaseManagerEngine pattern, so the dispatchers never see it. The render pass is bounded retry with `_MAX_VIDEO_RENDER_ATTEMPTS` and a per-render deadline; `NullVideoRenderer` raises on unconfigured so a misflagged trigger fails loud rather than silently no-op'ing. The approve critical section is heartbeat-mutex protected so a double-click can't double-post and a partial platform failure is recoverable, and (Wave 5, PR #551) `approve` now also refuses a CANCELLED draft outright rather than posting it. TikTok's OAuth2 secrets live Fernet-encrypted in a singleton row (migration 062); agents never hold creds or egress — `VideoPostService.approve` is the only caller of the posters.

## Related

- `docs/rag/architecture/video-engine.md` — the user-facing architecture doc
- `docs/rag/architecture/minio-storage.md` — the decoupled-durable render storage
- `docs/map/release-manager.md` — the sibling held-artifact engine whose lock shape `VideoPostService.approve` mirrors
- `docs/map/engines-heal-ciwatch-depupdate.md` — the other default-off originate-and-stop engines