Files
roboco/.github/workflows/agent-image-smoke.yml
T
f0f09b2204 [1197c975] Re-add Playwright chromium to FE/UX QA images and add browser-verification prompt guidance (#406)
* [3e552255] Re-add Playwright chromium to QA images + prompt guidance (#395) (#405)

* [3e552255] feat(docker): re-add Playwright chromium-headless-shell to QA images

* [3e552255] ci(docker): add Playwright QA image build + headless smoke check workflow

* [3e552255] fix(ci): scope agent-image-smoke.yml trigger to paths only, add PR comment

The workflow was gated by `branches: [master]` on both push and
pull_request, but this repo's task-hierarchy PRs open against nested
parent feature branches, not master, until root->master assembly - so
the workflow never fired on a dev-level PR and produced zero evidence.
Drop the branch filter (path scoping is sufficient) and post the
size-delta table + smoke-check output as a PR comment via
actions/github-script, since no agent role has gh CLI or GitHub API
read access to pull check-run output directly.

* [3e552255] fix(ci): post agent-image-smoke PR comment even on step failure

The 'Post results as a PR comment' step only had
`if: github.event_name == 'pull_request'`, which GitHub implicitly ANDs
with success() — so if the docker build or headless-launch smoke check
failed, the PR comment (the only evidence-delivery path QA/PM has, since
no agent role can read the Checks tab) silently never posted. Added
always() so a partial report always lands on the PR.

This commit also re-lands the branches-filter removal + PR-comment step
from 747e4d74 to make sure this fix actually reaches the remote PR
branch — QA's needs_revision at 05:48 came after that commit's local
timestamp (05:43) but still saw the pre-fix workflow, indicating the
prior push never reached GitHub.

* [3e552255] docs(qa): add browser verification guide and CHANGELOG entry for Playwright chromium

- Added comprehensive QA browser verification guide at docs/backend/qa/browser-verification.md covering setup, examples for fe-qa/ux-qa, limitations, and troubleshooting
- Updated CHANGELOG.md with Unreleased entry documenting Playwright chromium-headless-shell re-add to agent-qa-fe and agent-ux images, CI smoke workflow, and links to QA guidance
- Guide explains when to use browser verification (rendered output, computed styles, a11y, visual design), how to launch headless chromium, and provides practical examples for both FE and UX QA use cases
- References built-in guidance in fe-qa.md/ux-qa.md identity prompts and CI smoke workflow verification

---------

Co-authored-by: Backend Developer 1 <be-dev-1@roboco.tech>
Co-authored-by: Backend Documenter <be-doc@roboco.tech>

* [9aafe8f5] Revision: fix reflow-check CI failure and resolve orchestrator Playwright-allowance gap (#421)

* [9973237c] fix(docs): separate reflow-joined metadata lines in browser-verification.md (#419)

scripts/reflow_md.py --check treats two adjacent non-blank lines as one
paragraph and flags it as hard-wrapped prose needing a join. Insert a
blank line between the `**For:**` and `**Purpose:**` metadata lines so
each stays its own single-line paragraph; verified the reflow is now a
no-op and the non-whitespace token sequence is unchanged.

Co-authored-by: Backend Developer 1 <be-dev-1@roboco.tech>

* [cc355a2b] docs(qa): add no-op analysis for orchestrator.py Playwright allowance (#422)

Co-authored-by: Backend Developer 1 <be-dev-1@roboco.tech>

* [a506cc10] docs(qa): quote real Dockerfile Playwright snippets in no-op analysis (#424)

Co-authored-by: Backend Developer 1 <be-dev-1@roboco.tech>

---------

Co-authored-by: Backend Developer 1 <be-dev-1@roboco.tech>

* [2b35dd4e] Resolve merge conflict, confirm green CI, add Playwright MCP registration, re-verify 5 ACs (#447) (#450)

* [2b35dd4e] docs(changelog): resolve Unreleased/0.22.0 ordering conflict, keep Playwright entry

* [2b35dd4e] feat(runtime): register Playwright MCP server for fe-qa/ux-qa, per CEO round-3 note

Adds @playwright/mcp to agent-qa-fe and agent-ux images, wired via a wrapper
entrypoint that points the server at each image's already-baked
chromium-headless-shell instead of downloading a second browser. The
orchestrator registers the `playwright` MCP server only for the qa role on
the frontend/ux_ui teams, so be-qa and ux-dev never see it. Updates the QA
identity prompts and docs/backend/qa/browser-verification.md to document the
structured mcp__playwright__* tools in place of hand-scripted Bash+Python,
adds CI smoke coverage (binary + baked-chromium resolution + a real
panel-page screenshot from inside the ux-qa image), and records the change
in CHANGELOG.md.

---------

Co-authored-by: Backend Developer 1 <be-dev-1@roboco.tech>

* [1197c975] type the mcpServers extraction so mypy's no-any-return passes

* [1197c975] extract role-scoped MCP registration — orchestrator back under the complexity budget

The playwright branch pushed _generate_mcp_config to rank C in the merge
ref; docs/research/playwright registrations move to one helper, behavior
identical.

* [1197c975] basename-sanitize the MCP config filename

CodeQL's path-injection query re-fired on the (moved, unchanged) config
write; agent ids are orchestrator-issued, but the filename now rides the
same os.path.basename sanitizer _grok_usage_json established.

* [1197c975] basename the agent id variable itself — the sanitizer shape CodeQL models

---------

Co-authored-by: Backend Developer 1 <be-dev-1@roboco.tech>
Co-authored-by: Backend Documenter <be-doc@roboco.tech>
Co-authored-by: Renn F <rennf93@users.noreply.github.com>
2026-07-11 07:39:53 +02:00

202 lines
8.8 KiB
YAML

name: Agent Image Smoke (Playwright)
# Builds the two images that carry Playwright + chromium-headless-shell
# (agent-qa-fe, agent-ux) with a real Docker daemon, verifies a headless
# chromium launch actually works inside each, verifies the `playwright` MCP
# server (registered for fe-qa/ux-qa only, see orchestrator.py
# _generate_mcp_config) is installed and wired to the baked browser, takes a
# real headless screenshot of a live panel page from inside the ux-qa image,
# and reports the real before/after image size delta against origin/master.
# This exists because dev/QA agent sandboxes have no Docker daemon of their
# own — this workflow is the only place that can produce non-estimated
# numbers for this change.
on:
push:
paths:
- 'docker/agent-base.Dockerfile'
- 'docker/agent-qa-fe.Dockerfile'
- 'docker/agent-ux.Dockerfile'
- 'docker/scripts/playwright-mcp-entrypoint.sh'
- '.github/workflows/agent-image-smoke.yml'
pull_request:
# No `branches:` filter — this repo's task hierarchy opens dev-level
# PRs against nested parent feature branches (e.g.
# feature/backend/<root>--<cell>--<subcell>), not directly against
# master, so a `branches: [master]` filter here would only ever fire
# once the assembled PR reaches root->master — too late to give QA/PM
# real evidence at the dev-PR review stage this task exists for.
paths:
- 'docker/agent-base.Dockerfile'
- 'docker/agent-qa-fe.Dockerfile'
- 'docker/agent-ux.Dockerfile'
- 'docker/scripts/playwright-mcp-entrypoint.sh'
- '.github/workflows/agent-image-smoke.yml'
workflow_dispatch:
jobs:
playwright-smoke:
name: Build QA images, verify headless chromium launch + size delta
runs-on: ubuntu-latest
permissions:
pull-requests: write
steps:
- name: Checkout code
uses: actions/checkout@v7
with:
# Need origin/master reachable to diff this branch's Dockerfiles
# against the pre-change baseline for the size-delta report below.
fetch-depth: 0
- name: Build agent-base (shared FROM target for both variants)
run: docker build -f docker/agent-base.Dockerfile -t roboco-agent-base .
- name: Build "after" images (this branch)
run: |
docker build -f docker/agent-qa-fe.Dockerfile -t agent-qa-fe:after .
docker build -f docker/agent-ux.Dockerfile -t agent-ux:after .
- name: Build "before" images (origin/master baseline)
run: |
mkdir -p /tmp/before
git show origin/master:docker/agent-qa-fe.Dockerfile > /tmp/before/agent-qa-fe.Dockerfile
git show origin/master:docker/agent-ux.Dockerfile > /tmp/before/agent-ux.Dockerfile
docker build -f /tmp/before/agent-qa-fe.Dockerfile -t agent-qa-fe:before .
docker build -f /tmp/before/agent-ux.Dockerfile -t agent-ux:before .
- name: Report real before/after image size delta
run: |
{
echo "### Playwright image size delta (real \`docker inspect\` sizes)"
echo
echo "| image | before | after | delta |"
echo "|---|---|---|---|"
for name in agent-qa-fe agent-ux; do
before_bytes=$(docker inspect -f '{{.Size}}' "${name}:before")
after_bytes=$(docker inspect -f '{{.Size}}' "${name}:after")
delta_mb=$(( (after_bytes - before_bytes) / 1024 / 1024 ))
echo "| ${name} | $(( before_bytes / 1024 / 1024 ))MB | $(( after_bytes / 1024 / 1024 ))MB | +${delta_mb}MB |"
done
} | tee -a "$GITHUB_STEP_SUMMARY" /tmp/smoke-report.md
- name: Headless chromium launch smoke check
run: |
{
echo
echo "### Headless chromium launch smoke check"
echo
} | tee -a /tmp/smoke-report.md
for name in agent-qa-fe agent-ux; do
echo "::group::${name} headless launch smoke"
docker run --rm --entrypoint /app/.venv/bin/python "${name}:after" -c '
from playwright.sync_api import sync_playwright
with sync_playwright() as p:
browser = p.chromium.launch()
page = browser.new_page()
page.goto("about:blank")
assert page.title() == ""
browser.close()
print("PLAYWRIGHT_SMOKE_OK")
' | tee -a /tmp/smoke-report.md
echo "::endgroup::"
done
- name: Playwright MCP server smoke check (binary + baked-chromium wiring)
run: |
{
echo
echo "### Playwright MCP server smoke check"
echo
} | tee -a /tmp/smoke-report.md
for name in agent-qa-fe agent-ux; do
echo "::group::${name} playwright-mcp binary"
docker run --rm --entrypoint playwright-mcp "${name}:after" --version | tee -a /tmp/smoke-report.md
# Confirms the exact chromium-headless-shell path the wrapper
# entrypoint (docker/scripts/playwright-mcp-entrypoint.sh) will
# resolve and pass to `playwright-mcp --executable-path` is a
# real, executable file baked into this image — not a second,
# separately-downloaded browser.
docker run --rm --entrypoint /app/.venv/bin/python "${name}:after" -c '
import os
from playwright.sync_api import sync_playwright
with sync_playwright() as p:
path = p.chromium.executable_path
assert os.path.isfile(path) and os.access(path, os.X_OK), f"not executable: {path}"
print("RESOLVED_CHROMIUM_PATH_OK:", path)
' | tee -a /tmp/smoke-report.md
echo "::endgroup::"
done
- name: Set up Node (for the panel screenshot step below)
uses: actions/setup-node@v6
with:
node-version: '22'
- name: Enable corepack (pnpm)
run: corepack enable
- name: Build & start the panel (headless browser verification target)
working-directory: panel
env:
# pnpm 11 prompts for confirmation on modules-purge unless CI=true
# (matches docker/panel.Dockerfile's build stage).
CI: 'true'
run: |
pnpm install --frozen-lockfile --shamefully-hoist
pnpm build
nohup pnpm start -p 3000 > /tmp/panel.log 2>&1 &
npx --yes wait-on http://localhost:3000/login -t 60000
- name: Headless browser verification screenshot (ux-qa image)
# Demonstrates the ux-qa image's browser actually renders a live
# panel page end-to-end, not just about:blank. /login needs no
# backend/auth (proxy.ts excludes it from its matcher) so this has
# no dependency on the orchestrator/DB being up.
run: |
mkdir -p /tmp/pw-artifacts
chmod 777 /tmp/pw-artifacts
docker run --rm --add-host=host.docker.internal:host-gateway \
-v /tmp/pw-artifacts:/tmp/out \
--entrypoint /app/.venv/bin/python agent-ux:after -c '
from playwright.sync_api import sync_playwright
with sync_playwright() as p:
browser = p.chromium.launch()
page = browser.new_page(viewport={"width": 1280, "height": 800})
page.goto("http://host.docker.internal:3000/login", wait_until="networkidle")
page.screenshot(path="/tmp/out/ux-qa-panel-login.png")
browser.close()
print("PANEL_SCREENSHOT_OK")
'
{
echo
echo "### Headless browser verification screenshot (ux-qa image)"
echo
echo "Screenshot of a live panel page (\`/login\`), taken from inside \`agent-ux:after\`. See the \`ux-qa-panel-screenshot\` build artifact."
} | tee -a /tmp/smoke-report.md
- name: Upload panel screenshot artifact
uses: actions/upload-artifact@v4
with:
name: ux-qa-panel-screenshot
path: /tmp/pw-artifacts/ux-qa-panel-login.png
- name: Post results as a PR comment
# always() so the PR comment (the only evidence-delivery path QA/PM
# has, since no agent role can read the Checks tab directly) still
# posts a partial report even if an earlier build/smoke step failed —
# an `if:` that doesn't call a status function is otherwise
# implicitly ANDed with success(), silently swallowing the report.
if: always() && github.event_name == 'pull_request'
uses: actions/github-script@v7
with:
script: |
const fs = require('fs');
const body = fs.readFileSync('/tmp/smoke-report.md', 'utf8');
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body: `## Agent Image Smoke (Playwright) results\n\n${body}`,
});