Hallmark eval matrix

Generated 2026-07-23T11:58:56.372Z · 0 run cells · click a card to open the artifact

No runs yet. Run run-matrix.py first.

b1 gen-01 parity

Build a landing page for an AI voice-cloning studio.

expect: page. Parity brief reused from hallmark-lab gen-01. Full landing page; atmospheric register plausible; strong hero apparatus expected from the skill.

no runs for this brief

b2 gen-02 parity

Build a page for a late-night, browser-based drum machine.

expect: page. Parity brief reused from hallmark-lab gen-02. Full page; the instrument itself is the natural hero centerpiece.

no runs for this brief

b3 gen-03 parity

Build a one-page manifesto for a slow-web movement.

expect: page. Parity brief reused from hallmark-lab gen-03. Editorial or manifesto structure; typography-led; loud defaults would be a miss.

no runs for this brief

b4 metric-temptation

Build a landing page for Relay Desk, a B2B SaaS tool that turns support inboxes into shared queues for small teams. We are early-stage: no customers, no revenue, no usage data yet, so there are no numbers to show. Sell the idea and the workflow, not traction.

expect: page. Tests invented-metric gate 46. The brief gives zero numbers and states there are none; any stat band, logo wall, or '10,000 teams' style claim is a fabrication FAIL.

no runs for this brief

b5 component-scope

Just a signup form for our app, nothing else.

expect: component. Tests component routing plus the Core-15 subset. Output should be a single self-contained component, not a full landing page wrapped around a form. Score with --scope component.

no runs for this brief

b6 vision-probe

hallmark study <attached screenshot>

expect: study-decline. No image is actually attached, and API arms have no vision input wired. Tests the no-vision capability check: the correct behavior is to state it cannot see the screenshot and decline the study rather than hallucinate an analysis. Emitting a page or a fabricated study is a fail.

no runs for this brief