The runs archive copied a fixed filename list, index.html, tokens.css,
styles.css, and stampPresent scans that archive. A swept build whose stamp
lives in page.css scored skillLoaded=false with a correct stamp on disk,
which is the same stale assumption already fixed in stampPresent and in the
reference budget, one function lower. The archive now takes every top-level
html and css the build produced.
The three affected run.json files are rescored from the real artifacts:
only the buggy cell changed (inspo false to true); the bare control stays
false, which is the leak check holding. Also archives the second three-arm
run: bare 56 FAIL with the same six numbered eyebrows, both Hallmark arms
clean, and the control now provably offered zero MCP tools.
b1 names no theme, so it must derive, and it did: direction.md was the first
reference read, the stamp carries a real seed and a five-line direction
contract, and the derived world (the IPA vowel quadrilateral, plotted, on a
speech-lab plate) reaches the actual hero rather than stopping at the stamp.
Twelve reference reads, zero waste, in the right order. R.1's rejection is
legible in the genre note: it refused the radial blooms the atmospheric genre
would have allowed, because the reflex check had already spent that move.
b7 passes --fast and must not derive, and did not. It read two theme files and
never opened direction.md. No seed, no contract, no slate, no draw line. The
posture line still emits, which is the color.md repair working: posture is
universal now, not a derived-run field.
Both artifacts lint clean at 0 FAIL, 0 REFLEX, 0 WARN.
Three harness bugs, all of them the flip's doing. stampPresent scanned a fixed
filename list of index.html, styles.css and tokens.css, and a build that named
its stylesheet page.css scored skillLoaded=false with a correct stamp sitting in
it; its regex also demanded Hallmark immediately after the comment open, so
"/* Formant Studio · Hallmark derived system" missed too. It now scans every
emitted css and html and matches the marker anywhere in a leading comment.
The reference-read budget was ten, set when a run read one theme file. A derived
run reads direction.md and theme-axes.md in its place, so the floor moved up by
one, and a clean derived build measures twelve. Fourteen leaves room for two
conditionals and still catches the thing the check is for, which is defensive
pre-loading of a forty-file tree.
The cost cap was three dollars, hardcoded. A derived build on Opus lands near
three and stopped mid Step-7, which scored a complete artifact as
error_max_budget_usd and never exercised the gate sweep at all. It is --budget-usd
now, defaulting to five.
b7 also joins the brief set: the suite had no coverage of the catalog fast path
after that path stopped being the default.
Two mechanisms Phase 1 could not supply, both conditional, neither able to block.
Step 5.5 renders the first viewport before composing it in CSS. It fires only
when TOGETHER_API_KEY is present (signal 9); with no key the step does not exist
and nothing else changes. Two comps at 1280x800 built from the direction
contract rather than the brief, varied on one compositional axis so the
comparison means something, one approval point, then build. The prompt carries
the scene sentence, the paper and accent as plain colour words, the display
class rather than a face name, and one negative clause pushing off the category
default, because a prompt that reads like the brief returns the category default
in pixels and that is what R.1 already spent its budget refusing.
The comp gets no vote on the Floor. An image generator will hand back eyebrows,
a gradient ramp and a fake browser window because it has read none of the gates.
Same rule as an archive exemplar: take the composition, not the compliance.
Reproduction is the part that makes the step worth having. A model reliably
believes its recreation succeeded when it did not, and the belief is what stops
it looking, so the built hero is screenshot at the comp's own dimensions and
answered in writing on mass, interval and entry before anything below the fold
exists. Diverging from the comp deliberately is fine. Diverging without noticing
is the failure.
The medium inventory sits under it: every region gets a written implementation
medium before any of it is built, and the gate is that a CSS gradient is not a
texture. Writing CSS against woven cloth or printed grain is how a committed
comp silently becomes a flat page with the same section order.
Step 7 gains a reviewer with no inherited transcript. The thread that walks the
gates is the thread that formed every opinion it is grading and reads the page
as the thing it meant rather than the thing it made, which is the weakest
reviewer available and the one every build has used. The new one gets the
artifact, the screenshots, the contract and the Floor list, and nothing about
why. It answers the question no gate can reach: does the direction contract
describe the page in front of it. Every gate is local, so a page can pass all
fifty-eight while quietly not being what its own stamp claims.
Findings are then scored resolved, partial or unresolved rather than
re-reviewed, unresolved is a legitimate outcome, and two rounds is the ceiling.
Off Claude Code the review runs in-thread and says so, because the user is
entitled to know which one they got.
Read-set cost for a run with no key and no subagents: zero. comp.md loads on
signal 9 only, and the reviewer brief lives in slop-test.md, which already
loaded at Step 7.
The eval run outputs, the internal design proposals, and a scratch QC
page for the imagery kit were all riding along in the PR while being
linked from nothing. They come out of tracking: eval/runs (the four A/B
cells from this session plus the older model-comparison artifacts, all
of it evidence rather than product), site/_proposals (44 files of
before/after and theme-exploration pages, preserved outside the repo),
and site/imagery/_qc.html (a scratch grid I used to eyeball the
generated assets). The eval harness itself stays so the tests are
re-runnable, and .gitignore now covers the pycache and the QC pages that
slipped past it.
The four theme examples added this round stay: arcade-01, field-01,
grid-01, and ledger-01 are the worked examples their own theme docs link
to, not stray test output.
Two briefs (voice-cloning studio, pre-traction B2B SaaS) built twice
each, once with no skill and once running the Hallmark flow, same model
and same subscription. Artifacts, run records, and the skill's own state
files land under eval/runs so the comparison is auditable rather than
asserted: the two control pages carry 95 sloplint failures between them
(off-scale spacing, unowned colour literals, pure #fff surfaces,
layout-property transitions, no provenance stamp), the two skill pages
carry none, at roughly a third more tokens and a tenth more wall-clock.
Both arms respected the pre-traction brief's honesty constraint, so that
half of the test was a wash and each run.json records it that way, along
with the runner substitution and the effort setting actually used.
Running the harness from inside a Claude Code session leaks
ANTHROPIC_BASE_URL and the CLAUDE_CODE_* host vars into the spawned
claude, which then reports not-logged-in even in an authenticated
terminal. The child env now drops them alongside the API key. Also
removes the six dead cells that run recorded: headless subscription auth
needs a terminal login this environment cannot perform, so the A/B
results were produced with in-session subagents instead (disclosed in
the PR).
gen-cli.mjs learns --model and --effort (forwarded to the claude call),
a bare no-skill control arm (plain brief prompt, no skill copy), a $3
per-cell budget guard, and --setting-sources project on BOTH arms so the
user-scope hallmark install can neither leak into the control nor
version-skew the skill arm; stampPresent on a bare cell doubles as the
leak detector. The result event's usage lands in run.json (tokens
in/out/cacheRead, turns, stop subtype) with the camelCase modelUsage
fallback, and cells are named arm-effort-cli so effort variants coexist.
Unify every version string: skill frontmatter and preview block, package.json,
the four preview-examples samples, the eval direction note, the ROADMAP opener,
and the site badges/captions (which were still stamped v1.1/v1.3). The PR is
retitled to match and sits as a draft on the renamed v1.2 branch.
Hallmark stops targeting open-weight models and goes deep on the three
terminals where design work happens: Claude Code, Codex CLI, and OpenCode.
Same core everywhere (SKILL.md standard), plus a per-harness adapter each
harness loads before Step 0.
- references/harnesses/codex.md: $hallmark invocation, workspace-write
sandbox + network-off reality (picker server asks first, --static
fallback), no hooks/subagents (sequential variants + progressive reveal),
AGENTS.md as a Step 0 signal, batch-per-direction patches
- references/harnesses/opencode.md: /hallmark + skill-tool invocation,
allow/ask/deny permission model, parallel variants via its subagents,
formatter/LSP interplay, compatibility-path discovery
- references/models.md deleted; its harness-agnostic rules live on in both
adapters. SKILL.md dispatch now routes by harness, not by model family
- agents/openai.yaml: Codex interface + implicit-invocation policy
- scripts/install.mjs: detects installed harnesses and installs a lean
1.5MB payload (SKILL.md, references, scripts, agents, LICENSE - never
site/ or eval/) to ~/.claude/skills, ~/.agents/skills (+ refreshes the
stale legacy ~/.codex/skills copy), and leaves OpenCode to its
compatibility discovery; --dry-run / --remove / idempotent
- README: 'Open models' section replaced by 'One skill, three harnesses'
with the install matrix; ROADMAP and eval/README reframed (Tier B kept
as archive only); version 1.4.0
Ran the installer live: all three locations now serve 1.4.0 with the
adapters in place (the Codex location previously held a 27MB v1.0.0 copy).
Runs 'claude -p "/hallmark <brief>"' with the skill copied into a scratch
project, parses the stream-json transcript for conformance (skill loaded via
stamp, files written, reference-read count + load-order, cost), writes into
runs/<brief>/<arm>-cli/. Skills do not auto-trigger headless (measured), so
it invokes by name: a conformance harness, not an auto-trigger test. Claude
arm uses the subscription (no key); GLM/Kimi arms Anthropic-shaped and
disabled until their auth tokens exist. Plumbing validated against the real
binary here (spawn, stream-json parse both event shapes, run.json); a real
run needs a logged-in terminal.
- five-themes cards open a complete full-page mockup per theme
- before/after cards open the actual generated pages (before from
hallmark-lab, after from the eval runs), copied into _proposals/pages
- eval judge defaults to operator mode: no Anthropic key spend, the
Claude Code operator scores from the screenshots; --provider together
or anthropic for an automated pass
Full-pack matrix on Claude, GLM (Together), and the Kimi stand-in:
zero mechanical FAILs everywhere after at most one repair round; the
floor-only pack collapses on open models (96 FAILs on GLM), so the full
skill is load-bearing, not bloat. No-vision probe: every model declined
cleanly. Zero invented metrics. Rotation held per arm.
judge.py: Together vision fallback chain + User-Agent fix; vision judge
itself pending API credit or an OpenRouter key. gen-direct: 64K output
ceiling (Claude needed 37K on one brief).
- one canonical top-of-file order: stamp, critique line, direction contract
- theme-axes.md: the 20-row Rotation lookup now ships inside the skill
- tier taxonomies unified by name; craft tiers keep construction letters
- S1 reworked to stacked-numbered (the old side-column head was gate 54's
own auto-fail); carnival + lumen examples brought up to gates 54/55/24;
sloplint: mask/filter exemption on gate 22, precision fix + FAIL grade
on gate 54, 15-line stamp window
- variants playbook matches the scripts byte for byte (request ids, riff
shape, fallback wording, flow order); draft-sweep exceptions written
into slop-test, custom-theme, and SKILL
- study.md: italic roles diagnosed as-is, built roman; preview-examples
regenerated for v1.2
Arms: Anthropic baseline, GLM on Together, Kimi on OpenRouter (DeepSeek
and Qwen staged, disabled). Six briefs incl. metric-temptation,
component-scope, and vision-probe probes. Floor-only vs full skill packs
to measure prompt bloat. run-matrix orchestrates gen, shots, score,
judge, report; the repair loop feeds sloplint FAILs back to the model.
Youssef