6 Commits
Author SHA1 Message Date
Youssef de6d5a4903 One release: everything on this branch is v1.2
Unify every version string: skill frontmatter and preview block, package.json,
the four preview-examples samples, the eval direction note, the ROADMAP opener,
and the site badges/captions (which were still stamped v1.1/v1.3). The PR is
retitled to match and sits as a draft on the renamed v1.2 branch.
2026-07-30 16:08:57 +01:00
Youssef 30c55128b2 v1.4.0: the harness pivot - one core, three first-class harnesses
Hallmark stops targeting open-weight models and goes deep on the three
terminals where design work happens: Claude Code, Codex CLI, and OpenCode.
Same core everywhere (SKILL.md standard), plus a per-harness adapter each
harness loads before Step 0.

- references/harnesses/codex.md: $hallmark invocation, workspace-write
  sandbox + network-off reality (picker server asks first, --static
  fallback), no hooks/subagents (sequential variants + progressive reveal),
  AGENTS.md as a Step 0 signal, batch-per-direction patches
- references/harnesses/opencode.md: /hallmark + skill-tool invocation,
  allow/ask/deny permission model, parallel variants via its subagents,
  formatter/LSP interplay, compatibility-path discovery
- references/models.md deleted; its harness-agnostic rules live on in both
  adapters. SKILL.md dispatch now routes by harness, not by model family
- agents/openai.yaml: Codex interface + implicit-invocation policy
- scripts/install.mjs: detects installed harnesses and installs a lean
  1.5MB payload (SKILL.md, references, scripts, agents, LICENSE - never
  site/ or eval/) to ~/.claude/skills, ~/.agents/skills (+ refreshes the
  stale legacy ~/.codex/skills copy), and leaves OpenCode to its
  compatibility discovery; --dry-run / --remove / idempotent
- README: 'Open models' section replaced by 'One skill, three harnesses'
  with the install matrix; ROADMAP and eval/README reframed (Tier B kept
  as archive only); version 1.4.0

Ran the installer live: all three locations now serve 1.4.0 with the
adapters in place (the Codex location previously held a 27MB v1.0.0 copy).
2026-07-30 12:08:56 +01:00
Youssef 272a55062a Tier A harness: gen-cli.mjs drives the real Claude Code binary
Runs 'claude -p "/hallmark <brief>"' with the skill copied into a scratch
project, parses the stream-json transcript for conformance (skill loaded via
stamp, files written, reference-read count + load-order, cost), writes into
runs/<brief>/<arm>-cli/. Skills do not auto-trigger headless (measured), so
it invokes by name: a conformance harness, not an auto-trigger test. Claude
arm uses the subscription (no key); GLM/Kimi arms Anthropic-shaped and
disabled until their auth tokens exist. Plumbing validated against the real
binary here (spawn, stream-json parse both event shapes, run.json); a real
run needs a logged-in terminal.
2026-07-24 14:42:13 +01:00
Youssef 3e6fe26e73 Proposal pages: clickable full-page views + generated-page browser
- five-themes cards open a complete full-page mockup per theme
- before/after cards open the actual generated pages (before from
  hallmark-lab, after from the eval runs), copied into _proposals/pages
- eval judge defaults to operator mode: no Anthropic key spend, the
  Claude Code operator scores from the screenshots; --provider together
  or anthropic for an automated pass
2026-07-24 10:17:20 +01:00
Youssef 01386239f3 Eval run 2026-07-23: findings, judge fallback chain, output-ceiling fix
Full-pack matrix on Claude, GLM (Together), and the Kimi stand-in:
zero mechanical FAILs everywhere after at most one repair round; the
floor-only pack collapses on open models (96 FAILs on GLM), so the full
skill is load-bearing, not bloat. No-vision probe: every model declined
cleanly. Zero invented metrics. Rotation held per arm.

judge.py: Together vision fallback chain + User-Agent fix; vision judge
itself pending API credit or an OpenRouter key. gen-direct: 64K output
ceiling (Claude needed 37K on one brief).
2026-07-23 14:41:11 +01:00
Youssef faf7d56233 Eval harness: direct-API bench matrix, screenshots, sloplint + vision-judge scoring, compare gallery
Arms: Anthropic baseline, GLM on Together, Kimi on OpenRouter (DeepSeek
and Qwen staged, disabled). Six briefs incl. metric-temptation,
component-scope, and vision-probe probes. Floor-only vs full skill packs
to measure prompt bloat. run-matrix orchestrates gen, shots, score,
judge, report; the repair loop feeds sloplint FAILs back to the model.
2026-07-23 13:08:45 +01:00