The generated site's whole value is that drift is loud, so the tests
are mostly about drift: a heading renamed in AGENTS.md must fail the
build naming the route and the heading, a slice emptied to its heading
must fail rather than emit a page, and a heading inside a code fence
must never match — AGENTS.md fences a task template whose first line is
'# Task title', and matching that would slice the document in half.
Alongside: link rewriting (GitHub blob, link_routes overrides, dead and
escaping links refused), manifest validation, the output-directory
guard, and the acceptance criterion that no built page fetches anything
third-party — mechanised as 'every rel that opens a connection is
same-origin', with rel=canonical excluded because it is a statement,
not a fetch.
The release artifact test now asserts site/ is absent from bench.tar.gz.
The manifest already excludes it by saying nothing about it, but
correct-by-omission is exactly what a future manifest edit undoes
silently.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
install.py shipped mode 644 in v0.1-alpha, so the README one-liner's
./install.py was permission-denied on every install. The repo modes and
update.sh's repair line were hotfixed already (all 14 shebang'd tracked
files are 100755; update.sh:165 names install.py) — this is the guard
that keeps them that way, and the patch release that heals the field.
- tests/test_release_artifact.py: the invariant, read from the tar
header rather than the repo — every member whose content starts `#!`
must carry the exec bit, failing by name. No exception list: there is
no shipped file that legitimately may not be run, and gaining one
means editing the test with a reason. Two tests keep it honest: the
guard is proven to bite by repacking the real artifact with
install.py's mode stripped, and an unpacked release must run
./install.py as a program, not via python3.
- tests/test_update_from_release.py: an install whose install.py is
mode 644 — the shape v0.1-alpha left in the field — is executable
again after any update. `cp` onto an existing file keeps the
destination's mode, so the chmod line is the only thing healing it;
removing that line fails this test.
- manager/core/VERSION → 0.1-alpha.1: cutting the patch is the honest
move over a release note telling users to work around it.
- manager/core/release-manifest: the invariant, stated where the
shipping list lives.
Verified: python3 -m unittest discover -s tests (267 tests, OK). Both
new assertions were watched failing first — a build-side `chmod -x` on
the staged install.py, and update.sh with install.py dropped from its
chmod list — then restored.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Resolutions honouring both sides:
- CLAUDE.md: main's pointer form wins; this branch's release-era wording
(tree diagram line, Updating section) ported into AGENTS.md where the
brief now lives.
- update.sh: this branch's release-based rewrite wins; main's additions
to the old file list (AGENTS.md from task 13, LICENSE) are carried by
the release-manifest instead: added copy AGENTS.md and copy LICENSE.
- Starter local/: manifest and release.sh now seed the split pair
(AGENTS.md notes + pointer CLAUDE.md) mirroring task 13's layout;
artifact test updated to match.
- tests/test_update_round_trip.py (main's, aimed at the removed
git-clone mechanism) retired; its two 13-specific intents ported into
test_update_from_release.py as
test_agents_brief_replaces_an_old_vendor_named_copy, its third already
covered there. 147 tests green.
- release.sh's lost executable bit restored.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Refuse assets whose tar members or manifest copy paths reach outside
their root, checked before anything is replaced; create parent dirs for
future nested copy entries. Test both refusals hermetically. Replace
setUpClass bare asserts with explicit checks that survive python -O.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both stops precede any gh call, so they run hermetically in a scratch
git repo carrying only release.sh and a VERSION.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The repo is two things at once — bench-the-project and bench-the-
distribution — and install/update conflated them by cloning the repo
and subtracting what should not have come along. Invert it: one
curated artifact, correct by construction.
- manager/core/release-manifest: the whole shipping list in one place
(copy/tree/once/keep/seed classes). update.sh's hardcoded top-level
file list, promoted.
- release.sh: builds bench.tar.gz from the manifest (contents at the
tarball root, stable asset name — the tokenless latest-release URL
depends on both), stamps the source repo into the shipped update.sh,
refuses on dirty tree or existing tag, tags v<VERSION> and publishes
via gh release create. Never ships in the artifact.
- update.sh: downloads the latest release (BENCH_REF pins a tag) via
gh with an anonymous curl fallback; refuses when the asset's VERSION
disagrees with its tag; replaces manager/core/ wholesale plus the
artifact manifest's `copy` files; touches nothing else. No release
published -> says so and changes nothing; no silent git fallback.
- tests: the tarball equals exactly the manifest (no cards, no local/
content beyond the generated starter, no state/tests/.claude), the
artifact installs pristine and boots the board, and update.sh's
replace/survive/refuse paths run hermetically against PATH-stubbed
gh and curl.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
