- SKILL.md gains a 'Discover the Stack First' step: identify the
language, build system, checked-in wrappers, test framework, and
documented commands before choosing any test command, and run the
repository's own focused/full-suite commands throughout the loop
- verification checklist no longer hardcodes 'npm test'; red flag added
for reaching for a default test command without checking the repo
- references/testing-patterns.md relabeled in place as JavaScript/
TypeScript examples illustrating the universal principles (no
relocation, per the #361/#236 sequencing agreed in the issue)
- new behavioral eval: a Python/unittest fixture verifying the skill
adapts to non-npm repository tooling
There was no way to measure whether skills trigger correctly, stay
distinct, or change agent behavior. This adds evals, aligned with what
the community has converged on, with a deterministic CI tier on top:
- evals/cases/<skill>.json for all 24 skills. The evals[] block uses
Anthropic skill-creator's evals.json schema verbatim (id, prompt,
expected_output, expectations[]) so its runner, benchmarks, and eval
viewer work against our files unmodified. A trigger block (this
repo's extension) adds positive/negative routing prompts per skill.
- scripts/run-evals.js, zero-dependency runner:
Tier 2 (CI): trigger evals via stemmed TF-IDF ranking over skill
descriptions (positive prompts must rank top-k, negative prompts
must not rank first), catalog collision detection between skill
descriptions, schema and coverage checks.
Tier 3 (opt-in): --behavioral <skill> executes each eval through
headless claude -p and grades the transcript against expectations[]
(superpowers-style); --dry-run previews without spending tokens.
- CI: run the deterministic tier in the validate-skills job.
- Docs: evals/README.md defines the framework and prior art;
CONTRIBUTING requires an eval file for new skills (warning-level in
the runner until in-flight skill PRs clear); CLAUDE.md pointers.
Current baseline: 120 checks pass, 85% trigger rank-1 rate across 72
positive prompts, zero catalog collisions.
Adds a pair of optional Claude Code hooks that cache WebFetch output
on disk but revalidate every reuse against the origin. Content is
served only when the server returns 304 Not Modified, so
source-driven-development's "verify against current docs" guarantee
still holds across sessions.
- hooks/sdd-cache-pre.sh: PreToolUse hook. For a cached entry, issues
a HEAD with If-None-Match / If-Modified-Since. On 304, blocks the
WebFetch (exit 2) and returns cached content via stderr; otherwise
allows the fetch through.
- hooks/sdd-cache-post.sh: PostToolUse hook. Captures response plus
current ETag / Last-Modified. Entries without a validator are
never stored — without one, the pre hook cannot verify freshness
and caching would amount to trusting memory.
- Cache key: sha256(url + normalized_prompt). Prompt is lowercased
and whitespace-collapsed so stylistic variants hit the same entry;
semantically different prompts still miss.
- Hard 24h TTL as a safety net against misbehaving origins.
- hooks/SDD-CACHE.md: opt-in setup, end-to-end testing, debugging.
- .gitignore: ignore the .claude/sdd-cache/ directory.
Hooks are opt-in: users register them in .claude/settings.json. The
source-driven-development skill itself is unchanged.
Add MIT LICENSE file, CONTRIBUTING.md with skill quality guidelines,
expand .gitignore for safety, and remove leftover idea-refine.zip artifact.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ayobamiseun