Gives contributors a short intake path when a skill assumes the wrong
ecosystem or outdated steps, so gaps like #404 reach maintainers via
the repo instead of external channels. Closes the intake-only scope
from #412.
Co-authored-by: Cursor <cursoragent@cursor.com>
Add docs/developer-onboarding.md: a guided tour for people working on the
repo itself (the five layers, local setup, the verification loop, the
contribution paths, and a suggested reading order), complementing the
authoritative rules in CONTRIBUTING.md, skill-anatomy.md, and evals/README.md.
Link it from the top of CONTRIBUTING.md as the map to its rulebook.
State in CONTRIBUTING.md that translations of docs and skills are not
accepted: translated copies drift as content evolves and can't be
maintained long-term without leaning on agent translations plus
community corrections, for limited value.
Add a Scope banner to AGENTS.md and CLAUDE.md stating they configure
agents working on the addyosmani/agent-skills repository itself, not
users' own projects, referencing the repo by its canonical GitHub URL
so the scope is unambiguous when the file is read out of context. Add a
matching "Repo-scoped files" note to CONTRIBUTING.md so setup-guide
authors don't instruct users to copy these files.
Wire the framework docs to the tracking issues: #351 (description
vocabulary gaps) and #352 (Tier 3 graduation + deterministic ratchets),
so warning-level checks have an explicit promotion path instead of
becoming permanent.
There was no way to measure whether skills trigger correctly, stay
distinct, or change agent behavior. This adds evals, aligned with what
the community has converged on, with a deterministic CI tier on top:
- evals/cases/<skill>.json for all 24 skills. The evals[] block uses
Anthropic skill-creator's evals.json schema verbatim (id, prompt,
expected_output, expectations[]) so its runner, benchmarks, and eval
viewer work against our files unmodified. A trigger block (this
repo's extension) adds positive/negative routing prompts per skill.
- scripts/run-evals.js, zero-dependency runner:
Tier 2 (CI): trigger evals via stemmed TF-IDF ranking over skill
descriptions (positive prompts must rank top-k, negative prompts
must not rank first), catalog collision detection between skill
descriptions, schema and coverage checks.
Tier 3 (opt-in): --behavioral <skill> executes each eval through
headless claude -p and grades the transcript against expectations[]
(superpowers-style); --dry-run previews without spending tokens.
- CI: run the deterministic tier in the validate-skills job.
- Docs: evals/README.md defines the framework and prior art;
CONTRIBUTING requires an eval file for new skills (warning-level in
the runner until in-flight skill PRs clear); CLAUDE.md pointers.
Current baseline: 120 checks pass, 85% trigger rank-1 rate across 72
positive prompts, zero catalog collisions.
Add a "Before proposing a new skill" subsection to CONTRIBUTING.md that
defines a checklist contributors run before opening a new-skill PR: search
the existing catalog, check open PRs for overlapping proposals, confirm the
idea fits the skill anatomy, and justify in the PR description why it isn't
already covered.
The goal is to set clear rules/guidelines that curb duplication, since the
open-PR backlog has accumulated clusters of near-duplicate skill proposals.
Reorganize the section into "Before proposing a new skill" and "Creating the
skill" so the decision step comes before the how-to.
Add MIT LICENSE file, CONTRIBUTING.md with skill quality guidelines,
expand .gitignore for safety, and remove leftover idea-refine.zip artifact.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
