sebastianbreguel 2b10489888 Add token-compression eval harness with three-arm methodology ...

Closes #18 by replacing the unverified "~75% savings" claim with a real,
auditable measurement.

Methodology:
- Three arms per prompt: baseline (no system prompt), terse control
  ("Answer concisely."), and terse+SKILL.md. The honest delta is
  skill vs terse — this isolates the skill's contribution from the
  generic "be terse" effect.
- Real LLM in the loop via `claude -p --system-prompt`. No hand-written
  baselines, no circularity.
- Snapshot of LLM outputs committed to git as the source of truth.
  measure.py runs in CI with no network and no auth.
- Reports median, mean, min, max, stdev across prompts so noise is visible.
- Metadata pinned in the snapshot: model, CLI version, generation timestamp.

Initial run on claude-opus-4-6 (n=10 prompts) shows the real numbers
sit in the −22% to −49% mean range, not the previously claimed ~75%.
The README's headline number should be updated to match.

2026-04-08 18:20:09 -04:00

..

en.txt

Add token-compression eval harness with three-arm methodology

2026-04-08 18:20:09 -04:00