Replace the ~50-65% range and the ~50%-vs-terse figure with a single 65%
(the measured average output reduction vs verbose baseline) on every product
surface: README hero + ASCII card + benchmark blurb, plugin.json,
marketplace.json, gemini-extension.json, caveman SKILL.md + skill README,
caveman-init rule frontmatter, and the docs site (index.html telemetry widget
+ marquee still showed the old 75%). Trim HONEST-NUMBERS.md to match. Raw
benchmark/eval snapshots and the eval-harness methodology docs are untouched.
Bump PINNED_REF v1.9.0 -> v1.9.1 so the standalone hook-download fallback
fetches cavecrew-model-overrides.js from the release tag instead of 404ing.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0163QczkBHKpYgxx3sBc7X3b
/caveman-stats credited ALL output tokens since session start to whatever
mode the flag held at stats time: activating caveman late inflated the
estimate (verbose tokens back-computed as compressed), deactivating it
zeroed honest savings — and the wrong number compounded into the lifetime
history and statusline.
The flag files carried no timestamps, so past sessions cannot be exactly
reconstructed. Fix going forward + honest fallback:
- caveman-config: recordModeChange() appends {ts, mode, prev} to
.caveman-mode-log.jsonl on every actual flag transition (deduped,
symlink-safe via appendFlag, best-effort)
- mode tracker + SessionStart activate hook log every flag mutation
(set, off, NL deactivation, one-shot restore, session-start reset)
- caveman-stats joins the log timestamps against the session JSONL
message timestamps and computes savings per mode span; the first
row's prev covers the pre-inception span
- with no log coverage: a flag written mid-session means the earlier
tokens have UNKNOWN mode — they are excluded and labeled, never
guessed (no-fake-savings); with no evidence of a mid-session change,
whole-session attribution stays (correct when the mode never changed)
- history rows, --share, and the statusline suffix all use the
attributed figure; mixed sessions render a per-mode breakdown with
the estimate basis stated
Old inflated history rows cannot be retroactively corrected (the data to
re-attribute them was never recorded).
Tests: mid-session activation (inflation case), mid-session deactivation
(zeroed case), tracker transition-log dedup, unattributable-prefix
exclusion. Checksums manifest refreshed for the four changed hook files.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011kmm2umRGb5nLxdimrwweZ
The files array omitted spawn-options.js, which index.js requires at
startup — the next npm publish would have shipped a package that crashes
with MODULE_NOT_FOUND on every launch (verified via npm pack --dry-run).
- add spawn-options.js to package.json files
- bump 0.1.1 (registry 0.1.0 is stale; the fix is unpublishable without
a version bump anyway — NOT published here)
- add a static packaging test that walks every relative require reachable
from the package entry points (bin + main) and fails if any resolved
module is missing from files
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011kmm2umRGb5nLxdimrwweZ
The limit-headroom meter (8a5ab60) printed saved/(saved+output) as
'Session budget saved: ~X% of your usage this session' and 'Est. budget
saved: ~X% of tracked usage'. That ratio is algebraically the output
reduction (always ~65% in full mode) with an output-only denominator —
input + cache tokens, which dominate agentic sessions and count against
Pro/Max limits, were excluded. docs/HONEST-NUMBERS.md on this same
branch says real session-level totals land ~14-21% and below zero on
terse workloads, so the label overstated limit relief.
Fix: say only what the math computes.
- Session view: drop the budget line; the saved line now reads
'(~X% of output)' and the footer states input/cache usage is
unchanged.
- Lifetime view: relabel to 'Est. output reduction: ~X% (output tokens
only, est.)'.
- budgetSavedPct -> outputReductionPct with a comment forbidding
usage/budget relabeling; tests assert no usage/budget claim appears.
- Hook checksum refreshed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011kmm2umRGb5nLxdimrwweZ
README gets a two-line Caveman Cloud section near the bottom; the
installer summary now points at /caveman-stats (labeled estimates)
and getcaveman.dev. One link, one honest sentence each — local
numbers are estimates, Cloud measures and verifies them.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011kmm2umRGb5nLxdimrwweZ
Extends the #614 SECURITY.md scaffolding: verified zero-network-after-
install statement (skill is a prompt; hooks/stats/statusline/shrink
have no http/https/fetch), exact install-time fetch list incl. the
detached curl fallback (release-tag-pinned, SHA-256-verified), what
stays local, air-gapped/enterprise note, and honest explanations for
the Defender (#383) and Snyk caveman-compress (#28) flags. Adds a
top-level README Privacy section (#504) and corrects INSTALL.md's
'no network calls' claim to name the curl-fallback exception.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011kmm2umRGb5nLxdimrwweZ
README headline now states the measured range with method (65% avg vs
verbose default from benchmarks/, ~50% median vs 'Answer concisely.'
control from evals/) plus explicit caveat: input tokens untouched,
session-level savings smaller. Stats box shows output vs input split,
drops the unmeasured ~3x speed row. Same ~75% claim corrected in
SKILL.md description (+ synced plugin copy and dist/caveman.skill),
plugin.json, marketplace.json, gemini-extension.json, caveman-init
rule frontmatter, skill README, and CLAUDE.md.
Note for maintainer: the GitHub repo description still says 'cuts 65%
of tokens' — should read 'cuts ~50-65% of output tokens (measured)'.
Not changeable from a local commit.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011kmm2umRGb5nLxdimrwweZ
Plain-truth page: measured output reduction (65% avg vs verbose,
~50% median vs terse control), the ~1-1.5k/turn input cost of the
injected rules, net-negative cases (terse Q&A per #145, per-request
billing per #506, adverse tool-side counters per #550), and how to
A/B it yourself. /caveman-stats savings labeled as estimates.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011kmm2umRGb5nLxdimrwweZ
Claude Code Pro/Max users don't spend dollars, they spend a 5-hour /
weekly usage limit. /caveman-stats now also reports the savings in
their currency: 'Session budget saved: ~X% of your usage this session
(est.)', and the lifetime view gets 'Est. budget saved: ~X% of tracked
usage (est.)'.
Honesty rules baked in:
- % = saved / (saved + used) from tokens we actually count — nothing
else. budgetSavedPct returns null (line omitted) when no savings are
measured: honest zero, no claim.
- no plan-limit sizes are assumed or hardcoded (Anthropic doesn't
publish token quotas); the footer says so explicitly.
- clearly labeled (est.); USD lines stay for API users.
Checksums refreshed for the caveman-stats.js change. 5 new tests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011kmm2umRGb5nLxdimrwweZ
The old loader setdefault'ed EVERY key found in repo-root .env.local
into os.environ. Security scanners (Hermes skill scan, issue #528) flag
that as a high-severity exfiltration surface: install caveman into a
profile with secrets in .env.local and the benchmark quietly pulls all
of them into its process environment.
The benchmark only ever needs ANTHROPIC_API_KEY (anthropic.Anthropic()
reads it implicitly), so read that one key and nothing else — skip the
file entirely when the var is already set.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011kmm2umRGb5nLxdimrwweZ
irm .../install.ps1 | iex crashed with "Cannot bind argument to
parameter 'Path' because it is null": under iex there is no script
file, so $MyInvocation.MyCommand.Path is $null and Split-Path threw.
The top-level param() block also can't receive arguments through a
pipe.
- wrap all logic in Install-Caveman, invoked at the bottom with the
automatic $args (populated for file runs, empty under iex)
- replace $MyInvocation.MyCommand.Path with $PSCommandPath, guarded —
pipe installs skip the local-clone branch and go straight to npx
- static regression tests in tests/installer/ps1-pipe.test.mjs (CI has
no pwsh, so pin the pipe-safety contract textually)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011kmm2umRGb5nLxdimrwweZ
Claude Code only scans commands/*.md (YAML frontmatter) for plugin slash
commands — the commands/*.toml files are the Gemini extension format and
are ignored, so /caveman, /caveman-commit, /caveman-review,
/caveman-stats and /caveman-init all returned Unknown command after a
plugin install.
- add commands/{caveman,caveman-commit,caveman-review,caveman-stats,
caveman-init}.md mirroring the toml prompts ({{args}} -> $ARGUMENTS)
- caveman-stats.md body still matches the mode-tracker intercept regex
so the hook injects real numbers, model computes nothing
- caveman-init.md uses the standalone-fallback body from #603
- keep every .toml — Gemini CLI extensions only read TOML
- pin the two-format contract + hook-regex + no-{{args}}-leak in
tests/installer/slash-commands.test.mjs
The namespaced-command side of #599 (/caveman:caveman-commit etc.) is
already handled by the mode-tracker (both bare and namespaced forms,
one-shot restore) — this closes the discovery half.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011kmm2umRGb5nLxdimrwweZ
commands/caveman-init.toml told every installed Codex/Gemini user to run
node src/tools/caveman-init.js — a path that only exists inside the
caveman clone, so the command failed with Cannot find module for
everyone but caveman developers.
- toml prompt now gates the repo-relative path on the file existing and
falls back to the standalone script via curl | node -
- src/tools/caveman-init.js: the documented curl|node stdin path was
silently a no-op — require.main is undefined under node -, so the
require.main === module guard never ran main() (exit 0, no output,
no files). Guard now also matches module.id === '[stdin]'.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011kmm2umRGb5nLxdimrwweZ
PR #560 shipped the regression test but its source change never made it
into the diff — the branch only adds tests/test_mode_tracker_stdin.js.
Applied the one-line silent-exit error listener the PR describes and
took the author's test verbatim.
Co-authored-by: ousamabenyounes <ousamabenyounes@users.noreply.github.com>
--force on an AGENTS.md containing the legacy un-fenced sentinel
replaced the entire file with the fenced block — destroying all
user-authored content around the legacy block. The installer's own
hint told users with mixed files to run exactly that.
Migrate instead: back up once to AGENTS.md.bak, remove the legacy
block (exact match of the current rule body when possible, otherwise
cut from the sentinel's paragraph start — the legacy path appended the
block, so user content precedes it), and write the fenced block after
the preserved user content.
Fixes#594
A truncated/stray marker (interrupted write, partial user edit) chained
into deleting user content: appendBootstrapToSoul saw 'no complete
block' and appended a second one; stripBootstrapFromSoul then cut from
the FIRST begin to the FIRST end — spanning everything between the
stray marker and the appended block. Reported reproduction ended with
the whole SOUL.md deleted.
Replace the single-span cut with a scan that pairs each begin with the
nearest end before the next begin; unpaired markers are removed as just
the marker text, never as a span. Append now detects damaged markers
(orphans, duplicates), strips them safely, and writes one clean block.
Fixes#596
Natural-language matching (#598):
- Deactivation computed first, word-order tolerant: 'turn caveman mode
off' used to ACTIVATE caveman (and reset the level to default),
'turn caveman off' was a no-op.
- 'enable caveman and stop apologizing' no longer deactivates (the old
stop-guard fired on 'stop' anywhere, then the deactivation regex
matched 'caveman and stop').
- Questions ('what is caveman mode?') no longer arm the mode.
- 'normal mode' deactivates only as a command or with caveman context
('how do I exit vim normal mode' no longer kills the session mode).
- Prompts normalized to one line so multiline input matches.
- Scoped brevity ('be brief in the summary section') is a one-off
instruction, not a session-wide switch.
One-shot modes (#599):
- /caveman-commit|-review|-compress save the displaced prose mode to
.caveman-active.prev and the next ordinary prompt restores it (or
deactivates if caveman wasn't active before) — SKILL.md's 'level
persist until changed or session end' holds again.
- Plugin-namespaced /caveman:caveman-commit and -review recognized
(only compress and stats had the variant).
- Deactivation clears the saved prev so nothing resurrects the mode.
Fixes#598, fixes#599
The trailing-comma sweep ran a global regex over the whole
comment-stripped output, including string contents — a JSONC
settings.json with a value containing ',}' or ',]' (shell brace
expansion, inline JSON in hook args) was silently corrupted on read
and persisted corrupted on the next write.
Replace the regex with a scan that tracks string state (same approach
as the comment stripper above it) and only drops commas outside
strings.
Fixes#595
detect.py listed .dockerfile/.makefile in SKIP_EXTENSIONS, but real
files are named Dockerfile/Makefile with no extension, so they fell
through to the content heuristic and came back compressible —
/caveman-compress Dockerfile overwrote a Dockerfile with caveman prose.
Add a basename guard (Dockerfile, Makefile, Jenkinsfile, Vagrantfile,
CMakeLists.txt, ...) checked before any extension rule — CMakeLists.txt
would otherwise ride the compressible .txt rule — and a shebang check
in the extensionless branch so executable scripts are always code.
Mirror synced to plugins/caveman (sync workflow only triggers on
SKILL.md changes, so scripts/ must ride along).
Fixes#600
removeCavemanHooks stripped any settings.json hook whose command
contained the substring 'caveman' anywhere — a user-authored hook like
'node ~/Projects/caveman-notes/my-hook.js' was silently deleted by
--uninstall.
Match tokens against MANAGED_HOOK_BASENAMES by exact basename instead
(win32.basename so Windows-written configs match anywhere), the same
pattern pruneOrphanedManagedHooks already uses. Hoist the tokenizer to
module scope and reuse it in the prune pass. Add caveman-statusline.ps1
to the managed set so the Windows statusline wiring is covered by
removal and orphan-pruning too.
Fixes#593
spawnSync reports a missing binary as { status: null, error }, and the
(r.status || 0) === 0 checks coerced that null to success. On a machine
without the claude CLI, --only claude printed 'installed: claude',
skipped the standalone-hook fallback (which works offline), and left
nothing installed. Same pattern at 8 sites: claude, gemini, npx-skills
providers, mcp-shrink, runInit (both paths), uninstall, skills-auto.
Route every spawn result through spawnOk() (!r.error && r.status === 0)
and warn when the claude CLI itself could not be spawned. Regression
test runs the installer with an empty PATH and asserts failure is
reported and standalone hooks get wired.
Fixes#592
Hook lives at <plugin_root>/src/hooks/ but read SKILL.md via a single
'..' — resolving to nonexistent src/skills/, so every plugin install
silently fell back to the stale hardcoded ruleset (missing language
preservation, no-self-reference, intensity table).
Resolve via candidates in order: $CLAUDE_PLUGIN_ROOT/skills/,
__dirname/../../skills/ (plugin + repo layout), __dirname/../skills/
(standalone $CLAUDE_CONFIG_DIR layout). Sync the two missing rules
into the fallback for installs with no SKILL.md at all.
Fixes#587, fixes#589
This fixes the edge case for the following error by instantiating the
paths differently.
✘ Failed to install plugin "caveman@caveman": EXDEV: cross-device link not permitted
withProtectedSegments built sentinels in a single pass and restored them
in a single pass. When two PROTECTED_PATTERNS matched the same span
(e.g. path rule swallows STARTER/BUSINESS, then function-call rule
swallows the resulting type ( 0 )), the outer sentinel restored to
"type ( 0 )" but the inner " 0 " was never substituted back — enum
values reaching the model became literal "( 0 )".
Restore now loops with MAX_RESTORE_PASSES bound; same depth is reached
on the actual #444 inputs in <=2 passes.
Co-Authored-By: Claude <noreply@anthropic.com>
Adversarial review against a live Hermes Agent install (v0.11.0) found the
forward install was correct (skills land in the real ~/.hermes/skills/
productivity/<skill>/ layout, all 7 load as 'enabled' via 'hermes skills
list' — verified empirically, no version: field required), but uninstall had
NO Hermes handling: --uninstall silently orphaned all 7 skill folders forever.
- add Hermes block to uninstall() honoring HERMES_HOME (mirrors opencode/openclaw)
- tests/installer/hermes.test.mjs: install lands 7 skills, uninstall removes
them (regression guard for the asymmetry), dry-run uninstall is a no-op
- INSTALL.md: add Hermes Agent row to the per-agent install table (CLAUDE.md
mandates the install table stay complete)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- restore 2-space PROVIDERS indent and 4-space dispatch-loop indent (PR
re-indented existing gemini/opencode/openclaw lines as churn)
- correct mech label: 'native hermes skills copy' (it does a native dir
copy via installHermes, not 'npx skills add')
- drop dead profile: 'hermes' — shadowed by the installHermes special-case
and not a valid upstream vercel-labs/skills slug
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PR #527 added cavecrew-model-overrides.js with a wrong sha256 (eae3... vs
actual 9f26...) and modified caveman-activate.js without regenerating its
entry. bin/install.js verifies downloaded hooks against this manifest and
aborts on mismatch, so both were latent install-breakers on the remote path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds Hermes Agent as a supported provider with native skill installation.
Hermes Agent uses SKILL.md format (same as upstream), so the installer
copies all 7 caveman skills directly to ~/.hermes/skills/productivity/.
Closes#45914 (related)
github-actions[bot]