mirror of
https://github.com/affaan-m/ECC.git
synced 2026-09-08 07:37:48 +02:00
Affaan MustafaandClaude Fable 5.1
20244f51b1
Port the first deployable slices of the harness engineering plan v2 into the ECC eval harness as runnable Node frameworks under scripts/lib/eval-harness/, with a CLI, tests, and an end-to-end example. - Envelope: capsule-envelope/v1 contract (schemas/capsule-envelope.schema.json), five lineages, effect classes SE0 to SE4, default-deny payload allowlist, secret canaries. - Capsule: append-only NDJSON journal with sha256 predecessor links; verify fails at the exact entry for tamper, truncation, reordering, and a partial trailing write; byte-stable projection; minimal export bundle. - Gate: smoke, selection, and held-out stages with the checker outside the candidate's sandbox, syntactic tripwires (hidden network, process spawn, sandbox weakening, checker probing, parent escape, effect-class expansion), a random sandbox marker, and a promotion receipt with regressions and a rollback target. A reward-hack fixture that scores as high as the honest candidate is rejected. - Replay: declared determinism and effect class per tool, content-addressed fixtures, fail-closed tool.fixture_missing and tool.effect_forbidden, and a --require effect fence that blocks network modules, fetch, and writes outside the sandbox root while logging every attempt. - Receipt: offline receipt over capsule root, entry count, artifact digest, and gate receipt with a detached signature interface; verification names the failing check, including stale checkpoints. Adds scripts/eval-harness.js (npm run harness:eval), examples/eval-harness with a baseline, an honest candidate, and a reward-hack variant, docs/architecture/eval-harness-frameworks.md, a Runnable Frameworks section in skills/eval-harness/SKILL.md, and 35 tests under tests/lib/eval-harness. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016UHxskuJiMLydwLrxDRqwy