4 Commits

Include renames

Author	SHA1	Message	Date
Violin	3807d22eaa	fix(benchmark): evidence-gated scorer contract, false-positive discriminators, runner validity ... - Confirm Validated hypotheses via canonical FIND-NNN.md matched in live-app terms (METHOD + endpoint route) alongside execution bundles - require/require_any discriminators for shared-endpoint challenges; absence-type scoring for no-rate-limiting - Runner: closeout timeout is a soft warning, not run invalidation - ai_judge: whitelist legitimate closeout artifacts in schema-drift audit - Align version surfaces to 3.1.0; move pyyaml to runtime deps Calibration: known-good 20/20, known-bad 0/0. Latest run re-scored 4 -> 9.	2026-08-10 20:52:02 +01:00
Violin	48119a64cf	feat(benchmark): implement proof verifier, AI judge refinements, and duck-store calibration evidence	2026-08-10 08:54:35 +01:00
Violin	4b21dce94a	feat: harden benchmark evaluation and guard runtime	2026-08-08 13:29:11 +01:00
3 people	3b21e0a92a	feat: Automated Benchmark Runner and Evaluation Framework (#68) ... * fix: bump distribution version to 3.0.1 * feat(benchmark): introduce benchmark runner engine, Docker containerization, and calibration datasets * Fix URL/path scope exclusions and update duck-store template - Modified targets.py to ensure that excluding a specific URL or path does not result in the entire host being blocked. Implemented a strict command payload check for excluded endpoints. - Updated duck-store benchmark scope template to explicitly block the /vulnerabilities endpoint (which leaks intentional challenges) while allowing access to endpoint documentation. * fix: correct scope.yaml indentation and refine guard blocking messages * fix: resolve ModuleNotFoundError when score.py is executed directly --------- Co-authored-by: Dan <dan@strategicautomation.local>	2026-08-08 13:29:11 +01:00