VIOLIN_BENCHMARK_RECEIPT_KEY -> VIOLIN_RECEIPT_KEY. The all-caps
'BENCHMARK' slipped past the original case-sensitive de-cheat grep in
plugins/violin_guard. No consumers hardcode the literal; all use the
RECEIPT_KEY_ENV constant.
Run 184313's agent canonized findings but never wrote the forward
'Linked findings' board field, so the scorer lost the confirmations.
The closeout goal now permits bookkeeping-only updates: link each
Validated hypothesis to its FIND-*.md via violin_record_hypothesis.
No new testing, findings, or evidence re-verification allowed.
- Remove challenge_ids seeding (vulnerability-name leak) from scope;
seed engagement.coverage_obligations (client-style in-scope endpoints)
+ generic engagement.audit_mode flag instead
- plugins/violin_guard now contains zero benchmark/run-id references;
gates are framework-owned and audit-mode-gated
- Cross-engagement guard blocks ANY foreign engagement dir, not just
benchmark-run-*
- Anti-cheat regression test asserts no vuln names in generated scope
- _engagement_needs_closeout no longer fires on leftover unchecked PTT rows:
a single '[ ]' task marker triggered a continuation pass that re-ran the
entire assessment (255 msgs / 20m50s) instead of closing out. Closeout is
now gated on the artifacts the scorer consumes (FIND files + report.md +
retrospective.md) only.
- Closeout goal rewritten to be surgical: assessment is COMPLETE, evidence is
final, no new tests/scans/verification, no evidence modification; only
generate missing report/retrospective (prefer generate-closeout) and update
PTT statuses.
- Friction-log instruction moved out of the general pentest SKILL.md (it is a
benchmark-only artifact) into the benchmark goal prompt so the agent
actually sees it.
- New pinned tests: leftover-PTT-row closeout detection, closeout-goal
wording.
* fix: bump distribution version to 3.0.1
* feat(benchmark): introduce benchmark runner engine, Docker containerization, and calibration datasets
* Fix URL/path scope exclusions and update duck-store template
- Modified targets.py to ensure that excluding a specific URL or path does not result in the entire host being blocked. Implemented a strict command payload check for excluded endpoints.
- Updated duck-store benchmark scope template to explicitly block the /vulnerabilities endpoint (which leaks intentional challenges) while allowing access to endpoint documentation.
* fix: correct scope.yaml indentation and refine guard blocking messages
* fix: resolve ModuleNotFoundError when score.py is executed directly
---------
Co-authored-by: Dan <dan@strategicautomation.local>
Violin