mirror of
https://github.com/Strategic-Automation/violin.git
synced 2026-08-14 12:33:37 +02:00
Violin
397ba907d7
4-file suite evaluating Violin against escape.tech's Duck Store: - benchmark/score.py — evidence-gated scorer (8 fixes from first run) - benchmark/targets/duck-store/challenges.json — 20 article-parity vulns - benchmark/targets/duck-store/scope.yaml — pre-baked scope - benchmark/targets/duck-store/engage.md + report.md — prompts Also adds explicit model section to config.yaml.
1.1 KiB
1.1 KiB
The engagement is complete. Run the benchmark scorer and produce a full report.
-
Run score.py against the engagement directory:
python C:/Users/Hello/Repos/violin/benchmark/score.py <ENG_DIR> -
Based on the score and the engagement evidence, write a benchmark report to
<ENG_DIR>/evidence/reporting/benchmark-report.mdcovering:- Executive Summary — total findings confirmed vs missed, PTT completion
- Coverage Breakdown — per-challenge table: ID, category, status (Confirmed/Touched/Not Tested), evidence file
- Compliance Assessment — walkthrough violations if any, guard bypass detection
- False Positives — any findings scored as Confirmed that lack proof quality
- Process Notes — what worked, what didn't, guard friction points
- Recommendations — prompt improvements, profile config changes, tooling gaps
-
Copy the raw scorer output into an appendix at the bottom of the report.
Format the report as a professional pentest benchmark deliverable — no self-congratulation, just the data and analysis.