Violin
3807d22eaa
fix(benchmark): evidence-gated scorer contract, false-positive discriminators, runner validity
...
- Confirm Validated hypotheses via canonical FIND-NNN.md matched in live-app
terms (METHOD + endpoint route) alongside execution bundles
- require/require_any discriminators for shared-endpoint challenges;
absence-type scoring for no-rate-limiting
- Runner: closeout timeout is a soft warning, not run invalidation
- ai_judge: whitelist legitimate closeout artifacts in schema-drift audit
- Align version surfaces to 3.1.0; move pyyaml to runtime deps
Calibration: known-good 20/20, known-bad 0/0. Latest run re-scored 4 -> 9.
2026-08-10 20:52:02 +01:00
3b21e0a92a
feat: Automated Benchmark Runner and Evaluation Framework ( #68 )
...
* fix: bump distribution version to 3.0.1
* feat(benchmark): introduce benchmark runner engine, Docker containerization, and calibration datasets
* Fix URL/path scope exclusions and update duck-store template
- Modified targets.py to ensure that excluding a specific URL or path does not result in the entire host being blocked. Implemented a strict command payload check for excluded endpoints.
- Updated duck-store benchmark scope template to explicitly block the /vulnerabilities endpoint (which leaks intentional challenges) while allowing access to endpoint documentation.
* fix: correct scope.yaml indentation and refine guard blocking messages
* fix: resolve ModuleNotFoundError when score.py is executed directly
---------
Co-authored-by: Dan <dan@strategicautomation.local >
2026-08-08 13:29:11 +01:00
Violin
021fad0c0d
chore: pin Python 3.11 in .python-version and pyproject.toml to match Hermes runtime
2026-07-26 17:25:27 +01:00
Violin
1e04f565b0
feat(guard): replace custom shell regexes with bashlex AST parsing
2026-07-26 17:11:05 +01:00
Violin
06a60afa47
feat(guard): upgrade IP/CIDR scope policy arithmetic to netaddr IPSet and yarl URL parsing
2026-07-26 17:10:27 +01:00
Violin
0b135660ee
feat(guard): replace platform-specific process termination with psutil tree cleanup
2026-07-26 17:09:55 +01:00
Violin
58f487f19d
feat(guard): migrate tool schemas and validation to Pydantic v2
2026-07-26 17:09:00 +01:00
Violin
5fc876367d
Release Violin 3.0.0 profile
2026-07-22 11:18:53 +01:00
Violin
5873cf9ddd
fix CodeQL quality findings (v2.0.6)
2026-07-19 00:03:08 +01:00
Violin
6e8ba6e77c
fix exact repeat history parsing ( #3 )
2026-07-18 23:31:16 +01:00
Violin
22560d57b4
block secondary-only primary targets ( #2 )
2026-07-18 23:19:50 +01:00
Violin
a7dffbd191
fix compound terminal command policy ( #1 )
2026-07-18 23:01:48 +01:00
Violin
e2be6759a5
restrict workflow token permissions
2026-07-18 22:54:16 +01:00
Violin
28c72446fa
upgrade pytest for CVE-2025-71176
2026-07-18 22:42:57 +01:00
Violin
698edc0a45
chore: bump version to v2.0.0 and update docs
2026-07-17 20:26:47 +01:00
Violin
5870509fdb
feat: implement violin_guard plugin for centralized command execution auditing, terminal policy enforcement, and history tracking.
2026-07-17 20:14:20 +01:00
Violin
a9130fe02c
Harden plugin release checks and CI
2026-07-13 08:53:57 +01:00
Violin
283f2683a9
feat: improve guard and workflow reliability
2026-07-12 09:39:55 +01:00
Violin