External/SnapOtter
1
0
Fork 0
mirror of https://github.com/snapotter-hq/SnapOtter.git synced 2026-08-03 07:46:42 +02:00
Code Issues Packages Projects Releases Wiki Activity

test(fixtures): add provenance manifest + LICENSES + consistency guard (phase 1)

Browse Source
This commit is contained in:
SnapOtter
2026-06-19 20:13:43 +08:00
parent 8f72a5152c
commit d23e72d8d4
5 changed files with 446 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/fixtures/LICENSES.md
Vendored
+62
View File
@@ -0,0 +1,62 @@
# Fixture Provenance & Licensing
All fixtures in `tests/fixtures/` must have verified provenance before the Phase 2
licensing hard gate. Only CC0, CC-BY, CC-BY-SA, and public-domain assets are
permitted in the final registry.
## Verified Assets
| Path | License | Source | Author |
|------|---------|--------|--------|
| (none yet -- Phase 2 audit populates this table) | | | |
## Synthetic / Project-Generated Assets
The following were generated by project scripts or test infrastructure and carry
no third-party copyright concern:
- Root synthetics: `test-200x150.png`, `test-100x100.jpg`, `test-50x50.webp`,
`test-100x100.svg`, `test-200x150.heic`, `test-1x1.png`, `test-blank.png`,
`test-portrait-tall.png`, `test-portrait-extreme.png`, `test-with-exif.jpg`,
`test-scene.png`, `test-fake-transparency.png`, `test-3page.pdf`
- `media/tiny.*` (29 formats), `documents/tiny.*` (17 formats), `data/tiny.*` (9 formats)
- `hostile/*` (generated via `scripts/generate-hostile-fixtures.mjs`)
- `security/*` (SVG XXE test vectors)
- `formats/sample.*` (format samples, 35 files)
## Provenance Unverified -- Replace in Phase 2
The following `content/` heroes have `UNVERIFIED` license status in `manifest.json`.
Phase 2 will either verify their provenance (CC0/CC-BY source) or replace them with
properly-licensed alternatives:
- `content/alt-2page.pdf`
- `content/animated-simpsons.gif`
- `content/audio-with-tags.mp3`
- `content/barcode.avif`
- `content/barcode.png`
- `content/cross-format-chat.webp`
- `content/media-30s.mp4`
- `content/media-30s.wav`
- `content/motorcycle.heif`
- `content/multi-face.webp`
- `content/multipage-6.pdf`
- `content/ocr-chat.jpeg`
- `content/ocr-clean.png`
- `content/ocr-japanese.png`
- `content/ocr-scanned.pdf`
- `content/portrait-bw.jpeg`
- `content/portrait-color-dup.jpg`
- `content/portrait-color.jpg`
- `content/portrait-headshot.heic`
- `content/portrait-isolated.png`
- `content/qr-code.avif`
- `content/qr-code.png`
- `content/qr-code.svg`
- `content/red-eye.jpg`
- `content/speech-10s.mp4`
- `content/speech-10s.wav`
- `content/stress-large.jpg`
- `content/svg-logo.svg`
- `content/video-with-meta.mp4`
- `content/watermark.jpg`
tests/fixtures/README.md
Vendored
+77
View File
@@ -0,0 +1,77 @@
# Test Fixtures
Shared test fixtures for the SnapOtter test suite. Organized in two tiers:
## Two-Tier Structure
**Tier 1: Tiny Synthetics (matrices/format coverage)**
Small, project-generated files for format-compatibility matrices and fast unit tests.
Located in `media/`, `documents/`, `data/`, `formats/`, and root-level `test-*` files.
**Tier 2: Real Heroes (depth/fidelity testing)**
Larger, content-representative files for integration and fidelity tests.
Located in `content/`. Each must be tracked in `manifest.json` with sha256 + provenance.
## Layout (Modality-First, Phase 6 Target)
```
tests/fixtures/
index.ts # Typed registry (import paths from here, not raw strings)
manifest.json # sha256 + bytes + provenance for content/ heroes
gen-manifest.mjs # Script to re-stamp manifest hashes
LICENSES.md # Provenance audit trail
content/ # Tier-2 real heroes
formats/ # Tier-1 format samples (35 formats incl. RAW)
media/ # Tier-1 tiny audio/video samples
documents/ # Tier-1 tiny document samples
data/ # Tier-1 tiny data samples (csv, json, xml, etc.)
hostile/ # Malformed/bomb/polyglot files for rejection tests
security/ # XXE, SSRF test vectors
image/ # (Phase 6) modality-first target dir
video/ # (Phase 6) modality-first target dir
audio/ # (Phase 6) modality-first target dir
document/ # (Phase 6) modality-first target dir
data/ # (Phase 6) modality-first target dir
```
## Using the Registry
Always import paths from `tests/fixtures/index.ts` rather than hard-coding paths:
```ts
import { fixtures, readFixture } from "../../fixtures/index.js";
// Direct path access
const pngPath = fixtures.image.base.png200;
// Read bytes
const buffer = readFixture(fixtures.image.base.png200);
// Format accessor (for matrix iteration)
const arwPath = fixtures.image.formats("arw");
const mp4Path = fixtures.video.tiny("mp4");
```
## Adding a Fixture
1. Add the file to the appropriate directory
2. Add a key in `tests/fixtures/index.ts` pointing to it
3. Run `node tests/fixtures/gen-manifest.mjs` if the file is in `content/`
4. Fill in `sourceUrl` and `license` in `manifest.json` (required for Phase 2+)
5. Update `LICENSES.md` with the provenance record
6. Run the guards: `pnpm vitest run tests/unit/fixtures/ tests/integration/fixtures/`
## Licensing Gate
Phase 1 allows `UNVERIFIED` provenance. Phase 2 enforces that every `content/` hero
has a verified license (CC0, CC-BY, CC-BY-SA, or public-domain). See `LICENSES.md`
for the current audit status.
## Guard Tests
Four guard tests protect fixture integrity:
- **fixture-resolve** (unit): every registry path exists and is non-empty
- **fixture-budget** (unit): per-extension size caps prevent bloat
- **fixture-manifest** (unit): sha256/bytes in manifest.json match disk
- **fixture-integrity** (integration): real heroes decode through Sharp/ffprobe/qpdf
tests/fixtures/gen-manifest.mjs
Vendored
+23
View File
@@ -0,0 +1,23 @@
#!/usr/bin/env node
// Stamps sha256 + bytes for the real `content/` heroes into manifest.json.
// Provenance (sourceUrl/license) is filled by hand in Step 2 -- this only fills hashes.
import { createHash } from "node:crypto";
import { readdirSync, readFileSync, writeFileSync } from "node:fs";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
const ROOT = dirname(fileURLToPath(import.meta.url));
const contentDir = join(ROOT, "content");
const existing = JSON.parse(readFileSync(join(ROOT, "manifest.json"), "utf8"));
const byPath = new Map(existing.assets.map((a) => [a.path, a]));
for (const name of readdirSync(contentDir)) {
const rel = `content/${name}`;
const bytes = readFileSync(join(contentDir, name));
const sha256 = createHash("sha256").update(bytes).digest("hex");
const prev = byPath.get(rel) ?? { path: rel, sourceUrl: "UNVERIFIED", license: "UNVERIFIED", toolsServed: [] };
byPath.set(rel, { ...prev, sha256, bytes: bytes.length });
}
const assets = [...byPath.values()].sort((a, b) => a.path.localeCompare(b.path));
writeFileSync(join(ROOT, "manifest.json"), JSON.stringify({ assets }, null, 2));
console.log(`manifest: ${assets.length} assets`);
tests/fixtures/manifest.json
Vendored
+244
View File
@@ -0,0 +1,244 @@
{
"assets": [
{
"path": "content/alt-2page.pdf",
"sourceUrl": "UNVERIFIED",
"license": "UNVERIFIED",
"toolsServed": [],
"sha256": "494c52ff3883d4fb6b019be71e0e29c4b27c5d5c48ba27e22d86295797114658",
"bytes": 1075
},
{
"path": "content/animated-simpsons.gif",
"sourceUrl": "UNVERIFIED",
"license": "UNVERIFIED",
"toolsServed": [],
"sha256": "34cb66ae913c928293af2b7d6ca159eb245bd8c2fa92c208ca6975758246b0ca",
"bytes": 109693
},
{
"path": "content/audio-with-tags.mp3",
"sourceUrl": "UNVERIFIED",
"license": "UNVERIFIED",
"toolsServed": [],
"sha256": "882c35013d3554cf32e37b0912abe14d88415efd28ade2be39e7935d7ce71a47",
"bytes": 9956
},
{
"path": "content/barcode.avif",
"sourceUrl": "UNVERIFIED",
"license": "UNVERIFIED",
"toolsServed": [],
"sha256": "870b71dcb8eea7cc69a612cd0dcd259f7a402220fcc3467c17ebf76dc9f52826",
"bytes": 1152
},
{
"path": "content/barcode.png",
"sourceUrl": "UNVERIFIED",
"license": "UNVERIFIED",
"toolsServed": [],
"sha256": "50bf07d3d746c63ccdcd87f40989b5f35b1e1a6e1f6e75b6851d550574b3f5b2",
"bytes": 12576
},
{
"path": "content/cross-format-chat.webp",
"sourceUrl": "UNVERIFIED",
"license": "UNVERIFIED",
"toolsServed": [],
"sha256": "de010e4433f6b20e9e7834bf8608f9a946971af5920d4fc53658194844686285",
"bytes": 181960
},
{
"path": "content/media-30s.mp4",
"sourceUrl": "UNVERIFIED",
"license": "UNVERIFIED",
"toolsServed": [],
"sha256": "b4504e28ddda52ee3cb9152bf32b8695f0a61f6f585042c293d3e13645aeb338",
"bytes": 6768813
},
{
"path": "content/media-30s.wav",
"sourceUrl": "UNVERIFIED",
"license": "UNVERIFIED",
"toolsServed": [],
"sha256": "92ac8e1dcb67e1601c92cef8c9da7118e03340e17645efe9e387db63af7968ff",
"bytes": 2646078
},
{
"path": "content/motorcycle.heif",
"sourceUrl": "UNVERIFIED",
"license": "UNVERIFIED",
"toolsServed": [],
"sha256": "90f3ae39a6ea1355f148712cd013f71111b2d74451ca07bcf8442489b0d1995e",
"bytes": 224154
},
{
"path": "content/multi-face.webp",
"sourceUrl": "UNVERIFIED",
"license": "UNVERIFIED",
"toolsServed": [],
"sha256": "a2b634fc99014bf08e924848790fe5bddff825fa56dcbd20ea71176fa3fcf334",
"bytes": 30040
},
{
"path": "content/multipage-6.pdf",
"sourceUrl": "UNVERIFIED",
"license": "UNVERIFIED",
"toolsServed": [],
"sha256": "f26d8cfff5ce1895a1e877bacd5bff024692085d8f30f378d7bc50d214e3fd72",
"bytes": 2239
},
{
"path": "content/ocr-chat.jpeg",
"sourceUrl": "UNVERIFIED",
"license": "UNVERIFIED",
"toolsServed": [],
"sha256": "97798605609879cbbefb4bdc94d4c8e47ac1e122d22d7951c33d37e63c5f4238",
"bytes": 1048725
},
{
"path": "content/ocr-clean.png",
"sourceUrl": "UNVERIFIED",
"license": "UNVERIFIED",
"toolsServed": [],
"sha256": "a07ea2e0c88f2765eb48bd6f261bf146581fc7d1a38b6d1a0384aeb0c4a7b5e7",
"bytes": 9176
},
{
"path": "content/ocr-japanese.png",
"sourceUrl": "UNVERIFIED",
"license": "UNVERIFIED",
"toolsServed": [],
"sha256": "d2bed03fd01e7a327d49a4abaa6b13c2507b585ba23a66f877ea78d68b022417",
"bytes": 10104
},
{
"path": "content/ocr-scanned.pdf",
"sourceUrl": "UNVERIFIED",
"license": "UNVERIFIED",
"toolsServed": [],
"sha256": "28d33f9f30a450943d59489345bf475b42ca0f73deffeb883df1d117d97e5d90",
"bytes": 1049673
},
{
"path": "content/portrait-bw.jpeg",
"sourceUrl": "UNVERIFIED",
"license": "UNVERIFIED",
"toolsServed": [],
"sha256": "0ca77a23c1af40d107f4f34eef048adef98f8b36c475723c6fe921ed3dec81bc",
"bytes": 2731390
},
{
"path": "content/portrait-color-dup.jpg",
"sourceUrl": "UNVERIFIED",
"license": "UNVERIFIED",
"toolsServed": [],
"sha256": "908893cc7c816e977f84af59237a1af29bab6e3ed801879cbe992f698161397e",
"bytes": 89084
},
{
"path": "content/portrait-color.jpg",
"sourceUrl": "UNVERIFIED",
"license": "UNVERIFIED",
"toolsServed": [],
"sha256": "908893cc7c816e977f84af59237a1af29bab6e3ed801879cbe992f698161397e",
"bytes": 89084
},
{
"path": "content/portrait-headshot.heic",
"sourceUrl": "UNVERIFIED",
"license": "UNVERIFIED",
"toolsServed": [],
"sha256": "1f59692ec10d5372e69ebc789c53ebe752f665ff87ede4d15817ce512e769732",
"bytes": 1489644
},
{
"path": "content/portrait-isolated.png",
"sourceUrl": "UNVERIFIED",
"license": "UNVERIFIED",
"toolsServed": [],
"sha256": "8d53453a5adaecfc3b6db83e85f732cbe67adc42e7c719d4c3ae8eb7777e0a8f",
"bytes": 80745
},
{
"path": "content/qr-code.avif",
"sourceUrl": "UNVERIFIED",
"license": "UNVERIFIED",
"toolsServed": [],
"sha256": "7d7d93d1f9221c434b99a8262d68aea94acd2de88d13a729eb3af1e3bf4f35c5",
"bytes": 1487
},
{
"path": "content/qr-code.png",
"sourceUrl": "UNVERIFIED",
"license": "UNVERIFIED",
"toolsServed": [],
"sha256": "a526c26245a3caab564fab64fde7f2d75d5f7c9dff53fcfe784d9119ebe5074f",
"bytes": 2438
},
{
"path": "content/qr-code.svg",
"sourceUrl": "UNVERIFIED",
"license": "UNVERIFIED",
"toolsServed": [],
"sha256": "23ae3e41f61df60db35cd65aa06b8842c19fc06857f0711d97a42fad496b6577",
"bytes": 2191
},
{
"path": "content/red-eye.jpg",
"sourceUrl": "UNVERIFIED",
"license": "UNVERIFIED",
"toolsServed": [],
"sha256": "04a993def90ffb471cbb0b4bbe46bd2aa9a2d344a9ab7ad187dc22f5feca23ed",
"bytes": 368937
},
{
"path": "content/speech-10s.mp4",
"sourceUrl": "UNVERIFIED",
"license": "UNVERIFIED",
"toolsServed": [],
"sha256": "e3196b8eb044feb7eba7ae55d931f5be31f6694186e8d67d6fb6d412a89f25e8",
"bytes": 2254674
},
{
"path": "content/speech-10s.wav",
"sourceUrl": "UNVERIFIED",
"license": "UNVERIFIED",
"toolsServed": [],
"sha256": "dde7726a10dc469dea2e306b94462b5fd4ec9b2f62e0b22401a018c6b27270fa",
"bytes": 320390
},
{
"path": "content/stress-large.jpg",
"sourceUrl": "UNVERIFIED",
"license": "UNVERIFIED",
"toolsServed": [],
"sha256": "53610a10526e1a4f42036a08091cbe8a79781f4522915f98f423cab14f6bca10",
"bytes": 6678906
},
{
"path": "content/svg-logo.svg",
"sourceUrl": "UNVERIFIED",
"license": "UNVERIFIED",
"toolsServed": [],
"sha256": "7a8f5e99e26fd660babe7084290827ed8faac053ec089cce31687307d736a6ca",
"bytes": 2906
},
{
"path": "content/video-with-meta.mp4",
"sourceUrl": "UNVERIFIED",
"license": "UNVERIFIED",
"toolsServed": [],
"sha256": "77b567b9e43b15d6edac80a445f29ee3f8affa77d7835442309cb9b0417cab5b",
"bytes": 13444
},
{
"path": "content/watermark.jpg",
"sourceUrl": "UNVERIFIED",
"license": "UNVERIFIED",
"toolsServed": [],
"sha256": "b8ddbceea01513f4a9ac6b0903b8071d8466ae88a8d5ef6fd84c33644ce22a4d",
"bytes": 87064
}
]
}
tests/unit/fixtures/fixture-manifest.test.ts
+40
View File
@@ -0,0 +1,40 @@
import { createHash } from "node:crypto";
import { readFileSync } from "node:fs";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
import { describe, expect, it } from "vitest";
const ROOT = join(dirname(fileURLToPath(import.meta.url)), "../../fixtures");
const manifest = JSON.parse(readFileSync(join(ROOT, "manifest.json"), "utf8"));
describe("fixture manifest is consistent with disk", () => {
it("has entries", () => expect(manifest.assets.length).toBeGreaterThan(20));
// Phase 1 hard-checks bytes + sha256; license may be "UNVERIFIED" (tracked in
// LICENSES.md). Phase 2 does the real provenance audit and removes that tolerance.
const ALLOWED = ["CC0", "CC-BY", "CC-BY-SA", "public-domain", "UNVERIFIED"];
it.each(manifest.assets)("$path matches sha256 + bytes and declares a license", (asset: {
path: string;
bytes: number;
sha256: string;
license: string;
}) => {
const buf = readFileSync(join(ROOT, asset.path));
expect(buf.length, `${asset.path} byte mismatch`).toBe(asset.bytes);
expect(createHash("sha256").update(buf).digest("hex"), `${asset.path} sha256 mismatch`).toBe(
asset.sha256,
);
expect(ALLOWED, `${asset.path} license '${asset.license}' not allowed`).toContain(
asset.license,
);
});
it("reports how many assets still need provenance (Phase 2 drives this to 0)", () => {
const unverified = manifest.assets
.filter((a: { license: string; path: string }) => a.license === "UNVERIFIED")
.map((a: { path: string }) => a.path);
if (unverified.length)
console.warn(`UNVERIFIED provenance (verify/replace in Phase 2): ${unverified.join(", ")}`);
expect(unverified.length).toBeLessThanOrEqual(manifest.assets.length); // informational; never fails Phase 1
});
});