SnapOtter/tests

SnapOtter Test Suite

Directory Structure

tests/
  unit/                    # Vitest unit tests (~185 files)
    ai/                    # AI sidecar bridge + bundle tests
    api/                   # API route/config/middleware tests
    doc-engine/            # doc-engine package tests
    features/              # Feature flag + enterprise tests
    fixtures/              # Fixture guard tests (resolve/budget/manifest)
    image-engine/          # image-engine operation tests
    media-engine/          # media-engine FFmpeg wrapper tests
    security/              # Security-focused unit tests
    shared/                # Shared constants/i18n/permissions tests
    web/                   # React component tests (tsx)
  integration/             # Full-API integration tests (~250 files)
    fixtures/              # Fixture integrity tests (decode through Sharp/ffprobe/qpdf)
    generated/             # Generated test matrices (format x tool, pairwise settings)
    platform/              # Platform tests (jobs, pipelines, batch, auth, OIDC)
    security/              # Security integration tests (SSRF, XXE, path traversal)
    tools/                 # Per-tool integration tests
  e2e/                     # Playwright specs (main app, Chromium primary)
  e2e-analytics/           # Analytics flow e2e specs
  e2e-docker/              # Docker container e2e specs
  e2e-docs/                # VitePress docs site e2e specs
  e2e-editor/              # Konva image editor e2e specs
  e2e-landing/             # Astro landing site e2e specs
  fixtures/                # Shared test fixtures (see below)
  helpers/                 # Test utilities (pairwise, zod-pict, tool defaults)
  setup/                   # Vitest setup (per-fork DB clone, BullMQ prefix isolation)
  baseline/                # Phase 0 baseline snapshots for parity verification
  benchmark/               # Performance benchmarks
  qa/                      # QA checklists and manual test plans

Fixtures

Fixtures live in tests/fixtures/ in a modality-first layout:

tests/fixtures/
  index.ts              # Typed registry -- import paths from here, not raw strings
  manifest.json         # sha256 + bytes + provenance for all valid/ assets
  gen-manifest.mjs      # Re-stamps manifest hashes from disk
  gen-synthetic-content.mjs  # Regenerates CC0 synthetic content (QR, barcode, OCR, etc.)
  LICENSES.md           # Provenance audit trail
  image/
    valid/              # Real images for depth tests (portraits, scenes, OCR, etc.)
    formats/            # Tier-1 tiny format samples (sample.png, sample.arw, ...)
    edge/               # Edge-case images (1x1, blank, extreme aspect)
    hostile/            # Malformed/bomb/polyglot files for rejection tests
  video/
    valid/              # Real videos (Big Buck Bunny CC-BY heroes, TTS speech)
    formats/            # Tier-1 tiny format samples (tiny.mp4, tiny.webm, ...)
    hostile/            # Truncated/malformed video files
  audio/
    valid/              # Real audio (TTS speech in 6 formats, tagged MP3)
    formats/            # Tier-1 tiny format samples (tiny.mp3, tiny.wav, ...)
    hostile/            # Zero-byte/malformed audio files
  document/
    valid/              # PDFs (3-page, 6-page, encrypted, OCR-scanned)
    formats/            # Tier-1 tiny doc samples (tiny.docx, tiny.epub, ...)
    edge/               # Edge-case docs (remote-img.html for SSRF tests)
    hostile/            # Truncated/garbage documents
  data/
    valid/              # Data files (CSV, JSON, XML, YAML, TSV, ZIP)
  security/             # XXE/SSRF test vectors (SVG)

Two-Tier Fixture Strategy

Tier-1: Tiny Synthetics (in formats/, edge/, hostile/) Small, project-generated files for format-compatibility matrices and fast unit tests. These are always-regeneratable by the generator scripts and are used for breadth.

Tier-2: Real Heroes (in valid/) Content-representative files for integration and fidelity tests. Includes:

• Big Buck Bunny video clips (CC-BY, Blender Foundation)
• macOS TTS speech in 6 audio formats (CC0)
• Project-generated synthetics (QR codes, barcodes, OCR text images, PDFs)
• Portrait/scene photos for AI tool tests (some UNVERIFIED-REVIEW, see manifest)

Every valid/ file is tracked in manifest.json with sha256 hash, byte count, source URL, and license. The manifest guard test (fixture-manifest.test.ts) verifies disk matches manifest on every run.

Typed Fixture Registry

Always import fixture paths from tests/fixtures/index.ts:

import { fixtures, readFixture, fixtureDir } from "../../fixtures/index.js";

// Named paths (type-checked, IDE autocomplete)
const pngPath = fixtures.image.base.png200;
const heroMp4 = fixtures.video.hero.mp4;

// Read file bytes
const buffer = readFixture(fixtures.image.base.png200);

// Format accessor (for matrix iteration)
const arwPath = fixtures.image.formats("arw");
const mp4Path = fixtures.video.tiny("mp4");

// Directory paths (for glob/scan patterns)
const hostileDir = fixtureDir.image.hostile;

Guard Tests

Four guard tests protect fixture integrity:

Guard	Location	What it checks
fixture-resolve	unit/fixtures/	Every registry path exists and is non-empty
fixture-budget	unit/fixtures/	Per-extension size caps prevent bloat
fixture-manifest	unit/fixtures/	sha256/bytes in manifest.json match disk
fixture-integrity	integration/fixtures/	Real heroes decode through Sharp/ffprobe/qpdf

Additional parity guards:

Guard	Location	What it checks
tool-registry-drift	unit/shared/	Frontend registry matches shared TOOLS catalog
tool-route-drift	integration/	API routes match shared TOOLS catalog
i18n-completeness	unit/shared/	All 21 locales have all required keys

Adding a Fixture

1. Add the file to the appropriate modality directory (image/valid/, video/formats/, etc.)
2. Add a named key in tests/fixtures/index.ts
3. Run node tests/fixtures/gen-manifest.mjs (for valid/ files)
4. Fill in sourceUrl and license in manifest.json
5. Update LICENSES.md if the file has third-party origin
6. Run guards: pnpm vitest run tests/unit/fixtures/

Generator Scripts

Three generators produce deterministic synthetics. All are idempotent and skip files that already exist (to avoid breaking manifest hashes from encoder-version differences). Pass --force to gen-synthetic-content.mjs to overwrite.

Script	Scope	Outputs
scripts/generate-test-fixtures.mjs	Tier-1 format samples	video/formats/tiny.mp4, audio/formats/tiny.{mp3,wav}, document/formats/tiny.{docx,xlsx,epub,html,md}, document/valid/encrypted.pdf
scripts/generate-hostile-fixtures.mjs	Hostile inputs	image/hostile/{truncated.jpg,zero-byte.png,garbage.jpg,png-bytes.jpg,bomb-50000x50000.png}
tests/fixtures/gen-synthetic-content.mjs	CC0 synthetics	QR codes, barcodes, OCR text images, PDFs, SVG logo, multi-face placeholder, animated GIF, tagged MP3, metadata MP4

The generators do NOT touch committed real heroes (BBB videos, TTS speech, UNVERIFIED-REVIEW photos).

Running Tests

# Full suite
pnpm test                    # All unit + integration tests
pnpm test:unit               # Unit tests only
pnpm test:integration        # Integration tests only

# Single file
pnpm vitest run tests/unit/my-test.test.ts
pnpm vitest run tests/integration/my-test.test.ts

# Fixture guards only
pnpm vitest run tests/unit/fixtures

# E2E (Playwright)
pnpm test:e2e                # Main app (chromium + serial + visual projects)
pnpm test:e2e:landing        # Landing site
pnpm test:e2e:docs           # VitePress docs
pnpm test:e2e:analytics      # Analytics flows
pnpm test:docker             # Full Docker container test

# Editor e2e (no root alias)
pnpm playwright test --config playwright.editor.config.ts

# Single e2e spec
pnpm playwright test tests/e2e/my-test.spec.ts

Environment Requirements

Unit and integration tests need Postgres + Redis:

docker compose -f docker-compose.dev.yml up -d
export DATA_DIR=/tmp/so-test-data && mkdir -p "$DATA_DIR"

Each Vitest fork gets its own Postgres DB clone + workspace + isolated BULLMQ_PREFIX via tests/setup/per-fork-env.ts (testcontainers for Postgres). 30s test timeouts.

E2E Projects

The main Playwright config defines three projects:

• chromium: Parallel-safe specs, 2 workers (override via PW_WORKERS)
• chromium-serial: Global-state specs, 1 worker
• chromium-visual: Screenshot comparison specs (platform-suffixed baselines)

Auth setup project logs in and saves storage state. The webServer block spins up both dev servers with a fresh test DB.

Environment Switches

Variable	Effect
FULL_MATRIX=1	Run all format x tool combinations (nightly)
FUZZ=1 + FUZZ_RUNS=N	Property-based fuzz testing (nightly)
PW_WORKERS=N	Override Playwright worker count

CI Pipeline

PRs run: lint, typecheck, unit, integration (4 shards), e2e smoke. Nightly: full e2e, cross-browser, docker suite, full matrix + fuzz, coverage.