mirror of
https://github.com/snapotter-hq/SnapOtter.git
synced 2026-08-03 07:46:42 +02:00
SnapOtterandGitHub
8f4235d2c6
* fix(enterprise): ship enterprise pkg in prod image, full license features, tracing key fallback docker/Dockerfile: COPY packages/enterprise manifest+src into the production stage. Without it, apps/api's workspace link to @snapotter/enterprise dangles and every import() throws (silently caught), so all 19 enterprise features failed closed (enterprise.active=false) regardless of a valid license. scripts/generate-license.mjs: sync PLAN_FEATURES with packages/enterprise/src/license.ts so a --plan enterprise license unlocks all 19 features (was 8) and team unlocks 8. apps/api/src/tracing.ts: accept SNAPOTTER_LICENSE_KEY as a fallback to LICENSE_KEY so distributed_tracing activates with the same key as the rest of the app. * fix(docker): keep scripts/bake-analytics.mjs in build context .dockerignore excluded the whole scripts/ dir (PR #82, V1 hardening), but docker/Dockerfile later added 'COPY scripts/bake-analytics.mjs' for the analytics bake step. A clean production image build therefore fails with 'scripts/bake-analytics.mjs: not found'. The published image build is gated off in CI so this latent break went unnoticed. Exclude scripts/* but re-include the one file the Dockerfile needs. * fix: S3 upload stream, analytics bake reaches API, dedupe retention field, reconcile orphan jobs storage-s3.ts: wrap the upload AsyncIterable in Readable.from() so @aws-sdk/lib-storage accepts it. STORAGE_MODE=s3 file uploads failed with 'Body Data is unsupported format' for every tool because a bare async generator is not a Readable. docker/Dockerfile: COPY the builder-baked analytics baked.ts into the API runtime stage. The API re-copied the committed (off) baked.ts from the build context, so the SNAPOTTER_ANALYTICS build arg had no effect on the API -- and since the SPA reads /api/v1/config/analytics, analytics was off everywhere regardless of the arg. settings-dialog.tsx: remove the duplicate tempFileMaxAgeHours control under Data Retention; it bound the same setting key as the File Management control with a different default, so editing either silently overwrote the other. apps/api/src/index.ts: reconcile orphaned job rows (empty tool_id, never enqueued to BullMQ) at boot so they don't sit in processing/queued forever and inflate the per-user concurrent-job count and the upgrade-check in-flight gate. * fix(web): style the SSO login buttons (they referenced undefined theme tokens) The OIDC/SAML 'Sign in with <provider>' buttons used bg-secondary / text-secondary-foreground, which the web theme never defines (it has primary, background, foreground, muted, border, card, primary-subtle). Those classes resolved to nothing, so the buttons rendered as bare unstyled text on the login page. Restyle: the optional (non-enforced) buttons become white-card outline buttons with a key icon and an orange hover tint, secondary to the primary Login button; the SSO-enforced buttons become solid primary with the icon. * fix: gate S3 behind license, custom-role enterprise perms, wire retention UI, cleanup S3 is a licensed feature, but shipping packages/enterprise in every image removed the implicit gate, so STORAGE_MODE=s3 worked without a license. Enforce isFeatureEnabled('s3_storage') at boot and fail fast if unlicensed. Custom roles can now be granted security:manage / compliance:manage / webhooks:manage (roles.ts ALL_PERMISSIONS + the Roles UI) so admins can build least-privilege compliance/security roles instead of only the built-in admin role. retentionSweep now reads the jobsRetentionDays / auditRetentionDays DB settings the System Settings UI writes (env vars become the fallback default), mirroring how the temp-file sweep reads tempFileMaxAgeHours. Previously those two UI controls were no-ops. Cleanup: drop the never-set snapotter_storage_bytes gauge and the unused MAX_WORKSPACE_SIZE_GB env var; emit tool_client_error to PostHog from the web ErrorBoundary (client crashes were not reaching analytics); add the Python OpenTelemetry packages so the innermost sidecar.<script> span exports; fix the stale 'only local storage' line in the docs; delete two e2e-analytics specs that tested the removed consent UI. * fix(env): restore MAX_WORKSPACE_SIZE_GB default security-auth-hardening.test.ts asserts env.MAX_WORKSPACE_SIZE_GB defaults to 10, so the var is an intentional (tested) default, not dead code. Removing it in the cleanup commit broke that unit test. Keep the declaration.
SnapOtter Test Suite
Directory Structure
tests/
unit/ # Vitest unit tests (~185 files)
ai/ # AI sidecar bridge + bundle tests
api/ # API route/config/middleware tests
doc-engine/ # doc-engine package tests
features/ # Feature flag + enterprise tests
fixtures/ # Fixture guard tests (resolve/budget/manifest)
image-engine/ # image-engine operation tests
media-engine/ # media-engine FFmpeg wrapper tests
security/ # Security-focused unit tests
shared/ # Shared constants/i18n/permissions tests
web/ # React component tests (tsx)
integration/ # Full-API integration tests (~250 files)
fixtures/ # Fixture integrity tests (decode through Sharp/ffprobe/qpdf)
generated/ # Generated test matrices (format x tool, pairwise settings)
platform/ # Platform tests (jobs, pipelines, batch, auth, OIDC)
security/ # Security integration tests (SSRF, XXE, path traversal)
tools/ # Per-tool integration tests
e2e/ # Playwright specs (main app, Chromium primary)
e2e-analytics/ # Analytics flow e2e specs
e2e-docker/ # Docker container e2e specs
e2e-docs/ # VitePress docs site e2e specs
e2e-editor/ # Konva image editor e2e specs
e2e-landing/ # Astro landing site e2e specs
fixtures/ # Shared test fixtures (see below)
helpers/ # Test utilities (pairwise, zod-pict, tool defaults)
setup/ # Vitest setup (per-fork DB clone, BullMQ prefix isolation)
baseline/ # Phase 0 baseline snapshots for parity verification
benchmark/ # Performance benchmarks
qa/ # QA checklists and manual test plans
Fixtures
Fixtures live in tests/fixtures/ in a modality-first layout:
tests/fixtures/
index.ts # Typed registry -- import paths from here, not raw strings
manifest.json # sha256 + bytes + provenance for all valid/ assets
gen-manifest.mjs # Re-stamps manifest hashes from disk
gen-synthetic-content.mjs # Regenerates CC0 synthetic content (QR, barcode, OCR, etc.)
LICENSES.md # Provenance audit trail
image/
valid/ # Real images for depth tests (portraits, scenes, OCR, etc.)
formats/ # Tier-1 tiny format samples (sample.png, sample.arw, ...)
edge/ # Edge-case images (1x1, blank, extreme aspect)
hostile/ # Malformed/bomb/polyglot files for rejection tests
video/
valid/ # Real videos (Big Buck Bunny CC-BY heroes, TTS speech)
formats/ # Tier-1 tiny format samples (tiny.mp4, tiny.webm, ...)
hostile/ # Truncated/malformed video files
audio/
valid/ # Real audio (TTS speech in 6 formats, tagged MP3)
formats/ # Tier-1 tiny format samples (tiny.mp3, tiny.wav, ...)
hostile/ # Zero-byte/malformed audio files
document/
valid/ # PDFs (3-page, 6-page, encrypted, OCR-scanned)
formats/ # Tier-1 tiny doc samples (tiny.docx, tiny.epub, ...)
edge/ # Edge-case docs (remote-img.html for SSRF tests)
hostile/ # Truncated/garbage documents
data/
valid/ # Data files (CSV, JSON, XML, YAML, TSV, ZIP)
security/ # XXE/SSRF test vectors (SVG)
Two-Tier Fixture Strategy
Tier-1: Tiny Synthetics (in formats/, edge/, hostile/)
Small, project-generated files for format-compatibility matrices and fast unit tests.
These are always-regeneratable by the generator scripts and are used for breadth.
Tier-2: Real Heroes (in valid/)
Content-representative files for integration and fidelity tests. Includes:
- Big Buck Bunny video clips (CC-BY, Blender Foundation)
- macOS TTS speech in 6 audio formats (CC0)
- Project-generated synthetics (QR codes, barcodes, OCR text images, PDFs)
- Portrait/scene photos for AI tool tests (some UNVERIFIED-REVIEW, see manifest)
Every valid/ file is tracked in manifest.json with sha256 hash, byte count,
source URL, and license. The manifest guard test (fixture-manifest.test.ts)
verifies disk matches manifest on every run.
Typed Fixture Registry
Always import fixture paths from tests/fixtures/index.ts:
import { fixtures, readFixture, fixtureDir } from "../../fixtures/index.js";
// Named paths (type-checked, IDE autocomplete)
const pngPath = fixtures.image.base.png200;
const heroMp4 = fixtures.video.hero.mp4;
// Read file bytes
const buffer = readFixture(fixtures.image.base.png200);
// Format accessor (for matrix iteration)
const arwPath = fixtures.image.formats("arw");
const mp4Path = fixtures.video.tiny("mp4");
// Directory paths (for glob/scan patterns)
const hostileDir = fixtureDir.image.hostile;
Guard Tests
Four guard tests protect fixture integrity:
| Guard | Location | What it checks |
|---|---|---|
| fixture-resolve | unit/fixtures/ |
Every registry path exists and is non-empty |
| fixture-budget | unit/fixtures/ |
Per-extension size caps prevent bloat |
| fixture-manifest | unit/fixtures/ |
sha256/bytes in manifest.json match disk |
| fixture-integrity | integration/fixtures/ |
Real heroes decode through Sharp/ffprobe/qpdf |
Additional parity guards:
| Guard | Location | What it checks |
|---|---|---|
| tool-registry-drift | unit/shared/ |
Frontend registry matches shared TOOLS catalog |
| tool-route-drift | integration/ |
API routes match shared TOOLS catalog |
| i18n-completeness | unit/shared/ |
All 21 locales have all required keys |
Adding a Fixture
- Add the file to the appropriate modality directory (
image/valid/,video/formats/, etc.) - Add a named key in
tests/fixtures/index.ts - Run
node tests/fixtures/gen-manifest.mjs(forvalid/files) - Fill in
sourceUrlandlicenseinmanifest.json - Update
LICENSES.mdif the file has third-party origin - Run guards:
pnpm vitest run tests/unit/fixtures/
Generator Scripts
Three generators produce deterministic synthetics. All are idempotent and skip
files that already exist (to avoid breaking manifest hashes from encoder-version
differences). Pass --force to gen-synthetic-content.mjs to overwrite.
| Script | Scope | Outputs |
|---|---|---|
scripts/generate-test-fixtures.mjs |
Tier-1 format samples | video/formats/tiny.mp4, audio/formats/tiny.{mp3,wav}, document/formats/tiny.{docx,xlsx,epub,html,md}, document/valid/encrypted.pdf |
scripts/generate-hostile-fixtures.mjs |
Hostile inputs | image/hostile/{truncated.jpg,zero-byte.png,garbage.jpg,png-bytes.jpg,bomb-50000x50000.png} |
tests/fixtures/gen-synthetic-content.mjs |
CC0 synthetics | QR codes, barcodes, OCR text images, PDFs, SVG logo, multi-face placeholder, animated GIF, tagged MP3, metadata MP4 |
The generators do NOT touch committed real heroes (BBB videos, TTS speech, UNVERIFIED-REVIEW photos).
Running Tests
# Full suite
pnpm test # All unit + integration tests
pnpm test:unit # Unit tests only
pnpm test:integration # Integration tests only
# Single file
pnpm vitest run tests/unit/my-test.test.ts
pnpm vitest run tests/integration/my-test.test.ts
# Fixture guards only
pnpm vitest run tests/unit/fixtures
# E2E (Playwright)
pnpm test:e2e # Main app (chromium + serial + visual projects)
pnpm test:e2e:landing # Landing site
pnpm test:e2e:docs # VitePress docs
pnpm test:e2e:analytics # Analytics flows
pnpm test:docker # Full Docker container test
# Editor e2e (no root alias)
pnpm playwright test --config playwright.editor.config.ts
# Single e2e spec
pnpm playwright test tests/e2e/my-test.spec.ts
Environment Requirements
Unit and integration tests need Postgres + Redis:
docker compose -f docker-compose.dev.yml up -d
export DATA_DIR=/tmp/so-test-data && mkdir -p "$DATA_DIR"
Each Vitest fork gets its own Postgres DB clone + workspace + isolated BULLMQ_PREFIX
via tests/setup/per-fork-env.ts (testcontainers for Postgres). 30s test timeouts.
E2E Projects
The main Playwright config defines three projects:
- chromium: Parallel-safe specs, 2 workers (override via
PW_WORKERS) - chromium-serial: Global-state specs, 1 worker
- chromium-visual: Screenshot comparison specs (platform-suffixed baselines)
Auth setup project logs in and saves storage state. The webServer block spins up
both dev servers with a fresh test DB.
Environment Switches
| Variable | Effect |
|---|---|
FULL_MATRIX=1 |
Run all format x tool combinations (nightly) |
FUZZ=1 + FUZZ_RUNS=N |
Property-based fuzz testing (nightly) |
PW_WORKERS=N |
Override Playwright worker count |
CI Pipeline
PRs run: lint, typecheck, unit, integration (4 shards), e2e smoke. Nightly: full e2e, cross-browser, docker suite, full matrix + fuzz, coverage.