- Remove media-30s.mp4 and media-30s.wav from gen-synthetic-content.mjs
(these are committed real heroes, not synthetics to regenerate)
- Add skip-if-exists guards to all generators to prevent manifest hash
breakage from encoder-version differences
- Add --force flag to gen-synthetic-content.mjs for deliberate overwrite
- Fix generate-test-fixtures.mjs to skip encrypted.pdf if it exists
(qpdf AES encryption uses random IVs, non-deterministic)
- Fill provenance for 14 newly-scanned manifest entries after Phase 6b moves
- Verify all three generators produce expected output against new layout
Move all fixture files from flat/mixed dirs (content/, media/, documents/,
formats/, hostile/, root loose) into the modality-first hierarchy:
image/{valid,formats,edge,hostile}, video/{valid,formats,hostile},
audio/{valid,formats,hostile}, document/{valid,formats,edge,hostile},
data/valid/, security/. Update index.ts paths, fixtureDir aliases,
all literal refs in 17 e2e/qa/script files, manifest.json, and the
three generator scripts. 163 files moved, 0 dropped, 100 new tests
from expanded document scan.
Add scoped axe accessibility audit (a11y.spec.ts, device-a11y.spec.ts)
scanning home, one tool per modality, editor, and login across desktop
chromium and mobile-chromium in EN and AR locales. Uses a committed
baseline (a11y-baseline.json) to gate on NEW critical/serious violations
while documenting existing debt.
Add device-visual.spec.ts with curated screenshots (home, resize tool,
settings dialog) on mobile-chromium and tablet-chromium. Six darwin
baselines generated; linux baselines deferred to the existing
update-visual-baselines workflow.
Trivial a11y fixes applied:
- Login page: outer div -> main (fixes landmark-one-main, reduces region)
- Editor page: outer div -> main for both desktop and mobile gate
- AppLayout main: add tabIndex={-1} for skip-link focusability
Updated DEVICE_SPECS regex to route device-visual and device-a11y specs.
Added @axe-core/playwright as a devDependency.
Phase 3 replaced these with real device-emulated specs
(device-mobile.spec.ts, device-tablet.spec.ts) that use actual Pixel 7,
iPhone 14, iPad, and Galaxy Tab emulation with touch, DPR, and proper
/:modality/:toolId routes. The old specs used bare viewport resizing and
bare routes (/resize) that 404 on the prod-build preview server.
Removed:
- gui-visual-mobile.spec.ts (27 tests)
- gui-visual-tablet.spec.ts (27 tests)
- gui-responsive.spec.ts (82 tests)
Four launch gates for v2.0.0:
1. Catalog integrity (catalog-integrity.test.ts): asserts every TOOLS entry
is fully wired end to end (API route + frontend registry + display mode +
process fn or REGISTRY_EXEMPT). Count checked dynamically against
TOOLS.length. All 157 tools pass.
2. i18n cross-locale parity (i18n-parity.test.ts): asserts every locale in
SUPPORTED_LOCALES has the same key set as en.ts. Found and fixed a real
bug: zh-CN and pt-BR exported only a camelCase named export (zhCN, ptBR)
with no default export, so loadTranslations silently fell back to English
for Chinese Simplified and Brazilian Portuguese users. Fixed by adding
export default to both files. All 20 non-en locales now pass parity.
3. Cross-modality smoke (cross-modality-smoke.test.ts): one fast tool per
modality (rotate/image, mute-video/video, convert-audio/audio,
rotate-pdf/document, csv-json/data) plus an auth gate. Tools needing
ffmpeg or qpdf are gated with skipIf. Ship/no-ship signal.
4. Migration launch gate: extended migrate-from-sqlite.test.ts with a
representative 1.x SQLite database (3 users, 3 teams, 3 settings,
2 roles, 2 sessions, 2 API keys, 2 pipelines, 4 jobs, 4 audit entries,
4 user files) covering boolean/timestamp/JSON/NULL type conversions,
column remapping (input_files->input_refs, progress real->jsonb), and
multi-row round-trip verification. 9 new test cases.
Parity: 13260 passed, 0 dropped.
- animated-simpsons.gif: was Homer Simpson (Fox/Disney copyright),
now ffmpeg testsrc color-cycling GIF
- svg-logo.svg: was ConvertICO brand logo (third-party trademark),
now geometric test-fixture SVG
- multi-face.webp: was Shutterstock stock photo #199321457
(watermarked, copyrighted), now Sharp-generated face-placeholder grid
Replace 13 content fixtures with deterministic, reproducible outputs
from gen-synthetic-content.mjs:
- QR codes (PNG/SVG/AVIF): qrcode lib encoding https://snapotter.com
- Barcodes (PNG/AVIF): Code 128B encoder for SNAPOTTER-TEST-123
- OCR text (English/Japanese PNG): Sharp SVG text rendering
- PDFs (2-page/6-page): minimal PDF byte generator
- Audio (MP3 with ID3 tags, 30s WAV): ffmpeg 440Hz sine wave
- Video (1s metadata, 30s hero): ffmpeg testsrc2 + sine tone
All outputs are CC0 with no third-party IP. The generator script is
idempotent and can be re-run to reproduce identical fixtures.
Fixes all integration CI failures on the 2.0 branch.
## What was broken
Two independent root causes:
1. **202 assertion failures** -- Under 4-fork CI parallel load, the 30s
`SYNC_WAIT_MS` sync window can expire before a BullMQ worker finishes a
heavy encode (avif, heic), returning a legitimate `202 {jobId, async: true}`
instead of `200`. Tests that hard-asserted `200` were spuriously failing.
2. **Vitest timeout race** -- `SYNC_WAIT_MS` (30s) and the default Vitest
`testTimeout` (also 30s) fired simultaneously. Vitest won the race,
reporting "Test timed out in 30000ms" instead of the test receiving the
202 response.
## Fixes
- Added `isAsyncFallback()` helper to four integration test files; validates
the `{async: true, jobId}` body shape and returns early so the synchronous
200 path runs full assertions only when warranted.
- Set `vi.setConfig({ testTimeout: 60_000 })` at module level in
`image-enhancement.test.ts` and `format-matrix-comprehensive.test.ts`,
giving a 30s buffer between when `waitForJob()` returns 202 and when
Vitest gives up.
- Bumped explicit matrix timeouts in `format-matrix.test.ts` and
`new-formats.test.ts` from 30s to 60s for the same reason.
- Installed missing CI doc-engine binaries (qpdf, pandoc, libreoffice,
pdfcpu) that were causing unrelated integration failures.
- Fixed E2E smoke specs for 2.0 UI changes (modality selector, tool routes,
validation behavior).
rest.md: correct the tool response shape (JSON {jobId, downloadUrl, ...} for fast / 202 {jobId, async} for long, not the raw file), fix the SSE jobId source (body, with X-Job-Id only on batch/pipeline ZIP streams), complete the permissions list to all 17, and add ai-canvas-expand.
ai.md: reconcile ~14 AI tool parameter tables to the real Zod schemas (remove-background, upscale, ocr, enhance-faces, colorize, noise-removal, red-eye-removal, restore-photo, passport-photo, erase-object, smart-crop, content-aware-resize, transparency-fixer, image-enhancement), add the 6 missing AI tools (background-replace, blur-background, ocr-pdf, ai-canvas-expand, transcribe-audio, auto-subtitles), and fix the dispatcher script diagram and feature-bundle table.
image-engine.md: add 4 undocumented operations (sharpen, sharpen-advanced, color-blindness, edit-metadata) and fix flip/crop/rotate/convert/compress/strip-metadata/color-channels params, output-format count (13), and the info JSON example. supported-formats.md: output count 14 -> 13. Verified with a clean docs build.
Reconcile every tool page's parameters, defaults, and response shape against the tool's Zod settings schema and executionHint in code. Notable fixes: color-palette (add count + format params, hex output, median-cut algorithm), favicon (add 5 params, was documented as having none), qr-generate (add logoDataUri), convert (add ppm/eps/tga formats), video-loudnorm (-16 LUFS not -14), smart-crop (async 202 not sync 200), images-to-video (1080x1080 square), and several output-filename and behavior-note corrections.
Also normalize API endpoint paths to /api/v1/tools/<id> (no modality segment) and standardize curl examples on the Docker API port 1349. Verified with a clean docs build.
Move all 157 tool pages into image/video/audio/pdf/data subfolders so URLs read /tools/<modality>/<id> (e.g. /tools/image/crop). Nest the image sub-categories under an Image group in the sidebar so the nav reads by modality. Add public/_redirects (301, clean + .html forms) mapping every old flat /tools/<id> URL to its new path so inbound links keep working. Rewrite all internal /tools links. Verified with a clean docs build (no dead links).
The Release workflow failed at startup ("workflow file issue", no jobs
created) whenever dispatched. Root cause: release.yml sets top-level
`permissions: {}`, so the ai-bundles reusable-workflow call inherited an
empty scope set. ai-bundles.yml's jobs declare `contents: read` /
`packages: read`, and GitHub rejects a called workflow requesting scopes
the caller never granted -- validated at startup, before any job runs, so
the whole run fails immediately regardless of the job's `if:`.
Grant the two scopes on the call job. Confirmed on a throwaway branch: a
neutralized dispatch went from startup_failure to a clean run with
ai-bundles correctly skipped.
This unblocks every real release; main's release.yml could not start.
Fixes a backlog of integration/unit specs that asserted pre-2.0 behavior and
were failing CI (not caused by recent feature work):
- modality-aware empty-input error is 'No file(s) provided', not /no image/i
(rotate, border, crop, resize, smart-crop, edge-cases, adversarial-extended,
api, tool-factory-route)
- input validation rejects pre-enqueue with a clean 400 in 'error' (was a worker
422 in 'details'): create-zip, extract-zip, merge-csvs
- resolveToolPool defaults unknown tools to the system pool (pool-routing)
- /upload and fetch-urls accept non-image content, validated per-tool at process
time (api, fetch-urls)
- color-adjust legacy aliases were consolidated into adjust-colors: drop the
removed-alias tests; retarget the format-preservation tests
- xml-to-csv gracefully converts a single non-repeating record to a 1-row CSV
- dropzone is multimodal; image-only filtering is opt-in via fileFilter
- factory-multi-input: register the synthetic test tools in the catalog so they
route correctly (file modality for concat; image for the validation-prefix test)
Verified locally: unit 4546 passed, integration 8332 passed, typecheck + lint green.
rembg 2.0.70+ requires numpy>=2.3.0, but the AI bundle pins numpy==1.26.4
(mediapipe, realesrgan/basicsr, codeformer, paddle all need numpy<2). The
unresolvable rembg==2.0.75 + numpy==1.26.4 combination broke pip-audit's
dependency resolution (CI red) and the background-removal bundle build. 2.0.69
is the newest rembg with an unconstrained numpy requirement. Verified: pip-audit
resolves with no unignored vulnerabilities on Python 3.11.
The batch endpoint became modality-aware, so an empty request returns the
generic "No files provided" rather than the legacy image-specific message.
The assertion still matched /no image/i (a pre-existing stale spec on main),
so it failed against the correct behavior. Match /no files/i instead.
Bump all workspace package versions and APP_VERSION to 2.0.0, marking the
official 2.0 release. Removes the stale 1.x .release-notes.md artifact
(semantic-release regenerates release notes). The 2.0/multimodality docs
and rebrand already landed on main via #254 and #261, so this carries only
the version designation forward from the rebrand branch.
BREAKING CHANGE: SnapOtter 2.0 - the platform re-architecture (Postgres 17 +
Redis 8 + BullMQ durable jobs, 157 tools across five modalities) is the 2.0
release line, replacing the 1.x SQLite single-container architecture.
These suites only ever skipped (pandoc is absent in every test env), which hid three latent bugs once the binaries are reachable:
- epub-convert and to-epub are executionHint "long" (async: 202 + jobId), but the tests asserted a synchronous 200 and read the response downloadUrl. Rewrite to the 202 -> poll job row -> download pattern.
- The job-row reads selected schema.jobs.errorMessage, which is not a column (the field is `error` jsonb {message,details}); drizzle threw on the bad select. Read error/error.message instead. (The pdf-chain test had the same bug.)
- The SSRF test asserted the remote <img> URL passes through, but pandoc strips the unmanifested remote ref. Assert the security-relevant property instead: the conversion completes from the book's own content and never fetches+inlines the remote resource (no data: URI).
Verified by building the test image with pandoc: all epub/to-epub tests pass.
The tool factory created its per-request validation scratch dir at tmpdir()/snapotter-scratch/<jobId> -- the same path the BullMQ worker uses for job processing (worker.ts scratchRoot()/<jobId>) whenever SCRATCH_PATH is unset. For sync tools this is harmless because the factory blocks on the worker before its finally cleanup runs. For "long" tools the factory returns 202 immediately and then rm's that dir, racing the worker that is writing the job's input there -- so tools that stage input in the worker scratch dir (epub-convert, to-epub) intermittently failed with ENOENT. Production never hit this because it sets SCRATCH_PATH, diverging the two paths. Give the factory's validation dir a -prep suffix so its cleanup can never touch the worker's job dir.
pnpm test:docker ran pnpm test:ci (vitest --coverage), but the lean test image deliberately skips binary-gated tools (AI model bundles, LibreOffice, etc.), so it can never meet the host-calibrated coverage thresholds -- the container exited non-zero on coverage even with zero test failures. Point the compose command at vitest run so test:docker is a clean functional pass/fail gate; coverage stays enforced on host CI where every tool is present.
Two test-image gaps surfaced by a full pnpm test:ci run:
- s3-storage.test.ts imports @aws-sdk/client-s3 (an enterprise dependency) at module load, but Dockerfile.test never copied packages/enterprise/package.json before pnpm install, so the suite failed to collect. Copy it so the dep installs; the suite then skips cleanly when MinIO is absent.
- EPS batch decode returned 422: ImageMagick reads EPS through the Ghostscript PS coder, but policy.xml left PS/PS2/PS3 at rights=none (only EPS was opened), so convert refused with a policy error before Ghostscript ran. Open the PostScript coders too.
tests/setup/per-fork-env.ts hardcoded SYNC_WAIT_MS=30000 on every fork, overriding whatever the container set, so the docker test image could never grant heavy ops a wider sync window. A 12MP stress-image enhance takes ~34s on the macOS Docker VM (Sharp runs 2-3x slower there), just past the 30s window, so the factory returned 202 and three sync-asserting image-enhancement tests failed.
Honor a higher SYNC_WAIT_MS when provided (30s floor preserved for host/CI), raise it to 120s in docker-compose.test.yml, and make the vitest test/hook timeouts env-overridable so a slow-but-correct job returns 200 rather than tripping a framework timeout. Host and CI behavior is unchanged.
SnapOtter