Sharp 0.35.1 moved FormatEnum to a namespace export and removed "avif"
from FormatEnum (now a separate literal in toFormat). BullMQ 5.78.1
bundles ioredis 5.10.1 while we have 5.11.1, causing structural type
mismatch. Also fixes new Biome 1.9 lint rules.
- Remove obsolete fullscreen redirect test (route deleted in Task 3)
- Update dropzone test strings to match new i18n values
- Fix categories count in i18n locale test (12 -> 24)
- Add chart-maker to no-dropzone exempt list in tool-registry test
- Auto-fix import ordering (Biome) in popular tools route and landing config
* feat(infra): add dev compose stack with postgres and redis
* fix(infra): comment dev env defaults until wired; harden dev compose restart and start_period
* chore(deps): add pg driver and testcontainers for postgres migration
* feat(db): translate schema to drizzle pg-core (timestamptz, boolean, pgEnum, jsonb)
Schema translation (apps/api/src/db/schema.ts):
- sqlite-core -> pg-core, all 10 tables preserved 1:1
- integer(mode:'timestamp') -> timestamp({ withTimezone: true })
- integer(mode:'boolean') -> boolean
- jobs.status text enum -> pgEnum('job_status') with same 4 values
- 7 columns changed from text to jsonb: jobs.inputFiles, jobs.settings,
pipelines.steps, apiKeys.permissions, roles.permissions,
auditLog.details, userFiles.toolChain
- settings.value stays text, jobs.error stays text, jobs.progress stays real
jsonb call-site sweep (removed JSON.stringify on writes, JSON.parse on reads):
- apps/api/src/routes/roles.ts: permissions read/write (3 sites)
- apps/api/src/routes/api-keys.ts: permissions write + read (2 sites)
- apps/api/src/routes/audit-log.ts: details read (1 site)
- apps/api/src/routes/pipeline.ts: steps write + read (2 sites)
- apps/api/src/routes/progress.ts: inputFiles write (2 sites)
- apps/api/src/routes/tool-factory.ts: toolChain read + write (2 sites)
- apps/api/src/routes/user-files.ts: toolChain read + write (4 sites)
- apps/api/src/permissions.ts: roles.permissions read (1 site)
- apps/api/src/lib/audit.ts: details write (1 site)
- apps/api/src/plugins/auth.ts: apiKeys.permissions read (1 site)
* refactor(db): type jsonb columns via $type and note raw CTE conversion requirements
* feat(db): archive sqlite migrations and generate postgres baseline
* chore(db): dockerignore legacy migrations, add archive breadcrumb, fix trailing newline
* feat(db): pg pool connection, advisory-locked boot migrations, DATABASE_URL config
* fix(db): friendly fatal on unreachable postgres, idempotent closeDb, lock-key convention note
* refactor(db): async drizzle calls in plugins, lib, permissions
* fix(api): analytics never throws, typed permission guard, single-query session invalidation
* refactor(db): async drizzle calls across all routes and bootstrap
Convert every route file and index.ts from sync SQLite drizzle
patterns to async node-postgres drizzle:
- .all() removed (bare await on select)
- .get() converted to destructured [row] = await ...
- .run() removed (bare await on insert/update/delete)
- .changes replaced with .rowCount (null-guarded) in progress.ts
- sqlite import removed from user-files.ts; raw CTEs converted to
await db.execute(sql`...`) with postgres-dialect recursive CTEs
- ChainRow types updated: tool_chain is parsed jsonb (string[] | null),
created_at is Date (timestamptz) with no * 1000 conversion
- All requirePermission() guard calls awaited (security: unawaited
async guard returns truthy Promise, bypassing permission check)
- All hasEffectivePermission() and getPermissions() calls awaited
- All auditLog() calls awaited (preserves write-before-response order)
- trackEvent() and captureException() left un-awaited (fire-and-forget
by design, guaranteed never-throw)
- ensureAnonymousUser(), startCleanupCron(), recoverStaleJobs() awaited
in bootstrap sequence
- ensureInstanceId() and ensureDefaultSettings() made async
Files converted: 14 (index.ts + 12 route files + tools/index.ts)
* fix(db): await async checkStorageQuota in user-files upload/save routes
* fix(db): await checkStorageQuota in save-result route (missed second call site)
* feat(db): sqlite-to-postgres migrator with CLI and first-boot import
* fix(db): migrator error context, honest force semantics, boot-hook fatal, null-variance tests
* test: run suite against per-file postgres databases via testcontainers
- Add tests/global-setup.ts: spins up a Postgres testcontainer,
creates a migrated template database once per vitest run.
- Rewrite tests/setup/per-fork-env.ts: each test file (forks pool)
clones the template into its own database via CREATE DATABASE ...
TEMPLATE, preserving the same per-file isolation granularity.
- Update vitest.config.ts: add globalSetup, pg alias, update comment.
- Fix tests/integration/test-server.ts: remove DB_PATH mkdir, async
runMigrations, async db operations, remove SQLite WAL checkpoint.
- Fix 21 unit test db/index mocks: add pool and closeDb exports.
- Fix 8 unit test files: add async/await for now-async permission,
audit, and analytics functions.
- Fix 18 integration test files: convert sync .run()/.all()/.get()
to async drizzle patterns, add async to callbacks.
- Production change: apps/api/src/routes/teams.ts: cast COUNT(*)
to ::int so Postgres returns a number instead of bigint string.
* fix(db): seed built-in roles, reject NUL bytes, cast COUNT, serialize job persists
- Seed built-in roles (admin, editor, user) at boot via ensureBuiltinRoles()
with onConflictDoNothing, restoring data that legacy SQLite migration 0007
provided via INSERT statements (the pg baseline is DDL-only).
- Reject NUL bytes in login credentials with 401 (postgres rejects \x00 in
text columns; valid usernames never contain NUL, matching 1.x behavior).
- Cast COUNT(*)::int in user-files, audit-log, and roles listing queries so
postgres returns a JS number instead of bigint-as-string.
- Serialize fire-and-forget job progress DB writes per jobId so the final
"completed" status is never overwritten by a late-arriving "processing"
write (race condition exposed by async postgres round-trips).
* test: fix teams race, seed roles in test server, poll for job status
- Add missing await to resetTeams() in teams PUT beforeEach (the async
delete raced with the subsequent insert under postgres).
- Call ensureBuiltinRoles() in test server bootstrap so integration tests
have the same built-in roles as production.
- Replace fixed 100ms flushPersist delay with a polling helper that waits
for terminal job status, eliminating timing-dependent failures caused by
postgres network round-trip latency.
* test: make heic temp-file cleanup assertion resilient to concurrent workers
Use a set-based diff instead of raw file count when checking that
decodeHeic cleans up temp files. Other concurrent test workers can
create heic-in-*/heic-out-* files in the shared tmpdir, inflating the
"after" count and causing spurious failures under full-suite load.
* fix(db): align builtin-role seed to post-0010 legacy state; test polish
* feat(docker): three-container compose (app, postgres, redis) with boot wait and migrations
* fix(docker): set TEST_DATABASE_URL so containerized tests skip testcontainers
* chore(docker): test compose project name, clearer 1.x upgrade comment, unref probe timer
* feat(enterprise): enforce D15 license boundary; move s3 storage into packages/enterprise
* fix(enterprise): restore lazy aws-sdk loading; community installs load no s3 code at boot
* fix(enterprise): boundary check catches dynamic imports; document getS3 concurrency
* feat(db)!: SnapOtter 2.0 phase 1 foundation: postgres, migrator, compose stack
BREAKING CHANGE: SQLite is no longer the runtime database. Deployments now
require Postgres (and Redis, used from phase 2). Existing installs migrate
with SQLITE_MIGRATE_PATH or 'pnpm --filter @snapotter/api migrate:sqlite'.
* fix(ci): postgres service + fresh e2e database per run; ignore unfixable torch CVE-2025-3000
Closes the "e2e never runs in CI" hole. Adds per-PR e2e smoke gate,
nightly full-suite workflows, parallel vitest forks (per-fork DBs),
Playwright parallel/serial/visual projects against production builds,
metadata-generated test suites (drift guards, hostile inputs, format
matrix, pairwise settings, property-based fuzz), Stryker mutation
testing, Schemathesis API fuzz, coverage ratchet, and fixes for three
session-poisoning bugs that caused 200+ serial-bucket failures.
Bug fix included: favicon/split/bulk-rename could hang clients forever
when ZIP streaming failed after reply.hijack().
- Create 53 per-tool VitePress documentation pages with accurate
parameters from Zod schemas, example requests, and response formats
- Add root llms.txt for LLM-friendly repo browsing
- Fix OpenAPI spec: add auth and 422 error schemas to
edit-metadata/inspect and strip-metadata/inspect sub-routes
- Fix tool count inconsistency (52 -> 53) across landing site,
e2e tests, and local docs
- Rename color-adjustments.ts to adjust-colors.ts to match tool ID
- Update VitePress sidebar with all 8 tool categories and top nav
- Fix selective metadata stripping (P1): use Sharp's keepExif()/keepIccProfile()
instead of broken withMetadata({}) that preserved everything
- Fix meme font mapping (P1): add ArchivoBlack and ComicNeue fonts, map
arial-black and comic-sans to correct TTF files instead of Anton
- Fix meme contentType (P2): detect actual output format from Sharp metadata
instead of hardcoding image/png
- Fix info/text-overlay/color-palette i18n (P2): wire up existing translation
keys that were imported but never used
- Fix info and color-palette displayMode (P2): change from before-after to
no-comparison since neither tool produces a processed image
- Add missing i18n keys across all 21 locales
- Update displayMode test assertions
- Expose birefnet-hr-matting in UI (People/Ultra) and fix model defaults
(People/Max now uses birefnet-matting for true alpha matting)
- Add output format selector (PNG/WebP/AVIF) with lossless alpha support
- Add edge smoothing post-processing (Off/Light/Medium/Strong) via
morphological mask refinement to reduce gray halo artifacts
- Add color decontamination to remove background color spill from
semi-transparent edge pixels
- Thread new settings through full stack: frontend -> API schema ->
Python sidecar -> Sharp effects pipeline
- Add i18n keys for all 21 locales
- Add unit tests for new option serialization (3 tests)
- Add integration tests for new settings validation (4 tests)
Sharp can read HEIF container headers (metadata succeeds) but cannot
decode HEVC pixels (stats fails with "No decoding plugin installed").
Unconditionally pre-decode HEIF files via heif-dec CLI before any Sharp
operations, matching the createToolRoute factory behavior.
- Filter known client-error noise (rate limit, empty body, unsupported
media type, content-length mismatch, premature close) from Sentry
via beforeSend to stop 644 events of non-actionable noise
- Sanitize x-output-filename header to prevent TypeError on non-ASCII
filenames in optimize-for-web preview (23 events)
- Handle EPIPE on Python dispatcher stdin write with graceful fallback
to per-request spawning instead of crashing (NODE-W)
- Map EACCES on storage directory/file write to proper 503 status
instead of generic 500 (NODE-P, 3 events)
- fix(db): migration 0012 column order mismatch causing NOT NULL
constraint failure on existing databases; use explicit column
mapping instead of SELECT *
- fix(db): disable FK checks during migrations to allow SQLite
table-recreation pattern (DROP + RENAME)
- fix(security): filter cookie_secret and instance_id from settings
API response for non-admin users
- fix(lint): resolve all 7 API lint warnings (noParameterAssign,
noImplicitAnyLet) in compose, image-enhancement, and workspace
- fix(docs): correct permission count from 16 to 14 in CLAUDE.md
- fix(e2e): resolve 44 Playwright test failures across 8 spec files
including locator specificity, compress mode defaults, format count,
restore-photo UI drift, stitch image count, GIF animated fixtures,
submit button timing, and processing timeouts
Tests expected 422 for corrupted image data, but the API correctly
returns 400 since corruption is caught during validation (unrecognized
format), not during processing. Also fix watermark-image route passing
a hardcoded "watermark" string instead of the actual uploaded filename
to validateImageBuffer, which broke SVGZ detection.
validateImageBuffer catches corrupt image data before processing
reaches the tool handler, so the correct status code is 400 (bad
request) rather than 422 (processing failure). Also fix SVGZ
watermark validation by returning early for compressed SVG (Sharp
cannot read gzip-compressed SVGZ directly) and passing the actual
watermark filename to validateImageBuffer for correct format
detection.
Auth: login rate limit 30/min (was 500), global rate limit 1000/min (was
unlimited), password/username max lengths on all Zod schemas, session
invalidation on role change, API key legacy scan bounded to 100 keys.
SVG: hardened regex sanitizer with CDATA stripping, XML entity decoding,
set/animate/iframe/embed blocking, comprehensive data: URI blocking,
use element external href blocking. 11 attack payload fixtures added.
SSRF: fixed DNS rebinding TOCTOU by pinning resolved IPs via custom
HTTP/HTTPS agents. Added 6to4 and NAT64 to blocked IPv6 ranges.
Docker: capability dropping (cap_drop ALL + minimal cap_add), resource
limits (4g/8g mem, 512/1024 pids), healthcheck timeout, password
removed from startup banner, default password warning comments.
Network: CSP and HSTS applied in all environments (not just production),
stack traces removed from all error responses, internal paths stripped
from error details, per-route rate limits on uploads (60/min) and URL
fetches (200/hour).
Files: exclusive temp file creation (O_EXCL), disk space circuit
breaker, per-user storage quotas, settings payload 64KB size guard.
Python sidecar: script name allowlist in dispatcher, minimal environment
for subprocess spawns.
Dependencies: fixed 6 production CVEs (drizzle-orm, fastify, fast-uri,
@fastify/static, next, archiver/lodash). Pinned all GitHub Actions to
SHA hashes.
114 security tests added. Full OWASP Top 10 penetration test matrix
verified against production Docker container (30/30 pass after
hardening).
The transparency-fixer now directly detects the baked-in checkerboard
pattern using per-pixel chroma analysis instead of BiRefNet AI matting.
Achromatic pixels in the gray range are classified as background
(transparent), chromatic pixels as foreground (opaque), with smooth
transitions at anti-aliased edges.
- No longer requires Python sidecar or background-removal bundle
- Watermark removal uses Sharp median(5) filter pre-processing
- Moved tool from "ai" to "utilities" category
- Removed from PYTHON_SIDECAR_TOOLS and background-removal enablesTools
- Near-instant processing (pure Sharp, no model inference)
The luminance anomaly detection + LaMa inpainting approach failed because
watermark signal on the matted foreground was too weak (10-15 units vs
threshold of 25). Median filter with kernel=5 effectively removes
semi-transparent watermark text while preserving the stamp structure.
Pipeline is now: median filter (if toggle on) -> BiRefNet matting -> defringe.
No longer requires object-eraser-colorize bundle for watermark removal.
Add tier enum (fast/balanced/high) with balanced default to the Zod
settings schema. Pass tier through to outpaint options in both the HTTP
route and the pipeline/batch registry. Fix log message to say
"Starting AI canvas expand" and include tier in structured log fields.
- Fix dispatcher pipe deadlock: drain stdout pipe in a background thread
to prevent blocking when ONNX runtime output exceeds 64KB pipe buffer
- Add 5-minute SSE stall timeout so the UI shows an error instead of
hanging forever when async AI processing stalls
- Guard CPU colorization: skip for images >2MP on CPU and when DDColor
model is not installed, with clear user-facing messages
- Add AVIF decode fallback via ImageMagick for bitstream variants that
Sharp's bundled libheif cannot decode (affects all tools)
HDR and EXR files decoded by ImageMagick can produce 16-bit PNG buffers.
Sharp's CLAHE operation (hist_local) requires VIPS_FORMAT_UCHAR (8-bit).
Check the buffer depth and convert to 8-bit sRGB before processing.
Sharp's TIFF encoder silently strips the alpha channel, flattening
transparency against black. This caused border (corner radius, shadow),
beautify, and replace-color tools to produce wrong output for TIFF
inputs when the operation needs transparency. Remove TIFF from
ALPHA_FORMATS so these tools fall back to PNG output.
The find-duplicates tool failed entirely when any uploaded file couldn't
be processed, returning "Duplicate detection failed" or a format-specific
error that aborted the whole batch. With mixed-format uploads (77 files),
this made the tool unusable.
- Skip unprocessable files instead of aborting; return skippedFiles in response
- Switch from fetch() to XHR with upload progress tracking (Uploading X%)
- Add Vite proxy timeout config (5min) to prevent connection drops on large uploads
- Add "Download Grouped" button: ZIP with each duplicate group in its own folder
- Add collapsible skipped-files section in the results UI
- Add 3 integration tests for skip behavior (43 total)
Custom-route tools (split, compare, collage, find-duplicates, etc.) only
handled HEIC via ensureSharpCompat, failing on BMP, PSD, RAW, TGA, EXR,
HDR, JXL, and other formats Sharp cannot decode natively. Added the full
decode pipeline from createToolRoute to all 16 affected routes: format
validation via validateImageBuffer, HEIC decoding with actionable error
messages, CLI-based exotic format decoding with nested fallback, and SVG
sanitization to prevent XXE/SSRF/script injection.
The vectorize tool had a custom processing flow that bypassed the
standard useToolProcessor hook -- no progress indication, no server-side
batch, and the Download All ZIP relied on a client-side sequential loop.
Backend: extract core logic into vectorizeBuffer(), register via
registerToolProcessFn() so the /batch endpoint works with p-queue
concurrency and SSE progress events.
Frontend: replace custom fetch loop with useToolProcessor hook and
ProgressCard, giving upload progress, per-file batch status, and
automatic Download All ZIP via the existing tool-page infrastructure.
Also set image/svg+xml MIME type on SVG blobs during batch ZIP
extraction to ensure reliable rendering in <img> tags across browsers.
The stitch route only decoded HEIC via ensureSharpCompat, so exotic
formats (DNG, PSD, TGA, BMP, JXL, HDR, QOI, DDS, ICO, JP2, DPX, etc.)
crashed Sharp at metadata read time, causing "Stitch creation failed".
Add the full CLI decode pipeline (decodeToSharpCompat) matching the
tool-factory pattern, plus SVG sanitization with proper error handling
for each format category.
Replace fetch() with XMLHttpRequest in the frontend to surface upload
progress via onprogress events, and add a progress bar that shows upload
percentage then pulses during server-side stitching.
The X-File-Results header contained raw JSON with non-ASCII characters
from filenames (Chinese, Japanese, etc.), violating RFC 7230. Node.js
threw ERR_INVALID_CHAR on writeHead(). Fixed by wrapping the JSON in
encodeURIComponent() on the backend and decodeURIComponent() on the
frontend, ensuring only ASCII goes into the header while preserving
the original filenames after decoding.
Closes#133
Sharp can read TIFF-based RAW files (DNG, CR2, NEF) directly for
metadata without requiring ImageMagick/darktable to fully decode them.
Try Sharp on the raw buffer first; only fall back to the decode pipeline
for formats Sharp cannot open (PSD, ICO, TGA). For RAW files, enrich
with ExifTool to get real sensor dimensions instead of thumbnail size.
