All /api/ routes are already covered by the global @fastify/rate-limit
(index.ts:283), but CodeQL doesn't model the global registration and flagged
every route without an explicit per-route limit. Added tuned config.rateLimit
to 45 routes: stricter on auth/credential routes (mfa/saml/oidc/api-keys,
15-30/min), moderate on writes (60/min), generous on reads/polls (300/min =
the global default). Real defense-in-depth on sensitive routes. 3 alerts on
non-route code (a preHandler hook, the rate-limiter's own DB lookup, a test
helper) are documented false-positives covered by the global limiter.
* feat(infra): add dev compose stack with postgres and redis
* fix(infra): comment dev env defaults until wired; harden dev compose restart and start_period
* chore(deps): add pg driver and testcontainers for postgres migration
* feat(db): translate schema to drizzle pg-core (timestamptz, boolean, pgEnum, jsonb)
Schema translation (apps/api/src/db/schema.ts):
- sqlite-core -> pg-core, all 10 tables preserved 1:1
- integer(mode:'timestamp') -> timestamp({ withTimezone: true })
- integer(mode:'boolean') -> boolean
- jobs.status text enum -> pgEnum('job_status') with same 4 values
- 7 columns changed from text to jsonb: jobs.inputFiles, jobs.settings,
pipelines.steps, apiKeys.permissions, roles.permissions,
auditLog.details, userFiles.toolChain
- settings.value stays text, jobs.error stays text, jobs.progress stays real
jsonb call-site sweep (removed JSON.stringify on writes, JSON.parse on reads):
- apps/api/src/routes/roles.ts: permissions read/write (3 sites)
- apps/api/src/routes/api-keys.ts: permissions write + read (2 sites)
- apps/api/src/routes/audit-log.ts: details read (1 site)
- apps/api/src/routes/pipeline.ts: steps write + read (2 sites)
- apps/api/src/routes/progress.ts: inputFiles write (2 sites)
- apps/api/src/routes/tool-factory.ts: toolChain read + write (2 sites)
- apps/api/src/routes/user-files.ts: toolChain read + write (4 sites)
- apps/api/src/permissions.ts: roles.permissions read (1 site)
- apps/api/src/lib/audit.ts: details write (1 site)
- apps/api/src/plugins/auth.ts: apiKeys.permissions read (1 site)
* refactor(db): type jsonb columns via $type and note raw CTE conversion requirements
* feat(db): archive sqlite migrations and generate postgres baseline
* chore(db): dockerignore legacy migrations, add archive breadcrumb, fix trailing newline
* feat(db): pg pool connection, advisory-locked boot migrations, DATABASE_URL config
* fix(db): friendly fatal on unreachable postgres, idempotent closeDb, lock-key convention note
* refactor(db): async drizzle calls in plugins, lib, permissions
* fix(api): analytics never throws, typed permission guard, single-query session invalidation
* refactor(db): async drizzle calls across all routes and bootstrap
Convert every route file and index.ts from sync SQLite drizzle
patterns to async node-postgres drizzle:
- .all() removed (bare await on select)
- .get() converted to destructured [row] = await ...
- .run() removed (bare await on insert/update/delete)
- .changes replaced with .rowCount (null-guarded) in progress.ts
- sqlite import removed from user-files.ts; raw CTEs converted to
await db.execute(sql`...`) with postgres-dialect recursive CTEs
- ChainRow types updated: tool_chain is parsed jsonb (string[] | null),
created_at is Date (timestamptz) with no * 1000 conversion
- All requirePermission() guard calls awaited (security: unawaited
async guard returns truthy Promise, bypassing permission check)
- All hasEffectivePermission() and getPermissions() calls awaited
- All auditLog() calls awaited (preserves write-before-response order)
- trackEvent() and captureException() left un-awaited (fire-and-forget
by design, guaranteed never-throw)
- ensureAnonymousUser(), startCleanupCron(), recoverStaleJobs() awaited
in bootstrap sequence
- ensureInstanceId() and ensureDefaultSettings() made async
Files converted: 14 (index.ts + 12 route files + tools/index.ts)
* fix(db): await async checkStorageQuota in user-files upload/save routes
* fix(db): await checkStorageQuota in save-result route (missed second call site)
* feat(db): sqlite-to-postgres migrator with CLI and first-boot import
* fix(db): migrator error context, honest force semantics, boot-hook fatal, null-variance tests
* test: run suite against per-file postgres databases via testcontainers
- Add tests/global-setup.ts: spins up a Postgres testcontainer,
creates a migrated template database once per vitest run.
- Rewrite tests/setup/per-fork-env.ts: each test file (forks pool)
clones the template into its own database via CREATE DATABASE ...
TEMPLATE, preserving the same per-file isolation granularity.
- Update vitest.config.ts: add globalSetup, pg alias, update comment.
- Fix tests/integration/test-server.ts: remove DB_PATH mkdir, async
runMigrations, async db operations, remove SQLite WAL checkpoint.
- Fix 21 unit test db/index mocks: add pool and closeDb exports.
- Fix 8 unit test files: add async/await for now-async permission,
audit, and analytics functions.
- Fix 18 integration test files: convert sync .run()/.all()/.get()
to async drizzle patterns, add async to callbacks.
- Production change: apps/api/src/routes/teams.ts: cast COUNT(*)
to ::int so Postgres returns a number instead of bigint string.
* fix(db): seed built-in roles, reject NUL bytes, cast COUNT, serialize job persists
- Seed built-in roles (admin, editor, user) at boot via ensureBuiltinRoles()
with onConflictDoNothing, restoring data that legacy SQLite migration 0007
provided via INSERT statements (the pg baseline is DDL-only).
- Reject NUL bytes in login credentials with 401 (postgres rejects \x00 in
text columns; valid usernames never contain NUL, matching 1.x behavior).
- Cast COUNT(*)::int in user-files, audit-log, and roles listing queries so
postgres returns a JS number instead of bigint-as-string.
- Serialize fire-and-forget job progress DB writes per jobId so the final
"completed" status is never overwritten by a late-arriving "processing"
write (race condition exposed by async postgres round-trips).
* test: fix teams race, seed roles in test server, poll for job status
- Add missing await to resetTeams() in teams PUT beforeEach (the async
delete raced with the subsequent insert under postgres).
- Call ensureBuiltinRoles() in test server bootstrap so integration tests
have the same built-in roles as production.
- Replace fixed 100ms flushPersist delay with a polling helper that waits
for terminal job status, eliminating timing-dependent failures caused by
postgres network round-trip latency.
* test: make heic temp-file cleanup assertion resilient to concurrent workers
Use a set-based diff instead of raw file count when checking that
decodeHeic cleans up temp files. Other concurrent test workers can
create heic-in-*/heic-out-* files in the shared tmpdir, inflating the
"after" count and causing spurious failures under full-suite load.
* fix(db): align builtin-role seed to post-0010 legacy state; test polish
* feat(docker): three-container compose (app, postgres, redis) with boot wait and migrations
* fix(docker): set TEST_DATABASE_URL so containerized tests skip testcontainers
* chore(docker): test compose project name, clearer 1.x upgrade comment, unref probe timer
* feat(enterprise): enforce D15 license boundary; move s3 storage into packages/enterprise
* fix(enterprise): restore lazy aws-sdk loading; community installs load no s3 code at boot
* fix(enterprise): boundary check catches dynamic imports; document getS3 concurrency
* feat(db)!: SnapOtter 2.0 phase 1 foundation: postgres, migrator, compose stack
BREAKING CHANGE: SQLite is no longer the runtime database. Deployments now
require Postgres (and Redis, used from phase 2). Existing installs migrate
with SQLITE_MIGRATE_PATH or 'pnpm --filter @snapotter/api migrate:sqlite'.
* fix(ci): postgres service + fresh e2e database per run; ignore unfixable torch CVE-2025-3000
* fix: extend SSE result retention from 2 to 10 minutes for mobile recovery
* fix: check server health on tab visibility change for mobile recovery
* fix: recover feature install SSE on tab visibility change
* fix: reconnect SSE on tab visibility change in tool processor
* fix: reconnect SSE on tab visibility change in pipeline processor
* test: add visibility recovery tests for connection monitor
The Open File button in the Files section did nothing due to a race
condition where the home page reset the file store on mount before files
from handleOpenFile could render. Upload on the files page used fetch
with no timeout, progress, or retry, causing silent failures on mobile
and slow connections. SSE connections for job progress had no keepalive
pings, allowing reverse proxies to kill idle streams.
* feat: add 52 per-tool SEO landing pages and public changelog
- Create individual landing pages for all 52 tools at /tools/{slug}
with search-intent title tags, unique FAQs (156 Q&As), per-tool
features, and rich schema markup (BreadcrumbList, WebApplication,
HowTo, FAQPage)
- Replace static sitemap.xml with dynamic generation including all
tool URLs
- Refactor bento grid to import from @snapotter/shared (eliminates
330 lines of duplicated tool data) and link cards to tool pages
- Add @snapotter/shared as workspace dependency to landing site
- Add public changelog page to docs site with curated release notes
from v1.8 through v1.17
- Update docs nav and sidebar with changelog link
* feat: enhance SEO and performance with updated metadata, robots.txt, and llms.txt
- Fix SSE write-after-end crash in progress.ts (remove callback before ending stream)
- Fix blob URL memory leaks: revoke processedPreviewUrl and old HEIC preview URLs
- Add AbortController to batch fetch in use-tool-processor and use-pipeline-processor
- Fix TGA format misidentified as CUR (extension overrides magic bytes)
- Add libheif-plugin-libde265 to Docker for HEIC/HEIF decode support
- Remove unused imports and state (AppLayout, setSampledColor, useEffect)
- Fix non-null assertions in meme-text-renderer and meme-generator
- Fix confusing void type in meme-templates
- Remove unnecessary useEffect deps in adjustments-panel
- Fix Playwright strict mode violations in 5 E2E tests
The upscale route held the HTTP connection open for the full duration of
Python sidecar processing (30-300s). Behind proxies with connection
timeouts (Cloudflare Tunnel: 100s), this caused HTTP 524 errors.
The route now returns 202 Accepted immediately after upload validation
and processes in the background. The result (downloadUrl, sizes, etc.)
is delivered via the existing SSE progress channel. The frontend detects
the 202 and waits for the SSE completion event instead of reading the
XHR response body. A reconnect-safe completion store ensures results
survive brief SSE disconnects.
Closes#106
Batch progress was broken because JobProgress events lacked a `type`
field. The frontend checks `data.type === "batch"` to distinguish batch
from single-file SSE events, so batch progress was silently discarded
and multi-file processing appeared stuck at 15%.
Also improves the processing UX for non-AI (Sharp-based) tools: the
progress bar now pulses during the server processing phase and shows
a "This may take a moment" hint after 10 seconds.
Co-authored-by: Siddharth Kumar Sah <siddharth123sk@gmail.com>
- Remove @fastify/swagger and @fastify/swagger-ui (API docs live on GitHub Pages)
- Run typecheck, build, and docker CI jobs in parallel instead of sequentially
Backend: POST /api/v1/tools/:toolId/batch accepts multiple files +
settings, processes via p-queue with CONCURRENT_JOBS concurrency limit,
streams ZIP response using archiver. Tool registry in tool-factory
enables batch to reuse any registered tool's process function. SSE
endpoint at GET /api/v1/jobs/:jobId/progress provides real-time updates.
Handles partial failures gracefully, preserves filenames, deduplicates
collisions. Frontend: use-batch-processor hook handles upload, SSE
progress tracking, and automatic ZIP download.
