- Add POST/DELETE /api/v1/enterprise/scim/token for SCIM bearer token management
- Cancel active BullMQ jobs via requestCancel() before GDPR purge deletes DB rows
- Reject duplicate MFA enrollment when a pending (unverified) secret exists
- Add webhook_destinations to config export REDACTED_KEYS (contains auth headers)
- Validate DATA_ENCRYPTION_KEY and DATA_ENCRYPTION_KEY_PREVIOUS are 64-char hex at startup
- Add 1000 req/min Redis counter rate limit to SCIM auth middleware
- Document SIEM/webhook system coexistence in siem-forward.ts
Redis sliding window (sorted set) enforces per-user request rate limits
via the rateLimitPerUser DB setting. Concurrent job limits checked at
the HTTP layer before enqueue via maxConcurrentJobsPerUser setting.
Both default to 0 (unlimited).
Handle MFA challenge in login flow (TOTP code input after password
verification, recovery code hint, back navigation). Add admin security
settings section with session idle timeout, max sessions per user,
MFA policy selector, SSO enforcement toggle with break-glass username,
and password policy controls. Propagate new i18n keys to all 21 locales.
Move user resolution logic (match by externalId, auto-link by email,
auto-create with user limit check) into a shared module that both
OIDC and SAML callbacks can use. Includes sanitizeUsername and
findUniqueUsername helpers. Preserves all existing OIDC behavior
and audit events.
Idle timeout: reads `sessionIdleTimeoutMinutes` from settings, tracks
last activity in Redis (with Postgres fallback on cache miss), and
invalidates sessions that exceed the configured idle window.
Concurrent session limit: reads `maxSessionsPerUser` from settings
and evicts oldest sessions (FIFO) when a new login exceeds the cap.
Both features are opt-in (disabled when value is 0 or absent).
Add per-team storage breakdown table to the usage section (from
GET /api/v1/admin/usage teamStorage data). Add storage quota (MB)
and retention (hours) inline fields to the teams section, saving
via PUT /api/v1/teams/:id. Add i18n keys to all 21 locale files.
Compute a deleteAfter timestamp on job creation when the enterprise
team_retention_overrides feature is enabled. The cleanup sweep now
deletes storage for jobs past their deleteAfter deadline, running
independently of the global TTL setting.
Increment users.storageUsed on file upload/save, decrement on delete
(per-user via GREATEST to prevent negatives). Add per-team storage
breakdown to GET /api/v1/admin/usage. Weekly reconciliation job
(3 AM Sunday) recomputes counters from actual userFiles sums.
Add isToolAuditEnabled() helper that checks the auditToolOperations
DB setting (off by default) or falls back to the enterprise
audit_export feature flag. The createToolRoute factory now emits a
TOOL_EXECUTED audit entry on successful tool execution when enabled,
using a fire-and-forget pattern so a failed audit write never blocks
the tool response.
The home-page preview panel always rendered ImageViewer regardless of the
uploaded file's modality, causing videos, audio, PDFs, and data files to
show "Preview not available". Root cause: the preview branching only
checked for a blob URL and assumed all files were images.
Fix: branch on currentEntry.previewKind so the existing MediaPlayerView,
DocumentView, and a file-info fallback are activated for their respective
modalities. Also widen the AppLayout dropzone from image-only to all file
types so non-image files can reach the home page in the first place.
Add horizontal filter tabs (All, Image, Video, Audio, Documents, Files)
above the search bar so users can instantly scope the 157-tool catalog to
one modality. Active tab uses the modality accent color. In "all" mode
file-modality tools remain merged into the document section; selecting a
specific modality tab shows only that modality with its own header.
Add a row of filter tabs (All, Image, Video, Audio, Documents,
Data & Files) at the top of the fullscreen tool grid. Selecting a
tab filters the visible tools by modality, and the existing search
still works within the filtered set.
Also registers the Wand2 icon needed by the new video-effects
category header.
Static data-driven test covering 90 tools with 551 variations. For each
tool that has configurable settings, every enum value, boundary number,
and boolean flag is exercised through the HTTP endpoint. The test asserts
no 500s occur and validates response shape for 200/202 results.
Coverage spans image (29 tools), video (20), audio (15), PDF/document
(21), and data (5) modalities.
Edge worker 301-redirects www.snapotter.com to snapotter.com to stop
Google from splitting domain authority. Adds X-Robots-Tag: noindex on
/_next/static/ and /_next/data/ paths to reclaim crawl budget wasted
on JS chunks. Adds snapotter.com website badge to README so the
highest-authority backlink (GitHub) points to the main site.
The refactored CTE used = ANY(${validIds}::uuid[]) which Drizzle
serializes as a scalar text parameter with a ::uuid[] cast. Since
the user_files.id column is text (not uuid), PostgreSQL rejects
the text = uuid comparison. Replace with IN (...) via sql.join()
to pass each ID as a plain text parameter matching the column type.
Closes#71. Several error paths in the worker could leak internal
filesystem paths (/tmp/workspace, /data/ai/venv, /app) through SSE
frames, resultPayload objects, and Redis batch-error lists. The
existing stripInternalPaths call at worker.ts line 345 only covered
the single-file processToolJob catch block.
Wrapped 6 additional call sites:
- processPipelineStep: prevError from DB and catch errorMsg
- processPipelineFinalize: composed errorMsg reaching SSE,
recordChildOutcome, and resultPayload
- processBatchChild: catch error reaching recordChildOutcome
and resultPayload
- processBatchFinalize: manifest errorMsg from DB rows
- recordChildOutcome (batch-progress.ts): defense-in-depth
strip before Redis rpush
Added 10 unit tests for stripInternalPaths covering /tmp, /data,
/app, /opt, /home, /workspace, multi-path messages, safe passthrough,
and pipeline-step wrapping.
On 1.x-to-2.0 upgrade the /data/ai/venv already existed with stale
packages, so the entrypoint skipped the cp from /opt/venv and 2.0
Python tools failed with ImportError (#85).
Dockerfile now writes a SHA-256 of `pip freeze` to /opt/venv/.venv-version
at build time. The entrypoint compares that stamp against the copy on the
volume; on mismatch it does a clean nuke + re-copy and resets
installed.json so the UI correctly shows bundles as needing reinstall.
Models in /data/ai/models survive the refresh, so bundle reinstall only
reruns pip (model downloads are idempotent).
Backward compatible: if the image has no stamp (old build), the
entrypoint falls through to the existing skip-if-exists behavior.
* feat(infra): add dev compose stack with postgres and redis
* fix(infra): comment dev env defaults until wired; harden dev compose restart and start_period
* chore(deps): add pg driver and testcontainers for postgres migration
* feat(db): translate schema to drizzle pg-core (timestamptz, boolean, pgEnum, jsonb)
Schema translation (apps/api/src/db/schema.ts):
- sqlite-core -> pg-core, all 10 tables preserved 1:1
- integer(mode:'timestamp') -> timestamp({ withTimezone: true })
- integer(mode:'boolean') -> boolean
- jobs.status text enum -> pgEnum('job_status') with same 4 values
- 7 columns changed from text to jsonb: jobs.inputFiles, jobs.settings,
pipelines.steps, apiKeys.permissions, roles.permissions,
auditLog.details, userFiles.toolChain
- settings.value stays text, jobs.error stays text, jobs.progress stays real
jsonb call-site sweep (removed JSON.stringify on writes, JSON.parse on reads):
- apps/api/src/routes/roles.ts: permissions read/write (3 sites)
- apps/api/src/routes/api-keys.ts: permissions write + read (2 sites)
- apps/api/src/routes/audit-log.ts: details read (1 site)
- apps/api/src/routes/pipeline.ts: steps write + read (2 sites)
- apps/api/src/routes/progress.ts: inputFiles write (2 sites)
- apps/api/src/routes/tool-factory.ts: toolChain read + write (2 sites)
- apps/api/src/routes/user-files.ts: toolChain read + write (4 sites)
- apps/api/src/permissions.ts: roles.permissions read (1 site)
- apps/api/src/lib/audit.ts: details write (1 site)
- apps/api/src/plugins/auth.ts: apiKeys.permissions read (1 site)
* refactor(db): type jsonb columns via $type and note raw CTE conversion requirements
* feat(db): archive sqlite migrations and generate postgres baseline
* chore(db): dockerignore legacy migrations, add archive breadcrumb, fix trailing newline
* feat(db): pg pool connection, advisory-locked boot migrations, DATABASE_URL config
* fix(db): friendly fatal on unreachable postgres, idempotent closeDb, lock-key convention note
* refactor(db): async drizzle calls in plugins, lib, permissions
* fix(api): analytics never throws, typed permission guard, single-query session invalidation
* refactor(db): async drizzle calls across all routes and bootstrap
Convert every route file and index.ts from sync SQLite drizzle
patterns to async node-postgres drizzle:
- .all() removed (bare await on select)
- .get() converted to destructured [row] = await ...
- .run() removed (bare await on insert/update/delete)
- .changes replaced with .rowCount (null-guarded) in progress.ts
- sqlite import removed from user-files.ts; raw CTEs converted to
await db.execute(sql`...`) with postgres-dialect recursive CTEs
- ChainRow types updated: tool_chain is parsed jsonb (string[] | null),
created_at is Date (timestamptz) with no * 1000 conversion
- All requirePermission() guard calls awaited (security: unawaited
async guard returns truthy Promise, bypassing permission check)
- All hasEffectivePermission() and getPermissions() calls awaited
- All auditLog() calls awaited (preserves write-before-response order)
- trackEvent() and captureException() left un-awaited (fire-and-forget
by design, guaranteed never-throw)
- ensureAnonymousUser(), startCleanupCron(), recoverStaleJobs() awaited
in bootstrap sequence
- ensureInstanceId() and ensureDefaultSettings() made async
Files converted: 14 (index.ts + 12 route files + tools/index.ts)
* fix(db): await async checkStorageQuota in user-files upload/save routes
* fix(db): await checkStorageQuota in save-result route (missed second call site)
* feat(db): sqlite-to-postgres migrator with CLI and first-boot import
* fix(db): migrator error context, honest force semantics, boot-hook fatal, null-variance tests
* test: run suite against per-file postgres databases via testcontainers
- Add tests/global-setup.ts: spins up a Postgres testcontainer,
creates a migrated template database once per vitest run.
- Rewrite tests/setup/per-fork-env.ts: each test file (forks pool)
clones the template into its own database via CREATE DATABASE ...
TEMPLATE, preserving the same per-file isolation granularity.
- Update vitest.config.ts: add globalSetup, pg alias, update comment.
- Fix tests/integration/test-server.ts: remove DB_PATH mkdir, async
runMigrations, async db operations, remove SQLite WAL checkpoint.
- Fix 21 unit test db/index mocks: add pool and closeDb exports.
- Fix 8 unit test files: add async/await for now-async permission,
audit, and analytics functions.
- Fix 18 integration test files: convert sync .run()/.all()/.get()
to async drizzle patterns, add async to callbacks.
- Production change: apps/api/src/routes/teams.ts: cast COUNT(*)
to ::int so Postgres returns a number instead of bigint string.
* fix(db): seed built-in roles, reject NUL bytes, cast COUNT, serialize job persists
- Seed built-in roles (admin, editor, user) at boot via ensureBuiltinRoles()
with onConflictDoNothing, restoring data that legacy SQLite migration 0007
provided via INSERT statements (the pg baseline is DDL-only).
- Reject NUL bytes in login credentials with 401 (postgres rejects \x00 in
text columns; valid usernames never contain NUL, matching 1.x behavior).
- Cast COUNT(*)::int in user-files, audit-log, and roles listing queries so
postgres returns a JS number instead of bigint-as-string.
- Serialize fire-and-forget job progress DB writes per jobId so the final
"completed" status is never overwritten by a late-arriving "processing"
write (race condition exposed by async postgres round-trips).
* test: fix teams race, seed roles in test server, poll for job status
- Add missing await to resetTeams() in teams PUT beforeEach (the async
delete raced with the subsequent insert under postgres).
- Call ensureBuiltinRoles() in test server bootstrap so integration tests
have the same built-in roles as production.
- Replace fixed 100ms flushPersist delay with a polling helper that waits
for terminal job status, eliminating timing-dependent failures caused by
postgres network round-trip latency.
* test: make heic temp-file cleanup assertion resilient to concurrent workers
Use a set-based diff instead of raw file count when checking that
decodeHeic cleans up temp files. Other concurrent test workers can
create heic-in-*/heic-out-* files in the shared tmpdir, inflating the
"after" count and causing spurious failures under full-suite load.
* fix(db): align builtin-role seed to post-0010 legacy state; test polish
* feat(docker): three-container compose (app, postgres, redis) with boot wait and migrations
* fix(docker): set TEST_DATABASE_URL so containerized tests skip testcontainers
* chore(docker): test compose project name, clearer 1.x upgrade comment, unref probe timer
* feat(enterprise): enforce D15 license boundary; move s3 storage into packages/enterprise
* fix(enterprise): restore lazy aws-sdk loading; community installs load no s3 code at boot
* fix(enterprise): boundary check catches dynamic imports; document getS3 concurrency
* feat(db)!: SnapOtter 2.0 phase 1 foundation: postgres, migrator, compose stack
BREAKING CHANGE: SQLite is no longer the runtime database. Deployments now
require Postgres (and Redis, used from phase 2). Existing installs migrate
with SQLITE_MIGRATE_PATH or 'pnpm --filter @snapotter/api migrate:sqlite'.
* fix(ci): postgres service + fresh e2e database per run; ignore unfixable torch CVE-2025-3000
Closes the "e2e never runs in CI" hole. Adds per-PR e2e smoke gate,
nightly full-suite workflows, parallel vitest forks (per-fork DBs),
Playwright parallel/serial/visual projects against production builds,
metadata-generated test suites (drift guards, hostile inputs, format
matrix, pairwise settings, property-based fuzz), Stryker mutation
testing, Schemathesis API fuzz, coverage ratchet, and fixes for three
session-poisoning bugs that caused 200+ serial-bucket failures.
Bug fix included: favicon/split/bulk-rename could hang clients forever
when ZIP streaming failed after reply.hijack().
rembg 2.0.75 pulls in numpy>=2.3.0 which conflicts with our pinned
numpy==1.26.4 and would break the entire AI dependency chain. The two
rembg CVEs (SSRF + path traversal) are in its server/CLI components
which we don't use; they're already in the pip-audit ignore list.
Proxmox LXC and other headless Linux installs lack libgl1, causing
all AI tools to show a misleading "install opencv-python-headless"
error even though the pip package is already installed. Detect the
libGL ImportError and suggest `apt-get install -y libgl1` instead.
The pipeline endpoints were documented as /api/v1/pipelines CRUD routes
that do not exist; the real routes live under /api/v1/pipeline/
(execute, save, list, delete, batch). The X-API-Key header was never
supported; API keys authenticate via Authorization: Bearer. Also adds
the openapi.yaml and llms-full.txt endpoints and the explicit batch
route.
generate-meme-thumbs.ts is superseded by build-templates.mjs which
generates thumbnails and the manifest in one pass. test-docker-fixes.sh
was a one-off Docker debugging script with no remaining references.
Five unit tests crashed because they imported modules that transitively
reached db/index.ts, which eagerly calls `new Database()` at module load
time. Added vi.mock for db/index.js matching the pattern used by 16
other passing API unit tests.
Moved cleanup.test.ts to tests/integration/ since it uses a real SQLite
database, runs migrations, and inserts rows.
Path resolution for the feature manifest and install script was hardcoded
to /app/..., which only works inside the Docker container. Native installs
(e.g. Proxmox at /opt/snapotter) hit "No such file or directory" errors.
Resolve both paths relative to the source file location via import.meta.url
so they work regardless of where the project is installed.
Also loosen mediapipe==0.10.21 to >=0.10.21 in requirements.txt and
requirements-gpu.txt to match the feature manifest. The exact pin has no
cp313 wheel, so it fails on Python 3.13 (Debian 13 default). mediapipe
0.10.35 ships py3-none universal wheels that resolve cleanly.
Reported-by: MickLesk (community-scripts/ProxmoxVE#14720)
The empty-state dropzone in the Automate preview panel was expanding
to fill the full 38% panel height, causing overlap and clipping of
the "Import from Library" button. Constrain to max 120px and tighten
spacing for a cleaner layout.
The security hardening commit changed this to false, which prevents
the analytics consent page from appearing on first login. Users see
"disabled by administrator" instead of being asked to opt in/out.
With ANALYTICS_ENABLED=true, the SDK initializes but NO user data is
sent until the user explicitly accepts on the consent page. The consent
page is the privacy safeguard, not the server-side flag. Self-hosters
who want to fully disable analytics can still set ANALYTICS_ENABLED=false.
The docs version badge was reading from apps/docs/package.json which
gets out of sync during manual releases. Read from the monorepo root
package.json instead so it always matches the released version.
SnapOtter