674 Commits
Author SHA1 Message Date
SnapOtterandGitHub 6683ee8c30 test: clear the sync-wait floor in the format matrix per-case timeout (#381)
The matrix per-case timeout equaled the 30s SYNC_WAIT_MS test floor, so a contended job that rode the full sync-wait window before returning a valid 202 raced the timeout and was flagged as a hang. Raise it to 60s. extract-pages on tiny.pdf took 30051ms in the PR run, just over the old 30000ms limit.
2026-06-29 23:46:59 +08:00
SnapOtterandGitHub 37dc0098ba docs: sync API and documentation coverage (#379)
* docs: sync API and docs coverage

* ci: pin pandoc for sandboxed conversions
2026-06-29 22:35:05 +08:00
SnapOtterandGitHub fd6ebe77b5 test: expand coverage across jobs and tools (#380) 2026-06-29 22:06:24 +08:00
SnapOtter 1d99acf9ee feat: merge PostHog customer feedback 2026-06-29 18:19:18 +08:00
SnapOtter 649e65b035 feat: add PostHog customer feedback 2026-06-29 18:16:33 +08:00
SnapOtter c6319cf8a9 fix(security): harden auth and outbound fetches 2026-06-29 17:54:12 +08:00
SnapOtterandGitHub 084a9c9faa feat: all-in-one embedded single-container mode (#377)
Embedded Postgres 17 + Redis via s6-overlay when DATABASE_URL/REDIS_URL are unset; restores the one-command docker run for 2.0. EMBEDDED=0 disables; Compose stays the production path. Verified arm64 (14/14 lifecycle + Compose regression) and amd64 (build + embedded smoke).
2026-06-29 15:49:09 +08:00
SnapOtterandGitHub 0cdd560ac4 feat: add Sign PDF tool (draw/type/upload signatures, place on a PDF) (#370)
Draw, type, or upload a signature and place resizable/rotatable copies across PDF pages; output flattened server-side with PyMuPDF. Visual electronic signature, not cryptographic. New interactive-sign display mode (pdf.js + Konva) and a custom docs-pool route.
2026-06-29 11:01:25 +08:00
SnapOtterandGitHub 1c202c6ef0 feat(analytics): upload web source maps to Sentry + tie release to build (#369)
* feat(analytics): upload web source maps to Sentry + tie release to build

Web crash reports were unusable: the bundle ships minified with no source
maps uploaded, and every build reported as the frozen APP_VERSION, so a
Sentry error showed an unreadable stack under a single release.

- Add @sentry/vite-plugin: emit hidden source maps and upload them by debug
  id when SENTRY_AUTH_TOKEN is present (published Docker build only), then
  delete the maps so they never ship. No-op for dev and the source archive.
- Set the Sentry release from SENTRY_RELEASE / VITE_SENTRY_RELEASE (the Docker
  build passes the release version), falling back to APP_VERSION.
- Relax beforeSend so app bundle frames keep a host-stripped path (Sentry needs
  it to match the uploaded map) while the instance hostname, error message, and
  PII stay stripped. Filesystem paths still collapse to the basename.
- Wire the Dockerfile (sentry_auth_token build secret + SENTRY_RELEASE arg/env)
  and the release docker job.

* fix(analytics): point source map upload at the snapotter org (project node)
2026-06-29 10:47:30 +08:00
SnapOtterandGitHub 19b515dc5f chore: use "200+ tools" for the tool-count claim across public surfaces (#364)
* fix(landing): correct PDF tool count to 28 in alternatives copy

The pdf section has 28 tools (section.test.ts asserts bySection('pdf')=28). PR #363 corrected the docs breakdown but the alternatives pages still said 40 PDF tools (the document-modality count, not the pdf section) with 200 for the rest. Update to 28 PDF tools and 212 for the non-PDF remainder.

* chore: use "200+ tools" for the tool-count claim across public surfaces

Replaces the exact '240 tools' count (which drifts as tools are added) with the stable '200+ tools' on README, the Docker Hub overview, landing pages, the docs site (meta, homepage, search), the API self-description, the demo OG tag, llms.txt, package.json, the branding readme, and the en/nl app strings. Per-modality breakdown tables stay exact. Leaves the architecture doc's technical 'tool routes' figure, an internal vitest comment, and a QA report line unchanged. Updates the two tests that assert the docs strings.
2026-06-28 21:49:14 +08:00
SnapOtterandGitHub 8e9452e650 fix: settings dialog and settings API correctness bugs
Seven correctness fixes in the admin settings dialog and settings API: AdminSecuritySettings save echoing read-only/redacted keys; the server persisting the ******** mask over real OIDC/SIEM secrets on a settings round trip; the Tools panel missing its settings:write gate; three swallowed errors (ToolsSection save, ApiKeys generate and delete); and generatePassword omitting a special char under passwordRequireSpecial. Adds an integration regression test for the secret-mask no-op.
2026-06-28 21:37:38 +08:00
SnapOtterandGitHub 63a03d26f2 feat: pipeline templates, analytics opt-out, 83 conversion presets, positioning + e2e modernization
Lands five integrated branches: pipeline templates (#355), analytics opt-out (#354), 83 conversion presets bringing the catalog to 240 tools (#356), self-hosted positioning (#353), and e2e modernization (#351).

Integration fixes: aligned stale web analytics tests with the opt-out/allow-list model, closed 3 CodeQL incomplete-sanitization alerts in the i18n generator, resolved settings/index/docs/format-matrix conflicts, and corrected tool counts to 240.
2026-06-28 18:57:53 +08:00
SnapOtterandGitHub 88af8d46fb fix(tools): honest content-type for edit-metadata pass-through (#350)
edit-metadata writes EXIF tags in place and streams the original bytes
back, but the response content-type defaulted to image/jpeg for any
format outside a small map. A BMP/PSD/PPM download then claimed to be a
JPEG, and the nightly tool-x-format matrix tried to Sharp-decode it and
threw "unsupported image format".

Map every format validateImageBuffer can report to its real MIME type,
and switch the matrix's pixel-decode gate from a fragile denylist to an
allowlist of formats this libvips build is guaranteed to decode. Niche
raster types streamed back untouched now carry an honest content-type
we simply don't pixel-verify.

Verified green across all 157 tools x 34 formats with FULL_MATRIX=1.
2026-06-25 02:20:41 +08:00
SnapOtterandGitHub bbad953e79 fix(nightly): de-flake matrix timeouts, exclude browser tool from fuzz (#349)
Systematic round - attack the failure classes, not one bug at a time:

- Flakiness (Coverage/Extended Matrix flipped green<->red): the generated
  format-matrix conversion tests hardcode a 30s/60s per-test timeout that the
  jobs' VITEST_TEST_TIMEOUT can't override. Heavy conversions under coverage
  instrumentation / full matrix intermittently exceed it. Raise to 120s.
- Schemathesis: html-to-image returns 503 when its headless browser isn't
  present (always, in the fuzz env) - same expected-unavailable class as the
  AI 501s. Exclude it from the fuzz (it can't be exercised without a browser).
- Baseline regen: bump timeout 60->120 min so it can finish now that the
  route sweep (#347) lets the visual specs pass instead of failing+retrying.
2026-06-25 00:14:45 +08:00
SnapOtterandGitHub 0f98f60c33 test(e2e): modernize stale routes for 2.0 section URLs (#347)
The e2e specs predate the 2.0 section-route migration and navigated to
single-segment tool URLs (/resize) that now 404 (App.tsx only mounts
/:section/:toolId). This broke the whole e2e suite (Cross-Browser, Device
Matrix, E2E Full, Visual) - part of the known stale-spec backlog.

- Sweep 929 goto("/<tool>") -> goto("/<section>/<tool>") across 45 spec
  files, using the authoritative TOOLS + toolSection() mapping. Two-segment
  routes, top-level routes (/automate, /editor, ...), and intentional 404
  tests (/nonexistent-*) are untouched.
- Implement the legacy redirects the specs already assert: the 1.x color
  tools (brightness-contrast, saturation, color-channels, color-effects)
  redirect to /image/adjust-colors (App.tsx). Good for old bookmarks too.
- Remove the analytics-consent page tests (gui-navigation + gui-visual);
  #336 deleted that page.

Mechanical + biome-clean + web typecheck passes. Browser-specific behavior
can only be confirmed by the nightly e2e jobs.
2026-06-24 20:19:37 +08:00
SnapOtterandGitHub 3dd0ebb49d test(nightly): fix Schemathesis/Coverage/Docker/fuzz root causes (#346)
Second round of nightly fixes, each root-caused from the post-fix run:

- Docker E2E (real bug): Dockerfile.test never copied patches/, so pnpm
  install hit 'ENOENT patches/gray-matter@4.0.3.patch' and exited 254.
  Copy patches/ like the prod Dockerfile does. (My earlier network-retry
  guess was a misdiagnosis; reverted.)
- fuzz-settings (real bug): the graceful-skip regex matched 'precondition'
  but fast-check v4 says 'pre-condition' (hyphenated), so 3 constrained PDF
  tools (extract/remove/organize-pages) errored instead of skipping. Match
  the hyphen. Verified locally: 3 failed -> 3 passed.
- Schemathesis: AI tool endpoints return 501 FEATURE_NOT_INSTALLED when the
  ML bundle is absent (always, in CI). That is expected, not a server bug,
  and the endpoints cannot be fuzzed without the bundle, so exclude them.
  (The ASCII spec-load fix already landed in #344.)
- Extended Matrix: bump the per-test timeout to 600s; edit-metadata over
  every format still exceeded 300s even at 2 forks.
- Coverage: tests now pass (video-speed + timeout fixes); re-baseline the
  branches/functions thresholds to the measured floor with a written reason.
2026-06-24 20:08:40 +08:00
SnapOtterandGitHub 33dfcecd6a test(nightly): stabilize the exhaustive nightly suite (#345)
Triaged the nightly failures (all pre-existing, unrelated to the analytics
work) and fixed the ones with clear root causes:

- video-speed: a 1s tiny.mp4 sped up 2x rounds to ~0.75s, flaking the +/-25%
  duration assertion under heavy CI load. Use the 8s hero.mp4 (still 44.1kHz)
  so rounding is negligible. Verified locally.
- Extended Matrix + Coverage timeouts: full-matrix / coverage-instrumented runs
  starve the heavy media tests under 4 forks at the 30s default. Make maxForks
  env-overridable (VITEST_MAX_FORKS) and run those jobs with 2 forks + a 300s
  timeout so format-matrix conversions and qr-generate stop timing out.
- Device Matrix visual baselines: the update-visual-baselines workflow could
  not start the app ('failed to create database') because it never provisioned
  Postgres/Redis. Add the same services block the e2e jobs use.
- Docker E2E: a container pnpm install network blip exits 254. Add fetch
  retries + a longer network timeout (frozen-lockfile already passes locally).
- Cross-browser: the home page is the tool catalog now (no dropzone), and the
  tool routes moved to /<section>/<toolId>. Point the upload test at a real
  tool page and fix the stale single-segment routes (/resize -> /image/resize,
  etc.).

The flaky/timeout and cross-browser fixes can only be confirmed by the nightly
(they are load- and browser-specific); a fresh nightly run will verify.
2026-06-24 18:23:23 +08:00
SnapOtterandGitHub 8f4235d2c6 fix(enterprise): ship enterprise package in prod image + S3, analytics, tracing, queue fixes (#342)
* fix(enterprise): ship enterprise pkg in prod image, full license features, tracing key fallback

docker/Dockerfile: COPY packages/enterprise manifest+src into the production stage.
Without it, apps/api's workspace link to @snapotter/enterprise dangles and every
import() throws (silently caught), so all 19 enterprise features failed closed
(enterprise.active=false) regardless of a valid license.

scripts/generate-license.mjs: sync PLAN_FEATURES with packages/enterprise/src/license.ts
so a --plan enterprise license unlocks all 19 features (was 8) and team unlocks 8.

apps/api/src/tracing.ts: accept SNAPOTTER_LICENSE_KEY as a fallback to LICENSE_KEY so
distributed_tracing activates with the same key as the rest of the app.

* fix(docker): keep scripts/bake-analytics.mjs in build context

.dockerignore excluded the whole scripts/ dir (PR #82, V1 hardening), but
docker/Dockerfile later added 'COPY scripts/bake-analytics.mjs' for the analytics
bake step. A clean production image build therefore fails with
'scripts/bake-analytics.mjs: not found'. The published image build is gated off in
CI so this latent break went unnoticed. Exclude scripts/* but re-include the one
file the Dockerfile needs.

* fix: S3 upload stream, analytics bake reaches API, dedupe retention field, reconcile orphan jobs

storage-s3.ts: wrap the upload AsyncIterable in Readable.from() so @aws-sdk/lib-storage
accepts it. STORAGE_MODE=s3 file uploads failed with 'Body Data is unsupported format'
for every tool because a bare async generator is not a Readable.

docker/Dockerfile: COPY the builder-baked analytics baked.ts into the API runtime stage.
The API re-copied the committed (off) baked.ts from the build context, so the
SNAPOTTER_ANALYTICS build arg had no effect on the API -- and since the SPA reads
/api/v1/config/analytics, analytics was off everywhere regardless of the arg.

settings-dialog.tsx: remove the duplicate tempFileMaxAgeHours control under Data
Retention; it bound the same setting key as the File Management control with a different
default, so editing either silently overwrote the other.

apps/api/src/index.ts: reconcile orphaned job rows (empty tool_id, never enqueued to
BullMQ) at boot so they don't sit in processing/queued forever and inflate the per-user
concurrent-job count and the upgrade-check in-flight gate.

* fix(web): style the SSO login buttons (they referenced undefined theme tokens)

The OIDC/SAML 'Sign in with <provider>' buttons used bg-secondary /
text-secondary-foreground, which the web theme never defines (it has primary,
background, foreground, muted, border, card, primary-subtle). Those classes resolved
to nothing, so the buttons rendered as bare unstyled text on the login page.

Restyle: the optional (non-enforced) buttons become white-card outline buttons with a
key icon and an orange hover tint, secondary to the primary Login button; the
SSO-enforced buttons become solid primary with the icon.

* fix: gate S3 behind license, custom-role enterprise perms, wire retention UI, cleanup

S3 is a licensed feature, but shipping packages/enterprise in every image removed the
implicit gate, so STORAGE_MODE=s3 worked without a license. Enforce
isFeatureEnabled('s3_storage') at boot and fail fast if unlicensed.

Custom roles can now be granted security:manage / compliance:manage / webhooks:manage
(roles.ts ALL_PERMISSIONS + the Roles UI) so admins can build least-privilege
compliance/security roles instead of only the built-in admin role.

retentionSweep now reads the jobsRetentionDays / auditRetentionDays DB settings the
System Settings UI writes (env vars become the fallback default), mirroring how the
temp-file sweep reads tempFileMaxAgeHours. Previously those two UI controls were no-ops.

Cleanup: drop the never-set snapotter_storage_bytes gauge and the unused
MAX_WORKSPACE_SIZE_GB env var; emit tool_client_error to PostHog from the web
ErrorBoundary (client crashes were not reaching analytics); add the Python
OpenTelemetry packages so the innermost sidecar.<script> span exports; fix the stale
'only local storage' line in the docs; delete two e2e-analytics specs that tested the
removed consent UI.

* fix(env): restore MAX_WORKSPACE_SIZE_GB default

security-auth-hardening.test.ts asserts env.MAX_WORKSPACE_SIZE_GB defaults to 10, so
the var is an intentional (tested) default, not dead code. Removing it in the cleanup
commit broke that unit test. Keep the declaration.
2026-06-24 17:27:59 +08:00
SnapOtterandGitHub c2ae334c81 fix(api): make OpenAPI spec ASCII-only so Schemathesis can load it (#344)
The nightly Schemathesis job failed with a schema-loading error:
'unacceptable character #x0080: control characters are not allowed'. The
served openapi.yaml contained 215 em dashes (U+2014) and box-drawing
section dividers (U+2500); Schemathesis's strict YAML parser mis-decodes
those multi-byte UTF-8 sequences as C1 control chars and refuses to load
the schema, so no fuzz checks ran. (PyYAML is lenient, which is why the
local yaml.safe_load check passed.)

Replace every non-ASCII char with ASCII '-'. This also clears an
em-dash style-rule violation. Add a docs.test.ts guard asserting the
served spec is ASCII-only, with a clear message, so this can't regress.

Pre-existing issue (the dashes predate this branch); surfaced while
verifying CI is green.
2026-06-24 17:14:10 +08:00
SnapOtterandGitHub 5b75d813b8 fix(e2e): wait for post-login redirect before forcing navigation (#341)
Removing the analytics-consent PUT (#340) also removed the async HTTP
round-trip that incidentally let the app's own post-login redirect
(/login -> /) commit before the explicit page.goto("/"). Without it,
page.goto raced the in-flight client-side redirect and aborted with
'Navigation to / is interrupted by another navigation to /', failing the
auth setup projects. Desktop smoke passed on timing luck; mobile
emulation is slower and lost the race.

Wait for the app's redirect to settle (waitForURL) before the explicit
goto so they no longer race. Platform-timing-independent.
2026-06-24 14:45:37 +08:00
SnapOtterandGitHub 6917a8b0c7 fix(test): repair integration suite after analytics column/endpoint removal (#340)
* fix(test): repair integration suite after analytics column/endpoint removal

#336 moved analytics to a build-time bake: migration 0005 dropped the
users.analytics_enabled and analytics_consent_* columns and removed the
PUT /api/v1/user/analytics endpoint. Two integration tests were left
referencing the old shape and went red on main (13 failures):

- migrate-from-sqlite.test.ts built 1.x SQLite fixtures whose users table
  declared the analytics columns. The generic SELECT *-based importer then
  tried to INSERT them into the 2.0 target, which no longer has those
  columns, failing with Postgres 42703 and rolling back the whole import
  (cascading to all 12 assertions). 1.x never had analytics columns, so the
  fixtures are corrected to drop them. Also removed the now-dead analytics
  entries from the importer's TS/BOOL conversion sets.

- analytics.test.ts asserted the removed PUT endpoint returns 404 but sent
  the request unauthenticated, so the global auth preHandler answered 401
  first. It now authenticates, reaching Fastify's not-found handler (404).

Also removed the stale /api/v1/user/analytics path from openapi.yaml.

Verified locally: full platform integration bucket 1029 passed / 0 failed;
monorepo typecheck clean.

* test(e2e): drop orphaned analytics-consent dismissal calls

#336 deleted the entire analytics consent system (consent page, consent
module, and PUT /api/v1/user/analytics), but six tests/e2e files still
PUT to that removed endpoint to 'dismiss analytics consent.' The calls
were silent no-ops (Playwright request.put / fetch don't throw on 4xx),
so they passed while hitting a dead route.

There is no consent prompt to dismiss anymore, so remove the calls:
- auth.setup.ts / qa-auth.setup.ts: keep the waitForFunction that syncs on
  login completion, drop the now-unused token capture, the dead PUT, and
  the stale 'consent guard' comments.
- rbac / rbac-full / gui-settings-rbac / gui-settings-expanded specs: the
  re-login blocks existed solely to obtain a token for the PUT (reLoginData
  was used nowhere else and the block was the tail of each helper), so
  remove the whole block. The meaningful create-user/login/change-password
  work is untouched.

Verified: no /api/v1/user/analytics refs remain in tests/e2e; biome clean
(no unused vars).
2026-06-24 13:30:18 +08:00
SnapOtterandGitHub 5d36ac06d8 feat(analytics): build-time bake + telemetry depth (#336)
Bake PostHog + Sentry into the published Docker image (SNAPOTTER_ANALYTICS
build arg, codegen script). Delete entire consent system. Move event emission
to BullMQ worker. Add cross-tier identity stitching, Sentry performance
tracing on both tiers, frontend funnel events. Fix stateful regex bug.

86 files changed, 1593 insertions(+), 3747 deletions(-)
2026-06-24 11:05:39 +08:00
SnapOtterandGitHub a53038ed96 feat(automate): make the pipeline builder fully multi-modal (#335)
* feat(shared): add outputModality to Tool metadata for crossing tools

* feat(shared): add modalityForExtension, toolInputModality, toolOutputModality

* fix(pipeline): route finalize and parent jobs to the pipeline's modality pool

* feat(automate): add ConvertAudioControls pipeline step (exemplar)

* feat(automate): add video tool settings controls to pipelines

* feat(automate): add audio tool settings controls to pipelines

* feat(automate): add document tool settings controls to pipelines

* feat(automate): add chart-maker settings control to pipelines

* feat(automate): warn on modality-incompatible pipeline steps

* feat(automate): add single-file download button for pipeline results

* refactor(automate): modality-aware icons, nav handler rename, mobile size bar

* test(pipeline): cover audio, document, file, and cross-modality chains

* i18n(automate): translate the modality-warning tooltip

* test(pipeline): gate media-pool routing assertion on ffmpeg availability
2026-06-24 00:21:41 +08:00
SnapOtterandGitHub 7a70affac5 fix(ai): enforce the feature gate on the per-request fallback path (#331)
The persistent Python dispatcher rejects scripts whose feature bundle is not
installed, but the per-request fallback (used when the dispatcher is down, e.g.
restarting right after a model repair) spawned scripts directly and bypassed
that gate. Behavior was therefore inconsistent: a gated script would fail under
the dispatcher but run under the fallback -- the "works once after a repair"
symptom from the original report.

- add packages/ai/src/feature-gate.ts: SCRIPT_BUNDLE_MAP + missingBundleForScript,
  mirroring TOOL_BUNDLE_MAP in dispatcher.py, reading the same installed.json and
  failing closed exactly like dispatcher._get_installed_bundles()
- runPerRequest now rejects with "feature_not_installed" (the same message the
  dispatcher path surfaces) when a gated script's bundle is not installed
- unit tests for the gate, plus a drift test pinning the TS map to dispatcher.py

Closes #327
2026-06-22 23:49:41 +08:00
SnapOtterandGitHub 32c1192d63 fix(passport-photo): require the face-detection bundle, not just background-removal (#329)
* fix(passport-photo): require the face-detection bundle, not just background-removal

Passport Photo runs face-landmark detection (face_landmarks.py, gated to the
face-detection bundle) before background removal (background-removal bundle),
but it was only declared under and guarded against background-removal. A user
who installed only Background Removal passed every JS-side check, then hit a
late "feature_not_installed" from the Python dispatcher gate when the analyze
step ran face landmarks, and the UI never told them Face Detection was needed.

- shared: add TOOL_EXTRA_BUNDLES + getRequiredBundlesForTool so a tool can
  declare more than one required bundle (passport-photo needs background-removal
  and face-detection). enablesTools is untouched, so the one-tool-per-bundle
  invariant still holds.
- api: isToolInstalled() now checks every required bundle; add
  getFirstMissingBundleForTool() so the analyze and base routes, pipeline (both
  guards) and batch report the bundle the user actually still needs.
- web: the proactive install prompt (tool-page) and features-store treat a tool
  as installed only when all required bundles are present, and point the prompt
  at the first missing one (sequential install, no new UI).

Refs #327

* test(passport-photo): deterministic integration coverage for the two-bundle guard

Boots the real API with an isolated DATA_DIR and controls installed.json to
prove the HTTP route behavior end-to-end:
- nothing installed -> 501 naming background-removal
- only background-removal installed -> 501 naming face-detection (issue #327)
- both installed -> guard passes (not 501)
- base route reports face-detection too

Refs #327
2026-06-22 23:31:18 +08:00
SnapOtterandGitHub 8952e9ba47 fix: harden against three production Sentry crashes (#328)
Three production crashes from the snapotter/node Sentry project.

feature-status (NODE-12): a valid-JSON-but-wrong-shape installed.json
crashed boot via Object.keys(data.bundles). readInstalled() now
normalizes any unusable shape to { bundles: {} }, and the boot recovery
call is wrapped so cleanup can never fatal startup.

image-viewer (NODE-15/17/18): drag-to-pan read .x off an undefined
use-gesture memo on pointerUp or a pinch-into-pan. A guarded pure helper
(resolvePanStart) now falls back to the live pan offset.

Fastify (NODE-14): raised pluginTimeout to 60s so slow self-hosted boots
do not fatal at @fastify/static.
2026-06-22 23:25:22 +08:00
SnapOtterandGitHub 95d100c20b feat(web): in-canvas zoom & pan for the object eraser and split tools (#320)
* feat(web): add pure zoom/pan math module with unit tests

* feat(i18n): add a11y.pan key across all locales (English, matching adjacent zoom labels)

* feat(web): add useZoomPan hook (state + gestures over pure math)

* feat(web): add ZoomToolbar component

* feat(web): zoom & pan in the object eraser canvas

* feat(web): zoom & pan in the split tool preview

* fix(web): synchronous pan-mode refs so drag-pan is race-free under fast input

* test(e2e): zoom & pan acceptance (split always-on, eraser bundle-gated)
2026-06-22 20:50:28 +08:00
SnapOtterandGitHub 313d4ae04e test(e2e): fix stale route + fixture paths in erase-object spec (#321)
The spec used the bare /erase-object route (404 under section URLs) and the
old flat fixture path (test-200x150.png moved to image/valid/), so every test
silently skipped. Point it at /image/erase-object and the correct fixture, and
add a 404 guard so future route rot fails loudly instead of skipping.
2026-06-22 20:24:07 +08:00
SnapOtterandGitHub 1fec97111b fix(docker): make storage writable under non-root/foreign UIDs (TrueNAS, OpenShift) (#299)
The entrypoint only fixed volume permissions when started as root (chown +
gosu-drop to snapotter). Launched under a non-root/foreign UID (TrueNAS app
user, Kubernetes runAsUser, OpenShift) it did no permission setup, so /data and
/tmp/workspace -- owned by uid 999 from the image -- were not writable by the
running user. Uploads and processing then failed with a cryptic EACCES
("workspace folder is not writable") and AI bundle installs failed the same way,
while health checks still reported the container healthy.

- entrypoint: source new entrypoint-lib.sh; verify writability up front when
  non-root, and as snapotter after chown when root (catches root-squashed
  mounts), failing fast with an actionable message (which dir, uid/gid, how to
  fix) instead of a late, cryptic EACCES
- Dockerfile: own /data and /tmp/workspace as snapotter:0, group-writable with
  setgid, so an arbitrary UID with the root supplementary group (OpenShift /
  Kubernetes fsGroup) can write; keep /opt/venv world-readable for the AI venv
  bootstrap under arbitrary UIDs
- api: assert storage writability at boot (lib/storage-writable.ts), failing
  fast with the same guidance even when the entrypoint is bypassed
- docs: add a Storage permissions section (named volumes, bind mounts, TrueNAS,
  Kubernetes/OpenShift) and cross-link it from the security guide

Fixes #230
2026-06-22 16:58:59 +08:00
SnapOtterandGitHub 33671d2d92 feat(landing): auto-refresh live GitHub stars and image-pull stats (#292)
Fetch landing stars + image pulls at build time via a shared stats lib, refreshed by a daily cron + authenticated GITHUB_TOKEN. Image Pulls totals live Docker Hub pull_count + a maintained GHCR estimate (ghcr.io has no public pull-count API).
2026-06-22 14:11:48 +08:00
SnapOtterandGitHub 9b1c105089 test(integration): accept 202 async for DNG in format-matrix smoke tests (#291)
* test(integration): accept 202 async for DNG in format-matrix smoke tests

Follow-up to #290. Full-resolution DNG decode (3474x2314 vs the old 1024px
preview) pushes expensive operations (AVIF encode, image-enhancement) past
the 8s sync window on CI runners, so the API correctly returns 202 Accepted
and processes the job asynchronously. The format-matrix smoke tests only
allowed [200, 400, 422] and required a clean error body on any non-200, so
they failed on the 202 (Integration shard 4/4 went red on main).

202 (accepted, async) is a valid clean response for these "no crash / clean
response" checks. Make the matrix allowlists 202-tolerant and require an
error body only for true error codes:
- add 202 to ACCEPTABLE_CODES / ACCEPTABLE_FALLBACK_CODES + inline allowlists
- change `if (statusCode !== 200)` error-body checks to `>= 400`

Verified locally against Postgres+Redis: DNG tests pass both normally (200)
and with SYNC_WAIT_MS=1 forcing 202 (68 passed, 0 failed each run).

* test(integration): treat 202 as non-error in DNG conversion else-branches

The first pass added 202 to status allowlists and switched `if (!== 200)`
error checks to `>= 400`, but missed the `if (200) {...} else {...}` shape in
the exotic conversion matrix and the expanded color-blindness test: their
`else` caught 202 and then asserted body.error (which async responses lack).

Reproduced locally by temporarily lowering the test sync-window floor to force
202 on every DNG op, then fixed every flagged assertion. Change the two
`else` branches to `else if (statusCode >= 400)` so 202 (accepted, async) is a
valid outcome with no sync body to verify.

Verified: forced-202 across all 4 DNG matrix files = 123 passed / 0 failed;
normal sync window = 34 DNG tests passed; typecheck + biome clean.
2026-06-22 11:39:13 +08:00
SnapOtterandGitHub 3d9ff1e0d2 fix(api): decode RAW via LibRaw first so DNG processes at full resolution (#289) (#290)
RAW (DNG) processing crashed on ImageMagick's deprecated ufraw-batch
delegate, which fails on modern formats such as iPhone ProRAW DNG.

Root cause: the dcraw_emu (LibRaw) decode tier read the wrong output path.
dcraw_emu APPENDS the output extension (raw-in-X.dng -> raw-in-X.dng.tiff)
but the code looked for raw-in-X.tiff (replaced extension), so readFile threw
on every RAW, the tier silently fell through to ufraw, and the 24MB TIFF
leaked into the temp dir on each attempt.

- Repair the dcraw_emu output path; clean it up in finally (fixes the leak)
- Prefer LibRaw full decode over embedded-preview extraction so a
  full-resolution RAW is never silently returned as a reduced-size preview
  (sample DNG: was 1024x683 preview, now 3474x2314 full)
- Add RAW decode regression tests (DNG full-resolution + all 6 RAW formats);
  these were absent, which let the bug ship
- Install libraw-bin on CI test runners so dcraw_emu is actually exercised
2026-06-22 09:54:04 +08:00
Luciano GodoyandGitHub ce02ce1348 fix(api): respect RATE_LIMIT_PER_MIN for tool routes (#272)
Tool endpoints (/api/v1/tools/*) now honor the RATE_LIMIT_PER_MIN env var instead of a hardcoded 60/min: `0` disables per-tool limiting, `>0` uses the configured value, and unset falls back to 60. Merged on top of the section-based route refactor (#280).

Fixes #271.
2026-06-21 23:39:55 +08:00
SnapOtterandGitHub dba8a85a80 fix(jobs): pre-warm QueueEvents to kill first-sync-wait flake (#285)
The csv-json integration test intermittently timed out at 30000ms on
the first worker-backed job in a fork. Root cause: waitForJob() creates
the BullMQ QueueEvents consumer lazily on first use, and a fresh consumer
reads the Redis events stream from "$" (the tail at the moment its run
loop starts). A trivial tool can publish its completed:<id> event before
the brand-new consumer positions itself, so waitUntilFinished() never
sees the event and blocks for the full sync-wait window. In tests
SYNC_WAIT_MS is floored at 30000ms, exactly the vitest per-test budget,
so the stall surfaces as an opaque timeout instead of a 202 fallback.
This is also a latent production latency bug: the first synchronous tool
request after each boot could hang up to the 8s prod window.

Fix: warmQueueEvents() eagerly constructs and connects every pool's
consumer at spine startup, before any job is enqueued, so each consumer
is positioned at the stream tail up front and never misses a completion.
Awaited in the test spine (deterministic for the first request) and fired
non-blocking at prod boot (a slow Redis must not stall startup).

Adds a regression guard in job-spine.test.ts that drops the cached
consumers, warms explicitly, and asserts a fast job's completion is
captured on the first sync-wait.

Verified: 3 parallel stress runs (276 file-runs across all pools), zero
timeouts; targeted job-spine + csv-json suites green; typecheck clean.
2026-06-21 23:22:59 +08:00
SnapOtterandGitHub f75cc328ac fix(security): numeric CIDR matching for IPv6 SSRF allow/deny
Replace textual prefix matching in isPrivateIPv6() with numeric bit-prefix matching via ipaddr.js. Closes reachable classifier bypasses for IPv4-compatible IPv6 (::a.b.c.d), site-local fec0::/10, and the full fe80::/10 link-local span, on top of the IPv4-mapped forms. Embedded IPv4 (mapped and compatible) is run through the full isPrivateIPv4 classifier; unparseable input fails closed.

Follow-up hardening to f64cbdda.

Reported-by: tonghuaroot
2026-06-21 23:20:28 +08:00
SnapOtter bdadb843d8 fix(security): close remaining high-severity CodeQL alerts
- svg-sanitize.ts: strip each dangerous element repeatedly until stable with
  whitespace-tolerant end tags, defeating nested/overlapping tags (closes 5
  incomplete-multi-character-sanitization + 1 bad-tag-filter; the prior
  single-pass regex could leave a residual <script>/<iframe>).
- file-preview.ts: add a resolve()+containment barrier (the path-traversal
  guard CodeQL recognizes) on top of the id charset check (closes 9
  path-injection).
- metadata.ts: bound the XMP namespace:name key segments so parseXmp cannot
  backtrack polynomially (closes js/polynomial-redos).
- analytics-disabled.spec.ts: match analytics by URL host, not substring
  (closes 4 incomplete-url-substring-sanitization).

typecheck + lint green; svg (119), preview (22), metadata (164) tests pass.
2026-06-21 13:47:22 +08:00
SnapOtter 76fc695233 fix(benchmark): use section-prefixed tool URLs for 2.0 route scheme
The 2.0 API routes changed from /api/v1/tools/<toolId> to
/api/v1/tools/<section>/<toolId>. Updated all bench script call sites
to pass section-prefixed tool IDs (e.g. "image/resize" instead of
"resize"), matching the authoritative toolSection() mapping.
2026-06-21 12:18:47 +08:00
SnapOtter a6837b687a fix(e2e,landing): robust path escaping (CodeQL) + unique file-tools card
- tests/e2e/helpers.ts: build the sharp script path via JSON.stringify
  instead of single-quote-only replace (CodeQL js/incomplete-sanitization,
  high: backslashes were not escaped). Proper fix, no suppression.
- landing CategoryCards: rename the file-modality marketing card to
  "File Tools" (matches the Image/Video/Audio Tools siblings and is
  unique vs the 23 "Files" tool pills, which broke the e2e locator).
  Modality label stays "Files" everywhere it is the actual modality.
2026-06-21 03:25:45 +08:00
SnapOtter 71fefc05b0 feat(modality): rename "file" modality label "Data" -> "Files"
The fifth user-facing group is now Image, Video, Audio, PDF, Files
(internal modality id stays "file"; section.ts "files" was already
"Files"). Updates modality.ts label + comment, all 21 i18n locales
(categories.data "Data Files"->"Files", modalities.documentsAndFiles
"PDF & Data"->"PDF & Files", dead homePage.data), landing cards/hero
search/tools filter, docs headings, and e2e modality-tab assertions
(/^Data/ -> /^Files/, which had been failing).
2026-06-21 02:45:56 +08:00
SnapOtter d4dc2eae5d fix(tests): section-prefix tool API URLs across overhauled suite
#276's overhauled integration tests + QA sweep scripts hit the old
sectionless /api/v1/tools/<id> routes, which 404 after #280 made routes
section-prefixed. Ran scripts/rewrite-tool-paths.ts for literal URLs and
converted dynamic /api/v1/tools/${id} constructions to apiToolPath().
Negative-test paths (nonexistent/fake tools) intentionally left raw.
2026-06-21 02:30:58 +08:00
SnapOtter 5ffa1d55ea Merge branch 'worktree-test+suite-overhaul-and-real-fixtures' into chore/consolidate-v2.0.0
# Conflicts:
#	tests/integration/generated/settings-matrix.test.ts
#	tests/integration/platform/api.test.ts
#	tests/integration/platform/concurrent.test.ts
#	tests/integration/platform/factory-multi-input.test.ts
#	tests/integration/security/adversarial-comprehensive.test.ts
#	tests/integration/security/adversarial-coverage-gaps.test.ts
#	tests/integration/security/adversarial-extended.test.ts
#	tests/integration/security/adversarial-final-gaps.test.ts
#	tests/integration/security/adversarial-matrix.test.ts
#	tests/integration/security/adversarial-security.test.ts
#	tests/integration/security/adversarial.test.ts
#	tests/integration/tools/image/color-adjustments.test.ts
2026-06-21 02:18:53 +08:00
SnapOtter 719457e111 Merge branch 'feat/section-based-tool-urls' into chore/consolidate-v2.0.0 2026-06-21 02:09:14 +08:00
SnapOtter 37841e3369 Merge branch 'security/hardening-audit' into chore/consolidate-v2.0.0 2026-06-21 02:08:50 +08:00
SnapOtter 0ddcc7aef5 test(api): fall back to raw path for unknown tool ids in adversarial negative tests 2026-06-21 00:47:06 +08:00
SnapOtter c1c66efd3f chore: section-prefix remaining tool paths in docs, examples, benchmarks, and multi-input test 2026-06-21 00:30:10 +08:00
SnapOtter 5dbbac6c43 test(web): align unit + e2e expectations with section routes 2026-06-20 23:35:37 +08:00
SnapOtter f80444791b test(api): section-prefix tool URLs across integration, docker, and e2e api specs 2026-06-20 12:24:50 +08:00
SnapOtter 414f126a33 docs(api): section-prefix OpenAPI tool paths 2026-06-20 11:55:22 +08:00
SnapOtter 68dd7dabaa feat(api): section-prefix factory + batch routes via apiToolPath (+section validation) 2026-06-20 11:44:36 +08:00
SnapOtter c17a4a4225 feat(scripts): add segment-anchored tool-path rewrite transform 2026-06-20 11:22:33 +08:00