674 Commits
Author SHA1 Message Date
SnapOtterandGitHub 04ef1141fb feat(telemetry): readable Sentry errors, Python tracebacks, and diagnostic mode
Keeps a real, redacted error message instead of "Error: Error", surfaces Python tracebacks in Sentry as a vetted context, and adds an opt-in SNAPOTTER_SENTRY_DIAGNOSTIC verbose mode plus SNAPOTTER_SENTRY_DSN_OVERRIDE. The default fleet path ships nothing on the never-collect list; raw detail is reachable only via the opt-in flag. Also classifies Redis OOM/READONLY replies as operational and removes a ReDoS in stack-frame extraction.
2026-08-03 13:13:38 +08:00
SnapOtterandGitHub 865390ce66 fix(api): match collage pan limits to the preview and harden cell encoding (#739)
The preview lets a cell be dragged to +/-200% but the schema rejected
anything past +/-100, failing half the draggable range with a 400. The
schema now mirrors the drag clamp; cover-mode values past 100 saturate
at the image edge, contain-mode values keep panning like the preview.

Cover cells and the cornerRadius mask re-encode kept the input format,
so JPEG inputs lost the masked alpha and corners flattened to black.
Both now encode PNG explicitly.

Unreadable image dimensions used to silently substitute the cell's own
size, fill-stretching the image; they now throw and surface as a 422
naming the file.

Fixes #718
2026-08-02 20:37:33 +08:00
SnapOtterandGitHub a9bb76fbc7 fix(api): honour zoom and pan in collage contain cells (#717)
The contain branch chained a zoom resize into the contain resize on one Sharp pipeline; Sharp keeps a single set of resize options, so per-cell zoom was silently discarded and pan was equally dead. Splitting the chain would not have fixed it: the contain step scales the zoomed image straight back down.

The backend now implements the preview's semantics exactly: contain-fit, scale about the cell centre by zoom, offset by pan as a fraction of the cell, clip, background behind. Defaults keep the old single contain resize (now PNG-encoded so JPEG inputs no longer flatten cornerRadius masks to black corners), and the general path centres with floor to match Sharp's contain placement.

Nine marker-geometry tests, seven mutants each killed by the test built for them, window math fuzzed over 2.59M combinations.

Fixes #711
2026-08-02 15:26:33 +08:00
SnapOtterandGitHub 14f6c09805 fix(api): stop palette-quantising PNG output across image routes (#716)
resolveOutputFormat handed every route a default quality of 95, and Sharp reads quality on PNG as palette quantisation, so every PNG through the factory-route family came back dithered and often bigger. quality is now optional and stays undefined for PNG unless a caller passes an explicit override; smart-crop's user-chosen quality still quantises on request.

Two branches bypassed the resolver and carried their own copy of the bug, both fixed: image-pad hardcoded quality 95 for transparent padding, replace-color hardcoded quality 100 when forcing PNG for transparency. pixelate drops the local special case #709 added for the same bug.

Four new integration oracles on >256-colour inputs, each watched failing against the old code.

Fixes #710
2026-08-02 15:05:27 +08:00
SnapOtterandGitHub 5ffede05ef fix(api): make pixelate actually pixelate (#709)
Sharp keeps one set of resize options per pipeline, so the chained shrink-then-grow collapsed into a single resize back to the original size and both the full-image and region paths returned the input untouched.

Run the two resizes as separate pipelines, pinned to fit: "fill" so a skewed block grid cannot crop the picture.

Also in this change:
- stop passing quality to the PNG encoder, which Sharp reads as "quantise to a palette" and which dithered the flat blocks and inflated the file
- replace the region instead of blending into it, so a part-transparent image no longer shows the original through the mosaic
- throw on unreadable dimensions rather than falling back to a 1x1 image

Five new integration tests on content-bearing fixtures, each verified against a deliberate mutant.

Fixes #678
2026-08-02 12:03:04 +08:00
SnapOtterandGitHub 50d12c6aba fix(doc-engine): keep protect-pdf passwords out of qpdf's argv (#708)
qpdf expands argument files before parsing options, so a bare positional password
beginning with @ was resolved as a path and the file's contents became the
encryption key: exit 0, and the user's own password no longer opened the PDF.

Drive the encrypt through a job-JSON file so neither password reaches argv. The
=-joined flag form needs qpdf 11.7+, and the released image carries 11.3.0 which
rejects it; job JSON works on both (verified 11.3.0 and 12.1.0, R = 6 each).
2026-08-01 14:54:56 +08:00
SnapOtterandGitHub 1544966b52 fix(api): authorize the standalone upload and preview routes (#707)
POST /api/v1/upload, POST /api/v1/preview and POST /api/v1/preview/generate
authenticated but never authorized, sitting between requireFileAccess (which
guards /api/v1/files) and toolAccessMiddleware (scoped to /api/v1/tools/). A key
scoped to settings:read alone could still stage bytes behind the unauthenticated
download URL and drive Sharp, libheif, LibRaw and FFmpeg.

Upload now takes requireFileAccess; both preview routes take tools:use.
requireFileAccess moves to permissions.ts so the modules share one definition.
2026-08-01 14:40:09 +08:00
SnapOtterandGitHub 059af34ace fix(gdpr): stop exporting MFA credentials and gate exports on role authority (#706)
The subject-access export selected the whole users row and subtracted only
passwordHash, so profile.json carried totpSecret and recoveryCodesHash. On a
default install DATA_ENCRYPTION_KEY is empty and the TOTP seed is stored as
cleartext base32; recovery codes are 32-bit values behind an unsalted SHA-256.

Name the profile columns instead, add the canManageTargetRole gate the sibling
purge routes already apply, and scope the export status lookup to the user in
the path plus the gdpr-export tool id.
2026-08-01 14:16:36 +08:00
SnapOtterandGitHub d88031ac0f test(qa): decode subtitle output as text in the sweep oracle (#704)
The canonical lane classified .srt/.vtt/.ass artifacts served as
application/octet-stream as binary and passed them on a 16-byte floor, so
auto-subtitles and extract-subtitles only proved their output was
non-trivial, not that it was a real subtitle. classify() already keys
several formats off the filename extension; extend the text branch the same
way so these decode as text and the oracle asserts real content.

Verified on a live v2.2.0 container: auto-subtitles now reports
'text 180chars 8lines' and extract-subtitles 'text 101chars 9lines' (both
were binary no-signature), still passing, and an empty subtitle would now
fail as whitespace-only instead of sliding through.
2026-07-31 16:05:29 +08:00
SnapOtterandGitHub 447190afda fix(qa): give the canonical lane meaningful inputs for six tools (#703)
On a bundles-installed container the canonical lane reported six false
failures, all harness gaps rather than product bugs:

- remove-gif-background got a still GIF and correctly refused it; pin the
  animated fixture it actually needs.
- erase-object got no mask part, so the route's 400 was correct; wire the
  mask through SECONDARY_INPUTS the way the other multi-input tools are.
- passport-photo's base route refuses by design (use /analyze or /generate);
  record that in EXPECTED_SELF_REJECT.
- ocr, transcribe-audio and auto-subtitles ran on fixtures carrying none of
  the text or speech they extract, so they emitted empty but valid artifacts
  the oracle read as zero-byte failures. Pin fixtures that carry real content.

Verified red to green against a live v2.2.0 container with every bundle
installed: the six went from 6 fail to 5 pass plus 1 expected-reject, the
multi lane stayed clean (0 fail across 20 tools), and a direct probe showed
the download path was never at fault (ocr on a text-free image returns a
correct 0-byte result; on ocr-clean.png it returns the real text).

Fixes #690
2026-07-31 15:06:24 +08:00
SnapOtterandGitHub 24868ac5b4 test(e2e): make the pin-tools spec idempotent across retries (#697)
The nightly E2E Full (4/4) intermittently failed pin-tools at its opening no-Pinned-section assertion: the spec mutates the shared pinnedTools preference, and with CI retries:1 an attempt that failed after pinning left a server-side pin the retry saw. Reset pinnedTools via the preferences API at the start so the test is idempotent. Confirmed intermittent (the same job passed on rerun of unchanged code) and PR CI green.
2026-07-31 08:19:47 +08:00
SnapOtterandGitHub c0dc037860 fix(fuzz): bound settings-fuzz inputs, cap split tiles, budget the slow codecs, steady flakes (#696)
Follow-up to #695, greening the last nightly jobs it exposed. split gains a 400-tile product cap (a 100x100 split was a 10,000-file ZIP and ~20s of work). The settings-fuzz bounds its image inputs to 640px and gives the tools whose cost is output-driven (border, gif-tools, split) or codec-driven (heic/webp-to-avif) honest per-case budgets, since #649's settle-job wiring made every case wait for the real encode. The delete-team serial spec waits with toHaveCount(0) so the success toast can't trip strict mode, and type-to-search allows a route announcer's programmatic reading focus so it works on WebKit. Confirmed on a nightly dispatch: Extended Matrix (all 4 shards), Serial Bucket, Cross-Browser, and Coverage all green; Docker Container E2E's failures were GitHub runner reclamation (exit 137, tests passing throughout), which cleared on the #695 dispatch and is unaffected by this change.
2026-07-31 06:56:44 +08:00
SnapOtterandGitHub 1b41da7615 ci(nightly): repair five job classes broken by the #649 QA hardening (#695)
Nightly has been red since 07-28; per-PR CI and main are green. Two investigations traced all five failing classes to #649: extended-matrix required AI bundles it never installs (removed), nightly SYSTEM_DEPS drifted from ci.yml (added libreoffice + a doc-binaries composite for pandoc/pdfcpu), the generated-case classifier only skipped ffmpeg (widened to pdfcpu/soffice/pandoc + excluded repo-audit specs from the lean docker image), a settings spec capped loginAttemptLimit and 429-cascaded the serial bucket (restore via API), and type-to-search refused keystrokes under a route announcer's programmatic focus (guard added). A nightly dispatch on the branch confirmed all five classes green.
2026-07-31 02:09:39 +08:00
SnapOtterandGitHub 6c2fa307f4 test(qa): align AI oracles with fixtures and honor QA_OUT_DIR (#686)
The ocr oracle demanded six pangram words when ocr-clean.png prints five; the ocr-pdf oracle expected a searchable PDF when the tool's contract is text extraction; media-30s.mp4 was eight seconds long. Fix all three, regenerate tool-contract.json from current schemas, record the new fixture in the manifest, and add a QA_OUT_DIR override so parallel machines stop clobbering each other's lane output. Fixes #677.
2026-07-30 10:32:07 +08:00
SnapOtterandGitHub 2d39881d16 fix(api): refuse HQ inpainting on CPU hosts below the measured memory floor (#685)
SD1.5 inpainting is OOM-killed at the stock 6g compose limit on CPU hosts and completes at 8g. Read the cgroup limit and refuse hq up front with an actionable message; GPU hosts and unlimited containers are untouched. Fixes #670.
2026-07-30 10:17:39 +08:00
SnapOtterandGitHub 470a0a4acb fix(ai): pin huggingface-hub so bundle rebuilds cannot strand hub 1.x (#683)
The arm64 transcription bundle baked huggingface_hub 1.22.0 while inpaint-hq's transformers needs hub <1.0; last-writer-wins in the shared venv made the hq install fail on arm64. Constrain hub at build time, pin it in transcription's package list, and lock both invariants with manifest unit tests. The arm64 transcription bundle still needs a rebuild and republish to ship the fix. Fixes #669.
2026-07-30 10:04:15 +08:00
SnapOtterandGitHub 890b38a71a fix(api): stop claiming no GPU exists before the first AI bundle install (#682)
A fresh GPU deployment has working passthrough but no torch/ONNX runtime until a bundle installs, so the dispatcher reports gpu=false and the boot banner said no GPU was detected. Add the hardware-present state, driven by the same /dev/nvidia0 check the instance census uses. Fixes #673.
2026-07-30 09:49:45 +08:00
SnapOtterandGitHub b6c69b7aeb feat(telemetry): carry app_version on instance_started (#681)
app_version was attached only to feedback events; every other allowlist stripped it, so the install base could not be segmented by release. One property on the once-per-boot census event covers it. Fixes #674.
2026-07-30 09:40:53 +08:00
SnapOtterandGitHub fe21f352f6 fix(image): validate caire settings as integers (#680)
caire's -width/-height/-blur/-sobel flags are integer-only; a schema-valid float crashed the binary and surfaced as a corrupt-file 422. Reject fractional values at validation time with a settings-shaped 400. Fixes #672.
2026-07-30 09:36:54 +08:00
SnapOtterandGitHub 192d56e2ca fix(audio): refuse trim windows shorter than one codec frame (#679)
trim-audio stream-copies, and a window below one frame ships a container with zero audio frames as a 200 success. Floor the window at 0.1s in the schema, and refuse after endS clamps to the real duration if the window drops under the floor. Fixes #671.
2026-07-30 09:34:56 +08:00
SnapOtterandGitHub 44081e6a3f ci(ocr): verify the baked trust file, not unset image env vars (#666)
Fourth latent bug in the OCR publish chain (#649/#519 added it, never ran).
verify-ocr's trust-identity step read OCR_RUNTIME_INDEX_KEY_ID from the image env,
which the official image leaves unset by design (operator-override path;
runtime-index.ts reads the baked file when env is unset). It compared "" to
snapotter-ocr-2026-07 and failed a correct image. Now verifies the baked
/app/docker/ocr-runtime-trust.json, proven against the built release image.

A full audit of the remaining chain (verify, sign, verify-signed, publish)
reproduced the sign+verify path end-to-end inside the image and confirmed nothing
else breaks for v2.2.0. One latent non-ASCII canonicalization mismatch filed as
#667 (cannot fire on v2.2.0). Non-releasable type so the re-dispatch re-runs 2.2.0.
2026-07-29 20:21:16 +08:00
SnapOtterandGitHub 5c75a93484 ci(release): fix the archive tsx path and patch the go-tools x/text HIGH (#665)
Two latent bugs the v2.2.0 release run surfaced, both added by #649 and never run
in a real release (v2.1.0 had no archive-security job).

archive-security asserted tsx at the workspace root, but tsx is a prod dependency
of apps/api, so pnpm places its bin at apps/api/node_modules/.bin/tsx, where the
Docker CMD runs it. The root path never existed and failed the extract step on
both arches. Fixed to the apps/api path, proven against the real prebuilt-amd64
artifact.

The blocking Trivy scans would then have failed on CVE-2026-56852,
golang.org/x/text v0.38.0 -> v0.39.0, the only fixed CRITICAL/HIGH in the image,
compiled into caire and pdfcpu. Pinned to v0.39.0 in both go-tools modules,
verified building in the golang:1.25.12 toolchain with -mod=readonly and linking
v0.39.0, with a clean Trivy rescan.

Guards added for both the tsx path and the x/text pin. Non-releasable type so a
re-dispatch re-runs 2.2.0.
2026-07-29 19:05:30 +08:00
SnapOtterandGitHub 935861bced fix(release): resolve the release by id, and make the vulnerability gate cover HIGH (#661)
Two release-pipeline defects found while pre-flighting 2.2.0, plus the image
hardening that the second one exposed.

The release job would have died immediately after pushing the v2.2.0 tag.
draftRelease was turned on in #649 and never executed, and GitHub's
/releases/tags/{tag} endpoint does not return draft releases, so all nine tag
lookups in release.yml would have 404'd against the draft semantic-release had
just created. Verified against this repo with a throwaway draft: the tag
endpoint 404s while gh release view reads it and /releases/{id} returns the same
REST shape. Every site now resolves the numeric id first, so existing jq
expressions are untouched.

The unfixed-vulnerability gate was measuring almost nothing. The blocking Trivy
steps run ignore-unfixed, and trivy-unfixed-gate.mjs was meant to cover the
remainder but defaults to CRITICAL with neither call site passing --severity. An
unfixed HIGH was gated by nothing, and the arm64 image carried 79 of them while
the summary read clean.

Rather than document 79 findings, the image lost what it did not need:
libde265 1.1.1 and libheif 1.23.1 are now built from source (the old libheif pin
was itself affected by CVE-2026-3950, and Debian's libde265 1.0.11 was the
decoder every .heic upload actually reached), and xvfb, wget and openssh-client
are purged. 15 CVEs left the image outright and the HIGH gap fell to 65, each
now carrying a rationale verified against the running container.

curl gets its own section: bookworm-backports has a fixed 8.14.1, so claiming no
fix was available would have been false. It is recorded as a declined fix.

Verified on both architectures: gate exits 0, the source-built libde265 is the
one libheif links, and HEIC, RAW, ImageMagick, Sharp AVIF and headless chromium
all still work after the purge.
2026-07-28 22:49:56 +08:00
SnapOtterandGitHub 42e1dc9799 fix(docs): keep the translated locale trees out of the search index (#662)
Search Console flagged four reasons on 2026-07-28: soft 404, both
duplicate-canonical variants, and noindex. All four sat on
docs.snapotter.com; the landing site was clean.

The cause was boilerplate dominance rather than a broken tag.
/tools/video/crop-video carries 1.3 KB of unique body against 3.5 KB of
identical chrome (nav, sidebar, 21-language switcher), so unrelated tool
docs measured 52-60% full-page similarity. Across 20 locales that was
3,640 of 3,822 submitted URLs. Google read the lot as one duplicate
cluster and began electing arbitrary representatives: /changelog became
the canonical for /tools/image/favicon, and /uk/guide/getting-started for
/nl/tools/image/resize. English tool docs indexed 2 of 10 and localized 4
of 10, while the landing page for those same five tools indexed 5 of 5.

Translated pages now emit `noindex, follow` with a self-canonical, and
sitemap.transformItems drops them from the sitemap. Self-canonical rather
than pointing at English, since noindex paired with a cross-canonical
sends two conflicting instructions. hreflang is removed outright: the
annotation only means something between pages that can all be indexed.

Readers see no change. The language switcher and every in-page link
behave exactly as before.

Verified against a real build: sitemap 3,822 to 182 URLs with zero
translated entries, all 3,640 translated files carrying the noindex and
no English file doing so, docs e2e 100 passed.
2026-07-28 17:08:32 +08:00
SnapOtterandGitHub 5f21588f6c chore: prepare the 2.2.0 release (#660)
Bumps every version surface to 2.2.0, fixes a latent version-coupling bug in the
OCR runtime tests, and stops an absent GPU runner from silently stalling a
release.

Version surfaces: scripts/sync-version.sh covers the 11 workspaces, APP_VERSION,
and the docs release commands across all locales. Root package.json plus the
three surfaces the script never reaches are done by hand: the DOCKERHUB.md banner
and tag table, the docker-tags.md pinning table in 21 locales, and the example
runtimeVersion in tools/image/ocr.md in 21 locales. The release-notes archive step
is deliberately not pre-run, so the notes text stays editable until the release.

Latent bug: runtime-state rejects any runtime whose compatibility.snapotterVersion
is not exactly APP_VERSION, and five fixtures pinned the literal 2.1.0. Since
semantic-release rewrites APP_VERSION on every release, the first PR after any
bump would have gone red for a reason nobody would trace to the release. The
fixtures now derive from APP_VERSION.

GPU runner: sign-ocr-index needs verify-ocr-nvidia on self-hosted hardware, and
the gated manifest job needs ai-bundles, so a missing runner queued instead of
failing and produced no image tags. preflight-gpu-runner claims the same labels
with no dependencies, so it is scheduled first and validates the GPU before the
90-minute build. An API preflight is impossible because listing self-hosted
runners needs Administration:read, which GITHUB_TOKEN cannot hold, so RELEASE.md
carries the maintainer-side check.
2026-07-27 22:09:31 +08:00
SnapOtterandGitHub d10d0f544f fix: release QA hardening across processing, media, security, and CI gates (#649)
A release-readiness QA pass over the whole product. The commits split into
defects a user would hit and gates that were reporting green while measuring
nothing.

## Fixes that change behaviour

Rate limiting was bypassable on every install: TRUST_PROXY defaulted to true, so
request.ip came from a client-set header and a forged X-Forwarded-For got past
the login limiter. The default is now a private-network trust list.

A transient Postgres outage stranded in-flight jobs, leaving finished output on
disk with no row pointing at it. A reconciler now resolves those rows and adopts
the bytes rather than dropping the work.

A Redis connection that moved to a new address wedged every read-blocked
consumer, so completions stopped signalling while health still answered 200.
Socket timeouts plus subscriber pings recover it.

Installing more than one AI bundle left the shared venv multi-versioned and
silently broke three tools. The installer now reconciles distributions to one
version each.

Converting an image to JXL at quality 1 through 4 returned a 500, because
libjxl 0.7 rejects the distance those values compute. The quality is floored at
what the encoder honours. A missing ffmpeg was also reported to the user as a
corrupt upload; it now says the engine is unavailable.

RAW uploads reached an unpatched LibRaw on arm64, so it is built from source at
0.22.2, and the release scan was split so it can fail on an unfixed critical
instead of hiding it behind ignore-unfixed.

## Gates that could not fail

Two mutation lanes ran zero mutants because Stryker crawled the gitignored docs
build; coverage discarded its whole report on any failing test; the lint gate
skipped root tests, scripts, and two workspaces; and several generated matrices
counted a host missing ffmpeg as a passing tool. Each now measures what it
claims.

Full evidence and the outstanding release items are tracked locally and are not
part of this branch.
2026-07-27 15:37:30 +08:00
SnapOtterandGitHub bc32f86a07 test: correct the SYNC_WAIT_MS comment and guard six 202 branches (#656)
The comment added in #652 was wrong. It said SYNC_WAIT_MS=0 drives every tool through its 202 path; it does the opposite. BullMQ arms its timer under `if (ttl)`, so 0 is falsy, waitUntilFinished waits forever and every route answers 200, matching the repo-wide convention that 0 means unlimited. A small positive value such as 1 is what forces the async path.

That error mattered: the local validation claimed for #652 exercised the synchronous path throughout and never reached settleAsyncFallback. Redone with SYNC_WAIT_MS=1, it surfaced six specs whose status gate accepts 202 but whose else branch then demands an error body a 202 never carries, a latent flake on a slow runner.

Each now settles the job first. Under a forced 1ms window the 'expected undefined to be defined' failures drop from 55 to 0, and all 1183 tests still pass on the normal 30s window. Per-shard totals unchanged at 9903 tests, 9435 passed, 468 skipped.

Specs asserting a bare 200 without listing 202 are deliberately untouched: they own the synchronous contract.
2026-07-27 14:20:29 +08:00
SnapOtterandGitHub f1ec3beaf7 test: settle 202 jobs instead of returning without asserting (#652)
A 202 means the sync window expired while the job was still running. Tests treated it as a terminal pass: `if (isAsyncFallback(res)) return;` checked the envelope and returned, asserting nothing about the outcome and leaving the job running into the next test, which is the leak cancelAcceptedJobAndWait exists to prevent.

Because the window only expires under load, coverage tracked runner load. On CI 44 tests took this path and verified nothing; the same tests on a dev machine asserted in full (one measured 6.4s locally against 31s on CI).

settleAsyncFallback waits for a terminal state and asserts the job finished, and that a failure carries a message rather than being a crash. A clean failure stays valid, since the exotic-format fixtures are meant to be rejected. All 82 call sites moved over.

per-fork-env no longer floors SYNC_WAIT_MS, so forcing it to 0 drives every request through its 202 path. 570 tests were validated that way and matched their normal-window results exactly.

The 29-34s band dropped from 44 tests (23.4% of test time) to 6 (3.0%). Total test time rose 5.8% and CI wall went 12.8 to 13.1 min: the forks were doing real work during that wait, so this buys determinism, not speed. Per-shard totals unchanged at 9903 tests, 9435 passed, 468 skipped.
2026-07-27 12:16:48 +08:00
SnapOtterandGitHub d9978525fe test: split the three oversized format-matrix specs (#651)
Vitest shards by file and runs a file's tests serially in one fork, so a single spec set the floor for the whole Integration job no matter how many shards or forks it got. Cost-aware sharding (#650) balanced the shards but could not get under that floor.

Split the three specs that exceeded it: format-matrix-comprehensive (1365s) into 4 by describe, format-matrix (1130s) into 4 with Cross-format matrix striped over FORMAT_SAMPLES, and format-matrix-generated (779s) into 3 striped over TOOLS. Largest spec is now 370s. Each preamble moved verbatim into a sibling .shared.ts exposing setupMatrixApp().

Integration shards went from 20m59s/17m55s/16m33s/9m19s to 11m44s/12m31s/10m37s/11m1s.

Coverage checked, not assumed: the set of test names collected by vitest list is byte-identical across the split, 2151 before and 2151 after. Per-shard totals matched the baseline exactly at 9903 tests, 9435 passed, 468 skipped.
2026-07-27 00:36:51 +08:00
SnapOtterandGitHub d91abc8a23 test: shard vitest by measured cost instead of path hash (#650)
Vitest's BaseSequencer sorts specs by SHA1 of their path and slices an equal number of files per shard, which is blind to how long each one takes. That dropped the four costliest generated matrix specs into a single shard: 24m47s against shard 1's 3m47s.

Partition by measured cost instead, greedy longest-processing-time-first. CI wall goes 25 min to 21.3 min. The remaining ceiling is that `format-matrix-comprehensive` and `format-matrix` are each one indivisible file, and tests within a file run sequentially in a single fork.

Coverage is unchanged by construction. The partition is total and disjoint, guarded over the real spec list for shard counts 1 through 8. Per-shard totals on the PR run matched the baseline exactly: 297 files, 9903 tests, 9435 passed, 468 skipped.
2026-07-27 00:06:04 +08:00
SnapOtterandGitHub 2848dd0e53 fix(help): render the help dialog from i18n instead of hardcoded English (#647)
The help dialog carried its 13 shortcut labels, its getting-started
paragraph and its version line as hardcoded English, while fully
translated strings for exactly those labels sat unused in all 21 locale
files. Every non-English user read English there. The translations did
not need writing, only reading: t.help.keyboardShortcuts already had
focusSearchBar, goToTools, processFile and the rest, in every locale.

Labels now index into t.help.keyboardShortcuts by key rather than
carrying text. Getting-started reads t.help.gettingStarted.description,
which drops the inline Kbd chip the hardcoded copy had, matching what all
21 locales already say. The version line goes through
t.help.versionLabel.

Also adds the type-to-search row that #644 left out, keyed
help.keyboardShortcuts.typeToSearch, translated into all 21 locales, and
regenerates the two darwin help-dialog baselines for the extra row.

Nothing caught the original bug because the i18n context defaults to en,
so asserting on English text passes whether or not the component reads
i18n at all. The new test mocks the context with sentinel values instead:
putting the hardcoded labels back fails 15 of its 19 cases.

Verified: 19 new unit tests, full unit suite 7576 passed, help-dialog
visual 3/3 against regenerated baselines, help accessibility e2e 7/7,
typecheck and lint clean, all 18 CI checks green.
2026-07-26 10:28:02 +08:00
SnapOtterandGitHub 2d8b57c57f fix(api): gate every tool endpoint and stop ZIP streams failing quietly (#646)
Three defects from #645, all of which let the server report something
that was not true.

Tool access was enforced per route, so it drifted. createToolRoute calls
requireToolAccess and the factory tools were fine, but all 45
hand-written routes had to remember the same call and none of them did.
A role without tools:use could run image-to-pdf, svg-to-raster,
erase-object, favicon, qr-generate, upscale, sign-pdf and the rest. The
issue described this as affecting two routes; it was every one of them.

The check now lives in a single preHandler keyed off the tool the router
matched, so it covers sub-paths (/batch, /info, /preview, /analyze,
/inspect) and any route added later without that route opting in. Ids no
tool claims stay unresolved, which keeps an unknown or misfiled tool a
404 rather than telling an unauthorized caller which ids exist.

Resolution reads request.routeOptions.url, the pattern the app itself
registered, rather than parsing request.url a second time. find-my-way
decodes before matching, so an independent parse disagrees with the
router and the router wins: `/api/v1/tools/image/%66avicon` ran favicon
while the gate saw no tool at all. Absolute-form request targets slipped
it the same way. Taking the router's own answer removes the disagreement.

A ZIP stream that failed after the 200 headers were out called
reply.raw.end(). On a chunked response that is indistinguishable from
success, so a client kept an archive with no central directory believing
it whole. Worse, a source stream that errored had no listener: the
request hung until it timed out and the error surfaced as unhandled. A
poisoned-storage probe reproduced both. The socket is destroyed instead,
and every source stream is listened to. svg-to-raster additionally ran
its append loop past the hijack with no try/catch, where a throw leaves
Fastify logging and walking away with the socket neither ended nor
destroyed. pdf-to-image is fixed alongside the other two: it shipped in
#643 with the destroy half but not the listener, so it hung the same way.

A zero-byte upload was dropped during parsing. The client pairs results
with its own file list by index, so every later result shifted onto the
wrong file: one document's output was presented as another's, under
another's name, while the file that actually converted was marked "not
found in batch results". Empty parts now keep their slot and fail in
place with a reason.

Two tests in adversarial-extended.test.ts asserted the old zero-byte
behavior, including a comment that batch "silently skips zero-byte
parts". They now pin the replacement: still rejected, nothing processed,
but the caller is told which files were empty instead of being told it
sent none.

A guard walks the whole catalog and fails if any of the 241 tools answers
anything but 403 for a role without tools:use, so a tool cannot escape
the gate by being registered in a shape nobody thought to sample.

Fixes #645
2026-07-26 09:57:44 +08:00
SnapOtterandGitHub 0058fc610f feat(search): start typing anywhere to fill the search box (#644)
Type a printable character on the landing homepage or the app's home
dashboard and it lands in the search box, provided the box is on screen
and nothing else holds focus. Mod+K keeps working unchanged.

The parts that are easy to get wrong live in
packages/shared/src/search/type-to-search.ts so the two surfaces cannot
drift. isTypeToSearchKey decides whether a keystroke is text.
isSearchBoxTypeable decides whether the box is reachable, via one
elementFromPoint hit test at its center, which folds off-screen,
covered-by-a-modal and hidden into a single check that leans on no one's
aria markup. It fails closed where there is no layout engine, so jsdom
tests that mount the search bar do not blow up on it.

Modifier handling reads getModifierState("AltGraph") rather than
inferring AltGr from ctrl+alt. That inference reads correctly on Windows
and is backwards on macOS, where Option alone types accented characters
and ctrl+alt is a shortcut prefix, VoiceOver's included.

Focus is claimed before the keystroke is committed. Browsers silently
refuse focus inside inert or visibility:hidden subtrees, and without the
check an entire query drains into a box the user cannot see.

Scope comes from where the hook is mounted rather than a route check that
could rot, so tool pages, the editor, Files and Automate get nothing. No
new i18n strings, and no new analytics event, since
ANALYTICS_EVENTS.SEARCH already fires off the same state change.

Verified: 44 new unit tests, full unit suite 7557 passed, landing
homepage 24/24, home-page 19/19, gui-keyboard 41/41, typecheck and lint
clean, all 18 CI checks green.
2026-07-26 08:27:17 +08:00
SnapOtterandGitHub a7137958a1 fix(pdf): pdf-to-image presets no longer 404 on 2+ files (#643)
Upload two PDFs to pdf-to-jpg and it answered `Tool "pdf-to-jpg" not
found`. pdf-to-jpg, pdf-to-png and pdf-to-tiff share
registerPdfToImageRoute, which registered a single-file endpoint and
nothing else, so the shared preset settings component's 2+-file
submission fell through to the generic `:section/:toolId/batch` route,
whose registry lookup misses every tool outside
createToolRoute/registerToolProcessFn.

Mirror of #627, different fix. image-to-pdf is many-to-one, so #633 sent
every file in one request. This direction is one-to-many: separate PDFs
want separate conversions, which is what /batch is for. The route now
serves its own /batch, the shape svg-to-raster already uses, and the
literal path beats the generic parametric one.

One PDF fans out to many page images, so a per-file result is a ZIP, same
as the single-file route. A batch returns a ZIP of per-document ZIPs in
upload order, keyed by X-File-Results so each result pairs with the file
it came from. A document that is unreadable, locked, empty, short of the
requested page range, or carrying no pages at all fails alone; 422 with a
reason per file when none survive.

That literal path also shadows the generic route's requireToolAccess
call, which would have turned a 403 into a converted ZIP for roles
without tools:use. All four endpoints in this file now gate.

Four ways the batch path could have reported something untrue are closed
with it: a storage fault blamed on the document (statusCode-carrying
errors now reach the error handler, the rest are logged before being
reduced to a generic message), per-file reasons stranded in a field
parseApiError never reads, a zero-byte upload dropped so that later
results landed on the wrong file, and a mid-stream failure ended cleanly
enough to pass for success (the socket is destroyed instead).

Page rendering and ZIP assembly are shared helpers now, createUniqueNamer
moves to lib/filename.ts next to its two existing copies, and
tool-route-drift fails if any batch-dispatched preset loses its /batch
route. Follow-up for the same defects in the sibling custom routes: #645.

Fixes #632
2026-07-26 01:35:42 +08:00
SnapOtterandGitHub d690a6e26d feat(landing): add a live system status indicator to the footer (#641)
Adds /api/status to the landing Pages worker, HEAD-probing demo.snapotter.com and
docs.snapotter.com with a 2 second per-attempt deadline and one retry. snapotter.com is
not probed; the worker answering the request is the proof it is up.

The footer badge ships grey in the static HTML and only upgrades once the route answers.
A rejected fetch, a non-ok response, an unparseable body, and an unrecognized verdict all
leave it grey, so it never claims green on its own.

Color lives in the dot, never the label: `--color-success` scores 4.498:1 against the
footer's `--color-background-alt`, just under AA.

Four labels across 21 locales.
2026-07-25 20:52:42 +08:00
SnapOtterandGitHub d9a8ae7b7e test(web): stop the onboarding Escape test racing the passive effect (#640)
Unit Tests went red on main after #639 with one failure: the Escape test saw a
fully rendered dialog but zero apiPut calls, so the handler never ran. Escape is
the only control in that file wired through a document listener in a passive
effect; every other control is an onClick prop attached at commit, which is why
the sibling close-button test passed in the same run.

Could not reproduce it: 50 isolated and whole-file runs stayed green, and a probe
counting document keydown registrations shows the listener is already attached
when findByText resolves locally, so the window only opens under CI contention.
Rather than bet on an unconfirmed mechanism, settle effects and retry the
dispatch until the dismissal lands. handleDismiss guards on `busy` and the
settings write is idempotent, so repeats are harmless.

Still non-vacuous: pointing the handler at a key that never fires fails the test
in about a second. 30 consecutive whole-file runs clean, full unit suite 7,474
passing.
2026-07-25 19:50:53 +08:00
SnapOtterandGitHub 5cc0a850c6 feat: add landing testimonial wall, unblock the onboarding survey (#639)
Testimonial wall: new landing section between Feature Highlights and Pricing.
Two rows drifting in opposite directions, CSS-only to keep the zero-runtime
rule. 22 quotes, all verbatim and traceable to a public URL or a
feedback_submitted event. In-app quotes ship unattributed because the feedback
dialog only ever promised "You can contact me about this feedback". Marquee
traps documented in the CSS: a track gap also sits between the last original
and the first clone, so the -50% translate jumped half a gap per loop; and
under dir="rtl" the flex track drifted itself off-screen while "@amn-96"
bidi-reordered to "amn-96@".

Landing stats: DOCKER_FALLBACK read 104,000 against a real 233,057, but the
stale constant was the symptom. Both fetchers swallowed failures in a bare
catch, so a degraded build never announced itself. That warning then exposed
the real bug: getStarCount runs from Navbar and TrustSignals on all 798 pages,
firing ~800 unauthenticated GitHub calls per build and 403ing partway through,
so early pages carried the live count and later pages the fallback. Both
fetchers now memoize the promise.

Onboarding survey: the shipped gate has no activity condition, so it fires on
first admin login; 1,105 of 1,287 surveyed instances never processed a file.
The opaque fixed inset-0 aria-modal with a focus trap and no Escape becomes a
corner card at 12% of the screen, Escape closes, and the optional questions
stay collapsed until the one required answer. Its title was an h1, which
RouteAnnouncer focuses and announces on every route change, so navigating
anywhere announced the survey instead of the page. Now an h2.
2026-07-25 18:45:38 +08:00
SnapOtterandGitHub 025851beef fix: honor unlimited processing timeout (#638)
* fix(web): recover stalled job progress streams

* fix(ai): honor unlimited processing timeout

* fix(web): keep retrying stalled progress streams
2026-07-25 11:36:02 +08:00
SnapOtterandGitHub 841f47f6ca fix(files): decode CLI-decoded formats before URL-import preview generation (#637)
fetch-urls.ts called sharp(buffer).webp() directly on the raw, undecoded buffer for CLI-decoded formats (HEIC, RAW, PSD, ...), which throws and gets silently swallowed, so URL-imported files in those formats came back with no preview and 0x0 dimensions. Decodes once before both preview generation and dimension lookup, mirroring the pattern already used elsewhere in the codebase.

Fixes #634
2026-07-25 10:46:50 +08:00
SnapOtterandGitHub 511633fa1c fix(files): store null instead of 0x0 for undecoded upload dimensions (#636)
validateImageBuffer() intentionally reports {width: 0, height: 0} for every CLI_DECODED_FORMATS member. The file library's upload and save-result endpoints treated that 0 as a real measurement and wrote it into the DB. Adds a measuredDimensions() helper that treats non-positive width/height as unmeasured and stores null instead.

Fixes #635
2026-07-25 10:46:11 +08:00
SnapOtterandGitHub 098ed50d06 fix(image): decode real iPhone HEIC files instead of rejecting them at validation (#631)
validateImageBuffer() never listed heif in CLI_DECODED_FORMATS, so real iPhone HEIC uploads hit Sharp's own metadata probe (its bundled libheif only supports AV1/AVIF) and got rejected before reaching the working heif-convert/heif-dec decode path already wired up downstream. Adds heif to that set, same as raw/psd/tga/bmp/etc.

Also fixes the same gap on erase-object's mask input, which validates through the same function but had no matching decode step, so a HEIC mask reached an unguarded sharp() call and came back as a misclassified server error instead of a clean 422.

Fixes #622
2026-07-25 10:45:12 +08:00
SnapOtterandGitHub 330cf559e0 fix(image): image-to-pdf presets no longer 404 on 2+ files (#633)
jpg-to-pdf and its six image-to-pdf-group siblings share the base tool's
registerImageToPdfRoute, which never registers into the toolRegistry the
generic /batch endpoint reads from. The shared conversion-preset settings
component routed any 2+-file submission to /batch regardless of tool, so
these presets 404'd with `Tool "<id>" not found` past the first file, while
the base image-to-pdf tool stayed unaffected because it bypasses that
dispatch entirely with its own settings component.

MULTI_FILE_TOOLS now includes every image-to-pdf-group preset, derived from
BASE_CONFIG instead of hardcoded, and the preset settings component checks
that set before choosing batch vs. a single combined request.

Fixes #627
2026-07-25 09:18:18 +08:00
EuanandGitHub e0a7aecde8 fix(pdf): restore downloads on PDF conversion preset pages (#629)
Adds downloadUrl/originalSize/processedSize to the pdf-to-image route's synchronous response so PDF conversion presets (pdf-to-png, pdf-to-jpg, pdf-to-tiff) satisfy the standard tool-result contract and show their download action again.

Fixes #623

Co-authored-by: EuanTop <euan@mail.bnu.edu.cn>
2026-07-24 18:53:19 +08:00
SnapOtterandGitHub 301e6eb01a test: coverage campaign and mutation testing across five packages (#628)
Coverage 83.6 to 87.36% lines, 81.63 to 84.14% branches. Mutation testing across five packages: image-engine 85, media-engine 92, doc-engine 87, shared+enterprise 86, apps/api security and jobs slice. Runs all five lanes weekly. Fixes the silently-broken mutation CI (babel pin), a redact-pdf envelope-shape test bug, an untested enterprise license valid-signature path, and an audit test that only exercised a hand-copied reproduction. Test and config only, no product code changes beyond the babel pin and one test-only oidc export. Full suite: 16,712 pass, 0 fail.
2026-07-24 17:36:57 +08:00
SnapOtterandGitHub 44f5aea326 fix(ci): repair the chronically-failing nightly workflow (#624)
The scheduled Nightly had been red for over a week across nearly every job. This
root-causes and fixes each one. All were pre-existing: missing CI provisioning,
specs that drifted as the app grew, a job too heavy for its timeout, and a fuzz
that was never configured for file-upload endpoints. None came from the recent
security merge.

- Coverage + Docker Container E2E: install tesseract and its language packs so
  the built-in Fast OCR tests stop throwing spawn ENOENT; gate two repo-file and
  release-workflow tests that cannot run inside the slimmed container image.
- E2E (Full, Serial, Cross-Browser, Device Matrix): refresh specs that drifted
  behind the app (tool renames, the now admin-only Tools tab, dropped About copy,
  locator collisions scoped to the right region). One real product fix rode
  along: /config/auth was refetched six times per tool-page load, so cache it
  behind a single shared fetch, dropping the tool page from 13 to 8 API calls.
- Extended Matrix + Fuzz: shard the integration suite four ways so the full
  format x tool matrix plus property fuzz fits its budget instead of overrunning
  the 90-minute ceiling every night.
- Schemathesis: exclude the tools with bespoke handlers that process
  synchronously in-request (they hang the fuzz on adversarial input) and suppress
  Hypothesis's data-generation health checks, which fire because file-upload
  endpoints reject the fuzzer's random bytes. not_a_server_error still runs on
  every generated case (5000+ per run).
- Stabilize two long-tail flakes: raise the avif matrix per-test cap from 240s to
  600s, and assert toHaveCount(0) on the deleted user row so a transient success
  toast no longer trips a strict-mode violation.

Verified end to end: the full Nightly workflow is green on this branch (all 14
jobs), and PR CI is green.
2026-07-24 03:54:50 +08:00
SnapOtterandGitHub 079fcd2631 fix(security): close the gaps a full 2.0 re-audit left open (#620)
Follow-up to a full re-audit of the 2.0 tree. Most prior findings were already
fixed; this closes the ones that were not:

- SAML assertion replay: validateInResponseTo ifPresent plus a Redis-backed
  CacheProvider, so a captured signed assertion cannot be replayed. ifPresent
  keeps IdP-initiated SSO working.
- MFA login challenge burned after 5 wrong TOTP codes.
- api_keys.key_prefix indexed; the per-request lookup was a full table scan.
- MAX_AI_JOBS_PER_USER caps a user's in-flight single-file AI jobs (the AI pool
  runs at concurrency 1). Batch and pipeline AI stay uncapped.
- MAX_WORKSPACE_SIZE_GB enforced instead of being dead config.
- SUBPROCESS_MEMORY_LIMIT_MB (default off) for the native media and doc engines;
  not applied to the AI sidecar.
- SVG sanitizer closes unquoted and whitespace-prefixed javascript: hrefs and
  the animateTransform/animateMotion/handler/mpath elements.
- Windows-style paths stripped from error output to match the Sentry scrubber.
- Postgres and Redis compose services get cap_drop plus pids_limit and cpus.
- .env.example ships MAX_SVG_SIZE_MB=50 (0 disabled the cap).

Adds security-focused unit and integration tests. typecheck, biome, and the
full unit and integration suites pass.
2026-07-23 00:18:16 +08:00
SnapOtterandGitHub 44d8109486 fix: enforce settings authority boundaries (#618)
Close generic settings authorization bypasses and enforce per-setting authority, validation, redaction, transactional config import, and route-local write rate limiting.
2026-07-22 20:15:38 +08:00
0467e87bfe fix(download): reset the socket when a stream is shorter than Content-Length (#617)
The download route sets Content-Length from a stat and then streams the
object; when the stat size exceeds the bytes the stream yields (#590
"cause 2"), the client hangs on keep-alive framing waiting for a tail
that never arrives. Both send paths now run through a backpressure-safe
byte-counting Transform that resets the socket on a shortfall, so the
download fails at once instead of hanging. Adds a real-socket regression
test at the generic download route, the coverage gap #590 named.

Refs #590

Co-authored-by: harshjainnn <170849281+harshjainnn@users.noreply.github.com>
2026-07-22 02:03:54 +00:00
SnapOtterandGitHub 1f8a42e548 fix: enforce role authority for user management (#616)
Centralize role-authority enforcement across user management, role management, configuration import, SCIM, GDPR, and MFA mutations. Add regression coverage for delegated custom roles and protect higher-privilege accounts from reset, deletion, or takeover.
2026-07-22 01:23:15 +08:00
SnapOtterandGitHub 129e42b95c feat(feedback): gate onboarding survey on first processing, add prompt lifecycle events (#615)
Defers the onboarding usage survey to the instance's first successful processing (the worker writes a one-time onboarding.firstProcessedAt marker and the overlay gates on it), so it reaches engaged users instead of first-landing visitors.

Replaces the two questions telemetry already answers (modality preference from tool_used, install method from instance_started) with what it can't infer: prior tool, self-host motivation, and discovery source.

Adds feedback_prompt_shown and feedback_prompt_dismissed on all five feedback surfaces (usage survey, per-job prompt, admin install card, global nav dialog, search-miss) so skip and completion rates are measurable, not just submissions. New survey strings translated into all 20 non-English locales.
2026-07-21 16:31:30 +00:00