mirror of
https://github.com/snapotter-hq/SnapOtter.git
synced 2026-08-03 07:46:42 +02:00
2d8b57c57fad4e1d98ab7eb327198809aa64bd09
29
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 SnapOtterandGitHub
|
2d8b57c57f |
fix(api): gate every tool endpoint and stop ZIP streams failing quietly (#646)
Three defects from #645, all of which let the server report something that was not true. Tool access was enforced per route, so it drifted. createToolRoute calls requireToolAccess and the factory tools were fine, but all 45 hand-written routes had to remember the same call and none of them did. A role without tools:use could run image-to-pdf, svg-to-raster, erase-object, favicon, qr-generate, upscale, sign-pdf and the rest. The issue described this as affecting two routes; it was every one of them. The check now lives in a single preHandler keyed off the tool the router matched, so it covers sub-paths (/batch, /info, /preview, /analyze, /inspect) and any route added later without that route opting in. Ids no tool claims stay unresolved, which keeps an unknown or misfiled tool a 404 rather than telling an unauthorized caller which ids exist. Resolution reads request.routeOptions.url, the pattern the app itself registered, rather than parsing request.url a second time. find-my-way decodes before matching, so an independent parse disagrees with the router and the router wins: `/api/v1/tools/image/%66avicon` ran favicon while the gate saw no tool at all. Absolute-form request targets slipped it the same way. Taking the router's own answer removes the disagreement. A ZIP stream that failed after the 200 headers were out called reply.raw.end(). On a chunked response that is indistinguishable from success, so a client kept an archive with no central directory believing it whole. Worse, a source stream that errored had no listener: the request hung until it timed out and the error surfaced as unhandled. A poisoned-storage probe reproduced both. The socket is destroyed instead, and every source stream is listened to. svg-to-raster additionally ran its append loop past the hijack with no try/catch, where a throw leaves Fastify logging and walking away with the socket neither ended nor destroyed. pdf-to-image is fixed alongside the other two: it shipped in #643 with the destroy half but not the listener, so it hung the same way. A zero-byte upload was dropped during parsing. The client pairs results with its own file list by index, so every later result shifted onto the wrong file: one document's output was presented as another's, under another's name, while the file that actually converted was marked "not found in batch results". Empty parts now keep their slot and fail in place with a reason. Two tests in adversarial-extended.test.ts asserted the old zero-byte behavior, including a comment that batch "silently skips zero-byte parts". They now pin the replacement: still rejected, nothing processed, but the caller is told which files were empty instead of being told it sent none. A guard walks the whole catalog and fails if any of the 241 tools answers anything but 403 for a role without tools:use, so a tool cannot escape the gate by being registered in a shape nobody thought to sample. Fixes #645 |
||
|
SnapOtterandGitHub
|
a7137958a1 |
fix(pdf): pdf-to-image presets no longer 404 on 2+ files (#643)
Upload two PDFs to pdf-to-jpg and it answered `Tool "pdf-to-jpg" not found`. pdf-to-jpg, pdf-to-png and pdf-to-tiff share registerPdfToImageRoute, which registered a single-file endpoint and nothing else, so the shared preset settings component's 2+-file submission fell through to the generic `:section/:toolId/batch` route, whose registry lookup misses every tool outside createToolRoute/registerToolProcessFn. Mirror of #627, different fix. image-to-pdf is many-to-one, so #633 sent every file in one request. This direction is one-to-many: separate PDFs want separate conversions, which is what /batch is for. The route now serves its own /batch, the shape svg-to-raster already uses, and the literal path beats the generic parametric one. One PDF fans out to many page images, so a per-file result is a ZIP, same as the single-file route. A batch returns a ZIP of per-document ZIPs in upload order, keyed by X-File-Results so each result pairs with the file it came from. A document that is unreadable, locked, empty, short of the requested page range, or carrying no pages at all fails alone; 422 with a reason per file when none survive. That literal path also shadows the generic route's requireToolAccess call, which would have turned a 403 into a converted ZIP for roles without tools:use. All four endpoints in this file now gate. Four ways the batch path could have reported something untrue are closed with it: a storage fault blamed on the document (statusCode-carrying errors now reach the error handler, the rest are logged before being reduced to a generic message), per-file reasons stranded in a field parseApiError never reads, a zero-byte upload dropped so that later results landed on the wrong file, and a mid-stream failure ended cleanly enough to pass for success (the socket is destroyed instead). Page rendering and ZIP assembly are shared helpers now, createUniqueNamer moves to lib/filename.ts next to its two existing copies, and tool-route-drift fails if any batch-dispatched preset loses its /batch route. Follow-up for the same defects in the sibling custom routes: #645. Fixes #632 |
||
|
SnapOtterandGitHub
|
098ed50d06 |
fix(image): decode real iPhone HEIC files instead of rejecting them at validation (#631)
validateImageBuffer() never listed heif in CLI_DECODED_FORMATS, so real iPhone HEIC uploads hit Sharp's own metadata probe (its bundled libheif only supports AV1/AVIF) and got rejected before reaching the working heif-convert/heif-dec decode path already wired up downstream. Adds heif to that set, same as raw/psd/tga/bmp/etc. Also fixes the same gap on erase-object's mask input, which validates through the same function but had no matching decode step, so a HEIC mask reached an unguarded sharp() call and came back as a misclassified server error instead of a clean 422. Fixes #622 |
||
|
EuanandGitHub
|
e0a7aecde8 |
fix(pdf): restore downloads on PDF conversion preset pages (#629)
Adds downloadUrl/originalSize/processedSize to the pdf-to-image route's synchronous response so PDF conversion presets (pdf-to-png, pdf-to-jpg, pdf-to-tiff) satisfy the standard tool-result contract and show their download action again. Fixes #623 Co-authored-by: EuanTop <euan@mail.bnu.edu.cn> |
||
|
SnapOtterandGitHub
|
301e6eb01a |
test: coverage campaign and mutation testing across five packages (#628)
Coverage 83.6 to 87.36% lines, 81.63 to 84.14% branches. Mutation testing across five packages: image-engine 85, media-engine 92, doc-engine 87, shared+enterprise 86, apps/api security and jobs slice. Runs all five lanes weekly. Fixes the silently-broken mutation CI (babel pin), a redact-pdf envelope-shape test bug, an untested enterprise license valid-signature path, and an audit test that only exercised a hand-copied reproduction. Test and config only, no product code changes beyond the babel pin and one test-only oidc export. Full suite: 16,712 pass, 0 fail. |
||
|
SnapOtterandGitHub
|
89d75853f4 |
fix(download): ask reverse proxies not to buffer file downloads (#604)
Send X-Accel-Buffering: no on file download responses so nginx and compatible reverse proxies stream them through instead of buffering, the usual cause of a self-hosted download that "starts but never finishes". The app already delivers exactly Content-Length bytes; a new real-socket test proves it for both the collated PDF and the multi-file ZIP. Refs #590 |
||
|
SnapOtterandGitHub
|
e7ffb37e98 |
feat(image): add rounded-square and squircle crop tool (#602)
Adds a Rounded Crop image tool for logo, favicon, and app-icon work. It masks the framed square to a rounded rectangle (with a corner-radius control) or an iOS-style squircle, reusing circle-crop's zoom/offset framing, border ring, background fill, and output-size options. Includes translations across all 21 locales. Closes #601 |
||
|
SnapOtterandGitHub
|
7d37f6e6f5 |
fix(pdf): flag scanned PDFs in pdf-to-text and serve text as UTF-8 (#603)
When a PDF has no text layer (scanned or image-only), pdf-to-text now returns a 422 that points at the OCR tool instead of a silent empty file, and text downloads carry charset=utf-8 so UTF-8 Arabic renders correctly when the .txt is viewed inline. Fixes #589 |
||
|
SnapOtterandGitHub
|
73df107758 |
fix(pdf): stop page tools failing on short and encrypted PDFs (#594)
Empty the hardcoded page-range default in remove/split/extract PDF tools (remove-pages defaulted to "2,4-6", out of range for any PDF under 6 pages) and disable submit until a range is entered. Reject password-protected PDFs up front for PDF-only tools with guidance to unlock first, instead of failing cryptically in the qpdf worker. Adds integration + e2e coverage. |
||
|
SnapOtterandGitHub
|
df92f7ee42 |
fix(video): write faststart mp4/mov output from stabilize-video (#593)
Add -movflags +faststart for mp4/mov/m4v output from stabilize-video so the stabilized result streams and previews progressively instead of appearing broken or corrupted (moov atom was landing after mdat). Fixes #588 |
||
|
SnapOtterandGitHub
|
1113c761ea |
feat(library): wire save-mode into the five custom-client tool submitters (#577)
Closes #565. Wires the fileId/saveMode pair into the ocr, erase-object, remove-background, background-replace, and blur-background submitters so the library save-mode selector works for them; remove-background's two-phase effects route now auto-saves the final composite instead of the transparent intermediate. |
||
|
SnapOtterandGitHub
|
1bac663a2e |
feat(erase-object): optional high-quality diffusion inpainting bundle (#566)
Adds an opt-in High Quality mode to the Object Eraser, backed by a new inpaint-hq feature bundle (Stable Diffusion 1.5 inpainting via diffusers). The default fast LaMa path is unchanged. Both arch archives are published to deepsafe/feature-bundles and the manifest carries their real sha256/sizes. Verified end to end: a fresh container pulls the bundle from HuggingFace, checksum-verifies it, extracts torch/diffusers plus the fp16 model, and the HQ sidecar erases a large object with a plausible fill. Refs #141 |
||
|
SnapOtterandGitHub
|
d4eaa655b2 |
fix(audio): expose sample rate setting in Convert Audio (#561)
The Convert Audio tool promised configurable bitrate, sample rate, and channel count, but only format and bitrate were exposed. Adds an optional sampleRate setting (8000 to 96000 Hz, omitted = preserve source) wired through the Zod schema, the FFmpeg -ar flag, the standalone settings panel, and the pipeline builder controls. Impossible combinations fail loudly instead of degrading silently: MP3 + 96000 Hz is rejected (libmp3lame caps at 48 kHz), and MP3 bitrates above the encoder ceiling at low rates (64 kbps at 8 kHz, 160 kbps at 16/22.05 kHz) are rejected rather than clamped. The UI offers only legal combinations and sanitizes stored pipeline settings on load. Docs updated in English plus all 20 localized pages with refreshed i18n_source_hash stamps; two new UI strings added to all 21 locales. Fixes #558 |
||
|
SnapOtterandGitHub
|
7d938af1f9 |
fix(compress-pdf): land close to the target size, honestly (#522)
Target-size compression had only a coarse DPI lever, so it undershot badly (a 350KB target could land at 216KB) and silently missed unreachable targets. Adds JPEG quality as a second lever (forced re-encode so it bites on JPEG scans), folds both into one monotonic quality axis that target-size binary-searches, reports targetMet honestly in the panel across 21 locales, and flips the tool to async for the extra passes. Quality-mode output sizes shift intentionally (slider now drives JPEG quality at full resolution in its top half). |
||
|
SnapOtterandGitHub
|
991c981529 |
fix: make OCR portable and reliable across AMD64 and ARM64 (#519)
* fix: make OCR portable and reliable * fix: harden OCR installation portability * fix: pin OCR partials across downloads * fix: make OCR execution reliably asynchronous * fix: harden OCR portability and docs routes * fix: preserve decoder and docs safeguards |
||
|
SnapOtterandGitHub
|
cb5db59f77 |
feat(tools): remove background from animated GIFs (WebP, APNG) (#502)
Adds a dedicated remove-gif-background AI tool: removes the background from an animated GIF, WebP, or APNG frame by frame and reassembles a transparent (or composited) animation in WebP, APNG, or GIF, with full per-frame effects. Reuses the background-removal bundle. Verified end-to-end with the real rembg model. Closes #496. |
||
|
SnapOtterandGitHub
|
e7cfc00fe1 | fix: preserve colored blocks in PDF-to-Word (#500) | ||
|
SnapOtterandGitHub
|
60d01ab2dd |
fix: release-acceptance QA follow-ups (upload crash, scipy ABI conflict, rate limit, OCR fallback) (#458)
* fix(api): prevent a crash when an over-limit upload stream has no consumer yet busboy's "limit" handler destroyed the file stream with an error but never attached its own error listener, relying entirely on whatever consumes part.file downstream to do so. On a fast enough connection (or a fully buffered body, e.g. Fastify inject()), busboy can process enough bytes to hit the size limit before the route handler's receiveUpload() call has attached its own stream listener, leaving the resulting "error" event with zero listeners -- which crashes the whole process by default in Node. Surfaced by tonight's FULL_MATRIX+FUZZ integration run (880 uncaught exceptions, all the same root cause). Reproduces deterministically in isolation; unrelated to this release's actual code delta (file untouched since PR #413, well before the baseline QA pass). Fix: attach a baseline no-op error listener the moment the stream is created, guaranteeing at least one listener always exists. EventEmitter delivers "error" to every registered listener, so the real consumer's own error handling is unaffected. * fix(ai-bundles): rebuild upscale-enhance and photo-restoration to reconcile scipy ABI upscale-enhance and photo-restoration both depend on codeformer-pip, whose transitive closure (basicsr -> realesrgan -> gfpgan) pulls in an unpinned scipy. Both bundles were last built ~June 18-19, before PR #437 added the manifest's `constraints` array (numpy==1.26.4, scipy==1.12.0, etc.) to pin exactly this kind of dependency during bundle builds. Only the ocr bundle was rebuilt after that fix landed. install_feature.py has no pip install step -- it's a raw tarfile extraction with no cross-bundle conflict resolution, so installing OCR alongside either stale bundle left three incompatible scipy versions' files mixed in the same site-packages directory (a compiled _rotation.*.so from one release next to Python files expecting a different release's API), breaking the `upscale` tool and OCR's higher-quality tiers with an ImportError. Rebuilt both bundles for amd64-gpu and arm64-cpu from the current manifest, verified scipy/scikit-learn/scikit-image/pandas all resolve to the pinned versions in the tarballs themselves, then verified end-to-end on real hardware (Mac arm64 CPU and ubuntu_gpu .248 RTX 4070): installing all affected bundles together now yields exactly one version of each constrained package, `upscale` produces correct output, and OCR's balanced/best tiers correctly use PaddleOCR-GPU instead of erroring out. Published the rebuilt tarballs to the public deepsafe/feature-bundles HuggingFace repo and updated this manifest's sha256/sizes to match. Also adds verify-bundle-compatibility.sh: verify-bundle.sh checks each bundle in isolation (a fresh venv per bundle), which is exactly why this shipped twice -- nothing ever checked that bundles built at different times agree once layered into the one shared venv real installs use. The new script installs every bundle for an arch into one venv and asserts each constrained package has exactly one, correct version. Known follow-up (not fixed here, needs separate discussion): uninstalling a bundle only removes its downloaded model weights, never the site-packages it added, so existing installations that already hit this bug have no clean self-service fix via uninstall+reinstall -- they need a full AI-venv wipe. * fix(docker): bake a real rate limit default for the all-in-one one-liner The documented single-container `docker run` install had RATE_LIMIT_PER_MIN=0 (effectively unlimited, ~50k/min) baked in, since only docker-compose.yml carried a hardened override. A self-hoster following the one-liner path got no meaningful throttling anywhere, including auth-adjacent routes with no dedicated per-route limit. Bakes a generous-but-real 1000/min default into the Dockerfile, raises both compose files' fallback to match so the two documented install paths converge on the same posture, and updates the Zod schema default plus docs that quoted the old value. * fix(api): boot log undercounted tool routes by the conversion-preset total The "Tool routes: N active" line logged before registerConversionPresets(app) ran, so it only ever reported the base 158 tools, 83 short of the real 241-tool total. Presets have to register after the base loop (they delegate to each base tool's own processV2), so the fix moves the log line to after that call and has registerConversionPresets return its count instead of reordering the dependency. * fix(ai): forward {info}/{warning} stderr JSON instead of dropping it The dispatcher stderr parser only recognized {ready} and {progress,stage} shaped JSON lines; anything else that parsed as valid JSON (like ocr.py's GPU-to-tesseract downgrade notice, an {"info": ...} line) matched neither branch and fell through silently, never reaching docker logs. Adds explicit {info}/{warning} handling that forwards to console.log/console.warn, same as the existing [prefix]-tagged non-JSON path. * fix(api): fall back to a lower OCR tier when PaddleOCR itself is unusable ocr.ts already retries lower quality tiers on a crashed dispatcher, but the condition only matched crash-style messages (segfault, exited unexpectedly). ocr.py's own ImportError/exception handlers already produce messages telling the caller to use a lower tier (e.g. on the scipy ABI conflict class of bug), but nothing ever acted on them, so a broken PaddleOCR hard-failed with 422 instead of degrading to Tesseract like ocr-pdf effectively does. Broadens the retry condition to also catch PaddleOCR-engine-unusable messages. Note: ocr-pdf's tesseract-only behavior turned out to be an unrelated, pre-existing, deliberate design choice (PaddleOCR segfaults on rasterized PDF pages on arm64), not a graceful-fallback mechanism to copy -- the two tools weren't actually solving the same problem, so this fixes ocr.ts's own gap rather than trying to mirror ocr-pdf. |
||
|
SnapOtterandGitHub
|
bf417a509e |
fix: first-run QA sweep of the single-container image (#413)
Fixes found by manually testing a fresh install end to end: - auth: the must-change-password gate returned 403 on public routes including /api/v1/health, so every fresh install showed a false "Reconnecting to server" banner on the forced password change screen. Public routes are now exempt (they need no session at all). Adds the gate's first direct tests. - multipart: @fastify/multipart's parts() iterator (9.4.0 and 10.0.0) ends on the request stream's "close", which on a reused keep-alive connection fires while an earlier part is still streaming to storage, silently dropping the parts behind it. The object eraser lost its mask file on every second POST per connection. Replaced with a busboy-driven iterator (lib/multipart-parts.ts) that ends on busboy's own "finish", installed for all routes via a preValidation hook; the tool-factory field-recovery workaround for the same bug is now unnecessary and removed. - eraser: the mask canvas backing store is natural resolution, but "absolute inset-0" does not stretch replaced elements, so the canvas rendered at intrinsic size and the brush ring, strokes, and exported mask were all misscaled on photos larger than the viewport. The canvas now gets an explicit CSS box at the fitted size. - compare slider: solid white divider with a dark halo so it stays visible over light images; still initialised at the painted region. - tool page: the AI bundle install prompt now centers in the content area instead of hugging the top. - api docs: disabled Scalar's cloud features (Ask AI, Generate MCP, Open API Client, dev toolbar), hid the "Powered by Scalar" footer link, and set the page title to "SnapOtter API Reference". The docs CSP blocks those cloud calls by design, so the buttons were dead UI. - docker: embedded Redis comes from packages.redis.io pinned to the 8.x major (was Debian's 7.0.15), matching the Compose stack and the documented claim. Build fails fast if the major ever drifts. - docs: DOCKERHUB.md quick start now leads with the one-command docker run (matching the README) with Compose as the production path; README says embedded Postgres 17 + Redis 8. Claude-Session: https://claude.ai/code/session_01XGB4pGvTvb7sUX4JN745U7 |
||
|
SnapOtterandGitHub
|
b4375e558d |
fix: harden install queue/dispatcher lifecycle and repair review-sweep regressions (#395)
Fixes 15 defects found by a max-effort multi-agent review of the last 6 merged PRs (#388, #390, #391, #392, #393, #394), all adversarially verified before fixing. Install queue + dispatcher (the serious cluster): - features.ts: finalize the installer child exactly once. A failed spawn fires both "error" and "close", and the second event released the file lock and active slot that pump() had just handed to the next queued bundle, letting two pip processes write the same venv concurrently. Outcome recording now happens before pump() so the next bundle's first progress frame cannot race the previous install's bookkeeping. - feature-status.ts: keep failed-install errors in a per-bundle map instead of the single progress slot. With the queue auto-starting the next install, the slot was overwritten within seconds and a failed install vanished without ever surfacing to GET /features. - bridge.ts: scope child lifecycle per process (stopped-children set + request generation tags) instead of an instance-wide shuttingDown flag that the next spawn reset. A stale SIGTERMed child's late close event could record a phantom crash (5 of which permanently disable the dispatcher), null out the freshly spawned child, and reject the new child's pending requests. The request-timeout kill path still counts as a real crash. - install_feature.py: the pre-write disk re-check measured ai_dir's filesystem even when budgeting the cross-filesystem copy that lands on the venv's disk; now each budget is checked against the filesystem the bytes actually land on, so ENOSPC cannot strike mid-write and leave site-packages half overwritten. Behavior regressions: - embed-subtitles: preserve pre-existing subtitle tracks (0:s?) and MKV attachments (0:t?) that the -map 0:v:0/0:a? rewrite silently dropped; data streams stay unmapped on purpose (the actual MPEG remux fix). The new subtitle maps first so the language tag hits the right stream. - usage-survey-overlay: fail closed when the settings fetch fails; the fail-open path rendered the blocking survey against an unhealthy API and soft-locked admins, the lock-out class #392 fixed. - features-store: queued bundles poll instead of each holding an SSE connection (Install All could pin 7 EventSources and exhaust the browser's 6-per-origin HTTP/1.1 limit, hanging the whole app); listenToProgress closes any prior stream and stops any poll before subscribing; installAll skips bundles already installing or queued. Contracts, tests, i18n: - openapi.yaml: add "queued" to the features status enum and document downloadBytes/installedBytes (Schemathesis conformance). - feature-lifecycle e2e: queue transcription (~0.5 GB) instead of ocr (~6 GB) and give the test a budget that covers both install drains (the stacked waits exceeded the old 900s timeout). - docker-compose.qa.yml: parameterize the host port (QA_APP_PORT) so QA_PROJECT_NAME concurrent stacks can actually bind. - compare + watermark-image: restore per-input error attribution ("Invalid first/second image", "Invalid watermark image") lost in the shared-handler migration. - ai-features-section: the "{size} on disk" suffix now goes through i18n; key added to all 21 locales. - watermark-image + content-aware-resize: migrate to the shared inputHandlerFor("image") chain like compare/vectorize/compose, fixing drift in the inline copies (no SVG sanitize, no RAW extension hint, no AVIF probe). Verified: typecheck across 9 workspaces, Biome clean on all changed files, 584 targeted unit tests and 249 integration tests green (including real-ffmpeg embed-subtitles runs). One unit test updated to the new poll-while-queued contract with a single-EventSource assertion. Claude-Session: https://claude.ai/code/session_017mR1HiHaf3a1BmUtrHX4j3 |
||
|
SnapOtterandGitHub
|
b37faed95f |
fix: QA sweep - tool routes, security, i18n, a11y, + AI bundle install hardening (#393)
* fix(api): correct format/filename/container handling across tool routes Found during a comprehensive QA sweep exercising every tool against its full accepted-format matrix: - watermark-image, compose: preserve the requested output format and a matching download filename/extension instead of always emitting the source format - compose: crop oversized overlays to the visible base area instead of crashing Sharp's composite, and reject only overlays fully outside the base image instead of any oversized one - compare, vectorize: switch to the shared image input handler so filenames and formats like .svgz/.tga/RAW survive validation instead of being rejected pre-processing - tool-factory, images-to-video: normalize frames through Sharp before handing them to FFmpeg, fixing GIF/AVIF/RAW image-to-video jobs that previously failed or hung - media-tool, replace-audio, embed-subtitles: fix legacy container MIME/codec handling for MPEG sources and subtitle remux cases - files: expand download MIME mapping for text/data/document/video/audio outputs that were falling back to a generic content type - convert-document/presentation/spreadsheet: same-format conversions now return the original validated file instead of erroring or producing corrupt tiny output Claude-Session: https://claude.ai/code/session_019fpSXhLGLXWwfyZY2tWhLG * fix(web): dropzone a11y, stale localStorage getter, dead code - dropzone: stop making the whole drop-zone section clickable/focusable. A section acting as an interactive element around a real upload button is a nested-interactive-element anti-pattern that confuses screen readers; drag-and-drop doesn't need focus semantics, only the button fallback does. Keeps that button semantic and keyboard-reachable. Updates the two e2e call sites that clicked the section directly. - api, use-auth: read through window.localStorage via the existing API storage helper instead of the bare global, which resolves to Node's experimental localStorage getter under Vitest and threw - find-duplicates-settings, info-settings, login-page: remove dead code (unused zip-download handler, a stale mount-only effect dependency that left cached info stuck at reused indices, an unused response variable) Claude-Session: https://claude.ai/code/session_019fpSXhLGLXWwfyZY2tWhLG * fix(i18n): pt-BR, zh-CN, zh-TW were silently falling back to English The locale loader looked up dynamic-import exports by the raw locale code (mod["pt-BR"], mod["zh-CN"], mod["zh-TW"]), but those three modules export camelCased bindings (ptBR, zhCN, zhTW) since identifiers can't contain hyphens. The lookup returned undefined and every consumer silently fell back to English for these three locales. Replaces the generic lookup with explicit per-locale loaders so the mapping can't drift out of sync again. Also updates the dropzone helper copy across all 21 locales to match the drag-only dropzone wording from the previous commit. Claude-Session: https://claude.ai/code/session_019fpSXhLGLXWwfyZY2tWhLG * fix(docs): clear build warnings in the VitePress site - config.mts: add an onwarn handler for the @vueuse INVALID_ANNOTATION warnings emitted during the docs build - deployment.md: the caddyfile code fence language isn't a shiki grammar VitePress ships with, so it warned on every build; use txt instead Claude-Session: https://claude.ai/code/session_019fpSXhLGLXWwfyZY2tWhLG * test(qa): update QA harness for the drag-only dropzone and regen metadata - api-sweep, qa-helpers, verify-ai: add JSON-body tools, multi-input secondary fixtures, async polling for slow valid jobs, 501 FEATURE_NOT_INSTALLED skip handling, and safer per-tool settings - input-preview, pipeline-ui specs: update upload flow for the drag-only dropzone surface - add tests/fixtures/data/valid/chart.json, a valid chart fixture the updated helpers route to - regenerate tools-meta.json against current TOOLS[] Claude-Session: https://claude.ai/code/session_019fpSXhLGLXWwfyZY2tWhLG * fix(security): close a login timing side-channel, harden zip-slip tests Found during a black-box security sweep of the real auth-enabled production container: a nonexistent username returned 401 in ~3-10ms, while a wrong password for a real user took ~35-42ms, because scrypt verification only ran when a user row existed. That timing gap lets an attacker enumerate valid usernames without ever guessing a password. Now runs verification against a cached dummy hash on the unknown-user path too, so both cases cost the same regardless of outcome. extract-zip already had a relative-traversal regression test (../evil.txt), but its absolute-path rejection branches (name.startsWith("/") / startsWith("\\")) had none. Added the three missing cases: deep relative traversal, absolute Unix path, and Windows-style absolute path. Claude-Session: https://claude.ai/code/session_019fpSXhLGLXWwfyZY2tWhLG * test(qa): add UI-driven AI bundle install scripts QA_PROMPT.md's Phase 2 requires installing AI models the way a user does -- through the UI, on demand from HuggingFace -- and treats the curl-based admin install endpoint as fallback-only. Nothing in the harness actually drove that flow; tests/qa/seed-ai-models.sh installs via docker exec + pip, which is further from a real user than even the API fallback. install-ai-bundles-ui.mts logs in, opens Settings > AI Features, screenshots the pre-install state, clicks Install All, and screenshots progress -- then exits, since installs continue server-side once triggered. verify-ai-install-complete.mts polls bundle status, screenshots the completed state, and runs one real tool per installed bundle to prove the freshly-downloaded model actually executes. Claude-Session: https://claude.ai/code/session_019fpSXhLGLXWwfyZY2tWhLG * fix(qa): correct the apiToolPath import in the AI verify script Dynamic import of the package name failed under tsx's module resolution from apps/api's node_modules context; use the same relative-path import api-sweep.mts already uses successfully. Claude-Session: https://claude.ai/code/session_019fpSXhLGLXWwfyZY2tWhLG * fix(web): correct AI bundle size estimates shown before install Measured real downloads during GPU-node QA verification: photo-restoration pulls ~4.4GB (was advertised as 800MB-1GB, off by 4-5x) and ocr pulls ~5.5GB (was advertised as 3-4GB). Both estimates only accounted for model weights, not the pip dependencies (torch/paddle) that come down with them. Updated to reflect actual total download size, since that's what a user deciding whether they have the disk/bandwidth actually needs to know. Claude-Session: https://claude.ai/code/session_019fpSXhLGLXWwfyZY2tWhLG * fix(web): make desktop Settings reachable when auth is disabled AvatarDropdown (the only desktop entry point to Settings) was gated behind `!isMobile && authEnabled`. With AUTH_ENABLED=false the synthetic anonymous admin user should have full Settings access per how auth.ts documents this mode -- and the mobile bottom nav already worked this way, showing Settings unconditionally. Desktop just had a stray extra gate the component doesn't need: AvatarDropdown already resolves its own username internally (falling back to "admin") and reads authEnabled itself where it actually matters (hiding the Logout button). Removed the outer gate; verified end-to-end against a fresh AUTH_ENABLED=false instance -- avatar now renders, Settings opens, shows the anonymous/Admin identity correctly. Also documents (not changes) a related finding in install_feature.py: detect_arch() always resolves amd64 hosts to the GPU-bundled archive variant regardless of actual GPU presence, since no CPU-only amd64 archive is published to the bundle repo yet. Left as a code comment rather than a behavior change, since requesting an unpublished archive key would hard-fail installs entirely -- worse than the current oversized-but-working download. Full detail in the QA report. Claude-Session: https://claude.ai/code/session_019fpSXhLGLXWwfyZY2tWhLG * fix(ai): stop logging expected dispatcher reloads as crashes After each AI bundle install the Python dispatcher reloads because the venv changed, and after every app shutdown it's SIGTERMed. Both took the close handler's `code !== 0` branch (SIGTERM makes the exit code null), so they were counted as crashes -- producing an alarming "crash" line in the logs and a pointless ~1s recovery backoff after each of 7 installs. A `stopping` flag set in shutdown() lets the close handler tell an intentional stop apart from a real crash. The request-timeout kill path deliberately does not set it, so a genuinely hung script still records a crash and the 5-in-60s permanent-disable threshold is untouched. Claude-Session: https://claude.ai/code/session_019fpSXhLGLXWwfyZY2tWhLG * fix(api): return a clean message when content-aware resize times out Carving a very high-resolution image down to a tiny target could exceed the caire subprocess timeout, and the raw error forwarded to the user was caire's terminal output -- ANSI color codes and progress-spinner control characters -- instead of anything actionable. Now: the timeout path throws a clear "timed out; try a smaller image or larger target" message (keeping the raw stderr as `cause` for server logs); friendlyError() strips ANSI/control chars centrally so any subprocess dump surfaced through the shared sanitizer is plain text; and the content-aware-resize route (a custom route that bypassed the sanitizer) now routes its error paths through friendlyError like every other tool. Claude-Session: https://claude.ai/code/session_019fpSXhLGLXWwfyZY2tWhLG * fix(ai): stop bundle installs from exhausting host disk Installing an AI bundle on a tight-disk host could push the root filesystem to zero bytes free after the preflight check had already passed. Two root causes: - move_tree used copytree+rmtree, so during the move the extracted payload existed in both staging and the venv at once -- a full transient doubling on disk. Rewrote it to rename entries (a cheap metadata op on the same filesystem, no copy), falling back to a copy only across filesystems. - the preflight budget used the manifest's extractedSize verbatim, which is 0 for several archives, collapsing the estimate to just the compressed size. Added a conservative fallback (3x compressed) so a missing value can't under-reserve. Also added a real-on-disk re-check immediately before the first destructive venv write (measuring the actual extracted payload and whether the move needs extra space for a cross-filesystem copy), which also now covers the offline-import path that previously skipped the disk check entirely; wrapped the moves so an out-of-space failure returns a clean actionable error instead of a traceback; and made the disk check resolve the nearest existing ancestor so it never throws on a not-yet-created venv path. Claude-Session: https://claude.ai/code/session_019fpSXhLGLXWwfyZY2tWhLG * feat(web): show the real per-arch AI bundle download size The bundle cards and install prompt showed a hardcoded, architecture-blind estimatedSize string. That's misleading: amd64 hosts always pull the CUDA-inclusive archive (there's no CPU-only amd64 variant published), so a bundle labelled "1-2 GB" can actually download several times that, while arm64 pulls a much smaller archive for the same label. The manifest already carries the real per-arch compressedSize (and extractedSize where measured), so surface those: a new optional downloadBytes/installedBytes on FeatureBundleState, populated in getFeatureStates() for this host's arch (resolver mirrors install_feature.py detect_arch), shown by the UI when present with estimatedSize kept as the fallback label. Also nudged upscale-enhance's fallback string (4-5 -> 5-6 GB) to match its real compressed size, consistent with the earlier photo-restoration/ocr fixes. Fields are optional so demo/mock and existing tests stay compiling; the manifest's extractedSize is 0 for a few archives, which now surfaces as null rather than a bogus 0. Claude-Session: https://claude.ai/code/session_019fpSXhLGLXWwfyZY2tWhLG * fix(web): move the AI install queue to the server so it survives tab close Installing multiple bundles could silently lose all but the first. The server rejected a concurrent install with 409, so the client worked around it by queueing the rest in browser-local state and only POSTing each once it saw the previous finish. A single POSTed install is durable (the installer child is detached from the request), but a queued one had zero server footprint -- close the tab mid-queue and those installs vanished with no error, while the UI still showed them "Queued". The client "mutex" didn't even serialize: the queued bundles' local waits all resolved at once and raced into concurrent POSTs that 409'd each other. Now the queue lives on the server (a small in-memory FIFO leaf module). The install endpoint enqueues instead of 409-ing and returns 202 {jobId, queued}; a pump starts the next bundle when the current one's child exits (and after an offline import releases the lock), all behind the existing venv + file locks, which are unchanged. The client just POSTs every bundle immediately and reflects the server-reported queued/installing status; Install All fires all POSTs and lets the server serialize them, keeping the one-shot retry-on-failure. Adds "queued" to FeatureStatus (the bundle card already rendered that state) and surfaces it from getFeatureStates. In-memory is deliberate: it matches the existing contract (survives a tab close, not a server restart, which already clears the lock on boot). Claude-Session: https://claude.ai/code/session_019fpSXhLGLXWwfyZY2tWhLG * fix(qa): don't log env-derived credentials in the AI-install script CodeQL flagged clear-text logging of sensitive information: the login status line interpolated the QA base URL and username (both read from the process environment) into a console.log. Replaced with a static message. QA helper only, but it's a real hygiene issue and cleared the high-severity code-scanning alert on the PR. Claude-Session: https://claude.ai/code/session_019fpSXhLGLXWwfyZY2tWhLG |
||
|
SnapOtterandGitHub
|
f3342a1e57 |
fix: harden Docker image and async job responses
Harden Docker runtime packaging, preserve async job response semantics, fix Redis subscriber startup connections, clear lint warnings, and harden enterprise S3 object body handling. |
||
|
SnapOtterandGitHub
|
fd6ebe77b5 | test: expand coverage across jobs and tools (#380) | ||
|
SnapOtterandGitHub
|
63a03d26f2 |
feat: pipeline templates, analytics opt-out, 83 conversion presets, positioning + e2e modernization
Lands five integrated branches: pipeline templates (#355), analytics opt-out (#354), 83 conversion presets bringing the catalog to 240 tools (#356), self-hosted positioning (#353), and e2e modernization (#351). Integration fixes: aligned stale web analytics tests with the opt-out/allow-list model, closed 3 CodeQL incomplete-sanitization alerts in the i18n generator, resolved settings/index/docs/format-matrix conflicts, and corrected tool counts to 240. |
||
|
SnapOtterandGitHub
|
33dfcecd6a |
test(nightly): stabilize the exhaustive nightly suite (#345)
Triaged the nightly failures (all pre-existing, unrelated to the analytics
work) and fixed the ones with clear root causes:
- video-speed: a 1s tiny.mp4 sped up 2x rounds to ~0.75s, flaking the +/-25%
duration assertion under heavy CI load. Use the 8s hero.mp4 (still 44.1kHz)
so rounding is negligible. Verified locally.
- Extended Matrix + Coverage timeouts: full-matrix / coverage-instrumented runs
starve the heavy media tests under 4 forks at the 30s default. Make maxForks
env-overridable (VITEST_MAX_FORKS) and run those jobs with 2 forks + a 300s
timeout so format-matrix conversions and qr-generate stop timing out.
- Device Matrix visual baselines: the update-visual-baselines workflow could
not start the app ('failed to create database') because it never provisioned
Postgres/Redis. Add the same services block the e2e jobs use.
- Docker E2E: a container pnpm install network blip exits 254. Add fetch
retries + a longer network timeout (frozen-lockfile already passes locally).
- Cross-browser: the home page is the tool catalog now (no dropzone), and the
tool routes moved to /<section>/<toolId>. Point the upload test at a real
tool page and fix the stale single-segment routes (/resize -> /image/resize,
etc.).
The flaky/timeout and cross-browser fixes can only be confirmed by the nightly
(they are load- and browser-specific); a fresh nightly run will verify.
|
||
|
SnapOtterandGitHub
|
32c1192d63 |
fix(passport-photo): require the face-detection bundle, not just background-removal (#329)
* fix(passport-photo): require the face-detection bundle, not just background-removal Passport Photo runs face-landmark detection (face_landmarks.py, gated to the face-detection bundle) before background removal (background-removal bundle), but it was only declared under and guarded against background-removal. A user who installed only Background Removal passed every JS-side check, then hit a late "feature_not_installed" from the Python dispatcher gate when the analyze step ran face landmarks, and the UI never told them Face Detection was needed. - shared: add TOOL_EXTRA_BUNDLES + getRequiredBundlesForTool so a tool can declare more than one required bundle (passport-photo needs background-removal and face-detection). enablesTools is untouched, so the one-tool-per-bundle invariant still holds. - api: isToolInstalled() now checks every required bundle; add getFirstMissingBundleForTool() so the analyze and base routes, pipeline (both guards) and batch report the bundle the user actually still needs. - web: the proactive install prompt (tool-page) and features-store treat a tool as installed only when all required bundles are present, and point the prompt at the first missing one (sequential install, no new UI). Refs #327 * test(passport-photo): deterministic integration coverage for the two-bundle guard Boots the real API with an isolated DATA_DIR and controls installed.json to prove the HTTP route behavior end-to-end: - nothing installed -> 501 naming background-removal - only background-removal installed -> 501 naming face-detection (issue #327) - both installed -> guard passes (not 501) - base route reports face-detection too Refs #327 |
||
|
SnapOtter
|
5ffa1d55ea |
Merge branch 'worktree-test+suite-overhaul-and-real-fixtures' into chore/consolidate-v2.0.0
# Conflicts: # tests/integration/generated/settings-matrix.test.ts # tests/integration/platform/api.test.ts # tests/integration/platform/concurrent.test.ts # tests/integration/platform/factory-multi-input.test.ts # tests/integration/security/adversarial-comprehensive.test.ts # tests/integration/security/adversarial-coverage-gaps.test.ts # tests/integration/security/adversarial-extended.test.ts # tests/integration/security/adversarial-final-gaps.test.ts # tests/integration/security/adversarial-matrix.test.ts # tests/integration/security/adversarial-security.test.ts # tests/integration/security/adversarial.test.ts # tests/integration/tools/image/color-adjustments.test.ts |
||
|
SnapOtter
|
941ed27912 |
test: reorganize fixture files into modality-first layout (phase 6b)
Move all fixture files from flat/mixed dirs (content/, media/, documents/,
formats/, hostile/, root loose) into the modality-first hierarchy:
image/{valid,formats,edge,hostile}, video/{valid,formats,hostile},
audio/{valid,formats,hostile}, document/{valid,formats,edge,hostile},
data/valid/, security/. Update index.ts paths, fixtureDir aliases,
all literal refs in 17 e2e/qa/script files, manifest.json, and the
three generator scripts. 163 files moved, 0 dropped, 100 new tests
from expanded document scan.
|
||
|
SnapOtter
|
1727a7a73e |
test: reorganize flat test files into purpose-based subdirectories (phase 6)
Group 245 flat integration tests and 25 loose unit tests into
discoverable subdirectories per spec section 6:
integration/tools/{image,video,audio,document,data}/ (156 files)
integration/platform/ (64 files)
integration/generated/ (14 files)
integration/security/ (10 files)
unit/security/ (8 files, new subdir)
unit/api/ (7 files moved in)
unit/web/ (3 files moved in)
unit/shared/ (6 files moved in)
unit/image-engine/ (1 file moved in)
All moves via git mv (history preserved). Relative imports repaired
for both depth levels (platform/generated/security = +1, tools/ = +2):
static from-imports, dynamic import() calls, vi.mock() paths,
import.meta.dirname joins, and __dirname joins.
Vitest discovery unchanged (no test.include in config, recursive glob
matches subdirs, shard-by-hash unaffected). test-server.ts and
tool-route-drift.test.ts stay at integration root. fixtures/ untouched.
Parity gate: 13189 passing test names before = 13189 after (0 dropped).
|