mirror of
https://github.com/snapotter-hq/SnapOtter.git
synced 2026-08-03 07:46:42 +02:00
main
17
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
ae6a4c8b7c |
fix: error-only Sentry telemetry, storm-proof capture, and crash fixes (#476)
Removes Sentry tracing entirely (BullMQ idle polling burned 4.8M transactions in 2 days at the baked 0.1 rate), decouples PostHog sampling, and replaces the type-only error scrub with a vetted-field sanitizer plus SafeError/ToolInputError contracts. One classified capture path with per-signature throttles and a per-process ceiling makes storms impossible (NODE-1E was 4,541 events from one 30s loop). Browser errors move to a dedicated web Sentry project with their own source maps. Adds the SNAPOTTER_TELEMETRY runtime kill switch and silences test fleets. Crash fixes: remote 204/304 SSRF process kill (NODE-20), conversion-preset boot crash loop (NODE-21), Redis version preflight + unhandled subscribe rejection (NODE-1T), Sign PDF on plain-http origins (NODE-1K/1M), wavesurfer/pdf.js teardown rejections (NODE-1P/1N), bundle-import ZlibError to 400 (NODE-1Z), chart-maker input errors declassified (NODE-1H/1J), asset requests skip the session DB lookup (NODE-1D). |
||
|
|
b37faed95f |
fix: QA sweep - tool routes, security, i18n, a11y, + AI bundle install hardening (#393)
* fix(api): correct format/filename/container handling across tool routes Found during a comprehensive QA sweep exercising every tool against its full accepted-format matrix: - watermark-image, compose: preserve the requested output format and a matching download filename/extension instead of always emitting the source format - compose: crop oversized overlays to the visible base area instead of crashing Sharp's composite, and reject only overlays fully outside the base image instead of any oversized one - compare, vectorize: switch to the shared image input handler so filenames and formats like .svgz/.tga/RAW survive validation instead of being rejected pre-processing - tool-factory, images-to-video: normalize frames through Sharp before handing them to FFmpeg, fixing GIF/AVIF/RAW image-to-video jobs that previously failed or hung - media-tool, replace-audio, embed-subtitles: fix legacy container MIME/codec handling for MPEG sources and subtitle remux cases - files: expand download MIME mapping for text/data/document/video/audio outputs that were falling back to a generic content type - convert-document/presentation/spreadsheet: same-format conversions now return the original validated file instead of erroring or producing corrupt tiny output Claude-Session: https://claude.ai/code/session_019fpSXhLGLXWwfyZY2tWhLG * fix(web): dropzone a11y, stale localStorage getter, dead code - dropzone: stop making the whole drop-zone section clickable/focusable. A section acting as an interactive element around a real upload button is a nested-interactive-element anti-pattern that confuses screen readers; drag-and-drop doesn't need focus semantics, only the button fallback does. Keeps that button semantic and keyboard-reachable. Updates the two e2e call sites that clicked the section directly. - api, use-auth: read through window.localStorage via the existing API storage helper instead of the bare global, which resolves to Node's experimental localStorage getter under Vitest and threw - find-duplicates-settings, info-settings, login-page: remove dead code (unused zip-download handler, a stale mount-only effect dependency that left cached info stuck at reused indices, an unused response variable) Claude-Session: https://claude.ai/code/session_019fpSXhLGLXWwfyZY2tWhLG * fix(i18n): pt-BR, zh-CN, zh-TW were silently falling back to English The locale loader looked up dynamic-import exports by the raw locale code (mod["pt-BR"], mod["zh-CN"], mod["zh-TW"]), but those three modules export camelCased bindings (ptBR, zhCN, zhTW) since identifiers can't contain hyphens. The lookup returned undefined and every consumer silently fell back to English for these three locales. Replaces the generic lookup with explicit per-locale loaders so the mapping can't drift out of sync again. Also updates the dropzone helper copy across all 21 locales to match the drag-only dropzone wording from the previous commit. Claude-Session: https://claude.ai/code/session_019fpSXhLGLXWwfyZY2tWhLG * fix(docs): clear build warnings in the VitePress site - config.mts: add an onwarn handler for the @vueuse INVALID_ANNOTATION warnings emitted during the docs build - deployment.md: the caddyfile code fence language isn't a shiki grammar VitePress ships with, so it warned on every build; use txt instead Claude-Session: https://claude.ai/code/session_019fpSXhLGLXWwfyZY2tWhLG * test(qa): update QA harness for the drag-only dropzone and regen metadata - api-sweep, qa-helpers, verify-ai: add JSON-body tools, multi-input secondary fixtures, async polling for slow valid jobs, 501 FEATURE_NOT_INSTALLED skip handling, and safer per-tool settings - input-preview, pipeline-ui specs: update upload flow for the drag-only dropzone surface - add tests/fixtures/data/valid/chart.json, a valid chart fixture the updated helpers route to - regenerate tools-meta.json against current TOOLS[] Claude-Session: https://claude.ai/code/session_019fpSXhLGLXWwfyZY2tWhLG * fix(security): close a login timing side-channel, harden zip-slip tests Found during a black-box security sweep of the real auth-enabled production container: a nonexistent username returned 401 in ~3-10ms, while a wrong password for a real user took ~35-42ms, because scrypt verification only ran when a user row existed. That timing gap lets an attacker enumerate valid usernames without ever guessing a password. Now runs verification against a cached dummy hash on the unknown-user path too, so both cases cost the same regardless of outcome. extract-zip already had a relative-traversal regression test (../evil.txt), but its absolute-path rejection branches (name.startsWith("/") / startsWith("\\")) had none. Added the three missing cases: deep relative traversal, absolute Unix path, and Windows-style absolute path. Claude-Session: https://claude.ai/code/session_019fpSXhLGLXWwfyZY2tWhLG * test(qa): add UI-driven AI bundle install scripts QA_PROMPT.md's Phase 2 requires installing AI models the way a user does -- through the UI, on demand from HuggingFace -- and treats the curl-based admin install endpoint as fallback-only. Nothing in the harness actually drove that flow; tests/qa/seed-ai-models.sh installs via docker exec + pip, which is further from a real user than even the API fallback. install-ai-bundles-ui.mts logs in, opens Settings > AI Features, screenshots the pre-install state, clicks Install All, and screenshots progress -- then exits, since installs continue server-side once triggered. verify-ai-install-complete.mts polls bundle status, screenshots the completed state, and runs one real tool per installed bundle to prove the freshly-downloaded model actually executes. Claude-Session: https://claude.ai/code/session_019fpSXhLGLXWwfyZY2tWhLG * fix(qa): correct the apiToolPath import in the AI verify script Dynamic import of the package name failed under tsx's module resolution from apps/api's node_modules context; use the same relative-path import api-sweep.mts already uses successfully. Claude-Session: https://claude.ai/code/session_019fpSXhLGLXWwfyZY2tWhLG * fix(web): correct AI bundle size estimates shown before install Measured real downloads during GPU-node QA verification: photo-restoration pulls ~4.4GB (was advertised as 800MB-1GB, off by 4-5x) and ocr pulls ~5.5GB (was advertised as 3-4GB). Both estimates only accounted for model weights, not the pip dependencies (torch/paddle) that come down with them. Updated to reflect actual total download size, since that's what a user deciding whether they have the disk/bandwidth actually needs to know. Claude-Session: https://claude.ai/code/session_019fpSXhLGLXWwfyZY2tWhLG * fix(web): make desktop Settings reachable when auth is disabled AvatarDropdown (the only desktop entry point to Settings) was gated behind `!isMobile && authEnabled`. With AUTH_ENABLED=false the synthetic anonymous admin user should have full Settings access per how auth.ts documents this mode -- and the mobile bottom nav already worked this way, showing Settings unconditionally. Desktop just had a stray extra gate the component doesn't need: AvatarDropdown already resolves its own username internally (falling back to "admin") and reads authEnabled itself where it actually matters (hiding the Logout button). Removed the outer gate; verified end-to-end against a fresh AUTH_ENABLED=false instance -- avatar now renders, Settings opens, shows the anonymous/Admin identity correctly. Also documents (not changes) a related finding in install_feature.py: detect_arch() always resolves amd64 hosts to the GPU-bundled archive variant regardless of actual GPU presence, since no CPU-only amd64 archive is published to the bundle repo yet. Left as a code comment rather than a behavior change, since requesting an unpublished archive key would hard-fail installs entirely -- worse than the current oversized-but-working download. Full detail in the QA report. Claude-Session: https://claude.ai/code/session_019fpSXhLGLXWwfyZY2tWhLG * fix(ai): stop logging expected dispatcher reloads as crashes After each AI bundle install the Python dispatcher reloads because the venv changed, and after every app shutdown it's SIGTERMed. Both took the close handler's `code !== 0` branch (SIGTERM makes the exit code null), so they were counted as crashes -- producing an alarming "crash" line in the logs and a pointless ~1s recovery backoff after each of 7 installs. A `stopping` flag set in shutdown() lets the close handler tell an intentional stop apart from a real crash. The request-timeout kill path deliberately does not set it, so a genuinely hung script still records a crash and the 5-in-60s permanent-disable threshold is untouched. Claude-Session: https://claude.ai/code/session_019fpSXhLGLXWwfyZY2tWhLG * fix(api): return a clean message when content-aware resize times out Carving a very high-resolution image down to a tiny target could exceed the caire subprocess timeout, and the raw error forwarded to the user was caire's terminal output -- ANSI color codes and progress-spinner control characters -- instead of anything actionable. Now: the timeout path throws a clear "timed out; try a smaller image or larger target" message (keeping the raw stderr as `cause` for server logs); friendlyError() strips ANSI/control chars centrally so any subprocess dump surfaced through the shared sanitizer is plain text; and the content-aware-resize route (a custom route that bypassed the sanitizer) now routes its error paths through friendlyError like every other tool. Claude-Session: https://claude.ai/code/session_019fpSXhLGLXWwfyZY2tWhLG * fix(ai): stop bundle installs from exhausting host disk Installing an AI bundle on a tight-disk host could push the root filesystem to zero bytes free after the preflight check had already passed. Two root causes: - move_tree used copytree+rmtree, so during the move the extracted payload existed in both staging and the venv at once -- a full transient doubling on disk. Rewrote it to rename entries (a cheap metadata op on the same filesystem, no copy), falling back to a copy only across filesystems. - the preflight budget used the manifest's extractedSize verbatim, which is 0 for several archives, collapsing the estimate to just the compressed size. Added a conservative fallback (3x compressed) so a missing value can't under-reserve. Also added a real-on-disk re-check immediately before the first destructive venv write (measuring the actual extracted payload and whether the move needs extra space for a cross-filesystem copy), which also now covers the offline-import path that previously skipped the disk check entirely; wrapped the moves so an out-of-space failure returns a clean actionable error instead of a traceback; and made the disk check resolve the nearest existing ancestor so it never throws on a not-yet-created venv path. Claude-Session: https://claude.ai/code/session_019fpSXhLGLXWwfyZY2tWhLG * feat(web): show the real per-arch AI bundle download size The bundle cards and install prompt showed a hardcoded, architecture-blind estimatedSize string. That's misleading: amd64 hosts always pull the CUDA-inclusive archive (there's no CPU-only amd64 variant published), so a bundle labelled "1-2 GB" can actually download several times that, while arm64 pulls a much smaller archive for the same label. The manifest already carries the real per-arch compressedSize (and extractedSize where measured), so surface those: a new optional downloadBytes/installedBytes on FeatureBundleState, populated in getFeatureStates() for this host's arch (resolver mirrors install_feature.py detect_arch), shown by the UI when present with estimatedSize kept as the fallback label. Also nudged upscale-enhance's fallback string (4-5 -> 5-6 GB) to match its real compressed size, consistent with the earlier photo-restoration/ocr fixes. Fields are optional so demo/mock and existing tests stay compiling; the manifest's extractedSize is 0 for a few archives, which now surfaces as null rather than a bogus 0. Claude-Session: https://claude.ai/code/session_019fpSXhLGLXWwfyZY2tWhLG * fix(web): move the AI install queue to the server so it survives tab close Installing multiple bundles could silently lose all but the first. The server rejected a concurrent install with 409, so the client worked around it by queueing the rest in browser-local state and only POSTing each once it saw the previous finish. A single POSTed install is durable (the installer child is detached from the request), but a queued one had zero server footprint -- close the tab mid-queue and those installs vanished with no error, while the UI still showed them "Queued". The client "mutex" didn't even serialize: the queued bundles' local waits all resolved at once and raced into concurrent POSTs that 409'd each other. Now the queue lives on the server (a small in-memory FIFO leaf module). The install endpoint enqueues instead of 409-ing and returns 202 {jobId, queued}; a pump starts the next bundle when the current one's child exits (and after an offline import releases the lock), all behind the existing venv + file locks, which are unchanged. The client just POSTs every bundle immediately and reflects the server-reported queued/installing status; Install All fires all POSTs and lets the server serialize them, keeping the one-shot retry-on-failure. Adds "queued" to FeatureStatus (the bundle card already rendered that state) and surfaces it from getFeatureStates. In-memory is deliberate: it matches the existing contract (survives a tab close, not a server restart, which already clears the lock on boot). Claude-Session: https://claude.ai/code/session_019fpSXhLGLXWwfyZY2tWhLG * fix(qa): don't log env-derived credentials in the AI-install script CodeQL flagged clear-text logging of sensitive information: the login status line interpolated the QA base URL and username (both read from the process environment) into a console.log. Replaced with a static message. QA helper only, but it's a real hygiene issue and cleared the high-severity code-scanning alert on the PR. Claude-Session: https://claude.ai/code/session_019fpSXhLGLXWwfyZY2tWhLG |
||
|
|
63a03d26f2 |
feat: pipeline templates, analytics opt-out, 83 conversion presets, positioning + e2e modernization
Lands five integrated branches: pipeline templates (#355), analytics opt-out (#354), 83 conversion presets bringing the catalog to 240 tools (#356), self-hosted positioning (#353), and e2e modernization (#351). Integration fixes: aligned stale web analytics tests with the opt-out/allow-list model, closed 3 CodeQL incomplete-sanitization alerts in the i18n generator, resolved settings/index/docs/format-matrix conflicts, and corrected tool counts to 240. |
||
|
|
5d36ac06d8 |
feat(analytics): build-time bake + telemetry depth (#336)
Bake PostHog + Sentry into the published Docker image (SNAPOTTER_ANALYTICS build arg, codegen script). Delete entire consent system. Move event emission to BullMQ worker. Add cross-tier identity stitching, Sentry performance tracing on both tiers, frontend funnel events. Fix stateful regex bug. 86 files changed, 1593 insertions(+), 3747 deletions(-) |
||
|
|
492da820f8 | fix(web): resolve feature install status sync and mutual exclusivity (#214) | ||
|
|
436576fe15 |
fix: resolve features-store test timeouts and dynamic icon-map test (#151)
- Fix installAll tests timing out by returning installed status from refreshBundles mock (prevents infinite retry loop) and increasing waitFor/test timeouts to accommodate the 2s inter-install delay - Fix zustand-stores installAll test with same refreshBundles mock fix - Icon-map test already fixed to derive from shared constants |
||
|
|
e1ca06cd95 | fix: update unit tests for DecodedPreview return type and outpaint tier arg | ||
|
|
a0556772e8 |
test: expand coverage across all layers -- 1,268 new tests, fix replace-color div-by-zero
14-agent parallel test expansion covering integration, unit, E2E, E2E-Docker, cross-format matrix, adversarial, GUI navigation/tools/settings/visual/a11y/perf. - Integration: expand 23 tool test files with HEIC, stress, batch, edge cases - Unit: close coverage gaps in image-engine, stores, lib (metadata, auto-enhance, connection-store, lazy-with-retry, collage/file-store HEIC preview) - AI bridge: 141 new tests for dispatcher buffering, crash recovery, OOM/segfault - Cross-format: 794 parameterized tests (16 formats x 12 tools + no-crash matrix) - Adversarial: memory stress (50x large file), zero-byte, corrupted headers, unicode - E2E-Docker: expand 8 spec files with dimension verification, pipeline chains - GUI E2E: tool UI settings/interactions for all 47 tools, remove all test.skip, RBAC per-role verification, visual screenshot naming, cross-browser smoke tests, a11y ARIA/focus/contrast, performance budgets, 15-tool stability test - Fix: replace-color.ts tolerance=0 caused division-by-zero producing NaN pixels Total: 8,958 tests passing across 202 files. Zero failures, zero skips. |
||
|
|
649ad5db9e |
test: massive test coverage expansion (+1,437 tests, 22 new files)
Expand test coverage across all layers via 14 parallel agents: Unit tests (3,378 total, +534): - First-ever AI sidecar tests (157 tests covering bridge lifecycle, all 12 tool modules) - API route infrastructure (auth, pipeline, batch, settings, teams, roles, audit, api-keys, files, docs) - Lib coverage improvements (audit 7%->95%, worker-pool 33%->100%) - Web store/lib gap fills (features-store, tool-registry) Integration tests (4,403 total, +903): - Expanded 19 tool test files with parameter variations, format edge cases, boundary values - Cross-format matrix: 290 tests covering 14 tools x 17 formats - Adversarial/edge cases: 63 tests for extreme inputs, concurrent requests, corrupted files E2E-Docker (125 new tests): - Expanded 8 spec files + 1 new file covering all 49 tools - Added HEIC/format handling, auth failures, download verification GUI E2E (expanded 28 spec files): - Navigation, responsive layout, keyboard shortcuts - All 51 tool UIs with settings, processing, display modes - Batch/pipeline workflows, settings/RBAC, visual regression - Resilience, accessibility (ARIA, contrast, focus), performance budgets |
||
|
|
8f9ba701be |
fix: QR code logo causes preview to vanish and become unrecoverable
The QR code generator's logo feature was broken in production (Docker) due to three interacting issues: 1. The CSP connect-src directive did not include data:, so the qr-code-styling library's internal XHR to convert logo data URLs to blobs was silently blocked. The library has no onerror handler, so the render promise hung forever after the container was already cleared. 2. crossOrigin: "anonymous" was unnecessarily set on imageOptions for data URLs, which can cause canvas taint issues. 3. The logo options used a conditional spread that omitted the image key when no logo was set. The library's update() deep-merges options, so removing the logo preserved the stale data URL and the QR stayed broken even after logo removal. Closes #121 |
||
|
|
e358634f8b |
fix: production CSP blocking PostHog/Sentry/Scalar and silent failure hardening
The production CSP had connect-src/script-src/font-src set to 'self' only, silently blocking all analytics and error reporting in production while working fine in dev (where CSP is not applied). CSP fixes: - Add PostHog ingest + assets origins to connect-src and script-src - Add Sentry ingest origin to connect-src - Add Scalar fonts origin to font-src for API docs pages - Extract CSP construction into testable buildCsp() function Silent failure hardening: - Settings/features stores now set loadError flag and allow retry on subsequent fetch() calls instead of permanently caching failed state - Analytics init no longer sets initialized=true before the try block, allowing retry on failure - Settings dialog Tools section disables save button when settings failed to load, preventing accidental config wipe - Branding logo storage moved from process.cwd() to FILES_STORAGE_PATH so logos persist across Docker container recreation Test coverage: - 16 CSP directive tests covering all external service domains - Store retry-on-error behavior tests for settings and features stores - Analytics init retry-after-failure test |
||
|
|
4f81b29fbc |
fix: AI feature install failures — missing rembg session and Fastify 415 (#102, #103)
Register custom BiRefNet-matting ONNX session in install_feature.py so
rembg.new_session("birefnet-matting") no longer raises ValueError during
on-demand installs. The session was already registered in remove_bg.py
(runtime) and download_models.py (build-time) but was missed in the
install path, causing background-removal bundle installs to always fail.
Send JSON body on install/uninstall POST requests to avoid Fastify 5's
strict content-type parser rejecting body-less POSTs with 415.
Fix error message extraction to preserve structured {"error": ...} JSON
from the Python script and filter out pthread_setaffinity_np noise.
|
||
|
|
733ebe8010 |
test: major coverage expansion — 18 new test files, ~830 new tests
Unit tests: 1354 → 1781 (+427) - 11 new AI bridge module tests (packages/ai/ from 2/13 → 13/13 files) - files-page-store (0% → full), pdf-to-image-store, features-store expanded - saturation and edit-metadata image-engine operations - analytics route, features route, web analytics lib, api-extended Integration tests: ~2070 → 2320 (+250) - 31 integration files expanded with branch-coverage-targeted tests - progress.ts SSE endpoints (28% → comprehensive, +18 tests) - gif-tools all modes (+18), pdf-to-image format variants (+13) - Cross-format matrix expanded to 17 tools × 17 formats (467 tests) - Adversarial: concurrent, memory pressure, unicode filenames, pipeline limits E2E-Docker: +1020 lines across 6 spec files - Info, colors, sharpening, base64, QR read, JXL/ICO/SVG formats - Strip-metadata, image-enhancement, content-aware-resize expanded - Batch pipelines, multi-format batches, HEIC input coverage |
||
|
|
c061ad13ce |
fix: default theme setting not persisting across sessions (#98)
Three disconnected systems caused the theme to never apply from server settings: the DEFAULT_THEME env var was parsed but never seeded to the database, the settings store ignored defaultTheme from the API, and the settings dialog wrote to the DB without updating the active theme store. - Seed DEFAULT_THEME and DEFAULT_LOCALE env vars into the settings table on first startup (ensureDefaultSettings in index.ts) - Add applyServerDefault() to theme store that applies the server's default theme only when the user hasn't made an explicit choice - Extract defaultTheme from the settings API response and apply it on fresh sessions (no localStorage preference) - Apply theme immediately when admin saves settings - Allow "system" as a valid DEFAULT_THEME env var value |
||
|
|
7f62bc32db |
test: expand coverage to 3,382 tests across all layers
- Unit: 1,353 tests (42 files) — +256 new tests covering AI bridge modules, image-engine sharpen/optimize-for-web, Zustand stores, and icon-map validation - Integration: 1,640 tests (57 files) — +826 new tests across all tool routes, pipeline/progress/batch infrastructure, user-files, edit-metadata, and a 321-test cross-format matrix - E2E-Docker: 389 passing (20 spec files) — 6 new spec files for batch processing, format conversion, layout, optimization, watermark/overlay, and pipeline chains. Tests verified against fresh Docker container with all 6 AI bundles installed. Bug fixes discovered during testing: - fix(compress): SVG/BMP/exotic formats crashed Sharp encoder — added format-safety fallback to PNG - fix(rate-limit): increase default login attempt limit from 10 to 500 per minute — previous value caused false test failures and is too restrictive for a self-hosted app - fix(auth.setup): wait for consent button visibility before clicking to prevent flaky E2E-Docker auth setup |
||
|
|
0309e0f680 |
chore: deploy to Cloudflare Pages and update branding
- Add Cloudflare Pages deployment for landing page (snapotter.com) and docs (docs.snapotter.com) - Create deploy-landing.yml and update deploy-docs.yml workflows - Update CI to ignore apps/landing/** paths - Fix logo transparency (remove white background) across all apps - Recreate social-preview.png with SnapOtter branding - Update all docs URLs from GitHub Pages to docs.snapotter.com - Update VitePress config: light theme default, fix llms.txt paths - Add .vitepress/cache/ and .env.* to gitignore |
||
|
|
babca4cf97 |
test: comprehensive test coverage expansion (+965 tests)
Add 42 new test files covering all untested tool routes, image engine internals, AI sidecar bridge, Zustand stores, and cross-format compatibility. Expand e2e-docker suite with 7 spec files covering all 48 tools against a real Docker container. Unit tests: - Image engine: format detection, MIME mapping, metadata parsing, pipeline - AI bridge: sidecar lifecycle, all 11 tool functions (mocked) - Web stores: 14 Zustand stores (collage, settings, features, analytics, etc.) - API helpers: format decoders, page range, file validation Integration tests: - 25 tool routes that had zero dedicated tests - Cross-format matrix: 17 input formats x 3 tools - Edge cases: zero-byte files, corrupted headers, path traversal, XSS, SQL injection - Concurrent request handling and pipeline edge cases E2E-Docker (Playwright against real container): - 7 spec files: essential, adjustment, conversion, creative, utility, AI, pipeline - Custom buildMultipart helper for multi-file tool uploads - AI tools gracefully skip when sidecar not installed Fixtures: - Organized test media: formats/ (18 formats) + content/ (17 content types) - Reduced from 3.1 GB unorganized samples to 33 MB structured fixtures Bug fix: - color-adjustments: gamma exposure used invalid single-param gamma() for positive values; fixed to use two-param gamma(gammaIn, gammaOut) form |