Commit Graph
58 Commits
Author SHA1 Message Date
SnapOtterandGitHub 35e18d8b79 fix: GPU deployment robustness (6 fixes from end-to-end testing on an RTX 4070) (#334)
* fix(docker): pin CUDA base to 12.6 so the GPU image starts on R560+ drivers

The amd64 base nvidia/cuda:12.9.2-cudnn-runtime bakes a cuda>=12.9 driver gate enforced by nvidia-container-toolkit at container start, so the image fails to launch on common production drivers (e.g. 570.x / CUDA 12.8). The AI bundles are all cu126 wheels and the image installs libcublas-12-6, so 12.9 was misaligned with the workload. Pin to nvidia/cuda:12.6.3-cudnn-runtime-ubuntu24.04 to match the wheels and lower the driver floor to R560+.

* fix(ai): broaden OOM detection so the rembg lighter-model fallback fires

onnxruntime/CUDA allocation failures surface as 'Failed to allocate memory for requested buffer', CUBLAS_STATUS_ALLOC_FAILED, or bad_alloc, not just 'out of memory'. The background-removal and transparency-fixer fallback-to-lighter-model paths only matched the literal 'out of memory', so the fallback was dead code and transparency-fixer (default birefnet-hr-matting) always failed with an allocation error. Add isMemoryAllocError() and use it in both checks.

* fix(ai): use bundled PaddleOCR models so OCR runs offline

ocr.py passed no model dirs to PaddleOCR, so PaddleX resolved models from ~/.paddlex and downloaded them from HuggingFace at runtime (slow first use, broken air-gapped), ignoring the models the OCR bundle ships in MODELS_PATH; it also pulled doc-orientation/unwarping models that are not bundled. Pin detection, recognition and textline models to the bundled dirs in MODELS_PATH (per language) and disable use_doc_orientation_classify / use_doc_unwarping, with per-component fallback when a model is absent. Verified: OCR runs with zero HuggingFace requests.

* fix(docker): add CAP_KILL so container shutdown is graceful

cap_drop: ALL without re-adding KILL meant tini (PID 1, root) could not forward SIGTERM to the gosu-dropped snapotter process (root minus CAP_KILL cannot signal a different UID). docker stop logged '[FATAL tini] forwarding signal: Operation not permitted', never delivered the signal, and fell back to SIGKILL after the 10s timeout. Add KILL to cap_add in both compose files. Verified: docker stop completes in 0s with SIGTERM delivered (exit 143) and no FATAL tini.

* fix(ai): serialize bundle installs against AI jobs to prevent sidecar segfault

A feature bundle install rewrites the shared Python venv (pip + copytree of site-packages/*.so) as a background subprocess, with no coordination against AI tool jobs that dlopen native libs (torch / onnxruntime CUDA) from the same venv; a job loading a shared object while it is overwritten segfaults the sidecar. Add a process-wide async mutex (venv-lock.ts): bridge.run() acquires it before every AI script and the install route holds it across the installer subprocess. Both run in the same Node process so a module-level lock suffices. Verified: concurrent install + AI job produces zero segfaults and the job serializes behind the install.

* fix(ai): make the venv lock read/write so concurrent AI jobs are not serialized

The first cut used an exclusive mutex, which (a) deferred the dispatcher spawn by a microtask and broke unit tests that synchronously drive the mocked spawn, and (b) serialized AI jobs against each other, removing the dispatcher's by-id request multiplexing. Make it a writer-preferring read/write lock: AI jobs are shared readers (with a synchronous fast path so spawn still happens in-tick) and a bundle install is the exclusive writer. Verified: all 764 AI unit tests pass.

* fix(ai): degrade OCR to Tesseract on CPU-only hosts instead of segfaulting

The amd64 AI bundle ships paddlepaddle-gpu, whose native libs dlopen
libcuda.so.1 at import and segfault on a host without a GPU (libcuda is the
driver lib, injected only by nvidia-container-toolkit on GPU hosts). The
segfault crashed the shared long-lived AI dispatcher and, after a few attempts,
tripped the bridge crash-recovery permanent-disable, wedging all AI until a
container restart. The standalone ocr tool defaults to quality=balanced
(PaddleOCR), so it hit this on every CPU-only deployment; ocr-pdf already
hardcoded Tesseract and was unaffected.

ocr.py now gates the PaddleOCR tiers on gpu_available(): balanced/best
transparently fall back to fast (Tesseract, CPU-capable) when no usable GPU is
present, and run_paddleocr_v5/run_paddleocr_vl refuse before importing paddle so
the GPU build is never dlopen'd on CPU. GPU hosts are unchanged.

Verified on a CPU-only Windows/WSL2 box: ocr returns Tesseract text across
repeated runs with the dispatcher staying healthy (no wedge).
2026-06-23 18:39:51 +08:00
SnapOtterandGitHub 7a70affac5 fix(ai): enforce the feature gate on the per-request fallback path (#331)
The persistent Python dispatcher rejects scripts whose feature bundle is not
installed, but the per-request fallback (used when the dispatcher is down, e.g.
restarting right after a model repair) spawned scripts directly and bypassed
that gate. Behavior was therefore inconsistent: a gated script would fail under
the dispatcher but run under the fallback -- the "works once after a repair"
symptom from the original report.

- add packages/ai/src/feature-gate.ts: SCRIPT_BUNDLE_MAP + missingBundleForScript,
  mirroring TOOL_BUNDLE_MAP in dispatcher.py, reading the same installed.json and
  failing closed exactly like dispatcher._get_installed_bundles()
- runPerRequest now rejects with "feature_not_installed" (the same message the
  dispatcher path surfaces) when a gated script's bundle is not installed
- unit tests for the gate, plus a drift test pinning the TS map to dispatcher.py

Closes #327
2026-06-22 23:49:41 +08:00
SnapOtterandGitHub 3fb8164fa5 feat: add OpenTelemetry distributed tracing (enterprise) (#232)
* feat(tracing): add OpenTelemetry dependencies and --import preload flag

* feat(enterprise): add distributed_tracing feature gate

* feat(tracing): add SDK bootstrap with enterprise gating

* fix(tracing): correct test coverage for enterprise-unavailable path and prevent double-init

Test 2 now mocks @snapotter/enterprise to throw an import error, exercising
the catch block in the preload. Test 3 imports with no endpoint so the preload
is a no-op, avoiding leaked SDK from double-initialization. Added idempotency
guard to initTracing() as a safety net.

* feat(tracing): add Pino trace mixin and shared logger

When OTel tracing is active, every Pino log line now includes traceId,
spanId, and traceFlags fields for log-to-trace correlation. The mixin
is a no-op when no SDK is registered (community users).

* feat(tracing): add _otel to ToolJobData and inject trace context at enqueue

Add optional _otel carrier field to ToolJobData for W3C trace context
propagation across BullMQ job boundaries. When an active OTel span exists,
propagation.inject() writes traceparent/tracestate into the job data before
queue.add(). When no SDK is registered (community edition), the carrier
stays empty and _otel remains undefined -- zero overhead.

* feat(tracing): extract trace context and create spans in BullMQ worker

* feat(tracing): inject trace context into Python sidecar calls

* feat(tracing): add trace context extraction to Python sidecar

* feat(tracing): add shutdownTracing to graceful shutdown sequence

* feat(tracing): enrich HTTP spans with tool_id and user_id attributes

* docs: add OpenTelemetry env var documentation to .env.example

* test(tracing): add lifecycle integration tests for trace propagation

* fix(tracing): inject trace context into pipeline and batch flow jobs

* fix(tracing): add sidecar.execute Node-side span and remove unnecessary comment

Wraps PythonDispatcher.run() with a sidecar.execute span on the Node
side so traces show the full round-trip (Node span -> Python span).
Also removes an obvious comment from logger.ts.
2026-06-15 12:53:06 +08:00
SnapOtter 51666cdd5f feat(tools): 2.0 phase 5 wave 5b - ai pool: ocr-pdf, transcription, background composites (5 tools) (#226) 2026-06-13 10:19:47 +08:00
SnapOtter d647d8ed19 feat(modality)!: SnapOtter 2.0 phase 3 modality framework: media/doc engines, pool routing, display modes (#218) 2026-06-13 10:18:39 +08:00
SnapOtter 66e503730d fix: resolve 6 production Sentry errors
- Prevent @fastify/static double-registration crash via decorateReply guard
- Fix non-ASCII filename header encoding (X-Output-Filename + RFC 5987 Content-Disposition)
- Add EACCES error handling to all startup mkdir calls with actionable messages
- Add WAL autocheckpoint and journal size limit to prevent unbounded SQLite growth
- Fix Python sidecar EPIPE handling to reject pending requests and trigger restart
- Ensure Docker entrypoint creates all subdirectories before chown
2026-06-06 16:05:59 +08:00
SnapOtter 80957f6e10 feat: improve remove background with edge smoothing, color decontamination, output formats
- Expose birefnet-hr-matting in UI (People/Ultra) and fix model defaults
  (People/Max now uses birefnet-matting for true alpha matting)
- Add output format selector (PNG/WebP/AVIF) with lossless alpha support
- Add edge smoothing post-processing (Off/Light/Medium/Strong) via
  morphological mask refinement to reduce gray halo artifacts
- Add color decontamination to remove background color spill from
  semi-transparent edge pixels
- Thread new settings through full stack: frontend -> API schema ->
  Python sidecar -> Sharp effects pipeline
- Add i18n keys for all 21 locales
- Add unit tests for new option serialization (3 tests)
- Add integration tests for new settings validation (4 tests)
2026-06-05 23:05:25 +08:00
SnapOtter 212ef653b5 fix: resolve 5 Sentry production errors (668 total events)
- Filter known client-error noise (rate limit, empty body, unsupported
  media type, content-length mismatch, premature close) from Sentry
  via beforeSend to stop 644 events of non-actionable noise
- Sanitize x-output-filename header to prevent TypeError on non-ASCII
  filenames in optimize-for-web preview (23 events)
- Handle EPIPE on Python dispatcher stdin write with graceful fallback
  to per-request spawning instead of crashing (NODE-W)
- Map EACCES on storage directory/file write to proper 503 status
  instead of generic 500 (NODE-P, 3 events)
2026-05-16 12:22:24 +08:00
SnapOtter 19a607454a fix: improve GPU detection diagnostics and fallback for container environments
The GPU detection in gpu.py had two issues preventing GPU usage in
containers (especially rootless podman with CDI):

1. When torch was installed but torch.cuda.is_available() returned
   False, the function returned immediately without trying the
   ONNX Runtime + nvidia-smi fallback. This meant a CPU-only torch
   build (installed before GPU was available) would block all GPU
   detection, even for ONNX-based tools.

2. The failure logged a generic "torch loaded but CUDA not available"
   with no diagnostic information, making it impossible to debug
   whether the issue was a CPU-only build, missing libraries, or
   device permissions.

The fix restructures gpu_available() into three detection tiers
(torch -> ONNX Runtime -> nvidia-smi) that always fall through on
failure. When torch CUDA fails, it now checks torch.version.cuda to
distinguish CPU-only builds from CUDA builds that can't access the
GPU, and logs LD_LIBRARY_PATH, torch.cuda.init() errors, and
nvidia-smi results.

Also fixes two env var passthrough bugs in buildMinimalEnv():
- SNAPOTTER_GPU was never passed to the Python subprocess, so the
  user-facing GPU override env var had no effect
- MODELS_DIR was a dead entry (never set as env var); replaced with
  MODELS_PATH which the Dockerfile sets and Python scripts read

Closes #134
2026-05-14 23:17:21 +08:00
SnapOtter 4e64ee2779 fix(security): comprehensive security audit and hardening
Auth: login rate limit 30/min (was 500), global rate limit 1000/min (was
unlimited), password/username max lengths on all Zod schemas, session
invalidation on role change, API key legacy scan bounded to 100 keys.

SVG: hardened regex sanitizer with CDATA stripping, XML entity decoding,
set/animate/iframe/embed blocking, comprehensive data: URI blocking,
use element external href blocking. 11 attack payload fixtures added.

SSRF: fixed DNS rebinding TOCTOU by pinning resolved IPs via custom
HTTP/HTTPS agents. Added 6to4 and NAT64 to blocked IPv6 ranges.

Docker: capability dropping (cap_drop ALL + minimal cap_add), resource
limits (4g/8g mem, 512/1024 pids), healthcheck timeout, password
removed from startup banner, default password warning comments.

Network: CSP and HSTS applied in all environments (not just production),
stack traces removed from all error responses, internal paths stripped
from error details, per-route rate limits on uploads (60/min) and URL
fetches (200/hour).

Files: exclusive temp file creation (O_EXCL), disk space circuit
breaker, per-user storage quotas, settings payload 64KB size guard.

Python sidecar: script name allowlist in dispatcher, minimal environment
for subprocess spawns.

Dependencies: fixed 6 production CVEs (drizzle-orm, fastify, fast-uri,
@fastify/static, next, archiver/lodash). Pinned all GitHub Actions to
SHA hashes.

114 security tests added. Full OWASP Top 10 penetration test matrix
verified against production Docker container (30/30 pass after
hardening).
2026-05-13 21:33:50 +08:00
SnapOtter 693b441aa0 refactor(ai): remove mode, add colorizeStrength to restore options 2026-05-13 16:55:38 +08:00
SnapOtter cc2c182e67 feat(ai): pass quality tier through Node outpaint bridge
Add optional tier field to OutpaintOptions interface and forward it as
the 7th argument to the Python outpaint script, defaulting to balanced.
2026-05-13 15:46:02 +08:00
SnapOtter 648c8c12f5 fix: convert input buffer to PNG before passing to face landmarks Python sidecar
AVIF (and other Sharp-native formats) were written as raw bytes to a
.png temp file, causing PIL to fail with "cannot identify image file".
Every other AI module wrapper already converts via sharp().png().toBuffer()
before writing; face-landmarks was the only one that skipped this step.
2026-05-13 14:18:49 +08:00
SnapOtter b06906025c refactor: improve tool processing, dropzone, seam carving, and format encoding
- Refactor use-tool-processor and use-pipeline-processor hooks
- Enhance dropzone component with improved UX
- Improve seam carving with better error handling and tests
- Add JXL format encoding support to format-encoders
- Update tool routes for consistent format handling
- Add dropzone unit tests
2026-05-11 21:57:40 +08:00
SnapOtter 4a863e1ce8 feat: add Node bridge for outpainting 2026-05-11 21:08:08 +08:00
SnapOtter f856c26fcb fix: upscale tool times out on CPU-only systems (NAS/low-power hardware)
The upscale function called runPythonWithProgress without a timeout parameter,
defaulting to the bridge's 10-minute hard limit. On CPU-only systems like
Synology NAS devices, Real-ESRGAN 4x upscaling easily exceeds this for modest
images. Additionally, when the timeout fired on the dispatcher path, the Python
process was left running and blocked all subsequent AI operations.

This fix adds an adaptive timeout based on input megapixels, scale factor, and
GPU availability (180s/effective-MP on CPU, 30s/effective-MP on GPU, floor of
10 minutes). It also kills the dispatcher on timeout so subsequent requests can
proceed via a fresh restart.

Closes #119
2026-05-05 21:14:56 +08:00
SnapOtter b344edf416 feat: add initDispatcher() for eager sidecar startup
The dispatcher was lazy-initialized on first AI request, but a race
condition meant the first call always missed it (dispatcherReady still
false) and fell through to cold per-request Python. initDispatcher()
starts the dispatcher eagerly and returns a Promise that resolves with
GPU status once ready (or after a timeout).
2026-04-30 18:48:15 +08:00
SnapOtter 6d5d0a3673 fix: do not count normal dispatcher exits as crashes
The close handler called recordCrash() unconditionally, even for exit
code 0 (normal MAX_REQUESTS restart). After 5 normal cycles within 60s
the dispatcher was permanently disabled. Now only non-zero exits count.
2026-04-30 18:45:55 +08:00
SnapOtter 4acec0846c test: add comprehensive AI feature install/uninstall test coverage
- Add 55 unit tests for feature-status.ts (installed.json CRUD, cache
  behavior, install lock, model verification, crash recovery, composite
  state) using real temp directories
- Add 36 integration tests for full install/uninstall lifecycle against
  Docker containers (face-detection bundle, SSE progress, tool gates,
  shared model protection, concurrent install prevention, auth guards,
  container restart recovery)
- Fix noise-removal CPU timeout by adding megapixel-based timeout
  calculation (120s/MP, min 5 minutes)
- Fix Playwright auth storage state race condition (mkdirSync before
  saving analytics-user.json)
- Fix 2 skipped tests in fixes-verification.spec.ts by replacing
  external ~/Downloads/sample dependency with existing test fixtures
- Enable skipped analytics-consent settings toggle test
- Restructure features.spec.ts to manage bundle state (uninstall/
  reinstall OCR) so 501 guard tests run instead of skipping
- Update noise-removal test mock to include sharp metadata() method
2026-04-28 13:27:54 +08:00
SnapOtter c6c78dc76f fix: prevent OOM kills during background removal on CPU
Skip alpha matting on CPU (pymatting's sparse matrices are the main
memory hog), auto-downscale images above 2048px before sending to
rembg, and retry with the lighter u2net model when OOM is detected.
2026-04-28 02:20:48 +08:00
SnapOtter dee9452c48 fix: format preservation, dispatcher stability, and health reporting
Closes #17, #18, #19, #31, #32, #33, #34

Format preservation (#17, #18, #19):
- Add resolveOutputFormat to rotate, resize, text-overlay, watermark-text,
  border, replace-color, blur-faces, upscale, erase-object, restore-photo
- Alpha-aware fallback: border with corner radius/shadow and replace-color
  with makeTransparent fall back to PNG for non-alpha formats (JPEG)
- Python sidecar tools (blur-faces, upscale, erase-object) now convert
  PNG output back to input format, matching restore-photo/colorize pattern
- Upscale and erase-object default to "auto" format detection instead of PNG

Dispatcher stability (#31, #32):
- Add gc.collect() and torch.cuda.empty_cache() after each dispatcher request
- Add configurable max_requests (default 50) for periodic dispatcher restart
- Add exponential backoff to dispatcher crash recovery in bridge.ts
- Circuit breaker: 5 crashes within 60s permanently disables dispatcher
- Reset crash counter on successful dispatcher startup

Health & security (#33, #34):
- Export getDispatcherStatus() from @snapotter/ai with running/ready/failed/
  gpu/pid/consecutiveCrashes fields
- Admin health endpoint now includes full dispatcher status
- Add pip-audit job to CI workflow for Python dependency scanning
2026-04-26 03:22:26 +08:00
SnapOtter bf0307d87d fix: QA sweep — 7 bugs fixed, 17 test corrections
Code fixes:
- Sidebar state bleed: reset file store on HomePage mount
- restore-photo: raise error instead of silently skipping colorize
  when DDColor model missing
- PaddleOCR OOM: cap input images to 2048px before OCR inference
- Torch CPU optimization: use --index-url .../whl/cpu on CPU nodes

Test fixes:
- upscale: add exact:true to scale factor button locators
- smart-crop: add exact:true to "Pad to square" locator
- colorize: use regex for model button names (Best/Balanced/Fast)
- enhance-faces: use .first() for ambiguous percentage display
- passport-photo: fix DPI locator, .or() compound, generate fallback
- people: update maxUsers assertions for unlimited (0) default
- automate: "Save Pipeline" → "Save" matching actual button text
- tools.test: add resize to Sharp mock chain for OCR tests
2026-04-25 07:23:58 +08:00
SnapOtter e3259d163a fix: PaddleOCR CPU crash, content-aware-resize limits, barcode fixtures
- Add enable_mkldnn=False to PaddleOCR constructor to bypass PaddlePaddle
  3.3+ OneDNN/PIR crash on CPU-only systems
- Add 25MP and 75% max-reduction guard to seam carving with clear error
  messages instead of silent timeout/crash
- Replace barcode/QR AVIF test fixtures with actual scannable codes
  (old fixtures did not contain real barcodes)
2026-04-24 23:58:06 +08:00
ashim-hq 9a015c8501 fix: AVIF sidecar crash, edit-metadata silent no-op, passport batch blank images, color-palette hex overflow, OCR log noise
- Convert all AI bridge inputs to PNG before writing to disk so PIL can
  read AVIF/WebP/TIFF (7 bridge files; face-detection and OCR already
  had this pattern)
- Add title/author aliases to edit-metadata schema so common field names
  actually write EXIF tags instead of being silently stripped by Zod
- Port extend/pad crop logic from passport-photo single endpoint to the
  batch pipeline so crop regions extending beyond the image get filled
  with background color instead of producing all-white output
- Clamp quantized color channels to 255 in color-palette to prevent
  Math.round(255/16)*16=256 from producing invalid hex like #100100100
- Compare OCR fallback warning against expected engine name per tier
  instead of comparing engine name against tier name (always mismatch)
2026-04-21 23:54:25 +08:00
ashim-hq 6746989aa1 feat: make all hardcoded limits configurable via env vars
- bodyLimit: conditional on MAX_UPLOAD_SIZE_MB (0 = 1GB practical max)
- rate limiting: disabled when RATE_LIMIT_PER_MIN=0
- shutdown timeout: 8s → 30s
- upload plugin: no fileSize/files cap when env=0
- session duration: configurable via SESSION_DURATION_HOURS (default 168h)
- login attempts: configurable via LOGIN_ATTEMPT_LIMIT
- batch/pipeline/svg-to-raster: skip guard when MAX_BATCH_SIZE=0
- pipeline steps: configurable via MAX_PIPELINE_STEPS (0 = unlimited)
- user-files: remove 200 hard cap
- stitch canvas: configurable via MAX_CANVAS_PIXELS (0 = unlimited)
- PDF pages: configurable via MAX_PDF_PAGES (0 = unlimited)
- SVG size: configurable via MAX_SVG_SIZE_MB (0 = unlimited)
- logo size: configurable via MAX_LOGO_SIZE_KB (default 2048)
- worker threads: auto-detect via resolveWorkerThreads (0 = auto)
- megapixels: skip validation when MAX_MEGAPIXELS=0
- seam carving: remove 1200px dimension cap
- concurrency: auto-detect via resolveConcurrency (0 = auto)
2026-04-20 21:50:17 +08:00
ashim-hq be254f9ca6 feat: dynamic timeouts — scale with image size, respect PROCESSING_TIMEOUT_S
Create timeout.ts utility for dynamic timeout computation.
Replace hardcoded timeouts across the stack:
- tool-factory worker: 30s → dynamic based on megapixels
- Python bridge default: 300s → 600s (or env override)
- background-removal: fixed → dynamic based on image size
- OCR: fixed 600s → dynamic based on image size
- seam-carving: 120s → dynamic based on image size
- ExifTool: 30s → 60s
- HEIC converter: 30s → 120s
- SQLite busy_timeout: 5s → 10s
2026-04-20 21:46:07 +08:00
ashim-hq 00041d535d feat: kill all silent fallbacks — fail clearly, never degrade silently
Remove 9 silent fallback chains in the Python sidecar:
- upscale: RealESRGAN→Lanczos (now errors with install guidance)
- upscale: GFPGAN skip (now errors with install guidance)
- gpu: GPU→CPU (now reports device in response, never silent)
- remove_bg: alpha matting fallback (now errors with retry guidance)
- remove_bg: GPU→CPU session (now reports device)
- colorize: DDColor→OpenCV (now errors with install guidance)
- enhance_faces: CodeFormer→GFPGAN (now errors with install guidance)
- ocr: quality cascade (now errors at requested level)
- bridge: dispatcher crash retry (now reports retry in stderr)

Also: raise red_eye max_faces 10→50, face_landmarks max_num_faces configurable,
restore.py min face size 48→24px.
2026-04-20 21:42:19 +08:00
ashim-hq 8c83d7efcd fix: handle HEIC images in blur-faces and red-eye-removal, show warning when no faces detected 2026-04-19 19:52:23 +08:00
ashim-hq 12c4d4de6f fix: update tool installation checks and refactor stdout JSON parsing in AI modules 2026-04-19 12:13:26 +08:00
Ashim 08a7ffe403 Enhance logging and error handling across tools; add full tool audit and Playwright tests
- Added model mismatch warnings in colorize, enhance-faces, and upscale routes.
- Improved error handling in colorize, enhance_faces, remove_bg, restore, and upscale scripts with detailed logging.
- Updated Dockerfile to align NCCL versions for compatibility.
- Introduced a new full tool audit script to test all tools for functionality and GPU usage.
- Created Playwright E2E tests for GPU-dependent tools to ensure proper functionality and performance.
2026-04-17 23:06:31 +08:00
ashim-hq 32239600ae fix: verbose error handling, batch processing, and multi-file support
- Replace [object Object] errors with readable messages across all 20+ API
  routes by normalizing Zod validation errors to strings (formatZodErrors)
- Add parseApiError() on frontend to defensively handle any details type
- Add global Fastify error handler with full stack traces in logs
- Fix image-to-pdf auth: Object.entries(headers) → headers.forEach()
- Fix passport-photo: safeParse + formatZodErrors, safe error extraction
- Fix OCR silent fallbacks: log exception type/message when falling back,
  include actual engine used in API response and Docker logs
- Fix split tool: process all uploaded images, combine into ZIP with
  subfolders per image
- Fix batch support for blur-faces, strip-metadata, edit-metadata,
  vectorize: add processAllFiles branch for multi-file uploads
- Docker: LOG_LEVEL=debug, PYTHONWARNINGS=default for visibility
- Add Playwright e2e tests verifying all fixes against Docker container
2026-04-17 14:15:27 +08:00
ashim-hq f28792a5ed fix: resolve runtime model path mismatch for non-root Docker user
Set U2NET_HOME=/opt/models/rembg so rembg models pre-downloaded at
build time as root are found at runtime by the non-root ashim user.
Without this every fresh container re-downloaded the 973 MB BiRefNet
models on first background-removal request.

Apply the same fix to PaddleOCR: download to /opt/models/paddlex and
symlink into both /root/.paddlex and /app/.paddlex so PaddleX finds
models regardless of which HOME gosu resolves at runtime.

Fall back to per-request spawning in bridge.ts when the persistent
dispatcher crashes mid-request (e.g. OOM loading a large ONNX model),
so the operation succeeds instead of surfacing "Python dispatcher
exited unexpectedly" to the user.

Improve entrypoint.sh permission warning to mention Windows bind mounts
as the likely cause.
2026-04-16 23:45:02 +08:00
2f11b9e101 feat(passport-photo): SOTA passport photo maker with compliance validation (#64)
* feat(passport-photo): add passport specs database and tool constants

* feat(passport-photo): add MediaPipe FaceMesh landmark detection script

* feat(passport-photo): add TypeScript bridge for face landmark detection

* feat(passport-photo): add API routes with analyze and generate endpoints

* fix(passport-photo): accept landmarks from request body and fix pixel coordinate conversion

- Generate endpoint now accepts landmarks + imageWidth/imageHeight in request body
  instead of re-running AI face detection (makes generate phase instant)
- Fixed bug where normalized landmark coordinates (0-1) were used directly
  as pixel values in crop computation - now properly multiplied by imgW/imgH
- Fixed same bug in pipeline process function

* feat(passport-photo): add UI component with live preview and compliance overlay

---------

Co-authored-by: stirling-image <stirling-image@users.noreply.github.com>
2026-04-14 09:59:48 +08:00
6a43cc1b77 feat: SOTA AI photo restoration with multi-step pipeline (#58) (#62)
Add comprehensive photo restoration tool that chains multiple AI models:
- Scratch/tear/spot detection via morphological analysis (top-hat/black-hat transforms)
- Damage inpainting via LaMa ONNX model (reuses existing infrastructure)
- Face enhancement via CodeFormer ONNX (~377MB, from facefusion/models-3.0.0)
- Noise reduction via OpenCV NLMeans in LAB color space
- Optional B&W auto-colorization via DDColor (reuses existing model)

Settings: 3 restoration modes (Light/Auto/Heavy), individual feature toggles
for scratch removal, face enhancement (with fidelity slider), denoising
(with strength slider), and auto-colorize. Before/after comparison view.

Handles HEIC, HEIF, and all standard formats. Batch processing supported.
No new Python dependencies - reuses onnxruntime, cv2, mediapipe, PIL.

Co-authored-by: stirling-image <stirling-image@users.noreply.github.com>
2026-04-13 21:57:51 +08:00
8071fe61c5 feat: AI face enhancement with GFPGAN and CodeFormer (#61)
* feat(shared): add enhance-faces tool definition and i18n strings

* feat(ai): add face enhancement script with GFPGAN and CodeFormer support

Detects faces via MediaPipe dual-model approach, then enhances using
GFPGAN (proven) or CodeFormer (via codeformer-pip) with auto fallback.
Supports strength-based alpha blending with original image.

* feat(ai): add TypeScript bridge for face enhancement

* feat(api): add enhance-faces route with GFPGAN/CodeFormer support

* feat(web): add enhance-faces settings component and register in tool registry

* feat(docker): add CodeFormer dependency and model download

- Add codeformer-pip to both CPU and GPU requirements
- Download CodeFormer model (~375MB) at Docker build time
- Add CodeFormer to smoke test verification

* fix(enhance-faces): address code review findings

- Skip alpha blend for CodeFormer (strength already applied via fidelity weight)
- Hide "only enhance main face" checkbox when Best (CodeFormer) is selected
- Fix sensitivity slider labels (swap More/Fewer faces to match actual behavior)
- Register EnhanceFacesControls in pipeline step settings
- Remove model names from user-facing descriptions

* fix(enhance-faces): fix CodeFormer integration and Docker setup

- Add codeformer-pip install to Dockerfile with --no-deps to avoid numpy 2.x conflict
- Re-pin numpy==1.26.4 after codeformer-pip install
- Pin codeformer-pip==0.0.4 in requirements files
- Broaden auto-mode fallback to catch any Exception from CodeFormer

---------

Co-authored-by: stirling-image <stirling-image@users.noreply.github.com>
2026-04-13 21:56:59 +08:00
9ddeac92b6 feat(red-eye-removal): SOTA red eye removal with MediaPipe Face Mesh + OpenCV LAB correction (#60)
Uses MediaPipe Face Mesh (refine_landmarks=True) for precise iris localization
and OpenCV LAB color space for accurate red-eye detection and luminance-preserving
correction. Zero new dependencies - leverages existing MediaPipe + OpenCV stack.

- Sensitivity slider (LAB 'a' channel threshold)
- Correction strength slider (pupil darkening factor)
- Output format selector (Original/PNG/JPEG/WebP)
- Before/after preview, progress stages, batch processing
- Pipeline support via Controls/Settings split

Co-authored-by: stirling-image <stirling-image@users.noreply.github.com>
2026-04-13 20:22:30 +08:00
dfffc0a8cc feat(noise-removal): SOTA noise removal with 4 quality tiers (#57)
* feat(noise-removal): register tool in shared constants and i18n

* feat(noise-removal): add SCUNet and NAFNet model architectures

* feat(noise-removal): add Python denoising engine with 4 quality tiers

* feat(noise-removal): add TypeScript bridge for Python sidecar

* feat(noise-removal): add frontend settings with 4-tier selector

* feat(noise-removal): register in tool registry and pipeline

* feat(noise-removal): add Fastify API route with Zod validation

* feat(noise-removal): add SCUNet and NAFNet model downloads to Docker build

* test(noise-removal): add to e2e tool page rendering tests

* test(noise-removal): add integration tests for API endpoint

* style: fix biome formatting and import ordering

* fix(noise-removal): use correct model download URLs

NAFNet model is hosted on HuggingFace, not GitHub releases.
Also align SCUNet URL to use the KAIR releases (same as Docker build).

* fix(noise-removal): remove emojis from tier selector, simplify labels

Drop emoji icons from Quick/Balanced/Quality/Maximum buttons. Replace
technical algorithm names with plain descriptions users can understand.

---------

Co-authored-by: stirling-image <stirling-image@users.noreply.github.com>
2026-04-13 19:50:23 +08:00
c280076098 feat: SOTA AI photo colorization with DDColor deep learning model (#57) (#58)
Add AI-powered photo colorization that converts B&W/grayscale images to
full color using DDColor (ICCV 2023 dual-decoder architecture) via ONNX
Runtime. Includes model selection (Auto/DDColor/Classic), adjustable color
intensity, batch processing, before/after preview, and full HEIC/HEIF support.

Co-authored-by: stirling-image <stirling-image@users.noreply.github.com>
2026-04-13 19:40:55 +08:00
Siddharth Kumar Sah 92d4d2d9c6 feat(smart-crop): overhaul with face detection, social presets, and 3 modes
Replace the confusing 2-mode smart crop with a clear 3-mode system:
- Subject Focus: Sharp attention/entropy saliency crop with social media presets
- Face Focus: MediaPipe face detection with headshot framing presets
- Auto Trim: Border removal with optional pad-to-square

Adds detectFaces() to AI package, face preset constants, backward
compatibility for old mode names, and comprehensive integration tests.
2026-04-13 00:47:53 +08:00
Siddharth Kumar Sah 29fafd0722 fix(ocr): fix PaddleOCR crashes, add multi-image and auto-detect language
- Pin PaddlePaddle to 3.0.0 on ARM64 to fix segfault in PIR inference
  engine (3.1+ crashes on aarch64 Debian Bookworm)
- Fix text extraction for PaddleOCR 3.4.x result format (rec_texts)
- Add Node.js-level fallback chain (best -> balanced -> fast) when
  Python subprocess crashes
- Add multi-image OCR: processes all uploaded files sequentially with
  per-file progress and filename headers in combined output
- Convert input images to PNG via Sharp before OCR so HEIC, AVIF, WebP,
  TIFF all work transparently
- Implement real auto-detect language using Tesseract multi-lang script
  detection (analyzes Unicode ranges for Hangul, CJK, Kana, Latin)
- Default enhance to off (hurts clean digital images)
2026-04-12 23:46:39 +08:00
stirling-imageandGitHub ed5f71e2fc Merge pull request #48 from stirling-image/fix/upscale-bugs-and-features
feat: overhaul upscale with bug fixes and advanced features
2026-04-12 19:09:22 +08:00
Siddharth Kumar Sah fe376aebd2 feat: overhaul upscale with bug fixes and advanced features
- Fix multi-image: process selected file, not always first
- Fix progress bar: asymptotic fill prevents visual stalling
- Fix slider: write results to captured index, not current selection
- Add model selection (Auto/AI/Fast), face enhancement, denoise
- Add output format (PNG/JPEG/WebP) with quality control
- Add Upscale All for sequential batch processing with queue
- More granular Python progress stages for smoother UX
2026-04-12 19:04:23 +08:00
Siddharth Kumar Sah 4235875f78 feat(ocr): update TypeScript bridge for quality tiers 2026-04-12 18:34:47 +08:00
Siddharth Kumar Sah 6f5283019b feat: full HEIF/HEIC support, content-aware resize performance fix, UI improvements
- Add bidirectional HEIF support: decode (input) and encode (output) via system heif-convert/heif-enc
- Add server-side WebP preview generation for non-browser-previewable formats (HEIC, TIFF)
- Fix content-aware resize failing on HEIF input (decode before passing to caire)
- Fix content-aware resize timeout on large images by downscaling to max 1200px and using JPEG intermediate
- Add HEIF as target format in convert tool
- Add loading spinner for HEIF preview decode in file store
- Fix file picker not accepting HEIF files (explicit .heic,.heif,.hif extensions)
- Extend frontend timeout for medium tools to 180s with 45s progress animation
- Redesign rotate controls with preset buttons and compact flip section
- Remove misleading savings percentage from convert tool
2026-04-11 23:27:44 +08:00
Siddharth Kumar Sah 1707521f3a feat: replace Python seam carving with caire Go binary
Replace the Python seam-carving library with caire (esimov/caire v1.5.0),
a Go-based content-aware resize engine that is faster and supports both
shrinking and enlarging via seam insertion.

- Add Go builder stage in Dockerfile to compile caire from source
- Rewrite seam-carving.ts to call caire via execFile (no Python sidecar)
- Remove content-aware-resize from PYTHON_SIDECAR_TOOLS (60s timeout)
- Add new options: blur radius, edge sensitivity, square mode, face detection
- Move content-aware toggle below standard resize in UI (subtler placement)
- Rename "Don't enlarge" to "Limit to original size" with hover tooltip
- Add smooth progress bar for medium-duration tools
- Delete seam_carve.py and remove seam-carving pip dependency
- Update integration tests and visual regression screenshots
2026-04-11 17:49:28 +08:00
Siddharth Kumar Sah d3b646207d feat: add seam carving AI bridge module 2026-04-07 23:26:47 +08:00
Siddharth Kumar Sah 29a382e9e0 feat: add GPU/CUDA acceleration support (:cuda Docker tag)
Add a :cuda Docker image tag that auto-detects NVIDIA GPU at runtime
and falls back gracefully to CPU. Same pattern as Immich.

- New gpu.py shared utility for cached CUDA detection
- Background removal (rembg): pass CUDAExecutionProvider to ONNX Runtime
- Upscaling (Real-ESRGAN): use CUDA device + FP16 when GPU available
- OCR (PaddleOCR): enable use_gpu when CUDA detected
- Dispatcher reports GPU status at startup via readiness signal
- Admin health endpoint exposes GPU availability
- Dockerfile uses ARG GPU=false with conditional NVIDIA CUDA base image
- docker-compose.gpu.yml override for GPU users
- CI/CD workflows build and publish :cuda tag (amd64 only)

Three tags: :latest (CPU), :lite (no AI), :cuda (GPU with CPU fallback)
2026-04-05 19:12:45 +08:00
Siddharth Kumar Sah 1cbdfa1590 feat: add worker threads, persistent Python sidecar, graceful shutdown, and architectural improvements
- Graceful shutdown: SIGTERM/SIGINT handlers drain HTTP, stop workers, close DB
- Thumbnail caching: disk-cached thumbnails with immutable Cache-Control headers
- Worker thread pool: Piscina offloads Sharp processing off the main event loop
- Persistent Python dispatcher: pre-imports ML libraries, eliminates cold-start latency
- Tool page registry: declarative tool-to-component mapping replaces 750-line switch
- File store cleanup: remove dead derived fields, stable files array reference
- Job persistence: progress written to SQLite jobs table, stale jobs recovered on startup
2026-03-29 17:23:41 +08:00
Siddharth Kumar Sah 585d66f0c9 refactor: rename Tool.alpha to Tool.experimental 2026-03-26 01:10:51 +08:00
Siddharth Kumar Sah 80e536bcf8 chore: remove dead code, add test infrastructure, update docs
- Delete 3 dead files: use-batch-processor.ts, use-i18n.ts, smart-crop.ts (AI package)
- Remove dead getJobProgress function and unused runPythonScript wrapper
- Remove 6 unused imports across API and web apps
- Remove unused shared types (ImageFormat, AppConfig, ApiError, HealthResponse, JobProgress)
  and constants (SUPPORTED_INPUT_FORMATS/OUTPUT_FORMATS, DEFAULT_OUTPUT_FORMAT)
- Remove unused store method (setOriginalBlobUrl) and clean AI package re-exports
- Add test infrastructure: vitest config, unit/integration/e2e tests, fixtures, screenshots
- Add Docker test infrastructure: Dockerfile.test, docker-compose.test.yml
- Add download_models.py for pre-baking AI model weights in Docker
- Add filename sanitization utility (apps/api/src/lib/filename.ts)
- Update .gitignore to exclude coverage/, *.tsbuildinfo, .superpowers/, test artifacts
- Update .dockerignore to exclude test/coverage/IDE artifacts from builds
- Update docs: remove smart crop from AI docs (uses Sharp directly), update bridge docs
2026-03-23 11:46:45 +08:00