- Add safe_onnx_session() to gpu.py with graceful CUDA EP → CPU fallback
- Replace bare ort.InferenceSession() calls across colorize, restore, inpaint, remove_bg
- Add libcublas-12-6 to production Dockerfile for ONNX Runtime CUDA EP
- Add skipIfFeatureNotInstalled guards to remove-bg, blur-faces, smart-crop, ocr, noise-removal e2e specs
- Add AI tool install prompt detection in tools-all.spec.ts
- Add smart-crop to PYTHON_SIDECAR_TOOLS so frontend shows install prompt correctly
- Create Dockerfile.test.dockerignore to include tests/ in test image builds
- Add libheif-examples and exiftool to Dockerfile.test for HEIC and metadata tests
- Regenerate visual regression baselines for Docker/Linux and skip on non-Docker platforms
- Add 8 new E2E specs for AI tools (upscale, enhance-faces, colorize,
restore-photo, erase-object, smart-crop, passport-photo, red-eye-removal)
closing all HIGH/MEDIUM coverage gaps from the test matrix audit
- Fix ensureAiDirs() crash on non-Docker environments by gating on
isDockerEnvironment() — prevents ENOENT when /data doesn't exist
- Bump torch 2.6.0→2.7.0 and torchvision 0.21.0→0.22.0 in feature
manifest for broader Python version compatibility
- Add Python 3.14 version guard warning in install_feature.py
- Remove duplicate torchvision shims from upscale.py and enhance_faces.py
(dispatcher.py already handles this at startup)
- Remove orphaned tools.batch i18n key and dead pipeline-builder filter
- Regenerate 4 visual regression baselines for current UI state
- Add data-testid to passport-photo generate button for E2E testability
The torchvision compatibility shim for basicsr 1.4.2 was missing the
parent-package binding and only proxied a single attribute, causing
upscale and enhance-faces to fail at import time. The fix adds a
__getattr__ proxy for all attributes, binds the shim to the parent
package, and installs it in the dispatcher at startup for defense-in-depth.
Also removes unused anyInstalling variable, redundant `as any` cast,
and applies Biome formatting fixes across the codebase.
- Fix NameError in restore.py: face enhancement loop used undefined
variable `i`, now uses enumerate()
- Fix gpu.py ONNX fallback: previous smoke-test with empty bytes
always raised, making GPU detection unreachable via the ONNX path.
Now uses nvidia-smi hardware check after confirming CUDA EP is
compiled in — works on Linux, Windows, and gracefully fails on macOS
- Fix cpu_fallback_packages stripping CUDA-specific index URLs when
replacing paddlepaddle-gpu with paddlepaddle for CPU-only systems
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Pin torch==2.6.0+cu126 and torchvision==0.21.0+cu126 in feature
manifest to prevent NCCL symbol mismatch on CUDA 12.6 base images
- Move lpips after torch in install order to prevent wrong version
resolution from PyPI
- Add einops to upscale-enhance common deps (required by SCUNet)
- Update cpu_fallback_packages to handle multi-package CUDA torch
entries on amd64 without GPU
- Fix gpu.py ONNX CUDA detection: replace hardcoded .so path with
cross-platform session smoke-test
- Fix os.dup(1) crashes on Windows in upscale, enhance_faces, and
noise_removal by wrapping in try/except with sys.stderr fallback
- Guard top-level numpy/cv2 imports in colorize.py and restore.py
with helpful error messages
- Add weights_only=False fallback for torch.load in noise_removal
- Fix integration tests to accept 501 for uninstalled AI features
and 422 for missing system tools (exiftool, libheif)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add tool-specific suffix to output filenames so downloads don't overwrite originals (batch & single-tool routes)
- Skip deleting shared models when uninstalling a bundle that shares models with another installed bundle
- Auto-detect NVIDIA GPU and swap GPU-only pip packages (onnxruntime-gpu, paddlepaddle-gpu) for CPU equivalents
- Refactor docker-compose with YAML anchors and explicit cpu/gpu profiles
- Add libheif-plugin-x265 to Dockerfile
- Fix install-all queue logic to handle concurrent individual installs and clear stale errors
- Unify playwright docker config to use same test dir with API_URL env var
- Fix flaky e2e selectors, rename Strip Metadata → Remove Metadata, handle collage custom dropzone, improve fallback test image generation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The on-demand feature download system stores models at /data/ai/models/
(set via MODELS_PATH env var), but all Python scripts hardcoded
/opt/models/ as the base path. Each script now reads MODELS_PATH and
falls back to /opt/models for backward compatibility.
Check installed.json before exec()-ing AI scripts so that requests
for uninstalled feature bundles return a structured error instead
of crashing with an ImportError. Also sets U2NET_HOME to the
bundled model directory when present.
Reads the feature manifest, installs pip packages (common + arch-specific),
downloads models in parallel with atomic rename, and writes installed.json.
Includes disk space pre-check, NCCL conflict handling, retry logic, and
progress reporting via stderr JSON lines. Also updates the feature route
to pass manifestPath and modelsDir as CLI arguments.
- Added model mismatch warnings in colorize, enhance-faces, and upscale routes.
- Improved error handling in colorize, enhance_faces, remove_bg, restore, and upscale scripts with detailed logging.
- Updated Dockerfile to align NCCL versions for compatibility.
- Introduced a new full tool audit script to test all tools for functionality and GPU usage.
- Created Playwright E2E tests for GPU-dependent tools to ensure proper functionality and performance.
- Replace [object Object] errors with readable messages across all 20+ API
routes by normalizing Zod validation errors to strings (formatZodErrors)
- Add parseApiError() on frontend to defensively handle any details type
- Add global Fastify error handler with full stack traces in logs
- Fix image-to-pdf auth: Object.entries(headers) → headers.forEach()
- Fix passport-photo: safeParse + formatZodErrors, safe error extraction
- Fix OCR silent fallbacks: log exception type/message when falling back,
include actual engine used in API response and Docker logs
- Fix split tool: process all uploaded images, combine into ZIP with
subfolders per image
- Fix batch support for blur-faces, strip-metadata, edit-metadata,
vectorize: add processAllFiles branch for multi-file uploads
- Docker: LOG_LEVEL=debug, PYTHONWARNINGS=default for visibility
- Add Playwright e2e tests verifying all fixes against Docker container
Set U2NET_HOME=/opt/models/rembg so rembg models pre-downloaded at
build time as root are found at runtime by the non-root ashim user.
Without this every fresh container re-downloaded the 973 MB BiRefNet
models on first background-removal request.
Apply the same fix to PaddleOCR: download to /opt/models/paddlex and
symlink into both /root/.paddlex and /app/.paddlex so PaddleX finds
models regardless of which HOME gosu resolves at runtime.
Fall back to per-request spawning in bridge.ts when the persistent
dispatcher crashes mid-request (e.g. OOM loading a large ONNX model),
so the operation succeeds instead of surfacing "Python dispatcher
exited unexpectedly" to the user.
Improve entrypoint.sh permission warning to mention Windows bind mounts
as the likely cause.
Run both short-range and full-range MediaPipe models and merge results,
then apply non-maximum suppression to remove duplicate bounding boxes.
Fixes missed faces in group photos where the single-model loop exited
early after the first positive detection.
Three fixes to ensure zero network access after docker pull:
1. rembg model allowlist: validate model parameter against the 7
pre-downloaded models, preventing rembg from attempting to download
unknown models via a raw API call.
2. GFPGAN/CodeFormer auxiliary models: pre-download facexlib's
detection_Resnet50_Final.pth and parsing_parsenet.pth at build time.
These were previously downloaded on first use via basicsr. Symlinks
in /app/gfpgan/weights/ ensure codeformer-pip also finds them.
3. OpenCV colorize models: pre-download the prototxt, caffemodel, and
pts_in_hull.npy so the lightweight OpenCV colorizer fallback works
in addition to the primary DDColor method.
Co-authored-by: stirling-image <stirling-image@users.noreply.github.com>
MediaPipe >= 0.10.30 removed the mp.solutions namespace. This broke
face blur, face enhance, red-eye removal, and photo restoration for
users running newer mediapipe versions (closes#43).
All 5 Python scripts that use mediapipe now try the legacy mp.solutions
API first and fall back to the new mp.tasks API on AttributeError.
Model files (blaze_face_short_range.task, face_landmarker.task) are
pre-downloaded during Docker build into /opt/models/mediapipe/ so the
image works fully airgapped. Local dev auto-downloads to .models/.
Co-authored-by: stirling-image <stirling-image@users.noreply.github.com>
- Old API (mp.solutions.face_mesh) for Docker with mediapipe < 0.10.30
- New API (mp.tasks.vision.FaceLandmarker) for newer mediapipe >= 0.10.30
- Auto-downloads face_landmarker.task model on first use with new API
- Extracted shared landmark index constants and key point extraction
* feat(passport-photo): add passport specs database and tool constants
* feat(passport-photo): add MediaPipe FaceMesh landmark detection script
* feat(passport-photo): add TypeScript bridge for face landmark detection
* feat(passport-photo): add API routes with analyze and generate endpoints
* fix(passport-photo): accept landmarks from request body and fix pixel coordinate conversion
- Generate endpoint now accepts landmarks + imageWidth/imageHeight in request body
instead of re-running AI face detection (makes generate phase instant)
- Fixed bug where normalized landmark coordinates (0-1) were used directly
as pixel values in crop computation - now properly multiplied by imgW/imgH
- Fixed same bug in pipeline process function
* feat(passport-photo): add UI component with live preview and compliance overlay
---------
Co-authored-by: stirling-image <stirling-image@users.noreply.github.com>
enhance_faces.py relied on implicit PyTorch auto-detection for both
GFPGAN and CodeFormer, bypassing the centralized gpu.py module.
inpaint.py queried ort.get_available_providers() directly, which
reports compiled-in backends rather than actual hardware.
Both tools now go through gpu.py so STIRLING_GPU=false correctly
forces CPU across every AI tool.
Co-authored-by: stirling-image <stirling-image@users.noreply.github.com>
Add comprehensive photo restoration tool that chains multiple AI models:
- Scratch/tear/spot detection via morphological analysis (top-hat/black-hat transforms)
- Damage inpainting via LaMa ONNX model (reuses existing infrastructure)
- Face enhancement via CodeFormer ONNX (~377MB, from facefusion/models-3.0.0)
- Noise reduction via OpenCV NLMeans in LAB color space
- Optional B&W auto-colorization via DDColor (reuses existing model)
Settings: 3 restoration modes (Light/Auto/Heavy), individual feature toggles
for scratch removal, face enhancement (with fidelity slider), denoising
(with strength slider), and auto-colorize. Before/after comparison view.
Handles HEIC, HEIF, and all standard formats. Batch processing supported.
No new Python dependencies - reuses onnxruntime, cv2, mediapipe, PIL.
Co-authored-by: stirling-image <stirling-image@users.noreply.github.com>
* feat(shared): add enhance-faces tool definition and i18n strings
* feat(ai): add face enhancement script with GFPGAN and CodeFormer support
Detects faces via MediaPipe dual-model approach, then enhances using
GFPGAN (proven) or CodeFormer (via codeformer-pip) with auto fallback.
Supports strength-based alpha blending with original image.
* feat(ai): add TypeScript bridge for face enhancement
* feat(api): add enhance-faces route with GFPGAN/CodeFormer support
* feat(web): add enhance-faces settings component and register in tool registry
* feat(docker): add CodeFormer dependency and model download
- Add codeformer-pip to both CPU and GPU requirements
- Download CodeFormer model (~375MB) at Docker build time
- Add CodeFormer to smoke test verification
* fix(enhance-faces): address code review findings
- Skip alpha blend for CodeFormer (strength already applied via fidelity weight)
- Hide "only enhance main face" checkbox when Best (CodeFormer) is selected
- Fix sensitivity slider labels (swap More/Fewer faces to match actual behavior)
- Register EnhanceFacesControls in pipeline step settings
- Remove model names from user-facing descriptions
* fix(enhance-faces): fix CodeFormer integration and Docker setup
- Add codeformer-pip install to Dockerfile with --no-deps to avoid numpy 2.x conflict
- Re-pin numpy==1.26.4 after codeformer-pip install
- Pin codeformer-pip==0.0.4 in requirements files
- Broaden auto-mode fallback to catch any Exception from CodeFormer
---------
Co-authored-by: stirling-image <stirling-image@users.noreply.github.com>
* feat(noise-removal): register tool in shared constants and i18n
* feat(noise-removal): add SCUNet and NAFNet model architectures
* feat(noise-removal): add Python denoising engine with 4 quality tiers
* feat(noise-removal): add TypeScript bridge for Python sidecar
* feat(noise-removal): add frontend settings with 4-tier selector
* feat(noise-removal): register in tool registry and pipeline
* feat(noise-removal): add Fastify API route with Zod validation
* feat(noise-removal): add SCUNet and NAFNet model downloads to Docker build
* test(noise-removal): add to e2e tool page rendering tests
* test(noise-removal): add integration tests for API endpoint
* style: fix biome formatting and import ordering
* fix(noise-removal): use correct model download URLs
NAFNet model is hosted on HuggingFace, not GitHub releases.
Also align SCUNet URL to use the KAIR releases (same as Docker build).
* fix(noise-removal): remove emojis from tier selector, simplify labels
Drop emoji icons from Quick/Balanced/Quality/Maximum buttons. Replace
technical algorithm names with plain descriptions users can understand.
---------
Co-authored-by: stirling-image <stirling-image@users.noreply.github.com>
Add AI-powered photo colorization that converts B&W/grayscale images to
full color using DDColor (ICCV 2023 dual-decoder architecture) via ONNX
Runtime. Includes model selection (Auto/DDColor/Classic), adjustable color
intensity, batch processing, before/after preview, and full HEIC/HEIF support.
Co-authored-by: stirling-image <stirling-image@users.noreply.github.com>
Update erase-object pipeline, eraser canvas, and inpainting Python script.
Add LaMa model download script and update Dockerfile for model support.
Update multi-file tool routes for consistency.
Replace the confusing 2-mode smart crop with a clear 3-mode system:
- Subject Focus: Sharp attention/entropy saliency crop with social media presets
- Face Focus: MediaPipe face detection with headshot framing presets
- Auto Trim: Border removal with optional pad-to-square
Adds detectFaces() to AI package, face preset constants, backward
compatibility for old mode names, and comprehensive integration tests.
- Pin PaddlePaddle to 3.0.0 on ARM64 to fix segfault in PIR inference
engine (3.1+ crashes on aarch64 Debian Bookworm)
- Fix text extraction for PaddleOCR 3.4.x result format (rec_texts)
- Add Node.js-level fallback chain (best -> balanced -> fast) when
Python subprocess crashes
- Add multi-image OCR: processes all uploaded files sequentially with
per-file progress and filename headers in combined output
- Convert input images to PNG via Sharp before OCR so HEIC, AVIF, WebP,
TIFF all work transparently
- Implement real auto-detect language using Tesseract multi-lang script
detection (analyzes Unicode ranges for Hangul, CJK, Kana, Latin)
- Default enhance to off (hurts clean digital images)
- Replace Auto/AI/Fast buttons with Fast/Balanced/Best (consistent with other tools)
- Rename "Denoise" to "Noise Reduction" with explanatory subtitle
- Change output format from 3 buttons to dropdown with all formats (PNG, JPG, WebP, AVIF, TIFF, GIF, HEIC, HEIF)
- Add HEIC/HEIF input decoding (was missing unlike other tools)
- Add HEIC/HEIF/AVIF output conversion via Sharp and heif-enc
- Generate browser-compatible WebP preview for non-previewable output formats
- Fix torchvision compatibility shim so Real-ESRGAN actually loads (was silently falling back to Lanczos)
- Fix denoise crash: Image.fromarray() instead of type(img).fromarray()
- Redirect stdout for entire AI pipeline to prevent library messages corrupting JSON output
- Add GFPGAN model download for face enhancement
- Use batch endpoint for multi-file uploads (enables Download All ZIP)
- Fix multi-image: process selected file, not always first
- Fix progress bar: asymptotic fill prevents visual stalling
- Fix slider: write results to captured index, not current selection
- Add model selection (Auto/AI/Fast), face enhancement, denoise
- Add output format (PNG/JPEG/WebP) with quality control
- Add Upscale All for sequential batch processing with queue
- More granular Python progress stages for smoother UX
- Fix temp file leak: clean up preprocessed image in finally block
- Log warning instead of silently swallowing preprocessing failures
- Simplify auto_detect_language to honest default (was a stub that
wasted time loading a model but always returned "en")
Ultra quality (People only):
- BiRefNet-matting ONNX (928MB) for true alpha matting with per-pixel
transparency on hair wisps and fine edges
- Custom rembg session class, zero new Python dependencies
- Model pre-downloaded in Docker build for immediate availability
Quality tier labels: Fast / HD / Max / Ultra (shorter, fits 4-col grid)
ashim-hq