RealESRGAN's enhance() and rembg's remove() run in one opaque call, so the
progress bar froze at 30% for the whole inference. Add a time-based
heartbeat that advances the bar in a background thread while the model runs
and stops when it returns, so the bar moves instead of freezing. Verified
end to end on a GPU box (forced CPU): a 55s upscale emitted 26 steady ticks
then completed; background removal too.
Fixes#591
* fix: remove all automatic third-party egress (OSM tiles, Scalar fonts, editor Google Fonts, AI model download fallbacks)
Phone-home audit follow-up. The product no longer makes any automatic
third-party request; user-initiated click-outs stay, and production now
fails closed on missing AI models.
1. GPS leak via OSM tiles: the strip-metadata panel auto-loaded
tile.openstreetmap.org tiles encoding the photo's GPS position. The
Leaflet mini-map is gone; coordinates render as text plus an explicit
View on map link (openstreetmap.org, opens on click only). Removed
tile.openstreetmap.org from the CSP img-src, dropped the leaflet
dependency, added the viewOnMap i18n key to all 21 locales.
2. Scalar docs fonts: /api/docs loaded Inter and JetBrains Mono from
fonts.scalar.com. Scalar now renders with withDefaultFonts: false and
both --scalar-font and --scalar-font-code pinned to system stacks;
fonts.scalar.com removed from the docs CSP font-src. Verified by
injecting GET /api/docs/: config carries withDefaultFonts false and
the served page has no fonts.scalar.com reference.
3. Editor Google Fonts: the editor font picker built
fonts.googleapis.com stylesheet URLs for 25 web fonts the served CSP
already blocked. The remote loading path is deleted; the picker now
offers system fonts only, with a SELF_HOSTED_FONTS seam (FontFace API,
same origin) for bundling fonts later. Unknown families saved in old
documents fall back to the browser default.
4. Python sidecar fails closed on model downloads: new
packages/ai/python/offline_guard.py gates every runtime download
fallback (inpaint, outpaint, restore, noise_removal, detect_faces,
enhance_faces, face_landmarks, red_eye_removal, remove_bg, ocr,
transcribe, upscale) behind SNAPOTTER_ALLOW_MODEL_DOWNLOAD=1 with an
actionable error. Bundled models keep working untouched.
5. OCR and transcription library-internal downloads: unbundled PaddleOCR
language and detection fallbacks now raise the guard error naming the
language instead of resolving models over the network; faster-whisper
gets local_files_only when downloads are off.
6. GFPGAN and CodeFormer cwd-relative weights: facexlib and
codeformer-pip resolve helper weights relative to the process cwd and
fetch them from GitHub when absent. They are now symlinked from the
installed bundle files under MODELS_PATH/gfpgan/facelib before the
libraries load, failing closed when unresolvable.
Defense in depth: HF_HUB_OFFLINE=1 and TRANSFORMERS_OFFLINE=1 are set in
the runtime image and in the sidecar spawn env; install_feature.py lifts
them for user-initiated bundle installs and restores them afterwards
(it can run in-process inside the dispatcher). SNAPOTTER_ALLOW_MODEL_DOWNLOAD
is documented in .env.example, default off.
Validation: typecheck 9/9 workspaces, Biome clean on touched files,
5178 unit tests pass, py_compile on all touched scripts, guard behavior
exercised in both dispatcher exec and per-request import modes, zero
remaining runtime references to the three hosts. Docker build and live
AI inference need post-merge verification on the GPU host.
Claude-Session: https://claude.ai/code/session_01XGB4pGvTvb7sUX4JN745U7
* fix: allow AI model downloads by default, make strict offline mode opt-in
Product call: ease of use first. The download gating from the previous
commit inverts its default: runtime model fetches (public model weights
only, never user data) are allowed out of the box so AI tools self-heal,
and SNAPOTTER_ALLOW_MODEL_DOWNLOAD=0 becomes the explicit strict offline
mode for airgapped deployments, where every fallback raises the
actionable error instead of fetching.
Changes: offline_guard blocks only on an explicit 0/false; the
unconditional HF_HUB_OFFLINE/TRANSFORMERS_OFFLINE image ENV is removed
and bridge.ts sets those flags for the sidecar only in strict mode;
.env.example documents the new default; install_feature's lift/restore
stays. All bundled-path preferences, pre-existence checks, and symlink
pre-placement remain, so installed bundles never trigger a download.
The OSM, Scalar font, and editor font fixes are unchanged.
Validation rerun: typecheck 9/9, Biome clean on touched files, 5178
unit tests pass, py_compile on touched scripts, guard behavior verified
for unset/1 (allowed) and 0/false (blocked with the new message).
Claude-Session: https://claude.ai/code/session_01XGB4pGvTvb7sUX4JN745U7
- Expose birefnet-hr-matting in UI (People/Ultra) and fix model defaults
(People/Max now uses birefnet-matting for true alpha matting)
- Add output format selector (PNG/WebP/AVIF) with lossless alpha support
- Add edge smoothing post-processing (Off/Light/Medium/Strong) via
morphological mask refinement to reduce gray halo artifacts
- Add color decontamination to remove background color spill from
semi-transparent edge pixels
- Thread new settings through full stack: frontend -> API schema ->
Python sidecar -> Sharp effects pipeline
- Add i18n keys for all 21 locales
- Add unit tests for new option serialization (3 tests)
- Add integration tests for new settings validation (4 tests)
The BiRefNetHRMattingSession.predict normalization crashes when all
pixels share the same value (ma == mi). Use a guarded denominator so
uniform-alpha inputs produce a zero mask instead of a NaN explosion.
Also adds _register_birefnet_hr_matting() to install_feature.py so the
HR-matting model can be downloaded during feature installation, matching
the existing registration in remove_bg.py.
Skip alpha matting on CPU (pymatting's sparse matrices are the main
memory hog), auto-downscale images above 2048px before sending to
rembg, and retry with the lighter u2net model when OOM is detected.
- Add safe_onnx_session() to gpu.py with graceful CUDA EP → CPU fallback
- Replace bare ort.InferenceSession() calls across colorize, restore, inpaint, remove_bg
- Add libcublas-12-6 to production Dockerfile for ONNX Runtime CUDA EP
- Add skipIfFeatureNotInstalled guards to remove-bg, blur-faces, smart-crop, ocr, noise-removal e2e specs
- Add AI tool install prompt detection in tools-all.spec.ts
- Add smart-crop to PYTHON_SIDECAR_TOOLS so frontend shows install prompt correctly
- Create Dockerfile.test.dockerignore to include tests/ in test image builds
- Add libheif-examples and exiftool to Dockerfile.test for HEIC and metadata tests
- Regenerate visual regression baselines for Docker/Linux and skip on non-Docker platforms
- Added model mismatch warnings in colorize, enhance-faces, and upscale routes.
- Improved error handling in colorize, enhance_faces, remove_bg, restore, and upscale scripts with detailed logging.
- Updated Dockerfile to align NCCL versions for compatibility.
- Introduced a new full tool audit script to test all tools for functionality and GPU usage.
- Created Playwright E2E tests for GPU-dependent tools to ensure proper functionality and performance.
Three fixes to ensure zero network access after docker pull:
1. rembg model allowlist: validate model parameter against the 7
pre-downloaded models, preventing rembg from attempting to download
unknown models via a raw API call.
2. GFPGAN/CodeFormer auxiliary models: pre-download facexlib's
detection_Resnet50_Final.pth and parsing_parsenet.pth at build time.
These were previously downloaded on first use via basicsr. Symlinks
in /app/gfpgan/weights/ ensure codeformer-pip also finds them.
3. OpenCV colorize models: pre-download the prototxt, caffemodel, and
pts_in_hull.npy so the lightweight OpenCV colorizer fallback works
in addition to the primary DDColor method.
Co-authored-by: stirling-image <stirling-image@users.noreply.github.com>
Ultra quality (People only):
- BiRefNet-matting ONNX (928MB) for true alpha matting with per-pixel
transparency on hair wisps and fine edges
- Custom rembg session class, zero new Python dependencies
- Model pre-downloaded in Docker build for immediate availability
Quality tier labels: Fast / HD / Max / Ultra (shorter, fits 4-col grid)
Remove Background:
- Two-phase flow: AI removes bg once, then effects adjust instantly
- Blur background effect with real-time CSS preview (portrait mode)
- Drop shadow effect with opacity control
- Gradient backgrounds with presets, custom colors, and angle
- Custom background image upload (including HEIC/HEIF)
- Solid color backgrounds moved from Python to Node.js/Sharp
- Effects-only API endpoint for instant re-renders without AI re-run
- HEIC/HEIF input support (decoded before passing to Python/rembg)
- Passport/ID photo checkbox defaults ON for People subject
- Before/after slider preserved when no effects active
- 15 comprehensive Playwright e2e tests
Color Tools:
- Consolidated 4 tools (brightness-contrast, saturation, color-channels,
color-effects) into single "Adjust Colors" tool
- Added exposure, temperature, tint, hue, sharpness controls
- SVG filter-based live preview for all adjustments
- Backward-compatible URL redirects from old tool paths
Other fixes:
- Favicon tool: download button instead of auto-download
- Batch processing: HEIC filename extension fix
- File store: processedFilename field for proper batch downloads
- Replace OpenCV Haar Cascades with MediaPipe for face detection, using
short-range model first with full-range fallback for better accuracy
- Add auto-orient to remove-background route for EXIF-rotated photos
- Change default background removal model from u2net to birefnet-general-lite
- Fix flaky test by setting SQLite busy_timeout before journal_mode pragma
Co-authored-by: Siddharth Kumar Sah <siddharth123sk@gmail.com>
Add a :cuda Docker image tag that auto-detects NVIDIA GPU at runtime
and falls back gracefully to CPU. Same pattern as Immich.
- New gpu.py shared utility for cached CUDA detection
- Background removal (rembg): pass CUDAExecutionProvider to ONNX Runtime
- Upscaling (Real-ESRGAN): use CUDA device + FP16 when GPU available
- OCR (PaddleOCR): enable use_gpu when CUDA detected
- Dispatcher reports GPU status at startup via readiness signal
- Admin health endpoint exposes GPU availability
- Dockerfile uses ARG GPU=false with conditional NVIDIA CUDA base image
- docker-compose.gpu.yml override for GPU users
- CI/CD workflows build and publish :cuda tag (amd64 only)
Three tags: :latest (CPU), :lite (no AI), :cuda (GPU with CPU fallback)
- Set up semantic-release with zero-touch CI pipeline on push to main
- Add version sync script to keep all package.json files and APP_VERSION
constant in sync automatically
- Consolidate Docker publishing into single tag-triggered workflow that
pushes to both Docker Hub and ghcr.io with semver tags
- Add help dialog with keyboard shortcuts, getting started guide, and
resource links
- Sync all versions to 0.2.1 to match Docker Hub latest
BiRefNet-Lite times out on first load (~60s+ for 973MB model).
U2-Net works in 2 seconds. Users can still select BiRefNet for
higher quality when they're willing to wait. Added timing hints
in model descriptions and increased timeout for BiRefNet models.
- Switch default from birefnet-general (973MB, 4min) to
birefnet-general-lite (faster, still SOTA quality)
- Fix Python script stdout pollution — progress messages now go to
stderr so the JSON result parser doesn't break
- Pre-download birefnet-general-lite in Docker build
- Switch default model from U2-Net to BiRefNet (state-of-the-art)
- Add 6 model options: BiRefNet, BiRefNet Lite, BiRefNet Portrait,
BRIA RMBG, IS-Net, U2-Net
- Add animated progress bar with stage indicators (loading model,
analyzing, removing, refining edges) and elapsed timer
- Add intuitive background color presets (Transparent, White, Black,
Red, Green, Blue) as clickable buttons + custom color picker
- Handle background color compositing in Python (PIL alpha composite)
- Add checkerboard pattern to before/after slider for transparency
- Pre-bake BiRefNet model (973MB) in Docker image for instant use
- Add user management endpoints (register, list, delete, change password)
- Add API key management (create, list, delete)
- Add settings persistence endpoints (get, put)
- Wire settings dialog to real backend (People, API Keys, System, Security)
- Fix login auth flow (window.location.href for full reload)
- Fix download URLs returning 401 (make public since UUIDs are unguessable)
- Fix border tool shadowColor validation (accept 6-8 hex digits)
- Fix remove-bg alpha matting fallback (retry without on failure)
- Fix AI tool silent fallbacks (report errors instead of no-ops)
- Add checkerboard background to before/after slider for transparency
- Add progress bars to all AI tool components
- Add Playwright E2E test suite (131 tests across 9 test files)
- Rewrite Dockerfile for production (tsx runtime, pre-baked AI models)
- Add .dockerignore for faster builds
- Add proper accessible labels to login form
