fix: make OCR portable and reliable across AMD64 and ARM64 (#519)

* fix: make OCR portable and reliable * fix: harden OCR installation portability * fix: pin OCR partials across downloads * fix: make OCR execution reliably asynchronous * fix: harden OCR portability and docs routes * fix: preserve decoder and docs safeguards
2026-08-03 07:46:42 +02:00 · 2026-07-15 03:34:24 +08:00
parent 58121f205f
commit 991c981529
409 changed files with 67151 additions and 8076 deletions
@@ -33,13 +33,13 @@ This package has no network dependencies and runs entirely in-process.

 ### `@snapotter/ai` {#snapotter-ai}

-A bridge layer that calls Python scripts for ML operations. On first use, the bridge starts a persistent Python dispatcher process that pre-imports heavy libraries (PIL, NumPy, MediaPipe, rembg) so subsequent AI calls skip the import overhead. If the dispatcher is not yet ready, the bridge falls back to spawning a fresh Python subprocess per request.
+A bridge layer that calls native and Python ML runtimes. Most Python tools use a persistent dispatcher that pre-imports heavy libraries (PIL, NumPy, MediaPipe, rembg) so subsequent calls skip the import overhead. OCR is isolated from that mutable shared environment: `fast` invokes native Tesseract, while `balanced` and `best` use a dedicated persistent JSONL dispatcher pinned to the active immutable RapidOCR/ONNX generation. Each request holds a generation lease. Activation first runs a smoke test on a candidate, then atomically switches to its dispatcher. The prior dispatcher drains before its generation is garbage-collected.

 **Models are not pre-loaded.** Each tool script loads its model weights from disk at request time and discards them when the request finishes. See [Resource footprint](#resource-footprint) for the full memory profile.

-Supported operations: background removal (rembg/BiRefNet), upscaling (RealESRGAN), face blur (MediaPipe), face enhancement (GFPGAN/CodeFormer), object erasing (LaMa ONNX), OCR (PaddleOCR/Tesseract), colorization (DDColor), noise removal, red eye removal, photo restoration, passport photo generation, transparency fixing (BiRefNet HR-matting), and content-aware resize (Go caire binary).
+Supported operations: background removal (rembg/BiRefNet), upscaling (RealESRGAN), face blur (MediaPipe), face enhancement (GFPGAN/CodeFormer), object erasing (LaMa ONNX), OCR (Tesseract and RapidOCR with PP-OCR ONNX models), colorization (DDColor), noise removal, red eye removal, photo restoration, passport photo generation, transparency fixing (BiRefNet HR-matting), and content-aware resize (Go caire binary).

-Python scripts live in `packages/ai/python/`. The Docker image pre-downloads all model weights during the build so the container works fully offline.
+Python scripts live in `packages/ai/python/`. Large optional model packs are installed on demand into the persistent `/data/ai` volume. Accurate OCR uses signed, platform-specific artifacts; the built-in Tesseract tier requires no model-pack download.

 ### `@snapotter/shared` {#snapotter-shared}

@@ -84,7 +84,7 @@ This VitePress site. Deployed to Cloudflare Pages automatically on push to `main
 2. The frontend sends a multipart POST to `/api/v1/tools/:section/:toolId` with the file and settings.
 3. The API route validates the input with Zod, then dispatches processing.
 4. For standard tools, the job is enqueued to the appropriate BullMQ pool (image, media, or docs based on modality). The in-process BullMQ worker auto-orients the image based on EXIF metadata, runs the tool's process function, and returns the result.
-5. For AI tools, the TypeScript bridge sends a request to the persistent Python dispatcher (or spawns a fresh subprocess as fallback), waits for it to finish, and reads the output file.
+5. For most AI tools, the TypeScript bridge sends a request to the persistent Python dispatcher. Fast OCR instead invokes Tesseract, and accurate OCR starts the pinned executable from the active immutable OCR generation. The requested OCR tier is fixed at ingress and is never silently changed during execution.
 6. Job progress is persisted to the `jobs` table in PostgreSQL so state survives container restarts. Real-time updates are delivered via SSE at `/api/v1/jobs/:jobId/progress`.
 7. The API returns a `jobId` and `downloadUrl`. The user downloads the processed file from `/api/v1/download/:jobId/:filename`.

@@ -110,7 +110,7 @@ The app is then available at `http://localhost:1349`.

 ## Quick Start (NVIDIA CUDA) {#quick-start-nvidia-cuda}

-For NVIDIA CUDA acceleration on AI tools (background removal, upscaling, face enhancement, OCR):
+For NVIDIA CUDA acceleration on supported AI tools (background removal, upscaling, face enhancement):

 ```yaml
 # docker-compose-gpu.yml - Requires: NVIDIA GPU + nvidia-container-toolkit
@@ -248,10 +248,10 @@ deploy:
 |---|---|
 | CPU | 4 cores |
 | RAM | 4 GB |
-| Disk | 3 GB (image) + 24 GB (AI models) + workspace |
+| Disk | 3 GB (image) + about 20 GB (all optional AI packs) + workspace |
 | GPU | Not required (CPU fallback) |

-**Installing the AI bundles is what pushes RAM to 4 GB.** With no AI installed the app idles around 360 MB; with all seven bundles installed it holds ~2.6 GB resident, because the Python AI sidecar pre-loads its models (background removal, upscaling, OCR, transcription, face detection, restoration) at startup. Non-AI installs stay light; AI installs need ≥4 GB.
+**Installing and running the larger AI bundles is what pushes the recommendation to 4 GB of RAM.** With no optional packs installed the app idles around 360 MB. Legacy Python tools share a sidecar, while accurate OCR uses a dedicated long-lived dispatcher pinned to the active immutable generation. Before activation, the installer runs a smoke test on the candidate. It then atomically switches to the new dispatcher and drains the prior dispatcher before garbage collection. Every official accurate-OCR artifact must pass its worst-case release suite inside a 4 GiB cgroup, while the 4 GB host recommendation leaves headroom for the Node.js application, Postgres, Redis, queues, and concurrent work.

 Most AI tools are perfectly usable on CPU; a couple really want a GPU. Measured on a modern 4-core CPU:

@@ -268,7 +268,7 @@ SnapOtter intentionally does not bake these model downloads into the Docker imag

 Some tools depend on more than one shared bundle. For example, Passport Photo needs both `background-removal` and `face-detection`; if `background-removal` is already installed, enabling Passport Photo only downloads the missing `face-detection` bundle. The same reuse applies across all AI tools.

-AI model download sizes:
+Optional AI pack storage estimates:

 | Bundle | Disk Size |
 |---|---|
@@ -276,9 +276,16 @@ AI model download sizes:
 | Upscale + Face enhance + Noise removal | 5-6 GB |
 | Face detection | 200-300 MB |
 | Object eraser + Colorize | 1-2 GB |
-| OCR | 5-6 GB |
+| Accurate OCR (`balanced`/`best`) | ~208-234 MiB download / ~409-488 MiB installed |
 | Photo restoration | 4-5 GB |
-| **All bundles** | **~24 GB** |
+| Transcription | ~600 MB |
+| **All bundles** | **~20 GB installed** |
+
+Fast OCR is built into the image through Tesseract, adds about 25 MiB, and does not require the optional OCR pack or its 4 GiB memory requirement. Fast supports `auto`, `en`, `de`, `es`, `fr`, `zh`, and `ja`, but not Korean (`ko`). Korean uses `balanced` or `best` and therefore requires the accurate pack. The accurate pack is available in the official Linux amd64 and arm64 containers and runs ONNX Runtime on CPU. NVIDIA hosts use that same CPU OCR runtime, so OCR does not depend on the CUDA version or GPU architecture. The accurate runtime requires at least 4 GiB of effective memory: the configured container cgroup limit, otherwise host memory. SnapOtter rejects systems below that signed compatibility minimum before downloading the pack. Accurate-pack installation is also rejected on bare-metal/prebuilt archives whose libc and Python ABI cannot be guaranteed. On those unsupported hosts, Korean OCR returns an explicit incompatibility error and never silently falls back to Fast.
+
+Replicas that share the same `DATA_DIR` must use the same CPU architecture; pin multi-replica deployments with node affinity. Mixed amd64/arm64 replicas need separate data volumes and independent SnapOtter deployments.
+
+The accurate runtime keeps one active generation and purges its download cache after activation. For this release, a first install temporarily needs roughly 620-720 MiB for the archive plus staging, and an upgrade can peak near 1.2 GiB while the old generation remains active. The installer computes the exact requirement from the signed index and current generations before downloading or extracting, and fails early if the data volume is too small.

 ```yaml
 deploy:
@@ -350,7 +357,6 @@ See the [complete format list](/guide/supported-formats) for details on every su

 - **Content-aware resize** crashes on large images (>5 MP) due to a limitation in the caire binary. Works fine with smaller images.
 - **HEIF decode** takes 13-23 seconds. HEIC (Apple's variant) is much faster at 0.3-0.9 seconds.
- **OCR Japanese** fails on CPU due to a PaddlePaddle MKLDNN bug. Works on GPU.
 - **Upscale** times out on CPU for anything beyond small images. GPU required for practical use.
 - **CodeFormer** face enhancement is significantly slower than GFPGAN (53s vs 2s on GPU). GFPGAN is recommended for most use cases.

@@ -431,6 +437,26 @@ The startup error names the exact UID to use, so the quickest path is to start t
 | `SESSION_DURATION_HOURS` | `168` | Login session lifetime (7 days) |
 | `CORS_ORIGIN` | (empty) | Comma-separated allowed origins, or empty for same-origin |

+### Outbound proxy and private CA {#outbound-proxy-and-private-ca}
+
+The official container enables Node's environment-proxy support. If SnapOtter must reach the OCR runtime repository or other HTTPS services through a corporate proxy, set `HTTPS_PROXY` (and `HTTP_PROXY` when needed). Set `NO_PROXY` to a comma-separated list of hosts that must be reached directly, such as Postgres, Redis, and internal object storage.
+
+If the proxy or an internal service is signed by a private certificate authority, mount the CA certificate read-only and point `NODE_EXTRA_CA_CERTS` to it. The file must exist when the Node process starts:
+
+```yaml
+services:
+  app:
+    environment:
+      HTTPS_PROXY: http://proxy.example.internal:3128
+      HTTP_PROXY: http://proxy.example.internal:3128
+      NO_PROXY: postgres,redis,minio,localhost,127.0.0.1
+      NODE_EXTRA_CA_CERTS: /etc/snapotter/custom-ca.pem
+    volumes:
+      - ./company-ca.pem:/etc/snapotter/custom-ca.pem:ro
+```
+
+Keep the proxy credentials outside the Compose file (for example in a protected `.env` file or secret). Do not disable TLS verification: the signed OCR index authenticates release metadata, while normal TLS validation still protects transport and every other outbound request.
+
 ## Health Check {#health-check}

 The container includes a built-in health check:
@@ -38,7 +38,6 @@ Tested on an NVIDIA RTX 4070 (12 GB VRAM) with a 572x1024 JPEG portrait.
 | Background removal (isnet) | 2,457ms | 1,137ms | 2.2x |
 | Upscale 2x | 350ms | 309ms | 1.1x |
 | Upscale 4x | 910ms | 310ms | 2.9x |
-| OCR (PaddleOCR) | 137ms | 94ms | 1.5x |
 | Face blur | 139ms | 122ms | 1.1x |

 #### Cold start (first request after container start) {#cold-start-first-request-after-container-start}
@@ -47,7 +46,8 @@ Tested on an NVIDIA RTX 4070 (12 GB VRAM) with a 572x1024 JPEG portrait.
 |------|-----|-----|---------|
 | Background removal | 22,286ms | 4,792ms | 4.7x |
 | Upscale 2x | 3,957ms | 2,318ms | 1.7x |
-| OCR (PaddleOCR) | 1,469ms | 1,090ms | 1.3x |
+
+OCR is not included in the CUDA comparison. Both the built-in Tesseract tier and the optional RapidOCR/ONNX tiers use CPU, including when the container has NVIDIA GPU access.

 ### CUDA health check {#cuda-health-check}

@@ -29,7 +29,7 @@ For details about what is collected, see [What SnapOtter collects](/guide/teleme
 :::

 ::: tip NVIDIA CUDA acceleration
-Add `--gpus all` for NVIDIA CUDA-accelerated background removal, upscaling, OCR, face enhancement, and restoration:
+Add `--gpus all` for NVIDIA CUDA-accelerated background removal, upscaling, face enhancement, and restoration. OCR remains CPU-based and works in the same image with or without GPU access:

 ```bash
 docker run -d --name SnapOtter -p 1349:1349 --gpus all -v SnapOtter-data:/data snapotter/snapotter:latest