External/SnapOtter
1
0
Fork 0
mirror of https://github.com/snapotter-hq/SnapOtter.git synced 2026-08-03 07:46:42 +02:00
Code Issues Packages Projects Releases Wiki Activity

docs: clarify nvidia cuda ai support (#378)

Browse Source
This commit is contained in:
SnapOtter
2026-06-29 21:29:00 +08:00
committed by GitHub
parent 1d99acf9ee
commit ef342c268b
10 changed files with 44 additions and 38 deletions
Download Patch File Download Diff File Expand all files Collapse all files
DOCKERHUB.md
+4 -4
View File
@@ -79,10 +79,10 @@ You will be asked to change your password on first login. Change `DEFAULT_PASSWO
| Architecture | GPU support | Notes | | Architecture | GPU support | Notes |
|--------------|-------------|-------| |--------------|-------------|-------|
| `linux/amd64` | NVIDIA CUDA | Full GPU acceleration for AI tools | | `linux/amd64` | NVIDIA CUDA | Full CUDA acceleration for AI tools |
| `linux/arm64` | CPU only | Raspberry Pi 4/5, Apple Silicon via Docker Desktop | | `linux/arm64` | CPU only | Raspberry Pi 4/5, Apple Silicon via Docker Desktop |
The same image runs on CPU or GPU. See [Docker Tags](https://docs.snapotter.com/guide/docker-tags) for benchmarks and version-pinning details. The same image runs on CPU or NVIDIA CUDA. Intel/AMD iGPU acceleration through VA-API, Quick Sync, or OpenCL is not supported for AI inference today; those systems run AI tools on CPU. See [Docker Tags](https://docs.snapotter.com/guide/docker-tags) for benchmarks and version-pinning details.
## Features ## Features
@@ -139,7 +139,7 @@ PostgreSQL and Redis keep their own volumes (`snapotter-pgdata`, `snapotter-redi
|------|---------| |------|---------|
| `1349` | Web UI and REST API | | `1349` | Web UI and REST API |
## GPU acceleration ## NVIDIA CUDA acceleration
The `amd64` image bundles CUDA. With an NVIDIA GPU and the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) installed, add this to the `snapotter` service to accelerate background removal, upscaling, OCR, and transcription: The `amd64` image bundles CUDA. With an NVIDIA GPU and the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) installed, add this to the `snapotter` service to accelerate background removal, upscaling, OCR, and transcription:
@@ -153,7 +153,7 @@ The `amd64` image bundles CUDA. With an NVIDIA GPU and the [NVIDIA Container Too
capabilities: [gpu] capabilities: [gpu]
``` ```
The image auto-detects the GPU at runtime and falls back to CPU when none is present. Benchmarks are in [Docker Tags](https://docs.snapotter.com/guide/docker-tags). The image auto-detects NVIDIA CUDA at runtime and falls back to CPU when CUDA is unavailable. Mapping `/dev/dri` for Intel or AMD GPUs does not accelerate SnapOtter AI tools today. Benchmarks are in [Docker Tags](https://docs.snapotter.com/guide/docker-tags).
## Upgrading from SnapOtter 1.x ## Upgrading from SnapOtter 1.x
README.md
+3 -3
View File
@@ -92,10 +92,10 @@ docker compose up -d
``` ```
<details> <details>
<summary><sub>Have an NVIDIA GPU? Click here for GPU acceleration.</sub></summary> <summary><sub>Have an NVIDIA GPU? Click here for CUDA acceleration.</sub></summary>
<br> <br>
Use the GPU Compose file for GPU-accelerated background removal, upscaling, transcription, and OCR. See [Docker Tags](https://docs.snapotter.com/guide/docker-tags) for the GPU Compose example and benchmarks. Use the GPU Compose file for NVIDIA CUDA-accelerated background removal, upscaling, transcription, and OCR. Intel/AMD iGPU acceleration through VA-API, Quick Sync, or OpenCL is not supported for AI inference today; those systems run AI tools on CPU. See [Docker Tags](https://docs.snapotter.com/guide/docker-tags) for the GPU Compose example and benchmarks.
</details> </details>
@@ -108,7 +108,7 @@ Use the GPU Compose file for GPU-accelerated background removal, upscaling, tran
You will be asked to change your password on first login. You will be asked to change your password on first login.
For Docker Compose, persistent storage, and other setup options, see the [Getting Started Guide](https://docs.snapotter.com/guide/getting-started). For GPU acceleration and tag details, see [Docker Tags](https://docs.snapotter.com/guide/docker-tags). For Docker Compose, persistent storage, and other setup options, see the [Getting Started Guide](https://docs.snapotter.com/guide/getting-started). For NVIDIA CUDA acceleration and tag details, see [Docker Tags](https://docs.snapotter.com/guide/docker-tags).
## Documentation ## Documentation
apps/docs/api/ai.md
+4 -2
View File
@@ -4,7 +4,9 @@ description: AI engine reference with all local ML tools. Background removal, up
# AI Engine Reference # AI Engine Reference
The `@snapotter/ai` package bridges Node.js to a **persistent Python sidecar** for all ML operations. The dispatcher process stays alive between requests for fast warm-start performance. GPU is auto-detected at startup and used when available. The `@snapotter/ai` package bridges Node.js to a **persistent Python sidecar** for all ML operations. The dispatcher process stays alive between requests for fast warm-start performance. NVIDIA CUDA is auto-detected at startup and used when available; otherwise AI tools run on CPU.
Intel/AMD iGPU acceleration through VA-API, Quick Sync, or OpenCL is not supported for AI inference today. Mapping `/dev/dri` into a container does not accelerate these Python sidecar tools unless a CUDA-capable NVIDIA GPU is available.
19 Python sidecar AI tools across four modalities (image, audio, video, document), plus 2 tools with optional AI capabilities. All models run locally - no internet required after initial model download. 19 Python sidecar AI tools across four modalities (image, audio, video, document), plus 2 tools with optional AI capabilities. All models run locally - no internet required after initial model download.
@@ -278,7 +280,7 @@ The mask is sent as a **second file part** (fieldname `mask`), not as base64. Wh
| `format` | string | `"auto"` | Output format: `auto`, `png`, `jpg`, `jpeg`, `webp`, `tiff`, `gif`, `avif`, `heic`, `heif`, `jxl` | | `format` | string | `"auto"` | Output format: `auto`, `png`, `jpg`, `jpeg`, `webp`, `tiff`, `gif`, `avif`, `heic`, `heif`, `jxl` |
| `quality` | integer (1-100) | `95` | Output quality | | `quality` | integer (1-100) | `95` | Output quality |
GPU-accelerated when an NVIDIA GPU is available. CUDA-accelerated when an NVIDIA GPU is available.
## AI Canvas Expand ## AI Canvas Expand
apps/docs/api/rest.md
+1 -1
View File
@@ -145,7 +145,7 @@ curl -X POST http://localhost:1349/api/v1/tools/<section>/<toolId>/batch \
### AI Tools ### AI Tools
All AI tools run on your hardware (CPU or NVIDIA GPU). No internet required. All AI tools run on your hardware: CPU by default, or NVIDIA CUDA when a supported NVIDIA GPU is available. Intel/AMD iGPU acceleration through VA-API, Quick Sync, or OpenCL is not supported for AI inference today. No internet required.
| Tool ID | Name | AI Model | Key settings | | Tool ID | Name | AI Model | Key settings |
|---------|------|---------|-------------| |---------|------|---------|-------------|
apps/docs/guide/deployment.md
+11 -9
View File
@@ -4,7 +4,7 @@ description: Deploy SnapOtter to production with Docker. Hardware requirements,
# Deployment # Deployment
SnapOtter deploys as a 3-container Docker Compose stack: the SnapOtter app image, PostgreSQL 17, and Redis 8. The app image supports **linux/amd64** (with NVIDIA CUDA) and **linux/arm64** (CPU), so it runs natively on Intel/AMD servers, Apple Silicon Macs, and ARM devices like the Raspberry Pi 4/5. SnapOtter deploys as a 3-container Docker Compose stack: the SnapOtter app image, PostgreSQL 17, and Redis 8. The app image supports **linux/amd64** (with NVIDIA CUDA for AI acceleration) and **linux/arm64** (CPU), so it runs natively on Intel/AMD servers, Apple Silicon Macs, and ARM devices like the Raspberry Pi 4/5. Intel/AMD iGPU acceleration through VA-API, Quick Sync, or OpenCL is not supported for AI inference today.
See [Docker Image](./docker-tags) for GPU setup, Docker Compose examples, and version pinning. See [Docker Image](./docker-tags) for GPU setup, Docker Compose examples, and version pinning.
@@ -108,9 +108,9 @@ The app is then available at `http://localhost:1349`.
> **Docker Hub rate limits?** Replace `snapotter/snapotter:latest` with `ghcr.io/snapotter-hq/snapotter:latest` to pull from GitHub Container Registry instead. Both registries receive the same image on every release. > **Docker Hub rate limits?** Replace `snapotter/snapotter:latest` with `ghcr.io/snapotter-hq/snapotter:latest` to pull from GitHub Container Registry instead. Both registries receive the same image on every release.
## Quick Start (GPU) ## Quick Start (NVIDIA CUDA)
For NVIDIA GPU acceleration on AI tools (background removal, upscaling, face enhancement, OCR): For NVIDIA CUDA acceleration on AI tools (background removal, upscaling, face enhancement, OCR):
```yaml ```yaml
# docker-compose-gpu.yml - Requires: NVIDIA GPU + nvidia-container-toolkit # docker-compose-gpu.yml - Requires: NVIDIA GPU + nvidia-container-toolkit
@@ -198,11 +198,11 @@ volumes:
docker compose -f docker-compose-gpu.yml up -d docker compose -f docker-compose-gpu.yml up -d
``` ```
Check GPU detection in the logs: Check CUDA detection in the logs:
```bash ```bash
docker logs SnapOtter 2>&1 | head -20 docker logs SnapOtter 2>&1 | head -20
# Look for: [INFO] GPU detected — AI tools will use CUDA acceleration # Look for: [gpu] CUDA available via torch
``` ```
## Hardware Requirements ## Hardware Requirements
@@ -215,7 +215,7 @@ These numbers come from benchmarks run across four systems (Apple M2 Max, AMD Ry
|------|----------|-----|-----|-----|---------| |------|----------|-----|-----|-----|---------|
| Minimum | Core tools, single user | 1 core | 1 GB | None | 5 GB | | Minimum | Core tools, single user | 1 core | 1 GB | None | 5 GB |
| Recommended | All tools + AI on CPU | 4 cores | 4 GB | None | 20 GB | | Recommended | All tools + AI on CPU | 4 cores | 4 GB | None | 20 GB |
| Full | All tools + AI on GPU | 4+ cores | 8 GB | NVIDIA 8 GB+ | 30 GB | | Full | All tools + AI on NVIDIA CUDA | 4+ cores | 8 GB | NVIDIA 8 GB+ | 30 GB |
### Minimum (core tools, no AI) ### Minimum (core tools, no AI)
@@ -275,7 +275,7 @@ deploy:
memory: 4G memory: 4G
``` ```
### Full (AI tools on GPU) ### Full (AI tools on NVIDIA CUDA)
| Resource | Requirement | | Resource | Requirement |
|---|---| |---|---|
@@ -284,7 +284,7 @@ deploy:
| GPU | NVIDIA with 8+ GB VRAM (12 GB recommended) | | GPU | NVIDIA with 8+ GB VRAM (12 GB recommended) |
| Disk | 30 GB total | | Disk | 30 GB total |
GPU acceleration gives 3-13,000x speedup depending on the operation. Measured on an RTX 4070 vs Intel i7-7600U: NVIDIA CUDA acceleration gives 3-13,000x speedup depending on the operation. Measured on an RTX 4070 vs Intel i7-7600U:
| AI Tool | GPU Time | CPU Time | Speedup | | AI Tool | GPU Time | CPU Time | Speedup |
|---|---|---|---| |---|---|---|---|
@@ -297,7 +297,9 @@ GPU acceleration gives 3-13,000x speedup depending on the operation. Measured on
| restore-photo | 31s | 90s | 2.9x | | restore-photo | 31s | 90s | 2.9x |
| colorize | 10s | 13s | 1.3x | | colorize | 10s | 13s | 1.3x |
Peak VRAM usage reaches 7.5 GB during upscale with face enhancement. A 6 GB GPU works for most AI tools individually but will fail on upscale. 8-12 GB VRAM handles everything. Peak VRAM usage reaches 7.5 GB during upscale with face enhancement. A 6 GB NVIDIA GPU works for most AI tools individually but will fail on upscale. 8-12 GB VRAM handles everything.
Intel/AMD iGPU acceleration through VA-API, Quick Sync, or OpenCL is not supported for AI inference today. Mapping `/dev/dri` into the container does not enable AI GPU acceleration; SnapOtter will run AI tools on CPU unless NVIDIA CUDA is available.
```yaml ```yaml
deploy: deploy:
apps/docs/guide/docker-tags.md
+9 -7
View File
@@ -14,15 +14,17 @@ docker run -d --name SnapOtter -p 1349:1349 -v SnapOtter-data:/data snapotter/sn
With no `DATABASE_URL` set, this runs in embedded mode: PostgreSQL and Redis start inside the container on loopback, with all data under the `SnapOtter-data` volume. Set `DATABASE_URL` and `REDIS_URL` (as the [Compose](#docker-compose) stack does) to use external services instead. See [Configuration](/guide/configuration#embedded-mode). With no `DATABASE_URL` set, this runs in embedded mode: PostgreSQL and Redis start inside the container on loopback, with all data under the `SnapOtter-data` volume. Set `DATABASE_URL` and `REDIS_URL` (as the [Compose](#docker-compose) stack does) to use external services instead. See [Configuration](/guide/configuration#embedded-mode).
## GPU acceleration ## NVIDIA CUDA acceleration
The image includes CUDA support on amd64. If you have an NVIDIA GPU with the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) installed, add `--gpus all`: The image includes NVIDIA CUDA support on amd64. If you have an NVIDIA GPU with the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) installed, add `--gpus all`:
```bash ```bash
docker run -d --name SnapOtter --gpus all -p 1349:1349 -v SnapOtter-data:/data snapotter/snapotter:latest docker run -d --name SnapOtter --gpus all -p 1349:1349 -v SnapOtter-data:/data snapotter/snapotter:latest
``` ```
The image auto-detects your GPU at runtime. Without `--gpus all`, it runs on CPU. Same image either way. The image auto-detects CUDA at runtime. Without `--gpus all`, or when CUDA is unavailable, AI tools run on CPU. Same image either way.
Intel/AMD iGPU acceleration through VA-API, Quick Sync, or OpenCL is not supported for SnapOtter AI inference today. Mapping `/dev/dri` into the container can expose the render device, but the AI runtime will still use CPU unless CUDA is available.
### Benchmarks ### Benchmarks
@@ -47,9 +49,9 @@ Tested on an NVIDIA RTX 4070 (12 GB VRAM) with a 572x1024 JPEG portrait.
| Upscale 2x | 3,957ms | 2,318ms | 1.7x | | Upscale 2x | 3,957ms | 2,318ms | 1.7x |
| OCR (PaddleOCR) | 1,469ms | 1,090ms | 1.3x | | OCR (PaddleOCR) | 1,469ms | 1,090ms | 1.3x |
### GPU health check ### CUDA health check
After the first AI request, the admin health endpoint reports GPU status: After the first AI request, the admin health endpoint reports CUDA GPU status:
``` ```
GET /api/v1/admin/health GET /api/v1/admin/health
@@ -118,7 +120,7 @@ volumes:
SnapOtter-redisdata: SnapOtter-redisdata:
``` ```
For GPU acceleration via Docker Compose, add the deploy section to the SnapOtter service: For NVIDIA CUDA acceleration via Docker Compose, add the deploy section to the SnapOtter service:
```yaml ```yaml
deploy: deploy:
@@ -143,7 +145,7 @@ For GPU acceleration via Docker Compose, add the deploy section to the SnapOtter
| Architecture | GPU support | Notes | | Architecture | GPU support | Notes |
|---|---|---| |---|---|---|
| linux/amd64 | NVIDIA CUDA | Full GPU acceleration for AI tools | | linux/amd64 | NVIDIA CUDA | Full CUDA acceleration for AI tools |
| linux/arm64 | CPU only | Raspberry Pi 4/5, Apple Silicon via Docker Desktop | | linux/arm64 | CPU only | Raspberry Pi 4/5, Apple Silicon via Docker Desktop |
## Migration from previous tags ## Migration from previous tags
apps/docs/guide/getting-started.md
+3 -3
View File
@@ -18,14 +18,14 @@ This single container runs everything it needs: with no `DATABASE_URL` set, it s
You will be asked to change your password on first login. You will be asked to change your password on first login.
::: tip NVIDIA GPU acceleration ::: tip NVIDIA CUDA acceleration
Add `--gpus all` for GPU-accelerated background removal, upscaling, OCR, face enhancement, and restoration: Add `--gpus all` for NVIDIA CUDA-accelerated background removal, upscaling, OCR, face enhancement, and restoration:
```bash ```bash
docker run -d --name SnapOtter -p 1349:1349 --gpus all -v SnapOtter-data:/data snapotter/snapotter:latest docker run -d --name SnapOtter -p 1349:1349 --gpus all -v SnapOtter-data:/data snapotter/snapotter:latest
``` ```
Requires the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html). Falls back to CPU automatically. See [Docker Tags](/guide/docker-tags) for benchmarks. Requires the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html). Falls back to CPU automatically when CUDA is unavailable. Intel/AMD iGPU acceleration through VA-API, Quick Sync, or OpenCL is not supported for AI inference today. See [Docker Tags](/guide/docker-tags) for benchmarks.
::: :::
::: details Also on GHCR ::: details Also on GHCR
apps/landing/src/components/FeatureHighlights.astro
+1 -1
View File
@@ -141,7 +141,7 @@ const aiPillColors = [
<p class="mt-4 text-base leading-relaxed text-muted"> <p class="mt-4 text-base leading-relaxed text-muted">
19 local AI tools for OCR, transcription, background removal, upscaling, and more. 19 local AI tools for OCR, transcription, background removal, upscaling, and more.
No external API keys, no per-request charges, no data sent to third-party AI providers. No external API keys, no per-request charges, no data sent to third-party AI providers.
Runs on your GPU or CPU within your own infrastructure. Runs on CPU or NVIDIA CUDA within your own infrastructure.
</p> </p>
</div> </div>
<div class="md:order-2"> <div class="md:order-2">
apps/landing/src/data/alternatives.ts
+3 -3
View File
@@ -195,7 +195,7 @@ export const ALTERNATIVES: Alternative[] = [
pageTitle: "The Open-Source, Self-Hosted Alternative to Otter.ai", pageTitle: "The Open-Source, Self-Hosted Alternative to Otter.ai",
h1: "The open-source, self-hosted alternative to Otter.ai", h1: "The open-source, self-hosted alternative to Otter.ai",
metaDescription: metaDescription:
"Otter.ai transcribes audio in the cloud. SnapOtter transcribes on your own GPU or CPU with local Whisper models. No upload, no per-minute fees. Open source, AGPLv3.", "Otter.ai transcribes audio in the cloud. SnapOtter transcribes on your own CPU or NVIDIA CUDA GPU with local Whisper models. No upload, no per-minute fees. Open source, AGPLv3.",
intro: intro:
"Otter.ai uploads your recordings to transcribe them in the cloud. SnapOtter runs speech-to-text on your own hardware with local models, so sensitive recordings never leave your network.", "Otter.ai uploads your recordings to transcribe them in the cloud. SnapOtter runs speech-to-text on your own hardware with local models, so sensitive recordings never leave your network.",
breadth: breadth:
@@ -204,7 +204,7 @@ export const ALTERNATIVES: Alternative[] = [
rows: [ rows: [
{ {
feature: "Where your audio goes", feature: "Where your audio goes",
snapotter: "Your own server / GPU", snapotter: "Your own server",
competitor: "Their cloud", competitor: "Their cloud",
snapotterWins: true, snapotterWins: true,
}, },
@@ -242,7 +242,7 @@ export const ALTERNATIVES: Alternative[] = [
faqs: [ faqs: [
{ {
q: "Is there a self-hosted Otter.ai alternative?", q: "Is there a self-hosted Otter.ai alternative?",
a: "Yes. SnapOtter runs transcription locally with Whisper models on your own GPU or CPU. No audio is uploaded to a third party.", a: "Yes. SnapOtter runs transcription locally with Whisper models on your own CPU or NVIDIA CUDA GPU. No audio is uploaded to a third party.",
}, },
{ {
q: "Do I need a GPU?", q: "Do I need a GPU?",
apps/landing/src/data/tool-seo.ts
+5 -5
View File
@@ -3012,14 +3012,14 @@ export const TOOL_SEO: Record<string, ToolSeo> = {
}, },
{ {
q: "Can I remove backgrounds from hundreds of product images at once?", q: "Can I remove backgrounds from hundreds of product images at once?",
a: "Yes. Upload your entire product catalog and batch process them all. Each image gets the same AI treatment. Processing time depends on your hardware, but GPU acceleration is supported.", a: "Yes. Upload your entire product catalog and batch process them all. Each image gets the same AI treatment. Processing time depends on your hardware, and NVIDIA CUDA acceleration is supported.",
}, },
], ],
}, },
upscale: { upscale: {
searchTitle: "Upscale Image with AI - Enhance Resolution", searchTitle: "Upscale Image with AI - Enhance Resolution",
longDescription: longDescription:
"Upscale images using AI super-resolution models that add genuine detail, not just interpolated blur. Supports 2x and 4x scaling with models trained on real-world photography. Runs locally with GPU acceleration when available.", "Upscale images using AI super-resolution models that add genuine detail, not just interpolated blur. Supports 2x and 4x scaling with models trained on real-world photography. Runs locally with NVIDIA CUDA acceleration when available.",
useCases: [ useCases: [
"Upscale low-resolution product images for high-DPI displays", "Upscale low-resolution product images for high-DPI displays",
"Enlarge social media photos for print without losing quality", "Enlarge social media photos for print without losing quality",
@@ -3029,7 +3029,7 @@ export const TOOL_SEO: Record<string, ToolSeo> = {
features: [ features: [
"2x and 4x AI upscaling with RealESRGAN", "2x and 4x AI upscaling with RealESRGAN",
"Adds genuine detail, not interpolation blur", "Adds genuine detail, not interpolation blur",
"GPU acceleration (CUDA) when available", "NVIDIA CUDA acceleration when available",
"Optimized models for photos, anime, and general content", "Optimized models for photos, anime, and general content",
"Runs 100% locally with no cloud dependency", "Runs 100% locally with no cloud dependency",
], ],
@@ -3040,11 +3040,11 @@ export const TOOL_SEO: Record<string, ToolSeo> = {
}, },
{ {
q: "How large can I upscale an image?", q: "How large can I upscale an image?",
a: "SnapOtter supports 2x and 4x scaling. A 500x500 image becomes 1000x1000 at 2x or 2000x2000 at 4x. Processing time increases with the output size and depends on whether you have GPU acceleration.", a: "SnapOtter supports 2x and 4x scaling. A 500x500 image becomes 1000x1000 at 2x or 2000x2000 at 4x. Processing time increases with the output size and depends on whether NVIDIA CUDA acceleration is available.",
}, },
{ {
q: "Do I need a GPU for AI upscaling?", q: "Do I need a GPU for AI upscaling?",
a: "A GPU (NVIDIA CUDA) significantly speeds up processing, but it is not required. CPU-only mode works on any hardware, just slower. A typical photo takes seconds on GPU, minutes on CPU.", a: "An NVIDIA CUDA GPU significantly speeds up processing, but it is not required. CPU-only mode works on any hardware, just slower. A typical photo takes seconds on CUDA, minutes on CPU.",
}, },
], ],
}, },