mirror of
https://github.com/snapotter-hq/SnapOtter.git
synced 2026-08-03 07:46:42 +02:00
docs: clarify nvidia cuda ai support (#378)
This commit is contained in:
SnapOtter
+4
-2
@@ -4,7 +4,9 @@ description: AI engine reference with all local ML tools. Background removal, up
|
||||
|
||||
# AI Engine Reference
|
||||
|
||||
The `@snapotter/ai` package bridges Node.js to a **persistent Python sidecar** for all ML operations. The dispatcher process stays alive between requests for fast warm-start performance. GPU is auto-detected at startup and used when available.
|
||||
The `@snapotter/ai` package bridges Node.js to a **persistent Python sidecar** for all ML operations. The dispatcher process stays alive between requests for fast warm-start performance. NVIDIA CUDA is auto-detected at startup and used when available; otherwise AI tools run on CPU.
|
||||
|
||||
Intel/AMD iGPU acceleration through VA-API, Quick Sync, or OpenCL is not supported for AI inference today. Mapping `/dev/dri` into a container does not accelerate these Python sidecar tools unless a CUDA-capable NVIDIA GPU is available.
|
||||
|
||||
19 Python sidecar AI tools across four modalities (image, audio, video, document), plus 2 tools with optional AI capabilities. All models run locally - no internet required after initial model download.
|
||||
|
||||
@@ -278,7 +280,7 @@ The mask is sent as a **second file part** (fieldname `mask`), not as base64. Wh
|
||||
| `format` | string | `"auto"` | Output format: `auto`, `png`, `jpg`, `jpeg`, `webp`, `tiff`, `gif`, `avif`, `heic`, `heif`, `jxl` |
|
||||
| `quality` | integer (1-100) | `95` | Output quality |
|
||||
|
||||
GPU-accelerated when an NVIDIA GPU is available.
|
||||
CUDA-accelerated when an NVIDIA GPU is available.
|
||||
|
||||
## AI Canvas Expand
|
||||
|
||||
|
||||
@@ -145,7 +145,7 @@ curl -X POST http://localhost:1349/api/v1/tools/<section>/<toolId>/batch \
|
||||
|
||||
### AI Tools
|
||||
|
||||
All AI tools run on your hardware (CPU or NVIDIA GPU). No internet required.
|
||||
All AI tools run on your hardware: CPU by default, or NVIDIA CUDA when a supported NVIDIA GPU is available. Intel/AMD iGPU acceleration through VA-API, Quick Sync, or OpenCL is not supported for AI inference today. No internet required.
|
||||
|
||||
| Tool ID | Name | AI Model | Key settings |
|
||||
|---------|------|---------|-------------|
|
||||
|
||||
@@ -4,7 +4,7 @@ description: Deploy SnapOtter to production with Docker. Hardware requirements,
|
||||
|
||||
# Deployment
|
||||
|
||||
SnapOtter deploys as a 3-container Docker Compose stack: the SnapOtter app image, PostgreSQL 17, and Redis 8. The app image supports **linux/amd64** (with NVIDIA CUDA) and **linux/arm64** (CPU), so it runs natively on Intel/AMD servers, Apple Silicon Macs, and ARM devices like the Raspberry Pi 4/5.
|
||||
SnapOtter deploys as a 3-container Docker Compose stack: the SnapOtter app image, PostgreSQL 17, and Redis 8. The app image supports **linux/amd64** (with NVIDIA CUDA for AI acceleration) and **linux/arm64** (CPU), so it runs natively on Intel/AMD servers, Apple Silicon Macs, and ARM devices like the Raspberry Pi 4/5. Intel/AMD iGPU acceleration through VA-API, Quick Sync, or OpenCL is not supported for AI inference today.
|
||||
|
||||
See [Docker Image](./docker-tags) for GPU setup, Docker Compose examples, and version pinning.
|
||||
|
||||
@@ -108,9 +108,9 @@ The app is then available at `http://localhost:1349`.
|
||||
|
||||
> **Docker Hub rate limits?** Replace `snapotter/snapotter:latest` with `ghcr.io/snapotter-hq/snapotter:latest` to pull from GitHub Container Registry instead. Both registries receive the same image on every release.
|
||||
|
||||
## Quick Start (GPU)
|
||||
## Quick Start (NVIDIA CUDA)
|
||||
|
||||
For NVIDIA GPU acceleration on AI tools (background removal, upscaling, face enhancement, OCR):
|
||||
For NVIDIA CUDA acceleration on AI tools (background removal, upscaling, face enhancement, OCR):
|
||||
|
||||
```yaml
|
||||
# docker-compose-gpu.yml - Requires: NVIDIA GPU + nvidia-container-toolkit
|
||||
@@ -198,11 +198,11 @@ volumes:
|
||||
docker compose -f docker-compose-gpu.yml up -d
|
||||
```
|
||||
|
||||
Check GPU detection in the logs:
|
||||
Check CUDA detection in the logs:
|
||||
|
||||
```bash
|
||||
docker logs SnapOtter 2>&1 | head -20
|
||||
# Look for: [INFO] GPU detected — AI tools will use CUDA acceleration
|
||||
# Look for: [gpu] CUDA available via torch
|
||||
```
|
||||
|
||||
## Hardware Requirements
|
||||
@@ -215,7 +215,7 @@ These numbers come from benchmarks run across four systems (Apple M2 Max, AMD Ry
|
||||
|------|----------|-----|-----|-----|---------|
|
||||
| Minimum | Core tools, single user | 1 core | 1 GB | None | 5 GB |
|
||||
| Recommended | All tools + AI on CPU | 4 cores | 4 GB | None | 20 GB |
|
||||
| Full | All tools + AI on GPU | 4+ cores | 8 GB | NVIDIA 8 GB+ | 30 GB |
|
||||
| Full | All tools + AI on NVIDIA CUDA | 4+ cores | 8 GB | NVIDIA 8 GB+ | 30 GB |
|
||||
|
||||
### Minimum (core tools, no AI)
|
||||
|
||||
@@ -275,7 +275,7 @@ deploy:
|
||||
memory: 4G
|
||||
```
|
||||
|
||||
### Full (AI tools on GPU)
|
||||
### Full (AI tools on NVIDIA CUDA)
|
||||
|
||||
| Resource | Requirement |
|
||||
|---|---|
|
||||
@@ -284,7 +284,7 @@ deploy:
|
||||
| GPU | NVIDIA with 8+ GB VRAM (12 GB recommended) |
|
||||
| Disk | 30 GB total |
|
||||
|
||||
GPU acceleration gives 3-13,000x speedup depending on the operation. Measured on an RTX 4070 vs Intel i7-7600U:
|
||||
NVIDIA CUDA acceleration gives 3-13,000x speedup depending on the operation. Measured on an RTX 4070 vs Intel i7-7600U:
|
||||
|
||||
| AI Tool | GPU Time | CPU Time | Speedup |
|
||||
|---|---|---|---|
|
||||
@@ -297,7 +297,9 @@ GPU acceleration gives 3-13,000x speedup depending on the operation. Measured on
|
||||
| restore-photo | 31s | 90s | 2.9x |
|
||||
| colorize | 10s | 13s | 1.3x |
|
||||
|
||||
Peak VRAM usage reaches 7.5 GB during upscale with face enhancement. A 6 GB GPU works for most AI tools individually but will fail on upscale. 8-12 GB VRAM handles everything.
|
||||
Peak VRAM usage reaches 7.5 GB during upscale with face enhancement. A 6 GB NVIDIA GPU works for most AI tools individually but will fail on upscale. 8-12 GB VRAM handles everything.
|
||||
|
||||
Intel/AMD iGPU acceleration through VA-API, Quick Sync, or OpenCL is not supported for AI inference today. Mapping `/dev/dri` into the container does not enable AI GPU acceleration; SnapOtter will run AI tools on CPU unless NVIDIA CUDA is available.
|
||||
|
||||
```yaml
|
||||
deploy:
|
||||
|
||||
@@ -14,15 +14,17 @@ docker run -d --name SnapOtter -p 1349:1349 -v SnapOtter-data:/data snapotter/sn
|
||||
|
||||
With no `DATABASE_URL` set, this runs in embedded mode: PostgreSQL and Redis start inside the container on loopback, with all data under the `SnapOtter-data` volume. Set `DATABASE_URL` and `REDIS_URL` (as the [Compose](#docker-compose) stack does) to use external services instead. See [Configuration](/guide/configuration#embedded-mode).
|
||||
|
||||
## GPU acceleration
|
||||
## NVIDIA CUDA acceleration
|
||||
|
||||
The image includes CUDA support on amd64. If you have an NVIDIA GPU with the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) installed, add `--gpus all`:
|
||||
The image includes NVIDIA CUDA support on amd64. If you have an NVIDIA GPU with the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) installed, add `--gpus all`:
|
||||
|
||||
```bash
|
||||
docker run -d --name SnapOtter --gpus all -p 1349:1349 -v SnapOtter-data:/data snapotter/snapotter:latest
|
||||
```
|
||||
|
||||
The image auto-detects your GPU at runtime. Without `--gpus all`, it runs on CPU. Same image either way.
|
||||
The image auto-detects CUDA at runtime. Without `--gpus all`, or when CUDA is unavailable, AI tools run on CPU. Same image either way.
|
||||
|
||||
Intel/AMD iGPU acceleration through VA-API, Quick Sync, or OpenCL is not supported for SnapOtter AI inference today. Mapping `/dev/dri` into the container can expose the render device, but the AI runtime will still use CPU unless CUDA is available.
|
||||
|
||||
### Benchmarks
|
||||
|
||||
@@ -47,9 +49,9 @@ Tested on an NVIDIA RTX 4070 (12 GB VRAM) with a 572x1024 JPEG portrait.
|
||||
| Upscale 2x | 3,957ms | 2,318ms | 1.7x |
|
||||
| OCR (PaddleOCR) | 1,469ms | 1,090ms | 1.3x |
|
||||
|
||||
### GPU health check
|
||||
### CUDA health check
|
||||
|
||||
After the first AI request, the admin health endpoint reports GPU status:
|
||||
After the first AI request, the admin health endpoint reports CUDA GPU status:
|
||||
|
||||
```
|
||||
GET /api/v1/admin/health
|
||||
@@ -118,7 +120,7 @@ volumes:
|
||||
SnapOtter-redisdata:
|
||||
```
|
||||
|
||||
For GPU acceleration via Docker Compose, add the deploy section to the SnapOtter service:
|
||||
For NVIDIA CUDA acceleration via Docker Compose, add the deploy section to the SnapOtter service:
|
||||
|
||||
```yaml
|
||||
deploy:
|
||||
@@ -143,7 +145,7 @@ For GPU acceleration via Docker Compose, add the deploy section to the SnapOtter
|
||||
|
||||
| Architecture | GPU support | Notes |
|
||||
|---|---|---|
|
||||
| linux/amd64 | NVIDIA CUDA | Full GPU acceleration for AI tools |
|
||||
| linux/amd64 | NVIDIA CUDA | Full CUDA acceleration for AI tools |
|
||||
| linux/arm64 | CPU only | Raspberry Pi 4/5, Apple Silicon via Docker Desktop |
|
||||
|
||||
## Migration from previous tags
|
||||
|
||||
@@ -18,14 +18,14 @@ This single container runs everything it needs: with no `DATABASE_URL` set, it s
|
||||
|
||||
You will be asked to change your password on first login.
|
||||
|
||||
::: tip NVIDIA GPU acceleration
|
||||
Add `--gpus all` for GPU-accelerated background removal, upscaling, OCR, face enhancement, and restoration:
|
||||
::: tip NVIDIA CUDA acceleration
|
||||
Add `--gpus all` for NVIDIA CUDA-accelerated background removal, upscaling, OCR, face enhancement, and restoration:
|
||||
|
||||
```bash
|
||||
docker run -d --name SnapOtter -p 1349:1349 --gpus all -v SnapOtter-data:/data snapotter/snapotter:latest
|
||||
```
|
||||
|
||||
Requires the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html). Falls back to CPU automatically. See [Docker Tags](/guide/docker-tags) for benchmarks.
|
||||
Requires the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html). Falls back to CPU automatically when CUDA is unavailable. Intel/AMD iGPU acceleration through VA-API, Quick Sync, or OpenCL is not supported for AI inference today. See [Docker Tags](/guide/docker-tags) for benchmarks.
|
||||
:::
|
||||
|
||||
::: details Also on GHCR
|
||||
|
||||
@@ -141,7 +141,7 @@ const aiPillColors = [
|
||||
<p class="mt-4 text-base leading-relaxed text-muted">
|
||||
19 local AI tools for OCR, transcription, background removal, upscaling, and more.
|
||||
No external API keys, no per-request charges, no data sent to third-party AI providers.
|
||||
Runs on your GPU or CPU within your own infrastructure.
|
||||
Runs on CPU or NVIDIA CUDA within your own infrastructure.
|
||||
</p>
|
||||
</div>
|
||||
<div class="md:order-2">
|
||||
|
||||
@@ -195,7 +195,7 @@ export const ALTERNATIVES: Alternative[] = [
|
||||
pageTitle: "The Open-Source, Self-Hosted Alternative to Otter.ai",
|
||||
h1: "The open-source, self-hosted alternative to Otter.ai",
|
||||
metaDescription:
|
||||
"Otter.ai transcribes audio in the cloud. SnapOtter transcribes on your own GPU or CPU with local Whisper models. No upload, no per-minute fees. Open source, AGPLv3.",
|
||||
"Otter.ai transcribes audio in the cloud. SnapOtter transcribes on your own CPU or NVIDIA CUDA GPU with local Whisper models. No upload, no per-minute fees. Open source, AGPLv3.",
|
||||
intro:
|
||||
"Otter.ai uploads your recordings to transcribe them in the cloud. SnapOtter runs speech-to-text on your own hardware with local models, so sensitive recordings never leave your network.",
|
||||
breadth:
|
||||
@@ -204,7 +204,7 @@ export const ALTERNATIVES: Alternative[] = [
|
||||
rows: [
|
||||
{
|
||||
feature: "Where your audio goes",
|
||||
snapotter: "Your own server / GPU",
|
||||
snapotter: "Your own server",
|
||||
competitor: "Their cloud",
|
||||
snapotterWins: true,
|
||||
},
|
||||
@@ -242,7 +242,7 @@ export const ALTERNATIVES: Alternative[] = [
|
||||
faqs: [
|
||||
{
|
||||
q: "Is there a self-hosted Otter.ai alternative?",
|
||||
a: "Yes. SnapOtter runs transcription locally with Whisper models on your own GPU or CPU. No audio is uploaded to a third party.",
|
||||
a: "Yes. SnapOtter runs transcription locally with Whisper models on your own CPU or NVIDIA CUDA GPU. No audio is uploaded to a third party.",
|
||||
},
|
||||
{
|
||||
q: "Do I need a GPU?",
|
||||
|
||||
@@ -3012,14 +3012,14 @@ export const TOOL_SEO: Record<string, ToolSeo> = {
|
||||
},
|
||||
{
|
||||
q: "Can I remove backgrounds from hundreds of product images at once?",
|
||||
a: "Yes. Upload your entire product catalog and batch process them all. Each image gets the same AI treatment. Processing time depends on your hardware, but GPU acceleration is supported.",
|
||||
a: "Yes. Upload your entire product catalog and batch process them all. Each image gets the same AI treatment. Processing time depends on your hardware, and NVIDIA CUDA acceleration is supported.",
|
||||
},
|
||||
],
|
||||
},
|
||||
upscale: {
|
||||
searchTitle: "Upscale Image with AI - Enhance Resolution",
|
||||
longDescription:
|
||||
"Upscale images using AI super-resolution models that add genuine detail, not just interpolated blur. Supports 2x and 4x scaling with models trained on real-world photography. Runs locally with GPU acceleration when available.",
|
||||
"Upscale images using AI super-resolution models that add genuine detail, not just interpolated blur. Supports 2x and 4x scaling with models trained on real-world photography. Runs locally with NVIDIA CUDA acceleration when available.",
|
||||
useCases: [
|
||||
"Upscale low-resolution product images for high-DPI displays",
|
||||
"Enlarge social media photos for print without losing quality",
|
||||
@@ -3029,7 +3029,7 @@ export const TOOL_SEO: Record<string, ToolSeo> = {
|
||||
features: [
|
||||
"2x and 4x AI upscaling with RealESRGAN",
|
||||
"Adds genuine detail, not interpolation blur",
|
||||
"GPU acceleration (CUDA) when available",
|
||||
"NVIDIA CUDA acceleration when available",
|
||||
"Optimized models for photos, anime, and general content",
|
||||
"Runs 100% locally with no cloud dependency",
|
||||
],
|
||||
@@ -3040,11 +3040,11 @@ export const TOOL_SEO: Record<string, ToolSeo> = {
|
||||
},
|
||||
{
|
||||
q: "How large can I upscale an image?",
|
||||
a: "SnapOtter supports 2x and 4x scaling. A 500x500 image becomes 1000x1000 at 2x or 2000x2000 at 4x. Processing time increases with the output size and depends on whether you have GPU acceleration.",
|
||||
a: "SnapOtter supports 2x and 4x scaling. A 500x500 image becomes 1000x1000 at 2x or 2000x2000 at 4x. Processing time increases with the output size and depends on whether NVIDIA CUDA acceleration is available.",
|
||||
},
|
||||
{
|
||||
q: "Do I need a GPU for AI upscaling?",
|
||||
a: "A GPU (NVIDIA CUDA) significantly speeds up processing, but it is not required. CPU-only mode works on any hardware, just slower. A typical photo takes seconds on GPU, minutes on CPU.",
|
||||
a: "An NVIDIA CUDA GPU significantly speeds up processing, but it is not required. CPU-only mode works on any hardware, just slower. A typical photo takes seconds on CUDA, minutes on CPU.",
|
||||
},
|
||||
],
|
||||
},
|
||||
|
||||
Reference in New Issue
Block a user