trawl/apps/docs/deployment/docker-compose.md

title, description

title	description
Docker Compose	Run TRAWL with the supplied minimal, cached, or production Docker Compose setup.

Docker Compose

Three Compose files live in the repository root.

Scraper only

Minimal

docker-compose.minimal.yml — single service, no Redis. Fastest to get started, no session caching.

docker compose -f docker-compose.minimal.yml up -d

Cached (default)

docker-compose.yml — scraper + Redis session cache, with API port 8191 and optional proxy port 8192. Repeat requests can reuse accepted sessions instead of solving the challenge again.

docker compose up -d

Production

docker-compose.prod.yml — same as cached but with restart: always, a memory limit, and a healthcheck.

docker compose -f docker-compose.prod.yml up -d

trawl:
  restart: always
  mem_limit: 3g
  environment:
    BROWSER_POOL_SIZE: 3
  healthcheck:
    test: ["CMD", "curl", "-sf", "http://localhost:8191/health"]
    interval: 30s

To update to the latest image:

docker compose pull && docker compose up -d

Baseline (older CPUs / Synology NAS)

If your CPU doesn't support AVX2 — older Synology NAS units, Atom/Celeron-era hardware — override the image tag to :baseline in any compose file above. Nothing else changes:

services:
  trawl:
    image: ghcr.io/germondai/trawl:baseline
    # ...rest of the service definition unchanged

::: tip See Standalone Containers → Older CPUs & Synology NAS for how to tell if you need this, and the README for the full tag comparison. :::

Environment variables

Variable	Default	Description
BROWSER_POOL_SIZE	3	Warm browsers; supplied minimal/cached Compose files override this to 1
BROWSER_ACQUIRE_TIMEOUT_MS	15000	How long acquire() polls for a free browser before returning HTTP 429
BROWSER_RECYCLE_AFTER_CONTEXTS	8	Rolling-replace after this many Tier 3/4 contexts; 0 disables it
REDIS_URL	redis://redis:6379	Redis connection (set automatically in compose)
PROXY_URL	—	Optional Tier 3 datacenter proxy or pool
RESIDENTIAL_PROXY_URL	—	Enables Tier 4 proxy escalation
MITM_PROXY_ENABLED	false	Starts the general HTTP/HTTPS proxy
MITM_PROXY_PORT	8192	Proxy listen and published port
MITM_PROXY_HOST	0.0.0.0	Proxy bind address
MITM_PROXY_CA_DIR	/data/proxy-ca	Persistent root CA directory

All supplied Compose files publish port 8192 and mount the trawl_proxy_ca volume. The listener does not start until MITM_PROXY_ENABLED=true. See Proxy Configuration.

All supplied Compose files also pass PROXY_URL, PROXY_LIST_FILE, RESIDENTIAL_PROXY_URL, and RESIDENTIAL_PROXY_LIST_FILE from the local environment or .env file. For a single residential endpoint:

# .env
RESIDENTIAL_PROXY_URL=http://user:pass@residential.example.com:8080

For supported endpoint formats, pools, and mounted list files, see Configuration → Proxies.

Route TRAWL through Gluetun

To route all TRAWL traffic through a Gluetun VPN, share Gluetun's network namespace and wait for its healthcheck before starting TRAWL. Because TRAWL no longer has its own network namespace, publish both TRAWL ports on the gluetun service:

services:
  gluetun:
    image: qmcgaw/gluetun
    # Configure your VPN provider and credentials here.
    ports:
      - "8191:8191"
      - "8192:8192"

  trawl:
    image: ghcr.io/germondai/trawl:latest
    network_mode: service:gluetun
    shm_size: 1gb
    depends_on:
      gluetun:
        condition: service_healthy

Do not also publish 8191 or 8192 on trawl; Compose does not allow port publishing with network_mode: service:gluetun. Gluetun routing applies to all outbound container traffic. PROXY_URL, by contrast, is optional application-level proxy configuration used by TRAWL's escalation tiers; it is not required to send the container through the VPN.

Logs

docker compose logs -f trawl
docker compose logs -f redis

Memory guide

BROWSER_POOL_SIZE	Approx. RAM	Recommended host RAM
1	~500 MB	1 GB
3	~1.2 GB	2 GB
5	~2 GB	3 GB
10	~4 GB	6 GB

Each Camoufox Firefox instance uses ~350–500 MB under load.

Reverse proxy

To expose TRAWL over HTTPS, proxy port 8191. Set proxy_read_timeout longer than your maxTimeout — challenge solves can take up to 15s.

server {
  listen 443 ssl;
  server_name trawl.yourdomain.com;

  location / {
    proxy_pass http://localhost:8191;
    proxy_http_version 1.1;
    proxy_set_header Host $host;
    proxy_read_timeout 120s;
  }
}