trawl/apps/docs/deployment/troubleshooting.md

---
title: Troubleshooting
description: Common issues and how to fix them.
---

# Troubleshooting

## API never becomes ready

**Symptom:** `docker compose logs api` shows browser launches but the API never prints `ready — all N browsers warm`.

**Causes:**

1. **Redis not reachable** — Check `REDIS_URL`. From inside Docker, use `redis://redis:6379` not `redis://localhost:6379`.
2. **Camoufox binary not installed** — The API Dockerfile runs `bunx camoufox-js fetch`. If this step was skipped (e.g. build cache reuse), rebuild: `docker compose build --no-cache api`.
3. **shm_size too small** — Ensure `shm_size: 1gb` is set on the API service.

### Startup timeout behind Gluetun

**Symptom:** TRAWL logs `browser launch exceeded ...` during startup and never becomes healthy.

Camoufox performs outbound work, including GeoIP lookup, while launching. If TRAWL starts before
Gluetun has established its VPN connection, that work can hit the browser **launch timeout**. Set
`depends_on.gluetun.condition: service_healthy` as shown in
[Docker Compose → Route TRAWL through Gluetun](/deployment/docker-compose#route-trawl-through-gluetun).

This differs from the **acquire timeout** (`BROWSER_ACQUIRE_TIMEOUT_MS`): a launch timeout means a
browser could not start, often because outbound networking or a GeoIP endpoint was unavailable; an
acquire timeout means the browsers started but all pool capacity remained busy, and `/v1` returns
HTTP 429.

### Requests through the VPN return HTTP 403

If TRAWL starts successfully but a target returns HTTP 403, the VPN is already routing traffic and
the target is likely rejecting the VPN exit IP. Waiting longer for startup will not fix that. Try a
different Gluetun server or exit region, or configure an appropriate application proxy.

Gluetun and `PROXY_URL` operate at different layers: `network_mode: service:gluetun` routes all
container traffic through the VPN, while the optional `PROXY_URL` is used explicitly by TRAWL's
proxy escalation tier. Setting `PROXY_URL` is not necessary for Gluetun routing, and setting it
means that proxied requests still reach that proxy through Gluetun's network namespace.

## Container crash-loops with `EISDIR` or `Cannot find module` errors

**Symptom:** The container restarts continuously, logging one of:

```
error: EISDIR reading "/app/packages/browser/node_modules/camoufox-js"
```

```
error: Cannot find module '@sinclair/typebox' from '/app/apps/api/node_modules/elysia/dist/index.mjs'
```

or, on older images, `error: Cannot find package 'memoirist' from '/app/apps/api/node_modules/elysia/dist/index.mjs'`.

**Cause:** A bug in Bun's default "isolated" install linker corrupted `node_modules` during the Docker build, leaving transitive dependencies (`camoufox-js`, `@sinclair/typebox`) missing or broken inside the image ([oven-sh/bun#23524](https://github.com/oven-sh/bun/issues/23524), [oven-sh/bun#29489](https://github.com/oven-sh/bun/issues/29489)).

::: tip Already fixed — just re-pull
This was fixed by switching the image build to `bun install --linker=hoisted`. Every `:latest` and `:baseline` image published after the fix is unaffected. If you're hitting this, re-pull rather than patching your container:

```bash
docker pull ghcr.io/germondai/trawl:latest   # or :baseline
docker compose up -d --force-recreate
```
:::

## All requests return Tier 3 (never hitting cache)

**Symptom:** every request takes 10–30s, even for the same domain.

**Causes:**

1. **Redis session data is not persisting** — Run `docker compose exec redis redis-cli keys "session:*"` after a successful scrape. If empty, the session cache write is failing. Check API logs for Redis connection errors.
2. **`SESSION_TTL_SECONDS` set too low** — If it's shorter than Cloudflare's challenge interval, the cache expires before the next request.
3. **Domain key mismatch** — The key is the hostname only. `sub.example.com` and `www.example.com` are separate sessions.

## POST /v1 returns HTTP 429 with `status: "error"`

**Symptom:** Request returns **HTTP 429** (not 500) with a FlareSolverr v2 envelope and `message: "Browser pool saturated, retry shortly"`.

**Cause:** TRAWL polled for `BROWSER_ACQUIRE_TIMEOUT_MS` (default 15s) without finding an idle browser. With `BROWSER_POOL_SIZE=3` and 10 concurrent requests, this only fires under sustained burst pressure.

**Fixes (in order of preference):**

1. **Raise `BROWSER_ACQUIRE_TIMEOUT_MS`** if your upstream target legitimately takes >5s per scrape — bumps the queue wait before 429 fires.
2. **Raise `BROWSER_POOL_SIZE`** if you're consistently saturating — each browser uses ~350–500 MB RAM.
3. **Reduce incoming request rate** if you control the client (Prowlarr's indexer interval, etc.).

## POST /v1 returns `status: "error"` with message `"timeout"`

**Symptom:** `maxTimeout` exceeded (per-request timeout set by the client).

**Causes:**

1. **Cloudflare introduced a harder challenge** — Some sites use Turnstile or WAF rules that are harder to bypass. Check the API logs for the actual error.
2. **Pool exhausted** — All browsers are busy. Increase `BROWSER_POOL_SIZE`.
3. **Proxy not working** — If `PROXY_URL` is configured and invalid, Tier 3 will fail consistently. Test the proxy directly: `curl --proxy $PROXY_URL https://nowsecure.nl`.

## Prowlarr FlareSolverr test fails

**Symptom:** Green test in isolation but Prowlarr reports the FlareSolverr test as failed.

**Check:**
1. The URL in Prowlarr includes no trailing slash: `http://trawl:8191`
2. Prowlarr can reach the TRAWL container. If they're in different Docker networks, add TRAWL to Prowlarr's network (see [Prowlarr docs](/integrations/prowlarr)).
3. Run `docker exec prowlarr curl -s http://trawl:8191/health` to verify network reachability from inside the Prowlarr container.

## High memory usage / OOM kills

Each Camoufox instance uses 350–500 MB. With 3 browsers, expect ~1.5 GB total. If the API is being killed:

1. Reduce `BROWSER_POOL_SIZE` to 1 or 2
2. Upgrade the server (more RAM or more cores)
3. Ensure `shm_size: 1gb` is set — Firefox uses `/dev/shm` heavily

## Debugging tips

```bash
# Live API logs
docker compose logs -f api

# Check Redis keys
docker compose exec redis redis-cli keys "*"

# Test scrape endpoint
curl -s -X POST http://localhost:8191/scrape \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com"}' | jq '{tier, totalMs}'

# Verify FlareSolverr compat response shape
curl -s -X POST http://localhost:8191/v1 \
  -H "Content-Type: application/json" \
  -d '{"cmd":"request.get","url":"https://nowsecure.nl"}' | jq '{status, version}'
```