2.5 KiB
title, description
| title | description |
|---|---|
| Architecture Overview | How TRAWL's components fit together — embedded browser pool, session cache, and tiered execution. |
Architecture Overview
Client (Prowlarr, curl, your code)
│
▼
Elysia API (apps/api)
┌──────────────────────────┐
│ POST /v1 POST /scrape │
│ GET /health GET /stats │
└────────────┬─────────────┘
│ direct call
▼
Orchestrator (packages/tiers)
┌──────────────────────────────────────┐
│ ├── Tier 1: plain Bun fetch │
│ ├── Tier 2: cached session + pool │
│ ├── Tier 3: fresh solve + pool │
│ └── Tier 4: residential proxy │
└──────────────┬───────────────────────┘
┌────────┴──────────┐
▼ ▼
BrowserPool SessionCache
(packages/browser) (packages/browser)
│ │
▼ ▼
Camoufox Redis (cache)
Firefox N×
Components
API (apps/api)
An Elysia HTTP server. Accepts scrape requests and calls the orchestrator inline — all browser work happens in the same process. Exposes /health and /stats for monitoring.
Browser Pool (packages/browser/src/pool.ts)
Maintains a fixed set of { browser, context } pairs using Camoufox (Firefox with fingerprint patching at the C++/Juggler level). Acquisition is sticky — if a browser last served example.com, it is preferred for the next request to example.com. Browsers accumulate domain cookies across requests, which makes subsequent challenges faster.
Session Cache (packages/browser/src/session.ts)
Stores { cookies, userAgent, savedAt } in Redis, keyed by hostname (session:example.com). The TTL is configurable (default 1 hour). Tier 3 writes to it on every successful challenge solve. Tier 2 reads from it at the start of every request.
Redis is optional — if REDIS_URL is not set, the session cache is disabled and every request escalates to Tier 3.
Tiers (packages/tiers)
The escalation logic. See Tiered Execution for the full breakdown.