diff --git a/apps/docs/architecture/session-cache.md b/apps/docs/architecture/session-cache.md new file mode 100644 index 0000000..90f184b --- /dev/null +++ b/apps/docs/architecture/session-cache.md @@ -0,0 +1,70 @@ +--- +title: Session Cache +description: How TRAWL caches Cloudflare cookies in Redis to make repeat requests fast. +--- + +# Session Cache + +The session cache is what makes Tier 2 possible. After every successful Tier 3 solve, the extracted Cloudflare cookies are saved to Redis. The next request to the same domain injects those cookies into a browser context, skipping the challenge entirely. + +## Storage format + +Key: `session:{hostname}` (e.g. `session:nowsecure.nl`) + +Value (JSON): +```typescript +interface SessionData { + cookies: Cookie[] + userAgent: string + savedAt: number // unix timestamp ms +} +``` + +TTL: `SESSION_TTL_SECONDS` (default 3600 seconds / 1 hour). + +## Session key + +The key is the **hostname only** — no path, no port, no protocol. This means all pages on a domain share one session: + +``` +https://example.com/ → session:example.com +https://example.com/page → session:example.com (same key) +https://sub.example.com/ → session:sub.example.com (different key) +``` + +Subdomains have separate sessions because Cloudflare can issue different challenge cookies per subdomain. + +## Lifecycle + +``` +Tier 3 succeeds + │ + ├── extract cookies from browser context + ├── REDIS SET session:hostname → JSON EX SESSION_TTL_SECONDS + │ + └── next request to same domain: + REDIS GET session:hostname + ├── hit → Tier 2: inject cookies, navigate (500ms) + └── miss → Tier 3: fresh solve, save to cache +``` + +## Invalidation + +If Tier 2 navigates with the cached cookies and the result is still a Cloudflare interstitial (the session expired before Redis TTL), the orchestrator: + +1. Calls `sessionCache.invalidate(domain)` — deletes the Redis key +2. Escalates to Tier 3 to get a fresh session + +This handles the case where Cloudflare's `cf_clearance` cookie (30-minute expiry) expires before the Redis TTL does. + +## Redis client + +TRAWL uses `new RedisClient(REDIS_URL)` from Bun's native Redis client (not ioredis). Bun's Redis client is ~7.9× faster than ioredis for simple GET/SET operations, which matters since every request does at least one Redis read. + +```typescript +import { RedisClient } from 'bun' + +const redis = new RedisClient('redis://localhost:6379') +await redis.set('session:example.com', JSON.stringify(data), 'EX', 3600) +const raw = await redis.get('session:example.com') +```