docs: add session cache design documentation

This commit is contained in:
germondai
2026-06-19 09:40:00 +02:00
parent 2a28054e1a
commit 8bd83862e4
+70
View File
@@ -0,0 +1,70 @@
---
title: Session Cache
description: How TRAWL caches Cloudflare cookies in Redis to make repeat requests fast.
---
# Session Cache
The session cache is what makes Tier 2 possible. After every successful Tier 3 solve, the extracted Cloudflare cookies are saved to Redis. The next request to the same domain injects those cookies into a browser context, skipping the challenge entirely.
## Storage format
Key: `session:{hostname}` (e.g. `session:nowsecure.nl`)
Value (JSON):
```typescript
interface SessionData {
cookies: Cookie[]
userAgent: string
savedAt: number // unix timestamp ms
}
```
TTL: `SESSION_TTL_SECONDS` (default 3600 seconds / 1 hour).
## Session key
The key is the **hostname only** — no path, no port, no protocol. This means all pages on a domain share one session:
```
https://example.com/ → session:example.com
https://example.com/page → session:example.com (same key)
https://sub.example.com/ → session:sub.example.com (different key)
```
Subdomains have separate sessions because Cloudflare can issue different challenge cookies per subdomain.
## Lifecycle
```
Tier 3 succeeds
├── extract cookies from browser context
├── REDIS SET session:hostname → JSON EX SESSION_TTL_SECONDS
└── next request to same domain:
REDIS GET session:hostname
├── hit → Tier 2: inject cookies, navigate (500ms)
└── miss → Tier 3: fresh solve, save to cache
```
## Invalidation
If Tier 2 navigates with the cached cookies and the result is still a Cloudflare interstitial (the session expired before Redis TTL), the orchestrator:
1. Calls `sessionCache.invalidate(domain)` — deletes the Redis key
2. Escalates to Tier 3 to get a fresh session
This handles the case where Cloudflare's `cf_clearance` cookie (30-minute expiry) expires before the Redis TTL does.
## Redis client
TRAWL uses `new RedisClient(REDIS_URL)` from Bun's native Redis client (not ioredis). Bun's Redis client is ~7.9× faster than ioredis for simple GET/SET operations, which matters since every request does at least one Redis read.
```typescript
import { RedisClient } from 'bun'
const redis = new RedisClient('redis://localhost:6379')
await redis.set('session:example.com', JSON.stringify(data), 'EX', 3600)
const raw = await redis.get('session:example.com')
```