`BrowserPool.restartEntry` awaited `context.close()`, `browser.close()` and the
Camoufox launch with no timeout on any of them. Camoufox hangs on close when a
content process is wedged — tiers/3.ts and tiers/4.ts already guard their
*temporary* contexts against exactly this with a 5s `Promise.race` — but the
persistent context and browser the pool owns had no such guard, and launches can
hang too.
When any one of those hangs, the entry is pinned at `restarting = true` forever.
From then on the health check hits its own `if (entry.restarting) return` guard,
so every 30s tick logs "browser N disconnected, restarting" and does nothing.
The pool silently loses that slot permanently: `restartCount` never increments,
so the restart counter sits frozen while the log implies furious activity. With
enough uptime every entry ends up in this state and the pool is inert.
Changes:
* every await in `restartEntry`, `init()` and `shutdown()` is bounded. On
timeout the entry is left unhealthy with `restarting` cleared, so the next
health-check tick retries it from scratch instead of wedging.
* `runHealthCheck` reclaims checkouts past their deadline. Previously busy
entries were skipped entirely, so an entry whose request wedged was never
examined again.
* a per-checkout `lease`, returned on the handle and passed back to
`release()`, so a request that outlives its checkout cannot free — or
recycle, via `noteTemporaryContext` — a browser the pool has since handed to
someone else.
* `release()` hands its in-flight page closes to `restartEntry` rather than
racing them, since closing a context underneath in-flight `page.close()`
calls is one way to wedge the transport in the first place.
* abandoned launches are counted and capped. A timeout can only stop *waiting*
for a launch, not cancel it, so retrying without a cap could pile up hung
Firefox processes; past the cap the entry stays down and `live` reflects it.
Timeouts are configurable (`closeTimeoutMs`, `launchTimeoutMs`, `stallAfterMs`,
`healthIntervalMs`) with the API exposing them as BROWSER_*_MS env vars.
Adds regression tests for the hung close, the hung launch, stall accounting,
budget-aware stall deadlines, disconnected-but-busy entries, and stale releases.
The hung-close and hung-launch tests both fail against the unpatched pool.
`/health` returns 200 as soon as `pool` is non-null, which happens before
`await pool.init()` has warmed any browser — so a readiness probe on /health
passes before the process can solve anything.
The obvious fix, `available + busy > 0`, is also wrong, and fails in a much
worse way. A request that hangs mid-solve never reaches the orchestrator's
`finally`, so it never calls `release()` and its entry stays `busy` for the life
of the process. `busy` therefore counts dead entries as capacity, and /health
can report 200/"ok" indefinitely on a pool with zero usable browsers — the
failure is completely invisible to any external check.
Adds `stalled` and `live` to PoolStats:
* an entry is `stalled` once its checkout outlives the caller's own budget
(req.maxTimeout, threaded through acquire()) plus a grace period, so a slow
but genuinely live request is never miscounted
* `live` counts entries that can serve work now or are genuinely mid-request:
idle-and-connected, plus busy-and-connected-and-not-stalled
/health now gates on `live > 0`. A fully utilised pool still reports ready, so
this does not flap under load, but a wedged one cannot report ready at all.
`isUsable()` also checks `browser.isConnected()` rather than trusting the
`healthy` flag, which is only refreshed on the 30s health-check tick and is
never refreshed at all for busy entries.
trawl returned some Akamai-fronted pages as 200 'success' with only the
~2KB sec-cpt behavioral interstitial as content, because tier detection
knew Cloudflare/Imperva but not Akamai.
- detect.ts: hasAkamaiChallenge() + 'akamai' ChallengeType (sec-if-cpt-container
/ behavioral-content markers, size-gated sensor fallback); wired into
detectChallengeType/isBlocked/needsJs.
- akamaiWait.ts (new): Akamai analogue of challengeWait/impervaWait — drives
human-like mouse motion, press-and-hold on the behavioral widget, waits for
the sensor's location.reload() into real content.
- tiers 1-4: escalate the 200 interstitial (needs-js), invalidate a stale
cached-session interstitial, dispatch the resolver, report akamai-persistent.
Additive; Cloudflare/Imperva paths untouched. Verified against Edmunds.
The FlareSolverr /v1 contract only returns cookies + user-agent. Clients like
Prowlarr take those and re-fetch the target with their own HTTP stack, which is
re-challenged on sites whose Cloudflare clearance is bound to the solving
browser's connection fingerprint (e.g. 1337x) — no cookie is portable to a
plain HTTP client, so those indexers can't be used at all.
Add an optional HTTP(S) forward proxy (MITM_PROXY_ENABLED). Point such a client's
proxy at it (per-indexer HTTP proxy in Prowlarr) and every request — search and
the .torrent/magnet grab — is transparently re-issued through the browser pool,
returning the RAW response bytes so binary downloads pass through intact.
- ca.ts: self-generated CA (persisted) + on-demand per-host leaf certs
- server.ts: per-host loopback-TLS termination (Bun's node:tls can't drive a
handshake via emit("connection") or honor SNICallback, so one listening TLS
server per host is the reliable path); raw-byte capture via page.goto response
body, with the download-event path for binaries; scrape() fallback solves CF
- /proxy-ca.crt route to fetch the CA for the client's trust store
- New env: MITM_PROXY_{ENABLED,PORT,CA_DIR,MAX_TIER,DEBUG}
Off by default; localhost-only by design (a MITM proxy can impersonate any host
to a client that trusts its CA).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Splits the single [Unreleased] CHANGELOG block into dated 0.1.0-1.0.0
sections matching the milestone commits being tagged for issue #24
(numeric release tags), and bumps every package.json to 1.0.0.
germondai