feat(lambda): cold-start hardening + handler-side retry orchestration (#180)

* feat(lambda): cold-start hardening + handler-side retry orchestration

Two related improvements based on benchmarking the integration at scale
(3454-site sample, multiple iterations).

Cold-start hardening (lambda-entrypoint.sh + lambda_handler.py):
  - Clean stale Xvfb lock file before starting the X server. We observed
    that under cold-start storms, a previous Xvfb sometimes died and left
    /tmp/.X99-lock + /tmp/.X11-unix/X99 behind, so the next start failed
    with "Server is already active for display 99". Removing both files
    makes Xvfb start cleanly every time.
  - Replace `sleep 0.5` with a poll-for-X11-socket loop (up to 10s) plus
    a 200ms post-socket buffer for listen()/accept() to settle. The
    fixed sleep lost the race during concurrent cold inits, surfacing as
    "Looks like you launched a headed browser without having a XServer
    running" failures (~10% rate at 100-concurrent cold-start storm).
  - Add _launch_with_retry helper in the handler: 3 attempts with linear
    backoff (0.3s, 0.6s) on launch_context_async failures. Belt-and-
    suspenders for whatever the entrypoint fix doesn't catch — a retry on
    a now-warm container almost always succeeds.

Handler-side retry orchestration (lambda_handler.py):
  - Add _classify_error() — maps Playwright errors to retry-strategy
    overrides:
      ERR_CERT_*                -> --ignore-certificate-errors + 60s goto
      Timeout exceeded          -> 90s goto + 25s smart_wait cap
      ERR_CONNECTION_TIMED_OUT  -> same as Timeout
    Returns None for unrecoverable site issues (DNS, SSL, refused, HTTP
    4xx/5xx) — those bail immediately without burning a retry slot.
  - Add _attempt_scrape() — extracted scrape body so the retry loop can
    call it with overridden event dicts. Each attempt relaunches the
    browser; uniform behavior across strategies.
  - Rewrite _run() as a retry loop: first attempt uses event verbatim;
    on a classifiable failure, merge the strategy's overrides into the
    event and retry. Bounded by the new `retries` event field (default 1;
    set to 0 to disable retry).
  - Add _raise_with_history() — surfaces a final failure with a
    retry_history block embedded in the error message so callers see
    exactly what was tried before bailing. Successful invocations return
    the standard response shape unchanged — no surprise fields.

INSTRUCTIONS.md updates:
  - Bump function timeout recommendation from 60-120s to 120-180s. Under
    retry, a Timeout-class first failure (30s) plus a longer-budget retry
    (90s) plus cleanup can total ~120-130s; 180s leaves headroom.
  - Document the new `retries` event field in the schema.
  - Add a "Retry orchestration" subsection covering both layers (launch
    retries and strategy retries) with the full strategy table.

Bench results on the 3454-site sample (seed=1):
  v1 baseline (no fixes, c=100):           13.5% failure rate, $1.07
  v2 (entrypoint Xvfb poll only, c=100):    9.9% failure rate, $1.11
  v3 (cold-start fix + bench-side retry):   3.3% failure rate, $1.32
  This change (handler retry, c=250):       2.1% failure rate, $1.13

The remaining 2.1% are all genuinely unrecoverable: DNS doesn't exist,
broken SSL, connection refused, 4xx/5xx responses, payload >6MB Lambda
limit. No retry logic can fix those.

* fix(lambda): merge extra_args on strategy retry instead of clobbering

A flat dict spread replaced caller-supplied extra_args (e.g.
--proxy-server=...) with the strategy's extra_args on a cert retry.
Append both lists so caller flags survive the merge.

examples/integrations/aws_lambda/INSTRUCTIONS.md

@@ -113,6 +113,29 @@ Only `url` is required. Everything else is optional.
 | `screenshot` | bool | `true` |
 | `full_page_screenshot` | bool | `false` |
 
+### Retry orchestration
+
+The handler retries transient navigation failures inline within the same Lambda invocation. Two layers, both built-in:
+
+- **Launch retries** — 3 attempts with 0.3 s + 0.6 s backoff. Recovers Xvfb / Chromium spawn races at cold start. Fast and cheap; not configurable.
+- **Strategy retries** — default 1 attempt, configurable via the `retries` event field. Recovers specific post-launch error classes by relaunching with adjusted Chromium args / page-load budgets.
+
+| Field | Type | Default |
+|---|---|---|
+| `retries` | int | `1` — number of strategy-retry attempts after the first failure. Set to `0` to disable retry entirely. |
+
+Strategies (priority order — first match wins):
+
+| Error pattern | Strategy applied |
+|---|---|
+| `ERR_CERT_*` (any cert error) | `extra_args: ["--ignore-certificate-errors"]`, `goto_timeout_ms: 60000` |
+| `Timeout … exceeded` | `goto_timeout_ms: 90000`, `max_settle_ms: 25000` |
+| `ERR_CONNECTION_TIMED_OUT` | same as `Timeout … exceeded` |
+
+Errors that are **not retried** (no anonymous scraper can recover): `ERR_NAME_NOT_RESOLVED`, `ERR_SSL_PROTOCOL_ERROR`, `ERR_CONNECTION_REFUSED`, `ERR_HTTP_RESPONSE_CODE_FAILURE`. These bail immediately.
+
+On final failure, the raised `RuntimeError`'s message includes a `retry_history` block listing every attempt (strategy applied + error seen). Successful invocations return the standard response shape unchanged — no surprise fields when retries didn't fire.
+
 ### Response
 
 ```json
@@ -140,7 +163,7 @@ Whatever tool you use to create the Lambda function (CLI, CDK, Terraform, SAM, c
 | Package type | Image | Required — this is a container image, not a zip. |
 | Architecture | `arm64` | Roughly 20% cheaper than x86_64. Native build on Apple Silicon. Match the architecture you built for. |
 | Memory | 3008 MB | Memory in Lambda is tied to vCPU. Below ~1769 MB Chromium starts noticeably slower. |
-| Timeout | 60–120 s | Cold start can hit 80+ s on this image; warm invocations are 3–15 s depending on site. |
+| Timeout | 120–180 s | Single-attempt scrapes complete in 3–15 s warm; under retry, a `Timeout`-class first failure (30 s default) plus a longer-budget retry (90 s) plus cleanup can total ~120-130 s. 180 s leaves headroom; below 120 s the function will time out before the retry completes. Cold-start init adds 5-10 s on top. |
 | Ephemeral storage (`/tmp`) | 1024 MB | Chromium profile dirs and screenshots can fill the 512 MB default. |
 | Networking | Default (no VPC) | Binary is baked in, no network needed at cold start. Add VPC + NAT only if your proxy egress requires it. |
 | Execution role | `AWSLambdaBasicExecutionRole` | Just CloudWatch Logs. Add more permissions only if your handler needs them. |