- Extract HumanActionOptions type alias to replace ~40 inline copies
- Frame check/uncheck: let isChecked errors propagate instead of silently clicking non-checkbox elements
- SOCKS5 credential log: console.debug → console.info (parity with Python logger.info)
- Add contributors to README
* fix(proxy): log when SOCKS5 credential auto-encoding rewrites URL (#157)
Auto URL-encoding of SOCKS5 credentials (added in v0.3.26 to fix Chromium's
'=' truncation bug) currently happens silently. Users debugging connectivity
have no way to know the wrapper rewrote their proxy URL — the original #157
thread took 8 round-trips to surface this exact ambiguity.
Emit a log when re-encoding actually changes the URL: INFO on Python's
'cloakbrowser' logger, console.debug in JavaScript. Stays silent on
already-encoded inputs and credential-less URLs to avoid false-positive
noise. Credentials are not included in the log message.
Tests: 3 new cases per language (Python caplog, JS vi.spyOn console.debug)
covering trigger / silent-when-encoded / silent-when-no-creds.
* fix(proxy): gate log on credential change, not full URL diff
Per Copilot review on #209: urlparse cosmetically lowercases scheme and
hostname, so comparing the full reconstructed URL to the input would emit
"Auto URL-encoded SOCKS5..." even for inputs like
`socks5://USER:pass@HOST.com:1080` where no credential encoding happened.
Compare raw vs encoded user/password substrings instead. Mirror the same
condition in JS for parity (JS's manual parser preserves case today, but the
credential-level compare is more robust against future changes).
Adds one regression test per language.
The humanize layer hardcoded timeout=2000ms for element lookups, causing
locator.click() and page.click() to fail instantly instead of retrying
for 30s like standard Playwright. Aligned all defaults to 30000ms across
Python sync/async, JS Playwright, and JS Puppeteer paths. Bumped the
outer retry sleep from 200ms to 500ms for DOM mutation settle time.
* feat(lambda): cold-start hardening + handler-side retry orchestration
Two related improvements based on benchmarking the integration at scale
(3454-site sample, multiple iterations).
Cold-start hardening (lambda-entrypoint.sh + lambda_handler.py):
- Clean stale Xvfb lock file before starting the X server. We observed
that under cold-start storms, a previous Xvfb sometimes died and left
/tmp/.X99-lock + /tmp/.X11-unix/X99 behind, so the next start failed
with "Server is already active for display 99". Removing both files
makes Xvfb start cleanly every time.
- Replace `sleep 0.5` with a poll-for-X11-socket loop (up to 10s) plus
a 200ms post-socket buffer for listen()/accept() to settle. The
fixed sleep lost the race during concurrent cold inits, surfacing as
"Looks like you launched a headed browser without having a XServer
running" failures (~10% rate at 100-concurrent cold-start storm).
- Add _launch_with_retry helper in the handler: 3 attempts with linear
backoff (0.3s, 0.6s) on launch_context_async failures. Belt-and-
suspenders for whatever the entrypoint fix doesn't catch — a retry on
a now-warm container almost always succeeds.
Handler-side retry orchestration (lambda_handler.py):
- Add _classify_error() — maps Playwright errors to retry-strategy
overrides:
ERR_CERT_* -> --ignore-certificate-errors + 60s goto
Timeout exceeded -> 90s goto + 25s smart_wait cap
ERR_CONNECTION_TIMED_OUT -> same as Timeout
Returns None for unrecoverable site issues (DNS, SSL, refused, HTTP
4xx/5xx) — those bail immediately without burning a retry slot.
- Add _attempt_scrape() — extracted scrape body so the retry loop can
call it with overridden event dicts. Each attempt relaunches the
browser; uniform behavior across strategies.
- Rewrite _run() as a retry loop: first attempt uses event verbatim;
on a classifiable failure, merge the strategy's overrides into the
event and retry. Bounded by the new `retries` event field (default 1;
set to 0 to disable retry).
- Add _raise_with_history() — surfaces a final failure with a
retry_history block embedded in the error message so callers see
exactly what was tried before bailing. Successful invocations return
the standard response shape unchanged — no surprise fields.
INSTRUCTIONS.md updates:
- Bump function timeout recommendation from 60-120s to 120-180s. Under
retry, a Timeout-class first failure (30s) plus a longer-budget retry
(90s) plus cleanup can total ~120-130s; 180s leaves headroom.
- Document the new `retries` event field in the schema.
- Add a "Retry orchestration" subsection covering both layers (launch
retries and strategy retries) with the full strategy table.
Bench results on the 3454-site sample (seed=1):
v1 baseline (no fixes, c=100): 13.5% failure rate, $1.07
v2 (entrypoint Xvfb poll only, c=100): 9.9% failure rate, $1.11
v3 (cold-start fix + bench-side retry): 3.3% failure rate, $1.32
This change (handler retry, c=250): 2.1% failure rate, $1.13
The remaining 2.1% are all genuinely unrecoverable: DNS doesn't exist,
broken SSL, connection refused, 4xx/5xx responses, payload >6MB Lambda
limit. No retry logic can fix those.
* fix(lambda): merge extra_args on strategy retry instead of clobbering
A flat dict spread replaced caller-supplied extra_args (e.g.
--proxy-server=...) with the strategy's extra_args on a cert retry.
Append both lists so caller flags survive the merge.
Chromium's --proxy-server parser truncates passwords at '=' and other
special chars, causing SOCKS5 auth to silently fail and fall back to
direct connection. The dict path already encoded creds; now the string
path does too. Idempotent: pre-encoded input stays encoded.
Python: add async counterpart to launch_context(). Forwards all kwargs to
browser.new_context() — enables storage_state, permissions, extra_http_headers,
etc. without needing a persistent profile folder.
JS: launchContext() and launchPersistentContext() silently dropped unknown
options. New contextOptions field in LaunchContextOptions is spread into
newContext() to forward arbitrary Playwright context options (e.g.
storageState, permissions, geolocation).
Route SOCKS5/SOCKS5h proxies via --proxy-server Chrome arg instead of
Playwright's proxy dict (which rejects SOCKS5 with credentials).
Handles string URLs, Playwright dicts, IPv6, bypass lists.
SOCKS5 geoip exit IP resolution uses socks-proxy-agent (optional peer
dep). Falls back to DNS if not installed.
CLI args like --fingerprint-locale were passed as raw passthrough args
to Chrome, missing the companion --lang flag that build_args() normally
adds. Caused Intl API to default to en-US while navigator.language
showed the correct locale — a detectable mismatch.
Fixes#130
Node 22 ships npm v10 which lacks OIDC support. The upgrade step was
removed in 02359f6 but is required for provenance-based publishing.
Pin to npm@11 instead of @latest to avoid future breakage.
Remove --disable-blink-features=AutomationControlled (dead, binary handles
navigator.webdriver at source level) and hardcoded GPU vendor/renderer flags.
Binary auto-generates diverse GPU profiles from fingerprint seed. Improves
fingerprint diversity -- previously every user shared the same GPU string.
Bump to v0.3.21.
viewport=None now disables viewport emulation via Playwright's
no_viewport=True, matching the JS wrapper's viewport: null behavior.
Uses a sentinel to distinguish "not provided" from explicit None.
Co-authored-by: kitiho <51785099+kitiho@users.noreply.github.com>
Two ways to spoof WebRTC ICE candidate IPs:
1. --fingerprint-webrtc-ip=auto in args: resolves proxy exit IP via
HTTP call through the proxy (ipify.org). No extra deps needed.
2. geoip=True: auto-injects the flag for free (exit IP already
resolved during timezone/locale lookup, zero extra network cost).
Explicit IP (--fingerprint-webrtc-ip=1.2.3.4) also supported.
User-provided values always take precedence.
Python + JS wrappers, README docs, tests.
- Move `import websockets` to top-level (guaranteed by [serve] extra)
- Add --data-dir flag with smart default (Docker → /tmp/cloakserve, bare metal → ~/.cloakbrowser/cloakserve)
- Store launch params (tz/locale/proxy) on ChromeProcess for conflict logging
- Enhance GET / to return per-process detail (pid, port, seed, connections, config)
- Add connection refcounting in WS handlers for status visibility
- Add first-launch-wins note to README
- Add tests for data-dir, Docker detection, and connection tracking
Spawns a separate Chrome process per unique fingerprint seed, all behind
a single port (9222). Clients specify seeds and fingerprint params via
query string on the CDP URL:
connect_over_cdp("http://host:9222?fingerprint=12345&timezone=Asia/Tokyo")
Supports all --fingerprint-* flags as query params, geoip=true for
auto timezone/locale from proxy IP, and proxy= for per-process proxies.
- Rewrite bin/cloakserve from 57-line wrapper to aiohttp CDP multiplexer
- Add ChromePool with per-seed process management and port allocation
- Bidirectional WebSocket proxy for CDP traffic
- URL rewriting for /json/version, /json/list, and WS paths
- Rename _build_args -> build_args, _maybe_resolve_geoip -> maybe_resolve_geoip
- Add aiohttp + websockets to serve optional deps
- Dockerfile installs .[serve] extras
- Add 20 unit tests for cloakserve (param parsing, CLI args, URL rewriting)
CloakHQ