Frame-level methods (click, dblclick, hover, dragAndDrop) passed the raw
timeout to each sequential operation independently, causing 3x actual
wait time when elements don't exist. ElementHandle methods had a similar
2x issue between actionability and pointer-events checks.
Port the deadline + remainingMs() pattern already used by page-level
methods. Also fix bot detection test selector after site added a hidden
duplicate submit button.
Bypass Playwright's CDP Fetch.authRequired interceptor for authenticated
HTTP proxies by passing inline credentials via Chrome's --proxy-server
flag. Chrome sends Proxy-Authorization preemptively, avoiding the 407
round-trip that breaks on some proxies and Google domains (#182).
Gated on platform (linux-x64, windows-x64) and binary version >= 146.0.7680.177.5.
Unsupported platforms fall back to Playwright's proxy dict.
Puppeteer falls back to page.authenticate() on unsupported platforms.
On `docker restart`, `/tmp` is preserved across container instances, so the
previous Xvfb's `/tmp/.X99-lock` survives into the new container. The new
Xvfb sees the existing lock and refuses to start, leaving the container
with no X server. Every Chrome launch then dies with "Missing X server or
$DISPLAY", and `cloakserve` returns 502 from `/json/version` forever.
Any orchestrator that restarts on unhealthy (the README-recommended
healthcheck + `restart: always`, autoheal sidecars, etc.) then enters a
permanent restart loop because every restart hits the same broken state.
This is silent for first-time users: the container appears to start
successfully (Xvfb did launch *once*), then degrades only after the first
restart. The fix is one line in the entrypoint: remove the stale lock
before starting Xvfb.
Fixes#283
Expose userDataDir support via launchPersistentContext() for
cloakbrowser/puppeteer, matching the existing Playwright API.
Includes proxy auth, geoip, and humanize support.
Add `extension_paths` parameter to all launch functions (Python + JS) for loading Chrome extensions.
Resolves paths to absolute, injects `--load-extension` and `--disable-extensions-except` flags via `build_args()`.
Note: Extensions require a persistent context (`launch_persistent_context`) to function — this is a Chromium limitation.
Co-authored-by: zackycodes <75211659+zackycodes@users.noreply.github.com>
* feat: add flake.nix
* refactor: improve code style and add more information to flake.nix
* chore(nix): ignore build result symlink
* chore(nix): use unversioned pytest packages
Detected by Aeon + osv-scanner.
Severity: high (runtime tar) / high+moderate (dev deps)
Patches 8 of 14 CVEs flagged by osv-scanner — all that can be fixed
within current semver ranges via `npm audit fix --package-lock-only`.
The remaining 6 are gated on a puppeteer-core/vitest major-version
bump (out of scope for this PR).
Runtime (shipped to users):
- tar 7.5.9 -> 7.5.15
- GHSA-9ppj-qmqm-q256 HIGH: Symlink Path Traversal via Drive-Relative Linkpath
- GHSA-qffp-2rhf-9h96 HIGH: Hardlink Path Traversal via Drive-Relative Linkpath
- Reachable in js/src/download.ts (extractTar) — the existing filter() rejects
absolute paths and "..", but does not inspect linkpath, so a malicious
Chromium tarball could write outside the cache dir on Windows.
Dev (build-time only):
- basic-ftp 5.2.0 -> 5.3.1 (4 HIGH: CRLF injection x2, DoS x2)
- ip-address 10.1.0 -> 10.2.0 (1 MOD: XSS in Address6 HTML methods)
- postcss 8.5.6 -> 8.5.14 (1 MOD: XSS via unescaped </style>)
Lockfile metadata side-effects (npm-regenerated, not editorial):
- name@version block synced from package.json (0.3.23 -> 0.3.28)
- devDependencies + peerDependencies version ranges synced to current
package.json (the lockfile was stale relative to head package.json)
Verification:
- `npm test` -> 320 passed / 11 skipped / 0 failed (9 test files)
- `npm run typecheck` -> clean
- osv-scanner before: 14 CVEs; after: 6 (those 6 need a breaking
major-version bump to land — happy to follow up if you want it)
Co-authored-by: Aeon <aeon@aaronjmars.eth>
Remove dead null checks, document CLOAKBROWSER_GEOIP_TIMEOUT_SECONDS
env var, credit contributor.
Fix review findings:
- Use timeout-bounded resolve_proxy_exit_ip in _resolve_webrtc_args
- Add missing timeout handler on tunneled HTTPS request in JS
- Reject nan/inf in Python timeout parsing (parity with JS)
- Recompute deadline after CONNECT succeeds in JS proxy tunnel
- Extract HumanActionOptions type alias to replace ~40 inline copies
- Frame check/uncheck: let isChecked errors propagate instead of silently clicking non-checkbox elements
- SOCKS5 credential log: console.debug → console.info (parity with Python logger.info)
- Add contributors to README
* fix(proxy): log when SOCKS5 credential auto-encoding rewrites URL (#157)
Auto URL-encoding of SOCKS5 credentials (added in v0.3.26 to fix Chromium's
'=' truncation bug) currently happens silently. Users debugging connectivity
have no way to know the wrapper rewrote their proxy URL — the original #157
thread took 8 round-trips to surface this exact ambiguity.
Emit a log when re-encoding actually changes the URL: INFO on Python's
'cloakbrowser' logger, console.debug in JavaScript. Stays silent on
already-encoded inputs and credential-less URLs to avoid false-positive
noise. Credentials are not included in the log message.
Tests: 3 new cases per language (Python caplog, JS vi.spyOn console.debug)
covering trigger / silent-when-encoded / silent-when-no-creds.
* fix(proxy): gate log on credential change, not full URL diff
Per Copilot review on #209: urlparse cosmetically lowercases scheme and
hostname, so comparing the full reconstructed URL to the input would emit
"Auto URL-encoded SOCKS5..." even for inputs like
`socks5://USER:pass@HOST.com:1080` where no credential encoding happened.
Compare raw vs encoded user/password substrings instead. Mirror the same
condition in JS for parity (JS's manual parser preserves case today, but the
credential-level compare is more robust against future changes).
Adds one regression test per language.
The humanize layer hardcoded timeout=2000ms for element lookups, causing
locator.click() and page.click() to fail instantly instead of retrying
for 30s like standard Playwright. Aligned all defaults to 30000ms across
Python sync/async, JS Playwright, and JS Puppeteer paths. Bumped the
outer retry sleep from 200ms to 500ms for DOM mutation settle time.
* feat(lambda): cold-start hardening + handler-side retry orchestration
Two related improvements based on benchmarking the integration at scale
(3454-site sample, multiple iterations).
Cold-start hardening (lambda-entrypoint.sh + lambda_handler.py):
- Clean stale Xvfb lock file before starting the X server. We observed
that under cold-start storms, a previous Xvfb sometimes died and left
/tmp/.X99-lock + /tmp/.X11-unix/X99 behind, so the next start failed
with "Server is already active for display 99". Removing both files
makes Xvfb start cleanly every time.
- Replace `sleep 0.5` with a poll-for-X11-socket loop (up to 10s) plus
a 200ms post-socket buffer for listen()/accept() to settle. The
fixed sleep lost the race during concurrent cold inits, surfacing as
"Looks like you launched a headed browser without having a XServer
running" failures (~10% rate at 100-concurrent cold-start storm).
- Add _launch_with_retry helper in the handler: 3 attempts with linear
backoff (0.3s, 0.6s) on launch_context_async failures. Belt-and-
suspenders for whatever the entrypoint fix doesn't catch — a retry on
a now-warm container almost always succeeds.
Handler-side retry orchestration (lambda_handler.py):
- Add _classify_error() — maps Playwright errors to retry-strategy
overrides:
ERR_CERT_* -> --ignore-certificate-errors + 60s goto
Timeout exceeded -> 90s goto + 25s smart_wait cap
ERR_CONNECTION_TIMED_OUT -> same as Timeout
Returns None for unrecoverable site issues (DNS, SSL, refused, HTTP
4xx/5xx) — those bail immediately without burning a retry slot.
- Add _attempt_scrape() — extracted scrape body so the retry loop can
call it with overridden event dicts. Each attempt relaunches the
browser; uniform behavior across strategies.
- Rewrite _run() as a retry loop: first attempt uses event verbatim;
on a classifiable failure, merge the strategy's overrides into the
event and retry. Bounded by the new `retries` event field (default 1;
set to 0 to disable retry).
- Add _raise_with_history() — surfaces a final failure with a
retry_history block embedded in the error message so callers see
exactly what was tried before bailing. Successful invocations return
the standard response shape unchanged — no surprise fields.
INSTRUCTIONS.md updates:
- Bump function timeout recommendation from 60-120s to 120-180s. Under
retry, a Timeout-class first failure (30s) plus a longer-budget retry
(90s) plus cleanup can total ~120-130s; 180s leaves headroom.
- Document the new `retries` event field in the schema.
- Add a "Retry orchestration" subsection covering both layers (launch
retries and strategy retries) with the full strategy table.
Bench results on the 3454-site sample (seed=1):
v1 baseline (no fixes, c=100): 13.5% failure rate, $1.07
v2 (entrypoint Xvfb poll only, c=100): 9.9% failure rate, $1.11
v3 (cold-start fix + bench-side retry): 3.3% failure rate, $1.32
This change (handler retry, c=250): 2.1% failure rate, $1.13
The remaining 2.1% are all genuinely unrecoverable: DNS doesn't exist,
broken SSL, connection refused, 4xx/5xx responses, payload >6MB Lambda
limit. No retry logic can fix those.
* fix(lambda): merge extra_args on strategy retry instead of clobbering
A flat dict spread replaced caller-supplied extra_args (e.g.
--proxy-server=...) with the strategy's extra_args on a cert retry.
Append both lists so caller flags survive the merge.
Chromium's --proxy-server parser truncates passwords at '=' and other
special chars, causing SOCKS5 auth to silently fail and fall back to
direct connection. The dict path already encoded creds; now the string
path does too. Idempotent: pre-encoded input stays encoded.
Python: add async counterpart to launch_context(). Forwards all kwargs to
browser.new_context() — enables storage_state, permissions, extra_http_headers,
etc. without needing a persistent profile folder.
JS: launchContext() and launchPersistentContext() silently dropped unknown
options. New contextOptions field in LaunchContextOptions is spread into
newContext() to forward arbitrary Playwright context options (e.g.
storageState, permissions, geolocation).
