Compare commits

..
13 Commits
Author SHA1 Message Date
CloakHQ 0d6ce76b1d release: v0.3.27 — per-call human_config, humanize timeout fix, scrollIntoViewIfNeeded 2026-05-06 18:32:41 +02:00
CloakHQ f01902025a fix: align humanize timeout default with Playwright's 30s auto-retry (#172)
The humanize layer hardcoded timeout=2000ms for element lookups, causing
locator.click() and page.click() to fail instantly instead of retrying
for 30s like standard Playwright. Aligned all defaults to 30000ms across
Python sync/async, JS Playwright, and JS Puppeteer paths. Bumped the
outer retry sleep from 200ms to 500ms for DOM mutation settle time.
2026-05-01 20:48:06 +02:00
CloakHQ 2df8c7e2d1 fix(js): correct issue references #137#172 in humanize comments 2026-04-28 20:37:09 +02:00
lilosandGitHub 661b873dad feat: per-call human_config, timeout forwarding, humanized scrollIntoViewIfNeeded (#183) 2026-04-28 20:34:17 +02:00
CloakHQ ee346a6a57 release: v0.3.26 — Windows x64 upgraded to Chromium 146, SOCKS5 credential encoding, Lambda integration 2026-04-28 05:38:06 +02:00
CloakHQ 3e699f554c fix(docker): add emoji and extended font packages to resolve Kasada/Akamai canvas blocks (#179)
Dockerfile: add fonts-noto-color-emoji, fonts-freefont-ttf, fonts-unifont,
fonts-ipafont-gothic, fonts-wqy-zenhei, fonts-tlwg-loma-otf.
README: separate anti-bot font fix (apt packages) from CreepJS font
enumeration (Windows fonts + --fingerprint-fonts-dir).
2026-04-28 04:11:19 +02:00
Alex StepanskyandGitHub 9eb90da012 feat(lambda): cold-start hardening + handler-side retry orchestration (#180)
* feat(lambda): cold-start hardening + handler-side retry orchestration

Two related improvements based on benchmarking the integration at scale
(3454-site sample, multiple iterations).

Cold-start hardening (lambda-entrypoint.sh + lambda_handler.py):
  - Clean stale Xvfb lock file before starting the X server. We observed
    that under cold-start storms, a previous Xvfb sometimes died and left
    /tmp/.X99-lock + /tmp/.X11-unix/X99 behind, so the next start failed
    with "Server is already active for display 99". Removing both files
    makes Xvfb start cleanly every time.
  - Replace `sleep 0.5` with a poll-for-X11-socket loop (up to 10s) plus
    a 200ms post-socket buffer for listen()/accept() to settle. The
    fixed sleep lost the race during concurrent cold inits, surfacing as
    "Looks like you launched a headed browser without having a XServer
    running" failures (~10% rate at 100-concurrent cold-start storm).
  - Add _launch_with_retry helper in the handler: 3 attempts with linear
    backoff (0.3s, 0.6s) on launch_context_async failures. Belt-and-
    suspenders for whatever the entrypoint fix doesn't catch — a retry on
    a now-warm container almost always succeeds.

Handler-side retry orchestration (lambda_handler.py):
  - Add _classify_error() — maps Playwright errors to retry-strategy
    overrides:
      ERR_CERT_*                -> --ignore-certificate-errors + 60s goto
      Timeout exceeded          -> 90s goto + 25s smart_wait cap
      ERR_CONNECTION_TIMED_OUT  -> same as Timeout
    Returns None for unrecoverable site issues (DNS, SSL, refused, HTTP
    4xx/5xx) — those bail immediately without burning a retry slot.
  - Add _attempt_scrape() — extracted scrape body so the retry loop can
    call it with overridden event dicts. Each attempt relaunches the
    browser; uniform behavior across strategies.
  - Rewrite _run() as a retry loop: first attempt uses event verbatim;
    on a classifiable failure, merge the strategy's overrides into the
    event and retry. Bounded by the new `retries` event field (default 1;
    set to 0 to disable retry).
  - Add _raise_with_history() — surfaces a final failure with a
    retry_history block embedded in the error message so callers see
    exactly what was tried before bailing. Successful invocations return
    the standard response shape unchanged — no surprise fields.

INSTRUCTIONS.md updates:
  - Bump function timeout recommendation from 60-120s to 120-180s. Under
    retry, a Timeout-class first failure (30s) plus a longer-budget retry
    (90s) plus cleanup can total ~120-130s; 180s leaves headroom.
  - Document the new `retries` event field in the schema.
  - Add a "Retry orchestration" subsection covering both layers (launch
    retries and strategy retries) with the full strategy table.

Bench results on the 3454-site sample (seed=1):
  v1 baseline (no fixes, c=100):           13.5% failure rate, $1.07
  v2 (entrypoint Xvfb poll only, c=100):    9.9% failure rate, $1.11
  v3 (cold-start fix + bench-side retry):   3.3% failure rate, $1.32
  This change (handler retry, c=250):       2.1% failure rate, $1.13

The remaining 2.1% are all genuinely unrecoverable: DNS doesn't exist,
broken SSL, connection refused, 4xx/5xx responses, payload >6MB Lambda
limit. No retry logic can fix those.

* fix(lambda): merge extra_args on strategy retry instead of clobbering

A flat dict spread replaced caller-supplied extra_args (e.g.
--proxy-server=...) with the strategy's extra_args on a cert retry.
Append both lists so caller flags survive the merge.
2026-04-28 03:33:04 +02:00
CloakHQ 6b8d8b6378 docs: add Font Setup on Linux section to README (#179) 2026-04-28 00:23:08 +02:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
252e79b17d chore(deps): bump the actions group across 1 directory with 3 updates (#178)
Bumps the actions group with 3 updates in the / directory: [actions/setup-node](https://github.com/actions/setup-node), [pypa/gh-action-pypi-publish](https://github.com/pypa/gh-action-pypi-publish) and [docker/build-push-action](https://github.com/docker/build-push-action).


Updates `actions/setup-node` from 6.3.0 to 6.4.0
- [Release notes](https://github.com/actions/setup-node/releases)
- [Commits](https://github.com/actions/setup-node/compare/53b83947a5a98c8d113130e565377fae1a50d02f...48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e)

Updates `pypa/gh-action-pypi-publish` from 1.13.0 to 1.14.0
- [Release notes](https://github.com/pypa/gh-action-pypi-publish/releases)
- [Commits](https://github.com/pypa/gh-action-pypi-publish/compare/ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e...cef221092ed1bacb1cc03d23a2d87d1d172e277b)

Updates `docker/build-push-action` from 7.0.0 to 7.1.0
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](https://github.com/docker/build-push-action/compare/d08e5c354a6adb9ed34480a06d141179aa583294...bcafcacb16a39f128d818304e6c9c0c18556b85f)

---
updated-dependencies:
- dependency-name: actions/setup-node
  dependency-version: 6.4.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: actions
- dependency-name: pypa/gh-action-pypi-publish
  dependency-version: 1.14.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: actions
- dependency-name: docker/build-push-action
  dependency-version: 7.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-27 17:32:05 +02:00
CloakHQ 0ccdc71e47 docs: add @AlexTech314 to contributors, add Deployment Integrations section (#177) 2026-04-27 17:23:24 +02:00
Alex StepanskyandGitHub 74b1ff64db feat(lambda): add AWS Lambda integration in examples/integrations/aws_lambda/ (#177)
feat(lambda): add AWS Lambda one-shot scrape integration

Self-contained example in examples/integrations/aws_lambda/ — Dockerfile,
entrypoint, handler, and docs for running CloakBrowser stealth scrapes in
AWS Lambda (container image). Includes smart_wait DOM-stability polling,
Xvfb headed mode, and Lambda-specific Chromium flags.

Contributed by @AlexTech314.
2026-04-27 17:20:02 +02:00
CloakHQ a9a0ba13ba fix(proxy): auto URL-encode SOCKS5 credentials in string URLs (#157)
Chromium's --proxy-server parser truncates passwords at '=' and other
special chars, causing SOCKS5 auth to silently fail and fall back to
direct connection. The dict path already encoded creds; now the string
path does too. Idempotent: pre-encoded input stays encoded.
2026-04-25 23:01:16 +02:00
CloakHQ b04ad6ec2a docs: credit @eofreternal for humanConfig type fix (#151) 2026-04-16 23:12:57 +02:00
34 changed files with 2536 additions and 276 deletions
+1 -1
View File
@@ -23,7 +23,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 20
- name: Install and build
+4 -4
View File
@@ -32,7 +32,7 @@ jobs:
run: |
pip install -e ".[dev]" pytest pytest-asyncio
pytest tests/ -v -m "not slow"
- uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 22
- name: JavaScript tests
@@ -71,7 +71,7 @@ jobs:
pip install build
python -m build
- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e # v1
uses: pypa/gh-action-pypi-publish@cef221092ed1bacb1cc03d23a2d87d1d172e277b # v1
publish-npm:
needs: [test, validate-version]
@@ -81,7 +81,7 @@ jobs:
id-token: write # OIDC trusted publishing + provenance — no NPM_TOKEN needed
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 24 # npm 11.11.0 native — no upgrade needed (Node 22.22.2 has broken npm)
registry-url: 'https://registry.npmjs.org'
@@ -113,7 +113,7 @@ jobs:
password: ${{ secrets.DOCKER_PAT }}
- name: Build and push
id: build
uses: docker/build-push-action@d08e5c354a6adb9ed34480a06d141179aa583294 # v7.0.0
uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
with:
context: .
platforms: linux/amd64,linux/arm64
+17
View File
@@ -8,6 +8,23 @@ Changes are tagged: **[wrapper]** for Python/JS wrapper, **[binary]** for Chromi
## [Unreleased]
## [0.3.27] — 2026-05-06
- **[wrapper]** Per-call `human_config` override — pass `human_config={...}` to individual humanized methods to override global HumanConfig on a per-action basis (#183)
- **[wrapper]** Humanized `scrollIntoViewIfNeeded` — auto-scrolls with human-like behavior when `humanize=True` (#183)
- **[wrapper]** Forward `timeout` parameter through humanized Playwright methods (#183)
- **[wrapper]** Fix humanize timeout default to align with Playwright's 30s auto-retry instead of custom 2s (#172)
## [0.3.26] — 2026-04-28
- **[binary]** Windows x64 upgraded to Chromium 146.0.7680.177.4 — 57 source-level fingerprint patches (up from 33 on 145.0.7632.159.7), now matches Linux. Includes all binary improvements from 0.3.180.3.25: native SOCKS5 proxy with UDP ASSOCIATE (QUIC/HTTP3), WebRTC IP spoofing, proxy signal removal, CDP input stealth, storage quota normalization, WebAuthn/AAC/window position patches, WebGL and canvas consistency fixes, expanded GPU model database
- **[wrapper]** Auto URL-encode SOCKS5 credentials containing special characters in string URLs (#157)
- **[wrapper]** AWS Lambda integration example with cold-start hardening and handler-side retry orchestration (#177, thanks [@AlexTech314](https://github.com/AlexTech314))
- **[docker]** Add emoji and extended font packages to resolve Kasada/Akamai canvas fingerprint blocks (#179)
- **[docs]** Add Font Setup on Linux section to README (#179)
- **[docs]** Add Deployment Integrations section to README (#177)
- **[meta]** Bump GitHub Actions dependencies (#178)
## [0.3.25] — 2026-04-16
- **[wrapper]** Python: add `launch_context_async()` — async counterpart to `launch_context()`. Returns a BrowserContext with all kwargs forwarded to `browser.new_context()`, enabling `storage_state`, `permissions`, `extra_http_headers`, etc. without a persistent profile folder. Closes #141.
+2
View File
@@ -9,6 +9,8 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
libxcb1 libxext6 libxshmfence1 \
libglib2.0-0 libgtk-3-0 libpangocairo-1.0-0 libcairo-gobject2 \
libgdk-pixbuf-2.0-0 libxss1 libxtst6 fonts-liberation \
fonts-noto-color-emoji fonts-unifont fonts-freefont-ttf \
fonts-ipafont-gothic fonts-wqy-zenhei fonts-tlwg-loma-otf \
xvfb xdotool \
curl ca-certificates \
&& curl -fsSL https://deb.nodesource.com/setup_20.x | bash - \
+50 -8
View File
@@ -128,7 +128,7 @@ Open [http://localhost:8080](http://localhost:8080). Create a profile. Click **L
---
## Latest: v0.3.25 (Chromium 146.0.7680.177.3)
## Latest: v0.3.26 (Chromium 146.0.7680.177.4)
- **`launch_context_async()`** — async counterpart to `launch_context()`. Forwards kwargs to `browser.new_context()` for `storage_state`, `permissions`, `extra_http_headers` without a persistent profile folder.
- **JS `contextOptions` escape hatch** — forward arbitrary options (including `storageState`) to Playwright's `newContext()` from `launchContext()` / `launchPersistentContext()`.
@@ -626,13 +626,39 @@ Supported by the binary but **not set by default** — pass via `args` to custom
| `--fingerprint-locale` | Locale (e.g. `en-US`) |
| `--fingerprint-storage-quota` | Override storage quota in MB — affects `storage.estimate()`, `storageBuckets`, and legacy webkit APIs. Auto-normalized when `--fingerprint` is set |
| `--fingerprint-taskbar-height` | Override taskbar height (binary defaults: Win=48, Mac=95, Linux=0) |
| `--fingerprint-fonts-dir` | Path to cross-platform font directory |
| `--fingerprint-fonts-dir` | Path to directory containing target-platform fonts (see [Font Setup on Linux](#font-setup-on-linux)) |
| `--fingerprint-webrtc-ip` | WebRTC ICE candidate IP replacement. Use `auto` to resolve from proxy exit IP (makes an HTTP call through the proxy), or pass an explicit IP. Auto-injected when `geoip=True` |
| `--fingerprint-noise=false` | Disable noise injection (canvas, WebGL, audio, client rects) while keeping the deterministic fingerprint seed active |
| `--enable-blink-features=FakeShadowRoot` | Access closed shadow DOM elements |
> **Note:** All stealth tests were verified with the default fingerprint config above. Changing these flags may affect detection results — test your configuration before using in production.
### Font Setup on Linux
**Required for aggressive anti-bot sites (Kasada, Akamai).** These systems render emoji on a hidden canvas and hash the pixel output. Minimal Linux environments (Docker, cloud VMs) often lack emoji and extended fonts, producing hashes that don't match any real browser. Install standard font packages to fix this:
```bash
sudo apt install -y fonts-noto-color-emoji fonts-freefont-ttf fonts-unifont \
fonts-ipafont-gothic fonts-wqy-zenhei fonts-tlwg-loma-otf
```
The Docker image (`cloakhq/cloakbrowser`) ships with these pre-installed. If you run the binary directly on a Linux server or in a custom Docker image, install them manually.
**Optional: Windows fonts for CreepJS font enumeration.** The packages above fix anti-bot canvas checks but won't improve your CreepJS font score. For that, you need actual Windows fonts (Segoe UI, Calibri, Bahnschrift, etc.) from a Windows machine's `C:\Windows\Fonts\` directory — `ttf-mscorefonts-installer` only has old XP-era fonts and isn't enough.
```bash
mkdir -p ~/.local/share/fonts/windows
cp /path/to/windows/fonts/*.ttf ~/.local/share/fonts/windows/
cp /path/to/windows/fonts/*.TTF ~/.local/share/fonts/windows/
fc-cache -f # mandatory for manually copied fonts
```
```python
browser = launch(
args=["--fingerprint-fonts-dir=/home/user/.local/share/fonts/windows"],
)
```
### Examples
```python
@@ -703,15 +729,21 @@ browser = await launch_async(args=["--remote-debugging-port=9242"])
| [undetected-chromedriver](https://github.com/ultrafunkamsterdam/undetected-chromedriver) | 12K | Python | [`undetected_chromedriver.py`](examples/integrations/undetected_chromedriver.py) |
| [agent-browser](https://github.com/nichochar/agent-browser) | — | Shell | [`agent_browser.sh`](examples/integrations/agent_browser.sh) |
### Deployment Integrations
| Platform | Example |
|----------|---------|
| [AWS Lambda](https://aws.amazon.com/lambda/) | [`aws_lambda/`](examples/integrations/aws_lambda/) — One-shot scrapes in Lambda (container image) |
## Platforms
| Platform | Chromium | Patches | Status |
|---|---|---|---|
| Linux x86_64 | 146 | 49 | ✅ Latest |
| Linux arm64 (RPi, Graviton) | 146 | 49 | ✅ Latest |
| Linux x86_64 | 146 | 57 | ✅ Latest |
| Linux arm64 (RPi, Graviton) | 146 | 57 | ✅ Latest |
| macOS arm64 (Apple Silicon) | 145 | 26 | ✅ |
| macOS x86_64 (Intel) | 145 | 26 | ✅ |
| Windows x86_64 | 145 | 48 | ✅ |
| Windows x86_64 | 146 | 57 | ✅ Latest |
The wrapper auto-downloads the correct binary for your platform.
@@ -940,7 +972,15 @@ If your proxy supports SOCKS5, use it for better compatibility — SOCKS5 tunnel
browser = launch(proxy="socks5://user:pass@proxy:1080", geoip=True, headless=False, humanize=True)
```
If you're still blocked after this, the issue is almost always IP reputation — try a different proxy region or test from a home ISP to confirm the browser itself is clean.
If you're still blocked after this, check the font setup below.
---
### Blocked on Kasada / Akamai sites despite correct config?
On minimal Linux environments, missing font packages cause canvas emoji rendering to produce hashes that anti-bot systems don't recognize. This is the most common cause of blocks on aggressive sites after proxy, geoip, and headed mode are already set up correctly.
Install the font packages listed in [Font Setup on Linux](#font-setup-on-linux) above.
---
@@ -1102,9 +1142,9 @@ A: Yes. Pass `proxy="http://user:pass@host:port"` or `proxy="socks5://user:pass@
| Feature | Status |
|---------|--------|
| Linux x64 — Chromium 146 (49 patches) | ✅ Released |
| Linux x64 — Chromium 146 (57 patches) | ✅ Released |
| macOS arm64/x64 — Chromium 145 (26 patches) | ✅ Released |
| Windows x64 — Chromium 145 (33 patches) | ✅ Released |
| Windows x64 — Chromium 146 (57 patches) | ✅ Released |
| JavaScript/Puppeteer + Playwright support | ✅ Released |
| Fingerprint rotation per session | ✅ Released |
| Built-in proxy rotation | 📋 Planned |
@@ -1152,3 +1192,5 @@ Issues and PRs welcome. If something isn't working, [open an issue](https://gith
- [@evelaa123](https://github.com/evelaa123) — humanize behavior, persistent contexts, Windows fix
- [@yahooguntu](https://github.com/yahooguntu) — persistent contexts
- [@kitiho](https://github.com/kitiho) — null viewport fix
- [@eofreternal](https://github.com/eofreternal) — humanConfig type fix
- [@AlexTech314](https://github.com/AlexTech314) — AWS Lambda integration
+2 -2
View File
@@ -36,7 +36,7 @@ import aiohttp
import websockets
from aiohttp import web
from cloakbrowser.browser import build_args, maybe_resolve_geoip, _resolve_webrtc_args
from cloakbrowser.browser import build_args, maybe_resolve_geoip, _resolve_webrtc_args, _normalize_socks_string_url
from cloakbrowser.download import ensure_binary
logging.basicConfig(
@@ -189,7 +189,7 @@ class ChromePool:
if extra_args:
fp_extra.extend(extra_args)
if proxy:
fp_extra.append(f"--proxy-server={proxy}")
fp_extra.append(f"--proxy-server={_normalize_socks_string_url(proxy)}")
# WebRTC IP spoofing: resolve auto, inject geoip exit IP
fp_extra = _resolve_webrtc_args(fp_extra, proxy)
+1 -1
View File
@@ -1 +1 @@
__version__ = "0.3.25"
__version__ = "0.3.27"
+80 -14
View File
@@ -760,6 +760,37 @@ def _ensure_proxy_scheme(proxy_url: str) -> str:
return proxy_url if "://" in proxy_url else f"http://{proxy_url}"
def _assemble_socks_url(
scheme: str,
host: str,
port: int | None,
enc_user: str,
enc_pass: str | None,
path: str = "",
params: str = "",
query: str = "",
fragment: str = "",
) -> str:
"""Build a SOCKS URL from already-percent-encoded credentials and host parts.
``enc_pass is None`` means no password (no colon in userinfo). Empty string
means present-but-empty (colon preserved). This mirrors the distinction
urlparse makes between ``user@host`` and ``user:@host``.
"""
if ":" in host: # IPv6 literal — re-add brackets
host = f"[{host}]"
if enc_pass is not None:
userinfo = f"{enc_user}:{enc_pass}@"
elif enc_user:
userinfo = f"{enc_user}@"
else:
userinfo = ""
netloc = f"{userinfo}{host}"
if port is not None:
netloc += f":{port}"
return urlunparse((scheme, netloc, path, params, query, fragment))
def _reconstruct_socks_url(proxy: ProxySettings) -> str:
"""Reconstruct a SOCKS5 URL with inline credentials from a Playwright proxy dict."""
server = proxy.get("server", "")
@@ -768,16 +799,47 @@ def _reconstruct_socks_url(proxy: ProxySettings) -> str:
if not username:
return server
parsed = urlparse(server)
creds = quote(username, safe="")
if password:
creds += f":{quote(password, safe='')}"
host = parsed.hostname or ""
if ":" in host: # IPv6 literal — re-add brackets
host = f"[{host}]"
netloc = f"{creds}@{host}"
if parsed.port:
netloc += f":{parsed.port}"
return urlunparse((parsed.scheme, netloc, parsed.path, "", "", ""))
enc_user = quote(username, safe="")
# Dict convention: empty/missing password → no colon.
enc_pass = quote(password, safe="") if password else None
return _assemble_socks_url(
parsed.scheme, parsed.hostname or "", parsed.port,
enc_user, enc_pass, parsed.path,
)
def _normalize_socks_string_url(url: str) -> str:
"""Re-encode credentials in a SOCKS5 URL string so Chromium's parser doesn't
truncate them at special chars like '='. Idempotent: pre-encoded input stays
the same (decoded then re-encoded).
On unparseable input (invalid port, broken IPv6 literal, etc.) logs a
warning and returns the original string — preserves pre-fix pass-through
behavior so Chromium's own error handling kicks in.
"""
try:
parsed = urlparse(url)
# Accessing .port raises ValueError on invalid port strings.
_ = parsed.port
except ValueError as e:
logger.warning("Malformed SOCKS5 proxy URL, passing through unchanged: %s", e)
return url
# Skip only if no credentials at all (username AND password both absent).
# urlparse returns None for absent components, "" for present-but-empty.
if parsed.username is None and parsed.password is None:
return url
enc_user = quote(unquote(parsed.username), safe="") if parsed.username else ""
# Preserve the colon separator when password component is present, even if
# empty, so `user:@host` stays `user:@host`.
if parsed.password is not None:
enc_pass = quote(unquote(parsed.password), safe="") if parsed.password else ""
else:
enc_pass = None
return _assemble_socks_url(
parsed.scheme, parsed.hostname or "", parsed.port,
enc_user, enc_pass,
parsed.path, parsed.params, parsed.query, parsed.fragment,
)
def _extract_proxy_url(proxy: str | ProxySettings | None) -> str | None:
@@ -926,11 +988,14 @@ def build_args(
def _parse_proxy_url(proxy: str) -> dict[str, Any]:
"""Parse proxy URL, extracting credentials into separate Playwright fields.
"""Parse HTTP(S) proxy URL, extracting credentials into separate Playwright fields.
Handles: http://user:pass@host:port -> {server: "http://host:port", username: "user", password: "pass"}
Also handles: no credentials, URL-encoded special chars, socks5://, missing port,
Also handles: no credentials, URL-encoded special chars, missing port,
and bare proxy strings without a scheme (e.g. 'user:pass@host:port' -> treated as http).
SOCKS5 URLs are NOT handled here — they take a dedicated path via
``_normalize_socks_string_url`` in ``_resolve_proxy_config``.
"""
# Bare format: "user:pass@host:port" — urlparse needs a scheme to extract credentials.
normalized = proxy
@@ -988,8 +1053,9 @@ def _resolve_proxy_config(
if proxy.get("bypass"):
extra_args.append(f"--proxy-bypass-list={proxy['bypass']}")
return {}, extra_args
# String URL — pass as-is (Chrome handles user:pass@ in the URL)
return {}, [f"--proxy-server={proxy}"]
# String URL — re-encode creds to work around Chromium parser truncating
# passwords at '=' and other special chars (#157).
return {}, [f"--proxy-server={_normalize_socks_string_url(proxy)}"]
# HTTP/HTTPS: use Playwright's proxy dict as before
if isinstance(proxy, dict):
+1 -1
View File
@@ -22,7 +22,7 @@ PLATFORM_CHROMIUM_VERSIONS: dict[str, str] = {
"linux-arm64": "146.0.7680.177.3",
"darwin-arm64": "145.0.7632.109.2",
"darwin-x64": "145.0.7632.109.2",
"windows-x64": "145.0.7632.159.7",
"windows-x64": "146.0.7680.177.4",
}
# ---------------------------------------------------------------------------
-1
View File
@@ -60,7 +60,6 @@ def _show_welcome() -> None:
sys.stderr.write(" CloakBrowser — stealth Chromium for automation\n")
sys.stderr.write(" https://github.com/CloakHQ/CloakBrowser\n")
sys.stderr.write("\n")
sys.stderr.write(" Issues? https://github.com/CloakHQ/CloakBrowser/issues\n")
sys.stderr.write(" Donate? https://ko-fi.com/cloakhq\n")
sys.stderr.write(" Star us if CloakBrowser helps your project!\n")
sys.stderr.write("\n")
+304 -95
View File
@@ -18,23 +18,23 @@ import logging
import sys
from typing import Any, Optional
from .config import HumanConfig, HumanPreset, resolve_config
from .config import HumanConfig, HumanPreset, resolve_config, merge_config
from .config import rand, rand_range, sleep_ms, async_sleep_ms
from .mouse import RawMouse, human_move, human_click, click_target, human_idle
from .keyboard import RawKeyboard, human_type
from .scroll import scroll_to_element
from .scroll import scroll_to_element, human_scroll_into_view
from .mouse_async import AsyncRawMouse, async_human_move, async_human_click, async_human_idle
from .keyboard_async import AsyncRawKeyboard, async_human_type
from .scroll_async import async_scroll_to_element
from .scroll_async import async_scroll_to_element, async_human_scroll_into_view
_SELECT_ALL = "Meta+a" if sys.platform == "darwin" else "Control+a"
__all__ = [
"patch_browser", "patch_context", "patch_page",
"patch_browser_async", "patch_context_async", "patch_page_async",
"HumanConfig", "resolve_config",
"HumanConfig", "resolve_config", "merge_config",
"human_move", "human_click", "click_target", "human_idle",
"human_type", "scroll_to_element",
"human_type", "scroll_to_element", "human_scroll_into_view",
]
logger = logging.getLogger("cloakbrowser.human")
@@ -356,6 +356,7 @@ def _patch_locator_class_sync():
_orig_tap = Locator.tap
_orig_drag_to = Locator.drag_to
_orig_clear = Locator.clear
_orig_scroll_into_view = getattr(Locator, 'scroll_into_view_if_needed', None)
def _get_selector(self):
return self._impl_obj._selector
@@ -366,36 +367,76 @@ def _patch_locator_class_sync():
def _get_cfg(self):
return getattr(self.page, '_human_cfg', None)
# Forward only options the page-level humanized methods understand
# (timeout, human_config). Other Locator-specific kwargs (force, trial,
# noWaitAfter, ...) are silently dropped — the humanized path doesn't
# consult them.
def _forward_kwargs(kwargs):
out = {}
if "timeout" in kwargs:
out["timeout"] = kwargs["timeout"]
if "human_config" in kwargs:
out["human_config"] = kwargs["human_config"]
return out
def _humanized_fill(self, value, **kwargs):
if _is_humanized(self):
self.page.fill(_get_selector(self), value)
self.page.fill(_get_selector(self), value, **_forward_kwargs(kwargs))
else:
_orig_fill(self, value, **kwargs)
def _humanized_click(self, **kwargs):
if _is_humanized(self):
self.page.click(_get_selector(self))
self.page.click(_get_selector(self), **_forward_kwargs(kwargs))
else:
_orig_click(self, **kwargs)
def _humanized_type(self, text, **kwargs):
if _is_humanized(self):
self.page.type(_get_selector(self), text)
self.page.type(_get_selector(self), text, **_forward_kwargs(kwargs))
else:
_orig_type(self, text, **kwargs)
def _humanized_dblclick(self, **kwargs):
if _is_humanized(self):
self.page.dblclick(_get_selector(self))
self.page.dblclick(_get_selector(self), **_forward_kwargs(kwargs))
else:
_orig_dblclick(self, **kwargs)
def _humanized_hover(self, **kwargs):
if _is_humanized(self):
self.page.hover(_get_selector(self))
self.page.hover(_get_selector(self), **_forward_kwargs(kwargs))
else:
_orig_hover(self, **kwargs)
def _humanized_scroll_into_view_if_needed(self, **kwargs):
if _is_humanized(self):
page = self.page
cfg = _get_cfg(self)
cursor = getattr(page, '_human_cursor', None)
raw = getattr(page, '_human_raw_mouse', None)
call_cfg = merge_config(cfg, kwargs.get("human_config")) if cfg else None
if call_cfg is None or cursor is None or raw is None:
if _orig_scroll_into_view is not None:
native_kwargs = {k: v for k, v in kwargs.items() if k != "human_config"}
return _orig_scroll_into_view(self, **native_kwargs)
return
timeout = kwargs.get("timeout", 30000)
try:
_, nx, ny = human_scroll_into_view(
page, raw,
lambda: self.bounding_box(timeout=timeout),
cursor.x, cursor.y, call_cfg,
)
cursor.x = nx
cursor.y = ny
except Exception:
if _orig_scroll_into_view is not None:
native_kwargs = {k: v for k, v in kwargs.items() if k != "human_config"}
_orig_scroll_into_view(self, **native_kwargs)
elif _orig_scroll_into_view is not None:
_orig_scroll_into_view(self, **kwargs)
def _humanized_check(self, **kwargs):
if _is_humanized(self):
cfg = _get_cfg(self)
@@ -512,6 +553,8 @@ def _patch_locator_class_sync():
Locator.tap = _humanized_tap
Locator.drag_to = _humanized_drag_to
Locator.clear = _humanized_clear
if _orig_scroll_into_view is not None:
Locator.scroll_into_view_if_needed = _humanized_scroll_into_view_if_needed
# ============================================================================
@@ -544,6 +587,7 @@ def _patch_locator_class_async():
_orig_tap = AsyncLocator.tap
_orig_drag_to = AsyncLocator.drag_to
_orig_clear = AsyncLocator.clear
_orig_scroll_into_view = getattr(AsyncLocator, 'scroll_into_view_if_needed', None)
def _get_selector(self):
return self._impl_obj._selector
@@ -554,36 +598,74 @@ def _patch_locator_class_async():
def _get_cfg(self):
return getattr(self.page, '_human_cfg', None)
def _forward_kwargs(kwargs):
out = {}
if "timeout" in kwargs:
out["timeout"] = kwargs["timeout"]
if "human_config" in kwargs:
out["human_config"] = kwargs["human_config"]
return out
async def _humanized_fill(self, value, **kwargs):
if _is_humanized(self):
await self.page.fill(_get_selector(self), value)
await self.page.fill(_get_selector(self), value, **_forward_kwargs(kwargs))
else:
await _orig_fill(self, value, **kwargs)
async def _humanized_click(self, **kwargs):
if _is_humanized(self):
await self.page.click(_get_selector(self))
await self.page.click(_get_selector(self), **_forward_kwargs(kwargs))
else:
await _orig_click(self, **kwargs)
async def _humanized_type(self, text, **kwargs):
if _is_humanized(self):
await self.page.type(_get_selector(self), text)
await self.page.type(_get_selector(self), text, **_forward_kwargs(kwargs))
else:
await _orig_type(self, text, **kwargs)
async def _humanized_dblclick(self, **kwargs):
if _is_humanized(self):
await self.page.dblclick(_get_selector(self))
await self.page.dblclick(_get_selector(self), **_forward_kwargs(kwargs))
else:
await _orig_dblclick(self, **kwargs)
async def _humanized_hover(self, **kwargs):
if _is_humanized(self):
await self.page.hover(_get_selector(self))
await self.page.hover(_get_selector(self), **_forward_kwargs(kwargs))
else:
await _orig_hover(self, **kwargs)
async def _humanized_scroll_into_view_if_needed(self, **kwargs):
if _is_humanized(self):
page = self.page
cfg = _get_cfg(self)
cursor = getattr(page, '_human_cursor', None)
raw = getattr(page, '_human_raw_mouse', None)
call_cfg = merge_config(cfg, kwargs.get("human_config")) if cfg else None
if call_cfg is None or cursor is None or raw is None:
if _orig_scroll_into_view is not None:
native_kwargs = {k: v for k, v in kwargs.items() if k != "human_config"}
await _orig_scroll_into_view(self, **native_kwargs)
return
timeout = kwargs.get("timeout", 30000)
async def _get_box():
return await self.bounding_box(timeout=timeout)
try:
_, nx, ny = await async_human_scroll_into_view(
page, raw, _get_box,
cursor.x, cursor.y, call_cfg,
)
cursor.x = nx
cursor.y = ny
except Exception:
if _orig_scroll_into_view is not None:
native_kwargs = {k: v for k, v in kwargs.items() if k != "human_config"}
await _orig_scroll_into_view(self, **native_kwargs)
elif _orig_scroll_into_view is not None:
await _orig_scroll_into_view(self, **kwargs)
async def _humanized_check(self, **kwargs):
if _is_humanized(self):
cfg = _get_cfg(self)
@@ -708,6 +790,8 @@ def _patch_locator_class_async():
AsyncLocator.tap = _humanized_tap
AsyncLocator.drag_to = _humanized_drag_to
AsyncLocator.clear = _humanized_clear
if _orig_scroll_into_view is not None:
AsyncLocator.scroll_into_view_if_needed = _humanized_scroll_into_view_if_needed
# ============================================================================
@@ -738,6 +822,7 @@ def patch_page(page: Any, cfg: HumanConfig, cursor: _CursorState) -> None:
page._original = originals
page._human_cfg = cfg
page._human_cursor = cursor
# --- Stealth infrastructure ---
try:
@@ -764,6 +849,8 @@ def patch_page(page: Any, cfg: HumanConfig, cursor: _CursorState) -> None:
"insert_text": originals.keyboard_insert_text,
})()
page._human_raw_mouse = raw_mouse
def _ensure_cursor_init() -> None:
if not cursor.initialized:
cursor.x = rand(cfg.initial_cursor_x[0], cfg.initial_cursor_x[1])
@@ -780,32 +867,36 @@ def patch_page(page: Any, cfg: HumanConfig, cursor: _CursorState) -> None:
def _human_click(selector: str, **kwargs: Any) -> None:
_ensure_cursor_init()
if cfg.idle_between_actions:
human_idle(raw_mouse, rand(cfg.idle_between_duration[0], cfg.idle_between_duration[1]), cursor.x, cursor.y, cfg)
call_cfg = merge_config(cfg, kwargs.get("human_config"))
timeout = kwargs.get("timeout", 30000)
if call_cfg.idle_between_actions:
human_idle(raw_mouse, rand(call_cfg.idle_between_duration[0], call_cfg.idle_between_duration[1]), cursor.x, cursor.y, call_cfg)
box, cx, cy = scroll_to_element(
page, raw_mouse, selector, cursor.x, cursor.y, cfg
page, raw_mouse, selector, cursor.x, cursor.y, call_cfg, timeout=timeout,
)
cursor.x = cx
cursor.y = cy
is_input = _is_input_element(page, selector)
target = click_target(box, is_input, cfg)
human_move(raw_mouse, cursor.x, cursor.y, target.x, target.y, cfg)
target = click_target(box, is_input, call_cfg)
human_move(raw_mouse, cursor.x, cursor.y, target.x, target.y, call_cfg)
cursor.x = target.x
cursor.y = target.y
human_click(raw_mouse, is_input, cfg)
human_click(raw_mouse, is_input, call_cfg)
def _human_dblclick(selector: str, **kwargs: Any) -> None:
_ensure_cursor_init()
if cfg.idle_between_actions:
human_idle(raw_mouse, rand(cfg.idle_between_duration[0], cfg.idle_between_duration[1]), cursor.x, cursor.y, cfg)
call_cfg = merge_config(cfg, kwargs.get("human_config"))
timeout = kwargs.get("timeout", 30000)
if call_cfg.idle_between_actions:
human_idle(raw_mouse, rand(call_cfg.idle_between_duration[0], call_cfg.idle_between_duration[1]), cursor.x, cursor.y, call_cfg)
box, cx, cy = scroll_to_element(
page, raw_mouse, selector, cursor.x, cursor.y, cfg
page, raw_mouse, selector, cursor.x, cursor.y, call_cfg, timeout=timeout,
)
cursor.x = cx
cursor.y = cy
is_input = _is_input_element(page, selector)
target = click_target(box, is_input, cfg)
human_move(raw_mouse, cursor.x, cursor.y, target.x, target.y, cfg)
target = click_target(box, is_input, call_cfg)
human_move(raw_mouse, cursor.x, cursor.y, target.x, target.y, call_cfg)
cursor.x = target.x
cursor.y = target.y
raw_mouse.down(click_count=2)
@@ -814,33 +905,39 @@ def patch_page(page: Any, cfg: HumanConfig, cursor: _CursorState) -> None:
def _human_hover(selector: str, **kwargs: Any) -> None:
_ensure_cursor_init()
if cfg.idle_between_actions:
human_idle(raw_mouse, rand(cfg.idle_between_duration[0], cfg.idle_between_duration[1]), cursor.x, cursor.y, cfg)
call_cfg = merge_config(cfg, kwargs.get("human_config"))
timeout = kwargs.get("timeout", 30000)
if call_cfg.idle_between_actions:
human_idle(raw_mouse, rand(call_cfg.idle_between_duration[0], call_cfg.idle_between_duration[1]), cursor.x, cursor.y, call_cfg)
box, cx, cy = scroll_to_element(
page, raw_mouse, selector, cursor.x, cursor.y, cfg
page, raw_mouse, selector, cursor.x, cursor.y, call_cfg, timeout=timeout,
)
cursor.x = cx
cursor.y = cy
target = click_target(box, False, cfg)
human_move(raw_mouse, cursor.x, cursor.y, target.x, target.y, cfg)
target = click_target(box, False, call_cfg)
human_move(raw_mouse, cursor.x, cursor.y, target.x, target.y, call_cfg)
cursor.x = target.x
cursor.y = target.y
def _human_type(selector: str, text: str, **kwargs: Any) -> None:
sleep_ms(rand_range(cfg.field_switch_delay))
_human_click(selector)
call_cfg = merge_config(cfg, kwargs.get("human_config"))
sleep_ms(rand_range(call_cfg.field_switch_delay))
# Forward kwargs so timeout / human_config also propagate to the click
# that focuses the field.
_human_click(selector, **kwargs)
sleep_ms(rand(100, 250))
human_type(page, raw_keyboard, text, cfg, cdp_session=cdp_session)
human_type(page, raw_keyboard, text, call_cfg, cdp_session=cdp_session)
def _human_fill(selector: str, value: str, **kwargs: Any) -> None:
sleep_ms(rand_range(cfg.field_switch_delay))
_human_click(selector)
call_cfg = merge_config(cfg, kwargs.get("human_config"))
sleep_ms(rand_range(call_cfg.field_switch_delay))
_human_click(selector, **kwargs)
sleep_ms(rand(100, 250))
originals.keyboard_press(_SELECT_ALL)
sleep_ms(rand(30, 80))
originals.keyboard_press("Backspace")
sleep_ms(rand(50, 150))
human_type(page, raw_keyboard, value, cfg, cdp_session=cdp_session)
human_type(page, raw_keyboard, value, call_cfg, cdp_session=cdp_session)
def _human_check(selector: str, **kwargs: Any) -> None:
try:
@@ -958,6 +1055,7 @@ def _patch_single_element_handle_sync(
_orig_set_checked = getattr(el, 'set_checked', None)
_orig_tap = el.tap
_orig_focus = el.focus
_orig_scroll_into_view = getattr(el, 'scroll_into_view_if_needed', None)
# Nested selectors
_orig_qs = el.query_selector
@@ -992,39 +1090,57 @@ def _patch_single_element_handle_sync(
el.query_selector_all = _patched_qsa
el.wait_for_selector = _patched_wfs
# Helper: move cursor to element
def _move_to_element():
# Helper: move cursor to element. Accepts optional ``call_cfg`` so per-call
# ``human_config`` overrides on type/fill carry through to mouse timing.
# Also scrolls into view first so off-screen elements don't silently fall
# back to the unpatched native method (#129, #172 follow-up).
def _move_to_element(call_cfg: HumanConfig = cfg):
if not cursor.initialized:
cursor.x = rand(cfg.initial_cursor_x[0], cfg.initial_cursor_x[1])
cursor.y = rand(cfg.initial_cursor_y[0], cfg.initial_cursor_y[1])
cursor.x = rand(call_cfg.initial_cursor_x[0], call_cfg.initial_cursor_x[1])
cursor.y = rand(call_cfg.initial_cursor_y[0], call_cfg.initial_cursor_y[1])
originals.mouse_move(cursor.x, cursor.y)
cursor.initialized = True
# Scroll into view first — best-effort. If the element can't be located
# we fall through to bounding_box() below which returns None and lets
# the caller fall back to the original Playwright method.
try:
_, nx, ny = human_scroll_into_view(
page, raw_mouse, lambda: el.bounding_box(),
cursor.x, cursor.y, call_cfg,
)
cursor.x = nx
cursor.y = ny
except Exception:
pass
box = el.bounding_box()
if not box:
return None
is_inp = _is_input_element_handle_sync(el)
target = click_target(box, is_inp, cfg)
target = click_target(box, is_inp, call_cfg)
if cfg.idle_between_actions:
human_idle(raw_mouse, rand(cfg.idle_between_duration[0], cfg.idle_between_duration[1]), cursor.x, cursor.y, cfg)
if call_cfg.idle_between_actions:
human_idle(raw_mouse, rand(call_cfg.idle_between_duration[0], call_cfg.idle_between_duration[1]), cursor.x, cursor.y, call_cfg)
human_move(raw_mouse, cursor.x, cursor.y, target.x, target.y, cfg)
human_move(raw_mouse, cursor.x, cursor.y, target.x, target.y, call_cfg)
cursor.x = target.x
cursor.y = target.y
return {'box': box, 'is_inp': is_inp}
# --- el.click() ---
def _human_el_click(**kwargs: Any) -> None:
info = _move_to_element()
call_cfg = merge_config(cfg, kwargs.get("human_config"))
info = _move_to_element(call_cfg)
if info is None:
return _orig_click(**kwargs)
human_click(raw_mouse, info['is_inp'], cfg)
human_click(raw_mouse, info['is_inp'], call_cfg)
# --- el.dblclick() ---
def _human_el_dblclick(**kwargs: Any) -> None:
info = _move_to_element()
call_cfg = merge_config(cfg, kwargs.get("human_config"))
info = _move_to_element(call_cfg)
if info is None:
return _orig_dblclick(**kwargs)
raw_mouse.down(click_count=2)
@@ -1033,32 +1149,62 @@ def _patch_single_element_handle_sync(
# --- el.hover() ---
def _human_el_hover(**kwargs: Any) -> None:
info = _move_to_element()
call_cfg = merge_config(cfg, kwargs.get("human_config"))
info = _move_to_element(call_cfg)
if info is None:
return _orig_hover(**kwargs)
# Just move, no click
# --- el.type() ---
def _human_el_type(text: str, **kwargs: Any) -> None:
info = _move_to_element()
call_cfg = merge_config(cfg, kwargs.get("human_config"))
info = _move_to_element(call_cfg)
if info is None:
return _orig_type(text, **kwargs)
human_click(raw_mouse, info['is_inp'], cfg)
human_click(raw_mouse, info['is_inp'], call_cfg)
sleep_ms(rand(100, 250))
human_type(page, raw_keyboard, text, cfg, cdp_session=cdp_session)
human_type(page, raw_keyboard, text, call_cfg, cdp_session=cdp_session)
# --- el.fill() ---
def _human_el_fill(value: str, **kwargs: Any) -> None:
info = _move_to_element()
call_cfg = merge_config(cfg, kwargs.get("human_config"))
info = _move_to_element(call_cfg)
if info is None:
return _orig_fill(value, **kwargs)
human_click(raw_mouse, info['is_inp'], cfg)
human_click(raw_mouse, info['is_inp'], call_cfg)
sleep_ms(rand(100, 250))
originals.keyboard_press(_SELECT_ALL)
sleep_ms(rand(30, 80))
originals.keyboard_press("Backspace")
sleep_ms(rand(50, 150))
human_type(page, raw_keyboard, value, cfg, cdp_session=cdp_session)
human_type(page, raw_keyboard, value, call_cfg, cdp_session=cdp_session)
# --- el.scroll_into_view_if_needed() ---
# Playwright's native version snaps the page — a strong bot signal.
# Replace with the same accelerate → cruise → decelerate → overshoot wheel
# sequence used by page.click(). Falls back to the native method if the
# element is detached or scrolling fails.
def _human_el_scroll_into_view_if_needed(**kwargs: Any) -> None:
call_cfg = merge_config(cfg, kwargs.get("human_config"))
if not cursor.initialized:
cursor.x = rand(call_cfg.initial_cursor_x[0], call_cfg.initial_cursor_x[1])
cursor.y = rand(call_cfg.initial_cursor_y[0], call_cfg.initial_cursor_y[1])
try:
originals.mouse_move(cursor.x, cursor.y)
cursor.initialized = True
except Exception:
pass
try:
_, nx, ny = human_scroll_into_view(
page, raw_mouse, lambda: el.bounding_box(),
cursor.x, cursor.y, call_cfg,
)
cursor.x = nx
cursor.y = ny
except Exception:
if _orig_scroll_into_view is not None:
native_kwargs = {k: v for k, v in kwargs.items() if k != "human_config"}
_orig_scroll_into_view(**native_kwargs)
# --- el.press() ---
def _human_el_press(key: str, **kwargs: Any) -> None:
@@ -1142,6 +1288,8 @@ def _patch_single_element_handle_sync(
el.set_checked = _human_el_set_checked
el.tap = _human_el_tap
el.focus = _human_el_focus
if _orig_scroll_into_view is not None:
el.scroll_into_view_if_needed = _human_el_scroll_into_view_if_needed
def _patch_page_element_handles_sync(
@@ -1425,6 +1573,7 @@ def patch_page_async(page: Any, cfg: HumanConfig, cursor: _CursorState) -> None:
page._original = originals
page._human_cfg = cfg
page._human_cursor = cursor
# --- Stealth infrastructure (lazy-initialized, async) ---
stealth = _AsyncIsolatedWorld(page)
@@ -1454,6 +1603,8 @@ def patch_page_async(page: Any, cfg: HumanConfig, cursor: _CursorState) -> None:
"insert_text": originals.keyboard_insert_text,
})()
page._human_raw_mouse = raw_mouse
async def _ensure_cursor_init() -> None:
if not cursor.initialized:
cursor.x = rand(cfg.initial_cursor_x[0], cfg.initial_cursor_x[1])
@@ -1469,32 +1620,36 @@ def patch_page_async(page: Any, cfg: HumanConfig, cursor: _CursorState) -> None:
async def _human_click(selector: str, **kwargs: Any) -> None:
await _ensure_cursor_init()
if cfg.idle_between_actions:
await async_human_idle(raw_mouse, rand(cfg.idle_between_duration[0], cfg.idle_between_duration[1]), cursor.x, cursor.y, cfg)
call_cfg = merge_config(cfg, kwargs.get("human_config"))
timeout = kwargs.get("timeout", 30000)
if call_cfg.idle_between_actions:
await async_human_idle(raw_mouse, rand(call_cfg.idle_between_duration[0], call_cfg.idle_between_duration[1]), cursor.x, cursor.y, call_cfg)
box, cx, cy = await async_scroll_to_element(
page, raw_mouse, selector, cursor.x, cursor.y, cfg
page, raw_mouse, selector, cursor.x, cursor.y, call_cfg, timeout=timeout,
)
cursor.x = cx
cursor.y = cy
is_input = await _async_is_input_element(page, selector)
target = click_target(box, is_input, cfg)
await async_human_move(raw_mouse, cursor.x, cursor.y, target.x, target.y, cfg)
target = click_target(box, is_input, call_cfg)
await async_human_move(raw_mouse, cursor.x, cursor.y, target.x, target.y, call_cfg)
cursor.x = target.x
cursor.y = target.y
await async_human_click(raw_mouse, is_input, cfg)
await async_human_click(raw_mouse, is_input, call_cfg)
async def _human_dblclick(selector: str, **kwargs: Any) -> None:
await _ensure_cursor_init()
if cfg.idle_between_actions:
await async_human_idle(raw_mouse, rand(cfg.idle_between_duration[0], cfg.idle_between_duration[1]), cursor.x, cursor.y, cfg)
call_cfg = merge_config(cfg, kwargs.get("human_config"))
timeout = kwargs.get("timeout", 30000)
if call_cfg.idle_between_actions:
await async_human_idle(raw_mouse, rand(call_cfg.idle_between_duration[0], call_cfg.idle_between_duration[1]), cursor.x, cursor.y, call_cfg)
box, cx, cy = await async_scroll_to_element(
page, raw_mouse, selector, cursor.x, cursor.y, cfg
page, raw_mouse, selector, cursor.x, cursor.y, call_cfg, timeout=timeout,
)
cursor.x = cx
cursor.y = cy
is_input = await _async_is_input_element(page, selector)
target = click_target(box, is_input, cfg)
await async_human_move(raw_mouse, cursor.x, cursor.y, target.x, target.y, cfg)
target = click_target(box, is_input, call_cfg)
await async_human_move(raw_mouse, cursor.x, cursor.y, target.x, target.y, call_cfg)
cursor.x = target.x
cursor.y = target.y
await raw_mouse.down(click_count=2)
@@ -1503,35 +1658,39 @@ def patch_page_async(page: Any, cfg: HumanConfig, cursor: _CursorState) -> None:
async def _human_hover(selector: str, **kwargs: Any) -> None:
await _ensure_cursor_init()
if cfg.idle_between_actions:
await async_human_idle(raw_mouse, rand(cfg.idle_between_duration[0], cfg.idle_between_duration[1]), cursor.x, cursor.y, cfg)
call_cfg = merge_config(cfg, kwargs.get("human_config"))
timeout = kwargs.get("timeout", 30000)
if call_cfg.idle_between_actions:
await async_human_idle(raw_mouse, rand(call_cfg.idle_between_duration[0], call_cfg.idle_between_duration[1]), cursor.x, cursor.y, call_cfg)
box, cx, cy = await async_scroll_to_element(
page, raw_mouse, selector, cursor.x, cursor.y, cfg
page, raw_mouse, selector, cursor.x, cursor.y, call_cfg, timeout=timeout,
)
cursor.x = cx
cursor.y = cy
target = click_target(box, False, cfg)
await async_human_move(raw_mouse, cursor.x, cursor.y, target.x, target.y, cfg)
target = click_target(box, False, call_cfg)
await async_human_move(raw_mouse, cursor.x, cursor.y, target.x, target.y, call_cfg)
cursor.x = target.x
cursor.y = target.y
async def _human_type(selector: str, text: str, **kwargs: Any) -> None:
await async_sleep_ms(rand_range(cfg.field_switch_delay))
await _human_click(selector)
call_cfg = merge_config(cfg, kwargs.get("human_config"))
await async_sleep_ms(rand_range(call_cfg.field_switch_delay))
await _human_click(selector, **kwargs)
await async_sleep_ms(rand(100, 250))
cdp = await _ensure_cdp()
await async_human_type(page, raw_keyboard, text, cfg, cdp_session=cdp)
await async_human_type(page, raw_keyboard, text, call_cfg, cdp_session=cdp)
async def _human_fill(selector: str, value: str, **kwargs: Any) -> None:
await async_sleep_ms(rand_range(cfg.field_switch_delay))
await _human_click(selector)
call_cfg = merge_config(cfg, kwargs.get("human_config"))
await async_sleep_ms(rand_range(call_cfg.field_switch_delay))
await _human_click(selector, **kwargs)
await async_sleep_ms(rand(100, 250))
await originals.keyboard_press(_SELECT_ALL)
await async_sleep_ms(rand(30, 80))
await originals.keyboard_press("Backspace")
await async_sleep_ms(rand(50, 150))
cdp = await _ensure_cdp()
await async_human_type(page, raw_keyboard, value, cfg, cdp_session=cdp)
await async_human_type(page, raw_keyboard, value, call_cfg, cdp_session=cdp)
async def _human_check(selector: str, **kwargs: Any) -> None:
try:
@@ -1637,6 +1796,7 @@ def _patch_single_element_handle_async(
_orig_set_checked = getattr(el, 'set_checked', None)
_orig_tap = el.tap
_orig_focus = el.focus
_orig_scroll_into_view = getattr(el, 'scroll_into_view_if_needed', None)
# Nested selectors
_orig_qs = el.query_selector
@@ -1671,25 +1831,41 @@ def _patch_single_element_handle_async(
el.query_selector_all = _patched_qsa
el.wait_for_selector = _patched_wfs
# Helper: move cursor to element (async)
async def _move_to_element():
# Helper: move cursor to element (async). Accepts optional ``call_cfg`` so
# per-call ``human_config`` overrides on type/fill carry through to mouse
# timing. Also scrolls into view first so off-screen elements work
# (#129, #172 follow-up).
async def _move_to_element(call_cfg: HumanConfig = cfg):
if not cursor.initialized:
cursor.x = rand(cfg.initial_cursor_x[0], cfg.initial_cursor_x[1])
cursor.y = rand(cfg.initial_cursor_y[0], cfg.initial_cursor_y[1])
cursor.x = rand(call_cfg.initial_cursor_x[0], call_cfg.initial_cursor_x[1])
cursor.y = rand(call_cfg.initial_cursor_y[0], call_cfg.initial_cursor_y[1])
await originals.mouse_move(cursor.x, cursor.y)
cursor.initialized = True
# Scroll into view first — best-effort.
async def _get_box():
return await el.bounding_box()
try:
_, nx, ny = await async_human_scroll_into_view(
page, raw_mouse, _get_box,
cursor.x, cursor.y, call_cfg,
)
cursor.x = nx
cursor.y = ny
except Exception:
pass
box = await el.bounding_box()
if not box:
return None
is_inp = await _async_is_input_element_handle(el)
target = click_target(box, is_inp, cfg)
target = click_target(box, is_inp, call_cfg)
if cfg.idle_between_actions:
await async_human_idle(raw_mouse, rand(cfg.idle_between_duration[0], cfg.idle_between_duration[1]), cursor.x, cursor.y, cfg)
if call_cfg.idle_between_actions:
await async_human_idle(raw_mouse, rand(call_cfg.idle_between_duration[0], call_cfg.idle_between_duration[1]), cursor.x, cursor.y, call_cfg)
await async_human_move(raw_mouse, cursor.x, cursor.y, target.x, target.y, cfg)
await async_human_move(raw_mouse, cursor.x, cursor.y, target.x, target.y, call_cfg)
cursor.x = target.x
cursor.y = target.y
return {'box': box, 'is_inp': is_inp}
@@ -1704,14 +1880,16 @@ def _patch_single_element_handle_async(
# --- el.click() ---
async def _human_el_click(**kwargs: Any) -> None:
info = await _move_to_element()
call_cfg = merge_config(cfg, kwargs.get("human_config"))
info = await _move_to_element(call_cfg)
if info is None:
return await _orig_click(**kwargs)
await async_human_click(raw_mouse, info['is_inp'], cfg)
await async_human_click(raw_mouse, info['is_inp'], call_cfg)
# --- el.dblclick() ---
async def _human_el_dblclick(**kwargs: Any) -> None:
info = await _move_to_element()
call_cfg = merge_config(cfg, kwargs.get("human_config"))
info = await _move_to_element(call_cfg)
if info is None:
return await _orig_dblclick(**kwargs)
await raw_mouse.down(click_count=2)
@@ -1720,33 +1898,62 @@ def _patch_single_element_handle_async(
# --- el.hover() ---
async def _human_el_hover(**kwargs: Any) -> None:
info = await _move_to_element()
call_cfg = merge_config(cfg, kwargs.get("human_config"))
info = await _move_to_element(call_cfg)
if info is None:
return await _orig_hover(**kwargs)
# --- el.type() ---
async def _human_el_type(text: str, **kwargs: Any) -> None:
info = await _move_to_element()
call_cfg = merge_config(cfg, kwargs.get("human_config"))
info = await _move_to_element(call_cfg)
if info is None:
return await _orig_type(text, **kwargs)
await async_human_click(raw_mouse, info['is_inp'], cfg)
await async_human_click(raw_mouse, info['is_inp'], call_cfg)
await async_sleep_ms(rand(100, 250))
cdp = await _get_cdp()
await async_human_type(page, raw_keyboard, text, cfg, cdp_session=cdp)
await async_human_type(page, raw_keyboard, text, call_cfg, cdp_session=cdp)
# --- el.fill() ---
async def _human_el_fill(value: str, **kwargs: Any) -> None:
info = await _move_to_element()
call_cfg = merge_config(cfg, kwargs.get("human_config"))
info = await _move_to_element(call_cfg)
if info is None:
return await _orig_fill(value, **kwargs)
await async_human_click(raw_mouse, info['is_inp'], cfg)
await async_human_click(raw_mouse, info['is_inp'], call_cfg)
await async_sleep_ms(rand(100, 250))
await originals.keyboard_press(_SELECT_ALL)
await async_sleep_ms(rand(30, 80))
await originals.keyboard_press("Backspace")
await async_sleep_ms(rand(50, 150))
cdp = await _get_cdp()
await async_human_type(page, raw_keyboard, value, cfg, cdp_session=cdp)
await async_human_type(page, raw_keyboard, value, call_cfg, cdp_session=cdp)
# --- el.scroll_into_view_if_needed() ---
async def _human_el_scroll_into_view_if_needed(**kwargs: Any) -> None:
call_cfg = merge_config(cfg, kwargs.get("human_config"))
if not cursor.initialized:
cursor.x = rand(call_cfg.initial_cursor_x[0], call_cfg.initial_cursor_x[1])
cursor.y = rand(call_cfg.initial_cursor_y[0], call_cfg.initial_cursor_y[1])
try:
await originals.mouse_move(cursor.x, cursor.y)
cursor.initialized = True
except Exception:
pass
async def _get_box():
return await el.bounding_box()
try:
_, nx, ny = await async_human_scroll_into_view(
page, raw_mouse, _get_box,
cursor.x, cursor.y, call_cfg,
)
cursor.x = nx
cursor.y = ny
except Exception:
if _orig_scroll_into_view is not None:
native_kwargs = {k: v for k, v in kwargs.items() if k != "human_config"}
await _orig_scroll_into_view(**native_kwargs)
# --- el.press() ---
async def _human_el_press(key: str, **kwargs: Any) -> None:
@@ -1830,6 +2037,8 @@ def _patch_single_element_handle_async(
el.set_checked = _human_el_set_checked
el.tap = _human_el_tap
el.focus = _human_el_focus
if _orig_scroll_into_view is not None:
el.scroll_into_view_if_needed = _human_el_scroll_into_view_if_needed
def _patch_page_element_handles_async(
+19
View File
@@ -201,6 +201,25 @@ def resolve_config(
return HumanConfig(**merged)
def merge_config(base: HumanConfig, overrides: dict | None) -> HumanConfig:
"""Merge ``overrides`` (a dict of HumanConfig field names → values) on top of
``base``. Returns a new HumanConfig — ``base`` is never mutated.
Used by per-call overrides like ``page.type(sel, text, human_config={...})``
so the same page can use different timings for different inputs without
re-patching.
Unknown keys are ignored silently to keep this forgiving for callers.
"""
if not overrides:
return base
merged = {k: getattr(base, k) for k in base.__dataclass_fields__}
for k, v in overrides.items():
if k in base.__dataclass_fields__:
merged[k] = v
return HumanConfig(**merged)
# ---------------------------------------------------------------------------
# Utility functions
# ---------------------------------------------------------------------------
+44 -13
View File
@@ -4,7 +4,7 @@ from __future__ import annotations
import math
import random
from typing import Any, Optional, Tuple
from typing import Any, Callable, Optional, Tuple
from .config import HumanConfig, rand, rand_range, rand_int_range, sleep_ms
from .mouse import RawMouse, human_move
@@ -18,10 +18,15 @@ def _is_in_viewport(bounds: dict, viewport_height: int, cfg: HumanConfig) -> boo
return top_edge >= zone_top and bottom_edge <= zone_bottom
def _get_element_box(page: Any, selector: str) -> Optional[dict]:
def _get_element_box(page: Any, selector: str, timeout: float = 30000) -> Optional[dict]:
"""Locate ``selector`` and return its bounding box.
The ``timeout`` is forwarded to Playwright's ``boundingBox(timeout=...)``
so callers can extend it for slow-loading elements (#172).
"""
try:
el = page.locator(selector).first
return el.bounding_box(timeout=2000)
return el.bounding_box(timeout=timeout)
except Exception:
return None
@@ -39,13 +44,21 @@ def _smooth_wheel(raw: RawMouse, delta: int, cfg: HumanConfig) -> None:
sleep_ms(rand(8, 20))
def scroll_to_element(
def human_scroll_into_view(
page: Any,
raw: RawMouse,
selector: str,
get_box: Callable[[], Optional[dict]],
cursor_x: float, cursor_y: float,
cfg: HumanConfig,
) -> Tuple[dict, float, float]:
"""Humanized scrolling that uses an arbitrary ``get_box`` callable
instead of a CSS selector.
Used both by ``scroll_to_element`` (selector-based) and by
``ElementHandle.scroll_into_view_if_needed`` / ``Locator.scroll_into_view_if_needed``
(handle-based) so the same accelerate \u2192 cruise \u2192 decelerate \u2192 overshoot
behavior runs everywhere.
"""
viewport = page.viewport_size
if not viewport:
raise RuntimeError("Viewport size not available")
@@ -53,12 +66,9 @@ def scroll_to_element(
viewport_height = viewport["height"]
viewport_width = viewport["width"]
box = _get_element_box(page, selector)
box = get_box()
if box is None:
sleep_ms(200)
box = _get_element_box(page, selector)
if box is None:
raise RuntimeError(f"Element not found: {selector}")
raise RuntimeError("Element not found while scrolling into view")
if _is_in_viewport(box, viewport_height, cfg):
return box, cursor_x, cursor_y
@@ -105,7 +115,7 @@ def scroll_to_element(
# Check visibility every 3 steps
if i % 3 == 2 or i == total_clicks - 1:
box = _get_element_box(page, selector)
box = get_box()
if box and _is_in_viewport(box, viewport_height, cfg):
break
if scrolled >= abs_distance * 1.1:
@@ -125,8 +135,29 @@ def scroll_to_element(
# Settle
sleep_ms(rand_range(cfg.scroll_settle_delay))
box = _get_element_box(page, selector)
box = get_box()
if box is None:
raise RuntimeError(f"Element lost after scrolling: {selector}")
raise RuntimeError("Element lost after scrolling into view")
return box, cursor_x, cursor_y
def scroll_to_element(
page: Any,
raw: RawMouse,
selector: str,
cursor_x: float, cursor_y: float,
cfg: HumanConfig,
timeout: float = 30000,
) -> Tuple[dict, float, float]:
"""Selector-based humanized scroll.
``timeout`` is forwarded to ``locator.bounding_box(timeout=...)`` so callers
such as ``page.click('#x', timeout=5000)`` can wait longer for slow elements
(#172). Default matches Playwright's 30000ms when not specified.
"""
return human_scroll_into_view(
page, raw,
lambda: _get_element_box(page, selector, timeout),
cursor_x, cursor_y, cfg,
)
+43 -13
View File
@@ -8,17 +8,22 @@ from __future__ import annotations
import math
import random
from typing import Any, Optional, Tuple
from typing import Any, Awaitable, Callable, Optional, Tuple
from .config import HumanConfig, rand, rand_range, rand_int_range, async_sleep_ms
from .mouse_async import AsyncRawMouse, async_human_move
from .scroll import _is_in_viewport
async def _get_element_box_async(page: Any, selector: str) -> Optional[dict]:
async def _get_element_box_async(
page: Any, selector: str, timeout: float = 30000,
) -> Optional[dict]:
"""Async variant. ``timeout`` is forwarded to Playwright's
``boundingBox(timeout=...)`` so callers can extend it for slow-loading
elements (#172)."""
try:
el = page.locator(selector).first
return await el.bounding_box(timeout=2000)
return await el.bounding_box(timeout=timeout)
except Exception:
return None
@@ -36,13 +41,20 @@ async def _async_smooth_wheel(raw: AsyncRawMouse, delta: int, cfg: HumanConfig)
await async_sleep_ms(rand(8, 20))
async def async_scroll_to_element(
async def async_human_scroll_into_view(
page: Any,
raw: AsyncRawMouse,
selector: str,
get_box: Callable[[], Awaitable[Optional[dict]]],
cursor_x: float, cursor_y: float,
cfg: HumanConfig,
) -> Tuple[dict, float, float]:
"""Humanized scrolling using an arbitrary async ``get_box`` callable.
Used by both ``async_scroll_to_element`` (selector-based) and the
ElementHandle / Locator ``scroll_into_view_if_needed`` patches so all
scrolling paths share the same accelerate \u2192 cruise \u2192 decelerate
\u2192 overshoot behavior.
"""
viewport = page.viewport_size
if not viewport:
raise RuntimeError("Viewport size not available")
@@ -50,12 +62,9 @@ async def async_scroll_to_element(
viewport_height = viewport["height"]
viewport_width = viewport["width"]
box = await _get_element_box_async(page, selector)
box = await get_box()
if box is None:
await async_sleep_ms(200)
box = await _get_element_box_async(page, selector)
if box is None:
raise RuntimeError(f"Element not found: {selector}")
raise RuntimeError("Element not found while scrolling into view")
if _is_in_viewport(box, viewport_height, cfg):
return box, cursor_x, cursor_y
@@ -102,7 +111,7 @@ async def async_scroll_to_element(
# Check visibility every 3 steps
if i % 3 == 2 or i == total_clicks - 1:
box = await _get_element_box_async(page, selector)
box = await get_box()
if box and _is_in_viewport(box, viewport_height, cfg):
break
if scrolled >= abs_distance * 1.1:
@@ -122,8 +131,29 @@ async def async_scroll_to_element(
# Settle
await async_sleep_ms(rand_range(cfg.scroll_settle_delay))
box = await _get_element_box_async(page, selector)
box = await get_box()
if box is None:
raise RuntimeError(f"Element lost after scrolling: {selector}")
raise RuntimeError("Element lost after scrolling into view")
return box, cursor_x, cursor_y
async def async_scroll_to_element(
page: Any,
raw: AsyncRawMouse,
selector: str,
cursor_x: float, cursor_y: float,
cfg: HumanConfig,
timeout: float = 30000,
) -> Tuple[dict, float, float]:
"""Selector-based humanized scroll (async).
``timeout`` is forwarded to ``locator.bounding_box(timeout=...)`` so callers
such as ``page.click('#x', timeout=5000)`` can wait longer for slow elements
(#172). Default matches Playwright's 30000ms when not specified.
"""
async def _get():
return await _get_element_box_async(page, selector, timeout)
return await async_human_scroll_into_view(
page, raw, _get, cursor_x, cursor_y, cfg,
)
@@ -0,0 +1,79 @@
# CloakBrowser on AWS Lambda — derived from the official CloakHQ image.
#
# `FROM cloakhq/cloakbrowser:<tag>` is an official distribution channel under
# the CloakBrowser Binary License — pulling it isn't redistribution. We just
# layer Lambda glue on top: the Lambda Runtime Interface Client (awslambdaric),
# the Lambda Runtime Interface Emulator (for local `docker run` testing), the
# dual-mode entrypoint, and the handler module.
#
# This directory is self-contained — copy/clone it anywhere and build from
# inside it. No files outside this directory are referenced.
#
# ─── Lambda invocation (default CMD) ──────────────────────────────────────────
# # From inside this directory:
# docker buildx build --platform linux/arm64 -t cloakbrowser-lambda:arm64 --load .
#
# # Or from a parent dir, pointing at this directory as the build context:
# docker buildx build --platform linux/arm64 \
# -f path/to/aws_lambda/Dockerfile -t cloakbrowser-lambda:arm64 --load \
# path/to/aws_lambda
#
# docker run --rm -p 9000:8080 cloakbrowser-lambda:arm64
# curl -XPOST http://localhost:9000/2015-03-31/functions/function/invocations \
# -d '{"url":"https://example.com"}'
#
# ─── Same as the canonical CloakHQ image (CMD overridden) ─────────────────────
# docker run --rm -it cloakbrowser-lambda:arm64 python # REPL
# docker run --rm cloakbrowser-lambda:arm64 python examples/basic.py # examples
# docker run --rm -p 9222:9222 cloakbrowser-lambda:arm64 cloakserve --port=9222 # CDP server
# docker run --rm cloakbrowser-lambda:arm64 cloaktest # stealth tests
# docker run --rm -it cloakbrowser-lambda:arm64 node # JS wrapper
# docker run --rm -it cloakbrowser-lambda:arm64 bash # shell
#
# Pin a specific tag (e.g. cloakhq/cloakbrowser:0.3.25) for reproducible builds;
# `latest` floats with CloakHQ's release cadence.
FROM cloakhq/cloakbrowser:latest
# ─── Lambda Runtime Interface Client ──────────────────────────────────────────
RUN pip install --no-cache-dir awslambdaric
# ─── Lambda Runtime Interface Emulator (local `docker run` testing) ───────────
# Bundled into the image so users can hit the standard local-invoke endpoint
# without mounting the RIE separately. TARGETARCH is provided by buildx.
ARG TARGETARCH
ADD https://github.com/aws/aws-lambda-runtime-interface-emulator/releases/latest/download/aws-lambda-rie-${TARGETARCH} \
/usr/local/bin/aws-lambda-rie
RUN chmod +x /usr/local/bin/aws-lambda-rie
# ─── Lambda glue ──────────────────────────────────────────────────────────────
# Dual-mode entrypoint replaces the canonical bin/docker-entrypoint.sh: same
# Xvfb startup, plus routing for `module.func` CMDs through awslambdaric.
COPY lambda-entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh
# Handler sits at /app (already on Python's import path in the canonical image,
# WORKDIR=/app), imports cloakbrowser as a normal library.
COPY lambda_handler.py /app/lambda_handler.py
# ─── Lambda non-root readability fix ──────────────────────────────────────────
# The canonical image bakes the Chromium binary at /root/.cloakbrowser/ (root's
# HOME at build time). Lambda runs the container as a non-root user that can't
# read /root by default (mode 750). Make the whole binary tree world-readable
# and traversable. Also restore the .welcome_shown marker the canonical image
# rm's (Lambda's read-only runtime FS can't recreate it, so the welcome would
# print to CloudWatch on every cold start otherwise).
RUN touch /root/.cloakbrowser/.welcome_shown \
&& chmod -R o+rX /root /root/.cloakbrowser
# ─── Lambda runtime env ───────────────────────────────────────────────────────
# HOME=/tmp gives Chromium a writable scratch dir (Lambda only allows writes
# under /tmp). CLOAKBROWSER_CACHE_DIR points at the baked binary location since
# HOME=/tmp would otherwise make get_cache_dir() resolve to /tmp/.cloakbrowser
# (empty). Auto-update is disabled because the runtime FS is read-only.
ENV HOME=/tmp \
CLOAKBROWSER_CACHE_DIR=/root/.cloakbrowser \
CLOAKBROWSER_AUTO_UPDATE=false
ENTRYPOINT ["/entrypoint.sh"]
CMD ["lambda_handler.handler"]
@@ -0,0 +1,181 @@
# CloakBrowser on AWS Lambda
Run stealth Chromium one-shot scrapes inside an AWS Lambda function (container image package type). The image derives directly from the official CloakHQ Docker Hub image (`cloakhq/cloakbrowser`) and adds Lambda runtime support on top — Lambda is an additional invocation surface, not a replacement. Every other surface from the canonical image (`python`, `cloakserve`, `cloaktest`, `node`, `bash`, examples) keeps working.
This document covers what the image is, how to build and locally test it, and the event/response contract. **It does not prescribe a deployment method** — push the resulting image to ECR and create the Lambda function however you prefer (AWS CLI, CDK, Terraform, SAM, console, etc.). Configuration tips for whichever tool you use are at the bottom.
## Files in this directory
| File | Purpose |
|---|---|
| `Dockerfile` | `FROM cloakhq/cloakbrowser` plus a thin Lambda layer. Self-contained — no files outside this directory are referenced. |
| `lambda-entrypoint.sh` | Dual-mode entrypoint. Starts Xvfb, then routes `module.func` CMDs through `awslambdaric` (via the bundled `aws-lambda-rie` locally, or the AWS Runtime API in production), and execs everything else (`python`, `cloakserve`, `cloaktest`, `node`, `bash`) directly. |
| `lambda_handler.py` | Default handler. Takes `{url, ...}`, returns `{title, url, html, screenshot_b64?}`. Always headed via Xvfb. |
| `INSTRUCTIONS.md` | This file. |
The Lambda layer is ~30 lines on top of the official image — no apt list, no Node install, no JS-wrapper build, no Chromium download. The canonical CloakHQ image owns those.
This directory is **standalone**: copy or clone it anywhere (its own repo, a subdirectory of an existing project, a CI artifact bundle) and the build still works. It depends only on the upstream `cloakhq/cloakbrowser` image on Docker Hub and the `aws-lambda-rie` binary on GitHub Releases — both fetched at build time.
## Build
From inside this directory:
```bash
docker buildx build --platform linux/arm64 -t cloakbrowser-lambda:arm64 --load .
```
Or from anywhere, pointing at this directory as the build context:
```bash
docker buildx build --platform linux/arm64 \
-f path/to/aws_lambda/Dockerfile \
-t cloakbrowser-lambda:arm64 --load \
path/to/aws_lambda
```
The build pulls `cloakhq/cloakbrowser:latest` from Docker Hub and adds the Lambda layer on top. Pin a specific tag (e.g. `cloakhq/cloakbrowser:0.3.25`) in the `FROM` line for reproducible builds; `latest` floats with the upstream release cadence.
For x86_64, switch `--platform linux/amd64` (slower on Apple Silicon under emulation).
## Local smoke test (no AWS account needed)
> **What's the RIE?** Lambda container images can't be run with a plain `docker run` — they expect to talk to AWS's Runtime API (the HTTP service Lambda exposes inside its sandbox to deliver events and collect responses). AWS publishes a small binary called the **Runtime Interface Emulator** that stands up a fake Runtime API on localhost so you can test the container exactly the way Lambda will invoke it, without deploying. We bake the RIE into the image, and the dual-mode entrypoint uses it automatically when `AWS_LAMBDA_RUNTIME_API` isn't set (i.e. you're not running in real Lambda).
The image bakes in `aws-lambda-rie`, so the standard Lambda local-invoke endpoint works without mounting anything:
```bash
docker run --rm -p 9000:8080 cloakbrowser-lambda:arm64
# In another shell:
curl -sS -XPOST "http://localhost:9000/2015-03-31/functions/function/invocations" \
-d '{"url":"https://example.com"}'
```
Other invocation surfaces stay intact (these match the canonical CloakHQ image):
```bash
docker run --rm -it cloakbrowser-lambda:arm64 python # REPL
docker run --rm cloakbrowser-lambda:arm64 python examples/basic.py # examples
docker run --rm -p 9222:9222 cloakbrowser-lambda:arm64 cloakserve --port=9222 # CDP server
docker run --rm cloakbrowser-lambda:arm64 cloaktest # stealth tests
docker run --rm -it cloakbrowser-lambda:arm64 node # JS wrapper
```
## Event schema
Only `url` is required. Everything else is optional.
### Launch options (forwarded to `cloakbrowser.launch_context_async`)
| Field | Type | Default |
|---|---|---|
| `url` | str | required |
| `proxy` | str / dict | none — `http://user:pass@host:port` or a Playwright proxy dict |
| `humanize` | bool | `false` — enable human-like mouse / keyboard / scroll |
| `human_preset` | str | `"default"` or `"careful"` |
| `geoip` | bool | `false` — auto timezone+locale from proxy IP |
| `timezone` | str | none — IANA tz, e.g. `"America/New_York"` |
| `locale` | str | none — BCP-47, e.g. `"en-US"` |
| `viewport` | `{width,height}` | `1920x947` (cloakbrowser default) |
| `user_agent` | str | none |
| `extra_args` | `list[str]` | `[]` — extra Chromium CLI flags |
### Navigation
| Field | Type | Default |
|---|---|---|
| `wait_until` | str | `"domcontentloaded"``load` / `domcontentloaded` / `networkidle` / `commit` |
| `goto_timeout_ms` | int | `30000` |
### Post-navigation waits
`smart_wait` is the default when no other wait is specified. It polls `document.documentElement.outerHTML.length` and returns when the size hasn't changed for `dom_stable_ms`. Robust for at-scale scraping because it ignores network activity (analytics beacons, long-poll, websockets) that doesn't mutate the DOM — `wait_until: "networkidle"` is unreliable on modern SPAs for exactly this reason.
| Field | Type | Default |
|---|---|---|
| `smart_wait` | bool | `true` if no other wait is set |
| `dom_stable_ms` | int | `1500` |
| `max_settle_ms` | int | `15000` |
| `wait_for_load_state` | str | none — `load` / `domcontentloaded` / `networkidle` |
| `wait_for_load_state_timeout_ms` | int | `30000` |
| `wait_for_selector` | str | none — CSS or XPath |
| `wait_for_selector_state` | str | `"visible"` — also `attached` / `detached` / `hidden` |
| `wait_for_selector_timeout_ms` | int | `30000` |
| `wait_for_function` | str | none — JS expression returning truthy when ready |
| `wait_for_function_timeout_ms` | int | `30000` |
| `wait_ms` | int | none — fixed pause |
### Capture
| Field | Type | Default |
|---|---|---|
| `screenshot` | bool | `true` |
| `full_page_screenshot` | bool | `false` |
### Retry orchestration
The handler retries transient navigation failures inline within the same Lambda invocation. Two layers, both built-in:
- **Launch retries** — 3 attempts with 0.3 s + 0.6 s backoff. Recovers Xvfb / Chromium spawn races at cold start. Fast and cheap; not configurable.
- **Strategy retries** — default 1 attempt, configurable via the `retries` event field. Recovers specific post-launch error classes by relaunching with adjusted Chromium args / page-load budgets.
| Field | Type | Default |
|---|---|---|
| `retries` | int | `1` — number of strategy-retry attempts after the first failure. Set to `0` to disable retry entirely. |
Strategies (priority order — first match wins):
| Error pattern | Strategy applied |
|---|---|
| `ERR_CERT_*` (any cert error) | `extra_args: ["--ignore-certificate-errors"]`, `goto_timeout_ms: 60000` |
| `Timeout … exceeded` | `goto_timeout_ms: 90000`, `max_settle_ms: 25000` |
| `ERR_CONNECTION_TIMED_OUT` | same as `Timeout … exceeded` |
Errors that are **not retried** (no anonymous scraper can recover): `ERR_NAME_NOT_RESOLVED`, `ERR_SSL_PROTOCOL_ERROR`, `ERR_CONNECTION_REFUSED`, `ERR_HTTP_RESPONSE_CODE_FAILURE`. These bail immediately.
On final failure, the raised `RuntimeError`'s message includes a `retry_history` block listing every attempt (strategy applied + error seen). Successful invocations return the standard response shape unchanged — no surprise fields when retries didn't fire.
### Response
```json
{
"title": "...",
"url": "https://example.com/",
"html": "<!DOCTYPE html>...",
"screenshot_b64": "<base64 PNG>"
}
```
## Lambda-specific Chromium hardening (baked in, do not remove)
Two flags are forced on every launch by `lambda_handler.py`:
- `--disable-dev-shm-usage` — Lambda's `/dev/shm` is ~64 MB; Chromium's renderer crashes mid-paint without this.
- `--no-zygote` — Lambda's restricted process model can't fork from Chromium's zygote process; without this the browser launches but child renderers fail to spawn and the first `page.new_page()` raises `TargetClosedError`.
## Function configuration recommendations
Whatever tool you use to create the Lambda function (CLI, CDK, Terraform, SAM, console), apply these settings:
| Setting | Value | Why |
|---|---|---|
| Package type | Image | Required — this is a container image, not a zip. |
| Architecture | `arm64` | Roughly 20% cheaper than x86_64. Native build on Apple Silicon. Match the architecture you built for. |
| Memory | 3008 MB | Memory in Lambda is tied to vCPU. Below ~1769 MB Chromium starts noticeably slower. |
| Timeout | 120180 s | Single-attempt scrapes complete in 315 s warm; under retry, a `Timeout`-class first failure (30 s default) plus a longer-budget retry (90 s) plus cleanup can total ~120-130 s. 180 s leaves headroom; below 120 s the function will time out before the retry completes. Cold-start init adds 5-10 s on top. |
| Ephemeral storage (`/tmp`) | 1024 MB | Chromium profile dirs and screenshots can fill the 512 MB default. |
| Networking | Default (no VPC) | Binary is baked in, no network needed at cold start. Add VPC + NAT only if your proxy egress requires it. |
| Execution role | `AWSLambdaBasicExecutionRole` | Just CloudWatch Logs. Add more permissions only if your handler needs them. |
## Cold start
First invocation in a new container takes ~8090 s (image extraction, Chromium binary mmap, JS engine warmup, no DNS/TLS caches). Subsequent warm invocations on the same container are 315 s.
For latency-sensitive use cases: provision concurrency, schedule a CloudWatch/EventBridge warmer ping, or accept the cold tail.
If you see empty/missing dynamic content on cold-start invocations, raise `max_settle_ms` in the event payload (e.g. `25000`) — the default `15000` is tuned for warm runs.
## License
The patched Chromium binary inside the upstream `cloakhq/cloakbrowser` image is governed by the **CloakBrowser Binary License** (published at https://github.com/CloakHQ/CloakBrowser/blob/main/BINARY-LICENSE.md). Internal organizational use (private ECR, your own scraping pipelines, your own business) is free. Exposing this Lambda as a paid API to third-party customers — i.e. browser-as-a-service — requires an OEM/SaaS license from CloakHQ (`cloakhq@pm.me`). Do not push the resulting image to a public registry; that would be redistribution and is prohibited.
@@ -0,0 +1,52 @@
#!/bin/sh
# Dual-mode entrypoint for the CloakBrowser Lambda image.
#
# 1. Always start Xvfb on :99 (same as the canonical bin/docker-entrypoint.sh)
# so headed Chromium works no matter how the container is invoked.
# 2. Detect whether the CMD looks like a Lambda handler (a single
# `module.func`-shaped argument). If yes, route through the Lambda runtime
# client (using the bundled aws-lambda-rie locally, or talking to the real
# Lambda Runtime API when AWS_LAMBDA_RUNTIME_API is set in production).
# 3. Otherwise exec the CMD directly — preserving the canonical Dockerfile's
# interaction surface (`python`, `cloakserve`, `cloaktest`, `node`, `bash`,
# `python examples/basic.py`, etc.).
set -e
mkdir -p /tmp/.X11-unix
chmod 1777 /tmp/.X11-unix 2>/dev/null || true
# Clean any stale Xvfb state. If a previous Xvfb died and left its lock file
# behind (we observed this in cold-start storms), a new Xvfb refuses to start
# with "Server is already active for display 99". Removing both files makes
# Xvfb start cleanly every time.
rm -f /tmp/.X99-lock /tmp/.X11-unix/X99
Xvfb :99 -screen 0 1920x1080x24 -nolisten tcp >/tmp/Xvfb.log 2>&1 &
# Wait for the X11 socket to appear AND for Xvfb to be ready to serve. The
# socket file appears at bind(), but listen() and the first accept() come
# slightly later — under cold-start CPU contention this gap matters.
i=0
while [ ! -e /tmp/.X11-unix/X99 ] && [ "$i" -lt 200 ]; do
i=$((i + 1))
sleep 0.05
done
# Small buffer after the socket appears so Xvfb has a moment to call listen()
# and start accepting clients. Cheap insurance against the bind/listen gap.
sleep 0.2
# Lambda handler shape: exactly one arg, dotted identifier (no spaces, no slashes,
# no leading dot). `python`, `cloakserve`, `cloaktest`, `bash`, `node` all fail
# this test and pass through to plain exec.
if [ $# -eq 1 ] && \
echo "$1" | grep -qE '^[a-zA-Z_][a-zA-Z0-9_]*(\.[a-zA-Z_][a-zA-Z0-9_]*)+$'; then
if [ -z "${AWS_LAMBDA_RUNTIME_API}" ]; then
# Local invocation via bundled RIE.
exec /usr/local/bin/aws-lambda-rie /usr/local/bin/python -m awslambdaric "$@"
else
# Real Lambda — runtime API endpoint already provided by the platform.
exec /usr/local/bin/python -m awslambdaric "$@"
fi
fi
exec "$@"
@@ -0,0 +1,336 @@
"""AWS Lambda handler for one-off stealth-browser invocations.
Always runs **headed** via the Xvfb display started by `lambda-entrypoint.sh`.
Event schema (all fields except `url` are optional):
Launch options (passed to cloakbrowser.launch_context_async):
url str required, the page to scrape
proxy str|dict http://user:pass@host:port or Playwright proxy dict
humanize bool False enable human-like mouse/keyboard/scroll
human_preset str "default" | "careful"
geoip bool False auto timezone+locale from proxy IP
timezone str IANA tz, e.g. "America/New_York"
locale str BCP-47, e.g. "en-US"
viewport {width,height} defaults to 1920x947 (cloakbrowser DEFAULT_VIEWPORT)
user_agent str custom UA (rare cloakbrowser sets one already)
extra_args list[str] additional Chromium CLI flags
Navigation options (passed to page.goto):
wait_until str "load"|"domcontentloaded"|"networkidle"|"commit"
default "domcontentloaded"
goto_timeout_ms int 30000
Post-navigation waits (run in this order if specified):
smart_wait bool ON by default if no other wait is set.
Polls document.outerHTML.length and bails when it
hasn't changed for `dom_stable_ms`. Handles lazy
hydration, async chunks, and lazy images, and is
immune to analytics beacons / long-poll that keep
the network busy without mutating the DOM.
dom_stable_ms int 1500 how long DOM must be quiet
max_settle_ms int 15000 hard cap on smart_wait
wait_for_load_state str "load"|"domcontentloaded"|"networkidle"
wait_for_load_state_timeout_ms int 30000
wait_for_selector str CSS or XPath selector
wait_for_selector_state str "attached"|"detached"|"visible"|"hidden", default "visible"
wait_for_selector_timeout_ms int 30000
wait_for_function str JS expression that returns truthy when ready
wait_for_function_timeout_ms int 30000
wait_ms int fixed pause in ms (page.wait_for_timeout)
Capture options:
screenshot bool True
full_page_screenshot bool False capture entire scrollable page
Retry orchestration:
retries int default 1. Number of retry attempts after the first
failure. Set to 0 to disable retries entirely (the
handler will fail fast on the first error).
Retried errors:
ERR_CERT_* -> retry with --ignore-certificate-errors
Timeout exceeded -> retry with goto_timeout_ms=90000, max_settle_ms=25000
ERR_CONNECTION_TIMED_OUT -> same as Timeout
Not retried (unrecoverable): ERR_NAME_NOT_RESOLVED,
ERR_SSL_PROTOCOL_ERROR, generic ERR_CONNECTION_REFUSED.
On final failure, the error message includes a
retry_history block with strategy + error per attempt.
Returns:
{"title": ..., "url": ..., "html": ..., "screenshot_b64"?: ...}
"""
from __future__ import annotations
import asyncio
import base64
import json
import logging
import subprocess
from pathlib import Path
from typing import Any
from cloakbrowser import launch_context_async
logger = logging.getLogger("cloakbrowser.lambda")
logger.setLevel(logging.INFO)
def _diag_snapshot() -> str:
"""Capture Xvfb status, Xvfb log, X11 socket state, and env for error reports."""
import os
parts = []
try:
r = subprocess.run(["pgrep", "-fa", "Xvfb"], capture_output=True, text=True)
parts.append(f"pgrep Xvfb: rc={r.returncode} stdout={r.stdout.strip()!r}")
except Exception as e:
parts.append(f"pgrep failed: {e}")
try:
r = subprocess.run(["ls", "-la", "/tmp/.X11-unix"], capture_output=True, text=True)
parts.append(f"ls /tmp/.X11-unix:\n{r.stdout}{r.stderr}")
except Exception as e:
parts.append(f"ls /tmp/.X11-unix failed: {e}")
try:
log = Path("/tmp/Xvfb.log").read_text()
parts.append(f"/tmp/Xvfb.log:\n{log}")
except Exception as e:
parts.append(f"Xvfb log unreadable: {e}")
parts.append(f"env: DISPLAY={os.environ.get('DISPLAY')!r} HOME={os.environ.get('HOME')!r}")
return "\n".join(parts)
def handler(event: dict, context: Any) -> dict:
return asyncio.run(_run(event))
def _build_launch_kwargs(event: dict) -> dict:
"""Translate the event dict into kwargs for launch_context_async.
Only includes keys explicitly set in the event so cloakbrowser's defaults
(DEFAULT_VIEWPORT etc.) kick in when fields are absent passing
viewport=None would *disable* viewport emulation, which we don't want.
"""
kwargs: dict = {
"headless": False, # always headed via Xvfb
"args": [
# Lambda /dev/shm is ~64 MB — Chromium crashes mid-render without this.
"--disable-dev-shm-usage",
# Lambda's restricted process model can't fork from Chromium's zygote
# — without this, child renderer processes fail to spawn.
"--no-zygote",
*event.get("extra_args", []),
],
}
for key in ("proxy", "humanize", "human_preset", "geoip",
"timezone", "locale", "viewport", "user_agent"):
if key in event:
kwargs[key] = event[key]
return kwargs
async def _smart_wait(page, dom_stable_ms: int = 1500, max_settle_ms: int = 15000) -> None:
"""Wait until the document HTML hasn't changed for `dom_stable_ms`.
Generic stopping condition for at-scale scraping when you can't tune
selectors per site. More robust than `networkidle` because it ignores
network activity that doesn't mutate the DOM (analytics beacons,
long-poll, websockets, web vitals streams).
"""
js = f"""
(() => {{
if (!window.__cb_settle) {{
window.__cb_settle = {{ len: -1, since: Date.now() }};
}}
const cur = document.documentElement.outerHTML.length;
const s = window.__cb_settle;
if (cur !== s.len) {{
s.len = cur;
s.since = Date.now();
return false;
}}
return (Date.now() - s.since) >= {int(dom_stable_ms)};
}})()
"""
try:
await page.wait_for_function(js, timeout=max_settle_ms, polling=200)
except Exception:
# Hit max_settle_ms cap — return what we have rather than fail the whole invoke
logger.warning("smart_wait hit max_settle_ms=%d cap", max_settle_ms)
_EXPLICIT_WAIT_KEYS = (
"wait_for_load_state", "wait_for_selector", "wait_for_function", "wait_ms",
)
async def _post_nav_waits(page, event: dict) -> None:
"""Run waits in priority order. smart_wait is the default unless the
caller asked for a more specific stopping condition."""
explicit = any(k in event for k in _EXPLICIT_WAIT_KEYS)
if event.get("smart_wait", not explicit):
await _smart_wait(
page,
dom_stable_ms=event.get("dom_stable_ms", 1500),
max_settle_ms=event.get("max_settle_ms", 15000),
)
if "wait_for_load_state" in event:
await page.wait_for_load_state(
event["wait_for_load_state"],
timeout=event.get("wait_for_load_state_timeout_ms", 30000),
)
if "wait_for_selector" in event:
await page.wait_for_selector(
event["wait_for_selector"],
state=event.get("wait_for_selector_state", "visible"),
timeout=event.get("wait_for_selector_timeout_ms", 30000),
)
if "wait_for_function" in event:
await page.wait_for_function(
event["wait_for_function"],
timeout=event.get("wait_for_function_timeout_ms", 30000),
)
if "wait_ms" in event:
await page.wait_for_timeout(event["wait_ms"])
async def _launch_with_retry(event: dict, attempts: int = 3, backoff_s: float = 0.3):
"""Retry launch_context_async up to `attempts` times with linear backoff.
Lambda cold-start storms occasionally race Xvfb readiness or hit transient
Chromium spawn failures both surface as "Target page, context or browser
has been closed" at launch. The failure is fast (~0.5s) so retries are
cheap, and a retry on a now-warm container almost always succeeds.
Pairs with the lock-cleanup + socket-poll in lambda-entrypoint.sh: the
entrypoint catches the common case at container init; this catches the
residual race when the first invocation hits before Xvfb is fully ready.
"""
last_err: Exception | None = None
for i in range(attempts):
try:
return await launch_context_async(**_build_launch_kwargs(event))
except Exception as e:
last_err = e
logger.warning("launch attempt %d/%d failed: %s",
i + 1, attempts, str(e)[:200])
if i + 1 < attempts:
await asyncio.sleep(backoff_s * (i + 1)) # 0.3s, 0.6s
raise last_err # type: ignore[misc]
def _classify_error(err: Exception) -> dict | None:
"""Map a Playwright error to a retry-strategy override dict, or None
if the error is unrecoverable.
Match on str(e) because Playwright errors carry their codes inside the
message (Error.__str__ includes ERR_CERT_AUTHORITY_INVALID etc.); there
is no stable structured `.error_code` attribute to rely on.
Strategies (priority order first match wins):
ERR_CERT_* -> --ignore-certificate-errors + 60s goto budget
Timeout exceeded -> 90s goto budget + 25s smart_wait cap
ERR_CONNECTION_TIMED_OUT -> same as Timeout
Returns None for unrecoverable site issues (DNS, SSL, refused, HTTP 4xx/5xx).
"""
msg = str(err)
if "ERR_CERT" in msg:
return {
"extra_args": ["--ignore-certificate-errors"],
"goto_timeout_ms": 60000,
}
if ("Timeout" in msg and "exceeded" in msg) or "ERR_CONNECTION_TIMED_OUT" in msg:
return {
"goto_timeout_ms": 90000,
"max_settle_ms": 25000,
}
return None
async def _attempt_scrape(url: str, event: dict) -> dict:
"""One self-contained scrape attempt: launch, navigate, wait, capture, close.
Extracted from `_run` so the retry loop can call it repeatedly with an
overridden event dict. Each attempt relaunches the browser uniform
behavior across strategies (the cert-bypass strategy *requires* a relaunch
because `--ignore-certificate-errors` is a Chromium CLI arg, not a per-
context switch), and the ~3-5s relaunch cost is fine on the slow path.
"""
ctx = await _launch_with_retry(event)
try:
page = await ctx.new_page()
await page.goto(
url,
wait_until=event.get("wait_until", "domcontentloaded"),
timeout=event.get("goto_timeout_ms", 30000),
)
await _post_nav_waits(page, event)
result: dict = {
"title": await page.title(),
"url": page.url,
"html": await page.content(),
}
if event.get("screenshot", True):
png = await page.screenshot(
full_page=event.get("full_page_screenshot", False),
)
result["screenshot_b64"] = base64.b64encode(png).decode()
return result
finally:
try:
await ctx.close()
except Exception:
pass
def _raise_with_history(err: Exception, history: list[dict]) -> None:
"""Surface a final failure with a retry_history block embedded in the
error message, so callers see what was tried before bailing."""
diag = _diag_snapshot()
if history:
diag = "retry_history: " + json.dumps(history, default=str) + "\n\n" + diag
logger.error("scrape failed (after %d retries): %s\nDIAG:\n%s",
len(history), err, diag)
raise RuntimeError(f"scrape failed: {err}\n--- DIAG ---\n{diag}") from err
async def _run(event: dict) -> dict:
"""Top-level scrape with strategy-based retry orchestration.
First attempt uses the event verbatim. If it fails with a classifiable
error (cert / timeout), retry with that strategy's overrides merged into
the event. `retries` bounds the number of strategy retries (default 1;
set to 0 to disable retry entirely).
"""
url = event["url"]
retries_left = max(0, int(event.get("retries", 1)))
history: list[dict] = []
current_event = event
while True:
try:
return await _attempt_scrape(url, current_event)
except Exception as e:
if retries_left <= 0:
_raise_with_history(e, history)
strategy = _classify_error(e)
if strategy is None:
_raise_with_history(e, history)
history.append({
"attempt": len(history) + 1,
"error": str(e)[:300],
"strategy": strategy,
})
logger.warning("attempt %d failed (%s); retrying with strategy=%s",
len(history), str(e)[:120], strategy)
merged_args = list(current_event.get("extra_args", [])) + list(strategy.get("extra_args", []))
current_event = {**current_event, **strategy, "extra_args": merged_args}
retries_left -= 1
# No backoff: strategy overrides change goto budget directly;
# the prior failure was either fast (cert reject) or already
# waited its full timeout. Container is warm.
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "cloakbrowser",
"version": "0.3.25",
"version": "0.3.27",
"description": "Stealth Chromium that passes every bot detection test. Drop-in Playwright/Puppeteer replacement with source-level fingerprint patches.",
"type": "module",
"main": "dist/index.js",
+1 -1
View File
@@ -34,7 +34,7 @@ export const PLATFORM_CHROMIUM_VERSIONS: Record<string, string> = {
"linux-arm64": "146.0.7680.177.3",
"darwin-arm64": "145.0.7632.109.2",
"darwin-x64": "145.0.7632.109.2",
"windows-x64": "145.0.7632.159.7",
"windows-x64": "146.0.7680.177.4",
};
// ---------------------------------------------------------------------------
+77 -31
View File
@@ -48,16 +48,16 @@
import type { Browser, Page, Frame, CDPSession, ElementHandle, BrowserContext } from 'puppeteer-core';
import type { HumanConfig } from '../human/config.js';
import { resolveConfig, rand, randRange, sleep } from '../human/config.js';
import { resolveConfig, mergeConfig, rand, randRange, sleep } from '../human/config.js';
import { RawMouse, RawKeyboard, humanMove, humanClick, clickTarget, humanIdle } from '../human/mouse.js';
import { humanType } from './keyboard.js';
import { scrollToElement, smoothWheel } from './scroll.js';
import { scrollToElement, humanScrollIntoView, smoothWheel } from './scroll.js';
export type { HumanConfig } from '../human/config.js';
export { resolveConfig } from '../human/config.js';
export { resolveConfig, mergeConfig } from '../human/config.js';
export { humanMove, humanClick, clickTarget, humanIdle } from '../human/mouse.js';
export { humanType } from './keyboard.js';
export { scrollToElement } from './scroll.js';
export { scrollToElement, humanScrollIntoView } from './scroll.js';
// ============================================================================
@@ -329,52 +329,55 @@ function patchPage(page: Page, cfg: HumanConfig, cursor: CursorState): void {
// ==== click (with clickCount support for dblclick) ====
const humanClickFn = async (selector: string, options?: any) => {
await ensureCursorInit();
if (cfg.idle_between_actions) {
await humanIdle(raw, rand(cfg.idle_between_duration[0], cfg.idle_between_duration[1]), cursor.x, cursor.y, cfg);
const callCfg = mergeConfig(cfg, options?.human_config);
if (callCfg.idle_between_actions) {
await humanIdle(raw, rand(callCfg.idle_between_duration[0], callCfg.idle_between_duration[1]), cursor.x, cursor.y, callCfg);
}
const { box, cursorX, cursorY } = await scrollToElement(page, raw, selector, cursor.x, cursor.y, cfg);
const { box, cursorX, cursorY } = await scrollToElement(page, raw, selector, cursor.x, cursor.y, callCfg, options?.timeout);
cursor.x = cursorX;
cursor.y = cursorY;
const isInput = await isInputElement(stealth, page, selector);
const target = clickTarget(box, isInput, cfg);
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, cfg);
const target = clickTarget(box, isInput, callCfg);
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, callCfg);
cursor.x = target.x;
cursor.y = target.y;
const clickCount = options?.clickCount ?? options?.count ?? 1;
if (clickCount >= 2) {
await humanClick(raw, isInput, cfg);
await humanClick(raw, isInput, callCfg);
await sleep(rand(40, 90));
await raw.down({ clickCount: 2 });
await sleep(rand(30, 60));
await raw.up({ clickCount: 2 });
} else {
await humanClick(raw, isInput, cfg);
await humanClick(raw, isInput, callCfg);
}
};
// ==== hover ====
const humanHoverFn = async (selector: string, options?: any) => {
await ensureCursorInit();
if (cfg.idle_between_actions) {
await humanIdle(raw, rand(cfg.idle_between_duration[0], cfg.idle_between_duration[1]), cursor.x, cursor.y, cfg);
const callCfg = mergeConfig(cfg, options?.human_config);
if (callCfg.idle_between_actions) {
await humanIdle(raw, rand(callCfg.idle_between_duration[0], callCfg.idle_between_duration[1]), cursor.x, cursor.y, callCfg);
}
const { box, cursorX, cursorY } = await scrollToElement(page, raw, selector, cursor.x, cursor.y, cfg);
const { box, cursorX, cursorY } = await scrollToElement(page, raw, selector, cursor.x, cursor.y, callCfg, options?.timeout);
cursor.x = cursorX;
cursor.y = cursorY;
const target = clickTarget(box, false, cfg);
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, cfg);
const target = clickTarget(box, false, callCfg);
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, callCfg);
cursor.x = target.x;
cursor.y = target.y;
};
// ==== type ====
const humanTypeFn = async (selector: string, text: string, options?: any) => {
await sleep(randRange(cfg.field_switch_delay));
await humanClickFn(selector);
const callCfg = mergeConfig(cfg, options?.human_config);
await sleep(randRange(callCfg.field_switch_delay));
await humanClickFn(selector, options);
await sleep(rand(100, 250));
const cdp = await ensureCdp();
await humanType(page, rawKb, text, cfg, cdp);
await humanType(page, rawKb, text, callCfg, cdp);
};
// ==== select ====
@@ -577,6 +580,9 @@ function patchSingleElementHandle(
const origElDragAndDrop = (el as any).dragAndDrop?.bind(el);
const origElSelect = (el as any).select?.bind(el);
const origElDrop = (el as any).drop?.bind(el);
// Puppeteer v22+ adds ElementHandle.scrollIntoView(); earlier versions
// expose it implicitly via evaluate(node => node.scrollIntoView()).
const origElScrollIntoView = (el as any).scrollIntoView?.bind(el);
// --- Nested selectors ---
const origEl$ = el.$.bind(el);
@@ -603,20 +609,34 @@ function patchSingleElementHandle(
return child;
};
// --- Helper: get box and move cursor ---
const moveToElement = async () => {
// --- Helper: get box and move cursor. Accepts a per-call ``callCfg``
// so type/fill overrides like ``el.type(text, { human_config: {...} })``
// carry through to mouse timing for that single call. Also scrolls into
// view first so off-screen elements work (#129, #172 follow-up).
const moveToElement = async (callCfg: HumanConfig = cfg) => {
await (page as any)._ensureCursorInit();
try {
const { cursorX, cursorY } = await humanScrollIntoView(
page, raw,
() => el.boundingBox().then(b => b ?? null),
cursor.x, cursor.y, callCfg,
);
cursor.x = cursorX;
cursor.y = cursorY;
} catch { /* let boundingBox() decide */ }
const box = await el.boundingBox();
if (!box) return null;
const isInp = await isInputElementHandle(stealth, el);
const target = clickTarget(box, isInp, cfg);
const target = clickTarget(box, isInp, callCfg);
if (cfg.idle_between_actions) {
await humanIdle(raw, rand(cfg.idle_between_duration[0], cfg.idle_between_duration[1]), cursor.x, cursor.y, cfg);
if (callCfg.idle_between_actions) {
await humanIdle(raw, rand(callCfg.idle_between_duration[0], callCfg.idle_between_duration[1]), cursor.x, cursor.y, callCfg);
}
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, cfg);
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, callCfg);
cursor.x = target.x;
cursor.y = target.y;
return { box, isInp };
@@ -624,18 +644,19 @@ function patchSingleElementHandle(
// --- el.click() ---
(el as any).click = async (options?: any) => {
const info = await moveToElement();
const callCfg = mergeConfig(cfg, options?.human_config);
const info = await moveToElement(callCfg);
if (!info) return origElClick(options);
const clickCount = options?.clickCount ?? options?.count ?? 1;
if (clickCount >= 2) {
await humanClick(raw, info.isInp, cfg);
await humanClick(raw, info.isInp, callCfg);
await sleep(rand(40, 90));
await raw.down({ clickCount: 2 });
await sleep(rand(30, 60));
await raw.up({ clickCount: 2 });
} else {
await humanClick(raw, info.isInp, cfg);
await humanClick(raw, info.isInp, callCfg);
}
};
@@ -647,14 +668,39 @@ function patchSingleElementHandle(
// --- el.type() ---
(el as any).type = async (text: string, options?: any) => {
const info = await moveToElement();
const callCfg = mergeConfig(cfg, options?.human_config);
const info = await moveToElement(callCfg);
if (!info) return origElType(text, options);
await humanClick(raw, info.isInp, cfg);
await humanClick(raw, info.isInp, callCfg);
await sleep(rand(100, 250));
const cdp = await stealth.getCdpSession().catch(() => null);
await humanType(page, rawKb, text, cfg, cdp);
await humanType(page, rawKb, text, callCfg, cdp);
};
// --- el.scrollIntoView() ---
// Puppeteer-only equivalent of Playwright's scrollIntoViewIfNeeded.
// Replaces the native snap-scroll (a strong bot signal) with the same
// accelerate → cruise → decelerate → overshoot wheel sequence used by
// page.click(). Only patched when the underlying ElementHandle exposes
// ``scrollIntoView`` (Puppeteer v22+).
if (origElScrollIntoView) {
(el as any).scrollIntoView = async (options?: any) => {
const callCfg = mergeConfig(cfg, options?.human_config);
await (page as any)._ensureCursorInit();
try {
const { cursorX, cursorY } = await humanScrollIntoView(
page, raw,
() => el.boundingBox().then(b => b ?? null),
cursor.x, cursor.y, callCfg,
);
cursor.x = cursorX;
cursor.y = cursorY;
} catch {
return origElScrollIntoView(options);
}
};
}
// --- el.press() ---
if (origElPress) {
(el as any).press = async (key: string, options?: any) => {
+58 -21
View File
@@ -5,7 +5,7 @@
* Changes from Playwright version:
* - page.viewport() instead of page.viewportSize()
* - page.$(selector) + el.boundingBox() instead of page.locator().boundingBox()
* - No timeout parameter on boundingBox()
* - boundingBox() has no timeout param we poll page.$() up to ``timeout`` ms
*/
import type { Page } from 'puppeteer-core';
@@ -55,22 +55,40 @@ export async function smoothWheel(
}
}
async function getElementBox(page: Page, selector: string): Promise<ElementBounds | null> {
try {
const el = await page.$(selector);
if (!el) return null;
const box = await el.boundingBox();
if (!box) return null;
return { x: box.x, y: box.y, width: box.width, height: box.height };
} catch {
return null;
/**
* Poll ``page.$(selector)`` for up to ``timeout`` ms, returning the element's
* bounding box when found. ``timeout`` defaults to 30000ms when not specified.
*/
async function getElementBox(
page: Page,
selector: string,
timeout: number = 30000,
): Promise<ElementBounds | null> {
const start = Date.now();
const pollInterval = 100;
while (true) {
try {
const el = await page.$(selector);
if (el) {
const box = await el.boundingBox();
if (box) return { x: box.x, y: box.y, width: box.width, height: box.height };
}
} catch { /* keep polling */ }
if (Date.now() - start >= timeout) return null;
await sleep(pollInterval);
}
}
export async function scrollToElement(
/**
* Humanized scrolling that takes an arbitrary ``getBox`` callable.
* Used by both ``scrollToElement`` (selector-based) and the ElementHandle
* ``scrollIntoView`` patch.
*/
export async function humanScrollIntoView(
page: Page,
raw: RawMouse,
selector: string,
getBox: () => Promise<ElementBounds | null>,
cursorX: number,
cursorY: number,
cfg: HumanConfig,
@@ -78,12 +96,8 @@ export async function scrollToElement(
const viewport = page.viewport();
if (!viewport) throw new Error('Viewport size not available');
let box = await getElementBox(page, selector);
if (!box) {
await sleep(200);
box = await getElementBox(page, selector);
if (!box) throw new Error(`Element not found: ${selector}`);
}
let box = await getBox();
if (!box) throw new Error('Element not found while scrolling into view');
if (isInViewport(box, viewport.height, cfg)) {
return { box, cursorX, cursorY };
@@ -134,7 +148,7 @@ export async function scrollToElement(
await sleep(pause);
if (i % 3 === 2 || i === totalClicks - 1) {
box = await getElementBox(page, selector);
box = await getBox();
if (box && isInViewport(box, viewport.height, cfg)) {
break;
}
@@ -159,8 +173,31 @@ export async function scrollToElement(
await sleep(randRange(cfg.scroll_settle_delay));
box = await getElementBox(page, selector);
if (!box) throw new Error(`Element lost after scrolling: ${selector}`);
box = await getBox();
if (!box) throw new Error('Element lost after scrolling into view');
return { box, cursorX, cursorY };
}
/**
* Selector-based humanized scroll (Puppeteer).
*
* ``timeout`` controls how long we poll ``page.$(selector)`` before giving up,
* so callers like ``page.click('#x', { timeout: 5000 })`` can wait longer for
* slow-loading elements (#172). Default matches Playwright's 30000ms when not specified.
*/
export async function scrollToElement(
page: Page,
raw: RawMouse,
selector: string,
cursorX: number,
cursorY: number,
cfg: HumanConfig,
timeout?: number,
): Promise<{ box: ElementBounds; cursorX: number; cursorY: number }> {
return humanScrollIntoView(
page, raw,
() => getElementBox(page, selector, timeout),
cursorX, cursorY, cfg,
);
}
+16
View File
@@ -201,6 +201,22 @@ export function resolveConfig(
return { ...base, ...overrides };
}
/**
* Merge a partial overrides object on top of an existing HumanConfig.
* Returns a new object the original ``cfg`` is never mutated.
*
* Used by per-call overrides such as ``page.type(sel, text, { human_config: { typing_delay: 30 } })``
* so the same patched page can type different fields at different speeds
* without re-patching.
*/
export function mergeConfig(
cfg: HumanConfig,
overrides?: Partial<HumanConfig> | null,
): HumanConfig {
if (!overrides) return cfg;
return { ...cfg, ...overrides };
}
// ---------------------------------------------------------------------------
// Utility: random number in range
+67 -16
View File
@@ -18,9 +18,10 @@
import type { Page, Frame, ElementHandle, CDPSession } from 'playwright-core';
import type { HumanConfig } from './config.js';
import { rand, randRange, sleep } from './config.js';
import { rand, randRange, sleep, mergeConfig } from './config.js';
import { RawMouse, RawKeyboard, humanMove, humanClick, clickTarget, humanIdle } from './mouse.js';
import { humanType } from './keyboard.js';
import { humanScrollIntoView } from './scroll.js';
// --- Platform-aware select-all shortcut ---
const SELECT_ALL = process.platform === 'darwin' ? 'Meta+a' : 'Control+a';
@@ -102,6 +103,7 @@ export function patchSingleElementHandle(
const origElSetChecked = (el as any).setChecked?.bind(el);
const origElTap = el.tap.bind(el);
const origElFocus = el.focus.bind(el);
const origElScrollIntoViewIfNeeded = (el as any).scrollIntoViewIfNeeded?.bind(el);
// Nested selectors
const origEl$ = el.$.bind(el);
@@ -130,22 +132,42 @@ export function patchSingleElementHandle(
};
// --- Helper: get bounding box and move cursor to element ---
const moveToElement = async () => {
// Accepts a per-call ``callCfg`` so type/fill overrides like
// ``el.type(text, { human_config: { typing_delay: 30 } })`` carry through to
// mouse movement & idle timing for that single call.
// Also scrolls the element into view first so off-screen elements work
// (#129, #172 follow-up): otherwise boundingBox() returns null and we'd
// silently fall back to the unpatched native method.
const moveToElement = async (callCfg: HumanConfig = cfg) => {
// Ensure cursor is initialized
const ensureCursorInit = (page as any)._ensureCursorInit;
if (ensureCursorInit) await ensureCursorInit();
// Scroll into view first so boundingBox() returns coordinates even when
// the element starts below the fold. Best-effort — if humanScrollIntoView
// throws (e.g. detached element), we let boundingBox() decide whether to
// proceed or fall back to the original method.
try {
const { cursorX, cursorY } = await humanScrollIntoView(
page, raw,
() => el.boundingBox(),
cursor.x, cursor.y, callCfg,
);
cursor.x = cursorX;
cursor.y = cursorY;
} catch { /* let boundingBox() decide */ }
const box = await el.boundingBox();
if (!box) return null;
const isInp = await isInputElementHandle(stealth, el);
const target = clickTarget(box, isInp, cfg);
const target = clickTarget(box, isInp, callCfg);
if (cfg.idle_between_actions) {
await humanIdle(raw, rand(cfg.idle_between_duration[0], cfg.idle_between_duration[1]), cursor.x, cursor.y, cfg);
if (callCfg.idle_between_actions) {
await humanIdle(raw, rand(callCfg.idle_between_duration[0], callCfg.idle_between_duration[1]), cursor.x, cursor.y, callCfg);
}
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, cfg);
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, callCfg);
cursor.x = target.x;
cursor.y = target.y;
return { box, isInp };
@@ -153,14 +175,16 @@ export function patchSingleElementHandle(
// --- el.click() ---
(el as any).click = async (options?: any) => {
const info = await moveToElement();
const callCfg = mergeConfig(cfg, options?.human_config);
const info = await moveToElement(callCfg);
if (!info) return origElClick(options);
await humanClick(raw, info.isInp, cfg);
await humanClick(raw, info.isInp, callCfg);
};
// --- el.dblclick() ---
(el as any).dblclick = async (options?: any) => {
const info = await moveToElement();
const callCfg = mergeConfig(cfg, options?.human_config);
const info = await moveToElement(callCfg);
if (!info) return origElDblclick(options);
await raw.down({ clickCount: 2 });
await sleep(rand(30, 60));
@@ -169,27 +193,30 @@ export function patchSingleElementHandle(
// --- el.hover() ---
(el as any).hover = async (options?: any) => {
const info = await moveToElement();
const callCfg = mergeConfig(cfg, options?.human_config);
const info = await moveToElement(callCfg);
if (!info) return origElHover(options);
// Just move — no click
};
// --- el.type() ---
(el as any).type = async (text: string, options?: any) => {
const info = await moveToElement();
const callCfg = mergeConfig(cfg, options?.human_config);
const info = await moveToElement(callCfg);
if (!info) return origElType(text, options);
await humanClick(raw, info.isInp, cfg);
await humanClick(raw, info.isInp, callCfg);
await sleep(rand(100, 250));
let cdpSession: CDPSession | null = null;
try { cdpSession = await stealth?.getCdpSession(); } catch {}
await humanType(page, rawKb, text, cfg, cdpSession);
await humanType(page, rawKb, text, callCfg, cdpSession);
};
// --- el.fill() ---
(el as any).fill = async (value: string, options?: any) => {
const info = await moveToElement();
const callCfg = mergeConfig(cfg, options?.human_config);
const info = await moveToElement(callCfg);
if (!info) return origElFill(value, options);
await humanClick(raw, info.isInp, cfg);
await humanClick(raw, info.isInp, callCfg);
await sleep(rand(100, 250));
// Clear existing content
await originals.keyboardPress(SELECT_ALL);
@@ -198,7 +225,7 @@ export function patchSingleElementHandle(
await sleep(rand(50, 150));
let cdpSession: CDPSession | null = null;
try { cdpSession = await stealth?.getCdpSession(); } catch {}
await humanType(page, rawKb, value, cfg, cdpSession);
await humanType(page, rawKb, value, callCfg, cdpSession);
};
// --- el.press() ---
@@ -268,6 +295,30 @@ export function patchSingleElementHandle(
await moveToElement(); // human-like Bézier cursor movement
await origElFocus(); // programmatic focus, no click
};
// --- el.scrollIntoViewIfNeeded() ---
// Playwright's native version snaps the page — a strong bot signal.
// Replace with the same accelerate → cruise → decelerate → overshoot
// wheel sequence used by page.click() etc. Falls back to the native
// method if the element is detached or scrolling fails.
if (origElScrollIntoViewIfNeeded) {
(el as any).scrollIntoViewIfNeeded = async (options?: any) => {
const callCfg = mergeConfig(cfg, options?.human_config);
const ensureCursorInit = (page as any)._ensureCursorInit;
if (ensureCursorInit) await ensureCursorInit();
try {
const { cursorX, cursorY } = await humanScrollIntoView(
page, raw,
() => el.boundingBox(),
cursor.x, cursor.y, callCfg,
);
cursor.x = cursorX;
cursor.y = cursorY;
} catch {
return origElScrollIntoViewIfNeeded(options);
}
};
}
}
+34 -28
View File
@@ -23,16 +23,16 @@
*/
import type { Browser, BrowserContext, Page, Frame, CDPSession } from 'playwright-core';
import { HumanConfig, resolveConfig, rand, randRange, sleep } from './config.js';
import { HumanConfig, resolveConfig, mergeConfig, rand, randRange, sleep } from './config.js';
import { RawMouse, RawKeyboard, humanMove, humanClick, clickTarget, humanIdle } from './mouse.js';
import { humanType } from './keyboard.js';
import { scrollToElement } from './scroll.js';
import { scrollToElement, humanScrollIntoView } from './scroll.js';
import { patchPageElementHandles, patchFrameElementHandles, patchSingleElementHandle } from './elementhandle.js';
export { HumanConfig, resolveConfig } from './config.js';
export { HumanConfig, resolveConfig, mergeConfig } from './config.js';
export { humanMove, humanClick, clickTarget, humanIdle } from './mouse.js';
export { humanType } from './keyboard.js';
export { scrollToElement } from './scroll.js';
export { scrollToElement, humanScrollIntoView } from './scroll.js';
export { patchSingleElementHandle } from './elementhandle.js';
// --- Platform-aware select-all shortcut (macOS uses Meta, others use Control) ---
@@ -305,32 +305,34 @@ function patchPage(page: Page, cfg: HumanConfig, cursor: CursorState): void {
// --- click ---
const humanClickFn = async (selector: string, options?: any) => {
await ensureCursorInit();
if (cfg.idle_between_actions) {
await humanIdle(raw, rand(cfg.idle_between_duration[0], cfg.idle_between_duration[1]), cursor.x, cursor.y, cfg);
const callCfg = mergeConfig(cfg, options?.human_config);
if (callCfg.idle_between_actions) {
await humanIdle(raw, rand(callCfg.idle_between_duration[0], callCfg.idle_between_duration[1]), cursor.x, cursor.y, callCfg);
}
const { box, cursorX, cursorY } = await scrollToElement(page, raw, selector, cursor.x, cursor.y, cfg);
const { box, cursorX, cursorY } = await scrollToElement(page, raw, selector, cursor.x, cursor.y, callCfg, options?.timeout);
cursor.x = cursorX;
cursor.y = cursorY;
const isInput = await isInputElement(stealth, page, selector);
const target = clickTarget(box, isInput, cfg);
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, cfg);
const target = clickTarget(box, isInput, callCfg);
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, callCfg);
cursor.x = target.x;
cursor.y = target.y;
await humanClick(raw, isInput, cfg);
await humanClick(raw, isInput, callCfg);
};
// --- dblclick ---
const humanDblclickFn = async (selector: string, options?: any) => {
await ensureCursorInit();
if (cfg.idle_between_actions) {
await humanIdle(raw, rand(cfg.idle_between_duration[0], cfg.idle_between_duration[1]), cursor.x, cursor.y, cfg);
const callCfg = mergeConfig(cfg, options?.human_config);
if (callCfg.idle_between_actions) {
await humanIdle(raw, rand(callCfg.idle_between_duration[0], callCfg.idle_between_duration[1]), cursor.x, cursor.y, callCfg);
}
const { box, cursorX, cursorY } = await scrollToElement(page, raw, selector, cursor.x, cursor.y, cfg);
const { box, cursorX, cursorY } = await scrollToElement(page, raw, selector, cursor.x, cursor.y, callCfg, options?.timeout);
cursor.x = cursorX;
cursor.y = cursorY;
const isInput = await isInputElement(stealth, page, selector);
const target = clickTarget(box, isInput, cfg);
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, cfg);
const target = clickTarget(box, isInput, callCfg);
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, callCfg);
cursor.x = target.x;
cursor.y = target.y;
await raw.down({ clickCount: 2 });
@@ -341,38 +343,41 @@ function patchPage(page: Page, cfg: HumanConfig, cursor: CursorState): void {
// --- hover ---
const humanHoverFn = async (selector: string, options?: any) => {
await ensureCursorInit();
if (cfg.idle_between_actions) {
await humanIdle(raw, rand(cfg.idle_between_duration[0], cfg.idle_between_duration[1]), cursor.x, cursor.y, cfg);
const callCfg = mergeConfig(cfg, options?.human_config);
if (callCfg.idle_between_actions) {
await humanIdle(raw, rand(callCfg.idle_between_duration[0], callCfg.idle_between_duration[1]), cursor.x, cursor.y, callCfg);
}
const { box, cursorX, cursorY } = await scrollToElement(page, raw, selector, cursor.x, cursor.y, cfg);
const { box, cursorX, cursorY } = await scrollToElement(page, raw, selector, cursor.x, cursor.y, callCfg, options?.timeout);
cursor.x = cursorX;
cursor.y = cursorY;
const target = clickTarget(box, false, cfg);
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, cfg);
const target = clickTarget(box, false, callCfg);
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, callCfg);
cursor.x = target.x;
cursor.y = target.y;
};
// --- type ---
const humanTypeFn = async (selector: string, text: string, options?: any) => {
await sleep(randRange(cfg.field_switch_delay));
await humanClickFn(selector);
const callCfg = mergeConfig(cfg, options?.human_config);
await sleep(randRange(callCfg.field_switch_delay));
await humanClickFn(selector, options);
await sleep(rand(100, 250));
const cdp = await ensureCdp();
await humanType(page, rawKb, text, cfg, cdp);
await humanType(page, rawKb, text, callCfg, cdp);
};
// --- fill (clears existing content first) ---
const humanFillFn = async (selector: string, value: string, options?: any) => {
await sleep(randRange(cfg.field_switch_delay));
await humanClickFn(selector);
const callCfg = mergeConfig(cfg, options?.human_config);
await sleep(randRange(callCfg.field_switch_delay));
await humanClickFn(selector, options);
await sleep(rand(100, 250));
await originals.keyboardPress(SELECT_ALL);
await sleep(rand(30, 80));
await originals.keyboardPress('Backspace');
await sleep(rand(50, 150));
const cdp = await ensureCdp();
await humanType(page, rawKb, value, cfg, cdp);
await humanType(page, rawKb, value, callCfg, cdp);
};
// --- clear ---
@@ -426,12 +431,13 @@ function patchPage(page: Page, cfg: HumanConfig, cursor: CursorState): void {
// --- pressSequentially ---
const humanPressSequentiallyFn = async (selector: string, text: string, options?: any) => {
const callCfg = mergeConfig(cfg, options?.human_config);
if (!await isSelectorFocused(stealth, page, selector)) {
await humanClickFn(selector);
await humanClickFn(selector, options);
}
await sleep(rand(100, 250));
const cdp = await ensureCdp();
await humanType(page, rawKb, text, cfg, cdp);
await humanType(page, rawKb, text, callCfg, cdp);
};
// --- tap ---
+43 -13
View File
@@ -38,10 +38,17 @@ async function smoothWheel(raw: RawMouse, delta: number, cfg: HumanConfig): Prom
}
}
export async function scrollToElement(
/**
* Humanized scrolling that takes an arbitrary ``getBox`` callable.
*
* Used by both ``scrollToElement`` (selector-based) and the ElementHandle
* ``scrollIntoViewIfNeeded`` patch so the same accelerate cruise
* decelerate overshoot behavior runs everywhere.
*/
export async function humanScrollIntoView(
page: Page,
raw: RawMouse,
selector: string,
getBox: () => Promise<ElementBounds | null>,
cursorX: number,
cursorY: number,
cfg: HumanConfig,
@@ -49,12 +56,8 @@ export async function scrollToElement(
const viewport = page.viewportSize();
if (!viewport) throw new Error('Viewport size not available');
let box = await getElementBox(page, selector);
if (!box) {
await sleep(200);
box = await getElementBox(page, selector);
if (!box) throw new Error(`Element not found: ${selector}`);
}
let box = await getBox();
if (!box) throw new Error('Element not found while scrolling into view');
if (isInViewport(box, viewport.height, cfg)) {
return { box, cursorX, cursorY };
@@ -107,7 +110,7 @@ export async function scrollToElement(
// Check visibility every 3 steps
if (i % 3 === 2 || i === totalClicks - 1) {
box = await getElementBox(page, selector);
box = await getBox();
if (box && isInViewport(box, viewport.height, cfg)) {
break;
}
@@ -133,16 +136,43 @@ export async function scrollToElement(
// Settle
await sleep(randRange(cfg.scroll_settle_delay));
box = await getElementBox(page, selector);
if (!box) throw new Error(`Element lost after scrolling: ${selector}`);
box = await getBox();
if (!box) throw new Error('Element lost after scrolling into view');
return { box, cursorX, cursorY };
}
async function getElementBox(page: Page, selector: string): Promise<ElementBounds | null> {
/**
* Selector-based humanized scroll.
*
* ``timeout`` is forwarded to Playwright's ``boundingBox({ timeout })`` so
* callers like ``page.click('#x', { timeout: 5000 })`` can wait longer for
* slow-loading elements (#172). Default matches Playwright's 30000ms when not specified.
*/
export async function scrollToElement(
page: Page,
raw: RawMouse,
selector: string,
cursorX: number,
cursorY: number,
cfg: HumanConfig,
timeout?: number,
): Promise<{ box: ElementBounds; cursorX: number; cursorY: number }> {
return humanScrollIntoView(
page, raw,
() => getElementBox(page, selector, timeout),
cursorX, cursorY, cfg,
);
}
async function getElementBox(
page: Page,
selector: string,
timeout: number = 30000,
): Promise<ElementBounds | null> {
const el = page.locator(selector).first();
try {
const box = await el.boundingBox({ timeout: 2000 });
const box = await el.boundingBox({ timeout });
return box;
} catch {
return null;
+91 -1
View File
@@ -43,6 +43,40 @@ export function isSocksProxy(proxy: string | ProxyDict | undefined | null): bool
return /^socks5h?:\/\//i.test(url);
}
/**
* Build a SOCKS URL from already-percent-encoded credentials and a host suffix.
*
* `encPass === null` means no password (no colon in userinfo). Empty string
* means present-but-empty (colon preserved).
*/
function assembleSocksUrl(
scheme: string,
encUser: string,
encPass: string | null,
hostAndRest: string,
): string {
let userinfo: string;
if (encPass !== null) {
userinfo = `${encUser}:${encPass}@`;
} else if (encUser) {
userinfo = `${encUser}@`;
} else {
userinfo = "";
}
return `${scheme}://${userinfo}${hostAndRest}`;
}
/**
* Lenient percent-decode that handles malformed escapes gracefully, matching
* Python's ``urllib.parse.unquote``: valid ``%XX`` sequences are decoded,
* bare ``%`` not followed by two hex digits is left as a literal ``%``.
*/
function lenientDecodeURIComponent(s: string): string {
return s.replace(/%([0-9A-Fa-f]{2})|%/g, (match, hex) =>
hex ? String.fromCharCode(parseInt(hex, 16)) : "%",
);
}
/**
* Reconstruct a SOCKS5 URL with inline credentials from a proxy dict.
*/
@@ -55,6 +89,60 @@ export function reconstructSocksUrl(proxy: ProxyDict): string {
return url.href.replace(/\/$/, "");
}
/**
* Re-encode credentials in a SOCKS5 URL string so Chromium's parser doesn't
* truncate them at special chars like '='. Idempotent: pre-encoded input stays
* the same (decoded then re-encoded).
*
* Parsing is done manually rather than via `new URL` + setters, because WHATWG
* URL's username/password setters re-encode `%` on assignment, causing
* double-encoding when we round-trip decode-then-encode.
*
* On any unexpected failure, logs a warning and returns the original string
* so Chromium's own error handling can surface the real problem.
*/
export function normalizeSocksStringUrl(urlStr: string): string {
// Split userinfo from host at the LAST '@' (RFC 3986), so a raw '@' inside
// a password like `socks5://user:p@ss@host:1080` parses correctly. Matches
// Python urlparse's rpartition('@') behavior.
const schemeMatch = urlStr.match(/^([a-z][a-z0-9+\-.]*):\/\/(.*)$/i);
if (!schemeMatch) return urlStr;
const [, scheme, rest] = schemeMatch;
const hostStart = rest.search(/[/?#]/);
const authority = hostStart === -1 ? rest : rest.slice(0, hostStart);
const suffix = hostStart === -1 ? "" : rest.slice(hostStart);
const atIdx = authority.lastIndexOf("@");
if (atIdx === -1) return urlStr; // no creds
const userinfo = authority.slice(0, atIdx);
const hostPart = authority.slice(atIdx + 1);
// Validate port (matches Python's urlparse().port ValueError guard).
// Extract port after last ':' — but skip IPv6 brackets (e.g. [::1]:1080).
const bracketEnd = hostPart.lastIndexOf("]");
const portColonIdx = hostPart.indexOf(":", Math.max(bracketEnd, 0));
if (portColonIdx !== -1) {
const portStr = hostPart.slice(portColonIdx + 1);
if (portStr && !/^\d+$/.test(portStr)) {
console.warn(`[cloakbrowser] Malformed SOCKS5 proxy URL, passing through unchanged: invalid port`);
return urlStr;
}
}
const hostAndRest = hostPart + suffix;
const colonIdx = userinfo.indexOf(":");
const rawUserEnc = colonIdx === -1 ? userinfo : userinfo.slice(0, colonIdx);
const hasPassword = colonIdx !== -1;
const rawPassEnc = hasPassword ? userinfo.slice(colonIdx + 1) : "";
try {
const encUser = rawUserEnc ? encodeURIComponent(lenientDecodeURIComponent(rawUserEnc)) : "";
const encPass = hasPassword
? (rawPassEnc ? encodeURIComponent(lenientDecodeURIComponent(rawPassEnc)) : "")
: null;
return assembleSocksUrl(scheme, encUser, encPass, hostAndRest);
} catch (e) {
console.warn(`[cloakbrowser] Could not normalize SOCKS5 proxy URL, passing through unchanged: ${(e as Error).message}`);
return urlStr;
}
}
/**
* Resolve proxy into Playwright option and/or Chrome args.
*
@@ -67,7 +155,9 @@ export function resolveProxyConfig(proxy: string | ProxyDict | undefined): Proxy
if (isSocksProxy(proxy)) {
// SOCKS5: bypass Playwright, pass directly to Chrome via --proxy-server.
if (typeof proxy === "string") {
return { proxyArgs: [`--proxy-server=${proxy}`] };
// Re-encode creds to work around Chromium parser truncating passwords
// at '=' and other special chars (#157).
return { proxyArgs: [`--proxy-server=${normalizeSocksStringUrl(proxy)}`] };
}
const socksUrl = reconstructSocksUrl(proxy);
const args = [`--proxy-server=${socksUrl}`];
+323
View File
@@ -1052,3 +1052,326 @@ function buildMockFrame(): any {
childFrames: vi.fn(() => []),
};
}
// =========================================================================
// mergeConfig
// =========================================================================
describe("mergeConfig", () => {
it("returns base unchanged when overrides is undefined/null", async () => {
const { mergeConfig, resolveConfig: rc } = await import("../src/human/config.js");
const base = rc("default");
expect(mergeConfig(base, undefined)).toBe(base);
expect(mergeConfig(base, null)).toBe(base);
});
it("creates a new object — base is never mutated", async () => {
const { mergeConfig, resolveConfig: rc } = await import("../src/human/config.js");
const base = rc("default");
const before = base.typing_delay;
const merged = mergeConfig(base, { typing_delay: 30 });
expect(merged.typing_delay).toBe(30);
expect(base.typing_delay).toBe(before);
expect(merged).not.toBe(base);
});
it("preserves non-overridden fields", async () => {
const { mergeConfig, resolveConfig: rc } = await import("../src/human/config.js");
const base = rc("default");
const merged = mergeConfig(base, { typing_delay: 30 });
expect(merged.mouse_min_steps).toBe(base.mouse_min_steps);
expect(merged.mistype_chance).toBe(base.mistype_chance);
});
});
// =========================================================================
// Per-call timeout forwarding (issue #137)
// =========================================================================
describe("page.click(selector, { timeout }) forwards timeout to scroll", () => {
it("scrollToElement passes timeout to locator.boundingBox()", async () => {
const { scrollToElement } = await import("../src/human/scroll.js");
const cfg = resolveConfig("default");
const boundingBox = vi.fn(async () => ({ x: 100, y: 200, width: 50, height: 30 }));
const page: any = {
viewportSize: () => ({ width: 1280, height: 720 }),
locator: vi.fn(() => ({ first: () => ({ boundingBox }) })),
};
const raw = {
move: vi.fn(async () => {}),
down: vi.fn(async () => {}),
up: vi.fn(async () => {}),
wheel: vi.fn(async () => {}),
};
await scrollToElement(page, raw, "#x", 0, 0, cfg, 5000);
expect(boundingBox).toHaveBeenCalledWith({ timeout: 5000 });
});
it("default timeout matches Playwright's 30000ms when not specified", async () => {
const { scrollToElement } = await import("../src/human/scroll.js");
const cfg = resolveConfig("default");
const boundingBox = vi.fn(async () => ({ x: 100, y: 200, width: 50, height: 30 }));
const page: any = {
viewportSize: () => ({ width: 1280, height: 720 }),
locator: vi.fn(() => ({ first: () => ({ boundingBox }) })),
};
const raw = {
move: vi.fn(async () => {}),
down: vi.fn(async () => {}),
up: vi.fn(async () => {}),
wheel: vi.fn(async () => {}),
};
await scrollToElement(page, raw, "#x", 0, 0, cfg);
expect(boundingBox).toHaveBeenCalledWith({ timeout: 30000 });
});
it("page.click({ timeout }) reaches scrollToElement", async () => {
const scrollMod = await import("../src/human/scroll.js");
const { patchPage } = await import("../src/human/index.js");
const cfg = resolveConfig("default", { idle_between_actions: false });
let captured = -1;
const spy = vi.spyOn(scrollMod, "scrollToElement").mockImplementation(
async (_page, _raw, _sel, cx, cy, _cfg, timeout?: number) => {
captured = timeout ?? -1;
return { box: { x: 100, y: 100, width: 50, height: 30 }, cursorX: cx, cursorY: cy };
},
);
const page = buildMockPage();
const cursor = { x: 100, y: 100, initialized: true };
patchPage(page as any, cfg, cursor as any);
await (page as any).click("#slow", { timeout: 5000 });
expect(captured).toBe(5000);
spy.mockRestore();
});
});
// =========================================================================
// Per-call human_config override
// =========================================================================
describe("page.type / page.fill accept per-call human_config override", () => {
it("page.type forwards merged config to humanType", async () => {
const keyboardMod = await import("../src/human/keyboard.js");
const scrollMod = await import("../src/human/scroll.js");
const { patchPage } = await import("../src/human/index.js");
// Make field_switch_delay tiny so the test runs fast
const cfg = resolveConfig("default", {
idle_between_actions: false,
field_switch_delay: [0, 1],
});
expect(cfg.typing_delay).toBe(70); // baseline
let captured: any = null;
const typeSpy = vi.spyOn(keyboardMod, "humanType").mockImplementation(
async (_page, _raw, _text, callCfg) => { captured = callCfg; },
);
const scrollSpy = vi.spyOn(scrollMod, "scrollToElement").mockImplementation(
async (_page, _raw, _sel, cx, cy) => ({
box: { x: 100, y: 100, width: 50, height: 30 },
cursorX: cx, cursorY: cy,
}),
);
const page = buildMockPage();
const cursor = { x: 100, y: 100, initialized: true };
patchPage(page as any, cfg, cursor as any);
await (page as any).type("#email", "hi", {
human_config: { typing_delay: 30, mistype_chance: 0 },
});
expect(captured.typing_delay).toBe(30);
expect(captured.mistype_chance).toBe(0);
// Global cfg untouched
expect(cfg.typing_delay).toBe(70);
typeSpy.mockRestore();
scrollSpy.mockRestore();
}, 30000);
it("page.fill forwards merged config to humanType", async () => {
const keyboardMod = await import("../src/human/keyboard.js");
const scrollMod = await import("../src/human/scroll.js");
const { patchPage } = await import("../src/human/index.js");
const cfg = resolveConfig("default", {
idle_between_actions: false,
field_switch_delay: [0, 1],
});
let captured: any = null;
const typeSpy = vi.spyOn(keyboardMod, "humanType").mockImplementation(
async (_page, _raw, _text, callCfg) => { captured = callCfg; },
);
const scrollSpy = vi.spyOn(scrollMod, "scrollToElement").mockImplementation(
async (_page, _raw, _sel, cx, cy) => ({
box: { x: 100, y: 100, width: 50, height: 30 },
cursorX: cx, cursorY: cy,
}),
);
const page = buildMockPage();
const cursor = { x: 100, y: 100, initialized: true };
patchPage(page as any, cfg, cursor as any);
await (page as any).fill("#password", "secret", {
human_config: { typing_delay: 150 },
});
expect(captured.typing_delay).toBe(150);
typeSpy.mockRestore();
scrollSpy.mockRestore();
}, 30000);
it("el.type forwards human_config to humanType", async () => {
const keyboardMod = await import("../src/human/keyboard.js");
const { patchSingleElementHandle } = await import("../src/human/elementhandle.js");
const cfg = resolveConfig("default", { idle_between_actions: false, mistype_chance: 0 });
const cursor = { x: 50, y: 50, initialized: true };
let captured: any = null;
const typeSpy = vi.spyOn(keyboardMod, "humanType").mockImplementation(
async (_page, _raw, _text, callCfg) => { captured = callCfg; },
);
const raw = { move: vi.fn(async () => {}), down: vi.fn(async () => {}), up: vi.fn(async () => {}), wheel: vi.fn(async () => {}) };
const rawKb = { down: vi.fn(async () => {}), up: vi.fn(async () => {}), type: vi.fn(async () => {}), insertText: vi.fn(async () => {}) };
const originals = { keyboardPress: vi.fn(async () => {}), keyboardDown: vi.fn(async () => {}), keyboardUp: vi.fn(async () => {}) };
const el = buildMockElementHandle({ evaluate: vi.fn(async () => true) });
const page = buildMockPage();
(page as any)._ensureCursorInit = vi.fn(async () => {});
patchSingleElementHandle(el, page as any, cfg, cursor as any, raw, rawKb, originals, null);
await el.type("abc", { human_config: { typing_delay: 25 } });
expect(captured.typing_delay).toBe(25);
typeSpy.mockRestore();
}, 30000);
});
// =========================================================================
// scrollIntoViewIfNeeded humanization
// =========================================================================
describe("humanScrollIntoView", () => {
it("skips wheel events when element is already in viewport", async () => {
const { humanScrollIntoView } = await import("../src/human/scroll.js");
const cfg = resolveConfig("default");
const page: any = { viewportSize: () => ({ width: 1280, height: 720 }) };
const raw = {
move: vi.fn(async () => {}),
down: vi.fn(async () => {}),
up: vi.fn(async () => {}),
wheel: vi.fn(async () => {}),
};
// Box centered in viewport — squarely in scroll_target_zone
const inViewBox = { x: 200, y: 300, width: 50, height: 30 };
const result = await humanScrollIntoView(page, raw, async () => inViewBox, 0, 0, cfg);
expect(result.box).toEqual(inViewBox);
expect(raw.wheel).not.toHaveBeenCalled();
});
it("fires wheel events when element is below the fold", async () => {
const { humanScrollIntoView } = await import("../src/human/scroll.js");
const cfg = resolveConfig("default", {
scroll_overshoot_chance: 0,
scroll_pre_move_delay: [0, 1],
scroll_pause_fast: [0, 1],
scroll_pause_slow: [0, 1],
scroll_settle_delay: [0, 1],
});
const page: any = { viewportSize: () => ({ width: 1280, height: 720 }) };
const raw = {
move: vi.fn(async () => {}),
down: vi.fn(async () => {}),
up: vi.fn(async () => {}),
wheel: vi.fn(async () => {}),
};
const boxes = [
{ x: 200, y: 2000, width: 50, height: 30 }, // far below
{ x: 200, y: 1500, width: 50, height: 30 },
{ x: 200, y: 1000, width: 50, height: 30 },
{ x: 200, y: 400, width: 50, height: 30 }, // in view
{ x: 200, y: 400, width: 50, height: 30 },
{ x: 200, y: 400, width: 50, height: 30 },
];
let i = 0;
const getBox = async () => boxes[Math.min(i++, boxes.length - 1)];
await humanScrollIntoView(page, raw, getBox, 0, 0, cfg);
expect(raw.wheel).toHaveBeenCalled();
}, 15000);
});
describe("el.scrollIntoViewIfNeeded humanization", () => {
it("calls humanScrollIntoView instead of native snap-scroll", async () => {
const scrollMod = await import("../src/human/scroll.js");
const { patchSingleElementHandle } = await import("../src/human/elementhandle.js");
let called = 0;
const spy = vi.spyOn(scrollMod, "humanScrollIntoView").mockImplementation(
async (_p, _raw, _gb, cx, cy) => {
called++;
return { box: { x: 200, y: 200, width: 50, height: 30 }, cursorX: cx, cursorY: cy };
},
);
const cfg = resolveConfig("default", { idle_between_actions: false });
const cursor = { x: 50, y: 50, initialized: true };
const raw = { move: vi.fn(async () => {}), down: vi.fn(async () => {}), up: vi.fn(async () => {}), wheel: vi.fn(async () => {}) };
const rawKb = { down: vi.fn(async () => {}), up: vi.fn(async () => {}), type: vi.fn(async () => {}), insertText: vi.fn(async () => {}) };
const originals = { keyboardPress: vi.fn(async () => {}), keyboardDown: vi.fn(async () => {}), keyboardUp: vi.fn(async () => {}) };
const el = buildMockElementHandle();
el.scrollIntoViewIfNeeded = vi.fn(async () => {});
const page = buildMockPage();
(page as any)._ensureCursorInit = vi.fn(async () => {});
patchSingleElementHandle(el, page as any, cfg, cursor as any, raw, rawKb, originals, null);
await el.scrollIntoViewIfNeeded();
expect(called).toBeGreaterThan(0);
spy.mockRestore();
});
it("falls back to native scrollIntoViewIfNeeded if humanized helper throws", async () => {
const scrollMod = await import("../src/human/scroll.js");
const { patchSingleElementHandle } = await import("../src/human/elementhandle.js");
const spy = vi.spyOn(scrollMod, "humanScrollIntoView").mockImplementation(
async () => { throw new Error("detached"); },
);
const cfg = resolveConfig("default", { idle_between_actions: false });
const cursor = { x: 50, y: 50, initialized: true };
const raw = { move: vi.fn(async () => {}), down: vi.fn(async () => {}), up: vi.fn(async () => {}), wheel: vi.fn(async () => {}) };
const rawKb = { down: vi.fn(async () => {}), up: vi.fn(async () => {}), type: vi.fn(async () => {}), insertText: vi.fn(async () => {}) };
const originals = { keyboardPress: vi.fn(async () => {}), keyboardDown: vi.fn(async () => {}), keyboardUp: vi.fn(async () => {}) };
const nativeFallback = vi.fn(async () => {});
const el = buildMockElementHandle();
el.scrollIntoViewIfNeeded = nativeFallback;
const page = buildMockPage();
(page as any)._ensureCursorInit = vi.fn(async () => {});
patchSingleElementHandle(el, page as any, cfg, cursor as any, raw, rawKb, originals, null);
await el.scrollIntoViewIfNeeded();
expect(nativeFallback).toHaveBeenCalled();
spy.mockRestore();
});
});
+13 -7
View File
@@ -4,14 +4,20 @@ import { DEFAULT_VIEWPORT, getChromiumVersion } from "../src/config.js";
describe("binaryInfo", () => {
it("returns correct structure", () => {
const info = binaryInfo();
const orig = process.env.CLOAKBROWSER_CACHE_DIR;
process.env.CLOAKBROWSER_CACHE_DIR = `/tmp/cloakbrowser-test-${Date.now()}`;
try {
const info = binaryInfo();
expect(info.version).toBe(getChromiumVersion());
expect(info.platform).toMatch(/^(linux|darwin|windows)-(x64|arm64)$/);
expect(info.binaryPath).toBeTruthy();
expect(typeof info.installed).toBe("boolean");
expect(info.cacheDir).toContain("cloakbrowser");
expect(info.downloadUrl).toContain(".tar.gz");
expect(info.version).toBe(getChromiumVersion());
expect(info.platform).toMatch(/^(linux|darwin|windows)-(x64|arm64)$/);
expect(info.binaryPath).toBeTruthy();
expect(typeof info.installed).toBe("boolean");
expect(info.cacheDir).toContain("cloakbrowser");
} finally {
if (orig) process.env.CLOAKBROWSER_CACHE_DIR = orig;
else delete process.env.CLOAKBROWSER_CACHE_DIR;
}
});
});
+61 -2
View File
@@ -191,8 +191,7 @@ describe("resolveProxyConfig", () => {
password: "p@ss",
});
expect(proxyOption).toBeUndefined();
expect(proxyArgs.length).toBe(1);
expect(proxyArgs[0]).toContain("--proxy-server=socks5://user:p%40ss@host:1080");
expect(proxyArgs).toEqual(["--proxy-server=socks5://user:p%40ss@host:1080"]);
});
it("includes bypass for socks5 dict", () => {
@@ -203,4 +202,64 @@ describe("resolveProxyConfig", () => {
expect(proxyArgs).toContain("--proxy-server=socks5://host:1080");
expect(proxyArgs).toContain("--proxy-bypass-list=.example.com");
});
// Chromium's --proxy-server parser truncates passwords at '=' (#157).
// Wrapper must auto URL-encode before passing to Chrome.
it("encodes '=' in socks5 string password", () => {
const { proxyArgs } = resolveProxyConfig("socks5://user:pass=123@host:1080");
expect(proxyArgs).toEqual(["--proxy-server=socks5://user:pass%3D123@host:1080"]);
});
it("encoding is idempotent for already-encoded socks5 string", () => {
const { proxyArgs } = resolveProxyConfig("socks5://user:pass%3D123@host:1080");
expect(proxyArgs).toEqual(["--proxy-server=socks5://user:pass%3D123@host:1080"]);
});
it("leaves socks5 string without creds unchanged", () => {
const { proxyArgs } = resolveProxyConfig("socks5://host:1080");
expect(proxyArgs).toEqual(["--proxy-server=socks5://host:1080"]);
});
it("encodes password even with empty username (password-only userinfo)", () => {
// Regression: empty-username bypass would skip encoding, leaving the
// Chromium truncation bug alive for this userinfo shape.
const { proxyArgs } = resolveProxyConfig("socks5://:pass=123@host:1080");
expect(proxyArgs).toEqual(["--proxy-server=socks5://:pass%3D123@host:1080"]);
});
it("handles literal '%' in password without throwing (malformed escape)", () => {
// JS's decodeURIComponent throws on '%sure' (% not followed by 2 hex digits).
// Must fall back to treating '%' as literal and percent-encoding it.
const { proxyArgs } = resolveProxyConfig("socks5://user:100%sure@host:1080");
expect(proxyArgs).toEqual(["--proxy-server=socks5://user:100%25sure@host:1080"]);
});
it("passes malformed SOCKS5 URLs through unchanged (no throw)", () => {
// Broken IPv6 bracket — wrapper must not throw;
// Chromium will surface its own error.
const { proxyArgs: a1 } = resolveProxyConfig("socks5://user:pass@[::1");
expect(a1).toEqual(["--proxy-server=socks5://user:pass@[::1"]);
});
it("passes non-numeric port through unchanged", () => {
const { proxyArgs } = resolveProxyConfig("socks5://user:pass@host:abc");
expect(proxyArgs).toEqual(["--proxy-server=socks5://user:pass@host:abc"]);
});
it("encodes special chars in IPv6 SOCKS5 string password", () => {
const { proxyArgs } = resolveProxyConfig("socks5://user:pass=eq@[::1]:1080");
expect(proxyArgs).toEqual(["--proxy-server=socks5://user:pass%3Deq@[::1]:1080"]);
});
// Regression #157: userinfo must be split at the LAST '@' (RFC 3986),
// not the first, so raw '@' in a password parses correctly.
it("encodes raw '@' in socks5 string password (last-@ split)", () => {
const { proxyArgs } = resolveProxyConfig("socks5://user:p@ss@host:1080");
expect(proxyArgs).toEqual(["--proxy-server=socks5://user:p%40ss@host:1080"]);
});
it("handles multiple raw '@' in password (splits at last)", () => {
const { proxyArgs } = resolveProxyConfig("socks5://user:a@b@c@host:1080");
expect(proxyArgs).toEqual(["--proxy-server=socks5://user:a%40b%40c@host:1080"]);
});
});
+2
View File
@@ -1671,11 +1671,13 @@ describe("Puppeteer: isInputElement stealth integration via patchPage", () => {
}),
});
const mockEl = buildMockElementHandle();
const page = buildMockPage({
evaluate: vi.fn(async (...args: any[]) => {
evaluateCalls.push(args);
return false;
}),
$: vi.fn(async () => mockEl),
});
page.createCDPSession = vi.fn(async () => mockCdp);
+8 -2
View File
@@ -320,8 +320,14 @@ describe("download fallback", () => {
describe("effective version", () => {
it("returns platform version when no marker exists", () => {
// Default behavior — no marker file in test environment
expect(getEffectiveVersion()).toBe(getChromiumVersion());
const orig = process.env.CLOAKBROWSER_CACHE_DIR;
process.env.CLOAKBROWSER_CACHE_DIR = `/tmp/cloakbrowser-test-${Date.now()}`;
try {
expect(getEffectiveVersion()).toBe(getChromiumVersion());
} finally {
if (orig) process.env.CLOAKBROWSER_CACHE_DIR = orig;
else delete process.env.CLOAKBROWSER_CACHE_DIR;
}
});
});
+453
View File
@@ -1296,6 +1296,459 @@ class TestAsyncElementHandle:
await browser.close()
# =========================================================================
# 15. Per-call timeout forwarding (issue #137)
# =========================================================================
class TestPerCallTimeoutForwarding:
"""page.click('#x', timeout=5000) must forward 5000 to bounding_box(),
not silently use the hardcoded 2000ms in scroll."""
def test_get_element_box_default_timeout(self):
"""Default timeout matches Playwright's 30000ms."""
from cloakbrowser.human.scroll import _get_element_box
from unittest.mock import MagicMock
page = MagicMock()
loc = MagicMock()
loc.bounding_box = MagicMock(return_value={"x": 0, "y": 0, "width": 1, "height": 1})
page.locator = MagicMock(return_value=MagicMock(first=loc))
_get_element_box(page, "#x")
loc.bounding_box.assert_called_once_with(timeout=30000)
def test_get_element_box_custom_timeout(self):
"""Caller can pass a custom timeout that overrides the default."""
from cloakbrowser.human.scroll import _get_element_box
from unittest.mock import MagicMock
page = MagicMock()
loc = MagicMock()
loc.bounding_box = MagicMock(return_value={"x": 0, "y": 0, "width": 1, "height": 1})
page.locator = MagicMock(return_value=MagicMock(first=loc))
_get_element_box(page, "#x", timeout=5000)
loc.bounding_box.assert_called_once_with(timeout=5000)
def test_scroll_to_element_forwards_timeout(self):
"""scroll_to_element passes timeout through to bounding_box()."""
from cloakbrowser.human.scroll import scroll_to_element
from cloakbrowser.human.config import resolve_config
from unittest.mock import MagicMock
cfg = resolve_config("default", None)
page = MagicMock()
page.viewport_size = {"width": 1280, "height": 720}
loc = MagicMock()
# Already in viewport so we don't actually scroll — just verify
# the timeout was forwarded on the first bounding_box() call.
loc.bounding_box = MagicMock(return_value={"x": 100, "y": 200, "width": 50, "height": 30})
page.locator = MagicMock(return_value=MagicMock(first=loc))
raw = MagicMock()
scroll_to_element(page, raw, "#x", 0, 0, cfg, timeout=7500)
loc.bounding_box.assert_called_with(timeout=7500)
def test_page_click_forwards_timeout_kwarg(self):
"""page.click(selector, timeout=...) reaches scroll_to_element.
Patches scroll_to_element module-side via monkey-patching the
cloakbrowser.human module attribute used by patch_page.
"""
import cloakbrowser.human as h
from cloakbrowser.human import _CursorState
from cloakbrowser.human.config import resolve_config
from unittest.mock import MagicMock, patch
cfg = resolve_config("default", {"idle_between_actions": False})
cursor = _CursorState()
cursor.initialized = True
cursor.x = 100
cursor.y = 100
# Build a minimal page mock
page = MagicMock()
page.click = MagicMock()
page.dblclick = MagicMock()
page.hover = MagicMock()
page.type = MagicMock()
page.fill = MagicMock()
page.goto = MagicMock()
page.is_checked = MagicMock(return_value=False)
page.viewport_size = {"width": 1280, "height": 720}
page.evaluate = MagicMock(return_value=False)
page.context.new_cdp_session = MagicMock(side_effect=Exception("no cdp"))
page.mouse = MagicMock()
page.keyboard = MagicMock()
page.query_selector = MagicMock(return_value=None)
page.query_selector_all = MagicMock(return_value=[])
page.wait_for_selector = MagicMock(return_value=None)
page.main_frame = MagicMock()
page.main_frame.child_frames = []
captured = {}
def fake_scroll(page_arg, raw, selector, cx, cy, cfg_arg, timeout=30000):
captured["timeout"] = timeout
return ({"x": 100, "y": 100, "width": 50, "height": 30}, cx, cy)
with patch.object(h, "scroll_to_element", side_effect=fake_scroll):
h.patch_page(page, cfg, cursor)
page.click("#slow-button", timeout=5000)
assert captured.get("timeout") == 5000, f"expected 5000, got {captured}"
# =========================================================================
# 16. Per-call human_config override (typing speed customization)
# =========================================================================
class TestPerCallHumanConfigOverride:
"""page.type('#email', text, human_config={'typing_delay': 30}) lets users
override typing speed (and any other HumanConfig field) on a per-call
basis without re-patching the page."""
def test_merge_config_creates_new_instance(self):
from cloakbrowser.human.config import resolve_config, merge_config
base = resolve_config("default", None)
merged = merge_config(base, {"typing_delay": 30})
assert merged.typing_delay == 30
assert base.typing_delay != 30 # not mutated
# Non-overridden fields are preserved
assert merged.mouse_min_steps == base.mouse_min_steps
def test_merge_config_none_returns_base(self):
from cloakbrowser.human.config import resolve_config, merge_config
base = resolve_config("default", None)
merged = merge_config(base, None)
assert merged is base
def test_merge_config_ignores_unknown_keys(self):
from cloakbrowser.human.config import resolve_config, merge_config
base = resolve_config("default", None)
# ``not_a_real_field`` is silently dropped — callers shouldn't crash
# if they pass typos or future field names.
merged = merge_config(base, {"typing_delay": 30, "not_a_real_field": 99})
assert merged.typing_delay == 30
def test_page_type_uses_per_call_typing_delay(self):
"""page.type(..., human_config={'typing_delay': 30}) reaches human_type
with cfg.typing_delay == 30 even when patch was done with default 70."""
import cloakbrowser.human as h
from cloakbrowser.human import _CursorState
from cloakbrowser.human.config import resolve_config
from unittest.mock import MagicMock, patch
cfg = resolve_config("default", {
"idle_between_actions": False,
"field_switch_delay": (0, 1),
})
assert cfg.typing_delay == 70 # baseline
cursor = _CursorState()
cursor.initialized = True
cursor.x = 100
cursor.y = 100
page = MagicMock()
page.click = MagicMock()
page.dblclick = MagicMock()
page.hover = MagicMock()
page.type = MagicMock()
page.fill = MagicMock()
page.goto = MagicMock()
page.is_checked = MagicMock(return_value=False)
page.viewport_size = {"width": 1280, "height": 720}
page.evaluate = MagicMock(return_value=False)
page.context.new_cdp_session = MagicMock(side_effect=Exception("no cdp"))
page.mouse = MagicMock()
page.keyboard = MagicMock()
page.query_selector = MagicMock(return_value=None)
page.query_selector_all = MagicMock(return_value=[])
page.wait_for_selector = MagicMock(return_value=None)
page.main_frame = MagicMock()
page.main_frame.child_frames = []
captured = {}
def fake_human_type(page_arg, raw, text, cfg_arg, cdp_session=None):
captured["typing_delay"] = cfg_arg.typing_delay
captured["mistype_chance"] = cfg_arg.mistype_chance
def fake_scroll(*args, **kwargs):
return ({"x": 100, "y": 100, "width": 50, "height": 30}, 100, 100)
with patch.object(h, "human_type", side_effect=fake_human_type), \
patch.object(h, "scroll_to_element", side_effect=fake_scroll):
h.patch_page(page, cfg, cursor)
page.type(
"#email", "hi",
human_config={"typing_delay": 30, "mistype_chance": 0},
)
assert captured["typing_delay"] == 30
assert captured["mistype_chance"] == 0
# Global cfg untouched — per-call override doesn't leak
assert cfg.typing_delay == 70
def test_page_fill_uses_per_call_typing_delay(self):
"""Same as type, but for fill (which also clears the field first)."""
import cloakbrowser.human as h
from cloakbrowser.human import _CursorState
from cloakbrowser.human.config import resolve_config
from unittest.mock import MagicMock, patch
cfg = resolve_config("default", {
"idle_between_actions": False,
"field_switch_delay": (0, 1),
})
cursor = _CursorState()
cursor.initialized = True
cursor.x = 100
cursor.y = 100
page = MagicMock()
page.viewport_size = {"width": 1280, "height": 720}
page.is_checked = MagicMock(return_value=False)
page.evaluate = MagicMock(return_value=False)
page.context.new_cdp_session = MagicMock(side_effect=Exception("no cdp"))
page.mouse = MagicMock()
page.keyboard = MagicMock()
page.query_selector = MagicMock(return_value=None)
page.query_selector_all = MagicMock(return_value=[])
page.wait_for_selector = MagicMock(return_value=None)
page.main_frame = MagicMock()
page.main_frame.child_frames = []
captured = {}
def fake_human_type(page_arg, raw, text, cfg_arg, cdp_session=None):
captured["typing_delay"] = cfg_arg.typing_delay
def fake_scroll(*args, **kwargs):
return ({"x": 100, "y": 100, "width": 50, "height": 30}, 100, 100)
with patch.object(h, "human_type", side_effect=fake_human_type), \
patch.object(h, "scroll_to_element", side_effect=fake_scroll):
h.patch_page(page, cfg, cursor)
page.fill("#password", "secret", human_config={"typing_delay": 150})
assert captured["typing_delay"] == 150
def test_element_handle_type_uses_per_call_human_config(self):
"""el.type(text, human_config={...}) merges per-call overrides on the
ElementHandle path (which doesn't go through page.type)."""
from cloakbrowser.human import _patch_single_element_handle_sync, _CursorState
from cloakbrowser.human.config import resolve_config
import cloakbrowser.human as h
from unittest.mock import MagicMock, patch
cfg = resolve_config("default", {
"idle_between_actions": False,
"field_switch_delay": (0, 1),
})
cursor = _CursorState()
cursor.initialized = True
cursor.x = 50
cursor.y = 50
page = MagicMock()
page.viewport_size = {"width": 1280, "height": 720}
page._original = MagicMock()
el = MagicMock()
el._human_patched = False
el.bounding_box = MagicMock(
return_value={"x": 200, "y": 200, "width": 100, "height": 30}
)
el.evaluate = MagicMock(return_value=True)
el.is_checked = MagicMock(return_value=False)
el.query_selector = MagicMock(return_value=None)
el.query_selector_all = MagicMock(return_value=[])
el.wait_for_selector = MagicMock(return_value=None)
el.scroll_into_view_if_needed = MagicMock()
raw_mouse = MagicMock()
raw_keyboard = MagicMock()
captured = {}
def fake_human_type(page_arg, raw, text, cfg_arg, cdp_session=None):
captured["typing_delay"] = cfg_arg.typing_delay
with patch.object(h, "human_type", side_effect=fake_human_type):
_patch_single_element_handle_sync(
el, page, cfg, cursor, raw_mouse, raw_keyboard,
page._original, None, None,
)
el.type("abc", human_config={"typing_delay": 25})
assert captured["typing_delay"] == 25
# =========================================================================
# 17. scroll_into_view_if_needed humanization
# =========================================================================
class TestScrollIntoViewIfNeeded:
"""scroll_into_view_if_needed should run through the same
accelerate cruise decelerate overshoot wheel sequence as page.click
not Playwright's instant-snap default."""
def test_human_scroll_into_view_skips_when_in_viewport(self):
"""Already-visible elements: no wheel events, just return."""
from cloakbrowser.human.scroll import human_scroll_into_view
from cloakbrowser.human.config import resolve_config
from unittest.mock import MagicMock
cfg = resolve_config("default", None)
page = MagicMock()
page.viewport_size = {"width": 1280, "height": 720}
raw = MagicMock()
# Box is dead-center of viewport — squarely in scroll_target_zone
in_view_box = {"x": 200, "y": 300, "width": 50, "height": 30}
box, cx, cy = human_scroll_into_view(
page, raw, lambda: in_view_box, 0, 0, cfg,
)
assert box == in_view_box
assert not raw.wheel.called, "In-viewport elements shouldn't trigger wheel events"
def test_human_scroll_into_view_scrolls_when_below_fold(self):
"""Below-fold elements: wheel events fire, eventually box becomes visible."""
from cloakbrowser.human.scroll import human_scroll_into_view
from cloakbrowser.human.config import resolve_config
from unittest.mock import MagicMock
cfg = resolve_config("default", {
"scroll_overshoot_chance": 0, # deterministic
"scroll_pre_move_delay": (0, 1),
"scroll_pause_fast": (0, 1),
"scroll_pause_slow": (0, 1),
"scroll_settle_delay": (0, 1),
})
page = MagicMock()
page.viewport_size = {"width": 1280, "height": 720}
raw = MagicMock()
# First box is far below the fold; subsequent boxes "come into view"
# so the loop terminates after a few wheel bursts.
boxes = [
{"x": 200, "y": 2000, "width": 50, "height": 30},
{"x": 200, "y": 1500, "width": 50, "height": 30},
{"x": 200, "y": 1000, "width": 50, "height": 30},
{"x": 200, "y": 400, "width": 50, "height": 30}, # in viewport
{"x": 200, "y": 400, "width": 50, "height": 30},
{"x": 200, "y": 400, "width": 50, "height": 30},
]
idx = {"i": 0}
def get_box():
i = min(idx["i"], len(boxes) - 1)
idx["i"] += 1
return boxes[i]
human_scroll_into_view(page, raw, get_box, 0, 0, cfg)
assert raw.wheel.called, "Below-fold scroll should produce wheel events"
def test_element_handle_scroll_into_view_if_needed_humanized(self):
"""el.scroll_into_view_if_needed() routes through human_scroll_into_view."""
from cloakbrowser.human import _patch_single_element_handle_sync, _CursorState
from cloakbrowser.human.config import resolve_config
import cloakbrowser.human as h
from unittest.mock import MagicMock, patch
cfg = resolve_config("default", None)
cursor = _CursorState()
cursor.initialized = True
cursor.x = 50
cursor.y = 50
page = MagicMock()
page.viewport_size = {"width": 1280, "height": 720}
page._original = MagicMock()
el = MagicMock()
el._human_patched = False
el.bounding_box = MagicMock(
return_value={"x": 200, "y": 200, "width": 50, "height": 30}
)
el.evaluate = MagicMock(return_value=False)
el.is_checked = MagicMock(return_value=False)
el.query_selector = MagicMock(return_value=None)
el.query_selector_all = MagicMock(return_value=[])
el.wait_for_selector = MagicMock(return_value=None)
# Make sure the original method exists so the patch is wired up
el.scroll_into_view_if_needed = MagicMock()
called = {"count": 0}
def fake(*args, **kwargs):
called["count"] += 1
return ({"x": 200, "y": 200, "width": 50, "height": 30}, 100, 100)
with patch.object(h, "human_scroll_into_view", side_effect=fake):
_patch_single_element_handle_sync(
el, page, cfg, cursor, MagicMock(), MagicMock(),
page._original, None, None,
)
# Patched method should now invoke our humanized helper
el.scroll_into_view_if_needed()
assert called["count"] >= 1, "humanized scroll helper was never called"
def test_locator_scroll_into_view_if_needed_humanized(self):
"""Locator.scroll_into_view_if_needed() also goes through humanized scroll."""
import cloakbrowser.human as h
from cloakbrowser.human import _CursorState
from cloakbrowser.human.config import resolve_config
from unittest.mock import MagicMock, patch
# Patch Locator class fresh
_ensure_locator_patched()
from playwright.sync_api._generated import Locator
cfg = resolve_config("default", None)
cursor = _CursorState()
cursor.x = 50
cursor.y = 50
cursor.initialized = True
page = MagicMock()
page._original = MagicMock()
page._human_cfg = cfg
page._human_cursor = cursor
page._human_raw_mouse = MagicMock()
page.viewport_size = {"width": 1280, "height": 720}
# Build a Locator-like object satisfying the patched method
loc = MagicMock(spec=Locator)
loc.page = page
impl_obj = MagicMock()
impl_obj._selector = "#x"
loc._impl_obj = impl_obj
loc.bounding_box = MagicMock(
return_value={"x": 100, "y": 100, "width": 50, "height": 30}
)
called = {"count": 0, "cfg": None}
def fake(*args, **kwargs):
called["count"] += 1
# cfg is the 6th positional arg (page, raw, get_box, cx, cy, cfg)
called["cfg"] = args[5] if len(args) >= 6 else kwargs.get("cfg")
return ({"x": 100, "y": 100, "width": 50, "height": 30}, 200, 200)
with patch.object(h, "human_scroll_into_view", side_effect=fake):
Locator.scroll_into_view_if_needed(
loc, human_config={"scroll_overshoot_chance": 0.5},
)
assert called["count"] == 1
# Per-call override merged into the cfg passed downstream
assert called["cfg"].scroll_overshoot_chance == 0.5
# Cursor was updated from the helper's return value
assert cursor.x == 200 and cursor.y == 200
# =========================================================================
# Direct runner (backwards compat)
# =========================================================================
+72
View File
@@ -266,3 +266,75 @@ class TestResolveProxyConfig:
assert kwargs == {}
assert "--proxy-server=socks5://host:1080" in args
assert "--proxy-bypass-list=.example.com" in args
def test_socks5_string_encodes_equals_in_password(self):
# Chromium's --proxy-server parser truncates passwords at '=' (#157).
# Wrapper must auto URL-encode before passing to Chrome.
_, args = _resolve_proxy_config("socks5://user:pass=123@host:1080")
assert args == ["--proxy-server=socks5://user:pass%3D123@host:1080"]
def test_socks5_string_encodes_at_in_password(self):
_, args = _resolve_proxy_config("socks5://user:p@ss@host:1080")
# Note: parsing "user:p@ss@host" — urlparse takes everything up to LAST @
# as userinfo, so password = "p@ss".
assert args == ["--proxy-server=socks5://user:p%40ss@host:1080"]
def test_socks5_string_encoding_idempotent(self):
# Already-encoded input should remain encoded (not double-encoded).
_, args = _resolve_proxy_config("socks5://user:pass%3D123@host:1080")
assert args == ["--proxy-server=socks5://user:pass%3D123@host:1080"]
def test_socks5_string_no_creds_unchanged(self):
_, args = _resolve_proxy_config("socks5://host:1080")
assert args == ["--proxy-server=socks5://host:1080"]
def test_socks5_string_password_only_still_encoded(self):
# Empty username with password: fix must still re-encode the password
# (regression test for empty-username bypass).
_, args = _resolve_proxy_config("socks5://:pass=123@host:1080")
assert args == ["--proxy-server=socks5://:pass%3D123@host:1080"]
def test_socks5_string_empty_password_preserves_colon(self):
# `user:@host` (empty password) must NOT collapse to `user@host` —
# semantics differ between the two forms.
_, args = _resolve_proxy_config("socks5://user:@host:1080")
assert args == ["--proxy-server=socks5://user:@host:1080"]
def test_socks5_string_literal_percent_in_password(self):
# Literal '%' not followed by 2 hex digits must be encoded as '%25'
# so Chrome decodes it back to '%'. Must not crash.
_, args = _resolve_proxy_config("socks5://user:100%sure@host:1080")
assert args == ["--proxy-server=socks5://user:100%25sure@host:1080"]
def test_socks5_string_malformed_port_passes_through(self, caplog):
# Invalid port (non-numeric) raises in urlparse.port. Wrapper should
# log a warning and pass original through to Chromium.
import logging
with caplog.at_level(logging.WARNING, logger="cloakbrowser"):
_, args = _resolve_proxy_config("socks5://user:pass@host:abc")
assert args == ["--proxy-server=socks5://user:pass@host:abc"]
assert any("Malformed SOCKS5" in r.message for r in caplog.records)
def test_socks5_string_malformed_ipv6_passes_through(self, caplog):
# Broken IPv6 bracket — must not crash, and must reach Chromium
# verbatim so its own error surfaces instead of a silent rewrite.
import logging
with caplog.at_level(logging.WARNING, logger="cloakbrowser"):
_, args = _resolve_proxy_config("socks5://user:pass@[::1")
assert args == ["--proxy-server=socks5://user:pass@[::1"]
def test_socks5_string_preserves_path_and_query(self):
# Nonstandard for SOCKS5, but don't silently drop user-supplied suffixes.
# Matches JS behavior.
_, args = _resolve_proxy_config("socks5://user:pass@host:1080/p?x=1#f")
assert args[0] == "--proxy-server=socks5://user:pass@host:1080/p?x=1#f"
def test_socks5_string_ipv6_with_special_char_password(self):
# IPv6 host + special char in password — both must be handled.
_, args = _resolve_proxy_config("socks5://user:pass=eq@[::1]:1080")
assert args[0] == "--proxy-server=socks5://user:pass%3Deq@[::1]:1080"
def test_socks5_string_port_zero_preserved(self):
# Port 0 is an unusual but valid URL component; don't silently strip it.
_, args = _resolve_proxy_config("socks5://user:pass=1@host:0")
assert args[0] == "--proxy-server=socks5://user:pass%3D1@host:0"