Compare commits

..
Author SHA1 Message Date
CloakHQ 749ed561a6 fix(humanize): strip custom kwargs before forwarding to Playwright select_option
originals.select_option(**kwargs) passes human_config/force to Playwright
which rejects unknown kwargs with TypeError.
2026-05-15 20:51:30 +02:00
CloakHQ f4f6a3f8fd fix(humanize): forward human_config in all chained methods, use evaluate args in handle pointer checks
- Add human_config=kwargs.get("human_config") to check/uncheck/select_option/press inner calls (sync+async+JS)
- Convert check_pointer_events_handle from f-string interpolation to evaluate args pattern (sync+async+JS)
2026-05-15 19:17:11 +02:00
CloakHQ 39f807f246 feat(humanize): add Playwright-style actionability checks to all interaction methods
Humanized locator/page methods now perform pre-action validation matching
Playwright's native behavior: attached, visible, enabled, editable, stable,
and receives-pointer-events checks with retry loop and backoff.

- New error hierarchy: ActionabilityError base with ElementNotAttachedError,
  ElementNotVisibleError, ElementNotStableError, ElementNotEnabledError,
  ElementNotEditableError, ElementNotReceivingEventsError
- force=True parameter skips all actionability checks (matches Playwright)
- Shared deadline across all steps (checks + scroll + stable + pointer)
- Post-scroll stability check only runs when scroll actually happened
- Chained methods (type/fill/check/uncheck/press) skip inner click checks
  but still run pointer-events check at actual click coordinates
- Frame methods now forward kwargs (force, timeout, human_config)
- Locator patches forward force via _forward_kwargs
- Python sync + async, JS/TS implementation
2026-05-12 22:58:03 +02:00
26 changed files with 152 additions and 1162 deletions
-1
View File
@@ -46,7 +46,6 @@ js/dist/
*.whl
AGENTS.md
.beads
result
# Private docs (launch posts, strategy)
docs/
-22
View File
@@ -8,28 +8,6 @@ Changes are tagged: **[wrapper]** for Python/JS wrapper, **[binary]** for Chromi
## [Unreleased]
## [0.3.30] — 2026-05-21
- **[binary]** New build 146.0.7680.177.5 for Linux x64 + Windows x64 — 58 source-level fingerprint patches (up from 57)
- **[binary]** Rendering consistency improvements across Linux and Windows — corrected GPU, display, and graphics parameters to match stock Chrome 146 profiles
- **[binary]** Windows: native GPU/rendering values now pass through directly instead of being spoofed, matching real hardware behavior
- **[binary]** Storage normalization fix for Windows
- **[binary]** HTTP proxy inline credential support at the network layer
- **[wrapper]** Update `PLATFORM_CHROMIUM_VERSIONS` for linux-x64 and windows-x64 to 146.0.7680.177.5
## [0.3.29] — 2026-05-20
- **[wrapper]** **Security**: `cloakserve` — guard WebSocket origins to prevent browser-origin CSRF via CDP proxy (thanks [@0xlally](https://github.com/0xlally) for the report, [@honor2030](https://github.com/honor2030) for the fix, #239, #240)
- **[wrapper]** **Security**: Lambda example — add URL scheme validation, SSRF protection, post-navigation re-validation, remove unsafe caller-controlled options (#233)
- **[wrapper]** **Security**: CI — isolate `workflow_dispatch` input to avoid shell injection in attest-release (thanks [@aaronjmars](https://github.com/aaronjmars), #223)
- **[wrapper]** **Security**: JS — bump tar + transitive deps via npm audit fix (thanks [@aaronjmars](https://github.com/aaronjmars), #222)
- **[wrapper]** Add `extension_paths` parameter for loading Chrome extensions in all launch functions (thanks [@zackycodes](https://github.com/zackycodes), #210)
- **[wrapper]** Humanize: add Playwright-style actionability checks — auto-wait for visible, enabled, stable elements before humanized actions (#228)
- **[wrapper]** JS: export composable launch helpers — `buildLaunchOptions()` and `humanizeBrowser()` for custom Playwright integrations (thanks [@honor2030](https://github.com/honor2030), #244)
- **[wrapper]** JS: add `launchPersistentContext()` to Puppeteer wrapper (#261)
- **[wrapper]** Add `flake.nix` for Nix/NixOS (thanks [@Seryiza](https://github.com/Seryiza), #220)
- **[meta]** JS: sync package-lock metadata (thanks [@245678000000](https://github.com/245678000000), #219)
## [0.3.28] — 2026-05-11
- **[wrapper]** **Security**: `cloakserve` — sanitize fingerprint seed to prevent path traversal, bind to `127.0.0.1` on bare metal, detect Podman containers (#217)
+19 -35
View File
@@ -40,7 +40,7 @@ Drop-in Playwright/Puppeteer replacement for Python and JavaScript.<br>
Same API, same code — just swap the import. <strong>3 lines of code, 30 seconds to unblock.</strong>
</p>
- **58 source-level C++ patches** — canvas, WebGL, audio, fonts, GPU, screen, WebRTC, network timing, automation signals, CDP input behavior
- **49 source-level C++ patches** — canvas, WebGL, audio, fonts, GPU, screen, WebRTC, network timing, automation signals, CDP input behavior
- **`humanize=True`** — human-like mouse curves, keyboard timing, and scroll patterns. One flag, behavioral detection passes
- **0.9 reCAPTCHA v3 score** — human-level, server-verified
- **Passes Cloudflare Turnstile**, FingerprintJS, BrowserScan — tested against 30+ detection sites
@@ -128,19 +128,17 @@ Open [http://localhost:8080](http://localhost:8080). Create a profile. Click **L
---
## Latest: v0.3.30 (Chromium 146.0.7680.177.5)
## Latest: v0.3.28 (Chromium 146.0.7680.177.4)
- **58 fingerprint patches** — rendering consistency improvements across Linux and Windows, corrected GPU/display/graphics parameters to match stock Chrome 146 profiles
- **Windows native GPU passthrough** — real hardware values pass through directly instead of being spoofed, matching real browser behavior
- **HTTP proxy inline credentials** — new network-layer support for proxies with inline authentication
- **`extension_paths`** — load Chrome extensions in all launch functions
- **Humanize actionability** — auto-wait for visible, enabled, stable elements before humanized actions
- **Per-call `human_config`** — override humanize settings on individual method calls
- **Composable JS helpers** — `buildLaunchOptions()` and `humanizeBrowser()` for custom Playwright integrations
- **Native SOCKS5 proxy** — `proxy="socks5://user:pass@host:port"` works directly in all launch functions, Python + JS. QUIC/HTTP3 tunnels through SOCKS5 via UDP ASSOCIATE
- **Proxy signal removal** — DNS/connect/SSL timing zeroed, proxy cache headers stripped, Proxy-Connection header leak removed
- **`launch_context_async()`** — async counterpart to `launch_context()`. Forwards kwargs to `browser.new_context()` for `storage_state`, `permissions`, `extra_http_headers` without a persistent profile folder.
- **JS `contextOptions` escape hatch** — forward arbitrary options (including `storageState`) to Playwright's `newContext()` from `launchContext()` / `launchPersistentContext()`.
- **Native SOCKS5 proxy** — `proxy="socks5://user:pass@host:port"` works directly in all launch functions, Python + JS. QUIC/HTTP3 tunnels through SOCKS5 via UDP ASSOCIATE.
- **Chromium 146 upgrade** — rebased all patches from 145.0.7632.x to 146.0.7680.177
- **57 fingerprint patches** — additional detection-vector coverage (WebAuthn, AAC audio, window position) and WebGL/canvas consistency fixes
- **WebRTC IP spoofing** — `--fingerprint-webrtc-ip=auto` resolves your proxy's exit IP and spoofs WebRTC ICE candidates. Auto-injected when using `geoip=True` (no extra network call)
- **Proxy signal removal** — DNS/connect/SSL timing zeroed, proxy cache headers stripped, Proxy-Connection header leak removed
- **`cloakserve` CDP multiplexer** — rewritten as a multi-connection CDP proxy with per-connection fingerprint seeds
- **Humanize CDP isolation** — keyboard events now use isolated worlds and trusted dispatch for better behavioral stealth
- **`humanize=True`** — one flag makes all mouse, keyboard, and scroll interactions behave like a real user. Bézier curves, per-character typing, realistic scroll patterns
- **Stealthy with zero flags** — binary auto-generates a random fingerprint seed at startup. No configuration required
- **Timezone & locale from proxy IP** — `launch(proxy="...", geoip=True)` auto-detects timezone and locale
@@ -228,7 +226,7 @@ CloakBrowser is a thin wrapper (Python + JavaScript) around a custom-built Chrom
3. **Every launch** → Playwright or Puppeteer starts with our binary + stealth args
4. **You write code** → standard Playwright/Puppeteer API, nothing new to learn
The binary includes 58 source-level patches covering canvas, WebGL, audio, fonts, GPU, screen properties, WebRTC, network timing, hardware reporting, automation signal removal, and CDP input behavior mimicking.
The binary includes 49 source-level patches covering canvas, WebGL, audio, fonts, GPU, screen properties, WebRTC, network timing, hardware reporting, automation signal removal, and CDP input behavior mimicking.
These are compiled into the Chromium binary — not injected via JavaScript, not set via flags.
@@ -373,16 +371,9 @@ ctx.close() # profile saved
# Next run — cookies, localStorage restored automatically
ctx = launch_persistent_context("./my-profile", headless=False)
# Load Chrome extensions
ctx = launch_persistent_context(
"./my-profile",
headless=False,
extension_paths=["./my-extension"],
)
```
Supports all the same options as `launch_context()`: `proxy`, `user_agent`, `viewport`, `locale`, `timezone`, `color_scheme`, `geoip`, `extension_paths`.
Supports all the same options as `launch_context()`: `proxy`, `user_agent`, `viewport`, `locale`, `timezone`, `color_scheme`, `geoip`.
Async version: `launch_persistent_context_async()`.
@@ -415,7 +406,7 @@ from cloakbrowser import binary_info, clear_cache, ensure_binary
# Check binary installation status
print(binary_info())
# {'version': '146.0.7680.177.5', 'platform': 'linux-x64', 'installed': True, ...}
# {'version': '146.0.7680.177.3', 'platform': 'linux-x64', 'installed': True, ...}
# Force re-download
clear_cache()
@@ -749,11 +740,11 @@ browser = await launch_async(args=["--remote-debugging-port=9242"])
| Platform | Chromium | Patches | Status |
|---|---|---|---|
| Linux x86_64 | 146 | 58 | ✅ Latest |
| Linux arm64 (RPi, Graviton) | 146 | 58 | ✅ |
| Linux x86_64 | 146 | 57 | ✅ Latest |
| Linux arm64 (RPi, Graviton) | 146 | 57 | ✅ Latest |
| macOS arm64 (Apple Silicon) | 145 | 26 | ✅ |
| macOS x86_64 (Intel) | 145 | 26 | ✅ |
| Windows x86_64 | 146 | 58 | ✅ Latest |
| Windows x86_64 | 146 | 57 | ✅ Latest |
The wrapper auto-downloads the correct binary for your platform.
@@ -1152,9 +1143,9 @@ A: Yes. Pass `proxy="http://user:pass@host:port"` or `proxy="socks5://user:pass@
| Feature | Status |
|---------|--------|
| Linux x64 — Chromium 146 (58 patches) | ✅ Released |
| Linux x64 — Chromium 146 (57 patches) | ✅ Released |
| macOS arm64/x64 — Chromium 145 (26 patches) | ✅ Released |
| Windows x64 — Chromium 146 (58 patches) | ✅ Released |
| Windows x64 — Chromium 146 (57 patches) | ✅ Released |
| JavaScript/Puppeteer + Playwright support | ✅ Released |
| Fingerprint rotation per session | ✅ Released |
| Built-in proxy rotation | 📋 Planned |
@@ -1176,7 +1167,7 @@ All releases are signed for supply chain verification.
```bash
# Verify GPG signature (binary release tag)
gpg --keyserver keyserver.ubuntu.com --recv-keys C60C0DDC9D0DE2DD
git verify-tag chromium-v146.0.7680.177.5
git verify-tag chromium-v146.0.7680.177.3
# Verify GitHub binary attestation (Sigstore)
gh attestation verify cloakbrowser-linux-x64.tar.gz --repo CloakHQ/cloakbrowser
@@ -1205,11 +1196,4 @@ Issues and PRs welcome. If something isn't working, [open an issue](https://gith
- [@eofreternal](https://github.com/eofreternal) — humanConfig type fix, humanized method option types
- [@manaskarra](https://github.com/manaskarra) — iframe scope fix for humanized frame actions, GeoIP timeout guard
- [@Youhai020616](https://github.com/Youhai020616) — SOCKS5 credential encoding logging
- [@AlexTech314](https://github.com/AlexTech314) — AWS Lambda integration, cold-start hardening
- [@dgtlmoon](https://github.com/dgtlmoon) — graceful pw.stop() cleanup
- [@zackycodes](https://github.com/zackycodes) — Chrome extension loading
- [@aaronjmars](https://github.com/aaronjmars) — security fixes (shell injection, dep bumps)
- [@Seryiza](https://github.com/Seryiza) — Nix/NixOS flake
- [@245678000000](https://github.com/245678000000) — package-lock sync
- [@honor2030](https://github.com/honor2030) — cloakserve WebSocket origin guard, composable JS launch helpers
- [@0xlally](https://github.com/0xlally) — security reports (cloakserve path traversal, WebSocket origin bypass)
- [@AlexTech314](https://github.com/AlexTech314) — AWS Lambda integration
+3 -97
View File
@@ -18,7 +18,6 @@ Client:
from __future__ import annotations
import asyncio
import ipaddress
import json
import logging
import os
@@ -30,7 +29,7 @@ import subprocess
import sys
import time
from dataclasses import dataclass
from urllib.parse import parse_qs, urlparse
from urllib.parse import parse_qs
from pathlib import Path
@@ -65,91 +64,6 @@ BASE_CDP_PORT = 5100
SAFE_SEED_RE = re.compile(r"^[A-Za-z0-9_-]{1,128}$")
RESERVED_SEEDS = {"__default__"}
TRUSTED_WS_ORIGINS = {"devtools://devtools", "chrome-devtools://devtools"}
def _host_port_from_netloc(netloc: str, default_port: int) -> tuple[str, int] | None:
"""Return a normalized (host, port) pair for an Origin/Host netloc."""
if "," in netloc:
return None
try:
parsed = urlparse(f"//{netloc.strip()}")
authority = parsed.netloc.rsplit("@", 1)[-1]
if (
not parsed.hostname
or parsed.username is not None
or parsed.password is not None
or authority.endswith(":")
or parsed.path
or parsed.params
or parsed.query
or parsed.fragment
):
return None
return (parsed.hostname.lower(), parsed.port if parsed.port is not None else default_port)
except ValueError:
return None
def _is_loopback_host(hostname: str) -> bool:
"""Return True for localhost and loopback IP literals."""
hostname = hostname.strip("[]").rstrip(".").lower()
if hostname == "localhost":
return True
try:
return ipaddress.ip_address(hostname).is_loopback
except ValueError:
return False
def _origin_is_allowed(
origin: str | None,
host: str | None,
request_scheme: str = "http",
) -> bool:
"""Return True when a WebSocket Origin is safe to proxy to local CDP."""
if origin is None:
# Playwright/Puppeteer and other non-browser CDP clients commonly omit
# Origin. Keep those clients working while rejecting browser-origin CSRF.
return True
origin = origin.strip()
if not origin or origin.lower() == "null":
return False
if origin in TRUSTED_WS_ORIGINS:
return True
try:
parsed = urlparse(origin)
except ValueError:
return False
if parsed.scheme not in ("http", "https"):
return False
if parsed.path or parsed.params or parsed.query or parsed.fragment:
return False
origin_default_port = 443 if parsed.scheme == "https" else 80
request_scheme = request_scheme.split(",", 1)[0].strip().lower()
request_default_port = 443 if request_scheme in ("https", "wss") else 80
origin_host = _host_port_from_netloc(parsed.netloc, origin_default_port)
request_host = _host_port_from_netloc(host or "", request_default_port)
if origin_host is None or request_host is None:
return False
if not _is_loopback_host(request_host[0]):
return False
return origin_host == request_host
def _reject_untrusted_origin(request: web.Request) -> web.Response | None:
"""Reject browser-origin WebSocket upgrades that would expose local CDP."""
origin = request.headers.get("Origin")
host = request.headers.get("Host")
scheme = request.headers.get("X-Forwarded-Proto", getattr(request, "scheme", "http"))
if _origin_is_allowed(origin, host, request_scheme=scheme):
return None
logger.warning("Rejected CDP WebSocket from untrusted Origin %r for Host %r", origin, host)
return web.Response(status=403, text="Forbidden: untrusted WebSocket origin\n")
# ---------------------------------------------------------------------------
@@ -613,12 +527,8 @@ async def proxy_cdp_websocket(
logger.error("%s error: %s", label, exc)
async def handle_ws_default(request: web.Request) -> web.StreamResponse:
async def handle_ws_default(request: web.Request) -> web.WebSocketResponse:
"""WebSocket proxy for default (no-seed) Chrome: /devtools/{type}/{guid}"""
rejected = _reject_untrusted_origin(request)
if rejected is not None:
return rejected
pool: ChromePool = request.app["pool"]
path = request.match_info.get("path", "")
@@ -636,12 +546,8 @@ async def handle_ws_default(request: web.Request) -> web.StreamResponse:
return ws
async def handle_ws_seed(request: web.Request) -> web.StreamResponse:
async def handle_ws_seed(request: web.Request) -> web.WebSocketResponse:
"""WebSocket proxy for seed-specific Chrome: /fingerprint/{seed}/devtools/{type}/{guid}"""
rejected = _reject_untrusted_origin(request)
if rejected is not None:
return rejected
pool: ChromePool = request.app["pool"]
seed = request.match_info["seed"]
path = request.match_info.get("path", "")
+1 -1
View File
@@ -1 +1 @@
__version__ = "0.3.30"
__version__ = "0.3.28"
+6 -29
View File
@@ -64,7 +64,6 @@ def launch(
humanize: bool = False,
human_preset: HumanPreset = "default",
human_config: HumanConfigOverrides | None = None,
extension_paths: list[str] | None = None,
**kwargs: Any,
) -> Any:
"""Launch stealth Chromium browser. Returns a Playwright Browser object.
@@ -76,7 +75,6 @@ def launch(
Dict: {"server": "http://proxy:8080", "bypass": ".google.com", ...}
passed directly to Playwright.
args: Additional Chromium CLI arguments to pass.
extension_paths: List of Chrome extension paths to load.
stealth_args: Include default stealth fingerprint args (default True).
Set to False if you want to pass your own --fingerprint flags.
timezone: IANA timezone (e.g. 'America/New_York'). Sets --fingerprint-timezone binary flag.
@@ -114,8 +112,7 @@ def launch(
if exit_ip and not (args and any(a.startswith("--fingerprint-webrtc-ip") for a in args)):
args = list(args or [])
args.append(f"--fingerprint-webrtc-ip={exit_ip}")
chrome_args = build_args(stealth_args, (args or []) + proxy_extra_args, timezone=timezone, locale=locale, headless=headless, extension_paths=extension_paths)
chrome_args = build_args(stealth_args, (args or []) + proxy_extra_args, timezone=timezone, locale=locale, headless=headless)
logger.debug("Launching stealth Chromium (headless=%s, args=%d)", headless, len(chrome_args))
@@ -162,7 +159,6 @@ async def launch_async( # noqa: C901
humanize: bool = False,
human_preset: HumanPreset = "default",
human_config: HumanConfigOverrides | None = None,
extension_paths: list[str] | None = None,
**kwargs: Any,
) -> Any:
"""Async version of launch(). Returns a Playwright Browser object.
@@ -171,7 +167,6 @@ async def launch_async( # noqa: C901
headless: Run in headless mode (default True).
proxy: Proxy URL string or Playwright proxy dict (see launch() for details).
args: Additional Chromium CLI arguments to pass.
extension_paths: List of Chrome extension paths to load.
stealth_args: Include default stealth fingerprint args (default True).
timezone: IANA timezone (e.g. 'America/New_York'). Sets --fingerprint-timezone binary flag.
locale: BCP 47 locale (e.g. 'en-US'). Sets --lang binary flag.
@@ -207,7 +202,7 @@ async def launch_async( # noqa: C901
if exit_ip and not (args and any(a.startswith("--fingerprint-webrtc-ip") for a in args)):
args = list(args or [])
args.append(f"--fingerprint-webrtc-ip={exit_ip}")
chrome_args = build_args(stealth_args, (args or []) + proxy_extra_args, timezone=timezone, locale=locale, headless=headless, extension_paths=extension_paths)
chrome_args = build_args(stealth_args, (args or []) + proxy_extra_args, timezone=timezone, locale=locale, headless=headless)
logger.debug("Launching stealth Chromium async (headless=%s, args=%d)", headless, len(chrome_args))
@@ -258,7 +253,6 @@ def launch_persistent_context(
humanize: bool = False,
human_preset: HumanPreset = "default",
human_config: HumanConfigOverrides | None = None,
extension_paths: list[str] | None = None,
**kwargs: Any,
) -> Any:
"""Launch stealth browser with a persistent profile and return a BrowserContext.
@@ -274,7 +268,6 @@ def launch_persistent_context(
headless: Run in headless mode (default True).
proxy: Proxy URL string or Playwright proxy dict (see launch() for details).
args: Additional Chromium CLI arguments.
extension_paths: List of Chrome extension paths to load.
stealth_args: Include default stealth fingerprint args (default True).
user_agent: Custom user agent string.
viewport: Viewport size dict, e.g. {"width": 1920, "height": 1080}.
@@ -313,7 +306,7 @@ def launch_persistent_context(
if exit_ip and not (args and any(a.startswith("--fingerprint-webrtc-ip") for a in args)):
args = list(args or [])
args.append(f"--fingerprint-webrtc-ip={exit_ip}")
chrome_args = build_args(stealth_args, (args or []) + proxy_extra_args, timezone=timezone, locale=locale, headless=headless, extension_paths=extension_paths)
chrome_args = build_args(stealth_args, (args or []) + proxy_extra_args, timezone=timezone, locale=locale, headless=headless)
logger.debug(
"Launching persistent stealth Chromium (headless=%s, user_data_dir=%s)",
@@ -384,7 +377,6 @@ async def launch_persistent_context_async(
humanize: bool = False,
human_preset: HumanPreset = "default",
human_config: HumanConfigOverrides | None = None,
extension_paths: list[str] | None = None,
**kwargs: Any,
) -> Any:
"""Async version of launch_persistent_context().
@@ -399,7 +391,6 @@ async def launch_persistent_context_async(
headless: Run in headless mode (default True).
proxy: Proxy URL string or Playwright proxy dict (see launch() for details).
args: Additional Chromium CLI arguments.
extension_paths: List of Chrome extension paths to load.
stealth_args: Include default stealth fingerprint args (default True).
user_agent: Custom user agent string.
viewport: Viewport size dict, e.g. {"width": 1920, "height": 1080}.
@@ -441,7 +432,7 @@ async def launch_persistent_context_async(
if exit_ip and not (args and any(a.startswith("--fingerprint-webrtc-ip") for a in args)):
args = list(args or [])
args.append(f"--fingerprint-webrtc-ip={exit_ip}")
chrome_args = build_args(stealth_args, (args or []) + proxy_extra_args, timezone=timezone, locale=locale, headless=headless, extension_paths=extension_paths)
chrome_args = build_args(stealth_args, (args or []) + proxy_extra_args, timezone=timezone, locale=locale, headless=headless)
logger.debug(
"Launching persistent stealth Chromium async (headless=%s, user_data_dir=%s)",
@@ -511,7 +502,6 @@ def launch_context(
humanize: bool = False,
human_preset: HumanPreset = "default",
human_config: HumanConfigOverrides | None = None,
extension_paths: list[str] | None = None,
**kwargs: Any,
) -> Any:
"""Launch stealth browser and return a BrowserContext with common options pre-set.
@@ -523,7 +513,6 @@ def launch_context(
headless: Run in headless mode (default True).
proxy: Proxy URL string or Playwright proxy dict (see launch() for details).
args: Additional Chromium CLI arguments.
extension_paths: List of Chrome extension paths to load.
stealth_args: Include default stealth fingerprint args (default True).
user_agent: Custom user agent string.
viewport: Viewport size dict, e.g. {"width": 1920, "height": 1080}.
@@ -555,7 +544,7 @@ def launch_context(
# so it applies to ALL contexts, not just the default one.
# locale and timezone are set via binary flags only — no CDP emulation.
browser = launch(headless=headless, proxy=proxy, args=args, stealth_args=stealth_args,
timezone=timezone, locale=locale, backend=backend, extension_paths=extension_paths)
timezone=timezone, locale=locale, backend=backend)
context_kwargs: dict[str, Any] = {}
if user_agent:
@@ -612,7 +601,6 @@ async def launch_context_async(
humanize: bool = False,
human_preset: HumanPreset = "default",
human_config: HumanConfigOverrides | None = None,
extension_paths: list[str] | None = None,
**kwargs: Any,
) -> Any:
"""Async version of launch_context().
@@ -626,7 +614,6 @@ async def launch_context_async(
headless: Run in headless mode (default True).
proxy: Proxy URL string or Playwright proxy dict (see launch() for details).
args: Additional Chromium CLI arguments.
extension_paths: List of Chrome extension paths to load.
stealth_args: Include default stealth fingerprint args (default True).
user_agent: Custom user agent string.
viewport: Viewport size dict, e.g. {"width": 1920, "height": 1080}.
@@ -675,7 +662,7 @@ async def launch_context_async(
# so it applies to ALL contexts, not just the default one.
# locale and timezone are set via binary flags only — no CDP emulation.
browser = await launch_async(headless=headless, proxy=proxy, args=args, stealth_args=stealth_args,
timezone=timezone, locale=locale, backend=backend, extension_paths=extension_paths)
timezone=timezone, locale=locale, backend=backend)
context_kwargs: dict[str, Any] = {}
if user_agent:
@@ -970,7 +957,6 @@ def build_args(
timezone: str | None = None,
locale: str | None = None,
headless: bool = True,
extension_paths: list[str] | None = None,
) -> list[str]:
"""Combine stealth args with user-provided args and locale flags.
@@ -1014,15 +1000,6 @@ def build_args(
logger.debug("Arg override: %s -> %s", seen[key], flag)
seen[key] = flag
if extension_paths:
abs_paths = [os.path.abspath(p) for p in extension_paths]
ext_val = ",".join(abs_paths)
seen["--load-extension"] = f"--load-extension={ext_val}"
seen["--disable-extensions-except"] = (
f"--disable-extensions-except={ext_val}"
)
return list(seen.values())
+3 -3
View File
@@ -15,14 +15,14 @@ from ._version import __version__
# CHROMIUM_VERSION is the latest across all platforms (for display/reference).
# Use get_chromium_version() for the current platform's actual version.
# ---------------------------------------------------------------------------
CHROMIUM_VERSION = "146.0.7680.177.5"
CHROMIUM_VERSION = "146.0.7680.177.3"
PLATFORM_CHROMIUM_VERSIONS: dict[str, str] = {
"linux-x64": "146.0.7680.177.5",
"linux-x64": "146.0.7680.177.3",
"linux-arm64": "146.0.7680.177.3",
"darwin-arm64": "145.0.7632.109.2",
"darwin-x64": "145.0.7632.109.2",
"windows-x64": "146.0.7680.177.5",
"windows-x64": "146.0.7680.177.4",
}
# ---------------------------------------------------------------------------
@@ -70,7 +70,7 @@ Only `url` is required. Everything else is optional.
| Field | Type | Default |
|---|---|---|
| `url` | str | required `http://` and `https://` only |
| `url` | str | required |
| `proxy` | str / dict | none — `http://user:pass@host:port` or a Playwright proxy dict |
| `humanize` | bool | `false` — enable human-like mouse / keyboard / scroll |
| `human_preset` | str | `"default"` or `"careful"` |
@@ -79,6 +79,7 @@ Only `url` is required. Everything else is optional.
| `locale` | str | none — BCP-47, e.g. `"en-US"` |
| `viewport` | `{width,height}` | `1920x947` (cloakbrowser default) |
| `user_agent` | str | none |
| `extra_args` | `list[str]` | `[]` — extra Chromium CLI flags |
### Navigation
@@ -101,6 +102,8 @@ Only `url` is required. Everything else is optional.
| `wait_for_selector` | str | none — CSS or XPath |
| `wait_for_selector_state` | str | `"visible"` — also `attached` / `detached` / `hidden` |
| `wait_for_selector_timeout_ms` | int | `30000` |
| `wait_for_function` | str | none — JS expression returning truthy when ready |
| `wait_for_function_timeout_ms` | int | `30000` |
| `wait_ms` | int | none — fixed pause |
### Capture
@@ -115,7 +118,7 @@ Only `url` is required. Everything else is optional.
The handler retries transient navigation failures inline within the same Lambda invocation. Two layers, both built-in:
- **Launch retries** — 3 attempts with 0.3 s + 0.6 s backoff. Recovers Xvfb / Chromium spawn races at cold start. Fast and cheap; not configurable.
- **Strategy retries** — default 1 attempt, configurable via the `retries` event field. Recovers specific post-launch error classes by relaunching with adjusted internal Chromium args / page-load budgets.
- **Strategy retries** — default 1 attempt, configurable via the `retries` event field. Recovers specific post-launch error classes by relaunching with adjusted Chromium args / page-load budgets.
| Field | Type | Default |
|---|---|---|
@@ -173,22 +176,6 @@ For latency-sensitive use cases: provision concurrency, schedule a CloudWatch/Ev
If you see empty/missing dynamic content on cold-start invocations, raise `max_settle_ms` in the event payload (e.g. `25000`) — the default `15000` is tuned for warm runs.
## Security
The handler validates all incoming URLs before navigation:
- **Scheme restriction** — only `http://` and `https://` are accepted. `file://`, `data:`, `javascript:`, and other schemes are rejected.
- **SSRF protection** — hostnames are resolved before navigation and checked against private, loopback, link-local, reserved, and multicast IP ranges. This blocks access to cloud metadata endpoints (e.g. `169.254.169.254`), localhost services, and internal networks.
- **Post-navigation re-validation** — the final URL is re-checked after page load and after post-navigation waits to catch server-side redirects to blocked destinations.
- **No caller-controlled Chromium flags** — the handler does not accept arbitrary CLI flags from the event. Internal retry strategies add flags as needed (e.g. `--ignore-certificate-errors` for cert errors).
- **No arbitrary JS execution**`wait_for_function` is not exposed. Use `wait_for_selector` or `smart_wait` instead.
**Limitations**:
- Post-navigation re-validation prevents response *exfiltration*, but does not prevent the browser from *making* the request. If an internal endpoint has side effects on GET, the request will still reach it before validation rejects the response. Use network-level controls (security groups, VPC) to protect side-effect-bearing internal endpoints.
- DNS rebinding attacks can bypass pre-navigation IP checks in theory, though the post-navigation re-validation provides a second layer of defense.
**Trust boundary**: if this handler is exposed to untrusted callers (Lambda Function URL, API Gateway without auth, public ALB), add an authentication layer (API Gateway authorizer, IAM auth, etc.). The URL validation above is defense-in-depth, not a substitute for access control.
## License
The patched Chromium binary inside the upstream `cloakhq/cloakbrowser` image is governed by the **CloakBrowser Binary License** (published at https://github.com/CloakHQ/CloakBrowser/blob/main/BINARY-LICENSE.md). Internal organizational use (private ECR, your own scraping pipelines, your own business) is free. Exposing this Lambda as a paid API to third-party customers — i.e. browser-as-a-service — requires an OEM/SaaS license from CloakHQ (`cloakhq@pm.me`). Do not push the resulting image to a public registry; that would be redistribution and is prohibited.
@@ -5,7 +5,7 @@ Always runs **headed** via the Xvfb display started by `lambda-entrypoint.sh`.
Event schema (all fields except `url` are optional):
Launch options (passed to cloakbrowser.launch_context_async):
url str required, the page to scrape (http/https only)
url str required, the page to scrape
proxy str|dict http://user:pass@host:port or Playwright proxy dict
humanize bool False enable human-like mouse/keyboard/scroll
human_preset str "default" | "careful"
@@ -14,6 +14,7 @@ Event schema (all fields except `url` are optional):
locale str BCP-47, e.g. "en-US"
viewport {width,height} defaults to 1920x947 (cloakbrowser DEFAULT_VIEWPORT)
user_agent str custom UA (rare cloakbrowser sets one already)
extra_args list[str] additional Chromium CLI flags
Navigation options (passed to page.goto):
wait_until str "load"|"domcontentloaded"|"networkidle"|"commit"
@@ -34,6 +35,8 @@ Event schema (all fields except `url` are optional):
wait_for_selector str CSS or XPath selector
wait_for_selector_state str "attached"|"detached"|"visible"|"hidden", default "visible"
wait_for_selector_timeout_ms int 30000
wait_for_function str JS expression that returns truthy when ready
wait_for_function_timeout_ms int 30000
wait_ms int fixed pause in ms (page.wait_for_timeout)
Capture options:
@@ -61,14 +64,11 @@ from __future__ import annotations
import asyncio
import base64
import ipaddress
import json
import logging
import socket
import subprocess
from pathlib import Path
from typing import Any
from urllib.parse import urlparse
from cloakbrowser import launch_context_async
@@ -76,26 +76,6 @@ logger = logging.getLogger("cloakbrowser.lambda")
logger.setLevel(logging.INFO)
def _validate_url(url: str) -> None:
"""Reject non-HTTP schemes and URLs that resolve to private/internal IPs."""
parsed = urlparse(url)
if parsed.scheme.lower() not in ("http", "https"):
raise ValueError(
f"Only http:// and https:// URLs are supported, got: {parsed.scheme!r}"
)
hostname = parsed.hostname
if not hostname:
raise ValueError("URL has no hostname")
try:
infos = socket.getaddrinfo(hostname, None, socket.AF_UNSPEC, socket.SOCK_STREAM)
except socket.gaierror:
raise ValueError(f"Cannot resolve hostname: {hostname}")
for info in infos:
addr = ipaddress.ip_address(info[4][0])
if not addr.is_global:
raise ValueError("URLs targeting private/internal networks are blocked")
def _diag_snapshot() -> str:
"""Capture Xvfb status, Xvfb log, X11 socket state, and env for error reports."""
import os
@@ -138,7 +118,7 @@ def _build_launch_kwargs(event: dict) -> dict:
# Lambda's restricted process model can't fork from Chromium's zygote
# — without this, child renderer processes fail to spawn.
"--no-zygote",
*event.get("_strategy_args", []),
*event.get("extra_args", []),
],
}
for key in ("proxy", "humanize", "human_preset", "geoip",
@@ -179,7 +159,7 @@ async def _smart_wait(page, dom_stable_ms: int = 1500, max_settle_ms: int = 1500
_EXPLICIT_WAIT_KEYS = (
"wait_for_load_state", "wait_for_selector", "wait_ms",
"wait_for_load_state", "wait_for_selector", "wait_for_function", "wait_ms",
)
@@ -204,6 +184,11 @@ async def _post_nav_waits(page, event: dict) -> None:
state=event.get("wait_for_selector_state", "visible"),
timeout=event.get("wait_for_selector_timeout_ms", 30000),
)
if "wait_for_function" in event:
await page.wait_for_function(
event["wait_for_function"],
timeout=event.get("wait_for_function_timeout_ms", 30000),
)
if "wait_ms" in event:
await page.wait_for_timeout(event["wait_ms"])
@@ -250,7 +235,7 @@ def _classify_error(err: Exception) -> dict | None:
msg = str(err)
if "ERR_CERT" in msg:
return {
"_strategy_args": ["--ignore-certificate-errors"],
"extra_args": ["--ignore-certificate-errors"],
"goto_timeout_ms": 60000,
}
if ("Timeout" in msg and "exceeded" in msg) or "ERR_CONNECTION_TIMED_OUT" in msg:
@@ -278,10 +263,8 @@ async def _attempt_scrape(url: str, event: dict) -> dict:
wait_until=event.get("wait_until", "domcontentloaded"),
timeout=event.get("goto_timeout_ms", 30000),
)
_validate_url(page.url)
await _post_nav_waits(page, event)
_validate_url(page.url)
result: dict = {
"title": await page.title(),
@@ -323,8 +306,6 @@ async def _run(event: dict) -> dict:
set to 0 to disable retry entirely).
"""
url = event["url"]
_validate_url(url)
event = {k: v for k, v in event.items() if k not in ("extra_args", "_strategy_args")}
retries_left = max(0, int(event.get("retries", 1)))
history: list[dict] = []
current_event = event
@@ -345,8 +326,8 @@ async def _run(event: dict) -> dict:
})
logger.warning("attempt %d failed (%s); retrying with strategy=%s",
len(history), str(e)[:120], strategy)
merged_args = list(current_event.get("_strategy_args", [])) + list(strategy.get("_strategy_args", []))
current_event = {**current_event, **strategy, "_strategy_args": merged_args}
merged_args = list(current_event.get("extra_args", [])) + list(strategy.get("extra_args", []))
current_event = {**current_event, **strategy, "extra_args": merged_args}
retries_left -= 1
# No backoff: strategy overrides change goto budget directly;
# the prior failure was either fast (cert reject) or already
Generated
-27
View File
@@ -1,27 +0,0 @@
{
"nodes": {
"nixpkgs": {
"locked": {
"lastModified": 1777954456,
"narHash": "sha256-hGdgeU2Nk87RAuZyYjyDjFL6LK7dAZN5RE9+hrDTkDU=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "549bd84d6279f9852cae6225e372cc67fb91a4c1",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixos-unstable",
"repo": "nixpkgs",
"type": "github"
}
},
"root": {
"inputs": {
"nixpkgs": "nixpkgs"
}
}
},
"root": "root",
"version": 7
}
-237
View File
@@ -1,237 +0,0 @@
{
description = "CloakBrowser development shell with Nix-packaged Chromium binaries";
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
};
outputs = { self, nixpkgs }:
let
inherit (nixpkgs) lib;
supportedSystems = [
"x86_64-linux"
"aarch64-linux"
];
forAllSystems = lib.genAttrs supportedSystems;
packageInfo = {
x86_64-linux = {
platformTag = "linux-x64";
version = "146.0.7680.177.5";
hash = "sha256-ShK83pX6G7G+7ytBq15cJ8Nr544749DayMZNcFIWZw4=";
};
aarch64-linux = {
platformTag = "linux-arm64";
version = "146.0.7680.177.3";
hash = "sha256-i3HOU7T9ExMnMxox+6ODXXGILRm/qr3njdD1OQvRb0U=";
};
};
cloakbrowserBinaryLicense = {
shortName = "cloakbrowser-binary";
fullName = "CloakBrowser Binary License";
url = "https://github.com/CloakHQ/CloakBrowser/blob/main/BINARY-LICENSE.md";
free = false;
redistributable = false;
};
mkPkgs = system: import nixpkgs {
inherit system;
config.allowUnfree = true;
};
runtimeLibraries = pkgs: with pkgs; [
alsa-lib
at-spi2-atk
at-spi2-core
atk
cairo
cups
dbus
expat
fontconfig
freetype
gdk-pixbuf
glib
gtk3
libdrm
libgbm
libGL
libpulseaudio
libxkbcommon
mesa
nspr
nss
pango
systemd
wayland
libx11
libxcb
libxcomposite
libxcursor
libxdamage
libxext
libxfixes
libxi
libxrandr
libxrender
libxscrnsaver
libxshmfence
libxtst
];
fontPackages = pkgs: with pkgs; [
freefont_ttf
ipafont
liberation_ttf
noto-fonts
noto-fonts-cjk-sans
noto-fonts-color-emoji
tlwg
unifont
wqy_zenhei
];
desktopPackages = pkgs: with pkgs; [
adwaita-icon-theme
gsettings-desktop-schemas
xdg-utils
];
mkCloakBrowserChromium = pkgs: system:
let
info = packageInfo.${system} or (throw "CloakBrowser flake package currently supports only x86_64-linux and aarch64-linux.");
archiveName = "cloakbrowser-${info.platformTag}.tar.gz";
chromiumVersion = info.version;
libs = runtimeLibraries pkgs;
desktopDeps = desktopPackages pkgs;
fonts = fontPackages pkgs;
fontsConf = pkgs.makeFontsConf {
fontDirectories = fonts;
};
in
pkgs.stdenvNoCC.mkDerivation {
pname = "cloakbrowser-chromium";
version = chromiumVersion;
src = pkgs.fetchurl {
url = "https://cloakbrowser.dev/chromium-v${chromiumVersion}/${archiveName}";
inherit (info) hash;
};
dontUnpack = true;
nativeBuildInputs = with pkgs; [
autoPatchelfHook
makeWrapper
];
buildInputs = libs ++ desktopDeps;
runtimeDependencies = libs;
installPhase = ''
runHook preInstall
mkdir -p "$out/lib/cloakbrowser" "$out/bin"
tar -xzf "$src" -C "$out/lib/cloakbrowser"
chmod +x "$out/lib/cloakbrowser/chrome"
chmod +x "$out/lib/cloakbrowser/chromedriver"
runHook postInstall
'';
postFixup = ''
makeWrapper "$out/lib/cloakbrowser/chrome" "$out/bin/cloakbrowser-chrome" \
--prefix LD_LIBRARY_PATH : "${lib.makeLibraryPath libs}" \
--prefix XDG_DATA_DIRS : "$GSETTINGS_SCHEMAS_PATH:$XDG_ICON_DIRS" \
--suffix PATH : "${lib.makeBinPath [ pkgs.xdg-utils ]}" \
--set FONTCONFIG_FILE "${fontsConf}" \
--set CHROME_WRAPPER "cloakbrowser-chrome"
makeWrapper "$out/lib/cloakbrowser/chromedriver" "$out/bin/cloakbrowser-chromedriver" \
--prefix LD_LIBRARY_PATH : "${lib.makeLibraryPath libs}"
'';
meta = {
description = "Official CloakBrowser patched Chromium binary";
homepage = "https://github.com/CloakHQ/CloakBrowser";
license = cloakbrowserBinaryLicense;
mainProgram = "cloakbrowser-chrome";
platforms = supportedSystems;
sourceProvenance = [ lib.sourceTypes.binaryNativeCode ];
};
};
in
{
packages = forAllSystems (system:
let
pkgs = mkPkgs system;
cloakbrowserChromium = mkCloakBrowserChromium pkgs system;
in
{
inherit cloakbrowserChromium;
default = cloakbrowserChromium;
});
apps = forAllSystems (system:
let
cloakbrowserChromium = self.packages.${system}.cloakbrowserChromium;
in
{
default = {
type = "app";
program = "${cloakbrowserChromium}/bin/cloakbrowser-chrome";
meta.description = "Run CloakBrowser Chromium";
};
cloakbrowser-chrome = {
type = "app";
program = "${cloakbrowserChromium}/bin/cloakbrowser-chrome";
meta.description = "Run CloakBrowser Chromium";
};
cloakbrowser-chromedriver = {
type = "app";
program = "${cloakbrowserChromium}/bin/cloakbrowser-chromedriver";
meta.description = "Run the CloakBrowser Chromedriver binary";
};
});
devShells = forAllSystems (system:
let
pkgs = mkPkgs system;
cloakbrowserChromium = self.packages.${system}.cloakbrowserChromium;
python = pkgs.python312.withPackages (ps: with ps; [
aiohttp
geoip2
hatchling
httpx
playwright
pytest
pytest-asyncio
socksio
websockets
]);
in
{
default = pkgs.mkShell {
packages = [
cloakbrowserChromium
python
pkgs.cacert
pkgs.curl
pkgs.git
pkgs.jq
pkgs.nodejs_20
pkgs.which
pkgs.xdotool
pkgs.xvfb-run
]
++ runtimeLibraries pkgs
++ fontPackages pkgs;
CLOAKBROWSER_BINARY_PATH = "${cloakbrowserChromium}/bin/cloakbrowser-chrome";
};
});
};
}
-7
View File
@@ -230,13 +230,6 @@ const ctx = await launchPersistentContext({
userDataDir: './my-profile',
headless: false,
});
// Load Chrome extensions
const ctx = await launchPersistentContext({
userDataDir: './my-profile',
headless: false,
extensionPaths: ['./my-extension'],
});
```
This also gives you cookie and localStorage persistence across sessions.
+4 -3
View File
@@ -1,12 +1,12 @@
{
"name": "cloakbrowser",
"version": "0.3.30",
"version": "0.3.28",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "cloakbrowser",
"version": "0.3.30",
"version": "0.3.28",
"license": "MIT",
"dependencies": {
"tar": "^7.0.0"
@@ -1329,7 +1329,8 @@
"resolved": "https://registry.npmjs.org/devtools-protocol/-/devtools-protocol-0.0.1232444.tgz",
"integrity": "sha512-pM27vqEfxSxRkTMnF+XCmxSEb6duO5R+t8A9DEEJgy4Wz2RVanje2mmj99B6A3zv2r/qGfYlOvYznUhuokizmg==",
"dev": true,
"license": "BSD-3-Clause"
"license": "BSD-3-Clause",
"peer": true
},
"node_modules/diff-sequences": {
"version": "29.6.3",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "cloakbrowser",
"version": "0.3.30",
"version": "0.3.28",
"description": "Stealth Chromium that passes every bot detection test. Drop-in Playwright/Puppeteer replacement with source-level fingerprint patches.",
"type": "module",
"main": "dist/index.js",
+1 -12
View File
@@ -1,7 +1,7 @@
/**
* Shared argument builder for Playwright and Puppeteer wrappers.
*/
import path from "path";
import type { LaunchOptions } from "./types.js";
import { getDefaultStealthArgs } from "./config.js";
@@ -55,16 +55,5 @@ export function buildArgs(options: LaunchOptions): string[] {
seen.set(k, flag);
}
}
if (options.extensionPaths?.length) {
const absPaths = options.extensionPaths.map(p => path.resolve(p));
const joined = absPaths.join(",");
seen.set("--load-extension", `--load-extension=${joined}`);
seen.set(
"--disable-extensions-except",
`--disable-extensions-except=${joined}`
);
}
return [...seen.values()];
}
+3 -3
View File
@@ -27,14 +27,14 @@ export { WRAPPER_VERSION };
// CHROMIUM_VERSION is the latest across all platforms (for display/reference).
// Use getChromiumVersion() for the current platform's actual version.
// ---------------------------------------------------------------------------
export const CHROMIUM_VERSION = "146.0.7680.177.5";
export const CHROMIUM_VERSION = "146.0.7680.177.3";
export const PLATFORM_CHROMIUM_VERSIONS: Record<string, string> = {
"linux-x64": "146.0.7680.177.5",
"linux-x64": "146.0.7680.177.3",
"linux-arm64": "146.0.7680.177.3",
"darwin-arm64": "145.0.7632.109.2",
"darwin-x64": "145.0.7632.109.2",
"windows-x64": "146.0.7680.177.5",
"windows-x64": "146.0.7680.177.4",
};
// ---------------------------------------------------------------------------
+1 -1
View File
@@ -16,7 +16,7 @@
*/
// Launch functions (Playwright API)
export { launch, launchContext, launchPersistentContext, buildLaunchOptions, humanizeBrowser } from "./playwright.js";
export { launch, launchContext, launchPersistentContext } from "./playwright.js";
// Binary management
export { ensureBinary, clearCache, binaryInfo, checkForUpdate } from "./download.js";
+31 -49
View File
@@ -3,7 +3,7 @@
* Mirrors Python cloakbrowser/browser.py.
*/
import type { Browser, BrowserContext, BrowserContextOptions, LaunchOptions as PlaywrightLaunchOptions } from "playwright-core";
import type { Browser, BrowserContext, BrowserContextOptions } from "playwright-core";
import type { LaunchOptions, LaunchContextOptions, LaunchPersistentContextOptions } from "./types.js";
import { DEFAULT_VIEWPORT, IGNORE_DEFAULT_ARGS } from "./config.js";
import { buildArgs } from "./args.js";
@@ -44,52 +44,6 @@ function filterStealthCtxOptions(ctx?: BrowserContextOptions): Partial<BrowserCo
return rest;
}
/**
* Build Playwright launch options for CloakBrowser without starting Chromium.
*
* Useful when integrating CloakBrowser with a custom Playwright build or another
* wrapper that needs to call `chromium.launch()` itself.
*/
export async function buildLaunchOptions(
options: LaunchOptions = {}
): Promise<PlaywrightLaunchOptions> {
const binaryPath = process.env.CLOAKBROWSER_BINARY_PATH || (await ensureBinary());
const { exitIp, ...resolved } = await maybeResolveGeoip(options);
const { proxyOption, proxyArgs } = resolveProxyConfig(options.proxy);
let resolvedArgs = await resolveWebrtcArgs(options);
if (exitIp && !(resolvedArgs ?? []).some(a => a.startsWith("--fingerprint-webrtc-ip"))) {
resolvedArgs = [...(resolvedArgs ?? []), `--fingerprint-webrtc-ip=${exitIp}`];
}
const args = buildArgs({ ...options, ...resolved, args: [...(resolvedArgs ?? []), ...proxyArgs] });
return {
executablePath: binaryPath,
headless: options.headless ?? true,
args,
ignoreDefaultArgs: IGNORE_DEFAULT_ARGS,
...(proxyOption ? { proxy: proxyOption } : {}),
...options.launchOptions,
} as PlaywrightLaunchOptions;
}
/**
* Apply CloakBrowser's human-like behavioral layer to an existing Playwright browser.
*/
export async function humanizeBrowser(
browser: Browser,
options: LaunchOptions = {}
): Promise<void> {
if (!options.humanize) return;
const { patchBrowser } = await import('./human/index.js');
const { resolveConfig } = await import('./human/config.js');
const cfg = resolveConfig(
options.humanPreset ?? 'default',
options.humanConfig,
);
patchBrowser(browser, cfg);
}
/**
* Launch stealth Chromium browser via Playwright.
*
@@ -105,8 +59,36 @@ export async function humanizeBrowser(
*/
export async function launch(options: LaunchOptions = {}): Promise<Browser> {
const { chromium } = await import("playwright-core");
const browser = await chromium.launch(await buildLaunchOptions(options));
await humanizeBrowser(browser, options);
const binaryPath = process.env.CLOAKBROWSER_BINARY_PATH || (await ensureBinary());
const { exitIp, ...resolved } = await maybeResolveGeoip(options);
const { proxyOption, proxyArgs } = resolveProxyConfig(options.proxy);
let resolvedArgs = await resolveWebrtcArgs(options);
if (exitIp && !(resolvedArgs ?? []).some(a => a.startsWith("--fingerprint-webrtc-ip"))) {
resolvedArgs = [...(resolvedArgs ?? []), `--fingerprint-webrtc-ip=${exitIp}`];
}
const args = buildArgs({ ...options, ...resolved, args: [...(resolvedArgs ?? []), ...proxyArgs] });
const browser = await chromium.launch({
executablePath: binaryPath,
headless: options.headless ?? true,
args,
ignoreDefaultArgs: IGNORE_DEFAULT_ARGS,
...(proxyOption ? { proxy: proxyOption } : {}),
...options.launchOptions,
});
// Human-like behavioral patching
if (options.humanize) {
const { patchBrowser } = await import('./human/index.js');
const { resolveConfig } = await import('./human/config.js');
const cfg = resolveConfig(
options.humanPreset ?? 'default',
options.humanConfig,
);
patchBrowser(browser, cfg);
}
return browser;
}
+58 -106
View File
@@ -12,55 +12,71 @@ import { ensureBinary } from "./download.js";
import { isSocksProxy, parseProxyUrl, resolveProxyConfig } from "./proxy.js";
import { maybeResolveGeoip, resolveWebrtcArgs } from "./geoip.js";
/** Resolve binary path, geoip, webrtc, and build final Chrome args. */
async function resolveArgs(options: LaunchOptions): Promise<{ binaryPath: string; args: string[] }> {
/**
* Launch stealth Chromium browser via Puppeteer.
*
* @example
* ```ts
* import { launch } from 'cloakbrowser/puppeteer';
* * // With humanize — human-like mouse, keyboard, scroll
* const browser = await launch({ humanize: true });
* const page = await browser.newPage();
* await page.goto('[https://example.com](https://example.com)');
* await page.click('#login'); // Bézier curve mouse movement
* await page.type('#email', 'user@example.com'); // Per-character timing
* ```
*/
export async function launch(options: LaunchOptions = {}): Promise<Browser> {
const puppeteer = await import("puppeteer-core");
const binaryPath = process.env.CLOAKBROWSER_BINARY_PATH || (await ensureBinary());
const { exitIp, ...resolved } = (await maybeResolveGeoip(options)) ?? {};
let resolvedArgs = (await resolveWebrtcArgs(options)) ?? options.args;
if (exitIp && !(resolvedArgs ?? []).some(a => a.startsWith("--fingerprint-webrtc-ip"))) {
resolvedArgs = [...(resolvedArgs ?? []), `--fingerprint-webrtc-ip=${exitIp}`];
}
return { binaryPath, args: buildArgs({ ...options, ...resolved, args: resolvedArgs }) };
}
const args = buildArgs({ ...options, ...resolved, args: resolvedArgs });
/**
* Resolve proxy into Chrome CLI args and optional HTTP auth credentials.
* SOCKS5: Chrome supports inline credentials natively (RFC 1929 auth).
* HTTP: Chrome does NOT support inline credentials strip them and
* use page.authenticate() for Proxy-Authorization headers instead.
*/
function resolveProxy(options: LaunchOptions, args: string[]): { username: string; password: string } | undefined {
if (!options.proxy) return undefined;
if (isSocksProxy(options.proxy)) {
const { proxyArgs } = resolveProxyConfig(options.proxy);
args.push(...proxyArgs);
return undefined;
// Puppeteer handles proxy via CLI args, not a separate option.
// SOCKS5: Chrome supports inline credentials natively (RFC 1929 auth).
// HTTP: Chrome does NOT support inline credentials — strip them and
// use page.authenticate() for Proxy-Authorization headers instead.
let proxyAuth: { username: string; password: string } | undefined;
if (options.proxy) {
if (isSocksProxy(options.proxy)) {
// SOCKS5: pass full URL with credentials to Chrome directly
const { proxyArgs } = resolveProxyConfig(options.proxy);
args.push(...proxyArgs);
} else if (typeof options.proxy === "string") {
const { server, username, password } = parseProxyUrl(options.proxy);
args.push(`--proxy-server=${server}`);
if (username) {
proxyAuth = { username, password: password ?? "" };
}
} else {
const parsed = parseProxyUrl(options.proxy.server);
args.push(`--proxy-server=${parsed.server}`);
if (options.proxy.bypass) {
args.push(`--proxy-bypass-list=${options.proxy.bypass}`);
}
const username = options.proxy.username ?? parsed.username;
const password = options.proxy.password ?? parsed.password;
if (username) {
proxyAuth = { username, password: password ?? "" };
}
}
}
if (typeof options.proxy === "string") {
const { server, username, password } = parseProxyUrl(options.proxy);
args.push(`--proxy-server=${server}`);
return username ? { username, password: password ?? "" } : undefined;
}
const browser = await puppeteer.default.launch({
executablePath: binaryPath,
headless: options.headless ?? true,
args,
ignoreDefaultArgs: IGNORE_DEFAULT_ARGS,
...options.launchOptions,
});
const parsed = parseProxyUrl(options.proxy.server);
args.push(`--proxy-server=${parsed.server}`);
if (options.proxy.bypass) {
args.push(`--proxy-bypass-list=${options.proxy.bypass}`);
}
const username = options.proxy.username ?? parsed.username;
const password = options.proxy.password ?? parsed.password;
return username ? { username, password: password ?? "" } : undefined;
}
/** Apply proxy auth monkey-patch and humanize behavioral patching. */
async function applyPostLaunch(
browser: Browser,
options: LaunchOptions,
proxyAuth?: { username: string; password: string },
): Promise<void> {
// Monkey-patch newPage() to auto-authenticate proxy credentials
if (proxyAuth) {
const origNewPage = browser.newPage.bind(browser);
const auth = proxyAuth;
@@ -71,6 +87,9 @@ async function applyPostLaunch(
};
}
// Human-like behavioral patching — FULL coverage, same as Playwright.
// This enables Bézier mouse movements, organic typing rhythms, and
// natural scrolling to bypass advanced anti-bot detection.
if (options.humanize) {
const { patchBrowser } = await import('./human-puppeteer/index.js');
const { resolveConfig } = await import('./human/config.js');
@@ -80,73 +99,6 @@ async function applyPostLaunch(
);
patchBrowser(browser, cfg);
}
}
/**
* Launch stealth Chromium browser via Puppeteer.
*
* @example
* ```ts
* import { launch } from 'cloakbrowser/puppeteer';
* // With humanize — human-like mouse, keyboard, scroll
* const browser = await launch({ humanize: true });
* const page = await browser.newPage();
* await page.goto('https://example.com');
* await page.click('#login'); // Bézier curve mouse movement
* await page.type('#email', 'user@example.com'); // Per-character timing
* ```
*/
export async function launch(options: LaunchOptions = {}): Promise<Browser> {
const puppeteer = await import("puppeteer-core");
const { binaryPath, args } = await resolveArgs(options);
const proxyAuth = resolveProxy(options, args);
const browser = await puppeteer.default.launch({
...options.launchOptions,
executablePath: binaryPath,
headless: options.headless ?? true,
args,
ignoreDefaultArgs: IGNORE_DEFAULT_ARGS,
});
await applyPostLaunch(browser, options, proxyAuth);
return browser;
}
/**
* Launch stealth Chromium with a persistent user profile via Puppeteer.
* Passes `userDataDir` to Puppeteer's launch options so cookies,
* localStorage, and session data persist across launches.
*
* @example
* ```ts
* import { launchPersistentContext } from 'cloakbrowser/puppeteer';
* const browser = await launchPersistentContext({
* userDataDir: './chrome-profile',
* headless: false,
* proxy: 'http://user:pass@proxy:8080',
* });
* const page = await browser.newPage();
* await page.goto('https://example.com');
* await browser.close();
* ```
*/
export async function launchPersistentContext(
options: LaunchOptions & { userDataDir: string }
): Promise<Browser> {
const puppeteer = await import("puppeteer-core");
const { binaryPath, args } = await resolveArgs(options);
const proxyAuth = resolveProxy(options, args);
const browser = await puppeteer.default.launch({
...options.launchOptions,
executablePath: binaryPath,
headless: options.headless ?? true,
args,
ignoreDefaultArgs: IGNORE_DEFAULT_ARGS,
userDataDir: options.userDataDir,
});
await applyPostLaunch(browser, options, proxyAuth);
return browser;
}
-2
View File
@@ -17,8 +17,6 @@ export interface LaunchOptions {
proxy?: string | { server: string; bypass?: string; username?: string; password?: string };
/** Additional Chromium CLI arguments. */
args?: string[];
/** Chrome extension paths to load. */
extensionPaths?: string[];
/** Include default stealth fingerprint args (default: true). Set false to use custom --fingerprint flags. */
stealthArgs?: boolean;
/** IANA timezone, e.g. "America/New_York". Sets --fingerprint-timezone binary flag. */
-17
View File
@@ -1,17 +0,0 @@
import { test, expect } from "vitest";
import path from "path";
import { _buildArgsForTest } from "../src/playwright.js";
test("extension paths inject chrome flags", () => {
const args = _buildArgsForTest({
extensionPaths: ["./ext"],
});
const abs = path.resolve("./ext");
expect(args).toContain(`--load-extension=${abs}`);
expect(args).toContain(
`--disable-extensions-except=${abs}`
);
});
-64
View File
@@ -21,70 +21,6 @@ describe("binaryInfo", () => {
});
});
describe("composable Playwright launch helpers", () => {
const origBinaryPath = process.env.CLOAKBROWSER_BINARY_PATH;
beforeEach(() => {
process.env.CLOAKBROWSER_BINARY_PATH = "/fake/chrome";
vi.resetModules();
});
afterEach(() => {
vi.restoreAllMocks();
vi.resetModules();
if (origBinaryPath) {
process.env.CLOAKBROWSER_BINARY_PATH = origBinaryPath;
} else {
delete process.env.CLOAKBROWSER_BINARY_PATH;
}
});
it("exports buildLaunchOptions and humanizeBrowser from the package entrypoint", async () => {
const entry = await import("../src/index.js");
expect(entry.buildLaunchOptions).toBeTypeOf("function");
expect(entry.humanizeBrowser).toBeTypeOf("function");
});
it("buildLaunchOptions returns Playwright options without launching a browser", async () => {
const { buildLaunchOptions } = await import("../src/index.js");
const options = await buildLaunchOptions({
headless: false,
proxy: "http://user:pass@proxy.example:8080",
args: ["--custom-flag"],
launchOptions: { timeout: 1234 },
});
expect(options.executablePath).toBe("/fake/chrome");
expect(options.headless).toBe(false);
expect(options.args).toContain("--custom-flag");
expect(options.ignoreDefaultArgs).toContain("--enable-automation");
expect(options.proxy).toEqual({
server: "http://proxy.example:8080",
username: "user",
password: "pass",
});
expect(options.timeout).toBe(1234);
});
it("humanizeBrowser patches an existing browser only when requested", async () => {
const { humanizeBrowser } = await import("../src/index.js");
const browser = {
contexts: () => [],
newContext: vi.fn(async () => ({})),
newPage: vi.fn(async () => ({ context: () => ({}) })),
};
const originalNewContext = browser.newContext;
await humanizeBrowser(browser as any, { humanize: false });
expect(browser.newContext).toBe(originalNewContext);
await humanizeBrowser(browser as any, { humanize: true });
expect(browser.newContext).not.toBe(originalNewContext);
});
});
// Integration tests require the binary — run with:
// CLOAKBROWSER_BINARY_PATH=/path/to/chrome npm test
describe.skipIf(!process.env.CLOAKBROWSER_BINARY_PATH)(
-100
View File
@@ -126,14 +126,6 @@ describe("puppeteer launch", () => {
expect(page.authenticate).not.toHaveBeenCalled();
});
it("forwards launchOptions to puppeteer launch", async () => {
const { launch } = await import("../src/puppeteer.js");
await launch({ launchOptions: { slowMo: 50 } });
const callArgs = vi.mocked(puppeteerMock.default.launch).mock.calls[0][0];
expect(callArgs.slowMo).toBe(50);
});
it("reconstructs SOCKS5 dict with auth into --proxy-server URL", async () => {
const { launch } = await import("../src/puppeteer.js");
const browser = await launch({
@@ -147,95 +139,3 @@ describe("puppeteer launch", () => {
expect(page.authenticate).not.toHaveBeenCalled();
});
});
describe("puppeteer launchPersistentContext", () => {
let puppeteerMock: any;
let mockBrowser: any;
beforeEach(async () => {
delete process.env.CLOAKBROWSER_BINARY_PATH;
puppeteerMock = await import("puppeteer-core");
mockBrowser = {
newPage: vi.fn().mockResolvedValue({
authenticate: vi.fn(),
}),
close: vi.fn(),
};
vi.mocked(puppeteerMock.default.launch).mockResolvedValue(mockBrowser);
});
afterEach(() => {
vi.restoreAllMocks();
});
it("passes userDataDir to puppeteer launch", async () => {
process.env.CLOAKBROWSER_BINARY_PATH = "/fake/chrome";
const { launchPersistentContext } = await import("../src/puppeteer.js");
await launchPersistentContext({ userDataDir: "./my-profile" });
expect(puppeteerMock.default.launch).toHaveBeenCalledWith(
expect.objectContaining({
userDataDir: "./my-profile",
executablePath: "/fake/chrome",
})
);
});
it("includes stealth args", async () => {
const { launchPersistentContext } = await import("../src/puppeteer.js");
await launchPersistentContext({ userDataDir: "./my-profile" });
const callArgs = vi.mocked(puppeteerMock.default.launch).mock.calls[0][0];
expect(callArgs.args.some((a: string) => a.startsWith("--fingerprint="))).toBe(true);
});
it("handles proxy auth with persistent context", async () => {
const { launchPersistentContext } = await import("../src/puppeteer.js");
const browser = await launchPersistentContext({
userDataDir: "./my-profile",
proxy: "http://user:pass@proxy:8080",
});
const page = await browser.newPage();
expect(page.authenticate).toHaveBeenCalledWith({
username: "user",
password: "pass",
});
});
it("keeps SOCKS5 credentials in --proxy-server URL", async () => {
const { launchPersistentContext } = await import("../src/puppeteer.js");
const browser = await launchPersistentContext({
userDataDir: "./my-profile",
proxy: "socks5://user:pass@proxy:1080",
});
const callArgs = vi.mocked(puppeteerMock.default.launch).mock.calls[0][0];
expect(callArgs.args).toContain("--proxy-server=socks5://user:pass@proxy:1080");
const page = await browser.newPage();
expect(page.authenticate).not.toHaveBeenCalled();
});
it("forwards launchOptions to puppeteer launch", async () => {
const { launchPersistentContext } = await import("../src/puppeteer.js");
await launchPersistentContext({ userDataDir: "./my-profile", launchOptions: { slowMo: 50 } });
const callArgs = vi.mocked(puppeteerMock.default.launch).mock.calls[0][0];
expect(callArgs.slowMo).toBe(50);
expect(callArgs.userDataDir).toBe("./my-profile");
});
it("injects timezone and locale as binary flags", async () => {
const { launchPersistentContext } = await import("../src/puppeteer.js");
await launchPersistentContext({
userDataDir: "./my-profile",
timezone: "Asia/Tokyo",
locale: "ja-JP",
});
const callArgs = vi.mocked(puppeteerMock.default.launch).mock.calls[0][0];
expect(callArgs.args).toContain("--fingerprint-timezone=Asia/Tokyo");
expect(callArgs.args).toContain("--lang=ja-JP");
});
});
+2 -90
View File
@@ -1,11 +1,9 @@
"""Unit tests for cloakserve — parse_connection_params, parse_cli_args, URL rewriting, connection tracking."""
import asyncio
import importlib.machinery
import importlib.util
import sys
from pathlib import Path
from types import SimpleNamespace
from unittest.mock import patch
import pytest
@@ -143,94 +141,8 @@ class TestParseCliArgs:
# ---------------------------------------------------------------------------
class TestWebSocketOriginGuard:
"""Verify cloakserve rejects browser-origin CDP WebSocket hijacks."""
def test_absent_origin_allowed_for_non_browser_cdp_clients(self):
assert _mod._origin_is_allowed(None, "127.0.0.1:9555")
def test_matching_origin_host_allowed(self):
assert _mod._origin_is_allowed("http://127.0.0.1:9555", "127.0.0.1:9555")
def test_chrome_devtools_origin_allowed(self):
assert _mod._origin_is_allowed("devtools://devtools", "127.0.0.1:9555")
assert _mod._origin_is_allowed("chrome-devtools://devtools", "127.0.0.1:9555")
@pytest.mark.parametrize("origin", [
"http://attacker.example",
"https://attacker.example",
"http://PUBLIC_HOST:9555",
"http://attacker.example:9555",
"http://127.0.0.1:9555/",
"http://127.0.0.1:9555/path",
"http://127.0.0.1:9555?q=1",
"http://127.0.0.1:9555#fragment",
"http://user@127.0.0.1:9555",
"http://@127.0.0.1:9555",
"http://:@127.0.0.1:9555",
"http://127.0.0.1:",
"null",
"file://",
])
def test_untrusted_browser_origins_rejected(self, origin):
assert not _mod._origin_is_allowed(origin, "127.0.0.1:9555")
def test_public_origin_matching_host_is_still_rejected(self):
assert not _mod._origin_is_allowed("http://attacker.example:9555", "attacker.example:9555")
@pytest.mark.parametrize("host", [
"user@127.0.0.1:9555",
"127.0.0.1:9555/path",
"127.0.0.1:9555?x=1",
"127.0.0.1:9555#fragment",
"127.0.0.1:9555, attacker.example:9555",
"@127.0.0.1:9555",
":@127.0.0.1:9555",
"127.0.0.1:",
"[::1]:",
])
def test_malformed_host_is_rejected_even_when_hostname_is_loopback(self, host):
assert not _mod._origin_is_allowed("http://127.0.0.1:9555", host)
def test_request_scheme_controls_host_default_port(self):
assert _mod._origin_is_allowed("https://localhost", "localhost", request_scheme="https")
assert not _mod._origin_is_allowed("https://localhost", "localhost", request_scheme="http")
def test_ws_handler_rejects_untrusted_origin_before_launching_chrome(self):
class RejectingPool:
async def get_or_launch(self, **_kwargs):
raise AssertionError("untrusted origin should be rejected before launching Chrome")
request = SimpleNamespace(
headers={"Host": "127.0.0.1:9555", "Origin": "http://attacker.example"},
app={"pool": RejectingPool()},
match_info={"path": "browser/browser-guid"},
)
response = asyncio.run(_mod.handle_ws_default(request))
assert response.status == 403
assert "untrusted" in response.text.lower()
def test_seed_ws_handler_rejects_untrusted_origin_before_launching_chrome(self):
class RejectingPool:
async def get_or_launch(self, **_kwargs):
raise AssertionError("untrusted origin should be rejected before launching Chrome")
request = SimpleNamespace(
headers={"Host": "127.0.0.1:9555", "Origin": "http://attacker.example"},
app={"pool": RejectingPool()},
match_info={"seed": "abc123", "path": "page/page-guid"},
)
response = asyncio.run(_mod.handle_ws_seed(request))
assert response.status == 403
assert "untrusted" in response.text.lower()
class TestHandlerURLRewriting:
"""Verify handlers rewrite CDP WebSocket URLs to the public cloakserve endpoint."""
class TestURLRewriting:
"""Test the URL rewriting logic used by /json/version and /json/list."""
def _rewrite_version(self, orig_ws: str, host: str, seed: str | None, scheme: str = "ws") -> str:
"""Replicate the URL rewrite logic from handle_json_version."""
-33
View File
@@ -1,33 +0,0 @@
import os
from unittest.mock import MagicMock, patch
from cloakbrowser import launch
@patch("cloakbrowser.browser.ensure_binary")
@patch("cloakbrowser.browser._import_sync_playwright")
def test_extension_loading(mock_playwright_import, mock_ensure_binary):
mock_ensure_binary.return_value = "/fake/chrome"
mock_browser = MagicMock()
mock_pw = MagicMock()
mock_pw.chromium.launch.return_value = mock_browser
mock_pw_manager = MagicMock()
mock_pw_manager.return_value.start.return_value = mock_pw
mock_playwright_import.return_value = mock_pw_manager
launch(extension_paths=["./ext"])
mock_pw.chromium.launch.assert_called_once()
launch_call = mock_pw.chromium.launch.call_args
args = launch_call.kwargs["args"]
abs_path = os.path.abspath("./ext")
assert f"--load-extension={abs_path}" in args
assert f"--disable-extensions-except={abs_path}" in args
-171
View File
@@ -1,171 +0,0 @@
"""Security tests for the AWS Lambda handler URL validation."""
from __future__ import annotations
import sys
from pathlib import Path
from unittest.mock import patch
import pytest
sys.path.insert(
0, str(Path(__file__).resolve().parent.parent / "examples" / "integrations" / "aws_lambda")
)
from lambda_handler import _build_launch_kwargs, _classify_error, _validate_url
class TestSchemeValidation:
"""Fix 1: only http:// and https:// are accepted."""
@pytest.mark.parametrize("url", [
"file:///etc/passwd",
"file:///proc/self/environ",
"data:text/html,<h1>pwned</h1>",
"javascript:alert(1)",
"chrome://settings",
"about:blank",
"ftp://example.com/file",
"",
])
def test_rejects_non_http_schemes(self, url):
with pytest.raises(ValueError, match="Only http"):
_validate_url(url)
@pytest.mark.parametrize("url", [
"https://example.com",
"http://example.com",
"https://example.com/path?q=1",
"HTTP://EXAMPLE.COM",
])
def test_accepts_http_and_https(self, url):
_validate_url(url)
def test_rejects_missing_hostname(self):
with pytest.raises(ValueError, match="no hostname"):
_validate_url("http://")
class TestSSRFProtection:
"""Fix 2: block private, loopback, link-local, reserved, and metadata IPs."""
@pytest.mark.parametrize("url,label", [
("http://169.254.169.254", "AWS metadata"),
("http://169.254.169.254/latest/meta-data/", "AWS metadata path"),
("http://127.0.0.1", "loopback"),
("http://127.0.0.2", "loopback range"),
("http://localhost", "localhost"),
("http://10.0.0.1", "private 10.x"),
("http://172.16.0.1", "private 172.16"),
("http://192.168.1.1", "private 192.168"),
("http://0.0.0.0", "unspecified"),
("http://[::1]", "IPv6 loopback"),
])
def test_rejects_private_ips(self, url, label):
with pytest.raises(ValueError, match="private/internal"):
_validate_url(url)
def test_rejects_carrier_grade_nat(self):
with pytest.raises(ValueError, match="private/internal"):
_validate_url("http://100.64.0.1")
def test_rejects_unresolvable_hostname(self):
with pytest.raises(ValueError, match="Cannot resolve"):
_validate_url("http://this-host-does-not-exist-cb-test.invalid")
def test_rejects_ipv4_mapped_ipv6(self):
"""::ffff:127.0.0.1 should be blocked even though it's technically IPv6."""
with pytest.raises(ValueError, match="private/internal"):
_validate_url("http://[::ffff:127.0.0.1]")
class TestExtraArgsRemoval:
"""Fix 3: caller-controlled extra_args are ignored; internal _strategy_args work."""
def test_ignores_caller_extra_args(self):
event = {"url": "https://example.com", "extra_args": ["--remote-debugging-port=9222"]}
kwargs = _build_launch_kwargs(event)
assert "--remote-debugging-port=9222" not in kwargs["args"]
def test_includes_strategy_args(self):
event = {"url": "https://example.com", "_strategy_args": ["--ignore-certificate-errors"]}
kwargs = _build_launch_kwargs(event)
assert "--ignore-certificate-errors" in kwargs["args"]
def test_classify_error_uses_strategy_args(self):
result = _classify_error(Exception("ERR_CERT_AUTHORITY_INVALID"))
assert "_strategy_args" in result
assert "extra_args" not in result
def test_always_includes_lambda_hardening_flags(self):
kwargs = _build_launch_kwargs({"url": "https://example.com"})
assert "--disable-dev-shm-usage" in kwargs["args"]
assert "--no-zygote" in kwargs["args"]
def test_caller_cannot_inject_strategy_args(self):
"""_strategy_args in the caller event must be stripped by _run() before launch."""
from lambda_handler import _run
import inspect
source = inspect.getsource(_run)
assert '"_strategy_args"' in source and "extra_args" in source, \
"_run must strip both _strategy_args and extra_args from caller event"
class TestRedirectSSRF:
"""Fix 5: post-navigation re-validation catches redirects to blocked IPs.
These mock socket.getaddrinfo to simulate redirect scenarios without
needing a real browser or HTTP server.
"""
def test_validate_url_catches_redirect_target(self):
"""If Chromium followed a redirect to 169.254.169.254, the post-nav
_validate_url(page.url) call should reject it."""
with pytest.raises(ValueError, match="private/internal"):
_validate_url("http://169.254.169.254/latest/meta-data/iam/security-credentials/")
def test_validate_url_catches_localhost_redirect(self):
with pytest.raises(ValueError, match="private/internal"):
_validate_url("http://127.0.0.1:8080/admin")
def test_code_flow_validates_before_content(self):
"""Verify that _attempt_scrape calls _validate_url(page.url) at line 282
BEFORE building the result dict at line 290 (sequential code path)."""
import ast
handler_path = (
Path(__file__).resolve().parent.parent
/ "examples" / "integrations" / "aws_lambda" / "lambda_handler.py"
)
source = handler_path.read_text()
tree = ast.parse(source)
for node in ast.walk(tree):
if isinstance(node, ast.AsyncFunctionDef) and node.name == "_attempt_scrape":
body = node.body
# Find the try block
for stmt in body:
if isinstance(stmt, ast.Try):
try_body = stmt.body
validate_lines = []
content_line = None
for s in try_body:
if isinstance(s, ast.Expr) and isinstance(s.value, ast.Call):
func = s.value.func
if isinstance(func, ast.Name) and func.id == "_validate_url":
validate_lines.append(s.lineno)
if isinstance(s, ast.AnnAssign):
if isinstance(s.target, ast.Name) and s.target.id == "result":
content_line = s.lineno
elif isinstance(s, ast.Assign):
for target in s.targets:
if isinstance(target, ast.Name) and target.id == "result":
content_line = s.lineno
assert len(validate_lines) >= 2, (
f"Expected 2 _validate_url calls, found {len(validate_lines)}"
)
assert content_line is not None
assert all(v < content_line for v in validate_lines), (
f"_validate_url (lines {validate_lines}) must come before "
f"result assignment (line {content_line})"
)
return
pytest.fail("Could not find _attempt_scrape function in source")