mirror of
https://github.com/CloakHQ/CloakBrowser.git
synced 2026-06-23 11:41:46 +02:00
Compare commits
26
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
b91274cc98 | ||
|
|
7a9a61d4de | ||
|
|
34bc095b65 | ||
|
|
a23268c9e9 | ||
|
|
0437a3f1f5 | ||
|
|
8fdaa5a2d3 | ||
|
|
b0ea580cba | ||
|
|
6f4f92e7c7 | ||
|
|
ad4d946ca6 | ||
|
|
95a98b6747 | ||
|
|
23f1d4098c | ||
|
|
d45d7de9a9 | ||
|
|
db0b5f1946 | ||
|
|
babef04e07 | ||
|
|
f8026a7b39 | ||
|
|
71f57d00d1 | ||
|
|
e9735392e8 | ||
|
|
0d41a4f023 | ||
|
|
80d9f7c14e | ||
|
|
114b3c826b | ||
|
|
c07c2b6b4a | ||
|
|
13b1b98b68 | ||
|
|
0d6ce76b1d | ||
|
|
f01902025a | ||
|
|
2df8c7e2d1 | ||
|
|
661b873dad |
@@ -16,9 +16,10 @@ jobs:
|
||||
contents: write # Download release assets
|
||||
steps:
|
||||
- name: Download release binaries
|
||||
run: gh release download ${{ github.event.inputs.tag }} --repo CloakHQ/cloakbrowser --pattern "cloakbrowser-*.tar.gz" --pattern "cloakbrowser-*.zip"
|
||||
run: gh release download "$RELEASE_TAG" --repo CloakHQ/cloakbrowser --pattern "cloakbrowser-*.tar.gz" --pattern "cloakbrowser-*.zip"
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
RELEASE_TAG: ${{ github.event.inputs.tag }}
|
||||
|
||||
- name: Attest build provenance
|
||||
uses: actions/attest-build-provenance@a2bbfa25375fe432b6a289bc6b6cd05ecd0c4c32 # v4.1.0
|
||||
|
||||
@@ -123,7 +123,7 @@ jobs:
|
||||
cloakhq/cloakbrowser:latest
|
||||
provenance: true
|
||||
sbom: true
|
||||
- uses: sigstore/cosign-installer@cad07c2e89fa2edd6e2d7bab4c1aa38e53f76003 # v4.1.1
|
||||
- uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6 # v4.1.2
|
||||
- name: Sign image
|
||||
run: cosign sign --yes cloakhq/cloakbrowser@${{ steps.build.outputs.digest }}
|
||||
- name: Attest build provenance
|
||||
|
||||
@@ -46,6 +46,7 @@ js/dist/
|
||||
*.whl
|
||||
AGENTS.md
|
||||
.beads
|
||||
result
|
||||
|
||||
# Private docs (launch posts, strategy)
|
||||
docs/
|
||||
|
||||
@@ -8,6 +8,36 @@ Changes are tagged: **[wrapper]** for Python/JS wrapper, **[binary]** for Chromi
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [0.3.29] — 2026-05-20
|
||||
|
||||
- **[wrapper]** **Security**: `cloakserve` — guard WebSocket origins to prevent browser-origin CSRF via CDP proxy (thanks [@0xlally](https://github.com/0xlally) for the report, [@honor2030](https://github.com/honor2030) for the fix, #239, #240)
|
||||
- **[wrapper]** **Security**: Lambda example — add URL scheme validation, SSRF protection, post-navigation re-validation, remove unsafe caller-controlled options (#233)
|
||||
- **[wrapper]** **Security**: CI — isolate `workflow_dispatch` input to avoid shell injection in attest-release (thanks [@aaronjmars](https://github.com/aaronjmars), #223)
|
||||
- **[wrapper]** **Security**: JS — bump tar + transitive deps via npm audit fix (thanks [@aaronjmars](https://github.com/aaronjmars), #222)
|
||||
- **[wrapper]** Add `extension_paths` parameter for loading Chrome extensions in all launch functions (thanks [@zackycodes](https://github.com/zackycodes), #210)
|
||||
- **[wrapper]** Humanize: add Playwright-style actionability checks — auto-wait for visible, enabled, stable elements before humanized actions (#228)
|
||||
- **[wrapper]** JS: export composable launch helpers — `buildLaunchOptions()` and `humanizeBrowser()` for custom Playwright integrations (thanks [@honor2030](https://github.com/honor2030), #244)
|
||||
- **[wrapper]** JS: add `launchPersistentContext()` to Puppeteer wrapper (#261)
|
||||
- **[wrapper]** Add `flake.nix` for Nix/NixOS (thanks [@Seryiza](https://github.com/Seryiza), #220)
|
||||
- **[meta]** JS: sync package-lock metadata (thanks [@245678000000](https://github.com/245678000000), #219)
|
||||
|
||||
## [0.3.28] — 2026-05-11
|
||||
|
||||
- **[wrapper]** **Security**: `cloakserve` — sanitize fingerprint seed to prevent path traversal, bind to `127.0.0.1` on bare metal, detect Podman containers (#217)
|
||||
- **[wrapper]** Fix GeoIP resolution hanging indefinitely — bounded with 10s timeout so `launch()` cannot stall (thanks [@manaskarra](https://github.com/manaskarra), #213)
|
||||
- **[wrapper]** JS: preserve iframe scope in humanized frame actions — `check()`, `uncheck()`, `selectOption()` now execute in the correct frame (thanks [@manaskarra](https://github.com/manaskarra), #201)
|
||||
- **[wrapper]** JS: add TypeScript types to humanized method options — `HumanActionOptions` type for `human_config` and `timeout` overrides (thanks [@eofreternal](https://github.com/eofreternal), #205)
|
||||
- **[wrapper]** Log when SOCKS5 credential auto-encoding rewrites a proxy URL (thanks [@Youhai020616](https://github.com/Youhai020616), #209)
|
||||
- **[wrapper]** JS: bump `playwright-core` peer dependency minimum to >=1.53.0 (#200)
|
||||
- **[meta]** Bump sigstore/cosign-installer in CI (#214)
|
||||
|
||||
## [0.3.27] — 2026-05-06
|
||||
|
||||
- **[wrapper]** Per-call `human_config` override — pass `human_config={...}` to individual humanized methods to override global HumanConfig on a per-action basis (#183)
|
||||
- **[wrapper]** Humanized `scrollIntoViewIfNeeded` — auto-scrolls with human-like behavior when `humanize=True` (#183)
|
||||
- **[wrapper]** Forward `timeout` parameter through humanized Playwright methods (#183)
|
||||
- **[wrapper]** Fix humanize timeout default to align with Playwright's 30s auto-retry instead of custom 2s (#172)
|
||||
|
||||
## [0.3.26] — 2026-04-28
|
||||
|
||||
- **[binary]** Windows x64 upgraded to Chromium 146.0.7680.177.4 — 57 source-level fingerprint patches (up from 33 on 145.0.7632.159.7), now matches Linux. Includes all binary improvements from 0.3.18–0.3.25: native SOCKS5 proxy with UDP ASSOCIATE (QUIC/HTTP3), WebRTC IP spoofing, proxy signal removal, CDP input stealth, storage quota normalization, WebAuthn/AAC/window position patches, WebGL and canvas consistency fixes, expanded GPU model database
|
||||
|
||||
@@ -128,7 +128,7 @@ Open [http://localhost:8080](http://localhost:8080). Create a profile. Click **L
|
||||
|
||||
---
|
||||
|
||||
## Latest: v0.3.26 (Chromium 146.0.7680.177.4)
|
||||
## Latest: v0.3.29 (Chromium 146.0.7680.177.4)
|
||||
|
||||
- **`launch_context_async()`** — async counterpart to `launch_context()`. Forwards kwargs to `browser.new_context()` for `storage_state`, `permissions`, `extra_http_headers` without a persistent profile folder.
|
||||
- **JS `contextOptions` escape hatch** — forward arbitrary options (including `storageState`) to Playwright's `newContext()` from `launchContext()` / `launchPersistentContext()`.
|
||||
@@ -371,9 +371,16 @@ ctx.close() # profile saved
|
||||
|
||||
# Next run — cookies, localStorage restored automatically
|
||||
ctx = launch_persistent_context("./my-profile", headless=False)
|
||||
|
||||
# Load Chrome extensions
|
||||
ctx = launch_persistent_context(
|
||||
"./my-profile",
|
||||
headless=False,
|
||||
extension_paths=["./my-extension"],
|
||||
)
|
||||
```
|
||||
|
||||
Supports all the same options as `launch_context()`: `proxy`, `user_agent`, `viewport`, `locale`, `timezone`, `color_scheme`, `geoip`.
|
||||
Supports all the same options as `launch_context()`: `proxy`, `user_agent`, `viewport`, `locale`, `timezone`, `color_scheme`, `geoip`, `extension_paths`.
|
||||
|
||||
Async version: `launch_persistent_context_async()`.
|
||||
|
||||
@@ -570,6 +577,7 @@ Access the original un-patched Playwright page at `page._original` if you need r
|
||||
| `CLOAKBROWSER_DOWNLOAD_URL` | `cloakbrowser.dev` | Custom download URL for binary |
|
||||
| `CLOAKBROWSER_AUTO_UPDATE` | `true` | Set to `false` to disable background update checks |
|
||||
| `CLOAKBROWSER_SKIP_CHECKSUM` | `false` | Set to `true` to skip SHA-256 verification after download |
|
||||
| `CLOAKBROWSER_GEOIP_TIMEOUT_SECONDS` | `5` | Max seconds for GeoIP resolution before continuing without it |
|
||||
|
||||
## Fingerprint Management
|
||||
|
||||
@@ -1192,5 +1200,14 @@ Issues and PRs welcome. If something isn't working, [open an issue](https://gith
|
||||
- [@evelaa123](https://github.com/evelaa123) — humanize behavior, persistent contexts, Windows fix
|
||||
- [@yahooguntu](https://github.com/yahooguntu) — persistent contexts
|
||||
- [@kitiho](https://github.com/kitiho) — null viewport fix
|
||||
- [@eofreternal](https://github.com/eofreternal) — humanConfig type fix
|
||||
- [@AlexTech314](https://github.com/AlexTech314) — AWS Lambda integration
|
||||
- [@eofreternal](https://github.com/eofreternal) — humanConfig type fix, humanized method option types
|
||||
- [@manaskarra](https://github.com/manaskarra) — iframe scope fix for humanized frame actions, GeoIP timeout guard
|
||||
- [@Youhai020616](https://github.com/Youhai020616) — SOCKS5 credential encoding logging
|
||||
- [@AlexTech314](https://github.com/AlexTech314) — AWS Lambda integration, cold-start hardening
|
||||
- [@dgtlmoon](https://github.com/dgtlmoon) — graceful pw.stop() cleanup
|
||||
- [@zackycodes](https://github.com/zackycodes) — Chrome extension loading
|
||||
- [@aaronjmars](https://github.com/aaronjmars) — security fixes (shell injection, dep bumps)
|
||||
- [@Seryiza](https://github.com/Seryiza) — Nix/NixOS flake
|
||||
- [@245678000000](https://github.com/245678000000) — package-lock sync
|
||||
- [@honor2030](https://github.com/honor2030) — cloakserve WebSocket origin guard, composable JS launch helpers
|
||||
- [@0xlally](https://github.com/0xlally) — security reports (cloakserve path traversal, WebSocket origin bypass)
|
||||
|
||||
+128
-9
@@ -18,17 +18,19 @@ Client:
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import ipaddress
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import random
|
||||
import re
|
||||
import shutil
|
||||
import socket
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
from urllib.parse import parse_qs
|
||||
from urllib.parse import parse_qs, urlparse
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
@@ -61,6 +63,94 @@ BASE_CHROME_ARGS = [
|
||||
|
||||
BASE_CDP_PORT = 5100
|
||||
|
||||
SAFE_SEED_RE = re.compile(r"^[A-Za-z0-9_-]{1,128}$")
|
||||
RESERVED_SEEDS = {"__default__"}
|
||||
TRUSTED_WS_ORIGINS = {"devtools://devtools", "chrome-devtools://devtools"}
|
||||
|
||||
|
||||
def _host_port_from_netloc(netloc: str, default_port: int) -> tuple[str, int] | None:
|
||||
"""Return a normalized (host, port) pair for an Origin/Host netloc."""
|
||||
if "," in netloc:
|
||||
return None
|
||||
try:
|
||||
parsed = urlparse(f"//{netloc.strip()}")
|
||||
authority = parsed.netloc.rsplit("@", 1)[-1]
|
||||
if (
|
||||
not parsed.hostname
|
||||
or parsed.username is not None
|
||||
or parsed.password is not None
|
||||
or authority.endswith(":")
|
||||
or parsed.path
|
||||
or parsed.params
|
||||
or parsed.query
|
||||
or parsed.fragment
|
||||
):
|
||||
return None
|
||||
return (parsed.hostname.lower(), parsed.port if parsed.port is not None else default_port)
|
||||
except ValueError:
|
||||
return None
|
||||
|
||||
|
||||
def _is_loopback_host(hostname: str) -> bool:
|
||||
"""Return True for localhost and loopback IP literals."""
|
||||
hostname = hostname.strip("[]").rstrip(".").lower()
|
||||
if hostname == "localhost":
|
||||
return True
|
||||
try:
|
||||
return ipaddress.ip_address(hostname).is_loopback
|
||||
except ValueError:
|
||||
return False
|
||||
|
||||
|
||||
def _origin_is_allowed(
|
||||
origin: str | None,
|
||||
host: str | None,
|
||||
request_scheme: str = "http",
|
||||
) -> bool:
|
||||
"""Return True when a WebSocket Origin is safe to proxy to local CDP."""
|
||||
if origin is None:
|
||||
# Playwright/Puppeteer and other non-browser CDP clients commonly omit
|
||||
# Origin. Keep those clients working while rejecting browser-origin CSRF.
|
||||
return True
|
||||
|
||||
origin = origin.strip()
|
||||
if not origin or origin.lower() == "null":
|
||||
return False
|
||||
if origin in TRUSTED_WS_ORIGINS:
|
||||
return True
|
||||
|
||||
try:
|
||||
parsed = urlparse(origin)
|
||||
except ValueError:
|
||||
return False
|
||||
|
||||
if parsed.scheme not in ("http", "https"):
|
||||
return False
|
||||
if parsed.path or parsed.params or parsed.query or parsed.fragment:
|
||||
return False
|
||||
|
||||
origin_default_port = 443 if parsed.scheme == "https" else 80
|
||||
request_scheme = request_scheme.split(",", 1)[0].strip().lower()
|
||||
request_default_port = 443 if request_scheme in ("https", "wss") else 80
|
||||
origin_host = _host_port_from_netloc(parsed.netloc, origin_default_port)
|
||||
request_host = _host_port_from_netloc(host or "", request_default_port)
|
||||
if origin_host is None or request_host is None:
|
||||
return False
|
||||
if not _is_loopback_host(request_host[0]):
|
||||
return False
|
||||
return origin_host == request_host
|
||||
|
||||
|
||||
def _reject_untrusted_origin(request: web.Request) -> web.Response | None:
|
||||
"""Reject browser-origin WebSocket upgrades that would expose local CDP."""
|
||||
origin = request.headers.get("Origin")
|
||||
host = request.headers.get("Host")
|
||||
scheme = request.headers.get("X-Forwarded-Proto", getattr(request, "scheme", "http"))
|
||||
if _origin_is_allowed(origin, host, request_scheme=scheme):
|
||||
return None
|
||||
logger.warning("Rejected CDP WebSocket from untrusted Origin %r for Host %r", origin, host)
|
||||
return web.Response(status=403, text="Forbidden: untrusted WebSocket origin\n")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# ChromeProcess — one running Chrome instance
|
||||
@@ -111,6 +201,14 @@ class ChromePool:
|
||||
self._locks[seed] = asyncio.Lock()
|
||||
return self._locks[seed]
|
||||
|
||||
def _safe_rmtree(self, path: str) -> None:
|
||||
resolved = Path(path).resolve()
|
||||
data_resolved = Path(self._data_dir).resolve()
|
||||
if resolved == data_resolved or not resolved.is_relative_to(data_resolved):
|
||||
logger.error("Refusing to delete path outside data_dir: %s", resolved)
|
||||
return
|
||||
shutil.rmtree(path, True)
|
||||
|
||||
def _allocate_port(self) -> int:
|
||||
"""Find a free port starting from _next_port."""
|
||||
for _ in range(100):
|
||||
@@ -159,6 +257,11 @@ class ChromePool:
|
||||
seed_key = "__default__"
|
||||
actual_seed = str(random.randint(10000, 99999))
|
||||
else:
|
||||
if not SAFE_SEED_RE.match(seed) or seed in RESERVED_SEEDS:
|
||||
raise web.HTTPBadRequest(
|
||||
text=json.dumps({"error": "Invalid fingerprint seed"}),
|
||||
content_type="application/json",
|
||||
)
|
||||
seed_key = seed
|
||||
actual_seed = seed
|
||||
|
||||
@@ -232,7 +335,7 @@ class ChromePool:
|
||||
if not await self._wait_for_cdp(port):
|
||||
process.kill()
|
||||
await asyncio.to_thread(process.wait, timeout=5)
|
||||
await asyncio.to_thread(shutil.rmtree, user_data_dir, True)
|
||||
await asyncio.to_thread(self._safe_rmtree, user_data_dir)
|
||||
raise web.HTTPBadGateway(
|
||||
text=json.dumps({"error": "Chrome failed to start"}),
|
||||
content_type="application/json",
|
||||
@@ -266,8 +369,7 @@ class ChromePool:
|
||||
await asyncio.to_thread(proc.process.wait, timeout=5)
|
||||
except subprocess.TimeoutExpired:
|
||||
proc.process.kill()
|
||||
# Clean up user data dir (can be slow for large profiles)
|
||||
await asyncio.to_thread(shutil.rmtree, proc.user_data_dir, True)
|
||||
await asyncio.to_thread(self._safe_rmtree, proc.user_data_dir)
|
||||
if self._default is proc:
|
||||
self._default = None
|
||||
self._locks.pop(key, None)
|
||||
@@ -511,8 +613,12 @@ async def proxy_cdp_websocket(
|
||||
logger.error("%s error: %s", label, exc)
|
||||
|
||||
|
||||
async def handle_ws_default(request: web.Request) -> web.WebSocketResponse:
|
||||
async def handle_ws_default(request: web.Request) -> web.StreamResponse:
|
||||
"""WebSocket proxy for default (no-seed) Chrome: /devtools/{type}/{guid}"""
|
||||
rejected = _reject_untrusted_origin(request)
|
||||
if rejected is not None:
|
||||
return rejected
|
||||
|
||||
pool: ChromePool = request.app["pool"]
|
||||
path = request.match_info.get("path", "")
|
||||
|
||||
@@ -530,8 +636,12 @@ async def handle_ws_default(request: web.Request) -> web.WebSocketResponse:
|
||||
return ws
|
||||
|
||||
|
||||
async def handle_ws_seed(request: web.Request) -> web.WebSocketResponse:
|
||||
async def handle_ws_seed(request: web.Request) -> web.StreamResponse:
|
||||
"""WebSocket proxy for seed-specific Chrome: /fingerprint/{seed}/devtools/{type}/{guid}"""
|
||||
rejected = _reject_untrusted_origin(request)
|
||||
if rejected is not None:
|
||||
return rejected
|
||||
|
||||
pool: ChromePool = request.app["pool"]
|
||||
seed = request.match_info["seed"]
|
||||
path = request.match_info.get("path", "")
|
||||
@@ -559,8 +669,8 @@ async def on_shutdown(app: web.Application) -> None:
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _default_data_dir() -> str:
|
||||
"""Smart default: Docker → /tmp/cloakserve, bare metal → ~/.cloakbrowser/cloakserve."""
|
||||
if os.path.exists("/.dockerenv"):
|
||||
"""Smart default: container → /tmp/cloakserve, bare metal → ~/.cloakbrowser/cloakserve."""
|
||||
if os.path.exists("/.dockerenv") or os.path.exists("/run/.containerenv"):
|
||||
return "/tmp/cloakserve"
|
||||
return str(Path.home() / ".cloakbrowser" / "cloakserve")
|
||||
|
||||
@@ -624,6 +734,13 @@ def main() -> None:
|
||||
binary = ensure_binary()
|
||||
config, global_args = parse_cli_args(sys.argv[1:])
|
||||
|
||||
if config["default_seed"] and (
|
||||
not SAFE_SEED_RE.match(config["default_seed"])
|
||||
or config["default_seed"] in RESERVED_SEEDS
|
||||
):
|
||||
logger.error("Invalid --fingerprint seed: %s", config["default_seed"])
|
||||
sys.exit(1)
|
||||
|
||||
pool = ChromePool(
|
||||
binary=binary,
|
||||
global_args=global_args,
|
||||
@@ -662,7 +779,9 @@ def main() -> None:
|
||||
port,
|
||||
)
|
||||
|
||||
web.run_app(app, host="0.0.0.0", port=port, print=None)
|
||||
in_container = os.path.exists("/.dockerenv") or os.path.exists("/run/.containerenv")
|
||||
host = "0.0.0.0" if in_container else "127.0.0.1"
|
||||
web.run_app(app, host=host, port=port, print=None)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -1 +1 @@
|
||||
__version__ = "0.3.26"
|
||||
__version__ = "0.3.29"
|
||||
|
||||
+53
-14
@@ -25,6 +25,7 @@ from .human.config import HumanConfigOverrides, HumanPreset
|
||||
|
||||
logger = logging.getLogger("cloakbrowser")
|
||||
|
||||
|
||||
# Sentinel to distinguish "viewport not provided" from "viewport=None" (disable emulation)
|
||||
_VIEWPORT_UNSET = object()
|
||||
|
||||
@@ -63,6 +64,7 @@ def launch(
|
||||
humanize: bool = False,
|
||||
human_preset: HumanPreset = "default",
|
||||
human_config: HumanConfigOverrides | None = None,
|
||||
extension_paths: list[str] | None = None,
|
||||
**kwargs: Any,
|
||||
) -> Any:
|
||||
"""Launch stealth Chromium browser. Returns a Playwright Browser object.
|
||||
@@ -74,6 +76,7 @@ def launch(
|
||||
Dict: {"server": "http://proxy:8080", "bypass": ".google.com", ...}
|
||||
— passed directly to Playwright.
|
||||
args: Additional Chromium CLI arguments to pass.
|
||||
extension_paths: List of Chrome extension paths to load.
|
||||
stealth_args: Include default stealth fingerprint args (default True).
|
||||
Set to False if you want to pass your own --fingerprint flags.
|
||||
timezone: IANA timezone (e.g. 'America/New_York'). Sets --fingerprint-timezone binary flag.
|
||||
@@ -111,7 +114,8 @@ def launch(
|
||||
if exit_ip and not (args and any(a.startswith("--fingerprint-webrtc-ip") for a in args)):
|
||||
args = list(args or [])
|
||||
args.append(f"--fingerprint-webrtc-ip={exit_ip}")
|
||||
chrome_args = build_args(stealth_args, (args or []) + proxy_extra_args, timezone=timezone, locale=locale, headless=headless)
|
||||
|
||||
chrome_args = build_args(stealth_args, (args or []) + proxy_extra_args, timezone=timezone, locale=locale, headless=headless, extension_paths=extension_paths)
|
||||
|
||||
logger.debug("Launching stealth Chromium (headless=%s, args=%d)", headless, len(chrome_args))
|
||||
|
||||
@@ -158,6 +162,7 @@ async def launch_async( # noqa: C901
|
||||
humanize: bool = False,
|
||||
human_preset: HumanPreset = "default",
|
||||
human_config: HumanConfigOverrides | None = None,
|
||||
extension_paths: list[str] | None = None,
|
||||
**kwargs: Any,
|
||||
) -> Any:
|
||||
"""Async version of launch(). Returns a Playwright Browser object.
|
||||
@@ -166,6 +171,7 @@ async def launch_async( # noqa: C901
|
||||
headless: Run in headless mode (default True).
|
||||
proxy: Proxy URL string or Playwright proxy dict (see launch() for details).
|
||||
args: Additional Chromium CLI arguments to pass.
|
||||
extension_paths: List of Chrome extension paths to load.
|
||||
stealth_args: Include default stealth fingerprint args (default True).
|
||||
timezone: IANA timezone (e.g. 'America/New_York'). Sets --fingerprint-timezone binary flag.
|
||||
locale: BCP 47 locale (e.g. 'en-US'). Sets --lang binary flag.
|
||||
@@ -201,7 +207,7 @@ async def launch_async( # noqa: C901
|
||||
if exit_ip and not (args and any(a.startswith("--fingerprint-webrtc-ip") for a in args)):
|
||||
args = list(args or [])
|
||||
args.append(f"--fingerprint-webrtc-ip={exit_ip}")
|
||||
chrome_args = build_args(stealth_args, (args or []) + proxy_extra_args, timezone=timezone, locale=locale, headless=headless)
|
||||
chrome_args = build_args(stealth_args, (args or []) + proxy_extra_args, timezone=timezone, locale=locale, headless=headless, extension_paths=extension_paths)
|
||||
|
||||
logger.debug("Launching stealth Chromium async (headless=%s, args=%d)", headless, len(chrome_args))
|
||||
|
||||
@@ -252,6 +258,7 @@ def launch_persistent_context(
|
||||
humanize: bool = False,
|
||||
human_preset: HumanPreset = "default",
|
||||
human_config: HumanConfigOverrides | None = None,
|
||||
extension_paths: list[str] | None = None,
|
||||
**kwargs: Any,
|
||||
) -> Any:
|
||||
"""Launch stealth browser with a persistent profile and return a BrowserContext.
|
||||
@@ -267,6 +274,7 @@ def launch_persistent_context(
|
||||
headless: Run in headless mode (default True).
|
||||
proxy: Proxy URL string or Playwright proxy dict (see launch() for details).
|
||||
args: Additional Chromium CLI arguments.
|
||||
extension_paths: List of Chrome extension paths to load.
|
||||
stealth_args: Include default stealth fingerprint args (default True).
|
||||
user_agent: Custom user agent string.
|
||||
viewport: Viewport size dict, e.g. {"width": 1920, "height": 1080}.
|
||||
@@ -305,7 +313,7 @@ def launch_persistent_context(
|
||||
if exit_ip and not (args and any(a.startswith("--fingerprint-webrtc-ip") for a in args)):
|
||||
args = list(args or [])
|
||||
args.append(f"--fingerprint-webrtc-ip={exit_ip}")
|
||||
chrome_args = build_args(stealth_args, (args or []) + proxy_extra_args, timezone=timezone, locale=locale, headless=headless)
|
||||
chrome_args = build_args(stealth_args, (args or []) + proxy_extra_args, timezone=timezone, locale=locale, headless=headless, extension_paths=extension_paths)
|
||||
|
||||
logger.debug(
|
||||
"Launching persistent stealth Chromium (headless=%s, user_data_dir=%s)",
|
||||
@@ -376,6 +384,7 @@ async def launch_persistent_context_async(
|
||||
humanize: bool = False,
|
||||
human_preset: HumanPreset = "default",
|
||||
human_config: HumanConfigOverrides | None = None,
|
||||
extension_paths: list[str] | None = None,
|
||||
**kwargs: Any,
|
||||
) -> Any:
|
||||
"""Async version of launch_persistent_context().
|
||||
@@ -390,6 +399,7 @@ async def launch_persistent_context_async(
|
||||
headless: Run in headless mode (default True).
|
||||
proxy: Proxy URL string or Playwright proxy dict (see launch() for details).
|
||||
args: Additional Chromium CLI arguments.
|
||||
extension_paths: List of Chrome extension paths to load.
|
||||
stealth_args: Include default stealth fingerprint args (default True).
|
||||
user_agent: Custom user agent string.
|
||||
viewport: Viewport size dict, e.g. {"width": 1920, "height": 1080}.
|
||||
@@ -431,7 +441,7 @@ async def launch_persistent_context_async(
|
||||
if exit_ip and not (args and any(a.startswith("--fingerprint-webrtc-ip") for a in args)):
|
||||
args = list(args or [])
|
||||
args.append(f"--fingerprint-webrtc-ip={exit_ip}")
|
||||
chrome_args = build_args(stealth_args, (args or []) + proxy_extra_args, timezone=timezone, locale=locale, headless=headless)
|
||||
chrome_args = build_args(stealth_args, (args or []) + proxy_extra_args, timezone=timezone, locale=locale, headless=headless, extension_paths=extension_paths)
|
||||
|
||||
logger.debug(
|
||||
"Launching persistent stealth Chromium async (headless=%s, user_data_dir=%s)",
|
||||
@@ -501,6 +511,7 @@ def launch_context(
|
||||
humanize: bool = False,
|
||||
human_preset: HumanPreset = "default",
|
||||
human_config: HumanConfigOverrides | None = None,
|
||||
extension_paths: list[str] | None = None,
|
||||
**kwargs: Any,
|
||||
) -> Any:
|
||||
"""Launch stealth browser and return a BrowserContext with common options pre-set.
|
||||
@@ -512,6 +523,7 @@ def launch_context(
|
||||
headless: Run in headless mode (default True).
|
||||
proxy: Proxy URL string or Playwright proxy dict (see launch() for details).
|
||||
args: Additional Chromium CLI arguments.
|
||||
extension_paths: List of Chrome extension paths to load.
|
||||
stealth_args: Include default stealth fingerprint args (default True).
|
||||
user_agent: Custom user agent string.
|
||||
viewport: Viewport size dict, e.g. {"width": 1920, "height": 1080}.
|
||||
@@ -543,7 +555,7 @@ def launch_context(
|
||||
# so it applies to ALL contexts, not just the default one.
|
||||
# locale and timezone are set via binary flags only — no CDP emulation.
|
||||
browser = launch(headless=headless, proxy=proxy, args=args, stealth_args=stealth_args,
|
||||
timezone=timezone, locale=locale, backend=backend)
|
||||
timezone=timezone, locale=locale, backend=backend, extension_paths=extension_paths)
|
||||
|
||||
context_kwargs: dict[str, Any] = {}
|
||||
if user_agent:
|
||||
@@ -600,6 +612,7 @@ async def launch_context_async(
|
||||
humanize: bool = False,
|
||||
human_preset: HumanPreset = "default",
|
||||
human_config: HumanConfigOverrides | None = None,
|
||||
extension_paths: list[str] | None = None,
|
||||
**kwargs: Any,
|
||||
) -> Any:
|
||||
"""Async version of launch_context().
|
||||
@@ -613,6 +626,7 @@ async def launch_context_async(
|
||||
headless: Run in headless mode (default True).
|
||||
proxy: Proxy URL string or Playwright proxy dict (see launch() for details).
|
||||
args: Additional Chromium CLI arguments.
|
||||
extension_paths: List of Chrome extension paths to load.
|
||||
stealth_args: Include default stealth fingerprint args (default True).
|
||||
user_agent: Custom user agent string.
|
||||
viewport: Viewport size dict, e.g. {"width": 1920, "height": 1080}.
|
||||
@@ -661,7 +675,7 @@ async def launch_context_async(
|
||||
# so it applies to ALL contexts, not just the default one.
|
||||
# locale and timezone are set via binary flags only — no CDP emulation.
|
||||
browser = await launch_async(headless=headless, proxy=proxy, args=args, stealth_args=stealth_args,
|
||||
timezone=timezone, locale=locale, backend=backend)
|
||||
timezone=timezone, locale=locale, backend=backend, extension_paths=extension_paths)
|
||||
|
||||
context_kwargs: dict[str, Any] = {}
|
||||
if user_agent:
|
||||
@@ -813,6 +827,10 @@ def _normalize_socks_string_url(url: str) -> str:
|
||||
truncate them at special chars like '='. Idempotent: pre-encoded input stays
|
||||
the same (decoded then re-encoded).
|
||||
|
||||
Emits an INFO log when re-encoding actually changes the URL, so users who
|
||||
previously hit silent SOCKS5 fallback (#157) can see what the wrapper did.
|
||||
Silent on already-encoded inputs (no false-positive noise).
|
||||
|
||||
On unparseable input (invalid port, broken IPv6 literal, etc.) logs a
|
||||
warning and returns the original string — preserves pre-fix pass-through
|
||||
behavior so Chromium's own error handling kicks in.
|
||||
@@ -828,18 +846,30 @@ def _normalize_socks_string_url(url: str) -> str:
|
||||
# urlparse returns None for absent components, "" for present-but-empty.
|
||||
if parsed.username is None and parsed.password is None:
|
||||
return url
|
||||
enc_user = quote(unquote(parsed.username), safe="") if parsed.username else ""
|
||||
raw_user = parsed.username or ""
|
||||
enc_user = quote(unquote(raw_user), safe="") if raw_user else ""
|
||||
# Preserve the colon separator when password component is present, even if
|
||||
# empty, so `user:@host` stays `user:@host`.
|
||||
if parsed.password is not None:
|
||||
enc_pass = quote(unquote(parsed.password), safe="") if parsed.password else ""
|
||||
raw_pass = parsed.password
|
||||
enc_pass = quote(unquote(raw_pass), safe="") if raw_pass else ""
|
||||
else:
|
||||
raw_pass = None
|
||||
enc_pass = None
|
||||
return _assemble_socks_url(
|
||||
normalized = _assemble_socks_url(
|
||||
parsed.scheme, parsed.hostname or "", parsed.port,
|
||||
enc_user, enc_pass,
|
||||
parsed.path, parsed.params, parsed.query, parsed.fragment,
|
||||
)
|
||||
# Compare credentials, not the full URL: urlparse cosmetically lowercases
|
||||
# scheme and hostname, so a full-string compare would falsely fire on
|
||||
# `socks5://USER:pass@HOST.com:1080` even when no encoding work happened.
|
||||
if enc_user != raw_user or enc_pass != raw_pass:
|
||||
logger.info(
|
||||
"Auto URL-encoded SOCKS5 proxy credentials (special characters "
|
||||
"detected). Pre-encode the URL to suppress this notice."
|
||||
)
|
||||
return normalized
|
||||
|
||||
|
||||
def _extract_proxy_url(proxy: str | ProxySettings | None) -> str | None:
|
||||
@@ -874,7 +904,7 @@ def maybe_resolve_geoip(
|
||||
if not geoip or not proxy:
|
||||
return timezone, locale, None
|
||||
|
||||
from .geoip import resolve_proxy_geo_with_ip
|
||||
from .geoip import resolve_proxy_exit_ip, resolve_proxy_geo_with_ip
|
||||
|
||||
proxy_url = _extract_proxy_url(proxy)
|
||||
if not proxy_url:
|
||||
@@ -882,8 +912,7 @@ def maybe_resolve_geoip(
|
||||
|
||||
# When both tz/locale are explicit, still resolve exit IP for WebRTC
|
||||
if timezone is not None and locale is not None:
|
||||
from .geoip import _resolve_exit_ip
|
||||
exit_ip = _resolve_exit_ip(proxy_url)
|
||||
exit_ip = resolve_proxy_exit_ip(proxy_url)
|
||||
return timezone, locale, exit_ip
|
||||
|
||||
geo_tz, geo_locale, exit_ip = resolve_proxy_geo_with_ip(proxy_url)
|
||||
@@ -918,8 +947,8 @@ def _resolve_webrtc_args(
|
||||
del args[idx]
|
||||
return args
|
||||
try:
|
||||
from .geoip import _resolve_exit_ip
|
||||
exit_ip = _resolve_exit_ip(proxy_url)
|
||||
from .geoip import resolve_proxy_exit_ip
|
||||
exit_ip = resolve_proxy_exit_ip(proxy_url)
|
||||
except Exception:
|
||||
logger.warning("Failed to resolve proxy exit IP for WebRTC spoofing; removing --fingerprint-webrtc-ip=auto")
|
||||
args = list(args)
|
||||
@@ -941,6 +970,7 @@ def build_args(
|
||||
timezone: str | None = None,
|
||||
locale: str | None = None,
|
||||
headless: bool = True,
|
||||
extension_paths: list[str] | None = None,
|
||||
) -> list[str]:
|
||||
"""Combine stealth args with user-provided args and locale flags.
|
||||
|
||||
@@ -984,6 +1014,15 @@ def build_args(
|
||||
logger.debug("Arg override: %s -> %s", seen[key], flag)
|
||||
seen[key] = flag
|
||||
|
||||
if extension_paths:
|
||||
abs_paths = [os.path.abspath(p) for p in extension_paths]
|
||||
ext_val = ",".join(abs_paths)
|
||||
|
||||
seen["--load-extension"] = f"--load-extension={ext_val}"
|
||||
seen["--disable-extensions-except"] = (
|
||||
f"--disable-extensions-except={ext_val}"
|
||||
)
|
||||
|
||||
return list(seen.values())
|
||||
|
||||
|
||||
|
||||
@@ -60,7 +60,6 @@ def _show_welcome() -> None:
|
||||
sys.stderr.write(" CloakBrowser — stealth Chromium for automation\n")
|
||||
sys.stderr.write(" https://github.com/CloakHQ/CloakBrowser\n")
|
||||
sys.stderr.write("\n")
|
||||
sys.stderr.write(" Issues? https://github.com/CloakHQ/CloakBrowser/issues\n")
|
||||
sys.stderr.write(" Donate? https://ko-fi.com/cloakhq\n")
|
||||
sys.stderr.write(" Star us if CloakBrowser helps your project!\n")
|
||||
sys.stderr.write("\n")
|
||||
|
||||
+65
-5
@@ -12,6 +12,8 @@ from __future__ import annotations
|
||||
|
||||
import ipaddress
|
||||
import logging
|
||||
import math
|
||||
import os
|
||||
import socket
|
||||
import tempfile
|
||||
import threading
|
||||
@@ -27,6 +29,8 @@ GEOIP_DB_URL = (
|
||||
)
|
||||
GEOIP_DB_FILENAME = "GeoLite2-City.mmdb"
|
||||
GEOIP_UPDATE_INTERVAL = 30 * 86_400 # 30 days
|
||||
DEFAULT_GEOIP_TIMEOUT_SECONDS = 5.0
|
||||
GEOIP_TIMEOUT_ENV = "CLOAKBROWSER_GEOIP_TIMEOUT_SECONDS"
|
||||
|
||||
# Country ISO code → BCP 47 locale (covers ~90 % of proxy traffic)
|
||||
COUNTRY_LOCALE_MAP: dict[str, str] = {
|
||||
@@ -77,11 +81,16 @@ def resolve_proxy_geo_with_ip(
|
||||
if db_path is None:
|
||||
return None, None, None
|
||||
|
||||
timeout = _get_geoip_timeout_seconds()
|
||||
deadline = _deadline_from_timeout(timeout)
|
||||
|
||||
# Exit IP (through proxy) is most accurate — gateway DNS may differ from exit
|
||||
ip = _resolve_exit_ip(proxy_url)
|
||||
if ip is None:
|
||||
ip = _resolve_exit_ip(proxy_url, timeout=_remaining_seconds(deadline))
|
||||
if ip is None and not _deadline_expired(deadline):
|
||||
ip = _resolve_proxy_ip(proxy_url)
|
||||
if ip is None:
|
||||
if ip is None or _deadline_expired(deadline):
|
||||
if deadline is not None and _deadline_expired(deadline):
|
||||
logger.warning("GeoIP resolution timed out after %.1fs; continuing without GeoIP", timeout)
|
||||
return None, None, None
|
||||
|
||||
try:
|
||||
@@ -152,13 +161,64 @@ _IP_ECHO_URLS = [
|
||||
]
|
||||
|
||||
|
||||
def _resolve_exit_ip(proxy_url: str) -> str | None:
|
||||
def _get_geoip_timeout_seconds() -> float:
|
||||
raw = os.getenv(GEOIP_TIMEOUT_ENV)
|
||||
if not raw:
|
||||
return DEFAULT_GEOIP_TIMEOUT_SECONDS
|
||||
try:
|
||||
timeout = float(raw)
|
||||
except ValueError:
|
||||
timeout = float("nan")
|
||||
if not math.isfinite(timeout):
|
||||
logger.warning(
|
||||
"Invalid %s=%r; using %.1fs",
|
||||
GEOIP_TIMEOUT_ENV,
|
||||
raw,
|
||||
DEFAULT_GEOIP_TIMEOUT_SECONDS,
|
||||
)
|
||||
return DEFAULT_GEOIP_TIMEOUT_SECONDS
|
||||
return max(timeout, 0.0)
|
||||
|
||||
|
||||
def _deadline_from_timeout(timeout: float) -> float | None:
|
||||
if timeout <= 0:
|
||||
return None
|
||||
return time.monotonic() + timeout
|
||||
|
||||
|
||||
def _remaining_seconds(deadline: float | None) -> float | None:
|
||||
if deadline is None:
|
||||
return None
|
||||
return max(deadline - time.monotonic(), 0.0)
|
||||
|
||||
|
||||
def _deadline_expired(deadline: float | None) -> bool:
|
||||
return deadline is not None and time.monotonic() >= deadline
|
||||
|
||||
|
||||
def resolve_proxy_exit_ip(proxy_url: str) -> str | None:
|
||||
"""Resolve only the proxy exit IP, bounded by the GeoIP timeout."""
|
||||
timeout = _get_geoip_timeout_seconds()
|
||||
deadline = _deadline_from_timeout(timeout)
|
||||
ip = _resolve_exit_ip(proxy_url, timeout=timeout)
|
||||
if ip is None and _deadline_expired(deadline):
|
||||
logger.warning("GeoIP resolution timed out after %.1fs; continuing without GeoIP", timeout)
|
||||
return ip
|
||||
|
||||
|
||||
def _resolve_exit_ip(proxy_url: str, timeout: float | None = None) -> str | None:
|
||||
"""Discover the proxy's actual exit IP by connecting through it."""
|
||||
import httpx
|
||||
|
||||
deadline = _deadline_from_timeout(timeout or 0)
|
||||
|
||||
for url in _IP_ECHO_URLS:
|
||||
try:
|
||||
resp = httpx.get(url, proxy=proxy_url, timeout=10.0)
|
||||
remaining = _remaining_seconds(deadline)
|
||||
if remaining is not None and remaining <= 0:
|
||||
return None
|
||||
request_timeout = min(10.0, remaining) if remaining is not None else 10.0
|
||||
resp = httpx.get(url, proxy=proxy_url, timeout=request_timeout)
|
||||
resp.raise_for_status()
|
||||
ip = resp.text.strip()
|
||||
# Validate it looks like an IP
|
||||
|
||||
+680
-146
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,342 @@
|
||||
"""Playwright-style actionability checks for the humanize layer (sync).
|
||||
|
||||
Checks: attached, visible, stable, enabled, editable, receives pointer events.
|
||||
Retry loop with backoff matching Playwright internals: [100, 250, 500, 1000]ms.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import time
|
||||
from typing import Any, FrozenSet, Optional, Tuple
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Error hierarchy — all subclass RuntimeError for backward compat
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class ActionabilityError(RuntimeError):
|
||||
"""Base for all actionability failures."""
|
||||
|
||||
def __init__(self, selector: str, check: str, message: str):
|
||||
self.selector = selector
|
||||
self.check = check
|
||||
super().__init__(f"Element {selector!r} failed {check} check: {message}")
|
||||
|
||||
|
||||
class ElementNotAttachedError(ActionabilityError):
|
||||
def __init__(self, selector: str):
|
||||
super().__init__(selector, "attached", "element not found in DOM")
|
||||
|
||||
|
||||
class ElementNotVisibleError(ActionabilityError):
|
||||
def __init__(self, selector: str):
|
||||
super().__init__(selector, "visible", "element is not visible")
|
||||
|
||||
|
||||
class ElementNotStableError(ActionabilityError):
|
||||
def __init__(self, selector: str):
|
||||
super().__init__(selector, "stable", "element position is still changing")
|
||||
|
||||
|
||||
class ElementNotEnabledError(ActionabilityError):
|
||||
def __init__(self, selector: str):
|
||||
super().__init__(selector, "enabled", "element is disabled")
|
||||
|
||||
|
||||
class ElementNotEditableError(ActionabilityError):
|
||||
def __init__(self, selector: str):
|
||||
super().__init__(selector, "editable", "element is not editable")
|
||||
|
||||
|
||||
class ElementNotReceivingEventsError(ActionabilityError):
|
||||
def __init__(self, selector: str, covering_tag: str = "unknown"):
|
||||
super().__init__(
|
||||
selector,
|
||||
"pointer_events",
|
||||
f"element is covered by <{covering_tag}>",
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Check-set constants
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
CHECKS_CLICK: FrozenSet[str] = frozenset({"attached", "visible", "enabled", "pointer_events"})
|
||||
CHECKS_HOVER: FrozenSet[str] = frozenset({"attached", "visible", "pointer_events"})
|
||||
CHECKS_INPUT: FrozenSet[str] = frozenset({"attached", "visible", "enabled", "editable", "pointer_events"})
|
||||
CHECKS_FOCUS: FrozenSet[str] = frozenset({"attached", "visible", "enabled"})
|
||||
CHECKS_CHECK: FrozenSet[str] = frozenset({"attached", "visible", "enabled", "pointer_events"})
|
||||
|
||||
_BACKOFF_MS = [100, 250, 500, 1000]
|
||||
|
||||
|
||||
def _backoff_sleep(attempt: int) -> None:
|
||||
idx = min(attempt, len(_BACKOFF_MS) - 1)
|
||||
time.sleep(_BACKOFF_MS[idx] / 1000.0)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Pre-scroll actionability: attached, visible, enabled, editable
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def ensure_actionable(
|
||||
page: Any,
|
||||
selector: str,
|
||||
checks: FrozenSet[str],
|
||||
timeout: float = 30000,
|
||||
force: bool = False,
|
||||
) -> None:
|
||||
"""Wait for element to pass actionability checks (pre-scroll).
|
||||
|
||||
Retries with backoff until *timeout* ms elapsed.
|
||||
Raises a specific ``ActionabilityError`` subclass on failure.
|
||||
If *force* is True, returns immediately.
|
||||
"""
|
||||
if force:
|
||||
return
|
||||
|
||||
deadline = time.monotonic() + timeout / 1000.0
|
||||
attempt = 0
|
||||
last_error: Optional[ActionabilityError] = None
|
||||
|
||||
while True:
|
||||
remaining_ms = max(0, (deadline - time.monotonic()) * 1000)
|
||||
if remaining_ms <= 0:
|
||||
if last_error is not None:
|
||||
raise last_error
|
||||
raise ActionabilityError(selector, "timeout", "timeout expired before first check")
|
||||
|
||||
try:
|
||||
loc = page.locator(selector).first
|
||||
|
||||
if "attached" in checks:
|
||||
try:
|
||||
loc.wait_for(state="attached", timeout=max(1, min(remaining_ms, 2000)))
|
||||
except Exception:
|
||||
raise ElementNotAttachedError(selector)
|
||||
|
||||
if "visible" in checks:
|
||||
if not loc.is_visible():
|
||||
raise ElementNotVisibleError(selector)
|
||||
|
||||
if "enabled" in checks:
|
||||
if not loc.is_enabled():
|
||||
raise ElementNotEnabledError(selector)
|
||||
|
||||
if "editable" in checks:
|
||||
if not loc.is_editable():
|
||||
raise ElementNotEditableError(selector)
|
||||
|
||||
return
|
||||
|
||||
except ActionabilityError as e:
|
||||
last_error = e
|
||||
if time.monotonic() >= deadline:
|
||||
raise last_error
|
||||
_backoff_sleep(attempt)
|
||||
attempt += 1
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Post-scroll stability check
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _boxes_differ(a: dict, b: dict) -> bool:
|
||||
return (
|
||||
abs(a["x"] - b["x"]) > 1
|
||||
or abs(a["y"] - b["y"]) > 1
|
||||
or abs(a["width"] - b["width"]) > 1
|
||||
or abs(a["height"] - b["height"]) > 1
|
||||
)
|
||||
|
||||
|
||||
def ensure_stable(
|
||||
page: Any,
|
||||
selector: str,
|
||||
timeout: float = 5000,
|
||||
) -> None:
|
||||
"""Wait for element position to stabilize (two samples 100ms apart).
|
||||
|
||||
Only call after scroll — skip if element was already in viewport.
|
||||
"""
|
||||
deadline = time.monotonic() + timeout / 1000.0
|
||||
attempt = 0
|
||||
|
||||
while True:
|
||||
remaining_ms = max(0, (deadline - time.monotonic()) * 1000)
|
||||
if remaining_ms <= 0:
|
||||
raise ElementNotStableError(selector)
|
||||
|
||||
loc = page.locator(selector).first
|
||||
box1 = loc.bounding_box(timeout=max(1, min(remaining_ms, 1000)))
|
||||
if box1 is None:
|
||||
raise ElementNotAttachedError(selector)
|
||||
|
||||
time.sleep(0.1)
|
||||
|
||||
box2 = loc.bounding_box(timeout=max(1, min(remaining_ms, 1000)))
|
||||
if box2 is None:
|
||||
raise ElementNotAttachedError(selector)
|
||||
|
||||
if not _boxes_differ(box1, box2):
|
||||
return
|
||||
|
||||
if time.monotonic() >= deadline:
|
||||
raise ElementNotStableError(selector)
|
||||
|
||||
_backoff_sleep(attempt)
|
||||
attempt += 1
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Pointer-events check (post-scroll, at actual click coordinates)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_POINTER_EVENTS_LOCATOR_JS = """(expected, coords) => {
|
||||
const target = document.elementFromPoint(coords.x, coords.y);
|
||||
if (!target) return { hit: false, reason: 'no_element_at_point', covering: 'none' };
|
||||
let node = target;
|
||||
while (node) { if (node === expected) return { hit: true }; node = node.parentNode; }
|
||||
if (expected.contains(target)) return { hit: true };
|
||||
return { hit: false, reason: 'covered', covering: target.tagName || 'unknown' };
|
||||
}"""
|
||||
|
||||
_POINTER_EVENTS_HANDLE_JS = """(expected, coords) => {
|
||||
const target = document.elementFromPoint(coords.x, coords.y);
|
||||
if (!target) return { hit: false, reason: 'no_element_at_point', covering: 'none' };
|
||||
let node = target;
|
||||
while (node) { if (node === expected) return { hit: true }; node = node.parentNode; }
|
||||
if (expected.contains(target)) return { hit: true };
|
||||
return { hit: false, reason: 'covered', covering: target.tagName || 'unknown' };
|
||||
}"""
|
||||
|
||||
|
||||
def check_pointer_events(
|
||||
page: Any,
|
||||
selector: str,
|
||||
x: float,
|
||||
y: float,
|
||||
stealth: Any = None,
|
||||
timeout: float = 5000,
|
||||
) -> None:
|
||||
"""Check that elementFromPoint(x, y) hits the expected element.
|
||||
|
||||
Uses locator.evaluate() so all Playwright selector types work
|
||||
(text=, role=, XPath, CSS, etc.). Retries with backoff for transient overlays.
|
||||
"""
|
||||
deadline = time.monotonic() + timeout / 1000.0
|
||||
attempt = 0
|
||||
coords = {"x": x, "y": y}
|
||||
|
||||
while True:
|
||||
try:
|
||||
loc = page.locator(selector).first
|
||||
result = loc.evaluate(_POINTER_EVENTS_LOCATOR_JS, coords)
|
||||
except Exception as exc:
|
||||
logger.debug("pointer_events check failed for %r: %s", selector, exc)
|
||||
result = None
|
||||
|
||||
if result and result.get("hit", False):
|
||||
return
|
||||
|
||||
covering = (result or {}).get("covering", "unknown")
|
||||
|
||||
if time.monotonic() >= deadline:
|
||||
raise ElementNotReceivingEventsError(selector, covering)
|
||||
|
||||
_backoff_sleep(attempt)
|
||||
attempt += 1
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# ElementHandle variant
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def ensure_actionable_handle(
|
||||
page: Any,
|
||||
el: Any,
|
||||
checks: FrozenSet[str],
|
||||
timeout: float = 30000,
|
||||
force: bool = False,
|
||||
) -> None:
|
||||
"""Actionability checks for ElementHandle (no selector needed).
|
||||
|
||||
Uses Playwright's wait_for_element_state where available.
|
||||
"""
|
||||
if force:
|
||||
return
|
||||
|
||||
deadline = time.monotonic() + timeout / 1000.0
|
||||
attempt = 0
|
||||
last_error: Optional[ActionabilityError] = None
|
||||
label = "<ElementHandle>"
|
||||
|
||||
while True:
|
||||
remaining_ms = max(0, (deadline - time.monotonic()) * 1000)
|
||||
if remaining_ms <= 0:
|
||||
if last_error is not None:
|
||||
raise last_error
|
||||
raise ActionabilityError(label, "timeout", "timeout expired before first check")
|
||||
|
||||
try:
|
||||
if "visible" in checks:
|
||||
try:
|
||||
el.wait_for_element_state("visible", timeout=max(1, min(remaining_ms, 2000)))
|
||||
except Exception:
|
||||
raise ElementNotVisibleError(label)
|
||||
|
||||
if "enabled" in checks:
|
||||
try:
|
||||
el.wait_for_element_state("enabled", timeout=max(1, min(remaining_ms, 2000)))
|
||||
except Exception:
|
||||
raise ElementNotEnabledError(label)
|
||||
|
||||
if "editable" in checks:
|
||||
try:
|
||||
el.wait_for_element_state("editable", timeout=max(1, min(remaining_ms, 2000)))
|
||||
except Exception:
|
||||
raise ElementNotEditableError(label)
|
||||
|
||||
return
|
||||
|
||||
except ActionabilityError as e:
|
||||
last_error = e
|
||||
if time.monotonic() >= deadline:
|
||||
raise last_error
|
||||
_backoff_sleep(attempt)
|
||||
attempt += 1
|
||||
|
||||
|
||||
def check_pointer_events_handle(
|
||||
page: Any,
|
||||
el: Any,
|
||||
x: float,
|
||||
y: float,
|
||||
timeout: float = 5000,
|
||||
) -> None:
|
||||
"""Pointer-events check for ElementHandle."""
|
||||
deadline = time.monotonic() + timeout / 1000.0
|
||||
attempt = 0
|
||||
|
||||
coords = {"x": x, "y": y}
|
||||
|
||||
while True:
|
||||
try:
|
||||
result = el.evaluate(_POINTER_EVENTS_HANDLE_JS, coords)
|
||||
except Exception:
|
||||
result = None
|
||||
|
||||
if result and result.get("hit", False):
|
||||
return
|
||||
|
||||
covering = (result or {}).get("covering", "unknown")
|
||||
|
||||
if time.monotonic() >= deadline:
|
||||
raise ElementNotReceivingEventsError("<ElementHandle>", covering)
|
||||
|
||||
_backoff_sleep(attempt)
|
||||
attempt += 1
|
||||
@@ -0,0 +1,247 @@
|
||||
"""Playwright-style actionability checks for the humanize layer (async).
|
||||
|
||||
Async mirror of actionability.py — same logic, uses asyncio.sleep and await.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import time
|
||||
from typing import Any, FrozenSet, Optional
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
from .actionability import (
|
||||
ActionabilityError,
|
||||
ElementNotAttachedError,
|
||||
ElementNotVisibleError,
|
||||
ElementNotStableError,
|
||||
ElementNotEnabledError,
|
||||
ElementNotEditableError,
|
||||
ElementNotReceivingEventsError,
|
||||
_BACKOFF_MS,
|
||||
_boxes_differ,
|
||||
_POINTER_EVENTS_LOCATOR_JS,
|
||||
_POINTER_EVENTS_HANDLE_JS,
|
||||
)
|
||||
|
||||
|
||||
async def _async_backoff_sleep(attempt: int) -> None:
|
||||
idx = min(attempt, len(_BACKOFF_MS) - 1)
|
||||
await asyncio.sleep(_BACKOFF_MS[idx] / 1000.0)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Pre-scroll actionability
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
async def async_ensure_actionable(
|
||||
page: Any,
|
||||
selector: str,
|
||||
checks: FrozenSet[str],
|
||||
timeout: float = 30000,
|
||||
force: bool = False,
|
||||
) -> None:
|
||||
if force:
|
||||
return
|
||||
|
||||
deadline = time.monotonic() + timeout / 1000.0
|
||||
attempt = 0
|
||||
last_error: Optional[ActionabilityError] = None
|
||||
|
||||
while True:
|
||||
remaining_ms = max(0, (deadline - time.monotonic()) * 1000)
|
||||
if remaining_ms <= 0:
|
||||
if last_error is not None:
|
||||
raise last_error
|
||||
raise ActionabilityError(selector, "timeout", "timeout expired before first check")
|
||||
|
||||
try:
|
||||
loc = page.locator(selector).first
|
||||
|
||||
if "attached" in checks:
|
||||
try:
|
||||
await loc.wait_for(state="attached", timeout=max(1, min(remaining_ms, 2000)))
|
||||
except Exception:
|
||||
raise ElementNotAttachedError(selector)
|
||||
|
||||
if "visible" in checks:
|
||||
if not await loc.is_visible():
|
||||
raise ElementNotVisibleError(selector)
|
||||
|
||||
if "enabled" in checks:
|
||||
if not await loc.is_enabled():
|
||||
raise ElementNotEnabledError(selector)
|
||||
|
||||
if "editable" in checks:
|
||||
if not await loc.is_editable():
|
||||
raise ElementNotEditableError(selector)
|
||||
|
||||
return
|
||||
|
||||
except ActionabilityError as e:
|
||||
last_error = e
|
||||
if time.monotonic() >= deadline:
|
||||
raise last_error
|
||||
await _async_backoff_sleep(attempt)
|
||||
attempt += 1
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Post-scroll stability check
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
async def async_ensure_stable(
|
||||
page: Any,
|
||||
selector: str,
|
||||
timeout: float = 5000,
|
||||
) -> None:
|
||||
deadline = time.monotonic() + timeout / 1000.0
|
||||
attempt = 0
|
||||
|
||||
while True:
|
||||
remaining_ms = max(0, (deadline - time.monotonic()) * 1000)
|
||||
if remaining_ms <= 0:
|
||||
raise ElementNotStableError(selector)
|
||||
|
||||
loc = page.locator(selector).first
|
||||
box1 = await loc.bounding_box(timeout=max(1, min(remaining_ms, 1000)))
|
||||
if box1 is None:
|
||||
raise ElementNotAttachedError(selector)
|
||||
|
||||
await asyncio.sleep(0.1)
|
||||
|
||||
box2 = await loc.bounding_box(timeout=max(1, min(remaining_ms, 1000)))
|
||||
if box2 is None:
|
||||
raise ElementNotAttachedError(selector)
|
||||
|
||||
if not _boxes_differ(box1, box2):
|
||||
return
|
||||
|
||||
if time.monotonic() >= deadline:
|
||||
raise ElementNotStableError(selector)
|
||||
|
||||
await _async_backoff_sleep(attempt)
|
||||
attempt += 1
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Pointer-events check
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
async def async_check_pointer_events(
|
||||
page: Any,
|
||||
selector: str,
|
||||
x: float,
|
||||
y: float,
|
||||
stealth: Any = None,
|
||||
timeout: float = 5000,
|
||||
) -> None:
|
||||
deadline = time.monotonic() + timeout / 1000.0
|
||||
attempt = 0
|
||||
coords = {"x": x, "y": y}
|
||||
|
||||
while True:
|
||||
try:
|
||||
loc = page.locator(selector).first
|
||||
result = await loc.evaluate(_POINTER_EVENTS_LOCATOR_JS, coords)
|
||||
except Exception as exc:
|
||||
logger.debug("pointer_events check failed for %r: %s", selector, exc)
|
||||
result = None
|
||||
|
||||
if result and result.get("hit", False):
|
||||
return
|
||||
|
||||
covering = (result or {}).get("covering", "unknown")
|
||||
|
||||
if time.monotonic() >= deadline:
|
||||
raise ElementNotReceivingEventsError(selector, covering)
|
||||
|
||||
await _async_backoff_sleep(attempt)
|
||||
attempt += 1
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# ElementHandle variant
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
async def async_ensure_actionable_handle(
|
||||
page: Any,
|
||||
el: Any,
|
||||
checks: FrozenSet[str],
|
||||
timeout: float = 30000,
|
||||
force: bool = False,
|
||||
) -> None:
|
||||
if force:
|
||||
return
|
||||
|
||||
deadline = time.monotonic() + timeout / 1000.0
|
||||
attempt = 0
|
||||
last_error: Optional[ActionabilityError] = None
|
||||
label = "<ElementHandle>"
|
||||
|
||||
while True:
|
||||
remaining_ms = max(0, (deadline - time.monotonic()) * 1000)
|
||||
if remaining_ms <= 0:
|
||||
if last_error is not None:
|
||||
raise last_error
|
||||
raise ActionabilityError(label, "timeout", "timeout expired before first check")
|
||||
|
||||
try:
|
||||
if "visible" in checks:
|
||||
try:
|
||||
await el.wait_for_element_state("visible", timeout=max(1, min(remaining_ms, 2000)))
|
||||
except Exception:
|
||||
raise ElementNotVisibleError(label)
|
||||
|
||||
if "enabled" in checks:
|
||||
try:
|
||||
await el.wait_for_element_state("enabled", timeout=max(1, min(remaining_ms, 2000)))
|
||||
except Exception:
|
||||
raise ElementNotEnabledError(label)
|
||||
|
||||
if "editable" in checks:
|
||||
try:
|
||||
await el.wait_for_element_state("editable", timeout=max(1, min(remaining_ms, 2000)))
|
||||
except Exception:
|
||||
raise ElementNotEditableError(label)
|
||||
|
||||
return
|
||||
|
||||
except ActionabilityError as e:
|
||||
last_error = e
|
||||
if time.monotonic() >= deadline:
|
||||
raise last_error
|
||||
await _async_backoff_sleep(attempt)
|
||||
attempt += 1
|
||||
|
||||
|
||||
async def async_check_pointer_events_handle(
|
||||
page: Any,
|
||||
el: Any,
|
||||
x: float,
|
||||
y: float,
|
||||
timeout: float = 5000,
|
||||
) -> None:
|
||||
deadline = time.monotonic() + timeout / 1000.0
|
||||
attempt = 0
|
||||
|
||||
coords = {"x": x, "y": y}
|
||||
|
||||
while True:
|
||||
try:
|
||||
result = await el.evaluate(_POINTER_EVENTS_HANDLE_JS, coords)
|
||||
except Exception:
|
||||
result = None
|
||||
|
||||
if result and result.get("hit", False):
|
||||
return
|
||||
|
||||
covering = (result or {}).get("covering", "unknown")
|
||||
|
||||
if time.monotonic() >= deadline:
|
||||
raise ElementNotReceivingEventsError("<ElementHandle>", covering)
|
||||
|
||||
await _async_backoff_sleep(attempt)
|
||||
attempt += 1
|
||||
@@ -201,6 +201,25 @@ def resolve_config(
|
||||
return HumanConfig(**merged)
|
||||
|
||||
|
||||
def merge_config(base: HumanConfig, overrides: dict | None) -> HumanConfig:
|
||||
"""Merge ``overrides`` (a dict of HumanConfig field names → values) on top of
|
||||
``base``. Returns a new HumanConfig — ``base`` is never mutated.
|
||||
|
||||
Used by per-call overrides like ``page.type(sel, text, human_config={...})``
|
||||
so the same page can use different timings for different inputs without
|
||||
re-patching.
|
||||
|
||||
Unknown keys are ignored silently to keep this forgiving for callers.
|
||||
"""
|
||||
if not overrides:
|
||||
return base
|
||||
merged = {k: getattr(base, k) for k in base.__dataclass_fields__}
|
||||
for k, v in overrides.items():
|
||||
if k in base.__dataclass_fields__:
|
||||
merged[k] = v
|
||||
return HumanConfig(**merged)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Utility functions
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -4,7 +4,7 @@ from __future__ import annotations
|
||||
|
||||
import math
|
||||
import random
|
||||
from typing import Any, Optional, Tuple
|
||||
from typing import Any, Callable, Optional, Tuple
|
||||
|
||||
from .config import HumanConfig, rand, rand_range, rand_int_range, sleep_ms
|
||||
from .mouse import RawMouse, human_move
|
||||
@@ -18,10 +18,15 @@ def _is_in_viewport(bounds: dict, viewport_height: int, cfg: HumanConfig) -> boo
|
||||
return top_edge >= zone_top and bottom_edge <= zone_bottom
|
||||
|
||||
|
||||
def _get_element_box(page: Any, selector: str) -> Optional[dict]:
|
||||
def _get_element_box(page: Any, selector: str, timeout: float = 30000) -> Optional[dict]:
|
||||
"""Locate ``selector`` and return its bounding box.
|
||||
|
||||
The ``timeout`` is forwarded to Playwright's ``boundingBox(timeout=...)``
|
||||
so callers can extend it for slow-loading elements (#172).
|
||||
"""
|
||||
try:
|
||||
el = page.locator(selector).first
|
||||
return el.bounding_box(timeout=2000)
|
||||
return el.bounding_box(timeout=max(1, timeout))
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
@@ -39,13 +44,24 @@ def _smooth_wheel(raw: RawMouse, delta: int, cfg: HumanConfig) -> None:
|
||||
sleep_ms(rand(8, 20))
|
||||
|
||||
|
||||
def scroll_to_element(
|
||||
def human_scroll_into_view(
|
||||
page: Any,
|
||||
raw: RawMouse,
|
||||
selector: str,
|
||||
get_box: Callable[[], Optional[dict]],
|
||||
cursor_x: float, cursor_y: float,
|
||||
cfg: HumanConfig,
|
||||
) -> Tuple[dict, float, float]:
|
||||
) -> Tuple[dict, float, float, bool]:
|
||||
"""Humanized scrolling that uses an arbitrary ``get_box`` callable
|
||||
instead of a CSS selector.
|
||||
|
||||
Used both by ``scroll_to_element`` (selector-based) and by
|
||||
``ElementHandle.scroll_into_view_if_needed`` / ``Locator.scroll_into_view_if_needed``
|
||||
(handle-based) so the same accelerate \u2192 cruise \u2192 decelerate \u2192 overshoot
|
||||
behavior runs everywhere.
|
||||
|
||||
Returns ``(box, cursor_x, cursor_y, did_scroll)`` \u2014 *did_scroll* is False
|
||||
when the element was already in the viewport.
|
||||
"""
|
||||
viewport = page.viewport_size
|
||||
if not viewport:
|
||||
raise RuntimeError("Viewport size not available")
|
||||
@@ -53,15 +69,12 @@ def scroll_to_element(
|
||||
viewport_height = viewport["height"]
|
||||
viewport_width = viewport["width"]
|
||||
|
||||
box = _get_element_box(page, selector)
|
||||
box = get_box()
|
||||
if box is None:
|
||||
sleep_ms(200)
|
||||
box = _get_element_box(page, selector)
|
||||
if box is None:
|
||||
raise RuntimeError(f"Element not found: {selector}")
|
||||
raise RuntimeError("Element not found while scrolling into view")
|
||||
|
||||
if _is_in_viewport(box, viewport_height, cfg):
|
||||
return box, cursor_x, cursor_y
|
||||
return box, cursor_x, cursor_y, False
|
||||
|
||||
# Move cursor into scroll area
|
||||
scroll_area_x = round(viewport_width * rand(0.3, 0.7))
|
||||
@@ -105,7 +118,7 @@ def scroll_to_element(
|
||||
|
||||
# Check visibility every 3 steps
|
||||
if i % 3 == 2 or i == total_clicks - 1:
|
||||
box = _get_element_box(page, selector)
|
||||
box = get_box()
|
||||
if box and _is_in_viewport(box, viewport_height, cfg):
|
||||
break
|
||||
if scrolled >= abs_distance * 1.1:
|
||||
@@ -125,8 +138,31 @@ def scroll_to_element(
|
||||
# Settle
|
||||
sleep_ms(rand_range(cfg.scroll_settle_delay))
|
||||
|
||||
box = _get_element_box(page, selector)
|
||||
box = get_box()
|
||||
if box is None:
|
||||
raise RuntimeError(f"Element lost after scrolling: {selector}")
|
||||
raise RuntimeError("Element lost after scrolling into view")
|
||||
|
||||
return box, cursor_x, cursor_y
|
||||
return box, cursor_x, cursor_y, True
|
||||
|
||||
|
||||
def scroll_to_element(
|
||||
page: Any,
|
||||
raw: RawMouse,
|
||||
selector: str,
|
||||
cursor_x: float, cursor_y: float,
|
||||
cfg: HumanConfig,
|
||||
timeout: float = 30000,
|
||||
) -> Tuple[dict, float, float, bool]:
|
||||
"""Selector-based humanized scroll.
|
||||
|
||||
``timeout`` is forwarded to ``locator.bounding_box(timeout=...)`` so callers
|
||||
such as ``page.click('#x', timeout=5000)`` can wait longer for slow elements
|
||||
(#172). Default matches Playwright's 30000ms when not specified.
|
||||
|
||||
Returns ``(box, cursor_x, cursor_y, did_scroll)``.
|
||||
"""
|
||||
return human_scroll_into_view(
|
||||
page, raw,
|
||||
lambda: _get_element_box(page, selector, timeout),
|
||||
cursor_x, cursor_y, cfg,
|
||||
)
|
||||
|
||||
@@ -8,17 +8,22 @@ from __future__ import annotations
|
||||
|
||||
import math
|
||||
import random
|
||||
from typing import Any, Optional, Tuple
|
||||
from typing import Any, Awaitable, Callable, Optional, Tuple
|
||||
|
||||
from .config import HumanConfig, rand, rand_range, rand_int_range, async_sleep_ms
|
||||
from .mouse_async import AsyncRawMouse, async_human_move
|
||||
from .scroll import _is_in_viewport
|
||||
|
||||
|
||||
async def _get_element_box_async(page: Any, selector: str) -> Optional[dict]:
|
||||
async def _get_element_box_async(
|
||||
page: Any, selector: str, timeout: float = 30000,
|
||||
) -> Optional[dict]:
|
||||
"""Async variant. ``timeout`` is forwarded to Playwright's
|
||||
``boundingBox(timeout=...)`` so callers can extend it for slow-loading
|
||||
elements (#172)."""
|
||||
try:
|
||||
el = page.locator(selector).first
|
||||
return await el.bounding_box(timeout=2000)
|
||||
return await el.bounding_box(timeout=max(1, timeout))
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
@@ -36,13 +41,23 @@ async def _async_smooth_wheel(raw: AsyncRawMouse, delta: int, cfg: HumanConfig)
|
||||
await async_sleep_ms(rand(8, 20))
|
||||
|
||||
|
||||
async def async_scroll_to_element(
|
||||
async def async_human_scroll_into_view(
|
||||
page: Any,
|
||||
raw: AsyncRawMouse,
|
||||
selector: str,
|
||||
get_box: Callable[[], Awaitable[Optional[dict]]],
|
||||
cursor_x: float, cursor_y: float,
|
||||
cfg: HumanConfig,
|
||||
) -> Tuple[dict, float, float]:
|
||||
) -> Tuple[dict, float, float, bool]:
|
||||
"""Humanized scrolling using an arbitrary async ``get_box`` callable.
|
||||
|
||||
Used by both ``async_scroll_to_element`` (selector-based) and the
|
||||
ElementHandle / Locator ``scroll_into_view_if_needed`` patches so all
|
||||
scrolling paths share the same accelerate \u2192 cruise \u2192 decelerate
|
||||
\u2192 overshoot behavior.
|
||||
|
||||
Returns ``(box, cursor_x, cursor_y, did_scroll)`` \u2014 *did_scroll* is False
|
||||
when the element was already in the viewport.
|
||||
"""
|
||||
viewport = page.viewport_size
|
||||
if not viewport:
|
||||
raise RuntimeError("Viewport size not available")
|
||||
@@ -50,15 +65,12 @@ async def async_scroll_to_element(
|
||||
viewport_height = viewport["height"]
|
||||
viewport_width = viewport["width"]
|
||||
|
||||
box = await _get_element_box_async(page, selector)
|
||||
box = await get_box()
|
||||
if box is None:
|
||||
await async_sleep_ms(200)
|
||||
box = await _get_element_box_async(page, selector)
|
||||
if box is None:
|
||||
raise RuntimeError(f"Element not found: {selector}")
|
||||
raise RuntimeError("Element not found while scrolling into view")
|
||||
|
||||
if _is_in_viewport(box, viewport_height, cfg):
|
||||
return box, cursor_x, cursor_y
|
||||
return box, cursor_x, cursor_y, False
|
||||
|
||||
# Move cursor into scroll area
|
||||
scroll_area_x = round(viewport_width * rand(0.3, 0.7))
|
||||
@@ -102,7 +114,7 @@ async def async_scroll_to_element(
|
||||
|
||||
# Check visibility every 3 steps
|
||||
if i % 3 == 2 or i == total_clicks - 1:
|
||||
box = await _get_element_box_async(page, selector)
|
||||
box = await get_box()
|
||||
if box and _is_in_viewport(box, viewport_height, cfg):
|
||||
break
|
||||
if scrolled >= abs_distance * 1.1:
|
||||
@@ -122,8 +134,31 @@ async def async_scroll_to_element(
|
||||
# Settle
|
||||
await async_sleep_ms(rand_range(cfg.scroll_settle_delay))
|
||||
|
||||
box = await _get_element_box_async(page, selector)
|
||||
box = await get_box()
|
||||
if box is None:
|
||||
raise RuntimeError(f"Element lost after scrolling: {selector}")
|
||||
raise RuntimeError("Element lost after scrolling into view")
|
||||
|
||||
return box, cursor_x, cursor_y
|
||||
return box, cursor_x, cursor_y, True
|
||||
|
||||
|
||||
async def async_scroll_to_element(
|
||||
page: Any,
|
||||
raw: AsyncRawMouse,
|
||||
selector: str,
|
||||
cursor_x: float, cursor_y: float,
|
||||
cfg: HumanConfig,
|
||||
timeout: float = 30000,
|
||||
) -> Tuple[dict, float, float, bool]:
|
||||
"""Selector-based humanized scroll (async).
|
||||
|
||||
``timeout`` is forwarded to ``locator.bounding_box(timeout=...)`` so callers
|
||||
such as ``page.click('#x', timeout=5000)`` can wait longer for slow elements
|
||||
(#172). Default matches Playwright's 30000ms when not specified.
|
||||
|
||||
Returns ``(box, cursor_x, cursor_y, did_scroll)``.
|
||||
"""
|
||||
async def _get():
|
||||
return await _get_element_box_async(page, selector, timeout)
|
||||
return await async_human_scroll_into_view(
|
||||
page, raw, _get, cursor_x, cursor_y, cfg,
|
||||
)
|
||||
|
||||
@@ -70,7 +70,7 @@ Only `url` is required. Everything else is optional.
|
||||
|
||||
| Field | Type | Default |
|
||||
|---|---|---|
|
||||
| `url` | str | required |
|
||||
| `url` | str | required — `http://` and `https://` only |
|
||||
| `proxy` | str / dict | none — `http://user:pass@host:port` or a Playwright proxy dict |
|
||||
| `humanize` | bool | `false` — enable human-like mouse / keyboard / scroll |
|
||||
| `human_preset` | str | `"default"` or `"careful"` |
|
||||
@@ -79,7 +79,6 @@ Only `url` is required. Everything else is optional.
|
||||
| `locale` | str | none — BCP-47, e.g. `"en-US"` |
|
||||
| `viewport` | `{width,height}` | `1920x947` (cloakbrowser default) |
|
||||
| `user_agent` | str | none |
|
||||
| `extra_args` | `list[str]` | `[]` — extra Chromium CLI flags |
|
||||
|
||||
### Navigation
|
||||
|
||||
@@ -102,8 +101,6 @@ Only `url` is required. Everything else is optional.
|
||||
| `wait_for_selector` | str | none — CSS or XPath |
|
||||
| `wait_for_selector_state` | str | `"visible"` — also `attached` / `detached` / `hidden` |
|
||||
| `wait_for_selector_timeout_ms` | int | `30000` |
|
||||
| `wait_for_function` | str | none — JS expression returning truthy when ready |
|
||||
| `wait_for_function_timeout_ms` | int | `30000` |
|
||||
| `wait_ms` | int | none — fixed pause |
|
||||
|
||||
### Capture
|
||||
@@ -118,7 +115,7 @@ Only `url` is required. Everything else is optional.
|
||||
The handler retries transient navigation failures inline within the same Lambda invocation. Two layers, both built-in:
|
||||
|
||||
- **Launch retries** — 3 attempts with 0.3 s + 0.6 s backoff. Recovers Xvfb / Chromium spawn races at cold start. Fast and cheap; not configurable.
|
||||
- **Strategy retries** — default 1 attempt, configurable via the `retries` event field. Recovers specific post-launch error classes by relaunching with adjusted Chromium args / page-load budgets.
|
||||
- **Strategy retries** — default 1 attempt, configurable via the `retries` event field. Recovers specific post-launch error classes by relaunching with adjusted internal Chromium args / page-load budgets.
|
||||
|
||||
| Field | Type | Default |
|
||||
|---|---|---|
|
||||
@@ -176,6 +173,22 @@ For latency-sensitive use cases: provision concurrency, schedule a CloudWatch/Ev
|
||||
|
||||
If you see empty/missing dynamic content on cold-start invocations, raise `max_settle_ms` in the event payload (e.g. `25000`) — the default `15000` is tuned for warm runs.
|
||||
|
||||
## Security
|
||||
|
||||
The handler validates all incoming URLs before navigation:
|
||||
|
||||
- **Scheme restriction** — only `http://` and `https://` are accepted. `file://`, `data:`, `javascript:`, and other schemes are rejected.
|
||||
- **SSRF protection** — hostnames are resolved before navigation and checked against private, loopback, link-local, reserved, and multicast IP ranges. This blocks access to cloud metadata endpoints (e.g. `169.254.169.254`), localhost services, and internal networks.
|
||||
- **Post-navigation re-validation** — the final URL is re-checked after page load and after post-navigation waits to catch server-side redirects to blocked destinations.
|
||||
- **No caller-controlled Chromium flags** — the handler does not accept arbitrary CLI flags from the event. Internal retry strategies add flags as needed (e.g. `--ignore-certificate-errors` for cert errors).
|
||||
- **No arbitrary JS execution** — `wait_for_function` is not exposed. Use `wait_for_selector` or `smart_wait` instead.
|
||||
|
||||
**Limitations**:
|
||||
- Post-navigation re-validation prevents response *exfiltration*, but does not prevent the browser from *making* the request. If an internal endpoint has side effects on GET, the request will still reach it before validation rejects the response. Use network-level controls (security groups, VPC) to protect side-effect-bearing internal endpoints.
|
||||
- DNS rebinding attacks can bypass pre-navigation IP checks in theory, though the post-navigation re-validation provides a second layer of defense.
|
||||
|
||||
**Trust boundary**: if this handler is exposed to untrusted callers (Lambda Function URL, API Gateway without auth, public ALB), add an authentication layer (API Gateway authorizer, IAM auth, etc.). The URL validation above is defense-in-depth, not a substitute for access control.
|
||||
|
||||
## License
|
||||
|
||||
The patched Chromium binary inside the upstream `cloakhq/cloakbrowser` image is governed by the **CloakBrowser Binary License** (published at https://github.com/CloakHQ/CloakBrowser/blob/main/BINARY-LICENSE.md). Internal organizational use (private ECR, your own scraping pipelines, your own business) is free. Exposing this Lambda as a paid API to third-party customers — i.e. browser-as-a-service — requires an OEM/SaaS license from CloakHQ (`cloakhq@pm.me`). Do not push the resulting image to a public registry; that would be redistribution and is prohibited.
|
||||
|
||||
@@ -5,7 +5,7 @@ Always runs **headed** via the Xvfb display started by `lambda-entrypoint.sh`.
|
||||
Event schema (all fields except `url` are optional):
|
||||
|
||||
Launch options (passed to cloakbrowser.launch_context_async):
|
||||
url str required, the page to scrape
|
||||
url str required, the page to scrape (http/https only)
|
||||
proxy str|dict http://user:pass@host:port or Playwright proxy dict
|
||||
humanize bool False — enable human-like mouse/keyboard/scroll
|
||||
human_preset str "default" | "careful"
|
||||
@@ -14,7 +14,6 @@ Event schema (all fields except `url` are optional):
|
||||
locale str BCP-47, e.g. "en-US"
|
||||
viewport {width,height} defaults to 1920x947 (cloakbrowser DEFAULT_VIEWPORT)
|
||||
user_agent str custom UA (rare — cloakbrowser sets one already)
|
||||
extra_args list[str] additional Chromium CLI flags
|
||||
|
||||
Navigation options (passed to page.goto):
|
||||
wait_until str "load"|"domcontentloaded"|"networkidle"|"commit"
|
||||
@@ -35,8 +34,6 @@ Event schema (all fields except `url` are optional):
|
||||
wait_for_selector str CSS or XPath selector
|
||||
wait_for_selector_state str "attached"|"detached"|"visible"|"hidden", default "visible"
|
||||
wait_for_selector_timeout_ms int 30000
|
||||
wait_for_function str JS expression that returns truthy when ready
|
||||
wait_for_function_timeout_ms int 30000
|
||||
wait_ms int fixed pause in ms (page.wait_for_timeout)
|
||||
|
||||
Capture options:
|
||||
@@ -64,11 +61,14 @@ from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import base64
|
||||
import ipaddress
|
||||
import json
|
||||
import logging
|
||||
import socket
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
from urllib.parse import urlparse
|
||||
|
||||
from cloakbrowser import launch_context_async
|
||||
|
||||
@@ -76,6 +76,26 @@ logger = logging.getLogger("cloakbrowser.lambda")
|
||||
logger.setLevel(logging.INFO)
|
||||
|
||||
|
||||
def _validate_url(url: str) -> None:
|
||||
"""Reject non-HTTP schemes and URLs that resolve to private/internal IPs."""
|
||||
parsed = urlparse(url)
|
||||
if parsed.scheme.lower() not in ("http", "https"):
|
||||
raise ValueError(
|
||||
f"Only http:// and https:// URLs are supported, got: {parsed.scheme!r}"
|
||||
)
|
||||
hostname = parsed.hostname
|
||||
if not hostname:
|
||||
raise ValueError("URL has no hostname")
|
||||
try:
|
||||
infos = socket.getaddrinfo(hostname, None, socket.AF_UNSPEC, socket.SOCK_STREAM)
|
||||
except socket.gaierror:
|
||||
raise ValueError(f"Cannot resolve hostname: {hostname}")
|
||||
for info in infos:
|
||||
addr = ipaddress.ip_address(info[4][0])
|
||||
if not addr.is_global:
|
||||
raise ValueError("URLs targeting private/internal networks are blocked")
|
||||
|
||||
|
||||
def _diag_snapshot() -> str:
|
||||
"""Capture Xvfb status, Xvfb log, X11 socket state, and env for error reports."""
|
||||
import os
|
||||
@@ -118,7 +138,7 @@ def _build_launch_kwargs(event: dict) -> dict:
|
||||
# Lambda's restricted process model can't fork from Chromium's zygote
|
||||
# — without this, child renderer processes fail to spawn.
|
||||
"--no-zygote",
|
||||
*event.get("extra_args", []),
|
||||
*event.get("_strategy_args", []),
|
||||
],
|
||||
}
|
||||
for key in ("proxy", "humanize", "human_preset", "geoip",
|
||||
@@ -159,7 +179,7 @@ async def _smart_wait(page, dom_stable_ms: int = 1500, max_settle_ms: int = 1500
|
||||
|
||||
|
||||
_EXPLICIT_WAIT_KEYS = (
|
||||
"wait_for_load_state", "wait_for_selector", "wait_for_function", "wait_ms",
|
||||
"wait_for_load_state", "wait_for_selector", "wait_ms",
|
||||
)
|
||||
|
||||
|
||||
@@ -184,11 +204,6 @@ async def _post_nav_waits(page, event: dict) -> None:
|
||||
state=event.get("wait_for_selector_state", "visible"),
|
||||
timeout=event.get("wait_for_selector_timeout_ms", 30000),
|
||||
)
|
||||
if "wait_for_function" in event:
|
||||
await page.wait_for_function(
|
||||
event["wait_for_function"],
|
||||
timeout=event.get("wait_for_function_timeout_ms", 30000),
|
||||
)
|
||||
if "wait_ms" in event:
|
||||
await page.wait_for_timeout(event["wait_ms"])
|
||||
|
||||
@@ -235,7 +250,7 @@ def _classify_error(err: Exception) -> dict | None:
|
||||
msg = str(err)
|
||||
if "ERR_CERT" in msg:
|
||||
return {
|
||||
"extra_args": ["--ignore-certificate-errors"],
|
||||
"_strategy_args": ["--ignore-certificate-errors"],
|
||||
"goto_timeout_ms": 60000,
|
||||
}
|
||||
if ("Timeout" in msg and "exceeded" in msg) or "ERR_CONNECTION_TIMED_OUT" in msg:
|
||||
@@ -263,8 +278,10 @@ async def _attempt_scrape(url: str, event: dict) -> dict:
|
||||
wait_until=event.get("wait_until", "domcontentloaded"),
|
||||
timeout=event.get("goto_timeout_ms", 30000),
|
||||
)
|
||||
_validate_url(page.url)
|
||||
|
||||
await _post_nav_waits(page, event)
|
||||
_validate_url(page.url)
|
||||
|
||||
result: dict = {
|
||||
"title": await page.title(),
|
||||
@@ -306,6 +323,8 @@ async def _run(event: dict) -> dict:
|
||||
set to 0 to disable retry entirely).
|
||||
"""
|
||||
url = event["url"]
|
||||
_validate_url(url)
|
||||
event = {k: v for k, v in event.items() if k not in ("extra_args", "_strategy_args")}
|
||||
retries_left = max(0, int(event.get("retries", 1)))
|
||||
history: list[dict] = []
|
||||
current_event = event
|
||||
@@ -326,8 +345,8 @@ async def _run(event: dict) -> dict:
|
||||
})
|
||||
logger.warning("attempt %d failed (%s); retrying with strategy=%s",
|
||||
len(history), str(e)[:120], strategy)
|
||||
merged_args = list(current_event.get("extra_args", [])) + list(strategy.get("extra_args", []))
|
||||
current_event = {**current_event, **strategy, "extra_args": merged_args}
|
||||
merged_args = list(current_event.get("_strategy_args", [])) + list(strategy.get("_strategy_args", []))
|
||||
current_event = {**current_event, **strategy, "_strategy_args": merged_args}
|
||||
retries_left -= 1
|
||||
# No backoff: strategy overrides change goto budget directly;
|
||||
# the prior failure was either fast (cert reject) or already
|
||||
|
||||
Generated
+27
@@ -0,0 +1,27 @@
|
||||
{
|
||||
"nodes": {
|
||||
"nixpkgs": {
|
||||
"locked": {
|
||||
"lastModified": 1777954456,
|
||||
"narHash": "sha256-hGdgeU2Nk87RAuZyYjyDjFL6LK7dAZN5RE9+hrDTkDU=",
|
||||
"owner": "NixOS",
|
||||
"repo": "nixpkgs",
|
||||
"rev": "549bd84d6279f9852cae6225e372cc67fb91a4c1",
|
||||
"type": "github"
|
||||
},
|
||||
"original": {
|
||||
"owner": "NixOS",
|
||||
"ref": "nixos-unstable",
|
||||
"repo": "nixpkgs",
|
||||
"type": "github"
|
||||
}
|
||||
},
|
||||
"root": {
|
||||
"inputs": {
|
||||
"nixpkgs": "nixpkgs"
|
||||
}
|
||||
}
|
||||
},
|
||||
"root": "root",
|
||||
"version": 7
|
||||
}
|
||||
@@ -0,0 +1,237 @@
|
||||
{
|
||||
description = "CloakBrowser development shell with Nix-packaged Chromium binaries";
|
||||
|
||||
inputs = {
|
||||
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
|
||||
};
|
||||
|
||||
outputs = { self, nixpkgs }:
|
||||
let
|
||||
inherit (nixpkgs) lib;
|
||||
|
||||
supportedSystems = [
|
||||
"x86_64-linux"
|
||||
"aarch64-linux"
|
||||
];
|
||||
|
||||
forAllSystems = lib.genAttrs supportedSystems;
|
||||
|
||||
packageInfo = {
|
||||
x86_64-linux = {
|
||||
platformTag = "linux-x64";
|
||||
version = "146.0.7680.177.3";
|
||||
hash = "sha256-WvAn+q+x/vmTPreEwJS3ZHBt4io3KizuhLwRf8SrU38=";
|
||||
};
|
||||
aarch64-linux = {
|
||||
platformTag = "linux-arm64";
|
||||
version = "146.0.7680.177.3";
|
||||
hash = "sha256-i3HOU7T9ExMnMxox+6ODXXGILRm/qr3njdD1OQvRb0U=";
|
||||
};
|
||||
};
|
||||
|
||||
cloakbrowserBinaryLicense = {
|
||||
shortName = "cloakbrowser-binary";
|
||||
fullName = "CloakBrowser Binary License";
|
||||
url = "https://github.com/CloakHQ/CloakBrowser/blob/main/BINARY-LICENSE.md";
|
||||
free = false;
|
||||
redistributable = false;
|
||||
};
|
||||
|
||||
mkPkgs = system: import nixpkgs {
|
||||
inherit system;
|
||||
config.allowUnfree = true;
|
||||
};
|
||||
|
||||
runtimeLibraries = pkgs: with pkgs; [
|
||||
alsa-lib
|
||||
at-spi2-atk
|
||||
at-spi2-core
|
||||
atk
|
||||
cairo
|
||||
cups
|
||||
dbus
|
||||
expat
|
||||
fontconfig
|
||||
freetype
|
||||
gdk-pixbuf
|
||||
glib
|
||||
gtk3
|
||||
libdrm
|
||||
libgbm
|
||||
libGL
|
||||
libpulseaudio
|
||||
libxkbcommon
|
||||
mesa
|
||||
nspr
|
||||
nss
|
||||
pango
|
||||
systemd
|
||||
wayland
|
||||
libx11
|
||||
libxcb
|
||||
libxcomposite
|
||||
libxcursor
|
||||
libxdamage
|
||||
libxext
|
||||
libxfixes
|
||||
libxi
|
||||
libxrandr
|
||||
libxrender
|
||||
libxscrnsaver
|
||||
libxshmfence
|
||||
libxtst
|
||||
];
|
||||
|
||||
fontPackages = pkgs: with pkgs; [
|
||||
freefont_ttf
|
||||
ipafont
|
||||
liberation_ttf
|
||||
noto-fonts
|
||||
noto-fonts-cjk-sans
|
||||
noto-fonts-color-emoji
|
||||
tlwg
|
||||
unifont
|
||||
wqy_zenhei
|
||||
];
|
||||
|
||||
desktopPackages = pkgs: with pkgs; [
|
||||
adwaita-icon-theme
|
||||
gsettings-desktop-schemas
|
||||
xdg-utils
|
||||
];
|
||||
|
||||
mkCloakBrowserChromium = pkgs: system:
|
||||
let
|
||||
info = packageInfo.${system} or (throw "CloakBrowser flake package currently supports only x86_64-linux and aarch64-linux.");
|
||||
archiveName = "cloakbrowser-${info.platformTag}.tar.gz";
|
||||
chromiumVersion = info.version;
|
||||
libs = runtimeLibraries pkgs;
|
||||
desktopDeps = desktopPackages pkgs;
|
||||
fonts = fontPackages pkgs;
|
||||
fontsConf = pkgs.makeFontsConf {
|
||||
fontDirectories = fonts;
|
||||
};
|
||||
in
|
||||
pkgs.stdenvNoCC.mkDerivation {
|
||||
pname = "cloakbrowser-chromium";
|
||||
version = chromiumVersion;
|
||||
|
||||
src = pkgs.fetchurl {
|
||||
url = "https://cloakbrowser.dev/chromium-v${chromiumVersion}/${archiveName}";
|
||||
inherit (info) hash;
|
||||
};
|
||||
|
||||
dontUnpack = true;
|
||||
|
||||
nativeBuildInputs = with pkgs; [
|
||||
autoPatchelfHook
|
||||
makeWrapper
|
||||
];
|
||||
|
||||
buildInputs = libs ++ desktopDeps;
|
||||
runtimeDependencies = libs;
|
||||
|
||||
installPhase = ''
|
||||
runHook preInstall
|
||||
|
||||
mkdir -p "$out/lib/cloakbrowser" "$out/bin"
|
||||
tar -xzf "$src" -C "$out/lib/cloakbrowser"
|
||||
chmod +x "$out/lib/cloakbrowser/chrome"
|
||||
chmod +x "$out/lib/cloakbrowser/chromedriver"
|
||||
|
||||
runHook postInstall
|
||||
'';
|
||||
|
||||
postFixup = ''
|
||||
makeWrapper "$out/lib/cloakbrowser/chrome" "$out/bin/cloakbrowser-chrome" \
|
||||
--prefix LD_LIBRARY_PATH : "${lib.makeLibraryPath libs}" \
|
||||
--prefix XDG_DATA_DIRS : "$GSETTINGS_SCHEMAS_PATH:$XDG_ICON_DIRS" \
|
||||
--suffix PATH : "${lib.makeBinPath [ pkgs.xdg-utils ]}" \
|
||||
--set FONTCONFIG_FILE "${fontsConf}" \
|
||||
--set CHROME_WRAPPER "cloakbrowser-chrome"
|
||||
|
||||
makeWrapper "$out/lib/cloakbrowser/chromedriver" "$out/bin/cloakbrowser-chromedriver" \
|
||||
--prefix LD_LIBRARY_PATH : "${lib.makeLibraryPath libs}"
|
||||
'';
|
||||
|
||||
meta = {
|
||||
description = "Official CloakBrowser patched Chromium binary";
|
||||
homepage = "https://github.com/CloakHQ/CloakBrowser";
|
||||
license = cloakbrowserBinaryLicense;
|
||||
mainProgram = "cloakbrowser-chrome";
|
||||
platforms = supportedSystems;
|
||||
sourceProvenance = [ lib.sourceTypes.binaryNativeCode ];
|
||||
};
|
||||
};
|
||||
in
|
||||
{
|
||||
packages = forAllSystems (system:
|
||||
let
|
||||
pkgs = mkPkgs system;
|
||||
cloakbrowserChromium = mkCloakBrowserChromium pkgs system;
|
||||
in
|
||||
{
|
||||
inherit cloakbrowserChromium;
|
||||
default = cloakbrowserChromium;
|
||||
});
|
||||
|
||||
apps = forAllSystems (system:
|
||||
let
|
||||
cloakbrowserChromium = self.packages.${system}.cloakbrowserChromium;
|
||||
in
|
||||
{
|
||||
default = {
|
||||
type = "app";
|
||||
program = "${cloakbrowserChromium}/bin/cloakbrowser-chrome";
|
||||
meta.description = "Run CloakBrowser Chromium";
|
||||
};
|
||||
cloakbrowser-chrome = {
|
||||
type = "app";
|
||||
program = "${cloakbrowserChromium}/bin/cloakbrowser-chrome";
|
||||
meta.description = "Run CloakBrowser Chromium";
|
||||
};
|
||||
cloakbrowser-chromedriver = {
|
||||
type = "app";
|
||||
program = "${cloakbrowserChromium}/bin/cloakbrowser-chromedriver";
|
||||
meta.description = "Run the CloakBrowser Chromedriver binary";
|
||||
};
|
||||
});
|
||||
|
||||
devShells = forAllSystems (system:
|
||||
let
|
||||
pkgs = mkPkgs system;
|
||||
cloakbrowserChromium = self.packages.${system}.cloakbrowserChromium;
|
||||
python = pkgs.python312.withPackages (ps: with ps; [
|
||||
aiohttp
|
||||
geoip2
|
||||
hatchling
|
||||
httpx
|
||||
playwright
|
||||
pytest
|
||||
pytest-asyncio
|
||||
socksio
|
||||
websockets
|
||||
]);
|
||||
in
|
||||
{
|
||||
default = pkgs.mkShell {
|
||||
packages = [
|
||||
cloakbrowserChromium
|
||||
python
|
||||
pkgs.cacert
|
||||
pkgs.curl
|
||||
pkgs.git
|
||||
pkgs.jq
|
||||
pkgs.nodejs_20
|
||||
pkgs.which
|
||||
pkgs.xdotool
|
||||
pkgs.xvfb-run
|
||||
]
|
||||
++ runtimeLibraries pkgs
|
||||
++ fontPackages pkgs;
|
||||
|
||||
CLOAKBROWSER_BINARY_PATH = "${cloakbrowserChromium}/bin/cloakbrowser-chrome";
|
||||
};
|
||||
});
|
||||
};
|
||||
}
|
||||
+8
-1
@@ -215,7 +215,7 @@ const page = await browser.newPage();
|
||||
## Requirements
|
||||
|
||||
- Node.js >= 20
|
||||
- One of: `playwright-core` >= 1.40 or `puppeteer-core` >= 21
|
||||
- One of: `playwright-core` >= 1.53 or `puppeteer-core` >= 21
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
@@ -230,6 +230,13 @@ const ctx = await launchPersistentContext({
|
||||
userDataDir: './my-profile',
|
||||
headless: false,
|
||||
});
|
||||
|
||||
// Load Chrome extensions
|
||||
const ctx = await launchPersistentContext({
|
||||
userDataDir: './my-profile',
|
||||
headless: false,
|
||||
extensionPaths: ['./my-extension'],
|
||||
});
|
||||
```
|
||||
|
||||
This also gives you cookie and localStorage persistence across sessions.
|
||||
|
||||
Generated
+17
-17
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "cloakbrowser",
|
||||
"version": "0.3.23",
|
||||
"version": "0.3.29",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "cloakbrowser",
|
||||
"version": "0.3.23",
|
||||
"version": "0.3.29",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"tar": "^7.0.0"
|
||||
@@ -17,7 +17,7 @@
|
||||
"devDependencies": {
|
||||
"@types/node": "^20.10.0",
|
||||
"mmdb-lib": "^3.0.2",
|
||||
"playwright-core": "^1.40.0",
|
||||
"playwright-core": "^1.53.0",
|
||||
"puppeteer-core": "^21.0.0",
|
||||
"socks-proxy-agent": "^10.0.0",
|
||||
"typescript": "^5.3.0",
|
||||
@@ -28,9 +28,9 @@
|
||||
},
|
||||
"peerDependencies": {
|
||||
"mmdb-lib": ">=2.0.0",
|
||||
"playwright-core": ">=1.40.0",
|
||||
"playwright-core": ">=1.53.0",
|
||||
"puppeteer-core": ">=21.0.0",
|
||||
"socks-proxy-agent": ">=8.0.0"
|
||||
"socks-proxy-agent": ">=10.0.0"
|
||||
},
|
||||
"peerDependenciesMeta": {
|
||||
"mmdb-lib": {
|
||||
@@ -1092,9 +1092,9 @@
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/basic-ftp": {
|
||||
"version": "5.2.0",
|
||||
"resolved": "https://registry.npmjs.org/basic-ftp/-/basic-ftp-5.2.0.tgz",
|
||||
"integrity": "sha512-VoMINM2rqJwJgfdHq6RiUudKt2BV+FY5ZFezP/ypmwayk68+NzzAQy4XXLlqsGD4MCzq3DrmNFD/uUmBJuGoXw==",
|
||||
"version": "5.3.1",
|
||||
"resolved": "https://registry.npmjs.org/basic-ftp/-/basic-ftp-5.3.1.tgz",
|
||||
"integrity": "sha512-bopVNp6ugyA150DDuZfPFdt1KZ5a94ZDiwX4hMgZDzF+GttD80lEy8kj98kbyhLXnPvhtIo93mdnLIjpCAeeOw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
@@ -1684,9 +1684,9 @@
|
||||
"license": "BSD-3-Clause"
|
||||
},
|
||||
"node_modules/ip-address": {
|
||||
"version": "10.1.0",
|
||||
"resolved": "https://registry.npmjs.org/ip-address/-/ip-address-10.1.0.tgz",
|
||||
"integrity": "sha512-XXADHxXmvT9+CRxhXg56LJovE+bmWnEWB78LB83VZTprKTmaC5QfruXocxzTZ2Kl0DNwKuBdlIhjL8LeY8Sf8Q==",
|
||||
"version": "10.2.0",
|
||||
"resolved": "https://registry.npmjs.org/ip-address/-/ip-address-10.2.0.tgz",
|
||||
"integrity": "sha512-/+S6j4E9AHvW9SWMSEY9Xfy66O5PWvVEJ08O0y5JGyEKQpojb0K0GKpz/v5HJ/G0vi3D2sjGK78119oXZeE0qA==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
@@ -2114,9 +2114,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/postcss": {
|
||||
"version": "8.5.6",
|
||||
"resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.6.tgz",
|
||||
"integrity": "sha512-3Ybi1tAuwAP9s0r1UQ2J4n5Y0G05bJkpUIO0/bI9MhwmD70S5aTWbXGBwxHrelT+XM1k6dM0pk+SwNkpTRN7Pg==",
|
||||
"version": "8.5.14",
|
||||
"resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.14.tgz",
|
||||
"integrity": "sha512-SoSL4+OSEtR99LHFZQiJLkT59C5B1amGO1NzTwj7TT1qCUgUO6hxOvzkOYxD+vMrXBM3XJIKzokoERdqQq/Zmg==",
|
||||
"dev": true,
|
||||
"funding": [
|
||||
{
|
||||
@@ -2496,9 +2496,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/tar": {
|
||||
"version": "7.5.9",
|
||||
"resolved": "https://registry.npmjs.org/tar/-/tar-7.5.9.tgz",
|
||||
"integrity": "sha512-BTLcK0xsDh2+PUe9F6c2TlRp4zOOBMTkoQHQIWSIzI0R7KG46uEwq4OPk2W7bZcprBMsuaeFsqwYr7pjh6CuHg==",
|
||||
"version": "7.5.15",
|
||||
"resolved": "https://registry.npmjs.org/tar/-/tar-7.5.15.tgz",
|
||||
"integrity": "sha512-dzGK0boVlC4W5QFuQN1EFSl3bIDYsk7Tj40U6eIBnK2k/8ml7TZ5agbI5j5+qnoVcAA+rNtBml8SEiLxZpNqRQ==",
|
||||
"license": "BlueOak-1.0.0",
|
||||
"dependencies": {
|
||||
"@isaacs/fs-minipass": "^4.0.0",
|
||||
|
||||
+3
-3
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "cloakbrowser",
|
||||
"version": "0.3.26",
|
||||
"version": "0.3.29",
|
||||
"description": "Stealth Chromium that passes every bot detection test. Drop-in Playwright/Puppeteer replacement with source-level fingerprint patches.",
|
||||
"type": "module",
|
||||
"main": "dist/index.js",
|
||||
@@ -59,7 +59,7 @@
|
||||
},
|
||||
"peerDependencies": {
|
||||
"mmdb-lib": ">=2.0.0",
|
||||
"playwright-core": ">=1.40.0",
|
||||
"playwright-core": ">=1.53.0",
|
||||
"puppeteer-core": ">=21.0.0",
|
||||
"socks-proxy-agent": ">=10.0.0"
|
||||
},
|
||||
@@ -84,7 +84,7 @@
|
||||
"@types/node": "^20.10.0",
|
||||
"mmdb-lib": "^3.0.2",
|
||||
"socks-proxy-agent": "^10.0.0",
|
||||
"playwright-core": "^1.40.0",
|
||||
"playwright-core": "^1.53.0",
|
||||
"puppeteer-core": "^21.0.0",
|
||||
"typescript": "^5.3.0",
|
||||
"vitest": "^1.0.0"
|
||||
|
||||
+12
-1
@@ -1,7 +1,7 @@
|
||||
/**
|
||||
* Shared argument builder for Playwright and Puppeteer wrappers.
|
||||
*/
|
||||
|
||||
import path from "path";
|
||||
import type { LaunchOptions } from "./types.js";
|
||||
import { getDefaultStealthArgs } from "./config.js";
|
||||
|
||||
@@ -55,5 +55,16 @@ export function buildArgs(options: LaunchOptions): string[] {
|
||||
seen.set(k, flag);
|
||||
}
|
||||
}
|
||||
|
||||
if (options.extensionPaths?.length) {
|
||||
const absPaths = options.extensionPaths.map(p => path.resolve(p));
|
||||
const joined = absPaths.join(",");
|
||||
|
||||
seen.set("--load-extension", `--load-extension=${joined}`);
|
||||
seen.set(
|
||||
"--disable-extensions-except",
|
||||
`--disable-extensions-except=${joined}`
|
||||
);
|
||||
}
|
||||
return [...seen.values()];
|
||||
}
|
||||
|
||||
+53
-10
@@ -22,6 +22,7 @@ const GEOIP_DB_URL =
|
||||
"https://github.com/P3TERX/GeoLite.mmdb/raw/download/GeoLite2-City.mmdb";
|
||||
const GEOIP_DB_FILENAME = "GeoLite2-City.mmdb";
|
||||
const GEOIP_UPDATE_INTERVAL_MS = 30 * 86_400_000; // 30 days
|
||||
const DEFAULT_GEOIP_TIMEOUT_MS = 5_000;
|
||||
|
||||
/** Country ISO code → BCP 47 locale (covers ~90% of proxy traffic). */
|
||||
export const COUNTRY_LOCALE_MAP: Record<string, string> = {
|
||||
@@ -68,10 +69,18 @@ export async function resolveProxyGeo(
|
||||
const dbPath = await ensureGeoipDb();
|
||||
if (!dbPath) return { timezone: null, locale: null, exitIp: null };
|
||||
|
||||
const timeoutMs = getGeoipTimeoutMs();
|
||||
const deadline = deadlineFromTimeout(timeoutMs);
|
||||
|
||||
// Exit IP (through proxy) is most accurate — gateway DNS may differ from exit
|
||||
let ip = await resolveExitIp(proxyUrl);
|
||||
if (!ip) ip = await resolveProxyIp(proxyUrl);
|
||||
if (!ip) return { timezone: null, locale: null, exitIp: null };
|
||||
let ip = await resolveExitIp(proxyUrl, remainingMs(deadline));
|
||||
if (!ip && !deadlineExpired(deadline)) ip = await resolveProxyIp(proxyUrl);
|
||||
if (!ip || deadlineExpired(deadline)) {
|
||||
if (deadlineExpired(deadline)) {
|
||||
console.warn(`[cloakbrowser] GeoIP resolution timed out after ${timeoutMs}ms; continuing without GeoIP`);
|
||||
}
|
||||
return { timezone: null, locale: null, exitIp: null };
|
||||
}
|
||||
|
||||
try {
|
||||
const buf = fs.readFileSync(dbPath);
|
||||
@@ -87,6 +96,30 @@ export async function resolveProxyGeo(
|
||||
}
|
||||
}
|
||||
|
||||
function getGeoipTimeoutMs(): number {
|
||||
const raw = process.env.CLOAKBROWSER_GEOIP_TIMEOUT_SECONDS;
|
||||
if (!raw) return DEFAULT_GEOIP_TIMEOUT_MS;
|
||||
const timeoutSeconds = Number(raw);
|
||||
if (!Number.isFinite(timeoutSeconds)) {
|
||||
console.warn(`[cloakbrowser] Invalid CLOAKBROWSER_GEOIP_TIMEOUT_SECONDS=${raw}; using ${DEFAULT_GEOIP_TIMEOUT_MS / 1000}s`);
|
||||
return DEFAULT_GEOIP_TIMEOUT_MS;
|
||||
}
|
||||
return Math.max(timeoutSeconds, 0) * 1000;
|
||||
}
|
||||
|
||||
function deadlineFromTimeout(timeoutMs: number): number | null {
|
||||
return timeoutMs > 0 ? performance.now() + timeoutMs : null;
|
||||
}
|
||||
|
||||
function remainingMs(deadline: number | null): number | undefined {
|
||||
if (deadline === null) return undefined;
|
||||
return Math.max(deadline - performance.now(), 0);
|
||||
}
|
||||
|
||||
function deadlineExpired(deadline: number | null): boolean {
|
||||
return deadline !== null && performance.now() >= deadline;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Proxy IP resolution
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -128,7 +161,8 @@ const IP_ECHO_URLS = [
|
||||
"https://ifconfig.me/ip",
|
||||
];
|
||||
|
||||
async function resolveExitIp(proxyUrl: string): Promise<string | null> {
|
||||
async function resolveExitIp(proxyUrl: string, timeoutMs?: number): Promise<string | null> {
|
||||
const deadline = timeoutMs && timeoutMs > 0 ? performance.now() + timeoutMs : null;
|
||||
const isSocks = isSocksProxy(proxyUrl);
|
||||
|
||||
// SOCKS5: tunnel through the SOCKS5 proxy via socks-proxy-agent
|
||||
@@ -144,9 +178,11 @@ async function resolveExitIp(proxyUrl: string): Promise<string | null> {
|
||||
const agent = new SocksProxyAgent(proxyUrl);
|
||||
|
||||
for (const echoUrl of IP_ECHO_URLS) {
|
||||
const remaining = remainingMs(deadline);
|
||||
if (remaining !== undefined && remaining <= 0) return null;
|
||||
try {
|
||||
const ip = await new Promise<string | null>((resolve) => {
|
||||
const req = https.request(echoUrl, { agent, timeout: 10_000 }, (res) => {
|
||||
const req = https.request(echoUrl, { agent, timeout: Math.min(10_000, remaining ?? 10_000) }, (res) => {
|
||||
let data = "";
|
||||
res.on("data", (chunk: Buffer) => (data += chunk.toString()));
|
||||
res.on("end", () => {
|
||||
@@ -173,6 +209,8 @@ async function resolveExitIp(proxyUrl: string): Promise<string | null> {
|
||||
const proxyUrlObj = new URL(proxyUrl);
|
||||
|
||||
for (const echoUrl of IP_ECHO_URLS) {
|
||||
const remaining = remainingMs(deadline);
|
||||
if (remaining !== undefined && remaining <= 0) return null;
|
||||
try {
|
||||
const ip = await new Promise<string | null>((resolve, reject) => {
|
||||
const targetUrl = new URL(echoUrl);
|
||||
@@ -190,13 +228,14 @@ async function resolveExitIp(proxyUrl: string): Promise<string | null> {
|
||||
).toString("base64"),
|
||||
}
|
||||
: {},
|
||||
timeout: 10_000,
|
||||
timeout: Math.min(10_000, remaining ?? 10_000),
|
||||
});
|
||||
|
||||
connectReq.on("connect", (_res, socket) => {
|
||||
const innerRemaining = remainingMs(deadline);
|
||||
const req = https.request(
|
||||
echoUrl,
|
||||
{ socket, timeout: 5_000 } as any,
|
||||
{ socket, timeout: Math.min(5_000, innerRemaining ?? 5_000) } as any,
|
||||
(res) => {
|
||||
let data = "";
|
||||
res.on("data", (chunk: Buffer) => (data += chunk.toString()));
|
||||
@@ -207,6 +246,7 @@ async function resolveExitIp(proxyUrl: string): Promise<string | null> {
|
||||
}
|
||||
);
|
||||
req.on("error", () => resolve(null));
|
||||
req.on("timeout", () => { req.destroy(); resolve(null); });
|
||||
req.end();
|
||||
});
|
||||
|
||||
@@ -261,7 +301,9 @@ async function downloadGeoipDb(dest: string): Promise<void> {
|
||||
|
||||
const tmpPath = `${dest}.tmp.${Date.now()}`;
|
||||
try {
|
||||
const response = await fetch(GEOIP_DB_URL, { redirect: "follow" });
|
||||
const response = await fetch(GEOIP_DB_URL, {
|
||||
redirect: "follow",
|
||||
});
|
||||
if (!response.ok || !response.body) {
|
||||
throw new Error(`HTTP ${response.status}`);
|
||||
}
|
||||
@@ -329,7 +371,8 @@ export async function maybeResolveGeoip(
|
||||
|
||||
// When both tz/locale are explicit, still resolve exit IP for WebRTC
|
||||
if (options.timezone && options.locale) {
|
||||
const exitIp = await resolveExitIp(proxyUrl) ?? undefined;
|
||||
const timeoutMs = getGeoipTimeoutMs();
|
||||
const exitIp = await resolveExitIp(proxyUrl, timeoutMs) ?? undefined;
|
||||
return { timezone: options.timezone, locale: options.locale, exitIp };
|
||||
}
|
||||
|
||||
@@ -363,7 +406,7 @@ export async function resolveWebrtcArgs(
|
||||
}
|
||||
|
||||
try {
|
||||
const ip = await resolveExitIp(proxyUrl);
|
||||
const ip = await resolveExitIp(proxyUrl, getGeoipTimeoutMs());
|
||||
const result = [...args];
|
||||
if (ip) {
|
||||
result[idx] = `--fingerprint-webrtc-ip=${ip}`;
|
||||
|
||||
+141
-55
@@ -47,17 +47,17 @@
|
||||
*/
|
||||
|
||||
import type { Browser, Page, Frame, CDPSession, ElementHandle, BrowserContext } from 'puppeteer-core';
|
||||
import type { HumanConfig } from '../human/config.js';
|
||||
import { resolveConfig, rand, randRange, sleep } from '../human/config.js';
|
||||
import type { HumanConfig, HumanActionOptions } from '../human/config.js';
|
||||
import { resolveConfig, mergeConfig, rand, randRange, sleep } from '../human/config.js';
|
||||
import { RawMouse, RawKeyboard, humanMove, humanClick, clickTarget, humanIdle } from '../human/mouse.js';
|
||||
import { humanType } from './keyboard.js';
|
||||
import { scrollToElement, smoothWheel } from './scroll.js';
|
||||
import { scrollToElement, humanScrollIntoView, smoothWheel } from './scroll.js';
|
||||
|
||||
export type { HumanConfig } from '../human/config.js';
|
||||
export { resolveConfig } from '../human/config.js';
|
||||
export { resolveConfig, mergeConfig } from '../human/config.js';
|
||||
export { humanMove, humanClick, clickTarget, humanIdle } from '../human/mouse.js';
|
||||
export { humanType } from './keyboard.js';
|
||||
export { scrollToElement } from './scroll.js';
|
||||
export { scrollToElement, humanScrollIntoView } from './scroll.js';
|
||||
|
||||
|
||||
// ============================================================================
|
||||
@@ -319,7 +319,11 @@ function patchPage(page: Page, cfg: HumanConfig, cursor: CursorState): void {
|
||||
}
|
||||
|
||||
// ==== goto ====
|
||||
const humanGoto = async (url: string, options?: any) => {
|
||||
const humanGoto = async (url: string, options?: {
|
||||
referer?: string;
|
||||
timeout?: number;
|
||||
waitUntil?: 'load' | 'domcontentloaded' | 'networkidle0' | 'networkidle2';
|
||||
}) => {
|
||||
const response = await originals.goto(url, options);
|
||||
stealth.invalidate();
|
||||
patchFrames(page, cfg, cursor, raw, rawKb, originals, stealth);
|
||||
@@ -327,54 +331,64 @@ function patchPage(page: Page, cfg: HumanConfig, cursor: CursorState): void {
|
||||
};
|
||||
|
||||
// ==== click (with clickCount support for dblclick) ====
|
||||
const humanClickFn = async (selector: string, options?: any) => {
|
||||
const humanClickFn = async (selector: string, options?: HumanActionOptions & {
|
||||
button?: 'left' | 'right' | 'middle' | 'back' | 'forward';
|
||||
clickCount?: number;
|
||||
count?: number;
|
||||
delay?: number;
|
||||
}) => {
|
||||
await ensureCursorInit();
|
||||
if (cfg.idle_between_actions) {
|
||||
await humanIdle(raw, rand(cfg.idle_between_duration[0], cfg.idle_between_duration[1]), cursor.x, cursor.y, cfg);
|
||||
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
|
||||
if (callCfg.idle_between_actions) {
|
||||
await humanIdle(raw, cursor.x, cursor.y, callCfg);
|
||||
}
|
||||
const { box, cursorX, cursorY } = await scrollToElement(page, raw, selector, cursor.x, cursor.y, cfg);
|
||||
const { box, cursorX, cursorY } = await scrollToElement(page, raw, selector, cursor.x, cursor.y, callCfg, options?.timeout);
|
||||
cursor.x = cursorX;
|
||||
cursor.y = cursorY;
|
||||
const isInput = await isInputElement(stealth, page, selector);
|
||||
const target = clickTarget(box, isInput, cfg);
|
||||
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, cfg);
|
||||
const target = clickTarget(box, isInput, callCfg);
|
||||
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, callCfg);
|
||||
cursor.x = target.x;
|
||||
cursor.y = target.y;
|
||||
|
||||
const clickCount = options?.clickCount ?? options?.count ?? 1;
|
||||
if (clickCount >= 2) {
|
||||
await humanClick(raw, isInput, cfg);
|
||||
await humanClick(raw, isInput, callCfg);
|
||||
await sleep(rand(40, 90));
|
||||
await raw.down({ clickCount: 2 });
|
||||
await sleep(rand(30, 60));
|
||||
await raw.up({ clickCount: 2 });
|
||||
} else {
|
||||
await humanClick(raw, isInput, cfg);
|
||||
await humanClick(raw, isInput, callCfg);
|
||||
}
|
||||
};
|
||||
|
||||
// ==== hover ====
|
||||
const humanHoverFn = async (selector: string, options?: any) => {
|
||||
const humanHoverFn = async (selector: string, options?: HumanActionOptions) => {
|
||||
await ensureCursorInit();
|
||||
if (cfg.idle_between_actions) {
|
||||
await humanIdle(raw, rand(cfg.idle_between_duration[0], cfg.idle_between_duration[1]), cursor.x, cursor.y, cfg);
|
||||
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
|
||||
if (callCfg.idle_between_actions) {
|
||||
await humanIdle(raw, cursor.x, cursor.y, callCfg);
|
||||
}
|
||||
const { box, cursorX, cursorY } = await scrollToElement(page, raw, selector, cursor.x, cursor.y, cfg);
|
||||
const { box, cursorX, cursorY } = await scrollToElement(page, raw, selector, cursor.x, cursor.y, callCfg, options?.timeout);
|
||||
cursor.x = cursorX;
|
||||
cursor.y = cursorY;
|
||||
const target = clickTarget(box, false, cfg);
|
||||
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, cfg);
|
||||
const target = clickTarget(box, false, callCfg);
|
||||
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, callCfg);
|
||||
cursor.x = target.x;
|
||||
cursor.y = target.y;
|
||||
};
|
||||
|
||||
// ==== type ====
|
||||
const humanTypeFn = async (selector: string, text: string, options?: any) => {
|
||||
await sleep(randRange(cfg.field_switch_delay));
|
||||
await humanClickFn(selector);
|
||||
const humanTypeFn = async (selector: string, text: string, options?: HumanActionOptions & {
|
||||
delay?: number;
|
||||
}) => {
|
||||
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
|
||||
await sleep(randRange(callCfg.field_switch_delay));
|
||||
await humanClickFn(selector, options);
|
||||
await sleep(rand(100, 250));
|
||||
const cdp = await ensureCdp();
|
||||
await humanType(page, rawKb, text, cfg, cdp);
|
||||
await humanType(page, rawKb, text, callCfg, cdp);
|
||||
};
|
||||
|
||||
// ==== select ====
|
||||
@@ -392,7 +406,7 @@ function patchPage(page: Page, cfg: HumanConfig, cursor: CursorState): void {
|
||||
};
|
||||
|
||||
// ==== tap ====
|
||||
const humanTapFn = async (selector: string, options?: any) => {
|
||||
const humanTapFn = async (selector: string, options?: HumanActionOptions) => {
|
||||
await humanClickFn(selector, options);
|
||||
};
|
||||
|
||||
@@ -410,14 +424,19 @@ function patchPage(page: Page, cfg: HumanConfig, cursor: CursorState): void {
|
||||
// ============================================================
|
||||
// Mouse patches
|
||||
// ============================================================
|
||||
page.mouse.move = async (x: number, y: number, options?: any) => {
|
||||
page.mouse.move = async (x: number, y: number, options?: { steps?: number }) => {
|
||||
await ensureCursorInit();
|
||||
await humanMove(raw, cursor.x, cursor.y, x, y, cfg);
|
||||
cursor.x = x;
|
||||
cursor.y = y;
|
||||
};
|
||||
|
||||
page.mouse.click = async (x: number, y: number, options?: any) => {
|
||||
page.mouse.click = async (x: number, y: number, options?: {
|
||||
button?: 'left' | 'right' | 'middle' | 'back' | 'forward';
|
||||
clickCount?: number;
|
||||
count?: number;
|
||||
delay?: number;
|
||||
}) => {
|
||||
await ensureCursorInit();
|
||||
await humanMove(raw, cursor.x, cursor.y, x, y, cfg);
|
||||
cursor.x = x;
|
||||
@@ -452,7 +471,7 @@ function patchPage(page: Page, cfg: HumanConfig, cursor: CursorState): void {
|
||||
(page.mouse as any).dragAndDrop = async (
|
||||
start: { x: number; y: number },
|
||||
target: { x: number; y: number },
|
||||
options?: any,
|
||||
options?: { delay?: number },
|
||||
) => {
|
||||
await ensureCursorInit();
|
||||
await humanMove(raw, cursor.x, cursor.y, start.x, start.y, cfg);
|
||||
@@ -472,12 +491,12 @@ function patchPage(page: Page, cfg: HumanConfig, cursor: CursorState): void {
|
||||
// ============================================================
|
||||
// Keyboard patches
|
||||
// ============================================================
|
||||
page.keyboard.type = async (text: string, options?: any) => {
|
||||
page.keyboard.type = async (text: string, options?: { delay?: number }) => {
|
||||
const cdp = await ensureCdp();
|
||||
await humanType(page, rawKb, text, cfg, cdp);
|
||||
};
|
||||
|
||||
page.keyboard.press = async (key: any, options?: any) => {
|
||||
page.keyboard.press = async (key: any, options?: { delay?: number }) => {
|
||||
await sleep(rand(20, 60));
|
||||
await originals.keyboardDown(key as any);
|
||||
await sleep(randRange(cfg.key_hold));
|
||||
@@ -548,7 +567,11 @@ function patchElementHandle(
|
||||
return els;
|
||||
};
|
||||
|
||||
(page as any).waitForSelector = async (selector: string, options?: any) => {
|
||||
(page as any).waitForSelector = async (selector: string, options?: {
|
||||
hidden?: boolean;
|
||||
timeout?: number;
|
||||
visible?: boolean;
|
||||
}) => {
|
||||
const el = await origWaitForSelector(selector, options);
|
||||
if (el) patchSingleElementHandle(el, page, cfg, cursor, raw, rawKb, originals, stealth);
|
||||
return el;
|
||||
@@ -577,6 +600,9 @@ function patchSingleElementHandle(
|
||||
const origElDragAndDrop = (el as any).dragAndDrop?.bind(el);
|
||||
const origElSelect = (el as any).select?.bind(el);
|
||||
const origElDrop = (el as any).drop?.bind(el);
|
||||
// Puppeteer v22+ adds ElementHandle.scrollIntoView(); earlier versions
|
||||
// expose it implicitly via evaluate(node => node.scrollIntoView()).
|
||||
const origElScrollIntoView = (el as any).scrollIntoView?.bind(el);
|
||||
|
||||
// --- Nested selectors ---
|
||||
const origEl$ = el.$.bind(el);
|
||||
@@ -597,45 +623,69 @@ function patchSingleElementHandle(
|
||||
return children;
|
||||
};
|
||||
|
||||
(el as any).waitForSelector = async (selector: string, options?: any) => {
|
||||
(el as any).waitForSelector = async (selector: string, options?: {
|
||||
hidden?: boolean;
|
||||
timeout?: number;
|
||||
visible?: boolean;
|
||||
}) => {
|
||||
const child = await origElWaitForSelector(selector, options);
|
||||
if (child) patchSingleElementHandle(child, page, cfg, cursor, raw, rawKb, originals, stealth);
|
||||
return child;
|
||||
};
|
||||
|
||||
// --- Helper: get box and move cursor ---
|
||||
const moveToElement = async () => {
|
||||
// --- Helper: get box and move cursor. Accepts a per-call ``callCfg``
|
||||
// so type/fill overrides like ``el.type(text, { typing_delay: 30 })``
|
||||
// carry through to mouse timing for that single call. Also scrolls into
|
||||
// view first so off-screen elements work (#129, #172 follow-up).
|
||||
const moveToElement = async (callCfg: HumanConfig = cfg) => {
|
||||
await (page as any)._ensureCursorInit();
|
||||
|
||||
try {
|
||||
const { cursorX, cursorY } = await humanScrollIntoView(
|
||||
page, raw,
|
||||
() => el.boundingBox().then(b => b ?? null),
|
||||
cursor.x, cursor.y, callCfg,
|
||||
);
|
||||
cursor.x = cursorX;
|
||||
cursor.y = cursorY;
|
||||
} catch { /* let boundingBox() decide */ }
|
||||
|
||||
const box = await el.boundingBox();
|
||||
if (!box) return null;
|
||||
|
||||
const isInp = await isInputElementHandle(stealth, el);
|
||||
const target = clickTarget(box, isInp, cfg);
|
||||
const target = clickTarget(box, isInp, callCfg);
|
||||
|
||||
if (cfg.idle_between_actions) {
|
||||
await humanIdle(raw, rand(cfg.idle_between_duration[0], cfg.idle_between_duration[1]), cursor.x, cursor.y, cfg);
|
||||
if (callCfg.idle_between_actions) {
|
||||
await humanIdle(raw, cursor.x, cursor.y, callCfg);
|
||||
}
|
||||
|
||||
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, cfg);
|
||||
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, callCfg);
|
||||
cursor.x = target.x;
|
||||
cursor.y = target.y;
|
||||
return { box, isInp };
|
||||
};
|
||||
|
||||
// --- el.click() ---
|
||||
(el as any).click = async (options?: any) => {
|
||||
const info = await moveToElement();
|
||||
(el as any).click = async (options?: HumanActionOptions & {
|
||||
button?: 'left' | 'right' | 'middle' | 'back' | 'forward';
|
||||
clickCount?: number;
|
||||
count?: number;
|
||||
delay?: number;
|
||||
}) => {
|
||||
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
|
||||
const info = await moveToElement(callCfg);
|
||||
if (!info) return origElClick(options);
|
||||
|
||||
const clickCount = options?.clickCount ?? options?.count ?? 1;
|
||||
if (clickCount >= 2) {
|
||||
await humanClick(raw, info.isInp, cfg);
|
||||
await humanClick(raw, info.isInp, callCfg);
|
||||
await sleep(rand(40, 90));
|
||||
await raw.down({ clickCount: 2 });
|
||||
await sleep(rand(30, 60));
|
||||
await raw.up({ clickCount: 2 });
|
||||
} else {
|
||||
await humanClick(raw, info.isInp, cfg);
|
||||
await humanClick(raw, info.isInp, callCfg);
|
||||
}
|
||||
};
|
||||
|
||||
@@ -646,18 +696,43 @@ function patchSingleElementHandle(
|
||||
};
|
||||
|
||||
// --- el.type() ---
|
||||
(el as any).type = async (text: string, options?: any) => {
|
||||
const info = await moveToElement();
|
||||
(el as any).type = async (text: string, options?: HumanActionOptions & { delay?: number }) => {
|
||||
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
|
||||
const info = await moveToElement(callCfg);
|
||||
if (!info) return origElType(text, options);
|
||||
await humanClick(raw, info.isInp, cfg);
|
||||
await humanClick(raw, info.isInp, callCfg);
|
||||
await sleep(rand(100, 250));
|
||||
const cdp = await stealth.getCdpSession().catch(() => null);
|
||||
await humanType(page, rawKb, text, cfg, cdp);
|
||||
await humanType(page, rawKb, text, callCfg, cdp);
|
||||
};
|
||||
|
||||
// --- el.scrollIntoView() ---
|
||||
// Puppeteer-only equivalent of Playwright's scrollIntoViewIfNeeded.
|
||||
// Replaces the native snap-scroll (a strong bot signal) with the same
|
||||
// accelerate → cruise → decelerate → overshoot wheel sequence used by
|
||||
// page.click(). Only patched when the underlying ElementHandle exposes
|
||||
// ``scrollIntoView`` (Puppeteer v22+).
|
||||
if (origElScrollIntoView) {
|
||||
(el as any).scrollIntoView = async (options?: HumanActionOptions) => {
|
||||
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
|
||||
await (page as any)._ensureCursorInit();
|
||||
try {
|
||||
const { cursorX, cursorY } = await humanScrollIntoView(
|
||||
page, raw,
|
||||
() => el.boundingBox().then(b => b ?? null),
|
||||
cursor.x, cursor.y, callCfg,
|
||||
);
|
||||
cursor.x = cursorX;
|
||||
cursor.y = cursorY;
|
||||
} catch {
|
||||
return origElScrollIntoView(options);
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
// --- el.press() ---
|
||||
if (origElPress) {
|
||||
(el as any).press = async (key: string, options?: any) => {
|
||||
(el as any).press = async (key: string, options?: { delay?: number }) => {
|
||||
await sleep(rand(20, 60));
|
||||
await originals.keyboardDown(key as any);
|
||||
await sleep(randRange(cfg.key_hold));
|
||||
@@ -696,7 +771,7 @@ function patchSingleElementHandle(
|
||||
|
||||
// --- el.drop() ---
|
||||
if (origElDrop) {
|
||||
(el as any).drop = async (draggable: ElementHandle, options?: any) => {
|
||||
(el as any).drop = async (draggable: ElementHandle, options?: { delay?: number }) => {
|
||||
const srcBox = await draggable.boundingBox();
|
||||
const tgtBox = await el.boundingBox();
|
||||
|
||||
@@ -726,7 +801,7 @@ function patchSingleElementHandle(
|
||||
|
||||
// --- el.dragAndDrop() ---
|
||||
if (origElDragAndDrop) {
|
||||
(el as any).dragAndDrop = async (targetEl: ElementHandle, options?: any) => {
|
||||
(el as any).dragAndDrop = async (targetEl: ElementHandle, options?: { delay?: number }) => {
|
||||
const srcBox = await el.boundingBox();
|
||||
const tgtBox = await targetEl.boundingBox();
|
||||
|
||||
@@ -790,15 +865,22 @@ function patchSingleFrame(
|
||||
|
||||
const origFrameSelect = frame.select.bind(frame);
|
||||
|
||||
(frame as any).click = async (selector: string, options?: any) => {
|
||||
(frame as any).click = async (selector: string, options?: HumanActionOptions & {
|
||||
button?: 'left' | 'right' | 'middle' | 'back' | 'forward';
|
||||
clickCount?: number;
|
||||
count?: number;
|
||||
delay?: number;
|
||||
}) => {
|
||||
await (page as any).click(selector, options);
|
||||
};
|
||||
|
||||
(frame as any).hover = async (selector: string, options?: any) => {
|
||||
(frame as any).hover = async (selector: string, options?: HumanActionOptions) => {
|
||||
await (page as any).hover(selector, options);
|
||||
};
|
||||
|
||||
(frame as any).type = async (selector: string, text: string, options?: any) => {
|
||||
(frame as any).type = async (selector: string, text: string, options?: HumanActionOptions & {
|
||||
delay?: number;
|
||||
}) => {
|
||||
await (page as any).type(selector, text, options);
|
||||
};
|
||||
|
||||
@@ -812,7 +894,7 @@ function patchSingleFrame(
|
||||
await (page as any).focus(selector);
|
||||
};
|
||||
|
||||
(frame as any).tap = async (selector: string, options?: any) => {
|
||||
(frame as any).tap = async (selector: string, options?: HumanActionOptions) => {
|
||||
await (page as any).click(selector, options);
|
||||
};
|
||||
|
||||
@@ -835,7 +917,11 @@ function patchSingleFrame(
|
||||
return els;
|
||||
};
|
||||
|
||||
(frame as any).waitForSelector = async (selector: string, options?: any) => {
|
||||
(frame as any).waitForSelector = async (selector: string, options?: {
|
||||
hidden?: boolean;
|
||||
timeout?: number;
|
||||
visible?: boolean;
|
||||
}) => {
|
||||
const el = await origFrameWaitForSelector(selector, options);
|
||||
if (el) patchSingleElementHandle(el, page, cfg, cursor, raw, rawKb, originals, stealth);
|
||||
return el;
|
||||
@@ -881,7 +967,7 @@ export function patchBrowser(browser: Browser, cfg: HumanConfig): void {
|
||||
for (const methodName of ['createBrowserContext', 'createIncognitoBrowserContext'] as const) {
|
||||
if (typeof (browser as any)[methodName] === 'function') {
|
||||
const origCreateContext = (browser as any)[methodName].bind(browser);
|
||||
(browser as any)[methodName] = async (options?: any) => {
|
||||
(browser as any)[methodName] = async (options?: Parameters<typeof origCreateContext>[0]) => {
|
||||
const context: BrowserContext = await origCreateContext(options);
|
||||
|
||||
const origCtxNewPage = context.newPage.bind(context);
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
* Changes from Playwright version:
|
||||
* - page.viewport() instead of page.viewportSize()
|
||||
* - page.$(selector) + el.boundingBox() instead of page.locator().boundingBox()
|
||||
* - No timeout parameter on boundingBox()
|
||||
* - boundingBox() has no timeout param — we poll page.$() up to ``timeout`` ms
|
||||
*/
|
||||
|
||||
import type { Page } from 'puppeteer-core';
|
||||
@@ -55,22 +55,40 @@ export async function smoothWheel(
|
||||
}
|
||||
}
|
||||
|
||||
async function getElementBox(page: Page, selector: string): Promise<ElementBounds | null> {
|
||||
try {
|
||||
const el = await page.$(selector);
|
||||
if (!el) return null;
|
||||
const box = await el.boundingBox();
|
||||
if (!box) return null;
|
||||
return { x: box.x, y: box.y, width: box.width, height: box.height };
|
||||
} catch {
|
||||
return null;
|
||||
/**
|
||||
* Poll ``page.$(selector)`` for up to ``timeout`` ms, returning the element's
|
||||
* bounding box when found. ``timeout`` defaults to 30000ms when not specified.
|
||||
*/
|
||||
async function getElementBox(
|
||||
page: Page,
|
||||
selector: string,
|
||||
timeout: number = 30000,
|
||||
): Promise<ElementBounds | null> {
|
||||
const start = Date.now();
|
||||
const pollInterval = 100;
|
||||
while (true) {
|
||||
try {
|
||||
const el = await page.$(selector);
|
||||
if (el) {
|
||||
const box = await el.boundingBox();
|
||||
if (box) return { x: box.x, y: box.y, width: box.width, height: box.height };
|
||||
}
|
||||
} catch { /* keep polling */ }
|
||||
|
||||
if (Date.now() - start >= timeout) return null;
|
||||
await sleep(pollInterval);
|
||||
}
|
||||
}
|
||||
|
||||
export async function scrollToElement(
|
||||
/**
|
||||
* Humanized scrolling that takes an arbitrary ``getBox`` callable.
|
||||
* Used by both ``scrollToElement`` (selector-based) and the ElementHandle
|
||||
* ``scrollIntoView`` patch.
|
||||
*/
|
||||
export async function humanScrollIntoView(
|
||||
page: Page,
|
||||
raw: RawMouse,
|
||||
selector: string,
|
||||
getBox: () => Promise<ElementBounds | null>,
|
||||
cursorX: number,
|
||||
cursorY: number,
|
||||
cfg: HumanConfig,
|
||||
@@ -78,12 +96,8 @@ export async function scrollToElement(
|
||||
const viewport = page.viewport();
|
||||
if (!viewport) throw new Error('Viewport size not available');
|
||||
|
||||
let box = await getElementBox(page, selector);
|
||||
if (!box) {
|
||||
await sleep(200);
|
||||
box = await getElementBox(page, selector);
|
||||
if (!box) throw new Error(`Element not found: ${selector}`);
|
||||
}
|
||||
let box = await getBox();
|
||||
if (!box) throw new Error('Element not found while scrolling into view');
|
||||
|
||||
if (isInViewport(box, viewport.height, cfg)) {
|
||||
return { box, cursorX, cursorY };
|
||||
@@ -134,7 +148,7 @@ export async function scrollToElement(
|
||||
await sleep(pause);
|
||||
|
||||
if (i % 3 === 2 || i === totalClicks - 1) {
|
||||
box = await getElementBox(page, selector);
|
||||
box = await getBox();
|
||||
if (box && isInViewport(box, viewport.height, cfg)) {
|
||||
break;
|
||||
}
|
||||
@@ -159,8 +173,31 @@ export async function scrollToElement(
|
||||
|
||||
await sleep(randRange(cfg.scroll_settle_delay));
|
||||
|
||||
box = await getElementBox(page, selector);
|
||||
if (!box) throw new Error(`Element lost after scrolling: ${selector}`);
|
||||
box = await getBox();
|
||||
if (!box) throw new Error('Element lost after scrolling into view');
|
||||
|
||||
return { box, cursorX, cursorY };
|
||||
}
|
||||
|
||||
/**
|
||||
* Selector-based humanized scroll (Puppeteer).
|
||||
*
|
||||
* ``timeout`` controls how long we poll ``page.$(selector)`` before giving up,
|
||||
* so callers like ``page.click('#x', { timeout: 5000 })`` can wait longer for
|
||||
* slow-loading elements (#172). Default matches Playwright's 30000ms when not specified.
|
||||
*/
|
||||
export async function scrollToElement(
|
||||
page: Page,
|
||||
raw: RawMouse,
|
||||
selector: string,
|
||||
cursorX: number,
|
||||
cursorY: number,
|
||||
cfg: HumanConfig,
|
||||
timeout?: number,
|
||||
): Promise<{ box: ElementBounds; cursorX: number; cursorY: number }> {
|
||||
return humanScrollIntoView(
|
||||
page, raw,
|
||||
() => getElementBox(page, selector, timeout),
|
||||
cursorX, cursorY, cfg,
|
||||
);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,338 @@
|
||||
/**
|
||||
* Playwright-style actionability checks for the humanize layer.
|
||||
*
|
||||
* Checks: attached, visible, stable, enabled, editable, receives pointer events.
|
||||
* Retry loop with backoff matching Playwright internals: [100, 250, 500, 1000]ms.
|
||||
*/
|
||||
|
||||
import type { Page, Frame, ElementHandle } from 'playwright-core';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Error hierarchy
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export class ActionabilityError extends Error {
|
||||
selector: string;
|
||||
check: string;
|
||||
|
||||
constructor(selector: string, check: string, message: string) {
|
||||
super(`Element ${JSON.stringify(selector)} failed ${check} check: ${message}`);
|
||||
this.name = 'ActionabilityError';
|
||||
this.selector = selector;
|
||||
this.check = check;
|
||||
}
|
||||
}
|
||||
|
||||
export class ElementNotAttachedError extends ActionabilityError {
|
||||
constructor(selector: string) {
|
||||
super(selector, 'attached', 'element not found in DOM');
|
||||
this.name = 'ElementNotAttachedError';
|
||||
}
|
||||
}
|
||||
|
||||
export class ElementNotVisibleError extends ActionabilityError {
|
||||
constructor(selector: string) {
|
||||
super(selector, 'visible', 'element is not visible');
|
||||
this.name = 'ElementNotVisibleError';
|
||||
}
|
||||
}
|
||||
|
||||
export class ElementNotStableError extends ActionabilityError {
|
||||
constructor(selector: string) {
|
||||
super(selector, 'stable', 'element position is still changing');
|
||||
this.name = 'ElementNotStableError';
|
||||
}
|
||||
}
|
||||
|
||||
export class ElementNotEnabledError extends ActionabilityError {
|
||||
constructor(selector: string) {
|
||||
super(selector, 'enabled', 'element is disabled');
|
||||
this.name = 'ElementNotEnabledError';
|
||||
}
|
||||
}
|
||||
|
||||
export class ElementNotEditableError extends ActionabilityError {
|
||||
constructor(selector: string) {
|
||||
super(selector, 'editable', 'element is not editable');
|
||||
this.name = 'ElementNotEditableError';
|
||||
}
|
||||
}
|
||||
|
||||
export class ElementNotReceivingEventsError extends ActionabilityError {
|
||||
coveringTag: string;
|
||||
constructor(selector: string, coveringTag: string = 'unknown') {
|
||||
super(selector, 'pointer_events', `element is covered by <${coveringTag}>`);
|
||||
this.name = 'ElementNotReceivingEventsError';
|
||||
this.coveringTag = coveringTag;
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Check-set constants
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export type CheckName = 'attached' | 'visible' | 'enabled' | 'editable' | 'pointer_events';
|
||||
|
||||
export const CHECKS_CLICK: ReadonlySet<CheckName> = new Set(['attached', 'visible', 'enabled', 'pointer_events']);
|
||||
export const CHECKS_HOVER: ReadonlySet<CheckName> = new Set(['attached', 'visible', 'pointer_events']);
|
||||
export const CHECKS_INPUT: ReadonlySet<CheckName> = new Set(['attached', 'visible', 'enabled', 'editable', 'pointer_events']);
|
||||
export const CHECKS_FOCUS: ReadonlySet<CheckName> = new Set(['attached', 'visible', 'enabled']);
|
||||
export const CHECKS_CHECK: ReadonlySet<CheckName> = new Set(['attached', 'visible', 'enabled', 'pointer_events']);
|
||||
|
||||
const BACKOFF_MS = [100, 250, 500, 1000];
|
||||
|
||||
function backoffSleep(attempt: number): Promise<void> {
|
||||
const idx = Math.min(attempt, BACKOFF_MS.length - 1);
|
||||
return new Promise(resolve => setTimeout(resolve, BACKOFF_MS[idx]));
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Pre-scroll actionability
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export async function ensureActionable(
|
||||
pageOrFrame: Page | Frame,
|
||||
selector: string,
|
||||
checks: ReadonlySet<CheckName>,
|
||||
timeout: number = 30000,
|
||||
force: boolean = false,
|
||||
): Promise<void> {
|
||||
if (force) return;
|
||||
|
||||
const deadline = Date.now() + timeout;
|
||||
let attempt = 0;
|
||||
let lastError: ActionabilityError | null = null;
|
||||
|
||||
while (true) {
|
||||
const remainingMs = Math.max(0, deadline - Date.now());
|
||||
if (remainingMs <= 0) {
|
||||
if (lastError) throw lastError;
|
||||
throw new ActionabilityError(selector, 'timeout', 'timeout expired before first check');
|
||||
}
|
||||
|
||||
try {
|
||||
const loc = pageOrFrame.locator(selector).first();
|
||||
|
||||
if (checks.has('attached')) {
|
||||
try {
|
||||
await loc.waitFor({ state: 'attached', timeout: Math.max(1, Math.min(remainingMs, 2000)) });
|
||||
} catch {
|
||||
throw new ElementNotAttachedError(selector);
|
||||
}
|
||||
}
|
||||
|
||||
if (checks.has('visible')) {
|
||||
if (!await loc.isVisible()) throw new ElementNotVisibleError(selector);
|
||||
}
|
||||
|
||||
if (checks.has('enabled')) {
|
||||
if (!await loc.isEnabled()) throw new ElementNotEnabledError(selector);
|
||||
}
|
||||
|
||||
if (checks.has('editable')) {
|
||||
if (!await loc.isEditable()) throw new ElementNotEditableError(selector);
|
||||
}
|
||||
|
||||
return;
|
||||
} catch (e) {
|
||||
if (e instanceof ActionabilityError) {
|
||||
lastError = e;
|
||||
if (Date.now() >= deadline) throw lastError;
|
||||
await backoffSleep(attempt);
|
||||
attempt++;
|
||||
} else {
|
||||
throw e;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Post-scroll stability check
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function boxesDiffer(
|
||||
a: { x: number; y: number; width: number; height: number },
|
||||
b: { x: number; y: number; width: number; height: number },
|
||||
): boolean {
|
||||
return (
|
||||
Math.abs(a.x - b.x) > 1 ||
|
||||
Math.abs(a.y - b.y) > 1 ||
|
||||
Math.abs(a.width - b.width) > 1 ||
|
||||
Math.abs(a.height - b.height) > 1
|
||||
);
|
||||
}
|
||||
|
||||
export async function ensureStable(
|
||||
pageOrFrame: Page | Frame,
|
||||
selector: string,
|
||||
timeout: number = 5000,
|
||||
): Promise<void> {
|
||||
const deadline = Date.now() + timeout;
|
||||
let attempt = 0;
|
||||
|
||||
while (true) {
|
||||
const remainingMs = Math.max(0, deadline - Date.now());
|
||||
if (remainingMs <= 0) throw new ElementNotStableError(selector);
|
||||
|
||||
const loc = pageOrFrame.locator(selector).first();
|
||||
const box1 = await loc.boundingBox({ timeout: Math.max(1, Math.min(remainingMs, 1000)) });
|
||||
if (!box1) throw new ElementNotAttachedError(selector);
|
||||
|
||||
await new Promise(r => setTimeout(r, 100));
|
||||
|
||||
const box2 = await loc.boundingBox({ timeout: Math.max(1, Math.min(remainingMs, 1000)) });
|
||||
if (!box2) throw new ElementNotAttachedError(selector);
|
||||
|
||||
if (!boxesDiffer(box1, box2)) return;
|
||||
|
||||
if (Date.now() >= deadline) throw new ElementNotStableError(selector);
|
||||
|
||||
await backoffSleep(attempt);
|
||||
attempt++;
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Pointer-events check (post-scroll, at actual click coordinates)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const POINTER_EVENTS_LOCATOR_JS = `(expected, coords) => {
|
||||
const target = document.elementFromPoint(coords.x, coords.y);
|
||||
if (!target) return { hit: false, reason: 'no_element_at_point', covering: 'none' };
|
||||
let node = target;
|
||||
while (node) { if (node === expected) return { hit: true }; node = node.parentNode; }
|
||||
if (expected.contains(target)) return { hit: true };
|
||||
return { hit: false, reason: 'covered', covering: target.tagName || 'unknown' };
|
||||
}`;
|
||||
|
||||
const POINTER_EVENTS_HANDLE_JS = `(expected, coords) => {
|
||||
const target = document.elementFromPoint(coords.x, coords.y);
|
||||
if (!target) return { hit: false, reason: 'no_element_at_point', covering: 'none' };
|
||||
let node = target;
|
||||
while (node) { if (node === expected) return { hit: true }; node = node.parentNode; }
|
||||
if (expected.contains(target)) return { hit: true };
|
||||
return { hit: false, reason: 'covered', covering: target.tagName || 'unknown' };
|
||||
}`;
|
||||
|
||||
export async function checkPointerEvents(
|
||||
pageOrFrame: Page | Frame,
|
||||
selector: string,
|
||||
x: number,
|
||||
y: number,
|
||||
stealth?: { evaluate(expression: string): Promise<any> } | null,
|
||||
timeout: number = 5000,
|
||||
): Promise<void> {
|
||||
const deadline = Date.now() + timeout;
|
||||
let attempt = 0;
|
||||
const coords = { x, y };
|
||||
|
||||
while (true) {
|
||||
let result: any = null;
|
||||
try {
|
||||
const loc = pageOrFrame.locator(selector).first();
|
||||
result = await loc.evaluate(POINTER_EVENTS_LOCATOR_JS, coords);
|
||||
} catch {
|
||||
result = null;
|
||||
}
|
||||
|
||||
if (result && result.hit) return;
|
||||
const covering = (result as any)?.covering ?? 'unknown';
|
||||
if (Date.now() >= deadline) throw new ElementNotReceivingEventsError(selector, covering);
|
||||
|
||||
await backoffSleep(attempt);
|
||||
attempt++;
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// ElementHandle variant
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export async function ensureActionableHandle(
|
||||
el: ElementHandle,
|
||||
checks: ReadonlySet<CheckName>,
|
||||
timeout: number = 30000,
|
||||
force: boolean = false,
|
||||
): Promise<void> {
|
||||
if (force) return;
|
||||
|
||||
const deadline = Date.now() + timeout;
|
||||
let attempt = 0;
|
||||
let lastError: ActionabilityError | null = null;
|
||||
const label = '<ElementHandle>';
|
||||
|
||||
while (true) {
|
||||
const remainingMs = Math.max(0, deadline - Date.now());
|
||||
if (remainingMs <= 0) {
|
||||
if (lastError) throw lastError;
|
||||
throw new ActionabilityError(label, 'timeout', 'timeout expired before first check');
|
||||
}
|
||||
|
||||
try {
|
||||
if (checks.has('visible')) {
|
||||
try {
|
||||
await el.waitForElementState('visible', { timeout: Math.max(1, Math.min(remainingMs, 2000)) });
|
||||
} catch {
|
||||
throw new ElementNotVisibleError(label);
|
||||
}
|
||||
}
|
||||
|
||||
if (checks.has('enabled')) {
|
||||
try {
|
||||
await el.waitForElementState('enabled', { timeout: Math.max(1, Math.min(remainingMs, 2000)) });
|
||||
} catch {
|
||||
throw new ElementNotEnabledError(label);
|
||||
}
|
||||
}
|
||||
|
||||
if (checks.has('editable')) {
|
||||
try {
|
||||
await el.waitForElementState('editable', { timeout: Math.max(1, Math.min(remainingMs, 2000)) });
|
||||
} catch {
|
||||
throw new ElementNotEditableError(label);
|
||||
}
|
||||
}
|
||||
|
||||
return;
|
||||
} catch (e) {
|
||||
if (e instanceof ActionabilityError) {
|
||||
lastError = e;
|
||||
if (Date.now() >= deadline) throw lastError;
|
||||
await backoffSleep(attempt);
|
||||
attempt++;
|
||||
} else {
|
||||
throw e;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
export async function checkPointerEventsHandle(
|
||||
el: ElementHandle,
|
||||
x: number,
|
||||
y: number,
|
||||
timeout: number = 5000,
|
||||
): Promise<void> {
|
||||
const deadline = Date.now() + timeout;
|
||||
let attempt = 0;
|
||||
|
||||
const coords = { x, y };
|
||||
|
||||
while (true) {
|
||||
let result: any;
|
||||
try {
|
||||
result = await el.evaluate(POINTER_EVENTS_HANDLE_JS, coords);
|
||||
} catch {
|
||||
result = null;
|
||||
}
|
||||
|
||||
if (result && result.hit) return;
|
||||
|
||||
const covering = (result as any)?.covering ?? 'unknown';
|
||||
if (Date.now() >= deadline) throw new ElementNotReceivingEventsError('<ElementHandle>', covering);
|
||||
|
||||
await backoffSleep(attempt);
|
||||
attempt++;
|
||||
}
|
||||
}
|
||||
@@ -70,6 +70,12 @@ export interface HumanConfig {
|
||||
|
||||
export type HumanPreset = 'default' | 'careful';
|
||||
|
||||
export type HumanActionOptions = Partial<HumanConfig> & {
|
||||
timeout?: number;
|
||||
force?: boolean;
|
||||
human_config?: Partial<HumanConfig>;
|
||||
};
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Default preset
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -201,6 +207,22 @@ export function resolveConfig(
|
||||
return { ...base, ...overrides };
|
||||
}
|
||||
|
||||
/**
|
||||
* Merge a partial overrides object on top of an existing HumanConfig.
|
||||
* Returns a new object — the original ``cfg`` is never mutated.
|
||||
*
|
||||
* Used by per-call overrides such as ``page.type(sel, text, { human_config: { typing_delay: 30 } })``
|
||||
* so the same patched page can type different fields at different speeds
|
||||
* without re-patching.
|
||||
*/
|
||||
export function mergeConfig(
|
||||
cfg: HumanConfig,
|
||||
overrides?: Partial<HumanConfig> | null,
|
||||
): HumanConfig {
|
||||
if (!overrides) return cfg;
|
||||
return { ...cfg, ...overrides };
|
||||
}
|
||||
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Utility: random number in range
|
||||
|
||||
+195
-38
@@ -17,10 +17,15 @@
|
||||
*/
|
||||
|
||||
import type { Page, Frame, ElementHandle, CDPSession } from 'playwright-core';
|
||||
import type { HumanConfig } from './config.js';
|
||||
import { rand, randRange, sleep } from './config.js';
|
||||
import type { HumanConfig, HumanActionOptions } from './config.js';
|
||||
import { rand, randRange, sleep, mergeConfig } from './config.js';
|
||||
import { RawMouse, RawKeyboard, humanMove, humanClick, clickTarget, humanIdle } from './mouse.js';
|
||||
import { humanType } from './keyboard.js';
|
||||
import { humanScrollIntoView } from './scroll.js';
|
||||
import {
|
||||
ensureActionableHandle, checkPointerEventsHandle,
|
||||
CHECKS_CLICK, CHECKS_HOVER, CHECKS_INPUT, CHECKS_FOCUS, CHECKS_CHECK,
|
||||
} from './actionability.js';
|
||||
|
||||
// --- Platform-aware select-all shortcut ---
|
||||
const SELECT_ALL = process.platform === 'darwin' ? 'Meta+a' : 'Control+a';
|
||||
@@ -102,6 +107,7 @@ export function patchSingleElementHandle(
|
||||
const origElSetChecked = (el as any).setChecked?.bind(el);
|
||||
const origElTap = el.tap.bind(el);
|
||||
const origElFocus = el.focus.bind(el);
|
||||
const origElScrollIntoViewIfNeeded = (el as any).scrollIntoViewIfNeeded?.bind(el);
|
||||
|
||||
// Nested selectors
|
||||
const origEl$ = el.$.bind(el);
|
||||
@@ -123,86 +129,161 @@ export function patchSingleElementHandle(
|
||||
return children;
|
||||
};
|
||||
|
||||
(el as any).waitForSelector = async (selector: string, options?: any) => {
|
||||
const child = await origElWaitForSelector(selector, options);
|
||||
(el as any).waitForSelector = async (selector: string, options?: {
|
||||
state?: 'attached' | 'detached' | 'visible' | 'hidden';
|
||||
strict?: boolean;
|
||||
timeout?: number;
|
||||
}) => {
|
||||
const child = await origElWaitForSelector(selector, options ?? {});
|
||||
if (child) patchSingleElementHandle(child, page, cfg, cursor, raw, rawKb, originals, stealth);
|
||||
return child;
|
||||
};
|
||||
|
||||
// --- Helper: get bounding box and move cursor to element ---
|
||||
const moveToElement = async () => {
|
||||
// Accepts a per-call ``callCfg`` so type/fill overrides like
|
||||
// ``el.type(text, { human_config: { typing_delay: 30 } })`` or
|
||||
// ``el.type(text, { typing_delay: 30 })`` carry through to mouse movement
|
||||
// & idle timing for that single call.
|
||||
// Also scrolls the element into view first so off-screen elements work
|
||||
// (#129, #172 follow-up): otherwise boundingBox() returns null and we'd
|
||||
// silently fall back to the unpatched native method.
|
||||
const moveToElement = async (callCfg: HumanConfig = cfg) => {
|
||||
// Ensure cursor is initialized
|
||||
const ensureCursorInit = (page as any)._ensureCursorInit;
|
||||
if (ensureCursorInit) await ensureCursorInit();
|
||||
|
||||
// Scroll into view first so boundingBox() returns coordinates even when
|
||||
// the element starts below the fold. Best-effort — if humanScrollIntoView
|
||||
// throws (e.g. detached element), we let boundingBox() decide whether to
|
||||
// proceed or fall back to the original method.
|
||||
try {
|
||||
const { cursorX, cursorY } = await humanScrollIntoView(
|
||||
page, raw,
|
||||
() => el.boundingBox(),
|
||||
cursor.x, cursor.y, callCfg,
|
||||
);
|
||||
cursor.x = cursorX;
|
||||
cursor.y = cursorY;
|
||||
} catch { /* let boundingBox() decide */ }
|
||||
|
||||
const box = await el.boundingBox();
|
||||
if (!box) return null;
|
||||
|
||||
const isInp = await isInputElementHandle(stealth, el);
|
||||
const target = clickTarget(box, isInp, cfg);
|
||||
const target = clickTarget(box, isInp, callCfg);
|
||||
|
||||
if (cfg.idle_between_actions) {
|
||||
await humanIdle(raw, rand(cfg.idle_between_duration[0], cfg.idle_between_duration[1]), cursor.x, cursor.y, cfg);
|
||||
if (callCfg.idle_between_actions) {
|
||||
await humanIdle(raw, cursor.x, cursor.y, callCfg);
|
||||
}
|
||||
|
||||
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, cfg);
|
||||
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, callCfg);
|
||||
cursor.x = target.x;
|
||||
cursor.y = target.y;
|
||||
return { box, isInp };
|
||||
};
|
||||
|
||||
// --- el.click() ---
|
||||
(el as any).click = async (options?: any) => {
|
||||
const info = await moveToElement();
|
||||
(el as any).click = async (options?: HumanActionOptions & {
|
||||
button?: 'left' | 'right' | 'middle';
|
||||
clickCount?: number;
|
||||
delay?: number;
|
||||
force?: boolean;
|
||||
modifiers?: Array<'Alt' | 'Control' | 'ControlOrMeta' | 'Meta' | 'Shift'>;
|
||||
noWaitAfter?: boolean;
|
||||
position?: { x: number; y: number };
|
||||
trial?: boolean;
|
||||
}) => {
|
||||
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
|
||||
const force = options?.force ?? false;
|
||||
const timeout = options?.timeout ?? 30000;
|
||||
if (!force) await ensureActionableHandle(el, CHECKS_CLICK, timeout, force);
|
||||
const info = await moveToElement(callCfg);
|
||||
if (!info) return origElClick(options);
|
||||
await humanClick(raw, info.isInp, cfg);
|
||||
if (!force) await checkPointerEventsHandle(el, cursor.x, cursor.y, Math.min(timeout, 5000));
|
||||
await humanClick(raw, info.isInp, callCfg);
|
||||
};
|
||||
|
||||
// --- el.dblclick() ---
|
||||
(el as any).dblclick = async (options?: any) => {
|
||||
const info = await moveToElement();
|
||||
(el as any).dblclick = async (options?: HumanActionOptions & {
|
||||
button?: 'left' | 'right' | 'middle';
|
||||
delay?: number;
|
||||
force?: boolean;
|
||||
modifiers?: Array<'Alt' | 'Control' | 'ControlOrMeta' | 'Meta' | 'Shift'>;
|
||||
noWaitAfter?: boolean;
|
||||
position?: { x: number; y: number };
|
||||
trial?: boolean;
|
||||
}) => {
|
||||
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
|
||||
const force = options?.force ?? false;
|
||||
const timeout = options?.timeout ?? 30000;
|
||||
if (!force) await ensureActionableHandle(el, CHECKS_CLICK, timeout, force);
|
||||
const info = await moveToElement(callCfg);
|
||||
if (!info) return origElDblclick(options);
|
||||
if (!force) await checkPointerEventsHandle(el, cursor.x, cursor.y, Math.min(timeout, 5000));
|
||||
await raw.down({ clickCount: 2 });
|
||||
await sleep(rand(30, 60));
|
||||
await raw.up({ clickCount: 2 });
|
||||
};
|
||||
|
||||
// --- el.hover() ---
|
||||
(el as any).hover = async (options?: any) => {
|
||||
const info = await moveToElement();
|
||||
(el as any).hover = async (options?: HumanActionOptions & {
|
||||
force?: boolean;
|
||||
modifiers?: Array<'Alt' | 'Control' | 'ControlOrMeta' | 'Meta' | 'Shift'>;
|
||||
position?: { x: number; y: number };
|
||||
trial?: boolean;
|
||||
}) => {
|
||||
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
|
||||
const force = options?.force ?? false;
|
||||
const timeout = options?.timeout ?? 30000;
|
||||
if (!force) await ensureActionableHandle(el, CHECKS_HOVER, timeout, force);
|
||||
const info = await moveToElement(callCfg);
|
||||
if (!info) return origElHover(options);
|
||||
// Just move — no click
|
||||
};
|
||||
|
||||
// --- el.type() ---
|
||||
(el as any).type = async (text: string, options?: any) => {
|
||||
const info = await moveToElement();
|
||||
(el as any).type = async (text: string, options?: HumanActionOptions & {
|
||||
delay?: number;
|
||||
noWaitAfter?: boolean;
|
||||
}) => {
|
||||
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
|
||||
const force = (options as any)?.force ?? false;
|
||||
const timeout = options?.timeout ?? 30000;
|
||||
if (!force) await ensureActionableHandle(el, CHECKS_INPUT, timeout, force);
|
||||
const info = await moveToElement(callCfg);
|
||||
if (!info) return origElType(text, options);
|
||||
await humanClick(raw, info.isInp, cfg);
|
||||
if (!force) await checkPointerEventsHandle(el, cursor.x, cursor.y, Math.min(timeout, 5000));
|
||||
await humanClick(raw, info.isInp, callCfg);
|
||||
await sleep(rand(100, 250));
|
||||
let cdpSession: CDPSession | null = null;
|
||||
try { cdpSession = await stealth?.getCdpSession(); } catch {}
|
||||
await humanType(page, rawKb, text, cfg, cdpSession);
|
||||
await humanType(page, rawKb, text, callCfg, cdpSession);
|
||||
};
|
||||
|
||||
// --- el.fill() ---
|
||||
(el as any).fill = async (value: string, options?: any) => {
|
||||
const info = await moveToElement();
|
||||
(el as any).fill = async (value: string, options?: HumanActionOptions & {
|
||||
force?: boolean;
|
||||
noWaitAfter?: boolean;
|
||||
}) => {
|
||||
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
|
||||
const force = options?.force ?? false;
|
||||
const timeout = options?.timeout ?? 30000;
|
||||
if (!force) await ensureActionableHandle(el, CHECKS_INPUT, timeout, force);
|
||||
const info = await moveToElement(callCfg);
|
||||
if (!info) return origElFill(value, options);
|
||||
await humanClick(raw, info.isInp, cfg);
|
||||
if (!force) await checkPointerEventsHandle(el, cursor.x, cursor.y, Math.min(timeout, 5000));
|
||||
await humanClick(raw, info.isInp, callCfg);
|
||||
await sleep(rand(100, 250));
|
||||
// Clear existing content
|
||||
await originals.keyboardPress(SELECT_ALL);
|
||||
await sleep(rand(30, 80));
|
||||
await originals.keyboardPress('Backspace');
|
||||
await sleep(rand(50, 150));
|
||||
let cdpSession: CDPSession | null = null;
|
||||
try { cdpSession = await stealth?.getCdpSession(); } catch {}
|
||||
await humanType(page, rawKb, value, cfg, cdpSession);
|
||||
await humanType(page, rawKb, value, callCfg, cdpSession);
|
||||
};
|
||||
|
||||
// --- el.press() ---
|
||||
(el as any).press = async (key: string, options?: any) => {
|
||||
(el as any).press = async (key: string, options?: { delay?: number; noWaitAfter?: boolean; timeout?: number }) => {
|
||||
await sleep(rand(20, 60));
|
||||
await originals.keyboardDown(key);
|
||||
await sleep(randRange(cfg.key_hold));
|
||||
@@ -210,7 +291,14 @@ export function patchSingleElementHandle(
|
||||
};
|
||||
|
||||
// --- el.selectOption() ---
|
||||
(el as any).selectOption = async (values: any, options?: any) => {
|
||||
(el as any).selectOption = async (values: any, options?: {
|
||||
force?: boolean;
|
||||
noWaitAfter?: boolean;
|
||||
timeout?: number;
|
||||
}) => {
|
||||
const force = options?.force ?? false;
|
||||
const timeout = options?.timeout ?? 30000;
|
||||
if (!force) await ensureActionableHandle(el, CHECKS_FOCUS, timeout, force);
|
||||
const info = await moveToElement();
|
||||
if (!info) return origElSelectOption(values, options);
|
||||
await humanClick(raw, false, cfg);
|
||||
@@ -219,42 +307,79 @@ export function patchSingleElementHandle(
|
||||
};
|
||||
|
||||
// --- el.check() ---
|
||||
(el as any).check = async (options?: any) => {
|
||||
(el as any).check = async (options?: {
|
||||
force?: boolean;
|
||||
noWaitAfter?: boolean;
|
||||
position?: { x: number; y: number };
|
||||
timeout?: number;
|
||||
trial?: boolean;
|
||||
}) => {
|
||||
const force = options?.force ?? false;
|
||||
const timeout = options?.timeout ?? 30000;
|
||||
if (!force) await ensureActionableHandle(el, CHECKS_CHECK, timeout, force);
|
||||
try {
|
||||
const checked = await el.isChecked();
|
||||
if (checked) return; // Already checked
|
||||
if (checked) return;
|
||||
} catch {}
|
||||
const info = await moveToElement();
|
||||
if (!info) return origElCheck(options);
|
||||
if (!force) await checkPointerEventsHandle(el, cursor.x, cursor.y, Math.min(timeout, 5000));
|
||||
await humanClick(raw, info.isInp, cfg);
|
||||
};
|
||||
|
||||
// --- el.uncheck() ---
|
||||
(el as any).uncheck = async (options?: any) => {
|
||||
(el as any).uncheck = async (options?: {
|
||||
force?: boolean;
|
||||
noWaitAfter?: boolean;
|
||||
position?: { x: number; y: number };
|
||||
timeout?: number;
|
||||
trial?: boolean;
|
||||
}) => {
|
||||
const force = options?.force ?? false;
|
||||
const timeout = options?.timeout ?? 30000;
|
||||
if (!force) await ensureActionableHandle(el, CHECKS_CHECK, timeout, force);
|
||||
try {
|
||||
const checked = await el.isChecked();
|
||||
if (!checked) return; // Already unchecked
|
||||
if (!checked) return;
|
||||
} catch {}
|
||||
const info = await moveToElement();
|
||||
if (!info) return origElUncheck(options);
|
||||
if (!force) await checkPointerEventsHandle(el, cursor.x, cursor.y, Math.min(timeout, 5000));
|
||||
await humanClick(raw, info.isInp, cfg);
|
||||
};
|
||||
|
||||
// --- el.setChecked() ---
|
||||
if (origElSetChecked) {
|
||||
(el as any).setChecked = async (checked: boolean, options?: any) => {
|
||||
(el as any).setChecked = async (checked: boolean, options?: {
|
||||
force?: boolean;
|
||||
noWaitAfter?: boolean;
|
||||
position?: { x: number; y: number };
|
||||
timeout?: number;
|
||||
trial?: boolean;
|
||||
}) => {
|
||||
const force = options?.force ?? false;
|
||||
const timeout = options?.timeout ?? 30000;
|
||||
if (!force) await ensureActionableHandle(el, CHECKS_CHECK, timeout, force);
|
||||
try {
|
||||
const current = await el.isChecked();
|
||||
if (current === checked) return;
|
||||
} catch {}
|
||||
const info = await moveToElement();
|
||||
if (!info) return origElSetChecked(checked, options);
|
||||
if (!force) await checkPointerEventsHandle(el, cursor.x, cursor.y, Math.min(timeout, 5000));
|
||||
await humanClick(raw, info.isInp, cfg);
|
||||
};
|
||||
}
|
||||
|
||||
// --- el.tap() ---
|
||||
(el as any).tap = async (options?: any) => {
|
||||
(el as any).tap = async (options?: {
|
||||
force?: boolean;
|
||||
modifiers?: Array<'Alt' | 'Control' | 'ControlOrMeta' | 'Meta' | 'Shift'>;
|
||||
noWaitAfter?: boolean;
|
||||
position?: { x: number; y: number };
|
||||
timeout?: number;
|
||||
trial?: boolean;
|
||||
}) => {
|
||||
const info = await moveToElement();
|
||||
if (!info) return origElTap(options);
|
||||
await humanClick(raw, info.isInp, cfg);
|
||||
@@ -268,6 +393,30 @@ export function patchSingleElementHandle(
|
||||
await moveToElement(); // human-like Bézier cursor movement
|
||||
await origElFocus(); // programmatic focus, no click
|
||||
};
|
||||
|
||||
// --- el.scrollIntoViewIfNeeded() ---
|
||||
// Playwright's native version snaps the page — a strong bot signal.
|
||||
// Replace with the same accelerate → cruise → decelerate → overshoot
|
||||
// wheel sequence used by page.click() etc. Falls back to the native
|
||||
// method if the element is detached or scrolling fails.
|
||||
if (origElScrollIntoViewIfNeeded) {
|
||||
(el as any).scrollIntoViewIfNeeded = async (options?: HumanActionOptions) => {
|
||||
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
|
||||
const ensureCursorInit = (page as any)._ensureCursorInit;
|
||||
if (ensureCursorInit) await ensureCursorInit();
|
||||
try {
|
||||
const { cursorX, cursorY } = await humanScrollIntoView(
|
||||
page, raw,
|
||||
() => el.boundingBox(),
|
||||
cursor.x, cursor.y, callCfg,
|
||||
);
|
||||
cursor.x = cursorX;
|
||||
cursor.y = cursorY;
|
||||
} catch {
|
||||
return origElScrollIntoViewIfNeeded(options);
|
||||
}
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -309,8 +458,12 @@ export function patchPageElementHandles(
|
||||
// Patch page.waitForSelector()
|
||||
if (typeof page.waitForSelector === 'function') {
|
||||
const origWaitForSelector = page.waitForSelector.bind(page);
|
||||
(page as any).waitForSelector = async (selector: string, options?: any) => {
|
||||
const el = await origWaitForSelector(selector, options);
|
||||
(page as any).waitForSelector = async (selector: string, options?: {
|
||||
state?: 'attached' | 'detached' | 'visible' | 'hidden';
|
||||
strict?: boolean;
|
||||
timeout?: number;
|
||||
}) => {
|
||||
const el = await origWaitForSelector(selector, options ?? {});
|
||||
if (el) patchSingleElementHandle(el, page, cfg, cursor, raw, rawKb, originals, stealth);
|
||||
return el;
|
||||
};
|
||||
@@ -357,8 +510,12 @@ export function patchFrameElementHandles(
|
||||
// Patch frame.waitForSelector()
|
||||
if (typeof frame.waitForSelector === 'function') {
|
||||
const origFrameWaitForSelector = frame.waitForSelector.bind(frame);
|
||||
(frame as any).waitForSelector = async (selector: string, options?: any) => {
|
||||
const el = await origFrameWaitForSelector(selector, options);
|
||||
(frame as any).waitForSelector = async (selector: string, options?: {
|
||||
state?: 'attached' | 'detached' | 'visible' | 'hidden';
|
||||
strict?: boolean;
|
||||
timeout?: number;
|
||||
}) => {
|
||||
const el = await origFrameWaitForSelector(selector, options ?? {});
|
||||
if (el) patchSingleElementHandle(el, page, cfg, cursor, raw, rawKb, originals, stealth);
|
||||
return el;
|
||||
};
|
||||
|
||||
+304
-84
@@ -23,16 +23,21 @@
|
||||
*/
|
||||
|
||||
import type { Browser, BrowserContext, Page, Frame, CDPSession } from 'playwright-core';
|
||||
import { HumanConfig, resolveConfig, rand, randRange, sleep } from './config.js';
|
||||
import { HumanConfig, HumanActionOptions, resolveConfig, mergeConfig, rand, randRange, sleep } from './config.js';
|
||||
import { RawMouse, RawKeyboard, humanMove, humanClick, clickTarget, humanIdle } from './mouse.js';
|
||||
import { humanType } from './keyboard.js';
|
||||
import { scrollToElement } from './scroll.js';
|
||||
import { scrollToElement, humanScrollIntoView } from './scroll.js';
|
||||
import { patchPageElementHandles, patchFrameElementHandles, patchSingleElementHandle } from './elementhandle.js';
|
||||
import {
|
||||
ensureActionable, ensureStable, checkPointerEvents,
|
||||
CHECKS_CLICK, CHECKS_HOVER, CHECKS_INPUT, CHECKS_FOCUS, CHECKS_CHECK,
|
||||
type CheckName,
|
||||
} from './actionability.js';
|
||||
|
||||
export { HumanConfig, resolveConfig } from './config.js';
|
||||
export { HumanConfig, resolveConfig, mergeConfig } from './config.js';
|
||||
export { humanMove, humanClick, clickTarget, humanIdle } from './mouse.js';
|
||||
export { humanType } from './keyboard.js';
|
||||
export { scrollToElement } from './scroll.js';
|
||||
export { scrollToElement, humanScrollIntoView } from './scroll.js';
|
||||
export { patchSingleElementHandle } from './elementhandle.js';
|
||||
|
||||
// --- Platform-aware select-all shortcut (macOS uses Meta, others use Control) ---
|
||||
@@ -295,7 +300,11 @@ function patchPage(page: Page, cfg: HumanConfig, cursor: CursorState): void {
|
||||
}
|
||||
|
||||
// --- goto (invalidate isolated world on navigation) ---
|
||||
const humanGoto = async (url: string, options?: any) => {
|
||||
const humanGoto = async (url: string, options?: {
|
||||
referer?: string;
|
||||
timeout?: number;
|
||||
waitUntil?: 'load' | 'domcontentloaded' | 'networkidle' | 'commit';
|
||||
}) => {
|
||||
const response = await originals.goto(url, options);
|
||||
stealth.invalidate();
|
||||
patchFrames(page, cfg, cursor, raw, rawKb, originals, stealth);
|
||||
@@ -303,34 +312,67 @@ function patchPage(page: Page, cfg: HumanConfig, cursor: CursorState): void {
|
||||
};
|
||||
|
||||
// --- click ---
|
||||
const humanClickFn = async (selector: string, options?: any) => {
|
||||
const humanClickFn = async (selector: string, options?: HumanActionOptions & { _skipChecks?: boolean }) => {
|
||||
await ensureCursorInit();
|
||||
if (cfg.idle_between_actions) {
|
||||
await humanIdle(raw, rand(cfg.idle_between_duration[0], cfg.idle_between_duration[1]), cursor.x, cursor.y, cfg);
|
||||
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
|
||||
const timeout = options?.timeout ?? 30000;
|
||||
const force = options?.force ?? false;
|
||||
const skipChecks = (options as any)?._skipChecks ?? false;
|
||||
const deadline = Date.now() + timeout;
|
||||
const remainingMs = () => Math.max(0, deadline - Date.now());
|
||||
|
||||
if (!force && !skipChecks) {
|
||||
await ensureActionable(page, selector, CHECKS_CLICK, remainingMs(), force);
|
||||
}
|
||||
const { box, cursorX, cursorY } = await scrollToElement(page, raw, selector, cursor.x, cursor.y, cfg);
|
||||
if (callCfg.idle_between_actions) {
|
||||
await humanIdle(raw, cursor.x, cursor.y, callCfg);
|
||||
}
|
||||
const { box, cursorX, cursorY, didScroll } = await scrollToElement(page, raw, selector, cursor.x, cursor.y, callCfg, remainingMs());
|
||||
cursor.x = cursorX;
|
||||
cursor.y = cursorY;
|
||||
const isInput = await isInputElement(stealth, page, selector);
|
||||
const target = clickTarget(box, isInput, cfg);
|
||||
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, cfg);
|
||||
let finalBox = box;
|
||||
if (!force && didScroll) {
|
||||
await ensureStable(page, selector, remainingMs());
|
||||
finalBox = await page.locator(selector).first().boundingBox({ timeout: Math.max(1, remainingMs()) }) ?? box;
|
||||
}
|
||||
const target = clickTarget(finalBox, isInput, callCfg);
|
||||
if (!force) {
|
||||
await checkPointerEvents(page, selector, target.x, target.y, stealth, remainingMs());
|
||||
}
|
||||
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, callCfg);
|
||||
cursor.x = target.x;
|
||||
cursor.y = target.y;
|
||||
await humanClick(raw, isInput, cfg);
|
||||
await humanClick(raw, isInput, callCfg);
|
||||
};
|
||||
|
||||
// --- dblclick ---
|
||||
const humanDblclickFn = async (selector: string, options?: any) => {
|
||||
const humanDblclickFn = async (selector: string, options?: HumanActionOptions) => {
|
||||
await ensureCursorInit();
|
||||
if (cfg.idle_between_actions) {
|
||||
await humanIdle(raw, rand(cfg.idle_between_duration[0], cfg.idle_between_duration[1]), cursor.x, cursor.y, cfg);
|
||||
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
|
||||
const timeout = options?.timeout ?? 30000;
|
||||
const force = options?.force ?? false;
|
||||
const deadline = Date.now() + timeout;
|
||||
const remainingMs = () => Math.max(0, deadline - Date.now());
|
||||
|
||||
if (!force) await ensureActionable(page, selector, CHECKS_CLICK, remainingMs(), force);
|
||||
if (callCfg.idle_between_actions) {
|
||||
await humanIdle(raw, cursor.x, cursor.y, callCfg);
|
||||
}
|
||||
const { box, cursorX, cursorY } = await scrollToElement(page, raw, selector, cursor.x, cursor.y, cfg);
|
||||
const { box, cursorX, cursorY, didScroll } = await scrollToElement(page, raw, selector, cursor.x, cursor.y, callCfg, remainingMs());
|
||||
cursor.x = cursorX;
|
||||
cursor.y = cursorY;
|
||||
const isInput = await isInputElement(stealth, page, selector);
|
||||
const target = clickTarget(box, isInput, cfg);
|
||||
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, cfg);
|
||||
let finalBox = box;
|
||||
if (!force && didScroll) {
|
||||
await ensureStable(page, selector, remainingMs());
|
||||
finalBox = await page.locator(selector).first().boundingBox({ timeout: Math.max(1, remainingMs()) }) ?? box;
|
||||
}
|
||||
const target = clickTarget(finalBox, isInput, callCfg);
|
||||
if (!force) {
|
||||
await checkPointerEvents(page, selector, target.x, target.y, stealth, remainingMs());
|
||||
}
|
||||
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, callCfg);
|
||||
cursor.x = target.x;
|
||||
cursor.y = target.y;
|
||||
await raw.down({ clickCount: 2 });
|
||||
@@ -339,46 +381,82 @@ function patchPage(page: Page, cfg: HumanConfig, cursor: CursorState): void {
|
||||
};
|
||||
|
||||
// --- hover ---
|
||||
const humanHoverFn = async (selector: string, options?: any) => {
|
||||
const humanHoverFn = async (selector: string, options?: HumanActionOptions & { _skipChecks?: boolean }) => {
|
||||
await ensureCursorInit();
|
||||
if (cfg.idle_between_actions) {
|
||||
await humanIdle(raw, rand(cfg.idle_between_duration[0], cfg.idle_between_duration[1]), cursor.x, cursor.y, cfg);
|
||||
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
|
||||
const timeout = options?.timeout ?? 30000;
|
||||
const force = options?.force ?? false;
|
||||
const skipChecks = (options as any)?._skipChecks ?? false;
|
||||
const deadline = Date.now() + timeout;
|
||||
const remainingMs = () => Math.max(0, deadline - Date.now());
|
||||
|
||||
if (!force && !skipChecks) await ensureActionable(page, selector, CHECKS_HOVER, remainingMs(), force);
|
||||
if (callCfg.idle_between_actions) {
|
||||
await humanIdle(raw, cursor.x, cursor.y, callCfg);
|
||||
}
|
||||
const { box, cursorX, cursorY } = await scrollToElement(page, raw, selector, cursor.x, cursor.y, cfg);
|
||||
const { box, cursorX, cursorY, didScroll } = await scrollToElement(page, raw, selector, cursor.x, cursor.y, callCfg, remainingMs());
|
||||
cursor.x = cursorX;
|
||||
cursor.y = cursorY;
|
||||
const target = clickTarget(box, false, cfg);
|
||||
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, cfg);
|
||||
let finalBox = box;
|
||||
if (!force && didScroll) {
|
||||
await ensureStable(page, selector, remainingMs());
|
||||
finalBox = await page.locator(selector).first().boundingBox({ timeout: Math.max(1, remainingMs()) }) ?? box;
|
||||
}
|
||||
const target = clickTarget(finalBox, false, callCfg);
|
||||
if (!force) {
|
||||
await checkPointerEvents(page, selector, target.x, target.y, stealth, remainingMs());
|
||||
}
|
||||
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, callCfg);
|
||||
cursor.x = target.x;
|
||||
cursor.y = target.y;
|
||||
};
|
||||
|
||||
// --- type ---
|
||||
const humanTypeFn = async (selector: string, text: string, options?: any) => {
|
||||
await sleep(randRange(cfg.field_switch_delay));
|
||||
await humanClickFn(selector);
|
||||
const humanTypeFn = async (selector: string, text: string, options?: HumanActionOptions) => {
|
||||
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
|
||||
const timeout = options?.timeout ?? 30000;
|
||||
const force = options?.force ?? false;
|
||||
const deadline = Date.now() + timeout;
|
||||
const remainingMs = () => Math.max(0, deadline - Date.now());
|
||||
|
||||
if (!force) await ensureActionable(page, selector, CHECKS_INPUT, remainingMs(), force);
|
||||
await sleep(randRange(callCfg.field_switch_delay));
|
||||
await humanClickFn(selector, { _skipChecks: true, timeout: remainingMs(), force, human_config: options?.human_config } as any);
|
||||
await sleep(rand(100, 250));
|
||||
const cdp = await ensureCdp();
|
||||
await humanType(page, rawKb, text, cfg, cdp);
|
||||
await humanType(page, rawKb, text, callCfg, cdp);
|
||||
};
|
||||
|
||||
// --- fill (clears existing content first) ---
|
||||
const humanFillFn = async (selector: string, value: string, options?: any) => {
|
||||
await sleep(randRange(cfg.field_switch_delay));
|
||||
await humanClickFn(selector);
|
||||
const humanFillFn = async (selector: string, value: string, options?: HumanActionOptions) => {
|
||||
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
|
||||
const timeout = options?.timeout ?? 30000;
|
||||
const force = options?.force ?? false;
|
||||
const deadline = Date.now() + timeout;
|
||||
const remainingMs = () => Math.max(0, deadline - Date.now());
|
||||
|
||||
if (!force) await ensureActionable(page, selector, CHECKS_INPUT, remainingMs(), force);
|
||||
await sleep(randRange(callCfg.field_switch_delay));
|
||||
await humanClickFn(selector, { _skipChecks: true, timeout: remainingMs(), force, human_config: options?.human_config } as any);
|
||||
await sleep(rand(100, 250));
|
||||
await originals.keyboardPress(SELECT_ALL);
|
||||
await sleep(rand(30, 80));
|
||||
await originals.keyboardPress('Backspace');
|
||||
await sleep(rand(50, 150));
|
||||
const cdp = await ensureCdp();
|
||||
await humanType(page, rawKb, value, cfg, cdp);
|
||||
await humanType(page, rawKb, value, callCfg, cdp);
|
||||
};
|
||||
|
||||
// --- clear ---
|
||||
const humanClearFn = async (selector: string, options?: any) => {
|
||||
const humanClearFn = async (selector: string, options?: HumanActionOptions) => {
|
||||
const timeout = options?.timeout ?? 30000;
|
||||
const force = options?.force ?? false;
|
||||
const deadline = Date.now() + timeout;
|
||||
const remainingMs = () => Math.max(0, deadline - Date.now());
|
||||
|
||||
if (!force) await ensureActionable(page, selector, CHECKS_FOCUS, remainingMs(), force);
|
||||
if (!await isSelectorFocused(stealth, page, selector)) {
|
||||
await humanClickFn(selector);
|
||||
await humanClickFn(selector, { _skipChecks: true, timeout: remainingMs(), force, human_config: options?.human_config } as any);
|
||||
}
|
||||
await sleep(rand(50, 150));
|
||||
await originals.keyboardPress(SELECT_ALL);
|
||||
@@ -387,55 +465,88 @@ function patchPage(page: Page, cfg: HumanConfig, cursor: CursorState): void {
|
||||
};
|
||||
|
||||
// --- check ---
|
||||
const humanCheckFn = async (selector: string, options?: any) => {
|
||||
if (cfg.idle_between_actions) {
|
||||
await humanIdle(raw, rand(cfg.idle_between_duration[0], cfg.idle_between_duration[1]), cursor.x, cursor.y, cfg);
|
||||
const humanCheckFn = async (selector: string, options?: HumanActionOptions) => {
|
||||
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
|
||||
const timeout = options?.timeout ?? 30000;
|
||||
const force = options?.force ?? false;
|
||||
const deadline = Date.now() + timeout;
|
||||
const remainingMs = () => Math.max(0, deadline - Date.now());
|
||||
|
||||
if (!force) await ensureActionable(page, selector, CHECKS_CHECK, remainingMs(), force);
|
||||
if (callCfg.idle_between_actions) {
|
||||
await humanIdle(raw, cursor.x, cursor.y, callCfg);
|
||||
}
|
||||
const checked = await originals.isChecked(selector).catch(() => false);
|
||||
if (!checked) {
|
||||
await humanClickFn(selector);
|
||||
await humanClickFn(selector, { _skipChecks: true, timeout: remainingMs(), force, human_config: options?.human_config } as any);
|
||||
}
|
||||
};
|
||||
|
||||
// --- uncheck ---
|
||||
const humanUncheckFn = async (selector: string, options?: any) => {
|
||||
if (cfg.idle_between_actions) {
|
||||
await humanIdle(raw, rand(cfg.idle_between_duration[0], cfg.idle_between_duration[1]), cursor.x, cursor.y, cfg);
|
||||
const humanUncheckFn = async (selector: string, options?: HumanActionOptions) => {
|
||||
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
|
||||
const timeout = options?.timeout ?? 30000;
|
||||
const force = options?.force ?? false;
|
||||
const deadline = Date.now() + timeout;
|
||||
const remainingMs = () => Math.max(0, deadline - Date.now());
|
||||
|
||||
if (!force) await ensureActionable(page, selector, CHECKS_CHECK, remainingMs(), force);
|
||||
if (callCfg.idle_between_actions) {
|
||||
await humanIdle(raw, cursor.x, cursor.y, callCfg);
|
||||
}
|
||||
const checked = await originals.isChecked(selector).catch(() => true);
|
||||
if (checked) {
|
||||
await humanClickFn(selector);
|
||||
await humanClickFn(selector, { _skipChecks: true, timeout: remainingMs(), force, human_config: options?.human_config } as any);
|
||||
}
|
||||
};
|
||||
|
||||
// --- selectOption ---
|
||||
const humanSelectOptionFn = async (selector: string, values: any, options?: any) => {
|
||||
await humanHoverFn(selector);
|
||||
const humanSelectOptionFn = async (selector: string, values: any, options?: HumanActionOptions) => {
|
||||
const timeout = options?.timeout ?? 30000;
|
||||
const force = options?.force ?? false;
|
||||
const deadline = Date.now() + timeout;
|
||||
const remainingMs = () => Math.max(0, deadline - Date.now());
|
||||
|
||||
if (!force) await ensureActionable(page, selector, CHECKS_FOCUS, remainingMs(), force);
|
||||
await humanHoverFn(selector, { _skipChecks: true, timeout: remainingMs(), force, human_config: options?.human_config } as any);
|
||||
await sleep(rand(100, 300));
|
||||
return originals.selectOption(selector, values, options);
|
||||
};
|
||||
|
||||
// --- press (checks focus first — avoids redundant mouse moves) ---
|
||||
const humanPressFn = async (selector: string, key: string, options?: any) => {
|
||||
const humanPressFn = async (selector: string, key: string, options?: HumanActionOptions) => {
|
||||
const timeout = options?.timeout ?? 30000;
|
||||
const force = options?.force ?? false;
|
||||
const deadline = Date.now() + timeout;
|
||||
const remainingMs = () => Math.max(0, deadline - Date.now());
|
||||
|
||||
if (!force) await ensureActionable(page, selector, CHECKS_FOCUS, remainingMs(), force);
|
||||
if (!await isSelectorFocused(stealth, page, selector)) {
|
||||
await humanClickFn(selector);
|
||||
await humanClickFn(selector, { _skipChecks: true, timeout: remainingMs(), force, human_config: options?.human_config } as any);
|
||||
}
|
||||
await sleep(rand(50, 150));
|
||||
await originals.keyboardPress(key);
|
||||
};
|
||||
|
||||
// --- pressSequentially ---
|
||||
const humanPressSequentiallyFn = async (selector: string, text: string, options?: any) => {
|
||||
const humanPressSequentiallyFn = async (selector: string, text: string, options?: HumanActionOptions) => {
|
||||
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
|
||||
const timeout = options?.timeout ?? 30000;
|
||||
const force = options?.force ?? false;
|
||||
const deadline = Date.now() + timeout;
|
||||
const remainingMs = () => Math.max(0, deadline - Date.now());
|
||||
|
||||
if (!force) await ensureActionable(page, selector, CHECKS_FOCUS, remainingMs(), force);
|
||||
if (!await isSelectorFocused(stealth, page, selector)) {
|
||||
await humanClickFn(selector);
|
||||
await humanClickFn(selector, { _skipChecks: true, timeout: remainingMs(), force, human_config: options?.human_config } as any);
|
||||
}
|
||||
await sleep(rand(100, 250));
|
||||
const cdp = await ensureCdp();
|
||||
await humanType(page, rawKb, text, cfg, cdp);
|
||||
await humanType(page, rawKb, text, callCfg, cdp);
|
||||
};
|
||||
|
||||
// --- tap ---
|
||||
const humanTapFn = async (selector: string, options?: any) => {
|
||||
const humanTapFn = async (selector: string, options?: HumanActionOptions) => {
|
||||
await humanClickFn(selector, options);
|
||||
};
|
||||
|
||||
@@ -455,14 +566,20 @@ function patchPage(page: Page, cfg: HumanConfig, cursor: CursorState): void {
|
||||
(page as any).clear = humanClearFn;
|
||||
|
||||
// --- mouse patches ---
|
||||
page.mouse.move = async (x: number, y: number, options?: any) => {
|
||||
page.mouse.move = async (x: number, y: number, options?: {
|
||||
steps?: number;
|
||||
}) => {
|
||||
await ensureCursorInit();
|
||||
await humanMove(raw, cursor.x, cursor.y, x, y, cfg);
|
||||
cursor.x = x;
|
||||
cursor.y = y;
|
||||
};
|
||||
|
||||
page.mouse.click = async (x: number, y: number, options?: any) => {
|
||||
page.mouse.click = async (x: number, y: number, options?: {
|
||||
button?: 'left' | 'right' | 'middle';
|
||||
clickCount?: number;
|
||||
delay?: number;
|
||||
}) => {
|
||||
await ensureCursorInit();
|
||||
await humanMove(raw, cursor.x, cursor.y, x, y, cfg);
|
||||
cursor.x = x;
|
||||
@@ -471,7 +588,7 @@ function patchPage(page: Page, cfg: HumanConfig, cursor: CursorState): void {
|
||||
};
|
||||
|
||||
// --- keyboard patches ---
|
||||
page.keyboard.type = async (text: string, options?: any) => {
|
||||
page.keyboard.type = async (text: string, options?: { delay?: number }) => {
|
||||
const cdp = await ensureCdp();
|
||||
await humanType(page, rawKb, text, cfg, cdp);
|
||||
};
|
||||
@@ -523,16 +640,37 @@ function patchFrames(
|
||||
stealth: StealthEval,
|
||||
): void {
|
||||
for (const frame of iterFrames(page)) {
|
||||
patchSingleFrame(frame, page, cfg, originals, stealth);
|
||||
patchSingleFrame(frame, page, cfg, cursor, raw, rawKb, originals, stealth);
|
||||
// Patch frame-level ElementHandle selectors ($, $$, waitForSelector)
|
||||
patchFrameElementHandles(frame, page, cfg, cursor, raw, rawKb, originals, stealth);
|
||||
}
|
||||
}
|
||||
|
||||
function firstFrameLocator(frame: Frame, selector: string): any {
|
||||
const locator = frame.locator(selector) as any;
|
||||
return typeof locator.first === 'function' ? locator.first() : locator;
|
||||
}
|
||||
|
||||
async function isFrameInputElement(frame: Frame, selector: string): Promise<boolean> {
|
||||
return firstFrameLocator(frame, selector).evaluate((el: Element) => {
|
||||
const tag = el.tagName.toLowerCase();
|
||||
return tag === 'input' || tag === 'textarea'
|
||||
|| el.getAttribute('contenteditable') === 'true';
|
||||
}).catch(() => false);
|
||||
}
|
||||
|
||||
async function isFrameSelectorFocused(frame: Frame, selector: string): Promise<boolean> {
|
||||
return firstFrameLocator(frame, selector).evaluate((el: Element) => el === document.activeElement)
|
||||
.catch(() => false);
|
||||
}
|
||||
|
||||
function patchSingleFrame(
|
||||
frame: Frame,
|
||||
page: Page,
|
||||
cfg: HumanConfig,
|
||||
cursor: CursorState,
|
||||
raw: RawMouse,
|
||||
rawKb: RawKeyboard,
|
||||
originals: any,
|
||||
stealth: StealthEval,
|
||||
): void {
|
||||
@@ -540,58 +678,132 @@ function patchSingleFrame(
|
||||
(frame as any)._humanPatched = true;
|
||||
|
||||
// Save originals for methods that need fallback
|
||||
const origFrameClick = frame.click.bind(frame);
|
||||
const origFrameDblclick = frame.dblclick.bind(frame);
|
||||
const origFrameHover = frame.hover.bind(frame);
|
||||
const origFrameType = frame.type.bind(frame);
|
||||
const origFrameFill = frame.fill.bind(frame);
|
||||
const origFrameCheck = frame.check.bind(frame);
|
||||
const origFrameUncheck = frame.uncheck.bind(frame);
|
||||
const origFrameSelectOption = frame.selectOption.bind(frame);
|
||||
const origFramePress = frame.press.bind(frame);
|
||||
const origFramePressSequentially = (frame as any).pressSequentially?.bind(frame);
|
||||
const origFrameTap = (frame as any).tap?.bind(frame);
|
||||
const origFrameDragAndDrop = frame.dragAndDrop.bind(frame);
|
||||
|
||||
(frame as any).click = async (selector: string, options?: any) => {
|
||||
await (page as any).click(selector, options);
|
||||
const moveToFrameSelector = async (selector: string, options?: HumanActionOptions, inputBias = false) => {
|
||||
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
|
||||
if (callCfg.idle_between_actions) {
|
||||
await humanIdle(raw, cursor.x, cursor.y, callCfg);
|
||||
}
|
||||
|
||||
const locator = firstFrameLocator(frame, selector);
|
||||
if (typeof locator.scrollIntoViewIfNeeded === 'function') {
|
||||
await locator.scrollIntoViewIfNeeded({ timeout: options?.timeout }).catch(() => undefined);
|
||||
}
|
||||
const box = await locator.boundingBox({ timeout: options?.timeout ?? 30000 }).catch(() => null);
|
||||
if (!box) return null;
|
||||
|
||||
const isInput = inputBias || await isFrameInputElement(frame, selector);
|
||||
const target = clickTarget(box, isInput, callCfg);
|
||||
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, callCfg);
|
||||
cursor.x = target.x;
|
||||
cursor.y = target.y;
|
||||
return { callCfg, isInput };
|
||||
};
|
||||
|
||||
(frame as any).dblclick = async (selector: string, options?: any) => {
|
||||
await (page as any).dblclick(selector, options);
|
||||
const frameClick = async (selector: string, options?: HumanActionOptions) => {
|
||||
const moved = await moveToFrameSelector(selector, options);
|
||||
if (!moved) return origFrameClick(selector, options);
|
||||
await humanClick(raw, moved.isInput, moved.callCfg);
|
||||
};
|
||||
|
||||
(frame as any).hover = async (selector: string, options?: any) => {
|
||||
await (page as any).hover(selector, options);
|
||||
const getFrameCdp = async () => stealth.getCdpSession().catch(() => null);
|
||||
|
||||
const frameHover = async (selector: string, options?: HumanActionOptions) => {
|
||||
const moved = await moveToFrameSelector(selector, options, false);
|
||||
if (!moved) return origFrameHover(selector, options);
|
||||
};
|
||||
|
||||
(frame as any).type = async (selector: string, text: string, options?: any) => {
|
||||
await (page as any).type(selector, text, options);
|
||||
(frame as any).click = frameClick;
|
||||
|
||||
(frame as any).dblclick = async (selector: string, options?: HumanActionOptions) => {
|
||||
const moved = await moveToFrameSelector(selector, options);
|
||||
if (!moved) return origFrameDblclick(selector, options);
|
||||
await raw.down({ clickCount: 2 });
|
||||
await sleep(rand(30, 60));
|
||||
await raw.up({ clickCount: 2 });
|
||||
};
|
||||
|
||||
(frame as any).fill = async (selector: string, value: string, options?: any) => {
|
||||
await (page as any).fill(selector, value, options);
|
||||
(frame as any).hover = frameHover;
|
||||
|
||||
(frame as any).type = async (selector: string, text: string, options?: HumanActionOptions) => {
|
||||
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
|
||||
await sleep(randRange(callCfg.field_switch_delay));
|
||||
await frameClick(selector, options);
|
||||
await sleep(rand(100, 250));
|
||||
const cdp = await getFrameCdp();
|
||||
await humanType(page, rawKb, text, callCfg, cdp).catch(() => origFrameType(selector, text, options));
|
||||
};
|
||||
|
||||
(frame as any).check = async (selector: string, options?: any) => {
|
||||
await (page as any).check(selector, options);
|
||||
(frame as any).fill = async (selector: string, value: string, options?: HumanActionOptions) => {
|
||||
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
|
||||
await sleep(randRange(callCfg.field_switch_delay));
|
||||
await frameClick(selector, options);
|
||||
await sleep(rand(100, 250));
|
||||
await originals.keyboardPress(SELECT_ALL);
|
||||
await sleep(rand(30, 80));
|
||||
await originals.keyboardPress('Backspace');
|
||||
await sleep(rand(50, 150));
|
||||
const cdp = await getFrameCdp();
|
||||
await humanType(page, rawKb, value, callCfg, cdp).catch(() => origFrameFill(selector, value, options));
|
||||
};
|
||||
|
||||
(frame as any).uncheck = async (selector: string, options?: any) => {
|
||||
await (page as any).uncheck(selector, options);
|
||||
(frame as any).check = async (selector: string, options?: HumanActionOptions) => {
|
||||
const locator = firstFrameLocator(frame, selector);
|
||||
if (typeof locator.isChecked !== 'function') return origFrameCheck(selector, options);
|
||||
const checked = await locator.isChecked();
|
||||
if (!checked) await frameClick(selector, options).catch(() => origFrameCheck(selector, options));
|
||||
};
|
||||
|
||||
(frame as any).selectOption = async (selector: string, values: any, options?: any) => {
|
||||
await (page as any).hover(selector);
|
||||
(frame as any).uncheck = async (selector: string, options?: HumanActionOptions) => {
|
||||
const locator = firstFrameLocator(frame, selector);
|
||||
if (typeof locator.isChecked !== 'function') return origFrameUncheck(selector, options);
|
||||
const checked = await locator.isChecked();
|
||||
if (checked) await frameClick(selector, options).catch(() => origFrameUncheck(selector, options));
|
||||
};
|
||||
|
||||
(frame as any).selectOption = async (selector: string, values: any, options?: HumanActionOptions) => {
|
||||
await frameHover(selector, options);
|
||||
await sleep(rand(100, 300));
|
||||
return origFrameSelectOption(selector, values, options);
|
||||
};
|
||||
|
||||
(frame as any).press = async (selector: string, key: string, options?: any) => {
|
||||
await (page as any).press(selector, key, options);
|
||||
(frame as any).press = async (selector: string, key: string, options?: HumanActionOptions) => {
|
||||
if (!await isFrameSelectorFocused(frame, selector)) {
|
||||
await frameClick(selector, options);
|
||||
}
|
||||
await sleep(rand(50, 150));
|
||||
await originals.keyboardPress(key);
|
||||
};
|
||||
|
||||
(frame as any).pressSequentially = async (selector: string, text: string, options?: any) => {
|
||||
await (page as any).pressSequentially(selector, text, options);
|
||||
(frame as any).pressSequentially = async (selector: string, text: string, options?: HumanActionOptions) => {
|
||||
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
|
||||
if (!await isFrameSelectorFocused(frame, selector)) {
|
||||
await frameClick(selector, options);
|
||||
}
|
||||
await sleep(rand(100, 250));
|
||||
const cdp = await getFrameCdp();
|
||||
await humanType(page, rawKb, text, callCfg, cdp).catch(() => origFramePressSequentially?.(selector, text, options));
|
||||
};
|
||||
|
||||
(frame as any).tap = async (selector: string, options?: any) => {
|
||||
await (page as any).tap(selector, options);
|
||||
(frame as any).tap = async (selector: string, options?: HumanActionOptions) => {
|
||||
await frameClick(selector, options).catch(() => origFrameTap?.(selector, options));
|
||||
};
|
||||
|
||||
(frame as any).clear = async (selector: string, options?: any) => {
|
||||
if (!await isSelectorFocused(stealth, page, selector)) {
|
||||
await (page as any).click(selector);
|
||||
(frame as any).clear = async (selector: string, options?: HumanActionOptions) => {
|
||||
if (!await isFrameSelectorFocused(frame, selector)) {
|
||||
await frameClick(selector, options);
|
||||
}
|
||||
await sleep(rand(50, 150));
|
||||
await originals.keyboardPress(SELECT_ALL);
|
||||
@@ -599,9 +811,17 @@ function patchSingleFrame(
|
||||
await originals.keyboardPress('Backspace');
|
||||
};
|
||||
|
||||
(frame as any).dragAndDrop = async (source: string, target: string, options?: any) => {
|
||||
const srcBox = await frame.locator(source).boundingBox().catch(() => null);
|
||||
const tgtBox = await frame.locator(target).boundingBox().catch(() => null);
|
||||
(frame as any).dragAndDrop = async (source: string, target: string, options?: {
|
||||
force?: boolean;
|
||||
noWaitAfter?: boolean;
|
||||
sourcePosition?: { x: number; y: number };
|
||||
strict?: boolean;
|
||||
targetPosition?: { x: number; y: number };
|
||||
timeout?: number;
|
||||
trial?: boolean;
|
||||
}) => {
|
||||
const srcBox = await firstFrameLocator(frame, source).boundingBox({ timeout: options?.timeout ?? 30000 }).catch(() => null);
|
||||
const tgtBox = await firstFrameLocator(frame, target).boundingBox({ timeout: options?.timeout ?? 30000 }).catch(() => null);
|
||||
|
||||
if (srcBox && tgtBox) {
|
||||
const sx = srcBox.x + srcBox.width / 2;
|
||||
@@ -670,14 +890,14 @@ export function patchBrowser(browser: Browser, cfg: HumanConfig): void {
|
||||
}
|
||||
|
||||
const origNewContext = browser.newContext.bind(browser);
|
||||
(browser as any).newContext = async (options?: any) => {
|
||||
(browser as any).newContext = async (options?: Parameters<typeof origNewContext>[0]) => {
|
||||
const context = await origNewContext(options);
|
||||
patchContext(context, cfg);
|
||||
return context;
|
||||
};
|
||||
|
||||
const origNewPage = browser.newPage.bind(browser);
|
||||
(browser as any).newPage = async (options?: any) => {
|
||||
(browser as any).newPage = async (options?: Parameters<typeof origNewPage>[0]) => {
|
||||
const page = await origNewPage(options);
|
||||
if (!(page as any)._original) {
|
||||
const ctx = page.context();
|
||||
|
||||
+21
-1
@@ -172,13 +172,33 @@ export async function humanClick(
|
||||
// Human idle / drift
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export async function humanIdle(
|
||||
export function humanIdle(
|
||||
raw: RawMouse,
|
||||
cx: number,
|
||||
cy: number,
|
||||
cfg: HumanConfig,
|
||||
): Promise<void>;
|
||||
export function humanIdle(
|
||||
raw: RawMouse,
|
||||
seconds: number,
|
||||
cx: number,
|
||||
cy: number,
|
||||
cfg: HumanConfig,
|
||||
): Promise<void>;
|
||||
export async function humanIdle(
|
||||
raw: RawMouse,
|
||||
secondsOrCx: number,
|
||||
cxOrCy: number,
|
||||
cyOrCfg: number | HumanConfig,
|
||||
maybeCfg?: HumanConfig,
|
||||
): Promise<void> {
|
||||
const hasExplicitSeconds = maybeCfg !== undefined;
|
||||
const seconds = hasExplicitSeconds
|
||||
? secondsOrCx
|
||||
: rand((cyOrCfg as HumanConfig).idle_between_duration[0], (cyOrCfg as HumanConfig).idle_between_duration[1]);
|
||||
const cx = hasExplicitSeconds ? cxOrCy : secondsOrCx;
|
||||
const cy = hasExplicitSeconds ? (cyOrCfg as number) : cxOrCy;
|
||||
const cfg = hasExplicitSeconds ? maybeCfg! : (cyOrCfg as HumanConfig);
|
||||
const endTime = Date.now() + seconds * 1000;
|
||||
let x = cx;
|
||||
let y = cy;
|
||||
|
||||
+48
-16
@@ -38,26 +38,29 @@ async function smoothWheel(raw: RawMouse, delta: number, cfg: HumanConfig): Prom
|
||||
}
|
||||
}
|
||||
|
||||
export async function scrollToElement(
|
||||
/**
|
||||
* Humanized scrolling that takes an arbitrary ``getBox`` callable.
|
||||
*
|
||||
* Used by both ``scrollToElement`` (selector-based) and the ElementHandle
|
||||
* ``scrollIntoViewIfNeeded`` patch so the same accelerate → cruise →
|
||||
* decelerate → overshoot behavior runs everywhere.
|
||||
*/
|
||||
export async function humanScrollIntoView(
|
||||
page: Page,
|
||||
raw: RawMouse,
|
||||
selector: string,
|
||||
getBox: () => Promise<ElementBounds | null>,
|
||||
cursorX: number,
|
||||
cursorY: number,
|
||||
cfg: HumanConfig,
|
||||
): Promise<{ box: ElementBounds; cursorX: number; cursorY: number }> {
|
||||
): Promise<{ box: ElementBounds; cursorX: number; cursorY: number; didScroll: boolean }> {
|
||||
const viewport = page.viewportSize();
|
||||
if (!viewport) throw new Error('Viewport size not available');
|
||||
|
||||
let box = await getElementBox(page, selector);
|
||||
if (!box) {
|
||||
await sleep(200);
|
||||
box = await getElementBox(page, selector);
|
||||
if (!box) throw new Error(`Element not found: ${selector}`);
|
||||
}
|
||||
let box = await getBox();
|
||||
if (!box) throw new Error('Element not found while scrolling into view');
|
||||
|
||||
if (isInViewport(box, viewport.height, cfg)) {
|
||||
return { box, cursorX, cursorY };
|
||||
return { box, cursorX, cursorY, didScroll: false };
|
||||
}
|
||||
|
||||
// Move cursor into scroll area
|
||||
@@ -107,7 +110,7 @@ export async function scrollToElement(
|
||||
|
||||
// Check visibility every 3 steps
|
||||
if (i % 3 === 2 || i === totalClicks - 1) {
|
||||
box = await getElementBox(page, selector);
|
||||
box = await getBox();
|
||||
if (box && isInViewport(box, viewport.height, cfg)) {
|
||||
break;
|
||||
}
|
||||
@@ -133,16 +136,45 @@ export async function scrollToElement(
|
||||
// Settle
|
||||
await sleep(randRange(cfg.scroll_settle_delay));
|
||||
|
||||
box = await getElementBox(page, selector);
|
||||
if (!box) throw new Error(`Element lost after scrolling: ${selector}`);
|
||||
box = await getBox();
|
||||
if (!box) throw new Error('Element lost after scrolling into view');
|
||||
|
||||
return { box, cursorX, cursorY };
|
||||
return { box, cursorX, cursorY, didScroll: true };
|
||||
}
|
||||
|
||||
async function getElementBox(page: Page, selector: string): Promise<ElementBounds | null> {
|
||||
/**
|
||||
* Selector-based humanized scroll.
|
||||
*
|
||||
* ``timeout`` is forwarded to Playwright's ``boundingBox({ timeout })`` so
|
||||
* callers like ``page.click('#x', { timeout: 5000 })`` can wait longer for
|
||||
* slow-loading elements (#172). Default matches Playwright's 30000ms when not specified.
|
||||
*
|
||||
* Returns `{ box, cursorX, cursorY, didScroll }`.
|
||||
*/
|
||||
export async function scrollToElement(
|
||||
page: Page,
|
||||
raw: RawMouse,
|
||||
selector: string,
|
||||
cursorX: number,
|
||||
cursorY: number,
|
||||
cfg: HumanConfig,
|
||||
timeout?: number,
|
||||
): Promise<{ box: ElementBounds; cursorX: number; cursorY: number; didScroll: boolean }> {
|
||||
return humanScrollIntoView(
|
||||
page, raw,
|
||||
() => getElementBox(page, selector, timeout),
|
||||
cursorX, cursorY, cfg,
|
||||
);
|
||||
}
|
||||
|
||||
async function getElementBox(
|
||||
page: Page,
|
||||
selector: string,
|
||||
timeout: number = 30000,
|
||||
): Promise<ElementBounds | null> {
|
||||
const el = page.locator(selector).first();
|
||||
try {
|
||||
const box = await el.boundingBox({ timeout: 2000 });
|
||||
const box = await el.boundingBox({ timeout: Math.max(1, timeout) });
|
||||
return box;
|
||||
} catch {
|
||||
return null;
|
||||
|
||||
+1
-1
@@ -16,7 +16,7 @@
|
||||
*/
|
||||
|
||||
// Launch functions (Playwright API)
|
||||
export { launch, launchContext, launchPersistentContext } from "./playwright.js";
|
||||
export { launch, launchContext, launchPersistentContext, buildLaunchOptions, humanizeBrowser } from "./playwright.js";
|
||||
|
||||
// Binary management
|
||||
export { ensureBinary, clearCache, binaryInfo, checkForUpdate } from "./download.js";
|
||||
|
||||
+49
-31
@@ -3,7 +3,7 @@
|
||||
* Mirrors Python cloakbrowser/browser.py.
|
||||
*/
|
||||
|
||||
import type { Browser, BrowserContext, BrowserContextOptions } from "playwright-core";
|
||||
import type { Browser, BrowserContext, BrowserContextOptions, LaunchOptions as PlaywrightLaunchOptions } from "playwright-core";
|
||||
import type { LaunchOptions, LaunchContextOptions, LaunchPersistentContextOptions } from "./types.js";
|
||||
import { DEFAULT_VIEWPORT, IGNORE_DEFAULT_ARGS } from "./config.js";
|
||||
import { buildArgs } from "./args.js";
|
||||
@@ -44,6 +44,52 @@ function filterStealthCtxOptions(ctx?: BrowserContextOptions): Partial<BrowserCo
|
||||
return rest;
|
||||
}
|
||||
|
||||
/**
|
||||
* Build Playwright launch options for CloakBrowser without starting Chromium.
|
||||
*
|
||||
* Useful when integrating CloakBrowser with a custom Playwright build or another
|
||||
* wrapper that needs to call `chromium.launch()` itself.
|
||||
*/
|
||||
export async function buildLaunchOptions(
|
||||
options: LaunchOptions = {}
|
||||
): Promise<PlaywrightLaunchOptions> {
|
||||
const binaryPath = process.env.CLOAKBROWSER_BINARY_PATH || (await ensureBinary());
|
||||
const { exitIp, ...resolved } = await maybeResolveGeoip(options);
|
||||
const { proxyOption, proxyArgs } = resolveProxyConfig(options.proxy);
|
||||
let resolvedArgs = await resolveWebrtcArgs(options);
|
||||
if (exitIp && !(resolvedArgs ?? []).some(a => a.startsWith("--fingerprint-webrtc-ip"))) {
|
||||
resolvedArgs = [...(resolvedArgs ?? []), `--fingerprint-webrtc-ip=${exitIp}`];
|
||||
}
|
||||
const args = buildArgs({ ...options, ...resolved, args: [...(resolvedArgs ?? []), ...proxyArgs] });
|
||||
|
||||
return {
|
||||
executablePath: binaryPath,
|
||||
headless: options.headless ?? true,
|
||||
args,
|
||||
ignoreDefaultArgs: IGNORE_DEFAULT_ARGS,
|
||||
...(proxyOption ? { proxy: proxyOption } : {}),
|
||||
...options.launchOptions,
|
||||
} as PlaywrightLaunchOptions;
|
||||
}
|
||||
|
||||
/**
|
||||
* Apply CloakBrowser's human-like behavioral layer to an existing Playwright browser.
|
||||
*/
|
||||
export async function humanizeBrowser(
|
||||
browser: Browser,
|
||||
options: LaunchOptions = {}
|
||||
): Promise<void> {
|
||||
if (!options.humanize) return;
|
||||
|
||||
const { patchBrowser } = await import('./human/index.js');
|
||||
const { resolveConfig } = await import('./human/config.js');
|
||||
const cfg = resolveConfig(
|
||||
options.humanPreset ?? 'default',
|
||||
options.humanConfig,
|
||||
);
|
||||
patchBrowser(browser, cfg);
|
||||
}
|
||||
|
||||
/**
|
||||
* Launch stealth Chromium browser via Playwright.
|
||||
*
|
||||
@@ -59,36 +105,8 @@ function filterStealthCtxOptions(ctx?: BrowserContextOptions): Partial<BrowserCo
|
||||
*/
|
||||
export async function launch(options: LaunchOptions = {}): Promise<Browser> {
|
||||
const { chromium } = await import("playwright-core");
|
||||
|
||||
const binaryPath = process.env.CLOAKBROWSER_BINARY_PATH || (await ensureBinary());
|
||||
const { exitIp, ...resolved } = await maybeResolveGeoip(options);
|
||||
const { proxyOption, proxyArgs } = resolveProxyConfig(options.proxy);
|
||||
let resolvedArgs = await resolveWebrtcArgs(options);
|
||||
if (exitIp && !(resolvedArgs ?? []).some(a => a.startsWith("--fingerprint-webrtc-ip"))) {
|
||||
resolvedArgs = [...(resolvedArgs ?? []), `--fingerprint-webrtc-ip=${exitIp}`];
|
||||
}
|
||||
const args = buildArgs({ ...options, ...resolved, args: [...(resolvedArgs ?? []), ...proxyArgs] });
|
||||
|
||||
const browser = await chromium.launch({
|
||||
executablePath: binaryPath,
|
||||
headless: options.headless ?? true,
|
||||
args,
|
||||
ignoreDefaultArgs: IGNORE_DEFAULT_ARGS,
|
||||
...(proxyOption ? { proxy: proxyOption } : {}),
|
||||
...options.launchOptions,
|
||||
});
|
||||
|
||||
// Human-like behavioral patching
|
||||
if (options.humanize) {
|
||||
const { patchBrowser } = await import('./human/index.js');
|
||||
const { resolveConfig } = await import('./human/config.js');
|
||||
const cfg = resolveConfig(
|
||||
options.humanPreset ?? 'default',
|
||||
options.humanConfig,
|
||||
);
|
||||
patchBrowser(browser, cfg);
|
||||
}
|
||||
|
||||
const browser = await chromium.launch(await buildLaunchOptions(options));
|
||||
await humanizeBrowser(browser, options);
|
||||
return browser;
|
||||
}
|
||||
|
||||
|
||||
+13
-1
@@ -136,7 +136,19 @@ export function normalizeSocksStringUrl(urlStr: string): string {
|
||||
const encPass = hasPassword
|
||||
? (rawPassEnc ? encodeURIComponent(lenientDecodeURIComponent(rawPassEnc)) : "")
|
||||
: null;
|
||||
return assembleSocksUrl(scheme, encUser, encPass, hostAndRest);
|
||||
const normalized = assembleSocksUrl(scheme, encUser, encPass, hostAndRest);
|
||||
// Compare credentials, not the full URL: keeps the log condition focused
|
||||
// on real encoding work, not cosmetic differences (parity with the Python
|
||||
// implementation, which has to skip urlparse's hostname lowercasing).
|
||||
const credsChanged = encUser !== rawUserEnc
|
||||
|| (hasPassword ? encPass !== rawPassEnc : false);
|
||||
if (credsChanged) {
|
||||
console.info(
|
||||
"[cloakbrowser] Auto URL-encoded SOCKS5 proxy credentials (special " +
|
||||
"characters detected). Pre-encode the URL to suppress this notice.",
|
||||
);
|
||||
}
|
||||
return normalized;
|
||||
} catch (e) {
|
||||
console.warn(`[cloakbrowser] Could not normalize SOCKS5 proxy URL, passing through unchanged: ${(e as Error).message}`);
|
||||
return urlStr;
|
||||
|
||||
+106
-58
@@ -12,71 +12,55 @@ import { ensureBinary } from "./download.js";
|
||||
import { isSocksProxy, parseProxyUrl, resolveProxyConfig } from "./proxy.js";
|
||||
import { maybeResolveGeoip, resolveWebrtcArgs } from "./geoip.js";
|
||||
|
||||
/**
|
||||
* Launch stealth Chromium browser via Puppeteer.
|
||||
*
|
||||
* @example
|
||||
* ```ts
|
||||
* import { launch } from 'cloakbrowser/puppeteer';
|
||||
* * // With humanize — human-like mouse, keyboard, scroll
|
||||
* const browser = await launch({ humanize: true });
|
||||
* const page = await browser.newPage();
|
||||
* await page.goto('[https://example.com](https://example.com)');
|
||||
* await page.click('#login'); // Bézier curve mouse movement
|
||||
* await page.type('#email', 'user@example.com'); // Per-character timing
|
||||
* ```
|
||||
*/
|
||||
export async function launch(options: LaunchOptions = {}): Promise<Browser> {
|
||||
const puppeteer = await import("puppeteer-core");
|
||||
|
||||
/** Resolve binary path, geoip, webrtc, and build final Chrome args. */
|
||||
async function resolveArgs(options: LaunchOptions): Promise<{ binaryPath: string; args: string[] }> {
|
||||
const binaryPath = process.env.CLOAKBROWSER_BINARY_PATH || (await ensureBinary());
|
||||
const { exitIp, ...resolved } = (await maybeResolveGeoip(options)) ?? {};
|
||||
let resolvedArgs = (await resolveWebrtcArgs(options)) ?? options.args;
|
||||
|
||||
|
||||
if (exitIp && !(resolvedArgs ?? []).some(a => a.startsWith("--fingerprint-webrtc-ip"))) {
|
||||
resolvedArgs = [...(resolvedArgs ?? []), `--fingerprint-webrtc-ip=${exitIp}`];
|
||||
}
|
||||
const args = buildArgs({ ...options, ...resolved, args: resolvedArgs });
|
||||
return { binaryPath, args: buildArgs({ ...options, ...resolved, args: resolvedArgs }) };
|
||||
}
|
||||
|
||||
// Puppeteer handles proxy via CLI args, not a separate option.
|
||||
// SOCKS5: Chrome supports inline credentials natively (RFC 1929 auth).
|
||||
// HTTP: Chrome does NOT support inline credentials — strip them and
|
||||
// use page.authenticate() for Proxy-Authorization headers instead.
|
||||
let proxyAuth: { username: string; password: string } | undefined;
|
||||
if (options.proxy) {
|
||||
if (isSocksProxy(options.proxy)) {
|
||||
// SOCKS5: pass full URL with credentials to Chrome directly
|
||||
const { proxyArgs } = resolveProxyConfig(options.proxy);
|
||||
args.push(...proxyArgs);
|
||||
} else if (typeof options.proxy === "string") {
|
||||
const { server, username, password } = parseProxyUrl(options.proxy);
|
||||
args.push(`--proxy-server=${server}`);
|
||||
if (username) {
|
||||
proxyAuth = { username, password: password ?? "" };
|
||||
}
|
||||
} else {
|
||||
const parsed = parseProxyUrl(options.proxy.server);
|
||||
args.push(`--proxy-server=${parsed.server}`);
|
||||
if (options.proxy.bypass) {
|
||||
args.push(`--proxy-bypass-list=${options.proxy.bypass}`);
|
||||
}
|
||||
const username = options.proxy.username ?? parsed.username;
|
||||
const password = options.proxy.password ?? parsed.password;
|
||||
if (username) {
|
||||
proxyAuth = { username, password: password ?? "" };
|
||||
}
|
||||
}
|
||||
/**
|
||||
* Resolve proxy into Chrome CLI args and optional HTTP auth credentials.
|
||||
* SOCKS5: Chrome supports inline credentials natively (RFC 1929 auth).
|
||||
* HTTP: Chrome does NOT support inline credentials — strip them and
|
||||
* use page.authenticate() for Proxy-Authorization headers instead.
|
||||
*/
|
||||
function resolveProxy(options: LaunchOptions, args: string[]): { username: string; password: string } | undefined {
|
||||
if (!options.proxy) return undefined;
|
||||
|
||||
if (isSocksProxy(options.proxy)) {
|
||||
const { proxyArgs } = resolveProxyConfig(options.proxy);
|
||||
args.push(...proxyArgs);
|
||||
return undefined;
|
||||
}
|
||||
|
||||
const browser = await puppeteer.default.launch({
|
||||
executablePath: binaryPath,
|
||||
headless: options.headless ?? true,
|
||||
args,
|
||||
ignoreDefaultArgs: IGNORE_DEFAULT_ARGS,
|
||||
...options.launchOptions,
|
||||
});
|
||||
if (typeof options.proxy === "string") {
|
||||
const { server, username, password } = parseProxyUrl(options.proxy);
|
||||
args.push(`--proxy-server=${server}`);
|
||||
return username ? { username, password: password ?? "" } : undefined;
|
||||
}
|
||||
|
||||
// Monkey-patch newPage() to auto-authenticate proxy credentials
|
||||
const parsed = parseProxyUrl(options.proxy.server);
|
||||
args.push(`--proxy-server=${parsed.server}`);
|
||||
if (options.proxy.bypass) {
|
||||
args.push(`--proxy-bypass-list=${options.proxy.bypass}`);
|
||||
}
|
||||
const username = options.proxy.username ?? parsed.username;
|
||||
const password = options.proxy.password ?? parsed.password;
|
||||
return username ? { username, password: password ?? "" } : undefined;
|
||||
}
|
||||
|
||||
/** Apply proxy auth monkey-patch and humanize behavioral patching. */
|
||||
async function applyPostLaunch(
|
||||
browser: Browser,
|
||||
options: LaunchOptions,
|
||||
proxyAuth?: { username: string; password: string },
|
||||
): Promise<void> {
|
||||
if (proxyAuth) {
|
||||
const origNewPage = browser.newPage.bind(browser);
|
||||
const auth = proxyAuth;
|
||||
@@ -87,9 +71,6 @@ export async function launch(options: LaunchOptions = {}): Promise<Browser> {
|
||||
};
|
||||
}
|
||||
|
||||
// Human-like behavioral patching — FULL coverage, same as Playwright.
|
||||
// This enables Bézier mouse movements, organic typing rhythms, and
|
||||
// natural scrolling to bypass advanced anti-bot detection.
|
||||
if (options.humanize) {
|
||||
const { patchBrowser } = await import('./human-puppeteer/index.js');
|
||||
const { resolveConfig } = await import('./human/config.js');
|
||||
@@ -99,6 +80,73 @@ export async function launch(options: LaunchOptions = {}): Promise<Browser> {
|
||||
);
|
||||
patchBrowser(browser, cfg);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Launch stealth Chromium browser via Puppeteer.
|
||||
*
|
||||
* @example
|
||||
* ```ts
|
||||
* import { launch } from 'cloakbrowser/puppeteer';
|
||||
* // With humanize — human-like mouse, keyboard, scroll
|
||||
* const browser = await launch({ humanize: true });
|
||||
* const page = await browser.newPage();
|
||||
* await page.goto('https://example.com');
|
||||
* await page.click('#login'); // Bézier curve mouse movement
|
||||
* await page.type('#email', 'user@example.com'); // Per-character timing
|
||||
* ```
|
||||
*/
|
||||
export async function launch(options: LaunchOptions = {}): Promise<Browser> {
|
||||
const puppeteer = await import("puppeteer-core");
|
||||
const { binaryPath, args } = await resolveArgs(options);
|
||||
const proxyAuth = resolveProxy(options, args);
|
||||
|
||||
const browser = await puppeteer.default.launch({
|
||||
...options.launchOptions,
|
||||
executablePath: binaryPath,
|
||||
headless: options.headless ?? true,
|
||||
args,
|
||||
ignoreDefaultArgs: IGNORE_DEFAULT_ARGS,
|
||||
});
|
||||
|
||||
await applyPostLaunch(browser, options, proxyAuth);
|
||||
return browser;
|
||||
}
|
||||
|
||||
/**
|
||||
* Launch stealth Chromium with a persistent user profile via Puppeteer.
|
||||
* Passes `userDataDir` to Puppeteer's launch options so cookies,
|
||||
* localStorage, and session data persist across launches.
|
||||
*
|
||||
* @example
|
||||
* ```ts
|
||||
* import { launchPersistentContext } from 'cloakbrowser/puppeteer';
|
||||
* const browser = await launchPersistentContext({
|
||||
* userDataDir: './chrome-profile',
|
||||
* headless: false,
|
||||
* proxy: 'http://user:pass@proxy:8080',
|
||||
* });
|
||||
* const page = await browser.newPage();
|
||||
* await page.goto('https://example.com');
|
||||
* await browser.close();
|
||||
* ```
|
||||
*/
|
||||
export async function launchPersistentContext(
|
||||
options: LaunchOptions & { userDataDir: string }
|
||||
): Promise<Browser> {
|
||||
const puppeteer = await import("puppeteer-core");
|
||||
const { binaryPath, args } = await resolveArgs(options);
|
||||
const proxyAuth = resolveProxy(options, args);
|
||||
|
||||
const browser = await puppeteer.default.launch({
|
||||
...options.launchOptions,
|
||||
executablePath: binaryPath,
|
||||
headless: options.headless ?? true,
|
||||
args,
|
||||
ignoreDefaultArgs: IGNORE_DEFAULT_ARGS,
|
||||
userDataDir: options.userDataDir,
|
||||
});
|
||||
|
||||
await applyPostLaunch(browser, options, proxyAuth);
|
||||
return browser;
|
||||
}
|
||||
|
||||
@@ -17,6 +17,8 @@ export interface LaunchOptions {
|
||||
proxy?: string | { server: string; bypass?: string; username?: string; password?: string };
|
||||
/** Additional Chromium CLI arguments. */
|
||||
args?: string[];
|
||||
/** Chrome extension paths to load. */
|
||||
extensionPaths?: string[];
|
||||
/** Include default stealth fingerprint args (default: true). Set false to use custom --fingerprint flags. */
|
||||
stealthArgs?: boolean;
|
||||
/** IANA timezone, e.g. "America/New_York". Sets --fingerprint-timezone binary flag. */
|
||||
|
||||
@@ -0,0 +1,17 @@
|
||||
import { test, expect } from "vitest";
|
||||
import path from "path";
|
||||
import { _buildArgsForTest } from "../src/playwright.js";
|
||||
|
||||
test("extension paths inject chrome flags", () => {
|
||||
const args = _buildArgsForTest({
|
||||
extensionPaths: ["./ext"],
|
||||
});
|
||||
|
||||
const abs = path.resolve("./ext");
|
||||
|
||||
expect(args).toContain(`--load-extension=${abs}`);
|
||||
|
||||
expect(args).toContain(
|
||||
`--disable-extensions-except=${abs}`
|
||||
);
|
||||
});
|
||||
+59
-2
@@ -1,5 +1,17 @@
|
||||
import { describe, it, expect } from "vitest";
|
||||
import { COUNTRY_LOCALE_MAP, resolveProxyIp } from "../src/geoip.js";
|
||||
import { describe, it, expect, afterEach, vi } from "vitest";
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { COUNTRY_LOCALE_MAP, maybeResolveGeoip, resolveProxyGeo, resolveProxyIp } from "../src/geoip.js";
|
||||
|
||||
const tempDirs: string[] = [];
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
delete process.env.CLOAKBROWSER_GEOIP_TIMEOUT_SECONDS;
|
||||
delete process.env.CLOAKBROWSER_CACHE_DIR;
|
||||
for (const dir of tempDirs.splice(0)) fs.rmSync(dir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
describe("resolveProxyIp", () => {
|
||||
it("returns literal IPv4 from proxy URL", async () => {
|
||||
@@ -38,6 +50,51 @@ describe("resolveProxyIp", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("maybeResolveGeoip", () => {
|
||||
it("does not apply the GeoIP resolution timeout to first-use database download", async () => {
|
||||
const cacheDir = fs.mkdtempSync(path.join(os.tmpdir(), "cloak-geoip-download-"));
|
||||
tempDirs.push(cacheDir);
|
||||
process.env.CLOAKBROWSER_CACHE_DIR = cacheDir;
|
||||
process.env.CLOAKBROWSER_GEOIP_TIMEOUT_SECONDS = "0.001";
|
||||
|
||||
const fetchSpy = vi.spyOn(globalThis, "fetch").mockResolvedValue({
|
||||
ok: true,
|
||||
body: new ReadableStream({
|
||||
start(controller) {
|
||||
controller.enqueue(new Uint8Array([1, 2, 3]));
|
||||
controller.close();
|
||||
},
|
||||
}),
|
||||
} as Response);
|
||||
|
||||
const result = await resolveProxyGeo("http://203.0.113.10:8080");
|
||||
|
||||
expect(result).toEqual({ timezone: null, locale: null, exitIp: null });
|
||||
expect(fetchSpy).toHaveBeenCalledOnce();
|
||||
expect(fetchSpy.mock.calls[0][1]).toEqual({ redirect: "follow" });
|
||||
});
|
||||
|
||||
it("returns quickly when GeoIP resolution times out", async () => {
|
||||
const cacheDir = fs.mkdtempSync(path.join(os.tmpdir(), "cloak-geoip-timeout-"));
|
||||
tempDirs.push(cacheDir);
|
||||
process.env.CLOAKBROWSER_CACHE_DIR = cacheDir;
|
||||
process.env.CLOAKBROWSER_GEOIP_TIMEOUT_SECONDS = "0.025";
|
||||
|
||||
const start = performance.now();
|
||||
const result = await maybeResolveGeoip({
|
||||
geoip: true,
|
||||
proxy: "http://203.0.113.10:8080",
|
||||
timezone: "Europe/Paris",
|
||||
locale: "fr-FR",
|
||||
});
|
||||
const elapsed = performance.now() - start;
|
||||
|
||||
expect(result).toEqual({ timezone: "Europe/Paris", locale: "fr-FR", exitIp: undefined });
|
||||
expect(elapsed).toBeLessThan(500);
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
describe("COUNTRY_LOCALE_MAP", () => {
|
||||
it("contains common countries", () => {
|
||||
for (const code of ["US", "GB", "DE", "FR", "JP", "BR", "IL", "RU"]) {
|
||||
|
||||
+593
-166
File diff suppressed because it is too large
Load Diff
@@ -21,6 +21,70 @@ describe("binaryInfo", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("composable Playwright launch helpers", () => {
|
||||
const origBinaryPath = process.env.CLOAKBROWSER_BINARY_PATH;
|
||||
|
||||
beforeEach(() => {
|
||||
process.env.CLOAKBROWSER_BINARY_PATH = "/fake/chrome";
|
||||
vi.resetModules();
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
vi.resetModules();
|
||||
if (origBinaryPath) {
|
||||
process.env.CLOAKBROWSER_BINARY_PATH = origBinaryPath;
|
||||
} else {
|
||||
delete process.env.CLOAKBROWSER_BINARY_PATH;
|
||||
}
|
||||
});
|
||||
|
||||
it("exports buildLaunchOptions and humanizeBrowser from the package entrypoint", async () => {
|
||||
const entry = await import("../src/index.js");
|
||||
|
||||
expect(entry.buildLaunchOptions).toBeTypeOf("function");
|
||||
expect(entry.humanizeBrowser).toBeTypeOf("function");
|
||||
});
|
||||
|
||||
it("buildLaunchOptions returns Playwright options without launching a browser", async () => {
|
||||
const { buildLaunchOptions } = await import("../src/index.js");
|
||||
|
||||
const options = await buildLaunchOptions({
|
||||
headless: false,
|
||||
proxy: "http://user:pass@proxy.example:8080",
|
||||
args: ["--custom-flag"],
|
||||
launchOptions: { timeout: 1234 },
|
||||
});
|
||||
|
||||
expect(options.executablePath).toBe("/fake/chrome");
|
||||
expect(options.headless).toBe(false);
|
||||
expect(options.args).toContain("--custom-flag");
|
||||
expect(options.ignoreDefaultArgs).toContain("--enable-automation");
|
||||
expect(options.proxy).toEqual({
|
||||
server: "http://proxy.example:8080",
|
||||
username: "user",
|
||||
password: "pass",
|
||||
});
|
||||
expect(options.timeout).toBe(1234);
|
||||
});
|
||||
|
||||
it("humanizeBrowser patches an existing browser only when requested", async () => {
|
||||
const { humanizeBrowser } = await import("../src/index.js");
|
||||
const browser = {
|
||||
contexts: () => [],
|
||||
newContext: vi.fn(async () => ({})),
|
||||
newPage: vi.fn(async () => ({ context: () => ({}) })),
|
||||
};
|
||||
const originalNewContext = browser.newContext;
|
||||
|
||||
await humanizeBrowser(browser as any, { humanize: false });
|
||||
expect(browser.newContext).toBe(originalNewContext);
|
||||
|
||||
await humanizeBrowser(browser as any, { humanize: true });
|
||||
expect(browser.newContext).not.toBe(originalNewContext);
|
||||
});
|
||||
});
|
||||
|
||||
// Integration tests require the binary — run with:
|
||||
// CLOAKBROWSER_BINARY_PATH=/path/to/chrome npm test
|
||||
describe.skipIf(!process.env.CLOAKBROWSER_BINARY_PATH)(
|
||||
|
||||
+60
-1
@@ -1,4 +1,4 @@
|
||||
import { describe, it, expect } from "vitest";
|
||||
import { describe, it, expect, vi } from "vitest";
|
||||
import { parseProxyUrl, isSocksProxy, resolveProxyConfig } from "../src/proxy.js";
|
||||
import type { LaunchOptions } from "../src/types.js";
|
||||
|
||||
@@ -262,4 +262,63 @@ describe("resolveProxyConfig", () => {
|
||||
const { proxyArgs } = resolveProxyConfig("socks5://user:a@b@c@host:1080");
|
||||
expect(proxyArgs).toEqual(["--proxy-server=socks5://user:a%40b%40c@host:1080"]);
|
||||
});
|
||||
|
||||
// Visibility for #157: when wrapper actually rewrites the URL, surface an
|
||||
// info log so users debugging silent SOCKS5 fallback can see what happened.
|
||||
it("logs info message when SOCKS5 credentials get re-encoded", () => {
|
||||
const debugSpy = vi.spyOn(console, "info").mockImplementation(() => {});
|
||||
try {
|
||||
resolveProxyConfig("socks5://user:pass=123@host:1080");
|
||||
expect(debugSpy).toHaveBeenCalledWith(
|
||||
expect.stringContaining("Auto URL-encoded SOCKS5"),
|
||||
);
|
||||
// Credentials must not leak into the log.
|
||||
const calls = debugSpy.mock.calls.flat().join(" ");
|
||||
expect(calls).not.toContain("pass=123");
|
||||
expect(calls).not.toContain("pass%3D123");
|
||||
} finally {
|
||||
debugSpy.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
it("stays silent when SOCKS5 URL is already encoded (no log spam)", () => {
|
||||
const debugSpy = vi.spyOn(console, "info").mockImplementation(() => {});
|
||||
try {
|
||||
resolveProxyConfig("socks5://user:pass%3D123@host:1080");
|
||||
const reencodedCalls = debugSpy.mock.calls
|
||||
.flat()
|
||||
.filter((arg) => typeof arg === "string" && arg.includes("Auto URL-encoded SOCKS5"));
|
||||
expect(reencodedCalls).toHaveLength(0);
|
||||
} finally {
|
||||
debugSpy.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
it("stays silent when SOCKS5 URL has no credentials", () => {
|
||||
const debugSpy = vi.spyOn(console, "info").mockImplementation(() => {});
|
||||
try {
|
||||
resolveProxyConfig("socks5://host:1080");
|
||||
const reencodedCalls = debugSpy.mock.calls
|
||||
.flat()
|
||||
.filter((arg) => typeof arg === "string" && arg.includes("Auto URL-encoded SOCKS5"));
|
||||
expect(reencodedCalls).toHaveLength(0);
|
||||
} finally {
|
||||
debugSpy.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
it("stays silent when only host case differs (no credential rewrite)", () => {
|
||||
// Parity with Python: log condition must track credential changes, not
|
||||
// cosmetic URL-string differences (regression for Copilot's PR #209 review).
|
||||
const debugSpy = vi.spyOn(console, "info").mockImplementation(() => {});
|
||||
try {
|
||||
resolveProxyConfig("socks5://USER:pass@HOST.com:1080");
|
||||
const reencodedCalls = debugSpy.mock.calls
|
||||
.flat()
|
||||
.filter((arg) => typeof arg === "string" && arg.includes("Auto URL-encoded SOCKS5"));
|
||||
expect(reencodedCalls).toHaveLength(0);
|
||||
} finally {
|
||||
debugSpy.mockRestore();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
@@ -126,6 +126,14 @@ describe("puppeteer launch", () => {
|
||||
expect(page.authenticate).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("forwards launchOptions to puppeteer launch", async () => {
|
||||
const { launch } = await import("../src/puppeteer.js");
|
||||
await launch({ launchOptions: { slowMo: 50 } });
|
||||
|
||||
const callArgs = vi.mocked(puppeteerMock.default.launch).mock.calls[0][0];
|
||||
expect(callArgs.slowMo).toBe(50);
|
||||
});
|
||||
|
||||
it("reconstructs SOCKS5 dict with auth into --proxy-server URL", async () => {
|
||||
const { launch } = await import("../src/puppeteer.js");
|
||||
const browser = await launch({
|
||||
@@ -139,3 +147,95 @@ describe("puppeteer launch", () => {
|
||||
expect(page.authenticate).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
describe("puppeteer launchPersistentContext", () => {
|
||||
let puppeteerMock: any;
|
||||
let mockBrowser: any;
|
||||
|
||||
beforeEach(async () => {
|
||||
delete process.env.CLOAKBROWSER_BINARY_PATH;
|
||||
puppeteerMock = await import("puppeteer-core");
|
||||
mockBrowser = {
|
||||
newPage: vi.fn().mockResolvedValue({
|
||||
authenticate: vi.fn(),
|
||||
}),
|
||||
close: vi.fn(),
|
||||
};
|
||||
vi.mocked(puppeteerMock.default.launch).mockResolvedValue(mockBrowser);
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
it("passes userDataDir to puppeteer launch", async () => {
|
||||
process.env.CLOAKBROWSER_BINARY_PATH = "/fake/chrome";
|
||||
const { launchPersistentContext } = await import("../src/puppeteer.js");
|
||||
await launchPersistentContext({ userDataDir: "./my-profile" });
|
||||
|
||||
expect(puppeteerMock.default.launch).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
userDataDir: "./my-profile",
|
||||
executablePath: "/fake/chrome",
|
||||
})
|
||||
);
|
||||
});
|
||||
|
||||
it("includes stealth args", async () => {
|
||||
const { launchPersistentContext } = await import("../src/puppeteer.js");
|
||||
await launchPersistentContext({ userDataDir: "./my-profile" });
|
||||
|
||||
const callArgs = vi.mocked(puppeteerMock.default.launch).mock.calls[0][0];
|
||||
expect(callArgs.args.some((a: string) => a.startsWith("--fingerprint="))).toBe(true);
|
||||
});
|
||||
|
||||
it("handles proxy auth with persistent context", async () => {
|
||||
const { launchPersistentContext } = await import("../src/puppeteer.js");
|
||||
const browser = await launchPersistentContext({
|
||||
userDataDir: "./my-profile",
|
||||
proxy: "http://user:pass@proxy:8080",
|
||||
});
|
||||
|
||||
const page = await browser.newPage();
|
||||
expect(page.authenticate).toHaveBeenCalledWith({
|
||||
username: "user",
|
||||
password: "pass",
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps SOCKS5 credentials in --proxy-server URL", async () => {
|
||||
const { launchPersistentContext } = await import("../src/puppeteer.js");
|
||||
const browser = await launchPersistentContext({
|
||||
userDataDir: "./my-profile",
|
||||
proxy: "socks5://user:pass@proxy:1080",
|
||||
});
|
||||
|
||||
const callArgs = vi.mocked(puppeteerMock.default.launch).mock.calls[0][0];
|
||||
expect(callArgs.args).toContain("--proxy-server=socks5://user:pass@proxy:1080");
|
||||
|
||||
const page = await browser.newPage();
|
||||
expect(page.authenticate).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("forwards launchOptions to puppeteer launch", async () => {
|
||||
const { launchPersistentContext } = await import("../src/puppeteer.js");
|
||||
await launchPersistentContext({ userDataDir: "./my-profile", launchOptions: { slowMo: 50 } });
|
||||
|
||||
const callArgs = vi.mocked(puppeteerMock.default.launch).mock.calls[0][0];
|
||||
expect(callArgs.slowMo).toBe(50);
|
||||
expect(callArgs.userDataDir).toBe("./my-profile");
|
||||
});
|
||||
|
||||
it("injects timezone and locale as binary flags", async () => {
|
||||
const { launchPersistentContext } = await import("../src/puppeteer.js");
|
||||
await launchPersistentContext({
|
||||
userDataDir: "./my-profile",
|
||||
timezone: "Asia/Tokyo",
|
||||
locale: "ja-JP",
|
||||
});
|
||||
|
||||
const callArgs = vi.mocked(puppeteerMock.default.launch).mock.calls[0][0];
|
||||
expect(callArgs.args).toContain("--fingerprint-timezone=Asia/Tokyo");
|
||||
expect(callArgs.args).toContain("--lang=ja-JP");
|
||||
});
|
||||
});
|
||||
|
||||
@@ -532,6 +532,52 @@ describe("Puppeteer: non-ASCII text avoids CDP shift path", () => {
|
||||
});
|
||||
|
||||
|
||||
// =========================================================================
|
||||
// Per-call human config override (Puppeteer page-level)
|
||||
// =========================================================================
|
||||
describe("Puppeteer: page.type accepts per-call human config override", () => {
|
||||
it("page.type forwards merged config to humanType", async () => {
|
||||
const keyboardMod = await import("../src/human-puppeteer/keyboard.js");
|
||||
const scrollMod = await import("../src/human-puppeteer/scroll.js");
|
||||
|
||||
const cfg = resolveConfig("default", {
|
||||
idle_between_actions: false,
|
||||
field_switch_delay: [0, 1],
|
||||
});
|
||||
expect(cfg.typing_delay).toBe(70);
|
||||
|
||||
let captured: any = null;
|
||||
const typeSpy = vi.spyOn(keyboardMod, "humanType").mockImplementation(
|
||||
async (_page, _raw, _text, callCfg) => { captured = callCfg; },
|
||||
);
|
||||
const scrollSpy = vi.spyOn(scrollMod, "scrollToElement").mockImplementation(
|
||||
async (_page, _raw, _sel, cx, cy) => ({
|
||||
box: { x: 100, y: 100, width: 50, height: 30 },
|
||||
cursorX: cx,
|
||||
cursorY: cy,
|
||||
}),
|
||||
);
|
||||
|
||||
const { patchPage } = await import("../src/human-puppeteer/index.js");
|
||||
const page = buildMockPage();
|
||||
const cursor = { x: 100, y: 100, initialized: true };
|
||||
patchPage(page as any, cfg, cursor as any);
|
||||
|
||||
await (page as any).type("#email", "hi", {
|
||||
typing_delay: 30,
|
||||
mistype_chance: 0,
|
||||
});
|
||||
|
||||
expect(captured.typing_delay).toBe(30);
|
||||
expect(captured.mistype_chance).toBe(0);
|
||||
expect(cfg.typing_delay).toBe(70);
|
||||
|
||||
typeSpy.mockRestore();
|
||||
scrollSpy.mockRestore();
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
// =========================================================================
|
||||
// patchPage stealth infrastructure (Puppeteer)
|
||||
// =========================================================================
|
||||
@@ -1671,11 +1717,13 @@ describe("Puppeteer: isInputElement stealth integration via patchPage", () => {
|
||||
}),
|
||||
});
|
||||
|
||||
const mockEl = buildMockElementHandle();
|
||||
const page = buildMockPage({
|
||||
evaluate: vi.fn(async (...args: any[]) => {
|
||||
evaluateCalls.push(args);
|
||||
return false;
|
||||
}),
|
||||
$: vi.fn(async () => mockEl),
|
||||
});
|
||||
page.createCDPSession = vi.fn(async () => mockCdp);
|
||||
|
||||
|
||||
@@ -44,9 +44,14 @@ function buildMockPage(overrides: Record<string, any> = {}): any {
|
||||
|
||||
const makeLocator = () => {
|
||||
const loc: any = {
|
||||
boundingBox: vi.fn(async () => ({ x: 100, y: 100, width: 200, height: 30 })),
|
||||
boundingBox: vi.fn(async () => ({ x: 100, y: 300, width: 200, height: 30 })),
|
||||
scrollIntoViewIfNeeded: vi.fn(async () => {}),
|
||||
isChecked: overrides.isChecked ?? vi.fn(async () => false),
|
||||
waitFor: vi.fn(async () => {}),
|
||||
isVisible: vi.fn(async () => true),
|
||||
isEnabled: vi.fn(async () => true),
|
||||
isEditable: vi.fn(async () => true),
|
||||
evaluate: vi.fn(async () => ({ hit: true })),
|
||||
};
|
||||
loc.first = vi.fn(() => loc);
|
||||
return loc;
|
||||
@@ -687,6 +692,9 @@ describe("isInputElement stealth integration via patchPage", () => {
|
||||
}
|
||||
if (method === "Runtime.evaluate") {
|
||||
stealthEvaluateCalls.push(params.expression);
|
||||
if (params.expression.includes("elementFromPoint")) {
|
||||
return { result: { value: { hit: true } } };
|
||||
}
|
||||
return { result: { value: false } }; // not an input
|
||||
}
|
||||
return {};
|
||||
@@ -696,7 +704,7 @@ describe("isInputElement stealth integration via patchPage", () => {
|
||||
const page = buildMockPage({
|
||||
evaluate: vi.fn(async (...args: any[]) => {
|
||||
evaluateCalls.push(args);
|
||||
return false;
|
||||
return { hit: true };
|
||||
}),
|
||||
});
|
||||
page.context = vi.fn(() => ({
|
||||
|
||||
+184
-2
@@ -1,9 +1,11 @@
|
||||
"""Unit tests for cloakserve — parse_connection_params, parse_cli_args, URL rewriting, connection tracking."""
|
||||
|
||||
import asyncio
|
||||
import importlib.machinery
|
||||
import importlib.util
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
@@ -22,6 +24,8 @@ parse_connection_params = _mod.parse_connection_params
|
||||
parse_cli_args = _mod.parse_cli_args
|
||||
ChromePool = _mod.ChromePool
|
||||
_default_data_dir = _mod._default_data_dir
|
||||
SAFE_SEED_RE = _mod.SAFE_SEED_RE
|
||||
RESERVED_SEEDS = _mod.RESERVED_SEEDS
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -139,8 +143,94 @@ class TestParseCliArgs:
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestURLRewriting:
|
||||
"""Test the URL rewriting logic used by /json/version and /json/list."""
|
||||
class TestWebSocketOriginGuard:
|
||||
"""Verify cloakserve rejects browser-origin CDP WebSocket hijacks."""
|
||||
|
||||
def test_absent_origin_allowed_for_non_browser_cdp_clients(self):
|
||||
assert _mod._origin_is_allowed(None, "127.0.0.1:9555")
|
||||
|
||||
def test_matching_origin_host_allowed(self):
|
||||
assert _mod._origin_is_allowed("http://127.0.0.1:9555", "127.0.0.1:9555")
|
||||
|
||||
def test_chrome_devtools_origin_allowed(self):
|
||||
assert _mod._origin_is_allowed("devtools://devtools", "127.0.0.1:9555")
|
||||
assert _mod._origin_is_allowed("chrome-devtools://devtools", "127.0.0.1:9555")
|
||||
|
||||
@pytest.mark.parametrize("origin", [
|
||||
"http://attacker.example",
|
||||
"https://attacker.example",
|
||||
"http://PUBLIC_HOST:9555",
|
||||
"http://attacker.example:9555",
|
||||
"http://127.0.0.1:9555/",
|
||||
"http://127.0.0.1:9555/path",
|
||||
"http://127.0.0.1:9555?q=1",
|
||||
"http://127.0.0.1:9555#fragment",
|
||||
"http://user@127.0.0.1:9555",
|
||||
"http://@127.0.0.1:9555",
|
||||
"http://:@127.0.0.1:9555",
|
||||
"http://127.0.0.1:",
|
||||
"null",
|
||||
"file://",
|
||||
])
|
||||
def test_untrusted_browser_origins_rejected(self, origin):
|
||||
assert not _mod._origin_is_allowed(origin, "127.0.0.1:9555")
|
||||
|
||||
def test_public_origin_matching_host_is_still_rejected(self):
|
||||
assert not _mod._origin_is_allowed("http://attacker.example:9555", "attacker.example:9555")
|
||||
|
||||
@pytest.mark.parametrize("host", [
|
||||
"user@127.0.0.1:9555",
|
||||
"127.0.0.1:9555/path",
|
||||
"127.0.0.1:9555?x=1",
|
||||
"127.0.0.1:9555#fragment",
|
||||
"127.0.0.1:9555, attacker.example:9555",
|
||||
"@127.0.0.1:9555",
|
||||
":@127.0.0.1:9555",
|
||||
"127.0.0.1:",
|
||||
"[::1]:",
|
||||
])
|
||||
def test_malformed_host_is_rejected_even_when_hostname_is_loopback(self, host):
|
||||
assert not _mod._origin_is_allowed("http://127.0.0.1:9555", host)
|
||||
|
||||
def test_request_scheme_controls_host_default_port(self):
|
||||
assert _mod._origin_is_allowed("https://localhost", "localhost", request_scheme="https")
|
||||
assert not _mod._origin_is_allowed("https://localhost", "localhost", request_scheme="http")
|
||||
|
||||
def test_ws_handler_rejects_untrusted_origin_before_launching_chrome(self):
|
||||
class RejectingPool:
|
||||
async def get_or_launch(self, **_kwargs):
|
||||
raise AssertionError("untrusted origin should be rejected before launching Chrome")
|
||||
|
||||
request = SimpleNamespace(
|
||||
headers={"Host": "127.0.0.1:9555", "Origin": "http://attacker.example"},
|
||||
app={"pool": RejectingPool()},
|
||||
match_info={"path": "browser/browser-guid"},
|
||||
)
|
||||
|
||||
response = asyncio.run(_mod.handle_ws_default(request))
|
||||
|
||||
assert response.status == 403
|
||||
assert "untrusted" in response.text.lower()
|
||||
|
||||
def test_seed_ws_handler_rejects_untrusted_origin_before_launching_chrome(self):
|
||||
class RejectingPool:
|
||||
async def get_or_launch(self, **_kwargs):
|
||||
raise AssertionError("untrusted origin should be rejected before launching Chrome")
|
||||
|
||||
request = SimpleNamespace(
|
||||
headers={"Host": "127.0.0.1:9555", "Origin": "http://attacker.example"},
|
||||
app={"pool": RejectingPool()},
|
||||
match_info={"seed": "abc123", "path": "page/page-guid"},
|
||||
)
|
||||
|
||||
response = asyncio.run(_mod.handle_ws_seed(request))
|
||||
|
||||
assert response.status == 403
|
||||
assert "untrusted" in response.text.lower()
|
||||
|
||||
|
||||
class TestHandlerURLRewriting:
|
||||
"""Verify handlers rewrite CDP WebSocket URLs to the public cloakserve endpoint."""
|
||||
|
||||
def _rewrite_version(self, orig_ws: str, host: str, seed: str | None, scheme: str = "ws") -> str:
|
||||
"""Replicate the URL rewrite logic from handle_json_version."""
|
||||
@@ -244,3 +334,95 @@ class TestConnectionTracking:
|
||||
pool.disconnect("a")
|
||||
assert pool._connections["a"] == 1
|
||||
assert pool._connections["b"] == 1
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Seed validation (CVE fix — path traversal via fingerprint param)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestSeedValidation:
|
||||
"""Verify SAFE_SEED_RE rejects path traversal and reserved names."""
|
||||
|
||||
@pytest.mark.parametrize("seed", [
|
||||
"../foo", "../../etc", "/etc/passwd", "..", ".", "foo/bar",
|
||||
"foo\\bar", "\x00evil", "", "a" * 129,
|
||||
])
|
||||
def test_malicious_seeds_rejected(self, seed):
|
||||
assert not SAFE_SEED_RE.match(seed)
|
||||
|
||||
@pytest.mark.parametrize("seed", [
|
||||
"__default__",
|
||||
])
|
||||
def test_reserved_seeds_rejected(self, seed):
|
||||
assert seed in RESERVED_SEEDS
|
||||
|
||||
@pytest.mark.parametrize("seed", [
|
||||
"12345", "my-seed_01", "ABC", "a" * 128, "0", "test-seed",
|
||||
])
|
||||
def test_valid_seeds_accepted(self, seed):
|
||||
assert SAFE_SEED_RE.match(seed)
|
||||
assert seed not in RESERVED_SEEDS
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Path containment (_safe_rmtree)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestSafeRmtree:
|
||||
"""Verify _safe_rmtree refuses to delete outside data_dir."""
|
||||
|
||||
def _make_pool(self, data_dir: str):
|
||||
return ChromePool(
|
||||
binary="/fake/chrome",
|
||||
global_args=[],
|
||||
headless=True,
|
||||
data_dir=data_dir,
|
||||
)
|
||||
|
||||
def test_refuses_path_outside_data_dir(self, tmp_path):
|
||||
data_dir = tmp_path / "profiles"
|
||||
data_dir.mkdir()
|
||||
victim = tmp_path / "victim"
|
||||
victim.mkdir()
|
||||
(victim / "sentinel").touch()
|
||||
|
||||
pool = self._make_pool(str(data_dir))
|
||||
pool._safe_rmtree(str(victim))
|
||||
|
||||
assert victim.exists(), "Directory outside data_dir must not be deleted"
|
||||
|
||||
def test_refuses_data_dir_itself(self, tmp_path):
|
||||
data_dir = tmp_path / "profiles"
|
||||
data_dir.mkdir()
|
||||
(data_dir / "sentinel").touch()
|
||||
|
||||
pool = self._make_pool(str(data_dir))
|
||||
pool._safe_rmtree(str(data_dir))
|
||||
|
||||
assert data_dir.exists(), "data_dir itself must not be deleted"
|
||||
|
||||
def test_deletes_valid_subdirectory(self, tmp_path):
|
||||
data_dir = tmp_path / "profiles"
|
||||
data_dir.mkdir()
|
||||
subdir = data_dir / "seed-12345"
|
||||
subdir.mkdir()
|
||||
(subdir / "data").touch()
|
||||
|
||||
pool = self._make_pool(str(data_dir))
|
||||
pool._safe_rmtree(str(subdir))
|
||||
|
||||
assert not subdir.exists(), "Valid subdirectory should be deleted"
|
||||
|
||||
def test_refuses_traversal_path(self, tmp_path):
|
||||
data_dir = tmp_path / "profiles"
|
||||
data_dir.mkdir()
|
||||
victim = tmp_path / "victim"
|
||||
victim.mkdir()
|
||||
|
||||
traversal = str(data_dir / ".." / "victim")
|
||||
pool = self._make_pool(str(data_dir))
|
||||
pool._safe_rmtree(traversal)
|
||||
|
||||
assert victim.exists(), "Traversal path must not be deleted"
|
||||
|
||||
@@ -0,0 +1,33 @@
|
||||
import os
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from cloakbrowser import launch
|
||||
|
||||
|
||||
@patch("cloakbrowser.browser.ensure_binary")
|
||||
@patch("cloakbrowser.browser._import_sync_playwright")
|
||||
def test_extension_loading(mock_playwright_import, mock_ensure_binary):
|
||||
mock_ensure_binary.return_value = "/fake/chrome"
|
||||
|
||||
mock_browser = MagicMock()
|
||||
|
||||
mock_pw = MagicMock()
|
||||
mock_pw.chromium.launch.return_value = mock_browser
|
||||
|
||||
mock_pw_manager = MagicMock()
|
||||
mock_pw_manager.return_value.start.return_value = mock_pw
|
||||
|
||||
mock_playwright_import.return_value = mock_pw_manager
|
||||
|
||||
launch(extension_paths=["./ext"])
|
||||
|
||||
mock_pw.chromium.launch.assert_called_once()
|
||||
|
||||
launch_call = mock_pw.chromium.launch.call_args
|
||||
|
||||
args = launch_call.kwargs["args"]
|
||||
|
||||
abs_path = os.path.abspath("./ext")
|
||||
|
||||
assert f"--load-extension={abs_path}" in args
|
||||
assert f"--disable-extensions-except={abs_path}" in args
|
||||
@@ -1,6 +1,7 @@
|
||||
"""Unit tests for GeoIP-based timezone/locale detection."""
|
||||
|
||||
from unittest.mock import patch
|
||||
import time
|
||||
|
||||
import pytest
|
||||
|
||||
@@ -144,6 +145,20 @@ def test_maybe_resolve_fills_both():
|
||||
assert ip == "5.6.7.8"
|
||||
|
||||
|
||||
def test_maybe_resolve_geoip_timeout_returns_existing_values(monkeypatch):
|
||||
"""A stalled proxy lookup should not block launch indefinitely."""
|
||||
mock_geoip2 = type("module", (), {"database": type("db", (), {"Reader": None})})()
|
||||
monkeypatch.setenv("CLOAKBROWSER_GEOIP_TIMEOUT_SECONDS", "0.05")
|
||||
with patch.dict("sys.modules", {"geoip2": mock_geoip2, "geoip2.database": mock_geoip2.database}):
|
||||
with patch("cloakbrowser.geoip._ensure_geoip_db", return_value=object()):
|
||||
start = time.monotonic()
|
||||
tz, loc, ip = maybe_resolve_geoip(True, "http://203.0.113.10:8080", None, "fr-FR")
|
||||
elapsed = time.monotonic() - start
|
||||
|
||||
assert (tz, loc, ip) == (None, "fr-FR", None)
|
||||
assert elapsed < 0.5
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# _is_private_ip
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
+543
-11
@@ -14,6 +14,26 @@ import time
|
||||
import sys
|
||||
import asyncio
|
||||
import pytest
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
|
||||
def _mock_el_evaluate(is_input=False):
|
||||
"""Mock evaluate that returns is_input for tagName checks and {hit: True} for pointer events."""
|
||||
def _eval(js, *args, **kwargs):
|
||||
if isinstance(js, str) and "elementFromPoint" in js:
|
||||
return {"hit": True}
|
||||
return is_input
|
||||
return MagicMock(side_effect=_eval)
|
||||
|
||||
|
||||
def _async_mock_el_evaluate(is_input=False):
|
||||
"""Async version of _mock_el_evaluate."""
|
||||
from unittest.mock import AsyncMock
|
||||
async def _eval(js, *args, **kwargs):
|
||||
if isinstance(js, str) and "elementFromPoint" in js:
|
||||
return {"hit": True}
|
||||
return is_input
|
||||
return AsyncMock(side_effect=_eval)
|
||||
|
||||
|
||||
# =========================================================================
|
||||
@@ -192,6 +212,59 @@ class TestAsyncCompat:
|
||||
import asyncio
|
||||
assert asyncio.iscoroutinefunction(async_sleep_ms)
|
||||
|
||||
def test_patch_page_async_does_not_crash(self):
|
||||
"""patch_page_async must not raise NameError for missing definitions."""
|
||||
import cloakbrowser.human as h
|
||||
from cloakbrowser.human import _CursorState
|
||||
from cloakbrowser.human.config import resolve_config
|
||||
from unittest.mock import MagicMock, AsyncMock
|
||||
|
||||
cfg = resolve_config("default", {"idle_between_actions": False})
|
||||
cursor = _CursorState()
|
||||
cursor.initialized = True
|
||||
cursor.x = 100
|
||||
cursor.y = 100
|
||||
|
||||
page = MagicMock()
|
||||
page.click = AsyncMock()
|
||||
page.dblclick = AsyncMock()
|
||||
page.hover = AsyncMock()
|
||||
page.type = AsyncMock()
|
||||
page.fill = AsyncMock()
|
||||
page.goto = AsyncMock()
|
||||
page.check = AsyncMock()
|
||||
page.uncheck = AsyncMock()
|
||||
page.select_option = AsyncMock()
|
||||
page.press = AsyncMock()
|
||||
page.is_checked = AsyncMock(return_value=False)
|
||||
page.viewport_size = {"width": 1280, "height": 720}
|
||||
page.evaluate = AsyncMock(return_value={"hit": True})
|
||||
page.context.new_cdp_session = AsyncMock(side_effect=Exception("no cdp"))
|
||||
page.mouse = MagicMock()
|
||||
page.mouse.move = AsyncMock()
|
||||
page.mouse.click = AsyncMock()
|
||||
page.mouse.wheel = AsyncMock()
|
||||
page.mouse.down = AsyncMock()
|
||||
page.mouse.up = AsyncMock()
|
||||
page.keyboard = MagicMock()
|
||||
page.keyboard.type = AsyncMock()
|
||||
page.keyboard.down = AsyncMock()
|
||||
page.keyboard.up = AsyncMock()
|
||||
page.keyboard.press = AsyncMock()
|
||||
page.keyboard.insert_text = AsyncMock()
|
||||
page.query_selector = AsyncMock(return_value=None)
|
||||
page.query_selector_all = AsyncMock(return_value=[])
|
||||
page.wait_for_selector = AsyncMock(return_value=None)
|
||||
page.main_frame = MagicMock()
|
||||
page.main_frame.return_value = MagicMock()
|
||||
page.main_frame.return_value.child_frames = MagicMock(return_value=[])
|
||||
page.main_frame.child_frames = MagicMock(return_value=[])
|
||||
|
||||
h.patch_page_async(page, cfg, cursor)
|
||||
|
||||
assert hasattr(page, '_original')
|
||||
assert page.select_option != AsyncMock
|
||||
|
||||
|
||||
# =========================================================================
|
||||
# 4. Focus check — press / clear / pressSequentially
|
||||
@@ -707,7 +780,7 @@ class TestElementHandlePatchingSync:
|
||||
el = MagicMock()
|
||||
el._human_patched = False
|
||||
el.bounding_box = MagicMock(return_value={"x": 50, "y": 50, "width": 100, "height": 30})
|
||||
el.evaluate = MagicMock(return_value=True) # is_input
|
||||
el.evaluate = _mock_el_evaluate(is_input=True)
|
||||
el.is_checked = MagicMock(return_value=False)
|
||||
el.query_selector = MagicMock(return_value=None)
|
||||
el.query_selector_all = MagicMock(return_value=[])
|
||||
@@ -738,7 +811,7 @@ class TestElementHandlePatchingSync:
|
||||
el = MagicMock()
|
||||
el._human_patched = False
|
||||
el.bounding_box = MagicMock(return_value={"x": 200, "y": 200, "width": 100, "height": 30})
|
||||
el.evaluate = MagicMock(return_value=False)
|
||||
el.evaluate = _mock_el_evaluate(is_input=False)
|
||||
el.is_checked = MagicMock(return_value=False)
|
||||
el.query_selector = MagicMock(return_value=None)
|
||||
el.query_selector_all = MagicMock(return_value=[])
|
||||
@@ -779,7 +852,7 @@ class TestElementHandlePatchingSync:
|
||||
el = MagicMock()
|
||||
el._human_patched = False
|
||||
el.bounding_box = MagicMock(return_value={"x": 200, "y": 200, "width": 100, "height": 30})
|
||||
el.evaluate = MagicMock(return_value=False)
|
||||
el.evaluate = _mock_el_evaluate(is_input=False)
|
||||
el.is_checked = MagicMock(return_value=False)
|
||||
el.query_selector = MagicMock(return_value=None)
|
||||
el.query_selector_all = MagicMock(return_value=[])
|
||||
@@ -819,7 +892,7 @@ class TestElementHandlePatchingSync:
|
||||
el = MagicMock()
|
||||
el._human_patched = False
|
||||
el.bounding_box = MagicMock(return_value={"x": 200, "y": 200, "width": 100, "height": 30})
|
||||
el.evaluate = MagicMock(return_value=True) # is input
|
||||
el.evaluate = _mock_el_evaluate(is_input=True) # is input
|
||||
el.is_checked = MagicMock(return_value=False)
|
||||
el.query_selector = MagicMock(return_value=None)
|
||||
el.query_selector_all = MagicMock(return_value=[])
|
||||
@@ -867,7 +940,7 @@ class TestElementHandlePatchingSync:
|
||||
el = MagicMock()
|
||||
el._human_patched = False
|
||||
el.bounding_box = MagicMock(return_value={"x": 200, "y": 200, "width": 100, "height": 30})
|
||||
el.evaluate = MagicMock(return_value=True)
|
||||
el.evaluate = _mock_el_evaluate(is_input=True)
|
||||
el.is_checked = MagicMock(return_value=False)
|
||||
el.query_selector = MagicMock(return_value=None)
|
||||
el.query_selector_all = MagicMock(return_value=[])
|
||||
@@ -947,7 +1020,7 @@ class TestElementHandlePatchingSync:
|
||||
el = MagicMock()
|
||||
el._human_patched = False
|
||||
el.bounding_box = MagicMock(return_value={"x": 50, "y": 50, "width": 100, "height": 30})
|
||||
el.evaluate = MagicMock(return_value=False)
|
||||
el.evaluate = _mock_el_evaluate(is_input=False)
|
||||
el.is_checked = MagicMock(return_value=False)
|
||||
el.query_selector = MagicMock(return_value=child)
|
||||
el.query_selector_all = MagicMock(return_value=[])
|
||||
@@ -971,7 +1044,7 @@ class TestElementHandlePatchingSync:
|
||||
el = MagicMock()
|
||||
el._human_patched = False
|
||||
el.bounding_box = MagicMock(return_value={"x": 50, "y": 50, "width": 100, "height": 30})
|
||||
el.evaluate = MagicMock(return_value=False)
|
||||
el.evaluate = _mock_el_evaluate(is_input=False)
|
||||
el.is_checked = MagicMock(return_value=False)
|
||||
el.query_selector = MagicMock(return_value=None)
|
||||
el.query_selector_all = MagicMock(return_value=[])
|
||||
@@ -1036,7 +1109,7 @@ class TestElementHandlePatchingSync:
|
||||
el = MagicMock()
|
||||
el._human_patched = False
|
||||
el.bounding_box = MagicMock(return_value={"x": 50, "y": 50, "width": 100, "height": 30})
|
||||
el.evaluate = MagicMock(return_value=False)
|
||||
el.evaluate = _mock_el_evaluate(is_input=False)
|
||||
el.is_checked = MagicMock(return_value=False)
|
||||
el.query_selector = MagicMock(return_value=None)
|
||||
el.query_selector_all = MagicMock(return_value=[])
|
||||
@@ -1069,7 +1142,7 @@ class TestElementHandlePatchingSync:
|
||||
el = MagicMock()
|
||||
el._human_patched = False
|
||||
el.bounding_box = MagicMock(return_value={"x": 50, "y": 50, "width": 100, "height": 30})
|
||||
el.evaluate = MagicMock(return_value=False)
|
||||
el.evaluate = _mock_el_evaluate(is_input=False)
|
||||
el.is_checked = MagicMock(return_value=False)
|
||||
el.query_selector = MagicMock(return_value=None)
|
||||
el.query_selector_all = MagicMock(return_value=[])
|
||||
@@ -1114,8 +1187,9 @@ class TestElementHandlePatchingAsync:
|
||||
el = MagicMock()
|
||||
el._human_patched = False
|
||||
el.bounding_box = AsyncMock(return_value={"x": 200, "y": 200, "width": 100, "height": 30})
|
||||
el.evaluate = AsyncMock(return_value=False)
|
||||
el.evaluate = _async_mock_el_evaluate(is_input=False)
|
||||
el.is_checked = AsyncMock(return_value=False)
|
||||
el.wait_for_element_state = AsyncMock()
|
||||
el.query_selector = AsyncMock(return_value=None)
|
||||
el.query_selector_all = AsyncMock(return_value=[])
|
||||
el.wait_for_selector = AsyncMock(return_value=None)
|
||||
@@ -1155,8 +1229,9 @@ class TestElementHandlePatchingAsync:
|
||||
el = MagicMock()
|
||||
el._human_patched = False
|
||||
el.bounding_box = AsyncMock(return_value={"x": 50, "y": 50, "width": 100, "height": 30})
|
||||
el.evaluate = AsyncMock(return_value=False)
|
||||
el.evaluate = _async_mock_el_evaluate(is_input=False)
|
||||
el.is_checked = AsyncMock(return_value=False)
|
||||
el.wait_for_element_state = AsyncMock()
|
||||
el.query_selector = AsyncMock(return_value=None)
|
||||
el.query_selector_all = AsyncMock(return_value=[])
|
||||
el.wait_for_selector = AsyncMock(return_value=None)
|
||||
@@ -1296,6 +1371,463 @@ class TestAsyncElementHandle:
|
||||
await browser.close()
|
||||
|
||||
|
||||
# =========================================================================
|
||||
# 15. Per-call timeout forwarding (issue #137)
|
||||
# =========================================================================
|
||||
|
||||
class TestPerCallTimeoutForwarding:
|
||||
"""page.click('#x', timeout=5000) must forward 5000 to bounding_box(),
|
||||
not silently use the hardcoded 2000ms in scroll."""
|
||||
|
||||
def test_get_element_box_default_timeout(self):
|
||||
"""Default timeout matches Playwright's 30000ms."""
|
||||
from cloakbrowser.human.scroll import _get_element_box
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
page = MagicMock()
|
||||
loc = MagicMock()
|
||||
loc.bounding_box = MagicMock(return_value={"x": 0, "y": 0, "width": 1, "height": 1})
|
||||
page.locator = MagicMock(return_value=MagicMock(first=loc))
|
||||
|
||||
_get_element_box(page, "#x")
|
||||
loc.bounding_box.assert_called_once_with(timeout=30000)
|
||||
|
||||
def test_get_element_box_custom_timeout(self):
|
||||
"""Caller can pass a custom timeout that overrides the default."""
|
||||
from cloakbrowser.human.scroll import _get_element_box
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
page = MagicMock()
|
||||
loc = MagicMock()
|
||||
loc.bounding_box = MagicMock(return_value={"x": 0, "y": 0, "width": 1, "height": 1})
|
||||
page.locator = MagicMock(return_value=MagicMock(first=loc))
|
||||
|
||||
_get_element_box(page, "#x", timeout=5000)
|
||||
loc.bounding_box.assert_called_once_with(timeout=5000)
|
||||
|
||||
def test_scroll_to_element_forwards_timeout(self):
|
||||
"""scroll_to_element passes timeout through to bounding_box()."""
|
||||
from cloakbrowser.human.scroll import scroll_to_element
|
||||
from cloakbrowser.human.config import resolve_config
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
cfg = resolve_config("default", None)
|
||||
page = MagicMock()
|
||||
page.viewport_size = {"width": 1280, "height": 720}
|
||||
loc = MagicMock()
|
||||
# Already in viewport so we don't actually scroll — just verify
|
||||
# the timeout was forwarded on the first bounding_box() call.
|
||||
loc.bounding_box = MagicMock(return_value={"x": 100, "y": 200, "width": 50, "height": 30})
|
||||
page.locator = MagicMock(return_value=MagicMock(first=loc))
|
||||
|
||||
raw = MagicMock()
|
||||
scroll_to_element(page, raw, "#x", 0, 0, cfg, timeout=7500)
|
||||
loc.bounding_box.assert_called_with(timeout=7500)
|
||||
|
||||
def test_page_click_forwards_timeout_kwarg(self):
|
||||
"""page.click(selector, timeout=...) reaches scroll_to_element.
|
||||
|
||||
Patches scroll_to_element module-side via monkey-patching the
|
||||
cloakbrowser.human module attribute used by patch_page.
|
||||
"""
|
||||
import cloakbrowser.human as h
|
||||
from cloakbrowser.human import _CursorState
|
||||
from cloakbrowser.human.config import resolve_config
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
cfg = resolve_config("default", {"idle_between_actions": False})
|
||||
cursor = _CursorState()
|
||||
cursor.initialized = True
|
||||
cursor.x = 100
|
||||
cursor.y = 100
|
||||
|
||||
# Build a minimal page mock
|
||||
page = MagicMock()
|
||||
page.click = MagicMock()
|
||||
page.dblclick = MagicMock()
|
||||
page.hover = MagicMock()
|
||||
page.type = MagicMock()
|
||||
page.fill = MagicMock()
|
||||
page.goto = MagicMock()
|
||||
page.is_checked = MagicMock(return_value=False)
|
||||
page.viewport_size = {"width": 1280, "height": 720}
|
||||
page.evaluate = MagicMock(return_value={"hit": True})
|
||||
page.context.new_cdp_session = MagicMock(side_effect=Exception("no cdp"))
|
||||
page.mouse = MagicMock()
|
||||
page.keyboard = MagicMock()
|
||||
page.query_selector = MagicMock(return_value=None)
|
||||
page.query_selector_all = MagicMock(return_value=[])
|
||||
page.wait_for_selector = MagicMock(return_value=None)
|
||||
page.main_frame = MagicMock()
|
||||
page.main_frame.child_frames = []
|
||||
|
||||
captured = {}
|
||||
def fake_scroll(page_arg, raw, selector, cx, cy, cfg_arg, timeout=30000):
|
||||
captured["timeout"] = timeout
|
||||
return ({"x": 100, "y": 100, "width": 50, "height": 30}, cx, cy, False)
|
||||
|
||||
with patch.object(h, "scroll_to_element", side_effect=fake_scroll), \
|
||||
patch.object(h, "ensure_actionable"):
|
||||
h.patch_page(page, cfg, cursor)
|
||||
page.click("#slow-button", timeout=5000)
|
||||
|
||||
assert 4900 <= captured.get("timeout", 0) <= 5000, f"expected ~5000, got {captured}"
|
||||
|
||||
|
||||
# =========================================================================
|
||||
# 16. Per-call human_config override (typing speed customization)
|
||||
# =========================================================================
|
||||
|
||||
class TestPerCallHumanConfigOverride:
|
||||
"""page.type('#email', text, human_config={'typing_delay': 30}) lets users
|
||||
override typing speed (and any other HumanConfig field) on a per-call
|
||||
basis without re-patching the page."""
|
||||
|
||||
def test_merge_config_creates_new_instance(self):
|
||||
from cloakbrowser.human.config import resolve_config, merge_config
|
||||
|
||||
base = resolve_config("default", None)
|
||||
merged = merge_config(base, {"typing_delay": 30})
|
||||
|
||||
assert merged.typing_delay == 30
|
||||
assert base.typing_delay != 30 # not mutated
|
||||
# Non-overridden fields are preserved
|
||||
assert merged.mouse_min_steps == base.mouse_min_steps
|
||||
|
||||
def test_merge_config_none_returns_base(self):
|
||||
from cloakbrowser.human.config import resolve_config, merge_config
|
||||
|
||||
base = resolve_config("default", None)
|
||||
merged = merge_config(base, None)
|
||||
assert merged is base
|
||||
|
||||
def test_merge_config_ignores_unknown_keys(self):
|
||||
from cloakbrowser.human.config import resolve_config, merge_config
|
||||
|
||||
base = resolve_config("default", None)
|
||||
# ``not_a_real_field`` is silently dropped — callers shouldn't crash
|
||||
# if they pass typos or future field names.
|
||||
merged = merge_config(base, {"typing_delay": 30, "not_a_real_field": 99})
|
||||
assert merged.typing_delay == 30
|
||||
|
||||
def test_page_type_uses_per_call_typing_delay(self):
|
||||
"""page.type(..., human_config={'typing_delay': 30}) reaches human_type
|
||||
with cfg.typing_delay == 30 even when patch was done with default 70."""
|
||||
import cloakbrowser.human as h
|
||||
from cloakbrowser.human import _CursorState
|
||||
from cloakbrowser.human.config import resolve_config
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
cfg = resolve_config("default", {
|
||||
"idle_between_actions": False,
|
||||
"field_switch_delay": (0, 1),
|
||||
})
|
||||
assert cfg.typing_delay == 70 # baseline
|
||||
|
||||
cursor = _CursorState()
|
||||
cursor.initialized = True
|
||||
cursor.x = 100
|
||||
cursor.y = 100
|
||||
|
||||
page = MagicMock()
|
||||
page.click = MagicMock()
|
||||
page.dblclick = MagicMock()
|
||||
page.hover = MagicMock()
|
||||
page.type = MagicMock()
|
||||
page.fill = MagicMock()
|
||||
page.goto = MagicMock()
|
||||
page.is_checked = MagicMock(return_value=False)
|
||||
page.viewport_size = {"width": 1280, "height": 720}
|
||||
page.evaluate = MagicMock(return_value={"hit": True})
|
||||
page.context.new_cdp_session = MagicMock(side_effect=Exception("no cdp"))
|
||||
page.mouse = MagicMock()
|
||||
page.keyboard = MagicMock()
|
||||
page.query_selector = MagicMock(return_value=None)
|
||||
page.query_selector_all = MagicMock(return_value=[])
|
||||
page.wait_for_selector = MagicMock(return_value=None)
|
||||
page.main_frame = MagicMock()
|
||||
page.main_frame.child_frames = []
|
||||
|
||||
captured = {}
|
||||
def fake_human_type(page_arg, raw, text, cfg_arg, cdp_session=None):
|
||||
captured["typing_delay"] = cfg_arg.typing_delay
|
||||
captured["mistype_chance"] = cfg_arg.mistype_chance
|
||||
|
||||
def fake_scroll(*args, **kwargs):
|
||||
return ({"x": 100, "y": 100, "width": 50, "height": 30}, 100, 100, False)
|
||||
|
||||
with patch.object(h, "human_type", side_effect=fake_human_type), \
|
||||
patch.object(h, "scroll_to_element", side_effect=fake_scroll), \
|
||||
patch.object(h, "ensure_actionable"), \
|
||||
patch.object(h, "check_pointer_events"):
|
||||
h.patch_page(page, cfg, cursor)
|
||||
page.type(
|
||||
"#email", "hi",
|
||||
human_config={"typing_delay": 30, "mistype_chance": 0},
|
||||
)
|
||||
|
||||
assert captured["typing_delay"] == 30
|
||||
assert captured["mistype_chance"] == 0
|
||||
assert cfg.typing_delay == 70
|
||||
|
||||
def test_page_fill_uses_per_call_typing_delay(self):
|
||||
"""Same as type, but for fill (which also clears the field first)."""
|
||||
import cloakbrowser.human as h
|
||||
from cloakbrowser.human import _CursorState
|
||||
from cloakbrowser.human.config import resolve_config
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
cfg = resolve_config("default", {
|
||||
"idle_between_actions": False,
|
||||
"field_switch_delay": (0, 1),
|
||||
})
|
||||
cursor = _CursorState()
|
||||
cursor.initialized = True
|
||||
cursor.x = 100
|
||||
cursor.y = 100
|
||||
|
||||
page = MagicMock()
|
||||
page.viewport_size = {"width": 1280, "height": 720}
|
||||
page.is_checked = MagicMock(return_value=False)
|
||||
page.evaluate = MagicMock(return_value={"hit": True})
|
||||
page.context.new_cdp_session = MagicMock(side_effect=Exception("no cdp"))
|
||||
page.mouse = MagicMock()
|
||||
page.keyboard = MagicMock()
|
||||
page.query_selector = MagicMock(return_value=None)
|
||||
page.query_selector_all = MagicMock(return_value=[])
|
||||
page.wait_for_selector = MagicMock(return_value=None)
|
||||
page.main_frame = MagicMock()
|
||||
page.main_frame.child_frames = []
|
||||
|
||||
captured = {}
|
||||
def fake_human_type(page_arg, raw, text, cfg_arg, cdp_session=None):
|
||||
captured["typing_delay"] = cfg_arg.typing_delay
|
||||
|
||||
def fake_scroll(*args, **kwargs):
|
||||
return ({"x": 100, "y": 100, "width": 50, "height": 30}, 100, 100, False)
|
||||
|
||||
with patch.object(h, "human_type", side_effect=fake_human_type), \
|
||||
patch.object(h, "scroll_to_element", side_effect=fake_scroll), \
|
||||
patch.object(h, "ensure_actionable"), \
|
||||
patch.object(h, "check_pointer_events"):
|
||||
h.patch_page(page, cfg, cursor)
|
||||
page.fill("#password", "secret", human_config={"typing_delay": 150})
|
||||
|
||||
assert captured["typing_delay"] == 150
|
||||
|
||||
def test_element_handle_type_uses_per_call_human_config(self):
|
||||
"""el.type(text, human_config={...}) merges per-call overrides on the
|
||||
ElementHandle path (which doesn't go through page.type)."""
|
||||
from cloakbrowser.human import _patch_single_element_handle_sync, _CursorState
|
||||
from cloakbrowser.human.config import resolve_config
|
||||
import cloakbrowser.human as h
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
cfg = resolve_config("default", {
|
||||
"idle_between_actions": False,
|
||||
"field_switch_delay": (0, 1),
|
||||
})
|
||||
cursor = _CursorState()
|
||||
cursor.initialized = True
|
||||
cursor.x = 50
|
||||
cursor.y = 50
|
||||
|
||||
page = MagicMock()
|
||||
page.viewport_size = {"width": 1280, "height": 720}
|
||||
page._original = MagicMock()
|
||||
|
||||
el = MagicMock()
|
||||
el._human_patched = False
|
||||
el.bounding_box = MagicMock(
|
||||
return_value={"x": 200, "y": 200, "width": 100, "height": 30}
|
||||
)
|
||||
el.evaluate = _mock_el_evaluate(is_input=True)
|
||||
el.is_checked = MagicMock(return_value=False)
|
||||
el.query_selector = MagicMock(return_value=None)
|
||||
el.query_selector_all = MagicMock(return_value=[])
|
||||
el.wait_for_selector = MagicMock(return_value=None)
|
||||
el.scroll_into_view_if_needed = MagicMock()
|
||||
|
||||
raw_mouse = MagicMock()
|
||||
raw_keyboard = MagicMock()
|
||||
|
||||
captured = {}
|
||||
def fake_human_type(page_arg, raw, text, cfg_arg, cdp_session=None):
|
||||
captured["typing_delay"] = cfg_arg.typing_delay
|
||||
|
||||
with patch.object(h, "human_type", side_effect=fake_human_type):
|
||||
_patch_single_element_handle_sync(
|
||||
el, page, cfg, cursor, raw_mouse, raw_keyboard,
|
||||
page._original, None, None,
|
||||
)
|
||||
el.type("abc", human_config={"typing_delay": 25})
|
||||
|
||||
assert captured["typing_delay"] == 25
|
||||
|
||||
|
||||
# =========================================================================
|
||||
# 17. scroll_into_view_if_needed humanization
|
||||
# =========================================================================
|
||||
|
||||
class TestScrollIntoViewIfNeeded:
|
||||
"""scroll_into_view_if_needed should run through the same
|
||||
accelerate → cruise → decelerate → overshoot wheel sequence as page.click—
|
||||
not Playwright's instant-snap default."""
|
||||
|
||||
def test_human_scroll_into_view_skips_when_in_viewport(self):
|
||||
"""Already-visible elements: no wheel events, just return."""
|
||||
from cloakbrowser.human.scroll import human_scroll_into_view
|
||||
from cloakbrowser.human.config import resolve_config
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
cfg = resolve_config("default", None)
|
||||
page = MagicMock()
|
||||
page.viewport_size = {"width": 1280, "height": 720}
|
||||
raw = MagicMock()
|
||||
# Box is dead-center of viewport — squarely in scroll_target_zone
|
||||
in_view_box = {"x": 200, "y": 300, "width": 50, "height": 30}
|
||||
|
||||
box, cx, cy, did_scroll = human_scroll_into_view(
|
||||
page, raw, lambda: in_view_box, 0, 0, cfg,
|
||||
)
|
||||
assert not did_scroll, "In-viewport elements shouldn't report scrolling"
|
||||
assert box == in_view_box
|
||||
assert not raw.wheel.called, "In-viewport elements shouldn't trigger wheel events"
|
||||
|
||||
def test_human_scroll_into_view_scrolls_when_below_fold(self):
|
||||
"""Below-fold elements: wheel events fire, eventually box becomes visible."""
|
||||
from cloakbrowser.human.scroll import human_scroll_into_view
|
||||
from cloakbrowser.human.config import resolve_config
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
cfg = resolve_config("default", {
|
||||
"scroll_overshoot_chance": 0, # deterministic
|
||||
"scroll_pre_move_delay": (0, 1),
|
||||
"scroll_pause_fast": (0, 1),
|
||||
"scroll_pause_slow": (0, 1),
|
||||
"scroll_settle_delay": (0, 1),
|
||||
})
|
||||
page = MagicMock()
|
||||
page.viewport_size = {"width": 1280, "height": 720}
|
||||
raw = MagicMock()
|
||||
|
||||
# First box is far below the fold; subsequent boxes "come into view"
|
||||
# so the loop terminates after a few wheel bursts.
|
||||
boxes = [
|
||||
{"x": 200, "y": 2000, "width": 50, "height": 30},
|
||||
{"x": 200, "y": 1500, "width": 50, "height": 30},
|
||||
{"x": 200, "y": 1000, "width": 50, "height": 30},
|
||||
{"x": 200, "y": 400, "width": 50, "height": 30}, # in viewport
|
||||
{"x": 200, "y": 400, "width": 50, "height": 30},
|
||||
{"x": 200, "y": 400, "width": 50, "height": 30},
|
||||
]
|
||||
idx = {"i": 0}
|
||||
def get_box():
|
||||
i = min(idx["i"], len(boxes) - 1)
|
||||
idx["i"] += 1
|
||||
return boxes[i]
|
||||
|
||||
human_scroll_into_view(page, raw, get_box, 0, 0, cfg)
|
||||
assert raw.wheel.called, "Below-fold scroll should produce wheel events"
|
||||
|
||||
def test_element_handle_scroll_into_view_if_needed_humanized(self):
|
||||
"""el.scroll_into_view_if_needed() routes through human_scroll_into_view."""
|
||||
from cloakbrowser.human import _patch_single_element_handle_sync, _CursorState
|
||||
from cloakbrowser.human.config import resolve_config
|
||||
import cloakbrowser.human as h
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
cfg = resolve_config("default", None)
|
||||
cursor = _CursorState()
|
||||
cursor.initialized = True
|
||||
cursor.x = 50
|
||||
cursor.y = 50
|
||||
|
||||
page = MagicMock()
|
||||
page.viewport_size = {"width": 1280, "height": 720}
|
||||
page._original = MagicMock()
|
||||
|
||||
el = MagicMock()
|
||||
el._human_patched = False
|
||||
el.bounding_box = MagicMock(
|
||||
return_value={"x": 200, "y": 200, "width": 50, "height": 30}
|
||||
)
|
||||
el.evaluate = _mock_el_evaluate(is_input=False)
|
||||
el.is_checked = MagicMock(return_value=False)
|
||||
el.query_selector = MagicMock(return_value=None)
|
||||
el.query_selector_all = MagicMock(return_value=[])
|
||||
el.wait_for_selector = MagicMock(return_value=None)
|
||||
# Make sure the original method exists so the patch is wired up
|
||||
el.scroll_into_view_if_needed = MagicMock()
|
||||
|
||||
called = {"count": 0}
|
||||
def fake(*args, **kwargs):
|
||||
called["count"] += 1
|
||||
return ({"x": 200, "y": 200, "width": 50, "height": 30}, 100, 100, False)
|
||||
|
||||
with patch.object(h, "human_scroll_into_view", side_effect=fake):
|
||||
_patch_single_element_handle_sync(
|
||||
el, page, cfg, cursor, MagicMock(), MagicMock(),
|
||||
page._original, None, None,
|
||||
)
|
||||
el.scroll_into_view_if_needed()
|
||||
|
||||
assert called["count"] >= 1, "humanized scroll helper was never called"
|
||||
|
||||
def test_locator_scroll_into_view_if_needed_humanized(self):
|
||||
"""Locator.scroll_into_view_if_needed() also goes through humanized scroll."""
|
||||
import cloakbrowser.human as h
|
||||
from cloakbrowser.human import _CursorState
|
||||
from cloakbrowser.human.config import resolve_config
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
# Patch Locator class fresh
|
||||
_ensure_locator_patched()
|
||||
|
||||
from playwright.sync_api._generated import Locator
|
||||
|
||||
cfg = resolve_config("default", None)
|
||||
cursor = _CursorState()
|
||||
cursor.x = 50
|
||||
cursor.y = 50
|
||||
cursor.initialized = True
|
||||
|
||||
page = MagicMock()
|
||||
page._original = MagicMock()
|
||||
page._human_cfg = cfg
|
||||
page._human_cursor = cursor
|
||||
page._human_raw_mouse = MagicMock()
|
||||
page.viewport_size = {"width": 1280, "height": 720}
|
||||
|
||||
# Build a Locator-like object satisfying the patched method
|
||||
loc = MagicMock(spec=Locator)
|
||||
loc.page = page
|
||||
impl_obj = MagicMock()
|
||||
impl_obj._selector = "#x"
|
||||
loc._impl_obj = impl_obj
|
||||
loc.bounding_box = MagicMock(
|
||||
return_value={"x": 100, "y": 100, "width": 50, "height": 30}
|
||||
)
|
||||
|
||||
called = {"count": 0, "cfg": None}
|
||||
def fake(*args, **kwargs):
|
||||
called["count"] += 1
|
||||
# cfg is the 6th positional arg (page, raw, get_box, cx, cy, cfg)
|
||||
called["cfg"] = args[5] if len(args) >= 6 else kwargs.get("cfg")
|
||||
return ({"x": 100, "y": 100, "width": 50, "height": 30}, 200, 200, False)
|
||||
|
||||
with patch.object(h, "human_scroll_into_view", side_effect=fake):
|
||||
Locator.scroll_into_view_if_needed(
|
||||
loc, human_config={"scroll_overshoot_chance": 0.5},
|
||||
)
|
||||
|
||||
assert called["count"] == 1
|
||||
# Per-call override merged into the cfg passed downstream
|
||||
assert called["cfg"].scroll_overshoot_chance == 0.5
|
||||
# Cursor was updated from the helper's return value
|
||||
assert cursor.x == 200 and cursor.y == 200
|
||||
|
||||
|
||||
# =========================================================================
|
||||
# Direct runner (backwards compat)
|
||||
# =========================================================================
|
||||
|
||||
@@ -0,0 +1,171 @@
|
||||
"""Security tests for the AWS Lambda handler URL validation."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(
|
||||
0, str(Path(__file__).resolve().parent.parent / "examples" / "integrations" / "aws_lambda")
|
||||
)
|
||||
|
||||
from lambda_handler import _build_launch_kwargs, _classify_error, _validate_url
|
||||
|
||||
|
||||
class TestSchemeValidation:
|
||||
"""Fix 1: only http:// and https:// are accepted."""
|
||||
|
||||
@pytest.mark.parametrize("url", [
|
||||
"file:///etc/passwd",
|
||||
"file:///proc/self/environ",
|
||||
"data:text/html,<h1>pwned</h1>",
|
||||
"javascript:alert(1)",
|
||||
"chrome://settings",
|
||||
"about:blank",
|
||||
"ftp://example.com/file",
|
||||
"",
|
||||
])
|
||||
def test_rejects_non_http_schemes(self, url):
|
||||
with pytest.raises(ValueError, match="Only http"):
|
||||
_validate_url(url)
|
||||
|
||||
@pytest.mark.parametrize("url", [
|
||||
"https://example.com",
|
||||
"http://example.com",
|
||||
"https://example.com/path?q=1",
|
||||
"HTTP://EXAMPLE.COM",
|
||||
])
|
||||
def test_accepts_http_and_https(self, url):
|
||||
_validate_url(url)
|
||||
|
||||
def test_rejects_missing_hostname(self):
|
||||
with pytest.raises(ValueError, match="no hostname"):
|
||||
_validate_url("http://")
|
||||
|
||||
|
||||
class TestSSRFProtection:
|
||||
"""Fix 2: block private, loopback, link-local, reserved, and metadata IPs."""
|
||||
|
||||
@pytest.mark.parametrize("url,label", [
|
||||
("http://169.254.169.254", "AWS metadata"),
|
||||
("http://169.254.169.254/latest/meta-data/", "AWS metadata path"),
|
||||
("http://127.0.0.1", "loopback"),
|
||||
("http://127.0.0.2", "loopback range"),
|
||||
("http://localhost", "localhost"),
|
||||
("http://10.0.0.1", "private 10.x"),
|
||||
("http://172.16.0.1", "private 172.16"),
|
||||
("http://192.168.1.1", "private 192.168"),
|
||||
("http://0.0.0.0", "unspecified"),
|
||||
("http://[::1]", "IPv6 loopback"),
|
||||
])
|
||||
def test_rejects_private_ips(self, url, label):
|
||||
with pytest.raises(ValueError, match="private/internal"):
|
||||
_validate_url(url)
|
||||
|
||||
def test_rejects_carrier_grade_nat(self):
|
||||
with pytest.raises(ValueError, match="private/internal"):
|
||||
_validate_url("http://100.64.0.1")
|
||||
|
||||
def test_rejects_unresolvable_hostname(self):
|
||||
with pytest.raises(ValueError, match="Cannot resolve"):
|
||||
_validate_url("http://this-host-does-not-exist-cb-test.invalid")
|
||||
|
||||
def test_rejects_ipv4_mapped_ipv6(self):
|
||||
"""::ffff:127.0.0.1 should be blocked even though it's technically IPv6."""
|
||||
with pytest.raises(ValueError, match="private/internal"):
|
||||
_validate_url("http://[::ffff:127.0.0.1]")
|
||||
|
||||
|
||||
class TestExtraArgsRemoval:
|
||||
"""Fix 3: caller-controlled extra_args are ignored; internal _strategy_args work."""
|
||||
|
||||
def test_ignores_caller_extra_args(self):
|
||||
event = {"url": "https://example.com", "extra_args": ["--remote-debugging-port=9222"]}
|
||||
kwargs = _build_launch_kwargs(event)
|
||||
assert "--remote-debugging-port=9222" not in kwargs["args"]
|
||||
|
||||
def test_includes_strategy_args(self):
|
||||
event = {"url": "https://example.com", "_strategy_args": ["--ignore-certificate-errors"]}
|
||||
kwargs = _build_launch_kwargs(event)
|
||||
assert "--ignore-certificate-errors" in kwargs["args"]
|
||||
|
||||
def test_classify_error_uses_strategy_args(self):
|
||||
result = _classify_error(Exception("ERR_CERT_AUTHORITY_INVALID"))
|
||||
assert "_strategy_args" in result
|
||||
assert "extra_args" not in result
|
||||
|
||||
def test_always_includes_lambda_hardening_flags(self):
|
||||
kwargs = _build_launch_kwargs({"url": "https://example.com"})
|
||||
assert "--disable-dev-shm-usage" in kwargs["args"]
|
||||
assert "--no-zygote" in kwargs["args"]
|
||||
|
||||
def test_caller_cannot_inject_strategy_args(self):
|
||||
"""_strategy_args in the caller event must be stripped by _run() before launch."""
|
||||
from lambda_handler import _run
|
||||
import inspect
|
||||
source = inspect.getsource(_run)
|
||||
assert '"_strategy_args"' in source and "extra_args" in source, \
|
||||
"_run must strip both _strategy_args and extra_args from caller event"
|
||||
|
||||
|
||||
class TestRedirectSSRF:
|
||||
"""Fix 5: post-navigation re-validation catches redirects to blocked IPs.
|
||||
|
||||
These mock socket.getaddrinfo to simulate redirect scenarios without
|
||||
needing a real browser or HTTP server.
|
||||
"""
|
||||
|
||||
def test_validate_url_catches_redirect_target(self):
|
||||
"""If Chromium followed a redirect to 169.254.169.254, the post-nav
|
||||
_validate_url(page.url) call should reject it."""
|
||||
with pytest.raises(ValueError, match="private/internal"):
|
||||
_validate_url("http://169.254.169.254/latest/meta-data/iam/security-credentials/")
|
||||
|
||||
def test_validate_url_catches_localhost_redirect(self):
|
||||
with pytest.raises(ValueError, match="private/internal"):
|
||||
_validate_url("http://127.0.0.1:8080/admin")
|
||||
|
||||
def test_code_flow_validates_before_content(self):
|
||||
"""Verify that _attempt_scrape calls _validate_url(page.url) at line 282
|
||||
BEFORE building the result dict at line 290 (sequential code path)."""
|
||||
import ast
|
||||
handler_path = (
|
||||
Path(__file__).resolve().parent.parent
|
||||
/ "examples" / "integrations" / "aws_lambda" / "lambda_handler.py"
|
||||
)
|
||||
source = handler_path.read_text()
|
||||
tree = ast.parse(source)
|
||||
for node in ast.walk(tree):
|
||||
if isinstance(node, ast.AsyncFunctionDef) and node.name == "_attempt_scrape":
|
||||
body = node.body
|
||||
# Find the try block
|
||||
for stmt in body:
|
||||
if isinstance(stmt, ast.Try):
|
||||
try_body = stmt.body
|
||||
validate_lines = []
|
||||
content_line = None
|
||||
for s in try_body:
|
||||
if isinstance(s, ast.Expr) and isinstance(s.value, ast.Call):
|
||||
func = s.value.func
|
||||
if isinstance(func, ast.Name) and func.id == "_validate_url":
|
||||
validate_lines.append(s.lineno)
|
||||
if isinstance(s, ast.AnnAssign):
|
||||
if isinstance(s.target, ast.Name) and s.target.id == "result":
|
||||
content_line = s.lineno
|
||||
elif isinstance(s, ast.Assign):
|
||||
for target in s.targets:
|
||||
if isinstance(target, ast.Name) and target.id == "result":
|
||||
content_line = s.lineno
|
||||
assert len(validate_lines) >= 2, (
|
||||
f"Expected 2 _validate_url calls, found {len(validate_lines)}"
|
||||
)
|
||||
assert content_line is not None
|
||||
assert all(v < content_line for v in validate_lines), (
|
||||
f"_validate_url (lines {validate_lines}) must come before "
|
||||
f"result assignment (line {content_line})"
|
||||
)
|
||||
return
|
||||
pytest.fail("Could not find _attempt_scrape function in source")
|
||||
@@ -284,6 +284,42 @@ class TestResolveProxyConfig:
|
||||
_, args = _resolve_proxy_config("socks5://user:pass%3D123@host:1080")
|
||||
assert args == ["--proxy-server=socks5://user:pass%3D123@host:1080"]
|
||||
|
||||
def test_socks5_string_logs_info_when_reencoding(self, caplog):
|
||||
# When wrapper actually rewrites the URL (e.g. unencoded '=' in pwd),
|
||||
# surface an INFO log so users debugging SOCKS5 connectivity (#157)
|
||||
# can see what the wrapper did instead of being silently surprised.
|
||||
import logging
|
||||
with caplog.at_level(logging.INFO, logger="cloakbrowser"):
|
||||
_resolve_proxy_config("socks5://user:pass=123@host:1080")
|
||||
assert any("Auto URL-encoded SOCKS5" in r.message for r in caplog.records)
|
||||
# Credentials must not leak into the log.
|
||||
for r in caplog.records:
|
||||
assert "pass=123" not in r.message
|
||||
assert "pass%3D123" not in r.message
|
||||
|
||||
def test_socks5_string_silent_when_already_encoded(self, caplog):
|
||||
# Idempotent path: pre-encoded URL produces no log noise.
|
||||
import logging
|
||||
with caplog.at_level(logging.INFO, logger="cloakbrowser"):
|
||||
_resolve_proxy_config("socks5://user:pass%3D123@host:1080")
|
||||
assert not any("Auto URL-encoded SOCKS5" in r.message for r in caplog.records)
|
||||
|
||||
def test_socks5_string_silent_when_no_credentials(self, caplog):
|
||||
# No userinfo at all → no encoding work → no log.
|
||||
import logging
|
||||
with caplog.at_level(logging.INFO, logger="cloakbrowser"):
|
||||
_resolve_proxy_config("socks5://host:1080")
|
||||
assert not any("Auto URL-encoded SOCKS5" in r.message for r in caplog.records)
|
||||
|
||||
def test_socks5_string_silent_when_only_cosmetic_change(self, caplog):
|
||||
# urlparse lowercases scheme and hostname, but credentials are
|
||||
# untouched. The log must NOT fire for these cosmetic-only rewrites
|
||||
# (regression for Copilot's review on PR #209).
|
||||
import logging
|
||||
with caplog.at_level(logging.INFO, logger="cloakbrowser"):
|
||||
_resolve_proxy_config("socks5://USER:pass@HOST.com:1080")
|
||||
assert not any("Auto URL-encoded SOCKS5" in r.message for r in caplog.records)
|
||||
|
||||
def test_socks5_string_no_creds_unchanged(self):
|
||||
_, args = _resolve_proxy_config("socks5://host:1080")
|
||||
assert args == ["--proxy-server=socks5://host:1080"]
|
||||
|
||||
@@ -1010,7 +1010,11 @@ class TestPatchPageStealthWiring:
|
||||
fake_box = {"x": 100, "y": 200, "width": 200, "height": 30}
|
||||
with mock_patch(
|
||||
"cloakbrowser.human.scroll_to_element",
|
||||
return_value=(fake_box, 200.0, 215.0),
|
||||
return_value=(fake_box, 200.0, 215.0, False),
|
||||
), mock_patch(
|
||||
"cloakbrowser.human.ensure_actionable",
|
||||
), mock_patch(
|
||||
"cloakbrowser.human.check_pointer_events",
|
||||
):
|
||||
try:
|
||||
page.click("#btn")
|
||||
|
||||
Reference in New Issue
Block a user