Compare commits

...
34 Commits
Author SHA1 Message Date
CloakHQ 0caa14bf7b release: v0.3.31 — proxy credential routing, humanize iframe fixes 2026-05-26 20:30:28 +02:00
CloakHQ 2a99081850 docs: add recommended config to quick-start, FPJS troubleshooting, update contributors
- Add production-ready snippet (proxy, geoip, headless, humanize) near top of both READMEs
- Add "Detected by FingerprintJS?" troubleshooting with verified flags:
  noise=false, screen dimensions, storage quota, geoip, residential proxy
- Add @sparanoid to contributors (Docker Xvfb lock fix)
- Update @eofreternal credit (iframe pointer-events fix)
- Fix stale "48 patches" in JS README
2026-05-26 18:46:13 +02:00
CloakHQ 7fc577e5c6 fix(humanize): port #303 iframe pointer-events fix to Python
Mirror the JS fix from #303 in the sync and async Python actionability
checks: compute and apply the iframe coordinate offset before
elementFromPoint, and fail open when the check itself cannot run. Add
fail-open regression tests for both Python and JS.
2026-05-25 01:17:05 +02:00
EternalandGitHub 12d02c3547 fix(humanize): correct iframe coordinate offset in pointer-events check (#303)
The ElementHandle pointer-events check passed page-space click coordinates to elementFromPoint, which runs inside the target element's frame. For elements in an iframe the coordinate spaces differ, so the check looked at the wrong point and wrongly reported the element as covered. Now the iframe offset is computed and applied. Also fails open when the check itself cannot run.

Thanks @eofreternal for the fix.
2026-05-25 01:15:09 +02:00
243c1385a0 feat(js): export buildContextOptions helper (#262)
Co-authored-by: 이민재 <19909783+honor2030@users.noreply.github.com>
2026-05-25 00:15:27 +02:00
CloakHQ 0f3dc7201b chore(deps): bump JS dev dependencies
puppeteer-core 21→25 (fixes CVEs in tar-fs, ws),
typescript 5→6, @types/node 20→25, playwright-core 1.58→1.60.
Vitest stays on v1 (v4 breaks dynamic import mocking).
2026-05-24 23:57:47 +02:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
34d2f78e87 chore(deps): bump the actions group across 1 directory with 3 updates (#309)
Bumps the actions group with 3 updates in the / directory: [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action), [docker/login-action](https://github.com/docker/login-action) and [docker/build-push-action](https://github.com/docker/build-push-action).


Updates `docker/setup-buildx-action` from 4.0.0 to 4.1.0
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](https://github.com/docker/setup-buildx-action/compare/4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd...d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5)

Updates `docker/login-action` from 4.1.0 to 4.2.0
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](https://github.com/docker/login-action/compare/4907a6ddec9925e35a0a9e82d7399ccc52663121...650006c6eb7dba73a995cc03b0b2d7f5ca915bee)

Updates `docker/build-push-action` from 7.1.0 to 7.2.0
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](https://github.com/docker/build-push-action/compare/bcafcacb16a39f128d818304e6c9c0c18556b85f...f9f3042f7e2789586610d6e8b85c8f03e5195baf)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-version: 7.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: actions
- dependency-name: docker/login-action
  dependency-version: 4.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: actions
- dependency-name: docker/setup-buildx-action
  dependency-version: 4.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-24 23:40:28 +02:00
CloakHQ 41be4e0e30 chore(ci): add pip and npm ecosystems to Dependabot 2026-05-24 23:17:45 +02:00
CloakHQ 58ccdb683c fix(humanize): use shared deadline for timeout budget in frame and ElementHandle methods (#307)
Frame-level methods (click, dblclick, hover, dragAndDrop) passed the raw
timeout to each sequential operation independently, causing 3x actual
wait time when elements don't exist. ElementHandle methods had a similar
2x issue between actionability and pointer-events checks.

Port the deadline + remainingMs() pattern already used by page-level
methods. Also fix bot detection test selector after site added a hidden
duplicate submit button.
2026-05-24 23:12:07 +02:00
CloakHQ 8028ddefef feat: route HTTP proxy credentials through --proxy-server
Bypass Playwright's CDP Fetch.authRequired interceptor for authenticated
HTTP proxies by passing inline credentials via Chrome's --proxy-server
flag. Chrome sends Proxy-Authorization preemptively, avoiding the 407
round-trip that breaks on some proxies and Google domains (#182).

Gated on platform (linux-x64, windows-x64) and binary version >= 146.0.7680.177.5.
Unsupported platforms fall back to Playwright's proxy dict.
Puppeteer falls back to page.authenticate() on unsupported platforms.
2026-05-21 05:51:03 +02:00
sparanoidandGitHub 864cae2493 fix(docker): clean up stale Xvfb lock so container survives restarts (#284)
On `docker restart`, `/tmp` is preserved across container instances, so the
previous Xvfb's `/tmp/.X99-lock` survives into the new container. The new
Xvfb sees the existing lock and refuses to start, leaving the container
with no X server. Every Chrome launch then dies with "Missing X server or
$DISPLAY", and `cloakserve` returns 502 from `/json/version` forever.

Any orchestrator that restarts on unhealthy (the README-recommended
healthcheck + `restart: always`, autoheal sidecars, etc.) then enters a
permanent restart loop because every restart hits the same broken state.

This is silent for first-time users: the container appears to start
successfully (Xvfb did launch *once*), then degrades only after the first
restart. The fix is one line in the entrypoint: remove the stale lock
before starting Xvfb.

Fixes #283
2026-05-21 05:27:49 +02:00
CloakHQ 7e626ee7a1 release: v0.3.30 — binary 146.0.7680.177.5, rendering consistency fixes 2026-05-21 05:09:10 +02:00
CloakHQ b91274cc98 release: v0.3.29 — extension loading, composable JS helpers, cloakserve origin guard 2026-05-20 08:26:04 +02:00
CloakHQ 7a9a61d4de feat(js): add launchPersistentContext to Puppeteer wrapper (#261)
Expose userDataDir support via launchPersistentContext() for
cloakbrowser/puppeteer, matching the existing Playwright API.
Includes proxy auth, geoip, and humanize support.
2026-05-18 18:32:45 +02:00
34bc095b65 fix: guard cloakserve websocket origins (#240)
Co-authored-by: 이민재 <19909783+honor2030@users.noreply.github.com>
2026-05-17 19:40:44 +02:00
a23268c9e9 feat(js): export composable launch helpers (#244)
Co-authored-by: 이민재 <19909783+honor2030@users.noreply.github.com>
2026-05-17 19:15:39 +02:00
CloakHQ 0437a3f1f5 docs: update contributors and add extension_paths examples 2026-05-15 21:18:49 +02:00
zackandGitHub 8fdaa5a2d3 feat: add extension_paths parameter for loading Chrome extensions (#210)
Add `extension_paths` parameter to all launch functions (Python + JS) for loading Chrome extensions.

Resolves paths to absolute, injects `--load-extension` and `--disable-extensions-except` flags via `build_args()`.

Note: Extensions require a persistent context (`launch_persistent_context`) to function — this is a Chromium limitation.

Co-authored-by: zackycodes <75211659+zackycodes@users.noreply.github.com>
2026-05-15 21:08:42 +02:00
Cloak-HQandGitHub b0ea580cba feat(humanize): add Playwright-style actionability checks (#228)
* feat(humanize): add Playwright-style actionability checks to all interaction methods

Humanized locator/page methods now perform pre-action validation matching
Playwright's native behavior: attached, visible, enabled, editable, stable,
and receives-pointer-events checks with retry loop and backoff.

- New error hierarchy: ActionabilityError base with ElementNotAttachedError,
  ElementNotVisibleError, ElementNotStableError, ElementNotEnabledError,
  ElementNotEditableError, ElementNotReceivingEventsError
- force=True parameter skips all actionability checks (matches Playwright)
- Shared deadline across all steps (checks + scroll + stable + pointer)
- Post-scroll stability check only runs when scroll actually happened
- Chained methods (type/fill/check/uncheck/press) skip inner click checks
  but still run pointer-events check at actual click coordinates
- Frame methods now forward kwargs (force, timeout, human_config)
- Locator patches forward force via _forward_kwargs
- Python sync + async, JS/TS implementation

* fix(humanize): forward human_config in all chained methods, use evaluate args in handle pointer checks

- Add human_config=kwargs.get("human_config") to check/uncheck/select_option/press inner calls (sync+async+JS)
- Convert check_pointer_events_handle from f-string interpolation to evaluate args pattern (sync+async+JS)

* fix(humanize): strip custom kwargs before forwarding to Playwright select_option

originals.select_option(**kwargs) passes human_config/force to Playwright
which rejects unknown kwargs with TypeError.
2026-05-15 20:57:17 +02:00
CloakHQ 6f4f92e7c7 fix(security): add URL validation and SSRF protection to Lambda handler (#233)
Restrict Lambda handler to http/https URLs, block private/internal IPs,
remove caller-controlled extra_args and wait_for_function, re-validate
URL after navigation to catch redirect-based SSRF.
2026-05-13 18:55:07 +02:00
Sergey ZaborovskyandGitHub ad4d946ca6 Add flake.nix for Nix / NixOS (#220)
* feat: add flake.nix

* refactor: improve code style and add more information to flake.nix

* chore(nix): ignore build result symlink

* chore(nix): use unversioned pytest packages
2026-05-12 23:52:55 +02:00
@aaronjmarsandGitHub 95a98b6747 fix(security): isolate workflow_dispatch input to avoid shell injection in attest-release (#223)
Security hardening: route workflow_dispatch input through env var to prevent shell injection in attest-release workflow.
2026-05-12 15:53:31 +02:00
23f1d4098c fix(security): bump tar + transitive deps via npm audit fix (#222)
Detected by Aeon + osv-scanner.
Severity: high (runtime tar) / high+moderate (dev deps)

Patches 8 of 14 CVEs flagged by osv-scanner — all that can be fixed
within current semver ranges via `npm audit fix --package-lock-only`.
The remaining 6 are gated on a puppeteer-core/vitest major-version
bump (out of scope for this PR).

Runtime (shipped to users):
- tar 7.5.9 -> 7.5.15
  - GHSA-9ppj-qmqm-q256 HIGH: Symlink Path Traversal via Drive-Relative Linkpath
  - GHSA-qffp-2rhf-9h96 HIGH: Hardlink Path Traversal via Drive-Relative Linkpath
  - Reachable in js/src/download.ts (extractTar) — the existing filter() rejects
    absolute paths and "..", but does not inspect linkpath, so a malicious
    Chromium tarball could write outside the cache dir on Windows.

Dev (build-time only):
- basic-ftp 5.2.0 -> 5.3.1 (4 HIGH: CRLF injection x2, DoS x2)
- ip-address 10.1.0 -> 10.2.0 (1 MOD: XSS in Address6 HTML methods)
- postcss 8.5.6 -> 8.5.14 (1 MOD: XSS via unescaped </style>)

Lockfile metadata side-effects (npm-regenerated, not editorial):
- name@version block synced from package.json (0.3.23 -> 0.3.28)
- devDependencies + peerDependencies version ranges synced to current
  package.json (the lockfile was stale relative to head package.json)

Verification:
- `npm test` -> 320 passed / 11 skipped / 0 failed (9 test files)
- `npm run typecheck` -> clean
- osv-scanner before: 14 CVEs; after: 6 (those 6 need a breaking
  major-version bump to land — happy to follow up if you want it)

Co-authored-by: Aeon <aeon@aaronjmars.eth>
2026-05-12 15:47:26 +02:00
Novi Kurnia HutapeaandGitHub d45d7de9a9 chore(js): sync package-lock metadata (#219) 2026-05-12 15:39:22 +02:00
CloakHQ db0b5f1946 release: v0.3.28 — cloakserve path traversal fix, GeoIP timeout guard, humanize iframe scope 2026-05-11 21:43:14 +02:00
CloakHQ babef04e07 fix(cloakserve): sanitize fingerprint seed to prevent path traversal (#217)
Validate seed format with strict regex, add path containment check
before rmtree, and bind to 127.0.0.1 by default on bare metal.
2026-05-11 21:36:09 +02:00
CloakHQ f8026a7b39 chore: clean up GeoIP timeout follow-up (#213)
Remove dead null checks, document CLOAKBROWSER_GEOIP_TIMEOUT_SECONDS
env var, credit contributor.

Fix review findings:
- Use timeout-bounded resolve_proxy_exit_ip in _resolve_webrtc_args
- Add missing timeout handler on tunneled HTTPS request in JS
- Reject nan/inf in Python timeout parsing (parity with JS)
- Recompute deadline after CONNECT succeeds in JS proxy tunnel
2026-05-11 21:15:36 +02:00
manaskarraandGitHub 71f57d00d1 fix: bound GeoIP resolution so launch cannot hang (#213)
* Fix geoip resolution timeout

* fix: keep GeoIP timeout inside resolution path
2026-05-11 20:55:49 +02:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
e9735392e8 chore(deps): bump sigstore/cosign-installer in the actions group (#214)
Bumps the actions group with 1 update: [sigstore/cosign-installer](https://github.com/sigstore/cosign-installer).


Updates `sigstore/cosign-installer` from 4.1.1 to 4.1.2
- [Release notes](https://github.com/sigstore/cosign-installer/releases)
- [Commits](https://github.com/sigstore/cosign-installer/compare/cad07c2e89fa2edd6e2d7bab4c1aa38e53f76003...6f9f17788090df1f26f669e9d70d6ae9567deba6)

---
updated-dependencies:
- dependency-name: sigstore/cosign-installer
  dependency-version: 4.1.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-11 20:55:41 +02:00
CloakHQ 0d41a4f023 refactor(js): extract HumanActionOptions type, fix frame check/uncheck error handling, align SOCKS5 log level
- Extract HumanActionOptions type alias to replace ~40 inline copies
- Frame check/uncheck: let isChecked errors propagate instead of silently clicking non-checkbox elements
- SOCKS5 credential log: console.debug → console.info (parity with Python logger.info)
- Add contributors to README
2026-05-11 00:23:31 +02:00
EternalandCloakHQ 80d9f7c14e feat(js): add types to humanized method options (#205)
Replace `any` with proper TypeScript types on all humanized method options
(Playwright, Puppeteer, ElementHandle, Frame). Support flat per-call config
overrides alongside existing `human_config` style. Internalize `humanIdle`
duration computation with backward-compatible overloads.

Co-authored-by: Eternal <chasezou09@gmail.com>
2026-05-10 23:50:01 +02:00
114b3c826b fix(js): preserve iframe scope in humanized frame actions (#201)
fix(js): preserve iframe scope in humanized frame actions

Frame actions (click, type, fill, etc.) now resolve selectors through frame.locator() instead of delegating to page.* methods, fixing iframe-scoped interactions.

Closes #184

Co-authored-by: manaskarra <manas.karra@gmail.com>
2026-05-10 18:13:29 +02:00
YouhaiandGitHub c07c2b6b4a fix(proxy): log when SOCKS5 credential auto-encoding rewrites URL (#157) (#209)
* fix(proxy): log when SOCKS5 credential auto-encoding rewrites URL (#157)

Auto URL-encoding of SOCKS5 credentials (added in v0.3.26 to fix Chromium's
'=' truncation bug) currently happens silently. Users debugging connectivity
have no way to know the wrapper rewrote their proxy URL — the original #157
thread took 8 round-trips to surface this exact ambiguity.

Emit a log when re-encoding actually changes the URL: INFO on Python's
'cloakbrowser' logger, console.debug in JavaScript. Stays silent on
already-encoded inputs and credential-less URLs to avoid false-positive
noise. Credentials are not included in the log message.

Tests: 3 new cases per language (Python caplog, JS vi.spyOn console.debug)
covering trigger / silent-when-encoded / silent-when-no-creds.

* fix(proxy): gate log on credential change, not full URL diff

Per Copilot review on #209: urlparse cosmetically lowercases scheme and
hostname, so comparing the full reconstructed URL to the input would emit
"Auto URL-encoded SOCKS5..." even for inputs like
`socks5://USER:pass@HOST.com:1080` where no credential encoding happened.

Compare raw vs encoded user/password substrings instead. Mirror the same
condition in JS for parity (JS's manual parser preserves case today, but the
credential-level compare is more robust against future changes).

Adds one regression test per language.
2026-05-10 18:09:46 +02:00
CloakHQ 13b1b98b68 fix(js): bump playwright-core minimum to >=1.53.0 (#200)
playwright-core <=1.52.0 injects __pwInitScripts into window, which
deviceandbrowserinfo.com detects as isPlaywright:true. Fixed in 1.53.0.
2026-05-07 17:31:07 +02:00
55 changed files with 5262 additions and 1358 deletions
+18
View File
@@ -8,3 +8,21 @@ updates:
actions:
patterns:
- "*"
- package-ecosystem: "pip"
directory: "/"
schedule:
interval: "weekly"
groups:
python:
patterns:
- "*"
- package-ecosystem: "npm"
directory: "/js"
schedule:
interval: "weekly"
groups:
javascript:
patterns:
- "*"
+2 -1
View File
@@ -16,9 +16,10 @@ jobs:
contents: write # Download release assets
steps:
- name: Download release binaries
run: gh release download ${{ github.event.inputs.tag }} --repo CloakHQ/cloakbrowser --pattern "cloakbrowser-*.tar.gz" --pattern "cloakbrowser-*.zip"
run: gh release download "$RELEASE_TAG" --repo CloakHQ/cloakbrowser --pattern "cloakbrowser-*.tar.gz" --pattern "cloakbrowser-*.zip"
env:
GH_TOKEN: ${{ github.token }}
RELEASE_TAG: ${{ github.event.inputs.tag }}
- name: Attest build provenance
uses: actions/attest-build-provenance@a2bbfa25375fe432b6a289bc6b6cd05ecd0c4c32 # v4.1.0
+4 -4
View File
@@ -106,14 +106,14 @@ jobs:
VERSION=$(python -c 'import re; print(re.search(r"__version__\s*=\s*[\"'\'']([^\"'\'']+)", open("cloakbrowser/_version.py").read()).group(1))')
echo "VERSION=$VERSION" >> $GITHUB_ENV
- uses: docker/setup-qemu-action@ce360397dd3f832beb865e1373c09c0e9f86d70a # v4.0.0
- uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
- uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
- uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0
- uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0
with:
username: ${{ secrets.DOCKER_USER }}
password: ${{ secrets.DOCKER_PAT }}
- name: Build and push
id: build
uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
uses: docker/build-push-action@f9f3042f7e2789586610d6e8b85c8f03e5195baf # v7.2.0
with:
context: .
platforms: linux/amd64,linux/arm64
@@ -123,7 +123,7 @@ jobs:
cloakhq/cloakbrowser:latest
provenance: true
sbom: true
- uses: sigstore/cosign-installer@cad07c2e89fa2edd6e2d7bab4c1aa38e53f76003 # v4.1.1
- uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6 # v4.1.2
- name: Sign image
run: cosign sign --yes cloakhq/cloakbrowser@${{ steps.build.outputs.digest }}
- name: Attest build provenance
+1
View File
@@ -46,6 +46,7 @@ js/dist/
*.whl
AGENTS.md
.beads
result
# Private docs (launch posts, strategy)
docs/
+41
View File
@@ -8,6 +8,47 @@ Changes are tagged: **[wrapper]** for Python/JS wrapper, **[binary]** for Chromi
## [Unreleased]
## [0.3.31] — 2026-05-26
- **[wrapper]** Route HTTP proxy credentials through `--proxy-server` flag, removing the need for Playwright's proxy auth handler on HTTP proxies
- **[wrapper]** JS: export `buildContextOptions` helper for custom context creation (thanks [@honor2030](https://github.com/honor2030), #262)
- **[wrapper]** Humanize: fix iframe coordinate offset in pointer-events check (thanks [@eofreternal](https://github.com/eofreternal), #303)
- **[wrapper]** Humanize: use shared deadline for timeout budget in frame and ElementHandle methods (#307)
- **[docker]** Clean up stale Xvfb lock so container survives restarts (thanks [@sparanoid](https://github.com/sparanoid), #284)
- **[meta]** Add pip and npm ecosystems to Dependabot, bump GitHub Actions (#309)
## [0.3.30] — 2026-05-21
- **[binary]** New build 146.0.7680.177.5 for Linux x64 + Windows x64 — 58 source-level fingerprint patches (up from 57)
- **[binary]** Rendering consistency improvements across Linux and Windows — corrected GPU, display, and graphics parameters to match stock Chrome 146 profiles
- **[binary]** Windows: native GPU/rendering values now pass through directly instead of being spoofed, matching real hardware behavior
- **[binary]** Storage normalization fix for Windows
- **[binary]** HTTP proxy inline credential support at the network layer
- **[wrapper]** Update `PLATFORM_CHROMIUM_VERSIONS` for linux-x64 and windows-x64 to 146.0.7680.177.5
## [0.3.29] — 2026-05-20
- **[wrapper]** **Security**: `cloakserve` — guard WebSocket origins to prevent browser-origin CSRF via CDP proxy (thanks [@0xlally](https://github.com/0xlally) for the report, [@honor2030](https://github.com/honor2030) for the fix, #239, #240)
- **[wrapper]** **Security**: Lambda example — add URL scheme validation, SSRF protection, post-navigation re-validation, remove unsafe caller-controlled options (#233)
- **[wrapper]** **Security**: CI — isolate `workflow_dispatch` input to avoid shell injection in attest-release (thanks [@aaronjmars](https://github.com/aaronjmars), #223)
- **[wrapper]** **Security**: JS — bump tar + transitive deps via npm audit fix (thanks [@aaronjmars](https://github.com/aaronjmars), #222)
- **[wrapper]** Add `extension_paths` parameter for loading Chrome extensions in all launch functions (thanks [@zackycodes](https://github.com/zackycodes), #210)
- **[wrapper]** Humanize: add Playwright-style actionability checks — auto-wait for visible, enabled, stable elements before humanized actions (#228)
- **[wrapper]** JS: export composable launch helpers — `buildLaunchOptions()` and `humanizeBrowser()` for custom Playwright integrations (thanks [@honor2030](https://github.com/honor2030), #244)
- **[wrapper]** JS: add `launchPersistentContext()` to Puppeteer wrapper (#261)
- **[wrapper]** Add `flake.nix` for Nix/NixOS (thanks [@Seryiza](https://github.com/Seryiza), #220)
- **[meta]** JS: sync package-lock metadata (thanks [@245678000000](https://github.com/245678000000), #219)
## [0.3.28] — 2026-05-11
- **[wrapper]** **Security**: `cloakserve` — sanitize fingerprint seed to prevent path traversal, bind to `127.0.0.1` on bare metal, detect Podman containers (#217)
- **[wrapper]** Fix GeoIP resolution hanging indefinitely — bounded with 10s timeout so `launch()` cannot stall (thanks [@manaskarra](https://github.com/manaskarra), #213)
- **[wrapper]** JS: preserve iframe scope in humanized frame actions — `check()`, `uncheck()`, `selectOption()` now execute in the correct frame (thanks [@manaskarra](https://github.com/manaskarra), #201)
- **[wrapper]** JS: add TypeScript types to humanized method options — `HumanActionOptions` type for `human_config` and `timeout` overrides (thanks [@eofreternal](https://github.com/eofreternal), #205)
- **[wrapper]** Log when SOCKS5 credential auto-encoding rewrites a proxy URL (thanks [@Youhai020616](https://github.com/Youhai020616), #209)
- **[wrapper]** JS: bump `playwright-core` peer dependency minimum to >=1.53.0 (#200)
- **[meta]** Bump sigstore/cosign-installer in CI (#214)
## [0.3.27] — 2026-05-06
- **[wrapper]** Per-call `human_config` override — pass `human_config={...}` to individual humanized methods to override global HumanConfig on a per-action basis (#183)
+110 -23
View File
@@ -40,7 +40,7 @@ Drop-in Playwright/Puppeteer replacement for Python and JavaScript.<br>
Same API, same code — just swap the import. <strong>3 lines of code, 30 seconds to unblock.</strong>
</p>
- **49 source-level C++ patches** — canvas, WebGL, audio, fonts, GPU, screen, WebRTC, network timing, automation signals, CDP input behavior
- **58 source-level C++ patches** — canvas, WebGL, audio, fonts, GPU, screen, WebRTC, network timing, automation signals, CDP input behavior
- **`humanize=True`** — human-like mouse curves, keyboard timing, and scroll patterns. One flag, behavioral detection passes
- **0.9 reCAPTCHA v3 score** — human-level, server-verified
- **Passes Cloudflare Turnstile**, FingerprintJS, BrowserScan — tested against 30+ detection sites
@@ -59,7 +59,7 @@ from cloakbrowser import launch
browser = launch()
page = browser.new_page()
page.goto("https://protected-site.com") # no more blocks
page.goto("https://example.com")
browser.close()
```
@@ -69,12 +69,34 @@ import { launch } from 'cloakbrowser';
const browser = await launch();
const page = await browser.newPage();
await page.goto('https://protected-site.com');
await page.goto('https://example.com');
await browser.close();
```
Also works with Puppeteer: `import { launch } from 'cloakbrowser/puppeteer'` ([details](#puppeteer))
**For sites with anti-bot protection**, add a residential proxy and these flags:
```python
browser = launch(
proxy="http://user:pass@residential-proxy:port", # residential IP, not datacenter
geoip=True, # match timezone + locale to proxy IP
headless=False, # some sites detect headless even with C++ patches
humanize=True, # human-like mouse, keyboard, scroll
)
```
```javascript
const browser = await launch({
proxy: 'http://user:pass@residential-proxy:port',
geoip: true,
headless: false,
humanize: true,
});
```
See [Troubleshooting](#troubleshooting) for site-specific issues (FingerprintJS, Kasada, reCAPTCHA).
## Install
**Python:**
@@ -128,17 +150,19 @@ Open [http://localhost:8080](http://localhost:8080). Create a profile. Click **L
---
## Latest: v0.3.26 (Chromium 146.0.7680.177.4)
## Latest: v0.3.31 (Chromium 146.0.7680.177.5)
- **`launch_context_async()`** — async counterpart to `launch_context()`. Forwards kwargs to `browser.new_context()` for `storage_state`, `permissions`, `extra_http_headers` without a persistent profile folder.
- **JS `contextOptions` escape hatch** — forward arbitrary options (including `storageState`) to Playwright's `newContext()` from `launchContext()` / `launchPersistentContext()`.
- **Native SOCKS5 proxy** — `proxy="socks5://user:pass@host:port"` works directly in all launch functions, Python + JS. QUIC/HTTP3 tunnels through SOCKS5 via UDP ASSOCIATE.
- **Chromium 146 upgrade** — rebased all patches from 145.0.7632.x to 146.0.7680.177
- **57 fingerprint patches** — additional detection-vector coverage (WebAuthn, AAC audio, window position) and WebGL/canvas consistency fixes
- **WebRTC IP spoofing** — `--fingerprint-webrtc-ip=auto` resolves your proxy's exit IP and spoofs WebRTC ICE candidates. Auto-injected when using `geoip=True` (no extra network call)
- **58 fingerprint patches** — rendering consistency improvements across Linux and Windows, corrected GPU/display/graphics parameters to match stock Chrome 146 profiles
- **Windows native GPU passthrough** — real hardware values pass through directly instead of being spoofed, matching real browser behavior
- **HTTP proxy inline credentials** — new network-layer support for proxies with inline authentication
- **`extension_paths`** — load Chrome extensions in all launch functions
- **Humanize actionability** — auto-wait for visible, enabled, stable elements before humanized actions
- **Per-call `human_config`** — override humanize settings on individual method calls
- **Composable JS helpers** — `buildLaunchOptions()` and `humanizeBrowser()` for custom Playwright integrations
- **Native SOCKS5 proxy** — `proxy="socks5://user:pass@host:port"` works directly in all launch functions, Python + JS. QUIC/HTTP3 tunnels through SOCKS5 via UDP ASSOCIATE
- **Proxy signal removal** — DNS/connect/SSL timing zeroed, proxy cache headers stripped, Proxy-Connection header leak removed
- **`cloakserve` CDP multiplexer** — rewritten as a multi-connection CDP proxy with per-connection fingerprint seeds
- **Humanize CDP isolation** — keyboard events now use isolated worlds and trusted dispatch for better behavioral stealth
- **Chromium 146 upgrade** — rebased all patches from 145.0.7632.x to 146.0.7680.177
- **WebRTC IP spoofing** — `--fingerprint-webrtc-ip=auto` resolves your proxy's exit IP and spoofs WebRTC ICE candidates. Auto-injected when using `geoip=True` (no extra network call)
- **`humanize=True`** — one flag makes all mouse, keyboard, and scroll interactions behave like a real user. Bézier curves, per-character typing, realistic scroll patterns
- **Stealthy with zero flags** — binary auto-generates a random fingerprint seed at startup. No configuration required
- **Timezone & locale from proxy IP** — `launch(proxy="...", geoip=True)` auto-detects timezone and locale
@@ -226,7 +250,7 @@ CloakBrowser is a thin wrapper (Python + JavaScript) around a custom-built Chrom
3. **Every launch** → Playwright or Puppeteer starts with our binary + stealth args
4. **You write code** → standard Playwright/Puppeteer API, nothing new to learn
The binary includes 49 source-level patches covering canvas, WebGL, audio, fonts, GPU, screen properties, WebRTC, network timing, hardware reporting, automation signal removal, and CDP input behavior mimicking.
The binary includes 58 source-level patches covering canvas, WebGL, audio, fonts, GPU, screen properties, WebRTC, network timing, hardware reporting, automation signal removal, and CDP input behavior mimicking.
These are compiled into the Chromium binary — not injected via JavaScript, not set via flags.
@@ -371,9 +395,16 @@ ctx.close() # profile saved
# Next run — cookies, localStorage restored automatically
ctx = launch_persistent_context("./my-profile", headless=False)
# Load Chrome extensions
ctx = launch_persistent_context(
"./my-profile",
headless=False,
extension_paths=["./my-extension"],
)
```
Supports all the same options as `launch_context()`: `proxy`, `user_agent`, `viewport`, `locale`, `timezone`, `color_scheme`, `geoip`.
Supports all the same options as `launch_context()`: `proxy`, `user_agent`, `viewport`, `locale`, `timezone`, `color_scheme`, `geoip`, `extension_paths`.
Async version: `launch_persistent_context_async()`.
@@ -406,7 +437,7 @@ from cloakbrowser import binary_info, clear_cache, ensure_binary
# Check binary installation status
print(binary_info())
# {'version': '146.0.7680.177.3', 'platform': 'linux-x64', 'installed': True, ...}
# {'version': '146.0.7680.177.5', 'platform': 'linux-x64', 'installed': True, ...}
# Force re-download
clear_cache()
@@ -570,6 +601,7 @@ Access the original un-patched Playwright page at `page._original` if you need r
| `CLOAKBROWSER_DOWNLOAD_URL` | `cloakbrowser.dev` | Custom download URL for binary |
| `CLOAKBROWSER_AUTO_UPDATE` | `true` | Set to `false` to disable background update checks |
| `CLOAKBROWSER_SKIP_CHECKSUM` | `false` | Set to `true` to skip SHA-256 verification after download |
| `CLOAKBROWSER_GEOIP_TIMEOUT_SECONDS` | `5` | Max seconds for GeoIP resolution before continuing without it |
## Fingerprint Management
@@ -739,11 +771,11 @@ browser = await launch_async(args=["--remote-debugging-port=9242"])
| Platform | Chromium | Patches | Status |
|---|---|---|---|
| Linux x86_64 | 146 | 57 | ✅ Latest |
| Linux arm64 (RPi, Graviton) | 146 | 57 | ✅ Latest |
| Linux x86_64 | 146 | 58 | ✅ Latest |
| Linux arm64 (RPi, Graviton) | 146 | 58 | ✅ |
| macOS arm64 (Apple Silicon) | 145 | 26 | ✅ |
| macOS x86_64 (Intel) | 145 | 26 | ✅ |
| Windows x86_64 | 146 | 57 | ✅ Latest |
| Windows x86_64 | 146 | 58 | ✅ Latest |
The wrapper auto-downloads the correct binary for your platform.
@@ -976,6 +1008,51 @@ If you're still blocked after this, check the font setup below.
---
### Detected by FingerprintJS?
FingerprintJS (`demo.fingerprint.com/playground`) checks multiple signals. Each detection has a specific cause:
| Detection | Cause | Fix |
|-----------|-------|-----|
| **`nodriver` / bad bot** | IP reputation or missing flags | Residential proxy + config below |
| **Browser tampering** | Noise injection detected by ML | `--fingerprint-noise=false` |
| **Virtual machine** | Screen dimensions don't match viewport | `--fingerprint-screen-width/height` matching viewport |
| **Incognito** | Storage quota normalized to ~500MB | Expected tradeoff — see below |
Config that passes FPJS (verified on v0.3.30, Linux + Windows):
```python
browser = launch(
headless=False,
proxy="http://user:pass@residential-proxy:port",
geoip=True,
args=[
"--fingerprint-noise=false", # prevents tampering detection
"--fingerprint-screen-width=1920", # match your viewport
"--fingerprint-screen-height=1080",
],
)
```
```javascript
const browser = await launch({
headless: false,
proxy: 'http://user:pass@residential-proxy:port',
geoip: true,
args: [
'--fingerprint-noise=false',
'--fingerprint-screen-width=1920',
'--fingerprint-screen-height=1080',
],
});
```
For persistent contexts (`launch_persistent_context` / `launchPersistentContext`), also add `--fingerprint-storage-quota=500` to the args.
**Storage quota tradeoff:** The binary normalizes storage quota to ~500MB to pass FPJS, but this makes the session look like incognito to other detection services (e.g. BrowserScan's `notPrivate` check, -10 points). Setting `--fingerprint-storage-quota=5000` passes incognito checks but may trigger FPJS. You can't satisfy both simultaneously — choose based on what your target site checks. See the [storage quota tradeoff table](#launch_persistent_context) for details.
---
### Blocked on Kasada / Akamai sites despite correct config?
On minimal Linux environments, missing font packages cause canvas emoji rendering to produce hashes that anti-bot systems don't recognize. This is the most common cause of blocks on aggressive sites after proxy, geoip, and headed mode are already set up correctly.
@@ -1142,9 +1219,9 @@ A: Yes. Pass `proxy="http://user:pass@host:port"` or `proxy="socks5://user:pass@
| Feature | Status |
|---------|--------|
| Linux x64 — Chromium 146 (57 patches) | ✅ Released |
| Linux x64 — Chromium 146 (58 patches) | ✅ Released |
| macOS arm64/x64 — Chromium 145 (26 patches) | ✅ Released |
| Windows x64 — Chromium 146 (57 patches) | ✅ Released |
| Windows x64 — Chromium 146 (58 patches) | ✅ Released |
| JavaScript/Puppeteer + Playwright support | ✅ Released |
| Fingerprint rotation per session | ✅ Released |
| Built-in proxy rotation | 📋 Planned |
@@ -1166,7 +1243,7 @@ All releases are signed for supply chain verification.
```bash
# Verify GPG signature (binary release tag)
gpg --keyserver keyserver.ubuntu.com --recv-keys C60C0DDC9D0DE2DD
git verify-tag chromium-v146.0.7680.177.3
git verify-tag chromium-v146.0.7680.177.5
# Verify GitHub binary attestation (Sigstore)
gh attestation verify cloakbrowser-linux-x64.tar.gz --repo CloakHQ/cloakbrowser
@@ -1192,5 +1269,15 @@ Issues and PRs welcome. If something isn't working, [open an issue](https://gith
- [@evelaa123](https://github.com/evelaa123) — humanize behavior, persistent contexts, Windows fix
- [@yahooguntu](https://github.com/yahooguntu) — persistent contexts
- [@kitiho](https://github.com/kitiho) — null viewport fix
- [@eofreternal](https://github.com/eofreternal) — humanConfig type fix
- [@AlexTech314](https://github.com/AlexTech314) — AWS Lambda integration
- [@eofreternal](https://github.com/eofreternal) — humanConfig type fix, humanized method option types, iframe pointer-events fix
- [@manaskarra](https://github.com/manaskarra) — iframe scope fix for humanized frame actions, GeoIP timeout guard
- [@Youhai020616](https://github.com/Youhai020616) — SOCKS5 credential encoding logging
- [@AlexTech314](https://github.com/AlexTech314) — AWS Lambda integration, cold-start hardening
- [@dgtlmoon](https://github.com/dgtlmoon) — graceful pw.stop() cleanup
- [@zackycodes](https://github.com/zackycodes) — Chrome extension loading
- [@aaronjmars](https://github.com/aaronjmars) — security fixes (shell injection, dep bumps)
- [@Seryiza](https://github.com/Seryiza) — Nix/NixOS flake
- [@245678000000](https://github.com/245678000000) — package-lock sync
- [@honor2030](https://github.com/honor2030) — cloakserve WebSocket origin guard, composable JS launch helpers
- [@sparanoid](https://github.com/sparanoid) — Docker Xvfb lock cleanup
- [@0xlally](https://github.com/0xlally) — security reports (cloakserve path traversal, WebSocket origin bypass)
+128 -9
View File
@@ -18,17 +18,19 @@ Client:
from __future__ import annotations
import asyncio
import ipaddress
import json
import logging
import os
import random
import re
import shutil
import socket
import subprocess
import sys
import time
from dataclasses import dataclass
from urllib.parse import parse_qs
from urllib.parse import parse_qs, urlparse
from pathlib import Path
@@ -61,6 +63,94 @@ BASE_CHROME_ARGS = [
BASE_CDP_PORT = 5100
SAFE_SEED_RE = re.compile(r"^[A-Za-z0-9_-]{1,128}$")
RESERVED_SEEDS = {"__default__"}
TRUSTED_WS_ORIGINS = {"devtools://devtools", "chrome-devtools://devtools"}
def _host_port_from_netloc(netloc: str, default_port: int) -> tuple[str, int] | None:
"""Return a normalized (host, port) pair for an Origin/Host netloc."""
if "," in netloc:
return None
try:
parsed = urlparse(f"//{netloc.strip()}")
authority = parsed.netloc.rsplit("@", 1)[-1]
if (
not parsed.hostname
or parsed.username is not None
or parsed.password is not None
or authority.endswith(":")
or parsed.path
or parsed.params
or parsed.query
or parsed.fragment
):
return None
return (parsed.hostname.lower(), parsed.port if parsed.port is not None else default_port)
except ValueError:
return None
def _is_loopback_host(hostname: str) -> bool:
"""Return True for localhost and loopback IP literals."""
hostname = hostname.strip("[]").rstrip(".").lower()
if hostname == "localhost":
return True
try:
return ipaddress.ip_address(hostname).is_loopback
except ValueError:
return False
def _origin_is_allowed(
origin: str | None,
host: str | None,
request_scheme: str = "http",
) -> bool:
"""Return True when a WebSocket Origin is safe to proxy to local CDP."""
if origin is None:
# Playwright/Puppeteer and other non-browser CDP clients commonly omit
# Origin. Keep those clients working while rejecting browser-origin CSRF.
return True
origin = origin.strip()
if not origin or origin.lower() == "null":
return False
if origin in TRUSTED_WS_ORIGINS:
return True
try:
parsed = urlparse(origin)
except ValueError:
return False
if parsed.scheme not in ("http", "https"):
return False
if parsed.path or parsed.params or parsed.query or parsed.fragment:
return False
origin_default_port = 443 if parsed.scheme == "https" else 80
request_scheme = request_scheme.split(",", 1)[0].strip().lower()
request_default_port = 443 if request_scheme in ("https", "wss") else 80
origin_host = _host_port_from_netloc(parsed.netloc, origin_default_port)
request_host = _host_port_from_netloc(host or "", request_default_port)
if origin_host is None or request_host is None:
return False
if not _is_loopback_host(request_host[0]):
return False
return origin_host == request_host
def _reject_untrusted_origin(request: web.Request) -> web.Response | None:
"""Reject browser-origin WebSocket upgrades that would expose local CDP."""
origin = request.headers.get("Origin")
host = request.headers.get("Host")
scheme = request.headers.get("X-Forwarded-Proto", getattr(request, "scheme", "http"))
if _origin_is_allowed(origin, host, request_scheme=scheme):
return None
logger.warning("Rejected CDP WebSocket from untrusted Origin %r for Host %r", origin, host)
return web.Response(status=403, text="Forbidden: untrusted WebSocket origin\n")
# ---------------------------------------------------------------------------
# ChromeProcess — one running Chrome instance
@@ -111,6 +201,14 @@ class ChromePool:
self._locks[seed] = asyncio.Lock()
return self._locks[seed]
def _safe_rmtree(self, path: str) -> None:
resolved = Path(path).resolve()
data_resolved = Path(self._data_dir).resolve()
if resolved == data_resolved or not resolved.is_relative_to(data_resolved):
logger.error("Refusing to delete path outside data_dir: %s", resolved)
return
shutil.rmtree(path, True)
def _allocate_port(self) -> int:
"""Find a free port starting from _next_port."""
for _ in range(100):
@@ -159,6 +257,11 @@ class ChromePool:
seed_key = "__default__"
actual_seed = str(random.randint(10000, 99999))
else:
if not SAFE_SEED_RE.match(seed) or seed in RESERVED_SEEDS:
raise web.HTTPBadRequest(
text=json.dumps({"error": "Invalid fingerprint seed"}),
content_type="application/json",
)
seed_key = seed
actual_seed = seed
@@ -232,7 +335,7 @@ class ChromePool:
if not await self._wait_for_cdp(port):
process.kill()
await asyncio.to_thread(process.wait, timeout=5)
await asyncio.to_thread(shutil.rmtree, user_data_dir, True)
await asyncio.to_thread(self._safe_rmtree, user_data_dir)
raise web.HTTPBadGateway(
text=json.dumps({"error": "Chrome failed to start"}),
content_type="application/json",
@@ -266,8 +369,7 @@ class ChromePool:
await asyncio.to_thread(proc.process.wait, timeout=5)
except subprocess.TimeoutExpired:
proc.process.kill()
# Clean up user data dir (can be slow for large profiles)
await asyncio.to_thread(shutil.rmtree, proc.user_data_dir, True)
await asyncio.to_thread(self._safe_rmtree, proc.user_data_dir)
if self._default is proc:
self._default = None
self._locks.pop(key, None)
@@ -511,8 +613,12 @@ async def proxy_cdp_websocket(
logger.error("%s error: %s", label, exc)
async def handle_ws_default(request: web.Request) -> web.WebSocketResponse:
async def handle_ws_default(request: web.Request) -> web.StreamResponse:
"""WebSocket proxy for default (no-seed) Chrome: /devtools/{type}/{guid}"""
rejected = _reject_untrusted_origin(request)
if rejected is not None:
return rejected
pool: ChromePool = request.app["pool"]
path = request.match_info.get("path", "")
@@ -530,8 +636,12 @@ async def handle_ws_default(request: web.Request) -> web.WebSocketResponse:
return ws
async def handle_ws_seed(request: web.Request) -> web.WebSocketResponse:
async def handle_ws_seed(request: web.Request) -> web.StreamResponse:
"""WebSocket proxy for seed-specific Chrome: /fingerprint/{seed}/devtools/{type}/{guid}"""
rejected = _reject_untrusted_origin(request)
if rejected is not None:
return rejected
pool: ChromePool = request.app["pool"]
seed = request.match_info["seed"]
path = request.match_info.get("path", "")
@@ -559,8 +669,8 @@ async def on_shutdown(app: web.Application) -> None:
# ---------------------------------------------------------------------------
def _default_data_dir() -> str:
"""Smart default: Docker → /tmp/cloakserve, bare metal → ~/.cloakbrowser/cloakserve."""
if os.path.exists("/.dockerenv"):
"""Smart default: container → /tmp/cloakserve, bare metal → ~/.cloakbrowser/cloakserve."""
if os.path.exists("/.dockerenv") or os.path.exists("/run/.containerenv"):
return "/tmp/cloakserve"
return str(Path.home() / ".cloakbrowser" / "cloakserve")
@@ -624,6 +734,13 @@ def main() -> None:
binary = ensure_binary()
config, global_args = parse_cli_args(sys.argv[1:])
if config["default_seed"] and (
not SAFE_SEED_RE.match(config["default_seed"])
or config["default_seed"] in RESERVED_SEEDS
):
logger.error("Invalid --fingerprint seed: %s", config["default_seed"])
sys.exit(1)
pool = ChromePool(
binary=binary,
global_args=global_args,
@@ -662,7 +779,9 @@ def main() -> None:
port,
)
web.run_app(app, host="0.0.0.0", port=port, print=None)
in_container = os.path.exists("/.dockerenv") or os.path.exists("/run/.containerenv")
host = "0.0.0.0" if in_container else "127.0.0.1"
web.run_app(app, host=host, port=port, print=None)
if __name__ == "__main__":
+8
View File
@@ -1,4 +1,12 @@
#!/bin/bash
# Clean up any stale Xvfb lock left behind by a previous container instance.
# `/tmp` is not a tmpfs in this image, so on `docker restart` the previous
# container's `/tmp/.X99-lock` survives, and Xvfb refuses to start with an
# existing lock — leaving the container with no X server, every Chrome
# launch dying with "Missing X server or $DISPLAY", and `cloakserve`
# returning 502 forever. See CloakHQ/CloakBrowser#283.
rm -f /tmp/.X99-lock /tmp/.X11-unix/X99
# Start Xvfb for headed mode (Turnstile, CAPTCHAs), then run user command
Xvfb :99 -screen 0 1920x1080x24 -nolisten tcp &
sleep 1
+1 -1
View File
@@ -1 +1 @@
__version__ = "0.3.27"
__version__ = "0.3.31"
+148 -20
View File
@@ -25,6 +25,7 @@ from .human.config import HumanConfigOverrides, HumanPreset
logger = logging.getLogger("cloakbrowser")
# Sentinel to distinguish "viewport not provided" from "viewport=None" (disable emulation)
_VIEWPORT_UNSET = object()
@@ -63,6 +64,7 @@ def launch(
humanize: bool = False,
human_preset: HumanPreset = "default",
human_config: HumanConfigOverrides | None = None,
extension_paths: list[str] | None = None,
**kwargs: Any,
) -> Any:
"""Launch stealth Chromium browser. Returns a Playwright Browser object.
@@ -74,6 +76,7 @@ def launch(
Dict: {"server": "http://proxy:8080", "bypass": ".google.com", ...}
— passed directly to Playwright.
args: Additional Chromium CLI arguments to pass.
extension_paths: List of Chrome extension paths to load.
stealth_args: Include default stealth fingerprint args (default True).
Set to False if you want to pass your own --fingerprint flags.
timezone: IANA timezone (e.g. 'America/New_York'). Sets --fingerprint-timezone binary flag.
@@ -111,7 +114,8 @@ def launch(
if exit_ip and not (args and any(a.startswith("--fingerprint-webrtc-ip") for a in args)):
args = list(args or [])
args.append(f"--fingerprint-webrtc-ip={exit_ip}")
chrome_args = build_args(stealth_args, (args or []) + proxy_extra_args, timezone=timezone, locale=locale, headless=headless)
chrome_args = build_args(stealth_args, (args or []) + proxy_extra_args, timezone=timezone, locale=locale, headless=headless, extension_paths=extension_paths)
logger.debug("Launching stealth Chromium (headless=%s, args=%d)", headless, len(chrome_args))
@@ -158,6 +162,7 @@ async def launch_async( # noqa: C901
humanize: bool = False,
human_preset: HumanPreset = "default",
human_config: HumanConfigOverrides | None = None,
extension_paths: list[str] | None = None,
**kwargs: Any,
) -> Any:
"""Async version of launch(). Returns a Playwright Browser object.
@@ -166,6 +171,7 @@ async def launch_async( # noqa: C901
headless: Run in headless mode (default True).
proxy: Proxy URL string or Playwright proxy dict (see launch() for details).
args: Additional Chromium CLI arguments to pass.
extension_paths: List of Chrome extension paths to load.
stealth_args: Include default stealth fingerprint args (default True).
timezone: IANA timezone (e.g. 'America/New_York'). Sets --fingerprint-timezone binary flag.
locale: BCP 47 locale (e.g. 'en-US'). Sets --lang binary flag.
@@ -201,7 +207,7 @@ async def launch_async( # noqa: C901
if exit_ip and not (args and any(a.startswith("--fingerprint-webrtc-ip") for a in args)):
args = list(args or [])
args.append(f"--fingerprint-webrtc-ip={exit_ip}")
chrome_args = build_args(stealth_args, (args or []) + proxy_extra_args, timezone=timezone, locale=locale, headless=headless)
chrome_args = build_args(stealth_args, (args or []) + proxy_extra_args, timezone=timezone, locale=locale, headless=headless, extension_paths=extension_paths)
logger.debug("Launching stealth Chromium async (headless=%s, args=%d)", headless, len(chrome_args))
@@ -252,6 +258,7 @@ def launch_persistent_context(
humanize: bool = False,
human_preset: HumanPreset = "default",
human_config: HumanConfigOverrides | None = None,
extension_paths: list[str] | None = None,
**kwargs: Any,
) -> Any:
"""Launch stealth browser with a persistent profile and return a BrowserContext.
@@ -267,6 +274,7 @@ def launch_persistent_context(
headless: Run in headless mode (default True).
proxy: Proxy URL string or Playwright proxy dict (see launch() for details).
args: Additional Chromium CLI arguments.
extension_paths: List of Chrome extension paths to load.
stealth_args: Include default stealth fingerprint args (default True).
user_agent: Custom user agent string.
viewport: Viewport size dict, e.g. {"width": 1920, "height": 1080}.
@@ -305,7 +313,7 @@ def launch_persistent_context(
if exit_ip and not (args and any(a.startswith("--fingerprint-webrtc-ip") for a in args)):
args = list(args or [])
args.append(f"--fingerprint-webrtc-ip={exit_ip}")
chrome_args = build_args(stealth_args, (args or []) + proxy_extra_args, timezone=timezone, locale=locale, headless=headless)
chrome_args = build_args(stealth_args, (args or []) + proxy_extra_args, timezone=timezone, locale=locale, headless=headless, extension_paths=extension_paths)
logger.debug(
"Launching persistent stealth Chromium (headless=%s, user_data_dir=%s)",
@@ -376,6 +384,7 @@ async def launch_persistent_context_async(
humanize: bool = False,
human_preset: HumanPreset = "default",
human_config: HumanConfigOverrides | None = None,
extension_paths: list[str] | None = None,
**kwargs: Any,
) -> Any:
"""Async version of launch_persistent_context().
@@ -390,6 +399,7 @@ async def launch_persistent_context_async(
headless: Run in headless mode (default True).
proxy: Proxy URL string or Playwright proxy dict (see launch() for details).
args: Additional Chromium CLI arguments.
extension_paths: List of Chrome extension paths to load.
stealth_args: Include default stealth fingerprint args (default True).
user_agent: Custom user agent string.
viewport: Viewport size dict, e.g. {"width": 1920, "height": 1080}.
@@ -431,7 +441,7 @@ async def launch_persistent_context_async(
if exit_ip and not (args and any(a.startswith("--fingerprint-webrtc-ip") for a in args)):
args = list(args or [])
args.append(f"--fingerprint-webrtc-ip={exit_ip}")
chrome_args = build_args(stealth_args, (args or []) + proxy_extra_args, timezone=timezone, locale=locale, headless=headless)
chrome_args = build_args(stealth_args, (args or []) + proxy_extra_args, timezone=timezone, locale=locale, headless=headless, extension_paths=extension_paths)
logger.debug(
"Launching persistent stealth Chromium async (headless=%s, user_data_dir=%s)",
@@ -501,6 +511,7 @@ def launch_context(
humanize: bool = False,
human_preset: HumanPreset = "default",
human_config: HumanConfigOverrides | None = None,
extension_paths: list[str] | None = None,
**kwargs: Any,
) -> Any:
"""Launch stealth browser and return a BrowserContext with common options pre-set.
@@ -512,6 +523,7 @@ def launch_context(
headless: Run in headless mode (default True).
proxy: Proxy URL string or Playwright proxy dict (see launch() for details).
args: Additional Chromium CLI arguments.
extension_paths: List of Chrome extension paths to load.
stealth_args: Include default stealth fingerprint args (default True).
user_agent: Custom user agent string.
viewport: Viewport size dict, e.g. {"width": 1920, "height": 1080}.
@@ -543,7 +555,7 @@ def launch_context(
# so it applies to ALL contexts, not just the default one.
# locale and timezone are set via binary flags only — no CDP emulation.
browser = launch(headless=headless, proxy=proxy, args=args, stealth_args=stealth_args,
timezone=timezone, locale=locale, backend=backend)
timezone=timezone, locale=locale, backend=backend, extension_paths=extension_paths)
context_kwargs: dict[str, Any] = {}
if user_agent:
@@ -600,6 +612,7 @@ async def launch_context_async(
humanize: bool = False,
human_preset: HumanPreset = "default",
human_config: HumanConfigOverrides | None = None,
extension_paths: list[str] | None = None,
**kwargs: Any,
) -> Any:
"""Async version of launch_context().
@@ -613,6 +626,7 @@ async def launch_context_async(
headless: Run in headless mode (default True).
proxy: Proxy URL string or Playwright proxy dict (see launch() for details).
args: Additional Chromium CLI arguments.
extension_paths: List of Chrome extension paths to load.
stealth_args: Include default stealth fingerprint args (default True).
user_agent: Custom user agent string.
viewport: Viewport size dict, e.g. {"width": 1920, "height": 1080}.
@@ -661,7 +675,7 @@ async def launch_context_async(
# so it applies to ALL contexts, not just the default one.
# locale and timezone are set via binary flags only — no CDP emulation.
browser = await launch_async(headless=headless, proxy=proxy, args=args, stealth_args=stealth_args,
timezone=timezone, locale=locale, backend=backend)
timezone=timezone, locale=locale, backend=backend, extension_paths=extension_paths)
context_kwargs: dict[str, Any] = {}
if user_agent:
@@ -760,7 +774,7 @@ def _ensure_proxy_scheme(proxy_url: str) -> str:
return proxy_url if "://" in proxy_url else f"http://{proxy_url}"
def _assemble_socks_url(
def _assemble_proxy_url(
scheme: str,
host: str,
port: int | None,
@@ -771,7 +785,7 @@ def _assemble_socks_url(
query: str = "",
fragment: str = "",
) -> str:
"""Build a SOCKS URL from already-percent-encoded credentials and host parts.
"""Build a proxy URL from already-percent-encoded credentials and host parts.
``enc_pass is None`` means no password (no colon in userinfo). Empty string
means present-but-empty (colon preserved). This mirrors the distinction
@@ -802,7 +816,7 @@ def _reconstruct_socks_url(proxy: ProxySettings) -> str:
enc_user = quote(username, safe="")
# Dict convention: empty/missing password → no colon.
enc_pass = quote(password, safe="") if password else None
return _assemble_socks_url(
return _assemble_proxy_url(
parsed.scheme, parsed.hostname or "", parsed.port,
enc_user, enc_pass, parsed.path,
)
@@ -813,6 +827,10 @@ def _normalize_socks_string_url(url: str) -> str:
truncate them at special chars like '='. Idempotent: pre-encoded input stays
the same (decoded then re-encoded).
Emits an INFO log when re-encoding actually changes the URL, so users who
previously hit silent SOCKS5 fallback (#157) can see what the wrapper did.
Silent on already-encoded inputs (no false-positive noise).
On unparseable input (invalid port, broken IPv6 literal, etc.) logs a
warning and returns the original string — preserves pre-fix pass-through
behavior so Chromium's own error handling kicks in.
@@ -828,18 +846,30 @@ def _normalize_socks_string_url(url: str) -> str:
# urlparse returns None for absent components, "" for present-but-empty.
if parsed.username is None and parsed.password is None:
return url
enc_user = quote(unquote(parsed.username), safe="") if parsed.username else ""
raw_user = parsed.username or ""
enc_user = quote(unquote(raw_user), safe="") if raw_user else ""
# Preserve the colon separator when password component is present, even if
# empty, so `user:@host` stays `user:@host`.
if parsed.password is not None:
enc_pass = quote(unquote(parsed.password), safe="") if parsed.password else ""
raw_pass = parsed.password
enc_pass = quote(unquote(raw_pass), safe="") if raw_pass else ""
else:
raw_pass = None
enc_pass = None
return _assemble_socks_url(
normalized = _assemble_proxy_url(
parsed.scheme, parsed.hostname or "", parsed.port,
enc_user, enc_pass,
parsed.path, parsed.params, parsed.query, parsed.fragment,
)
# Compare credentials, not the full URL: urlparse cosmetically lowercases
# scheme and hostname, so a full-string compare would falsely fire on
# `socks5://USER:pass@HOST.com:1080` even when no encoding work happened.
if enc_user != raw_user or enc_pass != raw_pass:
logger.info(
"Auto URL-encoded SOCKS5 proxy credentials (special characters "
"detected). Pre-encode the URL to suppress this notice."
)
return normalized
def _extract_proxy_url(proxy: str | ProxySettings | None) -> str | None:
@@ -874,7 +904,7 @@ def maybe_resolve_geoip(
if not geoip or not proxy:
return timezone, locale, None
from .geoip import resolve_proxy_geo_with_ip
from .geoip import resolve_proxy_exit_ip, resolve_proxy_geo_with_ip
proxy_url = _extract_proxy_url(proxy)
if not proxy_url:
@@ -882,8 +912,7 @@ def maybe_resolve_geoip(
# When both tz/locale are explicit, still resolve exit IP for WebRTC
if timezone is not None and locale is not None:
from .geoip import _resolve_exit_ip
exit_ip = _resolve_exit_ip(proxy_url)
exit_ip = resolve_proxy_exit_ip(proxy_url)
return timezone, locale, exit_ip
geo_tz, geo_locale, exit_ip = resolve_proxy_geo_with_ip(proxy_url)
@@ -918,8 +947,8 @@ def _resolve_webrtc_args(
del args[idx]
return args
try:
from .geoip import _resolve_exit_ip
exit_ip = _resolve_exit_ip(proxy_url)
from .geoip import resolve_proxy_exit_ip
exit_ip = resolve_proxy_exit_ip(proxy_url)
except Exception:
logger.warning("Failed to resolve proxy exit IP for WebRTC spoofing; removing --fingerprint-webrtc-ip=auto")
args = list(args)
@@ -941,6 +970,7 @@ def build_args(
timezone: str | None = None,
locale: str | None = None,
headless: bool = True,
extension_paths: list[str] | None = None,
) -> list[str]:
"""Combine stealth args with user-provided args and locale flags.
@@ -984,6 +1014,15 @@ def build_args(
logger.debug("Arg override: %s -> %s", seen[key], flag)
seen[key] = flag
if extension_paths:
abs_paths = [os.path.abspath(p) for p in extension_paths]
ext_val = ",".join(abs_paths)
seen["--load-extension"] = f"--load-extension={ext_val}"
seen["--disable-extensions-except"] = (
f"--disable-extensions-except={ext_val}"
)
return list(seen.values())
@@ -1022,6 +1061,81 @@ def _parse_proxy_url(proxy: str) -> dict[str, Any]:
return result
def _has_credentials(proxy: str | ProxySettings) -> bool:
"""Check if the proxy has inline or dict-level credentials."""
if isinstance(proxy, dict):
return bool(proxy.get("username"))
return "@" in proxy
def _reconstruct_http_url(proxy: ProxySettings) -> str:
"""Reconstruct an HTTP(S) proxy URL with inline credentials from a Playwright proxy dict."""
server = proxy.get("server", "")
username = proxy.get("username", "")
password = proxy.get("password", "")
if not username:
return server
parsed = urlparse(_ensure_proxy_scheme(server))
enc_user = quote(username, safe="")
enc_pass = quote(password, safe="") if password else None
return _assemble_proxy_url(
parsed.scheme, parsed.hostname or "", parsed.port,
enc_user, enc_pass, parsed.path,
)
def _normalize_http_string_url(url: str) -> str:
"""Re-encode credentials in an HTTP(S) proxy URL string for --proxy-server.
Same pattern as ``_normalize_socks_string_url`` — decode then re-encode to
ensure Chromium's proxy URL parser handles special chars correctly.
"""
normalized = url if "://" in url else f"http://{url}"
try:
parsed = urlparse(normalized)
_ = parsed.port
except ValueError as e:
logger.warning("Malformed HTTP proxy URL, passing through unchanged: %s", e)
return normalized
if parsed.username is None and parsed.password is None:
return normalized
raw_user = parsed.username or ""
enc_user = quote(unquote(raw_user), safe="") if raw_user else ""
if parsed.password is not None:
raw_pass = parsed.password
enc_pass = quote(unquote(raw_pass), safe="") if raw_pass else ""
else:
raw_pass = None
enc_pass = None
result = _assemble_proxy_url(
parsed.scheme, parsed.hostname or "", parsed.port,
enc_user, enc_pass,
parsed.path, parsed.params, parsed.query, parsed.fragment,
)
if enc_user != raw_user or enc_pass != raw_pass:
logger.info(
"Auto URL-encoded HTTP proxy credentials (special characters "
"detected). Pre-encode the URL to suppress this notice."
)
return result
_HTTP_PROXY_INLINE_AUTH_MIN_VERSION = "146.0.7680.177.5"
_HTTP_PROXY_INLINE_AUTH_PLATFORMS = {"linux-x64", "windows-x64"}
def _supports_http_proxy_inline_auth() -> bool:
"""Check if the current platform's binary supports HTTP proxy inline credentials.
Requires both a supported platform AND a binary version with preemptive proxy auth.
"""
from .config import get_platform_tag, get_chromium_version, _version_tuple
tag = get_platform_tag()
if tag not in _HTTP_PROXY_INLINE_AUTH_PLATFORMS:
return False
return _version_tuple(get_chromium_version()) >= _version_tuple(_HTTP_PROXY_INLINE_AUTH_MIN_VERSION)
def _is_socks_proxy(proxy: str | ProxySettings | None) -> bool:
"""Check if the proxy uses SOCKS5 protocol."""
if proxy is None:
@@ -1035,8 +1149,9 @@ def _resolve_proxy_config(
) -> tuple[dict[str, Any], list[str]]:
"""Resolve proxy into Playwright kwargs and Chrome args.
Playwright rejects SOCKS5 proxies with credentials in its proxy dict,
so SOCKS5 is passed via --proxy-server Chrome arg instead.
Proxies with credentials (SOCKS5 or HTTP/HTTPS) are passed via Chrome's
--proxy-server flag with inline credentials, bypassing Playwright's CDP
auth interceptor which breaks on some proxies and Google domains (#182).
Returns:
(proxy_kwargs, extra_chrome_args) — one or both will be empty.
@@ -1057,7 +1172,20 @@ def _resolve_proxy_config(
# passwords at '=' and other special chars (#157).
return {}, [f"--proxy-server={_normalize_socks_string_url(proxy)}"]
# HTTP/HTTPS: use Playwright's proxy dict as before
# HTTP/HTTPS with credentials on supported platforms: bypass Playwright's
# CDP auth interceptor, pass directly to Chrome via --proxy-server with
# inline creds. Chrome sends Proxy-Authorization preemptively, avoiding
# the 407 round-trip that breaks on some proxies (#182).
if _has_credentials(proxy) and _supports_http_proxy_inline_auth():
if isinstance(proxy, dict):
url = _reconstruct_http_url(proxy)
extra_args = [f"--proxy-server={url}"]
if proxy.get("bypass"):
extra_args.append(f"--proxy-bypass-list={proxy['bypass']}")
return {}, extra_args
return {}, [f"--proxy-server={_normalize_http_string_url(proxy)}"]
# HTTP/HTTPS without credentials: use Playwright's proxy dict
if isinstance(proxy, dict):
return {"proxy": proxy}, []
return {"proxy": _parse_proxy_url(proxy)}, []
+3 -3
View File
@@ -15,14 +15,14 @@ from ._version import __version__
# CHROMIUM_VERSION is the latest across all platforms (for display/reference).
# Use get_chromium_version() for the current platform's actual version.
# ---------------------------------------------------------------------------
CHROMIUM_VERSION = "146.0.7680.177.3"
CHROMIUM_VERSION = "146.0.7680.177.5"
PLATFORM_CHROMIUM_VERSIONS: dict[str, str] = {
"linux-x64": "146.0.7680.177.3",
"linux-x64": "146.0.7680.177.5",
"linux-arm64": "146.0.7680.177.3",
"darwin-arm64": "145.0.7632.109.2",
"darwin-x64": "145.0.7632.109.2",
"windows-x64": "146.0.7680.177.4",
"windows-x64": "146.0.7680.177.5",
}
# ---------------------------------------------------------------------------
+65 -5
View File
@@ -12,6 +12,8 @@ from __future__ import annotations
import ipaddress
import logging
import math
import os
import socket
import tempfile
import threading
@@ -27,6 +29,8 @@ GEOIP_DB_URL = (
)
GEOIP_DB_FILENAME = "GeoLite2-City.mmdb"
GEOIP_UPDATE_INTERVAL = 30 * 86_400 # 30 days
DEFAULT_GEOIP_TIMEOUT_SECONDS = 5.0
GEOIP_TIMEOUT_ENV = "CLOAKBROWSER_GEOIP_TIMEOUT_SECONDS"
# Country ISO code → BCP 47 locale (covers ~90 % of proxy traffic)
COUNTRY_LOCALE_MAP: dict[str, str] = {
@@ -77,11 +81,16 @@ def resolve_proxy_geo_with_ip(
if db_path is None:
return None, None, None
timeout = _get_geoip_timeout_seconds()
deadline = _deadline_from_timeout(timeout)
# Exit IP (through proxy) is most accurate — gateway DNS may differ from exit
ip = _resolve_exit_ip(proxy_url)
if ip is None:
ip = _resolve_exit_ip(proxy_url, timeout=_remaining_seconds(deadline))
if ip is None and not _deadline_expired(deadline):
ip = _resolve_proxy_ip(proxy_url)
if ip is None:
if ip is None or _deadline_expired(deadline):
if deadline is not None and _deadline_expired(deadline):
logger.warning("GeoIP resolution timed out after %.1fs; continuing without GeoIP", timeout)
return None, None, None
try:
@@ -152,13 +161,64 @@ _IP_ECHO_URLS = [
]
def _resolve_exit_ip(proxy_url: str) -> str | None:
def _get_geoip_timeout_seconds() -> float:
raw = os.getenv(GEOIP_TIMEOUT_ENV)
if not raw:
return DEFAULT_GEOIP_TIMEOUT_SECONDS
try:
timeout = float(raw)
except ValueError:
timeout = float("nan")
if not math.isfinite(timeout):
logger.warning(
"Invalid %s=%r; using %.1fs",
GEOIP_TIMEOUT_ENV,
raw,
DEFAULT_GEOIP_TIMEOUT_SECONDS,
)
return DEFAULT_GEOIP_TIMEOUT_SECONDS
return max(timeout, 0.0)
def _deadline_from_timeout(timeout: float) -> float | None:
if timeout <= 0:
return None
return time.monotonic() + timeout
def _remaining_seconds(deadline: float | None) -> float | None:
if deadline is None:
return None
return max(deadline - time.monotonic(), 0.0)
def _deadline_expired(deadline: float | None) -> bool:
return deadline is not None and time.monotonic() >= deadline
def resolve_proxy_exit_ip(proxy_url: str) -> str | None:
"""Resolve only the proxy exit IP, bounded by the GeoIP timeout."""
timeout = _get_geoip_timeout_seconds()
deadline = _deadline_from_timeout(timeout)
ip = _resolve_exit_ip(proxy_url, timeout=timeout)
if ip is None and _deadline_expired(deadline):
logger.warning("GeoIP resolution timed out after %.1fs; continuing without GeoIP", timeout)
return ip
def _resolve_exit_ip(proxy_url: str, timeout: float | None = None) -> str | None:
"""Discover the proxy's actual exit IP by connecting through it."""
import httpx
deadline = _deadline_from_timeout(timeout or 0)
for url in _IP_ECHO_URLS:
try:
resp = httpx.get(url, proxy=proxy_url, timeout=10.0)
remaining = _remaining_seconds(deadline)
if remaining is not None and remaining <= 0:
return None
request_timeout = min(10.0, remaining) if remaining is not None else 10.0
resp = httpx.get(url, proxy=proxy_url, timeout=request_timeout)
resp.raise_for_status()
ip = resp.text.strip()
# Validate it looks like an IP
File diff suppressed because it is too large Load Diff
+354
View File
@@ -0,0 +1,354 @@
"""Playwright-style actionability checks for the humanize layer (sync).
Checks: attached, visible, stable, enabled, editable, receives pointer events.
Retry loop with backoff matching Playwright internals: [100, 250, 500, 1000]ms.
"""
from __future__ import annotations
import json
import logging
import time
from typing import Any, FrozenSet, Optional, Tuple
logger = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# Error hierarchy — all subclass RuntimeError for backward compat
# ---------------------------------------------------------------------------
class ActionabilityError(RuntimeError):
"""Base for all actionability failures."""
def __init__(self, selector: str, check: str, message: str):
self.selector = selector
self.check = check
super().__init__(f"Element {selector!r} failed {check} check: {message}")
class ElementNotAttachedError(ActionabilityError):
def __init__(self, selector: str):
super().__init__(selector, "attached", "element not found in DOM")
class ElementNotVisibleError(ActionabilityError):
def __init__(self, selector: str):
super().__init__(selector, "visible", "element is not visible")
class ElementNotStableError(ActionabilityError):
def __init__(self, selector: str):
super().__init__(selector, "stable", "element position is still changing")
class ElementNotEnabledError(ActionabilityError):
def __init__(self, selector: str):
super().__init__(selector, "enabled", "element is disabled")
class ElementNotEditableError(ActionabilityError):
def __init__(self, selector: str):
super().__init__(selector, "editable", "element is not editable")
class ElementNotReceivingEventsError(ActionabilityError):
def __init__(self, selector: str, covering_tag: str = "unknown"):
super().__init__(
selector,
"pointer_events",
f"element is covered by <{covering_tag}>",
)
# ---------------------------------------------------------------------------
# Check-set constants
# ---------------------------------------------------------------------------
CHECKS_CLICK: FrozenSet[str] = frozenset({"attached", "visible", "enabled", "pointer_events"})
CHECKS_HOVER: FrozenSet[str] = frozenset({"attached", "visible", "pointer_events"})
CHECKS_INPUT: FrozenSet[str] = frozenset({"attached", "visible", "enabled", "editable", "pointer_events"})
CHECKS_FOCUS: FrozenSet[str] = frozenset({"attached", "visible", "enabled"})
CHECKS_CHECK: FrozenSet[str] = frozenset({"attached", "visible", "enabled", "pointer_events"})
_BACKOFF_MS = [100, 250, 500, 1000]
def _backoff_sleep(attempt: int) -> None:
idx = min(attempt, len(_BACKOFF_MS) - 1)
time.sleep(_BACKOFF_MS[idx] / 1000.0)
# ---------------------------------------------------------------------------
# Pre-scroll actionability: attached, visible, enabled, editable
# ---------------------------------------------------------------------------
def ensure_actionable(
page: Any,
selector: str,
checks: FrozenSet[str],
timeout: float = 30000,
force: bool = False,
) -> None:
"""Wait for element to pass actionability checks (pre-scroll).
Retries with backoff until *timeout* ms elapsed.
Raises a specific ``ActionabilityError`` subclass on failure.
If *force* is True, returns immediately.
"""
if force:
return
deadline = time.monotonic() + timeout / 1000.0
attempt = 0
last_error: Optional[ActionabilityError] = None
while True:
remaining_ms = max(0, (deadline - time.monotonic()) * 1000)
if remaining_ms <= 0:
if last_error is not None:
raise last_error
raise ActionabilityError(selector, "timeout", "timeout expired before first check")
try:
loc = page.locator(selector).first
if "attached" in checks:
try:
loc.wait_for(state="attached", timeout=max(1, min(remaining_ms, 2000)))
except Exception:
raise ElementNotAttachedError(selector)
if "visible" in checks:
if not loc.is_visible():
raise ElementNotVisibleError(selector)
if "enabled" in checks:
if not loc.is_enabled():
raise ElementNotEnabledError(selector)
if "editable" in checks:
if not loc.is_editable():
raise ElementNotEditableError(selector)
return
except ActionabilityError as e:
last_error = e
if time.monotonic() >= deadline:
raise last_error
_backoff_sleep(attempt)
attempt += 1
# ---------------------------------------------------------------------------
# Post-scroll stability check
# ---------------------------------------------------------------------------
def _boxes_differ(a: dict, b: dict) -> bool:
return (
abs(a["x"] - b["x"]) > 1
or abs(a["y"] - b["y"]) > 1
or abs(a["width"] - b["width"]) > 1
or abs(a["height"] - b["height"]) > 1
)
def ensure_stable(
page: Any,
selector: str,
timeout: float = 5000,
) -> None:
"""Wait for element position to stabilize (two samples 100ms apart).
Only call after scroll skip if element was already in viewport.
"""
deadline = time.monotonic() + timeout / 1000.0
attempt = 0
while True:
remaining_ms = max(0, (deadline - time.monotonic()) * 1000)
if remaining_ms <= 0:
raise ElementNotStableError(selector)
loc = page.locator(selector).first
box1 = loc.bounding_box(timeout=max(1, min(remaining_ms, 1000)))
if box1 is None:
raise ElementNotAttachedError(selector)
time.sleep(0.1)
box2 = loc.bounding_box(timeout=max(1, min(remaining_ms, 1000)))
if box2 is None:
raise ElementNotAttachedError(selector)
if not _boxes_differ(box1, box2):
return
if time.monotonic() >= deadline:
raise ElementNotStableError(selector)
_backoff_sleep(attempt)
attempt += 1
# ---------------------------------------------------------------------------
# Pointer-events check (post-scroll, at actual click coordinates)
# ---------------------------------------------------------------------------
# data.box is page-space (from bounding_box); rect is frame-local. Their delta
# is the iframe offset, needed to map page-space click coords into the frame's
# own viewport before elementFromPoint. For main-frame elements the offset is 0.
_POINTER_EVENTS_LOCATOR_JS = """(expected, data) => {
const rect = expected.getBoundingClientRect();
const frameOffsetX = data.box ? data.box.x - rect.x : 0;
const frameOffsetY = data.box ? data.box.y - rect.y : 0;
const target = document.elementFromPoint(data.x - frameOffsetX, data.y - frameOffsetY);
if (!target) return { hit: false, reason: 'no_element_at_point', covering: 'none' };
let node = target;
while (node) { if (node === expected) return { hit: true }; node = node.parentNode; }
if (expected.contains(target)) return { hit: true };
return { hit: false, reason: 'covered', covering: target.tagName || 'unknown' };
}"""
_POINTER_EVENTS_HANDLE_JS = """(expected, data) => {
const rect = expected.getBoundingClientRect();
const frameOffsetX = data.box ? data.box.x - rect.x : 0;
const frameOffsetY = data.box ? data.box.y - rect.y : 0;
const target = document.elementFromPoint(data.x - frameOffsetX, data.y - frameOffsetY);
if (!target) return { hit: false, reason: 'no_element_at_point', covering: 'none' };
let node = target;
while (node) { if (node === expected) return { hit: true }; node = node.parentNode; }
if (expected.contains(target)) return { hit: true };
return { hit: false, reason: 'covered', covering: target.tagName || 'unknown' };
}"""
def check_pointer_events(
page: Any,
selector: str,
x: float,
y: float,
stealth: Any = None,
timeout: float = 5000,
) -> None:
"""Check that elementFromPoint(x, y) hits the expected element.
Uses locator.evaluate() so all Playwright selector types work
(text=, role=, XPath, CSS, etc.). Retries with backoff for transient overlays.
"""
deadline = time.monotonic() + timeout / 1000.0
attempt = 0
while True:
try:
loc = page.locator(selector).first
box = loc.bounding_box(timeout=max(1, min((deadline - time.monotonic()) * 1000, 1000)))
result = loc.evaluate(_POINTER_EVENTS_LOCATOR_JS, {"x": x, "y": y, "box": box})
except Exception as exc:
logger.debug("pointer_events check failed for %r: %s", selector, exc)
result = None
# Proceed if the check confirms a hit, or if it could not be determined
# (None) — failing closed would block legitimate clicks.
if result is None or result.get("hit", False):
return
covering = (result or {}).get("covering", "unknown")
if time.monotonic() >= deadline:
raise ElementNotReceivingEventsError(selector, covering)
_backoff_sleep(attempt)
attempt += 1
# ---------------------------------------------------------------------------
# ElementHandle variant
# ---------------------------------------------------------------------------
def ensure_actionable_handle(
page: Any,
el: Any,
checks: FrozenSet[str],
timeout: float = 30000,
force: bool = False,
) -> None:
"""Actionability checks for ElementHandle (no selector needed).
Uses Playwright's wait_for_element_state where available.
"""
if force:
return
deadline = time.monotonic() + timeout / 1000.0
attempt = 0
last_error: Optional[ActionabilityError] = None
label = "<ElementHandle>"
while True:
remaining_ms = max(0, (deadline - time.monotonic()) * 1000)
if remaining_ms <= 0:
if last_error is not None:
raise last_error
raise ActionabilityError(label, "timeout", "timeout expired before first check")
try:
if "visible" in checks:
try:
el.wait_for_element_state("visible", timeout=max(1, min(remaining_ms, 2000)))
except Exception:
raise ElementNotVisibleError(label)
if "enabled" in checks:
try:
el.wait_for_element_state("enabled", timeout=max(1, min(remaining_ms, 2000)))
except Exception:
raise ElementNotEnabledError(label)
if "editable" in checks:
try:
el.wait_for_element_state("editable", timeout=max(1, min(remaining_ms, 2000)))
except Exception:
raise ElementNotEditableError(label)
return
except ActionabilityError as e:
last_error = e
if time.monotonic() >= deadline:
raise last_error
_backoff_sleep(attempt)
attempt += 1
def check_pointer_events_handle(
page: Any,
el: Any,
x: float,
y: float,
timeout: float = 5000,
) -> None:
"""Pointer-events check for ElementHandle."""
deadline = time.monotonic() + timeout / 1000.0
attempt = 0
while True:
try:
box = el.bounding_box()
result = el.evaluate(_POINTER_EVENTS_HANDLE_JS, {"x": x, "y": y, "box": box})
except Exception:
result = None
# Proceed if the check confirms a hit, or if it could not be determined
# (None) — failing closed would block legitimate clicks.
if result is None or result.get("hit", False):
return
covering = (result or {}).get("covering", "unknown")
if time.monotonic() >= deadline:
raise ElementNotReceivingEventsError("<ElementHandle>", covering)
_backoff_sleep(attempt)
attempt += 1
+250
View File
@@ -0,0 +1,250 @@
"""Playwright-style actionability checks for the humanize layer (async).
Async mirror of actionability.py same logic, uses asyncio.sleep and await.
"""
from __future__ import annotations
import asyncio
import logging
import time
from typing import Any, FrozenSet, Optional
logger = logging.getLogger(__name__)
from .actionability import (
ActionabilityError,
ElementNotAttachedError,
ElementNotVisibleError,
ElementNotStableError,
ElementNotEnabledError,
ElementNotEditableError,
ElementNotReceivingEventsError,
_BACKOFF_MS,
_boxes_differ,
_POINTER_EVENTS_LOCATOR_JS,
_POINTER_EVENTS_HANDLE_JS,
)
async def _async_backoff_sleep(attempt: int) -> None:
idx = min(attempt, len(_BACKOFF_MS) - 1)
await asyncio.sleep(_BACKOFF_MS[idx] / 1000.0)
# ---------------------------------------------------------------------------
# Pre-scroll actionability
# ---------------------------------------------------------------------------
async def async_ensure_actionable(
page: Any,
selector: str,
checks: FrozenSet[str],
timeout: float = 30000,
force: bool = False,
) -> None:
if force:
return
deadline = time.monotonic() + timeout / 1000.0
attempt = 0
last_error: Optional[ActionabilityError] = None
while True:
remaining_ms = max(0, (deadline - time.monotonic()) * 1000)
if remaining_ms <= 0:
if last_error is not None:
raise last_error
raise ActionabilityError(selector, "timeout", "timeout expired before first check")
try:
loc = page.locator(selector).first
if "attached" in checks:
try:
await loc.wait_for(state="attached", timeout=max(1, min(remaining_ms, 2000)))
except Exception:
raise ElementNotAttachedError(selector)
if "visible" in checks:
if not await loc.is_visible():
raise ElementNotVisibleError(selector)
if "enabled" in checks:
if not await loc.is_enabled():
raise ElementNotEnabledError(selector)
if "editable" in checks:
if not await loc.is_editable():
raise ElementNotEditableError(selector)
return
except ActionabilityError as e:
last_error = e
if time.monotonic() >= deadline:
raise last_error
await _async_backoff_sleep(attempt)
attempt += 1
# ---------------------------------------------------------------------------
# Post-scroll stability check
# ---------------------------------------------------------------------------
async def async_ensure_stable(
page: Any,
selector: str,
timeout: float = 5000,
) -> None:
deadline = time.monotonic() + timeout / 1000.0
attempt = 0
while True:
remaining_ms = max(0, (deadline - time.monotonic()) * 1000)
if remaining_ms <= 0:
raise ElementNotStableError(selector)
loc = page.locator(selector).first
box1 = await loc.bounding_box(timeout=max(1, min(remaining_ms, 1000)))
if box1 is None:
raise ElementNotAttachedError(selector)
await asyncio.sleep(0.1)
box2 = await loc.bounding_box(timeout=max(1, min(remaining_ms, 1000)))
if box2 is None:
raise ElementNotAttachedError(selector)
if not _boxes_differ(box1, box2):
return
if time.monotonic() >= deadline:
raise ElementNotStableError(selector)
await _async_backoff_sleep(attempt)
attempt += 1
# ---------------------------------------------------------------------------
# Pointer-events check
# ---------------------------------------------------------------------------
async def async_check_pointer_events(
page: Any,
selector: str,
x: float,
y: float,
stealth: Any = None,
timeout: float = 5000,
) -> None:
deadline = time.monotonic() + timeout / 1000.0
attempt = 0
while True:
try:
loc = page.locator(selector).first
box = await loc.bounding_box(timeout=max(1, min((deadline - time.monotonic()) * 1000, 1000)))
result = await loc.evaluate(_POINTER_EVENTS_LOCATOR_JS, {"x": x, "y": y, "box": box})
except Exception as exc:
logger.debug("pointer_events check failed for %r: %s", selector, exc)
result = None
# Proceed if the check confirms a hit, or if it could not be determined
# (None) — failing closed would block legitimate clicks.
if result is None or result.get("hit", False):
return
covering = (result or {}).get("covering", "unknown")
if time.monotonic() >= deadline:
raise ElementNotReceivingEventsError(selector, covering)
await _async_backoff_sleep(attempt)
attempt += 1
# ---------------------------------------------------------------------------
# ElementHandle variant
# ---------------------------------------------------------------------------
async def async_ensure_actionable_handle(
page: Any,
el: Any,
checks: FrozenSet[str],
timeout: float = 30000,
force: bool = False,
) -> None:
if force:
return
deadline = time.monotonic() + timeout / 1000.0
attempt = 0
last_error: Optional[ActionabilityError] = None
label = "<ElementHandle>"
while True:
remaining_ms = max(0, (deadline - time.monotonic()) * 1000)
if remaining_ms <= 0:
if last_error is not None:
raise last_error
raise ActionabilityError(label, "timeout", "timeout expired before first check")
try:
if "visible" in checks:
try:
await el.wait_for_element_state("visible", timeout=max(1, min(remaining_ms, 2000)))
except Exception:
raise ElementNotVisibleError(label)
if "enabled" in checks:
try:
await el.wait_for_element_state("enabled", timeout=max(1, min(remaining_ms, 2000)))
except Exception:
raise ElementNotEnabledError(label)
if "editable" in checks:
try:
await el.wait_for_element_state("editable", timeout=max(1, min(remaining_ms, 2000)))
except Exception:
raise ElementNotEditableError(label)
return
except ActionabilityError as e:
last_error = e
if time.monotonic() >= deadline:
raise last_error
await _async_backoff_sleep(attempt)
attempt += 1
async def async_check_pointer_events_handle(
page: Any,
el: Any,
x: float,
y: float,
timeout: float = 5000,
) -> None:
deadline = time.monotonic() + timeout / 1000.0
attempt = 0
while True:
try:
box = await el.bounding_box()
result = await el.evaluate(_POINTER_EVENTS_HANDLE_JS, {"x": x, "y": y, "box": box})
except Exception:
result = None
# Proceed if the check confirms a hit, or if it could not be determined
# (None) — failing closed would block legitimate clicks.
if result is None or result.get("hit", False):
return
covering = (result or {}).get("covering", "unknown")
if time.monotonic() >= deadline:
raise ElementNotReceivingEventsError("<ElementHandle>", covering)
await _async_backoff_sleep(attempt)
attempt += 1
+10 -5
View File
@@ -26,7 +26,7 @@ def _get_element_box(page: Any, selector: str, timeout: float = 30000) -> Option
"""
try:
el = page.locator(selector).first
return el.bounding_box(timeout=timeout)
return el.bounding_box(timeout=max(1, timeout))
except Exception:
return None
@@ -50,7 +50,7 @@ def human_scroll_into_view(
get_box: Callable[[], Optional[dict]],
cursor_x: float, cursor_y: float,
cfg: HumanConfig,
) -> Tuple[dict, float, float]:
) -> Tuple[dict, float, float, bool]:
"""Humanized scrolling that uses an arbitrary ``get_box`` callable
instead of a CSS selector.
@@ -58,6 +58,9 @@ def human_scroll_into_view(
``ElementHandle.scroll_into_view_if_needed`` / ``Locator.scroll_into_view_if_needed``
(handle-based) so the same accelerate \u2192 cruise \u2192 decelerate \u2192 overshoot
behavior runs everywhere.
Returns ``(box, cursor_x, cursor_y, did_scroll)`` \u2014 *did_scroll* is False
when the element was already in the viewport.
"""
viewport = page.viewport_size
if not viewport:
@@ -71,7 +74,7 @@ def human_scroll_into_view(
raise RuntimeError("Element not found while scrolling into view")
if _is_in_viewport(box, viewport_height, cfg):
return box, cursor_x, cursor_y
return box, cursor_x, cursor_y, False
# Move cursor into scroll area
scroll_area_x = round(viewport_width * rand(0.3, 0.7))
@@ -139,7 +142,7 @@ def human_scroll_into_view(
if box is None:
raise RuntimeError("Element lost after scrolling into view")
return box, cursor_x, cursor_y
return box, cursor_x, cursor_y, True
def scroll_to_element(
@@ -149,12 +152,14 @@ def scroll_to_element(
cursor_x: float, cursor_y: float,
cfg: HumanConfig,
timeout: float = 30000,
) -> Tuple[dict, float, float]:
) -> Tuple[dict, float, float, bool]:
"""Selector-based humanized scroll.
``timeout`` is forwarded to ``locator.bounding_box(timeout=...)`` so callers
such as ``page.click('#x', timeout=5000)`` can wait longer for slow elements
(#172). Default matches Playwright's 30000ms when not specified.
Returns ``(box, cursor_x, cursor_y, did_scroll)``.
"""
return human_scroll_into_view(
page, raw,
+10 -5
View File
@@ -23,7 +23,7 @@ async def _get_element_box_async(
elements (#172)."""
try:
el = page.locator(selector).first
return await el.bounding_box(timeout=timeout)
return await el.bounding_box(timeout=max(1, timeout))
except Exception:
return None
@@ -47,13 +47,16 @@ async def async_human_scroll_into_view(
get_box: Callable[[], Awaitable[Optional[dict]]],
cursor_x: float, cursor_y: float,
cfg: HumanConfig,
) -> Tuple[dict, float, float]:
) -> Tuple[dict, float, float, bool]:
"""Humanized scrolling using an arbitrary async ``get_box`` callable.
Used by both ``async_scroll_to_element`` (selector-based) and the
ElementHandle / Locator ``scroll_into_view_if_needed`` patches so all
scrolling paths share the same accelerate \u2192 cruise \u2192 decelerate
\u2192 overshoot behavior.
Returns ``(box, cursor_x, cursor_y, did_scroll)`` \u2014 *did_scroll* is False
when the element was already in the viewport.
"""
viewport = page.viewport_size
if not viewport:
@@ -67,7 +70,7 @@ async def async_human_scroll_into_view(
raise RuntimeError("Element not found while scrolling into view")
if _is_in_viewport(box, viewport_height, cfg):
return box, cursor_x, cursor_y
return box, cursor_x, cursor_y, False
# Move cursor into scroll area
scroll_area_x = round(viewport_width * rand(0.3, 0.7))
@@ -135,7 +138,7 @@ async def async_human_scroll_into_view(
if box is None:
raise RuntimeError("Element lost after scrolling into view")
return box, cursor_x, cursor_y
return box, cursor_x, cursor_y, True
async def async_scroll_to_element(
@@ -145,12 +148,14 @@ async def async_scroll_to_element(
cursor_x: float, cursor_y: float,
cfg: HumanConfig,
timeout: float = 30000,
) -> Tuple[dict, float, float]:
) -> Tuple[dict, float, float, bool]:
"""Selector-based humanized scroll (async).
``timeout`` is forwarded to ``locator.bounding_box(timeout=...)`` so callers
such as ``page.click('#x', timeout=5000)`` can wait longer for slow elements
(#172). Default matches Playwright's 30000ms when not specified.
Returns ``(box, cursor_x, cursor_y, did_scroll)``.
"""
async def _get():
return await _get_element_box_async(page, selector, timeout)
@@ -70,7 +70,7 @@ Only `url` is required. Everything else is optional.
| Field | Type | Default |
|---|---|---|
| `url` | str | required |
| `url` | str | required `http://` and `https://` only |
| `proxy` | str / dict | none — `http://user:pass@host:port` or a Playwright proxy dict |
| `humanize` | bool | `false` — enable human-like mouse / keyboard / scroll |
| `human_preset` | str | `"default"` or `"careful"` |
@@ -79,7 +79,6 @@ Only `url` is required. Everything else is optional.
| `locale` | str | none — BCP-47, e.g. `"en-US"` |
| `viewport` | `{width,height}` | `1920x947` (cloakbrowser default) |
| `user_agent` | str | none |
| `extra_args` | `list[str]` | `[]` — extra Chromium CLI flags |
### Navigation
@@ -102,8 +101,6 @@ Only `url` is required. Everything else is optional.
| `wait_for_selector` | str | none — CSS or XPath |
| `wait_for_selector_state` | str | `"visible"` — also `attached` / `detached` / `hidden` |
| `wait_for_selector_timeout_ms` | int | `30000` |
| `wait_for_function` | str | none — JS expression returning truthy when ready |
| `wait_for_function_timeout_ms` | int | `30000` |
| `wait_ms` | int | none — fixed pause |
### Capture
@@ -118,7 +115,7 @@ Only `url` is required. Everything else is optional.
The handler retries transient navigation failures inline within the same Lambda invocation. Two layers, both built-in:
- **Launch retries** — 3 attempts with 0.3 s + 0.6 s backoff. Recovers Xvfb / Chromium spawn races at cold start. Fast and cheap; not configurable.
- **Strategy retries** — default 1 attempt, configurable via the `retries` event field. Recovers specific post-launch error classes by relaunching with adjusted Chromium args / page-load budgets.
- **Strategy retries** — default 1 attempt, configurable via the `retries` event field. Recovers specific post-launch error classes by relaunching with adjusted internal Chromium args / page-load budgets.
| Field | Type | Default |
|---|---|---|
@@ -176,6 +173,22 @@ For latency-sensitive use cases: provision concurrency, schedule a CloudWatch/Ev
If you see empty/missing dynamic content on cold-start invocations, raise `max_settle_ms` in the event payload (e.g. `25000`) — the default `15000` is tuned for warm runs.
## Security
The handler validates all incoming URLs before navigation:
- **Scheme restriction** — only `http://` and `https://` are accepted. `file://`, `data:`, `javascript:`, and other schemes are rejected.
- **SSRF protection** — hostnames are resolved before navigation and checked against private, loopback, link-local, reserved, and multicast IP ranges. This blocks access to cloud metadata endpoints (e.g. `169.254.169.254`), localhost services, and internal networks.
- **Post-navigation re-validation** — the final URL is re-checked after page load and after post-navigation waits to catch server-side redirects to blocked destinations.
- **No caller-controlled Chromium flags** — the handler does not accept arbitrary CLI flags from the event. Internal retry strategies add flags as needed (e.g. `--ignore-certificate-errors` for cert errors).
- **No arbitrary JS execution**`wait_for_function` is not exposed. Use `wait_for_selector` or `smart_wait` instead.
**Limitations**:
- Post-navigation re-validation prevents response *exfiltration*, but does not prevent the browser from *making* the request. If an internal endpoint has side effects on GET, the request will still reach it before validation rejects the response. Use network-level controls (security groups, VPC) to protect side-effect-bearing internal endpoints.
- DNS rebinding attacks can bypass pre-navigation IP checks in theory, though the post-navigation re-validation provides a second layer of defense.
**Trust boundary**: if this handler is exposed to untrusted callers (Lambda Function URL, API Gateway without auth, public ALB), add an authentication layer (API Gateway authorizer, IAM auth, etc.). The URL validation above is defense-in-depth, not a substitute for access control.
## License
The patched Chromium binary inside the upstream `cloakhq/cloakbrowser` image is governed by the **CloakBrowser Binary License** (published at https://github.com/CloakHQ/CloakBrowser/blob/main/BINARY-LICENSE.md). Internal organizational use (private ECR, your own scraping pipelines, your own business) is free. Exposing this Lambda as a paid API to third-party customers — i.e. browser-as-a-service — requires an OEM/SaaS license from CloakHQ (`cloakhq@pm.me`). Do not push the resulting image to a public registry; that would be redistribution and is prohibited.
@@ -5,7 +5,7 @@ Always runs **headed** via the Xvfb display started by `lambda-entrypoint.sh`.
Event schema (all fields except `url` are optional):
Launch options (passed to cloakbrowser.launch_context_async):
url str required, the page to scrape
url str required, the page to scrape (http/https only)
proxy str|dict http://user:pass@host:port or Playwright proxy dict
humanize bool False enable human-like mouse/keyboard/scroll
human_preset str "default" | "careful"
@@ -14,7 +14,6 @@ Event schema (all fields except `url` are optional):
locale str BCP-47, e.g. "en-US"
viewport {width,height} defaults to 1920x947 (cloakbrowser DEFAULT_VIEWPORT)
user_agent str custom UA (rare cloakbrowser sets one already)
extra_args list[str] additional Chromium CLI flags
Navigation options (passed to page.goto):
wait_until str "load"|"domcontentloaded"|"networkidle"|"commit"
@@ -35,8 +34,6 @@ Event schema (all fields except `url` are optional):
wait_for_selector str CSS or XPath selector
wait_for_selector_state str "attached"|"detached"|"visible"|"hidden", default "visible"
wait_for_selector_timeout_ms int 30000
wait_for_function str JS expression that returns truthy when ready
wait_for_function_timeout_ms int 30000
wait_ms int fixed pause in ms (page.wait_for_timeout)
Capture options:
@@ -64,11 +61,14 @@ from __future__ import annotations
import asyncio
import base64
import ipaddress
import json
import logging
import socket
import subprocess
from pathlib import Path
from typing import Any
from urllib.parse import urlparse
from cloakbrowser import launch_context_async
@@ -76,6 +76,26 @@ logger = logging.getLogger("cloakbrowser.lambda")
logger.setLevel(logging.INFO)
def _validate_url(url: str) -> None:
"""Reject non-HTTP schemes and URLs that resolve to private/internal IPs."""
parsed = urlparse(url)
if parsed.scheme.lower() not in ("http", "https"):
raise ValueError(
f"Only http:// and https:// URLs are supported, got: {parsed.scheme!r}"
)
hostname = parsed.hostname
if not hostname:
raise ValueError("URL has no hostname")
try:
infos = socket.getaddrinfo(hostname, None, socket.AF_UNSPEC, socket.SOCK_STREAM)
except socket.gaierror:
raise ValueError(f"Cannot resolve hostname: {hostname}")
for info in infos:
addr = ipaddress.ip_address(info[4][0])
if not addr.is_global:
raise ValueError("URLs targeting private/internal networks are blocked")
def _diag_snapshot() -> str:
"""Capture Xvfb status, Xvfb log, X11 socket state, and env for error reports."""
import os
@@ -118,7 +138,7 @@ def _build_launch_kwargs(event: dict) -> dict:
# Lambda's restricted process model can't fork from Chromium's zygote
# — without this, child renderer processes fail to spawn.
"--no-zygote",
*event.get("extra_args", []),
*event.get("_strategy_args", []),
],
}
for key in ("proxy", "humanize", "human_preset", "geoip",
@@ -159,7 +179,7 @@ async def _smart_wait(page, dom_stable_ms: int = 1500, max_settle_ms: int = 1500
_EXPLICIT_WAIT_KEYS = (
"wait_for_load_state", "wait_for_selector", "wait_for_function", "wait_ms",
"wait_for_load_state", "wait_for_selector", "wait_ms",
)
@@ -184,11 +204,6 @@ async def _post_nav_waits(page, event: dict) -> None:
state=event.get("wait_for_selector_state", "visible"),
timeout=event.get("wait_for_selector_timeout_ms", 30000),
)
if "wait_for_function" in event:
await page.wait_for_function(
event["wait_for_function"],
timeout=event.get("wait_for_function_timeout_ms", 30000),
)
if "wait_ms" in event:
await page.wait_for_timeout(event["wait_ms"])
@@ -235,7 +250,7 @@ def _classify_error(err: Exception) -> dict | None:
msg = str(err)
if "ERR_CERT" in msg:
return {
"extra_args": ["--ignore-certificate-errors"],
"_strategy_args": ["--ignore-certificate-errors"],
"goto_timeout_ms": 60000,
}
if ("Timeout" in msg and "exceeded" in msg) or "ERR_CONNECTION_TIMED_OUT" in msg:
@@ -263,8 +278,10 @@ async def _attempt_scrape(url: str, event: dict) -> dict:
wait_until=event.get("wait_until", "domcontentloaded"),
timeout=event.get("goto_timeout_ms", 30000),
)
_validate_url(page.url)
await _post_nav_waits(page, event)
_validate_url(page.url)
result: dict = {
"title": await page.title(),
@@ -306,6 +323,8 @@ async def _run(event: dict) -> dict:
set to 0 to disable retry entirely).
"""
url = event["url"]
_validate_url(url)
event = {k: v for k, v in event.items() if k not in ("extra_args", "_strategy_args")}
retries_left = max(0, int(event.get("retries", 1)))
history: list[dict] = []
current_event = event
@@ -326,8 +345,8 @@ async def _run(event: dict) -> dict:
})
logger.warning("attempt %d failed (%s); retrying with strategy=%s",
len(history), str(e)[:120], strategy)
merged_args = list(current_event.get("extra_args", [])) + list(strategy.get("extra_args", []))
current_event = {**current_event, **strategy, "extra_args": merged_args}
merged_args = list(current_event.get("_strategy_args", [])) + list(strategy.get("_strategy_args", []))
current_event = {**current_event, **strategy, "_strategy_args": merged_args}
retries_left -= 1
# No backoff: strategy overrides change goto budget directly;
# the prior failure was either fast (cert reject) or already
Generated
+27
View File
@@ -0,0 +1,27 @@
{
"nodes": {
"nixpkgs": {
"locked": {
"lastModified": 1777954456,
"narHash": "sha256-hGdgeU2Nk87RAuZyYjyDjFL6LK7dAZN5RE9+hrDTkDU=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "549bd84d6279f9852cae6225e372cc67fb91a4c1",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixos-unstable",
"repo": "nixpkgs",
"type": "github"
}
},
"root": {
"inputs": {
"nixpkgs": "nixpkgs"
}
}
},
"root": "root",
"version": 7
}
+237
View File
@@ -0,0 +1,237 @@
{
description = "CloakBrowser development shell with Nix-packaged Chromium binaries";
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
};
outputs = { self, nixpkgs }:
let
inherit (nixpkgs) lib;
supportedSystems = [
"x86_64-linux"
"aarch64-linux"
];
forAllSystems = lib.genAttrs supportedSystems;
packageInfo = {
x86_64-linux = {
platformTag = "linux-x64";
version = "146.0.7680.177.5";
hash = "sha256-ShK83pX6G7G+7ytBq15cJ8Nr544749DayMZNcFIWZw4=";
};
aarch64-linux = {
platformTag = "linux-arm64";
version = "146.0.7680.177.3";
hash = "sha256-i3HOU7T9ExMnMxox+6ODXXGILRm/qr3njdD1OQvRb0U=";
};
};
cloakbrowserBinaryLicense = {
shortName = "cloakbrowser-binary";
fullName = "CloakBrowser Binary License";
url = "https://github.com/CloakHQ/CloakBrowser/blob/main/BINARY-LICENSE.md";
free = false;
redistributable = false;
};
mkPkgs = system: import nixpkgs {
inherit system;
config.allowUnfree = true;
};
runtimeLibraries = pkgs: with pkgs; [
alsa-lib
at-spi2-atk
at-spi2-core
atk
cairo
cups
dbus
expat
fontconfig
freetype
gdk-pixbuf
glib
gtk3
libdrm
libgbm
libGL
libpulseaudio
libxkbcommon
mesa
nspr
nss
pango
systemd
wayland
libx11
libxcb
libxcomposite
libxcursor
libxdamage
libxext
libxfixes
libxi
libxrandr
libxrender
libxscrnsaver
libxshmfence
libxtst
];
fontPackages = pkgs: with pkgs; [
freefont_ttf
ipafont
liberation_ttf
noto-fonts
noto-fonts-cjk-sans
noto-fonts-color-emoji
tlwg
unifont
wqy_zenhei
];
desktopPackages = pkgs: with pkgs; [
adwaita-icon-theme
gsettings-desktop-schemas
xdg-utils
];
mkCloakBrowserChromium = pkgs: system:
let
info = packageInfo.${system} or (throw "CloakBrowser flake package currently supports only x86_64-linux and aarch64-linux.");
archiveName = "cloakbrowser-${info.platformTag}.tar.gz";
chromiumVersion = info.version;
libs = runtimeLibraries pkgs;
desktopDeps = desktopPackages pkgs;
fonts = fontPackages pkgs;
fontsConf = pkgs.makeFontsConf {
fontDirectories = fonts;
};
in
pkgs.stdenvNoCC.mkDerivation {
pname = "cloakbrowser-chromium";
version = chromiumVersion;
src = pkgs.fetchurl {
url = "https://cloakbrowser.dev/chromium-v${chromiumVersion}/${archiveName}";
inherit (info) hash;
};
dontUnpack = true;
nativeBuildInputs = with pkgs; [
autoPatchelfHook
makeWrapper
];
buildInputs = libs ++ desktopDeps;
runtimeDependencies = libs;
installPhase = ''
runHook preInstall
mkdir -p "$out/lib/cloakbrowser" "$out/bin"
tar -xzf "$src" -C "$out/lib/cloakbrowser"
chmod +x "$out/lib/cloakbrowser/chrome"
chmod +x "$out/lib/cloakbrowser/chromedriver"
runHook postInstall
'';
postFixup = ''
makeWrapper "$out/lib/cloakbrowser/chrome" "$out/bin/cloakbrowser-chrome" \
--prefix LD_LIBRARY_PATH : "${lib.makeLibraryPath libs}" \
--prefix XDG_DATA_DIRS : "$GSETTINGS_SCHEMAS_PATH:$XDG_ICON_DIRS" \
--suffix PATH : "${lib.makeBinPath [ pkgs.xdg-utils ]}" \
--set FONTCONFIG_FILE "${fontsConf}" \
--set CHROME_WRAPPER "cloakbrowser-chrome"
makeWrapper "$out/lib/cloakbrowser/chromedriver" "$out/bin/cloakbrowser-chromedriver" \
--prefix LD_LIBRARY_PATH : "${lib.makeLibraryPath libs}"
'';
meta = {
description = "Official CloakBrowser patched Chromium binary";
homepage = "https://github.com/CloakHQ/CloakBrowser";
license = cloakbrowserBinaryLicense;
mainProgram = "cloakbrowser-chrome";
platforms = supportedSystems;
sourceProvenance = [ lib.sourceTypes.binaryNativeCode ];
};
};
in
{
packages = forAllSystems (system:
let
pkgs = mkPkgs system;
cloakbrowserChromium = mkCloakBrowserChromium pkgs system;
in
{
inherit cloakbrowserChromium;
default = cloakbrowserChromium;
});
apps = forAllSystems (system:
let
cloakbrowserChromium = self.packages.${system}.cloakbrowserChromium;
in
{
default = {
type = "app";
program = "${cloakbrowserChromium}/bin/cloakbrowser-chrome";
meta.description = "Run CloakBrowser Chromium";
};
cloakbrowser-chrome = {
type = "app";
program = "${cloakbrowserChromium}/bin/cloakbrowser-chrome";
meta.description = "Run CloakBrowser Chromium";
};
cloakbrowser-chromedriver = {
type = "app";
program = "${cloakbrowserChromium}/bin/cloakbrowser-chromedriver";
meta.description = "Run the CloakBrowser Chromedriver binary";
};
});
devShells = forAllSystems (system:
let
pkgs = mkPkgs system;
cloakbrowserChromium = self.packages.${system}.cloakbrowserChromium;
python = pkgs.python312.withPackages (ps: with ps; [
aiohttp
geoip2
hatchling
httpx
playwright
pytest
pytest-asyncio
socksio
websockets
]);
in
{
default = pkgs.mkShell {
packages = [
cloakbrowserChromium
python
pkgs.cacert
pkgs.curl
pkgs.git
pkgs.jq
pkgs.nodejs_20
pkgs.which
pkgs.xdotool
pkgs.xvfb-run
]
++ runtimeLibraries pkgs
++ fontPackages pkgs;
CLOAKBROWSER_BINARY_PATH = "${cloakbrowserChromium}/bin/cloakbrowser-chrome";
};
});
};
}
+24 -4
View File
@@ -11,7 +11,7 @@
Drop-in Playwright/Puppeteer replacement. Same API, same code — just swap the import. **3 lines of code, 30 seconds to unblock.**
- **48 source-level C++ patches** — canvas, WebGL, audio, fonts, GPU, screen, WebRTC, network timing, automation signals
- **58 source-level C++ patches** — canvas, WebGL, audio, fonts, GPU, screen, WebRTC, network timing, automation signals
- **0.9 reCAPTCHA v3 score** — human-level, server-verified
- **Passes Cloudflare Turnstile**, FingerprintJS, BrowserScan — tested against 30+ detection sites
- **`npm install cloakbrowser`** — binary auto-downloads, auto-updates, zero config
@@ -39,11 +39,24 @@ import { launch } from 'cloakbrowser';
const browser = await launch();
const page = await browser.newPage();
await page.goto('https://protected-site.com');
await page.goto('https://example.com');
console.log(await page.title());
await browser.close();
```
**For sites with anti-bot protection**, add a residential proxy and these flags:
```javascript
const browser = await launch({
proxy: 'http://user:pass@residential-proxy:port',
geoip: true, // match timezone + locale to proxy IP
headless: false, // some sites detect headless even with C++ patches
humanize: true, // human-like mouse, keyboard, scroll
});
```
See the [main README](https://github.com/CloakHQ/CloakBrowser#troubleshooting) for site-specific troubleshooting (FingerprintJS, Kasada, reCAPTCHA).
### Puppeteer
> **Note:** Playwright is recommended for sites with reCAPTCHA Enterprise. Puppeteer's CDP protocol leaks automation signals that reCAPTCHA Enterprise can detect. This is a known Puppeteer limitation, not specific to CloakBrowser.
@@ -53,7 +66,7 @@ import { launch } from 'cloakbrowser/puppeteer';
const browser = await launch();
const page = await browser.newPage();
await page.goto('https://protected-site.com');
await page.goto('https://example.com');
console.log(await page.title());
await browser.close();
```
@@ -215,7 +228,7 @@ const page = await browser.newPage();
## Requirements
- Node.js >= 20
- One of: `playwright-core` >= 1.40 or `puppeteer-core` >= 21
- One of: `playwright-core` >= 1.53 or `puppeteer-core` >= 21
## Troubleshooting
@@ -230,6 +243,13 @@ const ctx = await launchPersistentContext({
userDataDir: './my-profile',
headless: false,
});
// Load Chrome extensions
const ctx = await launchPersistentContext({
userDataDir: './my-profile',
headless: false,
extensionPaths: ['./my-extension'],
});
```
This also gives you cookie and localStorage persistence across sessions.
+222 -613
View File
File diff suppressed because it is too large Load Diff
+6 -6
View File
@@ -1,6 +1,6 @@
{
"name": "cloakbrowser",
"version": "0.3.27",
"version": "0.3.31",
"description": "Stealth Chromium that passes every bot detection test. Drop-in Playwright/Puppeteer replacement with source-level fingerprint patches.",
"type": "module",
"main": "dist/index.js",
@@ -59,7 +59,7 @@
},
"peerDependencies": {
"mmdb-lib": ">=2.0.0",
"playwright-core": ">=1.40.0",
"playwright-core": ">=1.53.0",
"puppeteer-core": ">=21.0.0",
"socks-proxy-agent": ">=10.0.0"
},
@@ -81,12 +81,12 @@
"tar": "^7.0.0"
},
"devDependencies": {
"@types/node": "^20.10.0",
"@types/node": "^25.9.1",
"mmdb-lib": "^3.0.2",
"playwright-core": "1.60",
"puppeteer-core": "^25.0.4",
"socks-proxy-agent": "^10.0.0",
"playwright-core": "^1.40.0",
"puppeteer-core": "^21.0.0",
"typescript": "^5.3.0",
"typescript": "^6.0.3",
"vitest": "^1.0.0"
},
"scripts": {
+12 -1
View File
@@ -1,7 +1,7 @@
/**
* Shared argument builder for Playwright and Puppeteer wrappers.
*/
import path from "path";
import type { LaunchOptions } from "./types.js";
import { getDefaultStealthArgs } from "./config.js";
@@ -55,5 +55,16 @@ export function buildArgs(options: LaunchOptions): string[] {
seen.set(k, flag);
}
}
if (options.extensionPaths?.length) {
const absPaths = options.extensionPaths.map(p => path.resolve(p));
const joined = absPaths.join(",");
seen.set("--load-extension", `--load-extension=${joined}`);
seen.set(
"--disable-extensions-except",
`--disable-extensions-except=${joined}`
);
}
return [...seen.values()];
}
+3 -3
View File
@@ -27,14 +27,14 @@ export { WRAPPER_VERSION };
// CHROMIUM_VERSION is the latest across all platforms (for display/reference).
// Use getChromiumVersion() for the current platform's actual version.
// ---------------------------------------------------------------------------
export const CHROMIUM_VERSION = "146.0.7680.177.3";
export const CHROMIUM_VERSION = "146.0.7680.177.5";
export const PLATFORM_CHROMIUM_VERSIONS: Record<string, string> = {
"linux-x64": "146.0.7680.177.3",
"linux-x64": "146.0.7680.177.5",
"linux-arm64": "146.0.7680.177.3",
"darwin-arm64": "145.0.7632.109.2",
"darwin-x64": "145.0.7632.109.2",
"windows-x64": "146.0.7680.177.4",
"windows-x64": "146.0.7680.177.5",
};
// ---------------------------------------------------------------------------
+53 -10
View File
@@ -22,6 +22,7 @@ const GEOIP_DB_URL =
"https://github.com/P3TERX/GeoLite.mmdb/raw/download/GeoLite2-City.mmdb";
const GEOIP_DB_FILENAME = "GeoLite2-City.mmdb";
const GEOIP_UPDATE_INTERVAL_MS = 30 * 86_400_000; // 30 days
const DEFAULT_GEOIP_TIMEOUT_MS = 5_000;
/** Country ISO code → BCP 47 locale (covers ~90% of proxy traffic). */
export const COUNTRY_LOCALE_MAP: Record<string, string> = {
@@ -68,10 +69,18 @@ export async function resolveProxyGeo(
const dbPath = await ensureGeoipDb();
if (!dbPath) return { timezone: null, locale: null, exitIp: null };
const timeoutMs = getGeoipTimeoutMs();
const deadline = deadlineFromTimeout(timeoutMs);
// Exit IP (through proxy) is most accurate — gateway DNS may differ from exit
let ip = await resolveExitIp(proxyUrl);
if (!ip) ip = await resolveProxyIp(proxyUrl);
if (!ip) return { timezone: null, locale: null, exitIp: null };
let ip = await resolveExitIp(proxyUrl, remainingMs(deadline));
if (!ip && !deadlineExpired(deadline)) ip = await resolveProxyIp(proxyUrl);
if (!ip || deadlineExpired(deadline)) {
if (deadlineExpired(deadline)) {
console.warn(`[cloakbrowser] GeoIP resolution timed out after ${timeoutMs}ms; continuing without GeoIP`);
}
return { timezone: null, locale: null, exitIp: null };
}
try {
const buf = fs.readFileSync(dbPath);
@@ -87,6 +96,30 @@ export async function resolveProxyGeo(
}
}
function getGeoipTimeoutMs(): number {
const raw = process.env.CLOAKBROWSER_GEOIP_TIMEOUT_SECONDS;
if (!raw) return DEFAULT_GEOIP_TIMEOUT_MS;
const timeoutSeconds = Number(raw);
if (!Number.isFinite(timeoutSeconds)) {
console.warn(`[cloakbrowser] Invalid CLOAKBROWSER_GEOIP_TIMEOUT_SECONDS=${raw}; using ${DEFAULT_GEOIP_TIMEOUT_MS / 1000}s`);
return DEFAULT_GEOIP_TIMEOUT_MS;
}
return Math.max(timeoutSeconds, 0) * 1000;
}
function deadlineFromTimeout(timeoutMs: number): number | null {
return timeoutMs > 0 ? performance.now() + timeoutMs : null;
}
function remainingMs(deadline: number | null): number | undefined {
if (deadline === null) return undefined;
return Math.max(deadline - performance.now(), 0);
}
function deadlineExpired(deadline: number | null): boolean {
return deadline !== null && performance.now() >= deadline;
}
// ---------------------------------------------------------------------------
// Proxy IP resolution
// ---------------------------------------------------------------------------
@@ -128,7 +161,8 @@ const IP_ECHO_URLS = [
"https://ifconfig.me/ip",
];
async function resolveExitIp(proxyUrl: string): Promise<string | null> {
async function resolveExitIp(proxyUrl: string, timeoutMs?: number): Promise<string | null> {
const deadline = timeoutMs && timeoutMs > 0 ? performance.now() + timeoutMs : null;
const isSocks = isSocksProxy(proxyUrl);
// SOCKS5: tunnel through the SOCKS5 proxy via socks-proxy-agent
@@ -144,9 +178,11 @@ async function resolveExitIp(proxyUrl: string): Promise<string | null> {
const agent = new SocksProxyAgent(proxyUrl);
for (const echoUrl of IP_ECHO_URLS) {
const remaining = remainingMs(deadline);
if (remaining !== undefined && remaining <= 0) return null;
try {
const ip = await new Promise<string | null>((resolve) => {
const req = https.request(echoUrl, { agent, timeout: 10_000 }, (res) => {
const req = https.request(echoUrl, { agent, timeout: Math.min(10_000, remaining ?? 10_000) }, (res) => {
let data = "";
res.on("data", (chunk: Buffer) => (data += chunk.toString()));
res.on("end", () => {
@@ -173,6 +209,8 @@ async function resolveExitIp(proxyUrl: string): Promise<string | null> {
const proxyUrlObj = new URL(proxyUrl);
for (const echoUrl of IP_ECHO_URLS) {
const remaining = remainingMs(deadline);
if (remaining !== undefined && remaining <= 0) return null;
try {
const ip = await new Promise<string | null>((resolve, reject) => {
const targetUrl = new URL(echoUrl);
@@ -190,13 +228,14 @@ async function resolveExitIp(proxyUrl: string): Promise<string | null> {
).toString("base64"),
}
: {},
timeout: 10_000,
timeout: Math.min(10_000, remaining ?? 10_000),
});
connectReq.on("connect", (_res, socket) => {
const innerRemaining = remainingMs(deadline);
const req = https.request(
echoUrl,
{ socket, timeout: 5_000 } as any,
{ socket, timeout: Math.min(5_000, innerRemaining ?? 5_000) } as any,
(res) => {
let data = "";
res.on("data", (chunk: Buffer) => (data += chunk.toString()));
@@ -207,6 +246,7 @@ async function resolveExitIp(proxyUrl: string): Promise<string | null> {
}
);
req.on("error", () => resolve(null));
req.on("timeout", () => { req.destroy(); resolve(null); });
req.end();
});
@@ -261,7 +301,9 @@ async function downloadGeoipDb(dest: string): Promise<void> {
const tmpPath = `${dest}.tmp.${Date.now()}`;
try {
const response = await fetch(GEOIP_DB_URL, { redirect: "follow" });
const response = await fetch(GEOIP_DB_URL, {
redirect: "follow",
});
if (!response.ok || !response.body) {
throw new Error(`HTTP ${response.status}`);
}
@@ -329,7 +371,8 @@ export async function maybeResolveGeoip(
// When both tz/locale are explicit, still resolve exit IP for WebRTC
if (options.timezone && options.locale) {
const exitIp = await resolveExitIp(proxyUrl) ?? undefined;
const timeoutMs = getGeoipTimeoutMs();
const exitIp = await resolveExitIp(proxyUrl, timeoutMs) ?? undefined;
return { timezone: options.timezone, locale: options.locale, exitIp };
}
@@ -363,7 +406,7 @@ export async function resolveWebrtcArgs(
}
try {
const ip = await resolveExitIp(proxyUrl);
const ip = await resolveExitIp(proxyUrl, getGeoipTimeoutMs());
const result = [...args];
if (ip) {
result[idx] = `--fingerprint-webrtc-ip=${ip}`;
+75 -35
View File
@@ -47,7 +47,7 @@
*/
import type { Browser, Page, Frame, CDPSession, ElementHandle, BrowserContext } from 'puppeteer-core';
import type { HumanConfig } from '../human/config.js';
import type { HumanConfig, HumanActionOptions } from '../human/config.js';
import { resolveConfig, mergeConfig, rand, randRange, sleep } from '../human/config.js';
import { RawMouse, RawKeyboard, humanMove, humanClick, clickTarget, humanIdle } from '../human/mouse.js';
import { humanType } from './keyboard.js';
@@ -319,7 +319,11 @@ function patchPage(page: Page, cfg: HumanConfig, cursor: CursorState): void {
}
// ==== goto ====
const humanGoto = async (url: string, options?: any) => {
const humanGoto = async (url: string, options?: {
referer?: string;
timeout?: number;
waitUntil?: 'load' | 'domcontentloaded' | 'networkidle0' | 'networkidle2';
}) => {
const response = await originals.goto(url, options);
stealth.invalidate();
patchFrames(page, cfg, cursor, raw, rawKb, originals, stealth);
@@ -327,11 +331,16 @@ function patchPage(page: Page, cfg: HumanConfig, cursor: CursorState): void {
};
// ==== click (with clickCount support for dblclick) ====
const humanClickFn = async (selector: string, options?: any) => {
const humanClickFn = async (selector: string, options?: HumanActionOptions & {
button?: 'left' | 'right' | 'middle' | 'back' | 'forward';
clickCount?: number;
count?: number;
delay?: number;
}) => {
await ensureCursorInit();
const callCfg = mergeConfig(cfg, options?.human_config);
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
if (callCfg.idle_between_actions) {
await humanIdle(raw, rand(callCfg.idle_between_duration[0], callCfg.idle_between_duration[1]), cursor.x, cursor.y, callCfg);
await humanIdle(raw, cursor.x, cursor.y, callCfg);
}
const { box, cursorX, cursorY } = await scrollToElement(page, raw, selector, cursor.x, cursor.y, callCfg, options?.timeout);
cursor.x = cursorX;
@@ -355,11 +364,11 @@ function patchPage(page: Page, cfg: HumanConfig, cursor: CursorState): void {
};
// ==== hover ====
const humanHoverFn = async (selector: string, options?: any) => {
const humanHoverFn = async (selector: string, options?: HumanActionOptions) => {
await ensureCursorInit();
const callCfg = mergeConfig(cfg, options?.human_config);
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
if (callCfg.idle_between_actions) {
await humanIdle(raw, rand(callCfg.idle_between_duration[0], callCfg.idle_between_duration[1]), cursor.x, cursor.y, callCfg);
await humanIdle(raw, cursor.x, cursor.y, callCfg);
}
const { box, cursorX, cursorY } = await scrollToElement(page, raw, selector, cursor.x, cursor.y, callCfg, options?.timeout);
cursor.x = cursorX;
@@ -371,8 +380,10 @@ function patchPage(page: Page, cfg: HumanConfig, cursor: CursorState): void {
};
// ==== type ====
const humanTypeFn = async (selector: string, text: string, options?: any) => {
const callCfg = mergeConfig(cfg, options?.human_config);
const humanTypeFn = async (selector: string, text: string, options?: HumanActionOptions & {
delay?: number;
}) => {
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
await sleep(randRange(callCfg.field_switch_delay));
await humanClickFn(selector, options);
await sleep(rand(100, 250));
@@ -395,7 +406,7 @@ function patchPage(page: Page, cfg: HumanConfig, cursor: CursorState): void {
};
// ==== tap ====
const humanTapFn = async (selector: string, options?: any) => {
const humanTapFn = async (selector: string, options?: HumanActionOptions) => {
await humanClickFn(selector, options);
};
@@ -413,14 +424,19 @@ function patchPage(page: Page, cfg: HumanConfig, cursor: CursorState): void {
// ============================================================
// Mouse patches
// ============================================================
page.mouse.move = async (x: number, y: number, options?: any) => {
page.mouse.move = async (x: number, y: number, options?: { steps?: number }) => {
await ensureCursorInit();
await humanMove(raw, cursor.x, cursor.y, x, y, cfg);
cursor.x = x;
cursor.y = y;
};
page.mouse.click = async (x: number, y: number, options?: any) => {
page.mouse.click = async (x: number, y: number, options?: {
button?: 'left' | 'right' | 'middle' | 'back' | 'forward';
clickCount?: number;
count?: number;
delay?: number;
}) => {
await ensureCursorInit();
await humanMove(raw, cursor.x, cursor.y, x, y, cfg);
cursor.x = x;
@@ -455,7 +471,7 @@ function patchPage(page: Page, cfg: HumanConfig, cursor: CursorState): void {
(page.mouse as any).dragAndDrop = async (
start: { x: number; y: number },
target: { x: number; y: number },
options?: any,
options?: { delay?: number },
) => {
await ensureCursorInit();
await humanMove(raw, cursor.x, cursor.y, start.x, start.y, cfg);
@@ -475,12 +491,12 @@ function patchPage(page: Page, cfg: HumanConfig, cursor: CursorState): void {
// ============================================================
// Keyboard patches
// ============================================================
page.keyboard.type = async (text: string, options?: any) => {
page.keyboard.type = async (text: string, options?: { delay?: number }) => {
const cdp = await ensureCdp();
await humanType(page, rawKb, text, cfg, cdp);
};
page.keyboard.press = async (key: any, options?: any) => {
page.keyboard.press = async (key: any, options?: { delay?: number }) => {
await sleep(rand(20, 60));
await originals.keyboardDown(key as any);
await sleep(randRange(cfg.key_hold));
@@ -551,7 +567,11 @@ function patchElementHandle(
return els;
};
(page as any).waitForSelector = async (selector: string, options?: any) => {
(page as any).waitForSelector = async (selector: string, options?: {
hidden?: boolean;
timeout?: number;
visible?: boolean;
}) => {
const el = await origWaitForSelector(selector, options);
if (el) patchSingleElementHandle(el, page, cfg, cursor, raw, rawKb, originals, stealth);
return el;
@@ -603,14 +623,18 @@ function patchSingleElementHandle(
return children;
};
(el as any).waitForSelector = async (selector: string, options?: any) => {
(el as any).waitForSelector = async (selector: string, options?: {
hidden?: boolean;
timeout?: number;
visible?: boolean;
}) => {
const child = await origElWaitForSelector(selector, options);
if (child) patchSingleElementHandle(child, page, cfg, cursor, raw, rawKb, originals, stealth);
return child;
};
// --- Helper: get box and move cursor. Accepts a per-call ``callCfg``
// so type/fill overrides like ``el.type(text, { human_config: {...} })``
// so type/fill overrides like ``el.type(text, { typing_delay: 30 })``
// carry through to mouse timing for that single call. Also scrolls into
// view first so off-screen elements work (#129, #172 follow-up).
const moveToElement = async (callCfg: HumanConfig = cfg) => {
@@ -633,7 +657,7 @@ function patchSingleElementHandle(
const target = clickTarget(box, isInp, callCfg);
if (callCfg.idle_between_actions) {
await humanIdle(raw, rand(callCfg.idle_between_duration[0], callCfg.idle_between_duration[1]), cursor.x, cursor.y, callCfg);
await humanIdle(raw, cursor.x, cursor.y, callCfg);
}
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, callCfg);
@@ -643,8 +667,13 @@ function patchSingleElementHandle(
};
// --- el.click() ---
(el as any).click = async (options?: any) => {
const callCfg = mergeConfig(cfg, options?.human_config);
(el as any).click = async (options?: HumanActionOptions & {
button?: 'left' | 'right' | 'middle' | 'back' | 'forward';
clickCount?: number;
count?: number;
delay?: number;
}) => {
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
const info = await moveToElement(callCfg);
if (!info) return origElClick(options);
@@ -667,8 +696,8 @@ function patchSingleElementHandle(
};
// --- el.type() ---
(el as any).type = async (text: string, options?: any) => {
const callCfg = mergeConfig(cfg, options?.human_config);
(el as any).type = async (text: string, options?: HumanActionOptions & { delay?: number }) => {
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
const info = await moveToElement(callCfg);
if (!info) return origElType(text, options);
await humanClick(raw, info.isInp, callCfg);
@@ -684,8 +713,8 @@ function patchSingleElementHandle(
// page.click(). Only patched when the underlying ElementHandle exposes
// ``scrollIntoView`` (Puppeteer v22+).
if (origElScrollIntoView) {
(el as any).scrollIntoView = async (options?: any) => {
const callCfg = mergeConfig(cfg, options?.human_config);
(el as any).scrollIntoView = async (options?: HumanActionOptions) => {
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
await (page as any)._ensureCursorInit();
try {
const { cursorX, cursorY } = await humanScrollIntoView(
@@ -703,7 +732,7 @@ function patchSingleElementHandle(
// --- el.press() ---
if (origElPress) {
(el as any).press = async (key: string, options?: any) => {
(el as any).press = async (key: string, options?: { delay?: number }) => {
await sleep(rand(20, 60));
await originals.keyboardDown(key as any);
await sleep(randRange(cfg.key_hold));
@@ -742,7 +771,7 @@ function patchSingleElementHandle(
// --- el.drop() ---
if (origElDrop) {
(el as any).drop = async (draggable: ElementHandle, options?: any) => {
(el as any).drop = async (draggable: ElementHandle, options?: { delay?: number }) => {
const srcBox = await draggable.boundingBox();
const tgtBox = await el.boundingBox();
@@ -772,7 +801,7 @@ function patchSingleElementHandle(
// --- el.dragAndDrop() ---
if (origElDragAndDrop) {
(el as any).dragAndDrop = async (targetEl: ElementHandle, options?: any) => {
(el as any).dragAndDrop = async (targetEl: ElementHandle, options?: { delay?: number }) => {
const srcBox = await el.boundingBox();
const tgtBox = await targetEl.boundingBox();
@@ -836,15 +865,22 @@ function patchSingleFrame(
const origFrameSelect = frame.select.bind(frame);
(frame as any).click = async (selector: string, options?: any) => {
(frame as any).click = async (selector: string, options?: HumanActionOptions & {
button?: 'left' | 'right' | 'middle' | 'back' | 'forward';
clickCount?: number;
count?: number;
delay?: number;
}) => {
await (page as any).click(selector, options);
};
(frame as any).hover = async (selector: string, options?: any) => {
(frame as any).hover = async (selector: string, options?: HumanActionOptions) => {
await (page as any).hover(selector, options);
};
(frame as any).type = async (selector: string, text: string, options?: any) => {
(frame as any).type = async (selector: string, text: string, options?: HumanActionOptions & {
delay?: number;
}) => {
await (page as any).type(selector, text, options);
};
@@ -858,7 +894,7 @@ function patchSingleFrame(
await (page as any).focus(selector);
};
(frame as any).tap = async (selector: string, options?: any) => {
(frame as any).tap = async (selector: string, options?: HumanActionOptions) => {
await (page as any).click(selector, options);
};
@@ -881,7 +917,11 @@ function patchSingleFrame(
return els;
};
(frame as any).waitForSelector = async (selector: string, options?: any) => {
(frame as any).waitForSelector = async (selector: string, options?: {
hidden?: boolean;
timeout?: number;
visible?: boolean;
}) => {
const el = await origFrameWaitForSelector(selector, options);
if (el) patchSingleElementHandle(el, page, cfg, cursor, raw, rawKb, originals, stealth);
return el;
@@ -927,7 +967,7 @@ export function patchBrowser(browser: Browser, cfg: HumanConfig): void {
for (const methodName of ['createBrowserContext', 'createIncognitoBrowserContext'] as const) {
if (typeof (browser as any)[methodName] === 'function') {
const origCreateContext = (browser as any)[methodName].bind(browser);
(browser as any)[methodName] = async (options?: any) => {
(browser as any)[methodName] = async (options?: Parameters<typeof origCreateContext>[0]) => {
const context: BrowserContext = await origCreateContext(options);
const origCtxNewPage = context.newPage.bind(context);
+343
View File
@@ -0,0 +1,343 @@
/**
* Playwright-style actionability checks for the humanize layer.
*
* Checks: attached, visible, stable, enabled, editable, receives pointer events.
* Retry loop with backoff matching Playwright internals: [100, 250, 500, 1000]ms.
*/
import type { Page, Frame, ElementHandle } from 'playwright-core';
// ---------------------------------------------------------------------------
// Error hierarchy
// ---------------------------------------------------------------------------
export class ActionabilityError extends Error {
selector: string;
check: string;
constructor(selector: string, check: string, message: string) {
super(`Element ${JSON.stringify(selector)} failed ${check} check: ${message}`);
this.name = 'ActionabilityError';
this.selector = selector;
this.check = check;
}
}
export class ElementNotAttachedError extends ActionabilityError {
constructor(selector: string) {
super(selector, 'attached', 'element not found in DOM');
this.name = 'ElementNotAttachedError';
}
}
export class ElementNotVisibleError extends ActionabilityError {
constructor(selector: string) {
super(selector, 'visible', 'element is not visible');
this.name = 'ElementNotVisibleError';
}
}
export class ElementNotStableError extends ActionabilityError {
constructor(selector: string) {
super(selector, 'stable', 'element position is still changing');
this.name = 'ElementNotStableError';
}
}
export class ElementNotEnabledError extends ActionabilityError {
constructor(selector: string) {
super(selector, 'enabled', 'element is disabled');
this.name = 'ElementNotEnabledError';
}
}
export class ElementNotEditableError extends ActionabilityError {
constructor(selector: string) {
super(selector, 'editable', 'element is not editable');
this.name = 'ElementNotEditableError';
}
}
export class ElementNotReceivingEventsError extends ActionabilityError {
coveringTag: string;
constructor(selector: string, coveringTag: string = 'unknown') {
super(selector, 'pointer_events', `element is covered by <${coveringTag}>`);
this.name = 'ElementNotReceivingEventsError';
this.coveringTag = coveringTag;
}
}
// ---------------------------------------------------------------------------
// Check-set constants
// ---------------------------------------------------------------------------
export type CheckName = 'attached' | 'visible' | 'enabled' | 'editable' | 'pointer_events';
export const CHECKS_CLICK: ReadonlySet<CheckName> = new Set(['attached', 'visible', 'enabled', 'pointer_events']);
export const CHECKS_HOVER: ReadonlySet<CheckName> = new Set(['attached', 'visible', 'pointer_events']);
export const CHECKS_INPUT: ReadonlySet<CheckName> = new Set(['attached', 'visible', 'enabled', 'editable', 'pointer_events']);
export const CHECKS_FOCUS: ReadonlySet<CheckName> = new Set(['attached', 'visible', 'enabled']);
export const CHECKS_CHECK: ReadonlySet<CheckName> = new Set(['attached', 'visible', 'enabled', 'pointer_events']);
const BACKOFF_MS = [100, 250, 500, 1000];
function backoffSleep(attempt: number): Promise<void> {
const idx = Math.min(attempt, BACKOFF_MS.length - 1);
return new Promise(resolve => setTimeout(resolve, BACKOFF_MS[idx]));
}
// ---------------------------------------------------------------------------
// Pre-scroll actionability
// ---------------------------------------------------------------------------
export async function ensureActionable(
pageOrFrame: Page | Frame,
selector: string,
checks: ReadonlySet<CheckName>,
timeout: number = 30000,
force: boolean = false,
): Promise<void> {
if (force) return;
const deadline = Date.now() + timeout;
let attempt = 0;
let lastError: ActionabilityError | null = null;
while (true) {
const remainingMs = Math.max(0, deadline - Date.now());
if (remainingMs <= 0) {
if (lastError) throw lastError;
throw new ActionabilityError(selector, 'timeout', 'timeout expired before first check');
}
try {
const loc = pageOrFrame.locator(selector).first();
if (checks.has('attached')) {
try {
await loc.waitFor({ state: 'attached', timeout: Math.max(1, Math.min(remainingMs, 2000)) });
} catch {
throw new ElementNotAttachedError(selector);
}
}
if (checks.has('visible')) {
if (!await loc.isVisible()) throw new ElementNotVisibleError(selector);
}
if (checks.has('enabled')) {
if (!await loc.isEnabled()) throw new ElementNotEnabledError(selector);
}
if (checks.has('editable')) {
if (!await loc.isEditable()) throw new ElementNotEditableError(selector);
}
return;
} catch (e) {
if (e instanceof ActionabilityError) {
lastError = e;
if (Date.now() >= deadline) throw lastError;
await backoffSleep(attempt);
attempt++;
} else {
throw e;
}
}
}
}
// ---------------------------------------------------------------------------
// Post-scroll stability check
// ---------------------------------------------------------------------------
function boxesDiffer(
a: { x: number; y: number; width: number; height: number },
b: { x: number; y: number; width: number; height: number },
): boolean {
return (
Math.abs(a.x - b.x) > 1 ||
Math.abs(a.y - b.y) > 1 ||
Math.abs(a.width - b.width) > 1 ||
Math.abs(a.height - b.height) > 1
);
}
export async function ensureStable(
pageOrFrame: Page | Frame,
selector: string,
timeout: number = 5000,
): Promise<void> {
const deadline = Date.now() + timeout;
let attempt = 0;
while (true) {
const remainingMs = Math.max(0, deadline - Date.now());
if (remainingMs <= 0) throw new ElementNotStableError(selector);
const loc = pageOrFrame.locator(selector).first();
const box1 = await loc.boundingBox({ timeout: Math.max(1, Math.min(remainingMs, 1000)) });
if (!box1) throw new ElementNotAttachedError(selector);
await new Promise(r => setTimeout(r, 100));
const box2 = await loc.boundingBox({ timeout: Math.max(1, Math.min(remainingMs, 1000)) });
if (!box2) throw new ElementNotAttachedError(selector);
if (!boxesDiffer(box1, box2)) return;
if (Date.now() >= deadline) throw new ElementNotStableError(selector);
await backoffSleep(attempt);
attempt++;
}
}
// ---------------------------------------------------------------------------
// Pointer-events check (post-scroll, at actual click coordinates)
// ---------------------------------------------------------------------------
const POINTER_EVENTS_LOCATOR_JS = `(expected, data) => {
const rect = expected.getBoundingClientRect();
const frameOffsetX = data.box ? data.box.x - rect.x : 0;
const frameOffsetY = data.box ? data.box.y - rect.y : 0;
const target = document.elementFromPoint(data.x - frameOffsetX, data.y - frameOffsetY);
if (!target) return { hit: false, reason: 'no_element_at_point', covering: 'none' };
let node = target;
while (node) { if (node === expected) return { hit: true }; node = node.parentNode; }
if (expected.contains(target)) return { hit: true };
return { hit: false, reason: 'covered', covering: target.tagName || 'unknown' };
}`;
const POINTER_EVENTS_HANDLE_JS = `(expected, data) => {
const rect = expected.getBoundingClientRect();
const frameOffsetX = data.box ? data.box.x - rect.x : 0;
const frameOffsetY = data.box ? data.box.y - rect.y : 0;
const target = document.elementFromPoint(data.x - frameOffsetX, data.y - frameOffsetY);
if (!target) return { hit: false, reason: 'no_element_at_point', covering: 'none' };
let node = target;
while (node) { if (node === expected) return { hit: true }; node = node.parentNode; }
if (expected.contains(target)) return { hit: true };
return { hit: false, reason: 'covered', covering: target.tagName || 'unknown' };
}`;
export async function checkPointerEvents(
pageOrFrame: Page | Frame,
selector: string,
x: number,
y: number,
stealth?: { evaluate(expression: string): Promise<any> } | null,
timeout: number = 5000,
): Promise<void> {
const deadline = Date.now() + timeout;
let attempt = 0;
while (true) {
let result: any = null;
try {
const loc = pageOrFrame.locator(selector).first();
const box = await loc.boundingBox({ timeout: Math.max(1, Math.min(deadline - Date.now(), 1000)) });
result = await loc.evaluate(POINTER_EVENTS_LOCATOR_JS, { x, y, box });
} catch {
result = null;
}
if (!result || result.hit) return;
const covering = (result as any)?.covering ?? 'unknown';
if (Date.now() >= deadline) throw new ElementNotReceivingEventsError(selector, covering);
await backoffSleep(attempt);
attempt++;
}
}
// ---------------------------------------------------------------------------
// ElementHandle variant
// ---------------------------------------------------------------------------
export async function ensureActionableHandle(
el: ElementHandle,
checks: ReadonlySet<CheckName>,
timeout: number = 30000,
force: boolean = false,
): Promise<void> {
if (force) return;
const deadline = Date.now() + timeout;
let attempt = 0;
let lastError: ActionabilityError | null = null;
const label = '<ElementHandle>';
while (true) {
const remainingMs = Math.max(0, deadline - Date.now());
if (remainingMs <= 0) {
if (lastError) throw lastError;
throw new ActionabilityError(label, 'timeout', 'timeout expired before first check');
}
try {
if (checks.has('visible')) {
try {
await el.waitForElementState('visible', { timeout: Math.max(1, Math.min(remainingMs, 2000)) });
} catch {
throw new ElementNotVisibleError(label);
}
}
if (checks.has('enabled')) {
try {
await el.waitForElementState('enabled', { timeout: Math.max(1, Math.min(remainingMs, 2000)) });
} catch {
throw new ElementNotEnabledError(label);
}
}
if (checks.has('editable')) {
try {
await el.waitForElementState('editable', { timeout: Math.max(1, Math.min(remainingMs, 2000)) });
} catch {
throw new ElementNotEditableError(label);
}
}
return;
} catch (e) {
if (e instanceof ActionabilityError) {
lastError = e;
if (Date.now() >= deadline) throw lastError;
await backoffSleep(attempt);
attempt++;
} else {
throw e;
}
}
}
}
export async function checkPointerEventsHandle(
el: ElementHandle,
x: number,
y: number,
timeout: number = 5000,
): Promise<void> {
const deadline = Date.now() + timeout;
let attempt = 0;
while (true) {
let result: any;
try {
const box = await el.boundingBox();
result = await el.evaluate(POINTER_EVENTS_HANDLE_JS, { x, y, box });
} catch {
result = null;
}
if (!result || result.hit) return;
const covering = (result as any)?.covering ?? 'unknown';
if (Date.now() >= deadline) throw new ElementNotReceivingEventsError('<ElementHandle>', covering);
await backoffSleep(attempt);
attempt++;
}
}
+6
View File
@@ -70,6 +70,12 @@ export interface HumanConfig {
export type HumanPreset = 'default' | 'careful';
export type HumanActionOptions = Partial<HumanConfig> & {
timeout?: number;
force?: boolean;
human_config?: Partial<HumanConfig>;
};
// ---------------------------------------------------------------------------
// Default preset
// ---------------------------------------------------------------------------
+156 -32
View File
@@ -17,11 +17,15 @@
*/
import type { Page, Frame, ElementHandle, CDPSession } from 'playwright-core';
import type { HumanConfig } from './config.js';
import type { HumanConfig, HumanActionOptions } from './config.js';
import { rand, randRange, sleep, mergeConfig } from './config.js';
import { RawMouse, RawKeyboard, humanMove, humanClick, clickTarget, humanIdle } from './mouse.js';
import { humanType } from './keyboard.js';
import { humanScrollIntoView } from './scroll.js';
import {
ensureActionableHandle, checkPointerEventsHandle,
CHECKS_CLICK, CHECKS_HOVER, CHECKS_INPUT, CHECKS_FOCUS, CHECKS_CHECK,
} from './actionability.js';
// --- Platform-aware select-all shortcut ---
const SELECT_ALL = process.platform === 'darwin' ? 'Meta+a' : 'Control+a';
@@ -125,16 +129,21 @@ export function patchSingleElementHandle(
return children;
};
(el as any).waitForSelector = async (selector: string, options?: any) => {
const child = await origElWaitForSelector(selector, options);
(el as any).waitForSelector = async (selector: string, options?: {
state?: 'attached' | 'detached' | 'visible' | 'hidden';
strict?: boolean;
timeout?: number;
}) => {
const child = await origElWaitForSelector(selector, options ?? {});
if (child) patchSingleElementHandle(child, page, cfg, cursor, raw, rawKb, originals, stealth);
return child;
};
// --- Helper: get bounding box and move cursor to element ---
// Accepts a per-call ``callCfg`` so type/fill overrides like
// ``el.type(text, { human_config: { typing_delay: 30 } })`` carry through to
// mouse movement & idle timing for that single call.
// ``el.type(text, { human_config: { typing_delay: 30 } })`` or
// ``el.type(text, { typing_delay: 30 })`` carry through to mouse movement
// & idle timing for that single call.
// Also scrolls the element into view first so off-screen elements work
// (#129, #172 follow-up): otherwise boundingBox() returns null and we'd
// silently fall back to the unpatched native method.
@@ -164,7 +173,7 @@ export function patchSingleElementHandle(
const target = clickTarget(box, isInp, callCfg);
if (callCfg.idle_between_actions) {
await humanIdle(raw, rand(callCfg.idle_between_duration[0], callCfg.idle_between_duration[1]), cursor.x, cursor.y, callCfg);
await humanIdle(raw, cursor.x, cursor.y, callCfg);
}
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, callCfg);
@@ -174,36 +183,83 @@ export function patchSingleElementHandle(
};
// --- el.click() ---
(el as any).click = async (options?: any) => {
const callCfg = mergeConfig(cfg, options?.human_config);
(el as any).click = async (options?: HumanActionOptions & {
button?: 'left' | 'right' | 'middle';
clickCount?: number;
delay?: number;
force?: boolean;
modifiers?: Array<'Alt' | 'Control' | 'ControlOrMeta' | 'Meta' | 'Shift'>;
noWaitAfter?: boolean;
position?: { x: number; y: number };
trial?: boolean;
}) => {
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
const force = options?.force ?? false;
const timeout = options?.timeout ?? 30000;
const deadline = Date.now() + timeout;
const remainingMs = () => Math.max(0, deadline - Date.now());
if (!force) await ensureActionableHandle(el, CHECKS_CLICK, remainingMs(), force);
const info = await moveToElement(callCfg);
if (!info) return origElClick(options);
if (!force) await checkPointerEventsHandle(el, cursor.x, cursor.y, Math.min(remainingMs(), 5000));
await humanClick(raw, info.isInp, callCfg);
};
// --- el.dblclick() ---
(el as any).dblclick = async (options?: any) => {
const callCfg = mergeConfig(cfg, options?.human_config);
(el as any).dblclick = async (options?: HumanActionOptions & {
button?: 'left' | 'right' | 'middle';
delay?: number;
force?: boolean;
modifiers?: Array<'Alt' | 'Control' | 'ControlOrMeta' | 'Meta' | 'Shift'>;
noWaitAfter?: boolean;
position?: { x: number; y: number };
trial?: boolean;
}) => {
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
const force = options?.force ?? false;
const timeout = options?.timeout ?? 30000;
const deadline = Date.now() + timeout;
const remainingMs = () => Math.max(0, deadline - Date.now());
if (!force) await ensureActionableHandle(el, CHECKS_CLICK, remainingMs(), force);
const info = await moveToElement(callCfg);
if (!info) return origElDblclick(options);
if (!force) await checkPointerEventsHandle(el, cursor.x, cursor.y, Math.min(remainingMs(), 5000));
await raw.down({ clickCount: 2 });
await sleep(rand(30, 60));
await raw.up({ clickCount: 2 });
};
// --- el.hover() ---
(el as any).hover = async (options?: any) => {
const callCfg = mergeConfig(cfg, options?.human_config);
(el as any).hover = async (options?: HumanActionOptions & {
force?: boolean;
modifiers?: Array<'Alt' | 'Control' | 'ControlOrMeta' | 'Meta' | 'Shift'>;
position?: { x: number; y: number };
trial?: boolean;
}) => {
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
const force = options?.force ?? false;
const timeout = options?.timeout ?? 30000;
const deadline = Date.now() + timeout;
const remainingMs = () => Math.max(0, deadline - Date.now());
if (!force) await ensureActionableHandle(el, CHECKS_HOVER, remainingMs(), force);
const info = await moveToElement(callCfg);
if (!info) return origElHover(options);
// Just move — no click
};
// --- el.type() ---
(el as any).type = async (text: string, options?: any) => {
const callCfg = mergeConfig(cfg, options?.human_config);
(el as any).type = async (text: string, options?: HumanActionOptions & {
delay?: number;
noWaitAfter?: boolean;
}) => {
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
const force = (options as any)?.force ?? false;
const timeout = options?.timeout ?? 30000;
const deadline = Date.now() + timeout;
const remainingMs = () => Math.max(0, deadline - Date.now());
if (!force) await ensureActionableHandle(el, CHECKS_INPUT, remainingMs(), force);
const info = await moveToElement(callCfg);
if (!info) return origElType(text, options);
if (!force) await checkPointerEventsHandle(el, cursor.x, cursor.y, Math.min(remainingMs(), 5000));
await humanClick(raw, info.isInp, callCfg);
await sleep(rand(100, 250));
let cdpSession: CDPSession | null = null;
@@ -212,13 +268,21 @@ export function patchSingleElementHandle(
};
// --- el.fill() ---
(el as any).fill = async (value: string, options?: any) => {
const callCfg = mergeConfig(cfg, options?.human_config);
(el as any).fill = async (value: string, options?: HumanActionOptions & {
force?: boolean;
noWaitAfter?: boolean;
}) => {
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
const force = options?.force ?? false;
const timeout = options?.timeout ?? 30000;
const deadline = Date.now() + timeout;
const remainingMs = () => Math.max(0, deadline - Date.now());
if (!force) await ensureActionableHandle(el, CHECKS_INPUT, remainingMs(), force);
const info = await moveToElement(callCfg);
if (!info) return origElFill(value, options);
if (!force) await checkPointerEventsHandle(el, cursor.x, cursor.y, Math.min(remainingMs(), 5000));
await humanClick(raw, info.isInp, callCfg);
await sleep(rand(100, 250));
// Clear existing content
await originals.keyboardPress(SELECT_ALL);
await sleep(rand(30, 80));
await originals.keyboardPress('Backspace');
@@ -229,7 +293,7 @@ export function patchSingleElementHandle(
};
// --- el.press() ---
(el as any).press = async (key: string, options?: any) => {
(el as any).press = async (key: string, options?: { delay?: number; noWaitAfter?: boolean; timeout?: number }) => {
await sleep(rand(20, 60));
await originals.keyboardDown(key);
await sleep(randRange(cfg.key_hold));
@@ -237,7 +301,16 @@ export function patchSingleElementHandle(
};
// --- el.selectOption() ---
(el as any).selectOption = async (values: any, options?: any) => {
(el as any).selectOption = async (values: any, options?: {
force?: boolean;
noWaitAfter?: boolean;
timeout?: number;
}) => {
const force = options?.force ?? false;
const timeout = options?.timeout ?? 30000;
const deadline = Date.now() + timeout;
const remainingMs = () => Math.max(0, deadline - Date.now());
if (!force) await ensureActionableHandle(el, CHECKS_FOCUS, remainingMs(), force);
const info = await moveToElement();
if (!info) return origElSelectOption(values, options);
await humanClick(raw, false, cfg);
@@ -246,42 +319,85 @@ export function patchSingleElementHandle(
};
// --- el.check() ---
(el as any).check = async (options?: any) => {
(el as any).check = async (options?: {
force?: boolean;
noWaitAfter?: boolean;
position?: { x: number; y: number };
timeout?: number;
trial?: boolean;
}) => {
const force = options?.force ?? false;
const timeout = options?.timeout ?? 30000;
const deadline = Date.now() + timeout;
const remainingMs = () => Math.max(0, deadline - Date.now());
if (!force) await ensureActionableHandle(el, CHECKS_CHECK, remainingMs(), force);
try {
const checked = await el.isChecked();
if (checked) return; // Already checked
if (checked) return;
} catch {}
const info = await moveToElement();
if (!info) return origElCheck(options);
if (!force) await checkPointerEventsHandle(el, cursor.x, cursor.y, Math.min(remainingMs(), 5000));
await humanClick(raw, info.isInp, cfg);
};
// --- el.uncheck() ---
(el as any).uncheck = async (options?: any) => {
(el as any).uncheck = async (options?: {
force?: boolean;
noWaitAfter?: boolean;
position?: { x: number; y: number };
timeout?: number;
trial?: boolean;
}) => {
const force = options?.force ?? false;
const timeout = options?.timeout ?? 30000;
const deadline = Date.now() + timeout;
const remainingMs = () => Math.max(0, deadline - Date.now());
if (!force) await ensureActionableHandle(el, CHECKS_CHECK, remainingMs(), force);
try {
const checked = await el.isChecked();
if (!checked) return; // Already unchecked
if (!checked) return;
} catch {}
const info = await moveToElement();
if (!info) return origElUncheck(options);
if (!force) await checkPointerEventsHandle(el, cursor.x, cursor.y, Math.min(remainingMs(), 5000));
await humanClick(raw, info.isInp, cfg);
};
// --- el.setChecked() ---
if (origElSetChecked) {
(el as any).setChecked = async (checked: boolean, options?: any) => {
(el as any).setChecked = async (checked: boolean, options?: {
force?: boolean;
noWaitAfter?: boolean;
position?: { x: number; y: number };
timeout?: number;
trial?: boolean;
}) => {
const force = options?.force ?? false;
const timeout = options?.timeout ?? 30000;
const deadline = Date.now() + timeout;
const remainingMs = () => Math.max(0, deadline - Date.now());
if (!force) await ensureActionableHandle(el, CHECKS_CHECK, remainingMs(), force);
try {
const current = await el.isChecked();
if (current === checked) return;
} catch {}
const info = await moveToElement();
if (!info) return origElSetChecked(checked, options);
if (!force) await checkPointerEventsHandle(el, cursor.x, cursor.y, Math.min(remainingMs(), 5000));
await humanClick(raw, info.isInp, cfg);
};
}
// --- el.tap() ---
(el as any).tap = async (options?: any) => {
(el as any).tap = async (options?: {
force?: boolean;
modifiers?: Array<'Alt' | 'Control' | 'ControlOrMeta' | 'Meta' | 'Shift'>;
noWaitAfter?: boolean;
position?: { x: number; y: number };
timeout?: number;
trial?: boolean;
}) => {
const info = await moveToElement();
if (!info) return origElTap(options);
await humanClick(raw, info.isInp, cfg);
@@ -302,8 +418,8 @@ export function patchSingleElementHandle(
// wheel sequence used by page.click() etc. Falls back to the native
// method if the element is detached or scrolling fails.
if (origElScrollIntoViewIfNeeded) {
(el as any).scrollIntoViewIfNeeded = async (options?: any) => {
const callCfg = mergeConfig(cfg, options?.human_config);
(el as any).scrollIntoViewIfNeeded = async (options?: HumanActionOptions) => {
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
const ensureCursorInit = (page as any)._ensureCursorInit;
if (ensureCursorInit) await ensureCursorInit();
try {
@@ -360,8 +476,12 @@ export function patchPageElementHandles(
// Patch page.waitForSelector()
if (typeof page.waitForSelector === 'function') {
const origWaitForSelector = page.waitForSelector.bind(page);
(page as any).waitForSelector = async (selector: string, options?: any) => {
const el = await origWaitForSelector(selector, options);
(page as any).waitForSelector = async (selector: string, options?: {
state?: 'attached' | 'detached' | 'visible' | 'hidden';
strict?: boolean;
timeout?: number;
}) => {
const el = await origWaitForSelector(selector, options ?? {});
if (el) patchSingleElementHandle(el, page, cfg, cursor, raw, rawKb, originals, stealth);
return el;
};
@@ -408,8 +528,12 @@ export function patchFrameElementHandles(
// Patch frame.waitForSelector()
if (typeof frame.waitForSelector === 'function') {
const origFrameWaitForSelector = frame.waitForSelector.bind(frame);
(frame as any).waitForSelector = async (selector: string, options?: any) => {
const el = await origFrameWaitForSelector(selector, options);
(frame as any).waitForSelector = async (selector: string, options?: {
state?: 'attached' | 'detached' | 'visible' | 'hidden';
strict?: boolean;
timeout?: number;
}) => {
const el = await origFrameWaitForSelector(selector, options ?? {});
if (el) patchSingleElementHandle(el, page, cfg, cursor, raw, rawKb, originals, stealth);
return el;
};
+308 -77
View File
@@ -23,11 +23,16 @@
*/
import type { Browser, BrowserContext, Page, Frame, CDPSession } from 'playwright-core';
import { HumanConfig, resolveConfig, mergeConfig, rand, randRange, sleep } from './config.js';
import { HumanConfig, HumanActionOptions, resolveConfig, mergeConfig, rand, randRange, sleep } from './config.js';
import { RawMouse, RawKeyboard, humanMove, humanClick, clickTarget, humanIdle } from './mouse.js';
import { humanType } from './keyboard.js';
import { scrollToElement, humanScrollIntoView } from './scroll.js';
import { patchPageElementHandles, patchFrameElementHandles, patchSingleElementHandle } from './elementhandle.js';
import {
ensureActionable, ensureStable, checkPointerEvents,
CHECKS_CLICK, CHECKS_HOVER, CHECKS_INPUT, CHECKS_FOCUS, CHECKS_CHECK,
type CheckName,
} from './actionability.js';
export { HumanConfig, resolveConfig, mergeConfig } from './config.js';
export { humanMove, humanClick, clickTarget, humanIdle } from './mouse.js';
@@ -295,7 +300,11 @@ function patchPage(page: Page, cfg: HumanConfig, cursor: CursorState): void {
}
// --- goto (invalidate isolated world on navigation) ---
const humanGoto = async (url: string, options?: any) => {
const humanGoto = async (url: string, options?: {
referer?: string;
timeout?: number;
waitUntil?: 'load' | 'domcontentloaded' | 'networkidle' | 'commit';
}) => {
const response = await originals.goto(url, options);
stealth.invalidate();
patchFrames(page, cfg, cursor, raw, rawKb, originals, stealth);
@@ -303,17 +312,34 @@ function patchPage(page: Page, cfg: HumanConfig, cursor: CursorState): void {
};
// --- click ---
const humanClickFn = async (selector: string, options?: any) => {
const humanClickFn = async (selector: string, options?: HumanActionOptions & { _skipChecks?: boolean }) => {
await ensureCursorInit();
const callCfg = mergeConfig(cfg, options?.human_config);
if (callCfg.idle_between_actions) {
await humanIdle(raw, rand(callCfg.idle_between_duration[0], callCfg.idle_between_duration[1]), cursor.x, cursor.y, callCfg);
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
const timeout = options?.timeout ?? 30000;
const force = options?.force ?? false;
const skipChecks = (options as any)?._skipChecks ?? false;
const deadline = Date.now() + timeout;
const remainingMs = () => Math.max(0, deadline - Date.now());
if (!force && !skipChecks) {
await ensureActionable(page, selector, CHECKS_CLICK, remainingMs(), force);
}
const { box, cursorX, cursorY } = await scrollToElement(page, raw, selector, cursor.x, cursor.y, callCfg, options?.timeout);
if (callCfg.idle_between_actions) {
await humanIdle(raw, cursor.x, cursor.y, callCfg);
}
const { box, cursorX, cursorY, didScroll } = await scrollToElement(page, raw, selector, cursor.x, cursor.y, callCfg, remainingMs());
cursor.x = cursorX;
cursor.y = cursorY;
const isInput = await isInputElement(stealth, page, selector);
const target = clickTarget(box, isInput, callCfg);
let finalBox = box;
if (!force && didScroll) {
await ensureStable(page, selector, remainingMs());
finalBox = await page.locator(selector).first().boundingBox({ timeout: Math.max(1, remainingMs()) }) ?? box;
}
const target = clickTarget(finalBox, isInput, callCfg);
if (!force) {
await checkPointerEvents(page, selector, target.x, target.y, stealth, remainingMs());
}
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, callCfg);
cursor.x = target.x;
cursor.y = target.y;
@@ -321,17 +347,31 @@ function patchPage(page: Page, cfg: HumanConfig, cursor: CursorState): void {
};
// --- dblclick ---
const humanDblclickFn = async (selector: string, options?: any) => {
const humanDblclickFn = async (selector: string, options?: HumanActionOptions) => {
await ensureCursorInit();
const callCfg = mergeConfig(cfg, options?.human_config);
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
const timeout = options?.timeout ?? 30000;
const force = options?.force ?? false;
const deadline = Date.now() + timeout;
const remainingMs = () => Math.max(0, deadline - Date.now());
if (!force) await ensureActionable(page, selector, CHECKS_CLICK, remainingMs(), force);
if (callCfg.idle_between_actions) {
await humanIdle(raw, rand(callCfg.idle_between_duration[0], callCfg.idle_between_duration[1]), cursor.x, cursor.y, callCfg);
await humanIdle(raw, cursor.x, cursor.y, callCfg);
}
const { box, cursorX, cursorY } = await scrollToElement(page, raw, selector, cursor.x, cursor.y, callCfg, options?.timeout);
const { box, cursorX, cursorY, didScroll } = await scrollToElement(page, raw, selector, cursor.x, cursor.y, callCfg, remainingMs());
cursor.x = cursorX;
cursor.y = cursorY;
const isInput = await isInputElement(stealth, page, selector);
const target = clickTarget(box, isInput, callCfg);
let finalBox = box;
if (!force && didScroll) {
await ensureStable(page, selector, remainingMs());
finalBox = await page.locator(selector).first().boundingBox({ timeout: Math.max(1, remainingMs()) }) ?? box;
}
const target = clickTarget(finalBox, isInput, callCfg);
if (!force) {
await checkPointerEvents(page, selector, target.x, target.y, stealth, remainingMs());
}
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, callCfg);
cursor.x = target.x;
cursor.y = target.y;
@@ -341,36 +381,63 @@ function patchPage(page: Page, cfg: HumanConfig, cursor: CursorState): void {
};
// --- hover ---
const humanHoverFn = async (selector: string, options?: any) => {
const humanHoverFn = async (selector: string, options?: HumanActionOptions & { _skipChecks?: boolean }) => {
await ensureCursorInit();
const callCfg = mergeConfig(cfg, options?.human_config);
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
const timeout = options?.timeout ?? 30000;
const force = options?.force ?? false;
const skipChecks = (options as any)?._skipChecks ?? false;
const deadline = Date.now() + timeout;
const remainingMs = () => Math.max(0, deadline - Date.now());
if (!force && !skipChecks) await ensureActionable(page, selector, CHECKS_HOVER, remainingMs(), force);
if (callCfg.idle_between_actions) {
await humanIdle(raw, rand(callCfg.idle_between_duration[0], callCfg.idle_between_duration[1]), cursor.x, cursor.y, callCfg);
await humanIdle(raw, cursor.x, cursor.y, callCfg);
}
const { box, cursorX, cursorY } = await scrollToElement(page, raw, selector, cursor.x, cursor.y, callCfg, options?.timeout);
const { box, cursorX, cursorY, didScroll } = await scrollToElement(page, raw, selector, cursor.x, cursor.y, callCfg, remainingMs());
cursor.x = cursorX;
cursor.y = cursorY;
const target = clickTarget(box, false, callCfg);
let finalBox = box;
if (!force && didScroll) {
await ensureStable(page, selector, remainingMs());
finalBox = await page.locator(selector).first().boundingBox({ timeout: Math.max(1, remainingMs()) }) ?? box;
}
const target = clickTarget(finalBox, false, callCfg);
if (!force) {
await checkPointerEvents(page, selector, target.x, target.y, stealth, remainingMs());
}
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, callCfg);
cursor.x = target.x;
cursor.y = target.y;
};
// --- type ---
const humanTypeFn = async (selector: string, text: string, options?: any) => {
const callCfg = mergeConfig(cfg, options?.human_config);
const humanTypeFn = async (selector: string, text: string, options?: HumanActionOptions) => {
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
const timeout = options?.timeout ?? 30000;
const force = options?.force ?? false;
const deadline = Date.now() + timeout;
const remainingMs = () => Math.max(0, deadline - Date.now());
if (!force) await ensureActionable(page, selector, CHECKS_INPUT, remainingMs(), force);
await sleep(randRange(callCfg.field_switch_delay));
await humanClickFn(selector, options);
await humanClickFn(selector, { _skipChecks: true, timeout: remainingMs(), force, human_config: options?.human_config } as any);
await sleep(rand(100, 250));
const cdp = await ensureCdp();
await humanType(page, rawKb, text, callCfg, cdp);
};
// --- fill (clears existing content first) ---
const humanFillFn = async (selector: string, value: string, options?: any) => {
const callCfg = mergeConfig(cfg, options?.human_config);
const humanFillFn = async (selector: string, value: string, options?: HumanActionOptions) => {
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
const timeout = options?.timeout ?? 30000;
const force = options?.force ?? false;
const deadline = Date.now() + timeout;
const remainingMs = () => Math.max(0, deadline - Date.now());
if (!force) await ensureActionable(page, selector, CHECKS_INPUT, remainingMs(), force);
await sleep(randRange(callCfg.field_switch_delay));
await humanClickFn(selector, options);
await humanClickFn(selector, { _skipChecks: true, timeout: remainingMs(), force, human_config: options?.human_config } as any);
await sleep(rand(100, 250));
await originals.keyboardPress(SELECT_ALL);
await sleep(rand(30, 80));
@@ -381,9 +448,15 @@ function patchPage(page: Page, cfg: HumanConfig, cursor: CursorState): void {
};
// --- clear ---
const humanClearFn = async (selector: string, options?: any) => {
const humanClearFn = async (selector: string, options?: HumanActionOptions) => {
const timeout = options?.timeout ?? 30000;
const force = options?.force ?? false;
const deadline = Date.now() + timeout;
const remainingMs = () => Math.max(0, deadline - Date.now());
if (!force) await ensureActionable(page, selector, CHECKS_FOCUS, remainingMs(), force);
if (!await isSelectorFocused(stealth, page, selector)) {
await humanClickFn(selector);
await humanClickFn(selector, { _skipChecks: true, timeout: remainingMs(), force, human_config: options?.human_config } as any);
}
await sleep(rand(50, 150));
await originals.keyboardPress(SELECT_ALL);
@@ -392,48 +465,80 @@ function patchPage(page: Page, cfg: HumanConfig, cursor: CursorState): void {
};
// --- check ---
const humanCheckFn = async (selector: string, options?: any) => {
if (cfg.idle_between_actions) {
await humanIdle(raw, rand(cfg.idle_between_duration[0], cfg.idle_between_duration[1]), cursor.x, cursor.y, cfg);
const humanCheckFn = async (selector: string, options?: HumanActionOptions) => {
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
const timeout = options?.timeout ?? 30000;
const force = options?.force ?? false;
const deadline = Date.now() + timeout;
const remainingMs = () => Math.max(0, deadline - Date.now());
if (!force) await ensureActionable(page, selector, CHECKS_CHECK, remainingMs(), force);
if (callCfg.idle_between_actions) {
await humanIdle(raw, cursor.x, cursor.y, callCfg);
}
const checked = await originals.isChecked(selector).catch(() => false);
if (!checked) {
await humanClickFn(selector);
await humanClickFn(selector, { _skipChecks: true, timeout: remainingMs(), force, human_config: options?.human_config } as any);
}
};
// --- uncheck ---
const humanUncheckFn = async (selector: string, options?: any) => {
if (cfg.idle_between_actions) {
await humanIdle(raw, rand(cfg.idle_between_duration[0], cfg.idle_between_duration[1]), cursor.x, cursor.y, cfg);
const humanUncheckFn = async (selector: string, options?: HumanActionOptions) => {
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
const timeout = options?.timeout ?? 30000;
const force = options?.force ?? false;
const deadline = Date.now() + timeout;
const remainingMs = () => Math.max(0, deadline - Date.now());
if (!force) await ensureActionable(page, selector, CHECKS_CHECK, remainingMs(), force);
if (callCfg.idle_between_actions) {
await humanIdle(raw, cursor.x, cursor.y, callCfg);
}
const checked = await originals.isChecked(selector).catch(() => true);
if (checked) {
await humanClickFn(selector);
await humanClickFn(selector, { _skipChecks: true, timeout: remainingMs(), force, human_config: options?.human_config } as any);
}
};
// --- selectOption ---
const humanSelectOptionFn = async (selector: string, values: any, options?: any) => {
await humanHoverFn(selector);
const humanSelectOptionFn = async (selector: string, values: any, options?: HumanActionOptions) => {
const timeout = options?.timeout ?? 30000;
const force = options?.force ?? false;
const deadline = Date.now() + timeout;
const remainingMs = () => Math.max(0, deadline - Date.now());
if (!force) await ensureActionable(page, selector, CHECKS_FOCUS, remainingMs(), force);
await humanHoverFn(selector, { _skipChecks: true, timeout: remainingMs(), force, human_config: options?.human_config } as any);
await sleep(rand(100, 300));
return originals.selectOption(selector, values, options);
};
// --- press (checks focus first — avoids redundant mouse moves) ---
const humanPressFn = async (selector: string, key: string, options?: any) => {
const humanPressFn = async (selector: string, key: string, options?: HumanActionOptions) => {
const timeout = options?.timeout ?? 30000;
const force = options?.force ?? false;
const deadline = Date.now() + timeout;
const remainingMs = () => Math.max(0, deadline - Date.now());
if (!force) await ensureActionable(page, selector, CHECKS_FOCUS, remainingMs(), force);
if (!await isSelectorFocused(stealth, page, selector)) {
await humanClickFn(selector);
await humanClickFn(selector, { _skipChecks: true, timeout: remainingMs(), force, human_config: options?.human_config } as any);
}
await sleep(rand(50, 150));
await originals.keyboardPress(key);
};
// --- pressSequentially ---
const humanPressSequentiallyFn = async (selector: string, text: string, options?: any) => {
const callCfg = mergeConfig(cfg, options?.human_config);
const humanPressSequentiallyFn = async (selector: string, text: string, options?: HumanActionOptions) => {
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
const timeout = options?.timeout ?? 30000;
const force = options?.force ?? false;
const deadline = Date.now() + timeout;
const remainingMs = () => Math.max(0, deadline - Date.now());
if (!force) await ensureActionable(page, selector, CHECKS_FOCUS, remainingMs(), force);
if (!await isSelectorFocused(stealth, page, selector)) {
await humanClickFn(selector, options);
await humanClickFn(selector, { _skipChecks: true, timeout: remainingMs(), force, human_config: options?.human_config } as any);
}
await sleep(rand(100, 250));
const cdp = await ensureCdp();
@@ -441,7 +546,7 @@ function patchPage(page: Page, cfg: HumanConfig, cursor: CursorState): void {
};
// --- tap ---
const humanTapFn = async (selector: string, options?: any) => {
const humanTapFn = async (selector: string, options?: HumanActionOptions) => {
await humanClickFn(selector, options);
};
@@ -461,14 +566,20 @@ function patchPage(page: Page, cfg: HumanConfig, cursor: CursorState): void {
(page as any).clear = humanClearFn;
// --- mouse patches ---
page.mouse.move = async (x: number, y: number, options?: any) => {
page.mouse.move = async (x: number, y: number, options?: {
steps?: number;
}) => {
await ensureCursorInit();
await humanMove(raw, cursor.x, cursor.y, x, y, cfg);
cursor.x = x;
cursor.y = y;
};
page.mouse.click = async (x: number, y: number, options?: any) => {
page.mouse.click = async (x: number, y: number, options?: {
button?: 'left' | 'right' | 'middle';
clickCount?: number;
delay?: number;
}) => {
await ensureCursorInit();
await humanMove(raw, cursor.x, cursor.y, x, y, cfg);
cursor.x = x;
@@ -477,7 +588,7 @@ function patchPage(page: Page, cfg: HumanConfig, cursor: CursorState): void {
};
// --- keyboard patches ---
page.keyboard.type = async (text: string, options?: any) => {
page.keyboard.type = async (text: string, options?: { delay?: number }) => {
const cdp = await ensureCdp();
await humanType(page, rawKb, text, cfg, cdp);
};
@@ -529,16 +640,37 @@ function patchFrames(
stealth: StealthEval,
): void {
for (const frame of iterFrames(page)) {
patchSingleFrame(frame, page, cfg, originals, stealth);
patchSingleFrame(frame, page, cfg, cursor, raw, rawKb, originals, stealth);
// Patch frame-level ElementHandle selectors ($, $$, waitForSelector)
patchFrameElementHandles(frame, page, cfg, cursor, raw, rawKb, originals, stealth);
}
}
function firstFrameLocator(frame: Frame, selector: string): any {
const locator = frame.locator(selector) as any;
return typeof locator.first === 'function' ? locator.first() : locator;
}
async function isFrameInputElement(frame: Frame, selector: string): Promise<boolean> {
return firstFrameLocator(frame, selector).evaluate((el: Element) => {
const tag = el.tagName.toLowerCase();
return tag === 'input' || tag === 'textarea'
|| el.getAttribute('contenteditable') === 'true';
}).catch(() => false);
}
async function isFrameSelectorFocused(frame: Frame, selector: string): Promise<boolean> {
return firstFrameLocator(frame, selector).evaluate((el: Element) => el === document.activeElement)
.catch(() => false);
}
function patchSingleFrame(
frame: Frame,
page: Page,
cfg: HumanConfig,
cursor: CursorState,
raw: RawMouse,
rawKb: RawKeyboard,
originals: any,
stealth: StealthEval,
): void {
@@ -546,58 +678,146 @@ function patchSingleFrame(
(frame as any)._humanPatched = true;
// Save originals for methods that need fallback
const origFrameClick = frame.click.bind(frame);
const origFrameDblclick = frame.dblclick.bind(frame);
const origFrameHover = frame.hover.bind(frame);
const origFrameType = frame.type.bind(frame);
const origFrameFill = frame.fill.bind(frame);
const origFrameCheck = frame.check.bind(frame);
const origFrameUncheck = frame.uncheck.bind(frame);
const origFrameSelectOption = frame.selectOption.bind(frame);
const origFramePress = frame.press.bind(frame);
const origFramePressSequentially = (frame as any).pressSequentially?.bind(frame);
const origFrameTap = (frame as any).tap?.bind(frame);
const origFrameDragAndDrop = frame.dragAndDrop.bind(frame);
(frame as any).click = async (selector: string, options?: any) => {
await (page as any).click(selector, options);
const moveToFrameSelector = async (
selector: string,
options: HumanActionOptions | undefined,
inputBias: boolean,
remainingMs: () => number,
) => {
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
if (callCfg.idle_between_actions) {
await humanIdle(raw, cursor.x, cursor.y, callCfg);
}
const locator = firstFrameLocator(frame, selector);
if (typeof locator.scrollIntoViewIfNeeded === 'function') {
await locator.scrollIntoViewIfNeeded({ timeout: Math.max(1, remainingMs()) }).catch(() => undefined);
}
const box = await locator.boundingBox({ timeout: Math.max(1, remainingMs()) }).catch(() => null);
if (!box) return null;
const isInput = inputBias || await isFrameInputElement(frame, selector);
const target = clickTarget(box, isInput, callCfg);
await humanMove(raw, cursor.x, cursor.y, target.x, target.y, callCfg);
cursor.x = target.x;
cursor.y = target.y;
return { callCfg, isInput };
};
(frame as any).dblclick = async (selector: string, options?: any) => {
await (page as any).dblclick(selector, options);
const frameClick = async (selector: string, options?: HumanActionOptions) => {
const timeout = options?.timeout ?? 30000;
const deadline = Date.now() + timeout;
const remainingMs = () => Math.max(0, deadline - Date.now());
const moved = await moveToFrameSelector(selector, options, false, remainingMs);
if (!moved) return origFrameClick(selector, { ...options, timeout: Math.max(1, remainingMs()) });
await humanClick(raw, moved.isInput, moved.callCfg);
};
(frame as any).hover = async (selector: string, options?: any) => {
await (page as any).hover(selector, options);
const getFrameCdp = async () => stealth.getCdpSession().catch(() => null);
const frameHover = async (selector: string, options?: HumanActionOptions) => {
const timeout = options?.timeout ?? 30000;
const deadline = Date.now() + timeout;
const remainingMs = () => Math.max(0, deadline - Date.now());
const moved = await moveToFrameSelector(selector, options, false, remainingMs);
if (!moved) return origFrameHover(selector, { ...options, timeout: Math.max(1, remainingMs()) });
};
(frame as any).type = async (selector: string, text: string, options?: any) => {
await (page as any).type(selector, text, options);
(frame as any).click = frameClick;
(frame as any).dblclick = async (selector: string, options?: HumanActionOptions) => {
const timeout = options?.timeout ?? 30000;
const deadline = Date.now() + timeout;
const remainingMs = () => Math.max(0, deadline - Date.now());
const moved = await moveToFrameSelector(selector, options, false, remainingMs);
if (!moved) return origFrameDblclick(selector, { ...options, timeout: Math.max(1, remainingMs()) });
await raw.down({ clickCount: 2 });
await sleep(rand(30, 60));
await raw.up({ clickCount: 2 });
};
(frame as any).fill = async (selector: string, value: string, options?: any) => {
await (page as any).fill(selector, value, options);
(frame as any).hover = frameHover;
(frame as any).type = async (selector: string, text: string, options?: HumanActionOptions) => {
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
await sleep(randRange(callCfg.field_switch_delay));
await frameClick(selector, options);
await sleep(rand(100, 250));
const cdp = await getFrameCdp();
await humanType(page, rawKb, text, callCfg, cdp).catch(() => origFrameType(selector, text, options));
};
(frame as any).check = async (selector: string, options?: any) => {
await (page as any).check(selector, options);
(frame as any).fill = async (selector: string, value: string, options?: HumanActionOptions) => {
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
await sleep(randRange(callCfg.field_switch_delay));
await frameClick(selector, options);
await sleep(rand(100, 250));
await originals.keyboardPress(SELECT_ALL);
await sleep(rand(30, 80));
await originals.keyboardPress('Backspace');
await sleep(rand(50, 150));
const cdp = await getFrameCdp();
await humanType(page, rawKb, value, callCfg, cdp).catch(() => origFrameFill(selector, value, options));
};
(frame as any).uncheck = async (selector: string, options?: any) => {
await (page as any).uncheck(selector, options);
(frame as any).check = async (selector: string, options?: HumanActionOptions) => {
const locator = firstFrameLocator(frame, selector);
if (typeof locator.isChecked !== 'function') return origFrameCheck(selector, options);
const checked = await locator.isChecked();
if (!checked) await frameClick(selector, options).catch(() => origFrameCheck(selector, options));
};
(frame as any).selectOption = async (selector: string, values: any, options?: any) => {
await (page as any).hover(selector);
(frame as any).uncheck = async (selector: string, options?: HumanActionOptions) => {
const locator = firstFrameLocator(frame, selector);
if (typeof locator.isChecked !== 'function') return origFrameUncheck(selector, options);
const checked = await locator.isChecked();
if (checked) await frameClick(selector, options).catch(() => origFrameUncheck(selector, options));
};
(frame as any).selectOption = async (selector: string, values: any, options?: HumanActionOptions) => {
await frameHover(selector, options);
await sleep(rand(100, 300));
return origFrameSelectOption(selector, values, options);
};
(frame as any).press = async (selector: string, key: string, options?: any) => {
await (page as any).press(selector, key, options);
(frame as any).press = async (selector: string, key: string, options?: HumanActionOptions) => {
if (!await isFrameSelectorFocused(frame, selector)) {
await frameClick(selector, options);
}
await sleep(rand(50, 150));
await originals.keyboardPress(key);
};
(frame as any).pressSequentially = async (selector: string, text: string, options?: any) => {
await (page as any).pressSequentially(selector, text, options);
(frame as any).pressSequentially = async (selector: string, text: string, options?: HumanActionOptions) => {
const callCfg = mergeConfig(cfg, options?.human_config ?? options);
if (!await isFrameSelectorFocused(frame, selector)) {
await frameClick(selector, options);
}
await sleep(rand(100, 250));
const cdp = await getFrameCdp();
await humanType(page, rawKb, text, callCfg, cdp).catch(() => origFramePressSequentially?.(selector, text, options));
};
(frame as any).tap = async (selector: string, options?: any) => {
await (page as any).tap(selector, options);
(frame as any).tap = async (selector: string, options?: HumanActionOptions) => {
await frameClick(selector, options).catch(() => origFrameTap?.(selector, options));
};
(frame as any).clear = async (selector: string, options?: any) => {
if (!await isSelectorFocused(stealth, page, selector)) {
await (page as any).click(selector);
(frame as any).clear = async (selector: string, options?: HumanActionOptions) => {
if (!await isFrameSelectorFocused(frame, selector)) {
await frameClick(selector, options);
}
await sleep(rand(50, 150));
await originals.keyboardPress(SELECT_ALL);
@@ -605,9 +825,20 @@ function patchSingleFrame(
await originals.keyboardPress('Backspace');
};
(frame as any).dragAndDrop = async (source: string, target: string, options?: any) => {
const srcBox = await frame.locator(source).boundingBox().catch(() => null);
const tgtBox = await frame.locator(target).boundingBox().catch(() => null);
(frame as any).dragAndDrop = async (source: string, target: string, options?: {
force?: boolean;
noWaitAfter?: boolean;
sourcePosition?: { x: number; y: number };
strict?: boolean;
targetPosition?: { x: number; y: number };
timeout?: number;
trial?: boolean;
}) => {
const timeout = options?.timeout ?? 30000;
const deadline = Date.now() + timeout;
const remainingMs = () => Math.max(1, deadline - Date.now());
const srcBox = await firstFrameLocator(frame, source).boundingBox({ timeout: remainingMs() }).catch(() => null);
const tgtBox = await firstFrameLocator(frame, target).boundingBox({ timeout: remainingMs() }).catch(() => null);
if (srcBox && tgtBox) {
const sx = srcBox.x + srcBox.width / 2;
@@ -623,7 +854,7 @@ function patchSingleFrame(
await sleep(rand(80, 150));
await originals.mouseUp();
} else {
return origFrameDragAndDrop(source, target, options);
return origFrameDragAndDrop(source, target, { ...options, timeout: Math.max(1, remainingMs()) });
}
};
}
@@ -676,14 +907,14 @@ export function patchBrowser(browser: Browser, cfg: HumanConfig): void {
}
const origNewContext = browser.newContext.bind(browser);
(browser as any).newContext = async (options?: any) => {
(browser as any).newContext = async (options?: Parameters<typeof origNewContext>[0]) => {
const context = await origNewContext(options);
patchContext(context, cfg);
return context;
};
const origNewPage = browser.newPage.bind(browser);
(browser as any).newPage = async (options?: any) => {
(browser as any).newPage = async (options?: Parameters<typeof origNewPage>[0]) => {
const page = await origNewPage(options);
if (!(page as any)._original) {
const ctx = page.context();
+21 -1
View File
@@ -172,13 +172,33 @@ export async function humanClick(
// Human idle / drift
// ---------------------------------------------------------------------------
export async function humanIdle(
export function humanIdle(
raw: RawMouse,
cx: number,
cy: number,
cfg: HumanConfig,
): Promise<void>;
export function humanIdle(
raw: RawMouse,
seconds: number,
cx: number,
cy: number,
cfg: HumanConfig,
): Promise<void>;
export async function humanIdle(
raw: RawMouse,
secondsOrCx: number,
cxOrCy: number,
cyOrCfg: number | HumanConfig,
maybeCfg?: HumanConfig,
): Promise<void> {
const hasExplicitSeconds = maybeCfg !== undefined;
const seconds = hasExplicitSeconds
? secondsOrCx
: rand((cyOrCfg as HumanConfig).idle_between_duration[0], (cyOrCfg as HumanConfig).idle_between_duration[1]);
const cx = hasExplicitSeconds ? cxOrCy : secondsOrCx;
const cy = hasExplicitSeconds ? (cyOrCfg as number) : cxOrCy;
const cfg = hasExplicitSeconds ? maybeCfg! : (cyOrCfg as HumanConfig);
const endTime = Date.now() + seconds * 1000;
let x = cx;
let y = cy;
+7 -5
View File
@@ -52,7 +52,7 @@ export async function humanScrollIntoView(
cursorX: number,
cursorY: number,
cfg: HumanConfig,
): Promise<{ box: ElementBounds; cursorX: number; cursorY: number }> {
): Promise<{ box: ElementBounds; cursorX: number; cursorY: number; didScroll: boolean }> {
const viewport = page.viewportSize();
if (!viewport) throw new Error('Viewport size not available');
@@ -60,7 +60,7 @@ export async function humanScrollIntoView(
if (!box) throw new Error('Element not found while scrolling into view');
if (isInViewport(box, viewport.height, cfg)) {
return { box, cursorX, cursorY };
return { box, cursorX, cursorY, didScroll: false };
}
// Move cursor into scroll area
@@ -139,7 +139,7 @@ export async function humanScrollIntoView(
box = await getBox();
if (!box) throw new Error('Element lost after scrolling into view');
return { box, cursorX, cursorY };
return { box, cursorX, cursorY, didScroll: true };
}
/**
@@ -148,6 +148,8 @@ export async function humanScrollIntoView(
* ``timeout`` is forwarded to Playwright's ``boundingBox({ timeout })`` so
* callers like ``page.click('#x', { timeout: 5000 })`` can wait longer for
* slow-loading elements (#172). Default matches Playwright's 30000ms when not specified.
*
* Returns `{ box, cursorX, cursorY, didScroll }`.
*/
export async function scrollToElement(
page: Page,
@@ -157,7 +159,7 @@ export async function scrollToElement(
cursorY: number,
cfg: HumanConfig,
timeout?: number,
): Promise<{ box: ElementBounds; cursorX: number; cursorY: number }> {
): Promise<{ box: ElementBounds; cursorX: number; cursorY: number; didScroll: boolean }> {
return humanScrollIntoView(
page, raw,
() => getElementBox(page, selector, timeout),
@@ -172,7 +174,7 @@ async function getElementBox(
): Promise<ElementBounds | null> {
const el = page.locator(selector).first();
try {
const box = await el.boundingBox({ timeout });
const box = await el.boundingBox({ timeout: Math.max(1, timeout) });
return box;
} catch {
return null;
+1 -1
View File
@@ -16,7 +16,7 @@
*/
// Launch functions (Playwright API)
export { launch, launchContext, launchPersistentContext } from "./playwright.js";
export { launch, launchContext, launchPersistentContext, buildLaunchOptions, buildContextOptions, humanizeBrowser } from "./playwright.js";
// Binary management
export { ensureBinary, clearCache, binaryInfo, checkForUpdate } from "./download.js";
+71 -45
View File
@@ -3,7 +3,7 @@
* Mirrors Python cloakbrowser/browser.py.
*/
import type { Browser, BrowserContext, BrowserContextOptions } from "playwright-core";
import type { Browser, BrowserContext, BrowserContextOptions, LaunchOptions as PlaywrightLaunchOptions } from "playwright-core";
import type { LaunchOptions, LaunchContextOptions, LaunchPersistentContextOptions } from "./types.js";
import { DEFAULT_VIEWPORT, IGNORE_DEFAULT_ARGS } from "./config.js";
import { buildArgs } from "./args.js";
@@ -44,6 +44,72 @@ function filterStealthCtxOptions(ctx?: BrowserContextOptions): Partial<BrowserCo
return rest;
}
/**
* Build Playwright BrowserContext options for CloakBrowser without launching a browser
* or creating a context.
*
* Useful when integrating CloakBrowser with an existing Playwright Browser while
* keeping the wrapper's stealth-safe defaults for `newContext()`.
*/
export function buildContextOptions(
options: LaunchContextOptions = {}
): BrowserContextOptions {
return {
// contextOptions first — explicit wrapper fields below override it.
// filterStealthCtxOptions strips locale/timezoneId to prevent CDP detection.
...filterStealthCtxOptions(options.contextOptions),
...(options.userAgent ? { userAgent: options.userAgent } : {}),
viewport: options.viewport === undefined ? DEFAULT_VIEWPORT : options.viewport,
...(options.colorScheme ? { colorScheme: options.colorScheme } : {}),
} as BrowserContextOptions;
}
/**
* Build Playwright launch options for CloakBrowser without starting Chromium.
*
* Useful when integrating CloakBrowser with a custom Playwright build or another
* wrapper that needs to call `chromium.launch()` itself.
*/
export async function buildLaunchOptions(
options: LaunchOptions = {}
): Promise<PlaywrightLaunchOptions> {
const binaryPath = process.env.CLOAKBROWSER_BINARY_PATH || (await ensureBinary());
const { exitIp, ...resolved } = await maybeResolveGeoip(options);
const { proxyOption, proxyArgs } = resolveProxyConfig(options.proxy);
let resolvedArgs = await resolveWebrtcArgs(options);
if (exitIp && !(resolvedArgs ?? []).some(a => a.startsWith("--fingerprint-webrtc-ip"))) {
resolvedArgs = [...(resolvedArgs ?? []), `--fingerprint-webrtc-ip=${exitIp}`];
}
const args = buildArgs({ ...options, ...resolved, args: [...(resolvedArgs ?? []), ...proxyArgs] });
return {
executablePath: binaryPath,
headless: options.headless ?? true,
args,
ignoreDefaultArgs: IGNORE_DEFAULT_ARGS,
...(proxyOption ? { proxy: proxyOption } : {}),
...options.launchOptions,
} as PlaywrightLaunchOptions;
}
/**
* Apply CloakBrowser's human-like behavioral layer to an existing Playwright browser.
*/
export async function humanizeBrowser(
browser: Browser,
options: LaunchOptions = {}
): Promise<void> {
if (!options.humanize) return;
const { patchBrowser } = await import('./human/index.js');
const { resolveConfig } = await import('./human/config.js');
const cfg = resolveConfig(
options.humanPreset ?? 'default',
options.humanConfig,
);
patchBrowser(browser, cfg);
}
/**
* Launch stealth Chromium browser via Playwright.
*
@@ -59,36 +125,8 @@ function filterStealthCtxOptions(ctx?: BrowserContextOptions): Partial<BrowserCo
*/
export async function launch(options: LaunchOptions = {}): Promise<Browser> {
const { chromium } = await import("playwright-core");
const binaryPath = process.env.CLOAKBROWSER_BINARY_PATH || (await ensureBinary());
const { exitIp, ...resolved } = await maybeResolveGeoip(options);
const { proxyOption, proxyArgs } = resolveProxyConfig(options.proxy);
let resolvedArgs = await resolveWebrtcArgs(options);
if (exitIp && !(resolvedArgs ?? []).some(a => a.startsWith("--fingerprint-webrtc-ip"))) {
resolvedArgs = [...(resolvedArgs ?? []), `--fingerprint-webrtc-ip=${exitIp}`];
}
const args = buildArgs({ ...options, ...resolved, args: [...(resolvedArgs ?? []), ...proxyArgs] });
const browser = await chromium.launch({
executablePath: binaryPath,
headless: options.headless ?? true,
args,
ignoreDefaultArgs: IGNORE_DEFAULT_ARGS,
...(proxyOption ? { proxy: proxyOption } : {}),
...options.launchOptions,
});
// Human-like behavioral patching
if (options.humanize) {
const { patchBrowser } = await import('./human/index.js');
const { resolveConfig } = await import('./human/config.js');
const cfg = resolveConfig(
options.humanPreset ?? 'default',
options.humanConfig,
);
patchBrowser(browser, cfg);
}
const browser = await chromium.launch(await buildLaunchOptions(options));
await humanizeBrowser(browser, options);
return browser;
}
@@ -126,14 +164,7 @@ export async function launchContext(
let context: BrowserContext;
try {
context = await browser.newContext({
// contextOptions first — explicit wrapper fields below override it.
// filterStealthCtxOptions strips locale/timezoneId to prevent CDP detection.
...filterStealthCtxOptions(options.contextOptions),
...(options.userAgent ? { userAgent: options.userAgent } : {}),
viewport: options.viewport === undefined ? DEFAULT_VIEWPORT : options.viewport,
...(options.colorScheme ? { colorScheme: options.colorScheme } : {}),
});
context = await browser.newContext(buildContextOptions(options));
} catch (err) {
await browser.close();
throw err;
@@ -204,12 +235,7 @@ export async function launchPersistentContext(
args,
ignoreDefaultArgs: IGNORE_DEFAULT_ARGS,
...(proxyOption ? { proxy: proxyOption } : {}),
// contextOptions before explicit wrapper fields so explicit wins.
// filterStealthCtxOptions strips locale/timezoneId to prevent CDP detection.
...filterStealthCtxOptions(options.contextOptions),
...(options.userAgent ? { userAgent: options.userAgent } : {}),
viewport: options.viewport === undefined ? DEFAULT_VIEWPORT : options.viewport,
...(options.colorScheme ? { colorScheme: options.colorScheme } : {}),
...buildContextOptions(options),
...options.launchOptions,
});
+125 -4
View File
@@ -2,6 +2,8 @@
* Shared proxy URL parsing for Playwright and Puppeteer wrappers.
*/
import { getChromiumVersion, getPlatformTag, parseVersion } from "./config.js";
export interface ParsedProxy {
server: string;
username?: string;
@@ -136,18 +138,125 @@ export function normalizeSocksStringUrl(urlStr: string): string {
const encPass = hasPassword
? (rawPassEnc ? encodeURIComponent(lenientDecodeURIComponent(rawPassEnc)) : "")
: null;
return assembleSocksUrl(scheme, encUser, encPass, hostAndRest);
const normalized = assembleSocksUrl(scheme, encUser, encPass, hostAndRest);
// Compare credentials, not the full URL: keeps the log condition focused
// on real encoding work, not cosmetic differences (parity with the Python
// implementation, which has to skip urlparse's hostname lowercasing).
const credsChanged = encUser !== rawUserEnc
|| (hasPassword ? encPass !== rawPassEnc : false);
if (credsChanged) {
console.info(
"[cloakbrowser] Auto URL-encoded SOCKS5 proxy credentials (special " +
"characters detected). Pre-encode the URL to suppress this notice.",
);
}
return normalized;
} catch (e) {
console.warn(`[cloakbrowser] Could not normalize SOCKS5 proxy URL, passing through unchanged: ${(e as Error).message}`);
return urlStr;
}
}
const HTTP_PROXY_INLINE_AUTH_MIN_VERSION = "146.0.7680.177.5";
const HTTP_PROXY_INLINE_AUTH_PLATFORMS = new Set(["linux-x64", "windows-x64"]);
export function supportsHttpProxyInlineAuth(): boolean {
try {
const tag = getPlatformTag();
if (!HTTP_PROXY_INLINE_AUTH_PLATFORMS.has(tag)) return false;
const current = parseVersion(getChromiumVersion());
const minimum = parseVersion(HTTP_PROXY_INLINE_AUTH_MIN_VERSION);
for (let i = 0; i < Math.max(current.length, minimum.length); i++) {
if ((current[i] ?? 0) > (minimum[i] ?? 0)) return true;
if ((current[i] ?? 0) < (minimum[i] ?? 0)) return false;
}
return true; // equal = supported
} catch {
return false;
}
}
function hasCredentials(proxy: string | ProxyDict): boolean {
if (typeof proxy === "string") return proxy.includes("@");
return !!proxy.username;
}
/**
* Reconstruct an HTTP(S) proxy URL with inline credentials from a proxy dict.
*/
export function reconstructHttpUrl(proxy: ProxyDict): string {
if (!proxy.username) return proxy.server;
const url = new URL(ensureProxyScheme(proxy.server));
url.username = encodeURIComponent(proxy.username);
if (proxy.password) url.password = encodeURIComponent(proxy.password);
return url.href.replace(/\/$/, "");
}
/**
* Re-encode credentials in an HTTP(S) proxy URL string for --proxy-server.
* Same pattern as normalizeSocksStringUrl.
*/
export function normalizeHttpStringUrl(urlStr: string): string {
const normalized = urlStr.includes("://") ? urlStr : `http://${urlStr}`;
const schemeMatch = normalized.match(/^([a-z][a-z0-9+\-.]*):\/\/(.*)$/i);
if (!schemeMatch) return normalized;
const [, scheme, rest] = schemeMatch;
const hostStart = rest.search(/[/?#]/);
const authority = hostStart === -1 ? rest : rest.slice(0, hostStart);
const suffix = hostStart === -1 ? "" : rest.slice(hostStart);
const atIdx = authority.lastIndexOf("@");
if (atIdx === -1) return normalized;
const userinfo = authority.slice(0, atIdx);
const hostPart = authority.slice(atIdx + 1);
const bracketEnd = hostPart.lastIndexOf("]");
const portColonIdx = hostPart.indexOf(":", Math.max(bracketEnd, 0));
if (portColonIdx !== -1) {
const portStr = hostPart.slice(portColonIdx + 1);
if (portStr && !/^\d+$/.test(portStr)) {
console.warn(`[cloakbrowser] Malformed HTTP proxy URL, passing through unchanged: invalid port`);
return normalized;
}
}
const hostAndRest = hostPart + suffix;
const colonIdx = userinfo.indexOf(":");
const rawUserEnc = colonIdx === -1 ? userinfo : userinfo.slice(0, colonIdx);
const hasPassword = colonIdx !== -1;
const rawPassEnc = hasPassword ? userinfo.slice(colonIdx + 1) : "";
try {
const encUser = rawUserEnc ? encodeURIComponent(lenientDecodeURIComponent(rawUserEnc)) : "";
const encPass = hasPassword
? (rawPassEnc ? encodeURIComponent(lenientDecodeURIComponent(rawPassEnc)) : "")
: null;
let userinfoPart: string;
if (encPass !== null) {
userinfoPart = `${encUser}:${encPass}@`;
} else if (encUser) {
userinfoPart = `${encUser}@`;
} else {
userinfoPart = "";
}
const result = `${scheme}://${userinfoPart}${hostAndRest}`;
const credsChanged = encUser !== rawUserEnc
|| (hasPassword ? encPass !== rawPassEnc : false);
if (credsChanged) {
console.info(
"[cloakbrowser] Auto URL-encoded HTTP proxy credentials (special " +
"characters detected). Pre-encode the URL to suppress this notice.",
);
}
return result;
} catch (e) {
console.warn(`[cloakbrowser] Could not normalize HTTP proxy URL, passing through unchanged: ${(e as Error).message}`);
return normalized;
}
}
/**
* Resolve proxy into Playwright option and/or Chrome args.
*
* Playwright rejects SOCKS5 proxies with credentials in its proxy dict,
* so SOCKS5 is passed via --proxy-server Chrome arg instead.
* Proxies with credentials (SOCKS5 or HTTP/HTTPS on supported platforms) are
* passed via Chrome's --proxy-server flag with inline credentials, bypassing
* Playwright's CDP auth interceptor which breaks on some proxies (#182).
*/
export function resolveProxyConfig(proxy: string | ProxyDict | undefined): ProxyConfig {
if (!proxy) return { proxyArgs: [] };
@@ -165,7 +274,19 @@ export function resolveProxyConfig(proxy: string | ProxyDict | undefined): Proxy
return { proxyArgs: args };
}
// HTTP/HTTPS: use Playwright's proxy dict
// HTTP/HTTPS with credentials on supported platforms: bypass Playwright's
// CDP auth interceptor, use Chrome's preemptive Proxy-Authorization (#182).
if (hasCredentials(proxy) && supportsHttpProxyInlineAuth()) {
if (typeof proxy === "string") {
return { proxyArgs: [`--proxy-server=${normalizeHttpStringUrl(proxy)}`] };
}
const httpUrl = reconstructHttpUrl(proxy);
const args = [`--proxy-server=${httpUrl}`];
if (proxy.bypass) args.push(`--proxy-bypass-list=${proxy.bypass}`);
return { proxyArgs: args };
}
// HTTP/HTTPS without credentials (or unsupported platform): use Playwright's proxy dict
if (typeof proxy === "string") {
return { proxyOption: parseProxyUrl(proxy), proxyArgs: [] };
}
+124 -59
View File
@@ -9,74 +9,75 @@ import type { LaunchOptions } from "./types.js";
import { IGNORE_DEFAULT_ARGS } from "./config.js";
import { buildArgs } from "./args.js";
import { ensureBinary } from "./download.js";
import { isSocksProxy, parseProxyUrl, resolveProxyConfig } from "./proxy.js";
import { isSocksProxy, normalizeHttpStringUrl, parseProxyUrl, reconstructHttpUrl, resolveProxyConfig, supportsHttpProxyInlineAuth } from "./proxy.js";
import { maybeResolveGeoip, resolveWebrtcArgs } from "./geoip.js";
/**
* Launch stealth Chromium browser via Puppeteer.
*
* @example
* ```ts
* import { launch } from 'cloakbrowser/puppeteer';
* * // With humanize — human-like mouse, keyboard, scroll
* const browser = await launch({ humanize: true });
* const page = await browser.newPage();
* await page.goto('[https://example.com](https://example.com)');
* await page.click('#login'); // Bézier curve mouse movement
* await page.type('#email', 'user@example.com'); // Per-character timing
* ```
*/
export async function launch(options: LaunchOptions = {}): Promise<Browser> {
const puppeteer = await import("puppeteer-core");
/** Resolve binary path, geoip, webrtc, and build final Chrome args. */
async function resolveArgs(options: LaunchOptions): Promise<{ binaryPath: string; args: string[] }> {
const binaryPath = process.env.CLOAKBROWSER_BINARY_PATH || (await ensureBinary());
const { exitIp, ...resolved } = (await maybeResolveGeoip(options)) ?? {};
let resolvedArgs = (await resolveWebrtcArgs(options)) ?? options.args;
if (exitIp && !(resolvedArgs ?? []).some(a => a.startsWith("--fingerprint-webrtc-ip"))) {
resolvedArgs = [...(resolvedArgs ?? []), `--fingerprint-webrtc-ip=${exitIp}`];
}
const args = buildArgs({ ...options, ...resolved, args: resolvedArgs });
return { binaryPath, args: buildArgs({ ...options, ...resolved, args: resolvedArgs }) };
}
// Puppeteer handles proxy via CLI args, not a separate option.
// SOCKS5: Chrome supports inline credentials natively (RFC 1929 auth).
// HTTP: Chrome does NOT support inline credentials — strip them and
// use page.authenticate() for Proxy-Authorization headers instead.
let proxyAuth: { username: string; password: string } | undefined;
if (options.proxy) {
if (isSocksProxy(options.proxy)) {
// SOCKS5: pass full URL with credentials to Chrome directly
const { proxyArgs } = resolveProxyConfig(options.proxy);
args.push(...proxyArgs);
} else if (typeof options.proxy === "string") {
const { server, username, password } = parseProxyUrl(options.proxy);
args.push(`--proxy-server=${server}`);
if (username) {
proxyAuth = { username, password: password ?? "" };
}
} else {
const parsed = parseProxyUrl(options.proxy.server);
args.push(`--proxy-server=${parsed.server}`);
if (options.proxy.bypass) {
args.push(`--proxy-bypass-list=${options.proxy.bypass}`);
}
const username = options.proxy.username ?? parsed.username;
const password = options.proxy.password ?? parsed.password;
if (username) {
proxyAuth = { username, password: password ?? "" };
}
}
/**
* Resolve proxy into Chrome CLI args and optional HTTP auth credentials.
* SOCKS5: Chrome handles inline credentials natively (RFC 1929 auth).
* HTTP on supported platforms: inline credentials via --proxy-server.
* HTTP on unsupported platforms: strip credentials, use page.authenticate() fallback.
*/
function resolveProxy(options: LaunchOptions, args: string[]): { username: string; password: string } | undefined {
if (!options.proxy) return undefined;
if (isSocksProxy(options.proxy)) {
const { proxyArgs } = resolveProxyConfig(options.proxy);
args.push(...proxyArgs);
return undefined;
}
const browser = await puppeteer.default.launch({
executablePath: binaryPath,
headless: options.headless ?? true,
args,
ignoreDefaultArgs: IGNORE_DEFAULT_ARGS,
...options.launchOptions,
});
// On supported platforms: pass full URL with inline creds to --proxy-server
if (supportsHttpProxyInlineAuth()) {
if (typeof options.proxy === "string") {
args.push(`--proxy-server=${normalizeHttpStringUrl(options.proxy)}`);
return undefined;
}
const url = options.proxy.username
? reconstructHttpUrl(options.proxy)
: options.proxy.server;
args.push(`--proxy-server=${url}`);
if (options.proxy.bypass) {
args.push(`--proxy-bypass-list=${options.proxy.bypass}`);
}
return undefined;
}
// Monkey-patch newPage() to auto-authenticate proxy credentials
// Unsupported platform: strip credentials, fall back to page.authenticate()
if (typeof options.proxy === "string") {
const { server, username, password } = parseProxyUrl(options.proxy);
args.push(`--proxy-server=${server}`);
return username ? { username, password: password ?? "" } : undefined;
}
const parsed = parseProxyUrl(options.proxy.server);
args.push(`--proxy-server=${parsed.server}`);
if (options.proxy.bypass) {
args.push(`--proxy-bypass-list=${options.proxy.bypass}`);
}
const username = options.proxy.username ?? parsed.username;
const password = options.proxy.password ?? parsed.password;
return username ? { username, password: password ?? "" } : undefined;
}
/** Apply proxy auth fallback (unsupported platforms) and humanize patching. */
async function applyPostLaunch(
browser: Browser,
options: LaunchOptions,
proxyAuth?: { username: string; password: string },
): Promise<void> {
if (proxyAuth) {
const origNewPage = browser.newPage.bind(browser);
const auth = proxyAuth;
@@ -87,9 +88,6 @@ export async function launch(options: LaunchOptions = {}): Promise<Browser> {
};
}
// Human-like behavioral patching — FULL coverage, same as Playwright.
// This enables Bézier mouse movements, organic typing rhythms, and
// natural scrolling to bypass advanced anti-bot detection.
if (options.humanize) {
const { patchBrowser } = await import('./human-puppeteer/index.js');
const { resolveConfig } = await import('./human/config.js');
@@ -99,6 +97,73 @@ export async function launch(options: LaunchOptions = {}): Promise<Browser> {
);
patchBrowser(browser, cfg);
}
}
/**
* Launch stealth Chromium browser via Puppeteer.
*
* @example
* ```ts
* import { launch } from 'cloakbrowser/puppeteer';
* // With humanize — human-like mouse, keyboard, scroll
* const browser = await launch({ humanize: true });
* const page = await browser.newPage();
* await page.goto('https://example.com');
* await page.click('#login'); // Bézier curve mouse movement
* await page.type('#email', 'user@example.com'); // Per-character timing
* ```
*/
export async function launch(options: LaunchOptions = {}): Promise<Browser> {
const puppeteer = await import("puppeteer-core");
const { binaryPath, args } = await resolveArgs(options);
const proxyAuth = resolveProxy(options, args);
const browser = await puppeteer.default.launch({
...options.launchOptions,
executablePath: binaryPath,
headless: options.headless ?? true,
args,
ignoreDefaultArgs: IGNORE_DEFAULT_ARGS,
});
await applyPostLaunch(browser, options, proxyAuth);
return browser;
}
/**
* Launch stealth Chromium with a persistent user profile via Puppeteer.
* Passes `userDataDir` to Puppeteer's launch options so cookies,
* localStorage, and session data persist across launches.
*
* @example
* ```ts
* import { launchPersistentContext } from 'cloakbrowser/puppeteer';
* const browser = await launchPersistentContext({
* userDataDir: './chrome-profile',
* headless: false,
* proxy: 'http://user:pass@proxy:8080',
* });
* const page = await browser.newPage();
* await page.goto('https://example.com');
* await browser.close();
* ```
*/
export async function launchPersistentContext(
options: LaunchOptions & { userDataDir: string }
): Promise<Browser> {
const puppeteer = await import("puppeteer-core");
const { binaryPath, args } = await resolveArgs(options);
const proxyAuth = resolveProxy(options, args);
const browser = await puppeteer.default.launch({
...options.launchOptions,
executablePath: binaryPath,
headless: options.headless ?? true,
args,
ignoreDefaultArgs: IGNORE_DEFAULT_ARGS,
userDataDir: options.userDataDir,
});
await applyPostLaunch(browser, options, proxyAuth);
return browser;
}
+2
View File
@@ -17,6 +17,8 @@ export interface LaunchOptions {
proxy?: string | { server: string; bypass?: string; username?: string; password?: string };
/** Additional Chromium CLI arguments. */
args?: string[];
/** Chrome extension paths to load. */
extensionPaths?: string[];
/** Include default stealth fingerprint args (default: true). Set false to use custom --fingerprint flags. */
stealthArgs?: boolean;
/** IANA timezone, e.g. "America/New_York". Sets --fingerprint-timezone binary flag. */
+17
View File
@@ -0,0 +1,17 @@
import { test, expect } from "vitest";
import path from "path";
import { _buildArgsForTest } from "../src/playwright.js";
test("extension paths inject chrome flags", () => {
const args = _buildArgsForTest({
extensionPaths: ["./ext"],
});
const abs = path.resolve("./ext");
expect(args).toContain(`--load-extension=${abs}`);
expect(args).toContain(
`--disable-extensions-except=${abs}`
);
});
+59 -2
View File
@@ -1,5 +1,17 @@
import { describe, it, expect } from "vitest";
import { COUNTRY_LOCALE_MAP, resolveProxyIp } from "../src/geoip.js";
import { describe, it, expect, afterEach, vi } from "vitest";
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
import { COUNTRY_LOCALE_MAP, maybeResolveGeoip, resolveProxyGeo, resolveProxyIp } from "../src/geoip.js";
const tempDirs: string[] = [];
afterEach(() => {
vi.restoreAllMocks();
delete process.env.CLOAKBROWSER_GEOIP_TIMEOUT_SECONDS;
delete process.env.CLOAKBROWSER_CACHE_DIR;
for (const dir of tempDirs.splice(0)) fs.rmSync(dir, { recursive: true, force: true });
});
describe("resolveProxyIp", () => {
it("returns literal IPv4 from proxy URL", async () => {
@@ -38,6 +50,51 @@ describe("resolveProxyIp", () => {
});
});
describe("maybeResolveGeoip", () => {
it("does not apply the GeoIP resolution timeout to first-use database download", async () => {
const cacheDir = fs.mkdtempSync(path.join(os.tmpdir(), "cloak-geoip-download-"));
tempDirs.push(cacheDir);
process.env.CLOAKBROWSER_CACHE_DIR = cacheDir;
process.env.CLOAKBROWSER_GEOIP_TIMEOUT_SECONDS = "0.001";
const fetchSpy = vi.spyOn(globalThis, "fetch").mockResolvedValue({
ok: true,
body: new ReadableStream({
start(controller) {
controller.enqueue(new Uint8Array([1, 2, 3]));
controller.close();
},
}),
} as Response);
const result = await resolveProxyGeo("http://203.0.113.10:8080");
expect(result).toEqual({ timezone: null, locale: null, exitIp: null });
expect(fetchSpy).toHaveBeenCalledOnce();
expect(fetchSpy.mock.calls[0][1]).toEqual({ redirect: "follow" });
});
it("returns quickly when GeoIP resolution times out", async () => {
const cacheDir = fs.mkdtempSync(path.join(os.tmpdir(), "cloak-geoip-timeout-"));
tempDirs.push(cacheDir);
process.env.CLOAKBROWSER_CACHE_DIR = cacheDir;
process.env.CLOAKBROWSER_GEOIP_TIMEOUT_SECONDS = "0.025";
const start = performance.now();
const result = await maybeResolveGeoip({
geoip: true,
proxy: "http://203.0.113.10:8080",
timezone: "Europe/Paris",
locale: "fr-FR",
});
const elapsed = performance.now() - start;
expect(result).toEqual({ timezone: "Europe/Paris", locale: "fr-FR", exitIp: undefined });
expect(elapsed).toBeLessThan(500);
});
});
describe("COUNTRY_LOCALE_MAP", () => {
it("contains common countries", () => {
for (const code of ["US", "GB", "DE", "FR", "JP", "BR", "IL", "RU"]) {
+441 -217
View File
File diff suppressed because it is too large Load Diff
+123 -9
View File
@@ -1,6 +1,7 @@
import { describe, it, expect, vi, afterEach, beforeEach } from "vitest";
import { binaryInfo } from "../src/download.js";
import { DEFAULT_VIEWPORT, getChromiumVersion } from "../src/config.js";
import * as config from "../src/config.js";
describe("binaryInfo", () => {
it("returns correct structure", () => {
@@ -21,6 +22,113 @@ describe("binaryInfo", () => {
});
});
describe("composable Playwright launch helpers", () => {
const origBinaryPath = process.env.CLOAKBROWSER_BINARY_PATH;
beforeEach(() => {
process.env.CLOAKBROWSER_BINARY_PATH = "/fake/chrome";
vi.resetModules();
});
afterEach(() => {
vi.restoreAllMocks();
vi.resetModules();
if (origBinaryPath) {
process.env.CLOAKBROWSER_BINARY_PATH = origBinaryPath;
} else {
delete process.env.CLOAKBROWSER_BINARY_PATH;
}
});
it("exports composable helpers from the package entrypoint", async () => {
const entry = await import("../src/index.js");
expect(entry.buildLaunchOptions).toBeTypeOf("function");
expect(entry.buildContextOptions).toBeTypeOf("function");
expect(entry.humanizeBrowser).toBeTypeOf("function");
});
it("buildContextOptions returns Playwright context options without launching a browser", async () => {
const warnSpy = vi.spyOn(console, "warn").mockImplementation(() => {});
const { buildContextOptions } = await import("../src/index.js");
const options = buildContextOptions({
userAgent: "Explicit/1.0",
viewport: { width: 1280, height: 720 },
colorScheme: "dark",
contextOptions: {
userAgent: "Context/9.9",
viewport: { width: 9999, height: 9999 },
colorScheme: "light",
storageState: "state.json",
locale: "de-DE",
timezoneId: "Europe/Berlin",
},
});
expect(options).toMatchObject({
userAgent: "Explicit/1.0",
viewport: { width: 1280, height: 720 },
colorScheme: "dark",
storageState: "state.json",
});
expect(options.locale).toBeUndefined();
expect(options.timezoneId).toBeUndefined();
expect(warnSpy).toHaveBeenCalledTimes(2);
});
it("buildContextOptions applies DEFAULT_VIEWPORT by default and allows null viewport", async () => {
const { buildContextOptions } = await import("../src/index.js");
expect(buildContextOptions().viewport).toEqual(DEFAULT_VIEWPORT);
expect(buildContextOptions({ viewport: null }).viewport).toBeNull();
});
it("buildLaunchOptions returns Playwright options without launching a browser", async () => {
const freshConfig = await import("../src/config.js");
vi.spyOn(freshConfig, "getPlatformTag").mockReturnValue("darwin-arm64");
try {
const { buildLaunchOptions } = await import("../src/index.js");
const options = await buildLaunchOptions({
headless: false,
proxy: "http://user:pass@proxy.example:8080",
args: ["--custom-flag"],
launchOptions: { timeout: 1234 },
});
expect(options.executablePath).toBe("/fake/chrome");
expect(options.headless).toBe(false);
expect(options.args).toContain("--custom-flag");
expect(options.ignoreDefaultArgs).toContain("--enable-automation");
expect(options.proxy).toEqual({
server: "http://proxy.example:8080",
username: "user",
password: "pass",
});
expect(options.timeout).toBe(1234);
} finally {
vi.restoreAllMocks();
}
});
it("humanizeBrowser patches an existing browser only when requested", async () => {
const { humanizeBrowser } = await import("../src/index.js");
const browser = {
contexts: () => [],
newContext: vi.fn(async () => ({})),
newPage: vi.fn(async () => ({ context: () => ({}) })),
};
const originalNewContext = browser.newContext;
await humanizeBrowser(browser as any, { humanize: false });
expect(browser.newContext).toBe(originalNewContext);
await humanizeBrowser(browser as any, { humanize: true });
expect(browser.newContext).not.toBe(originalNewContext);
});
});
// Integration tests require the binary — run with:
// CLOAKBROWSER_BINARY_PATH=/path/to/chrome npm test
describe.skipIf(!process.env.CLOAKBROWSER_BINARY_PATH)(
@@ -243,16 +351,22 @@ describe("launchPersistentContext (unit)", () => {
});
it("forwards proxy string", async () => {
const { launchPersistentContext } = await import("../src/playwright.js");
await launchPersistentContext({
userDataDir: "/tmp/profile",
proxy: "http://user:pass@proxy:8080",
});
const freshConfig = await import("../src/config.js");
vi.spyOn(freshConfig, "getPlatformTag").mockReturnValue("darwin-arm64");
try {
const { launchPersistentContext } = await import("../src/playwright.js");
await launchPersistentContext({
userDataDir: "/tmp/profile",
proxy: "http://user:pass@proxy:8080",
});
const args = mockChromium.launchPersistentContext.mock.calls[0][1];
expect(args.proxy.server).toBe("http://proxy:8080");
expect(args.proxy.username).toBe("user");
expect(args.proxy.password).toBe("pass");
const args = mockChromium.launchPersistentContext.mock.calls[0][1];
expect(args.proxy.server).toBe("http://proxy:8080");
expect(args.proxy.username).toBe("user");
expect(args.proxy.password).toBe("pass");
} finally {
vi.restoreAllMocks();
}
});
it("forwards userAgent and colorScheme", async () => {
+163 -6
View File
@@ -1,5 +1,6 @@
import { describe, it, expect } from "vitest";
import { parseProxyUrl, isSocksProxy, resolveProxyConfig } from "../src/proxy.js";
import { describe, it, expect, vi } from "vitest";
import { parseProxyUrl, isSocksProxy, resolveProxyConfig, reconstructHttpUrl, normalizeHttpStringUrl } from "../src/proxy.js";
import * as config from "../src/config.js";
import type { LaunchOptions } from "../src/types.js";
describe("parseProxyUrl", () => {
@@ -153,10 +154,15 @@ describe("resolveProxyConfig", () => {
expect(proxyArgs).toEqual([]);
});
it("returns playwright dict for http string", () => {
const { proxyOption, proxyArgs } = resolveProxyConfig("http://user:pass@proxy:8080");
expect(proxyOption).toEqual({ server: "http://proxy:8080", username: "user", password: "pass" });
expect(proxyArgs).toEqual([]);
it("returns playwright dict for http string on unsupported platform", () => {
vi.spyOn(config, "getPlatformTag").mockReturnValue("darwin-arm64");
try {
const { proxyOption, proxyArgs } = resolveProxyConfig("http://user:pass@proxy:8080");
expect(proxyOption).toEqual({ server: "http://proxy:8080", username: "user", password: "pass" });
expect(proxyArgs).toEqual([]);
} finally {
vi.restoreAllMocks();
}
});
it("returns playwright dict for http dict", () => {
@@ -262,4 +268,155 @@ describe("resolveProxyConfig", () => {
const { proxyArgs } = resolveProxyConfig("socks5://user:a@b@c@host:1080");
expect(proxyArgs).toEqual(["--proxy-server=socks5://user:a%40b%40c@host:1080"]);
});
// Visibility for #157: when wrapper actually rewrites the URL, surface an
// info log so users debugging silent SOCKS5 fallback can see what happened.
it("logs info message when SOCKS5 credentials get re-encoded", () => {
const debugSpy = vi.spyOn(console, "info").mockImplementation(() => {});
try {
resolveProxyConfig("socks5://user:pass=123@host:1080");
expect(debugSpy).toHaveBeenCalledWith(
expect.stringContaining("Auto URL-encoded SOCKS5"),
);
// Credentials must not leak into the log.
const calls = debugSpy.mock.calls.flat().join(" ");
expect(calls).not.toContain("pass=123");
expect(calls).not.toContain("pass%3D123");
} finally {
debugSpy.mockRestore();
}
});
it("stays silent when SOCKS5 URL is already encoded (no log spam)", () => {
const debugSpy = vi.spyOn(console, "info").mockImplementation(() => {});
try {
resolveProxyConfig("socks5://user:pass%3D123@host:1080");
const reencodedCalls = debugSpy.mock.calls
.flat()
.filter((arg) => typeof arg === "string" && arg.includes("Auto URL-encoded SOCKS5"));
expect(reencodedCalls).toHaveLength(0);
} finally {
debugSpy.mockRestore();
}
});
it("stays silent when SOCKS5 URL has no credentials", () => {
const debugSpy = vi.spyOn(console, "info").mockImplementation(() => {});
try {
resolveProxyConfig("socks5://host:1080");
const reencodedCalls = debugSpy.mock.calls
.flat()
.filter((arg) => typeof arg === "string" && arg.includes("Auto URL-encoded SOCKS5"));
expect(reencodedCalls).toHaveLength(0);
} finally {
debugSpy.mockRestore();
}
});
it("stays silent when only host case differs (no credential rewrite)", () => {
// Parity with Python: log condition must track credential changes, not
// cosmetic URL-string differences (regression for Copilot's PR #209 review).
const debugSpy = vi.spyOn(console, "info").mockImplementation(() => {});
try {
resolveProxyConfig("socks5://USER:pass@HOST.com:1080");
const reencodedCalls = debugSpy.mock.calls
.flat()
.filter((arg) => typeof arg === "string" && arg.includes("Auto URL-encoded SOCKS5"));
expect(reencodedCalls).toHaveLength(0);
} finally {
debugSpy.mockRestore();
}
});
// --- HTTP with credentials → --proxy-server (supported platform + version) ---
it("routes http string with creds through --proxy-server on linux-x64 v177.5", () => {
vi.spyOn(config, "getPlatformTag").mockReturnValue("linux-x64");
vi.spyOn(config, "getChromiumVersion").mockReturnValue("146.0.7680.177.5");
try {
const { proxyOption, proxyArgs } = resolveProxyConfig("http://user:pass@proxy:8080");
expect(proxyOption).toBeUndefined();
expect(proxyArgs).toEqual(["--proxy-server=http://user:pass@proxy:8080"]);
} finally {
vi.restoreAllMocks();
}
});
it("routes http dict with creds through --proxy-server on linux-x64 v177.5", () => {
vi.spyOn(config, "getPlatformTag").mockReturnValue("linux-x64");
vi.spyOn(config, "getChromiumVersion").mockReturnValue("146.0.7680.177.5");
try {
const { proxyOption, proxyArgs } = resolveProxyConfig({
server: "http://proxy:8080",
username: "user",
password: "pass",
});
expect(proxyOption).toBeUndefined();
expect(proxyArgs).toEqual(["--proxy-server=http://user:pass@proxy:8080"]);
} finally {
vi.restoreAllMocks();
}
});
it("includes bypass for http dict with creds on windows-x64 v177.5", () => {
vi.spyOn(config, "getPlatformTag").mockReturnValue("windows-x64");
vi.spyOn(config, "getChromiumVersion").mockReturnValue("146.0.7680.177.5");
try {
const { proxyArgs } = resolveProxyConfig({
server: "http://proxy:8080",
username: "user",
password: "pass",
bypass: ".google.com",
});
expect(proxyArgs).toContain("--proxy-server=http://user:pass@proxy:8080");
expect(proxyArgs).toContain("--proxy-bypass-list=.google.com");
} finally {
vi.restoreAllMocks();
}
});
it("encodes special chars in http proxy password on supported platform v177.5", () => {
vi.spyOn(config, "getPlatformTag").mockReturnValue("linux-x64");
vi.spyOn(config, "getChromiumVersion").mockReturnValue("146.0.7680.177.5");
try {
const { proxyArgs } = resolveProxyConfig("http://user:pass=123@proxy:8080");
expect(proxyArgs).toEqual(["--proxy-server=http://user:pass%3D123@proxy:8080"]);
} finally {
vi.restoreAllMocks();
}
});
it("falls back on linux-x64 with old version (pre-inline-auth)", () => {
vi.spyOn(config, "getPlatformTag").mockReturnValue("linux-x64");
vi.spyOn(config, "getChromiumVersion").mockReturnValue("146.0.7680.177.3");
try {
const { proxyOption, proxyArgs } = resolveProxyConfig("http://user:pass@proxy:8080");
expect(proxyOption).toBeDefined();
expect(proxyArgs).toEqual([]);
} finally {
vi.restoreAllMocks();
}
});
it("falls back to playwright dict for http with creds on darwin-arm64", () => {
vi.spyOn(config, "getPlatformTag").mockReturnValue("darwin-arm64");
try {
const { proxyOption, proxyArgs } = resolveProxyConfig("http://user:pass@proxy:8080");
expect(proxyOption).toEqual({ server: "http://proxy:8080", username: "user", password: "pass" });
expect(proxyArgs).toEqual([]);
} finally {
vi.restoreAllMocks();
}
});
it("falls back to playwright dict for http with creds on linux-arm64", () => {
vi.spyOn(config, "getPlatformTag").mockReturnValue("linux-arm64");
try {
const { proxyOption, proxyArgs } = resolveProxyConfig("http://user:pass@proxy:8080");
expect(proxyOption).toBeDefined();
expect(proxyArgs).toEqual([]);
} finally {
vi.restoreAllMocks();
}
});
});
+138 -9
View File
@@ -84,16 +84,39 @@ describe("puppeteer launch", () => {
expect(callArgs.args).toContain("--proxy-bypass-list=.google.com,localhost");
});
it("monkey-patches newPage for proxy auth", async () => {
const { launch } = await import("../src/puppeteer.js");
const browser = await launch({ proxy: "http://user:pass@proxy:8080" });
it("uses page.authenticate fallback for http proxy on unsupported platform", async () => {
const config = await import("../src/config.js");
vi.spyOn(config, "getPlatformTag").mockReturnValue("darwin-arm64");
try {
const { launch } = await import("../src/puppeteer.js");
const browser = await launch({ proxy: "http://user:pass@proxy:8080" });
// newPage should auto-authenticate
const page = await browser.newPage();
expect(page.authenticate).toHaveBeenCalledWith({
username: "user",
password: "pass",
});
const page = await browser.newPage();
expect(page.authenticate).toHaveBeenCalledWith({
username: "user",
password: "pass",
});
} finally {
vi.restoreAllMocks();
}
});
it("passes inline creds via --proxy-server on supported platform (no page.authenticate)", async () => {
const config = await import("../src/config.js");
vi.spyOn(config, "getPlatformTag").mockReturnValue("linux-x64");
vi.spyOn(config, "getChromiumVersion").mockReturnValue("146.0.7680.177.5");
try {
const { launch } = await import("../src/puppeteer.js");
const browser = await launch({ proxy: "http://user:pass@proxy:8080" });
const callArgs = vi.mocked(puppeteerMock.default.launch).mock.calls[0][0];
expect(callArgs.args).toContain("--proxy-server=http://user:pass@proxy:8080");
const page = await browser.newPage();
expect(page.authenticate).not.toHaveBeenCalled();
} finally {
vi.restoreAllMocks();
}
});
it("injects timezone and locale as binary flags", async () => {
@@ -126,6 +149,14 @@ describe("puppeteer launch", () => {
expect(page.authenticate).not.toHaveBeenCalled();
});
it("forwards launchOptions to puppeteer launch", async () => {
const { launch } = await import("../src/puppeteer.js");
await launch({ launchOptions: { slowMo: 50 } });
const callArgs = vi.mocked(puppeteerMock.default.launch).mock.calls[0][0];
expect(callArgs.slowMo).toBe(50);
});
it("reconstructs SOCKS5 dict with auth into --proxy-server URL", async () => {
const { launch } = await import("../src/puppeteer.js");
const browser = await launch({
@@ -139,3 +170,101 @@ describe("puppeteer launch", () => {
expect(page.authenticate).not.toHaveBeenCalled();
});
});
describe("puppeteer launchPersistentContext", () => {
let puppeteerMock: any;
let mockBrowser: any;
beforeEach(async () => {
delete process.env.CLOAKBROWSER_BINARY_PATH;
puppeteerMock = await import("puppeteer-core");
mockBrowser = {
newPage: vi.fn().mockResolvedValue({
authenticate: vi.fn(),
}),
close: vi.fn(),
};
vi.mocked(puppeteerMock.default.launch).mockResolvedValue(mockBrowser);
});
afterEach(() => {
vi.restoreAllMocks();
});
it("passes userDataDir to puppeteer launch", async () => {
process.env.CLOAKBROWSER_BINARY_PATH = "/fake/chrome";
const { launchPersistentContext } = await import("../src/puppeteer.js");
await launchPersistentContext({ userDataDir: "./my-profile" });
expect(puppeteerMock.default.launch).toHaveBeenCalledWith(
expect.objectContaining({
userDataDir: "./my-profile",
executablePath: "/fake/chrome",
})
);
});
it("includes stealth args", async () => {
const { launchPersistentContext } = await import("../src/puppeteer.js");
await launchPersistentContext({ userDataDir: "./my-profile" });
const callArgs = vi.mocked(puppeteerMock.default.launch).mock.calls[0][0];
expect(callArgs.args.some((a: string) => a.startsWith("--fingerprint="))).toBe(true);
});
it("uses page.authenticate fallback for http proxy in persistent context on unsupported platform", async () => {
const config = await import("../src/config.js");
vi.spyOn(config, "getPlatformTag").mockReturnValue("darwin-arm64");
try {
const { launchPersistentContext } = await import("../src/puppeteer.js");
const browser = await launchPersistentContext({
userDataDir: "./my-profile",
proxy: "http://user:pass@proxy:8080",
});
const page = await browser.newPage();
expect(page.authenticate).toHaveBeenCalledWith({
username: "user",
password: "pass",
});
} finally {
vi.restoreAllMocks();
}
});
it("keeps SOCKS5 credentials in --proxy-server URL", async () => {
const { launchPersistentContext } = await import("../src/puppeteer.js");
const browser = await launchPersistentContext({
userDataDir: "./my-profile",
proxy: "socks5://user:pass@proxy:1080",
});
const callArgs = vi.mocked(puppeteerMock.default.launch).mock.calls[0][0];
expect(callArgs.args).toContain("--proxy-server=socks5://user:pass@proxy:1080");
const page = await browser.newPage();
expect(page.authenticate).not.toHaveBeenCalled();
});
it("forwards launchOptions to puppeteer launch", async () => {
const { launchPersistentContext } = await import("../src/puppeteer.js");
await launchPersistentContext({ userDataDir: "./my-profile", launchOptions: { slowMo: 50 } });
const callArgs = vi.mocked(puppeteerMock.default.launch).mock.calls[0][0];
expect(callArgs.slowMo).toBe(50);
expect(callArgs.userDataDir).toBe("./my-profile");
});
it("injects timezone and locale as binary flags", async () => {
const { launchPersistentContext } = await import("../src/puppeteer.js");
await launchPersistentContext({
userDataDir: "./my-profile",
timezone: "Asia/Tokyo",
locale: "ja-JP",
});
const callArgs = vi.mocked(puppeteerMock.default.launch).mock.calls[0][0];
expect(callArgs.args).toContain("--fingerprint-timezone=Asia/Tokyo");
expect(callArgs.args).toContain("--lang=ja-JP");
});
});
+46
View File
@@ -532,6 +532,52 @@ describe("Puppeteer: non-ASCII text avoids CDP shift path", () => {
});
// =========================================================================
// Per-call human config override (Puppeteer page-level)
// =========================================================================
describe("Puppeteer: page.type accepts per-call human config override", () => {
it("page.type forwards merged config to humanType", async () => {
const keyboardMod = await import("../src/human-puppeteer/keyboard.js");
const scrollMod = await import("../src/human-puppeteer/scroll.js");
const cfg = resolveConfig("default", {
idle_between_actions: false,
field_switch_delay: [0, 1],
});
expect(cfg.typing_delay).toBe(70);
let captured: any = null;
const typeSpy = vi.spyOn(keyboardMod, "humanType").mockImplementation(
async (_page, _raw, _text, callCfg) => { captured = callCfg; },
);
const scrollSpy = vi.spyOn(scrollMod, "scrollToElement").mockImplementation(
async (_page, _raw, _sel, cx, cy) => ({
box: { x: 100, y: 100, width: 50, height: 30 },
cursorX: cx,
cursorY: cy,
}),
);
const { patchPage } = await import("../src/human-puppeteer/index.js");
const page = buildMockPage();
const cursor = { x: 100, y: 100, initialized: true };
patchPage(page as any, cfg, cursor as any);
await (page as any).type("#email", "hi", {
typing_delay: 30,
mistype_chance: 0,
});
expect(captured.typing_delay).toBe(30);
expect(captured.mistype_chance).toBe(0);
expect(cfg.typing_delay).toBe(70);
typeSpy.mockRestore();
scrollSpy.mockRestore();
});
});
// =========================================================================
// patchPage stealth infrastructure (Puppeteer)
// =========================================================================
+10 -2
View File
@@ -44,9 +44,14 @@ function buildMockPage(overrides: Record<string, any> = {}): any {
const makeLocator = () => {
const loc: any = {
boundingBox: vi.fn(async () => ({ x: 100, y: 100, width: 200, height: 30 })),
boundingBox: vi.fn(async () => ({ x: 100, y: 300, width: 200, height: 30 })),
scrollIntoViewIfNeeded: vi.fn(async () => {}),
isChecked: overrides.isChecked ?? vi.fn(async () => false),
waitFor: vi.fn(async () => {}),
isVisible: vi.fn(async () => true),
isEnabled: vi.fn(async () => true),
isEditable: vi.fn(async () => true),
evaluate: vi.fn(async () => ({ hit: true })),
};
loc.first = vi.fn(() => loc);
return loc;
@@ -687,6 +692,9 @@ describe("isInputElement stealth integration via patchPage", () => {
}
if (method === "Runtime.evaluate") {
stealthEvaluateCalls.push(params.expression);
if (params.expression.includes("elementFromPoint")) {
return { result: { value: { hit: true } } };
}
return { result: { value: false } }; // not an input
}
return {};
@@ -696,7 +704,7 @@ describe("isInputElement stealth integration via patchPage", () => {
const page = buildMockPage({
evaluate: vi.fn(async (...args: any[]) => {
evaluateCalls.push(args);
return false;
return { hit: true };
}),
});
page.context = vi.fn(() => ({
+184 -2
View File
@@ -1,9 +1,11 @@
"""Unit tests for cloakserve — parse_connection_params, parse_cli_args, URL rewriting, connection tracking."""
import asyncio
import importlib.machinery
import importlib.util
import sys
from pathlib import Path
from types import SimpleNamespace
from unittest.mock import patch
import pytest
@@ -22,6 +24,8 @@ parse_connection_params = _mod.parse_connection_params
parse_cli_args = _mod.parse_cli_args
ChromePool = _mod.ChromePool
_default_data_dir = _mod._default_data_dir
SAFE_SEED_RE = _mod.SAFE_SEED_RE
RESERVED_SEEDS = _mod.RESERVED_SEEDS
# ---------------------------------------------------------------------------
@@ -139,8 +143,94 @@ class TestParseCliArgs:
# ---------------------------------------------------------------------------
class TestURLRewriting:
"""Test the URL rewriting logic used by /json/version and /json/list."""
class TestWebSocketOriginGuard:
"""Verify cloakserve rejects browser-origin CDP WebSocket hijacks."""
def test_absent_origin_allowed_for_non_browser_cdp_clients(self):
assert _mod._origin_is_allowed(None, "127.0.0.1:9555")
def test_matching_origin_host_allowed(self):
assert _mod._origin_is_allowed("http://127.0.0.1:9555", "127.0.0.1:9555")
def test_chrome_devtools_origin_allowed(self):
assert _mod._origin_is_allowed("devtools://devtools", "127.0.0.1:9555")
assert _mod._origin_is_allowed("chrome-devtools://devtools", "127.0.0.1:9555")
@pytest.mark.parametrize("origin", [
"http://attacker.example",
"https://attacker.example",
"http://PUBLIC_HOST:9555",
"http://attacker.example:9555",
"http://127.0.0.1:9555/",
"http://127.0.0.1:9555/path",
"http://127.0.0.1:9555?q=1",
"http://127.0.0.1:9555#fragment",
"http://user@127.0.0.1:9555",
"http://@127.0.0.1:9555",
"http://:@127.0.0.1:9555",
"http://127.0.0.1:",
"null",
"file://",
])
def test_untrusted_browser_origins_rejected(self, origin):
assert not _mod._origin_is_allowed(origin, "127.0.0.1:9555")
def test_public_origin_matching_host_is_still_rejected(self):
assert not _mod._origin_is_allowed("http://attacker.example:9555", "attacker.example:9555")
@pytest.mark.parametrize("host", [
"user@127.0.0.1:9555",
"127.0.0.1:9555/path",
"127.0.0.1:9555?x=1",
"127.0.0.1:9555#fragment",
"127.0.0.1:9555, attacker.example:9555",
"@127.0.0.1:9555",
":@127.0.0.1:9555",
"127.0.0.1:",
"[::1]:",
])
def test_malformed_host_is_rejected_even_when_hostname_is_loopback(self, host):
assert not _mod._origin_is_allowed("http://127.0.0.1:9555", host)
def test_request_scheme_controls_host_default_port(self):
assert _mod._origin_is_allowed("https://localhost", "localhost", request_scheme="https")
assert not _mod._origin_is_allowed("https://localhost", "localhost", request_scheme="http")
def test_ws_handler_rejects_untrusted_origin_before_launching_chrome(self):
class RejectingPool:
async def get_or_launch(self, **_kwargs):
raise AssertionError("untrusted origin should be rejected before launching Chrome")
request = SimpleNamespace(
headers={"Host": "127.0.0.1:9555", "Origin": "http://attacker.example"},
app={"pool": RejectingPool()},
match_info={"path": "browser/browser-guid"},
)
response = asyncio.run(_mod.handle_ws_default(request))
assert response.status == 403
assert "untrusted" in response.text.lower()
def test_seed_ws_handler_rejects_untrusted_origin_before_launching_chrome(self):
class RejectingPool:
async def get_or_launch(self, **_kwargs):
raise AssertionError("untrusted origin should be rejected before launching Chrome")
request = SimpleNamespace(
headers={"Host": "127.0.0.1:9555", "Origin": "http://attacker.example"},
app={"pool": RejectingPool()},
match_info={"seed": "abc123", "path": "page/page-guid"},
)
response = asyncio.run(_mod.handle_ws_seed(request))
assert response.status == 403
assert "untrusted" in response.text.lower()
class TestHandlerURLRewriting:
"""Verify handlers rewrite CDP WebSocket URLs to the public cloakserve endpoint."""
def _rewrite_version(self, orig_ws: str, host: str, seed: str | None, scheme: str = "ws") -> str:
"""Replicate the URL rewrite logic from handle_json_version."""
@@ -244,3 +334,95 @@ class TestConnectionTracking:
pool.disconnect("a")
assert pool._connections["a"] == 1
assert pool._connections["b"] == 1
# ---------------------------------------------------------------------------
# Seed validation (CVE fix — path traversal via fingerprint param)
# ---------------------------------------------------------------------------
class TestSeedValidation:
"""Verify SAFE_SEED_RE rejects path traversal and reserved names."""
@pytest.mark.parametrize("seed", [
"../foo", "../../etc", "/etc/passwd", "..", ".", "foo/bar",
"foo\\bar", "\x00evil", "", "a" * 129,
])
def test_malicious_seeds_rejected(self, seed):
assert not SAFE_SEED_RE.match(seed)
@pytest.mark.parametrize("seed", [
"__default__",
])
def test_reserved_seeds_rejected(self, seed):
assert seed in RESERVED_SEEDS
@pytest.mark.parametrize("seed", [
"12345", "my-seed_01", "ABC", "a" * 128, "0", "test-seed",
])
def test_valid_seeds_accepted(self, seed):
assert SAFE_SEED_RE.match(seed)
assert seed not in RESERVED_SEEDS
# ---------------------------------------------------------------------------
# Path containment (_safe_rmtree)
# ---------------------------------------------------------------------------
class TestSafeRmtree:
"""Verify _safe_rmtree refuses to delete outside data_dir."""
def _make_pool(self, data_dir: str):
return ChromePool(
binary="/fake/chrome",
global_args=[],
headless=True,
data_dir=data_dir,
)
def test_refuses_path_outside_data_dir(self, tmp_path):
data_dir = tmp_path / "profiles"
data_dir.mkdir()
victim = tmp_path / "victim"
victim.mkdir()
(victim / "sentinel").touch()
pool = self._make_pool(str(data_dir))
pool._safe_rmtree(str(victim))
assert victim.exists(), "Directory outside data_dir must not be deleted"
def test_refuses_data_dir_itself(self, tmp_path):
data_dir = tmp_path / "profiles"
data_dir.mkdir()
(data_dir / "sentinel").touch()
pool = self._make_pool(str(data_dir))
pool._safe_rmtree(str(data_dir))
assert data_dir.exists(), "data_dir itself must not be deleted"
def test_deletes_valid_subdirectory(self, tmp_path):
data_dir = tmp_path / "profiles"
data_dir.mkdir()
subdir = data_dir / "seed-12345"
subdir.mkdir()
(subdir / "data").touch()
pool = self._make_pool(str(data_dir))
pool._safe_rmtree(str(subdir))
assert not subdir.exists(), "Valid subdirectory should be deleted"
def test_refuses_traversal_path(self, tmp_path):
data_dir = tmp_path / "profiles"
data_dir.mkdir()
victim = tmp_path / "victim"
victim.mkdir()
traversal = str(data_dir / ".." / "victim")
pool = self._make_pool(str(data_dir))
pool._safe_rmtree(traversal)
assert victim.exists(), "Traversal path must not be deleted"
+33
View File
@@ -0,0 +1,33 @@
import os
from unittest.mock import MagicMock, patch
from cloakbrowser import launch
@patch("cloakbrowser.browser.ensure_binary")
@patch("cloakbrowser.browser._import_sync_playwright")
def test_extension_loading(mock_playwright_import, mock_ensure_binary):
mock_ensure_binary.return_value = "/fake/chrome"
mock_browser = MagicMock()
mock_pw = MagicMock()
mock_pw.chromium.launch.return_value = mock_browser
mock_pw_manager = MagicMock()
mock_pw_manager.return_value.start.return_value = mock_pw
mock_playwright_import.return_value = mock_pw_manager
launch(extension_paths=["./ext"])
mock_pw.chromium.launch.assert_called_once()
launch_call = mock_pw.chromium.launch.call_args
args = launch_call.kwargs["args"]
abs_path = os.path.abspath("./ext")
assert f"--load-extension={abs_path}" in args
assert f"--disable-extensions-except={abs_path}" in args
+15
View File
@@ -1,6 +1,7 @@
"""Unit tests for GeoIP-based timezone/locale detection."""
from unittest.mock import patch
import time
import pytest
@@ -144,6 +145,20 @@ def test_maybe_resolve_fills_both():
assert ip == "5.6.7.8"
def test_maybe_resolve_geoip_timeout_returns_existing_values(monkeypatch):
"""A stalled proxy lookup should not block launch indefinitely."""
mock_geoip2 = type("module", (), {"database": type("db", (), {"Reader": None})})()
monkeypatch.setenv("CLOAKBROWSER_GEOIP_TIMEOUT_SECONDS", "0.05")
with patch.dict("sys.modules", {"geoip2": mock_geoip2, "geoip2.database": mock_geoip2.database}):
with patch("cloakbrowser.geoip._ensure_geoip_db", return_value=object()):
start = time.monotonic()
tz, loc, ip = maybe_resolve_geoip(True, "http://203.0.113.10:8080", None, "fr-FR")
elapsed = time.monotonic() - start
assert (tz, loc, ip) == (None, "fr-FR", None)
assert elapsed < 0.5
# ---------------------------------------------------------------------------
# _is_private_ip
# ---------------------------------------------------------------------------
+221 -30
View File
@@ -14,6 +14,26 @@ import time
import sys
import asyncio
import pytest
from unittest.mock import MagicMock
def _mock_el_evaluate(is_input=False):
"""Mock evaluate that returns is_input for tagName checks and {hit: True} for pointer events."""
def _eval(js, *args, **kwargs):
if isinstance(js, str) and "elementFromPoint" in js:
return {"hit": True}
return is_input
return MagicMock(side_effect=_eval)
def _async_mock_el_evaluate(is_input=False):
"""Async version of _mock_el_evaluate."""
from unittest.mock import AsyncMock
async def _eval(js, *args, **kwargs):
if isinstance(js, str) and "elementFromPoint" in js:
return {"hit": True}
return is_input
return AsyncMock(side_effect=_eval)
# =========================================================================
@@ -192,6 +212,59 @@ class TestAsyncCompat:
import asyncio
assert asyncio.iscoroutinefunction(async_sleep_ms)
def test_patch_page_async_does_not_crash(self):
"""patch_page_async must not raise NameError for missing definitions."""
import cloakbrowser.human as h
from cloakbrowser.human import _CursorState
from cloakbrowser.human.config import resolve_config
from unittest.mock import MagicMock, AsyncMock
cfg = resolve_config("default", {"idle_between_actions": False})
cursor = _CursorState()
cursor.initialized = True
cursor.x = 100
cursor.y = 100
page = MagicMock()
page.click = AsyncMock()
page.dblclick = AsyncMock()
page.hover = AsyncMock()
page.type = AsyncMock()
page.fill = AsyncMock()
page.goto = AsyncMock()
page.check = AsyncMock()
page.uncheck = AsyncMock()
page.select_option = AsyncMock()
page.press = AsyncMock()
page.is_checked = AsyncMock(return_value=False)
page.viewport_size = {"width": 1280, "height": 720}
page.evaluate = AsyncMock(return_value={"hit": True})
page.context.new_cdp_session = AsyncMock(side_effect=Exception("no cdp"))
page.mouse = MagicMock()
page.mouse.move = AsyncMock()
page.mouse.click = AsyncMock()
page.mouse.wheel = AsyncMock()
page.mouse.down = AsyncMock()
page.mouse.up = AsyncMock()
page.keyboard = MagicMock()
page.keyboard.type = AsyncMock()
page.keyboard.down = AsyncMock()
page.keyboard.up = AsyncMock()
page.keyboard.press = AsyncMock()
page.keyboard.insert_text = AsyncMock()
page.query_selector = AsyncMock(return_value=None)
page.query_selector_all = AsyncMock(return_value=[])
page.wait_for_selector = AsyncMock(return_value=None)
page.main_frame = MagicMock()
page.main_frame.return_value = MagicMock()
page.main_frame.return_value.child_frames = MagicMock(return_value=[])
page.main_frame.child_frames = MagicMock(return_value=[])
h.patch_page_async(page, cfg, cursor)
assert hasattr(page, '_original')
assert page.select_option != AsyncMock
# =========================================================================
# 4. Focus check — press / clear / pressSequentially
@@ -635,7 +708,7 @@ class TestBrowserBotDetection:
time.sleep(0.3)
page.locator('#password').fill('SecurePass!123')
time.sleep(0.5)
page.locator('button[type="submit"]').click()
page.locator('#loginForm button[type="submit"]').click()
time.sleep(5)
body = page.locator('body').text_content()
assert '"superHumanSpeed": true' not in body
@@ -652,7 +725,7 @@ class TestBrowserBotDetection:
t0 = time.time()
page.locator('#email').fill('test@example.com')
page.locator('#password').fill('MyPassword!99')
page.locator('button[type="submit"]').click()
page.locator('#loginForm button[type="submit"]').click()
elapsed_ms = int((time.time() - t0) * 1000)
time.sleep(3)
assert elapsed_ms > 3000
@@ -707,7 +780,7 @@ class TestElementHandlePatchingSync:
el = MagicMock()
el._human_patched = False
el.bounding_box = MagicMock(return_value={"x": 50, "y": 50, "width": 100, "height": 30})
el.evaluate = MagicMock(return_value=True) # is_input
el.evaluate = _mock_el_evaluate(is_input=True)
el.is_checked = MagicMock(return_value=False)
el.query_selector = MagicMock(return_value=None)
el.query_selector_all = MagicMock(return_value=[])
@@ -738,7 +811,7 @@ class TestElementHandlePatchingSync:
el = MagicMock()
el._human_patched = False
el.bounding_box = MagicMock(return_value={"x": 200, "y": 200, "width": 100, "height": 30})
el.evaluate = MagicMock(return_value=False)
el.evaluate = _mock_el_evaluate(is_input=False)
el.is_checked = MagicMock(return_value=False)
el.query_selector = MagicMock(return_value=None)
el.query_selector_all = MagicMock(return_value=[])
@@ -779,7 +852,7 @@ class TestElementHandlePatchingSync:
el = MagicMock()
el._human_patched = False
el.bounding_box = MagicMock(return_value={"x": 200, "y": 200, "width": 100, "height": 30})
el.evaluate = MagicMock(return_value=False)
el.evaluate = _mock_el_evaluate(is_input=False)
el.is_checked = MagicMock(return_value=False)
el.query_selector = MagicMock(return_value=None)
el.query_selector_all = MagicMock(return_value=[])
@@ -819,7 +892,7 @@ class TestElementHandlePatchingSync:
el = MagicMock()
el._human_patched = False
el.bounding_box = MagicMock(return_value={"x": 200, "y": 200, "width": 100, "height": 30})
el.evaluate = MagicMock(return_value=True) # is input
el.evaluate = _mock_el_evaluate(is_input=True) # is input
el.is_checked = MagicMock(return_value=False)
el.query_selector = MagicMock(return_value=None)
el.query_selector_all = MagicMock(return_value=[])
@@ -867,7 +940,7 @@ class TestElementHandlePatchingSync:
el = MagicMock()
el._human_patched = False
el.bounding_box = MagicMock(return_value={"x": 200, "y": 200, "width": 100, "height": 30})
el.evaluate = MagicMock(return_value=True)
el.evaluate = _mock_el_evaluate(is_input=True)
el.is_checked = MagicMock(return_value=False)
el.query_selector = MagicMock(return_value=None)
el.query_selector_all = MagicMock(return_value=[])
@@ -947,7 +1020,7 @@ class TestElementHandlePatchingSync:
el = MagicMock()
el._human_patched = False
el.bounding_box = MagicMock(return_value={"x": 50, "y": 50, "width": 100, "height": 30})
el.evaluate = MagicMock(return_value=False)
el.evaluate = _mock_el_evaluate(is_input=False)
el.is_checked = MagicMock(return_value=False)
el.query_selector = MagicMock(return_value=child)
el.query_selector_all = MagicMock(return_value=[])
@@ -971,7 +1044,7 @@ class TestElementHandlePatchingSync:
el = MagicMock()
el._human_patched = False
el.bounding_box = MagicMock(return_value={"x": 50, "y": 50, "width": 100, "height": 30})
el.evaluate = MagicMock(return_value=False)
el.evaluate = _mock_el_evaluate(is_input=False)
el.is_checked = MagicMock(return_value=False)
el.query_selector = MagicMock(return_value=None)
el.query_selector_all = MagicMock(return_value=[])
@@ -1036,7 +1109,7 @@ class TestElementHandlePatchingSync:
el = MagicMock()
el._human_patched = False
el.bounding_box = MagicMock(return_value={"x": 50, "y": 50, "width": 100, "height": 30})
el.evaluate = MagicMock(return_value=False)
el.evaluate = _mock_el_evaluate(is_input=False)
el.is_checked = MagicMock(return_value=False)
el.query_selector = MagicMock(return_value=None)
el.query_selector_all = MagicMock(return_value=[])
@@ -1069,7 +1142,7 @@ class TestElementHandlePatchingSync:
el = MagicMock()
el._human_patched = False
el.bounding_box = MagicMock(return_value={"x": 50, "y": 50, "width": 100, "height": 30})
el.evaluate = MagicMock(return_value=False)
el.evaluate = _mock_el_evaluate(is_input=False)
el.is_checked = MagicMock(return_value=False)
el.query_selector = MagicMock(return_value=None)
el.query_selector_all = MagicMock(return_value=[])
@@ -1114,8 +1187,9 @@ class TestElementHandlePatchingAsync:
el = MagicMock()
el._human_patched = False
el.bounding_box = AsyncMock(return_value={"x": 200, "y": 200, "width": 100, "height": 30})
el.evaluate = AsyncMock(return_value=False)
el.evaluate = _async_mock_el_evaluate(is_input=False)
el.is_checked = AsyncMock(return_value=False)
el.wait_for_element_state = AsyncMock()
el.query_selector = AsyncMock(return_value=None)
el.query_selector_all = AsyncMock(return_value=[])
el.wait_for_selector = AsyncMock(return_value=None)
@@ -1155,8 +1229,9 @@ class TestElementHandlePatchingAsync:
el = MagicMock()
el._human_patched = False
el.bounding_box = AsyncMock(return_value={"x": 50, "y": 50, "width": 100, "height": 30})
el.evaluate = AsyncMock(return_value=False)
el.evaluate = _async_mock_el_evaluate(is_input=False)
el.is_checked = AsyncMock(return_value=False)
el.wait_for_element_state = AsyncMock()
el.query_selector = AsyncMock(return_value=None)
el.query_selector_all = AsyncMock(return_value=[])
el.wait_for_selector = AsyncMock(return_value=None)
@@ -1376,7 +1451,7 @@ class TestPerCallTimeoutForwarding:
page.goto = MagicMock()
page.is_checked = MagicMock(return_value=False)
page.viewport_size = {"width": 1280, "height": 720}
page.evaluate = MagicMock(return_value=False)
page.evaluate = MagicMock(return_value={"hit": True})
page.context.new_cdp_session = MagicMock(side_effect=Exception("no cdp"))
page.mouse = MagicMock()
page.keyboard = MagicMock()
@@ -1389,13 +1464,14 @@ class TestPerCallTimeoutForwarding:
captured = {}
def fake_scroll(page_arg, raw, selector, cx, cy, cfg_arg, timeout=30000):
captured["timeout"] = timeout
return ({"x": 100, "y": 100, "width": 50, "height": 30}, cx, cy)
return ({"x": 100, "y": 100, "width": 50, "height": 30}, cx, cy, False)
with patch.object(h, "scroll_to_element", side_effect=fake_scroll):
with patch.object(h, "scroll_to_element", side_effect=fake_scroll), \
patch.object(h, "ensure_actionable"):
h.patch_page(page, cfg, cursor)
page.click("#slow-button", timeout=5000)
assert captured.get("timeout") == 5000, f"expected 5000, got {captured}"
assert 4900 <= captured.get("timeout", 0) <= 5000, f"expected ~5000, got {captured}"
# =========================================================================
@@ -1462,7 +1538,7 @@ class TestPerCallHumanConfigOverride:
page.goto = MagicMock()
page.is_checked = MagicMock(return_value=False)
page.viewport_size = {"width": 1280, "height": 720}
page.evaluate = MagicMock(return_value=False)
page.evaluate = MagicMock(return_value={"hit": True})
page.context.new_cdp_session = MagicMock(side_effect=Exception("no cdp"))
page.mouse = MagicMock()
page.keyboard = MagicMock()
@@ -1478,10 +1554,12 @@ class TestPerCallHumanConfigOverride:
captured["mistype_chance"] = cfg_arg.mistype_chance
def fake_scroll(*args, **kwargs):
return ({"x": 100, "y": 100, "width": 50, "height": 30}, 100, 100)
return ({"x": 100, "y": 100, "width": 50, "height": 30}, 100, 100, False)
with patch.object(h, "human_type", side_effect=fake_human_type), \
patch.object(h, "scroll_to_element", side_effect=fake_scroll):
patch.object(h, "scroll_to_element", side_effect=fake_scroll), \
patch.object(h, "ensure_actionable"), \
patch.object(h, "check_pointer_events"):
h.patch_page(page, cfg, cursor)
page.type(
"#email", "hi",
@@ -1490,7 +1568,6 @@ class TestPerCallHumanConfigOverride:
assert captured["typing_delay"] == 30
assert captured["mistype_chance"] == 0
# Global cfg untouched — per-call override doesn't leak
assert cfg.typing_delay == 70
def test_page_fill_uses_per_call_typing_delay(self):
@@ -1512,7 +1589,7 @@ class TestPerCallHumanConfigOverride:
page = MagicMock()
page.viewport_size = {"width": 1280, "height": 720}
page.is_checked = MagicMock(return_value=False)
page.evaluate = MagicMock(return_value=False)
page.evaluate = MagicMock(return_value={"hit": True})
page.context.new_cdp_session = MagicMock(side_effect=Exception("no cdp"))
page.mouse = MagicMock()
page.keyboard = MagicMock()
@@ -1527,10 +1604,12 @@ class TestPerCallHumanConfigOverride:
captured["typing_delay"] = cfg_arg.typing_delay
def fake_scroll(*args, **kwargs):
return ({"x": 100, "y": 100, "width": 50, "height": 30}, 100, 100)
return ({"x": 100, "y": 100, "width": 50, "height": 30}, 100, 100, False)
with patch.object(h, "human_type", side_effect=fake_human_type), \
patch.object(h, "scroll_to_element", side_effect=fake_scroll):
patch.object(h, "scroll_to_element", side_effect=fake_scroll), \
patch.object(h, "ensure_actionable"), \
patch.object(h, "check_pointer_events"):
h.patch_page(page, cfg, cursor)
page.fill("#password", "secret", human_config={"typing_delay": 150})
@@ -1562,7 +1641,7 @@ class TestPerCallHumanConfigOverride:
el.bounding_box = MagicMock(
return_value={"x": 200, "y": 200, "width": 100, "height": 30}
)
el.evaluate = MagicMock(return_value=True)
el.evaluate = _mock_el_evaluate(is_input=True)
el.is_checked = MagicMock(return_value=False)
el.query_selector = MagicMock(return_value=None)
el.query_selector_all = MagicMock(return_value=[])
@@ -1608,9 +1687,10 @@ class TestScrollIntoViewIfNeeded:
# Box is dead-center of viewport — squarely in scroll_target_zone
in_view_box = {"x": 200, "y": 300, "width": 50, "height": 30}
box, cx, cy = human_scroll_into_view(
box, cx, cy, did_scroll = human_scroll_into_view(
page, raw, lambda: in_view_box, 0, 0, cfg,
)
assert not did_scroll, "In-viewport elements shouldn't report scrolling"
assert box == in_view_box
assert not raw.wheel.called, "In-viewport elements shouldn't trigger wheel events"
@@ -1672,7 +1752,7 @@ class TestScrollIntoViewIfNeeded:
el.bounding_box = MagicMock(
return_value={"x": 200, "y": 200, "width": 50, "height": 30}
)
el.evaluate = MagicMock(return_value=False)
el.evaluate = _mock_el_evaluate(is_input=False)
el.is_checked = MagicMock(return_value=False)
el.query_selector = MagicMock(return_value=None)
el.query_selector_all = MagicMock(return_value=[])
@@ -1683,14 +1763,13 @@ class TestScrollIntoViewIfNeeded:
called = {"count": 0}
def fake(*args, **kwargs):
called["count"] += 1
return ({"x": 200, "y": 200, "width": 50, "height": 30}, 100, 100)
return ({"x": 200, "y": 200, "width": 50, "height": 30}, 100, 100, False)
with patch.object(h, "human_scroll_into_view", side_effect=fake):
_patch_single_element_handle_sync(
el, page, cfg, cursor, MagicMock(), MagicMock(),
page._original, None, None,
)
# Patched method should now invoke our humanized helper
el.scroll_into_view_if_needed()
assert called["count"] >= 1, "humanized scroll helper was never called"
@@ -1735,7 +1814,7 @@ class TestScrollIntoViewIfNeeded:
called["count"] += 1
# cfg is the 6th positional arg (page, raw, get_box, cx, cy, cfg)
called["cfg"] = args[5] if len(args) >= 6 else kwargs.get("cfg")
return ({"x": 100, "y": 100, "width": 50, "height": 30}, 200, 200)
return ({"x": 100, "y": 100, "width": 50, "height": 30}, 200, 200, False)
with patch.object(h, "human_scroll_into_view", side_effect=fake):
Locator.scroll_into_view_if_needed(
@@ -1749,6 +1828,118 @@ class TestScrollIntoViewIfNeeded:
assert cursor.x == 200 and cursor.y == 200
# =========================================================================
# Issue #307: frame/page click timeout should not multiply
# =========================================================================
class TestTimeoutBudget307:
"""Verify timeout budget is shared across sequential operations."""
def test_page_click_total_time_within_budget(self):
"""page.click on a missing element should not exceed ~1x the timeout."""
import cloakbrowser.human as h
from cloakbrowser.human import _CursorState
from cloakbrowser.human.config import resolve_config
from unittest.mock import MagicMock, patch
TIMEOUT_MS = 500
cfg = resolve_config("default", {"idle_between_actions": False})
cursor = _CursorState()
cursor.initialized = True
cursor.x = 100
cursor.y = 100
page = MagicMock()
page.click = MagicMock()
page.dblclick = MagicMock()
page.hover = MagicMock()
page.type = MagicMock()
page.fill = MagicMock()
page.goto = MagicMock()
page.is_checked = MagicMock(return_value=False)
page.viewport_size = {"width": 1280, "height": 720}
page.evaluate = MagicMock(return_value={"hit": True})
page.context.new_cdp_session = MagicMock(side_effect=Exception("no cdp"))
page.mouse = MagicMock()
page.keyboard = MagicMock()
page.query_selector = MagicMock(return_value=None)
page.query_selector_all = MagicMock(return_value=[])
page.wait_for_selector = MagicMock(return_value=None)
page.main_frame = MagicMock()
page.main_frame.child_frames = []
loc = MagicMock()
loc.wait_for = MagicMock(side_effect=lambda **kw: time.sleep(kw.get("timeout", 30000) / 1000.0))
loc.is_visible = MagicMock(return_value=False)
loc.first = loc
page.locator = MagicMock(return_value=loc)
h.patch_page(page, cfg, cursor)
start = time.monotonic()
try:
page.click("#does-not-exist", timeout=TIMEOUT_MS)
except Exception:
pass
elapsed_ms = (time.monotonic() - start) * 1000
assert elapsed_ms < TIMEOUT_MS * 1.8, (
f"expected <{TIMEOUT_MS * 1.8}ms, got {elapsed_ms:.0f}ms"
)
class TestPointerEventsFailOpen:
"""The pointer-events check must fail open: when it cannot run (evaluate /
bounding_box throws -> result None), proceed with the click instead of
blocking it until the timeout expires."""
def test_handle_failopen_returns_on_evaluate_error(self):
from cloakbrowser.human.actionability import check_pointer_events_handle
el = MagicMock()
el.bounding_box = MagicMock(side_effect=Exception("stale handle"))
el.evaluate = MagicMock(side_effect=Exception("execution context destroyed"))
start = time.monotonic()
check_pointer_events_handle(MagicMock(), el, 100, 100, timeout=2000) # must not raise
elapsed_ms = (time.monotonic() - start) * 1000
assert elapsed_ms < 500, f"fail-open should return promptly, took {elapsed_ms:.0f}ms"
def test_locator_failopen_returns_on_evaluate_error(self):
from cloakbrowser.human.actionability import check_pointer_events
page = MagicMock()
loc = MagicMock()
loc.first = loc
loc.bounding_box = MagicMock(side_effect=Exception("no element"))
loc.evaluate = MagicMock(side_effect=Exception("no element"))
page.locator = MagicMock(return_value=loc)
start = time.monotonic()
check_pointer_events(page, "#x", 100, 100, timeout=2000) # must not raise
elapsed_ms = (time.monotonic() - start) * 1000
assert elapsed_ms < 500, f"fail-open should return promptly, took {elapsed_ms:.0f}ms"
def test_handle_still_raises_when_covered(self):
"""A genuine 'covered' result (not None) must still raise — fail-open
only applies when the check could not be determined."""
from cloakbrowser.human.actionability import (
check_pointer_events_handle, ElementNotReceivingEventsError,
)
el = MagicMock()
el.bounding_box = MagicMock(return_value={"x": 0, "y": 0, "width": 10, "height": 10})
el.evaluate = MagicMock(return_value={"hit": False, "covering": "DIV"})
with pytest.raises(ElementNotReceivingEventsError):
check_pointer_events_handle(MagicMock(), el, 5, 5, timeout=200)
def test_async_handle_failopen_returns_on_evaluate_error(self):
from cloakbrowser.human.actionability_async import async_check_pointer_events_handle
from unittest.mock import AsyncMock
el = MagicMock()
el.bounding_box = AsyncMock(side_effect=Exception("stale handle"))
el.evaluate = AsyncMock(side_effect=Exception("execution context destroyed"))
start = time.monotonic()
asyncio.run(async_check_pointer_events_handle(MagicMock(), el, 100, 100, timeout=2000))
elapsed_ms = (time.monotonic() - start) * 1000
assert elapsed_ms < 500, f"fail-open should return promptly, took {elapsed_ms:.0f}ms"
# =========================================================================
# Direct runner (backwards compat)
# =========================================================================
+171
View File
@@ -0,0 +1,171 @@
"""Security tests for the AWS Lambda handler URL validation."""
from __future__ import annotations
import sys
from pathlib import Path
from unittest.mock import patch
import pytest
sys.path.insert(
0, str(Path(__file__).resolve().parent.parent / "examples" / "integrations" / "aws_lambda")
)
from lambda_handler import _build_launch_kwargs, _classify_error, _validate_url
class TestSchemeValidation:
"""Fix 1: only http:// and https:// are accepted."""
@pytest.mark.parametrize("url", [
"file:///etc/passwd",
"file:///proc/self/environ",
"data:text/html,<h1>pwned</h1>",
"javascript:alert(1)",
"chrome://settings",
"about:blank",
"ftp://example.com/file",
"",
])
def test_rejects_non_http_schemes(self, url):
with pytest.raises(ValueError, match="Only http"):
_validate_url(url)
@pytest.mark.parametrize("url", [
"https://example.com",
"http://example.com",
"https://example.com/path?q=1",
"HTTP://EXAMPLE.COM",
])
def test_accepts_http_and_https(self, url):
_validate_url(url)
def test_rejects_missing_hostname(self):
with pytest.raises(ValueError, match="no hostname"):
_validate_url("http://")
class TestSSRFProtection:
"""Fix 2: block private, loopback, link-local, reserved, and metadata IPs."""
@pytest.mark.parametrize("url,label", [
("http://169.254.169.254", "AWS metadata"),
("http://169.254.169.254/latest/meta-data/", "AWS metadata path"),
("http://127.0.0.1", "loopback"),
("http://127.0.0.2", "loopback range"),
("http://localhost", "localhost"),
("http://10.0.0.1", "private 10.x"),
("http://172.16.0.1", "private 172.16"),
("http://192.168.1.1", "private 192.168"),
("http://0.0.0.0", "unspecified"),
("http://[::1]", "IPv6 loopback"),
])
def test_rejects_private_ips(self, url, label):
with pytest.raises(ValueError, match="private/internal"):
_validate_url(url)
def test_rejects_carrier_grade_nat(self):
with pytest.raises(ValueError, match="private/internal"):
_validate_url("http://100.64.0.1")
def test_rejects_unresolvable_hostname(self):
with pytest.raises(ValueError, match="Cannot resolve"):
_validate_url("http://this-host-does-not-exist-cb-test.invalid")
def test_rejects_ipv4_mapped_ipv6(self):
"""::ffff:127.0.0.1 should be blocked even though it's technically IPv6."""
with pytest.raises(ValueError, match="private/internal"):
_validate_url("http://[::ffff:127.0.0.1]")
class TestExtraArgsRemoval:
"""Fix 3: caller-controlled extra_args are ignored; internal _strategy_args work."""
def test_ignores_caller_extra_args(self):
event = {"url": "https://example.com", "extra_args": ["--remote-debugging-port=9222"]}
kwargs = _build_launch_kwargs(event)
assert "--remote-debugging-port=9222" not in kwargs["args"]
def test_includes_strategy_args(self):
event = {"url": "https://example.com", "_strategy_args": ["--ignore-certificate-errors"]}
kwargs = _build_launch_kwargs(event)
assert "--ignore-certificate-errors" in kwargs["args"]
def test_classify_error_uses_strategy_args(self):
result = _classify_error(Exception("ERR_CERT_AUTHORITY_INVALID"))
assert "_strategy_args" in result
assert "extra_args" not in result
def test_always_includes_lambda_hardening_flags(self):
kwargs = _build_launch_kwargs({"url": "https://example.com"})
assert "--disable-dev-shm-usage" in kwargs["args"]
assert "--no-zygote" in kwargs["args"]
def test_caller_cannot_inject_strategy_args(self):
"""_strategy_args in the caller event must be stripped by _run() before launch."""
from lambda_handler import _run
import inspect
source = inspect.getsource(_run)
assert '"_strategy_args"' in source and "extra_args" in source, \
"_run must strip both _strategy_args and extra_args from caller event"
class TestRedirectSSRF:
"""Fix 5: post-navigation re-validation catches redirects to blocked IPs.
These mock socket.getaddrinfo to simulate redirect scenarios without
needing a real browser or HTTP server.
"""
def test_validate_url_catches_redirect_target(self):
"""If Chromium followed a redirect to 169.254.169.254, the post-nav
_validate_url(page.url) call should reject it."""
with pytest.raises(ValueError, match="private/internal"):
_validate_url("http://169.254.169.254/latest/meta-data/iam/security-credentials/")
def test_validate_url_catches_localhost_redirect(self):
with pytest.raises(ValueError, match="private/internal"):
_validate_url("http://127.0.0.1:8080/admin")
def test_code_flow_validates_before_content(self):
"""Verify that _attempt_scrape calls _validate_url(page.url) at line 282
BEFORE building the result dict at line 290 (sequential code path)."""
import ast
handler_path = (
Path(__file__).resolve().parent.parent
/ "examples" / "integrations" / "aws_lambda" / "lambda_handler.py"
)
source = handler_path.read_text()
tree = ast.parse(source)
for node in ast.walk(tree):
if isinstance(node, ast.AsyncFunctionDef) and node.name == "_attempt_scrape":
body = node.body
# Find the try block
for stmt in body:
if isinstance(stmt, ast.Try):
try_body = stmt.body
validate_lines = []
content_line = None
for s in try_body:
if isinstance(s, ast.Expr) and isinstance(s.value, ast.Call):
func = s.value.func
if isinstance(func, ast.Name) and func.id == "_validate_url":
validate_lines.append(s.lineno)
if isinstance(s, ast.AnnAssign):
if isinstance(s.target, ast.Name) and s.target.id == "result":
content_line = s.lineno
elif isinstance(s, ast.Assign):
for target in s.targets:
if isinstance(target, ast.Name) and target.id == "result":
content_line = s.lineno
assert len(validate_lines) >= 2, (
f"Expected 2 _validate_url calls, found {len(validate_lines)}"
)
assert content_line is not None
assert all(v < content_line for v in validate_lines), (
f"_validate_url (lines {validate_lines}) must come before "
f"result assignment (line {content_line})"
)
return
pytest.fail("Could not find _attempt_scrape function in source")
+3 -2
View File
@@ -165,10 +165,11 @@ def test_persistent_context_close_stops_pw(_mock_geoip, _mock_bin):
pw.stop.assert_called_once()
@patch("cloakbrowser.config.get_platform_tag", return_value="darwin-arm64")
@patch("cloakbrowser.browser.ensure_binary", return_value="/fake/chrome")
@patch("cloakbrowser.browser.maybe_resolve_geoip", return_value=(None, None, None))
def test_persistent_context_proxy_string(_mock_geoip, _mock_bin):
"""Proxy string parsed and passed."""
def test_persistent_context_proxy_string(_mock_geoip, _mock_bin, _mock_platform):
"""Proxy string parsed and passed (unsupported platform → Playwright dict)."""
pw_cm, pw, context = _make_mock_pw_and_context()
with patch("playwright.sync_api.sync_playwright", return_value=pw_cm):
+154 -14
View File
@@ -55,12 +55,12 @@ class TestBuildProxyKwargs:
assert kwargs == {"proxy": {"server": "http://proxy:8080"}}
assert args == []
def test_proxy_with_auth(self):
@patch("cloakbrowser.config.get_chromium_version", return_value="146.0.7680.177.5")
@patch("cloakbrowser.config.get_platform_tag", return_value="linux-x64")
def test_proxy_with_auth(self, *_):
kwargs, args = _resolve_proxy_config("http://user:pass@proxy:8080")
assert kwargs == {
"proxy": {"server": "http://proxy:8080", "username": "user", "password": "pass"}
}
assert args == []
assert kwargs == {}
assert args == ["--proxy-server=http://user:pass@proxy:8080"]
def test_proxy_dict_passthrough(self):
proxy_dict = {"server": "http://proxy:8080", "bypass": ".google.com,localhost"}
@@ -68,7 +68,9 @@ class TestBuildProxyKwargs:
assert kwargs == {"proxy": proxy_dict}
assert args == []
def test_proxy_dict_with_auth(self):
@patch("cloakbrowser.config.get_chromium_version", return_value="146.0.7680.177.5")
@patch("cloakbrowser.config.get_platform_tag", return_value="linux-x64")
def test_proxy_dict_with_auth(self, *_):
proxy_dict = {
"server": "http://proxy:8080",
"username": "user",
@@ -76,8 +78,11 @@ class TestBuildProxyKwargs:
"bypass": ".example.com",
}
kwargs, args = _resolve_proxy_config(proxy_dict)
assert kwargs == {"proxy": proxy_dict}
assert args == []
assert kwargs == {}
assert args == [
"--proxy-server=http://user:pass@proxy:8080",
"--proxy-bypass-list=.example.com",
]
class TestMaybeResolveGeoip:
@@ -183,11 +188,12 @@ class TestBareProxyFormat:
r = _parse_proxy_url("proxy:8080")
assert r == {"server": "proxy:8080"}
def test_resolve_proxy_config_bare(self):
@patch("cloakbrowser.config.get_chromium_version", return_value="146.0.7680.177.5")
@patch("cloakbrowser.config.get_platform_tag", return_value="linux-x64")
def test_resolve_proxy_config_bare(self, *_):
kwargs, args = _resolve_proxy_config("user:pass@proxy:8080")
assert kwargs["proxy"]["username"] == "user"
assert kwargs["proxy"]["password"] == "pass"
assert "user" not in kwargs["proxy"]["server"]
assert kwargs == {}
assert args == ["--proxy-server=http://user:pass@proxy:8080"]
class TestIsSocksProxy:
@@ -219,11 +225,17 @@ class TestResolveProxyConfig:
assert kwargs == {}
assert args == []
def test_http_string_returns_playwright_dict(self):
@patch("cloakbrowser.config.get_chromium_version", return_value="146.0.7680.177.5")
@patch("cloakbrowser.config.get_platform_tag", return_value="linux-x64")
def test_http_string_with_creds_returns_chrome_arg(self, *_):
kwargs, args = _resolve_proxy_config("http://user:pass@proxy:8080")
assert kwargs == {}
assert args == ["--proxy-server=http://user:pass@proxy:8080"]
def test_http_string_no_creds_returns_playwright_dict(self):
kwargs, args = _resolve_proxy_config("http://proxy:8080")
assert "proxy" in kwargs
assert kwargs["proxy"]["server"] == "http://proxy:8080"
assert kwargs["proxy"]["username"] == "user"
assert args == []
def test_http_dict_passthrough(self):
@@ -284,6 +296,42 @@ class TestResolveProxyConfig:
_, args = _resolve_proxy_config("socks5://user:pass%3D123@host:1080")
assert args == ["--proxy-server=socks5://user:pass%3D123@host:1080"]
def test_socks5_string_logs_info_when_reencoding(self, caplog):
# When wrapper actually rewrites the URL (e.g. unencoded '=' in pwd),
# surface an INFO log so users debugging SOCKS5 connectivity (#157)
# can see what the wrapper did instead of being silently surprised.
import logging
with caplog.at_level(logging.INFO, logger="cloakbrowser"):
_resolve_proxy_config("socks5://user:pass=123@host:1080")
assert any("Auto URL-encoded SOCKS5" in r.message for r in caplog.records)
# Credentials must not leak into the log.
for r in caplog.records:
assert "pass=123" not in r.message
assert "pass%3D123" not in r.message
def test_socks5_string_silent_when_already_encoded(self, caplog):
# Idempotent path: pre-encoded URL produces no log noise.
import logging
with caplog.at_level(logging.INFO, logger="cloakbrowser"):
_resolve_proxy_config("socks5://user:pass%3D123@host:1080")
assert not any("Auto URL-encoded SOCKS5" in r.message for r in caplog.records)
def test_socks5_string_silent_when_no_credentials(self, caplog):
# No userinfo at all → no encoding work → no log.
import logging
with caplog.at_level(logging.INFO, logger="cloakbrowser"):
_resolve_proxy_config("socks5://host:1080")
assert not any("Auto URL-encoded SOCKS5" in r.message for r in caplog.records)
def test_socks5_string_silent_when_only_cosmetic_change(self, caplog):
# urlparse lowercases scheme and hostname, but credentials are
# untouched. The log must NOT fire for these cosmetic-only rewrites
# (regression for Copilot's review on PR #209).
import logging
with caplog.at_level(logging.INFO, logger="cloakbrowser"):
_resolve_proxy_config("socks5://USER:pass@HOST.com:1080")
assert not any("Auto URL-encoded SOCKS5" in r.message for r in caplog.records)
def test_socks5_string_no_creds_unchanged(self):
_, args = _resolve_proxy_config("socks5://host:1080")
assert args == ["--proxy-server=socks5://host:1080"]
@@ -338,3 +386,95 @@ class TestResolveProxyConfig:
# Port 0 is an unusual but valid URL component; don't silently strip it.
_, args = _resolve_proxy_config("socks5://user:pass=1@host:0")
assert args[0] == "--proxy-server=socks5://user:pass%3D1@host:0"
# --- HTTP with credentials → --proxy-server (supported platforms + version) ---
@patch("cloakbrowser.config.get_chromium_version", return_value="146.0.7680.177.5")
@patch("cloakbrowser.config.get_platform_tag", return_value="linux-x64")
def test_http_string_with_creds_on_supported_platform(self, *_):
kwargs, args = _resolve_proxy_config("http://user:pass@proxy:8080")
assert kwargs == {}
assert args == ["--proxy-server=http://user:pass@proxy:8080"]
@patch("cloakbrowser.config.get_chromium_version", return_value="146.0.7680.177.5")
@patch("cloakbrowser.config.get_platform_tag", return_value="linux-x64")
def test_http_dict_with_creds_on_supported_platform(self, *_):
proxy = {"server": "http://proxy:8080", "username": "user", "password": "pass"}
kwargs, args = _resolve_proxy_config(proxy)
assert kwargs == {}
assert args == ["--proxy-server=http://user:pass@proxy:8080"]
@patch("cloakbrowser.config.get_chromium_version", return_value="146.0.7680.177.5")
@patch("cloakbrowser.config.get_platform_tag", return_value="linux-x64")
def test_http_dict_with_creds_and_bypass(self, *_):
proxy = {
"server": "http://proxy:8080",
"username": "user",
"password": "pass",
"bypass": ".google.com",
}
kwargs, args = _resolve_proxy_config(proxy)
assert kwargs == {}
assert "--proxy-server=http://user:pass@proxy:8080" in args
assert "--proxy-bypass-list=.google.com" in args
@patch("cloakbrowser.config.get_chromium_version", return_value="146.0.7680.177.5")
@patch("cloakbrowser.config.get_platform_tag", return_value="linux-x64")
def test_http_string_encodes_special_chars_in_password(self, *_):
_, args = _resolve_proxy_config("http://user:pass=123@proxy:8080")
assert args == ["--proxy-server=http://user:pass%3D123@proxy:8080"]
@patch("cloakbrowser.config.get_chromium_version", return_value="146.0.7680.177.5")
@patch("cloakbrowser.config.get_platform_tag", return_value="linux-x64")
def test_http_string_encoding_idempotent(self, *_):
_, args = _resolve_proxy_config("http://user:pass%3D123@proxy:8080")
assert args == ["--proxy-server=http://user:pass%3D123@proxy:8080"]
@patch("cloakbrowser.config.get_chromium_version", return_value="146.0.7680.177.5")
@patch("cloakbrowser.config.get_platform_tag", return_value="windows-x64")
def test_http_string_with_creds_on_windows(self, *_):
kwargs, args = _resolve_proxy_config("http://user:pass@proxy:8080")
assert kwargs == {}
assert args == ["--proxy-server=http://user:pass@proxy:8080"]
@patch("cloakbrowser.config.get_chromium_version", return_value="146.0.7680.177.3")
@patch("cloakbrowser.config.get_platform_tag", return_value="linux-x64")
def test_http_with_creds_old_version_falls_back(self, *_):
kwargs, args = _resolve_proxy_config("http://user:pass@proxy:8080")
assert "proxy" in kwargs
assert args == []
# --- HTTP with credentials on unsupported platform → fallback to Playwright ---
@patch("cloakbrowser.config.get_platform_tag", return_value="darwin-arm64")
def test_http_string_with_creds_on_macos_falls_back(self, _mock):
kwargs, args = _resolve_proxy_config("http://user:pass@proxy:8080")
assert "proxy" in kwargs
assert kwargs["proxy"]["username"] == "user"
assert args == []
@patch("cloakbrowser.config.get_platform_tag", return_value="darwin-arm64")
def test_http_dict_with_creds_on_macos_falls_back(self, _mock):
proxy = {"server": "http://proxy:8080", "username": "user", "password": "pass"}
kwargs, args = _resolve_proxy_config(proxy)
assert kwargs == {"proxy": proxy}
assert args == []
@patch("cloakbrowser.config.get_platform_tag", return_value="linux-arm64")
def test_http_string_with_creds_on_linux_arm_falls_back(self, _mock):
kwargs, args = _resolve_proxy_config("http://user:pass@proxy:8080")
assert "proxy" in kwargs
assert args == []
# --- HTTP without credentials (all platforms) ---
def test_http_no_creds_returns_playwright_dict(self):
kwargs, args = _resolve_proxy_config("http://proxy:8080")
assert "proxy" in kwargs
assert args == []
def test_http_dict_no_creds_returns_playwright_dict(self):
proxy = {"server": "http://proxy:8080", "bypass": ".example.com"}
kwargs, args = _resolve_proxy_config(proxy)
assert kwargs == {"proxy": proxy}
assert args == []
+5 -1
View File
@@ -1010,7 +1010,11 @@ class TestPatchPageStealthWiring:
fake_box = {"x": 100, "y": 200, "width": 200, "height": 30}
with mock_patch(
"cloakbrowser.human.scroll_to_element",
return_value=(fake_box, 200.0, 215.0),
return_value=(fake_box, 200.0, 215.0, False),
), mock_patch(
"cloakbrowser.human.ensure_actionable",
), mock_patch(
"cloakbrowser.human.check_pointer_events",
):
try:
page.click("#btn")