mirror of
https://github.com/CloakHQ/CloakBrowser.git
synced 2026-06-23 11:41:46 +02:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
 CloakHQ
|
4c2e06682b | ||
|
CloakHQ
|
cb08a602b0 | ||
|
CloakHQ
|
8eb666885f | ||
|
CloakHQ
|
f417fe2530 | ||
|
CloakHQ
|
c65939af14 | ||
|
CloakHQ
|
2f1f592b3a | ||
|
CloakHQ
|
0bbc170747 | ||
|
CloakHQ
|
6506b5fe44 | ||
|
CloakHQ
|
1082c810af | ||
|
CloakHQ
|
67efadef26 | ||
|
CloakHQ
|
59b9d71684 | ||
|
CloakHQ
|
f480958ba7 |
@@ -40,6 +40,6 @@ jobs:
|
||||
# Binary auto-downloads on first launch
|
||||
```
|
||||
|
||||
> Checksums and platform list will be added after binary uploads.
|
||||
> Binary integrity is verified automatically via SHA-256 checksums on download.
|
||||
>
|
||||
> Release signed with CloakHQ GPG key: `C60C0DDC9D0DE2DD`
|
||||
|
||||
+113
@@ -0,0 +1,113 @@
|
||||
# Changelog
|
||||
|
||||
All notable changes to CloakBrowser — wrapper and binary — are documented here.
|
||||
|
||||
Changes are tagged: **[wrapper]** for Python/JS wrapper, **[binary]** for Chromium patches.
|
||||
|
||||
---
|
||||
|
||||
## [0.3.0] — Unreleased
|
||||
|
||||
Chromium v145 upgrade. 26 fingerprint patches (up from 16). New download verification and fallback system. Pending: macOS v145 binary builds.
|
||||
|
||||
### 2026-03-01
|
||||
|
||||
- **[wrapper]** Upgrade wrapper to Chromium v145.0.7632.109
|
||||
- **[wrapper]** Add GitHub Releases fallback when primary download mirror is unavailable
|
||||
- **[wrapper]** Add SHA-256 checksum verification for binary downloads
|
||||
- **[wrapper]** Wire timezone and locale params to Chromium binary flags
|
||||
- **[wrapper]** Add device memory to default stealth args
|
||||
- **[wrapper]** JS: add `colorScheme` support, guard download fallback against partial failures
|
||||
|
||||
### 2026-02-28
|
||||
|
||||
- **[binary]** Enforce strict flag discipline — patches only activate when explicitly configured via command-line flags
|
||||
- **[binary]** Improved fingerprint consistency across multiple browser APIs
|
||||
- **[binary]** 3 new fingerprint patches + bug fixes in existing patches
|
||||
- **[binary]** New command-line flag for device memory spoofing
|
||||
- **[infra]** Automated test matrix: 8 groups, 41+ tests across core stealth, fingerprint noise, bot detection, reCAPTCHA, TLS, Turnstile, residential proxy, and enterprise reCAPTCHA
|
||||
- **[infra]** Docker-based test runner with subprocess isolation per test group
|
||||
|
||||
### 2026-02-25
|
||||
|
||||
- **[binary]** Reduced automation markers visible to detection scripts
|
||||
- **[binary]** Added browser API support at build time
|
||||
- **[binary]** Improved screen property consistency
|
||||
|
||||
### 2026-02-24
|
||||
|
||||
- **[binary]** Comprehensive fingerprint audit and hardening pass
|
||||
- **[binary]** Fixed font rendering edge case on cross-platform spoofing
|
||||
- **[binary]** 4 new fingerprint patches
|
||||
|
||||
### 2026-02-22
|
||||
|
||||
- **[binary]** Start Chromium v145 build (v145.0.7632.109)
|
||||
- **[binary]** 24 fingerprint patches ported and adapted
|
||||
|
||||
---
|
||||
|
||||
## [0.2.2] — 2026-03-01
|
||||
|
||||
### 2026-03-01
|
||||
|
||||
- **[wrapper]** Fix: replace `page.wait_for_timeout()` with `time.sleep()` to avoid timing leak
|
||||
- **[wrapper]** Add auto-detect timezone and locale from proxy IP via GeoIP lookup
|
||||
- **[binary]** CDP detection vector audit and hardening
|
||||
|
||||
---
|
||||
|
||||
## [0.2.0] — 2026-02-27
|
||||
|
||||
macOS platform release. JavaScript/TypeScript wrapper. Self-hosted binary mirror.
|
||||
|
||||
### 2026-02-27
|
||||
|
||||
- **[wrapper]** Add macOS support: Apple Silicon (arm64) and Intel (x64) binary downloads
|
||||
- **[wrapper]** Add GPG-signed release workflow via GitHub Actions
|
||||
- **[wrapper]** Fix macOS binary download: preserve `.app` symlinks, remove quarantine xattrs
|
||||
- **[wrapper]** Add real bot detection assertions to stealth tests
|
||||
- **[wrapper]** Bump version to 0.2.0
|
||||
|
||||
### 2026-02-26
|
||||
|
||||
- **[wrapper]** Switch binary downloads to self-hosted mirror (`cloakbrowser.dev`) as GitHub backup
|
||||
- **[wrapper]** Set up GitLab mirror at `gitlab.com/CloakHQ/cloakbrowser`
|
||||
|
||||
### 2026-02-25
|
||||
|
||||
- **[wrapper]** Move binary releases from separate repo to wrapper repo
|
||||
- **[wrapper]** Add auto-update check on launch
|
||||
- **[infra]** Initial Docker test infrastructure + matrix test runner
|
||||
|
||||
### 2026-02-24
|
||||
|
||||
- **[wrapper]** Add JavaScript/TypeScript wrapper with Playwright + Puppeteer support (`npm install cloakbrowser`)
|
||||
- **[wrapper]** Fix proxy authentication credentials support in URL (closes #4)
|
||||
|
||||
---
|
||||
|
||||
## [0.1.4] — 2026-02-23
|
||||
|
||||
### 2026-02-23
|
||||
|
||||
- **[wrapper]** Stealth hardening: additional launch args and detection evasion improvements
|
||||
- **[wrapper]** Full test suite rewrite with real detection site assertions
|
||||
- **[wrapper]** Add Docker support with Dockerfile and compose config
|
||||
- **[wrapper]** Add headed mode documentation
|
||||
|
||||
---
|
||||
|
||||
## [0.1.0] — 2026-02-22
|
||||
|
||||
Initial release. Chromium v142 with 16 fingerprint patches.
|
||||
|
||||
### 2026-02-22
|
||||
|
||||
- **[binary]** Chromium v142.0.7444.175 with 16 source-level fingerprint patches
|
||||
- **[binary]** Fix browser brand string to match Chrome 142 format
|
||||
- **[wrapper]** `launch()` and `launch_async()` — drop-in Playwright replacements
|
||||
- **[wrapper]** Auto-download binary from GitHub Releases, cached in `~/.cloakbrowser/`
|
||||
- **[wrapper]** Linux x64 platform support
|
||||
- **[wrapper]** Passes 14/14 bot detection tests
|
||||
- **[wrapper]** reCAPTCHA v3: 0.9 (server-verified), Cloudflare Turnstile: pass
|
||||
@@ -11,8 +11,8 @@
|
||||
<a href="LICENSE"><img src="https://img.shields.io/github/license/CloakHQ/CloakBrowser" alt="License"></a>
|
||||
<br>
|
||||
<a href="https://github.com/CloakHQ/CloakBrowser"><img src="https://img.shields.io/github/stars/CloakHQ/CloakBrowser" alt="Stars"></a>
|
||||
<a href="https://pypi.org/project/cloakbrowser/"><img src="https://img.shields.io/pypi/dm/cloakbrowser" alt="PyPI Downloads"></a>
|
||||
<a href="https://www.npmjs.com/package/cloakbrowser"><img src="https://img.shields.io/npm/dm/cloakbrowser" alt="npm Downloads"></a>
|
||||
<a href="https://pepy.tech/projects/cloakbrowser"><img src="https://img.shields.io/pepy/dt/cloakbrowser?label=pypi&logo=pypi&logoColor=white" alt="PyPI Downloads"></a>
|
||||
<a href="https://www.npmjs.com/package/cloakbrowser"><img src="https://img.shields.io/npm/dt/cloakbrowser?label=npm&logo=npm&logoColor=white" alt="npm Downloads"></a>
|
||||
<a href="https://github.com/CloakHQ/CloakBrowser"><img src="https://img.shields.io/github/last-commit/CloakHQ/CloakBrowser" alt="Last Commit"></a>
|
||||
</p>
|
||||
|
||||
@@ -20,7 +20,8 @@
|
||||
|
||||
Drop-in Playwright/Puppeteer replacement for Python and JavaScript. Same API, same code — just swap the import. Your browser now scores **0.9 on reCAPTCHA v3**, passes **Cloudflare Turnstile**, and clears **30 out of 30** stealth detection tests.
|
||||
|
||||
- 🔒 **16 source-level C++ patches** — not JS injection, not config flags
|
||||
- 🔒 **26 source-level C++ patches** — not JS injection, not config flags
|
||||
- 🛡️ **CDP stealth built-in** — powered by [Patchright](https://github.com/Kaliiiiiiiiii-Vinyzu/patchright), hides Playwright's automation signals
|
||||
- 🎯 **0.9 reCAPTCHA v3 score** — human-level, server-verified
|
||||
- ☁️ **Passes Cloudflare Turnstile**, FingerprintJS, BrowserScan — 30/30 tests
|
||||
- 🔄 **Drop-in replacement** — works with Playwright (Python & JS) and Puppeteer (JS)
|
||||
@@ -75,15 +76,27 @@ npm install cloakbrowser puppeteer-core
|
||||
|
||||
On first run, the stealth Chromium binary is automatically downloaded (~200MB, cached locally).
|
||||
|
||||
## What's New in v0.3.0
|
||||
|
||||
- **Chromium 145** — latest stable, 26 fingerprint patches
|
||||
- **10 new patches** — screen dimensions, audio, WebGL, and more
|
||||
- **CDP hardening** — audited and patched known automation detection vectors
|
||||
- **Timezone & locale from proxy IP** — `launch(proxy="...", geoip=True)` auto-detects timezone and locale
|
||||
- **Improved cross-platform spoofing** — fixed edge cases in font rendering and GPU reporting
|
||||
- **Automated test matrix** — 41+ tests across 8 groups (stealth, fingerprint, reCAPTCHA, Turnstile, TLS, enterprise) running in Docker
|
||||
|
||||
See the full [CHANGELOG.md](CHANGELOG.md) for details.
|
||||
|
||||
## Why CloakBrowser?
|
||||
|
||||
- **Config-level patches break** — `playwright-stealth`, `undetected-chromedriver`, and `puppeteer-extra` inject JavaScript or tweak flags. Every Chrome update breaks them. Antibot systems detect the patches themselves.
|
||||
- **CloakBrowser patches Chromium source code** — fingerprints are modified at the C++ level, compiled into the binary. Detection sites see a real browser because it *is* a real browser.
|
||||
- **Two layers of stealth** — C++ patches handle fingerprints (GPU, screen, UA, hardware reporting), while the Patchright driver eliminates CDP automation leaks. Most stealth tools only do one or the other.
|
||||
- **One line to switch** — same Playwright API, no new abstractions, no CAPTCHA-solving services.
|
||||
|
||||
## Test Results
|
||||
|
||||
All tests verified against live detection services. Last tested: Feb 2026 (Chromium 142).
|
||||
All tests verified against live detection services. Last tested: Feb 2026 (Chromium 145).
|
||||
|
||||
| Detection Service | Stock Playwright | CloakBrowser | Notes |
|
||||
|---|---|---|---|
|
||||
@@ -98,7 +111,7 @@ All tests verified against live detection services. Last tested: Feb 2026 (Chrom
|
||||
| `navigator.webdriver` | `true` | **`false`** | Source-level patch |
|
||||
| `navigator.plugins.length` | 0 | **5** | Real plugin list |
|
||||
| `window.chrome` | `undefined` | **`object`** | Present like real Chrome |
|
||||
| UA string | `HeadlessChrome` | **`Chrome/142.0.0.0`** | No headless leak |
|
||||
| UA string | `HeadlessChrome` | **`Chrome/145.0.0.0`** | No headless leak |
|
||||
| CDP detection | Detected | **Not detected** | `isAutomatedWithCDP: false` |
|
||||
| TLS fingerprint | Mismatch | **Identical to Chrome** | ja3n/ja4/akamai match |
|
||||
|
||||
@@ -140,17 +153,7 @@ CloakBrowser is a thin wrapper (Python + JavaScript) around a custom-built Chrom
|
||||
3. **Every launch** → Playwright or Puppeteer starts with our binary + stealth args
|
||||
4. **You write code** → standard Playwright/Puppeteer API, nothing new to learn
|
||||
|
||||
The binary includes 16 source-level patches that modify:
|
||||
- Canvas fingerprint generation
|
||||
- WebGL renderer output
|
||||
- Audio processing fingerprint
|
||||
- Font enumeration results
|
||||
- Hardware concurrency reporting
|
||||
- Client rect measurements
|
||||
- GPU vendor/renderer strings
|
||||
- WebDriver flag
|
||||
- Headless detection signals
|
||||
- And more...
|
||||
The binary includes 26 source-level patches covering canvas, WebGL, audio, fonts, GPU, screen properties, hardware reporting, and automation signal removal.
|
||||
|
||||
These are compiled into the Chromium binary — not injected via JavaScript, not set via flags.
|
||||
|
||||
@@ -173,6 +176,12 @@ browser = launch(proxy="http://user:pass@proxy:8080")
|
||||
# With extra Chrome args
|
||||
browser = launch(args=["--disable-gpu", "--window-size=1920,1080"])
|
||||
|
||||
# With timezone and locale (sets both binary flags and Playwright context)
|
||||
browser = launch(timezone="America/New_York", locale="en-US")
|
||||
|
||||
# Auto-detect timezone/locale from proxy IP (requires: pip install cloakbrowser[geoip])
|
||||
browser = launch(proxy="http://proxy:8080", geoip=True)
|
||||
|
||||
# Without default stealth args (bring your own fingerprint flags)
|
||||
browser = launch(stealth_args=False, args=["--fingerprint=12345"])
|
||||
```
|
||||
@@ -211,6 +220,27 @@ context = launch_context(
|
||||
page = context.new_page()
|
||||
```
|
||||
|
||||
### Auto Timezone/Locale from Proxy IP
|
||||
|
||||
When using a proxy, antibot systems check that your browser's timezone and locale match the proxy's geographic location. CloakBrowser can auto-detect these from the proxy IP using an offline GeoIP database:
|
||||
|
||||
```bash
|
||||
pip install cloakbrowser[geoip] # installs geoip2 + downloads ~70 MB database on first use
|
||||
```
|
||||
|
||||
```python
|
||||
# Timezone and locale auto-set from proxy's IP geolocation
|
||||
browser = launch(proxy="http://proxy:8080", geoip=True)
|
||||
|
||||
# Works with launch_context too — sets both binary flags AND Playwright context
|
||||
context = launch_context(proxy="http://proxy:8080", geoip=True)
|
||||
|
||||
# Explicit values always win over auto-detection
|
||||
browser = launch(proxy="http://proxy:8080", geoip=True, timezone="Europe/London")
|
||||
```
|
||||
|
||||
> **Note:** For rotating residential proxies, the DNS-resolved IP may differ from the exit IP. Pass explicit `timezone`/`locale` in those cases.
|
||||
|
||||
### Utility Functions
|
||||
|
||||
```python
|
||||
@@ -299,6 +329,14 @@ clearCache();
|
||||
|
||||
Every launch automatically generates a **unique fingerprint**. A random seed (10000–99999) drives all seed-based patches — canvas, WebGL, audio, fonts, and client rects all produce consistent, correlated values derived from that single seed.
|
||||
|
||||
> **Tip: Use a fixed seed when revisiting the same site.** A random seed makes every session look like a different device — which can be suspicious when hitting the same site repeatedly from the same IP. For reCAPTCHA v3 Enterprise and similar scoring systems, a fixed seed produces a consistent fingerprint across sessions, making you look like a returning visitor:
|
||||
> ```python
|
||||
> browser = launch(args=["--fingerprint=12345"])
|
||||
> ```
|
||||
> ```javascript
|
||||
> const browser = await launch({ args: ['--fingerprint=12345'] });
|
||||
> ```
|
||||
|
||||
### Default Fingerprint
|
||||
|
||||
Every `launch()` call sets these automatically. Defaults are **platform-aware** — macOS runs as a native Mac browser, Linux spoofs Windows:
|
||||
@@ -403,6 +441,8 @@ page.goto("https://example.com")
|
||||
| macOS x86_64 (Intel) | ✅ Available |
|
||||
| Windows | Planned |
|
||||
|
||||
**macOS (early access):** macOS builds are new — tested but not yet battle-tested at scale like Linux. If you hit any issues, [please open a GitHub issue](https://github.com/CloakHQ/CloakBrowser/issues).
|
||||
|
||||
**macOS first launch:** The binary is ad-hoc signed. On first run, macOS Gatekeeper will block it. Right-click the app → **Open** → click **Open** in the dialog. This is only needed once.
|
||||
|
||||
**On Windows?** You can still use CloakBrowser via Docker or with your own Chromium binary by setting `CLOAKBROWSER_BINARY_PATH=/path/to/chrome`.
|
||||
@@ -425,13 +465,13 @@ page.goto("https://example.com")
|
||||
| Linux x64 binary | ✅ Released |
|
||||
| macOS arm64 (Apple Silicon) | ✅ Released |
|
||||
| macOS x64 (Intel) | ✅ Released |
|
||||
| Chromium 145 build | 🔜 In progress |
|
||||
| Chromium 145 build (26 patches) | 🔧 In progress |
|
||||
| JavaScript/Puppeteer + Playwright support | ✅ Released |
|
||||
| Fingerprint rotation per session | ✅ Released |
|
||||
| Built-in proxy rotation | 📋 Planned |
|
||||
| Windows support | 📋 Planned |
|
||||
|
||||
> ⭐ **Star this repo** to get notified when Chromium 145 and Windows builds drop.
|
||||
> ⭐ **Star this repo** to get notified when new builds drop.
|
||||
|
||||
## Docker
|
||||
|
||||
@@ -486,18 +526,71 @@ This runs a real headed browser rendered on a virtual display — no physical mo
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**Reddit or similar sites show CAPTCHA / "Prove your humanity"**
|
||||
|
||||
Some sites (notably Reddit homepage) use HTTP/2 fingerprinting that detects Playwright's connection layer. Pass `--disable-http2` to fall back to HTTP/1.1:
|
||||
|
||||
```python
|
||||
browser = launch(args=["--disable-http2"])
|
||||
```
|
||||
|
||||
```javascript
|
||||
const browser = await launch({ args: ['--disable-http2'] });
|
||||
```
|
||||
|
||||
Only use this flag for sites that require it — most sites work fine with HTTP/2.
|
||||
|
||||
**Binary download fails / timeout**
|
||||
Set a custom download URL or use a local binary:
|
||||
```bash
|
||||
export CLOAKBROWSER_BINARY_PATH=/path/to/your/chrome
|
||||
```
|
||||
|
||||
**macOS: "App is damaged" or Gatekeeper blocks launch**
|
||||
The binary is ad-hoc signed. macOS quarantines downloaded files. Run once to clear it:
|
||||
```bash
|
||||
xattr -cr ~/.cloakbrowser/chromium-*/Chromium.app
|
||||
```
|
||||
|
||||
**"playwright install" vs CloakBrowser binary**
|
||||
You do NOT need `playwright install chromium`. CloakBrowser downloads its own binary. You only need Playwright's system deps:
|
||||
```bash
|
||||
playwright install-deps chromium
|
||||
patchright install-deps chromium
|
||||
```
|
||||
|
||||
**reCAPTCHA v3 scores are low (0.1–0.3)**
|
||||
|
||||
Avoid `page.wait_for_timeout()` — it sends CDP protocol commands that reCAPTCHA detects. Use native sleep instead:
|
||||
|
||||
```python
|
||||
# Bad — sends CDP commands, reCAPTCHA detects this
|
||||
page.wait_for_timeout(3000)
|
||||
|
||||
# Good — invisible to the browser
|
||||
import time
|
||||
time.sleep(3)
|
||||
```
|
||||
|
||||
```javascript
|
||||
// Bad — sends CDP commands
|
||||
await page.waitForTimeout(3000);
|
||||
|
||||
// Good — invisible to the browser
|
||||
await new Promise(r => setTimeout(r, 3000));
|
||||
```
|
||||
|
||||
Other tips for maximizing reCAPTCHA scores:
|
||||
- **Use Playwright, not Puppeteer** — Puppeteer sends more CDP protocol traffic that reCAPTCHA detects ([details](#puppeteer))
|
||||
- **Use residential proxies** — datacenter IPs are flagged by IP reputation, not browser fingerprint
|
||||
- **Spend 15+ seconds on the page** before triggering reCAPTCHA — short visits score lower
|
||||
- **Space out requests** — back-to-back `grecaptcha.execute()` calls from the same session get penalized. Wait 30+ seconds between pages with reCAPTCHA
|
||||
- **Use a fixed fingerprint seed** (`--fingerprint=12345`) for consistent device identity across sessions
|
||||
- **Use `page.type()` instead of `page.fill()`** for form filling — `fill()` sets values directly without keyboard events, which reCAPTCHA's behavioral analysis flags. `type()` with a delay simulates real keystrokes:
|
||||
```python
|
||||
page.type("#email", "user@example.com", delay=50)
|
||||
```
|
||||
- **Minimize `page.evaluate()` calls** before the reCAPTCHA check fires — each one sends CDP traffic
|
||||
|
||||
## FAQ
|
||||
|
||||
**Q: Is this legal?**
|
||||
@@ -517,6 +610,7 @@ A: Yes. A ready-to-use Dockerfile is included — see the [Docker](#docker) sect
|
||||
|
||||
## Links
|
||||
|
||||
- 📋 **Changelog** — [CHANGELOG.md](CHANGELOG.md)
|
||||
- 🌐 **Website** — [cloakbrowser.dev](https://cloakbrowser.dev)
|
||||
- 🐛 **Bug reports & feature requests** — [GitHub Issues](https://github.com/CloakHQ/CloakBrowser/issues)
|
||||
- 📦 **PyPI** — [pypi.org/project/cloakbrowser](https://pypi.org/project/cloakbrowser/)
|
||||
|
||||
@@ -1 +1 @@
|
||||
__version__ = "0.2.0"
|
||||
__version__ = "0.3.0"
|
||||
|
||||
+69
-11
@@ -15,10 +15,10 @@ Usage:
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from typing import Any
|
||||
from typing import Any, Literal
|
||||
from urllib.parse import unquote, urlparse, urlunparse
|
||||
|
||||
from .config import get_default_stealth_args
|
||||
from .config import DEFAULT_VIEWPORT, get_default_stealth_args
|
||||
from .download import ensure_binary
|
||||
|
||||
logger = logging.getLogger("cloakbrowser")
|
||||
@@ -29,6 +29,9 @@ def launch(
|
||||
proxy: str | None = None,
|
||||
args: list[str] | None = None,
|
||||
stealth_args: bool = True,
|
||||
timezone: str | None = None,
|
||||
locale: str | None = None,
|
||||
geoip: bool = False,
|
||||
**kwargs: Any,
|
||||
) -> Any:
|
||||
"""Launch stealth Chromium browser. Returns a Playwright Browser object.
|
||||
@@ -39,6 +42,12 @@ def launch(
|
||||
args: Additional Chromium CLI arguments to pass.
|
||||
stealth_args: Include default stealth fingerprint args (default True).
|
||||
Set to False if you want to pass your own --fingerprint flags.
|
||||
timezone: IANA timezone (e.g. 'America/New_York'). Sets --timezone binary flag.
|
||||
locale: BCP 47 locale (e.g. 'en-US'). Sets --lang binary flag.
|
||||
geoip: Auto-detect timezone/locale from proxy IP (default False).
|
||||
Requires ``pip install cloakbrowser[geoip]``. Downloads ~70 MB
|
||||
GeoLite2-City database on first use. Explicit timezone/locale
|
||||
always override geoip results.
|
||||
**kwargs: Passed directly to playwright.chromium.launch().
|
||||
|
||||
Returns:
|
||||
@@ -52,10 +61,11 @@ def launch(
|
||||
>>> print(page.title())
|
||||
>>> browser.close()
|
||||
"""
|
||||
from playwright.sync_api import sync_playwright
|
||||
from patchright.sync_api import sync_playwright
|
||||
|
||||
binary_path = ensure_binary()
|
||||
chrome_args = _build_args(stealth_args, args)
|
||||
timezone, locale = _maybe_resolve_geoip(geoip, proxy, timezone, locale)
|
||||
chrome_args = _build_args(stealth_args, args, timezone=timezone, locale=locale)
|
||||
|
||||
logger.debug("Launching stealth Chromium (headless=%s, args=%d)", headless, len(chrome_args))
|
||||
|
||||
@@ -86,6 +96,9 @@ async def launch_async(
|
||||
proxy: str | None = None,
|
||||
args: list[str] | None = None,
|
||||
stealth_args: bool = True,
|
||||
timezone: str | None = None,
|
||||
locale: str | None = None,
|
||||
geoip: bool = False,
|
||||
**kwargs: Any,
|
||||
) -> Any:
|
||||
"""Async version of launch(). Returns a Playwright Browser object.
|
||||
@@ -95,6 +108,9 @@ async def launch_async(
|
||||
proxy: Proxy server URL (e.g. 'http://proxy:8080' or 'socks5://proxy:1080').
|
||||
args: Additional Chromium CLI arguments to pass.
|
||||
stealth_args: Include default stealth fingerprint args (default True).
|
||||
timezone: IANA timezone (e.g. 'America/New_York'). Sets --timezone binary flag.
|
||||
locale: BCP 47 locale (e.g. 'en-US'). Sets --lang binary flag.
|
||||
geoip: Auto-detect timezone/locale from proxy IP (default False).
|
||||
**kwargs: Passed directly to playwright.chromium.launch().
|
||||
|
||||
Returns:
|
||||
@@ -113,10 +129,11 @@ async def launch_async(
|
||||
>>>
|
||||
>>> asyncio.run(main())
|
||||
"""
|
||||
from playwright.async_api import async_playwright
|
||||
from patchright.async_api import async_playwright
|
||||
|
||||
binary_path = ensure_binary()
|
||||
chrome_args = _build_args(stealth_args, args)
|
||||
timezone, locale = _maybe_resolve_geoip(geoip, proxy, timezone, locale)
|
||||
chrome_args = _build_args(stealth_args, args, timezone=timezone, locale=locale)
|
||||
|
||||
logger.debug("Launching stealth Chromium async (headless=%s, args=%d)", headless, len(chrome_args))
|
||||
|
||||
@@ -151,6 +168,8 @@ def launch_context(
|
||||
viewport: dict | None = None,
|
||||
locale: str | None = None,
|
||||
timezone_id: str | None = None,
|
||||
color_scheme: Literal["light", "dark", "no-preference"] | None = None,
|
||||
geoip: bool = False,
|
||||
**kwargs: Any,
|
||||
) -> Any:
|
||||
"""Launch stealth browser and return a BrowserContext with common options pre-set.
|
||||
@@ -167,22 +186,31 @@ def launch_context(
|
||||
viewport: Viewport size dict, e.g. {"width": 1920, "height": 1080}.
|
||||
locale: Browser locale, e.g. "en-US".
|
||||
timezone_id: Timezone, e.g. "America/New_York".
|
||||
color_scheme: Color scheme preference — 'light', 'dark', or 'no-preference'.
|
||||
Default: None (uses Chromium default, which is 'light').
|
||||
Note: 'no-preference' doesn't work in Patchright (falls back to 'light').
|
||||
geoip: Auto-detect timezone/locale from proxy IP (default False).
|
||||
**kwargs: Passed to browser.new_context().
|
||||
|
||||
Returns:
|
||||
Playwright BrowserContext object.
|
||||
"""
|
||||
browser = launch(headless=headless, proxy=proxy, args=args, stealth_args=stealth_args)
|
||||
# Resolve geoip BEFORE launch() to avoid double-resolution and ensure
|
||||
# resolved values flow to both binary flags AND context params
|
||||
timezone_id, locale = _maybe_resolve_geoip(geoip, proxy, timezone_id, locale)
|
||||
browser = launch(headless=headless, proxy=proxy, args=args, stealth_args=stealth_args,
|
||||
timezone=timezone_id, locale=locale)
|
||||
|
||||
context_kwargs: dict[str, Any] = {}
|
||||
if user_agent:
|
||||
context_kwargs["user_agent"] = user_agent
|
||||
if viewport:
|
||||
context_kwargs["viewport"] = viewport
|
||||
context_kwargs["viewport"] = viewport or DEFAULT_VIEWPORT
|
||||
if locale:
|
||||
context_kwargs["locale"] = locale
|
||||
if timezone_id:
|
||||
context_kwargs["timezone_id"] = timezone_id
|
||||
if color_scheme:
|
||||
context_kwargs["color_scheme"] = color_scheme
|
||||
context_kwargs.update(kwargs)
|
||||
|
||||
try:
|
||||
@@ -208,13 +236,43 @@ def launch_context(
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _build_args(stealth_args: bool, extra_args: list[str] | None) -> list[str]:
|
||||
"""Combine stealth args with user-provided args."""
|
||||
def _maybe_resolve_geoip(
|
||||
geoip: bool,
|
||||
proxy: str | None,
|
||||
timezone: str | None,
|
||||
locale: str | None,
|
||||
) -> tuple[str | None, str | None]:
|
||||
"""Auto-fill timezone/locale from proxy IP when geoip is enabled."""
|
||||
if not geoip or not proxy or (timezone is not None and locale is not None):
|
||||
return timezone, locale
|
||||
|
||||
from .geoip import resolve_proxy_geo
|
||||
|
||||
geo_tz, geo_locale = resolve_proxy_geo(proxy)
|
||||
if timezone is None:
|
||||
timezone = geo_tz
|
||||
if locale is None:
|
||||
locale = geo_locale
|
||||
return timezone, locale
|
||||
|
||||
|
||||
def _build_args(
|
||||
stealth_args: bool,
|
||||
extra_args: list[str] | None,
|
||||
timezone: str | None = None,
|
||||
locale: str | None = None,
|
||||
) -> list[str]:
|
||||
"""Combine stealth args with user-provided args and locale flags."""
|
||||
result = []
|
||||
if stealth_args:
|
||||
result.extend(get_default_stealth_args())
|
||||
if extra_args:
|
||||
result.extend(extra_args)
|
||||
# Timezone/locale flags are independent of stealth_args — always inject when set
|
||||
if timezone:
|
||||
result.append(f"--timezone={timezone}")
|
||||
if locale:
|
||||
result.append(f"--lang={locale}")
|
||||
return result
|
||||
|
||||
|
||||
|
||||
+25
-1
@@ -12,7 +12,7 @@ from ._version import __version__
|
||||
# ---------------------------------------------------------------------------
|
||||
# Chromium version shipped with this release
|
||||
# ---------------------------------------------------------------------------
|
||||
CHROMIUM_VERSION = "142.0.7444.175"
|
||||
CHROMIUM_VERSION = "145.0.7632.109"
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Default stealth arguments passed to the patched Chromium binary.
|
||||
@@ -43,10 +43,23 @@ def get_default_stealth_args() -> list[str]:
|
||||
return base + [
|
||||
"--fingerprint-platform=windows",
|
||||
"--fingerprint-hardware-concurrency=8",
|
||||
"--fingerprint-device-memory=8",
|
||||
"--fingerprint-gpu-vendor=NVIDIA Corporation",
|
||||
"--fingerprint-gpu-renderer=NVIDIA GeForce RTX 3070",
|
||||
"--fingerprint-taskbar-height=40",
|
||||
"--fingerprint-screen-width=1920",
|
||||
"--fingerprint-screen-height=1080",
|
||||
"--window-size=1920,1080",
|
||||
]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Default viewport — realistic maximized Chrome on 1080p Windows
|
||||
# screen=1920x1080, availHeight=1040 (minus 40px taskbar),
|
||||
# innerHeight=955 (minus ~85px Chrome UI: tabs + address bar + bookmarks)
|
||||
# ---------------------------------------------------------------------------
|
||||
DEFAULT_VIEWPORT = {"width": 1920, "height": 955}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Platform detection
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -167,6 +180,10 @@ DOWNLOAD_BASE_URL = os.environ.get(
|
||||
|
||||
GITHUB_API_URL = "https://api.github.com/repos/CloakHQ/cloakbrowser/releases"
|
||||
|
||||
GITHUB_DOWNLOAD_BASE_URL = (
|
||||
"https://github.com/CloakHQ/cloakbrowser/releases/download"
|
||||
)
|
||||
|
||||
|
||||
def get_download_url(version: str | None = None) -> str:
|
||||
"""Return the full download URL for the current platform's binary archive."""
|
||||
@@ -175,6 +192,13 @@ def get_download_url(version: str | None = None) -> str:
|
||||
return f"{DOWNLOAD_BASE_URL}/chromium-v{v}/cloakbrowser-{tag}.tar.gz"
|
||||
|
||||
|
||||
def get_fallback_download_url(version: str | None = None) -> str:
|
||||
"""Return the GitHub Releases fallback URL for the binary archive."""
|
||||
v = version or CHROMIUM_VERSION
|
||||
tag = get_platform_tag()
|
||||
return f"{GITHUB_DOWNLOAD_BASE_URL}/chromium-v{v}/cloakbrowser-{tag}.tar.gz"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Local binary override (skip download, use your own build)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -6,6 +6,7 @@ Similar to how Playwright downloads its own bundled Chromium.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import logging
|
||||
import os
|
||||
import platform
|
||||
@@ -23,6 +24,7 @@ from .config import (
|
||||
CHROMIUM_VERSION,
|
||||
DOWNLOAD_BASE_URL,
|
||||
GITHUB_API_URL,
|
||||
GITHUB_DOWNLOAD_BASE_URL,
|
||||
_version_newer,
|
||||
check_platform_available,
|
||||
get_binary_dir,
|
||||
@@ -30,6 +32,7 @@ from .config import (
|
||||
get_cache_dir,
|
||||
get_download_url,
|
||||
get_effective_version,
|
||||
get_fallback_download_url,
|
||||
get_local_binary_override,
|
||||
get_platform_tag,
|
||||
)
|
||||
@@ -102,8 +105,14 @@ def ensure_binary() -> str:
|
||||
|
||||
|
||||
def _download_and_extract(version: str | None = None) -> None:
|
||||
"""Download the binary archive and extract to cache directory."""
|
||||
url = get_download_url(version)
|
||||
"""Download the binary archive and extract to cache directory.
|
||||
|
||||
Tries the primary server (cloakbrowser.dev) first, falls back to
|
||||
GitHub Releases if the primary is unreachable or returns an error.
|
||||
Verifies SHA-256 checksum before extraction when available.
|
||||
"""
|
||||
primary_url = get_download_url(version)
|
||||
fallback_url = get_fallback_download_url(version)
|
||||
binary_dir = get_binary_dir(version)
|
||||
binary_path = get_binary_path(version)
|
||||
|
||||
@@ -115,7 +124,22 @@ def _download_and_extract(version: str | None = None) -> None:
|
||||
tmp_path = Path(tmp.name)
|
||||
|
||||
try:
|
||||
_download_file(url, tmp_path)
|
||||
# Try primary, fall back to GitHub Releases (skip fallback if custom URL)
|
||||
try:
|
||||
_download_file(primary_url, tmp_path)
|
||||
except Exception as primary_err:
|
||||
if os.environ.get("CLOAKBROWSER_DOWNLOAD_URL"):
|
||||
raise
|
||||
logger.warning(
|
||||
"Primary download failed (%s), trying GitHub Releases...",
|
||||
primary_err,
|
||||
)
|
||||
_download_file(fallback_url, tmp_path)
|
||||
|
||||
# Verify checksum before extraction
|
||||
if os.environ.get("CLOAKBROWSER_SKIP_CHECKSUM", "").lower() != "true":
|
||||
_verify_download_checksum(tmp_path, version)
|
||||
|
||||
_extract_archive(tmp_path, binary_dir, binary_path)
|
||||
logger.info("Visit https://cloakbrowser.dev for docs and release notifications.")
|
||||
logger.info("Issues? https://github.com/CloakHQ/CloakBrowser/issues")
|
||||
@@ -125,6 +149,76 @@ def _download_and_extract(version: str | None = None) -> None:
|
||||
tmp_path.unlink(missing_ok=True)
|
||||
|
||||
|
||||
def _verify_download_checksum(file_path: Path, version: str | None = None) -> None:
|
||||
"""Fetch SHA256SUMS and verify the downloaded file. Warn if unavailable, fail on mismatch."""
|
||||
checksums = _fetch_checksums(version)
|
||||
tarball_name = f"cloakbrowser-{get_platform_tag()}.tar.gz"
|
||||
|
||||
if checksums is None:
|
||||
logger.warning("SHA256SUMS not available for this release — skipping checksum verification")
|
||||
return
|
||||
|
||||
expected = checksums.get(tarball_name)
|
||||
if expected is None:
|
||||
logger.warning("SHA256SUMS found but no entry for %s — skipping verification", tarball_name)
|
||||
return
|
||||
|
||||
_verify_checksum(file_path, expected)
|
||||
|
||||
|
||||
def _fetch_checksums(version: str | None = None) -> dict[str, str] | None:
|
||||
"""Fetch SHA256SUMS file for a version. Returns {filename: hash} or None."""
|
||||
v = version or CHROMIUM_VERSION
|
||||
has_custom_url = os.environ.get("CLOAKBROWSER_DOWNLOAD_URL")
|
||||
|
||||
# Build URL list — respect custom URL contract (no GitHub fallback)
|
||||
urls = [f"{DOWNLOAD_BASE_URL}/chromium-v{v}/SHA256SUMS"]
|
||||
if not has_custom_url:
|
||||
urls.append(f"{GITHUB_DOWNLOAD_BASE_URL}/chromium-v{v}/SHA256SUMS")
|
||||
|
||||
for url in urls:
|
||||
try:
|
||||
resp = httpx.get(url, follow_redirects=True, timeout=10.0)
|
||||
resp.raise_for_status()
|
||||
return _parse_checksums(resp.text)
|
||||
except Exception:
|
||||
continue
|
||||
return None
|
||||
|
||||
|
||||
def _parse_checksums(text: str) -> dict[str, str]:
|
||||
"""Parse SHA256SUMS format: 'hash filename' per line."""
|
||||
result = {}
|
||||
for line in text.strip().splitlines():
|
||||
line = line.strip()
|
||||
if not line:
|
||||
continue
|
||||
parts = line.split(None, 1)
|
||||
if len(parts) == 2:
|
||||
hash_val, filename = parts
|
||||
filename = filename.lstrip("*")
|
||||
result[filename] = hash_val.lower()
|
||||
return result
|
||||
|
||||
|
||||
def _verify_checksum(file_path: Path, expected_hash: str) -> None:
|
||||
"""Verify SHA-256 of a file. Raises RuntimeError on mismatch."""
|
||||
sha256 = hashlib.sha256()
|
||||
with open(file_path, "rb") as f:
|
||||
for chunk in iter(lambda: f.read(8192), b""):
|
||||
sha256.update(chunk)
|
||||
actual = sha256.hexdigest().lower()
|
||||
if actual != expected_hash:
|
||||
raise RuntimeError(
|
||||
f"Checksum verification failed!\n"
|
||||
f" Expected: {expected_hash}\n"
|
||||
f" Got: {actual}\n"
|
||||
f" File may be corrupted or tampered with. "
|
||||
f"Please retry or report at https://github.com/CloakHQ/cloakbrowser/issues"
|
||||
)
|
||||
logger.info("Checksum verified: SHA-256 OK")
|
||||
|
||||
|
||||
def _download_file(url: str, dest: Path) -> None:
|
||||
"""Download a file with progress logging."""
|
||||
logger.info("Downloading from %s", url)
|
||||
|
||||
@@ -0,0 +1,238 @@
|
||||
"""GeoIP-based timezone and locale detection from proxy IP.
|
||||
|
||||
Optional feature — requires ``geoip2`` package::
|
||||
|
||||
pip install cloakbrowser[geoip]
|
||||
|
||||
Downloads GeoLite2-City.mmdb (~70 MB) on first use, caches in
|
||||
``~/.cloakbrowser/geoip/``. Background re-download after 30 days.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import ipaddress
|
||||
import logging
|
||||
import socket
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
from pathlib import Path
|
||||
from urllib.parse import urlparse
|
||||
|
||||
logger = logging.getLogger("cloakbrowser")
|
||||
|
||||
# P3TERX mirror of MaxMind GeoLite2-City — no license key needed
|
||||
GEOIP_DB_URL = (
|
||||
"https://github.com/P3TERX/GeoLite.mmdb/raw/download/GeoLite2-City.mmdb"
|
||||
)
|
||||
GEOIP_DB_FILENAME = "GeoLite2-City.mmdb"
|
||||
GEOIP_UPDATE_INTERVAL = 30 * 86_400 # 30 days
|
||||
|
||||
# Country ISO code → BCP 47 locale (covers ~90 % of proxy traffic)
|
||||
COUNTRY_LOCALE_MAP: dict[str, str] = {
|
||||
"US": "en-US", "GB": "en-GB", "AU": "en-AU", "CA": "en-CA", "NZ": "en-NZ",
|
||||
"IE": "en-IE", "ZA": "en-ZA", "SG": "en-SG",
|
||||
"DE": "de-DE", "AT": "de-AT", "CH": "de-CH",
|
||||
"FR": "fr-FR", "BE": "fr-BE",
|
||||
"ES": "es-ES", "MX": "es-MX", "AR": "es-AR", "CO": "es-CO", "CL": "es-CL",
|
||||
"BR": "pt-BR", "PT": "pt-PT",
|
||||
"IT": "it-IT", "NL": "nl-NL",
|
||||
"JP": "ja-JP", "KR": "ko-KR", "CN": "zh-CN", "TW": "zh-TW", "HK": "zh-HK",
|
||||
"RU": "ru-RU", "UA": "uk-UA", "PL": "pl-PL", "CZ": "cs-CZ", "RO": "ro-RO",
|
||||
"IL": "he-IL", "TR": "tr-TR", "SA": "ar-SA", "AE": "ar-AE", "EG": "ar-EG",
|
||||
"IN": "hi-IN", "ID": "id-ID", "PH": "en-PH",
|
||||
"TH": "th-TH", "VN": "vi-VN", "MY": "ms-MY",
|
||||
"SE": "sv-SE", "NO": "nb-NO", "DK": "da-DK", "FI": "fi-FI",
|
||||
"GR": "el-GR", "HU": "hu-HU", "BG": "bg-BG",
|
||||
}
|
||||
|
||||
|
||||
def resolve_proxy_geo(proxy_url: str) -> tuple[str | None, str | None]:
|
||||
"""Resolve timezone and locale from a proxy's IP address.
|
||||
|
||||
Returns ``(timezone, locale)`` — either or both may be ``None`` on
|
||||
failure (missing dep, DB download error, lookup miss). Never raises.
|
||||
"""
|
||||
try:
|
||||
import geoip2.database # noqa: F811
|
||||
except ImportError:
|
||||
raise ImportError(
|
||||
"geoip2 is required for geoip=True. Install it with:\n"
|
||||
" pip install cloakbrowser[geoip]"
|
||||
) from None
|
||||
|
||||
db_path = _ensure_geoip_db()
|
||||
if db_path is None:
|
||||
return None, None
|
||||
|
||||
# Exit IP (through proxy) is most accurate — gateway DNS may differ from exit
|
||||
ip = _resolve_exit_ip(proxy_url)
|
||||
if ip is None:
|
||||
ip = _resolve_proxy_ip(proxy_url)
|
||||
if ip is None:
|
||||
return None, None
|
||||
|
||||
try:
|
||||
with geoip2.database.Reader(str(db_path)) as reader:
|
||||
resp = reader.city(ip)
|
||||
timezone = resp.location.time_zone
|
||||
country = resp.country.iso_code
|
||||
locale = COUNTRY_LOCALE_MAP.get(country) if country else None
|
||||
logger.debug(
|
||||
"GeoIP: %s → tz=%s, country=%s, locale=%s",
|
||||
ip, timezone, country, locale,
|
||||
)
|
||||
return timezone, locale
|
||||
except Exception as exc:
|
||||
logger.debug("GeoIP lookup failed for %s: %s", ip, exc)
|
||||
return None, None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Proxy IP resolution
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _resolve_proxy_ip(proxy_url: str) -> str | None:
|
||||
"""Extract proxy hostname from URL and resolve to an IP address."""
|
||||
try:
|
||||
hostname = urlparse(proxy_url).hostname
|
||||
if not hostname:
|
||||
return None
|
||||
|
||||
# Already a literal IP?
|
||||
try:
|
||||
socket.inet_pton(socket.AF_INET, hostname)
|
||||
return hostname
|
||||
except OSError:
|
||||
pass
|
||||
try:
|
||||
socket.inet_pton(socket.AF_INET6, hostname)
|
||||
return hostname
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
# DNS resolve (returns first result, handles both v4/v6)
|
||||
results = socket.getaddrinfo(hostname, None, socket.AF_UNSPEC, socket.SOCK_STREAM)
|
||||
if results:
|
||||
ip = results[0][4][0]
|
||||
logger.debug("Resolved proxy %s → %s", hostname, ip)
|
||||
return ip
|
||||
return None
|
||||
except Exception as exc:
|
||||
logger.debug("Failed to resolve proxy hostname: %s", exc)
|
||||
return None
|
||||
|
||||
|
||||
def _is_private_ip(ip: str) -> bool:
|
||||
"""Check if an IP address is private/internal (not routable on the internet)."""
|
||||
try:
|
||||
return ipaddress.ip_address(ip).is_private
|
||||
except ValueError:
|
||||
return False
|
||||
|
||||
|
||||
# IP echo services — fast, no auth, return just the IP
|
||||
_IP_ECHO_URLS = [
|
||||
"https://api.ipify.org",
|
||||
"https://checkip.amazonaws.com",
|
||||
"https://ifconfig.me/ip",
|
||||
]
|
||||
|
||||
|
||||
def _resolve_exit_ip(proxy_url: str) -> str | None:
|
||||
"""Discover the proxy's actual exit IP by connecting through it."""
|
||||
import httpx
|
||||
|
||||
for url in _IP_ECHO_URLS:
|
||||
try:
|
||||
resp = httpx.get(url, proxy=proxy_url, timeout=10.0)
|
||||
resp.raise_for_status()
|
||||
ip = resp.text.strip()
|
||||
# Validate it looks like an IP
|
||||
ipaddress.ip_address(ip)
|
||||
logger.debug("Exit IP via %s: %s", url, ip)
|
||||
return ip
|
||||
except Exception:
|
||||
continue
|
||||
logger.debug("Failed to discover exit IP through proxy")
|
||||
return None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# GeoIP database management
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _get_geoip_dir() -> Path:
|
||||
from .config import get_cache_dir
|
||||
|
||||
return get_cache_dir() / "geoip"
|
||||
|
||||
|
||||
def _ensure_geoip_db() -> Path | None:
|
||||
"""Return path to GeoLite2-City.mmdb, downloading on first use."""
|
||||
db_path = _get_geoip_dir() / GEOIP_DB_FILENAME
|
||||
|
||||
if db_path.exists():
|
||||
_maybe_trigger_update(db_path)
|
||||
return db_path
|
||||
|
||||
try:
|
||||
_download_geoip_db(db_path)
|
||||
return db_path
|
||||
except Exception as exc:
|
||||
logger.warning("Failed to download GeoIP database: %s", exc)
|
||||
return None
|
||||
|
||||
|
||||
def _download_geoip_db(dest: Path) -> None:
|
||||
"""Atomic download of GeoLite2-City.mmdb via httpx."""
|
||||
import httpx
|
||||
|
||||
dest.parent.mkdir(parents=True, exist_ok=True)
|
||||
logger.info("Downloading GeoIP database (~70 MB) …")
|
||||
|
||||
tmp_fd, tmp_name = tempfile.mkstemp(dir=dest.parent, suffix=".tmp")
|
||||
tmp_path = Path(tmp_name)
|
||||
try:
|
||||
with httpx.stream(
|
||||
"GET", GEOIP_DB_URL, follow_redirects=True, timeout=300.0
|
||||
) as resp:
|
||||
resp.raise_for_status()
|
||||
total = int(resp.headers.get("content-length", 0))
|
||||
downloaded = 0
|
||||
last_pct = -1
|
||||
with open(tmp_fd, "wb") as f:
|
||||
for chunk in resp.iter_bytes(chunk_size=65_536):
|
||||
f.write(chunk)
|
||||
downloaded += len(chunk)
|
||||
if total:
|
||||
pct = downloaded * 100 // total
|
||||
if pct >= last_pct + 10:
|
||||
last_pct = pct
|
||||
logger.info("GeoIP download: %d %%", pct)
|
||||
|
||||
tmp_path.rename(dest)
|
||||
logger.info("GeoIP database ready: %s", dest)
|
||||
except Exception:
|
||||
tmp_path.unlink(missing_ok=True)
|
||||
raise
|
||||
|
||||
|
||||
def _maybe_trigger_update(db_path: Path) -> None:
|
||||
"""Re-download in background if DB is older than 30 days."""
|
||||
try:
|
||||
age = time.time() - db_path.stat().st_mtime
|
||||
if age < GEOIP_UPDATE_INTERVAL:
|
||||
return
|
||||
except OSError:
|
||||
return
|
||||
|
||||
def _bg() -> None:
|
||||
try:
|
||||
_download_geoip_db(db_path)
|
||||
except Exception:
|
||||
logger.debug("Background GeoIP update failed", exc_info=True)
|
||||
|
||||
threading.Thread(target=_bg, daemon=True).start()
|
||||
@@ -0,0 +1,227 @@
|
||||
"""Test against fingerprint-scan.com and CreepJS.
|
||||
|
||||
Tests the specific headless detection signals flagged by the community:
|
||||
- noTaskbar, noContentIndex, noContactsManager, noDownlinkMax
|
||||
- Bot risk score (fingerprint-scan.com)
|
||||
- Headless/stealth percentages (CreepJS)
|
||||
- Full CreepJS signal breakdown (likeHeadless, headless, stealth)
|
||||
|
||||
Usage:
|
||||
python examples/fingerprint_scan_test.py
|
||||
python examples/fingerprint_scan_test.py --proxy http://10.50.96.5:8888
|
||||
python examples/fingerprint_scan_test.py --headless
|
||||
"""
|
||||
|
||||
import sys
|
||||
|
||||
from cloakbrowser import launch_context
|
||||
|
||||
HEADLESS = "--headless" in sys.argv
|
||||
PROXY = None
|
||||
for i, arg in enumerate(sys.argv):
|
||||
if arg == "--proxy" and i + 1 < len(sys.argv):
|
||||
PROXY = sys.argv[i + 1]
|
||||
|
||||
|
||||
def test_fingerprint_scan(page):
|
||||
"""fingerprint-scan.com — bot risk score + headless detection signals."""
|
||||
print("=== fingerprint-scan.com ===")
|
||||
page.goto("https://fingerprint-scan.com/", wait_until="domcontentloaded", timeout=30000)
|
||||
page.wait_for_timeout(20000) # Castle.js needs time to compute score
|
||||
|
||||
# Check bot risk score
|
||||
score = page.evaluate(
|
||||
'document.getElementById("fingerprintScore")?.textContent || "Score not rendered"'
|
||||
)
|
||||
print(f"Bot Risk Score: {score}")
|
||||
|
||||
# Check headless detection signals
|
||||
apis = page.evaluate("""() => ({
|
||||
noTaskbar: screen.height === screen.availHeight,
|
||||
taskbarSize: screen.height - screen.availHeight,
|
||||
noContentIndex: typeof window.ContentIndex === "undefined",
|
||||
noContactsManager: !("contacts" in navigator),
|
||||
noDownlinkMax: !("downlinkMax" in (navigator.connection || {})),
|
||||
downlinkMax: navigator.connection?.downlinkMax ?? null,
|
||||
timezone: Intl.DateTimeFormat().resolvedOptions().timeZone,
|
||||
webdriver: navigator.webdriver,
|
||||
isPlaywright: "__pwInitScripts" in window || "__playwright__binding__" in window,
|
||||
webgpu: typeof navigator.gpu !== "undefined" ? "available" : "NOT_AVAILABLE",
|
||||
scrollbarWidth: (() => { const d = document.createElement("div"); d.style.cssText = "overflow:scroll;width:100px;height:100px;position:absolute;top:-999px"; document.body.appendChild(d); const w = d.offsetWidth - d.clientWidth; d.remove(); return w; })()
|
||||
})""")
|
||||
|
||||
print("\nHeadless detection signals:")
|
||||
headless_fails = 0
|
||||
for k, v in apis.items():
|
||||
is_fail = k.startswith("no") and v is True
|
||||
if is_fail:
|
||||
headless_fails += 1
|
||||
flag = "FAIL" if is_fail else ""
|
||||
print(f" {k}: {v} {flag}")
|
||||
|
||||
# Extract bot test results from page
|
||||
bot_tests = page.evaluate("""() => {
|
||||
const text = document.body.innerText;
|
||||
const tests = {};
|
||||
for (const key of ['WebDriver', 'Is Selenium Chrome', 'CDP Check', 'Is Playwright']) {
|
||||
const match = text.match(new RegExp(key + '\\\\s+(true|false)'));
|
||||
if (match) tests[key] = match[1];
|
||||
}
|
||||
return tests;
|
||||
}""")
|
||||
print("\nBot Detection Tests:")
|
||||
for k, v in bot_tests.items():
|
||||
status = "PASS" if v == "false" else "FAIL"
|
||||
print(f" {k}: {v} [{status}]")
|
||||
|
||||
page.screenshot(path="/results/fingerprint-scan.png", full_page=True)
|
||||
print("\nScreenshot: /results/fingerprint-scan.png")
|
||||
|
||||
return {
|
||||
"score": score,
|
||||
"headless_fails": headless_fails,
|
||||
"apis": apis,
|
||||
"bot_tests": bot_tests,
|
||||
}
|
||||
|
||||
|
||||
def test_creepjs(page):
|
||||
"""abrahamjuliot.github.io/creepjs — comprehensive fingerprint analysis."""
|
||||
print("\n=== CreepJS ===")
|
||||
page.goto(
|
||||
"https://abrahamjuliot.github.io/creepjs/", wait_until="domcontentloaded", timeout=30000
|
||||
)
|
||||
print("Waiting 30s for CreepJS analysis...")
|
||||
page.wait_for_timeout(30000)
|
||||
|
||||
# Extract % scores from page text (matches test-infra/matrix_tests/group3_bot_detection.py)
|
||||
scores = page.evaluate("""() => {
|
||||
const text = document.body.innerText;
|
||||
const likeMatch = text.match(/(\\d+)%\\s*like headless/i);
|
||||
const headlessMatch = text.match(/(\\d+)%\\s*headless:/i);
|
||||
const stealthMatch = text.match(/(\\d+)%\\s*stealth:/i);
|
||||
return {
|
||||
likeHeadlessPct: likeMatch ? parseInt(likeMatch[1]) : null,
|
||||
headlessPct: headlessMatch ? parseInt(headlessMatch[1]) : null,
|
||||
stealthPct: stealthMatch ? parseInt(stealthMatch[1]) : null,
|
||||
};
|
||||
}""")
|
||||
|
||||
print(f"\nScores:")
|
||||
print(f" like-headless: {scores['likeHeadlessPct']}% (target: <=30%)")
|
||||
print(f" headless: {scores['headlessPct']}% (target: 0%)")
|
||||
print(f" stealth: {scores['stealthPct']}% (target: 0%)")
|
||||
|
||||
# Extract full signal breakdown from window.Fingerprint.headless (CreepJS internal object)
|
||||
signals = page.evaluate("""() => {
|
||||
try {
|
||||
const fp = window.Fingerprint;
|
||||
if (!fp || !fp.headless) return null;
|
||||
return {
|
||||
likeHeadless: fp.headless.likeHeadless || null,
|
||||
headless: fp.headless.headless || null,
|
||||
stealth: fp.headless.stealth || null,
|
||||
};
|
||||
} catch { return null; }
|
||||
}""")
|
||||
|
||||
if signals:
|
||||
if signals.get("likeHeadless"):
|
||||
print("\nlikeHeadless signals:")
|
||||
fails = 0
|
||||
for k, v in signals["likeHeadless"].items():
|
||||
is_fail = v is True
|
||||
if is_fail:
|
||||
fails += 1
|
||||
flag = " FAIL" if is_fail else ""
|
||||
print(f" {k}: {v}{flag}")
|
||||
print(f" ({fails} fails)")
|
||||
|
||||
if signals.get("headless"):
|
||||
print("\nheadless signals:")
|
||||
for k, v in signals["headless"].items():
|
||||
flag = " FAIL" if v is True else ""
|
||||
print(f" {k}: {v}{flag}")
|
||||
|
||||
if signals.get("stealth"):
|
||||
print("\nstealth signals:")
|
||||
for k, v in signals["stealth"].items():
|
||||
flag = " FAIL" if v is True else ""
|
||||
print(f" {k}: {v}{flag}")
|
||||
else:
|
||||
print("\n(window.Fingerprint.headless not available — signals not extracted)")
|
||||
|
||||
# Extract platform estimate
|
||||
platform = page.evaluate("""() => {
|
||||
try {
|
||||
const fp = window.Fingerprint;
|
||||
if (!fp || !fp.platformEstimate) return null;
|
||||
return fp.platformEstimate;
|
||||
} catch { return null; }
|
||||
}""")
|
||||
if platform:
|
||||
print(f"\nPlatform estimate: {platform}")
|
||||
|
||||
passed = (
|
||||
scores["headlessPct"] is not None
|
||||
and scores["headlessPct"] <= 30
|
||||
and scores["stealthPct"] is not None
|
||||
and scores["stealthPct"] <= 30
|
||||
)
|
||||
print(f"\nVerdict: {'PASS' if passed else 'FAIL'} (<=30% headless, <=30% stealth)")
|
||||
|
||||
page.screenshot(path="/results/creepjs.png", full_page=True)
|
||||
print("Screenshot: /results/creepjs.png")
|
||||
|
||||
return {**scores, "signals": signals, "platform": platform}
|
||||
|
||||
|
||||
def main():
|
||||
print("=" * 60)
|
||||
print("CloakBrowser — Fingerprint & Headless Detection Tests")
|
||||
print("=" * 60)
|
||||
print(f"Mode: {'headless' if HEADLESS else 'headed'}")
|
||||
print(f"Proxy: {PROXY or 'none'}")
|
||||
print()
|
||||
|
||||
context = launch_context(
|
||||
headless=HEADLESS,
|
||||
proxy=PROXY,
|
||||
args=[
|
||||
"--fingerprint-screen-width=1920",
|
||||
"--fingerprint-screen-height=1080",
|
||||
"--timezone=Asia/Jerusalem",
|
||||
],
|
||||
)
|
||||
page = context.new_page()
|
||||
|
||||
try:
|
||||
fp_result = test_fingerprint_scan(page)
|
||||
creep_result = test_creepjs(page)
|
||||
finally:
|
||||
context.close()
|
||||
|
||||
# Summary
|
||||
print("\n" + "=" * 60)
|
||||
print("SUMMARY")
|
||||
print("=" * 60)
|
||||
print(f"fingerprint-scan.com: {fp_result['score']}")
|
||||
print(f" Headless signal fails: {fp_result['headless_fails']}")
|
||||
like = creep_result["likeHeadlessPct"]
|
||||
headless = creep_result["headlessPct"]
|
||||
stealth = creep_result["stealthPct"]
|
||||
print(f"CreepJS: like-headless={like}%, headless={headless}%, stealth={stealth}%")
|
||||
|
||||
# Count CreepJS signal fails
|
||||
sigs = creep_result.get("signals")
|
||||
if sigs and sigs.get("likeHeadless"):
|
||||
fail_names = [k for k, v in sigs["likeHeadless"].items() if v is True]
|
||||
if fail_names:
|
||||
print(f" likeHeadless fails: {', '.join(fail_names)}")
|
||||
print("=" * 60)
|
||||
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -5,6 +5,8 @@ Expected: 0.9 (human-level) with cloakbrowser.
|
||||
Default Playwright typically scores 0.1-0.3.
|
||||
"""
|
||||
|
||||
import time
|
||||
|
||||
from cloakbrowser import launch
|
||||
|
||||
browser = launch(headless=True)
|
||||
@@ -18,7 +20,7 @@ page.wait_for_load_state("networkidle")
|
||||
button = page.query_selector("button")
|
||||
if button:
|
||||
button.click()
|
||||
page.wait_for_timeout(3000)
|
||||
time.sleep(3)
|
||||
|
||||
# Extract score from page
|
||||
content = page.content()
|
||||
|
||||
@@ -27,7 +27,7 @@ for i, arg in enumerate(sys.argv):
|
||||
def test_bot_sannysoft(page):
|
||||
"""bot.sannysoft.com — classic bot detection checks."""
|
||||
page.goto("https://bot.sannysoft.com", wait_until="networkidle", timeout=30000)
|
||||
page.wait_for_timeout(3000)
|
||||
time.sleep(3)
|
||||
|
||||
results = page.evaluate("""() => {
|
||||
const rows = document.querySelectorAll('table tr');
|
||||
@@ -53,7 +53,7 @@ def test_bot_sannysoft(page):
|
||||
def test_bot_incolumitas(page):
|
||||
"""bot.incolumitas.com — comprehensive 30+ check bot detection."""
|
||||
page.goto("https://bot.incolumitas.com", wait_until="networkidle", timeout=30000)
|
||||
page.wait_for_timeout(12000) # needs time to run all detection tests
|
||||
time.sleep(12) # needs time to run all detection tests
|
||||
|
||||
# Site outputs JSON blocks in page text, not HTML tables
|
||||
results = page.evaluate("""() => {
|
||||
@@ -74,7 +74,7 @@ def test_bot_incolumitas(page):
|
||||
def test_browserscan(page):
|
||||
"""browserscan.net/bot-detection — WebDriver, UA, CDP, Navigator checks."""
|
||||
page.goto("https://www.browserscan.net/bot-detection", wait_until="networkidle", timeout=30000)
|
||||
page.wait_for_timeout(5000)
|
||||
time.sleep(5)
|
||||
|
||||
results = page.evaluate("""() => {
|
||||
const items = document.querySelectorAll('[class*="result"], [class*="item"], [class*="check"]');
|
||||
@@ -95,7 +95,7 @@ def test_browserscan(page):
|
||||
def test_deviceandbrowserinfo(page):
|
||||
"""deviceandbrowserinfo.com/are_you_a_bot — fingerprint + behavioral detection."""
|
||||
page.goto("https://deviceandbrowserinfo.com/are_you_a_bot", wait_until="domcontentloaded", timeout=30000)
|
||||
page.wait_for_timeout(8000)
|
||||
time.sleep(8)
|
||||
|
||||
results = page.evaluate("""() => {
|
||||
const text = document.body.innerText;
|
||||
@@ -120,12 +120,12 @@ def test_deviceandbrowserinfo(page):
|
||||
def test_fingerprintjs(page):
|
||||
"""demo.fingerprint.com/web-scraping — industry-standard bot detection."""
|
||||
page.goto("https://demo.fingerprint.com/web-scraping", wait_until="domcontentloaded", timeout=30000)
|
||||
page.wait_for_timeout(8000)
|
||||
time.sleep(8)
|
||||
|
||||
# Click search to trigger bot detection — bots get blocked, humans see flights
|
||||
try:
|
||||
page.click("button:has-text('Search')", timeout=5000)
|
||||
page.wait_for_timeout(5000)
|
||||
time.sleep(5)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
@@ -147,7 +147,7 @@ def test_recaptcha(page):
|
||||
timeout=30000,
|
||||
)
|
||||
# Page auto-submits via grecaptcha.execute() — wait for backend response
|
||||
page.wait_for_timeout(8000)
|
||||
time.sleep(8)
|
||||
|
||||
results = page.evaluate("""() => {
|
||||
const text = document.body.innerText;
|
||||
|
||||
+57
-2
@@ -11,7 +11,7 @@
|
||||
|
||||
Drop-in Playwright/Puppeteer replacement. Same API — just swap the import. Scores **0.9 on reCAPTCHA v3**, passes **Cloudflare Turnstile**, and clears **30/30** stealth detection tests.
|
||||
|
||||
- 🔒 **16 source-level C++ patches** — not JS injection, not config flags
|
||||
- 🔒 **26 source-level C++ patches** — not JS injection, not config flags
|
||||
- 🎯 **0.9 reCAPTCHA v3 score** — human-level, server-verified
|
||||
- ☁️ **Passes Cloudflare Turnstile**, FingerprintJS, BrowserScan — 30/30 tests
|
||||
- 🔄 **Drop-in replacement** — works with both Playwright and Puppeteer
|
||||
@@ -75,7 +75,19 @@ const browser = await launch({
|
||||
args: ['--window-size=1920,1080'],
|
||||
});
|
||||
|
||||
// Browser + context in one call
|
||||
// With timezone and locale (sets --timezone and --lang binary flags)
|
||||
const browser = await launch({
|
||||
timezone: 'America/New_York',
|
||||
locale: 'en-US',
|
||||
});
|
||||
|
||||
// Auto-detect timezone/locale from proxy IP (requires: npm install mmdb-lib)
|
||||
const browser = await launch({
|
||||
proxy: 'http://proxy:8080',
|
||||
geoip: true,
|
||||
});
|
||||
|
||||
// Browser + context in one call (timezone/locale set both binary flags AND context)
|
||||
const context = await launchContext({
|
||||
userAgent: 'Custom UA',
|
||||
viewport: { width: 1920, height: 1080 },
|
||||
@@ -84,6 +96,27 @@ const context = await launchContext({
|
||||
});
|
||||
```
|
||||
|
||||
### Auto Timezone/Locale from Proxy IP
|
||||
|
||||
When using a proxy, antibot systems check that your browser's timezone and locale match the proxy's location. Install `mmdb-lib` to enable auto-detection from an offline GeoIP database (~70 MB, downloaded on first use):
|
||||
|
||||
```bash
|
||||
npm install mmdb-lib
|
||||
```
|
||||
|
||||
```javascript
|
||||
// Auto-detect — timezone and locale set from proxy's IP geolocation
|
||||
const browser = await launch({ proxy: 'http://proxy:8080', geoip: true });
|
||||
|
||||
// Works with launchContext too
|
||||
const context = await launchContext({ proxy: 'http://proxy:8080', geoip: true });
|
||||
|
||||
// Explicit values always win over auto-detection
|
||||
const browser = await launch({ proxy: 'http://proxy:8080', geoip: true, timezone: 'Europe/London' });
|
||||
```
|
||||
|
||||
> **Note:** For rotating residential proxies, the DNS-resolved IP may differ from the exit IP. Pass explicit `timezone`/`locale` in those cases.
|
||||
|
||||
### Utilities
|
||||
|
||||
```javascript
|
||||
@@ -151,6 +184,28 @@ const page = await browser.newPage();
|
||||
- Node.js >= 18
|
||||
- One of: `playwright-core` >= 1.40 or `puppeteer-core` >= 21
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**reCAPTCHA v3 scores are low (0.1–0.3)**
|
||||
|
||||
Avoid `page.waitForTimeout()` — it sends CDP protocol commands that reCAPTCHA detects. Use native sleep instead:
|
||||
|
||||
```javascript
|
||||
// Bad — sends CDP commands, reCAPTCHA detects this
|
||||
await page.waitForTimeout(3000);
|
||||
|
||||
// Good — invisible to the browser
|
||||
await new Promise(r => setTimeout(r, 3000));
|
||||
```
|
||||
|
||||
Other tips for maximizing reCAPTCHA scores:
|
||||
- **Use Playwright, not Puppeteer** — Puppeteer sends more CDP protocol traffic that reCAPTCHA detects ([details](#puppeteer))
|
||||
- **Use residential proxies** — datacenter IPs are flagged by IP reputation, not browser fingerprint
|
||||
- **Spend 15+ seconds on the page** before triggering reCAPTCHA — short visits score lower
|
||||
- **Space out requests** — back-to-back `grecaptcha.execute()` calls from the same session get penalized. Wait 30+ seconds between pages with reCAPTCHA
|
||||
- **Use a fixed fingerprint seed** (`--fingerprint=12345`) for consistent device identity across sessions
|
||||
- **Minimize `page.evaluate()` calls** before the reCAPTCHA check fires — each one sends CDP traffic
|
||||
|
||||
## Links
|
||||
|
||||
- 🌐 [Website](https://cloakbrowser.dev)
|
||||
|
||||
Generated
+18
-2
@@ -1,18 +1,19 @@
|
||||
{
|
||||
"name": "cloakbrowser",
|
||||
"version": "0.1.0",
|
||||
"version": "0.2.0",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "cloakbrowser",
|
||||
"version": "0.1.0",
|
||||
"version": "0.2.0",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"tar": "^7.0.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^20.10.0",
|
||||
"mmdb-lib": "^3.0.2",
|
||||
"playwright-core": "^1.40.0",
|
||||
"puppeteer-core": "^21.0.0",
|
||||
"typescript": "^5.3.0",
|
||||
@@ -22,10 +23,14 @@
|
||||
"node": ">=18.0.0"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"mmdb-lib": ">=2.0.0",
|
||||
"playwright-core": ">=1.40.0",
|
||||
"puppeteer-core": ">=21.0.0"
|
||||
},
|
||||
"peerDependenciesMeta": {
|
||||
"mmdb-lib": {
|
||||
"optional": true
|
||||
},
|
||||
"playwright-core": {
|
||||
"optional": true
|
||||
},
|
||||
@@ -1839,6 +1844,17 @@
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/mmdb-lib": {
|
||||
"version": "3.0.2",
|
||||
"resolved": "https://registry.npmjs.org/mmdb-lib/-/mmdb-lib-3.0.2.tgz",
|
||||
"integrity": "sha512-7e87vk0DdWT647wjcfEtWeMtjm+zVGqNohN/aeIymbUfjHQ2T4Sx5kM+1irVDBSloNC3CkGKxswdMoo8yhqTDg==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=10",
|
||||
"npm": ">=6"
|
||||
}
|
||||
},
|
||||
"node_modules/ms": {
|
||||
"version": "2.1.2",
|
||||
"resolved": "https://registry.npmjs.org/ms/-/ms-2.1.2.tgz",
|
||||
|
||||
+6
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "cloakbrowser",
|
||||
"version": "0.2.0",
|
||||
"version": "0.3.0",
|
||||
"description": "Stealth Chromium that passes every bot detection test. Drop-in Playwright/Puppeteer replacement with source-level fingerprint patches.",
|
||||
"type": "module",
|
||||
"main": "dist/index.js",
|
||||
@@ -43,6 +43,7 @@
|
||||
"node": ">=18.0.0"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"mmdb-lib": ">=2.0.0",
|
||||
"playwright-core": ">=1.40.0",
|
||||
"puppeteer-core": ">=21.0.0"
|
||||
},
|
||||
@@ -52,6 +53,9 @@
|
||||
},
|
||||
"puppeteer-core": {
|
||||
"optional": true
|
||||
},
|
||||
"mmdb-lib": {
|
||||
"optional": true
|
||||
}
|
||||
},
|
||||
"dependencies": {
|
||||
@@ -59,6 +63,7 @@
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^20.10.0",
|
||||
"mmdb-lib": "^3.0.2",
|
||||
"playwright-core": "^1.40.0",
|
||||
"puppeteer-core": "^21.0.0",
|
||||
"typescript": "^5.3.0",
|
||||
|
||||
+20
-1
@@ -10,7 +10,7 @@ import path from "node:path";
|
||||
// ---------------------------------------------------------------------------
|
||||
// Chromium version shipped with this release
|
||||
// ---------------------------------------------------------------------------
|
||||
export const CHROMIUM_VERSION = "142.0.7444.175";
|
||||
export const CHROMIUM_VERSION = "145.0.7632.109";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Platform detection
|
||||
@@ -91,12 +91,21 @@ export const DOWNLOAD_BASE_URL =
|
||||
export const GITHUB_API_URL =
|
||||
"https://api.github.com/repos/CloakHQ/cloakbrowser/releases";
|
||||
|
||||
export const GITHUB_DOWNLOAD_BASE_URL =
|
||||
"https://github.com/CloakHQ/cloakbrowser/releases/download";
|
||||
|
||||
export function getDownloadUrl(version?: string): string {
|
||||
const v = version || CHROMIUM_VERSION;
|
||||
const tag = getPlatformTag();
|
||||
return `${DOWNLOAD_BASE_URL}/chromium-v${v}/cloakbrowser-${tag}.tar.gz`;
|
||||
}
|
||||
|
||||
export function getFallbackDownloadUrl(version?: string): string {
|
||||
const v = version || CHROMIUM_VERSION;
|
||||
const tag = getPlatformTag();
|
||||
return `${GITHUB_DOWNLOAD_BASE_URL}/chromium-v${v}/cloakbrowser-${tag}.tar.gz`;
|
||||
}
|
||||
|
||||
export function getEffectiveVersion(): string {
|
||||
const marker = path.join(getCacheDir(), "latest_version");
|
||||
try {
|
||||
@@ -139,6 +148,11 @@ export function getLocalBinaryOverride(): string | undefined {
|
||||
// ---------------------------------------------------------------------------
|
||||
// Default stealth arguments
|
||||
// ---------------------------------------------------------------------------
|
||||
// Default viewport — realistic maximized Chrome on 1080p Windows
|
||||
// screen=1920x1080, availHeight=1040 (minus 40px taskbar),
|
||||
// innerHeight=955 (minus ~85px Chrome UI: tabs + address bar + bookmarks)
|
||||
export const DEFAULT_VIEWPORT = { width: 1920, height: 955 };
|
||||
|
||||
export function getDefaultStealthArgs(): string[] {
|
||||
const seed = Math.floor(Math.random() * 90000) + 10000; // 10000-99999
|
||||
const isMac = process.platform === "darwin";
|
||||
@@ -159,7 +173,12 @@ export function getDefaultStealthArgs(): string[] {
|
||||
...base,
|
||||
"--fingerprint-platform=windows",
|
||||
"--fingerprint-hardware-concurrency=8",
|
||||
"--fingerprint-device-memory=8",
|
||||
"--fingerprint-gpu-vendor=NVIDIA Corporation",
|
||||
"--fingerprint-gpu-renderer=NVIDIA GeForce RTX 3070",
|
||||
"--fingerprint-taskbar-height=40",
|
||||
"--fingerprint-screen-width=1920",
|
||||
"--fingerprint-screen-height=1080",
|
||||
"--window-size=1920,1080",
|
||||
];
|
||||
}
|
||||
|
||||
+99
-2
@@ -5,6 +5,7 @@
|
||||
*/
|
||||
|
||||
import { execFileSync } from "node:child_process";
|
||||
import { createHash } from "node:crypto";
|
||||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
import { pipeline } from "node:stream/promises";
|
||||
@@ -14,13 +15,16 @@ import { extract as tarExtract } from "tar";
|
||||
import type { BinaryInfo } from "./types.js";
|
||||
import {
|
||||
CHROMIUM_VERSION,
|
||||
DOWNLOAD_BASE_URL,
|
||||
GITHUB_API_URL,
|
||||
GITHUB_DOWNLOAD_BASE_URL,
|
||||
checkPlatformAvailable,
|
||||
getBinaryDir,
|
||||
getBinaryPath,
|
||||
getCacheDir,
|
||||
getDownloadUrl,
|
||||
getEffectiveVersion,
|
||||
getFallbackDownloadUrl,
|
||||
getLocalBinaryOverride,
|
||||
getPlatformTag,
|
||||
versionNewer,
|
||||
@@ -135,7 +139,8 @@ export async function checkForUpdate(): Promise<string | null> {
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
async function downloadAndExtract(version?: string): Promise<void> {
|
||||
const url = getDownloadUrl(version);
|
||||
const primaryUrl = getDownloadUrl(version);
|
||||
const fallbackUrl = getFallbackDownloadUrl(version);
|
||||
const binaryDir = getBinaryDir(version);
|
||||
const binaryPath = getBinaryPath(version);
|
||||
|
||||
@@ -149,7 +154,24 @@ async function downloadAndExtract(version?: string): Promise<void> {
|
||||
);
|
||||
|
||||
try {
|
||||
await downloadFile(url, tmpPath);
|
||||
// Try primary server, fall back to GitHub Releases (skip fallback if custom URL)
|
||||
try {
|
||||
await downloadFile(primaryUrl, tmpPath);
|
||||
} catch (primaryErr) {
|
||||
if (process.env.CLOAKBROWSER_DOWNLOAD_URL) {
|
||||
throw primaryErr;
|
||||
}
|
||||
console.warn(
|
||||
`[cloakbrowser] Primary download failed (${primaryErr instanceof Error ? primaryErr.message : primaryErr}), trying GitHub Releases...`
|
||||
);
|
||||
await downloadFile(fallbackUrl, tmpPath);
|
||||
}
|
||||
|
||||
// Verify checksum before extraction
|
||||
if (process.env.CLOAKBROWSER_SKIP_CHECKSUM?.toLowerCase() !== "true") {
|
||||
await verifyDownloadChecksum(tmpPath, version);
|
||||
}
|
||||
|
||||
await extractArchive(tmpPath, binaryDir, binaryPath);
|
||||
console.log(
|
||||
`[cloakbrowser] Visit https://cloakbrowser.dev for docs and release notifications.`
|
||||
@@ -168,6 +190,81 @@ async function downloadAndExtract(version?: string): Promise<void> {
|
||||
}
|
||||
}
|
||||
|
||||
async function verifyDownloadChecksum(filePath: string, version?: string): Promise<void> {
|
||||
const checksums = await fetchChecksums(version);
|
||||
const tarballName = `cloakbrowser-${getPlatformTag()}.tar.gz`;
|
||||
|
||||
if (!checksums) {
|
||||
console.warn("[cloakbrowser] SHA256SUMS not available for this release — skipping checksum verification");
|
||||
return;
|
||||
}
|
||||
|
||||
const expected = checksums.get(tarballName);
|
||||
if (!expected) {
|
||||
console.warn(`[cloakbrowser] SHA256SUMS found but no entry for ${tarballName} — skipping verification`);
|
||||
return;
|
||||
}
|
||||
|
||||
await verifyChecksum(filePath, expected);
|
||||
}
|
||||
|
||||
async function fetchChecksums(version?: string): Promise<Map<string, string> | null> {
|
||||
const v = version || CHROMIUM_VERSION;
|
||||
const hasCustomUrl = !!process.env.CLOAKBROWSER_DOWNLOAD_URL;
|
||||
|
||||
// Respect custom URL contract — no GitHub fallback when custom URL is set
|
||||
const urls = [`${DOWNLOAD_BASE_URL}/chromium-v${v}/SHA256SUMS`];
|
||||
if (!hasCustomUrl) {
|
||||
urls.push(`${GITHUB_DOWNLOAD_BASE_URL}/chromium-v${v}/SHA256SUMS`);
|
||||
}
|
||||
|
||||
for (const url of urls) {
|
||||
try {
|
||||
const resp = await fetch(url, {
|
||||
redirect: "follow",
|
||||
signal: AbortSignal.timeout(10_000),
|
||||
});
|
||||
if (!resp.ok) continue;
|
||||
return parseChecksums(await resp.text());
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function parseChecksums(text: string): Map<string, string> {
|
||||
const result = new Map<string, string>();
|
||||
for (const line of text.trim().split("\n")) {
|
||||
const trimmed = line.trim();
|
||||
if (!trimmed) continue;
|
||||
const match = trimmed.match(/^([a-f0-9]{64})\s+\*?(.+)$/);
|
||||
if (match) {
|
||||
result.set(match[2]!, match[1]!.toLowerCase());
|
||||
}
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
async function verifyChecksum(filePath: string, expectedHash: string): Promise<void> {
|
||||
const hash = createHash("sha256");
|
||||
const stream = fs.createReadStream(filePath);
|
||||
for await (const chunk of stream) {
|
||||
hash.update(chunk);
|
||||
}
|
||||
const actual = hash.digest("hex").toLowerCase();
|
||||
if (actual !== expectedHash) {
|
||||
throw new Error(
|
||||
`Checksum verification failed!\n` +
|
||||
` Expected: ${expectedHash}\n` +
|
||||
` Got: ${actual}\n` +
|
||||
` File may be corrupted or tampered with. ` +
|
||||
`Please retry or report at https://github.com/CloakHQ/cloakbrowser/issues`
|
||||
);
|
||||
}
|
||||
console.log("[cloakbrowser] Checksum verified: SHA-256 OK");
|
||||
}
|
||||
|
||||
async function downloadFile(url: string, dest: string): Promise<void> {
|
||||
console.log(`[cloakbrowser] Downloading from ${url}`);
|
||||
|
||||
|
||||
+262
@@ -0,0 +1,262 @@
|
||||
/**
|
||||
* GeoIP-based timezone and locale detection from proxy IP.
|
||||
*
|
||||
* Optional feature — requires `mmdb-lib` package:
|
||||
* npm install mmdb-lib
|
||||
*
|
||||
* Downloads GeoLite2-City.mmdb (~70 MB) on first use,
|
||||
* caches in `~/.cloakbrowser/geoip/`.
|
||||
*/
|
||||
|
||||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
import { createWriteStream } from "node:fs";
|
||||
import dns from "node:dns/promises";
|
||||
import net from "node:net";
|
||||
import { getCacheDir } from "./config.js";
|
||||
|
||||
// P3TERX mirror of MaxMind GeoLite2-City — no license key needed
|
||||
const GEOIP_DB_URL =
|
||||
"https://github.com/P3TERX/GeoLite.mmdb/raw/download/GeoLite2-City.mmdb";
|
||||
const GEOIP_DB_FILENAME = "GeoLite2-City.mmdb";
|
||||
const GEOIP_UPDATE_INTERVAL_MS = 30 * 86_400_000; // 30 days
|
||||
|
||||
/** Country ISO code → BCP 47 locale (covers ~90% of proxy traffic). */
|
||||
export const COUNTRY_LOCALE_MAP: Record<string, string> = {
|
||||
US: "en-US", GB: "en-GB", AU: "en-AU", CA: "en-CA", NZ: "en-NZ",
|
||||
IE: "en-IE", ZA: "en-ZA", SG: "en-SG",
|
||||
DE: "de-DE", AT: "de-AT", CH: "de-CH",
|
||||
FR: "fr-FR", BE: "fr-BE",
|
||||
ES: "es-ES", MX: "es-MX", AR: "es-AR", CO: "es-CO", CL: "es-CL",
|
||||
BR: "pt-BR", PT: "pt-PT",
|
||||
IT: "it-IT", NL: "nl-NL",
|
||||
JP: "ja-JP", KR: "ko-KR", CN: "zh-CN", TW: "zh-TW", HK: "zh-HK",
|
||||
RU: "ru-RU", UA: "uk-UA", PL: "pl-PL", CZ: "cs-CZ", RO: "ro-RO",
|
||||
IL: "he-IL", TR: "tr-TR", SA: "ar-SA", AE: "ar-AE", EG: "ar-EG",
|
||||
IN: "hi-IN", ID: "id-ID", PH: "en-PH",
|
||||
TH: "th-TH", VN: "vi-VN", MY: "ms-MY",
|
||||
SE: "sv-SE", NO: "nb-NO", DK: "da-DK", FI: "fi-FI",
|
||||
GR: "el-GR", HU: "hu-HU", BG: "bg-BG",
|
||||
};
|
||||
|
||||
export interface GeoResult {
|
||||
timezone: string | null;
|
||||
locale: string | null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve timezone and locale from a proxy's IP address.
|
||||
* Returns `{ timezone, locale }` — either may be null on failure.
|
||||
* Never throws.
|
||||
*/
|
||||
export async function resolveProxyGeo(
|
||||
proxyUrl: string
|
||||
): Promise<GeoResult> {
|
||||
let Reader: any;
|
||||
try {
|
||||
const mmdb = await import("mmdb-lib");
|
||||
Reader = mmdb.default?.Reader ?? mmdb.Reader;
|
||||
} catch {
|
||||
throw new Error(
|
||||
"mmdb-lib is required for geoip: true. Install it with:\n npm install mmdb-lib"
|
||||
);
|
||||
}
|
||||
|
||||
const dbPath = await ensureGeoipDb();
|
||||
if (!dbPath) return { timezone: null, locale: null };
|
||||
|
||||
// Exit IP (through proxy) is most accurate — gateway DNS may differ from exit
|
||||
let ip = await resolveExitIp(proxyUrl);
|
||||
if (!ip) ip = await resolveProxyIp(proxyUrl);
|
||||
if (!ip) return { timezone: null, locale: null };
|
||||
|
||||
try {
|
||||
const buf = fs.readFileSync(dbPath);
|
||||
const reader = new Reader(buf);
|
||||
const result = reader.get(ip) as any;
|
||||
const timezone: string | null = result?.location?.time_zone ?? null;
|
||||
const countryCode: string | null = result?.country?.iso_code ?? null;
|
||||
const locale =
|
||||
countryCode ? (COUNTRY_LOCALE_MAP[countryCode] ?? null) : null;
|
||||
return { timezone, locale };
|
||||
} catch {
|
||||
return { timezone: null, locale: null };
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Proxy IP resolution
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** @internal Exported for testing. */
|
||||
export async function resolveProxyIp(
|
||||
proxyUrl: string
|
||||
): Promise<string | null> {
|
||||
try {
|
||||
const url = new URL(proxyUrl);
|
||||
const hostname = url.hostname;
|
||||
if (!hostname) return null;
|
||||
|
||||
// Already a literal IP?
|
||||
if (net.isIP(hostname)) return hostname;
|
||||
|
||||
// DNS resolve
|
||||
const { address } = await dns.lookup(hostname);
|
||||
return address;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function isPrivateIp(ip: string): boolean {
|
||||
// Quick check for common private ranges
|
||||
if (ip.startsWith("10.") || ip.startsWith("127.") || ip === "::1") return true;
|
||||
if (ip.startsWith("172.")) {
|
||||
const second = parseInt(ip.split(".")[1], 10);
|
||||
if (second >= 16 && second <= 31) return true;
|
||||
}
|
||||
if (ip.startsWith("192.168.")) return true;
|
||||
return false;
|
||||
}
|
||||
|
||||
const IP_ECHO_URLS = [
|
||||
"https://api.ipify.org",
|
||||
"https://checkip.amazonaws.com",
|
||||
"https://ifconfig.me/ip",
|
||||
];
|
||||
|
||||
async function resolveExitIp(proxyUrl: string): Promise<string | null> {
|
||||
// Node.js fetch doesn't support proxy natively — use a CONNECT tunnel via http
|
||||
// For simplicity, use a direct HTTP request to a plain-text IP echo service
|
||||
// through the proxy using Node's http module
|
||||
try {
|
||||
const { default: http } = await import("node:http");
|
||||
const { default: https } = await import("node:https");
|
||||
const proxyUrlObj = new URL(proxyUrl);
|
||||
|
||||
for (const echoUrl of IP_ECHO_URLS) {
|
||||
try {
|
||||
const ip = await new Promise<string | null>((resolve, reject) => {
|
||||
const targetUrl = new URL(echoUrl);
|
||||
const connectReq = http.request({
|
||||
host: proxyUrlObj.hostname,
|
||||
port: parseInt(proxyUrlObj.port || "80", 10),
|
||||
method: "CONNECT",
|
||||
path: `${targetUrl.hostname}:443`,
|
||||
headers: proxyUrlObj.username
|
||||
? {
|
||||
"Proxy-Authorization":
|
||||
"Basic " +
|
||||
Buffer.from(
|
||||
`${decodeURIComponent(proxyUrlObj.username)}:${decodeURIComponent(proxyUrlObj.password || "")}`
|
||||
).toString("base64"),
|
||||
}
|
||||
: {},
|
||||
timeout: 10_000,
|
||||
});
|
||||
|
||||
connectReq.on("connect", (_res, socket) => {
|
||||
const req = https.request(
|
||||
echoUrl,
|
||||
{ socket, timeout: 5_000 } as any,
|
||||
(res) => {
|
||||
let data = "";
|
||||
res.on("data", (chunk: Buffer) => (data += chunk.toString()));
|
||||
res.on("end", () => {
|
||||
const ip = data.trim();
|
||||
resolve(net.isIP(ip) ? ip : null);
|
||||
});
|
||||
}
|
||||
);
|
||||
req.on("error", () => resolve(null));
|
||||
req.end();
|
||||
});
|
||||
|
||||
connectReq.on("error", () => resolve(null));
|
||||
connectReq.on("timeout", () => {
|
||||
connectReq.destroy();
|
||||
resolve(null);
|
||||
});
|
||||
connectReq.end();
|
||||
});
|
||||
|
||||
if (ip) return ip;
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
// Fallback: couldn't import http modules
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// GeoIP database management
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function getGeoipDir(): string {
|
||||
return path.join(getCacheDir(), "geoip");
|
||||
}
|
||||
|
||||
async function ensureGeoipDb(): Promise<string | null> {
|
||||
const dir = getGeoipDir();
|
||||
const dbPath = path.join(dir, GEOIP_DB_FILENAME);
|
||||
|
||||
if (fs.existsSync(dbPath)) {
|
||||
maybeTriggerUpdate(dbPath);
|
||||
return dbPath;
|
||||
}
|
||||
|
||||
try {
|
||||
await downloadGeoipDb(dbPath);
|
||||
return dbPath;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
async function downloadGeoipDb(dest: string): Promise<void> {
|
||||
const dir = path.dirname(dest);
|
||||
fs.mkdirSync(dir, { recursive: true });
|
||||
console.log("[cloakbrowser] Downloading GeoIP database (~70 MB)…");
|
||||
|
||||
const tmpPath = `${dest}.tmp.${Date.now()}`;
|
||||
try {
|
||||
const response = await fetch(GEOIP_DB_URL, { redirect: "follow" });
|
||||
if (!response.ok || !response.body) {
|
||||
throw new Error(`HTTP ${response.status}`);
|
||||
}
|
||||
|
||||
const fileStream = createWriteStream(tmpPath);
|
||||
const reader = response.body.getReader();
|
||||
|
||||
for (;;) {
|
||||
const { done, value } = await reader.read();
|
||||
if (done) break;
|
||||
fileStream.write(value);
|
||||
}
|
||||
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
fileStream.end(() => resolve());
|
||||
fileStream.on("error", reject);
|
||||
});
|
||||
|
||||
fs.renameSync(tmpPath, dest);
|
||||
console.log(`[cloakbrowser] GeoIP database ready: ${dest}`);
|
||||
} catch (err) {
|
||||
if (fs.existsSync(tmpPath)) fs.unlinkSync(tmpPath);
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
function maybeTriggerUpdate(dbPath: string): void {
|
||||
try {
|
||||
const age = Date.now() - fs.statSync(dbPath).mtimeMs;
|
||||
if (age < GEOIP_UPDATE_INTERVAL_MS) return;
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
// Fire-and-forget background update
|
||||
downloadGeoipDb(dbPath).catch(() => {});
|
||||
}
|
||||
+36
-6
@@ -5,7 +5,7 @@
|
||||
|
||||
import type { Browser, BrowserContext } from "playwright-core";
|
||||
import type { LaunchOptions, LaunchContextOptions } from "./types.js";
|
||||
import { getDefaultStealthArgs } from "./config.js";
|
||||
import { DEFAULT_VIEWPORT, getDefaultStealthArgs } from "./config.js";
|
||||
import { ensureBinary } from "./download.js";
|
||||
import { parseProxyUrl } from "./proxy.js";
|
||||
|
||||
@@ -26,7 +26,8 @@ export async function launch(options: LaunchOptions = {}): Promise<Browser> {
|
||||
const { chromium } = await import("playwright-core");
|
||||
|
||||
const binaryPath = process.env.CLOAKBROWSER_BINARY_PATH || (await ensureBinary());
|
||||
const args = buildArgs(options);
|
||||
const resolved = await maybeResolveGeoip(options);
|
||||
const args = buildArgs({ ...options, ...resolved });
|
||||
|
||||
const browser = await chromium.launch({
|
||||
executablePath: binaryPath,
|
||||
@@ -59,15 +60,18 @@ export async function launch(options: LaunchOptions = {}): Promise<Browser> {
|
||||
export async function launchContext(
|
||||
options: LaunchContextOptions = {}
|
||||
): Promise<BrowserContext> {
|
||||
const browser = await launch(options);
|
||||
// Resolve geoip BEFORE launch() to avoid double-resolution
|
||||
const resolved = await maybeResolveGeoip(options);
|
||||
const browser = await launch({ ...options, ...resolved, geoip: false });
|
||||
|
||||
let context: BrowserContext;
|
||||
try {
|
||||
context = await browser.newContext({
|
||||
...(options.userAgent ? { userAgent: options.userAgent } : {}),
|
||||
...(options.viewport ? { viewport: options.viewport } : {}),
|
||||
...(options.locale ? { locale: options.locale } : {}),
|
||||
...(options.timezoneId ? { timezoneId: options.timezoneId } : {}),
|
||||
viewport: options.viewport ?? DEFAULT_VIEWPORT,
|
||||
...(resolved.locale ? { locale: resolved.locale } : {}),
|
||||
...(resolved.timezone ? { timezoneId: resolved.timezone } : {}),
|
||||
...(options.colorScheme ? { colorScheme: options.colorScheme } : {}),
|
||||
});
|
||||
} catch (err) {
|
||||
await browser.close();
|
||||
@@ -88,6 +92,25 @@ export async function launchContext(
|
||||
// Internal
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
async function maybeResolveGeoip(
|
||||
options: LaunchOptions
|
||||
): Promise<{ timezone?: string; locale?: string }> {
|
||||
if (!options.geoip || !options.proxy) return { timezone: options.timezone, locale: options.locale };
|
||||
if (options.timezone && options.locale) return { timezone: options.timezone, locale: options.locale };
|
||||
|
||||
const { resolveProxyGeo } = await import("./geoip.js");
|
||||
const { timezone: geoTz, locale: geoLocale } = await resolveProxyGeo(options.proxy);
|
||||
return {
|
||||
timezone: options.timezone ?? geoTz ?? undefined,
|
||||
locale: options.locale ?? geoLocale ?? undefined,
|
||||
};
|
||||
}
|
||||
|
||||
/** @internal Exposed for unit tests only. */
|
||||
export function _buildArgsForTest(options: LaunchOptions): string[] {
|
||||
return buildArgs(options);
|
||||
}
|
||||
|
||||
function buildArgs(options: LaunchOptions): string[] {
|
||||
const args: string[] = [];
|
||||
if (options.stealthArgs !== false) {
|
||||
@@ -96,5 +119,12 @@ function buildArgs(options: LaunchOptions): string[] {
|
||||
if (options.args) {
|
||||
args.push(...options.args);
|
||||
}
|
||||
// Timezone/locale flags — always inject when set
|
||||
if (options.timezone) {
|
||||
args.push(`--timezone=${options.timezone}`);
|
||||
}
|
||||
if (options.locale) {
|
||||
args.push(`--lang=${options.locale}`);
|
||||
}
|
||||
return args;
|
||||
}
|
||||
|
||||
+22
-1
@@ -26,7 +26,8 @@ export async function launch(options: LaunchOptions = {}): Promise<Browser> {
|
||||
const puppeteer = await import("puppeteer-core");
|
||||
|
||||
const binaryPath = process.env.CLOAKBROWSER_BINARY_PATH || (await ensureBinary());
|
||||
const args = buildArgs(options);
|
||||
const resolved = await maybeResolveGeoip(options);
|
||||
const args = buildArgs({ ...options, ...resolved });
|
||||
|
||||
// Puppeteer handles proxy via CLI args, not a separate option.
|
||||
// Chromium's --proxy-server does NOT support inline credentials,
|
||||
@@ -66,6 +67,20 @@ export async function launch(options: LaunchOptions = {}): Promise<Browser> {
|
||||
// Internal
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
async function maybeResolveGeoip(
|
||||
options: LaunchOptions
|
||||
): Promise<{ timezone?: string; locale?: string }> {
|
||||
if (!options.geoip || !options.proxy) return { timezone: options.timezone, locale: options.locale };
|
||||
if (options.timezone && options.locale) return { timezone: options.timezone, locale: options.locale };
|
||||
|
||||
const { resolveProxyGeo } = await import("./geoip.js");
|
||||
const { timezone: geoTz, locale: geoLocale } = await resolveProxyGeo(options.proxy);
|
||||
return {
|
||||
timezone: options.timezone ?? geoTz ?? undefined,
|
||||
locale: options.locale ?? geoLocale ?? undefined,
|
||||
};
|
||||
}
|
||||
|
||||
function buildArgs(options: LaunchOptions): string[] {
|
||||
const args: string[] = [];
|
||||
if (options.stealthArgs !== false) {
|
||||
@@ -74,5 +89,11 @@ function buildArgs(options: LaunchOptions): string[] {
|
||||
if (options.args) {
|
||||
args.push(...options.args);
|
||||
}
|
||||
if (options.timezone) {
|
||||
args.push(`--timezone=${options.timezone}`);
|
||||
}
|
||||
if (options.locale) {
|
||||
args.push(`--lang=${options.locale}`);
|
||||
}
|
||||
return args;
|
||||
}
|
||||
|
||||
@@ -11,6 +11,12 @@ export interface LaunchOptions {
|
||||
args?: string[];
|
||||
/** Include default stealth fingerprint args (default: true). Set false to use custom --fingerprint flags. */
|
||||
stealthArgs?: boolean;
|
||||
/** IANA timezone, e.g. "America/New_York". Sets --timezone binary flag. */
|
||||
timezone?: string;
|
||||
/** BCP 47 locale, e.g. "en-US". Sets --lang binary flag. */
|
||||
locale?: string;
|
||||
/** Auto-detect timezone/locale from proxy IP (requires: npm install mmdb-lib). */
|
||||
geoip?: boolean;
|
||||
/** Raw options passed directly to playwright/puppeteer launch(). */
|
||||
launchOptions?: Record<string, unknown>;
|
||||
}
|
||||
@@ -24,6 +30,8 @@ export interface LaunchContextOptions extends LaunchOptions {
|
||||
locale?: string;
|
||||
/** Timezone, e.g. "America/New_York". */
|
||||
timezoneId?: string;
|
||||
/** Color scheme preference — 'light', 'dark', or 'no-preference'. */
|
||||
colorScheme?: "light" | "dark" | "no-preference";
|
||||
}
|
||||
|
||||
export interface BinaryInfo {
|
||||
|
||||
@@ -6,6 +6,7 @@ import {
|
||||
getBinaryDir,
|
||||
getDownloadUrl,
|
||||
} from "../src/config.js";
|
||||
import { _buildArgsForTest } from "../src/playwright.js";
|
||||
|
||||
describe("config", () => {
|
||||
it("CHROMIUM_VERSION matches expected format", () => {
|
||||
@@ -65,3 +66,34 @@ describe("config", () => {
|
||||
expect(url).toContain("cloakbrowser.dev");
|
||||
});
|
||||
});
|
||||
|
||||
describe("buildArgs timezone/locale", () => {
|
||||
it("injects --timezone when timezone is set", () => {
|
||||
const args = _buildArgsForTest({ timezone: "America/New_York" });
|
||||
expect(args).toContain("--timezone=America/New_York");
|
||||
});
|
||||
|
||||
it("injects --lang when locale is set", () => {
|
||||
const args = _buildArgsForTest({ locale: "en-US" });
|
||||
expect(args).toContain("--lang=en-US");
|
||||
});
|
||||
|
||||
it("injects both when both are set", () => {
|
||||
const args = _buildArgsForTest({ timezone: "Europe/Berlin", locale: "de-DE" });
|
||||
expect(args).toContain("--timezone=Europe/Berlin");
|
||||
expect(args).toContain("--lang=de-DE");
|
||||
});
|
||||
|
||||
it("injects timezone/locale even when stealthArgs=false", () => {
|
||||
const args = _buildArgsForTest({ stealthArgs: false, timezone: "America/New_York", locale: "en-US" });
|
||||
expect(args).toContain("--timezone=America/New_York");
|
||||
expect(args).toContain("--lang=en-US");
|
||||
expect(args.some(a => a.startsWith("--fingerprint="))).toBe(false);
|
||||
});
|
||||
|
||||
it("does not inject flags when not set", () => {
|
||||
const args = _buildArgsForTest({});
|
||||
expect(args.some(a => a.startsWith("--timezone="))).toBe(false);
|
||||
expect(args.some(a => a.startsWith("--lang="))).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
import { describe, it, expect } from "vitest";
|
||||
import { COUNTRY_LOCALE_MAP, resolveProxyIp } from "../src/geoip.js";
|
||||
|
||||
describe("resolveProxyIp", () => {
|
||||
it("returns literal IPv4 from proxy URL", async () => {
|
||||
expect(await resolveProxyIp("http://10.50.96.5:8888")).toBe("10.50.96.5");
|
||||
});
|
||||
|
||||
it("handles proxy URL with credentials", async () => {
|
||||
expect(await resolveProxyIp("http://user:pass@10.50.96.5:8888")).toBe(
|
||||
"10.50.96.5"
|
||||
);
|
||||
});
|
||||
|
||||
it("resolves localhost", async () => {
|
||||
const ip = await resolveProxyIp("http://localhost:8888");
|
||||
expect(ip).toBeTruthy();
|
||||
expect(["127.0.0.1", "::1"]).toContain(ip);
|
||||
});
|
||||
|
||||
it("returns null for invalid URL", async () => {
|
||||
expect(await resolveProxyIp("not-a-url")).toBeNull();
|
||||
});
|
||||
|
||||
it("returns null for empty string", async () => {
|
||||
expect(await resolveProxyIp("")).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe("COUNTRY_LOCALE_MAP", () => {
|
||||
it("contains common countries", () => {
|
||||
for (const code of ["US", "GB", "DE", "FR", "JP", "BR", "IL", "RU"]) {
|
||||
expect(COUNTRY_LOCALE_MAP[code]).toBeDefined();
|
||||
}
|
||||
});
|
||||
|
||||
it("values are BCP 47 language-REGION format", () => {
|
||||
for (const [code, locale] of Object.entries(COUNTRY_LOCALE_MAP)) {
|
||||
const parts = locale.split("-");
|
||||
expect(parts).toHaveLength(2);
|
||||
expect(parts[0]).toMatch(/^[a-z]{2,3}$/);
|
||||
expect(parts[1]).toMatch(/^[A-Z]{2}$/);
|
||||
}
|
||||
});
|
||||
});
|
||||
+4
-1
@@ -42,10 +42,13 @@ classifiers = [
|
||||
"Topic :: Software Development :: Testing",
|
||||
]
|
||||
dependencies = [
|
||||
"playwright>=1.40",
|
||||
"patchright>=1.40",
|
||||
"httpx>=0.24",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
geoip = ["geoip2>=4.0"]
|
||||
|
||||
[project.urls]
|
||||
Homepage = "https://github.com/CloakHQ/CloakBrowser"
|
||||
Documentation = "https://github.com/CloakHQ/CloakBrowser#readme"
|
||||
|
||||
@@ -0,0 +1,46 @@
|
||||
"""Unit tests for _build_args timezone/locale injection."""
|
||||
|
||||
from cloakbrowser.browser import _build_args
|
||||
|
||||
|
||||
def test_timezone_injected():
|
||||
"""--timezone flag should appear when timezone is set."""
|
||||
args = _build_args(stealth_args=True, extra_args=None, timezone="America/New_York")
|
||||
assert "--timezone=America/New_York" in args
|
||||
|
||||
|
||||
def test_locale_injected():
|
||||
"""--lang flag should appear when locale is set."""
|
||||
args = _build_args(stealth_args=True, extra_args=None, locale="en-US")
|
||||
assert "--lang=en-US" in args
|
||||
|
||||
|
||||
def test_both_injected():
|
||||
"""Both flags should appear when both are set."""
|
||||
args = _build_args(stealth_args=True, extra_args=None, timezone="Europe/Berlin", locale="de-DE")
|
||||
assert "--timezone=Europe/Berlin" in args
|
||||
assert "--lang=de-DE" in args
|
||||
|
||||
|
||||
def test_timezone_independent_of_stealth_args():
|
||||
"""--timezone should be injected even when stealth_args=False."""
|
||||
args = _build_args(stealth_args=False, extra_args=None, timezone="America/New_York", locale="en-US")
|
||||
assert "--timezone=America/New_York" in args
|
||||
assert "--lang=en-US" in args
|
||||
# No stealth fingerprint args
|
||||
assert not any(a.startswith("--fingerprint=") for a in args)
|
||||
|
||||
|
||||
def test_no_flags_when_not_set():
|
||||
"""No timezone/lang flags when params are None."""
|
||||
args = _build_args(stealth_args=True, extra_args=None)
|
||||
assert not any(a.startswith("--timezone=") for a in args)
|
||||
assert not any(a.startswith("--lang=") for a in args)
|
||||
|
||||
|
||||
def test_extra_args_preserved():
|
||||
"""Extra args should still be included alongside timezone/locale."""
|
||||
args = _build_args(stealth_args=True, extra_args=["--disable-gpu"], timezone="Asia/Tokyo", locale="ja-JP")
|
||||
assert "--disable-gpu" in args
|
||||
assert "--timezone=Asia/Tokyo" in args
|
||||
assert "--lang=ja-JP" in args
|
||||
@@ -0,0 +1,138 @@
|
||||
"""Unit tests for GeoIP-based timezone/locale detection."""
|
||||
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
from cloakbrowser.browser import _maybe_resolve_geoip
|
||||
from cloakbrowser.geoip import (
|
||||
COUNTRY_LOCALE_MAP,
|
||||
_resolve_proxy_ip,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# _resolve_proxy_ip
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_resolve_literal_ipv4():
|
||||
assert _resolve_proxy_ip("http://10.50.96.5:8888") == "10.50.96.5"
|
||||
|
||||
|
||||
def test_resolve_literal_ipv4_with_auth():
|
||||
assert _resolve_proxy_ip("http://user:pass@10.50.96.5:8888") == "10.50.96.5"
|
||||
|
||||
|
||||
def test_resolve_literal_ipv6():
|
||||
ip = _resolve_proxy_ip("http://[::1]:8888")
|
||||
assert ip == "::1"
|
||||
|
||||
|
||||
def test_resolve_hostname():
|
||||
"""DNS resolution of a known hostname should return an IP."""
|
||||
ip = _resolve_proxy_ip("http://localhost:8888")
|
||||
assert ip is not None
|
||||
assert ip in ("127.0.0.1", "::1")
|
||||
|
||||
|
||||
def test_resolve_invalid_url():
|
||||
assert _resolve_proxy_ip("not-a-url") is None
|
||||
|
||||
|
||||
def test_resolve_empty():
|
||||
assert _resolve_proxy_ip("") is None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# COUNTRY_LOCALE_MAP
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_locale_map_has_common_countries():
|
||||
for code in ("US", "GB", "DE", "FR", "JP", "BR", "IL", "RU"):
|
||||
assert code in COUNTRY_LOCALE_MAP, f"Missing {code}"
|
||||
|
||||
|
||||
def test_locale_map_values_are_bcp47():
|
||||
"""All locales should be language-REGION format."""
|
||||
for code, locale in COUNTRY_LOCALE_MAP.items():
|
||||
parts = locale.split("-")
|
||||
assert len(parts) == 2, f"{code}: {locale} not language-REGION"
|
||||
assert parts[0].islower(), f"{code}: language part should be lowercase"
|
||||
assert parts[1].isupper(), f"{code}: region part should be uppercase"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# resolve_proxy_geo fallbacks
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_resolve_geo_raises_when_geoip2_missing():
|
||||
"""Should raise ImportError with install instructions when geoip2 not installed."""
|
||||
with patch.dict("sys.modules", {"geoip2": None, "geoip2.database": None}):
|
||||
from importlib import reload
|
||||
import cloakbrowser.geoip as geoip_mod
|
||||
reload(geoip_mod)
|
||||
with pytest.raises(ImportError, match="pip install cloakbrowser"):
|
||||
geoip_mod.resolve_proxy_geo("http://10.50.96.5:8888")
|
||||
# Restore
|
||||
reload(geoip_mod)
|
||||
|
||||
|
||||
def test_resolve_geo_returns_none_when_db_missing():
|
||||
"""Should return (None, None) when DB file doesn't exist."""
|
||||
mock_geoip2 = type("module", (), {"database": type("db", (), {"Reader": None})})()
|
||||
with patch.dict("sys.modules", {"geoip2": mock_geoip2, "geoip2.database": mock_geoip2.database}):
|
||||
with patch("cloakbrowser.geoip._ensure_geoip_db", return_value=None):
|
||||
with patch("cloakbrowser.geoip._resolve_exit_ip", return_value=None):
|
||||
from cloakbrowser.geoip import resolve_proxy_geo
|
||||
assert resolve_proxy_geo("http://10.50.96.5:8888") == (None, None)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# _maybe_resolve_geoip (browser.py helper)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_maybe_resolve_skips_when_geoip_false():
|
||||
tz, loc = _maybe_resolve_geoip(False, "http://proxy:8080", None, None)
|
||||
assert tz is None
|
||||
assert loc is None
|
||||
|
||||
|
||||
def test_maybe_resolve_skips_when_no_proxy():
|
||||
tz, loc = _maybe_resolve_geoip(True, None, None, None)
|
||||
assert tz is None
|
||||
assert loc is None
|
||||
|
||||
|
||||
def test_maybe_resolve_skips_when_both_explicit():
|
||||
"""Explicit values should not trigger geoip resolution."""
|
||||
tz, loc = _maybe_resolve_geoip(True, "http://proxy:8080", "Europe/Berlin", "de-DE")
|
||||
assert tz == "Europe/Berlin"
|
||||
assert loc == "de-DE"
|
||||
|
||||
|
||||
def test_maybe_resolve_fills_missing_timezone():
|
||||
"""When only locale is explicit, geoip should fill timezone."""
|
||||
with patch("cloakbrowser.geoip.resolve_proxy_geo", return_value=("America/New_York", "en-US")):
|
||||
tz, loc = _maybe_resolve_geoip(True, "http://proxy:8080", None, "fr-FR")
|
||||
assert tz == "America/New_York"
|
||||
assert loc == "fr-FR" # Explicit wins
|
||||
|
||||
|
||||
def test_maybe_resolve_fills_missing_locale():
|
||||
"""When only timezone is explicit, geoip should fill locale."""
|
||||
with patch("cloakbrowser.geoip.resolve_proxy_geo", return_value=("America/New_York", "en-US")):
|
||||
tz, loc = _maybe_resolve_geoip(True, "http://proxy:8080", "Asia/Tokyo", None)
|
||||
assert tz == "Asia/Tokyo" # Explicit wins
|
||||
assert loc == "en-US"
|
||||
|
||||
|
||||
def test_maybe_resolve_fills_both():
|
||||
"""When neither is set, geoip should fill both."""
|
||||
with patch("cloakbrowser.geoip.resolve_proxy_geo", return_value=("Europe/Berlin", "de-DE")):
|
||||
tz, loc = _maybe_resolve_geoip(True, "http://proxy:8080", None, None)
|
||||
assert tz == "Europe/Berlin"
|
||||
assert loc == "de-DE"
|
||||
+130
-19
@@ -4,14 +4,19 @@ These tests verify that the stealth Chromium binary passes common
|
||||
bot detection checks. They require network access.
|
||||
"""
|
||||
|
||||
import os
|
||||
import time
|
||||
|
||||
import pytest
|
||||
from cloakbrowser import launch
|
||||
|
||||
PROXY = os.environ.get("CLOAKBROWSER_TEST_PROXY")
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def browser():
|
||||
"""Shared browser instance for stealth tests."""
|
||||
b = launch(headless=True)
|
||||
b = launch(headless=True, proxy=PROXY)
|
||||
yield b
|
||||
b.close()
|
||||
|
||||
@@ -48,7 +53,7 @@ class TestWebDriverDetection:
|
||||
"""Must have browser plugins (real Chrome has 5)."""
|
||||
page.goto("https://example.com")
|
||||
count = page.evaluate("navigator.plugins.length")
|
||||
assert count >= 1, f"Expected plugins, got {count}"
|
||||
assert count >= 5, f"Expected 5+ plugins (real Chrome), got {count}"
|
||||
|
||||
def test_languages_present(self, page):
|
||||
"""navigator.languages must be populated."""
|
||||
@@ -81,28 +86,134 @@ class TestBotDetectionSites:
|
||||
Mark with pytest -m slow to skip in CI.
|
||||
"""
|
||||
|
||||
@pytest.mark.slow
|
||||
def test_bot_sannysoft(self, page):
|
||||
"""bot.sannysoft.com — all checks should pass (0 failures)."""
|
||||
page.goto("https://bot.sannysoft.com", wait_until="networkidle", timeout=30000)
|
||||
time.sleep(3)
|
||||
|
||||
results = page.evaluate("""() => {
|
||||
const rows = document.querySelectorAll('table tr');
|
||||
const failed = [];
|
||||
let total = 0;
|
||||
rows.forEach(r => {
|
||||
const cells = r.querySelectorAll('td');
|
||||
if (cells.length >= 2) {
|
||||
total++;
|
||||
const cls = cells[1].className || '';
|
||||
if (cls.includes('failed')) {
|
||||
failed.push(cells[0].innerText.trim());
|
||||
}
|
||||
}
|
||||
});
|
||||
return {total, failed};
|
||||
}""")
|
||||
|
||||
failed = results["failed"]
|
||||
assert len(failed) == 0, f"Sannysoft failures: {', '.join(failed)}"
|
||||
|
||||
@pytest.mark.slow
|
||||
def test_bot_incolumitas(self, page):
|
||||
"""bot.incolumitas.com should detect minimal flags."""
|
||||
page.goto("https://bot.incolumitas.com", timeout=30000)
|
||||
page.wait_for_timeout(5000)
|
||||
# Check that we're not immediately flagged
|
||||
title = page.title()
|
||||
assert title # Page loaded successfully
|
||||
"""bot.incolumitas.com — max 1 failure (WEBDRIVER false positive expected)."""
|
||||
page.goto("https://bot.incolumitas.com", wait_until="networkidle", timeout=30000)
|
||||
time.sleep(12)
|
||||
|
||||
# Known acceptable failures (not browser fingerprint issues):
|
||||
# - WEBDRIVER: spec-level false positive across all builds
|
||||
# - connectionRTT: detects datacenter/proxy network latency, not browser
|
||||
KNOWN_ACCEPTABLE = {"WEBDRIVER", "connectionRTT"}
|
||||
|
||||
results = page.evaluate("""() => {
|
||||
const text = document.body.innerText;
|
||||
const okMatches = text.match(/"\\w+":\\s*"OK"/g) || [];
|
||||
const failMatches = text.match(/"\\w+":\\s*"FAIL"/g) || [];
|
||||
const failedTests = failMatches.map(m => m.match(/"(\\w+)"/)[1]);
|
||||
return {passed: okMatches.length, failed: failMatches.length, failedTests};
|
||||
}""")
|
||||
|
||||
failed_names = results["failedTests"]
|
||||
real_failures = [f for f in failed_names if f not in KNOWN_ACCEPTABLE]
|
||||
assert len(real_failures) == 0, f"Incolumitas unexpected failures: {', '.join(real_failures)}"
|
||||
|
||||
@pytest.mark.slow
|
||||
def test_browserscan(self, page):
|
||||
"""BrowserScan bot detection should show NORMAL."""
|
||||
page.goto("https://www.browserscan.net/bot-detection", timeout=30000)
|
||||
page.wait_for_timeout(5000)
|
||||
title = page.title()
|
||||
assert title # Page loaded
|
||||
"""BrowserScan bot detection — 0 abnormal checks."""
|
||||
page.goto("https://www.browserscan.net/bot-detection", wait_until="networkidle", timeout=30000)
|
||||
time.sleep(5)
|
||||
|
||||
results = page.evaluate("""() => {
|
||||
const text = document.body.innerText;
|
||||
const normalMatches = text.match(/Normal/g);
|
||||
const abnormalMatches = text.match(/Abnormal/g);
|
||||
return {
|
||||
normal: normalMatches ? normalMatches.length : 0,
|
||||
abnormal: abnormalMatches ? abnormalMatches.length : 0
|
||||
};
|
||||
}""")
|
||||
|
||||
assert results["abnormal"] == 0, \
|
||||
f"BrowserScan: {results['abnormal']} abnormal, {results['normal']} normal"
|
||||
|
||||
@pytest.mark.slow
|
||||
def test_device_and_browser_info(self, page):
|
||||
"""deviceandbrowserinfo.com should report isBot: false."""
|
||||
page.goto("https://deviceandbrowserinfo.com/are_you_a_bot", timeout=30000)
|
||||
page.wait_for_timeout(5000)
|
||||
content = page.content()
|
||||
# The page shows bot detection results
|
||||
assert "deviceandbrowserinfo" in page.url.lower()
|
||||
"""deviceandbrowserinfo.com — isBot must be false."""
|
||||
page.goto("https://deviceandbrowserinfo.com/are_you_a_bot", wait_until="domcontentloaded", timeout=30000)
|
||||
time.sleep(8)
|
||||
|
||||
results = page.evaluate("""() => {
|
||||
const text = document.body.innerText;
|
||||
const botMatch = text.match(/"isBot":\\s*(true|false)/);
|
||||
const isBot = botMatch ? botMatch[1] === 'true' : null;
|
||||
const checks = {};
|
||||
['isBot', 'hasBotUserAgent', 'hasWebdriverTrue', 'isHeadlessChrome',
|
||||
'isAutomatedWithCDP', 'hasSuspiciousWeakSignals', 'isPlaywright',
|
||||
'hasInconsistentChromeObject'].forEach(p => {
|
||||
const match = text.match(new RegExp('"' + p + '":\\s*(true|false)'));
|
||||
if (match) checks[p] = match[1] === 'true';
|
||||
});
|
||||
return {isBot, checks};
|
||||
}""")
|
||||
|
||||
assert results["isBot"] is False, f"Detected as bot! Checks: {results['checks']}"
|
||||
|
||||
@pytest.mark.slow
|
||||
def test_fingerprintjs(self, page):
|
||||
"""FingerprintJS — must not be blocked, should see flight data."""
|
||||
page.goto("https://demo.fingerprint.com/web-scraping", wait_until="domcontentloaded", timeout=30000)
|
||||
time.sleep(8)
|
||||
|
||||
try:
|
||||
page.click("button:has-text('Search')", timeout=5000)
|
||||
time.sleep(5)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
results = page.evaluate("""() => {
|
||||
const text = document.body.innerText;
|
||||
const hasFlights = text.includes('Price per adult') || text.includes('$');
|
||||
const isBlocked = text.includes('request was blocked') || text.includes('bot visit detected');
|
||||
return {passed: hasFlights && !isBlocked, isBlocked, hasFlights};
|
||||
}""")
|
||||
|
||||
assert not results["isBlocked"], "FingerprintJS blocked us as a bot"
|
||||
assert results["passed"], "FingerprintJS: no flight data shown"
|
||||
|
||||
@pytest.mark.slow
|
||||
def test_recaptcha_v3(self, page):
|
||||
"""reCAPTCHA v3 — score must be >= 0.7."""
|
||||
page.goto(
|
||||
"https://recaptcha-demo.appspot.com/recaptcha-v3-request-scores.php",
|
||||
wait_until="domcontentloaded",
|
||||
timeout=60000,
|
||||
)
|
||||
time.sleep(8)
|
||||
|
||||
results = page.evaluate("""() => {
|
||||
const text = document.body.innerText;
|
||||
const scoreMatch = text.match(/"score":\\s*(\\d+\\.\\d+)/);
|
||||
return {score: scoreMatch ? parseFloat(scoreMatch[1]) : null};
|
||||
}""")
|
||||
|
||||
score = results["score"]
|
||||
assert score is not None, "Could not extract reCAPTCHA score"
|
||||
assert score >= 0.7, f"reCAPTCHA score too low: {score}"
|
||||
|
||||
Reference in New Issue
Block a user