mirror of
https://github.com/CloakHQ/CloakBrowser.git
synced 2026-06-23 11:41:46 +02:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
4c2e06682b | ||
|
|
cb08a602b0 | ||
|
|
8eb666885f | ||
|
|
f417fe2530 | ||
|
|
c65939af14 | ||
|
|
2f1f592b3a | ||
|
|
0bbc170747 | ||
|
|
6506b5fe44 | ||
|
|
1082c810af | ||
|
|
67efadef26 | ||
|
|
59b9d71684 | ||
|
|
f480958ba7 | ||
|
|
8ffaf86abe | ||
|
|
8c76a68cb5 | ||
|
|
cee166c2d2 | ||
|
|
b07797f963 | ||
|
|
31c04d5bcc | ||
|
|
cc501d8ef6 | ||
|
|
8cecebf118 | ||
|
|
179531fd17 | ||
|
|
4d96db1448 | ||
|
|
4e809b9678 | ||
|
|
23ae521832 | ||
|
|
c2eb0ef21a | ||
|
|
2d7894ad68 | ||
|
|
efab732142 | ||
|
|
924f401265 | ||
|
|
6ea8391fac |
@@ -0,0 +1,45 @@
|
||||
name: Release Binary
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
tag:
|
||||
description: 'Release tag (e.g. chromium-v145.0.7718.0)'
|
||||
required: true
|
||||
title:
|
||||
description: 'Release title (e.g. Chromium v145 — Stealth Build)'
|
||||
required: true
|
||||
default: 'Stealth Chromium Build'
|
||||
patch_count:
|
||||
description: 'Number of fingerprint patches'
|
||||
required: true
|
||||
default: '16'
|
||||
|
||||
jobs:
|
||||
release:
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: write
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Create release
|
||||
uses: softprops/action-gh-release@v2
|
||||
with:
|
||||
tag_name: ${{ github.event.inputs.tag }}
|
||||
name: "${{ github.event.inputs.title }}"
|
||||
body: |
|
||||
## Stealth Chromium Build
|
||||
|
||||
Pre-built Chromium with ${{ github.event.inputs.patch_count }} source-level fingerprint patches.
|
||||
|
||||
### Install
|
||||
```bash
|
||||
pip install cloakbrowser # Python
|
||||
npm install cloakbrowser # JavaScript
|
||||
# Binary auto-downloads on first launch
|
||||
```
|
||||
|
||||
> Binary integrity is verified automatically via SHA-256 checksums on download.
|
||||
>
|
||||
> Release signed with CloakHQ GPG key: `C60C0DDC9D0DE2DD`
|
||||
+16
@@ -37,6 +37,10 @@ htmlcov/
|
||||
CLAUDE.md
|
||||
.claude/
|
||||
|
||||
# JavaScript / Node.js
|
||||
js/node_modules/
|
||||
js/dist/
|
||||
|
||||
# Distribution
|
||||
*.tar.gz
|
||||
*.whl
|
||||
@@ -45,3 +49,15 @@ AGENTS.md
|
||||
|
||||
# Private docs (launch posts, strategy)
|
||||
docs/
|
||||
|
||||
# Internal test infrastructure (Docker, VPS-specific)
|
||||
test-infra/
|
||||
|
||||
# Website (deployed separately)
|
||||
site/
|
||||
|
||||
# Release scripts
|
||||
publish.sh
|
||||
deploy.sh
|
||||
.env
|
||||
debug
|
||||
|
||||
+113
@@ -0,0 +1,113 @@
|
||||
# Changelog
|
||||
|
||||
All notable changes to CloakBrowser — wrapper and binary — are documented here.
|
||||
|
||||
Changes are tagged: **[wrapper]** for Python/JS wrapper, **[binary]** for Chromium patches.
|
||||
|
||||
---
|
||||
|
||||
## [0.3.0] — Unreleased
|
||||
|
||||
Chromium v145 upgrade. 26 fingerprint patches (up from 16). New download verification and fallback system. Pending: macOS v145 binary builds.
|
||||
|
||||
### 2026-03-01
|
||||
|
||||
- **[wrapper]** Upgrade wrapper to Chromium v145.0.7632.109
|
||||
- **[wrapper]** Add GitHub Releases fallback when primary download mirror is unavailable
|
||||
- **[wrapper]** Add SHA-256 checksum verification for binary downloads
|
||||
- **[wrapper]** Wire timezone and locale params to Chromium binary flags
|
||||
- **[wrapper]** Add device memory to default stealth args
|
||||
- **[wrapper]** JS: add `colorScheme` support, guard download fallback against partial failures
|
||||
|
||||
### 2026-02-28
|
||||
|
||||
- **[binary]** Enforce strict flag discipline — patches only activate when explicitly configured via command-line flags
|
||||
- **[binary]** Improved fingerprint consistency across multiple browser APIs
|
||||
- **[binary]** 3 new fingerprint patches + bug fixes in existing patches
|
||||
- **[binary]** New command-line flag for device memory spoofing
|
||||
- **[infra]** Automated test matrix: 8 groups, 41+ tests across core stealth, fingerprint noise, bot detection, reCAPTCHA, TLS, Turnstile, residential proxy, and enterprise reCAPTCHA
|
||||
- **[infra]** Docker-based test runner with subprocess isolation per test group
|
||||
|
||||
### 2026-02-25
|
||||
|
||||
- **[binary]** Reduced automation markers visible to detection scripts
|
||||
- **[binary]** Added browser API support at build time
|
||||
- **[binary]** Improved screen property consistency
|
||||
|
||||
### 2026-02-24
|
||||
|
||||
- **[binary]** Comprehensive fingerprint audit and hardening pass
|
||||
- **[binary]** Fixed font rendering edge case on cross-platform spoofing
|
||||
- **[binary]** 4 new fingerprint patches
|
||||
|
||||
### 2026-02-22
|
||||
|
||||
- **[binary]** Start Chromium v145 build (v145.0.7632.109)
|
||||
- **[binary]** 24 fingerprint patches ported and adapted
|
||||
|
||||
---
|
||||
|
||||
## [0.2.2] — 2026-03-01
|
||||
|
||||
### 2026-03-01
|
||||
|
||||
- **[wrapper]** Fix: replace `page.wait_for_timeout()` with `time.sleep()` to avoid timing leak
|
||||
- **[wrapper]** Add auto-detect timezone and locale from proxy IP via GeoIP lookup
|
||||
- **[binary]** CDP detection vector audit and hardening
|
||||
|
||||
---
|
||||
|
||||
## [0.2.0] — 2026-02-27
|
||||
|
||||
macOS platform release. JavaScript/TypeScript wrapper. Self-hosted binary mirror.
|
||||
|
||||
### 2026-02-27
|
||||
|
||||
- **[wrapper]** Add macOS support: Apple Silicon (arm64) and Intel (x64) binary downloads
|
||||
- **[wrapper]** Add GPG-signed release workflow via GitHub Actions
|
||||
- **[wrapper]** Fix macOS binary download: preserve `.app` symlinks, remove quarantine xattrs
|
||||
- **[wrapper]** Add real bot detection assertions to stealth tests
|
||||
- **[wrapper]** Bump version to 0.2.0
|
||||
|
||||
### 2026-02-26
|
||||
|
||||
- **[wrapper]** Switch binary downloads to self-hosted mirror (`cloakbrowser.dev`) as GitHub backup
|
||||
- **[wrapper]** Set up GitLab mirror at `gitlab.com/CloakHQ/cloakbrowser`
|
||||
|
||||
### 2026-02-25
|
||||
|
||||
- **[wrapper]** Move binary releases from separate repo to wrapper repo
|
||||
- **[wrapper]** Add auto-update check on launch
|
||||
- **[infra]** Initial Docker test infrastructure + matrix test runner
|
||||
|
||||
### 2026-02-24
|
||||
|
||||
- **[wrapper]** Add JavaScript/TypeScript wrapper with Playwright + Puppeteer support (`npm install cloakbrowser`)
|
||||
- **[wrapper]** Fix proxy authentication credentials support in URL (closes #4)
|
||||
|
||||
---
|
||||
|
||||
## [0.1.4] — 2026-02-23
|
||||
|
||||
### 2026-02-23
|
||||
|
||||
- **[wrapper]** Stealth hardening: additional launch args and detection evasion improvements
|
||||
- **[wrapper]** Full test suite rewrite with real detection site assertions
|
||||
- **[wrapper]** Add Docker support with Dockerfile and compose config
|
||||
- **[wrapper]** Add headed mode documentation
|
||||
|
||||
---
|
||||
|
||||
## [0.1.0] — 2026-02-22
|
||||
|
||||
Initial release. Chromium v142 with 16 fingerprint patches.
|
||||
|
||||
### 2026-02-22
|
||||
|
||||
- **[binary]** Chromium v142.0.7444.175 with 16 source-level fingerprint patches
|
||||
- **[binary]** Fix browser brand string to match Chrome 142 format
|
||||
- **[wrapper]** `launch()` and `launch_async()` — drop-in Playwright replacements
|
||||
- **[wrapper]** Auto-download binary from GitHub Releases, cached in `~/.cloakbrowser/`
|
||||
- **[wrapper]** Linux x64 platform support
|
||||
- **[wrapper]** Passes 14/14 bot detection tests
|
||||
- **[wrapper]** reCAPTCHA v3: 0.9 (server-verified), Cloudflare Turnstile: pass
|
||||
+5
-2
@@ -1,11 +1,14 @@
|
||||
FROM python:3.12-slim
|
||||
|
||||
# Playwright system deps
|
||||
# Chromium system deps (matches fingerprint-chromium 142+ requirements)
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
libnss3 libnspr4 libatk1.0-0 libatk-bridge2.0-0 libcups2 \
|
||||
libdbus-1-3 libdrm2 libxkbcommon0 libatspi2.0-0 libxcomposite1 \
|
||||
libxdamage1 libxfixes3 libxrandr2 libgbm1 libpango-1.0-0 \
|
||||
libcairo2 libasound2 libx11-xcb1 \
|
||||
libcairo2 libasound2 libx11-xcb1 libfontconfig1 libx11-6 \
|
||||
libxcb1 libxext6 libxshmfence1 \
|
||||
libglib2.0-0 libgtk-3-0 libpangocairo-1.0-0 libcairo-gobject2 \
|
||||
libgdk-pixbuf-2.0-0 libxss1 libxtst6 fonts-liberation \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
@@ -1,25 +1,34 @@
|
||||
<p align="center">
|
||||
<img src="https://i.imgur.com/cqkp6fG.png" width="500" alt="CloakBrowser">
|
||||
</p>
|
||||
|
||||
# CloakBrowser
|
||||
|
||||
[](https://pypi.org/project/cloakbrowser/)
|
||||
[](https://pypi.org/project/cloakbrowser/)
|
||||
[](LICENSE)
|
||||
<p align="center">
|
||||
<a href="https://pypi.org/project/cloakbrowser/"><img src="https://img.shields.io/pypi/v/cloakbrowser" alt="PyPI"></a>
|
||||
<a href="https://www.npmjs.com/package/cloakbrowser"><img src="https://img.shields.io/npm/v/cloakbrowser" alt="npm"></a>
|
||||
<a href="https://pypi.org/project/cloakbrowser/"><img src="https://img.shields.io/pypi/pyversions/cloakbrowser" alt="Python"></a>
|
||||
<a href="LICENSE"><img src="https://img.shields.io/github/license/CloakHQ/CloakBrowser" alt="License"></a>
|
||||
<br>
|
||||
<a href="https://github.com/CloakHQ/CloakBrowser"><img src="https://img.shields.io/github/stars/CloakHQ/CloakBrowser" alt="Stars"></a>
|
||||
<a href="https://pepy.tech/projects/cloakbrowser"><img src="https://img.shields.io/pepy/dt/cloakbrowser?label=pypi&logo=pypi&logoColor=white" alt="PyPI Downloads"></a>
|
||||
<a href="https://www.npmjs.com/package/cloakbrowser"><img src="https://img.shields.io/npm/dt/cloakbrowser?label=npm&logo=npm&logoColor=white" alt="npm Downloads"></a>
|
||||
<a href="https://github.com/CloakHQ/CloakBrowser"><img src="https://img.shields.io/github/last-commit/CloakHQ/CloakBrowser" alt="Last Commit"></a>
|
||||
</p>
|
||||
|
||||
**Stealth Chromium that passes every bot detection test.**
|
||||
|
||||
Drop-in Playwright replacement. Same API, same code — just swap the import. Your browser now scores **0.9 on reCAPTCHA v3**, passes **Cloudflare Turnstile**, and clears **14 out of 14** stealth detection tests.
|
||||
Drop-in Playwright/Puppeteer replacement for Python and JavaScript. Same API, same code — just swap the import. Your browser now scores **0.9 on reCAPTCHA v3**, passes **Cloudflare Turnstile**, and clears **30 out of 30** stealth detection tests.
|
||||
|
||||
> CloakBrowser is an open-source Python library that gives you an undetectable browser for web automation and data collection. It wraps a custom-built Chromium binary with 16 source-level fingerprint patches — modifications compiled directly into the browser's C++ code, not bolted on via JavaScript injection or config flags.
|
||||
>
|
||||
> Most stealth tools (playwright-stealth, undetected-chromedriver, puppeteer-extra) work at the config level: they override JavaScript properties, set browser flags, or inject scripts to mask automation signals. These approaches break every time Chrome updates its detection surface — and Cloudflare, reCAPTCHA, and other bot detection systems update constantly.
|
||||
>
|
||||
> CloakBrowser takes a fundamentally different approach. The fingerprint patches modify Chromium's rendering pipeline at the source level: canvas output, WebGL renderer strings, audio processing fingerprints, font enumeration, hardware concurrency, client rectangles, GPU info, timezone handling, and the WebDriver flag are all patched in C++ before compilation. The resulting binary produces fingerprints indistinguishable from a real user's Chrome installation. TLS fingerprints (ja3n, ja4, akamai hash) match genuine Chrome exactly because the browser IS Chrome — just with different fingerprint values compiled in.
|
||||
>
|
||||
> The result: a 0.9 score on Google's reCAPTCHA v3 (human-level, versus 0.1 for stock Playwright), clean passes on Cloudflare Turnstile managed challenges, zero bot flags on FingerprintJS, BrowserScan, and deviceandbrowserinfo.com, and undetected CDP (Chrome DevTools Protocol) usage. 14 out of 14 major detection tests passed.
|
||||
>
|
||||
> CloakBrowser fills the vacuum left by Camoufox (the popular Firefox-based stealth browser whose maintainer stepped down in 2025). Unlike Camoufox, CloakBrowser is Chromium-based — meaning native Playwright API support, a massive ecosystem of existing tools and scripts, and TLS fingerprints that match the browser 65%+ of the internet actually uses.
|
||||
>
|
||||
> Install with pip, write standard Playwright code, and let the binary handle the stealth. No new API to learn. No configuration to tune. No CAPTCHA-solving services to pay for. Just a browser that works on sites where others get blocked.
|
||||
- 🔒 **26 source-level C++ patches** — not JS injection, not config flags
|
||||
- 🛡️ **CDP stealth built-in** — powered by [Patchright](https://github.com/Kaliiiiiiiiii-Vinyzu/patchright), hides Playwright's automation signals
|
||||
- 🎯 **0.9 reCAPTCHA v3 score** — human-level, server-verified
|
||||
- ☁️ **Passes Cloudflare Turnstile**, FingerprintJS, BrowserScan — 30/30 tests
|
||||
- 🔄 **Drop-in replacement** — works with Playwright (Python & JS) and Puppeteer (JS)
|
||||
- 📦 **`pip install cloakbrowser`** or **`npm install cloakbrowser`** — binary auto-downloads, zero config
|
||||
- 🦊 **Fills the Camoufox vacuum** — Chromium-based, actively maintained
|
||||
|
||||
**Python:**
|
||||
```python
|
||||
from cloakbrowser import launch
|
||||
|
||||
@@ -29,21 +38,61 @@ page.goto("https://protected-site.com") # no more blocks
|
||||
browser.close()
|
||||
```
|
||||
|
||||
**JavaScript (Playwright):**
|
||||
```javascript
|
||||
import { launch } from 'cloakbrowser';
|
||||
|
||||
const browser = await launch();
|
||||
const page = await browser.newPage();
|
||||
await page.goto('https://protected-site.com');
|
||||
await browser.close();
|
||||
```
|
||||
|
||||
**JavaScript (Puppeteer):**
|
||||
```javascript
|
||||
import { launch } from 'cloakbrowser/puppeteer';
|
||||
|
||||
const browser = await launch();
|
||||
const page = await browser.newPage();
|
||||
await page.goto('https://protected-site.com');
|
||||
await browser.close();
|
||||
```
|
||||
|
||||
## Install
|
||||
|
||||
**Python:**
|
||||
```bash
|
||||
pip install cloakbrowser
|
||||
```
|
||||
|
||||
**JavaScript / Node.js:**
|
||||
```bash
|
||||
# With Playwright
|
||||
npm install cloakbrowser playwright-core
|
||||
|
||||
# With Puppeteer
|
||||
npm install cloakbrowser puppeteer-core
|
||||
```
|
||||
|
||||
On first run, the stealth Chromium binary is automatically downloaded (~200MB, cached locally).
|
||||
|
||||
## What's New in v0.3.0
|
||||
|
||||
- **Chromium 145** — latest stable, 26 fingerprint patches
|
||||
- **10 new patches** — screen dimensions, audio, WebGL, and more
|
||||
- **CDP hardening** — audited and patched known automation detection vectors
|
||||
- **Timezone & locale from proxy IP** — `launch(proxy="...", geoip=True)` auto-detects timezone and locale
|
||||
- **Improved cross-platform spoofing** — fixed edge cases in font rendering and GPU reporting
|
||||
- **Automated test matrix** — 41+ tests across 8 groups (stealth, fingerprint, reCAPTCHA, Turnstile, TLS, enterprise) running in Docker
|
||||
|
||||
See the full [CHANGELOG.md](CHANGELOG.md) for details.
|
||||
|
||||
## Why CloakBrowser?
|
||||
|
||||
Every bot detection system — reCAPTCHA, Cloudflare Turnstile, ShieldSquare, FingerprintJS — identifies automation browsers through **browser fingerprinting**: canvas rendering, WebGL output, audio processing, font enumeration, and dozens of other signals.
|
||||
|
||||
Tools like `playwright-stealth` or `undetected-chromedriver` try to fix this with **config-level patches** — JavaScript overrides, flag tweaks, UA spoofing. These work until the next Chrome update breaks them.
|
||||
|
||||
CloakBrowser patches **Chromium source code** — the fingerprint signals are modified at the C++ level, compiled into the binary. Detection sites see a real browser because, at the binary level, it *is* a real browser with different fingerprint values.
|
||||
- **Config-level patches break** — `playwright-stealth`, `undetected-chromedriver`, and `puppeteer-extra` inject JavaScript or tweak flags. Every Chrome update breaks them. Antibot systems detect the patches themselves.
|
||||
- **CloakBrowser patches Chromium source code** — fingerprints are modified at the C++ level, compiled into the binary. Detection sites see a real browser because it *is* a real browser.
|
||||
- **Two layers of stealth** — C++ patches handle fingerprints (GPU, screen, UA, hardware reporting), while the Patchright driver eliminates CDP automation leaks. Most stealth tools only do one or the other.
|
||||
- **One line to switch** — same Playwright API, no new abstractions, no CAPTCHA-solving services.
|
||||
|
||||
## Test Results
|
||||
|
||||
@@ -66,45 +115,45 @@ All tests verified against live detection services. Last tested: Feb 2026 (Chrom
|
||||
| CDP detection | Detected | **Not detected** | `isAutomatedWithCDP: false` |
|
||||
| TLS fingerprint | Mismatch | **Identical to Chrome** | ja3n/ja4/akamai match |
|
||||
|
||||
**14/14 tests passed.**
|
||||
**30/30 tests passed.**
|
||||
|
||||
### Proof
|
||||
|
||||
<p align="center">
|
||||
<img src="images/turnstile_non_interactive.png" width="600" alt="Cloudflare Turnstile — Success">
|
||||
<img src="https://i.imgur.com/IvB0It7.gif" width="600" alt="Cloudflare Turnstile — 3 Tests Passing (Headed Mode)">
|
||||
<br><em>Cloudflare Turnstile — 3 live tests passing in headed mode (macOS)</em>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<img src="https://i.imgur.com/hvIQyMv.png" width="600" alt="reCAPTCHA v3 — Score 0.9">
|
||||
<br><em>reCAPTCHA v3 score 0.9 — server-side verified (human-level)</em>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<img src="https://i.imgur.com/qMIRfhq.png" width="600" alt="Cloudflare Turnstile — Success">
|
||||
<br><em>Cloudflare Turnstile non-interactive challenge — auto-resolved</em>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<img src="images/browserscan_normal.png" width="600" alt="BrowserScan — Normal">
|
||||
<img src="https://i.imgur.com/PRsw6rT.png" width="600" alt="BrowserScan — Normal">
|
||||
<br><em>BrowserScan bot detection — NORMAL (4/4 checks passed)</em>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<img src="images/fingerprintjs_pass.png" width="600" alt="FingerprintJS — Passed">
|
||||
<img src="https://i.imgur.com/9n2C7tu.png" width="600" alt="FingerprintJS — Passed">
|
||||
<br><em>FingerprintJS web-scraping demo — data served, not blocked</em>
|
||||
</p>
|
||||
|
||||
## How It Works
|
||||
|
||||
CloakBrowser is a thin Python wrapper around a custom-built Chromium binary:
|
||||
CloakBrowser is a thin wrapper (Python + JavaScript) around a custom-built Chromium binary:
|
||||
|
||||
1. **You install** → `pip install cloakbrowser`
|
||||
2. **First launch** → binary auto-downloads for your platform (Linux x64 / macOS arm64)
|
||||
3. **Every launch** → Playwright starts with our binary + stealth args
|
||||
4. **You write code** → standard Playwright API, nothing new to learn
|
||||
1. **You install** → `pip install cloakbrowser` or `npm install cloakbrowser`
|
||||
2. **First launch** → binary auto-downloads for your platform (Linux x64, macOS arm64/x64)
|
||||
3. **Every launch** → Playwright or Puppeteer starts with our binary + stealth args
|
||||
4. **You write code** → standard Playwright/Puppeteer API, nothing new to learn
|
||||
|
||||
The binary includes 16 source-level patches that modify:
|
||||
- Canvas fingerprint generation
|
||||
- WebGL renderer output
|
||||
- Audio processing fingerprint
|
||||
- Font enumeration results
|
||||
- Hardware concurrency reporting
|
||||
- Client rect measurements
|
||||
- GPU vendor/renderer strings
|
||||
- WebDriver flag
|
||||
- Headless detection signals
|
||||
- And more...
|
||||
The binary includes 26 source-level patches covering canvas, WebGL, audio, fonts, GPU, screen properties, hardware reporting, and automation signal removal.
|
||||
|
||||
These are compiled into the Chromium binary — not injected via JavaScript, not set via flags.
|
||||
|
||||
@@ -127,6 +176,12 @@ browser = launch(proxy="http://user:pass@proxy:8080")
|
||||
# With extra Chrome args
|
||||
browser = launch(args=["--disable-gpu", "--window-size=1920,1080"])
|
||||
|
||||
# With timezone and locale (sets both binary flags and Playwright context)
|
||||
browser = launch(timezone="America/New_York", locale="en-US")
|
||||
|
||||
# Auto-detect timezone/locale from proxy IP (requires: pip install cloakbrowser[geoip])
|
||||
browser = launch(proxy="http://proxy:8080", geoip=True)
|
||||
|
||||
# Without default stealth args (bring your own fingerprint flags)
|
||||
browser = launch(stealth_args=False, args=["--fingerprint=12345"])
|
||||
```
|
||||
@@ -165,6 +220,27 @@ context = launch_context(
|
||||
page = context.new_page()
|
||||
```
|
||||
|
||||
### Auto Timezone/Locale from Proxy IP
|
||||
|
||||
When using a proxy, antibot systems check that your browser's timezone and locale match the proxy's geographic location. CloakBrowser can auto-detect these from the proxy IP using an offline GeoIP database:
|
||||
|
||||
```bash
|
||||
pip install cloakbrowser[geoip] # installs geoip2 + downloads ~70 MB database on first use
|
||||
```
|
||||
|
||||
```python
|
||||
# Timezone and locale auto-set from proxy's IP geolocation
|
||||
browser = launch(proxy="http://proxy:8080", geoip=True)
|
||||
|
||||
# Works with launch_context too — sets both binary flags AND Playwright context
|
||||
context = launch_context(proxy="http://proxy:8080", geoip=True)
|
||||
|
||||
# Explicit values always win over auto-detection
|
||||
browser = launch(proxy="http://proxy:8080", geoip=True, timezone="Europe/London")
|
||||
```
|
||||
|
||||
> **Note:** For rotating residential proxies, the DNS-resolved IP may differ from the exit IP. Pass explicit `timezone`/`locale` in those cases.
|
||||
|
||||
### Utility Functions
|
||||
|
||||
```python
|
||||
@@ -172,7 +248,7 @@ from cloakbrowser import binary_info, clear_cache, ensure_binary
|
||||
|
||||
# Check binary installation status
|
||||
print(binary_info())
|
||||
# {'version': '145.0.7723.116', 'platform': 'darwin-arm64', 'installed': True, ...}
|
||||
# {'version': '142.0.7444.175', 'platform': 'linux-x64', 'installed': True, ...}
|
||||
|
||||
# Force re-download
|
||||
clear_cache()
|
||||
@@ -181,13 +257,152 @@ clear_cache()
|
||||
ensure_binary()
|
||||
```
|
||||
|
||||
## JavaScript / Node.js API
|
||||
|
||||
CloakBrowser ships a TypeScript package with full type definitions. Choose Playwright or Puppeteer — same stealth binary underneath.
|
||||
|
||||
### Playwright (default)
|
||||
|
||||
```javascript
|
||||
import { launch, launchContext } from 'cloakbrowser';
|
||||
|
||||
// Basic
|
||||
const browser = await launch();
|
||||
|
||||
// With options
|
||||
const browser = await launch({
|
||||
headless: false,
|
||||
proxy: 'http://user:pass@proxy:8080',
|
||||
args: ['--window-size=1920,1080'],
|
||||
});
|
||||
|
||||
// Convenience: browser + context in one call
|
||||
const context = await launchContext({
|
||||
userAgent: 'Custom UA',
|
||||
viewport: { width: 1920, height: 1080 },
|
||||
locale: 'en-US',
|
||||
timezoneId: 'America/New_York',
|
||||
});
|
||||
const page = await context.newPage();
|
||||
```
|
||||
|
||||
> **Note:** Each example above is standalone — not meant to run as one block.
|
||||
|
||||
### Puppeteer
|
||||
|
||||
> **Note:** The Playwright wrapper is recommended for sites with reCAPTCHA Enterprise. Puppeteer's CDP protocol leaks automation signals that reCAPTCHA Enterprise can detect, causing intermittent 403 errors. This is a known Puppeteer limitation, not specific to CloakBrowser. Use Playwright for best results.
|
||||
|
||||
```javascript
|
||||
import { launch } from 'cloakbrowser/puppeteer';
|
||||
|
||||
const browser = await launch({ headless: true });
|
||||
const page = await browser.newPage();
|
||||
await page.goto('https://example.com');
|
||||
await browser.close();
|
||||
```
|
||||
|
||||
### Utility Functions (JS)
|
||||
|
||||
```javascript
|
||||
import { ensureBinary, clearCache, binaryInfo } from 'cloakbrowser';
|
||||
|
||||
// Pre-download binary (e.g., during Docker build)
|
||||
await ensureBinary();
|
||||
|
||||
// Check installation status
|
||||
console.log(binaryInfo());
|
||||
|
||||
// Force re-download
|
||||
clearCache();
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
| Env Variable | Default | Description |
|
||||
|---|---|---|
|
||||
| `CLOAKBROWSER_BINARY_PATH` | — | Skip download, use a local Chromium binary |
|
||||
| `CLOAKBROWSER_CACHE_DIR` | `~/.cloakbrowser` | Binary cache directory |
|
||||
| `CLOAKBROWSER_DOWNLOAD_URL` | GitHub Releases | Custom download URL for binary |
|
||||
| `CLOAKBROWSER_DOWNLOAD_URL` | `cloakbrowser.dev` | Custom download URL for binary |
|
||||
| `CLOAKBROWSER_AUTO_UPDATE` | `true` | Set to `false` to disable background update checks |
|
||||
|
||||
## Fingerprint Management
|
||||
|
||||
Every launch automatically generates a **unique fingerprint**. A random seed (10000–99999) drives all seed-based patches — canvas, WebGL, audio, fonts, and client rects all produce consistent, correlated values derived from that single seed.
|
||||
|
||||
> **Tip: Use a fixed seed when revisiting the same site.** A random seed makes every session look like a different device — which can be suspicious when hitting the same site repeatedly from the same IP. For reCAPTCHA v3 Enterprise and similar scoring systems, a fixed seed produces a consistent fingerprint across sessions, making you look like a returning visitor:
|
||||
> ```python
|
||||
> browser = launch(args=["--fingerprint=12345"])
|
||||
> ```
|
||||
> ```javascript
|
||||
> const browser = await launch({ args: ['--fingerprint=12345'] });
|
||||
> ```
|
||||
|
||||
### Default Fingerprint
|
||||
|
||||
Every `launch()` call sets these automatically. Defaults are **platform-aware** — macOS runs as a native Mac browser, Linux spoofs Windows:
|
||||
|
||||
| Flag | Linux Default | macOS Default | Controls |
|
||||
|------|--------------|---------------|----------|
|
||||
| `--fingerprint` | Random (10000–99999) | Random (10000–99999) | Master seed for canvas, WebGL, audio, fonts, client rects |
|
||||
| `--fingerprint-platform` | `windows` | `macos` | `navigator.platform`, User-Agent OS, GPU pool selection |
|
||||
| `--fingerprint-hardware-concurrency` | `8` | *(not set — uses real value)* | `navigator.hardwareConcurrency` |
|
||||
| `--fingerprint-gpu-vendor` | `NVIDIA Corporation` | *(not set — native Apple GPU)* | WebGL `UNMASKED_VENDOR_WEBGL` |
|
||||
| `--fingerprint-gpu-renderer` | `NVIDIA GeForce RTX 3070` | *(not set — native Metal renderer)* | WebGL `UNMASKED_RENDERER_WEBGL` |
|
||||
|
||||
> **Important:** `--fingerprint-platform` must always be set. The binary defaults to `windows` internally when this flag is missing, which causes GPU/UA mismatches on non-Windows systems. The wrapper handles this automatically.
|
||||
|
||||
### Additional Flags
|
||||
|
||||
Supported by the binary but **not set by default** — pass via `args` to customize:
|
||||
|
||||
| Flag | Controls |
|
||||
|------|----------|
|
||||
| `--fingerprint-brand` | Browser brand: `Chrome`, `Edge`, `Opera`, `Vivaldi` |
|
||||
| `--fingerprint-brand-version` | Brand version (UA + Client Hints) |
|
||||
| `--fingerprint-platform-version` | Client Hints platform version |
|
||||
| `--fingerprint-location` | Geolocation coordinates |
|
||||
| `--timezone` | Timezone (e.g. `America/New_York`) |
|
||||
|
||||
> **Note:** All stealth tests were verified with the default fingerprint config above. Changing these flags may affect detection results — test your configuration before using in production.
|
||||
|
||||
### Examples
|
||||
|
||||
```python
|
||||
# Default — unique fingerprint every launch
|
||||
browser = launch()
|
||||
|
||||
# Pin a seed for a persistent identity
|
||||
browser = launch(args=["--fingerprint=42069"])
|
||||
|
||||
# Full control — disable defaults, set everything yourself
|
||||
browser = launch(stealth_args=False, args=[
|
||||
"--fingerprint=42069",
|
||||
"--fingerprint-platform=windows",
|
||||
"--fingerprint-hardware-concurrency=8",
|
||||
"--fingerprint-gpu-vendor=NVIDIA Corporation",
|
||||
"--fingerprint-gpu-renderer=NVIDIA GeForce RTX 3070",
|
||||
])
|
||||
|
||||
# Add timezone and location on top of defaults
|
||||
browser = launch(args=[
|
||||
"--timezone=America/New_York",
|
||||
"--fingerprint-location=40.7128,-74.0060",
|
||||
])
|
||||
|
||||
# Override GPU to look like a different machine
|
||||
browser = launch(args=[
|
||||
"--fingerprint-gpu-vendor=Intel Inc.",
|
||||
"--fingerprint-gpu-renderer=Intel Iris OpenGL Engine",
|
||||
])
|
||||
```
|
||||
|
||||
```javascript
|
||||
// JavaScript — same flags
|
||||
const browser = await launch({
|
||||
args: ['--fingerprint=42069', '--timezone=Europe/London'],
|
||||
});
|
||||
```
|
||||
|
||||
|
||||
## Use With Existing Playwright Code
|
||||
|
||||
@@ -221,18 +436,161 @@ page.goto("https://example.com")
|
||||
|
||||
| Platform | Status |
|
||||
|---|---|
|
||||
| Linux x86_64 | Supported |
|
||||
| macOS arm64 (Apple Silicon) | Coming soon |
|
||||
| macOS x86_64 (Intel) | Coming soon |
|
||||
| Linux x86_64 | ✅ Available |
|
||||
| macOS arm64 (Apple Silicon) | ✅ Available |
|
||||
| macOS x86_64 (Intel) | ✅ Available |
|
||||
| Windows | Planned |
|
||||
|
||||
**macOS (early access):** macOS builds are new — tested but not yet battle-tested at scale like Linux. If you hit any issues, [please open a GitHub issue](https://github.com/CloakHQ/CloakBrowser/issues).
|
||||
|
||||
**macOS first launch:** The binary is ad-hoc signed. On first run, macOS Gatekeeper will block it. Right-click the app → **Open** → click **Open** in the dialog. This is only needed once.
|
||||
|
||||
**On Windows?** You can still use CloakBrowser via Docker or with your own Chromium binary by setting `CLOAKBROWSER_BINARY_PATH=/path/to/chrome`.
|
||||
## Examples
|
||||
|
||||
See the [`examples/`](examples/) directory:
|
||||
**Python** — see [`examples/`](examples/):
|
||||
- [`basic.py`](examples/basic.py) — Launch and load a page
|
||||
- [`recaptcha_score.py`](examples/recaptcha_score.py) — Check your reCAPTCHA v3 score
|
||||
- [`stealth_test.py`](examples/stealth_test.py) — Run against all detection services
|
||||
|
||||
**JavaScript** — see [`js/examples/`](js/examples/):
|
||||
- [`basic-playwright.ts`](js/examples/basic-playwright.ts) — Playwright launch and load
|
||||
- [`basic-puppeteer.ts`](js/examples/basic-puppeteer.ts) — Puppeteer launch and load
|
||||
- [`stealth-test.ts`](js/examples/stealth-test.ts) — Full 6-site detection test suite
|
||||
|
||||
## Roadmap
|
||||
|
||||
| Feature | Status |
|
||||
|---------|--------|
|
||||
| Linux x64 binary | ✅ Released |
|
||||
| macOS arm64 (Apple Silicon) | ✅ Released |
|
||||
| macOS x64 (Intel) | ✅ Released |
|
||||
| Chromium 145 build (26 patches) | 🔧 In progress |
|
||||
| JavaScript/Puppeteer + Playwright support | ✅ Released |
|
||||
| Fingerprint rotation per session | ✅ Released |
|
||||
| Built-in proxy rotation | 📋 Planned |
|
||||
| Windows support | 📋 Planned |
|
||||
|
||||
> ⭐ **Star this repo** to get notified when new builds drop.
|
||||
|
||||
## Docker
|
||||
|
||||
A ready-to-use [`Dockerfile`](Dockerfile) is included. It installs system deps, the package, and pre-downloads the stealth binary during build:
|
||||
|
||||
```bash
|
||||
docker build -t cloakbrowser .
|
||||
docker run --rm cloakbrowser python examples/basic.py
|
||||
```
|
||||
|
||||
The key steps in the Dockerfile:
|
||||
1. **System deps** — Chromium requires ~15 shared libraries (`libnss3`, `libgbm1`, etc.)
|
||||
2. **`pip install .`** — installs CloakBrowser + Playwright
|
||||
3. **`ensure_binary()`** — downloads the stealth Chromium binary at build time (~200MB), so containers start instantly
|
||||
|
||||
To extend with your own script, just add a `COPY` + `CMD`:
|
||||
|
||||
```dockerfile
|
||||
FROM cloakbrowser
|
||||
COPY your_script.py /app/
|
||||
CMD ["python", "your_script.py"]
|
||||
```
|
||||
|
||||
**Note:** If you run CloakBrowser inside a web server with uvloop (e.g., `uvicorn[standard]`), use `--loop asyncio` to avoid subprocess pipe hangs.
|
||||
|
||||
## Headed Mode (for aggressive bot detection)
|
||||
|
||||
Some sites using advanced bot detection (e.g., DataDome, Cloudflare Turnstile) can detect headless mode even with our C++ patches. For these sites, run in **headed mode** with a virtual display:
|
||||
|
||||
```bash
|
||||
# Install Xvfb (virtual framebuffer)
|
||||
sudo apt install xvfb
|
||||
|
||||
# Start virtual display
|
||||
Xvfb :99 -screen 0 1920x1080x24 &
|
||||
export DISPLAY=:99
|
||||
```
|
||||
|
||||
```python
|
||||
from cloakbrowser import launch
|
||||
|
||||
# Headed mode + residential proxy for maximum stealth
|
||||
browser = launch(headless=False, proxy="http://your-residential-proxy:port")
|
||||
page = browser.new_page()
|
||||
page.goto("https://heavily-protected-site.com") # passes DataDome, etc.
|
||||
browser.close()
|
||||
```
|
||||
|
||||
This runs a real headed browser rendered on a virtual display — no physical monitor needed. Combined with a residential proxy, this passes even the most aggressive detection services.
|
||||
|
||||
> **Tip:** Datacenter IPs are often flagged by IP reputation databases regardless of browser fingerprint. For sites with strict bot detection, a residential proxy makes the difference.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**Reddit or similar sites show CAPTCHA / "Prove your humanity"**
|
||||
|
||||
Some sites (notably Reddit homepage) use HTTP/2 fingerprinting that detects Playwright's connection layer. Pass `--disable-http2` to fall back to HTTP/1.1:
|
||||
|
||||
```python
|
||||
browser = launch(args=["--disable-http2"])
|
||||
```
|
||||
|
||||
```javascript
|
||||
const browser = await launch({ args: ['--disable-http2'] });
|
||||
```
|
||||
|
||||
Only use this flag for sites that require it — most sites work fine with HTTP/2.
|
||||
|
||||
**Binary download fails / timeout**
|
||||
Set a custom download URL or use a local binary:
|
||||
```bash
|
||||
export CLOAKBROWSER_BINARY_PATH=/path/to/your/chrome
|
||||
```
|
||||
|
||||
**macOS: "App is damaged" or Gatekeeper blocks launch**
|
||||
The binary is ad-hoc signed. macOS quarantines downloaded files. Run once to clear it:
|
||||
```bash
|
||||
xattr -cr ~/.cloakbrowser/chromium-*/Chromium.app
|
||||
```
|
||||
|
||||
**"playwright install" vs CloakBrowser binary**
|
||||
You do NOT need `playwright install chromium`. CloakBrowser downloads its own binary. You only need Playwright's system deps:
|
||||
```bash
|
||||
patchright install-deps chromium
|
||||
```
|
||||
|
||||
**reCAPTCHA v3 scores are low (0.1–0.3)**
|
||||
|
||||
Avoid `page.wait_for_timeout()` — it sends CDP protocol commands that reCAPTCHA detects. Use native sleep instead:
|
||||
|
||||
```python
|
||||
# Bad — sends CDP commands, reCAPTCHA detects this
|
||||
page.wait_for_timeout(3000)
|
||||
|
||||
# Good — invisible to the browser
|
||||
import time
|
||||
time.sleep(3)
|
||||
```
|
||||
|
||||
```javascript
|
||||
// Bad — sends CDP commands
|
||||
await page.waitForTimeout(3000);
|
||||
|
||||
// Good — invisible to the browser
|
||||
await new Promise(r => setTimeout(r, 3000));
|
||||
```
|
||||
|
||||
Other tips for maximizing reCAPTCHA scores:
|
||||
- **Use Playwright, not Puppeteer** — Puppeteer sends more CDP protocol traffic that reCAPTCHA detects ([details](#puppeteer))
|
||||
- **Use residential proxies** — datacenter IPs are flagged by IP reputation, not browser fingerprint
|
||||
- **Spend 15+ seconds on the page** before triggering reCAPTCHA — short visits score lower
|
||||
- **Space out requests** — back-to-back `grecaptcha.execute()` calls from the same session get penalized. Wait 30+ seconds between pages with reCAPTCHA
|
||||
- **Use a fixed fingerprint seed** (`--fingerprint=12345`) for consistent device identity across sessions
|
||||
- **Use `page.type()` instead of `page.fill()`** for form filling — `fill()` sets values directly without keyboard events, which reCAPTCHA's behavioral analysis flags. `type()` with a delay simulates real keystrokes:
|
||||
```python
|
||||
page.type("#email", "user@example.com", delay=50)
|
||||
```
|
||||
- **Minimize `page.evaluate()` calls** before the reCAPTCHA check fires — each one sends CDP traffic
|
||||
|
||||
## FAQ
|
||||
|
||||
**Q: Is this legal?**
|
||||
@@ -248,7 +606,16 @@ A: Possibly. Bot detection is an arms race. Source-level patches are harder to d
|
||||
A: Yes. Pass `proxy="http://user:pass@host:port"` to `launch()`.
|
||||
|
||||
**Q: Can I use this with Docker?**
|
||||
A: Yes. Use `ensure_binary()` in your Dockerfile to pre-download the binary during image build.
|
||||
A: Yes. A ready-to-use Dockerfile is included — see the [Docker](#docker) section above.
|
||||
|
||||
## Links
|
||||
|
||||
- 📋 **Changelog** — [CHANGELOG.md](CHANGELOG.md)
|
||||
- 🌐 **Website** — [cloakbrowser.dev](https://cloakbrowser.dev)
|
||||
- 🐛 **Bug reports & feature requests** — [GitHub Issues](https://github.com/CloakHQ/CloakBrowser/issues)
|
||||
- 📦 **PyPI** — [pypi.org/project/cloakbrowser](https://pypi.org/project/cloakbrowser/)
|
||||
- 📦 **npm** — [npmjs.com/package/cloakbrowser](https://www.npmjs.com/package/cloakbrowser)
|
||||
- 📧 **Contact** — cloakhq@pm.me
|
||||
|
||||
## License
|
||||
|
||||
@@ -256,4 +623,4 @@ MIT — see [LICENSE](LICENSE).
|
||||
|
||||
## Contributing
|
||||
|
||||
Issues and PRs welcome. Contact: cloakhq@pm.me
|
||||
Issues and PRs welcome. If something isn't working, [open an issue](https://github.com/CloakHQ/CloakBrowser/issues) — we respond fast.
|
||||
|
||||
@@ -12,8 +12,8 @@ Usage:
|
||||
"""
|
||||
|
||||
from .browser import launch, launch_async, launch_context
|
||||
from .config import CHROMIUM_VERSION, DEFAULT_STEALTH_ARGS
|
||||
from .download import binary_info, clear_cache, ensure_binary
|
||||
from .config import CHROMIUM_VERSION, get_default_stealth_args
|
||||
from .download import binary_info, check_for_update, clear_cache, ensure_binary
|
||||
from ._version import __version__
|
||||
|
||||
__all__ = [
|
||||
@@ -23,7 +23,8 @@ __all__ = [
|
||||
"ensure_binary",
|
||||
"clear_cache",
|
||||
"binary_info",
|
||||
"check_for_update",
|
||||
"CHROMIUM_VERSION",
|
||||
"DEFAULT_STEALTH_ARGS",
|
||||
"get_default_stealth_args",
|
||||
"__version__",
|
||||
]
|
||||
|
||||
@@ -1 +1 @@
|
||||
__version__ = "0.1.0"
|
||||
__version__ = "0.3.0"
|
||||
|
||||
+100
-13
@@ -15,9 +15,10 @@ Usage:
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from typing import Any
|
||||
from typing import Any, Literal
|
||||
from urllib.parse import unquote, urlparse, urlunparse
|
||||
|
||||
from .config import DEFAULT_STEALTH_ARGS
|
||||
from .config import DEFAULT_VIEWPORT, get_default_stealth_args
|
||||
from .download import ensure_binary
|
||||
|
||||
logger = logging.getLogger("cloakbrowser")
|
||||
@@ -28,6 +29,9 @@ def launch(
|
||||
proxy: str | None = None,
|
||||
args: list[str] | None = None,
|
||||
stealth_args: bool = True,
|
||||
timezone: str | None = None,
|
||||
locale: str | None = None,
|
||||
geoip: bool = False,
|
||||
**kwargs: Any,
|
||||
) -> Any:
|
||||
"""Launch stealth Chromium browser. Returns a Playwright Browser object.
|
||||
@@ -38,6 +42,12 @@ def launch(
|
||||
args: Additional Chromium CLI arguments to pass.
|
||||
stealth_args: Include default stealth fingerprint args (default True).
|
||||
Set to False if you want to pass your own --fingerprint flags.
|
||||
timezone: IANA timezone (e.g. 'America/New_York'). Sets --timezone binary flag.
|
||||
locale: BCP 47 locale (e.g. 'en-US'). Sets --lang binary flag.
|
||||
geoip: Auto-detect timezone/locale from proxy IP (default False).
|
||||
Requires ``pip install cloakbrowser[geoip]``. Downloads ~70 MB
|
||||
GeoLite2-City database on first use. Explicit timezone/locale
|
||||
always override geoip results.
|
||||
**kwargs: Passed directly to playwright.chromium.launch().
|
||||
|
||||
Returns:
|
||||
@@ -51,10 +61,11 @@ def launch(
|
||||
>>> print(page.title())
|
||||
>>> browser.close()
|
||||
"""
|
||||
from playwright.sync_api import sync_playwright
|
||||
from patchright.sync_api import sync_playwright
|
||||
|
||||
binary_path = ensure_binary()
|
||||
chrome_args = _build_args(stealth_args, args)
|
||||
timezone, locale = _maybe_resolve_geoip(geoip, proxy, timezone, locale)
|
||||
chrome_args = _build_args(stealth_args, args, timezone=timezone, locale=locale)
|
||||
|
||||
logger.debug("Launching stealth Chromium (headless=%s, args=%d)", headless, len(chrome_args))
|
||||
|
||||
@@ -63,6 +74,7 @@ def launch(
|
||||
executable_path=binary_path,
|
||||
headless=headless,
|
||||
args=chrome_args,
|
||||
ignore_default_args=["--enable-automation"],
|
||||
**_build_proxy_kwargs(proxy),
|
||||
**kwargs,
|
||||
)
|
||||
@@ -84,6 +96,9 @@ async def launch_async(
|
||||
proxy: str | None = None,
|
||||
args: list[str] | None = None,
|
||||
stealth_args: bool = True,
|
||||
timezone: str | None = None,
|
||||
locale: str | None = None,
|
||||
geoip: bool = False,
|
||||
**kwargs: Any,
|
||||
) -> Any:
|
||||
"""Async version of launch(). Returns a Playwright Browser object.
|
||||
@@ -93,6 +108,9 @@ async def launch_async(
|
||||
proxy: Proxy server URL (e.g. 'http://proxy:8080' or 'socks5://proxy:1080').
|
||||
args: Additional Chromium CLI arguments to pass.
|
||||
stealth_args: Include default stealth fingerprint args (default True).
|
||||
timezone: IANA timezone (e.g. 'America/New_York'). Sets --timezone binary flag.
|
||||
locale: BCP 47 locale (e.g. 'en-US'). Sets --lang binary flag.
|
||||
geoip: Auto-detect timezone/locale from proxy IP (default False).
|
||||
**kwargs: Passed directly to playwright.chromium.launch().
|
||||
|
||||
Returns:
|
||||
@@ -111,10 +129,11 @@ async def launch_async(
|
||||
>>>
|
||||
>>> asyncio.run(main())
|
||||
"""
|
||||
from playwright.async_api import async_playwright
|
||||
from patchright.async_api import async_playwright
|
||||
|
||||
binary_path = ensure_binary()
|
||||
chrome_args = _build_args(stealth_args, args)
|
||||
timezone, locale = _maybe_resolve_geoip(geoip, proxy, timezone, locale)
|
||||
chrome_args = _build_args(stealth_args, args, timezone=timezone, locale=locale)
|
||||
|
||||
logger.debug("Launching stealth Chromium async (headless=%s, args=%d)", headless, len(chrome_args))
|
||||
|
||||
@@ -123,6 +142,7 @@ async def launch_async(
|
||||
executable_path=binary_path,
|
||||
headless=headless,
|
||||
args=chrome_args,
|
||||
ignore_default_args=["--enable-automation"],
|
||||
**_build_proxy_kwargs(proxy),
|
||||
**kwargs,
|
||||
)
|
||||
@@ -148,6 +168,8 @@ def launch_context(
|
||||
viewport: dict | None = None,
|
||||
locale: str | None = None,
|
||||
timezone_id: str | None = None,
|
||||
color_scheme: Literal["light", "dark", "no-preference"] | None = None,
|
||||
geoip: bool = False,
|
||||
**kwargs: Any,
|
||||
) -> Any:
|
||||
"""Launch stealth browser and return a BrowserContext with common options pre-set.
|
||||
@@ -164,22 +186,31 @@ def launch_context(
|
||||
viewport: Viewport size dict, e.g. {"width": 1920, "height": 1080}.
|
||||
locale: Browser locale, e.g. "en-US".
|
||||
timezone_id: Timezone, e.g. "America/New_York".
|
||||
color_scheme: Color scheme preference — 'light', 'dark', or 'no-preference'.
|
||||
Default: None (uses Chromium default, which is 'light').
|
||||
Note: 'no-preference' doesn't work in Patchright (falls back to 'light').
|
||||
geoip: Auto-detect timezone/locale from proxy IP (default False).
|
||||
**kwargs: Passed to browser.new_context().
|
||||
|
||||
Returns:
|
||||
Playwright BrowserContext object.
|
||||
"""
|
||||
browser = launch(headless=headless, proxy=proxy, args=args, stealth_args=stealth_args)
|
||||
# Resolve geoip BEFORE launch() to avoid double-resolution and ensure
|
||||
# resolved values flow to both binary flags AND context params
|
||||
timezone_id, locale = _maybe_resolve_geoip(geoip, proxy, timezone_id, locale)
|
||||
browser = launch(headless=headless, proxy=proxy, args=args, stealth_args=stealth_args,
|
||||
timezone=timezone_id, locale=locale)
|
||||
|
||||
context_kwargs: dict[str, Any] = {}
|
||||
if user_agent:
|
||||
context_kwargs["user_agent"] = user_agent
|
||||
if viewport:
|
||||
context_kwargs["viewport"] = viewport
|
||||
context_kwargs["viewport"] = viewport or DEFAULT_VIEWPORT
|
||||
if locale:
|
||||
context_kwargs["locale"] = locale
|
||||
if timezone_id:
|
||||
context_kwargs["timezone_id"] = timezone_id
|
||||
if color_scheme:
|
||||
context_kwargs["color_scheme"] = color_scheme
|
||||
context_kwargs.update(kwargs)
|
||||
|
||||
try:
|
||||
@@ -205,13 +236,69 @@ def launch_context(
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _build_args(stealth_args: bool, extra_args: list[str] | None) -> list[str]:
|
||||
"""Combine stealth args with user-provided args."""
|
||||
def _maybe_resolve_geoip(
|
||||
geoip: bool,
|
||||
proxy: str | None,
|
||||
timezone: str | None,
|
||||
locale: str | None,
|
||||
) -> tuple[str | None, str | None]:
|
||||
"""Auto-fill timezone/locale from proxy IP when geoip is enabled."""
|
||||
if not geoip or not proxy or (timezone is not None and locale is not None):
|
||||
return timezone, locale
|
||||
|
||||
from .geoip import resolve_proxy_geo
|
||||
|
||||
geo_tz, geo_locale = resolve_proxy_geo(proxy)
|
||||
if timezone is None:
|
||||
timezone = geo_tz
|
||||
if locale is None:
|
||||
locale = geo_locale
|
||||
return timezone, locale
|
||||
|
||||
|
||||
def _build_args(
|
||||
stealth_args: bool,
|
||||
extra_args: list[str] | None,
|
||||
timezone: str | None = None,
|
||||
locale: str | None = None,
|
||||
) -> list[str]:
|
||||
"""Combine stealth args with user-provided args and locale flags."""
|
||||
result = []
|
||||
if stealth_args:
|
||||
result.extend(DEFAULT_STEALTH_ARGS)
|
||||
result.extend(get_default_stealth_args())
|
||||
if extra_args:
|
||||
result.extend(extra_args)
|
||||
# Timezone/locale flags are independent of stealth_args — always inject when set
|
||||
if timezone:
|
||||
result.append(f"--timezone={timezone}")
|
||||
if locale:
|
||||
result.append(f"--lang={locale}")
|
||||
return result
|
||||
|
||||
|
||||
def _parse_proxy_url(proxy: str) -> dict[str, Any]:
|
||||
"""Parse proxy URL, extracting credentials into separate Playwright fields.
|
||||
|
||||
Handles: http://user:pass@host:port -> {server: "http://host:port", username: "user", password: "pass"}
|
||||
Also handles: no credentials, URL-encoded special chars, socks5://, missing port.
|
||||
"""
|
||||
parsed = urlparse(proxy)
|
||||
|
||||
if not parsed.username:
|
||||
return {"server": proxy}
|
||||
|
||||
# Rebuild server URL without credentials
|
||||
netloc = parsed.hostname or ""
|
||||
if parsed.port:
|
||||
netloc += f":{parsed.port}"
|
||||
|
||||
server = urlunparse((parsed.scheme, netloc, parsed.path, "", "", ""))
|
||||
|
||||
result: dict[str, Any] = {"server": server}
|
||||
result["username"] = unquote(parsed.username)
|
||||
if parsed.password:
|
||||
result["password"] = unquote(parsed.password)
|
||||
|
||||
return result
|
||||
|
||||
|
||||
@@ -219,4 +306,4 @@ def _build_proxy_kwargs(proxy: str | None) -> dict[str, Any]:
|
||||
"""Build proxy kwargs for Playwright launch."""
|
||||
if proxy is None:
|
||||
return {}
|
||||
return {"proxy": {"server": proxy}}
|
||||
return {"proxy": _parse_proxy_url(proxy)}
|
||||
|
||||
+119
-20
@@ -4,6 +4,7 @@ from __future__ import annotations
|
||||
|
||||
import os
|
||||
import platform
|
||||
import random
|
||||
from pathlib import Path
|
||||
|
||||
from ._version import __version__
|
||||
@@ -11,22 +12,53 @@ from ._version import __version__
|
||||
# ---------------------------------------------------------------------------
|
||||
# Chromium version shipped with this release
|
||||
# ---------------------------------------------------------------------------
|
||||
CHROMIUM_VERSION = "142.0.7444.175"
|
||||
CHROMIUM_VERSION = "145.0.7632.109"
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Default stealth arguments passed to the patched Chromium binary.
|
||||
# These activate source-level fingerprint patches compiled into the binary.
|
||||
# ---------------------------------------------------------------------------
|
||||
DEFAULT_STEALTH_ARGS: list[str] = [
|
||||
"--no-sandbox",
|
||||
"--disable-blink-features=AutomationControlled",
|
||||
# Fingerprint overrides (activate compiled C++ patches)
|
||||
"--fingerprint=98765",
|
||||
"--fingerprint-platform=windows",
|
||||
"--fingerprint-hardware-concurrency=8",
|
||||
"--fingerprint-gpu-vendor=NVIDIA Corporation",
|
||||
"--fingerprint-gpu-renderer=NVIDIA GeForce RTX 4070",
|
||||
]
|
||||
def get_default_stealth_args() -> list[str]:
|
||||
"""Build stealth args with a random fingerprint seed per launch.
|
||||
|
||||
On macOS, skips platform/GPU spoofing — runs as a native Mac browser.
|
||||
Spoofing Windows on Mac creates detectable mismatches (fonts, GPU, etc.).
|
||||
"""
|
||||
seed = random.randint(10000, 99999)
|
||||
system = platform.system()
|
||||
|
||||
base = [
|
||||
"--no-sandbox",
|
||||
"--disable-blink-features=AutomationControlled",
|
||||
f"--fingerprint={seed}",
|
||||
]
|
||||
|
||||
if system == "Darwin":
|
||||
# Tell the fingerprint patches we're on macOS so GPU/UA match natively
|
||||
return base + [
|
||||
"--fingerprint-platform=macos",
|
||||
]
|
||||
|
||||
# Linux: spoof as Windows
|
||||
return base + [
|
||||
"--fingerprint-platform=windows",
|
||||
"--fingerprint-hardware-concurrency=8",
|
||||
"--fingerprint-device-memory=8",
|
||||
"--fingerprint-gpu-vendor=NVIDIA Corporation",
|
||||
"--fingerprint-gpu-renderer=NVIDIA GeForce RTX 3070",
|
||||
"--fingerprint-taskbar-height=40",
|
||||
"--fingerprint-screen-width=1920",
|
||||
"--fingerprint-screen-height=1080",
|
||||
"--window-size=1920,1080",
|
||||
]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Default viewport — realistic maximized Chrome on 1080p Windows
|
||||
# screen=1920x1080, availHeight=1040 (minus 40px taskbar),
|
||||
# innerHeight=955 (minus ~85px Chrome UI: tabs + address bar + bookmarks)
|
||||
# ---------------------------------------------------------------------------
|
||||
DEFAULT_VIEWPORT = {"width": 1920, "height": 955}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Platform detection
|
||||
@@ -38,6 +70,10 @@ SUPPORTED_PLATFORMS: dict[tuple[str, str], str] = {
|
||||
("Darwin", "x86_64"): "darwin-x64",
|
||||
}
|
||||
|
||||
# Platforms with pre-built binaries available for download.
|
||||
# Update this set as new platform builds are released.
|
||||
AVAILABLE_PLATFORMS: set[str] = {"linux-x64", "darwin-arm64", "darwin-x64"}
|
||||
|
||||
|
||||
def get_platform_tag() -> str:
|
||||
"""Return the platform tag for binary download (e.g. 'linux-x64', 'darwin-arm64')."""
|
||||
@@ -67,15 +103,15 @@ def get_cache_dir() -> Path:
|
||||
return Path.home() / ".cloakbrowser"
|
||||
|
||||
|
||||
def get_binary_dir() -> Path:
|
||||
"""Return the directory for the current Chromium version binary."""
|
||||
return get_cache_dir() / f"chromium-{CHROMIUM_VERSION}"
|
||||
def get_binary_dir(version: str | None = None) -> Path:
|
||||
"""Return the directory for a Chromium version binary."""
|
||||
v = version or CHROMIUM_VERSION
|
||||
return get_cache_dir() / f"chromium-{v}"
|
||||
|
||||
|
||||
def get_binary_path() -> Path:
|
||||
def get_binary_path(version: str | None = None) -> Path:
|
||||
"""Return the expected path to the chrome executable."""
|
||||
platform_tag = get_platform_tag()
|
||||
binary_dir = get_binary_dir()
|
||||
binary_dir = get_binary_dir(version)
|
||||
|
||||
if platform.system() == "Darwin":
|
||||
# macOS: Chromium.app bundle
|
||||
@@ -85,19 +121,82 @@ def get_binary_path() -> Path:
|
||||
return binary_dir / "chrome"
|
||||
|
||||
|
||||
def check_platform_available() -> None:
|
||||
"""Raise a clear error if no pre-built binary exists for this platform.
|
||||
|
||||
Skipped when CLOAKBROWSER_BINARY_PATH is set (user has their own build).
|
||||
"""
|
||||
if get_local_binary_override():
|
||||
return
|
||||
|
||||
tag = get_platform_tag() # raises if platform unsupported entirely
|
||||
if tag not in AVAILABLE_PLATFORMS:
|
||||
available = ", ".join(sorted(AVAILABLE_PLATFORMS))
|
||||
import sys
|
||||
sys.exit(
|
||||
f"\n\033[1mCloakBrowser\033[0m — Pre-built binaries are currently only available for: {available}.\n"
|
||||
f"Windows builds are coming soon.\n\n"
|
||||
f"To use CloakBrowser now, run in Docker (see README) or set CLOAKBROWSER_BINARY_PATH."
|
||||
)
|
||||
|
||||
|
||||
def get_effective_version() -> str:
|
||||
"""Return the best available version: auto-updated if available, else hardcoded.
|
||||
|
||||
Reads the latest_version marker file from the cache directory.
|
||||
Returns CHROMIUM_VERSION if no update has been downloaded.
|
||||
"""
|
||||
marker = get_cache_dir() / "latest_version"
|
||||
if marker.exists():
|
||||
try:
|
||||
version = marker.read_text().strip()
|
||||
if version and _version_newer(version, CHROMIUM_VERSION):
|
||||
# Verify the binary actually exists
|
||||
binary = get_binary_path(version)
|
||||
if binary.exists():
|
||||
return version
|
||||
except (ValueError, OSError):
|
||||
pass
|
||||
return CHROMIUM_VERSION
|
||||
|
||||
|
||||
def _version_tuple(v: str) -> tuple[int, ...]:
|
||||
"""Parse '145.0.7718.0' into (145, 0, 7718, 0) for comparison."""
|
||||
return tuple(int(x) for x in v.split("."))
|
||||
|
||||
|
||||
def _version_newer(a: str, b: str) -> bool:
|
||||
"""Return True if version a is strictly newer than version b."""
|
||||
return _version_tuple(a) > _version_tuple(b)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Download URL
|
||||
# ---------------------------------------------------------------------------
|
||||
DOWNLOAD_BASE_URL = os.environ.get(
|
||||
"CLOAKBROWSER_DOWNLOAD_URL",
|
||||
"https://github.com/CloakHQ/chromium-stealth-builds/releases/download",
|
||||
"https://cloakbrowser.dev",
|
||||
)
|
||||
|
||||
GITHUB_API_URL = "https://api.github.com/repos/CloakHQ/cloakbrowser/releases"
|
||||
|
||||
GITHUB_DOWNLOAD_BASE_URL = (
|
||||
"https://github.com/CloakHQ/cloakbrowser/releases/download"
|
||||
)
|
||||
|
||||
|
||||
def get_download_url() -> str:
|
||||
def get_download_url(version: str | None = None) -> str:
|
||||
"""Return the full download URL for the current platform's binary archive."""
|
||||
v = version or CHROMIUM_VERSION
|
||||
tag = get_platform_tag()
|
||||
return f"{DOWNLOAD_BASE_URL}/v{CHROMIUM_VERSION}/cloakbrowser-{tag}.tar.gz"
|
||||
return f"{DOWNLOAD_BASE_URL}/chromium-v{v}/cloakbrowser-{tag}.tar.gz"
|
||||
|
||||
|
||||
def get_fallback_download_url(version: str | None = None) -> str:
|
||||
"""Return the GitHub Releases fallback URL for the binary archive."""
|
||||
v = version or CHROMIUM_VERSION
|
||||
tag = get_platform_tag()
|
||||
return f"{GITHUB_DOWNLOAD_BASE_URL}/chromium-v{v}/cloakbrowser-{tag}.tar.gz"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
+303
-25
@@ -6,20 +6,33 @@ Similar to how Playwright downloads its own bundled Chromium.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import logging
|
||||
import os
|
||||
import platform
|
||||
import stat
|
||||
import subprocess
|
||||
import tarfile
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
|
||||
from .config import (
|
||||
CHROMIUM_VERSION,
|
||||
DOWNLOAD_BASE_URL,
|
||||
GITHUB_API_URL,
|
||||
GITHUB_DOWNLOAD_BASE_URL,
|
||||
_version_newer,
|
||||
check_platform_available,
|
||||
get_binary_dir,
|
||||
get_binary_path,
|
||||
get_cache_dir,
|
||||
get_download_url,
|
||||
get_effective_version,
|
||||
get_fallback_download_url,
|
||||
get_local_binary_override,
|
||||
get_platform_tag,
|
||||
)
|
||||
@@ -29,6 +42,9 @@ logger = logging.getLogger("cloakbrowser")
|
||||
# Timeout for download (large binary, allow 10 min)
|
||||
DOWNLOAD_TIMEOUT = 600.0
|
||||
|
||||
# Auto-update check interval (1 hour)
|
||||
UPDATE_CHECK_INTERVAL = 3600
|
||||
|
||||
|
||||
def ensure_binary() -> str:
|
||||
"""Ensure the stealth Chromium binary is available. Download if needed.
|
||||
@@ -48,13 +64,27 @@ def ensure_binary() -> str:
|
||||
logger.info("Using local binary override: %s", local_override)
|
||||
return str(path)
|
||||
|
||||
# Check if binary is already cached
|
||||
binary_path = get_binary_path()
|
||||
# Fail fast if no binary available for this platform
|
||||
check_platform_available()
|
||||
|
||||
# Check for auto-updated version first, then fall back to hardcoded
|
||||
effective = get_effective_version()
|
||||
binary_path = get_binary_path(effective)
|
||||
|
||||
if binary_path.exists() and _is_executable(binary_path):
|
||||
logger.debug("Binary found in cache: %s", binary_path)
|
||||
logger.debug("Binary found in cache: %s (version %s)", binary_path, effective)
|
||||
_maybe_trigger_update_check()
|
||||
return str(binary_path)
|
||||
|
||||
# Download
|
||||
# Fall back to hardcoded version if effective version binary doesn't exist
|
||||
if effective != CHROMIUM_VERSION:
|
||||
fallback_path = get_binary_path()
|
||||
if fallback_path.exists() and _is_executable(fallback_path):
|
||||
logger.debug("Binary found in cache: %s", fallback_path)
|
||||
_maybe_trigger_update_check()
|
||||
return str(fallback_path)
|
||||
|
||||
# Download hardcoded version
|
||||
logger.info(
|
||||
"Stealth Chromium %s not found. Downloading for %s...",
|
||||
CHROMIUM_VERSION,
|
||||
@@ -62,6 +92,7 @@ def ensure_binary() -> str:
|
||||
)
|
||||
_download_and_extract()
|
||||
|
||||
binary_path = get_binary_path()
|
||||
if not binary_path.exists():
|
||||
raise RuntimeError(
|
||||
f"Download completed but binary not found at expected path: {binary_path}. "
|
||||
@@ -69,13 +100,21 @@ def ensure_binary() -> str:
|
||||
f"https://github.com/CloakHQ/cloakbrowser/issues"
|
||||
)
|
||||
|
||||
_maybe_trigger_update_check()
|
||||
return str(binary_path)
|
||||
|
||||
|
||||
def _download_and_extract() -> None:
|
||||
"""Download the binary archive and extract to cache directory."""
|
||||
url = get_download_url()
|
||||
binary_dir = get_binary_dir()
|
||||
def _download_and_extract(version: str | None = None) -> None:
|
||||
"""Download the binary archive and extract to cache directory.
|
||||
|
||||
Tries the primary server (cloakbrowser.dev) first, falls back to
|
||||
GitHub Releases if the primary is unreachable or returns an error.
|
||||
Verifies SHA-256 checksum before extraction when available.
|
||||
"""
|
||||
primary_url = get_download_url(version)
|
||||
fallback_url = get_fallback_download_url(version)
|
||||
binary_dir = get_binary_dir(version)
|
||||
binary_path = get_binary_path(version)
|
||||
|
||||
# Create cache dir
|
||||
binary_dir.parent.mkdir(parents=True, exist_ok=True)
|
||||
@@ -85,13 +124,101 @@ def _download_and_extract() -> None:
|
||||
tmp_path = Path(tmp.name)
|
||||
|
||||
try:
|
||||
_download_file(url, tmp_path)
|
||||
_extract_archive(tmp_path, binary_dir)
|
||||
# Try primary, fall back to GitHub Releases (skip fallback if custom URL)
|
||||
try:
|
||||
_download_file(primary_url, tmp_path)
|
||||
except Exception as primary_err:
|
||||
if os.environ.get("CLOAKBROWSER_DOWNLOAD_URL"):
|
||||
raise
|
||||
logger.warning(
|
||||
"Primary download failed (%s), trying GitHub Releases...",
|
||||
primary_err,
|
||||
)
|
||||
_download_file(fallback_url, tmp_path)
|
||||
|
||||
# Verify checksum before extraction
|
||||
if os.environ.get("CLOAKBROWSER_SKIP_CHECKSUM", "").lower() != "true":
|
||||
_verify_download_checksum(tmp_path, version)
|
||||
|
||||
_extract_archive(tmp_path, binary_dir, binary_path)
|
||||
logger.info("Visit https://cloakbrowser.dev for docs and release notifications.")
|
||||
logger.info("Issues? https://github.com/CloakHQ/CloakBrowser/issues")
|
||||
logger.info("Star us if CloakBrowser helps: https://github.com/CloakHQ/CloakBrowser")
|
||||
finally:
|
||||
# Clean up temp file
|
||||
tmp_path.unlink(missing_ok=True)
|
||||
|
||||
|
||||
def _verify_download_checksum(file_path: Path, version: str | None = None) -> None:
|
||||
"""Fetch SHA256SUMS and verify the downloaded file. Warn if unavailable, fail on mismatch."""
|
||||
checksums = _fetch_checksums(version)
|
||||
tarball_name = f"cloakbrowser-{get_platform_tag()}.tar.gz"
|
||||
|
||||
if checksums is None:
|
||||
logger.warning("SHA256SUMS not available for this release — skipping checksum verification")
|
||||
return
|
||||
|
||||
expected = checksums.get(tarball_name)
|
||||
if expected is None:
|
||||
logger.warning("SHA256SUMS found but no entry for %s — skipping verification", tarball_name)
|
||||
return
|
||||
|
||||
_verify_checksum(file_path, expected)
|
||||
|
||||
|
||||
def _fetch_checksums(version: str | None = None) -> dict[str, str] | None:
|
||||
"""Fetch SHA256SUMS file for a version. Returns {filename: hash} or None."""
|
||||
v = version or CHROMIUM_VERSION
|
||||
has_custom_url = os.environ.get("CLOAKBROWSER_DOWNLOAD_URL")
|
||||
|
||||
# Build URL list — respect custom URL contract (no GitHub fallback)
|
||||
urls = [f"{DOWNLOAD_BASE_URL}/chromium-v{v}/SHA256SUMS"]
|
||||
if not has_custom_url:
|
||||
urls.append(f"{GITHUB_DOWNLOAD_BASE_URL}/chromium-v{v}/SHA256SUMS")
|
||||
|
||||
for url in urls:
|
||||
try:
|
||||
resp = httpx.get(url, follow_redirects=True, timeout=10.0)
|
||||
resp.raise_for_status()
|
||||
return _parse_checksums(resp.text)
|
||||
except Exception:
|
||||
continue
|
||||
return None
|
||||
|
||||
|
||||
def _parse_checksums(text: str) -> dict[str, str]:
|
||||
"""Parse SHA256SUMS format: 'hash filename' per line."""
|
||||
result = {}
|
||||
for line in text.strip().splitlines():
|
||||
line = line.strip()
|
||||
if not line:
|
||||
continue
|
||||
parts = line.split(None, 1)
|
||||
if len(parts) == 2:
|
||||
hash_val, filename = parts
|
||||
filename = filename.lstrip("*")
|
||||
result[filename] = hash_val.lower()
|
||||
return result
|
||||
|
||||
|
||||
def _verify_checksum(file_path: Path, expected_hash: str) -> None:
|
||||
"""Verify SHA-256 of a file. Raises RuntimeError on mismatch."""
|
||||
sha256 = hashlib.sha256()
|
||||
with open(file_path, "rb") as f:
|
||||
for chunk in iter(lambda: f.read(8192), b""):
|
||||
sha256.update(chunk)
|
||||
actual = sha256.hexdigest().lower()
|
||||
if actual != expected_hash:
|
||||
raise RuntimeError(
|
||||
f"Checksum verification failed!\n"
|
||||
f" Expected: {expected_hash}\n"
|
||||
f" Got: {actual}\n"
|
||||
f" File may be corrupted or tampered with. "
|
||||
f"Please retry or report at https://github.com/CloakHQ/cloakbrowser/issues"
|
||||
)
|
||||
logger.info("Checksum verified: SHA-256 OK")
|
||||
|
||||
|
||||
def _download_file(url: str, dest: Path) -> None:
|
||||
"""Download a file with progress logging."""
|
||||
logger.info("Downloading from %s", url)
|
||||
@@ -123,7 +250,9 @@ def _download_file(url: str, dest: Path) -> None:
|
||||
logger.info("Download complete: %d MB", dest.stat().st_size // (1024 * 1024))
|
||||
|
||||
|
||||
def _extract_archive(archive_path: Path, dest_dir: Path) -> None:
|
||||
def _extract_archive(
|
||||
archive_path: Path, dest_dir: Path, binary_path: Path | None = None
|
||||
) -> None:
|
||||
"""Extract tar.gz archive to destination directory."""
|
||||
logger.info("Extracting to %s", dest_dir)
|
||||
|
||||
@@ -135,28 +264,40 @@ def _extract_archive(archive_path: Path, dest_dir: Path) -> None:
|
||||
dest_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
with tarfile.open(archive_path, "r:gz") as tar:
|
||||
# Security: prevent path traversal and symlink attacks
|
||||
# Security: prevent path traversal
|
||||
safe_members = []
|
||||
for member in tar.getmembers():
|
||||
# Allow symlinks — macOS .app bundles require them (Framework layout)
|
||||
if member.issym() or member.islnk():
|
||||
logger.warning("Skipping symlink in archive: %s", member.name)
|
||||
continue
|
||||
member_path = (dest_dir / member.name).resolve()
|
||||
if not str(member_path).startswith(str(dest_dir.resolve())):
|
||||
raise RuntimeError(f"Archive contains path traversal: {member.name}")
|
||||
link_target = member.linkname
|
||||
# Reject symlinks that escape the dest dir
|
||||
if os.path.isabs(link_target) or ".." in link_target.split("/"):
|
||||
logger.warning("Skipping suspicious symlink: %s -> %s", member.name, link_target)
|
||||
continue
|
||||
else:
|
||||
member_path = (dest_dir / member.name).resolve()
|
||||
if not str(member_path).startswith(str(dest_dir.resolve())):
|
||||
raise RuntimeError(f"Archive contains path traversal: {member.name}")
|
||||
safe_members.append(member)
|
||||
|
||||
tar.extractall(dest_dir, members=safe_members)
|
||||
|
||||
# If tar extracted into a single subdirectory, flatten it
|
||||
# (e.g. fingerprint-chromium-142-custom-v2/chrome → chrome)
|
||||
# But never flatten .app bundles — macOS needs the bundle structure intact
|
||||
_flatten_single_subdir(dest_dir)
|
||||
|
||||
# Make binary executable
|
||||
binary_path = get_binary_path()
|
||||
if binary_path.exists():
|
||||
_make_executable(binary_path)
|
||||
logger.info("Binary ready: %s", binary_path)
|
||||
bp = binary_path or get_binary_path()
|
||||
if bp.exists():
|
||||
_make_executable(bp)
|
||||
|
||||
# macOS: remove quarantine/provenance xattrs to prevent Gatekeeper prompts
|
||||
if platform.system() == "Darwin":
|
||||
_remove_quarantine(dest_dir)
|
||||
|
||||
if bp.exists():
|
||||
logger.info("Binary ready: %s", bp)
|
||||
|
||||
|
||||
def _flatten_single_subdir(dest_dir: Path) -> None:
|
||||
@@ -170,6 +311,10 @@ def _flatten_single_subdir(dest_dir: Path) -> None:
|
||||
entries = list(dest_dir.iterdir())
|
||||
if len(entries) == 1 and entries[0].is_dir():
|
||||
subdir = entries[0]
|
||||
# Never flatten .app bundles — macOS needs the bundle structure
|
||||
if subdir.name.endswith(".app"):
|
||||
logger.debug("Keeping .app bundle intact: %s", subdir.name)
|
||||
return
|
||||
logger.debug("Flattening single subdirectory: %s", subdir.name)
|
||||
for item in subdir.iterdir():
|
||||
shutil.move(str(item), str(dest_dir / item.name))
|
||||
@@ -187,6 +332,19 @@ def _make_executable(path: Path) -> None:
|
||||
path.chmod(current | stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH)
|
||||
|
||||
|
||||
def _remove_quarantine(path: Path) -> None:
|
||||
"""Remove macOS quarantine/provenance xattrs so Gatekeeper doesn't block the binary."""
|
||||
try:
|
||||
subprocess.run(
|
||||
["xattr", "-cr", str(path)],
|
||||
capture_output=True,
|
||||
timeout=30,
|
||||
)
|
||||
logger.debug("Removed quarantine attributes from %s", path)
|
||||
except Exception:
|
||||
logger.debug("Failed to remove quarantine attributes", exc_info=True)
|
||||
|
||||
|
||||
def clear_cache() -> None:
|
||||
"""Remove all cached binaries. Forces re-download on next launch."""
|
||||
from .config import get_cache_dir
|
||||
@@ -200,12 +358,132 @@ def clear_cache() -> None:
|
||||
|
||||
def binary_info() -> dict:
|
||||
"""Return info about the current binary installation."""
|
||||
binary_path = get_binary_path()
|
||||
effective = get_effective_version()
|
||||
binary_path = get_binary_path(effective)
|
||||
return {
|
||||
"version": CHROMIUM_VERSION,
|
||||
"version": effective,
|
||||
"bundled_version": CHROMIUM_VERSION,
|
||||
"platform": get_platform_tag(),
|
||||
"binary_path": str(binary_path),
|
||||
"installed": binary_path.exists(),
|
||||
"cache_dir": str(get_binary_dir()),
|
||||
"download_url": get_download_url(),
|
||||
"cache_dir": str(get_binary_dir(effective)),
|
||||
"download_url": get_download_url(effective),
|
||||
}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Auto-update
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def check_for_update() -> str | None:
|
||||
"""Manually check for a newer Chromium version. Returns new version or None.
|
||||
|
||||
This is the public API for triggering an update check. Unlike the
|
||||
background check in ensure_binary(), this blocks until complete.
|
||||
"""
|
||||
latest = _get_latest_chromium_version()
|
||||
if latest is None:
|
||||
return None
|
||||
if not _version_newer(latest, CHROMIUM_VERSION):
|
||||
return None
|
||||
|
||||
binary_dir = get_binary_dir(latest)
|
||||
if binary_dir.exists():
|
||||
# Already downloaded
|
||||
_write_version_marker(latest)
|
||||
return latest
|
||||
|
||||
logger.info("Downloading Chromium %s...", latest)
|
||||
_download_and_extract(version=latest)
|
||||
_write_version_marker(latest)
|
||||
return latest
|
||||
|
||||
|
||||
def _should_check_for_update() -> bool:
|
||||
"""Check if auto-update is enabled and rate limit hasn't been hit."""
|
||||
if os.environ.get("CLOAKBROWSER_AUTO_UPDATE", "").lower() == "false":
|
||||
return False
|
||||
if get_local_binary_override():
|
||||
return False
|
||||
if os.environ.get("CLOAKBROWSER_DOWNLOAD_URL"):
|
||||
return False
|
||||
|
||||
check_file = get_cache_dir() / ".last_update_check"
|
||||
if check_file.exists():
|
||||
try:
|
||||
last_check = float(check_file.read_text().strip())
|
||||
if time.time() - last_check < UPDATE_CHECK_INTERVAL:
|
||||
return False
|
||||
except (ValueError, OSError):
|
||||
pass
|
||||
return True
|
||||
|
||||
|
||||
def _get_latest_chromium_version() -> str | None:
|
||||
"""Hit GitHub Releases API, return latest chromium-v* version string or None."""
|
||||
try:
|
||||
resp = httpx.get(
|
||||
GITHUB_API_URL, params={"per_page": 10}, timeout=10.0
|
||||
)
|
||||
resp.raise_for_status()
|
||||
for release in resp.json():
|
||||
tag = release.get("tag_name", "")
|
||||
if tag.startswith("chromium-v") and not release.get("draft"):
|
||||
return tag.removeprefix("chromium-v")
|
||||
return None
|
||||
except Exception:
|
||||
logger.debug("Auto-update check failed", exc_info=True)
|
||||
return None
|
||||
|
||||
|
||||
def _write_version_marker(version: str) -> None:
|
||||
"""Write the latest version marker to cache dir."""
|
||||
cache_dir = get_cache_dir()
|
||||
cache_dir.mkdir(parents=True, exist_ok=True)
|
||||
marker = cache_dir / "latest_version"
|
||||
# Write to temp file then rename for atomicity
|
||||
tmp = marker.with_suffix(".tmp")
|
||||
tmp.write_text(version)
|
||||
tmp.rename(marker)
|
||||
|
||||
|
||||
def _check_and_download_update() -> None:
|
||||
"""Background task: check for newer binary, download if available."""
|
||||
try:
|
||||
# Record check timestamp first (rate limiting)
|
||||
check_file = get_cache_dir() / ".last_update_check"
|
||||
check_file.parent.mkdir(parents=True, exist_ok=True)
|
||||
check_file.write_text(str(time.time()))
|
||||
|
||||
latest = _get_latest_chromium_version()
|
||||
if latest is None:
|
||||
return
|
||||
if not _version_newer(latest, CHROMIUM_VERSION):
|
||||
return
|
||||
|
||||
# Already downloaded?
|
||||
if get_binary_dir(latest).exists():
|
||||
_write_version_marker(latest)
|
||||
return
|
||||
|
||||
logger.info(
|
||||
"Newer Chromium available: %s (current: %s). Downloading in background...",
|
||||
latest,
|
||||
CHROMIUM_VERSION,
|
||||
)
|
||||
_download_and_extract(version=latest)
|
||||
_write_version_marker(latest)
|
||||
logger.info(
|
||||
"Background update complete: Chromium %s ready. Will use on next launch.",
|
||||
latest,
|
||||
)
|
||||
except Exception:
|
||||
logger.debug("Background update failed", exc_info=True)
|
||||
|
||||
|
||||
def _maybe_trigger_update_check() -> None:
|
||||
"""Fire-and-forget update check in a daemon thread."""
|
||||
if not _should_check_for_update():
|
||||
return
|
||||
t = threading.Thread(target=_check_and_download_update, daemon=True)
|
||||
t.start()
|
||||
|
||||
@@ -0,0 +1,238 @@
|
||||
"""GeoIP-based timezone and locale detection from proxy IP.
|
||||
|
||||
Optional feature — requires ``geoip2`` package::
|
||||
|
||||
pip install cloakbrowser[geoip]
|
||||
|
||||
Downloads GeoLite2-City.mmdb (~70 MB) on first use, caches in
|
||||
``~/.cloakbrowser/geoip/``. Background re-download after 30 days.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import ipaddress
|
||||
import logging
|
||||
import socket
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
from pathlib import Path
|
||||
from urllib.parse import urlparse
|
||||
|
||||
logger = logging.getLogger("cloakbrowser")
|
||||
|
||||
# P3TERX mirror of MaxMind GeoLite2-City — no license key needed
|
||||
GEOIP_DB_URL = (
|
||||
"https://github.com/P3TERX/GeoLite.mmdb/raw/download/GeoLite2-City.mmdb"
|
||||
)
|
||||
GEOIP_DB_FILENAME = "GeoLite2-City.mmdb"
|
||||
GEOIP_UPDATE_INTERVAL = 30 * 86_400 # 30 days
|
||||
|
||||
# Country ISO code → BCP 47 locale (covers ~90 % of proxy traffic)
|
||||
COUNTRY_LOCALE_MAP: dict[str, str] = {
|
||||
"US": "en-US", "GB": "en-GB", "AU": "en-AU", "CA": "en-CA", "NZ": "en-NZ",
|
||||
"IE": "en-IE", "ZA": "en-ZA", "SG": "en-SG",
|
||||
"DE": "de-DE", "AT": "de-AT", "CH": "de-CH",
|
||||
"FR": "fr-FR", "BE": "fr-BE",
|
||||
"ES": "es-ES", "MX": "es-MX", "AR": "es-AR", "CO": "es-CO", "CL": "es-CL",
|
||||
"BR": "pt-BR", "PT": "pt-PT",
|
||||
"IT": "it-IT", "NL": "nl-NL",
|
||||
"JP": "ja-JP", "KR": "ko-KR", "CN": "zh-CN", "TW": "zh-TW", "HK": "zh-HK",
|
||||
"RU": "ru-RU", "UA": "uk-UA", "PL": "pl-PL", "CZ": "cs-CZ", "RO": "ro-RO",
|
||||
"IL": "he-IL", "TR": "tr-TR", "SA": "ar-SA", "AE": "ar-AE", "EG": "ar-EG",
|
||||
"IN": "hi-IN", "ID": "id-ID", "PH": "en-PH",
|
||||
"TH": "th-TH", "VN": "vi-VN", "MY": "ms-MY",
|
||||
"SE": "sv-SE", "NO": "nb-NO", "DK": "da-DK", "FI": "fi-FI",
|
||||
"GR": "el-GR", "HU": "hu-HU", "BG": "bg-BG",
|
||||
}
|
||||
|
||||
|
||||
def resolve_proxy_geo(proxy_url: str) -> tuple[str | None, str | None]:
|
||||
"""Resolve timezone and locale from a proxy's IP address.
|
||||
|
||||
Returns ``(timezone, locale)`` — either or both may be ``None`` on
|
||||
failure (missing dep, DB download error, lookup miss). Never raises.
|
||||
"""
|
||||
try:
|
||||
import geoip2.database # noqa: F811
|
||||
except ImportError:
|
||||
raise ImportError(
|
||||
"geoip2 is required for geoip=True. Install it with:\n"
|
||||
" pip install cloakbrowser[geoip]"
|
||||
) from None
|
||||
|
||||
db_path = _ensure_geoip_db()
|
||||
if db_path is None:
|
||||
return None, None
|
||||
|
||||
# Exit IP (through proxy) is most accurate — gateway DNS may differ from exit
|
||||
ip = _resolve_exit_ip(proxy_url)
|
||||
if ip is None:
|
||||
ip = _resolve_proxy_ip(proxy_url)
|
||||
if ip is None:
|
||||
return None, None
|
||||
|
||||
try:
|
||||
with geoip2.database.Reader(str(db_path)) as reader:
|
||||
resp = reader.city(ip)
|
||||
timezone = resp.location.time_zone
|
||||
country = resp.country.iso_code
|
||||
locale = COUNTRY_LOCALE_MAP.get(country) if country else None
|
||||
logger.debug(
|
||||
"GeoIP: %s → tz=%s, country=%s, locale=%s",
|
||||
ip, timezone, country, locale,
|
||||
)
|
||||
return timezone, locale
|
||||
except Exception as exc:
|
||||
logger.debug("GeoIP lookup failed for %s: %s", ip, exc)
|
||||
return None, None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Proxy IP resolution
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _resolve_proxy_ip(proxy_url: str) -> str | None:
|
||||
"""Extract proxy hostname from URL and resolve to an IP address."""
|
||||
try:
|
||||
hostname = urlparse(proxy_url).hostname
|
||||
if not hostname:
|
||||
return None
|
||||
|
||||
# Already a literal IP?
|
||||
try:
|
||||
socket.inet_pton(socket.AF_INET, hostname)
|
||||
return hostname
|
||||
except OSError:
|
||||
pass
|
||||
try:
|
||||
socket.inet_pton(socket.AF_INET6, hostname)
|
||||
return hostname
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
# DNS resolve (returns first result, handles both v4/v6)
|
||||
results = socket.getaddrinfo(hostname, None, socket.AF_UNSPEC, socket.SOCK_STREAM)
|
||||
if results:
|
||||
ip = results[0][4][0]
|
||||
logger.debug("Resolved proxy %s → %s", hostname, ip)
|
||||
return ip
|
||||
return None
|
||||
except Exception as exc:
|
||||
logger.debug("Failed to resolve proxy hostname: %s", exc)
|
||||
return None
|
||||
|
||||
|
||||
def _is_private_ip(ip: str) -> bool:
|
||||
"""Check if an IP address is private/internal (not routable on the internet)."""
|
||||
try:
|
||||
return ipaddress.ip_address(ip).is_private
|
||||
except ValueError:
|
||||
return False
|
||||
|
||||
|
||||
# IP echo services — fast, no auth, return just the IP
|
||||
_IP_ECHO_URLS = [
|
||||
"https://api.ipify.org",
|
||||
"https://checkip.amazonaws.com",
|
||||
"https://ifconfig.me/ip",
|
||||
]
|
||||
|
||||
|
||||
def _resolve_exit_ip(proxy_url: str) -> str | None:
|
||||
"""Discover the proxy's actual exit IP by connecting through it."""
|
||||
import httpx
|
||||
|
||||
for url in _IP_ECHO_URLS:
|
||||
try:
|
||||
resp = httpx.get(url, proxy=proxy_url, timeout=10.0)
|
||||
resp.raise_for_status()
|
||||
ip = resp.text.strip()
|
||||
# Validate it looks like an IP
|
||||
ipaddress.ip_address(ip)
|
||||
logger.debug("Exit IP via %s: %s", url, ip)
|
||||
return ip
|
||||
except Exception:
|
||||
continue
|
||||
logger.debug("Failed to discover exit IP through proxy")
|
||||
return None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# GeoIP database management
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _get_geoip_dir() -> Path:
|
||||
from .config import get_cache_dir
|
||||
|
||||
return get_cache_dir() / "geoip"
|
||||
|
||||
|
||||
def _ensure_geoip_db() -> Path | None:
|
||||
"""Return path to GeoLite2-City.mmdb, downloading on first use."""
|
||||
db_path = _get_geoip_dir() / GEOIP_DB_FILENAME
|
||||
|
||||
if db_path.exists():
|
||||
_maybe_trigger_update(db_path)
|
||||
return db_path
|
||||
|
||||
try:
|
||||
_download_geoip_db(db_path)
|
||||
return db_path
|
||||
except Exception as exc:
|
||||
logger.warning("Failed to download GeoIP database: %s", exc)
|
||||
return None
|
||||
|
||||
|
||||
def _download_geoip_db(dest: Path) -> None:
|
||||
"""Atomic download of GeoLite2-City.mmdb via httpx."""
|
||||
import httpx
|
||||
|
||||
dest.parent.mkdir(parents=True, exist_ok=True)
|
||||
logger.info("Downloading GeoIP database (~70 MB) …")
|
||||
|
||||
tmp_fd, tmp_name = tempfile.mkstemp(dir=dest.parent, suffix=".tmp")
|
||||
tmp_path = Path(tmp_name)
|
||||
try:
|
||||
with httpx.stream(
|
||||
"GET", GEOIP_DB_URL, follow_redirects=True, timeout=300.0
|
||||
) as resp:
|
||||
resp.raise_for_status()
|
||||
total = int(resp.headers.get("content-length", 0))
|
||||
downloaded = 0
|
||||
last_pct = -1
|
||||
with open(tmp_fd, "wb") as f:
|
||||
for chunk in resp.iter_bytes(chunk_size=65_536):
|
||||
f.write(chunk)
|
||||
downloaded += len(chunk)
|
||||
if total:
|
||||
pct = downloaded * 100 // total
|
||||
if pct >= last_pct + 10:
|
||||
last_pct = pct
|
||||
logger.info("GeoIP download: %d %%", pct)
|
||||
|
||||
tmp_path.rename(dest)
|
||||
logger.info("GeoIP database ready: %s", dest)
|
||||
except Exception:
|
||||
tmp_path.unlink(missing_ok=True)
|
||||
raise
|
||||
|
||||
|
||||
def _maybe_trigger_update(db_path: Path) -> None:
|
||||
"""Re-download in background if DB is older than 30 days."""
|
||||
try:
|
||||
age = time.time() - db_path.stat().st_mtime
|
||||
if age < GEOIP_UPDATE_INTERVAL:
|
||||
return
|
||||
except OSError:
|
||||
return
|
||||
|
||||
def _bg() -> None:
|
||||
try:
|
||||
_download_geoip_db(db_path)
|
||||
except Exception:
|
||||
logger.debug("Background GeoIP update failed", exc_info=True)
|
||||
|
||||
threading.Thread(target=_bg, daemon=True).start()
|
||||
@@ -0,0 +1,227 @@
|
||||
"""Test against fingerprint-scan.com and CreepJS.
|
||||
|
||||
Tests the specific headless detection signals flagged by the community:
|
||||
- noTaskbar, noContentIndex, noContactsManager, noDownlinkMax
|
||||
- Bot risk score (fingerprint-scan.com)
|
||||
- Headless/stealth percentages (CreepJS)
|
||||
- Full CreepJS signal breakdown (likeHeadless, headless, stealth)
|
||||
|
||||
Usage:
|
||||
python examples/fingerprint_scan_test.py
|
||||
python examples/fingerprint_scan_test.py --proxy http://10.50.96.5:8888
|
||||
python examples/fingerprint_scan_test.py --headless
|
||||
"""
|
||||
|
||||
import sys
|
||||
|
||||
from cloakbrowser import launch_context
|
||||
|
||||
HEADLESS = "--headless" in sys.argv
|
||||
PROXY = None
|
||||
for i, arg in enumerate(sys.argv):
|
||||
if arg == "--proxy" and i + 1 < len(sys.argv):
|
||||
PROXY = sys.argv[i + 1]
|
||||
|
||||
|
||||
def test_fingerprint_scan(page):
|
||||
"""fingerprint-scan.com — bot risk score + headless detection signals."""
|
||||
print("=== fingerprint-scan.com ===")
|
||||
page.goto("https://fingerprint-scan.com/", wait_until="domcontentloaded", timeout=30000)
|
||||
page.wait_for_timeout(20000) # Castle.js needs time to compute score
|
||||
|
||||
# Check bot risk score
|
||||
score = page.evaluate(
|
||||
'document.getElementById("fingerprintScore")?.textContent || "Score not rendered"'
|
||||
)
|
||||
print(f"Bot Risk Score: {score}")
|
||||
|
||||
# Check headless detection signals
|
||||
apis = page.evaluate("""() => ({
|
||||
noTaskbar: screen.height === screen.availHeight,
|
||||
taskbarSize: screen.height - screen.availHeight,
|
||||
noContentIndex: typeof window.ContentIndex === "undefined",
|
||||
noContactsManager: !("contacts" in navigator),
|
||||
noDownlinkMax: !("downlinkMax" in (navigator.connection || {})),
|
||||
downlinkMax: navigator.connection?.downlinkMax ?? null,
|
||||
timezone: Intl.DateTimeFormat().resolvedOptions().timeZone,
|
||||
webdriver: navigator.webdriver,
|
||||
isPlaywright: "__pwInitScripts" in window || "__playwright__binding__" in window,
|
||||
webgpu: typeof navigator.gpu !== "undefined" ? "available" : "NOT_AVAILABLE",
|
||||
scrollbarWidth: (() => { const d = document.createElement("div"); d.style.cssText = "overflow:scroll;width:100px;height:100px;position:absolute;top:-999px"; document.body.appendChild(d); const w = d.offsetWidth - d.clientWidth; d.remove(); return w; })()
|
||||
})""")
|
||||
|
||||
print("\nHeadless detection signals:")
|
||||
headless_fails = 0
|
||||
for k, v in apis.items():
|
||||
is_fail = k.startswith("no") and v is True
|
||||
if is_fail:
|
||||
headless_fails += 1
|
||||
flag = "FAIL" if is_fail else ""
|
||||
print(f" {k}: {v} {flag}")
|
||||
|
||||
# Extract bot test results from page
|
||||
bot_tests = page.evaluate("""() => {
|
||||
const text = document.body.innerText;
|
||||
const tests = {};
|
||||
for (const key of ['WebDriver', 'Is Selenium Chrome', 'CDP Check', 'Is Playwright']) {
|
||||
const match = text.match(new RegExp(key + '\\\\s+(true|false)'));
|
||||
if (match) tests[key] = match[1];
|
||||
}
|
||||
return tests;
|
||||
}""")
|
||||
print("\nBot Detection Tests:")
|
||||
for k, v in bot_tests.items():
|
||||
status = "PASS" if v == "false" else "FAIL"
|
||||
print(f" {k}: {v} [{status}]")
|
||||
|
||||
page.screenshot(path="/results/fingerprint-scan.png", full_page=True)
|
||||
print("\nScreenshot: /results/fingerprint-scan.png")
|
||||
|
||||
return {
|
||||
"score": score,
|
||||
"headless_fails": headless_fails,
|
||||
"apis": apis,
|
||||
"bot_tests": bot_tests,
|
||||
}
|
||||
|
||||
|
||||
def test_creepjs(page):
|
||||
"""abrahamjuliot.github.io/creepjs — comprehensive fingerprint analysis."""
|
||||
print("\n=== CreepJS ===")
|
||||
page.goto(
|
||||
"https://abrahamjuliot.github.io/creepjs/", wait_until="domcontentloaded", timeout=30000
|
||||
)
|
||||
print("Waiting 30s for CreepJS analysis...")
|
||||
page.wait_for_timeout(30000)
|
||||
|
||||
# Extract % scores from page text (matches test-infra/matrix_tests/group3_bot_detection.py)
|
||||
scores = page.evaluate("""() => {
|
||||
const text = document.body.innerText;
|
||||
const likeMatch = text.match(/(\\d+)%\\s*like headless/i);
|
||||
const headlessMatch = text.match(/(\\d+)%\\s*headless:/i);
|
||||
const stealthMatch = text.match(/(\\d+)%\\s*stealth:/i);
|
||||
return {
|
||||
likeHeadlessPct: likeMatch ? parseInt(likeMatch[1]) : null,
|
||||
headlessPct: headlessMatch ? parseInt(headlessMatch[1]) : null,
|
||||
stealthPct: stealthMatch ? parseInt(stealthMatch[1]) : null,
|
||||
};
|
||||
}""")
|
||||
|
||||
print(f"\nScores:")
|
||||
print(f" like-headless: {scores['likeHeadlessPct']}% (target: <=30%)")
|
||||
print(f" headless: {scores['headlessPct']}% (target: 0%)")
|
||||
print(f" stealth: {scores['stealthPct']}% (target: 0%)")
|
||||
|
||||
# Extract full signal breakdown from window.Fingerprint.headless (CreepJS internal object)
|
||||
signals = page.evaluate("""() => {
|
||||
try {
|
||||
const fp = window.Fingerprint;
|
||||
if (!fp || !fp.headless) return null;
|
||||
return {
|
||||
likeHeadless: fp.headless.likeHeadless || null,
|
||||
headless: fp.headless.headless || null,
|
||||
stealth: fp.headless.stealth || null,
|
||||
};
|
||||
} catch { return null; }
|
||||
}""")
|
||||
|
||||
if signals:
|
||||
if signals.get("likeHeadless"):
|
||||
print("\nlikeHeadless signals:")
|
||||
fails = 0
|
||||
for k, v in signals["likeHeadless"].items():
|
||||
is_fail = v is True
|
||||
if is_fail:
|
||||
fails += 1
|
||||
flag = " FAIL" if is_fail else ""
|
||||
print(f" {k}: {v}{flag}")
|
||||
print(f" ({fails} fails)")
|
||||
|
||||
if signals.get("headless"):
|
||||
print("\nheadless signals:")
|
||||
for k, v in signals["headless"].items():
|
||||
flag = " FAIL" if v is True else ""
|
||||
print(f" {k}: {v}{flag}")
|
||||
|
||||
if signals.get("stealth"):
|
||||
print("\nstealth signals:")
|
||||
for k, v in signals["stealth"].items():
|
||||
flag = " FAIL" if v is True else ""
|
||||
print(f" {k}: {v}{flag}")
|
||||
else:
|
||||
print("\n(window.Fingerprint.headless not available — signals not extracted)")
|
||||
|
||||
# Extract platform estimate
|
||||
platform = page.evaluate("""() => {
|
||||
try {
|
||||
const fp = window.Fingerprint;
|
||||
if (!fp || !fp.platformEstimate) return null;
|
||||
return fp.platformEstimate;
|
||||
} catch { return null; }
|
||||
}""")
|
||||
if platform:
|
||||
print(f"\nPlatform estimate: {platform}")
|
||||
|
||||
passed = (
|
||||
scores["headlessPct"] is not None
|
||||
and scores["headlessPct"] <= 30
|
||||
and scores["stealthPct"] is not None
|
||||
and scores["stealthPct"] <= 30
|
||||
)
|
||||
print(f"\nVerdict: {'PASS' if passed else 'FAIL'} (<=30% headless, <=30% stealth)")
|
||||
|
||||
page.screenshot(path="/results/creepjs.png", full_page=True)
|
||||
print("Screenshot: /results/creepjs.png")
|
||||
|
||||
return {**scores, "signals": signals, "platform": platform}
|
||||
|
||||
|
||||
def main():
|
||||
print("=" * 60)
|
||||
print("CloakBrowser — Fingerprint & Headless Detection Tests")
|
||||
print("=" * 60)
|
||||
print(f"Mode: {'headless' if HEADLESS else 'headed'}")
|
||||
print(f"Proxy: {PROXY or 'none'}")
|
||||
print()
|
||||
|
||||
context = launch_context(
|
||||
headless=HEADLESS,
|
||||
proxy=PROXY,
|
||||
args=[
|
||||
"--fingerprint-screen-width=1920",
|
||||
"--fingerprint-screen-height=1080",
|
||||
"--timezone=Asia/Jerusalem",
|
||||
],
|
||||
)
|
||||
page = context.new_page()
|
||||
|
||||
try:
|
||||
fp_result = test_fingerprint_scan(page)
|
||||
creep_result = test_creepjs(page)
|
||||
finally:
|
||||
context.close()
|
||||
|
||||
# Summary
|
||||
print("\n" + "=" * 60)
|
||||
print("SUMMARY")
|
||||
print("=" * 60)
|
||||
print(f"fingerprint-scan.com: {fp_result['score']}")
|
||||
print(f" Headless signal fails: {fp_result['headless_fails']}")
|
||||
like = creep_result["likeHeadlessPct"]
|
||||
headless = creep_result["headlessPct"]
|
||||
stealth = creep_result["stealthPct"]
|
||||
print(f"CreepJS: like-headless={like}%, headless={headless}%, stealth={stealth}%")
|
||||
|
||||
# Count CreepJS signal fails
|
||||
sigs = creep_result.get("signals")
|
||||
if sigs and sigs.get("likeHeadless"):
|
||||
fail_names = [k for k, v in sigs["likeHeadless"].items() if v is True]
|
||||
if fail_names:
|
||||
print(f" likeHeadless fails: {', '.join(fail_names)}")
|
||||
print("=" * 60)
|
||||
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -5,6 +5,8 @@ Expected: 0.9 (human-level) with cloakbrowser.
|
||||
Default Playwright typically scores 0.1-0.3.
|
||||
"""
|
||||
|
||||
import time
|
||||
|
||||
from cloakbrowser import launch
|
||||
|
||||
browser = launch(headless=True)
|
||||
@@ -18,7 +20,7 @@ page.wait_for_load_state("networkidle")
|
||||
button = page.query_selector("button")
|
||||
if button:
|
||||
button.click()
|
||||
page.wait_for_timeout(3000)
|
||||
time.sleep(3)
|
||||
|
||||
# Extract score from page
|
||||
content = page.content()
|
||||
|
||||
+241
-27
@@ -1,57 +1,271 @@
|
||||
"""Run stealth tests against major bot detection services.
|
||||
|
||||
Tests cloakbrowser against multiple detection sites and reports results.
|
||||
Tests cloakbrowser against multiple detection sites, extracts pass/fail
|
||||
verdicts via JS evaluation, and reports results with screenshots.
|
||||
|
||||
Usage:
|
||||
python examples/stealth_test.py
|
||||
python examples/stealth_test.py --headed # watch in real-time
|
||||
python examples/stealth_test.py --no-screenshots
|
||||
python examples/stealth_test.py --proxy http://10.50.96.5:8888
|
||||
"""
|
||||
|
||||
import json
|
||||
import sys
|
||||
import time
|
||||
|
||||
from cloakbrowser import launch
|
||||
|
||||
HEADED = "--headed" in sys.argv
|
||||
SCREENSHOTS = "--no-screenshots" not in sys.argv
|
||||
PROXY = None
|
||||
for i, arg in enumerate(sys.argv):
|
||||
if arg == "--proxy" and i + 1 < len(sys.argv):
|
||||
PROXY = sys.argv[i + 1]
|
||||
|
||||
|
||||
def test_bot_sannysoft(page):
|
||||
"""bot.sannysoft.com — classic bot detection checks."""
|
||||
page.goto("https://bot.sannysoft.com", wait_until="networkidle", timeout=30000)
|
||||
time.sleep(3)
|
||||
|
||||
results = page.evaluate("""() => {
|
||||
const rows = document.querySelectorAll('table tr');
|
||||
const data = {};
|
||||
rows.forEach(r => {
|
||||
const cells = r.querySelectorAll('td');
|
||||
if (cells.length >= 2) {
|
||||
const key = cells[0].innerText.trim();
|
||||
const val = cells[1].innerText.trim();
|
||||
const cls = cells[1].className || '';
|
||||
data[key] = {value: val, passed: !cls.includes('failed')};
|
||||
}
|
||||
});
|
||||
return data;
|
||||
}""")
|
||||
|
||||
failed = [k for k, v in results.items() if not v["passed"]]
|
||||
total = len(results)
|
||||
passed = total - len(failed)
|
||||
return {"passed": passed, "total": total, "failed": failed}
|
||||
|
||||
|
||||
def test_bot_incolumitas(page):
|
||||
"""bot.incolumitas.com — comprehensive 30+ check bot detection."""
|
||||
page.goto("https://bot.incolumitas.com", wait_until="networkidle", timeout=30000)
|
||||
time.sleep(12) # needs time to run all detection tests
|
||||
|
||||
# Site outputs JSON blocks in page text, not HTML tables
|
||||
results = page.evaluate("""() => {
|
||||
const text = document.body.innerText;
|
||||
const okMatches = text.match(/"\\w+":\\s*"OK"/g) || [];
|
||||
const failMatches = text.match(/"\\w+":\\s*"FAIL"/g) || [];
|
||||
const failedTests = failMatches.map(m => m.match(/"(\\w+)"/)[1]);
|
||||
return {
|
||||
passed: okMatches.length,
|
||||
failed: failMatches.length,
|
||||
failedTests,
|
||||
total: okMatches.length + failMatches.length
|
||||
};
|
||||
}""")
|
||||
return results
|
||||
|
||||
|
||||
def test_browserscan(page):
|
||||
"""browserscan.net/bot-detection — WebDriver, UA, CDP, Navigator checks."""
|
||||
page.goto("https://www.browserscan.net/bot-detection", wait_until="networkidle", timeout=30000)
|
||||
time.sleep(5)
|
||||
|
||||
results = page.evaluate("""() => {
|
||||
const items = document.querySelectorAll('[class*="result"], [class*="item"], [class*="check"]');
|
||||
let normal = 0, abnormal = 0;
|
||||
const text = document.body.innerText;
|
||||
// Count "Normal" vs "Abnormal" verdicts
|
||||
const normalMatches = text.match(/Normal/g);
|
||||
const abnormalMatches = text.match(/Abnormal/g);
|
||||
return {
|
||||
normal: normalMatches ? normalMatches.length : 0,
|
||||
abnormal: abnormalMatches ? abnormalMatches.length : 0,
|
||||
pageText: text.substring(0, 500)
|
||||
};
|
||||
}""")
|
||||
return results
|
||||
|
||||
|
||||
def test_deviceandbrowserinfo(page):
|
||||
"""deviceandbrowserinfo.com/are_you_a_bot — fingerprint + behavioral detection."""
|
||||
page.goto("https://deviceandbrowserinfo.com/are_you_a_bot", wait_until="domcontentloaded", timeout=30000)
|
||||
time.sleep(8)
|
||||
|
||||
results = page.evaluate("""() => {
|
||||
const text = document.body.innerText;
|
||||
// Site outputs JSON with "isBot": false and detail checks
|
||||
const botMatch = text.match(/"isBot":\\s*(true|false)/);
|
||||
const isBot = botMatch ? botMatch[1] === 'true' : null;
|
||||
const checks = {};
|
||||
const patterns = [
|
||||
'isBot', 'hasBotUserAgent', 'hasWebdriverTrue',
|
||||
'isHeadlessChrome', 'isAutomatedWithCDP', 'hasSuspiciousWeakSignals',
|
||||
'isPlaywright', 'hasInconsistentChromeObject'
|
||||
];
|
||||
patterns.forEach(p => {
|
||||
const match = text.match(new RegExp('"' + p + '":\\s*(true|false)'));
|
||||
if (match) checks[p] = match[1] === 'true';
|
||||
});
|
||||
return {isBot, checks};
|
||||
}""")
|
||||
return results
|
||||
|
||||
|
||||
def test_fingerprintjs(page):
|
||||
"""demo.fingerprint.com/web-scraping — industry-standard bot detection."""
|
||||
page.goto("https://demo.fingerprint.com/web-scraping", wait_until="domcontentloaded", timeout=30000)
|
||||
time.sleep(8)
|
||||
|
||||
# Click search to trigger bot detection — bots get blocked, humans see flights
|
||||
try:
|
||||
page.click("button:has-text('Search')", timeout=5000)
|
||||
time.sleep(5)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
results = page.evaluate("""() => {
|
||||
const text = document.body.innerText;
|
||||
// Bots see error messages; humans see flight prices
|
||||
const hasFlights = text.includes('Price per adult') || text.includes('$');
|
||||
const isBlocked = text.includes('request was blocked') || text.includes('bot visit detected');
|
||||
return {passed: hasFlights && !isBlocked, isBlocked, hasFlights};
|
||||
}""")
|
||||
return results
|
||||
|
||||
|
||||
def test_recaptcha(page):
|
||||
"""recaptcha-demo.appspot.com — Google's official reCAPTCHA v3 score."""
|
||||
page.goto(
|
||||
"https://recaptcha-demo.appspot.com/recaptcha-v3-request-scores.php",
|
||||
wait_until="networkidle",
|
||||
timeout=30000,
|
||||
)
|
||||
# Page auto-submits via grecaptcha.execute() — wait for backend response
|
||||
time.sleep(8)
|
||||
|
||||
results = page.evaluate("""() => {
|
||||
const text = document.body.innerText;
|
||||
// Score appears in JSON response block: "score": 0.9
|
||||
const scoreMatch = text.match(/"score":\\s*(\\d+\\.\\d+)/);
|
||||
return {
|
||||
score: scoreMatch ? parseFloat(scoreMatch[1]) : null,
|
||||
pageText: text.substring(0, 500)
|
||||
};
|
||||
}""")
|
||||
return results
|
||||
|
||||
|
||||
TESTS = [
|
||||
{
|
||||
"name": "bot.sannysoft.com",
|
||||
"url": "https://bot.sannysoft.com",
|
||||
"runner": test_bot_sannysoft,
|
||||
"verdict": lambda r: f"{r['passed']}/{r['total']} passed"
|
||||
+ (f" (FAILED: {', '.join(r['failed'])})" if r["failed"] else " — ALL GREEN"),
|
||||
"pass": lambda r: len(r["failed"]) == 0,
|
||||
},
|
||||
{
|
||||
"name": "bot.incolumitas.com",
|
||||
"url": "https://bot.incolumitas.com",
|
||||
"check": "Bot detection analysis",
|
||||
"runner": test_bot_incolumitas,
|
||||
"verdict": lambda r: f"{r['passed']}/{r['total']} passed"
|
||||
+ (f" (FAILED: {', '.join(r.get('failedTests', []))})" if r.get("failed", 0) > 0 else " — ALL GREEN"),
|
||||
"pass": lambda r: r.get("failed", 0) <= 1, # fpscanner.WEBDRIVER false positive expected (all builds)
|
||||
},
|
||||
{
|
||||
"name": "BrowserScan",
|
||||
"url": "https://www.browserscan.net/bot-detection",
|
||||
"check": "Bot detection status",
|
||||
"runner": test_browserscan,
|
||||
"verdict": lambda r: f"Normal: {r['normal']}, Abnormal: {r['abnormal']}",
|
||||
"pass": lambda r: r.get("abnormal", 1) == 0,
|
||||
},
|
||||
{
|
||||
"name": "deviceandbrowserinfo.com",
|
||||
"url": "https://deviceandbrowserinfo.com/are_you_a_bot",
|
||||
"check": "isBot flag",
|
||||
"runner": test_deviceandbrowserinfo,
|
||||
"verdict": lambda r: f"isBot: {r.get('isBot', 'unknown')}"
|
||||
+ (f" checks: {json.dumps(r.get('checks', {}))}" if r.get("checks") else ""),
|
||||
"pass": lambda r: not r.get("isBot", True),
|
||||
},
|
||||
{
|
||||
"name": "FingerprintJS",
|
||||
"url": "https://demo.fingerprint.com/web-scraping",
|
||||
"check": "Bot detection result",
|
||||
"runner": test_fingerprintjs,
|
||||
"verdict": lambda r: "PASSED (flights shown)" if r.get("passed") else "BLOCKED" if r.get("isBlocked") else "NO FLIGHTS",
|
||||
"pass": lambda r: r.get("passed", False),
|
||||
},
|
||||
{
|
||||
"name": "reCAPTCHA v3 (Google)",
|
||||
"url": "https://recaptcha-demo.appspot.com/recaptcha-v3-request-scores.php",
|
||||
"runner": test_recaptcha,
|
||||
"verdict": lambda r: f"Score: {r.get('score', 'N/A')}",
|
||||
"pass": lambda r: (r.get("score") or 0) >= 0.7,
|
||||
},
|
||||
]
|
||||
|
||||
browser = launch(headless=True)
|
||||
page = browser.new_page()
|
||||
|
||||
print("=" * 60)
|
||||
print("CloakBrowser Stealth Test Suite")
|
||||
print("=" * 60)
|
||||
def main():
|
||||
print("=" * 60)
|
||||
print("CloakBrowser Stealth Test Suite")
|
||||
print("=" * 60)
|
||||
print(f"Mode: {'headed' if HEADED else 'headless'}")
|
||||
print(f"Screenshots: {'on' if SCREENSHOTS else 'off'}")
|
||||
print(f"Proxy: {PROXY or 'none'}")
|
||||
print()
|
||||
|
||||
for test in TESTS:
|
||||
print(f"\n--- {test['name']} ---")
|
||||
print(f"URL: {test['url']}")
|
||||
try:
|
||||
page.goto(test["url"], wait_until="networkidle", timeout=30000)
|
||||
page.wait_for_timeout(3000)
|
||||
browser = launch(headless=not HEADED, proxy=PROXY)
|
||||
page = browser.new_page()
|
||||
|
||||
# Screenshot each test
|
||||
filename = f"stealth_test_{test['name'].replace('.', '_').replace(' ', '_')}.png"
|
||||
page.screenshot(path=filename)
|
||||
print(f"Screenshot: {filename}")
|
||||
print(f"Title: {page.title()}")
|
||||
except Exception as e:
|
||||
print(f"Error: {e}")
|
||||
results_summary = []
|
||||
|
||||
browser.close()
|
||||
for test in TESTS:
|
||||
name = test["name"]
|
||||
print(f"--- {name} ---")
|
||||
print(f"URL: {test['url']}")
|
||||
|
||||
print("\n" + "=" * 60)
|
||||
print("Tests complete. Check screenshots for results.")
|
||||
print("=" * 60)
|
||||
try:
|
||||
result = test["runner"](page)
|
||||
passed = test["pass"](result)
|
||||
verdict = test["verdict"](result)
|
||||
status = "PASS" if passed else "FAIL"
|
||||
results_summary.append((name, status, verdict))
|
||||
|
||||
print(f"Result: [{status}] {verdict}")
|
||||
|
||||
if SCREENSHOTS:
|
||||
filename = f"stealth_test_{name.replace('.', '_').replace(' ', '_').replace('/', '_')}.png"
|
||||
page.screenshot(path=filename)
|
||||
print(f"Screenshot: {filename}")
|
||||
|
||||
except Exception as e:
|
||||
results_summary.append((name, "ERROR", str(e)))
|
||||
print(f"Error: {e}")
|
||||
|
||||
print()
|
||||
|
||||
browser.close()
|
||||
|
||||
# Summary table
|
||||
print("=" * 60)
|
||||
print("RESULTS SUMMARY")
|
||||
print("=" * 60)
|
||||
for name, status, verdict in results_summary:
|
||||
icon = {"PASS": "+", "FAIL": "!", "ERROR": "x"}[status]
|
||||
print(f" [{icon}] {name}: {verdict}")
|
||||
|
||||
passed_count = sum(1 for _, s, _ in results_summary if s == "PASS")
|
||||
total = len(results_summary)
|
||||
print(f"\n {passed_count}/{total} tests passed")
|
||||
print("=" * 60)
|
||||
|
||||
return 0 if passed_count == total else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 304 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 4.0 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 90 KiB |
+219
@@ -0,0 +1,219 @@
|
||||
<p align="center">
|
||||
<img src="https://i.imgur.com/cqkp6fG.png" width="500" alt="CloakBrowser">
|
||||
</p>
|
||||
|
||||
# CloakBrowser
|
||||
|
||||
[](https://www.npmjs.com/package/cloakbrowser)
|
||||
[](https://github.com/CloakHQ/CloakBrowser/blob/main/LICENSE)
|
||||
|
||||
**Stealth Chromium that passes every bot detection test.**
|
||||
|
||||
Drop-in Playwright/Puppeteer replacement. Same API — just swap the import. Scores **0.9 on reCAPTCHA v3**, passes **Cloudflare Turnstile**, and clears **30/30** stealth detection tests.
|
||||
|
||||
- 🔒 **26 source-level C++ patches** — not JS injection, not config flags
|
||||
- 🎯 **0.9 reCAPTCHA v3 score** — human-level, server-verified
|
||||
- ☁️ **Passes Cloudflare Turnstile**, FingerprintJS, BrowserScan — 30/30 tests
|
||||
- 🔄 **Drop-in replacement** — works with both Playwright and Puppeteer
|
||||
- 📦 **`npm install cloakbrowser`** — binary auto-downloads, zero config
|
||||
|
||||
## Install
|
||||
|
||||
```bash
|
||||
# With Playwright
|
||||
npm install cloakbrowser playwright-core
|
||||
|
||||
# With Puppeteer
|
||||
npm install cloakbrowser puppeteer-core
|
||||
```
|
||||
|
||||
On first launch, the stealth Chromium binary auto-downloads (~200MB, cached at `~/.cloakbrowser/`).
|
||||
|
||||
## Usage
|
||||
|
||||
### Playwright (default)
|
||||
|
||||
```javascript
|
||||
import { launch } from 'cloakbrowser';
|
||||
|
||||
const browser = await launch();
|
||||
const page = await browser.newPage();
|
||||
await page.goto('https://protected-site.com');
|
||||
console.log(await page.title());
|
||||
await browser.close();
|
||||
```
|
||||
|
||||
### Puppeteer
|
||||
|
||||
> **Note:** Playwright is recommended for sites with reCAPTCHA Enterprise. Puppeteer's CDP protocol leaks automation signals that reCAPTCHA Enterprise can detect. This is a known Puppeteer limitation, not specific to CloakBrowser.
|
||||
|
||||
```javascript
|
||||
import { launch } from 'cloakbrowser/puppeteer';
|
||||
|
||||
const browser = await launch();
|
||||
const page = await browser.newPage();
|
||||
await page.goto('https://protected-site.com');
|
||||
console.log(await page.title());
|
||||
await browser.close();
|
||||
```
|
||||
|
||||
### Options
|
||||
|
||||
```javascript
|
||||
import { launch, launchContext } from 'cloakbrowser';
|
||||
|
||||
// With proxy
|
||||
const browser = await launch({
|
||||
proxy: 'http://user:pass@proxy:8080',
|
||||
});
|
||||
|
||||
// Headed mode (visible browser window)
|
||||
const browser = await launch({ headless: false });
|
||||
|
||||
// Extra Chrome args
|
||||
const browser = await launch({
|
||||
args: ['--window-size=1920,1080'],
|
||||
});
|
||||
|
||||
// With timezone and locale (sets --timezone and --lang binary flags)
|
||||
const browser = await launch({
|
||||
timezone: 'America/New_York',
|
||||
locale: 'en-US',
|
||||
});
|
||||
|
||||
// Auto-detect timezone/locale from proxy IP (requires: npm install mmdb-lib)
|
||||
const browser = await launch({
|
||||
proxy: 'http://proxy:8080',
|
||||
geoip: true,
|
||||
});
|
||||
|
||||
// Browser + context in one call (timezone/locale set both binary flags AND context)
|
||||
const context = await launchContext({
|
||||
userAgent: 'Custom UA',
|
||||
viewport: { width: 1920, height: 1080 },
|
||||
locale: 'en-US',
|
||||
timezoneId: 'America/New_York',
|
||||
});
|
||||
```
|
||||
|
||||
### Auto Timezone/Locale from Proxy IP
|
||||
|
||||
When using a proxy, antibot systems check that your browser's timezone and locale match the proxy's location. Install `mmdb-lib` to enable auto-detection from an offline GeoIP database (~70 MB, downloaded on first use):
|
||||
|
||||
```bash
|
||||
npm install mmdb-lib
|
||||
```
|
||||
|
||||
```javascript
|
||||
// Auto-detect — timezone and locale set from proxy's IP geolocation
|
||||
const browser = await launch({ proxy: 'http://proxy:8080', geoip: true });
|
||||
|
||||
// Works with launchContext too
|
||||
const context = await launchContext({ proxy: 'http://proxy:8080', geoip: true });
|
||||
|
||||
// Explicit values always win over auto-detection
|
||||
const browser = await launch({ proxy: 'http://proxy:8080', geoip: true, timezone: 'Europe/London' });
|
||||
```
|
||||
|
||||
> **Note:** For rotating residential proxies, the DNS-resolved IP may differ from the exit IP. Pass explicit `timezone`/`locale` in those cases.
|
||||
|
||||
### Utilities
|
||||
|
||||
```javascript
|
||||
import { ensureBinary, clearCache, binaryInfo, checkForUpdate } from 'cloakbrowser';
|
||||
|
||||
// Pre-download binary (e.g., during Docker build)
|
||||
await ensureBinary();
|
||||
|
||||
// Check installation
|
||||
console.log(binaryInfo());
|
||||
|
||||
// Force re-download
|
||||
clearCache();
|
||||
|
||||
// Manually check for newer Chromium version
|
||||
const newVersion = await checkForUpdate();
|
||||
if (newVersion) console.log(`Updated to ${newVersion}`);
|
||||
```
|
||||
|
||||
## Test Results
|
||||
|
||||
| Detection Service | Stock Browser | CloakBrowser |
|
||||
|---|---|---|
|
||||
| **reCAPTCHA v3** | 0.1 (bot) | **0.9** (human) |
|
||||
| **Cloudflare Turnstile** | FAIL | **PASS** |
|
||||
| **FingerprintJS** | DETECTED | **PASS** |
|
||||
| **BrowserScan** | DETECTED | **NORMAL** (4/4) |
|
||||
| **bot.incolumitas.com** | 13 fails | **1 fail** |
|
||||
| `navigator.webdriver` | `true` | **`false`** |
|
||||
|
||||
## Configuration
|
||||
|
||||
| Env Variable | Default | Description |
|
||||
|---|---|---|
|
||||
| `CLOAKBROWSER_BINARY_PATH` | — | Skip download, use a local Chromium binary |
|
||||
| `CLOAKBROWSER_CACHE_DIR` | `~/.cloakbrowser` | Binary cache directory |
|
||||
| `CLOAKBROWSER_DOWNLOAD_URL` | `cloakbrowser.dev` | Custom download URL |
|
||||
| `CLOAKBROWSER_AUTO_UPDATE` | `true` | Set to `false` to disable background update checks |
|
||||
|
||||
## Migrate From Playwright
|
||||
|
||||
```diff
|
||||
- import { chromium } from 'playwright';
|
||||
- const browser = await chromium.launch();
|
||||
+ import { launch } from 'cloakbrowser';
|
||||
+ const browser = await launch();
|
||||
|
||||
const page = await browser.newPage();
|
||||
// ... rest of your code works unchanged
|
||||
```
|
||||
|
||||
## Platforms
|
||||
|
||||
| Platform | Status |
|
||||
|---|---|
|
||||
| Linux x86_64 | ✅ Available |
|
||||
| macOS arm64 (Apple Silicon) | ✅ Available |
|
||||
| macOS x86_64 (Intel) | ✅ Available |
|
||||
| Windows | Planned |
|
||||
|
||||
**On Windows?** You can still use CloakBrowser via Docker or with your own Chromium binary by setting `CLOAKBROWSER_BINARY_PATH=/path/to/chrome`.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Node.js >= 18
|
||||
- One of: `playwright-core` >= 1.40 or `puppeteer-core` >= 21
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**reCAPTCHA v3 scores are low (0.1–0.3)**
|
||||
|
||||
Avoid `page.waitForTimeout()` — it sends CDP protocol commands that reCAPTCHA detects. Use native sleep instead:
|
||||
|
||||
```javascript
|
||||
// Bad — sends CDP commands, reCAPTCHA detects this
|
||||
await page.waitForTimeout(3000);
|
||||
|
||||
// Good — invisible to the browser
|
||||
await new Promise(r => setTimeout(r, 3000));
|
||||
```
|
||||
|
||||
Other tips for maximizing reCAPTCHA scores:
|
||||
- **Use Playwright, not Puppeteer** — Puppeteer sends more CDP protocol traffic that reCAPTCHA detects ([details](#puppeteer))
|
||||
- **Use residential proxies** — datacenter IPs are flagged by IP reputation, not browser fingerprint
|
||||
- **Spend 15+ seconds on the page** before triggering reCAPTCHA — short visits score lower
|
||||
- **Space out requests** — back-to-back `grecaptcha.execute()` calls from the same session get penalized. Wait 30+ seconds between pages with reCAPTCHA
|
||||
- **Use a fixed fingerprint seed** (`--fingerprint=12345`) for consistent device identity across sessions
|
||||
- **Minimize `page.evaluate()` calls** before the reCAPTCHA check fires — each one sends CDP traffic
|
||||
|
||||
## Links
|
||||
|
||||
- 🌐 [Website](https://cloakbrowser.dev)
|
||||
- 🐛 [Bug reports & feature requests](https://github.com/CloakHQ/CloakBrowser/issues)
|
||||
- 📦 [PyPI (Python package)](https://pypi.org/project/cloakbrowser/)
|
||||
- 📖 [Full documentation](https://github.com/CloakHQ/CloakBrowser#readme)
|
||||
- 📧 Contact: cloakhq@pm.me
|
||||
|
||||
## License
|
||||
|
||||
MIT — see [LICENSE](https://github.com/CloakHQ/CloakBrowser/blob/main/LICENSE).
|
||||
@@ -0,0 +1,18 @@
|
||||
/**
|
||||
* Basic CloakBrowser example using Playwright API.
|
||||
*
|
||||
* Usage:
|
||||
* CLOAKBROWSER_BINARY_PATH=/path/to/chrome npx tsx examples/basic-playwright.ts
|
||||
*/
|
||||
|
||||
import { launch } from "../src/index.js";
|
||||
|
||||
const browser = await launch({ headless: true });
|
||||
const page = await browser.newPage();
|
||||
|
||||
await page.goto("https://example.com");
|
||||
console.log(`Title: ${await page.title()}`);
|
||||
console.log(`URL: ${page.url()}`);
|
||||
|
||||
await browser.close();
|
||||
console.log("Done.");
|
||||
@@ -0,0 +1,18 @@
|
||||
/**
|
||||
* Basic CloakBrowser example using Puppeteer API.
|
||||
*
|
||||
* Usage:
|
||||
* CLOAKBROWSER_BINARY_PATH=/path/to/chrome npx tsx examples/basic-puppeteer.ts
|
||||
*/
|
||||
|
||||
import { launch } from "../src/puppeteer.js";
|
||||
|
||||
const browser = await launch({ headless: true });
|
||||
const page = await browser.newPage();
|
||||
|
||||
await page.goto("https://example.com");
|
||||
console.log(`Title: ${await page.title()}`);
|
||||
console.log(`URL: ${page.url()}`);
|
||||
|
||||
await browser.close();
|
||||
console.log("Done.");
|
||||
@@ -0,0 +1,280 @@
|
||||
/**
|
||||
* Full stealth test suite — validates CloakBrowser against live detection services.
|
||||
* Mirrors Python examples/stealth_test.py.
|
||||
*
|
||||
* Usage:
|
||||
* CLOAKBROWSER_BINARY_PATH=/path/to/chrome npx tsx examples/stealth-test.ts
|
||||
* CLOAKBROWSER_BINARY_PATH=/path/to/chrome npx tsx examples/stealth-test.ts --proxy http://10.50.96.5:8888
|
||||
*/
|
||||
|
||||
import { launch } from "../src/index.js";
|
||||
|
||||
const PROXY = process.argv.includes("--proxy")
|
||||
? process.argv[process.argv.indexOf("--proxy") + 1]
|
||||
: undefined;
|
||||
|
||||
interface TestResult {
|
||||
name: string;
|
||||
status: "PASS" | "FAIL" | "ERROR";
|
||||
verdict: string;
|
||||
}
|
||||
|
||||
const results: TestResult[] = [];
|
||||
|
||||
console.log("=".repeat(60));
|
||||
console.log("CloakBrowser JS — Stealth Test Suite");
|
||||
console.log("=".repeat(60));
|
||||
console.log(`Proxy: ${PROXY || "none"}\n`);
|
||||
|
||||
const browser = await launch({ headless: true, proxy: PROXY });
|
||||
const page = await browser.newPage();
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Test 1: bot.sannysoft.com
|
||||
// ---------------------------------------------------------------------------
|
||||
async function testSannysoft() {
|
||||
console.log("--- bot.sannysoft.com ---");
|
||||
await page.goto("https://bot.sannysoft.com", {
|
||||
waitUntil: "networkidle",
|
||||
timeout: 30000,
|
||||
});
|
||||
await page.waitForTimeout(3000);
|
||||
|
||||
const result = await page.evaluate(() => {
|
||||
const rows = document.querySelectorAll("table tr");
|
||||
let passed = 0;
|
||||
let total = 0;
|
||||
const failed: string[] = [];
|
||||
rows.forEach((r) => {
|
||||
const cells = r.querySelectorAll("td");
|
||||
if (cells.length >= 2) {
|
||||
total++;
|
||||
const key = cells[0]!.innerText.trim();
|
||||
const cls = cells[1]!.className || "";
|
||||
if (cls.includes("failed")) {
|
||||
failed.push(key);
|
||||
} else {
|
||||
passed++;
|
||||
}
|
||||
}
|
||||
});
|
||||
return { passed, total, failed };
|
||||
});
|
||||
|
||||
const verdict =
|
||||
result.failed.length === 0
|
||||
? `${result.passed}/${result.total} — ALL GREEN`
|
||||
: `${result.passed}/${result.total} (FAILED: ${result.failed.join(", ")})`;
|
||||
const status = result.failed.length === 0 ? "PASS" : "FAIL";
|
||||
console.log(`Result: [${status}] ${verdict}\n`);
|
||||
results.push({ name: "bot.sannysoft.com", status, verdict });
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Test 2: bot.incolumitas.com
|
||||
// ---------------------------------------------------------------------------
|
||||
async function testIncolumitas() {
|
||||
console.log("--- bot.incolumitas.com ---");
|
||||
await page.goto("https://bot.incolumitas.com", {
|
||||
waitUntil: "networkidle",
|
||||
timeout: 30000,
|
||||
});
|
||||
await page.waitForTimeout(12000); // needs time for all detection tests
|
||||
|
||||
const result = await page.evaluate(() => {
|
||||
const text = document.body.innerText;
|
||||
const okMatches = text.match(/"(\w+)":\s*"OK"/g) || [];
|
||||
const failMatches = text.match(/"(\w+)":\s*"FAIL"/g) || [];
|
||||
const failedTests = failMatches.map((m) => {
|
||||
const match = m.match(/"(\w+)"/);
|
||||
return match ? match[1] : m;
|
||||
});
|
||||
return {
|
||||
passed: okMatches.length,
|
||||
failed: failMatches.length,
|
||||
failedTests,
|
||||
total: okMatches.length + failMatches.length,
|
||||
};
|
||||
});
|
||||
|
||||
const verdict =
|
||||
result.failed === 0
|
||||
? `${result.passed}/${result.total} — ALL GREEN`
|
||||
: `${result.passed}/${result.total} (FAILED: ${result.failedTests.join(", ")})`;
|
||||
// WEBDRIVER false positive is expected
|
||||
const status = result.failed <= 1 ? "PASS" : "FAIL";
|
||||
console.log(`Result: [${status}] ${verdict}\n`);
|
||||
results.push({ name: "bot.incolumitas.com", status, verdict });
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Test 3: BrowserScan
|
||||
// ---------------------------------------------------------------------------
|
||||
async function testBrowserScan() {
|
||||
console.log("--- BrowserScan ---");
|
||||
await page.goto("https://www.browserscan.net/bot-detection", {
|
||||
waitUntil: "networkidle",
|
||||
timeout: 30000,
|
||||
});
|
||||
await page.waitForTimeout(5000);
|
||||
|
||||
const result = await page.evaluate(() => {
|
||||
const text = document.body.innerText;
|
||||
const normalMatches = text.match(/Normal/g);
|
||||
const abnormalMatches = text.match(/Abnormal/g);
|
||||
return {
|
||||
normal: normalMatches ? normalMatches.length : 0,
|
||||
abnormal: abnormalMatches ? abnormalMatches.length : 0,
|
||||
};
|
||||
});
|
||||
|
||||
const verdict = `Normal: ${result.normal}, Abnormal: ${result.abnormal}`;
|
||||
const status = result.abnormal === 0 ? "PASS" : "FAIL";
|
||||
console.log(`Result: [${status}] ${verdict}\n`);
|
||||
results.push({ name: "BrowserScan", status, verdict });
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Test 4: deviceandbrowserinfo.com
|
||||
// ---------------------------------------------------------------------------
|
||||
async function testDeviceAndBrowserInfo() {
|
||||
console.log("--- deviceandbrowserinfo.com ---");
|
||||
await page.goto("https://deviceandbrowserinfo.com/are_you_a_bot", {
|
||||
waitUntil: "domcontentloaded",
|
||||
timeout: 30000,
|
||||
});
|
||||
await page.waitForTimeout(8000);
|
||||
|
||||
const result = await page.evaluate(() => {
|
||||
const text = document.body.innerText;
|
||||
const botMatch = text.match(/"isBot":\s*(true|false)/);
|
||||
const isBot = botMatch ? botMatch[1] === "true" : null;
|
||||
const checks: Record<string, boolean> = {};
|
||||
const patterns = [
|
||||
"isBot",
|
||||
"hasBotUserAgent",
|
||||
"hasWebdriverTrue",
|
||||
"isHeadlessChrome",
|
||||
"isAutomatedWithCDP",
|
||||
"hasSuspiciousWeakSignals",
|
||||
"isPlaywright",
|
||||
"hasInconsistentChromeObject",
|
||||
];
|
||||
patterns.forEach((p) => {
|
||||
const match = text.match(new RegExp('"' + p + '":\\s*(true|false)'));
|
||||
if (match) checks[p] = match[1] === "true";
|
||||
});
|
||||
return { isBot, checks };
|
||||
});
|
||||
|
||||
const trueFlags = Object.entries(result.checks)
|
||||
.filter(([, v]) => v)
|
||||
.map(([k]) => k);
|
||||
const verdict =
|
||||
`isBot: ${result.isBot}` +
|
||||
(trueFlags.length > 0 ? ` (flagged: ${trueFlags.join(", ")})` : " — all clear");
|
||||
const status = !result.isBot ? "PASS" : "FAIL";
|
||||
console.log(`Result: [${status}] ${verdict}\n`);
|
||||
results.push({ name: "deviceandbrowserinfo.com", status, verdict });
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Test 5: FingerprintJS
|
||||
// ---------------------------------------------------------------------------
|
||||
async function testFingerprintJS() {
|
||||
console.log("--- FingerprintJS ---");
|
||||
await page.goto("https://demo.fingerprint.com/web-scraping", {
|
||||
waitUntil: "networkidle",
|
||||
timeout: 30000,
|
||||
});
|
||||
await page.waitForTimeout(5000);
|
||||
|
||||
try {
|
||||
await page.click("button:has-text('Search')", { timeout: 5000 });
|
||||
await page.waitForTimeout(5000);
|
||||
} catch {
|
||||
// Search button may not be present
|
||||
}
|
||||
|
||||
const result = await page.evaluate(() => {
|
||||
const text = document.body.innerText;
|
||||
const hasFlights =
|
||||
text.includes("Price per adult") || text.includes("$");
|
||||
const isBlocked =
|
||||
text.includes("request was blocked") ||
|
||||
text.includes("bot visit detected");
|
||||
return { passed: hasFlights && !isBlocked, isBlocked, hasFlights };
|
||||
});
|
||||
|
||||
const verdict = result.passed
|
||||
? "PASSED (flights shown)"
|
||||
: result.isBlocked
|
||||
? "BLOCKED"
|
||||
: "NO FLIGHTS";
|
||||
const status = result.passed ? "PASS" : "FAIL";
|
||||
console.log(`Result: [${status}] ${verdict}\n`);
|
||||
results.push({ name: "FingerprintJS", status, verdict });
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Test 6: reCAPTCHA v3
|
||||
// ---------------------------------------------------------------------------
|
||||
async function testRecaptcha() {
|
||||
console.log("--- reCAPTCHA v3 (Google) ---");
|
||||
await page.goto(
|
||||
"https://recaptcha-demo.appspot.com/recaptcha-v3-request-scores.php",
|
||||
{ waitUntil: "networkidle", timeout: 30000 }
|
||||
);
|
||||
await page.waitForTimeout(8000);
|
||||
|
||||
const result = await page.evaluate(() => {
|
||||
const text = document.body.innerText;
|
||||
const scoreMatch = text.match(/"score":\s*(\d+\.\d+)/);
|
||||
return {
|
||||
score: scoreMatch ? parseFloat(scoreMatch[1]) : null,
|
||||
};
|
||||
});
|
||||
|
||||
const verdict = `Score: ${result.score ?? "N/A"}`;
|
||||
const status = (result.score ?? 0) >= 0.7 ? "PASS" : "FAIL";
|
||||
console.log(`Result: [${status}] ${verdict}\n`);
|
||||
results.push({ name: "reCAPTCHA v3", status, verdict });
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Run all tests
|
||||
// ---------------------------------------------------------------------------
|
||||
const tests = [
|
||||
testSannysoft,
|
||||
testIncolumitas,
|
||||
testBrowserScan,
|
||||
testDeviceAndBrowserInfo,
|
||||
testFingerprintJS,
|
||||
testRecaptcha,
|
||||
];
|
||||
|
||||
for (const test of tests) {
|
||||
try {
|
||||
await test();
|
||||
} catch (err) {
|
||||
const name = test.name.replace("test", "");
|
||||
console.log(`Error: ${err}\n`);
|
||||
results.push({ name, status: "ERROR", verdict: String(err) });
|
||||
}
|
||||
}
|
||||
|
||||
await browser.close();
|
||||
|
||||
// Summary
|
||||
console.log("=".repeat(60));
|
||||
console.log("RESULTS SUMMARY");
|
||||
console.log("=".repeat(60));
|
||||
for (const r of results) {
|
||||
const icon = { PASS: "+", FAIL: "!", ERROR: "x" }[r.status];
|
||||
console.log(` [${icon}] ${r.name}: ${r.verdict}`);
|
||||
}
|
||||
const passedCount = results.filter((r) => r.status === "PASS").length;
|
||||
console.log(`\n ${passedCount}/${results.length} tests passed`);
|
||||
console.log("=".repeat(60));
|
||||
|
||||
process.exit(passedCount === results.length ? 0 : 1);
|
||||
Generated
+2940
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,77 @@
|
||||
{
|
||||
"name": "cloakbrowser",
|
||||
"version": "0.3.0",
|
||||
"description": "Stealth Chromium that passes every bot detection test. Drop-in Playwright/Puppeteer replacement with source-level fingerprint patches.",
|
||||
"type": "module",
|
||||
"main": "dist/index.js",
|
||||
"types": "dist/index.d.ts",
|
||||
"exports": {
|
||||
".": {
|
||||
"types": "./dist/index.d.ts",
|
||||
"import": "./dist/index.js"
|
||||
},
|
||||
"./puppeteer": {
|
||||
"types": "./dist/puppeteer.d.ts",
|
||||
"import": "./dist/puppeteer.js"
|
||||
}
|
||||
},
|
||||
"files": [
|
||||
"dist"
|
||||
],
|
||||
"keywords": [
|
||||
"stealth",
|
||||
"browser",
|
||||
"chromium",
|
||||
"playwright",
|
||||
"puppeteer",
|
||||
"scraping",
|
||||
"anti-detect",
|
||||
"bot-detection",
|
||||
"fingerprint",
|
||||
"recaptcha",
|
||||
"cloudflare",
|
||||
"datadome"
|
||||
],
|
||||
"license": "MIT",
|
||||
"repository": {
|
||||
"type": "git",
|
||||
"url": "https://github.com/CloakHQ/cloakbrowser",
|
||||
"directory": "js"
|
||||
},
|
||||
"homepage": "https://github.com/CloakHQ/cloakbrowser#javascript--nodejs",
|
||||
"engines": {
|
||||
"node": ">=18.0.0"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"mmdb-lib": ">=2.0.0",
|
||||
"playwright-core": ">=1.40.0",
|
||||
"puppeteer-core": ">=21.0.0"
|
||||
},
|
||||
"peerDependenciesMeta": {
|
||||
"playwright-core": {
|
||||
"optional": true
|
||||
},
|
||||
"puppeteer-core": {
|
||||
"optional": true
|
||||
},
|
||||
"mmdb-lib": {
|
||||
"optional": true
|
||||
}
|
||||
},
|
||||
"dependencies": {
|
||||
"tar": "^7.0.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^20.10.0",
|
||||
"mmdb-lib": "^3.0.2",
|
||||
"playwright-core": "^1.40.0",
|
||||
"puppeteer-core": "^21.0.0",
|
||||
"typescript": "^5.3.0",
|
||||
"vitest": "^1.0.0"
|
||||
},
|
||||
"scripts": {
|
||||
"build": "tsc",
|
||||
"typecheck": "tsc --noEmit",
|
||||
"test": "vitest run"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,184 @@
|
||||
/**
|
||||
* Stealth configuration and platform detection for cloakbrowser.
|
||||
* Mirrors Python cloakbrowser/config.py.
|
||||
*/
|
||||
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Chromium version shipped with this release
|
||||
// ---------------------------------------------------------------------------
|
||||
export const CHROMIUM_VERSION = "145.0.7632.109";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Platform detection
|
||||
// ---------------------------------------------------------------------------
|
||||
const SUPPORTED_PLATFORMS: Record<string, string> = {
|
||||
"linux-x64": "linux-x64",
|
||||
"linux-arm64": "linux-arm64",
|
||||
"darwin-arm64": "darwin-arm64",
|
||||
"darwin-x64": "darwin-x64",
|
||||
};
|
||||
|
||||
// Platforms with pre-built binaries available for download.
|
||||
// Update this set as new platform builds are released.
|
||||
const AVAILABLE_PLATFORMS = new Set(["linux-x64", "darwin-arm64", "darwin-x64"]);
|
||||
|
||||
export function getPlatformTag(): string {
|
||||
const platform = process.platform;
|
||||
const arch = process.arch;
|
||||
|
||||
// Map Node.js platform/arch to our tag format
|
||||
let key: string;
|
||||
if (platform === "linux" && arch === "x64") key = "linux-x64";
|
||||
else if (platform === "linux" && arch === "arm64") key = "linux-arm64";
|
||||
else if (platform === "darwin" && arch === "arm64") key = "darwin-arm64";
|
||||
else if (platform === "darwin" && arch === "x64") key = "darwin-x64";
|
||||
else {
|
||||
const supported = Object.values(SUPPORTED_PLATFORMS).join(", ");
|
||||
throw new Error(
|
||||
`Unsupported platform: ${platform} ${arch}. Supported: ${supported}`
|
||||
);
|
||||
}
|
||||
|
||||
return SUPPORTED_PLATFORMS[key]!;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Binary cache paths
|
||||
// ---------------------------------------------------------------------------
|
||||
export function getCacheDir(): string {
|
||||
const custom = process.env.CLOAKBROWSER_CACHE_DIR;
|
||||
if (custom) return custom;
|
||||
return path.join(os.homedir(), ".cloakbrowser");
|
||||
}
|
||||
|
||||
export function getBinaryDir(version?: string): string {
|
||||
return path.join(getCacheDir(), `chromium-${version || CHROMIUM_VERSION}`);
|
||||
}
|
||||
|
||||
export function getBinaryPath(version?: string): string {
|
||||
const binaryDir = getBinaryDir(version);
|
||||
if (process.platform === "darwin") {
|
||||
return path.join(binaryDir, "Chromium.app", "Contents", "MacOS", "Chromium");
|
||||
}
|
||||
return path.join(binaryDir, "chrome");
|
||||
}
|
||||
|
||||
export function checkPlatformAvailable(): void {
|
||||
if (getLocalBinaryOverride()) return;
|
||||
|
||||
const tag = getPlatformTag(); // throws if unsupported entirely
|
||||
if (!AVAILABLE_PLATFORMS.has(tag)) {
|
||||
const available = [...AVAILABLE_PLATFORMS].sort().join(", ");
|
||||
throw new Error(
|
||||
`CloakBrowser — Pre-built binaries are currently only available for: ${available}.\n` +
|
||||
`Windows builds are coming soon.\n\n` +
|
||||
`To use CloakBrowser now, run in Docker (see README) or set CLOAKBROWSER_BINARY_PATH.`
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Download URL
|
||||
// ---------------------------------------------------------------------------
|
||||
export const DOWNLOAD_BASE_URL =
|
||||
process.env.CLOAKBROWSER_DOWNLOAD_URL ||
|
||||
"https://cloakbrowser.dev";
|
||||
|
||||
export const GITHUB_API_URL =
|
||||
"https://api.github.com/repos/CloakHQ/cloakbrowser/releases";
|
||||
|
||||
export const GITHUB_DOWNLOAD_BASE_URL =
|
||||
"https://github.com/CloakHQ/cloakbrowser/releases/download";
|
||||
|
||||
export function getDownloadUrl(version?: string): string {
|
||||
const v = version || CHROMIUM_VERSION;
|
||||
const tag = getPlatformTag();
|
||||
return `${DOWNLOAD_BASE_URL}/chromium-v${v}/cloakbrowser-${tag}.tar.gz`;
|
||||
}
|
||||
|
||||
export function getFallbackDownloadUrl(version?: string): string {
|
||||
const v = version || CHROMIUM_VERSION;
|
||||
const tag = getPlatformTag();
|
||||
return `${GITHUB_DOWNLOAD_BASE_URL}/chromium-v${v}/cloakbrowser-${tag}.tar.gz`;
|
||||
}
|
||||
|
||||
export function getEffectiveVersion(): string {
|
||||
const marker = path.join(getCacheDir(), "latest_version");
|
||||
try {
|
||||
if (fs.existsSync(marker)) {
|
||||
const version = fs.readFileSync(marker, "utf-8").trim();
|
||||
if (version && versionNewer(version, CHROMIUM_VERSION)) {
|
||||
const binary = getBinaryPath(version);
|
||||
if (fs.existsSync(binary)) {
|
||||
return version;
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
// Marker unreadable — fall back to hardcoded
|
||||
}
|
||||
return CHROMIUM_VERSION;
|
||||
}
|
||||
|
||||
export function parseVersion(v: string): number[] {
|
||||
return v.split(".").map(Number);
|
||||
}
|
||||
|
||||
export function versionNewer(a: string, b: string): boolean {
|
||||
const va = parseVersion(a);
|
||||
const vb = parseVersion(b);
|
||||
for (let i = 0; i < Math.max(va.length, vb.length); i++) {
|
||||
if ((va[i] ?? 0) > (vb[i] ?? 0)) return true;
|
||||
if ((va[i] ?? 0) < (vb[i] ?? 0)) return false;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Local binary override
|
||||
// ---------------------------------------------------------------------------
|
||||
export function getLocalBinaryOverride(): string | undefined {
|
||||
return process.env.CLOAKBROWSER_BINARY_PATH || undefined;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Default stealth arguments
|
||||
// ---------------------------------------------------------------------------
|
||||
// Default viewport — realistic maximized Chrome on 1080p Windows
|
||||
// screen=1920x1080, availHeight=1040 (minus 40px taskbar),
|
||||
// innerHeight=955 (minus ~85px Chrome UI: tabs + address bar + bookmarks)
|
||||
export const DEFAULT_VIEWPORT = { width: 1920, height: 955 };
|
||||
|
||||
export function getDefaultStealthArgs(): string[] {
|
||||
const seed = Math.floor(Math.random() * 90000) + 10000; // 10000-99999
|
||||
const isMac = process.platform === "darwin";
|
||||
|
||||
const base = [
|
||||
"--no-sandbox",
|
||||
"--disable-blink-features=AutomationControlled",
|
||||
`--fingerprint=${seed}`,
|
||||
];
|
||||
|
||||
if (isMac) {
|
||||
// macOS: run as native Mac browser — GPU/UA match natively
|
||||
return [...base, "--fingerprint-platform=macos"];
|
||||
}
|
||||
|
||||
// Linux: spoof as Windows
|
||||
return [
|
||||
...base,
|
||||
"--fingerprint-platform=windows",
|
||||
"--fingerprint-hardware-concurrency=8",
|
||||
"--fingerprint-device-memory=8",
|
||||
"--fingerprint-gpu-vendor=NVIDIA Corporation",
|
||||
"--fingerprint-gpu-renderer=NVIDIA GeForce RTX 3070",
|
||||
"--fingerprint-taskbar-height=40",
|
||||
"--fingerprint-screen-width=1920",
|
||||
"--fingerprint-screen-height=1080",
|
||||
"--window-size=1920,1080",
|
||||
];
|
||||
}
|
||||
@@ -0,0 +1,509 @@
|
||||
/**
|
||||
* Binary download and cache management for cloakbrowser.
|
||||
* Downloads the patched Chromium binary on first use, caches it locally.
|
||||
* Mirrors Python cloakbrowser/download.py.
|
||||
*/
|
||||
|
||||
import { execFileSync } from "node:child_process";
|
||||
import { createHash } from "node:crypto";
|
||||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
import { pipeline } from "node:stream/promises";
|
||||
import { createWriteStream } from "node:fs";
|
||||
import { extract as tarExtract } from "tar";
|
||||
|
||||
import type { BinaryInfo } from "./types.js";
|
||||
import {
|
||||
CHROMIUM_VERSION,
|
||||
DOWNLOAD_BASE_URL,
|
||||
GITHUB_API_URL,
|
||||
GITHUB_DOWNLOAD_BASE_URL,
|
||||
checkPlatformAvailable,
|
||||
getBinaryDir,
|
||||
getBinaryPath,
|
||||
getCacheDir,
|
||||
getDownloadUrl,
|
||||
getEffectiveVersion,
|
||||
getFallbackDownloadUrl,
|
||||
getLocalBinaryOverride,
|
||||
getPlatformTag,
|
||||
versionNewer,
|
||||
} from "./config.js";
|
||||
|
||||
const DOWNLOAD_TIMEOUT_MS = 600_000; // 10 minutes
|
||||
const UPDATE_CHECK_INTERVAL_MS = 3_600_000; // 1 hour
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Public API
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Ensure the stealth Chromium binary is available. Download if needed.
|
||||
* Returns the path to the chrome executable.
|
||||
*/
|
||||
export async function ensureBinary(): Promise<string> {
|
||||
// Check for local override
|
||||
const localOverride = getLocalBinaryOverride();
|
||||
if (localOverride) {
|
||||
if (!fs.existsSync(localOverride)) {
|
||||
throw new Error(
|
||||
`CLOAKBROWSER_BINARY_PATH set to '${localOverride}' but file does not exist`
|
||||
);
|
||||
}
|
||||
console.log(`[cloakbrowser] Using local binary override: ${localOverride}`);
|
||||
return localOverride;
|
||||
}
|
||||
|
||||
// Fail fast if no binary available for this platform
|
||||
checkPlatformAvailable();
|
||||
|
||||
// Check for auto-updated version first, then fall back to hardcoded
|
||||
const effective = getEffectiveVersion();
|
||||
const binaryPath = getBinaryPath(effective);
|
||||
|
||||
if (fs.existsSync(binaryPath) && isExecutable(binaryPath)) {
|
||||
maybeTriggerUpdateCheck();
|
||||
return binaryPath;
|
||||
}
|
||||
|
||||
// Fall back to hardcoded version if effective version binary doesn't exist
|
||||
if (effective !== CHROMIUM_VERSION) {
|
||||
const fallbackPath = getBinaryPath();
|
||||
if (fs.existsSync(fallbackPath) && isExecutable(fallbackPath)) {
|
||||
maybeTriggerUpdateCheck();
|
||||
return fallbackPath;
|
||||
}
|
||||
}
|
||||
|
||||
// Download hardcoded version
|
||||
console.log(
|
||||
`[cloakbrowser] Stealth Chromium ${CHROMIUM_VERSION} not found. Downloading for ${getPlatformTag()}...`
|
||||
);
|
||||
await downloadAndExtract();
|
||||
|
||||
const downloadedPath = getBinaryPath();
|
||||
if (!fs.existsSync(downloadedPath)) {
|
||||
throw new Error(
|
||||
`Download completed but binary not found at expected path: ${downloadedPath}. ` +
|
||||
`This may indicate a packaging issue. Please report at ` +
|
||||
`https://github.com/CloakHQ/cloakbrowser/issues`
|
||||
);
|
||||
}
|
||||
|
||||
maybeTriggerUpdateCheck();
|
||||
return downloadedPath;
|
||||
}
|
||||
|
||||
/** Remove all cached binaries. Forces re-download on next launch. */
|
||||
export function clearCache(): void {
|
||||
const cacheDir = getCacheDir();
|
||||
if (fs.existsSync(cacheDir)) {
|
||||
fs.rmSync(cacheDir, { recursive: true, force: true });
|
||||
console.log(`[cloakbrowser] Cache cleared: ${cacheDir}`);
|
||||
}
|
||||
}
|
||||
|
||||
/** Return info about the current binary installation. */
|
||||
export function binaryInfo(): BinaryInfo {
|
||||
const effective = getEffectiveVersion();
|
||||
const binaryPath = getBinaryPath(effective);
|
||||
return {
|
||||
version: effective,
|
||||
platform: getPlatformTag(),
|
||||
binaryPath,
|
||||
installed: fs.existsSync(binaryPath),
|
||||
cacheDir: getBinaryDir(effective),
|
||||
downloadUrl: getDownloadUrl(effective),
|
||||
};
|
||||
}
|
||||
|
||||
/** Manually check for a newer Chromium version. Returns new version or null. */
|
||||
export async function checkForUpdate(): Promise<string | null> {
|
||||
const latest = await getLatestChromiumVersion();
|
||||
if (!latest || !versionNewer(latest, CHROMIUM_VERSION)) return null;
|
||||
|
||||
const binaryDir = getBinaryDir(latest);
|
||||
if (fs.existsSync(binaryDir)) {
|
||||
writeVersionMarker(latest);
|
||||
return latest;
|
||||
}
|
||||
|
||||
console.log(`[cloakbrowser] Downloading Chromium ${latest}...`);
|
||||
await downloadAndExtract(latest);
|
||||
writeVersionMarker(latest);
|
||||
return latest;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Internal helpers
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
async function downloadAndExtract(version?: string): Promise<void> {
|
||||
const primaryUrl = getDownloadUrl(version);
|
||||
const fallbackUrl = getFallbackDownloadUrl(version);
|
||||
const binaryDir = getBinaryDir(version);
|
||||
const binaryPath = getBinaryPath(version);
|
||||
|
||||
// Create cache dir
|
||||
fs.mkdirSync(path.dirname(binaryDir), { recursive: true });
|
||||
|
||||
// Download to temp file (atomic — no partial downloads in cache)
|
||||
const tmpPath = path.join(
|
||||
path.dirname(binaryDir),
|
||||
`_download_${Date.now()}.tar.gz`
|
||||
);
|
||||
|
||||
try {
|
||||
// Try primary server, fall back to GitHub Releases (skip fallback if custom URL)
|
||||
try {
|
||||
await downloadFile(primaryUrl, tmpPath);
|
||||
} catch (primaryErr) {
|
||||
if (process.env.CLOAKBROWSER_DOWNLOAD_URL) {
|
||||
throw primaryErr;
|
||||
}
|
||||
console.warn(
|
||||
`[cloakbrowser] Primary download failed (${primaryErr instanceof Error ? primaryErr.message : primaryErr}), trying GitHub Releases...`
|
||||
);
|
||||
await downloadFile(fallbackUrl, tmpPath);
|
||||
}
|
||||
|
||||
// Verify checksum before extraction
|
||||
if (process.env.CLOAKBROWSER_SKIP_CHECKSUM?.toLowerCase() !== "true") {
|
||||
await verifyDownloadChecksum(tmpPath, version);
|
||||
}
|
||||
|
||||
await extractArchive(tmpPath, binaryDir, binaryPath);
|
||||
console.log(
|
||||
`[cloakbrowser] Visit https://cloakbrowser.dev for docs and release notifications.`
|
||||
);
|
||||
console.log(
|
||||
`[cloakbrowser] Issues? https://github.com/CloakHQ/CloakBrowser/issues`
|
||||
);
|
||||
console.log(
|
||||
`[cloakbrowser] Star us if CloakBrowser helps: https://github.com/CloakHQ/CloakBrowser`
|
||||
);
|
||||
} finally {
|
||||
// Clean up temp file
|
||||
if (fs.existsSync(tmpPath)) {
|
||||
fs.unlinkSync(tmpPath);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function verifyDownloadChecksum(filePath: string, version?: string): Promise<void> {
|
||||
const checksums = await fetchChecksums(version);
|
||||
const tarballName = `cloakbrowser-${getPlatformTag()}.tar.gz`;
|
||||
|
||||
if (!checksums) {
|
||||
console.warn("[cloakbrowser] SHA256SUMS not available for this release — skipping checksum verification");
|
||||
return;
|
||||
}
|
||||
|
||||
const expected = checksums.get(tarballName);
|
||||
if (!expected) {
|
||||
console.warn(`[cloakbrowser] SHA256SUMS found but no entry for ${tarballName} — skipping verification`);
|
||||
return;
|
||||
}
|
||||
|
||||
await verifyChecksum(filePath, expected);
|
||||
}
|
||||
|
||||
async function fetchChecksums(version?: string): Promise<Map<string, string> | null> {
|
||||
const v = version || CHROMIUM_VERSION;
|
||||
const hasCustomUrl = !!process.env.CLOAKBROWSER_DOWNLOAD_URL;
|
||||
|
||||
// Respect custom URL contract — no GitHub fallback when custom URL is set
|
||||
const urls = [`${DOWNLOAD_BASE_URL}/chromium-v${v}/SHA256SUMS`];
|
||||
if (!hasCustomUrl) {
|
||||
urls.push(`${GITHUB_DOWNLOAD_BASE_URL}/chromium-v${v}/SHA256SUMS`);
|
||||
}
|
||||
|
||||
for (const url of urls) {
|
||||
try {
|
||||
const resp = await fetch(url, {
|
||||
redirect: "follow",
|
||||
signal: AbortSignal.timeout(10_000),
|
||||
});
|
||||
if (!resp.ok) continue;
|
||||
return parseChecksums(await resp.text());
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function parseChecksums(text: string): Map<string, string> {
|
||||
const result = new Map<string, string>();
|
||||
for (const line of text.trim().split("\n")) {
|
||||
const trimmed = line.trim();
|
||||
if (!trimmed) continue;
|
||||
const match = trimmed.match(/^([a-f0-9]{64})\s+\*?(.+)$/);
|
||||
if (match) {
|
||||
result.set(match[2]!, match[1]!.toLowerCase());
|
||||
}
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
async function verifyChecksum(filePath: string, expectedHash: string): Promise<void> {
|
||||
const hash = createHash("sha256");
|
||||
const stream = fs.createReadStream(filePath);
|
||||
for await (const chunk of stream) {
|
||||
hash.update(chunk);
|
||||
}
|
||||
const actual = hash.digest("hex").toLowerCase();
|
||||
if (actual !== expectedHash) {
|
||||
throw new Error(
|
||||
`Checksum verification failed!\n` +
|
||||
` Expected: ${expectedHash}\n` +
|
||||
` Got: ${actual}\n` +
|
||||
` File may be corrupted or tampered with. ` +
|
||||
`Please retry or report at https://github.com/CloakHQ/cloakbrowser/issues`
|
||||
);
|
||||
}
|
||||
console.log("[cloakbrowser] Checksum verified: SHA-256 OK");
|
||||
}
|
||||
|
||||
async function downloadFile(url: string, dest: string): Promise<void> {
|
||||
console.log(`[cloakbrowser] Downloading from ${url}`);
|
||||
|
||||
const controller = new AbortController();
|
||||
const timeout = setTimeout(() => controller.abort(), DOWNLOAD_TIMEOUT_MS);
|
||||
|
||||
try {
|
||||
const response = await fetch(url, {
|
||||
signal: controller.signal,
|
||||
redirect: "follow",
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
throw new Error(`Download failed: HTTP ${response.status} ${response.statusText}`);
|
||||
}
|
||||
|
||||
if (!response.body) {
|
||||
throw new Error("Download failed: empty response body");
|
||||
}
|
||||
|
||||
const total = Number(response.headers.get("content-length") || 0);
|
||||
let downloaded = 0;
|
||||
let lastLoggedPct = -1;
|
||||
|
||||
const fileStream = createWriteStream(dest);
|
||||
const reader = response.body.getReader();
|
||||
|
||||
// Stream chunks to file with progress logging
|
||||
while (true) {
|
||||
const { done, value } = await reader.read();
|
||||
if (done) break;
|
||||
|
||||
fileStream.write(value);
|
||||
downloaded += value.length;
|
||||
|
||||
if (total > 0) {
|
||||
const pct = Math.floor((downloaded / total) * 100);
|
||||
if (pct >= lastLoggedPct + 10) {
|
||||
lastLoggedPct = pct;
|
||||
const dlMB = Math.floor(downloaded / (1024 * 1024));
|
||||
const totalMB = Math.floor(total / (1024 * 1024));
|
||||
console.log(
|
||||
`[cloakbrowser] Download progress: ${pct}% (${dlMB}/${totalMB} MB)`
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Wait for file stream to finish
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
fileStream.end(() => resolve());
|
||||
fileStream.on("error", reject);
|
||||
});
|
||||
|
||||
const sizeMB = Math.floor(fs.statSync(dest).size / (1024 * 1024));
|
||||
console.log(`[cloakbrowser] Download complete: ${sizeMB} MB`);
|
||||
} finally {
|
||||
clearTimeout(timeout);
|
||||
}
|
||||
}
|
||||
|
||||
async function extractArchive(
|
||||
archivePath: string,
|
||||
destDir: string,
|
||||
binaryPath?: string
|
||||
): Promise<void> {
|
||||
console.log(`[cloakbrowser] Extracting to ${destDir}`);
|
||||
|
||||
// Clean existing dir if partial download existed
|
||||
if (fs.existsSync(destDir)) {
|
||||
fs.rmSync(destDir, { recursive: true, force: true });
|
||||
}
|
||||
fs.mkdirSync(destDir, { recursive: true });
|
||||
|
||||
// Extract with tar — the 'tar' package handles symlink/traversal safety
|
||||
await tarExtract({
|
||||
file: archivePath,
|
||||
cwd: destDir,
|
||||
// Security: strip leading path components and reject absolute paths
|
||||
strip: 0,
|
||||
filter: (entryPath: string) => {
|
||||
// Reject absolute paths and path traversal
|
||||
if (path.isAbsolute(entryPath) || entryPath.includes("..")) {
|
||||
console.warn(
|
||||
`[cloakbrowser] Skipping suspicious archive entry: ${entryPath}`
|
||||
);
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
},
|
||||
});
|
||||
|
||||
// Flatten single subdirectory if needed
|
||||
flattenSingleSubdir(destDir);
|
||||
|
||||
// Make binary executable
|
||||
const bp = binaryPath || getBinaryPath();
|
||||
if (fs.existsSync(bp)) {
|
||||
fs.chmodSync(bp, 0o755);
|
||||
}
|
||||
|
||||
// macOS: remove quarantine/provenance xattrs to prevent Gatekeeper prompts
|
||||
if (process.platform === "darwin") {
|
||||
removeQuarantine(destDir);
|
||||
}
|
||||
|
||||
if (fs.existsSync(bp)) {
|
||||
console.log(`[cloakbrowser] Binary ready: ${bp}`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* If extraction created a single subdirectory, move its contents up.
|
||||
* Many tarballs wrap files in a top-level directory.
|
||||
*/
|
||||
function flattenSingleSubdir(destDir: string): void {
|
||||
const entries = fs.readdirSync(destDir);
|
||||
if (entries.length === 1) {
|
||||
const subdir = path.join(destDir, entries[0]!);
|
||||
// Never flatten .app bundles — macOS needs the bundle structure
|
||||
if (entries[0]!.endsWith(".app")) return;
|
||||
if (fs.statSync(subdir).isDirectory()) {
|
||||
const children = fs.readdirSync(subdir);
|
||||
for (const child of children) {
|
||||
fs.renameSync(
|
||||
path.join(subdir, child),
|
||||
path.join(destDir, child)
|
||||
);
|
||||
}
|
||||
fs.rmdirSync(subdir);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** Remove macOS quarantine/provenance xattrs so Gatekeeper doesn't block the binary. */
|
||||
function removeQuarantine(dirPath: string): void {
|
||||
try {
|
||||
execFileSync("xattr", ["-cr", dirPath], { timeout: 30_000 });
|
||||
} catch {
|
||||
// Non-fatal — user can manually run: xattr -cr ~/.cloakbrowser/
|
||||
}
|
||||
}
|
||||
|
||||
function isExecutable(filePath: string): boolean {
|
||||
try {
|
||||
fs.accessSync(filePath, fs.constants.X_OK);
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Auto-update
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function shouldCheckForUpdate(): boolean {
|
||||
if (process.env.CLOAKBROWSER_AUTO_UPDATE?.toLowerCase() === "false")
|
||||
return false;
|
||||
if (getLocalBinaryOverride()) return false;
|
||||
if (process.env.CLOAKBROWSER_DOWNLOAD_URL) return false;
|
||||
|
||||
const checkFile = path.join(getCacheDir(), ".last_update_check");
|
||||
try {
|
||||
const lastCheck = Number(fs.readFileSync(checkFile, "utf-8").trim());
|
||||
if (Date.now() - lastCheck < UPDATE_CHECK_INTERVAL_MS) return false;
|
||||
} catch {
|
||||
/* file doesn't exist or unreadable */
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
async function getLatestChromiumVersion(): Promise<string | null> {
|
||||
try {
|
||||
const resp = await fetch(`${GITHUB_API_URL}?per_page=10`, {
|
||||
signal: AbortSignal.timeout(10_000),
|
||||
});
|
||||
if (!resp.ok) return null;
|
||||
const releases = (await resp.json()) as Array<{
|
||||
tag_name: string;
|
||||
draft: boolean;
|
||||
}>;
|
||||
for (const release of releases) {
|
||||
if (release.tag_name.startsWith("chromium-v") && !release.draft) {
|
||||
return release.tag_name.replace("chromium-v", "");
|
||||
}
|
||||
}
|
||||
return null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function writeVersionMarker(version: string): void {
|
||||
const cacheDir = getCacheDir();
|
||||
fs.mkdirSync(cacheDir, { recursive: true });
|
||||
const marker = path.join(cacheDir, "latest_version");
|
||||
const tmp = `${marker}.tmp`;
|
||||
fs.writeFileSync(tmp, version);
|
||||
fs.renameSync(tmp, marker);
|
||||
}
|
||||
|
||||
async function checkAndDownloadUpdate(): Promise<void> {
|
||||
try {
|
||||
// Record check timestamp first (rate limiting)
|
||||
const cacheDir = getCacheDir();
|
||||
fs.mkdirSync(cacheDir, { recursive: true });
|
||||
fs.writeFileSync(
|
||||
path.join(cacheDir, ".last_update_check"),
|
||||
String(Date.now())
|
||||
);
|
||||
|
||||
const latest = await getLatestChromiumVersion();
|
||||
if (!latest || !versionNewer(latest, CHROMIUM_VERSION)) return;
|
||||
|
||||
// Already downloaded?
|
||||
if (fs.existsSync(getBinaryDir(latest))) {
|
||||
writeVersionMarker(latest);
|
||||
return;
|
||||
}
|
||||
|
||||
console.log(
|
||||
`[cloakbrowser] Newer Chromium available: ${latest} (current: ${CHROMIUM_VERSION}). Downloading in background...`
|
||||
);
|
||||
await downloadAndExtract(latest);
|
||||
writeVersionMarker(latest);
|
||||
console.log(
|
||||
`[cloakbrowser] Background update complete: Chromium ${latest} ready. Will use on next launch.`
|
||||
);
|
||||
} catch (err) {
|
||||
// Background update failed — don't disrupt the user
|
||||
if (process.env.DEBUG) {
|
||||
console.error("[cloakbrowser] Background update failed:", err);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function maybeTriggerUpdateCheck(): void {
|
||||
if (!shouldCheckForUpdate()) return;
|
||||
// Fire-and-forget — don't await
|
||||
checkAndDownloadUpdate().catch(() => {});
|
||||
}
|
||||
+262
@@ -0,0 +1,262 @@
|
||||
/**
|
||||
* GeoIP-based timezone and locale detection from proxy IP.
|
||||
*
|
||||
* Optional feature — requires `mmdb-lib` package:
|
||||
* npm install mmdb-lib
|
||||
*
|
||||
* Downloads GeoLite2-City.mmdb (~70 MB) on first use,
|
||||
* caches in `~/.cloakbrowser/geoip/`.
|
||||
*/
|
||||
|
||||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
import { createWriteStream } from "node:fs";
|
||||
import dns from "node:dns/promises";
|
||||
import net from "node:net";
|
||||
import { getCacheDir } from "./config.js";
|
||||
|
||||
// P3TERX mirror of MaxMind GeoLite2-City — no license key needed
|
||||
const GEOIP_DB_URL =
|
||||
"https://github.com/P3TERX/GeoLite.mmdb/raw/download/GeoLite2-City.mmdb";
|
||||
const GEOIP_DB_FILENAME = "GeoLite2-City.mmdb";
|
||||
const GEOIP_UPDATE_INTERVAL_MS = 30 * 86_400_000; // 30 days
|
||||
|
||||
/** Country ISO code → BCP 47 locale (covers ~90% of proxy traffic). */
|
||||
export const COUNTRY_LOCALE_MAP: Record<string, string> = {
|
||||
US: "en-US", GB: "en-GB", AU: "en-AU", CA: "en-CA", NZ: "en-NZ",
|
||||
IE: "en-IE", ZA: "en-ZA", SG: "en-SG",
|
||||
DE: "de-DE", AT: "de-AT", CH: "de-CH",
|
||||
FR: "fr-FR", BE: "fr-BE",
|
||||
ES: "es-ES", MX: "es-MX", AR: "es-AR", CO: "es-CO", CL: "es-CL",
|
||||
BR: "pt-BR", PT: "pt-PT",
|
||||
IT: "it-IT", NL: "nl-NL",
|
||||
JP: "ja-JP", KR: "ko-KR", CN: "zh-CN", TW: "zh-TW", HK: "zh-HK",
|
||||
RU: "ru-RU", UA: "uk-UA", PL: "pl-PL", CZ: "cs-CZ", RO: "ro-RO",
|
||||
IL: "he-IL", TR: "tr-TR", SA: "ar-SA", AE: "ar-AE", EG: "ar-EG",
|
||||
IN: "hi-IN", ID: "id-ID", PH: "en-PH",
|
||||
TH: "th-TH", VN: "vi-VN", MY: "ms-MY",
|
||||
SE: "sv-SE", NO: "nb-NO", DK: "da-DK", FI: "fi-FI",
|
||||
GR: "el-GR", HU: "hu-HU", BG: "bg-BG",
|
||||
};
|
||||
|
||||
export interface GeoResult {
|
||||
timezone: string | null;
|
||||
locale: string | null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve timezone and locale from a proxy's IP address.
|
||||
* Returns `{ timezone, locale }` — either may be null on failure.
|
||||
* Never throws.
|
||||
*/
|
||||
export async function resolveProxyGeo(
|
||||
proxyUrl: string
|
||||
): Promise<GeoResult> {
|
||||
let Reader: any;
|
||||
try {
|
||||
const mmdb = await import("mmdb-lib");
|
||||
Reader = mmdb.default?.Reader ?? mmdb.Reader;
|
||||
} catch {
|
||||
throw new Error(
|
||||
"mmdb-lib is required for geoip: true. Install it with:\n npm install mmdb-lib"
|
||||
);
|
||||
}
|
||||
|
||||
const dbPath = await ensureGeoipDb();
|
||||
if (!dbPath) return { timezone: null, locale: null };
|
||||
|
||||
// Exit IP (through proxy) is most accurate — gateway DNS may differ from exit
|
||||
let ip = await resolveExitIp(proxyUrl);
|
||||
if (!ip) ip = await resolveProxyIp(proxyUrl);
|
||||
if (!ip) return { timezone: null, locale: null };
|
||||
|
||||
try {
|
||||
const buf = fs.readFileSync(dbPath);
|
||||
const reader = new Reader(buf);
|
||||
const result = reader.get(ip) as any;
|
||||
const timezone: string | null = result?.location?.time_zone ?? null;
|
||||
const countryCode: string | null = result?.country?.iso_code ?? null;
|
||||
const locale =
|
||||
countryCode ? (COUNTRY_LOCALE_MAP[countryCode] ?? null) : null;
|
||||
return { timezone, locale };
|
||||
} catch {
|
||||
return { timezone: null, locale: null };
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Proxy IP resolution
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** @internal Exported for testing. */
|
||||
export async function resolveProxyIp(
|
||||
proxyUrl: string
|
||||
): Promise<string | null> {
|
||||
try {
|
||||
const url = new URL(proxyUrl);
|
||||
const hostname = url.hostname;
|
||||
if (!hostname) return null;
|
||||
|
||||
// Already a literal IP?
|
||||
if (net.isIP(hostname)) return hostname;
|
||||
|
||||
// DNS resolve
|
||||
const { address } = await dns.lookup(hostname);
|
||||
return address;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function isPrivateIp(ip: string): boolean {
|
||||
// Quick check for common private ranges
|
||||
if (ip.startsWith("10.") || ip.startsWith("127.") || ip === "::1") return true;
|
||||
if (ip.startsWith("172.")) {
|
||||
const second = parseInt(ip.split(".")[1], 10);
|
||||
if (second >= 16 && second <= 31) return true;
|
||||
}
|
||||
if (ip.startsWith("192.168.")) return true;
|
||||
return false;
|
||||
}
|
||||
|
||||
const IP_ECHO_URLS = [
|
||||
"https://api.ipify.org",
|
||||
"https://checkip.amazonaws.com",
|
||||
"https://ifconfig.me/ip",
|
||||
];
|
||||
|
||||
async function resolveExitIp(proxyUrl: string): Promise<string | null> {
|
||||
// Node.js fetch doesn't support proxy natively — use a CONNECT tunnel via http
|
||||
// For simplicity, use a direct HTTP request to a plain-text IP echo service
|
||||
// through the proxy using Node's http module
|
||||
try {
|
||||
const { default: http } = await import("node:http");
|
||||
const { default: https } = await import("node:https");
|
||||
const proxyUrlObj = new URL(proxyUrl);
|
||||
|
||||
for (const echoUrl of IP_ECHO_URLS) {
|
||||
try {
|
||||
const ip = await new Promise<string | null>((resolve, reject) => {
|
||||
const targetUrl = new URL(echoUrl);
|
||||
const connectReq = http.request({
|
||||
host: proxyUrlObj.hostname,
|
||||
port: parseInt(proxyUrlObj.port || "80", 10),
|
||||
method: "CONNECT",
|
||||
path: `${targetUrl.hostname}:443`,
|
||||
headers: proxyUrlObj.username
|
||||
? {
|
||||
"Proxy-Authorization":
|
||||
"Basic " +
|
||||
Buffer.from(
|
||||
`${decodeURIComponent(proxyUrlObj.username)}:${decodeURIComponent(proxyUrlObj.password || "")}`
|
||||
).toString("base64"),
|
||||
}
|
||||
: {},
|
||||
timeout: 10_000,
|
||||
});
|
||||
|
||||
connectReq.on("connect", (_res, socket) => {
|
||||
const req = https.request(
|
||||
echoUrl,
|
||||
{ socket, timeout: 5_000 } as any,
|
||||
(res) => {
|
||||
let data = "";
|
||||
res.on("data", (chunk: Buffer) => (data += chunk.toString()));
|
||||
res.on("end", () => {
|
||||
const ip = data.trim();
|
||||
resolve(net.isIP(ip) ? ip : null);
|
||||
});
|
||||
}
|
||||
);
|
||||
req.on("error", () => resolve(null));
|
||||
req.end();
|
||||
});
|
||||
|
||||
connectReq.on("error", () => resolve(null));
|
||||
connectReq.on("timeout", () => {
|
||||
connectReq.destroy();
|
||||
resolve(null);
|
||||
});
|
||||
connectReq.end();
|
||||
});
|
||||
|
||||
if (ip) return ip;
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
// Fallback: couldn't import http modules
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// GeoIP database management
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function getGeoipDir(): string {
|
||||
return path.join(getCacheDir(), "geoip");
|
||||
}
|
||||
|
||||
async function ensureGeoipDb(): Promise<string | null> {
|
||||
const dir = getGeoipDir();
|
||||
const dbPath = path.join(dir, GEOIP_DB_FILENAME);
|
||||
|
||||
if (fs.existsSync(dbPath)) {
|
||||
maybeTriggerUpdate(dbPath);
|
||||
return dbPath;
|
||||
}
|
||||
|
||||
try {
|
||||
await downloadGeoipDb(dbPath);
|
||||
return dbPath;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
async function downloadGeoipDb(dest: string): Promise<void> {
|
||||
const dir = path.dirname(dest);
|
||||
fs.mkdirSync(dir, { recursive: true });
|
||||
console.log("[cloakbrowser] Downloading GeoIP database (~70 MB)…");
|
||||
|
||||
const tmpPath = `${dest}.tmp.${Date.now()}`;
|
||||
try {
|
||||
const response = await fetch(GEOIP_DB_URL, { redirect: "follow" });
|
||||
if (!response.ok || !response.body) {
|
||||
throw new Error(`HTTP ${response.status}`);
|
||||
}
|
||||
|
||||
const fileStream = createWriteStream(tmpPath);
|
||||
const reader = response.body.getReader();
|
||||
|
||||
for (;;) {
|
||||
const { done, value } = await reader.read();
|
||||
if (done) break;
|
||||
fileStream.write(value);
|
||||
}
|
||||
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
fileStream.end(() => resolve());
|
||||
fileStream.on("error", reject);
|
||||
});
|
||||
|
||||
fs.renameSync(tmpPath, dest);
|
||||
console.log(`[cloakbrowser] GeoIP database ready: ${dest}`);
|
||||
} catch (err) {
|
||||
if (fs.existsSync(tmpPath)) fs.unlinkSync(tmpPath);
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
function maybeTriggerUpdate(dbPath: string): void {
|
||||
try {
|
||||
const age = Date.now() - fs.statSync(dbPath).mtimeMs;
|
||||
if (age < GEOIP_UPDATE_INTERVAL_MS) return;
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
// Fire-and-forget background update
|
||||
downloadGeoipDb(dbPath).catch(() => {});
|
||||
}
|
||||
@@ -0,0 +1,28 @@
|
||||
/**
|
||||
* CloakBrowser — Stealth Chromium for Node.js
|
||||
*
|
||||
* Default export uses Playwright. For Puppeteer, import from 'cloakbrowser/puppeteer'.
|
||||
*
|
||||
* @example
|
||||
* ```ts
|
||||
* // Playwright (default)
|
||||
* import { launch } from 'cloakbrowser';
|
||||
* const browser = await launch();
|
||||
*
|
||||
* // Puppeteer
|
||||
* import { launch } from 'cloakbrowser/puppeteer';
|
||||
* const browser = await launch();
|
||||
* ```
|
||||
*/
|
||||
|
||||
// Launch functions (Playwright API)
|
||||
export { launch, launchContext } from "./playwright.js";
|
||||
|
||||
// Binary management
|
||||
export { ensureBinary, clearCache, binaryInfo, checkForUpdate } from "./download.js";
|
||||
|
||||
// Config
|
||||
export { CHROMIUM_VERSION, getDefaultStealthArgs } from "./config.js";
|
||||
|
||||
// Types
|
||||
export type { LaunchOptions, LaunchContextOptions, BinaryInfo } from "./types.js";
|
||||
@@ -0,0 +1,130 @@
|
||||
/**
|
||||
* Playwright launch wrapper for cloakbrowser.
|
||||
* Mirrors Python cloakbrowser/browser.py.
|
||||
*/
|
||||
|
||||
import type { Browser, BrowserContext } from "playwright-core";
|
||||
import type { LaunchOptions, LaunchContextOptions } from "./types.js";
|
||||
import { DEFAULT_VIEWPORT, getDefaultStealthArgs } from "./config.js";
|
||||
import { ensureBinary } from "./download.js";
|
||||
import { parseProxyUrl } from "./proxy.js";
|
||||
|
||||
/**
|
||||
* Launch stealth Chromium browser via Playwright.
|
||||
*
|
||||
* @example
|
||||
* ```ts
|
||||
* import { launch } from 'cloakbrowser';
|
||||
* const browser = await launch();
|
||||
* const page = await browser.newPage();
|
||||
* await page.goto('https://bot.incolumitas.com');
|
||||
* console.log(await page.title());
|
||||
* await browser.close();
|
||||
* ```
|
||||
*/
|
||||
export async function launch(options: LaunchOptions = {}): Promise<Browser> {
|
||||
const { chromium } = await import("playwright-core");
|
||||
|
||||
const binaryPath = process.env.CLOAKBROWSER_BINARY_PATH || (await ensureBinary());
|
||||
const resolved = await maybeResolveGeoip(options);
|
||||
const args = buildArgs({ ...options, ...resolved });
|
||||
|
||||
const browser = await chromium.launch({
|
||||
executablePath: binaryPath,
|
||||
headless: options.headless ?? true,
|
||||
args,
|
||||
ignoreDefaultArgs: ["--enable-automation"],
|
||||
...(options.proxy ? { proxy: parseProxyUrl(options.proxy) } : {}),
|
||||
...options.launchOptions,
|
||||
});
|
||||
|
||||
return browser;
|
||||
}
|
||||
|
||||
/**
|
||||
* Launch stealth browser and return a BrowserContext with common options pre-set.
|
||||
* Closing the context also closes the browser.
|
||||
*
|
||||
* @example
|
||||
* ```ts
|
||||
* import { launchContext } from 'cloakbrowser';
|
||||
* const context = await launchContext({
|
||||
* userAgent: 'Mozilla/5.0...',
|
||||
* viewport: { width: 1920, height: 1080 },
|
||||
* });
|
||||
* const page = await context.newPage();
|
||||
* await page.goto('https://example.com');
|
||||
* await context.close(); // also closes browser
|
||||
* ```
|
||||
*/
|
||||
export async function launchContext(
|
||||
options: LaunchContextOptions = {}
|
||||
): Promise<BrowserContext> {
|
||||
// Resolve geoip BEFORE launch() to avoid double-resolution
|
||||
const resolved = await maybeResolveGeoip(options);
|
||||
const browser = await launch({ ...options, ...resolved, geoip: false });
|
||||
|
||||
let context: BrowserContext;
|
||||
try {
|
||||
context = await browser.newContext({
|
||||
...(options.userAgent ? { userAgent: options.userAgent } : {}),
|
||||
viewport: options.viewport ?? DEFAULT_VIEWPORT,
|
||||
...(resolved.locale ? { locale: resolved.locale } : {}),
|
||||
...(resolved.timezone ? { timezoneId: resolved.timezone } : {}),
|
||||
...(options.colorScheme ? { colorScheme: options.colorScheme } : {}),
|
||||
});
|
||||
} catch (err) {
|
||||
await browser.close();
|
||||
throw err;
|
||||
}
|
||||
|
||||
// Patch close() to also close the browser
|
||||
const origClose = context.close.bind(context);
|
||||
context.close = async () => {
|
||||
await origClose();
|
||||
await browser.close();
|
||||
};
|
||||
|
||||
return context;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Internal
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
async function maybeResolveGeoip(
|
||||
options: LaunchOptions
|
||||
): Promise<{ timezone?: string; locale?: string }> {
|
||||
if (!options.geoip || !options.proxy) return { timezone: options.timezone, locale: options.locale };
|
||||
if (options.timezone && options.locale) return { timezone: options.timezone, locale: options.locale };
|
||||
|
||||
const { resolveProxyGeo } = await import("./geoip.js");
|
||||
const { timezone: geoTz, locale: geoLocale } = await resolveProxyGeo(options.proxy);
|
||||
return {
|
||||
timezone: options.timezone ?? geoTz ?? undefined,
|
||||
locale: options.locale ?? geoLocale ?? undefined,
|
||||
};
|
||||
}
|
||||
|
||||
/** @internal Exposed for unit tests only. */
|
||||
export function _buildArgsForTest(options: LaunchOptions): string[] {
|
||||
return buildArgs(options);
|
||||
}
|
||||
|
||||
function buildArgs(options: LaunchOptions): string[] {
|
||||
const args: string[] = [];
|
||||
if (options.stealthArgs !== false) {
|
||||
args.push(...getDefaultStealthArgs());
|
||||
}
|
||||
if (options.args) {
|
||||
args.push(...options.args);
|
||||
}
|
||||
// Timezone/locale flags — always inject when set
|
||||
if (options.timezone) {
|
||||
args.push(`--timezone=${options.timezone}`);
|
||||
}
|
||||
if (options.locale) {
|
||||
args.push(`--lang=${options.locale}`);
|
||||
}
|
||||
return args;
|
||||
}
|
||||
@@ -0,0 +1,42 @@
|
||||
/**
|
||||
* Shared proxy URL parsing for Playwright and Puppeteer wrappers.
|
||||
*/
|
||||
|
||||
export interface ParsedProxy {
|
||||
server: string;
|
||||
username?: string;
|
||||
password?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse a proxy URL, extracting credentials into separate fields.
|
||||
*
|
||||
* Handles: "http://user:pass@host:port" -> { server: "http://host:port", username: "user", password: "pass" }
|
||||
* Also handles: no credentials, URL-encoded special chars, socks5://, missing port.
|
||||
*/
|
||||
export function parseProxyUrl(proxy: string): ParsedProxy {
|
||||
let url: URL;
|
||||
try {
|
||||
url = new URL(proxy);
|
||||
} catch {
|
||||
// Not a parseable URL (e.g. bare "host:port") — pass through as-is
|
||||
return { server: proxy };
|
||||
}
|
||||
|
||||
if (!url.username) {
|
||||
return { server: proxy };
|
||||
}
|
||||
|
||||
// Rebuild server URL without credentials
|
||||
const server = `${url.protocol}//${url.hostname}${url.port ? `:${url.port}` : ""}`;
|
||||
|
||||
const result: ParsedProxy = {
|
||||
server,
|
||||
username: decodeURIComponent(url.username),
|
||||
};
|
||||
if (url.password) {
|
||||
result.password = decodeURIComponent(url.password);
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
@@ -0,0 +1,99 @@
|
||||
/**
|
||||
* Puppeteer launch wrapper for cloakbrowser.
|
||||
* Alternative to the Playwright wrapper for users who prefer Puppeteer.
|
||||
*/
|
||||
|
||||
import type { Browser } from "puppeteer-core";
|
||||
import type { LaunchOptions } from "./types.js";
|
||||
import { getDefaultStealthArgs } from "./config.js";
|
||||
import { ensureBinary } from "./download.js";
|
||||
import { parseProxyUrl } from "./proxy.js";
|
||||
|
||||
/**
|
||||
* Launch stealth Chromium browser via Puppeteer.
|
||||
*
|
||||
* @example
|
||||
* ```ts
|
||||
* import { launch } from 'cloakbrowser/puppeteer';
|
||||
* const browser = await launch();
|
||||
* const page = await browser.newPage();
|
||||
* await page.goto('https://bot.incolumitas.com');
|
||||
* console.log(await page.title());
|
||||
* await browser.close();
|
||||
* ```
|
||||
*/
|
||||
export async function launch(options: LaunchOptions = {}): Promise<Browser> {
|
||||
const puppeteer = await import("puppeteer-core");
|
||||
|
||||
const binaryPath = process.env.CLOAKBROWSER_BINARY_PATH || (await ensureBinary());
|
||||
const resolved = await maybeResolveGeoip(options);
|
||||
const args = buildArgs({ ...options, ...resolved });
|
||||
|
||||
// Puppeteer handles proxy via CLI args, not a separate option.
|
||||
// Chromium's --proxy-server does NOT support inline credentials,
|
||||
// so we strip them and use page.authenticate() instead.
|
||||
let proxyAuth: { username: string; password: string } | undefined;
|
||||
if (options.proxy) {
|
||||
const { server, username, password } = parseProxyUrl(options.proxy);
|
||||
args.push(`--proxy-server=${server}`);
|
||||
if (username) {
|
||||
proxyAuth = { username, password: password || "" };
|
||||
}
|
||||
}
|
||||
|
||||
const browser = await puppeteer.default.launch({
|
||||
executablePath: binaryPath,
|
||||
headless: options.headless ?? true,
|
||||
args,
|
||||
ignoreDefaultArgs: ["--enable-automation"],
|
||||
...options.launchOptions,
|
||||
});
|
||||
|
||||
// Monkey-patch newPage() to auto-authenticate proxy credentials
|
||||
if (proxyAuth) {
|
||||
const origNewPage = browser.newPage.bind(browser);
|
||||
const auth = proxyAuth;
|
||||
browser.newPage = async (...pageArgs: Parameters<typeof origNewPage>) => {
|
||||
const page = await origNewPage(...pageArgs);
|
||||
await page.authenticate(auth);
|
||||
return page;
|
||||
};
|
||||
}
|
||||
|
||||
return browser;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Internal
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
async function maybeResolveGeoip(
|
||||
options: LaunchOptions
|
||||
): Promise<{ timezone?: string; locale?: string }> {
|
||||
if (!options.geoip || !options.proxy) return { timezone: options.timezone, locale: options.locale };
|
||||
if (options.timezone && options.locale) return { timezone: options.timezone, locale: options.locale };
|
||||
|
||||
const { resolveProxyGeo } = await import("./geoip.js");
|
||||
const { timezone: geoTz, locale: geoLocale } = await resolveProxyGeo(options.proxy);
|
||||
return {
|
||||
timezone: options.timezone ?? geoTz ?? undefined,
|
||||
locale: options.locale ?? geoLocale ?? undefined,
|
||||
};
|
||||
}
|
||||
|
||||
function buildArgs(options: LaunchOptions): string[] {
|
||||
const args: string[] = [];
|
||||
if (options.stealthArgs !== false) {
|
||||
args.push(...getDefaultStealthArgs());
|
||||
}
|
||||
if (options.args) {
|
||||
args.push(...options.args);
|
||||
}
|
||||
if (options.timezone) {
|
||||
args.push(`--timezone=${options.timezone}`);
|
||||
}
|
||||
if (options.locale) {
|
||||
args.push(`--lang=${options.locale}`);
|
||||
}
|
||||
return args;
|
||||
}
|
||||
@@ -0,0 +1,44 @@
|
||||
/**
|
||||
* Shared types for cloakbrowser launch wrappers.
|
||||
*/
|
||||
|
||||
export interface LaunchOptions {
|
||||
/** Run in headless mode (default: true). */
|
||||
headless?: boolean;
|
||||
/** Proxy server URL, e.g. 'http://proxy:8080' or 'socks5://proxy:1080'. */
|
||||
proxy?: string;
|
||||
/** Additional Chromium CLI arguments. */
|
||||
args?: string[];
|
||||
/** Include default stealth fingerprint args (default: true). Set false to use custom --fingerprint flags. */
|
||||
stealthArgs?: boolean;
|
||||
/** IANA timezone, e.g. "America/New_York". Sets --timezone binary flag. */
|
||||
timezone?: string;
|
||||
/** BCP 47 locale, e.g. "en-US". Sets --lang binary flag. */
|
||||
locale?: string;
|
||||
/** Auto-detect timezone/locale from proxy IP (requires: npm install mmdb-lib). */
|
||||
geoip?: boolean;
|
||||
/** Raw options passed directly to playwright/puppeteer launch(). */
|
||||
launchOptions?: Record<string, unknown>;
|
||||
}
|
||||
|
||||
export interface LaunchContextOptions extends LaunchOptions {
|
||||
/** Custom user agent string. */
|
||||
userAgent?: string;
|
||||
/** Viewport size. */
|
||||
viewport?: { width: number; height: number };
|
||||
/** Browser locale, e.g. "en-US". */
|
||||
locale?: string;
|
||||
/** Timezone, e.g. "America/New_York". */
|
||||
timezoneId?: string;
|
||||
/** Color scheme preference — 'light', 'dark', or 'no-preference'. */
|
||||
colorScheme?: "light" | "dark" | "no-preference";
|
||||
}
|
||||
|
||||
export interface BinaryInfo {
|
||||
version: string;
|
||||
platform: string;
|
||||
binaryPath: string;
|
||||
installed: boolean;
|
||||
cacheDir: string;
|
||||
downloadUrl: string;
|
||||
}
|
||||
@@ -0,0 +1,99 @@
|
||||
import { describe, it, expect } from "vitest";
|
||||
import {
|
||||
CHROMIUM_VERSION,
|
||||
getDefaultStealthArgs,
|
||||
getCacheDir,
|
||||
getBinaryDir,
|
||||
getDownloadUrl,
|
||||
} from "../src/config.js";
|
||||
import { _buildArgsForTest } from "../src/playwright.js";
|
||||
|
||||
describe("config", () => {
|
||||
it("CHROMIUM_VERSION matches expected format", () => {
|
||||
expect(CHROMIUM_VERSION).toMatch(/^\d+\.\d+\.\d+\.\d+$/);
|
||||
});
|
||||
|
||||
it("getDefaultStealthArgs returns expected flags", () => {
|
||||
const args = getDefaultStealthArgs();
|
||||
const isMac = process.platform === "darwin";
|
||||
|
||||
expect(args).toContain("--no-sandbox");
|
||||
expect(args).toContain("--disable-blink-features=AutomationControlled");
|
||||
|
||||
if (isMac) {
|
||||
expect(args).toContain("--fingerprint-platform=macos");
|
||||
// macOS: no hardware-concurrency or GPU spoofing (uses native values)
|
||||
expect(args.some((a) => a.includes("hardware-concurrency"))).toBe(false);
|
||||
} else {
|
||||
expect(args).toContain("--fingerprint-platform=windows");
|
||||
expect(args).toContain("--fingerprint-hardware-concurrency=8");
|
||||
}
|
||||
|
||||
// Should have a random fingerprint seed
|
||||
const fingerprintArg = args.find((a) => a.startsWith("--fingerprint="));
|
||||
expect(fingerprintArg).toBeDefined();
|
||||
const seed = Number(fingerprintArg!.split("=")[1]);
|
||||
expect(seed).toBeGreaterThanOrEqual(10000);
|
||||
expect(seed).toBeLessThanOrEqual(99999);
|
||||
});
|
||||
|
||||
it("getDefaultStealthArgs generates different seeds", () => {
|
||||
const seeds = new Set<string>();
|
||||
for (let i = 0; i < 10; i++) {
|
||||
const args = getDefaultStealthArgs();
|
||||
const fp = args.find((a) => a.startsWith("--fingerprint="))!;
|
||||
seeds.add(fp);
|
||||
}
|
||||
// With 90k possible seeds, 10 calls should produce at least 2 unique
|
||||
expect(seeds.size).toBeGreaterThan(1);
|
||||
});
|
||||
|
||||
it("getCacheDir returns ~/.cloakbrowser by default", () => {
|
||||
const dir = getCacheDir();
|
||||
expect(dir).toContain(".cloakbrowser");
|
||||
});
|
||||
|
||||
it("getBinaryDir includes version", () => {
|
||||
const dir = getBinaryDir();
|
||||
expect(dir).toContain(`chromium-${CHROMIUM_VERSION}`);
|
||||
});
|
||||
|
||||
it("getDownloadUrl contains version and platform tag", () => {
|
||||
const url = getDownloadUrl();
|
||||
expect(url).toContain(CHROMIUM_VERSION);
|
||||
expect(url).toContain("cloakbrowser-");
|
||||
expect(url).toContain(".tar.gz");
|
||||
expect(url).toContain("cloakbrowser.dev");
|
||||
});
|
||||
});
|
||||
|
||||
describe("buildArgs timezone/locale", () => {
|
||||
it("injects --timezone when timezone is set", () => {
|
||||
const args = _buildArgsForTest({ timezone: "America/New_York" });
|
||||
expect(args).toContain("--timezone=America/New_York");
|
||||
});
|
||||
|
||||
it("injects --lang when locale is set", () => {
|
||||
const args = _buildArgsForTest({ locale: "en-US" });
|
||||
expect(args).toContain("--lang=en-US");
|
||||
});
|
||||
|
||||
it("injects both when both are set", () => {
|
||||
const args = _buildArgsForTest({ timezone: "Europe/Berlin", locale: "de-DE" });
|
||||
expect(args).toContain("--timezone=Europe/Berlin");
|
||||
expect(args).toContain("--lang=de-DE");
|
||||
});
|
||||
|
||||
it("injects timezone/locale even when stealthArgs=false", () => {
|
||||
const args = _buildArgsForTest({ stealthArgs: false, timezone: "America/New_York", locale: "en-US" });
|
||||
expect(args).toContain("--timezone=America/New_York");
|
||||
expect(args).toContain("--lang=en-US");
|
||||
expect(args.some(a => a.startsWith("--fingerprint="))).toBe(false);
|
||||
});
|
||||
|
||||
it("does not inject flags when not set", () => {
|
||||
const args = _buildArgsForTest({});
|
||||
expect(args.some(a => a.startsWith("--timezone="))).toBe(false);
|
||||
expect(args.some(a => a.startsWith("--lang="))).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,45 @@
|
||||
import { describe, it, expect } from "vitest";
|
||||
import { COUNTRY_LOCALE_MAP, resolveProxyIp } from "../src/geoip.js";
|
||||
|
||||
describe("resolveProxyIp", () => {
|
||||
it("returns literal IPv4 from proxy URL", async () => {
|
||||
expect(await resolveProxyIp("http://10.50.96.5:8888")).toBe("10.50.96.5");
|
||||
});
|
||||
|
||||
it("handles proxy URL with credentials", async () => {
|
||||
expect(await resolveProxyIp("http://user:pass@10.50.96.5:8888")).toBe(
|
||||
"10.50.96.5"
|
||||
);
|
||||
});
|
||||
|
||||
it("resolves localhost", async () => {
|
||||
const ip = await resolveProxyIp("http://localhost:8888");
|
||||
expect(ip).toBeTruthy();
|
||||
expect(["127.0.0.1", "::1"]).toContain(ip);
|
||||
});
|
||||
|
||||
it("returns null for invalid URL", async () => {
|
||||
expect(await resolveProxyIp("not-a-url")).toBeNull();
|
||||
});
|
||||
|
||||
it("returns null for empty string", async () => {
|
||||
expect(await resolveProxyIp("")).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe("COUNTRY_LOCALE_MAP", () => {
|
||||
it("contains common countries", () => {
|
||||
for (const code of ["US", "GB", "DE", "FR", "JP", "BR", "IL", "RU"]) {
|
||||
expect(COUNTRY_LOCALE_MAP[code]).toBeDefined();
|
||||
}
|
||||
});
|
||||
|
||||
it("values are BCP 47 language-REGION format", () => {
|
||||
for (const [code, locale] of Object.entries(COUNTRY_LOCALE_MAP)) {
|
||||
const parts = locale.split("-");
|
||||
expect(parts).toHaveLength(2);
|
||||
expect(parts[0]).toMatch(/^[a-z]{2,3}$/);
|
||||
expect(parts[1]).toMatch(/^[A-Z]{2}$/);
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,39 @@
|
||||
import { describe, it, expect } from "vitest";
|
||||
import { binaryInfo } from "../src/download.js";
|
||||
import { CHROMIUM_VERSION } from "../src/config.js";
|
||||
|
||||
describe("binaryInfo", () => {
|
||||
it("returns correct structure", () => {
|
||||
const info = binaryInfo();
|
||||
|
||||
expect(info.version).toBe(CHROMIUM_VERSION);
|
||||
expect(info.platform).toMatch(/^(linux|darwin)-(x64|arm64)$/);
|
||||
expect(info.binaryPath).toBeTruthy();
|
||||
expect(typeof info.installed).toBe("boolean");
|
||||
expect(info.cacheDir).toContain("cloakbrowser");
|
||||
expect(info.downloadUrl).toContain(".tar.gz");
|
||||
});
|
||||
});
|
||||
|
||||
// Integration tests require the binary — run with:
|
||||
// CLOAKBROWSER_BINARY_PATH=/path/to/chrome npm test
|
||||
describe.skipIf(!process.env.CLOAKBROWSER_BINARY_PATH)(
|
||||
"launch (integration)",
|
||||
() => {
|
||||
it("launches browser and checks stealth", async () => {
|
||||
const { launch } = await import("../src/playwright.js");
|
||||
|
||||
const browser = await launch({ headless: true });
|
||||
const page = await browser.newPage();
|
||||
await page.goto("about:blank");
|
||||
|
||||
const webdriver = await page.evaluate(() => navigator.webdriver);
|
||||
expect(webdriver).toBeFalsy();
|
||||
|
||||
const plugins = await page.evaluate(() => navigator.plugins.length);
|
||||
expect(plugins).toBeGreaterThan(0);
|
||||
|
||||
await browser.close();
|
||||
}, 30_000);
|
||||
}
|
||||
);
|
||||
@@ -0,0 +1,49 @@
|
||||
import { describe, it, expect } from "vitest";
|
||||
import { parseProxyUrl } from "../src/proxy.js";
|
||||
|
||||
describe("parseProxyUrl", () => {
|
||||
it("passes through URL without credentials", () => {
|
||||
expect(parseProxyUrl("http://proxy:8080")).toEqual({
|
||||
server: "http://proxy:8080",
|
||||
});
|
||||
});
|
||||
|
||||
it("extracts credentials from URL", () => {
|
||||
expect(parseProxyUrl("http://user:pass@proxy:8080")).toEqual({
|
||||
server: "http://proxy:8080",
|
||||
username: "user",
|
||||
password: "pass",
|
||||
});
|
||||
});
|
||||
|
||||
it("decodes URL-encoded special chars", () => {
|
||||
const result = parseProxyUrl("http://user:p%40ss%3Aword@proxy:8080");
|
||||
expect(result.password).toBe("p@ss:word");
|
||||
expect(result.username).toBe("user");
|
||||
expect(result.server).toBe("http://proxy:8080");
|
||||
});
|
||||
|
||||
it("handles socks5 protocol", () => {
|
||||
const result = parseProxyUrl("socks5://user:pass@proxy:1080");
|
||||
expect(result.server).toBe("socks5://proxy:1080");
|
||||
expect(result.username).toBe("user");
|
||||
expect(result.password).toBe("pass");
|
||||
});
|
||||
|
||||
it("handles URL without port", () => {
|
||||
const result = parseProxyUrl("http://user:pass@proxy");
|
||||
expect(result.server).toBe("http://proxy");
|
||||
expect(result.username).toBe("user");
|
||||
});
|
||||
|
||||
it("handles username only (no password)", () => {
|
||||
const result = parseProxyUrl("http://user@proxy:8080");
|
||||
expect(result.server).toBe("http://proxy:8080");
|
||||
expect(result.username).toBe("user");
|
||||
expect(result.password).toBeUndefined();
|
||||
});
|
||||
|
||||
it("passes through unparseable string", () => {
|
||||
expect(parseProxyUrl("not-a-url")).toEqual({ server: "not-a-url" });
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,61 @@
|
||||
import { describe, it, expect } from "vitest";
|
||||
import {
|
||||
CHROMIUM_VERSION,
|
||||
getDownloadUrl,
|
||||
getEffectiveVersion,
|
||||
parseVersion,
|
||||
versionNewer,
|
||||
} from "../src/config.js";
|
||||
|
||||
describe("version comparison", () => {
|
||||
it("parseVersion handles 4-part versions", () => {
|
||||
expect(parseVersion("145.0.7718.0")).toEqual([145, 0, 7718, 0]);
|
||||
expect(parseVersion("142.0.7444.175")).toEqual([142, 0, 7444, 175]);
|
||||
});
|
||||
|
||||
it("detects newer version", () => {
|
||||
expect(versionNewer("145.0.7718.0", "142.0.7444.175")).toBe(true);
|
||||
});
|
||||
|
||||
it("detects older version", () => {
|
||||
expect(versionNewer("142.0.7444.175", "145.0.7718.0")).toBe(false);
|
||||
});
|
||||
|
||||
it("same version is not newer", () => {
|
||||
expect(versionNewer("142.0.7444.175", "142.0.7444.175")).toBe(false);
|
||||
});
|
||||
|
||||
it("patch bump detected", () => {
|
||||
expect(versionNewer("142.0.7444.176", "142.0.7444.175")).toBe(true);
|
||||
});
|
||||
|
||||
it("major bump wins over minor", () => {
|
||||
expect(versionNewer("143.0.0.0", "142.9.9999.999")).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe("download URL", () => {
|
||||
it("uses chromium-v prefix and cloakbrowser repo", () => {
|
||||
const url = getDownloadUrl();
|
||||
expect(url).toContain("cloakbrowser.dev");
|
||||
expect(url).toContain(`chromium-v${CHROMIUM_VERSION}`);
|
||||
expect(url.endsWith(".tar.gz")).toBe(true);
|
||||
});
|
||||
|
||||
it("accepts custom version", () => {
|
||||
const url = getDownloadUrl("145.0.7718.0");
|
||||
expect(url).toContain("chromium-v145.0.7718.0");
|
||||
});
|
||||
|
||||
it("does not reference old repo", () => {
|
||||
const url = getDownloadUrl();
|
||||
expect(url).not.toContain("chromium-stealth-builds");
|
||||
});
|
||||
});
|
||||
|
||||
describe("effective version", () => {
|
||||
it("returns CHROMIUM_VERSION when no marker exists", () => {
|
||||
// Default behavior — no marker file in test environment
|
||||
expect(getEffectiveVersion()).toBe(CHROMIUM_VERSION);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,19 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "ES2022",
|
||||
"module": "NodeNext",
|
||||
"moduleResolution": "NodeNext",
|
||||
"declaration": true,
|
||||
"declarationMap": true,
|
||||
"sourceMap": true,
|
||||
"outDir": "dist",
|
||||
"rootDir": "src",
|
||||
"strict": true,
|
||||
"esModuleInterop": true,
|
||||
"skipLibCheck": true,
|
||||
"forceConsistentCasingInFileNames": true,
|
||||
"resolveJsonModule": true
|
||||
},
|
||||
"include": ["src"],
|
||||
"exclude": ["dist", "node_modules", "tests", "examples"]
|
||||
}
|
||||
+9
-6
@@ -10,7 +10,7 @@ readme = "README.md"
|
||||
license = "MIT"
|
||||
requires-python = ">=3.9"
|
||||
authors = [
|
||||
{ name = "cloakbrowser" },
|
||||
{ name = "CloakHQ", email = "cloakhq@pm.me" },
|
||||
]
|
||||
keywords = [
|
||||
"stealth",
|
||||
@@ -42,15 +42,18 @@ classifiers = [
|
||||
"Topic :: Software Development :: Testing",
|
||||
]
|
||||
dependencies = [
|
||||
"playwright>=1.40",
|
||||
"patchright>=1.40",
|
||||
"httpx>=0.24",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
geoip = ["geoip2>=4.0"]
|
||||
|
||||
[project.urls]
|
||||
Homepage = "https://github.com/CloakHQ/cloakbrowser"
|
||||
Documentation = "https://github.com/CloakHQ/cloakbrowser#readme"
|
||||
Repository = "https://github.com/CloakHQ/cloakbrowser"
|
||||
Issues = "https://github.com/CloakHQ/cloakbrowser/issues"
|
||||
Homepage = "https://github.com/CloakHQ/CloakBrowser"
|
||||
Documentation = "https://github.com/CloakHQ/CloakBrowser#readme"
|
||||
Repository = "https://github.com/CloakHQ/CloakBrowser"
|
||||
Issues = "https://github.com/CloakHQ/CloakBrowser/issues"
|
||||
|
||||
[tool.hatch.version]
|
||||
path = "cloakbrowser/_version.py"
|
||||
|
||||
@@ -0,0 +1,46 @@
|
||||
"""Unit tests for _build_args timezone/locale injection."""
|
||||
|
||||
from cloakbrowser.browser import _build_args
|
||||
|
||||
|
||||
def test_timezone_injected():
|
||||
"""--timezone flag should appear when timezone is set."""
|
||||
args = _build_args(stealth_args=True, extra_args=None, timezone="America/New_York")
|
||||
assert "--timezone=America/New_York" in args
|
||||
|
||||
|
||||
def test_locale_injected():
|
||||
"""--lang flag should appear when locale is set."""
|
||||
args = _build_args(stealth_args=True, extra_args=None, locale="en-US")
|
||||
assert "--lang=en-US" in args
|
||||
|
||||
|
||||
def test_both_injected():
|
||||
"""Both flags should appear when both are set."""
|
||||
args = _build_args(stealth_args=True, extra_args=None, timezone="Europe/Berlin", locale="de-DE")
|
||||
assert "--timezone=Europe/Berlin" in args
|
||||
assert "--lang=de-DE" in args
|
||||
|
||||
|
||||
def test_timezone_independent_of_stealth_args():
|
||||
"""--timezone should be injected even when stealth_args=False."""
|
||||
args = _build_args(stealth_args=False, extra_args=None, timezone="America/New_York", locale="en-US")
|
||||
assert "--timezone=America/New_York" in args
|
||||
assert "--lang=en-US" in args
|
||||
# No stealth fingerprint args
|
||||
assert not any(a.startswith("--fingerprint=") for a in args)
|
||||
|
||||
|
||||
def test_no_flags_when_not_set():
|
||||
"""No timezone/lang flags when params are None."""
|
||||
args = _build_args(stealth_args=True, extra_args=None)
|
||||
assert not any(a.startswith("--timezone=") for a in args)
|
||||
assert not any(a.startswith("--lang=") for a in args)
|
||||
|
||||
|
||||
def test_extra_args_preserved():
|
||||
"""Extra args should still be included alongside timezone/locale."""
|
||||
args = _build_args(stealth_args=True, extra_args=["--disable-gpu"], timezone="Asia/Tokyo", locale="ja-JP")
|
||||
assert "--disable-gpu" in args
|
||||
assert "--timezone=Asia/Tokyo" in args
|
||||
assert "--lang=ja-JP" in args
|
||||
@@ -0,0 +1,138 @@
|
||||
"""Unit tests for GeoIP-based timezone/locale detection."""
|
||||
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
from cloakbrowser.browser import _maybe_resolve_geoip
|
||||
from cloakbrowser.geoip import (
|
||||
COUNTRY_LOCALE_MAP,
|
||||
_resolve_proxy_ip,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# _resolve_proxy_ip
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_resolve_literal_ipv4():
|
||||
assert _resolve_proxy_ip("http://10.50.96.5:8888") == "10.50.96.5"
|
||||
|
||||
|
||||
def test_resolve_literal_ipv4_with_auth():
|
||||
assert _resolve_proxy_ip("http://user:pass@10.50.96.5:8888") == "10.50.96.5"
|
||||
|
||||
|
||||
def test_resolve_literal_ipv6():
|
||||
ip = _resolve_proxy_ip("http://[::1]:8888")
|
||||
assert ip == "::1"
|
||||
|
||||
|
||||
def test_resolve_hostname():
|
||||
"""DNS resolution of a known hostname should return an IP."""
|
||||
ip = _resolve_proxy_ip("http://localhost:8888")
|
||||
assert ip is not None
|
||||
assert ip in ("127.0.0.1", "::1")
|
||||
|
||||
|
||||
def test_resolve_invalid_url():
|
||||
assert _resolve_proxy_ip("not-a-url") is None
|
||||
|
||||
|
||||
def test_resolve_empty():
|
||||
assert _resolve_proxy_ip("") is None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# COUNTRY_LOCALE_MAP
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_locale_map_has_common_countries():
|
||||
for code in ("US", "GB", "DE", "FR", "JP", "BR", "IL", "RU"):
|
||||
assert code in COUNTRY_LOCALE_MAP, f"Missing {code}"
|
||||
|
||||
|
||||
def test_locale_map_values_are_bcp47():
|
||||
"""All locales should be language-REGION format."""
|
||||
for code, locale in COUNTRY_LOCALE_MAP.items():
|
||||
parts = locale.split("-")
|
||||
assert len(parts) == 2, f"{code}: {locale} not language-REGION"
|
||||
assert parts[0].islower(), f"{code}: language part should be lowercase"
|
||||
assert parts[1].isupper(), f"{code}: region part should be uppercase"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# resolve_proxy_geo fallbacks
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_resolve_geo_raises_when_geoip2_missing():
|
||||
"""Should raise ImportError with install instructions when geoip2 not installed."""
|
||||
with patch.dict("sys.modules", {"geoip2": None, "geoip2.database": None}):
|
||||
from importlib import reload
|
||||
import cloakbrowser.geoip as geoip_mod
|
||||
reload(geoip_mod)
|
||||
with pytest.raises(ImportError, match="pip install cloakbrowser"):
|
||||
geoip_mod.resolve_proxy_geo("http://10.50.96.5:8888")
|
||||
# Restore
|
||||
reload(geoip_mod)
|
||||
|
||||
|
||||
def test_resolve_geo_returns_none_when_db_missing():
|
||||
"""Should return (None, None) when DB file doesn't exist."""
|
||||
mock_geoip2 = type("module", (), {"database": type("db", (), {"Reader": None})})()
|
||||
with patch.dict("sys.modules", {"geoip2": mock_geoip2, "geoip2.database": mock_geoip2.database}):
|
||||
with patch("cloakbrowser.geoip._ensure_geoip_db", return_value=None):
|
||||
with patch("cloakbrowser.geoip._resolve_exit_ip", return_value=None):
|
||||
from cloakbrowser.geoip import resolve_proxy_geo
|
||||
assert resolve_proxy_geo("http://10.50.96.5:8888") == (None, None)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# _maybe_resolve_geoip (browser.py helper)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_maybe_resolve_skips_when_geoip_false():
|
||||
tz, loc = _maybe_resolve_geoip(False, "http://proxy:8080", None, None)
|
||||
assert tz is None
|
||||
assert loc is None
|
||||
|
||||
|
||||
def test_maybe_resolve_skips_when_no_proxy():
|
||||
tz, loc = _maybe_resolve_geoip(True, None, None, None)
|
||||
assert tz is None
|
||||
assert loc is None
|
||||
|
||||
|
||||
def test_maybe_resolve_skips_when_both_explicit():
|
||||
"""Explicit values should not trigger geoip resolution."""
|
||||
tz, loc = _maybe_resolve_geoip(True, "http://proxy:8080", "Europe/Berlin", "de-DE")
|
||||
assert tz == "Europe/Berlin"
|
||||
assert loc == "de-DE"
|
||||
|
||||
|
||||
def test_maybe_resolve_fills_missing_timezone():
|
||||
"""When only locale is explicit, geoip should fill timezone."""
|
||||
with patch("cloakbrowser.geoip.resolve_proxy_geo", return_value=("America/New_York", "en-US")):
|
||||
tz, loc = _maybe_resolve_geoip(True, "http://proxy:8080", None, "fr-FR")
|
||||
assert tz == "America/New_York"
|
||||
assert loc == "fr-FR" # Explicit wins
|
||||
|
||||
|
||||
def test_maybe_resolve_fills_missing_locale():
|
||||
"""When only timezone is explicit, geoip should fill locale."""
|
||||
with patch("cloakbrowser.geoip.resolve_proxy_geo", return_value=("America/New_York", "en-US")):
|
||||
tz, loc = _maybe_resolve_geoip(True, "http://proxy:8080", "Asia/Tokyo", None)
|
||||
assert tz == "Asia/Tokyo" # Explicit wins
|
||||
assert loc == "en-US"
|
||||
|
||||
|
||||
def test_maybe_resolve_fills_both():
|
||||
"""When neither is set, geoip should fill both."""
|
||||
with patch("cloakbrowser.geoip.resolve_proxy_geo", return_value=("Europe/Berlin", "de-DE")):
|
||||
tz, loc = _maybe_resolve_geoip(True, "http://proxy:8080", None, None)
|
||||
assert tz == "Europe/Berlin"
|
||||
assert loc == "de-DE"
|
||||
@@ -0,0 +1,50 @@
|
||||
"""Tests for proxy URL parsing and credential extraction."""
|
||||
|
||||
from cloakbrowser.browser import _build_proxy_kwargs, _parse_proxy_url
|
||||
|
||||
|
||||
class TestParseProxyUrl:
|
||||
def test_no_credentials(self):
|
||||
assert _parse_proxy_url("http://proxy:8080") == {"server": "http://proxy:8080"}
|
||||
|
||||
def test_with_credentials(self):
|
||||
result = _parse_proxy_url("http://user:pass@proxy:8080")
|
||||
assert result == {"server": "http://proxy:8080", "username": "user", "password": "pass"}
|
||||
|
||||
def test_url_encoded_password(self):
|
||||
result = _parse_proxy_url("http://user:p%40ss%3Aword@proxy:8080")
|
||||
assert result["password"] == "p@ss:word"
|
||||
assert result["username"] == "user"
|
||||
assert result["server"] == "http://proxy:8080"
|
||||
|
||||
def test_socks5(self):
|
||||
result = _parse_proxy_url("socks5://user:pass@proxy:1080")
|
||||
assert result["server"] == "socks5://proxy:1080"
|
||||
assert result["username"] == "user"
|
||||
assert result["password"] == "pass"
|
||||
|
||||
def test_no_port(self):
|
||||
result = _parse_proxy_url("http://user:pass@proxy")
|
||||
assert result["server"] == "http://proxy"
|
||||
assert result["username"] == "user"
|
||||
|
||||
def test_username_only(self):
|
||||
result = _parse_proxy_url("http://user@proxy:8080")
|
||||
assert result["server"] == "http://proxy:8080"
|
||||
assert result["username"] == "user"
|
||||
assert "password" not in result
|
||||
|
||||
|
||||
class TestBuildProxyKwargs:
|
||||
def test_none(self):
|
||||
assert _build_proxy_kwargs(None) == {}
|
||||
|
||||
def test_simple_proxy(self):
|
||||
result = _build_proxy_kwargs("http://proxy:8080")
|
||||
assert result == {"proxy": {"server": "http://proxy:8080"}}
|
||||
|
||||
def test_proxy_with_auth(self):
|
||||
result = _build_proxy_kwargs("http://user:pass@proxy:8080")
|
||||
assert result == {
|
||||
"proxy": {"server": "http://proxy:8080", "username": "user", "password": "pass"}
|
||||
}
|
||||
+130
-19
@@ -4,14 +4,19 @@ These tests verify that the stealth Chromium binary passes common
|
||||
bot detection checks. They require network access.
|
||||
"""
|
||||
|
||||
import os
|
||||
import time
|
||||
|
||||
import pytest
|
||||
from cloakbrowser import launch
|
||||
|
||||
PROXY = os.environ.get("CLOAKBROWSER_TEST_PROXY")
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def browser():
|
||||
"""Shared browser instance for stealth tests."""
|
||||
b = launch(headless=True)
|
||||
b = launch(headless=True, proxy=PROXY)
|
||||
yield b
|
||||
b.close()
|
||||
|
||||
@@ -48,7 +53,7 @@ class TestWebDriverDetection:
|
||||
"""Must have browser plugins (real Chrome has 5)."""
|
||||
page.goto("https://example.com")
|
||||
count = page.evaluate("navigator.plugins.length")
|
||||
assert count >= 1, f"Expected plugins, got {count}"
|
||||
assert count >= 5, f"Expected 5+ plugins (real Chrome), got {count}"
|
||||
|
||||
def test_languages_present(self, page):
|
||||
"""navigator.languages must be populated."""
|
||||
@@ -81,28 +86,134 @@ class TestBotDetectionSites:
|
||||
Mark with pytest -m slow to skip in CI.
|
||||
"""
|
||||
|
||||
@pytest.mark.slow
|
||||
def test_bot_sannysoft(self, page):
|
||||
"""bot.sannysoft.com — all checks should pass (0 failures)."""
|
||||
page.goto("https://bot.sannysoft.com", wait_until="networkidle", timeout=30000)
|
||||
time.sleep(3)
|
||||
|
||||
results = page.evaluate("""() => {
|
||||
const rows = document.querySelectorAll('table tr');
|
||||
const failed = [];
|
||||
let total = 0;
|
||||
rows.forEach(r => {
|
||||
const cells = r.querySelectorAll('td');
|
||||
if (cells.length >= 2) {
|
||||
total++;
|
||||
const cls = cells[1].className || '';
|
||||
if (cls.includes('failed')) {
|
||||
failed.push(cells[0].innerText.trim());
|
||||
}
|
||||
}
|
||||
});
|
||||
return {total, failed};
|
||||
}""")
|
||||
|
||||
failed = results["failed"]
|
||||
assert len(failed) == 0, f"Sannysoft failures: {', '.join(failed)}"
|
||||
|
||||
@pytest.mark.slow
|
||||
def test_bot_incolumitas(self, page):
|
||||
"""bot.incolumitas.com should detect minimal flags."""
|
||||
page.goto("https://bot.incolumitas.com", timeout=30000)
|
||||
page.wait_for_timeout(5000)
|
||||
# Check that we're not immediately flagged
|
||||
title = page.title()
|
||||
assert title # Page loaded successfully
|
||||
"""bot.incolumitas.com — max 1 failure (WEBDRIVER false positive expected)."""
|
||||
page.goto("https://bot.incolumitas.com", wait_until="networkidle", timeout=30000)
|
||||
time.sleep(12)
|
||||
|
||||
# Known acceptable failures (not browser fingerprint issues):
|
||||
# - WEBDRIVER: spec-level false positive across all builds
|
||||
# - connectionRTT: detects datacenter/proxy network latency, not browser
|
||||
KNOWN_ACCEPTABLE = {"WEBDRIVER", "connectionRTT"}
|
||||
|
||||
results = page.evaluate("""() => {
|
||||
const text = document.body.innerText;
|
||||
const okMatches = text.match(/"\\w+":\\s*"OK"/g) || [];
|
||||
const failMatches = text.match(/"\\w+":\\s*"FAIL"/g) || [];
|
||||
const failedTests = failMatches.map(m => m.match(/"(\\w+)"/)[1]);
|
||||
return {passed: okMatches.length, failed: failMatches.length, failedTests};
|
||||
}""")
|
||||
|
||||
failed_names = results["failedTests"]
|
||||
real_failures = [f for f in failed_names if f not in KNOWN_ACCEPTABLE]
|
||||
assert len(real_failures) == 0, f"Incolumitas unexpected failures: {', '.join(real_failures)}"
|
||||
|
||||
@pytest.mark.slow
|
||||
def test_browserscan(self, page):
|
||||
"""BrowserScan bot detection should show NORMAL."""
|
||||
page.goto("https://www.browserscan.net/bot-detection", timeout=30000)
|
||||
page.wait_for_timeout(5000)
|
||||
title = page.title()
|
||||
assert title # Page loaded
|
||||
"""BrowserScan bot detection — 0 abnormal checks."""
|
||||
page.goto("https://www.browserscan.net/bot-detection", wait_until="networkidle", timeout=30000)
|
||||
time.sleep(5)
|
||||
|
||||
results = page.evaluate("""() => {
|
||||
const text = document.body.innerText;
|
||||
const normalMatches = text.match(/Normal/g);
|
||||
const abnormalMatches = text.match(/Abnormal/g);
|
||||
return {
|
||||
normal: normalMatches ? normalMatches.length : 0,
|
||||
abnormal: abnormalMatches ? abnormalMatches.length : 0
|
||||
};
|
||||
}""")
|
||||
|
||||
assert results["abnormal"] == 0, \
|
||||
f"BrowserScan: {results['abnormal']} abnormal, {results['normal']} normal"
|
||||
|
||||
@pytest.mark.slow
|
||||
def test_device_and_browser_info(self, page):
|
||||
"""deviceandbrowserinfo.com should report isBot: false."""
|
||||
page.goto("https://deviceandbrowserinfo.com/are_you_a_bot", timeout=30000)
|
||||
page.wait_for_timeout(5000)
|
||||
content = page.content()
|
||||
# The page shows bot detection results
|
||||
assert "deviceandbrowserinfo" in page.url.lower()
|
||||
"""deviceandbrowserinfo.com — isBot must be false."""
|
||||
page.goto("https://deviceandbrowserinfo.com/are_you_a_bot", wait_until="domcontentloaded", timeout=30000)
|
||||
time.sleep(8)
|
||||
|
||||
results = page.evaluate("""() => {
|
||||
const text = document.body.innerText;
|
||||
const botMatch = text.match(/"isBot":\\s*(true|false)/);
|
||||
const isBot = botMatch ? botMatch[1] === 'true' : null;
|
||||
const checks = {};
|
||||
['isBot', 'hasBotUserAgent', 'hasWebdriverTrue', 'isHeadlessChrome',
|
||||
'isAutomatedWithCDP', 'hasSuspiciousWeakSignals', 'isPlaywright',
|
||||
'hasInconsistentChromeObject'].forEach(p => {
|
||||
const match = text.match(new RegExp('"' + p + '":\\s*(true|false)'));
|
||||
if (match) checks[p] = match[1] === 'true';
|
||||
});
|
||||
return {isBot, checks};
|
||||
}""")
|
||||
|
||||
assert results["isBot"] is False, f"Detected as bot! Checks: {results['checks']}"
|
||||
|
||||
@pytest.mark.slow
|
||||
def test_fingerprintjs(self, page):
|
||||
"""FingerprintJS — must not be blocked, should see flight data."""
|
||||
page.goto("https://demo.fingerprint.com/web-scraping", wait_until="domcontentloaded", timeout=30000)
|
||||
time.sleep(8)
|
||||
|
||||
try:
|
||||
page.click("button:has-text('Search')", timeout=5000)
|
||||
time.sleep(5)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
results = page.evaluate("""() => {
|
||||
const text = document.body.innerText;
|
||||
const hasFlights = text.includes('Price per adult') || text.includes('$');
|
||||
const isBlocked = text.includes('request was blocked') || text.includes('bot visit detected');
|
||||
return {passed: hasFlights && !isBlocked, isBlocked, hasFlights};
|
||||
}""")
|
||||
|
||||
assert not results["isBlocked"], "FingerprintJS blocked us as a bot"
|
||||
assert results["passed"], "FingerprintJS: no flight data shown"
|
||||
|
||||
@pytest.mark.slow
|
||||
def test_recaptcha_v3(self, page):
|
||||
"""reCAPTCHA v3 — score must be >= 0.7."""
|
||||
page.goto(
|
||||
"https://recaptcha-demo.appspot.com/recaptcha-v3-request-scores.php",
|
||||
wait_until="domcontentloaded",
|
||||
timeout=60000,
|
||||
)
|
||||
time.sleep(8)
|
||||
|
||||
results = page.evaluate("""() => {
|
||||
const text = document.body.innerText;
|
||||
const scoreMatch = text.match(/"score":\\s*(\\d+\\.\\d+)/);
|
||||
return {score: scoreMatch ? parseFloat(scoreMatch[1]) : null};
|
||||
}""")
|
||||
|
||||
score = results["score"]
|
||||
assert score is not None, "Could not extract reCAPTCHA score"
|
||||
assert score >= 0.7, f"reCAPTCHA score too low: {score}"
|
||||
|
||||
@@ -0,0 +1,172 @@
|
||||
"""Tests for auto-update and version management."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from cloakbrowser.config import (
|
||||
CHROMIUM_VERSION,
|
||||
_version_newer,
|
||||
_version_tuple,
|
||||
get_download_url,
|
||||
get_effective_version,
|
||||
)
|
||||
from cloakbrowser.download import (
|
||||
_get_latest_chromium_version,
|
||||
_should_check_for_update,
|
||||
)
|
||||
|
||||
|
||||
class TestVersionComparison:
|
||||
def test_version_tuple_parsing(self):
|
||||
assert _version_tuple("145.0.7718.0") == (145, 0, 7718, 0)
|
||||
assert _version_tuple("142.0.7444.175") == (142, 0, 7444, 175)
|
||||
|
||||
def test_newer_version(self):
|
||||
assert _version_newer("145.0.7718.0", "142.0.7444.175") is True
|
||||
|
||||
def test_older_version(self):
|
||||
assert _version_newer("142.0.7444.175", "145.0.7718.0") is False
|
||||
|
||||
def test_same_version(self):
|
||||
assert _version_newer("142.0.7444.175", "142.0.7444.175") is False
|
||||
|
||||
def test_patch_bump(self):
|
||||
assert _version_newer("142.0.7444.176", "142.0.7444.175") is True
|
||||
|
||||
def test_major_bump(self):
|
||||
assert _version_newer("143.0.0.0", "142.9.9999.999") is True
|
||||
|
||||
|
||||
class TestDownloadUrl:
|
||||
def test_default_url_format(self):
|
||||
url = get_download_url()
|
||||
assert "cloakbrowser.dev" in url
|
||||
assert f"chromium-v{CHROMIUM_VERSION}" in url
|
||||
assert url.endswith(".tar.gz")
|
||||
|
||||
def test_custom_version_url(self):
|
||||
url = get_download_url("145.0.7718.0")
|
||||
assert "chromium-v145.0.7718.0" in url
|
||||
|
||||
def test_no_old_repo_reference(self):
|
||||
url = get_download_url()
|
||||
assert "chromium-stealth-builds" not in url
|
||||
|
||||
|
||||
class TestShouldCheckForUpdate:
|
||||
def test_disabled_by_env(self):
|
||||
with patch.dict(os.environ, {"CLOAKBROWSER_AUTO_UPDATE": "false"}):
|
||||
assert _should_check_for_update() is False
|
||||
|
||||
def test_disabled_by_env_case_insensitive(self):
|
||||
with patch.dict(os.environ, {"CLOAKBROWSER_AUTO_UPDATE": "False"}):
|
||||
assert _should_check_for_update() is False
|
||||
|
||||
def test_disabled_by_binary_override(self):
|
||||
with patch.dict(os.environ, {"CLOAKBROWSER_BINARY_PATH": "/some/path"}):
|
||||
assert _should_check_for_update() is False
|
||||
|
||||
def test_disabled_by_custom_download_url(self):
|
||||
with patch.dict(
|
||||
os.environ, {"CLOAKBROWSER_DOWNLOAD_URL": "https://my-mirror.com"}
|
||||
):
|
||||
assert _should_check_for_update() is False
|
||||
|
||||
def test_rate_limited(self, tmp_path):
|
||||
import time
|
||||
|
||||
with patch.dict(
|
||||
os.environ,
|
||||
{
|
||||
"CLOAKBROWSER_CACHE_DIR": str(tmp_path),
|
||||
"CLOAKBROWSER_BINARY_PATH": "",
|
||||
"CLOAKBROWSER_AUTO_UPDATE": "",
|
||||
"CLOAKBROWSER_DOWNLOAD_URL": "",
|
||||
},
|
||||
):
|
||||
check_file = tmp_path / ".last_update_check"
|
||||
check_file.write_text(str(time.time()))
|
||||
assert _should_check_for_update() is False
|
||||
|
||||
def test_stale_rate_limit_allows_check(self, tmp_path):
|
||||
import time
|
||||
|
||||
with patch.dict(
|
||||
os.environ,
|
||||
{
|
||||
"CLOAKBROWSER_CACHE_DIR": str(tmp_path),
|
||||
"CLOAKBROWSER_BINARY_PATH": "",
|
||||
"CLOAKBROWSER_AUTO_UPDATE": "",
|
||||
"CLOAKBROWSER_DOWNLOAD_URL": "",
|
||||
},
|
||||
):
|
||||
check_file = tmp_path / ".last_update_check"
|
||||
check_file.write_text(str(time.time() - 7200)) # 2 hours ago
|
||||
assert _should_check_for_update() is True
|
||||
|
||||
|
||||
class TestEffectiveVersion:
|
||||
def test_no_marker_returns_hardcoded(self, tmp_path):
|
||||
with patch.dict(os.environ, {"CLOAKBROWSER_CACHE_DIR": str(tmp_path)}):
|
||||
assert get_effective_version() == CHROMIUM_VERSION
|
||||
|
||||
def test_marker_with_newer_version(self, tmp_path):
|
||||
with patch.dict(os.environ, {"CLOAKBROWSER_CACHE_DIR": str(tmp_path)}):
|
||||
marker = tmp_path / "latest_version"
|
||||
marker.write_text("999.0.0.0")
|
||||
# Binary doesn't exist, so should fall back
|
||||
assert get_effective_version() == CHROMIUM_VERSION
|
||||
|
||||
def test_marker_with_older_version_ignored(self, tmp_path):
|
||||
with patch.dict(os.environ, {"CLOAKBROWSER_CACHE_DIR": str(tmp_path)}):
|
||||
marker = tmp_path / "latest_version"
|
||||
marker.write_text("100.0.0.0")
|
||||
assert get_effective_version() == CHROMIUM_VERSION
|
||||
|
||||
|
||||
class TestGetLatestVersion:
|
||||
def test_parses_chromium_tag(self):
|
||||
mock_response = MagicMock()
|
||||
mock_response.json.return_value = [
|
||||
{"tag_name": "chromium-v145.0.7718.0", "draft": False},
|
||||
{"tag_name": "chromium-v142.0.7444.175", "draft": False},
|
||||
]
|
||||
mock_response.raise_for_status = MagicMock()
|
||||
|
||||
with patch("cloakbrowser.download.httpx.get", return_value=mock_response):
|
||||
result = _get_latest_chromium_version()
|
||||
assert result == "145.0.7718.0"
|
||||
|
||||
def test_skips_draft_releases(self):
|
||||
mock_response = MagicMock()
|
||||
mock_response.json.return_value = [
|
||||
{"tag_name": "chromium-v999.0.0.0", "draft": True},
|
||||
{"tag_name": "chromium-v145.0.7718.0", "draft": False},
|
||||
]
|
||||
mock_response.raise_for_status = MagicMock()
|
||||
|
||||
with patch("cloakbrowser.download.httpx.get", return_value=mock_response):
|
||||
result = _get_latest_chromium_version()
|
||||
assert result == "145.0.7718.0"
|
||||
|
||||
def test_skips_non_chromium_tags(self):
|
||||
mock_response = MagicMock()
|
||||
mock_response.json.return_value = [
|
||||
{"tag_name": "v0.2.0", "draft": False},
|
||||
{"tag_name": "chromium-v145.0.7718.0", "draft": False},
|
||||
]
|
||||
mock_response.raise_for_status = MagicMock()
|
||||
|
||||
with patch("cloakbrowser.download.httpx.get", return_value=mock_response):
|
||||
result = _get_latest_chromium_version()
|
||||
assert result == "145.0.7718.0"
|
||||
|
||||
def test_network_error_returns_none(self):
|
||||
with patch("cloakbrowser.download.httpx.get", side_effect=Exception("timeout")):
|
||||
result = _get_latest_chromium_version()
|
||||
assert result is None
|
||||
Reference in New Issue
Block a user