External/trawl
1
0
Fork 0
mirror of https://github.com/germondai/trawl.git synced 2026-08-17 12:11:23 +02:00
Code Issues Packages Projects Releases Wiki Activity

docs: refresh feature overview and project contact addresses

Browse Source
This commit is contained in:
germondai
2026-08-02 02:46:53 +02:00
parent 7715f33dc7
commit 0fb4097388
3 changed files with 9 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CODE_OF_CONDUCT.md
+1 -1
View File
@@ -36,7 +36,7 @@ This Code of Conduct applies within all community spaces — the GitHub reposito
## Enforcement ## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the community leaders responsible for enforcement at **conduct@trawl.dev**. All complaints will be reviewed and investigated promptly and fairly. Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the community leaders responsible for enforcement at **conduct@trawl.germondai.com**. All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the reporter of any incident. All community leaders are obligated to respect the privacy and security of the reporter of any incident.
README.md
+7 -5
View File
@@ -8,18 +8,20 @@
## **Welcome** to <a href="https://trawl.germondai.com" target="_blank">**TRAWL**</a>! 👋 ## **Welcome** to <a href="https://trawl.germondai.com" target="_blank">**TRAWL**</a>! 👋
Self-hosted web scraping engine with best-effort JS challenge and CAPTCHA solving.\ Self-hosted web scraping engine with best-effort JS challenge and CAPTCHA solving.\
Support for: Cloudflare, Turnstile, Interstitial, reCAPTCHA, hCaptcha, GeeTest, Imperva (experimental).\ Dedicated flows for Cloudflare, Akamai Bot Manager, and Imperva/Incapsula (best effort), plus Turnstile, reCAPTCHA, hCaptcha, and GeeTest.\
Much faster and more reliable FlareSolverr & Byparr alternative and drop-in replacement for your \*arr stack. Much faster and more reliable FlareSolverr & Byparr alternative and drop-in replacement for your \*arr stack.
## Features ## Features
- **2-6x faster** - compared to FlareSolverr or Byparr it returns much faster with higher success rate - **2-6x faster** - compared to FlareSolverr or Byparr it returns much faster with higher success rate
- **4-tier execution** - plain HTTP fetch → cached browser session → fresh CF solve → residential proxy - **4-tier execution** - plain HTTP fetch → cached browser session → fresh challenge solve → residential proxy
- **Challenge-aware HTTP/HTTPS proxy** - direct forwarding for normal traffic, automatic tier escalation for detected walls, plus WebSockets, binary bodies, and Range/206 support
- **Multi-WAF handling** - dedicated Cloudflare, Akamai Bot Manager, and Imperva/Incapsula detection and browser flows
- **Native captcha solving** - CF Turnstile/Interstitial, reCAPTCHA v2 (free STT), hCaptcha, GeeTest v4 Slide - **Native captcha solving** - CF Turnstile/Interstitial, reCAPTCHA v2 (free STT), hCaptcha, GeeTest v4 Slide
- **Camoufox Firefox** - fingerprint-patched at the C++/Juggler level to reduce automation signals - **Camoufox Firefox** - fingerprint-patched at the C++/Juggler level to reduce automation signals
- **Session cache** - bypass cookies stored in Redis; repeat requests to the same domain return in ~500ms - **Session cache** - solved cookies and browser identity stored in Redis; accepted sessions can avoid a fresh solve
- **FlareSolverr compatible** - works with Prowlarr, Jackett, Sonarr, and the full \*arr ecosystem out of the box - **FlareSolverr compatible** - works with Prowlarr, Jackett, Sonarr, and the full \*arr ecosystem out of the box
- **No external APIs required** - reCAPTCHA audio transcription uses Google's free STT endpoint by default - **No paid solver API required** - reCAPTCHA audio can use Google's free STT endpoint or an optional local Whisper service
## Sponsors ## Sponsors
@@ -291,7 +293,7 @@ Tier 1: Plain HTTP fetch ────── success ──→ return (< 100ms)
Tier 2: Cached session ─────── success ──→ return (~500ms) Tier 2: Cached session ─────── success ──→ return (~500ms)
│ cache miss / expired │ cache miss / expired
▼ ▼
Tier 3: Fresh CF solve ─────── success ──→ cache + return (4–15s) Tier 3: Fresh challenge solve ─ success ──→ cache + return
│ IP flagged │ IP flagged
▼ ▼
Tier 4: Residential proxy ──── success ──→ cache + return (15–45s) Tier 4: Residential proxy ──── success ──→ cache + return (15–45s)
SECURITY.md
+1 -1
View File
@@ -16,7 +16,7 @@ TRAWL is currently in pre-1.0 development (`0.1.x`). Only the latest minor relea
Report privately through either of these channels (in order of preference): Report privately through either of these channels (in order of preference):
1. **GitHub Security Advisories** — open a [private security advisory](https://github.com/germondai/trawl/security/advisories/new) on this repository. 1. **GitHub Security Advisories** — open a [private security advisory](https://github.com/germondai/trawl/security/advisories/new) on this repository.
2. **Email** — `security@trawl.dev` 2. **Email** — `security@trawl.germondai.com`
Both channels reach the maintainer (@germondai). Please include: Both channels reach the maintainer (@germondai). Please include: