mirror of
https://github.com/germondai/trawl.git
synced 2026-08-17 12:11:23 +02:00
6.1 KiB
6.1 KiB
title, description
| title | description |
|---|---|
| FlareSolverr Compat | POST /v1 — the drop-in FlareSolverr v2 endpoint. |
POST /v1 — FlareSolverr Compatible
This endpoint implements the FlareSolverr v2 API contract. Any client that works with FlareSolverr works with TRAWL without code changes.
No authentication required.
Request
interface FlareSolverrRequest {
cmd?: 'request.get' | 'request.post' // default: 'request.get'
url: string
maxTimeout?: number // milliseconds, default 60000
postData?: string // body for request.post
headers?: Record<string, string>
proxy?: string // TRAWL extension — not part of the real FlareSolverr contract
}
Fields
| Field | Type | Required | Description |
|---|---|---|---|
cmd |
string | No | "request.get" or "request.post" (default "request.get") |
url |
string | Yes | The URL to scrape |
maxTimeout |
number | No | Max wait in ms (default 60000) |
postData |
string | No | POST body (only for request.post). On TRAWL's native /scrape endpoint this field is named body; the /v1 adapter maps postData → body internally so the FlareSolverr wire contract stays unchanged for existing callers. |
headers |
object | No | Custom headers forwarded to the target across all tiers — see Custom Headers. For Prowlarr compatibility, contentType is accepted as Content-Type; serialized contentLength is ignored and recalculated by the HTTP client. An explicit standard Content-Type takes precedence. |
proxy |
string | No | TRAWL-specific extension (not in the real FlareSolverr v2 contract) — per-request proxy override for Tier 3/4, see Configuration § Proxies |
Response
interface FlareSolverrResponse {
status: 'ok' | 'error'
message: string
startTimestamp: number // unix ms
endTimestamp: number // unix ms
version: '2.0.0'
solution: {
url: string // final URL after redirects
status: number // HTTP status code
headers: Record<string, string>
response: string // raw HTML body
cookies: Cookie[]
userAgent: string
}
}
interface Cookie {
name: string
value: string
domain: string
path: string
expires: number
httpOnly: boolean
secure: boolean
sameSite?: string
}
Examples
GET request (curl)
curl -s -X POST http://localhost:8191/v1 \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.get",
"url": "https://nowsecure.nl",
"maxTimeout": 60000
}'
GET request (JavaScript)
const res = await fetch('http://localhost:8191/v1', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
cmd: 'request.get',
url: 'https://nowsecure.nl',
maxTimeout: 60000,
}),
})
const data = await res.json()
// data.status === 'ok'
// data.solution.response → HTML string
// data.solution.cookies → Cookie[]
// data.solution.userAgent → browser UA
GET request (Python)
import requests
res = requests.post('http://localhost:8191/v1', json={
'cmd': 'request.get',
'url': 'https://nowsecure.nl',
'maxTimeout': 60000,
}, timeout=65)
data = res.json()
assert data['status'] == 'ok'
html = data['solution']['response']
cookies = data['solution']['cookies']
POST request
POST requests with postData must supply a content type. TRAWL accepts either a standard
Content-Type header or Prowlarr's serialized headers.contentType representation at this
compatibility endpoint. Native /scrape requests continue to require the standard header.
curl -s -X POST http://localhost:8191/v1 \
-H "Content-Type: application/json" \
-d '{
"cmd": "request.post",
"url": "https://example.com/api/login",
"postData": "username=user&password=pass",
"headers": { "Content-Type": "application/x-www-form-urlencoded" },
"maxTimeout": 30000
}'
Error response
When the request fails, the response is still a FlareSolverr v2 envelope with status: "error" and an empty solution. The HTTP status code carries the failure class:
| Code | Meaning |
|---|---|
| 200 | status: "ok" (request succeeded) |
| 400 | Malformed request body |
| 429 | Pool exhausted — all browsers busy past BROWSER_ACQUIRE_TIMEOUT_MS |
| 500 | Internal error |
Example — pool exhausted (HTTP 429):
{
"status": "error",
"message": "Browser pool saturated, retry shortly",
"startTimestamp": 1700000000000,
"endTimestamp": 1700000015000,
"version": "2.0.0",
"solution": {
"url": "https://nowsecure.nl",
"status": 0,
"headers": {},
"response": "",
"cookies": [],
"userAgent": ""
}
}