External/trawl
1
0
Fork 0
mirror of https://github.com/germondai/trawl.git synced 2026-08-17 12:11:23 +02:00
Code Issues Packages Projects Releases Wiki Activity
296 Commits 2 Branches 14 Tags
daff2c9f0e07fe67f327c1e55fecc7ebc15df7ea
Commit Graph
296 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
GermondandGitHub daff2c9f0e Merge pull request #58 from germondai/52-bug-ram-usage-has-increased-dramatically 
fix(browser): bound memory with rolling recycling
 2026-08-09 18:19:03 +02:00
germondai c0769af181 fix(browser): bound memory with rolling recycling 2026-08-09 18:15:48 +02:00
GermondandGitHub 34032cd296 Merge pull request #57 from germondai/50-bug-indexer-tests-in-prowlarr-return-http-400-bad-request 
fix(api): support Prowlarr POST content metadata
 2026-08-09 14:57:51 +02:00
germondai 66e48fd5fe fix(api): support Prowlarr POST content metadata 2026-08-09 05:15:49 +02:00
GermondandGitHub 92c016b4c4 Merge pull request #56 from germondai/54-bug-browser-crashs-on-loading 
fix(browser): clarify Gluetun startup failures
 2026-08-08 23:01:32 +02:00
germondai 2547daa41b fix(browser): clarify Gluetun startup failures 2026-08-08 22:58:10 +02:00
GermondandGitHub 83b86b3bb7 Merge pull request #55 from germondai/51-bug-url-returned-is-not-the-effective-url-after-redirects 
fix: report effective URL after redirects
 2026-08-08 18:30:08 +02:00
germondai c2ab109fe5 fix: report effective URL after redirects 2026-08-08 18:24:29 +02:00
germondai 5d19afc6e9 docs: add NAS app catalog links 2026-08-02 17:00:33 +02:00
germondai 097565a19f chore(release): prepare v1.3.1 v1.3.1 2026-08-02 15:51:35 +02:00
GermondandGitHub 9a2459e317 Merge pull request #49 from germondai/45-bug-mitm-proxy-in-prowlarr-fails-with-1337x 
fix(browser): keep persistent contexts pool-owned
 2026-08-02 15:46:43 +02:00
germondai 8b54abd4a1 fix(browser): keep persistent contexts pool-owned 2026-08-02 15:37:47 +02:00
germondai 121f57b681 chore(release): finalize v1.3.0 changelog v1.3.0 2026-08-02 03:37:05 +02:00
GermondandGitHub 1f38465ddb Merge pull request #47 from nandyalu/fix/tier1-full-body-response 
fix(tiers): return full body from Tier 1, not the 4 KiB detection preview
 2026-08-02 03:20:51 +02:00
germondai 3a3226ed20 chore(release): prepare v1.3.0 dependencies 2026-08-02 03:07:04 +02:00
germondai 0fb4097388 docs: refresh feature overview and project contact addresses 2026-08-02 02:46:53 +02:00
germondai 7715f33dc7 fix(web): update canonical site URL 2026-08-02 02:46:07 +02:00
germondai 7456e75990 fix(docker): preserve architecture-specific Camoufox metadata and tune browser pool defaults 2026-08-02 02:37:36 +02:00
germondai 2fc1031874 docs: update API, architecture, proxy, deployment, and integration guides 2026-08-02 02:35:35 +02:00
germondai 357767ce3c feat(web): refresh landing page content, comparisons, proxy showcase, and SEO 2026-08-02 02:34:34 +02:00
nandyaluandClaude Fable 5 de0bc788fc test(tiers): add full-body regression tests; reuse preview when body fits 
Address review feedback:
- Reuse previewText on success when the body fits inside the preview
  window, avoiding a redundant second decode for small responses.
- Add regression tests for #46: full html for >4 KiB responses, a
  multi-byte character straddling the preview boundary, and small
  responses returned unchanged. Verified the large-body test fails
  against the pre-fix Tier 1.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
 2026-07-29 00:30:13 -05:00
nandyaluandClaude Fable 5 53154aabe1 fix(tiers): return full body from Tier 1, not the 4 KiB detection preview 
Since d1ebbdd (1.2.0), Tier 1 reuses previewText — the 4096-byte slice
decoded for challenge detection — as the html field on success. The
/v1 and /scrape routes serve html as solution.response, so any
non-challenged text response larger than 4 KiB comes back silently
truncated: HTTP 200, status ok, no error anywhere.

Decode the full rawBytes buffer for html instead. previewText remains
bounded and is still used only for challenge/block detection. Tiers
2-4 are unaffected (their html is the browser-rendered DOM).

Fixes #46

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
 2026-07-29 00:23:37 -05:00
GermondandGitHub d899a507e1 Merge pull request #44 from germondai/26-doc-examples-for-residential_proxy_ 
docs(proxy): document residential proxy setup
 2026-07-27 03:29:59 +02:00
germondai 01b911d2e8 docs(proxy): document residential proxy setup 2026-07-27 03:27:57 +02:00
GermondandGitHub f036541045 Merge pull request #43 from germondai/40-bug-authenticated-proxy-fails-on-firefoxcamoufox-credentials-embedded-in-proxyserver-are-ignored-tier-34-ns_error_proxy_connection_refused 
fix(proxy): support authenticated browser proxies
 2026-07-27 03:11:12 +02:00
germondai 78836d6ed7 fix(proxy): support authenticated browser proxies 2026-07-27 03:08:16 +02:00
germondai 3a4d7491b3 Merge pull request #37 from funkypenguin/fix/bound-restart-and-reclaim-stalled 2026-07-27 02:40:41 +02:00
germondai 98e61d3770 Merge pull request #33 from edasque/feat/akamai-support 2026-07-26 17:27:02 +02:00
germondai 52e9700642 Merge pull request #42 from potatosips/fix/validate-request-bodies 2026-07-26 17:01:31 +02:00
germondai f397815fe7 fix(compose): pass upstream proxy configuration 2026-07-26 16:39:52 +02:00
germondai f08f5a2b44 chore(release): prepare v1.2.0 v1.2.0 2026-07-26 16:18:47 +02:00
germondai 6d49011114 refactor(web): simplify GitHub stars fallback 2026-07-26 15:12:09 +02:00
germondai d0fd300745 docs(changelog): record proxy architecture changes 2026-07-26 14:41:52 +02:00
germondai c528b45c98 docs(readme): document transparent proxy usage 2026-07-26 14:06:38 +02:00
germondai a71f1ac8d3 docs(proxy): add setup and operation guides 2026-07-26 13:18:27 +02:00
germondai a60635ea07 docs(config): clarify environment variables 2026-07-26 11:37:24 +02:00
germondai 1837293f5a chore(compose): expose and persist forward proxy 2026-07-26 10:54:43 +02:00
potatosips bcbdf13801 docs: note request validation fix 2026-07-26 14:10:11 +06:00
potatosips 09c8eb9608 fix(api): validate request bodies before scraping 2026-07-26 14:09:13 +06:00
germondai cbf6749103 ci: expand container publishing workflows 2026-07-26 08:29:16 +02:00
germondai 48ccac75ee test: add workspace test command 2026-07-25 19:06:37 +02:00
germondai eff5760ef5 feat(api): wire proxy lifecycle and context cache 2026-07-25 17:42:11 +02:00
germondai e62e73554c feat(proxy): route traffic through tiered fallback 2026-07-25 15:13:58 +02:00
germondai d7cca3ec44 feat(proxy): add challenge-aware response policy 2026-07-25 12:47:26 +02:00
germondai ce028e201c feat(proxy): add adaptive direct forwarding 2026-07-25 10:22:39 +02:00
germondai 748d89dfc5 refactor(proxy): normalize proxy pool fallbacks 2026-07-25 08:34:12 +02:00
germondai 1d752aee92 feat(proxy): add transparent header sanitization 2026-07-24 18:19:44 +02:00
germondai fa2bcbb64f refactor(solvers): simplify optional results 2026-07-24 16:51:07 +02:00
germondai d1ebbddf38 feat(tiers): preserve raw response payloads 2026-07-24 14:26:31 +02:00
germondai defade4324 refactor(browser): simplify optional pool state 2026-07-24 12:08:55 +02:00