Felipe
4160ff5e4a
Bumping version
2025-06-18 18:05:31 +00:00
Eli Dickinson
c3c5b8446a
don’t append /* when —exact-url
2025-06-15 13:26:11 -04:00
Felipe
18357a77ed
Correct file path and sanitization in Windows
...
Not only we weren't normalizing the file directories, we were also agressively sanitizing incorrect characters, leading to some funny stuff on Windows. Fixes #16
2025-06-15 13:48:11 +00:00
Felipe
3fdfd70fc1
Bump version
2025-06-05 22:34:40 +00:00
Eli Dickinson
79cbb639e7
Fix bug with archive urls containing square brackets
2025-06-03 16:36:03 -04:00
Eli Dickinson
1681a12579
workaround for API only showing html files for some domains
...
See https://github.com/StrawberryMaster/wayback-machine-downloader/issues/6
2025-05-30 12:50:48 -04:00
Felipe
f38756dd76
Correction for downloaded data folder
...
if you downloaded content from example.org/*, it would be listed in a folder titled * instead of the sitename. See #6 (and thanks to elidickinson for pointing it out!)
2025-05-30 14:00:32 +00:00
Felipe
9452411e32
Added nil checks
2025-05-30 13:52:25 +00:00
Felipe
61e22cfe25
Bump versions
2025-05-27 18:10:09 +00:00
Felipe
183ed61104
Attempt at fixing --all
...
I honestly don't recall if this was implemented in the original code, and I'm guessing this worked at *some point* during this fork. It seems to work correctly now, however. See #6 and #11
2025-05-27 17:17:34 +00:00
Felipe
ab4324c0eb
Bumping to 2.3.6
2025-05-18 16:49:44 +00:00
Felipe
e28d7d578b
Experimental ability to rewrite URLs to local browsing
2025-05-18 16:48:50 +00:00
adampweb
1ef8c14c48
Removed unused variable from if condition
2025-05-11 10:57:36 +02:00
Felipe
917f4f8798
Bumping version
2025-04-30 13:05:30 +00:00
Felipe
4db13a7792
Fix --all-timestamps
...
we were accidentally removing the timestamp prefix from `file_id`, rendering that option useless in 2.3.4. This should again now. This will fix #4
2025-04-30 13:01:29 +00:00
Felipe
31d51728af
Bump version
2025-04-19 14:07:05 +00:00
Felipe
febffe5de4
Added support for resuming incomplete downloads
2025-04-19 13:40:14 +00:00
Felipe
27dd619aa4
gzip support
2025-04-19 13:07:07 +00:00
Felipe
0c701ee890
Fetching API calls sequentially
...
although the WM API is particularly wonky and this will not prevent all errors, this aligns better with what we have here.
2025-03-29 22:27:01 +00:00
Felipe
2243958643
Fixes in cases of too many redirects or files not found
2025-02-09 16:48:52 +00:00
Felipe
46450d7c20
Refactoring tidy_bytes, part 2
2025-02-09 16:47:29 +00:00
Felipe
019534794c
Taking care of empty responses
...
fixes "unexpected token at ''" appearing after fetching a list of snapshots
2025-02-09 16:24:02 +00:00
Felipe
9283f04a57
Added ability to download rewritten Wayback Archive files
2025-01-02 12:17:20 +00:00
Felipe
b38d528656
typo fix
2025-01-01 12:20:06 +00:00
Felipe
fdcb81f1a0
Refactoring
2024-12-31 16:50:50 +00:00
Felipe
4d5f187f15
Proper connection pool lifecycle management
2024-12-31 16:48:29 +00:00
Felipe
7de1c5a028
typo fix
2024-12-31 15:03:28 +00:00
Felipe
9bbb67cd90
More testing
2024-12-31 00:11:58 +00:00
Felipe
75617060d7
Workflow fixes, pt.3
...
You've gotta be squidding me. How did I never notice this
2024-12-05 12:11:16 +00:00
Felipe
02785b2eba
Workflow fixes, pt. 1
2024-12-05 12:00:44 +00:00
Felipe
d1b70d83b1
Minor cleanup
2024-12-05 11:53:38 +00:00
Felipe
45fa2be573
Significant refactoring
...
including extra config settings, a proper rate limit, and a logger. Fixes : #307 #291 #281 #269 and probably others too
2024-12-03 00:23:47 +00:00
Felipe
a3ac4e0341
Minor cleanup
2024-06-26 20:30:59 +00:00
Felipe
93a6fb3c1b
typo
2024-06-26 19:52:34 +00:00
Felipe
509d7034a1
Setting file modified time to value reported by Wayback Machine
...
Implements 937306712c564e5757d898feacc14fbabd10722d, fixes Maintain original creation/modified dates of files while downloading #174
2024-06-26 19:52:12 +00:00
Felipe
0a7752eb41
Minor cleanup
2024-06-26 19:47:19 +00:00
Felipe
cff30f529e
Using net:HTTP and decompressing gzip content
...
see https://github.com/ShiftaDeband/wayback-machine-downloader and bf6e33c2fe
2024-06-26 16:54:55 +00:00
Felipe
466228fee4
Refactoring the archive API
2024-06-26 16:53:08 +00:00
hartator
cf770c2e55
Bump gem version
2021-09-04 01:51:08 -05:00
Paul Wise
9da87bfa74
Make URI#open cross Ruby versions compatible
...
Inspired-by: commit 30475c5c9e1d92d63b75dc5f22a40dd16c1aa23a
2021-06-08 07:59:38 +08:00
hartator
83b4f880b1
Bump Gem version
2021-06-06 19:47:48 -05:00
hartator
30475c5c9e
Make URI#open cross Ruby versions compatible
2021-06-06 19:47:11 -05:00
Paul Wise
ba4ca60377
Do not emit a comma for the final item in JSON output
...
This avoids producing JSON that is not parsable.
2021-05-03 20:54:29 +08:00
Paul Wise
06e25957b6
Print progress messages to stderr when printing JSON
...
This avoids the messages breaking JSON parsing when
the output is being redirected to a file and parsed.
2021-05-03 20:52:28 +08:00
Paul Wise
ea15965d6d
Fix typos
...
Suggested-by: codespell, spellintian
2021-05-03 20:20:09 +08:00
Paul Wise
cd29f79fd0
Switch to the JSON output format for easier parsing
2021-05-03 17:44:56 +08:00
Paul Wise
afab72c894
Construct the cdx API query using a URI object
...
This avoids problems related to URL encoding.
Obsoletes: https://github.com/hartator/wayback-machine-downloader/pull/116
2021-05-03 17:44:36 +08:00
DessertArbiter
15edae6a92
updated deprecated calls, changed URI to https
2020-05-27 20:28:06 -04:00
hartator
0a2ae60378
Bump Gem version
2017-10-26 20:09:23 -05:00
hartator
c360d4621f
Merge branch 'master' of https://github.com/niklasjansson/wayback-machine-downloader into niklasjansson-master
2017-10-26 20:05:10 -05:00
