External/wayback-machine-downloader
1
0
Fork 0
mirror of https://github.com/StrawberryMaster/wayback-machine-downloader.git synced 2025-12-18 02:06:35 +00:00
Code Issues Packages Projects Releases Wiki Activity
397 Commits 1 Branch 17 Tags
Commit Graph

397 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Oleg Pudeyev
aab9a49509 Get rid of assigned but unused variable warnings under ruby 2.4 2017-06-03 17:00:50 -04:00
Oleg Pudeyev
e6157c21b9 Parens are required before * when used for splatting. 
https://stackoverflow.com/questions/41821628/ruby-how-can-i-kill-warning-interpreted-as-argument-prefix
 2017-06-03 16:59:08 -04:00
Oleg Pudeyev
43ab0b2f48 Fix test name so that it does not shadow another test 2017-06-03 16:54:51 -04:00
Oleg Pudeyev
e73a88ab56 File.exists? causes warning is ruby 2.4.1, use exist? 2017-06-03 16:53:37 -04:00
Oleg Pudeyev
d926f965f9 Add exact_match option. 
With this option set, Wayback Machine Downloader
will only look for snapshots matching the exact base_url
passed in rather than base_url and its children.

This is useful when trying to download a single file
rather than mirroring a site.
 2017-03-15 17:58:05 -04:00
Oleg Pudeyev
65b1948517 Avoid interleaving status output with file listing. 
Before:

[
Getting snapshot pages.. found 1 snaphots to consider.

{"file_url":"http://www.trackpedia.com:80/forums/archive/index.php/f-115.html","timestamp":20131221124252,"file_id":"forums/archive/index.php/f-115.html"},
]

After:

Getting snapshot pages.. found 1 snaphots to consider.

[
{"file_url":"http://www.trackpedia.com:80/forums/archive/index.php/f-115.html","timestamp":20131221124252,"file_id":"forums/archive/index.php/f-115.html"},
]
 2017-03-15 17:19:34 -04:00
Oleg Pudeyev
6b8c1aa194 Remove list attribute from the downloader. 
Whether to list or download is a program option external to the downloader
 2017-03-15 17:12:43 -04:00
Oleg Pudeyev
ea73ed5ed6 Shorten some lines for readability 2017-03-15 17:10:16 -04:00
Oleg Pudeyev
6779971dc9 Fix whitespace 2017-03-15 17:08:40 -04:00
hartator
e95ade8079 Bump Gem version 2017-02-17 12:55:58 -06:00
hartator
6e3986b875 Merge branch 'pr/82' 2017-02-17 12:55:20 -06:00
hartator
4830913ed3 Add explicit variable current encoding 2017-02-17 12:54:12 -06:00
Ian Kirker
132e3fa5f8 Alters encoding of file_url to fix encoding incompatibilities 2017-02-15 11:01:26 +00:00
Ian Kirker
0a324016b8 Split encoding test into two tests 
(One for downloading, one for when files are already present)
 2017-02-15 10:58:40 +00:00
Ian Kirker
2fd50480ea Added prospective test for encoding conflict errors 2017-02-15 10:50:12 +00:00
insaner
5bd9fbffdd Update wayback_machine_downloader.rb 2017-01-24 05:03:07 -05:00
insaner
330c1888fb Update wayback_machine_downloader 2017-01-24 04:55:51 -05:00
hartator
63941f99dd Merge branch 'antoneliasson-master' 2016-11-14 18:20:42 -06:00
hartator
734521803a Bump Gem version 2016-11-14 18:20:16 -06:00
hartator
5195c36156 Remove Rubinius from Rubies following Travis CI issues 2016-11-14 18:20:06 -06:00
hartator
8d5be7a89e Fix compatibility with Ruby 1.9.x and proxies 2016-11-14 18:18:58 -06:00
Anton Eliasson
54bd5d3852 Support http(s)_proxy ENV variables 
Closes issue #65
 2016-10-31 17:49:30 +01:00
hartator
4177425a1b Fix Travis CI support of Rubinius by using precise supported version 2016-10-31 02:20:43 -05:00
hartator
6d7dbc13f6 Revert back to regular Rubinius version 2016-10-31 01:58:12 -05:00
hartator
ff05a2ab90 Fix Travis CI Ruby versions 2016-10-17 16:05:54 -05:00
hartator
ac759f3752 Replace Ruby 1.9.1 by 1.8.7 2016-10-17 13:26:36 -05:00
hartator
a3a02e0129 Add more Rubies to try out 2016-10-17 12:45:36 -05:00
hartator
7383bdfadc Bump Gem version 2016-10-17 12:42:05 -05:00
hartator
9fc60b5385 Require thread library explicitly for 1.9.x Ruby versions 2016-10-17 12:42:05 -05:00
hartator
1ea897d4e0 Align correctly CLI help texts 2016-10-03 08:45:57 -07:00
hartator
d254c5c541 Update file information to reflect new API selectors 2016-09-26 10:51:02 -07:00
hartator
c9be8ae945 Bump Gem version 2016-09-26 10:36:05 -07:00
hartator
c9e15636b1 Fix spelling in comment 2016-09-26 10:36:05 -07:00
hartator
5d1fefe252 Update tests to reflect new API selectors 2016-09-26 10:36:05 -07:00
hartator
fe96570c93 Fix spacing issues in CLI help text 2016-09-24 10:24:46 -07:00
hartator
c1d77259c9 Bump Gem version 2016-09-24 10:21:50 -07:00
hartator
7414c34d05 Add an additional request to archive.org API to get fresher snapshots 2016-09-24 10:21:17 -07:00
hartator
45f5f29826 Add new maximum snapshot page option 2016-09-24 10:06:58 -07:00
hartator
188b285848 Bump gem version 2016-09-24 10:06:35 -07:00
hartator
1b42f57e6f Add maximum snapshot page option 2016-09-24 10:06:27 -07:00
hartator
7eedc1a183 Get snapshot result page per page index 2016-09-24 10:04:57 -07:00
hartator
ec6aa17a07 Add option to set the maximum number of snapshot pages to consider 2016-09-23 18:19:37 -07:00
hartator
d37c56e7dc Force spacing for CLI help text 2016-09-23 18:18:34 -07:00
hartator
f7afca7d95 Bump Gem version 2016-09-18 12:24:29 -05:00
hartator
87eee70969 Show 404 archives when a resource had 200 response previously 2016-09-18 12:24:29 -05:00
hartator
6f10fc64ff Add Gem and build badges to Readme 2016-09-17 17:17:08 -05:00
hartator
a5b15d6d83 Fix formatting, spacing and spelling issues 2016-09-17 15:12:53 -05:00
hartator
21dd22f581 Disable gzip compression on API calls 2016-09-17 14:42:32 -05:00
hartator
2e7f8611ef Load early net/http library for Ruby 1.9.x 2016-09-17 14:08:29 -05:00
hartator
95eaa91715 Refactor archive API calls to own module 2016-09-17 13:37:17 -05:00