External/wayback-machine-downloader
1
0
Fork 0
mirror of https://github.com/StrawberryMaster/wayback-machine-downloader.git synced 2025-12-17 17:56:44 +00:00
Code Issues Packages Projects Releases Wiki Activity

Improve handling for wildcard URLs

Browse Source 
fixes #38
This commit is contained in:
Felipe 2025-11-15 12:45:34 +00:00 committed by GitHub
parent f7c0f1a964
commit 67685b781e
1 changed files with 12 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
lib/wayback_machine_downloader.rb
View File

@ -184,18 +184,29 @@ class WaybackMachineDownloader
end end
def backup_name def backup_name
url_to_process = @base_url.end_with?('/*') ? @base_url.chomp('/*') : @base_url url_to_process = @base_url
url_to_process = url_to_process.chomp('/*') if url_to_process&.end_with?('/*')
raw = if url_to_process.include?('//') raw = if url_to_process.include?('//')
url_to_process.split('/')[2] url_to_process.split('/')[2]
else else
url_to_process url_to_process
end end
# if it looks like a wildcard pattern, normalize to a safe host-ish name
if raw&.start_with?('*.')
raw = raw.sub(/\A\*\./, 'all-')
end
# sanitize for Windows (and safe cross-platform) to avoid ENOTDIR on mkdir (colon in host:port) # sanitize for Windows (and safe cross-platform) to avoid ENOTDIR on mkdir (colon in host:port)
if Gem.win_platform? if Gem.win_platform?
raw = raw.gsub(/[:*?"<>|]/, '_') raw = raw.gsub(/[:*?"<>|]/, '_')
raw = raw.gsub(/[ .]+\z/, '') raw = raw.gsub(/[ .]+\z/, '')
else
# still good practice to strip path separators (and maybe '*' for POSIX too)
raw = raw.gsub(/[\/:*?"<>|]/, '_')
end end
raw = 'site' if raw.nil? || raw.empty? raw = 'site' if raw.nil? || raw.empty?
raw raw
end end