mirror of
https://github.com/Yuvi9587/Kemono-Downloader.git
synced 2025-12-29 16:14:44 +00:00
Compare commits
22 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
8239fdb8f3 | ||
|
|
df8a305e81 | ||
|
|
090f1a638d | ||
|
|
871ee75a2a | ||
|
|
fea59c7903 | ||
|
|
a9b210b2ba | ||
|
|
ec94417569 | ||
|
|
0a902895a8 | ||
|
|
7217bfdb39 | ||
|
|
24880b5042 | ||
|
|
510ae5e1d1 | ||
|
|
65b4759bad | ||
|
|
6e993d88de | ||
|
|
cc3565b12b | ||
|
|
f8b150dfdb | ||
|
|
5f7b526852 | ||
|
|
b0a6c264e1 | ||
|
|
d9364f4f91 | ||
|
|
9cd48bb63a | ||
|
|
d0f11c4a06 | ||
|
|
26fa3b9bc1 | ||
|
|
f7c4d892a8 |
0
LinkMaker/hentai2read.py
Normal file
0
LinkMaker/hentai2read.py
Normal file
BIN
assets/Ko-fi.png
Normal file
BIN
assets/Ko-fi.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 2.9 KiB |
BIN
assets/buymeacoffee.png
Normal file
BIN
assets/buymeacoffee.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 3.2 KiB |
BIN
assets/patreon.png
Normal file
BIN
assets/patreon.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 978 B |
500
features.md
500
features.md
@@ -1,391 +1,159 @@
|
||||
<div>
|
||||
<h1>Kemono Downloader - Comprehensive Feature Guide</h1>
|
||||
<p>This guide provides a detailed overview of all user interface elements, input fields, buttons, popups, and functionalities available in the application.</p>
|
||||
<hr>
|
||||
|
||||
<h2><strong>1. URL Input (🔗)</strong></h2>
|
||||
<p>This is the primary input field where you specify the content you want to download.</p>
|
||||
|
||||
<p><strong>Functionality:</strong></p>
|
||||
<ul>
|
||||
<li><strong>Creator URL:</strong> A link to a creator's main page (e.g., https://kemono.su/patreon/user/12345). Downloads all posts from the creator.</li>
|
||||
<li><strong>Post URL:</strong> A direct link to a specific post (e.g., .../post/98765). Downloads only the specified post.</li>
|
||||
</ul>
|
||||
|
||||
<p><strong>Interaction with Other Features:</strong> The content of this field influences "Manga Mode" and "Page Range". "Page Range" is enabled only with a creator URL.</p>
|
||||
|
||||
<hr>
|
||||
|
||||
<h2><strong>2. Creator Selection & Update (🎨)</strong></h2>
|
||||
<p>The color palette emoji button opens the Creator Selection & Update dialog. This allows managing and downloading from a local creator database.</p>
|
||||
|
||||
<p><strong>Functionality:</strong></p>
|
||||
<ul>
|
||||
<li><strong>Creator Browser:</strong> Loads a list from <code>creators.json</code>. Search by name, service, or paste a URL to find creators.</li>
|
||||
<li><strong>Batch Selection:</strong> Select multiple creators and click "Add Selected" to add them to the batch download session.</li>
|
||||
<li><strong>Update Checker:</strong> Use a saved profile (.json) to download only new content based on previously fetched posts.</li>
|
||||
<li><strong>Post Fetching & Filtering:</strong> "Fetch Posts" loads post titles, allowing you to choose specific posts for download.</li>
|
||||
</ul>
|
||||
|
||||
<hr>
|
||||
|
||||
<h2><strong>3. Download Location Input (📁)</strong></h2>
|
||||
<p>This input defines the destination directory for downloaded files.</p>
|
||||
|
||||
<p><strong>Functionality:</strong></p>
|
||||
<ul>
|
||||
<li><strong>Manual Entry:</strong> Enter or paste the folder path.</li>
|
||||
<li><strong>Browse Button:</strong> Opens a system dialog to choose a folder.</li>
|
||||
<li><strong>Directory Creation:</strong> If the folder doesn't exist, the app can create it after user confirmation.</li>
|
||||
</ul>
|
||||
|
||||
<hr>
|
||||
|
||||
<h2><strong>4. Filter by Character(s) & Scope Button</strong></h2>
|
||||
<p>Used to download content for specific characters or series and organize them into subfolders.</p>
|
||||
|
||||
<p><strong>Input Field (Filter by Character(s)):</strong></p>
|
||||
<ul>
|
||||
<li>Enter comma-separated names (e.g., <code>Tifa, Aerith</code>).</li>
|
||||
<li>Group aliases using parentheses (e.g., <code>(Cloud, Zack)</code>).</li>
|
||||
<li>Names are matched against titles, filenames, or comments.</li>
|
||||
<li>If "Separate Folders by Known.txt" is enabled, the name becomes the subfolder name.</li>
|
||||
</ul>
|
||||
|
||||
<p><strong>Scope Button Modes:</strong></p>
|
||||
<ul>
|
||||
<li><strong>Filter: Title</strong> (default) – Match names in post titles only.</li>
|
||||
<li><strong>Filter: Files</strong> – Match names in filenames only.</li>
|
||||
<li><strong>Filter: Both</strong> – Try title match first, then filenames.</li>
|
||||
<li><strong>Filter: Comments</strong> – Try filenames first, then post comments if no match.</li>
|
||||
</ul>
|
||||
|
||||
<hr>
|
||||
|
||||
<h2><strong>5. Skip with Words & Scope Button</strong></h2>
|
||||
<p>Prevents downloading content based on keywords.</p>
|
||||
|
||||
<p><strong>Input Field (Skip with Words):</strong></p>
|
||||
<ul>
|
||||
<li>Enter comma-separated keywords (e.g., <code>WIP, sketch, preview</code>).</li>
|
||||
<li>Matching is case-insensitive.</li>
|
||||
<li>If a keyword matches, the file or post is skipped.</li>
|
||||
</ul>
|
||||
|
||||
<p><strong>Scope Button Modes:</strong></p>
|
||||
<ul>
|
||||
<li><strong>Scope: Posts</strong> (default) – Skips post if title contains a keyword.</li>
|
||||
<li><strong>Scope: Files</strong> – Skips individual files with keyword matches.</li>
|
||||
<li><strong>Scope: Both</strong> – Skips entire post if title matches, otherwise filters individual files.</li>
|
||||
</ul>
|
||||
</div>
|
||||
<div>
|
||||
<h2><strong>Filter File Section (Radio Buttons)</strong></h2>
|
||||
<p>This section uses a group of radio buttons to control the primary download mode, dictating which types of files are targeted. Only one of these modes can be active at a time.</p>
|
||||
|
||||
<ul>
|
||||
<li>
|
||||
<strong>All:</strong> Default mode. Downloads every file and attachment provided by the API, regardless of type.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Images/GIFs:</strong> Filters for common image formats (<code>.jpg</code>, <code>.png</code>, <code>.gif</code>, <code>.webp</code>), skipping non-image files.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Videos:</strong> Filters for common video formats like <code>.mp4</code>, <code>.webm</code>, and <code>.mov</code>, skipping all others.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Only Archives:</strong> Downloads only archive files (<code>.zip</code>, <code>.rar</code>). Disables "Compress to WebP" and unchecks "Skip Archives".
|
||||
</li>
|
||||
<li>
|
||||
<strong>Only Audio:</strong> Filters for common audio formats like <code>.mp3</code>, <code>.wav</code>, and <code>.flac</code>.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Only Links:</strong> Extracts external hyperlinks from post descriptions (e.g., Mega, Google Drive) and displays them in the log. Disables all download options.
|
||||
</li>
|
||||
<li>
|
||||
<strong>More:</strong> Opens the "More Options" dialog to download text-based content instead of media files.
|
||||
<ul>
|
||||
<li><strong>Scope:</strong> Choose to extract from post description or comments.</li>
|
||||
<li><strong>Export Format:</strong> Save text as PDF, DOCX, or TXT.</li>
|
||||
<li><strong>Single PDF:</strong> Optionally compile all text into one PDF.</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
<hr>
|
||||
|
||||
<h2><strong>Check Box Buttons</strong></h2>
|
||||
<p>These checkboxes provide additional toggles to refine the download behavior and enable special features.</p>
|
||||
|
||||
<ul>
|
||||
<li>
|
||||
<strong>⭐ Favorite Mode:</strong> Changes workflow to download from your personal favorites. Disables the URL input.
|
||||
<ul>
|
||||
<li><strong>Favorite Artists:</strong> Opens a dialog to select from your favorited creators.</li>
|
||||
<li><strong>Favorite Posts:</strong> Opens a dialog to select from your favorited posts on Kemono and Coomer.</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li>
|
||||
<strong>Skip Archives:</strong> When checked, archive files (<code>.zip</code>, <code>.rar</code>) are ignored. Disabled in "Only Archives" mode.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Download Thumbnail Only:</strong> Saves only thumbnail previews, not full-resolution files. Enables "Scan Content for Images".
|
||||
</li>
|
||||
<li>
|
||||
<strong>Scan Content for Images:</strong> Parses post HTML for embedded images not listed in the API. Looks for <code><img></code> tags and direct image links.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Compress to WebP:</strong> Converts large images (over 1.5 MB) to WebP format using the Pillow library for space-saving.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Keep Duplicates:</strong> Provides control over duplicate handling via the "Duplicate Handling Options" dialog.
|
||||
<ul>
|
||||
<li><strong>Skip by Hash:</strong> Default – skip identical files.</li>
|
||||
<li><strong>Keep Everything:</strong> Save all files regardless of duplication.</li>
|
||||
<li><strong>Limit:</strong> Set a limit on how many copies of the same file are saved. A limit of <code>0</code> means no limit.</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
<h2><strong>Folder Organization Checkboxes</strong></h2>
|
||||
<h1>Kemono Downloader - Comprehensive Feature Guide</h1>
|
||||
<p>This guide provides a detailed overview of all user interface elements, input fields, buttons, popups, and functionalities available in the application.</p>
|
||||
<hr>
|
||||
<h2>1. Core Concepts & Supported Sites</h2>
|
||||
<h3>URL Input (🔗)</h3>
|
||||
<p>This is the primary input field where you specify the content you want to download.</p>
|
||||
<p><strong>Supported URL Types:</strong></p>
|
||||
<ul>
|
||||
<li>
|
||||
<strong>Separate folders by Known.txt:</strong> Automatically organizes downloads into folders based on name matches.
|
||||
<ul>
|
||||
<li>Uses "Filter by Character(s)" input first, if available.</li>
|
||||
<li>Then checks names in <code>Known.txt</code>.</li>
|
||||
<li>Falls back to extracting from post title.</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li>
|
||||
<strong>Subfolder per post:</strong> Creates a unique folder per post, using the post’s title.
|
||||
<ul>
|
||||
<li>Prevents mixing files from multiple posts.</li>
|
||||
<li>Can be combined with Known.txt-based folders.</li>
|
||||
<li>Ensures uniqueness (e.g., <code>My Post Title_1</code>).</li>
|
||||
<li>Automatically removes empty folders.</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li>
|
||||
<strong>Date prefix:</strong> Enabled only with "Subfolder per post". Prepends the post date (e.g., <code>2025-08-03 My Post Title</code>) for chronological sorting.
|
||||
</li>
|
||||
<li><strong>Creator URL</strong>: A link to a creator's main page. Downloads all posts from that creator.</li>
|
||||
<li><strong>Post URL</strong>: A direct link to a specific post. Downloads only that single post.</li>
|
||||
<li><strong>Batch Command</strong>: Special keywords to trigger bulk downloading from a text file (see Batch Downloading section).</li>
|
||||
</ul>
|
||||
|
||||
<h2><strong>General Functionality Checkboxes</strong></h2>
|
||||
<p><strong>Supported Websites:</strong></p>
|
||||
<ul>
|
||||
<li>
|
||||
<strong>Use cookie:</strong> Enables login-based access via cookies.
|
||||
<li>Kemono (<code>kemono.su</code>, <code>kemono.party</code>, etc.)</li>
|
||||
<li>Coomer (<code>coomer.su</code>, <code>coomer.party</code>, etc.)</li>
|
||||
<li>Discord (via Kemono/Coomer API)</li>
|
||||
<li>Bunkr</li>
|
||||
<li>Erome</li>
|
||||
<li>Saint2.su</li>
|
||||
<li>nhentai</li>
|
||||
</ul>
|
||||
<hr>
|
||||
<h2>2. Main Download Controls & Inputs</h2>
|
||||
<h3>Download Location (📁)</h3>
|
||||
<p>This input defines the main folder where your files will be saved.</p>
|
||||
<ul>
|
||||
<li><strong>Browse Button</strong>: Opens a system dialog to choose a folder.</li>
|
||||
<li><strong>Directory Creation</strong>: If the folder doesn't exist, the app will ask for confirmation to create it.</li>
|
||||
</ul>
|
||||
<h3>Filter by Character(s) & Scope</h3>
|
||||
<p>Used to download content for specific characters or series and organize them into subfolders.</p>
|
||||
<ul>
|
||||
<li><strong>Input Field</strong>: Enter comma-separated names (e.g., <code>Tifa, Aerith</code>). Group aliases using parentheses for folder naming (e.g., <code>(Cloud, Zack)</code>).</li>
|
||||
<li><strong>Scope Button</strong>: Cycles through where to look for name matches:
|
||||
<ul>
|
||||
<li>Paste cookie string directly, or browse to select a <code>cookies.txt</code> file.</li>
|
||||
<li>Cookies are used in all authenticated API requests.</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li>
|
||||
<strong>Use Multithreading:</strong> Enables parallel downloading of posts.
|
||||
<ul>
|
||||
<li>Specify the number of worker threads (e.g., 10).</li>
|
||||
<li>Disabled for Manga Mode and Only Links mode.</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li>
|
||||
<strong>Show external links in log:</strong> Adds a secondary log that displays links (e.g., Mega, Dropbox) found in post text.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Manga/Comic mode:</strong> Sorts posts chronologically before download.
|
||||
<ul>
|
||||
<li>Ensures correct page order for comics/manga.</li>
|
||||
</ul>
|
||||
<strong>Scope Button (Name: ...):</strong> Controls filename style:
|
||||
<ul>
|
||||
<li><strong>Name: Post Title</strong> — e.g., <code>Chapter-1.jpg</code></li>
|
||||
<li><strong>Name: Date + Original</strong> — e.g., <code>2025-08-03_filename.png</code></li>
|
||||
<li><strong>Name: Date + Title</strong> — e.g., <code>2025-08-03_Chapter-1.jpg</code></li>
|
||||
<li><strong>Name: Title+G.Num</strong> — e.g., <code>Page_001.jpg</code></li>
|
||||
<li><strong>Name: Date Based</strong> — e.g., <code>001.jpg</code>, with optional prefix</li>
|
||||
<li><strong>Name: Post ID</strong> — uses unique post ID as filename</li>
|
||||
<li><strong>Filter: Title</strong>: Matches names in the post title.</li>
|
||||
<li><strong>Filter: Files</strong>: Matches names in the filenames.</li>
|
||||
<li><strong>Filter: Both</strong>: Checks the title first, then filenames.</li>
|
||||
<li><strong>Filter: Comments</strong>: Checks filenames first, then post comments.</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
<h2><strong>Start Download</strong></h2>
|
||||
<h3>Skip with Words & Scope</h3>
|
||||
<p>Prevents downloading content based on keywords or file size.</p>
|
||||
<ul>
|
||||
<li>
|
||||
<strong>Default State ("⬇️ Start Download"):</strong> When idle, this button gathers all current settings (URL, filters, checkboxes, etc.) and begins the download process via the DownloadManager.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Restore State:</strong> If an interrupted session is detected, the tooltip will indicate that starting a new download will discard previous session progress.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Update Mode (Phase 1 - "🔄 Check For Updates"):</strong> If a creator profile is loaded, clicking this button will fetch the creator's posts and compare them against your saved profile to identify new content.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Update Mode (Phase 2 - "⬇️ Start Download (X new)"):</strong> After new posts are found, the button text updates to reflect the number. Clicking it downloads only the new content.
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
<h2><strong>Pause / Resume Download</strong></h2>
|
||||
<ul>
|
||||
<li>
|
||||
<strong>While Downloading:</strong> The button toggles between:
|
||||
<li><strong>Input Field</strong>: Enter comma-separated keywords (e.g., <code>WIP, sketch, preview</code>).</li>
|
||||
<li><strong>Skip by Size</strong>: Enter a number in square brackets to skip any file <strong>smaller than</strong> that size in MB. For example, <code>WIP, [200]</code> skips files with "WIP" in the name AND any file smaller than 200 MB.</li>
|
||||
<li><strong>Scope Button</strong>: Cycles through where to apply keyword filters:
|
||||
<ul>
|
||||
<li><strong>"⏸️ Pause Download":</strong> Sets a <code>pause_event</code>, which tells all worker threads to halt their current task and wait.</li>
|
||||
<li><strong>"▶️ Resume Download":</strong> Clears the <code>pause_event</code>, allowing threads to resume their work.</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li>
|
||||
<strong>While Idle:</strong> The button is disabled.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Restore State:</strong> Changes to "🔄 Restore Download", which resumes the last session from saved data.
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
<h2><strong>Cancel & Reset UI</strong></h2>
|
||||
<ul>
|
||||
<li>
|
||||
<strong>Functionality:</strong> Stops downloads gracefully using a <code>cancellation_event</code>. Threads finish current tasks before shutting down.
|
||||
</li>
|
||||
<li>
|
||||
<strong>The Soft Reset:</strong> After cancellation is confirmed by background threads, the UI resets via the <code>download_finished</code> function. Input fields (URL and Download Location) are preserved for convenience.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Restore State:</strong> Changes to "🗑️ Discard Session", which deletes <code>session.json</code> and resets the UI.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Update State:</strong> Changes to "🗑️ Clear Selection", unloading the selected creator profile and returning to normal UI state.
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
<h2><strong>Error Button</strong></h2>
|
||||
<ul>
|
||||
<li>
|
||||
<strong>Error Counter:</strong> Shows how many files failed to download (e.g., <code>(3) Error</code>). Disabled if there are no errors.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Error Dialog:</strong> Clicking opens the "Files Skipped Due to Errors" dialog (defined in <code>ErrorFilesDialog.py</code>), listing all failed files.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Dialog Features:</strong>
|
||||
<ul>
|
||||
<li><strong>View Failed Files:</strong> Shows filenames and related post info.</li>
|
||||
<li><strong>Select and Retry:</strong> Retry selected failed files in a focused download session.</li>
|
||||
<li><strong>Export URLs:</strong> Save a <code>.txt</code> file of direct download links. Optionally include post metadata with each URL.</li>
|
||||
<li><strong>Scope: Posts</strong>: Skips the entire post if the title matches.</li>
|
||||
<li><strong>Scope: Files</strong>: Skips individual files if the filename matches.</li>
|
||||
<li><strong>Scope: Both</strong>: Checks the post title first, then individual files.</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
<h2><strong>"Known Area" and its Controls</strong></h2>
|
||||
<p>This section, located on the right side of the main window, manages your personal name database (<code>Known.txt</code>), which the app uses to organize downloads into subfolders.</p>
|
||||
|
||||
<h3>Remove Words from Name (✂️)</h3>
|
||||
<p>Enter comma-separated words to remove from final filenames (e.g., <code>patreon, [HD]</code>). This helps clean up file naming.</p>
|
||||
<hr>
|
||||
<h2>3. Primary Download Modes (Filter File Section)</h2>
|
||||
<p>This section uses radio buttons to set the main download mode. Only one can be active at a time.</p>
|
||||
<ul>
|
||||
<li>
|
||||
<strong>Open Known.txt:</strong> Opens the <code>Known.txt</code> file in your system's default text editor for manual editing, such as bulk changes or cleanup.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Search character input:</strong> A live search filter that hides any list items not matching your input text. Useful for quickly locating specific names in large lists.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Known Series/Characters Area:</strong> Displays all names currently stored in your <code>Known.txt</code>. These names are used when "Separate folders by Known.txt" is enabled.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Input at bottom & Add button:</strong> Type a new character or series name into the input field, then click "➕ Add". The app checks for duplicates, updates the list, and saves to <code>Known.txt</code>.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Add to Filter:</strong> Opens a dialog showing all entries from <code>Known.txt</code> with checkboxes. You can select one or more to auto-fill the "Filter by Character(s)" field at the top of the app.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Delete Selected:</strong> Select one or more entries from the list and click "🗑️ Delete Selected" to remove them from the app and update <code>Known.txt</code> accordingly.
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
<h2><strong>Other Buttons</strong></h2>
|
||||
<ul>
|
||||
<li>
|
||||
<strong>(?_?) mark button (Help Guide):</strong> Opens a multi-page help dialog with step-by-step instructions and explanations for all app features. Useful for new users.
|
||||
</li>
|
||||
<li>
|
||||
<strong>History Button:</strong> Opens the Download History dialog (from <code>DownloadHistoryDialog.py</code>), showing:
|
||||
<li><strong>All</strong>: Default mode. Downloads every file and attachment.</li>
|
||||
<li><strong>Images/GIFs</strong>: Downloads only common image formats.</li>
|
||||
<li><strong>Videos</strong>: Downloads only common video formats.</li>
|
||||
<li><strong>Only Archives</strong>: Downloads only <code>.zip</code>, <code>.rar</code>, etc.</li>
|
||||
<li><strong>Only Audio</strong>: Downloads only common audio formats.</li>
|
||||
<li><strong>Only Links</strong>: Extracts external hyperlinks (e.g., Mega, Google Drive) from post descriptions instead of downloading files. <strong>This mode unlocks special features</strong> (see section 6).</li>
|
||||
<li><strong>More</strong>: Opens a dialog to download text-based content.
|
||||
<ul>
|
||||
<li>Recently downloaded files</li>
|
||||
<li>The first few posts processed in the last session</li>
|
||||
</ul>
|
||||
This allows for a quick review of recent activity.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Settings Button:</strong> Opens the Settings dialog (from <code>FutureSettingsDialog.py</code>), where you can change app-wide settings such as theme (light/dark) and language.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Support Button:</strong> Opens the Support dialog (from <code>SupportDialog.py</code>), which includes developer info, source links, and donation platforms like Ko-fi or Patreon.
|
||||
</li>
|
||||
</ul>
|
||||
<h2><strong>Log Area Controls</strong></h2>
|
||||
<p>These controls are located around the main log panel and offer tools for managing downloads, configuring advanced options, and resetting the application.</p>
|
||||
|
||||
<ul>
|
||||
<li>
|
||||
<strong>Multi-part: OFF</strong><br>
|
||||
This button acts as both a status indicator and a configuration panel for multi-part downloading (parallel downloading of large files).
|
||||
<ul>
|
||||
<li><strong>Function:</strong> Opens the <code>Multipart Download Options</code> dialog (defined in <code>MultipartScopeDialog.py</code>).</li>
|
||||
<li><strong>Scope Options:</strong> Choose between "Videos Only", "Archives Only", or "Both".</li>
|
||||
<li><strong>Number of parts:</strong> Set how many simultaneous connections to use (2–16).</li>
|
||||
<li><strong>Minimum file size:</strong> Set a threshold (MB) below which files are downloaded normally.</li>
|
||||
<li><strong>Status:</strong> After applying settings, the button's text updates (e.g., <code>Multi-part: Both</code>); otherwise, it resets to <code>Multi-part: OFF</code>.</li>
|
||||
</ul>
|
||||
</li>
|
||||
|
||||
<li>
|
||||
<strong>👁️ Eye Emoji Button (Log View Toggle)</strong><br>
|
||||
Switches between two views in the log panel:
|
||||
<ul>
|
||||
<li><strong>👁️ Progress Log View:</strong> Shows real-time download progress, status messages, and errors.</li>
|
||||
<li><strong>🚫 Missed Character View:</strong> Displays names detected in posts that didn’t match the current filter — useful for updating <code>Known.txt</code>.</li>
|
||||
</ul>
|
||||
</li>
|
||||
|
||||
<li>
|
||||
<strong>Reset Button</strong><br>
|
||||
Performs a full "soft reset" of the UI when the application is idle.
|
||||
<ul>
|
||||
<li>Clears all inputs (except saved Download Location)</li>
|
||||
<li>Resets checkboxes, buttons, and logs</li>
|
||||
<li>Clears counters, queues, and restores the UI to its default state</li>
|
||||
<li><strong>Note:</strong> This is different from <em>Cancel & Reset UI</em>, which halts active downloads</li>
|
||||
<li><strong>Scope</strong>: Choose to extract text from the post description or comments.</li>
|
||||
<li><strong>Export Format</strong>: Save as PDF, DOCX, or TXT.</li>
|
||||
<li><strong>Single PDF</strong>: Compile all text from the session into one consolidated PDF file.</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
<h3><strong>The Progress Log and "Only Links" Mode Controls</strong></h3>
|
||||
|
||||
<hr>
|
||||
<h2>4. Advanced Features & Toggles (Checkboxes)</h2>
|
||||
<h3>Folder Organization</h3>
|
||||
<ul>
|
||||
<li>
|
||||
<strong>Standard Mode (Progress Log)</strong><br>
|
||||
This is the default behavior. The <code>main_log_output</code> field displays:
|
||||
<li><strong>Separate folders by Known.txt</strong>: Automatically organizes downloads into subfolders based on name matches from your <code>Known.txt</code> list or the "Filter by Character(s)" input.</li>
|
||||
<li><strong>Subfolder per post</strong>: Creates a unique folder for each post, named after the post's title. This prevents files from different posts from mixing.</li>
|
||||
<li><strong>Date prefix</strong>: (Only available with "Subfolder per post") Prepends the post date to the folder name (e.g., <code>2025-08-03 My Post Title</code>) for chronological sorting.</li>
|
||||
</ul>
|
||||
<h3>Special Modes</h3>
|
||||
<ul>
|
||||
<li><strong>⭐ Favorite Mode</strong>: Switches the UI to download from your personal favorites list instead of using the URL input.</li>
|
||||
<li><strong>Manga/Comic mode</strong>: Sorts a creator's posts from oldest to newest before downloading, ensuring correct page order. A scope button appears to control the filename style (e.g., using post title, date, or a global number).</li>
|
||||
</ul>
|
||||
<h3>File Handling</h3>
|
||||
<ul>
|
||||
<li><strong>Skip Archives</strong>: Ignores <code>.zip</code> and <code>.rar</code> files during downloads.</li>
|
||||
<li><strong>Download Thumbnail Only</strong>: Saves only the small preview images instead of full-resolution files.</li>
|
||||
<li><strong>Scan Content for Images</strong>: Parses post HTML to find embedded images that may not be listed in the API data.</li>
|
||||
<li><strong>Compress to WebP</strong>: Converts large images (over 1.5 MB) to the space-saving WebP format.</li>
|
||||
<li><strong>Keep Duplicates</strong>: Opens a dialog to control how duplicate files are handled (skip by default, keep all, or keep a specific number of copies).</li>
|
||||
</ul>
|
||||
<h3>General Functionality</h3>
|
||||
<ul>
|
||||
<li><strong>Use cookie</strong>: Enables login-based access. You can paste a cookie string or browse for a <code>cookies.txt</code> file.</li>
|
||||
<li><strong>Use Multithreading</strong>: Enables parallel processing of posts for faster downloads. You can set the number of concurrent worker threads.</li>
|
||||
<li><strong>Show external links in log</strong>: Opens a secondary log panel that displays external links found in post descriptions.</li>
|
||||
</ul>
|
||||
<hr>
|
||||
<h2>5. Specialized Downloaders & Batch Mode</h2>
|
||||
<h3>Discord Features</h3>
|
||||
<ul>
|
||||
<li>When a Discord URL is entered, a <strong>Scope</strong> button appears.
|
||||
<ul>
|
||||
<li>Post processing steps</li>
|
||||
<li>Download/skipped file notifications</li>
|
||||
<li>Error messages</li>
|
||||
<li>Session summaries</li>
|
||||
<li><strong>Scope: Files</strong>: Downloads all files from the channel/server.</li>
|
||||
<li><strong>Scope: Messages</strong>: Saves the entire message history of the channel/server as a formatted PDF.</li>
|
||||
</ul>
|
||||
</li>
|
||||
|
||||
<li>
|
||||
<strong>"Only Links" Mode</strong><br>
|
||||
When enabled, the log panel switches modes and reveals new controls.
|
||||
<li>A <strong>"Save as PDF"</strong> button also appears as a shortcut for the message saving feature.</li>
|
||||
</ul>
|
||||
<h3>Batch Downloading (<code>nhentai</code> & <code>saint2.su</code>)</h3>
|
||||
<p>This feature allows you to download hundreds of galleries or videos from a simple text file.</p>
|
||||
<ol>
|
||||
<li>In the <code>appdata</code> folder, create <code>nhentai.txt</code> or <code>saint2.su.txt</code>.</li>
|
||||
<li>Add one full URL per line to the corresponding file.</li>
|
||||
<li>In the app's URL input, type either <code>nhentai.net</code> or <code>saint2.su</code> and click "Start Download".</li>
|
||||
<li>The app will read the file and process every URL in the queue.</li>
|
||||
</ol>
|
||||
<hr>
|
||||
<h2>6. "Only Links" Mode: Extraction & Direct Download</h2>
|
||||
<p>When you select the <strong>"Only Links"</strong> radio button, the application's behavior changes significantly.</p>
|
||||
<ul>
|
||||
<li><strong>Link Extraction</strong>: Instead of downloading files, the main log panel will fill with all external links found (Mega, Google Drive, Dropbox, etc.).</li>
|
||||
<li><strong>Export Links</strong>: An "Export Links" button appears, allowing you to save the full list of extracted URLs to a <code>.txt</code> file.</li>
|
||||
<li><strong>Direct Cloud Download</strong>: A <strong>"Download"</strong> button appears next to the export button.
|
||||
<ul>
|
||||
<li><strong>📜 Extracted Links Log:</strong> Replaces progress info with a list of found external links (e.g., Mega, Dropbox).</li>
|
||||
<li><strong>Export Links Button:</strong> Saves the extracted links to a <code>.txt</code> file.</li>
|
||||
<li><strong>Download Button:</strong> Opens the <code>Download Selected External Links</code> dialog (from <code>DownloadExtractedLinksDialog.py</code>), where you can:
|
||||
<ul>
|
||||
<li>View all supported external links</li>
|
||||
<li>Select which ones to download</li>
|
||||
<li>Begin download directly from cloud services</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Links View Button:</strong> Toggles log display between:
|
||||
<ul>
|
||||
<li><strong>🔗 Links View:</strong> Shows all extracted links</li>
|
||||
<li><strong>⬇️ Progress View:</strong> Shows download progress from external services (e.g., Mega)</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li>Clicking this opens a new dialog listing all supported cloud links (Mega, G-Drive, Dropbox).</li>
|
||||
<li>You can select which files you want to download from this list.</li>
|
||||
<li>The application will then download the selected files directly from the cloud service to your chosen download location.</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
<hr>
|
||||
<h2>7. Session & Process Management</h2>
|
||||
<h3>Main Action Buttons</h3>
|
||||
<ul>
|
||||
<li><strong>Start Download</strong>: Begins the download process. This button's text changes contextually (e.g., "Extract Links", "Check for Updates").</li>
|
||||
<li><strong>Pause / Resume</strong>: Pauses or resumes the ongoing download. When paused, you can safely change some settings.</li>
|
||||
<li><strong>Cancel & Reset UI</strong>: Stops the current download and performs a soft reset of the UI, preserving your URL and download location.</li>
|
||||
</ul>
|
||||
<h3>Restore Interrupted Download</h3>
|
||||
<p>If the application is closed unexpectedly during a download, it will save its progress.</p>
|
||||
<ul>
|
||||
<li>On the next launch, the UI will be pre-filled with the settings from the interrupted session.</li>
|
||||
<li>The <strong>Pause</strong> button will change to <strong>"🔄 Restore Download"</strong>. Clicking it will resume the download exactly where it left off, skipping already processed posts.</li>
|
||||
<li>The <strong>Cancel</strong> button will change to <strong>"🗑️ Discard Session"</strong>, allowing you to clear the saved state and start fresh.</li>
|
||||
</ul>
|
||||
<h3>Other UI Controls</h3>
|
||||
<ul>
|
||||
<li><strong>Error Button</strong>: Shows a count of failed files. Clicking it opens a dialog where you can view, export, or retry the failed downloads.</li>
|
||||
<li><strong>History Button</strong>: Shows a log of recently downloaded files and processed posts.</li>
|
||||
<li><strong>Settings Button</strong>: Opens the settings dialog where you can change the theme, language, and <strong>check for application updates</strong>.</li>
|
||||
<li><strong>Support Button</strong>: Opens a dialog with links to the project's source code and developer support pages.</li>
|
||||
</ul>
|
||||
|
||||
203
readme.md
203
readme.md
@@ -1,162 +1,115 @@
|
||||
<h1 align="center">Kemono Downloader </h1>
|
||||
<h1 align="center">Kemono Downloader</h1>
|
||||
|
||||
<div align="center">
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<td align="center">
|
||||
<img src="Read/Read.png" alt="Default Mode" width="400"><br>
|
||||
<strong>Default</strong>
|
||||
</td>
|
||||
<td align="center">
|
||||
<img src="Read/Read1.png" alt="Favorite Mode" width="400"><br>
|
||||
<strong>Favorite Mode</strong>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center">
|
||||
<img src="Read/Read2.png" alt="Single Post" width="400"><br>
|
||||
<strong>Single Post</strong>
|
||||
</td>
|
||||
<td align="center">
|
||||
<img src="Read/Read3.png" alt="Manga/Comic Mode" width="400"><br>
|
||||
<strong>Manga/Comic Mode</strong>
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<table>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td align="center">
|
||||
<img src="Read/Read.png" alt="Default Mode" width="400"><br>
|
||||
<strong>Default Mode</strong>
|
||||
</td>
|
||||
<td align="center">
|
||||
<img src="Read/Read1.png" alt="Favorite Mode" width="400"><br>
|
||||
<strong>Favorite Mode</strong>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center">
|
||||
<img src="Read/Read2.png" alt="Single Post" width="400"><br>
|
||||
<strong>Single Post</strong>
|
||||
</td>
|
||||
<td align="center">
|
||||
<img src="Read/Read3.png" alt="Renaming Mode" width="400"><br>
|
||||
<strong>Renaming Mode</strong>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
---
|
||||
<hr>
|
||||
|
||||
A powerful, feature-rich GUI application for downloading content from **[Kemono.su](https://kemono.su)** (and its mirrors like kemono.party) and **[Coomer.party](https://coomer.party)** (and its mirrors like coomer.su).
|
||||
|
||||
Built with PyQt5, this tool is designed for users who want deep filtering capabilities, customizable folder structures, efficient downloads, and intelligent automation — all within a modern and user-friendly graphical interface.
|
||||
<p>A powerful, feature-rich GUI application for downloading content from a wide array of sites, including <strong>Kemono</strong>, <strong>Coomer</strong>, <strong>Bunkr</strong>, <strong>Erome</strong>, <strong>Saint2.su</strong>, and <strong>nhentai</strong>.</p>
|
||||
<p>Built with PyQt5, this tool is designed for users who want deep filtering capabilities, customizable folder structures, efficient downloads, and intelligent automation — all within a modern and user-friendly graphical interface.</p>
|
||||
|
||||
<div align="center">
|
||||
|
||||
[](features.md)
|
||||
[](LICENSE)
|
||||
[](note.md)
|
||||
|
||||
<a href="features.md"><img src="https://img.shields.io/badge/📚%20Full%20Feature%20List-FFD700?style=for-the-badge&logoColor=black&color=FFD700" alt="Full Feature List"></a>
|
||||
<a href="LICENSE"><img src="https://img.shields.io/badge/📝%20License-90EE90?style=for-the-badge&logoColor=black&color=90EE90" alt="License"></a>
|
||||
</div>
|
||||
|
||||
<h2><strong>Core Capabilities Overview</strong></h2>
|
||||
|
||||
<h3><strong>High-Performance Downloading</strong></h3>
|
||||
<h2>Core Capabilities Overview</h2>
|
||||
<h3>High-Performance & Resilient Downloading</h3>
|
||||
<ul>
|
||||
<li><strong>Multi-threading:</strong> Processes multiple posts simultaneously to greatly accelerate downloads from large creator profiles.</li>
|
||||
<li><strong>Multi-part Downloading:</strong> Splits large files into chunks and downloads them in parallel to maximize speed.</li>
|
||||
<li><strong>Resilience:</strong> Supports pausing, resuming, and restoring downloads after crashes or interruptions.</li>
|
||||
<li><strong>Session Management:</strong> Supports pausing, resuming, and <strong>restoring downloads</strong> after crashes or interruptions, so you never lose your progress.</li>
|
||||
</ul>
|
||||
|
||||
<h3><strong>Advanced Filtering & Content Control</strong></h3>
|
||||
<h3>Expanded Site Support</h3>
|
||||
<ul>
|
||||
<li><strong>Direct Downloading:</strong> Full support for Kemono, Coomer, Bunkr, Erome, Saint2.su, and nhentai.</li>
|
||||
<li><strong>Batch Mode:</strong> Download hundreds of URLs at once from <code>nhentai.txt</code> or <code>saint2.su.txt</code> files.</li>
|
||||
<li><strong>Discord Support:</strong> Download files or save entire channel histories as PDFs directly through the API.</li>
|
||||
</ul>
|
||||
<h3>Advanced Filtering & Content Control</h3>
|
||||
<ul>
|
||||
<li><strong>Content Type Filtering:</strong> Select whether to download all files or limit to images, videos, audio, or archives only.</li>
|
||||
<li><strong>Keyword Skipping:</strong> Automatically skips posts or files containing certain keywords (e.g., "WIP", "sketch").</li>
|
||||
<li><strong>Character Filtering:</strong> Restricts downloads to posts that match specific character or series names.</li>
|
||||
<li><strong>Skip by Size:</strong> Avoid small files by setting a minimum size threshold in MB (e.g., <code>[200]</code>).</li>
|
||||
<li><strong>Character Filtering:</strong> Restricts downloads to posts that match specific character or series names, with scope controls for title, filename, or comments.</li>
|
||||
</ul>
|
||||
|
||||
<h3><strong>File Organization & Renaming</strong></h3>
|
||||
<h3>Intelligent File Organization</h3>
|
||||
<ul>
|
||||
<li><strong>Automated Subfolders:</strong> Automatically organizes downloaded files into subdirectories based on character names or per post.</li>
|
||||
<li><strong>Advanced File Renaming:</strong> Flexible renaming options, especially in Manga Mode, including:
|
||||
<ul>
|
||||
<li><strong>Post Title:</strong> Uses the post's title (e.g., <code>Chapter-One.jpg</code>).</li>
|
||||
<li><strong>Date + Original Name:</strong> Prepends the publication date to the original filename.</li>
|
||||
<li><strong>Date + Title:</strong> Combines the date with the post title.</li>
|
||||
<li><strong>Sequential Numbering (Date Based):</strong> Simple sequence numbers (e.g., <code>001.jpg</code>, <code>002.jpg</code>).</li>
|
||||
<li><strong>Title + Global Numbering:</strong> Uses post title with a globally incrementing number across the session.</li>
|
||||
<li><strong>Post ID:</strong> Names files using the post’s unique ID.</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Advanced File Renaming:</strong> Flexible renaming options, especially in Manga Mode, including by post title, date, sequential numbering, or post ID.</li>
|
||||
<li><strong>Filename Cleaning:</strong> Automatically removes unwanted text from filenames.</li>
|
||||
</ul>
|
||||
|
||||
<h3><strong>Specialized Modes</strong></h3>
|
||||
<h3>Specialized Modes</h3>
|
||||
<ul>
|
||||
<li><strong>Manga/Comic Mode:</strong> Sorts posts chronologically before downloading to ensure pages appear in the correct sequence.</li>
|
||||
<li><strong>Renaming Mode:</strong> Sorts posts chronologically before downloading to ensure pages appear in the correct sequence.</li>
|
||||
<li><strong>Favorite Mode:</strong> Connects to your account and downloads from your favorites list (artists or posts).</li>
|
||||
<li><strong>Link Extraction Mode:</strong> Extracts external links from posts for export or targeted downloading.</li>
|
||||
<li><strong>Link Extraction Mode:</strong> Extracts external links (Mega, Google Drive) from posts for export or <strong>direct in-app downloading</strong>.</li>
|
||||
<li><strong>Text Extraction Mode:</strong> Saves post descriptions or comment sections as <code>PDF</code>, <code>DOCX</code>, or <code>TXT</code> files.</li>
|
||||
</ul>
|
||||
|
||||
<h3><strong>Utility & Advanced Features</strong></h3>
|
||||
<h3>Utility & Advanced Features</h3>
|
||||
<ul>
|
||||
<li><strong>In-App Updater:</strong> Check for new versions directly from the settings menu.</li>
|
||||
<li><strong>Cookie Support:</strong> Enables access to subscriber-only content via browser session cookies.</li>
|
||||
<li><strong>Duplicate Detection:</strong> Prevents saving duplicate files using content-based comparison, with configurable limits.</li>
|
||||
<li><strong>Image Compression:</strong> Automatically converts large images to <code>.webp</code> to reduce disk usage.</li>
|
||||
<li><strong>Creator Management:</strong> Built-in creator browser and update checker for downloading only new posts from saved profiles.</li>
|
||||
<li><strong>Error Handling:</strong> Tracks failed downloads and provides a retry dialog with options to export or redownload missing files.</li>
|
||||
</ul>
|
||||
|
||||
## 💻 Installation
|
||||
|
||||
### Requirements
|
||||
|
||||
- Python 3.6 or higher
|
||||
- pip (Python package installer)
|
||||
|
||||
### Install Dependencies
|
||||
|
||||
```bash
|
||||
pip install PyQt5 requests Pillow mega.py fpdf2 python-docx
|
||||
```
|
||||
|
||||
### Running the Application
|
||||
Navigate to the application's directory in your terminal and run:
|
||||
```bash
|
||||
python main.py
|
||||
```
|
||||
|
||||
### Optional Setup
|
||||
- **Main Inputs:**
|
||||
- Place your `cookies.txt` in the root directory (if using cookies).
|
||||
- Prepare your `Known.txt` and `creators.json` in the same directory for advanced filtering and selection features.
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### AttributeError: module 'asyncio' has no attribute 'coroutine'
|
||||
|
||||
If you encounter an error message similar to:
|
||||
```
|
||||
AttributeError: module 'asyncio' has no attribute 'coroutine'. Did you mean: 'coroutines'?
|
||||
```
|
||||
This usually means that a dependency, often `tenacity` (used by `mega.py`), is an older version that's incompatible with your Python version (typically Python 3.10+).
|
||||
|
||||
To fix this, activate your virtual environment and run the following commands to upgrade the libraries:
|
||||
|
||||
```bash
|
||||
pip install --upgrade tenacity
|
||||
pip install --upgrade mega.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Contribution
|
||||
|
||||
Feel free to fork this repo and submit pull requests for bug fixes, new features, or UI improvements!
|
||||
|
||||
---
|
||||
|
||||
## License
|
||||
|
||||
This project is under the MIT Licence
|
||||
|
||||
## Star History
|
||||
|
||||
<h2>💻 Installation</h2>
|
||||
<h3>Requirements</h3>
|
||||
<ul>
|
||||
<li>Python 3.6 or higher</li>
|
||||
<li>pip (Python package installer)</li>
|
||||
</ul>
|
||||
<h3>Install Dependencies</h3>
|
||||
<pre><code>pip install PyQt5 requests cloudscraper Pillow fpdf2 python-docx
|
||||
</code></pre>
|
||||
<h3>Running the Application</h3>
|
||||
<p>Navigate to the application's directory in your terminal and run:</p>
|
||||
<pre><code>python main.py
|
||||
</code></pre>
|
||||
<h2>Contribution</h2>
|
||||
<p>Feel free to fork this repo and submit pull requests for bug fixes, new features, or UI improvements!</p>
|
||||
<h2>License</h2>
|
||||
<p>This project is under the MIT Licence</p>
|
||||
<h2>Star History</h2>
|
||||
<table align="center" style="border-collapse: collapse; border: none; margin-left: auto; margin-right: auto;">
|
||||
<tr>
|
||||
<td align="center" valign="middle" style="padding: 10px; border: none;">
|
||||
<a href="https://www.star-history.com/#Yuvi9587/Kemono-Downloader&Date">
|
||||
<img src="https://api.star-history.com/svg?repos=Yuvi9587/Kemono-Downloader&type=Date" alt="Star History Chart" width="650">
|
||||
</a>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td align="center" valign="middle" style="padding: 10px; border: none;">
|
||||
<a href="https://www.star-history.com/#Yuvi9587/Kemono-Downloader&Date">
|
||||
<img src="https://api.star-history.com/svg?repos=Yuvi9587/Kemono-Downloader&type=Date" alt="Star History Chart" width="650">
|
||||
</a>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<p align="center">
|
||||
<a href="https://buymeacoffee.com/yuvi9587">
|
||||
<img src="https://img.shields.io/badge/🍺%20Buy%20Me%20a%20Coffee-FFCCCB?style=for-the-badge&logoColor=black&color=FFDD00" alt="Buy Me a Coffee">
|
||||
<img src="https://img.shields.io/badge/🍺%20Buy%20Me%20a%20Coffee-FFCCCB?style=for-the-badge&logoColor=black&color=FFDD00" alt="Buy Me a Coffee">
|
||||
</a>
|
||||
</p>
|
||||
|
||||
|
||||
@@ -6,9 +6,9 @@ We are committed to maintaining and improving the Kemono Downloader. For the bes
|
||||
|
||||
| Version | Supported Status |
|
||||
| -------------- | ------------------------------------ |
|
||||
| >= 5.0.0 | :white_check_mark: Actively Supported |
|
||||
| 4.0.0 - 4.x.x | :warning: Supported (Limited Features) |
|
||||
| < 4.0.0 | :x: End of Life (EOL) |
|
||||
| >= 7.0.0 | :white_check_mark: Actively Supported |
|
||||
| 6.0.0 - 6.x.x | :warning: Supported (Limited Features) |
|
||||
| < 5.0.0 | :x: End of Life (EOL) |
|
||||
|
||||
Users are encouraged to update to **v5.0.0 or newer** versions.
|
||||
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
# --- Application Metadata ---
|
||||
CONFIG_ORGANIZATION_NAME = "KemonoDownloader"
|
||||
CONFIG_APP_NAME_MAIN = "ApplicationSettings"
|
||||
CONFIG_APP_NAME_TOUR = "ApplicationTour"
|
||||
@@ -9,7 +8,7 @@ STYLE_ORIGINAL_NAME = "original_name"
|
||||
STYLE_DATE_BASED = "date_based"
|
||||
STYLE_DATE_POST_TITLE = "date_post_title"
|
||||
STYLE_POST_TITLE_GLOBAL_NUMBERING = "post_title_global_numbering"
|
||||
STYLE_POST_ID = "post_id" # Add this line
|
||||
STYLE_POST_ID = "post_id"
|
||||
MANGA_DATE_PREFIX_DEFAULT = ""
|
||||
|
||||
# --- Download Scopes ---
|
||||
@@ -48,6 +47,8 @@ MAX_PARTS_FOR_MULTIPART_DOWNLOAD = 15
|
||||
# --- UI and Settings Keys (for QSettings) ---
|
||||
TOUR_SHOWN_KEY = "neverShowTourAgainV19"
|
||||
MANGA_FILENAME_STYLE_KEY = "mangaFilenameStyleV1"
|
||||
MANGA_CUSTOM_FORMAT_KEY = "mangaCustomFormatV1"
|
||||
MANGA_CUSTOM_DATE_FORMAT_KEY = "mangaCustomDateFormatV1"
|
||||
SKIP_WORDS_SCOPE_KEY = "skipWordsScopeV1"
|
||||
ALLOW_MULTIPART_DOWNLOAD_KEY = "allowMultipartDownloadV1"
|
||||
USE_COOKIE_KEY = "useCookieV1"
|
||||
@@ -60,6 +61,12 @@ DOWNLOAD_LOCATION_KEY = "downloadLocationV1"
|
||||
RESOLUTION_KEY = "window_resolution"
|
||||
UI_SCALE_KEY = "ui_scale_factor"
|
||||
SAVE_CREATOR_JSON_KEY = "saveCreatorJsonProfile"
|
||||
DATE_PREFIX_FORMAT_KEY = "datePrefixFormatV1"
|
||||
AUTO_RETRY_ON_FINISH_KEY = "auto_retry_on_finish"
|
||||
FETCH_FIRST_KEY = "fetchAllPostsFirst"
|
||||
DISCORD_TOKEN_KEY = "discord/token"
|
||||
|
||||
POST_DOWNLOAD_ACTION_KEY = "postDownloadAction"
|
||||
|
||||
# --- UI Constants and Identifiers ---
|
||||
HTML_PREFIX = "<!HTML!>"
|
||||
@@ -81,7 +88,7 @@ VIDEO_EXTENSIONS = {
|
||||
'.mpg', '.m4v', '.3gp', '.ogv', '.ts', '.vob'
|
||||
}
|
||||
ARCHIVE_EXTENSIONS = {
|
||||
'.zip', '.rar', '.7z', '.tar', '.gz', '.bz2'
|
||||
'.zip', '.rar', '.7z', '.tar', '.gz', '.bz2', '.bin'
|
||||
}
|
||||
AUDIO_EXTENSIONS = {
|
||||
'.mp3', '.wav', '.aac', '.flac', '.ogg', '.wma', '.m4a', '.opus',
|
||||
@@ -111,10 +118,13 @@ CREATOR_DOWNLOAD_DEFAULT_FOLDER_IGNORE_WORDS = {
|
||||
"may", "jun", "june", "jul", "july", "aug", "august", "sep", "september",
|
||||
"oct", "october", "nov", "november", "dec", "december",
|
||||
"mon", "monday", "tue", "tuesday", "wed", "wednesday", "thu", "thursday",
|
||||
"fri", "friday", "sat", "saturday", "sun", "sunday"
|
||||
"fri", "friday", "sat", "saturday", "sun", "sunday", "Pack", "tier", "spoiler",
|
||||
|
||||
|
||||
# add more according to need
|
||||
}
|
||||
|
||||
# --- Duplicate Handling Modes ---
|
||||
DUPLICATE_HANDLING_HASH = "hash"
|
||||
DUPLICATE_HANDLING_KEEP_ALL = "keep_all"
|
||||
DUPLICATE_HANDLING_KEEP_ALL = "keep_all"
|
||||
STYLE_CUSTOM = "custom"
|
||||
207
src/core/Hentai2read_client.py
Normal file
207
src/core/Hentai2read_client.py
Normal file
@@ -0,0 +1,207 @@
|
||||
# src/core/Hentai2read_client.py
|
||||
|
||||
import re
|
||||
import os
|
||||
import time
|
||||
import cloudscraper
|
||||
from bs4 import BeautifulSoup
|
||||
from urllib.parse import urljoin
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
import queue
|
||||
|
||||
def run_hentai2read_download(start_url, output_dir, progress_callback, overall_progress_callback, check_pause_func):
|
||||
"""
|
||||
Orchestrates the download process using a producer-consumer model.
|
||||
The main thread scrapes image URLs and puts them in a queue.
|
||||
A pool of worker threads consumes from the queue to download images concurrently.
|
||||
"""
|
||||
scraper = cloudscraper.create_scraper()
|
||||
|
||||
try:
|
||||
progress_callback(" [Hentai2Read] Scraping series page for all metadata...")
|
||||
top_level_folder_name, chapters_to_process = _get_series_metadata(start_url, progress_callback, scraper)
|
||||
|
||||
if not chapters_to_process:
|
||||
progress_callback("❌ No chapters found to download. Aborting.")
|
||||
return 0, 0
|
||||
|
||||
total_chapters = len(chapters_to_process)
|
||||
overall_progress_callback(total_chapters, 0)
|
||||
|
||||
total_downloaded_count = 0
|
||||
total_skipped_count = 0
|
||||
|
||||
for idx, chapter in enumerate(chapters_to_process):
|
||||
if check_pause_func(): break
|
||||
|
||||
progress_callback(f"\n-- Processing and Downloading Chapter {idx + 1}/{total_chapters}: '{chapter['title']}' --")
|
||||
|
||||
series_folder = re.sub(r'[\\/*?:"<>|]', "", top_level_folder_name).strip()
|
||||
chapter_folder = re.sub(r'[\\/*?:"<>|]', "", chapter['title']).strip()
|
||||
final_save_path = os.path.join(output_dir, series_folder, chapter_folder)
|
||||
os.makedirs(final_save_path, exist_ok=True)
|
||||
|
||||
# This function now scrapes and downloads simultaneously
|
||||
dl_count, skip_count = _process_and_download_chapter(
|
||||
chapter_url=chapter['url'],
|
||||
save_path=final_save_path,
|
||||
scraper=scraper,
|
||||
progress_callback=progress_callback,
|
||||
check_pause_func=check_pause_func
|
||||
)
|
||||
|
||||
total_downloaded_count += dl_count
|
||||
total_skipped_count += skip_count
|
||||
|
||||
overall_progress_callback(total_chapters, idx + 1)
|
||||
if check_pause_func(): break
|
||||
|
||||
return total_downloaded_count, total_skipped_count
|
||||
|
||||
except Exception as e:
|
||||
progress_callback(f"❌ A critical error occurred in the Hentai2Read client: {e}")
|
||||
return 0, 0
|
||||
|
||||
def _get_series_metadata(start_url, progress_callback, scraper):
|
||||
"""
|
||||
Scrapes the main series page to get the Artist Name, Series Title, and chapter list.
|
||||
"""
|
||||
try:
|
||||
response = scraper.get(start_url, timeout=30)
|
||||
response.raise_for_status()
|
||||
soup = BeautifulSoup(response.text, 'html.parser')
|
||||
|
||||
series_title = "Unknown Series"
|
||||
artist_name = None
|
||||
metadata_list = soup.select_one("ul.list.list-simple-mini")
|
||||
|
||||
if metadata_list:
|
||||
first_li = metadata_list.find('li', recursive=False)
|
||||
if first_li and not first_li.find('a'):
|
||||
series_title = first_li.get_text(strip=True)
|
||||
|
||||
for b_tag in metadata_list.find_all('b'):
|
||||
label = b_tag.get_text(strip=True)
|
||||
if label in ("Artist", "Author"):
|
||||
a_tag = b_tag.find_next_sibling('a')
|
||||
if a_tag:
|
||||
artist_name = a_tag.get_text(strip=True)
|
||||
if label == "Artist":
|
||||
break
|
||||
|
||||
top_level_folder_name = artist_name if artist_name else series_title
|
||||
|
||||
chapter_links = soup.select("div.media a.pull-left.font-w600")
|
||||
if not chapter_links:
|
||||
chapters_to_process = [{'url': start_url, 'title': series_title}]
|
||||
else:
|
||||
chapters_to_process = [
|
||||
{'url': urljoin(start_url, link['href']), 'title': " ".join(link.stripped_strings)}
|
||||
for link in chapter_links
|
||||
]
|
||||
chapters_to_process.reverse()
|
||||
|
||||
progress_callback(f" [Hentai2Read] ✅ Found Artist/Series: '{top_level_folder_name}'")
|
||||
progress_callback(f" [Hentai2Read] ✅ Found {len(chapters_to_process)} chapters to process.")
|
||||
|
||||
return top_level_folder_name, chapters_to_process
|
||||
|
||||
except Exception as e:
|
||||
progress_callback(f" [Hentai2Read] ❌ Error getting series metadata: {e}")
|
||||
return "Unknown Series", []
|
||||
|
||||
### NEW: This function contains the pipeline logic ###
|
||||
def _process_and_download_chapter(chapter_url, save_path, scraper, progress_callback, check_pause_func):
|
||||
"""
|
||||
Uses a producer-consumer pattern to download a chapter.
|
||||
The main thread (producer) scrapes URLs one by one.
|
||||
Worker threads (consumers) download the URLs as they are found.
|
||||
"""
|
||||
task_queue = queue.Queue()
|
||||
num_download_threads = 8
|
||||
|
||||
# These will be updated by the worker threads
|
||||
download_stats = {'downloaded': 0, 'skipped': 0}
|
||||
|
||||
def downloader_worker():
|
||||
"""The function that each download thread will run."""
|
||||
# Create a unique session for each thread to avoid conflicts
|
||||
worker_scraper = cloudscraper.create_scraper()
|
||||
while True:
|
||||
try:
|
||||
# Get a task from the queue
|
||||
task = task_queue.get()
|
||||
# The sentinel value to signal the end
|
||||
if task is None:
|
||||
break
|
||||
|
||||
filepath, img_url = task
|
||||
if os.path.exists(filepath):
|
||||
progress_callback(f" -> Skip: '{os.path.basename(filepath)}'")
|
||||
download_stats['skipped'] += 1
|
||||
else:
|
||||
progress_callback(f" Downloading: '{os.path.basename(filepath)}'...")
|
||||
response = worker_scraper.get(img_url, stream=True, timeout=60, headers={'Referer': chapter_url})
|
||||
response.raise_for_status()
|
||||
with open(filepath, 'wb') as f:
|
||||
for chunk in response.iter_content(chunk_size=8192):
|
||||
f.write(chunk)
|
||||
download_stats['downloaded'] += 1
|
||||
except Exception as e:
|
||||
progress_callback(f" ❌ Download failed for task. Error: {e}")
|
||||
download_stats['skipped'] += 1
|
||||
finally:
|
||||
task_queue.task_done()
|
||||
|
||||
# --- Start the downloader threads ---
|
||||
executor = ThreadPoolExecutor(max_workers=num_download_threads, thread_name_prefix='H2R_Downloader')
|
||||
for _ in range(num_download_threads):
|
||||
executor.submit(downloader_worker)
|
||||
|
||||
# --- Main thread acts as the scraper (producer) ---
|
||||
page_number = 1
|
||||
while True:
|
||||
if check_pause_func(): break
|
||||
if page_number > 300: # Safety break
|
||||
progress_callback(" [Hentai2Read] ⚠️ Safety break: Reached 300 pages.")
|
||||
break
|
||||
|
||||
page_url_to_check = f"{chapter_url}{page_number}/"
|
||||
try:
|
||||
response = scraper.get(page_url_to_check, timeout=30)
|
||||
if response.history or response.status_code != 200:
|
||||
progress_callback(f" [Hentai2Read] End of chapter detected on page {page_number}.")
|
||||
break
|
||||
|
||||
soup = BeautifulSoup(response.text, 'html.parser')
|
||||
img_tag = soup.select_one("img#arf-reader")
|
||||
img_src = img_tag.get("src") if img_tag else None
|
||||
|
||||
if not img_tag or img_src == "https://static.hentai.direct/hentai":
|
||||
progress_callback(f" [Hentai2Read] End of chapter detected (Placeholder image on page {page_number}).")
|
||||
break
|
||||
|
||||
normalized_img_src = urljoin(response.url, img_src)
|
||||
ext = os.path.splitext(normalized_img_src.split('/')[-1])[-1] or ".jpg"
|
||||
filename = f"{page_number:03d}{ext}"
|
||||
filepath = os.path.join(save_path, filename)
|
||||
|
||||
# Put the download task into the queue for a worker to pick up
|
||||
task_queue.put((filepath, normalized_img_src))
|
||||
|
||||
page_number += 1
|
||||
time.sleep(0.1) # Small delay between scraping pages
|
||||
except Exception as e:
|
||||
progress_callback(f" [Hentai2Read] ❌ Error while scraping page {page_number}: {e}")
|
||||
break
|
||||
|
||||
# --- Shutdown sequence ---
|
||||
# Tell all worker threads to exit by sending the sentinel value
|
||||
for _ in range(num_download_threads):
|
||||
task_queue.put(None)
|
||||
|
||||
# Wait for all download tasks to be completed
|
||||
executor.shutdown(wait=True)
|
||||
|
||||
progress_callback(f" Found and processed {page_number - 1} images for this chapter.")
|
||||
return download_stats['downloaded'], download_stats['skipped']
|
||||
116
src/core/allcomic_client.py
Normal file
116
src/core/allcomic_client.py
Normal file
@@ -0,0 +1,116 @@
|
||||
import requests
|
||||
import re
|
||||
from bs4 import BeautifulSoup
|
||||
import cloudscraper
|
||||
import time
|
||||
from urllib.parse import urlparse
|
||||
|
||||
def get_chapter_list(series_url, logger_func):
|
||||
"""
|
||||
Checks if a URL is a series page and returns a list of all chapter URLs if it is.
|
||||
Includes a retry mechanism for robust connection.
|
||||
"""
|
||||
logger_func(f" [AllComic] Checking for chapter list at: {series_url}")
|
||||
|
||||
scraper = cloudscraper.create_scraper()
|
||||
response = None
|
||||
max_retries = 8
|
||||
|
||||
for attempt in range(max_retries):
|
||||
try:
|
||||
response = scraper.get(series_url, timeout=30)
|
||||
response.raise_for_status()
|
||||
logger_func(f" [AllComic] Successfully connected to series page on attempt {attempt + 1}.")
|
||||
break # Success, exit the loop
|
||||
except requests.RequestException as e:
|
||||
logger_func(f" [AllComic] ⚠️ Series page check attempt {attempt + 1}/{max_retries} failed: {e}")
|
||||
if attempt < max_retries - 1:
|
||||
wait_time = 2 * (attempt + 1)
|
||||
logger_func(f" Retrying in {wait_time} seconds...")
|
||||
time.sleep(wait_time)
|
||||
else:
|
||||
logger_func(f" [AllComic] ❌ All attempts to check series page failed.")
|
||||
return [] # Return empty on final failure
|
||||
|
||||
if not response:
|
||||
return []
|
||||
|
||||
try:
|
||||
soup = BeautifulSoup(response.text, 'html.parser')
|
||||
chapter_links = soup.select('li.wp-manga-chapter a')
|
||||
|
||||
if not chapter_links:
|
||||
logger_func(" [AllComic] ℹ️ No chapter list found. Assuming this is a single chapter page.")
|
||||
return []
|
||||
|
||||
chapter_urls = [link['href'] for link in chapter_links]
|
||||
chapter_urls.reverse() # Reverse for oldest-to-newest reading order
|
||||
|
||||
logger_func(f" [AllComic] ✅ Found {len(chapter_urls)} chapters.")
|
||||
return chapter_urls
|
||||
|
||||
except Exception as e:
|
||||
logger_func(f" [AllComic] ❌ Error parsing chapters after successful connection: {e}")
|
||||
return []
|
||||
|
||||
def fetch_chapter_data(chapter_url, logger_func):
|
||||
"""
|
||||
Fetches the comic title, chapter title, and image URLs for a single chapter page.
|
||||
"""
|
||||
logger_func(f" [AllComic] Fetching page: {chapter_url}")
|
||||
|
||||
scraper = cloudscraper.create_scraper(
|
||||
browser={'browser': 'firefox', 'platform': 'windows', 'desktop': True}
|
||||
)
|
||||
headers = {'Referer': 'https://allporncomic.com/'}
|
||||
|
||||
response = None
|
||||
max_retries = 8
|
||||
for attempt in range(max_retries):
|
||||
try:
|
||||
response = scraper.get(chapter_url, headers=headers, timeout=30)
|
||||
response.raise_for_status()
|
||||
break
|
||||
except requests.RequestException as e:
|
||||
if attempt < max_retries - 1:
|
||||
time.sleep(2 * (attempt + 1))
|
||||
else:
|
||||
logger_func(f" [AllComic] ❌ All connection attempts failed for chapter: {chapter_url}")
|
||||
return None, None, None
|
||||
|
||||
try:
|
||||
soup = BeautifulSoup(response.text, 'html.parser')
|
||||
title_element = soup.find('h1', class_='post-title')
|
||||
comic_title = None
|
||||
if title_element:
|
||||
comic_title = title_element.text.strip()
|
||||
else:
|
||||
try:
|
||||
path_parts = urlparse(chapter_url).path.strip('/').split('/')
|
||||
if len(path_parts) >= 3 and path_parts[-3] == 'porncomic':
|
||||
comic_slug = path_parts[-2]
|
||||
comic_title = comic_slug.replace('-', ' ').title()
|
||||
except Exception:
|
||||
comic_title = "Unknown Comic"
|
||||
|
||||
chapter_slug = chapter_url.strip('/').split('/')[-1]
|
||||
chapter_title = chapter_slug.replace('-', ' ').title()
|
||||
|
||||
reading_container = soup.find('div', class_='reading-content')
|
||||
list_of_image_urls = []
|
||||
if reading_container:
|
||||
image_elements = reading_container.find_all('img', class_='wp-manga-chapter-img')
|
||||
for img in image_elements:
|
||||
img_url = (img.get('data-src') or img.get('src', '')).strip()
|
||||
if img_url:
|
||||
list_of_image_urls.append(img_url)
|
||||
|
||||
if not comic_title or comic_title == "Unknown Comic" or not list_of_image_urls:
|
||||
logger_func(f" [AllComic] ❌ Could not find a valid title or images on the page. Title found: '{comic_title}'")
|
||||
return None, None, None
|
||||
|
||||
return comic_title, chapter_title, list_of_image_urls
|
||||
|
||||
except Exception as e:
|
||||
logger_func(f" [AllComic] ❌ An unexpected error occurred while parsing the page: {e}")
|
||||
return None, None, None
|
||||
@@ -1,8 +1,9 @@
|
||||
import time
|
||||
import traceback
|
||||
from urllib.parse import urlparse
|
||||
import json # Ensure json is imported
|
||||
import json
|
||||
import requests
|
||||
import cloudscraper
|
||||
from ..utils.network_utils import extract_post_info, prepare_cookies_for_request
|
||||
from ..config.constants import (
|
||||
STYLE_DATE_POST_TITLE
|
||||
@@ -12,7 +13,6 @@ from ..config.constants import (
|
||||
def fetch_posts_paginated(api_url_base, headers, offset, logger, cancellation_event=None, pause_event=None, cookies_dict=None):
|
||||
"""
|
||||
Fetches a single page of posts from the API with robust retry logic.
|
||||
NEW: Requests only essential fields to keep the response size small and reliable.
|
||||
"""
|
||||
if cancellation_event and cancellation_event.is_set():
|
||||
raise RuntimeError("Fetch operation cancelled by user.")
|
||||
@@ -33,7 +33,7 @@ def fetch_posts_paginated(api_url_base, headers, offset, logger, cancellation_ev
|
||||
if cancellation_event and cancellation_event.is_set():
|
||||
raise RuntimeError("Fetch operation cancelled by user during retry loop.")
|
||||
|
||||
log_message = f" Fetching post list: {api_url_base}?o={offset} (Page approx. {offset // 50 + 1})"
|
||||
log_message = f" Fetching post list: {api_url_base} (Page approx. {offset // 50 + 1})"
|
||||
if attempt > 0:
|
||||
log_message += f" (Attempt {attempt + 1}/{max_retries})"
|
||||
logger(log_message)
|
||||
@@ -41,9 +41,23 @@ def fetch_posts_paginated(api_url_base, headers, offset, logger, cancellation_ev
|
||||
try:
|
||||
response = requests.get(paginated_url, headers=headers, timeout=(15, 60), cookies=cookies_dict)
|
||||
response.raise_for_status()
|
||||
response.encoding = 'utf-8'
|
||||
return response.json()
|
||||
|
||||
except requests.exceptions.RequestException as e:
|
||||
# Handle 403 error on the FIRST page as a rate limit/block
|
||||
if e.response is not None and e.response.status_code == 403 and offset == 0:
|
||||
logger(" ❌ Access Denied (403 Forbidden) on the first page.")
|
||||
logger(" This is likely a rate limit or a Cloudflare block.")
|
||||
logger(" 💡 SOLUTION: Wait a while, use a VPN, or provide a valid session cookie.")
|
||||
return [] # Stop the process gracefully
|
||||
|
||||
# Handle 400 error as the end of pages
|
||||
if e.response is not None and e.response.status_code == 400:
|
||||
logger(f" ✅ Reached end of posts (API returned 400 Bad Request for offset {offset}).")
|
||||
return []
|
||||
|
||||
# Handle all other network errors with a retry
|
||||
logger(f" ⚠️ Retryable network error on page fetch (Attempt {attempt + 1}): {e}")
|
||||
if attempt < max_retries - 1:
|
||||
delay = retry_delay * (2 ** attempt)
|
||||
@@ -65,26 +79,28 @@ def fetch_posts_paginated(api_url_base, headers, offset, logger, cancellation_ev
|
||||
|
||||
raise RuntimeError(f"Failed to fetch page {paginated_url} after all attempts.")
|
||||
|
||||
|
||||
def fetch_single_post_data(api_domain, service, user_id, post_id, headers, logger, cookies_dict=None):
|
||||
"""
|
||||
--- NEW FUNCTION ---
|
||||
Fetches the full data, including the 'content' field, for a single post.
|
||||
--- MODIFIED FUNCTION ---
|
||||
Fetches the full data, including the 'content' field, for a single post using cloudscraper.
|
||||
"""
|
||||
post_api_url = f"https://{api_domain}/api/v1/{service}/user/{user_id}/post/{post_id}"
|
||||
logger(f" Fetching full content for post ID {post_id}...")
|
||||
|
||||
scraper = cloudscraper.create_scraper()
|
||||
|
||||
try:
|
||||
with requests.get(post_api_url, headers=headers, timeout=(15, 300), cookies=cookies_dict, stream=True) as response:
|
||||
response.raise_for_status()
|
||||
response_body = b""
|
||||
for chunk in response.iter_content(chunk_size=8192):
|
||||
response_body += chunk
|
||||
|
||||
full_post_data = json.loads(response_body)
|
||||
if isinstance(full_post_data, list) and full_post_data:
|
||||
return full_post_data[0]
|
||||
return full_post_data
|
||||
|
||||
response = scraper.get(post_api_url, headers=headers, timeout=(15, 300), cookies=cookies_dict)
|
||||
response.raise_for_status()
|
||||
|
||||
full_post_data = response.json()
|
||||
|
||||
if isinstance(full_post_data, list) and full_post_data:
|
||||
return full_post_data[0]
|
||||
if isinstance(full_post_data, dict) and 'post' in full_post_data:
|
||||
return full_post_data['post']
|
||||
return full_post_data
|
||||
|
||||
except Exception as e:
|
||||
logger(f" ❌ Failed to fetch full content for post {post_id}: {e}")
|
||||
return None
|
||||
@@ -101,6 +117,7 @@ def fetch_post_comments(api_domain, service, user_id, post_id, headers, logger,
|
||||
try:
|
||||
response = requests.get(comments_api_url, headers=headers, timeout=(10, 30), cookies=cookies_dict)
|
||||
response.raise_for_status()
|
||||
response.encoding = 'utf-8'
|
||||
return response.json()
|
||||
except requests.exceptions.RequestException as e:
|
||||
raise RuntimeError(f"Error fetching comments for post {post_id}: {e}")
|
||||
@@ -120,12 +137,18 @@ def download_from_api(
|
||||
selected_cookie_file=None,
|
||||
app_base_dir=None,
|
||||
manga_filename_style_for_sort_check=None,
|
||||
processed_post_ids=None
|
||||
):
|
||||
processed_post_ids=None,
|
||||
fetch_all_first=False
|
||||
):
|
||||
parsed_input_url_for_domain = urlparse(api_url_input)
|
||||
api_domain = parsed_input_url_for_domain.netloc
|
||||
|
||||
headers = {
|
||||
'User-Agent': 'Mozilla/5.0',
|
||||
'Accept': 'application/json'
|
||||
'User-Agent': 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)',
|
||||
'Referer': f'https://{api_domain}/',
|
||||
'Accept': 'text/css'
|
||||
}
|
||||
|
||||
if processed_post_ids is None:
|
||||
processed_post_ids = set()
|
||||
else:
|
||||
@@ -137,15 +160,11 @@ def download_from_api(
|
||||
logger(" Download_from_api cancelled at start.")
|
||||
return
|
||||
|
||||
parsed_input_url_for_domain = urlparse(api_url_input)
|
||||
api_domain = parsed_input_url_for_domain.netloc
|
||||
# The code that defined api_domain was moved from here to the top of the function
|
||||
|
||||
# --- START: MODIFIED LOGIC ---
|
||||
# This list is updated to include the new .cr and .st mirrors for validation.
|
||||
if not any(d in api_domain.lower() for d in ['kemono.su', 'kemono.party', 'kemono.cr', 'coomer.su', 'coomer.party', 'coomer.st']):
|
||||
logger(f"⚠️ Unrecognized domain '{api_domain}' from input URL. Defaulting to kemono.su for API calls.")
|
||||
api_domain = "kemono.su"
|
||||
# --- END: MODIFIED LOGIC ---
|
||||
|
||||
cookies_for_api = None
|
||||
if use_cookie and app_base_dir:
|
||||
@@ -159,6 +178,7 @@ def download_from_api(
|
||||
try:
|
||||
direct_response = requests.get(direct_post_api_url, headers=headers, timeout=(10, 30), cookies=cookies_for_api)
|
||||
direct_response.raise_for_status()
|
||||
direct_response.encoding = 'utf-8'
|
||||
direct_post_data = direct_response.json()
|
||||
if isinstance(direct_post_data, list) and direct_post_data:
|
||||
direct_post_data = direct_post_data[0]
|
||||
@@ -183,7 +203,8 @@ def download_from_api(
|
||||
logger("⚠️ Page range (start/end page) is ignored when a specific post URL is provided (searching all pages for the post).")
|
||||
|
||||
is_manga_mode_fetch_all_and_sort_oldest_first = manga_mode and (manga_filename_style_for_sort_check != STYLE_DATE_POST_TITLE) and not target_post_id
|
||||
api_base_url = f"https://{api_domain}/api/v1/{service}/user/{user_id}"
|
||||
should_fetch_all = fetch_all_first or is_manga_mode_fetch_all_and_sort_oldest_first
|
||||
api_base_url = f"https://{api_domain}/api/v1/{service}/user/{user_id}/posts"
|
||||
page_size = 50
|
||||
if is_manga_mode_fetch_all_and_sort_oldest_first:
|
||||
logger(f" Manga Mode (Style: {manga_filename_style_for_sort_check if manga_filename_style_for_sort_check else 'Default'} - Oldest First Sort Active): Fetching all posts to sort by date...")
|
||||
@@ -226,7 +247,7 @@ def download_from_api(
|
||||
break
|
||||
all_posts_for_manga_mode.extend(posts_batch_manga)
|
||||
|
||||
logger(f"MANGA_FETCH_PROGRESS:{len(all_posts_for_manga_mode)}:{current_page_num_manga}")
|
||||
logger(f"RENAMING_MODE_FETCH_PROGRESS:{len(all_posts_for_manga_mode)}:{current_page_num_manga}")
|
||||
|
||||
current_offset_manga += page_size
|
||||
time.sleep(0.6)
|
||||
@@ -244,7 +265,7 @@ def download_from_api(
|
||||
if cancellation_event and cancellation_event.is_set(): return
|
||||
|
||||
if all_posts_for_manga_mode:
|
||||
logger(f"MANGA_FETCH_COMPLETE:{len(all_posts_for_manga_mode)}")
|
||||
logger(f"RENAMING_MODE_FETCH_COMPLETE:{len(all_posts_for_manga_mode)}")
|
||||
|
||||
if all_posts_for_manga_mode:
|
||||
if processed_post_ids:
|
||||
@@ -354,3 +375,4 @@ def download_from_api(
|
||||
time.sleep(0.6)
|
||||
if target_post_id and not processed_target_post_flag and not (cancellation_event and cancellation_event.is_set()):
|
||||
logger(f"❌ Target post {target_post_id} could not be found after checking all relevant pages (final check after loop).")
|
||||
|
||||
|
||||
375
src/core/booru_client.py
Normal file
375
src/core/booru_client.py
Normal file
@@ -0,0 +1,375 @@
|
||||
# src/core/booru_client.py
|
||||
|
||||
import os
|
||||
import re
|
||||
import time
|
||||
import datetime
|
||||
import urllib.parse
|
||||
import requests
|
||||
import logging
|
||||
import cloudscraper
|
||||
# --- Start of Combined Code from 1.py ---
|
||||
|
||||
# Part 1: Essential Utilities & Exceptions
|
||||
|
||||
class BooruClientException(Exception):
|
||||
"""Base class for exceptions in this client."""
|
||||
pass
|
||||
|
||||
class HttpError(BooruClientException):
|
||||
"""HTTP request during data extraction failed."""
|
||||
def __init__(self, message="", response=None):
|
||||
self.response = response
|
||||
self.status = response.status_code if response else 0
|
||||
if response and not message:
|
||||
message = f"'{response.status_code} {response.reason}' for '{response.url}'"
|
||||
super().__init__(message)
|
||||
|
||||
class NotFoundError(BooruClientException):
|
||||
pass
|
||||
|
||||
def unquote(s):
|
||||
return urllib.parse.unquote(s)
|
||||
|
||||
def parse_datetime(date_string, fmt):
|
||||
try:
|
||||
# Assumes date_string is in a format that strptime can handle with timezone
|
||||
return datetime.datetime.strptime(date_string, fmt)
|
||||
except (ValueError, TypeError):
|
||||
return None
|
||||
|
||||
def nameext_from_url(url, data=None):
|
||||
if data is None: data = {}
|
||||
try:
|
||||
path = urllib.parse.urlparse(url).path
|
||||
filename = unquote(os.path.basename(path))
|
||||
if '.' in filename:
|
||||
name, ext = filename.rsplit('.', 1)
|
||||
data["filename"], data["extension"] = name, ext.lower()
|
||||
else:
|
||||
data["filename"], data["extension"] = filename, ""
|
||||
except Exception:
|
||||
data["filename"], data["extension"] = "", ""
|
||||
return data
|
||||
|
||||
USERAGENT_FIREFOX = "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/118.0"
|
||||
|
||||
# Part 2: Core Extractor Logic
|
||||
|
||||
class Extractor:
|
||||
category = ""
|
||||
subcategory = ""
|
||||
directory_fmt = ("{category}", "{id}")
|
||||
filename_fmt = "{filename}.{extension}"
|
||||
_retries = 3
|
||||
_timeout = 30
|
||||
|
||||
def __init__(self, match, logger_func=print):
|
||||
self.url = match.string
|
||||
self.match = match
|
||||
self.groups = match.groups()
|
||||
self.session = cloudscraper.create_scraper()
|
||||
self.session.headers["User-Agent"] = USERAGENT_FIREFOX
|
||||
self.log = logger_func
|
||||
self.api_key = None
|
||||
self.user_id = None
|
||||
|
||||
def set_auth(self, api_key, user_id):
|
||||
self.api_key = api_key
|
||||
self.user_id = user_id
|
||||
self._init_auth()
|
||||
|
||||
def _init_auth(self):
|
||||
"""Placeholder for extractor-specific auth setup."""
|
||||
pass
|
||||
|
||||
def request(self, url, method="GET", fatal=True, **kwargs):
|
||||
for attempt in range(self._retries + 1):
|
||||
try:
|
||||
response = self.session.request(method, url, timeout=self._timeout, **kwargs)
|
||||
if response.status_code < 400:
|
||||
return response
|
||||
if response.status_code == 404 and fatal:
|
||||
raise NotFoundError(f"Resource not found at {url}")
|
||||
self.log(f"Request for {url} failed with status {response.status_code}. Retrying...")
|
||||
except requests.exceptions.RequestException as e:
|
||||
self.log(f"Request for {url} failed: {e}. Retrying...")
|
||||
if attempt < self._retries:
|
||||
time.sleep(2 ** attempt)
|
||||
if fatal:
|
||||
raise HttpError(f"Failed to retrieve {url} after {self._retries} retries.")
|
||||
return None
|
||||
|
||||
def request_json(self, url, **kwargs):
|
||||
response = self.request(url, **kwargs)
|
||||
try:
|
||||
return response.json()
|
||||
except (ValueError, TypeError) as exc:
|
||||
self.log(f"Failed to decode JSON from {url}: {exc}")
|
||||
raise BooruClientException("Invalid JSON response")
|
||||
|
||||
def items(self):
|
||||
data = self.metadata()
|
||||
for item in self.posts():
|
||||
# Check for our special page update message
|
||||
if isinstance(item, tuple) and item[0] == 'PAGE_UPDATE':
|
||||
yield item
|
||||
continue
|
||||
|
||||
# Otherwise, process it as a post
|
||||
post = item
|
||||
url = post.get("file_url")
|
||||
if not url: continue
|
||||
|
||||
nameext_from_url(url, post)
|
||||
post["date"] = parse_datetime(post.get("created_at"), "%Y-%m-%dT%H:%M:%S.%f%z")
|
||||
|
||||
if url.startswith("/"):
|
||||
url = self.root + url
|
||||
post['file_url'] = url # Ensure full URL
|
||||
|
||||
post.update(data)
|
||||
yield post
|
||||
|
||||
class BaseExtractor(Extractor):
|
||||
instances = ()
|
||||
|
||||
def __init__(self, match, logger_func=print):
|
||||
super().__init__(match, logger_func)
|
||||
self._init_category()
|
||||
|
||||
def _init_category(self):
|
||||
parsed_url = urllib.parse.urlparse(self.url)
|
||||
self.root = f"{parsed_url.scheme}://{parsed_url.netloc}"
|
||||
for i, group in enumerate(self.groups):
|
||||
if group is not None:
|
||||
try:
|
||||
self.category = self.instances[i][0]
|
||||
return
|
||||
except IndexError:
|
||||
continue
|
||||
|
||||
@classmethod
|
||||
def update(cls, instances):
|
||||
pattern_list = []
|
||||
instance_list = cls.instances = []
|
||||
for category, info in instances.items():
|
||||
root = info["root"].rstrip("/") if info["root"] else ""
|
||||
instance_list.append((category, root, info))
|
||||
pattern = info.get("pattern", re.escape(root.partition("://")[2]))
|
||||
pattern_list.append(f"({pattern})")
|
||||
return r"(?:https?://)?(?:" + "|".join(pattern_list) + r")"
|
||||
|
||||
# Part 3: Danbooru Extractor
|
||||
|
||||
class DanbooruExtractor(BaseExtractor):
|
||||
filename_fmt = "{category}_{id}_{filename}.{extension}"
|
||||
per_page = 200
|
||||
|
||||
def __init__(self, match, logger_func=print):
|
||||
super().__init__(match, logger_func)
|
||||
self._auth_logged = False
|
||||
|
||||
def _init_auth(self):
|
||||
if self.user_id and self.api_key:
|
||||
if not self._auth_logged:
|
||||
self.log("Danbooru auth set.")
|
||||
self._auth_logged = True
|
||||
self.session.auth = (self.user_id, self.api_key)
|
||||
|
||||
|
||||
def items(self):
|
||||
data = self.metadata()
|
||||
for item in self.posts():
|
||||
# Check for our special page update message
|
||||
if isinstance(item, tuple) and item[0] == 'PAGE_UPDATE':
|
||||
yield item
|
||||
continue
|
||||
|
||||
# Otherwise, process it as a post
|
||||
post = item
|
||||
url = post.get("file_url")
|
||||
if not url: continue
|
||||
|
||||
nameext_from_url(url, post)
|
||||
post["date"] = parse_datetime(post.get("created_at"), "%Y-%m-%dT%H:%M:%S.%f%z")
|
||||
|
||||
if url.startswith("/"):
|
||||
url = self.root + url
|
||||
post['file_url'] = url # Ensure full URL
|
||||
|
||||
post.update(data)
|
||||
yield post
|
||||
|
||||
def metadata(self):
|
||||
return {}
|
||||
|
||||
def posts(self):
|
||||
return []
|
||||
|
||||
def _pagination(self, endpoint, params, prefix="b"):
|
||||
url = self.root + endpoint
|
||||
params["limit"] = self.per_page
|
||||
params["page"] = 1
|
||||
threshold = self.per_page - 20
|
||||
|
||||
while True:
|
||||
posts = self.request_json(url, params=params)
|
||||
if not posts: break
|
||||
yield ('PAGE_UPDATE', len(posts))
|
||||
yield from posts
|
||||
if len(posts) < threshold: return
|
||||
if prefix:
|
||||
params["page"] = f"{prefix}{posts[-1]['id']}"
|
||||
else:
|
||||
params["page"] += 1
|
||||
|
||||
BASE_PATTERN = DanbooruExtractor.update({
|
||||
"danbooru": {"root": None, "pattern": r"(?:danbooru|safebooru)\.donmai\.us"},
|
||||
})
|
||||
|
||||
class DanbooruTagExtractor(DanbooruExtractor):
|
||||
subcategory = "tag"
|
||||
directory_fmt = ("{category}", "{search_tags}")
|
||||
pattern = BASE_PATTERN + r"(/posts\?(?:[^&#]*&)*tags=([^&#]*))"
|
||||
|
||||
def metadata(self):
|
||||
self.tags = unquote(self.groups[-1].replace("+", " ")).strip()
|
||||
sanitized_tags = re.sub(r'[\\/*?:"<>|]', "_", self.tags)
|
||||
return {"search_tags": sanitized_tags}
|
||||
|
||||
def posts(self):
|
||||
return self._pagination("/posts.json", {"tags": self.tags})
|
||||
|
||||
class DanbooruPostExtractor(DanbooruExtractor):
|
||||
subcategory = "post"
|
||||
pattern = BASE_PATTERN + r"(/post(?:s|/show)/(\d+))"
|
||||
|
||||
def posts(self):
|
||||
post_id = self.groups[-1]
|
||||
url = f"{self.root}/posts/{post_id}.json"
|
||||
post = self.request_json(url)
|
||||
return (post,) if post else ()
|
||||
|
||||
class GelbooruBase(Extractor):
|
||||
category = "gelbooru"
|
||||
root = "https://gelbooru.com"
|
||||
|
||||
def __init__(self, match, logger_func=print):
|
||||
super().__init__(match, logger_func)
|
||||
self._auth_logged = False
|
||||
|
||||
def _api_request(self, params, key="post"):
|
||||
# Auth is now added dynamically
|
||||
if self.api_key and self.user_id:
|
||||
if not self._auth_logged:
|
||||
self.log("Gelbooru auth set.")
|
||||
self._auth_logged = True
|
||||
params.update({"api_key": self.api_key, "user_id": self.user_id})
|
||||
|
||||
url = self.root + "/index.php?page=dapi&q=index&json=1"
|
||||
data = self.request_json(url, params=params)
|
||||
|
||||
if not key: return data
|
||||
posts = data.get(key, [])
|
||||
return posts if isinstance(posts, list) else [posts] if posts else []
|
||||
|
||||
def items(self):
|
||||
base_data = self.metadata()
|
||||
base_data['category'] = self.category
|
||||
|
||||
for item in self.posts():
|
||||
# Check for our special page update message
|
||||
if isinstance(item, tuple) and item[0] == 'PAGE_UPDATE':
|
||||
yield item
|
||||
continue
|
||||
|
||||
# Otherwise, process it as a post
|
||||
post = item
|
||||
url = post.get("file_url")
|
||||
if not url: continue
|
||||
|
||||
data = base_data.copy()
|
||||
data.update(post)
|
||||
nameext_from_url(url, data)
|
||||
yield data
|
||||
|
||||
def metadata(self): return {}
|
||||
def posts(self): return []
|
||||
|
||||
GELBOORU_PATTERN = r"(?:https?://)?(?:www\.)?gelbooru\.com"
|
||||
|
||||
class GelbooruTagExtractor(GelbooruBase):
|
||||
subcategory = "tag"
|
||||
directory_fmt = ("{category}", "{search_tags}")
|
||||
filename_fmt = "{category}_{id}_{md5}.{extension}"
|
||||
pattern = GELBOORU_PATTERN + r"(/index\.php\?page=post&s=list&tags=([^&#]*))"
|
||||
|
||||
def metadata(self):
|
||||
self.tags = unquote(self.groups[-1].replace("+", " ")).strip()
|
||||
sanitized_tags = re.sub(r'[\\/*?:"<>|]', "_", self.tags)
|
||||
return {"search_tags": sanitized_tags}
|
||||
|
||||
def posts(self):
|
||||
"""Scrapes HTML search pages as API can be restrictive for tags."""
|
||||
pid = 0
|
||||
posts_per_page = 42
|
||||
search_url = self.root + "/index.php"
|
||||
params = {"page": "post", "s": "list", "tags": self.tags}
|
||||
|
||||
while True:
|
||||
params['pid'] = pid
|
||||
self.log(f"Scraping search results page (offset: {pid})...")
|
||||
response = self.request(search_url, params=params)
|
||||
html_content = response.text
|
||||
post_ids = re.findall(r'id="p(\d+)"', html_content)
|
||||
|
||||
if not post_ids:
|
||||
self.log("No more posts found on page. Ending scrape.")
|
||||
break
|
||||
yield ('PAGE_UPDATE', len(post_ids))
|
||||
for post_id in post_ids:
|
||||
post_data = self._api_request({"s": "post", "id": post_id})
|
||||
yield from post_data
|
||||
|
||||
pid += posts_per_page
|
||||
|
||||
class GelbooruPostExtractor(GelbooruBase):
|
||||
subcategory = "post"
|
||||
filename_fmt = "{category}_{id}_{md5}.{extension}"
|
||||
pattern = GELBOORU_PATTERN + r"(/index\.php\?page=post&s=view&id=(\d+))"
|
||||
|
||||
def posts(self):
|
||||
post_id = self.groups[-1]
|
||||
return self._api_request({"s": "post", "id": post_id})
|
||||
|
||||
# --- Main Entry Point ---
|
||||
|
||||
EXTRACTORS = [
|
||||
DanbooruTagExtractor,
|
||||
DanbooruPostExtractor,
|
||||
GelbooruTagExtractor,
|
||||
GelbooruPostExtractor,
|
||||
]
|
||||
|
||||
def find_extractor(url, logger_func):
|
||||
for extractor_cls in EXTRACTORS:
|
||||
match = re.search(extractor_cls.pattern, url)
|
||||
if match:
|
||||
return extractor_cls(match, logger_func)
|
||||
return None
|
||||
|
||||
def fetch_booru_data(url, api_key, user_id, logger_func):
|
||||
"""
|
||||
Main function to find an extractor and yield image data.
|
||||
"""
|
||||
extractor = find_extractor(url, logger_func)
|
||||
if not extractor:
|
||||
logger_func(f"No suitable Booru extractor found for URL: {url}")
|
||||
return
|
||||
|
||||
logger_func(f"Using extractor: {extractor.__class__.__name__}")
|
||||
extractor.set_auth(api_key, user_id)
|
||||
|
||||
# The 'items' method will now yield the data dictionaries directly
|
||||
yield from extractor.items()
|
||||
265
src/core/bunkr_client.py
Normal file
265
src/core/bunkr_client.py
Normal file
@@ -0,0 +1,265 @@
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import requests
|
||||
import html
|
||||
import time
|
||||
import datetime
|
||||
import urllib.parse
|
||||
import json
|
||||
import random
|
||||
import binascii
|
||||
import itertools
|
||||
|
||||
class MockMessage:
|
||||
Directory = 1
|
||||
Url = 2
|
||||
Version = 3
|
||||
|
||||
class AlbumException(Exception): pass
|
||||
class ExtractionError(AlbumException): pass
|
||||
class HttpError(ExtractionError):
|
||||
def __init__(self, message="", response=None):
|
||||
self.response = response
|
||||
self.status = response.status_code if response is not None else 0
|
||||
super().__init__(message)
|
||||
class ControlException(AlbumException): pass
|
||||
class AbortExtraction(ExtractionError, ControlException): pass
|
||||
|
||||
try:
|
||||
re_compile = re._compiler.compile
|
||||
except AttributeError:
|
||||
re_compile = re.sre_compile.compile
|
||||
HTML_RE = re_compile(r"<[^>]+>")
|
||||
def extr(txt, begin, end, default=""):
|
||||
try:
|
||||
first = txt.index(begin) + len(begin)
|
||||
return txt[first:txt.index(end, first)]
|
||||
except Exception: return default
|
||||
def extract_iter(txt, begin, end, pos=None):
|
||||
try:
|
||||
index = txt.index
|
||||
lbeg = len(begin)
|
||||
lend = len(end)
|
||||
while True:
|
||||
first = index(begin, pos) + lbeg
|
||||
last = index(end, first)
|
||||
pos = last + lend
|
||||
yield txt[first:last]
|
||||
except Exception: return
|
||||
def split_html(txt):
|
||||
try: return [html.unescape(x).strip() for x in HTML_RE.split(txt) if x and not x.isspace()]
|
||||
except TypeError: return []
|
||||
def parse_datetime(date_string, format="%Y-%m-%dT%H:%M:%S%z", utcoffset=0):
|
||||
try:
|
||||
d = datetime.datetime.strptime(date_string, format)
|
||||
o = d.utcoffset()
|
||||
if o is not None: d = d.replace(tzinfo=None, microsecond=0) - o
|
||||
else:
|
||||
if d.microsecond: d = d.replace(microsecond=0)
|
||||
if utcoffset: d += datetime.timedelta(0, utcoffset * -3600)
|
||||
return d
|
||||
except (TypeError, IndexError, KeyError, ValueError, OverflowError): return None
|
||||
unquote = urllib.parse.unquote
|
||||
unescape = html.unescape
|
||||
|
||||
def decrypt_xor(encrypted, key, base64=True, fromhex=False):
|
||||
if base64: encrypted = binascii.a2b_base64(encrypted)
|
||||
if fromhex: encrypted = bytes.fromhex(encrypted.decode())
|
||||
div = len(key)
|
||||
return bytes([encrypted[i] ^ key[i % div] for i in range(len(encrypted))]).decode()
|
||||
def advance(iterable, num):
|
||||
iterator = iter(iterable)
|
||||
next(itertools.islice(iterator, num, num), None)
|
||||
return iterator
|
||||
def json_loads(s): return json.loads(s)
|
||||
def json_dumps(obj): return json.dumps(obj, separators=(",", ":"))
|
||||
|
||||
class Extractor:
|
||||
def __init__(self, match, logger):
|
||||
self.log = logger
|
||||
self.url = match.string
|
||||
self.match = match
|
||||
self.groups = match.groups()
|
||||
self.session = requests.Session()
|
||||
self.session.headers["User-Agent"] = "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Firefox/102.0"
|
||||
@classmethod
|
||||
def from_url(cls, url, logger):
|
||||
if isinstance(cls.pattern, str): cls.pattern = re.compile(cls.pattern)
|
||||
match = cls.pattern.match(url)
|
||||
return cls(match, logger) if match else None
|
||||
def __iter__(self): return self.items()
|
||||
def items(self): yield MockMessage.Version, 1
|
||||
def request(self, url, method="GET", fatal=True, **kwargs):
|
||||
tries = 1
|
||||
while True:
|
||||
try:
|
||||
response = self.session.request(method, url, **kwargs)
|
||||
if response.status_code < 400: return response
|
||||
msg = f"'{response.status_code} {response.reason}' for '{response.url}'"
|
||||
except requests.exceptions.RequestException as exc:
|
||||
msg = str(exc)
|
||||
|
||||
self.log.info("%s (retrying...)", msg)
|
||||
if tries > 4: break
|
||||
time.sleep(tries)
|
||||
tries += 1
|
||||
if not fatal: return None
|
||||
raise HttpError(msg)
|
||||
def request_json(self, url, **kwargs):
|
||||
response = self.request(url, **kwargs)
|
||||
try: return json_loads(response.text)
|
||||
except Exception as exc:
|
||||
self.log.warning("%s: %s", exc.__class__.__name__, exc)
|
||||
if not kwargs.get("fatal", True): return {}
|
||||
raise
|
||||
|
||||
BASE_PATTERN_BUNKR = r"(?:https?://)?(?:[a-zA-Z0-9-]+\.)?(bunkr\.(?:si|la|ws|red|black|media|site|is|to|ac|cr|ci|fi|pk|ps|sk|ph|su)|bunkrr\.ru)"
|
||||
DOMAINS = ["bunkr.si", "bunkr.ws", "bunkr.la", "bunkr.red", "bunkr.black", "bunkr.media", "bunkr.site"]
|
||||
CF_DOMAINS = set()
|
||||
|
||||
class BunkrAlbumExtractor(Extractor):
|
||||
category = "bunkr"
|
||||
root = "https://bunkr.si"
|
||||
root_dl = "https://get.bunkrr.su"
|
||||
root_api = "https://apidl.bunkr.ru"
|
||||
pattern = re.compile(BASE_PATTERN_BUNKR + r"/a/([^/?#]+)")
|
||||
|
||||
def __init__(self, match, logger):
|
||||
super().__init__(match, logger)
|
||||
domain_match = re.search(BASE_PATTERN_BUNKR, match.string)
|
||||
if domain_match:
|
||||
self.root = "https://" + domain_match.group(1)
|
||||
self.endpoint = self.root_api + "/api/_001_v2"
|
||||
self.album_id = self.groups[-1]
|
||||
|
||||
def items(self):
|
||||
page = self.request(self.url).text
|
||||
title = unescape(unescape(extr(page, 'property="og:title" content="', '"')))
|
||||
items_html = list(extract_iter(page, '<div class="grid-images_box', "</a>"))
|
||||
|
||||
album_data = {
|
||||
"album_id": self.album_id,
|
||||
"album_name": title,
|
||||
"count": len(items_html),
|
||||
}
|
||||
yield MockMessage.Directory, album_data, {}
|
||||
|
||||
for item_html in items_html:
|
||||
try:
|
||||
webpage_url = unescape(extr(item_html, ' href="', '"'))
|
||||
if webpage_url.startswith("/"):
|
||||
webpage_url = self.root + webpage_url
|
||||
|
||||
file_data = self._extract_file(webpage_url)
|
||||
info = split_html(item_html)
|
||||
|
||||
if not file_data.get("name"):
|
||||
file_data["name"] = info[-3]
|
||||
|
||||
yield MockMessage.Url, file_data, {}
|
||||
except Exception as exc:
|
||||
self.log.error("%s: %s", exc.__class__.__name__, exc)
|
||||
|
||||
def _extract_file(self, webpage_url):
|
||||
page = self.request(webpage_url).text
|
||||
data_id = extr(page, 'data-file-id="', '"')
|
||||
referer = self.root_dl + "/file/" + data_id
|
||||
headers = {"Referer": referer, "Origin": self.root_dl}
|
||||
data = self.request_json(self.endpoint, method="POST", headers=headers, json={"id": data_id})
|
||||
|
||||
file_url = decrypt_xor(data["url"], f"SECRET_KEY_{data['timestamp'] // 3600}".encode()) if data.get("encrypted") else data["url"]
|
||||
file_name = extr(page, "<h1", "<").rpartition(">")[2]
|
||||
|
||||
return {
|
||||
"url": file_url,
|
||||
"name": unescape(file_name),
|
||||
"_http_headers": {"Referer": referer}
|
||||
}
|
||||
|
||||
class BunkrMediaExtractor(BunkrAlbumExtractor):
|
||||
pattern = re.compile(BASE_PATTERN_BUNKR + r"(/[fvid]/[^/?#]+)")
|
||||
def items(self):
|
||||
try:
|
||||
media_path = self.groups[-1]
|
||||
file_data = self._extract_file(self.root + media_path)
|
||||
album_data = {"album_name": file_data.get("name", "bunkr_media"), "count": 1}
|
||||
|
||||
yield MockMessage.Directory, album_data, {}
|
||||
yield MockMessage.Url, file_data, {}
|
||||
|
||||
except Exception as exc:
|
||||
self.log.error("%s: %s", exc.__class__.__name__, exc)
|
||||
yield MockMessage.Directory, {"album_name": "error", "count": 0}, {}
|
||||
|
||||
def get_bunkr_extractor(url, logger):
|
||||
"""Selects the correct Bunkr extractor based on the URL pattern."""
|
||||
if BunkrAlbumExtractor.pattern.match(url):
|
||||
logger.info("Bunkr Album URL detected.")
|
||||
return BunkrAlbumExtractor.from_url(url, logger)
|
||||
elif BunkrMediaExtractor.pattern.match(url):
|
||||
logger.info("Bunkr Media URL detected.")
|
||||
return BunkrMediaExtractor.from_url(url, logger)
|
||||
else:
|
||||
logger.error(f"No suitable Bunkr extractor found for URL: {url}")
|
||||
return None
|
||||
|
||||
def fetch_bunkr_data(url, logger):
|
||||
"""
|
||||
Main function to be called from the GUI.
|
||||
It extracts all file information from a Bunkr URL, now handling both albums and direct file links.
|
||||
|
||||
Returns:
|
||||
A tuple of (album_name, list_of_files)
|
||||
- album_name (str): The name of the album.
|
||||
- list_of_files (list): A list of dicts, each containing 'url', 'name', and '_http_headers'.
|
||||
Returns (None, None) on failure.
|
||||
"""
|
||||
# --- START: New logic to handle direct CDN file URLs ---
|
||||
try:
|
||||
parsed_url = urllib.parse.urlparse(url)
|
||||
# Check if the hostname contains 'cdn' and the path has a common file extension
|
||||
is_direct_cdn_file = (parsed_url.hostname and 'cdn' in parsed_url.hostname and 'bunkr' in parsed_url.hostname and
|
||||
any(parsed_url.path.lower().endswith(ext) for ext in ['.mp4', '.mkv', '.webm', '.jpg', '.jpeg', '.png', '.gif', '.zip', '.rar']))
|
||||
|
||||
if is_direct_cdn_file:
|
||||
logger.info("Bunkr direct file URL detected.")
|
||||
filename = os.path.basename(parsed_url.path)
|
||||
# Use the filename (without extension) as a sensible album name
|
||||
album_name = os.path.splitext(filename)[0]
|
||||
|
||||
files_to_download = [{
|
||||
'url': url,
|
||||
'name': filename,
|
||||
'_http_headers': {'Referer': 'https://bunkr.ru/'} # Use a generic Referer
|
||||
}]
|
||||
return album_name, files_to_download
|
||||
except Exception as e:
|
||||
logger.warning(f"Could not parse Bunkr URL for direct file check: {e}")
|
||||
# --- END: New logic ---
|
||||
|
||||
# This is the original logic for album and media pages
|
||||
extractor = get_bunkr_extractor(url, logger)
|
||||
if not extractor:
|
||||
return None, None
|
||||
|
||||
try:
|
||||
album_name = "default_bunkr_album"
|
||||
files_to_download = []
|
||||
for msg_type, data, metadata in extractor:
|
||||
if msg_type == MockMessage.Directory:
|
||||
raw_album_name = data.get('album_name', 'untitled')
|
||||
album_name = re.sub(r'[<>:"/\\|?*]', '_', raw_album_name).strip() or "untitled"
|
||||
logger.info(f"Processing Bunkr album: {album_name}")
|
||||
elif msg_type == MockMessage.Url:
|
||||
files_to_download.append(data)
|
||||
|
||||
if not files_to_download:
|
||||
logger.warning("No files found to download from the Bunkr URL.")
|
||||
return None, None
|
||||
|
||||
return album_name, files_to_download
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"An error occurred while extracting Bunkr info: {e}", exc_info=True)
|
||||
return None, None
|
||||
88
src/core/discord_client.py
Normal file
88
src/core/discord_client.py
Normal file
@@ -0,0 +1,88 @@
|
||||
import time
|
||||
import cloudscraper
|
||||
import json
|
||||
|
||||
def fetch_server_channels(server_id, logger=print, cookies_dict=None):
|
||||
"""
|
||||
Fetches all channels for a given Discord server ID from the API.
|
||||
Uses cloudscraper to bypass Cloudflare.
|
||||
"""
|
||||
api_url = f"https://kemono.cr/api/v1/discord/server/{server_id}"
|
||||
logger(f" Fetching channels for server: {api_url}")
|
||||
|
||||
scraper = cloudscraper.create_scraper()
|
||||
headers = {
|
||||
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36',
|
||||
'Referer': f'https://kemono.cr/discord/server/{server_id}',
|
||||
'Accept': 'text/css'
|
||||
}
|
||||
|
||||
try:
|
||||
response = scraper.get(api_url, headers=headers, cookies=cookies_dict, timeout=30)
|
||||
response.raise_for_status()
|
||||
channels = response.json()
|
||||
if isinstance(channels, list):
|
||||
logger(f" ✅ Found {len(channels)} channels for server {server_id}.")
|
||||
return channels
|
||||
return None
|
||||
except Exception as e:
|
||||
logger(f" ❌ Error fetching server channels for {server_id}: {e}")
|
||||
return None
|
||||
|
||||
def fetch_channel_messages(channel_id, logger=print, cancellation_event=None, pause_event=None, cookies_dict=None):
|
||||
"""
|
||||
A generator that fetches all messages for a specific Discord channel, handling pagination.
|
||||
Uses cloudscraper and proper headers to bypass server protection.
|
||||
"""
|
||||
scraper = cloudscraper.create_scraper()
|
||||
base_url = f"https://kemono.cr/api/v1/discord/channel/{channel_id}"
|
||||
headers = {
|
||||
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36',
|
||||
'Referer': f'https://kemono.cr/discord/channel/{channel_id}',
|
||||
'Accept': 'text/css'
|
||||
}
|
||||
|
||||
offset = 0
|
||||
page_size = 150
|
||||
|
||||
while True:
|
||||
if cancellation_event and cancellation_event.is_set():
|
||||
logger(" Discord message fetching cancelled.")
|
||||
break
|
||||
if pause_event and pause_event.is_set():
|
||||
logger(" Discord message fetching paused...")
|
||||
while pause_event.is_set():
|
||||
if cancellation_event and cancellation_event.is_set():
|
||||
break
|
||||
time.sleep(0.5)
|
||||
if not (cancellation_event and cancellation_event.is_set()):
|
||||
logger(" Discord message fetching resumed.")
|
||||
|
||||
paginated_url = f"{base_url}?o={offset}"
|
||||
logger(f" Fetching messages from API: page starting at offset {offset}")
|
||||
|
||||
try:
|
||||
response = scraper.get(paginated_url, headers=headers, cookies=cookies_dict, timeout=30)
|
||||
response.raise_for_status()
|
||||
messages_batch = response.json()
|
||||
|
||||
if not messages_batch:
|
||||
logger(f" ✅ Reached end of messages for channel {channel_id}.")
|
||||
break
|
||||
|
||||
logger(f" Fetched {len(messages_batch)} messages...")
|
||||
yield messages_batch
|
||||
|
||||
if len(messages_batch) < page_size:
|
||||
logger(f" ✅ Last page of messages received for channel {channel_id}.")
|
||||
break
|
||||
|
||||
offset += page_size
|
||||
time.sleep(0.5) # Be respectful to the API
|
||||
|
||||
except (cloudscraper.exceptions.CloudflareException, json.JSONDecodeError) as e:
|
||||
logger(f" ❌ Error fetching messages at offset {offset}: {e}")
|
||||
break
|
||||
except Exception as e:
|
||||
logger(f" ❌ An unexpected error occurred while fetching messages: {e}")
|
||||
break
|
||||
131
src/core/erome_client.py
Normal file
131
src/core/erome_client.py
Normal file
@@ -0,0 +1,131 @@
|
||||
|
||||
import os
|
||||
import re
|
||||
import html
|
||||
import time
|
||||
import urllib.parse
|
||||
import requests
|
||||
from datetime import datetime
|
||||
import cloudscraper
|
||||
|
||||
|
||||
def extr(txt, begin, end, default=""):
|
||||
"""Stripped-down version of 'extract()' to find text between two delimiters."""
|
||||
try:
|
||||
first = txt.index(begin) + len(begin)
|
||||
return txt[first:txt.index(end, first)]
|
||||
except (ValueError, IndexError):
|
||||
return default
|
||||
|
||||
def extract_iter(txt, begin, end):
|
||||
"""Yields all occurrences of text between two delimiters."""
|
||||
try:
|
||||
index = txt.index
|
||||
lbeg = len(begin)
|
||||
lend = len(end)
|
||||
pos = 0
|
||||
while True:
|
||||
first = index(begin, pos) + lbeg
|
||||
last = index(end, first)
|
||||
pos = last + lend
|
||||
yield txt[first:last]
|
||||
except (ValueError, IndexError):
|
||||
return
|
||||
|
||||
def nameext_from_url(url):
|
||||
"""Extracts filename and extension from a URL."""
|
||||
data = {}
|
||||
filename = urllib.parse.unquote(url.partition("?")[0].rpartition("/")[2])
|
||||
name, _, ext = filename.rpartition(".")
|
||||
if name and len(ext) <= 16:
|
||||
data["filename"], data["extension"] = name, ext.lower()
|
||||
else:
|
||||
data["filename"], data["extension"] = filename, ""
|
||||
return data
|
||||
|
||||
def parse_timestamp(ts, default=None):
|
||||
"""Creates a datetime object from a Unix timestamp."""
|
||||
try:
|
||||
return datetime.fromtimestamp(int(ts))
|
||||
except (ValueError, TypeError):
|
||||
return default
|
||||
|
||||
|
||||
def fetch_erome_data(url, logger):
|
||||
"""
|
||||
Identifies and extracts all media files from an Erome album URL.
|
||||
|
||||
Args:
|
||||
url (str): The Erome album URL (e.g., https://www.erome.com/a/albumID).
|
||||
logger (function): A function to log progress messages.
|
||||
|
||||
Returns:
|
||||
tuple: A tuple containing (album_folder_name, list_of_file_dicts).
|
||||
Returns (None, []) if data extraction fails.
|
||||
"""
|
||||
album_id_match = re.search(r"/a/(\w+)", url)
|
||||
if not album_id_match:
|
||||
logger(f"Error: The URL '{url}' does not appear to be a valid Erome album link.")
|
||||
return None, []
|
||||
|
||||
album_id = album_id_match.group(1)
|
||||
page_url = f"https://www.erome.com/a/{album_id}"
|
||||
|
||||
session = cloudscraper.create_scraper()
|
||||
|
||||
try:
|
||||
logger(f" Fetching Erome album page: {page_url}")
|
||||
for attempt in range(5):
|
||||
response = session.get(page_url, timeout=30)
|
||||
response.raise_for_status()
|
||||
page_content = response.text
|
||||
if "<title>Please wait a few moments</title>" in page_content:
|
||||
logger(f" Cloudflare check detected. Waiting 5 seconds... (Attempt {attempt + 1}/5)")
|
||||
time.sleep(5)
|
||||
continue
|
||||
break
|
||||
else:
|
||||
logger(" Error: Could not bypass Cloudflare check after several attempts.")
|
||||
return None, []
|
||||
|
||||
title = html.unescape(extr(page_content, 'property="og:title" content="', '"'))
|
||||
user = urllib.parse.unquote(extr(page_content, 'href="https://www.erome.com/', '"', default="unknown_user"))
|
||||
|
||||
sanitized_title = re.sub(r'[<>:"/\\|?*]', '_', title).strip()
|
||||
sanitized_user = re.sub(r'[<>:"/\\|?*]', '_', user).strip()
|
||||
|
||||
album_folder_name = f"Erome - {sanitized_user} - {sanitized_title} [{album_id}]"
|
||||
|
||||
urls = []
|
||||
media_groups = page_content.split('<div class="media-group"')
|
||||
for group in media_groups[1:]:
|
||||
video_url = extr(group, '<source src="', '"') or extr(group, 'data-src="', '"')
|
||||
if video_url:
|
||||
urls.append(video_url)
|
||||
|
||||
if not urls:
|
||||
logger(" Warning: No media URLs found on the album page.")
|
||||
return album_folder_name, []
|
||||
|
||||
logger(f" Found {len(urls)} media files in album '{title}'.")
|
||||
|
||||
file_list = []
|
||||
for i, file_url in enumerate(urls, 1):
|
||||
filename_info = nameext_from_url(file_url)
|
||||
filename = f"{album_id}_{sanitized_title}_{i:03d}.{filename_info.get('extension', 'mp4')}"
|
||||
|
||||
file_data = {
|
||||
"url": file_url,
|
||||
"filename": filename,
|
||||
"headers": {"Referer": page_url},
|
||||
}
|
||||
file_list.append(file_data)
|
||||
|
||||
return album_folder_name, file_list
|
||||
|
||||
except requests.exceptions.RequestException as e:
|
||||
logger(f" Error fetching Erome page: {e}")
|
||||
return None, []
|
||||
except Exception as e:
|
||||
logger(f" An unexpected error occurred during Erome extraction: {e}")
|
||||
return None, []
|
||||
125
src/core/fap_nation_client.py
Normal file
125
src/core/fap_nation_client.py
Normal file
@@ -0,0 +1,125 @@
|
||||
import re
|
||||
import os
|
||||
import cloudscraper
|
||||
from urllib.parse import urlparse, urljoin
|
||||
from ..utils.file_utils import clean_folder_name
|
||||
|
||||
def fetch_fap_nation_data(album_url, logger_func):
|
||||
"""
|
||||
Scrapes a fap-nation page by prioritizing HLS streams first, then falling
|
||||
back to direct download links. Selects the highest quality available.
|
||||
"""
|
||||
logger_func(f" [Fap-Nation] Fetching album data from: {album_url}")
|
||||
scraper = cloudscraper.create_scraper()
|
||||
|
||||
try:
|
||||
response = scraper.get(album_url, timeout=45)
|
||||
response.raise_for_status()
|
||||
html_content = response.text
|
||||
|
||||
title_match = re.search(r'<h1[^>]*itemprop="name"[^>]*>(.*?)</h1>', html_content, re.IGNORECASE)
|
||||
album_slug = clean_folder_name(os.path.basename(urlparse(album_url).path.strip('/')))
|
||||
album_title = clean_folder_name(title_match.group(1).strip()) if title_match else album_slug
|
||||
|
||||
files_to_download = []
|
||||
final_url = None
|
||||
link_type = None
|
||||
filename_from_video_tag = None
|
||||
|
||||
video_tag_title_match = re.search(r'data-plyr-config=.*?"title":.*?"([^&]+?\.mp4)"', html_content, re.IGNORECASE)
|
||||
if video_tag_title_match:
|
||||
filename_from_video_tag = clean_folder_name(video_tag_title_match.group(1))
|
||||
logger_func(f" [Fap-Nation] Found high-quality filename in video tag: {filename_from_video_tag}")
|
||||
|
||||
# --- REVISED LOGIC: HLS FIRST ---
|
||||
|
||||
# 1. Prioritize finding an HLS stream.
|
||||
logger_func(" [Fap-Nation] Priority 1: Searching for HLS stream...")
|
||||
iframe_match = re.search(r'<iframe[^>]+src="([^"]+mediadelivery\.net[^"]+)"', html_content, re.IGNORECASE)
|
||||
|
||||
if iframe_match:
|
||||
iframe_url = iframe_match.group(1)
|
||||
logger_func(f" [Fap-Nation] Found video iframe. Visiting: {iframe_url}")
|
||||
try:
|
||||
iframe_response = scraper.get(iframe_url, timeout=30)
|
||||
iframe_response.raise_for_status()
|
||||
iframe_html = iframe_response.text
|
||||
|
||||
playlist_match = re.search(r'<source[^>]+src="([^"]+\.m3u8)"', iframe_html, re.IGNORECASE)
|
||||
if playlist_match:
|
||||
final_url = playlist_match.group(1)
|
||||
link_type = 'hls'
|
||||
logger_func(f" [Fap-Nation] Found embedded HLS stream in iframe: {final_url}")
|
||||
except Exception as e:
|
||||
logger_func(f" [Fap-Nation] ⚠️ Error fetching or parsing iframe content: {e}")
|
||||
|
||||
if not final_url:
|
||||
logger_func(" [Fap-Nation] No stream found in iframe. Checking main page content as a last resort...")
|
||||
js_var_match = re.search(r'"(https?://[^"]+\.m3u8)"', html_content, re.IGNORECASE)
|
||||
if js_var_match:
|
||||
final_url = js_var_match.group(1)
|
||||
link_type = 'hls'
|
||||
logger_func(f" [Fap-Nation] Found HLS stream on main page: {final_url}")
|
||||
|
||||
# 2. Fallback: If no HLS stream was found, search for direct links.
|
||||
if not final_url:
|
||||
logger_func(" [Fap-Nation] No HLS stream found. Priority 2 (Fallback): Searching for direct download links...")
|
||||
direct_link_pattern = r'<a\s+[^>]*href="([^"]+\.(?:mp4|webm|mkv|mov))"[^>]*>'
|
||||
direct_links_found = re.findall(direct_link_pattern, html_content, re.IGNORECASE)
|
||||
|
||||
if direct_links_found:
|
||||
logger_func(f" [Fap-Nation] Found {len(direct_links_found)} direct media link(s). Selecting the best quality...")
|
||||
best_link = direct_links_found[0]
|
||||
for link in direct_links_found:
|
||||
if '1080p' in link.lower():
|
||||
best_link = link
|
||||
break
|
||||
final_url = best_link
|
||||
link_type = 'direct'
|
||||
logger_func(f" [Fap-Nation] Identified direct media link: {final_url}")
|
||||
|
||||
# If after all checks, we still have no URL, then fail.
|
||||
if not final_url:
|
||||
logger_func(" [Fap-Nation] ❌ Stage 1 Failed: Could not find any HLS stream or direct link.")
|
||||
return None, []
|
||||
|
||||
# --- HLS Quality Selection Logic ---
|
||||
if link_type == 'hls' and final_url:
|
||||
logger_func(" [Fap-Nation] HLS stream found. Checking for higher quality variants...")
|
||||
try:
|
||||
master_playlist_response = scraper.get(final_url, timeout=20)
|
||||
master_playlist_response.raise_for_status()
|
||||
playlist_content = master_playlist_response.text
|
||||
|
||||
streams = re.findall(r'#EXT-X-STREAM-INF:.*?RESOLUTION=(\d+)x(\d+).*?\n(.*?)\s', playlist_content)
|
||||
|
||||
if streams:
|
||||
best_stream = max(streams, key=lambda s: int(s[0]) * int(s[1]))
|
||||
height = best_stream[1]
|
||||
relative_path = best_stream[2]
|
||||
new_final_url = urljoin(final_url, relative_path)
|
||||
|
||||
logger_func(f" [Fap-Nation] ✅ Best quality found: {height}p. Updating URL to: {new_final_url}")
|
||||
final_url = new_final_url
|
||||
else:
|
||||
logger_func(" [Fap-Nation] ℹ️ No alternate quality streams found in playlist. Using original.")
|
||||
except Exception as e:
|
||||
logger_func(f" [Fap-Nation] ⚠️ Could not parse HLS master playlist for quality selection: {e}. Using original URL.")
|
||||
|
||||
if final_url and link_type:
|
||||
if filename_from_video_tag:
|
||||
base_name, _ = os.path.splitext(filename_from_video_tag)
|
||||
new_filename = f"{base_name}.mp4"
|
||||
else:
|
||||
new_filename = f"{album_slug}.mp4"
|
||||
|
||||
files_to_download.append({'url': final_url, 'filename': new_filename, 'type': link_type})
|
||||
logger_func(f" [Fap-Nation] ✅ Ready to download '{new_filename}' ({link_type} method).")
|
||||
return album_title, files_to_download
|
||||
|
||||
logger_func(f" [Fap-Nation] ❌ Could not determine a valid download link.")
|
||||
return None, []
|
||||
|
||||
except Exception as e:
|
||||
logger_func(f" [Fap-Nation] ❌ Error fetching Fap-Nation data: {e}")
|
||||
return None, []
|
||||
189
src/core/mangadex_client.py
Normal file
189
src/core/mangadex_client.py
Normal file
@@ -0,0 +1,189 @@
|
||||
# src/core/mangadex_client.py
|
||||
|
||||
import os
|
||||
import re
|
||||
import time
|
||||
import cloudscraper
|
||||
from collections import defaultdict
|
||||
from ..utils.file_utils import clean_folder_name
|
||||
|
||||
def fetch_mangadex_data(start_url, output_dir, logger_func, file_progress_callback, overall_progress_callback, pause_event, cancellation_event):
|
||||
"""
|
||||
Fetches and downloads all content from a MangaDex series or chapter URL.
|
||||
Returns a tuple of (downloaded_count, skipped_count).
|
||||
"""
|
||||
grand_total_dl = 0
|
||||
grand_total_skip = 0
|
||||
|
||||
api = _MangadexAPI(logger_func)
|
||||
|
||||
def _check_pause():
|
||||
if cancellation_event and cancellation_event.is_set(): return True
|
||||
if pause_event and pause_event.is_set():
|
||||
logger_func(" Download paused...")
|
||||
while pause_event.is_set():
|
||||
if cancellation_event and cancellation_event.is_set(): return True
|
||||
time.sleep(0.5)
|
||||
logger_func(" Download resumed.")
|
||||
return cancellation_event.is_set()
|
||||
|
||||
series_match = re.search(r"mangadex\.org/(?:title|manga)/([0-9a-f-]+)", start_url)
|
||||
chapter_match = re.search(r"mangadex\.org/chapter/([0-9a-f-]+)", start_url)
|
||||
|
||||
chapters_to_process = []
|
||||
if series_match:
|
||||
series_id = series_match.group(1)
|
||||
logger_func(f" Series detected. Fetching chapter list for ID: {series_id}")
|
||||
chapters_to_process = api.get_manga_chapters(series_id, cancellation_event, pause_event)
|
||||
elif chapter_match:
|
||||
chapter_id = chapter_match.group(1)
|
||||
logger_func(f" Single chapter detected. Fetching info for ID: {chapter_id}")
|
||||
chapter_info = api.get_chapter_info(chapter_id)
|
||||
if chapter_info:
|
||||
chapters_to_process = [chapter_info]
|
||||
|
||||
if not chapters_to_process:
|
||||
logger_func("❌ No chapters found or failed to fetch chapter info.")
|
||||
return 0, 0
|
||||
|
||||
logger_func(f"✅ Found {len(chapters_to_process)} chapter(s) to download.")
|
||||
if overall_progress_callback:
|
||||
overall_progress_callback.emit(len(chapters_to_process), 0)
|
||||
|
||||
for chap_idx, chapter_json in enumerate(chapters_to_process):
|
||||
if _check_pause(): break
|
||||
try:
|
||||
metadata = api.transform_chapter_data(chapter_json)
|
||||
logger_func("-" * 40)
|
||||
logger_func(f"Processing Chapter {chap_idx + 1}/{len(chapters_to_process)}: Vol. {metadata['volume']} Ch. {metadata['chapter']}{metadata['chapter_minor']} - {metadata['title']}")
|
||||
|
||||
server_info = api.get_at_home_server(chapter_json["id"])
|
||||
if not server_info:
|
||||
logger_func(" ❌ Could not get image server for this chapter. Skipping.")
|
||||
continue
|
||||
|
||||
base_url = f"{server_info['baseUrl']}/data/{server_info['chapter']['hash']}/"
|
||||
image_files = server_info['chapter']['data']
|
||||
|
||||
series_folder = clean_folder_name(metadata['manga'])
|
||||
chapter_folder_title = metadata['title'] or ''
|
||||
chapter_folder = clean_folder_name(f"Vol {metadata['volume']:02d} Chap {metadata['chapter']:03d}{metadata['chapter_minor']} - {chapter_folder_title}".strip().strip('-').strip())
|
||||
final_save_path = os.path.join(output_dir, series_folder, chapter_folder)
|
||||
os.makedirs(final_save_path, exist_ok=True)
|
||||
|
||||
for img_idx, filename in enumerate(image_files):
|
||||
if _check_pause(): break
|
||||
|
||||
full_img_url = base_url + filename
|
||||
img_path = os.path.join(final_save_path, f"{img_idx + 1:03d}{os.path.splitext(filename)[1]}")
|
||||
|
||||
if os.path.exists(img_path):
|
||||
logger_func(f" -> Skip ({img_idx+1}/{len(image_files)}): '{os.path.basename(img_path)}' already exists.")
|
||||
grand_total_skip += 1
|
||||
continue
|
||||
|
||||
logger_func(f" Downloading ({img_idx+1}/{len(image_files)}): '{os.path.basename(img_path)}'...")
|
||||
|
||||
try:
|
||||
response = api.session.get(full_img_url, stream=True, timeout=60, headers={'Referer': 'https://mangadex.org/'})
|
||||
response.raise_for_status()
|
||||
total_size = int(response.headers.get('content-length', 0))
|
||||
|
||||
if file_progress_callback:
|
||||
file_progress_callback.emit(os.path.basename(img_path), (0, total_size))
|
||||
|
||||
with open(img_path, 'wb') as f:
|
||||
downloaded_bytes = 0
|
||||
for chunk in response.iter_content(chunk_size=8192):
|
||||
if _check_pause(): break
|
||||
f.write(chunk)
|
||||
downloaded_bytes += len(chunk)
|
||||
if file_progress_callback:
|
||||
file_progress_callback.emit(os.path.basename(img_path), (downloaded_bytes, total_size))
|
||||
|
||||
if _check_pause():
|
||||
if os.path.exists(img_path): os.remove(img_path)
|
||||
break
|
||||
|
||||
grand_total_dl += 1
|
||||
except Exception as e:
|
||||
logger_func(f" ❌ Failed to download page {img_idx+1}: {e}")
|
||||
grand_total_skip += 1
|
||||
|
||||
if overall_progress_callback:
|
||||
overall_progress_callback.emit(len(chapters_to_process), chap_idx + 1)
|
||||
time.sleep(1)
|
||||
|
||||
except Exception as e:
|
||||
logger_func(f" ❌ An unexpected error occurred while processing chapter {chapter_json.get('id')}: {e}")
|
||||
|
||||
return grand_total_dl, grand_total_skip
|
||||
|
||||
class _MangadexAPI:
|
||||
def __init__(self, logger_func):
|
||||
self.logger_func = logger_func
|
||||
self.session = cloudscraper.create_scraper()
|
||||
self.root = "https://api.mangadex.org"
|
||||
|
||||
def _call(self, endpoint, params=None, cancellation_event=None):
|
||||
if cancellation_event and cancellation_event.is_set(): return None
|
||||
try:
|
||||
response = self.session.get(f"{self.root}{endpoint}", params=params, timeout=30)
|
||||
if response.status_code == 429:
|
||||
retry_after = int(response.headers.get("X-RateLimit-Retry-After", 5))
|
||||
self.logger_func(f" ⚠️ Rate limited. Waiting for {retry_after} seconds...")
|
||||
time.sleep(retry_after)
|
||||
return self._call(endpoint, params, cancellation_event)
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
except Exception as e:
|
||||
self.logger_func(f" ❌ API call to '{endpoint}' failed: {e}")
|
||||
return None
|
||||
|
||||
def get_manga_chapters(self, series_id, cancellation_event, pause_event):
|
||||
all_chapters = []
|
||||
offset = 0
|
||||
limit = 500
|
||||
base_params = {
|
||||
"limit": limit, "order[volume]": "asc", "order[chapter]": "asc",
|
||||
"translatedLanguage[]": ["en"], "includes[]": ["scanlation_group", "user", "manga"]
|
||||
}
|
||||
while True:
|
||||
if cancellation_event.is_set(): break
|
||||
while pause_event.is_set(): time.sleep(0.5)
|
||||
|
||||
params = {**base_params, "offset": offset}
|
||||
data = self._call(f"/manga/{series_id}/feed", params, cancellation_event)
|
||||
if not data or data.get("result") != "ok": break
|
||||
|
||||
results = data.get("data", [])
|
||||
all_chapters.extend(results)
|
||||
|
||||
if (offset + limit) >= data.get("total", 0): break
|
||||
offset += limit
|
||||
return all_chapters
|
||||
|
||||
def get_chapter_info(self, chapter_id):
|
||||
params = {"includes[]": ["scanlation_group", "user", "manga"]}
|
||||
data = self._call(f"/chapter/{chapter_id}", params)
|
||||
return data.get("data") if data and data.get("result") == "ok" else None
|
||||
|
||||
def get_at_home_server(self, chapter_id):
|
||||
return self._call(f"/at-home/server/{chapter_id}")
|
||||
|
||||
def transform_chapter_data(self, chapter):
|
||||
relationships = {item["type"]: item for item in chapter.get("relationships", [])}
|
||||
manga = relationships.get("manga", {})
|
||||
c_attrs = chapter.get("attributes", {})
|
||||
m_attrs = manga.get("attributes", {})
|
||||
|
||||
chapter_num_str = c_attrs.get("chapter", "0") or "0"
|
||||
chnum, sep, minor = chapter_num_str.partition(".")
|
||||
|
||||
return {
|
||||
"manga": (m_attrs.get("title", {}).get("en") or next(iter(m_attrs.get("title", {}).values()), "Unknown Series")),
|
||||
"title": c_attrs.get("title", ""),
|
||||
"volume": int(float(c_attrs.get("volume", 0) or 0)),
|
||||
"chapter": int(float(chnum or 0)),
|
||||
"chapter_minor": sep + minor if minor else ""
|
||||
}
|
||||
44
src/core/nhentai_client.py
Normal file
44
src/core/nhentai_client.py
Normal file
@@ -0,0 +1,44 @@
|
||||
import requests
|
||||
import cloudscraper
|
||||
import json
|
||||
|
||||
def fetch_nhentai_gallery(gallery_id, logger=print):
|
||||
"""
|
||||
Fetches the metadata for a single nhentai gallery using cloudscraper to bypass Cloudflare.
|
||||
|
||||
Args:
|
||||
gallery_id (str or int): The ID of the nhentai gallery.
|
||||
logger (function): A function to log progress and error messages.
|
||||
|
||||
Returns:
|
||||
dict: A dictionary containing the gallery's metadata if successful, otherwise None.
|
||||
"""
|
||||
api_url = f"https://nhentai.net/api/gallery/{gallery_id}"
|
||||
|
||||
scraper = cloudscraper.create_scraper()
|
||||
|
||||
logger(f" Fetching nhentai gallery metadata from: {api_url}")
|
||||
|
||||
try:
|
||||
# Use the scraper to make the GET request
|
||||
response = scraper.get(api_url, timeout=20)
|
||||
|
||||
if response.status_code == 404:
|
||||
logger(f" ❌ Gallery not found (404): ID {gallery_id}")
|
||||
return None
|
||||
|
||||
response.raise_for_status()
|
||||
|
||||
gallery_data = response.json()
|
||||
|
||||
if "id" in gallery_data and "media_id" in gallery_data and "images" in gallery_data:
|
||||
logger(f" ✅ Successfully fetched metadata for '{gallery_data['title']['english']}'")
|
||||
gallery_data['pages'] = gallery_data.pop('images')['pages']
|
||||
return gallery_data
|
||||
else:
|
||||
logger(" ❌ API response is missing essential keys (id, media_id, or images).")
|
||||
return None
|
||||
|
||||
except Exception as e:
|
||||
logger(f" ❌ An error occurred while fetching gallery {gallery_id}: {e}")
|
||||
return None
|
||||
93
src/core/pixeldrain_client.py
Normal file
93
src/core/pixeldrain_client.py
Normal file
@@ -0,0 +1,93 @@
|
||||
import os
|
||||
import re
|
||||
import cloudscraper
|
||||
from ..utils.file_utils import clean_folder_name
|
||||
# --- ADDED IMPORTS ---
|
||||
from requests.adapters import HTTPAdapter
|
||||
from urllib3.util.retry import Retry
|
||||
|
||||
def fetch_pixeldrain_data(url: str, logger):
|
||||
"""
|
||||
Scrapes a given Pixeldrain URL to extract album or file information.
|
||||
Handles single files (/u/), albums/lists (/l/), and folders (/d/).
|
||||
"""
|
||||
logger(f"Fetching data for Pixeldrain URL: {url}")
|
||||
scraper = cloudscraper.create_scraper()
|
||||
root = "https://pixeldrain.com"
|
||||
|
||||
# --- START OF FIX: Add a robust retry strategy ---
|
||||
try:
|
||||
retry_strategy = Retry(
|
||||
total=5, # Total number of retries
|
||||
backoff_factor=1, # Wait 1s, 2s, 4s, 8s between retries
|
||||
status_forcelist=[429, 500, 502, 503, 504], # Retry on these server errors
|
||||
allowed_methods=["HEAD", "GET"]
|
||||
)
|
||||
adapter = HTTPAdapter(max_retries=retry_strategy)
|
||||
scraper.mount("https://", adapter)
|
||||
scraper.mount("http://", adapter)
|
||||
logger(" [Pixeldrain] Configured retry strategy for network requests.")
|
||||
except Exception as e:
|
||||
logger(f" [Pixeldrain] ⚠️ Could not configure retry strategy: {e}")
|
||||
# --- END OF FIX ---
|
||||
|
||||
file_match = re.search(r"/u/(\w+)", url)
|
||||
album_match = re.search(r"/l/(\w+)", url)
|
||||
folder_match = re.search(r"/d/([^?]+)", url)
|
||||
|
||||
try:
|
||||
if file_match:
|
||||
file_id = file_match.group(1)
|
||||
logger(f" Detected Pixeldrain File ID: {file_id}")
|
||||
api_url = f"{root}/api/file/{file_id}/info"
|
||||
data = scraper.get(api_url).json()
|
||||
|
||||
title = data.get("name", file_id)
|
||||
|
||||
files = [{
|
||||
'url': f"{root}/api/file/{file_id}?download",
|
||||
'filename': data.get("name", f"{file_id}.tmp")
|
||||
}]
|
||||
return title, files
|
||||
|
||||
elif album_match:
|
||||
album_id = album_match.group(1)
|
||||
logger(f" Detected Pixeldrain Album ID: {album_id}")
|
||||
api_url = f"{root}/api/list/{album_id}"
|
||||
data = scraper.get(api_url).json()
|
||||
|
||||
title = data.get("title", album_id)
|
||||
|
||||
files = []
|
||||
for file_info in data.get("files", []):
|
||||
files.append({
|
||||
'url': f"{root}/api/file/{file_info['id']}?download",
|
||||
'filename': file_info.get("name", f"{file_info['id']}.tmp")
|
||||
})
|
||||
return title, files
|
||||
|
||||
elif folder_match:
|
||||
path_id = folder_match.group(1)
|
||||
logger(f" Detected Pixeldrain Folder Path: {path_id}")
|
||||
api_url = f"{root}/api/filesystem/{path_id}?stat"
|
||||
data = scraper.get(api_url).json()
|
||||
|
||||
path_info = data["path"][data["base_index"]]
|
||||
title = path_info.get("name", path_id)
|
||||
|
||||
files = []
|
||||
for child in data.get("children", []):
|
||||
if child.get("type") == "file":
|
||||
files.append({
|
||||
'url': f"{root}/api/filesystem{child['path']}?attach",
|
||||
'filename': child.get("name")
|
||||
})
|
||||
return title, files
|
||||
|
||||
else:
|
||||
logger(" ❌ Could not identify Pixeldrain URL type (file, album, or folder).")
|
||||
return None, []
|
||||
|
||||
except Exception as e:
|
||||
logger(f"❌ An error occurred while fetching Pixeldrain data: {e}")
|
||||
return None, []
|
||||
163
src/core/saint2_client.py
Normal file
163
src/core/saint2_client.py
Normal file
@@ -0,0 +1,163 @@
|
||||
import os
|
||||
import re as re_module
|
||||
import html
|
||||
import urllib.parse
|
||||
import requests
|
||||
|
||||
|
||||
PATTERN_CACHE = {}
|
||||
|
||||
def re(pattern):
|
||||
"""Compile a regular expression pattern and cache it."""
|
||||
try:
|
||||
return PATTERN_CACHE[pattern]
|
||||
except KeyError:
|
||||
p = PATTERN_CACHE[pattern] = re_module.compile(pattern)
|
||||
return p
|
||||
|
||||
def extract_from(txt, pos=None, default=""):
|
||||
"""Returns a function that extracts text between two delimiters from 'txt'."""
|
||||
def extr(begin, end, index=txt.find, txt=txt):
|
||||
nonlocal pos
|
||||
try:
|
||||
start_pos = pos if pos is not None else 0
|
||||
first = index(begin, start_pos) + len(begin)
|
||||
last = index(end, first)
|
||||
if pos is not None:
|
||||
pos = last + len(end)
|
||||
return txt[first:last]
|
||||
except (ValueError, IndexError):
|
||||
return default
|
||||
return extr
|
||||
|
||||
def nameext_from_url(url):
|
||||
"""Extract filename and extension from a URL."""
|
||||
data = {}
|
||||
filename = urllib.parse.unquote(url.partition("?")[0].rpartition("/")[2])
|
||||
name, _, ext = filename.rpartition(".")
|
||||
if name and len(ext) <= 16:
|
||||
data["filename"], data["extension"] = name, ext.lower()
|
||||
else:
|
||||
data["filename"], data["extension"] = filename, ""
|
||||
return data
|
||||
|
||||
class BaseExtractor:
|
||||
"""A simplified base class for extractors."""
|
||||
def __init__(self, match, session, logger):
|
||||
self.match = match
|
||||
self.groups = match.groups()
|
||||
self.session = session
|
||||
self.log = logger
|
||||
|
||||
def request(self, url, **kwargs):
|
||||
"""Makes an HTTP request using the session."""
|
||||
try:
|
||||
response = self.session.get(url, **kwargs)
|
||||
response.raise_for_status()
|
||||
return response
|
||||
except requests.exceptions.RequestException as e:
|
||||
self.log(f"Error making request to {url}: {e}")
|
||||
return None
|
||||
|
||||
class SaintAlbumExtractor(BaseExtractor):
|
||||
"""Extractor for saint.su albums."""
|
||||
root = "https://saint2.su"
|
||||
pattern = re(r"(?:https?://)?saint\d*\.(?:su|pk|cr|to)/a/([^/?#]+)")
|
||||
|
||||
def items(self):
|
||||
"""Generator that yields all files from an album."""
|
||||
album_id = self.groups[0]
|
||||
response = self.request(f"{self.root}/a/{album_id}")
|
||||
if not response:
|
||||
return None, []
|
||||
|
||||
extr = extract_from(response.text)
|
||||
title = extr("<title>", "<").rpartition(" - ")[0]
|
||||
self.log(f"Downloading album: {title}")
|
||||
|
||||
files_html = re_module.findall(r'<a class="image".*?</a>', response.text, re_module.DOTALL)
|
||||
file_list = []
|
||||
for i, file_html in enumerate(files_html, 1):
|
||||
file_extr = extract_from(file_html)
|
||||
file_url = html.unescape(file_extr("onclick=\"play('", "'"))
|
||||
if not file_url:
|
||||
continue
|
||||
|
||||
filename_info = nameext_from_url(file_url)
|
||||
filename = f"{filename_info['filename']}.{filename_info['extension']}"
|
||||
|
||||
file_data = {
|
||||
"url": file_url,
|
||||
"filename": filename,
|
||||
"headers": {"Referer": response.url},
|
||||
}
|
||||
file_list.append(file_data)
|
||||
|
||||
return title, file_list
|
||||
|
||||
class SaintMediaExtractor(BaseExtractor):
|
||||
"""Extractor for single saint.su media links."""
|
||||
root = "https://saint2.su"
|
||||
pattern = re(r"(?:https?://)?saint\d*\.(?:su|pk|cr|to)(/(embe)?d/([^/?#]+))")
|
||||
|
||||
def items(self):
|
||||
"""Generator that yields the single file from a media page."""
|
||||
path, embed, media_id = self.groups
|
||||
url = self.root + path
|
||||
response = self.request(url)
|
||||
if not response:
|
||||
return None, []
|
||||
|
||||
extr = extract_from(response.text)
|
||||
file_url = ""
|
||||
title = extr("<title>", "<").rpartition(" - ")[0] or media_id
|
||||
|
||||
if embed: # /embed/ link
|
||||
file_url = html.unescape(extr('<source src="', '"'))
|
||||
else: # /d/ link
|
||||
file_url = html.unescape(extr('<a href="', '"'))
|
||||
|
||||
if not file_url:
|
||||
self.log("Could not find video URL on the page.")
|
||||
return title, []
|
||||
|
||||
filename_info = nameext_from_url(file_url)
|
||||
filename = f"{filename_info['filename'] or media_id}.{filename_info['extension'] or 'mp4'}"
|
||||
|
||||
file_data = {
|
||||
"url": file_url,
|
||||
"filename": filename,
|
||||
"headers": {"Referer": response.url}
|
||||
}
|
||||
|
||||
return title, [file_data]
|
||||
|
||||
|
||||
def fetch_saint2_data(url, logger):
|
||||
"""
|
||||
Identifies the correct extractor for a saint2.su URL and returns the data.
|
||||
|
||||
Args:
|
||||
url (str): The saint2.su URL.
|
||||
logger (function): A function to log progress messages.
|
||||
|
||||
Returns:
|
||||
tuple: A tuple containing (album_title, list_of_file_dicts).
|
||||
Returns (None, []) if no data could be fetched.
|
||||
"""
|
||||
extractors = [SaintMediaExtractor, SaintAlbumExtractor]
|
||||
session = requests.Session()
|
||||
session.headers.update({
|
||||
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36'
|
||||
})
|
||||
|
||||
for extractor_cls in extractors:
|
||||
match = extractor_cls.pattern.match(url)
|
||||
if match:
|
||||
extractor = extractor_cls(match, session, logger)
|
||||
album_title, files = extractor.items()
|
||||
sanitized_title = re_module.sub(r'[<>:"/\\|?*]', '_', album_title) if album_title else "saint2_download"
|
||||
return sanitized_title, files
|
||||
|
||||
logger(f"Error: The URL '{url}' does not match a known saint2 pattern.")
|
||||
return None, []
|
||||
100
src/core/simpcity_client.py
Normal file
100
src/core/simpcity_client.py
Normal file
@@ -0,0 +1,100 @@
|
||||
# src/core/simpcity_client.py
|
||||
|
||||
import cloudscraper
|
||||
from bs4 import BeautifulSoup
|
||||
from urllib.parse import urlparse, unquote
|
||||
import os
|
||||
import re
|
||||
from ..utils.file_utils import clean_folder_name
|
||||
import urllib.parse
|
||||
|
||||
def fetch_single_simpcity_page(url, logger_func, cookies=None, post_id=None):
|
||||
"""
|
||||
Scrapes a single SimpCity page for images, external links, video tags, and iframes.
|
||||
"""
|
||||
scraper = cloudscraper.create_scraper()
|
||||
headers = {'Referer': 'https://simpcity.cr/'}
|
||||
|
||||
try:
|
||||
response = scraper.get(url, timeout=30, headers=headers, cookies=cookies)
|
||||
if response.status_code == 404:
|
||||
return None, []
|
||||
response.raise_for_status()
|
||||
soup = BeautifulSoup(response.text, 'html.parser')
|
||||
|
||||
album_title = None
|
||||
title_element = soup.find('h1', class_='p-title-value')
|
||||
if title_element:
|
||||
album_title = title_element.text.strip()
|
||||
|
||||
search_scope = soup
|
||||
if post_id:
|
||||
post_content_container = soup.find('div', attrs={'data-lb-id': f'post-{post_id}'})
|
||||
if post_content_container:
|
||||
logger_func(f" [SimpCity] ✅ Isolating search to post content container for ID {post_id}.")
|
||||
search_scope = post_content_container
|
||||
else:
|
||||
logger_func(f" [SimpCity] ⚠️ Could not find content container for post ID {post_id}.")
|
||||
|
||||
jobs_on_page = []
|
||||
|
||||
# Find native SimpCity images
|
||||
image_tags = search_scope.find_all('img', class_='bbImage')
|
||||
for img_tag in image_tags:
|
||||
thumbnail_url = img_tag.get('src')
|
||||
if not thumbnail_url or not isinstance(thumbnail_url, str) or 'saint2.su' in thumbnail_url: continue
|
||||
full_url = thumbnail_url.replace('.md.', '.')
|
||||
filename = img_tag.get('alt', '').replace('.md.', '.') or os.path.basename(unquote(urlparse(full_url).path))
|
||||
jobs_on_page.append({'type': 'image', 'filename': filename, 'url': full_url})
|
||||
|
||||
# Find links in <a> tags, now with redirect handling
|
||||
link_tags = search_scope.find_all('a', href=True)
|
||||
for link in link_tags:
|
||||
href = link.get('href', '')
|
||||
|
||||
actual_url = href
|
||||
if '/misc/goto?url=' in href:
|
||||
try:
|
||||
# Extract and decode the real URL from the 'url' parameter
|
||||
parsed_href = urlparse(href)
|
||||
query_params = dict(urllib.parse.parse_qsl(parsed_href.query))
|
||||
if 'url' in query_params:
|
||||
actual_url = unquote(query_params['url'])
|
||||
except Exception:
|
||||
actual_url = href # Fallback if parsing fails
|
||||
|
||||
# Perform all checks on the 'actual_url' which is now the real destination
|
||||
if re.search(r'pixeldrain\.com/[lud]/', actual_url): jobs_on_page.append({'type': 'pixeldrain', 'url': actual_url})
|
||||
elif re.search(r'saint2\.(su|pk|cr|to)/embed/', actual_url): jobs_on_page.append({'type': 'saint2', 'url': actual_url})
|
||||
elif re.search(r'bunkr\.(?:cr|si|la|ws|is|ru|su|red|black|media|site|to|ac|ci|fi|pk|ps|sk|ph)|bunkrr\.ru', actual_url): jobs_on_page.append({'type': 'bunkr', 'url': actual_url})
|
||||
elif re.search(r'mega\.(nz|io)', actual_url): jobs_on_page.append({'type': 'mega', 'url': actual_url})
|
||||
elif re.search(r'gofile\.io', actual_url): jobs_on_page.append({'type': 'gofile', 'url': actual_url})
|
||||
|
||||
# Find direct Saint2 video embeds in <video> tags
|
||||
video_tags = search_scope.find_all('video')
|
||||
for video in video_tags:
|
||||
source_tag = video.find('source')
|
||||
if source_tag and source_tag.get('src'):
|
||||
src_url = source_tag['src']
|
||||
if re.search(r'saint2\.(su|pk|cr|to)', src_url):
|
||||
jobs_on_page.append({'type': 'saint2_direct', 'url': src_url})
|
||||
|
||||
# Find embeds in <iframe> tags (as a fallback)
|
||||
iframe_tags = search_scope.find_all('iframe')
|
||||
for iframe in iframe_tags:
|
||||
src_url = iframe.get('src')
|
||||
if src_url and isinstance(src_url, str):
|
||||
if re.search(r'saint2\.(su|pk|cr|to)/embed/', src_url):
|
||||
jobs_on_page.append({'type': 'saint2', 'url': src_url})
|
||||
|
||||
if jobs_on_page:
|
||||
# We use a set to remove duplicate URLs that might be found in multiple ways
|
||||
unique_jobs = list({job['url']: job for job in jobs_on_page}.values())
|
||||
logger_func(f" [SimpCity] Scraper found jobs: {[job['type'] for job in unique_jobs]}")
|
||||
return album_title, unique_jobs
|
||||
|
||||
return album_title, []
|
||||
|
||||
except Exception as e:
|
||||
logger_func(f" [SimpCity] ❌ Error fetching page {url}: {e}")
|
||||
raise e
|
||||
73
src/core/toonily_client.py
Normal file
73
src/core/toonily_client.py
Normal file
@@ -0,0 +1,73 @@
|
||||
import cloudscraper
|
||||
from bs4 import BeautifulSoup
|
||||
import time
|
||||
|
||||
def get_chapter_list(series_url, logger_func):
|
||||
logger_func(f" [Toonily] Scraping series page for chapter list: {series_url}")
|
||||
scraper = cloudscraper.create_scraper()
|
||||
headers = {
|
||||
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36',
|
||||
'Referer': 'https://toonily.com/'
|
||||
}
|
||||
|
||||
try:
|
||||
response = scraper.get(series_url, timeout=30, headers=headers)
|
||||
response.raise_for_status()
|
||||
soup = BeautifulSoup(response.content, 'html.parser')
|
||||
chapter_links = soup.select('li.wp-manga-chapter > a')
|
||||
|
||||
if not chapter_links:
|
||||
logger_func(" [Toonily] ❌ Could not find any chapter links on the page.")
|
||||
return []
|
||||
|
||||
urls = [link['href'] for link in chapter_links]
|
||||
urls.reverse()
|
||||
logger_func(f" [Toonily] Found {len(urls)} chapters.")
|
||||
return urls
|
||||
|
||||
except Exception as e:
|
||||
logger_func(f" [Toonily] ❌ Error getting chapter list: {e}")
|
||||
return []
|
||||
|
||||
|
||||
def fetch_chapter_data(chapter_url, logger_func, scraper_session):
|
||||
"""
|
||||
Scrapes a single Toonily.com chapter page for its title and image URLs.
|
||||
"""
|
||||
main_series_url = chapter_url.rsplit('/', 2)[0] + '/'
|
||||
headers = {
|
||||
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36',
|
||||
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7',
|
||||
'Accept-Language': 'en-US,en;q=0.9',
|
||||
'Referer': main_series_url
|
||||
}
|
||||
|
||||
try:
|
||||
response = scraper_session.get(chapter_url, timeout=30, headers=headers)
|
||||
response.raise_for_status()
|
||||
soup = BeautifulSoup(response.content, 'html.parser')
|
||||
|
||||
title_element = soup.select_one('h1#chapter-heading')
|
||||
image_container = soup.select_one('div.reading-content')
|
||||
|
||||
if not title_element or not image_container:
|
||||
logger_func(" [Toonily] ❌ Page structure invalid. Could not find title or image container.")
|
||||
return None, None, []
|
||||
|
||||
full_chapter_title = title_element.text.strip()
|
||||
|
||||
if " - Chapter" in full_chapter_title:
|
||||
series_title = full_chapter_title.split(" - Chapter")[0].strip()
|
||||
else:
|
||||
series_title = full_chapter_title.strip()
|
||||
|
||||
chapter_title = full_chapter_title # The full string is best for the chapter folder name
|
||||
|
||||
image_elements = image_container.select('img')
|
||||
image_urls = [img.get('data-src', img.get('src')).strip() for img in image_elements if img.get('data-src') or img.get('src')]
|
||||
|
||||
return series_title, chapter_title, image_urls
|
||||
|
||||
except Exception as e:
|
||||
logger_func(f" [Toonily] ❌ An error occurred scraping chapter '{chapter_url}': {e}")
|
||||
return None, None, []
|
||||
1136
src/core/workers.py
1136
src/core/workers.py
File diff suppressed because it is too large
Load Diff
File diff suppressed because one or more lines are too long
@@ -5,9 +5,22 @@ import traceback
|
||||
import json
|
||||
import base64
|
||||
import time
|
||||
import zipfile
|
||||
import struct
|
||||
import sys
|
||||
import io
|
||||
import hashlib
|
||||
from contextlib import redirect_stdout
|
||||
from urllib.parse import urlparse, urlunparse, parse_qs, urlencode
|
||||
from concurrent.futures import ThreadPoolExecutor, as_completed
|
||||
from threading import Lock
|
||||
|
||||
# --- Third-party Library Imports ---
|
||||
import requests
|
||||
import cloudscraper
|
||||
from requests.adapters import HTTPAdapter
|
||||
from urllib3.util.retry import Retry
|
||||
from ..utils.file_utils import clean_folder_name
|
||||
|
||||
try:
|
||||
from Crypto.Cipher import AES
|
||||
@@ -22,245 +35,669 @@ except ImportError:
|
||||
GDRIVE_AVAILABLE = False
|
||||
|
||||
MEGA_API_URL = "https://g.api.mega.co.nz"
|
||||
MIN_SIZE_FOR_MULTIPART_MEGA = 20 * 1024 * 1024 # 20 MB
|
||||
NUM_PARTS_FOR_MEGA = 5
|
||||
|
||||
def _get_filename_from_headers(headers):
|
||||
"""
|
||||
Extracts a filename from the Content-Disposition header.
|
||||
(This is from your original file and is kept for Dropbox downloads)
|
||||
"""
|
||||
cd = headers.get('content-disposition')
|
||||
if not cd:
|
||||
return None
|
||||
|
||||
fname_match = re.findall('filename="?([^"]+)"?', cd)
|
||||
if fname_match:
|
||||
sanitized_name = re.sub(r'[<>:"/\\|?*]', '_', fname_match[0].strip())
|
||||
return sanitized_name
|
||||
|
||||
return None
|
||||
|
||||
# --- NEW: Helper functions for Mega decryption ---
|
||||
|
||||
def urlb64_to_b64(s):
|
||||
"""Converts a URL-safe base64 string to a standard base64 string."""
|
||||
s = s.replace('-', '+').replace('_', '/')
|
||||
s += '=' * (-len(s) % 4)
|
||||
return s
|
||||
return s.replace('-', '+').replace('_', '/')
|
||||
|
||||
def b64_to_bytes(s):
|
||||
"""Decodes a URL-safe base64 string to bytes."""
|
||||
return base64.b64decode(urlb64_to_b64(s))
|
||||
|
||||
def bytes_to_hex(b):
|
||||
"""Converts bytes to a hex string."""
|
||||
return b.hex()
|
||||
def bytes_to_b64(b):
|
||||
return base64.b64encode(b).decode('utf-8')
|
||||
|
||||
def hex_to_bytes(h):
|
||||
"""Converts a hex string to bytes."""
|
||||
return bytes.fromhex(h)
|
||||
|
||||
def hrk2hk(hex_raw_key):
|
||||
"""Derives the final AES key from the raw key components for Mega."""
|
||||
key_part1 = int(hex_raw_key[0:16], 16)
|
||||
key_part2 = int(hex_raw_key[16:32], 16)
|
||||
key_part3 = int(hex_raw_key[32:48], 16)
|
||||
key_part4 = int(hex_raw_key[48:64], 16)
|
||||
|
||||
final_key_part1 = key_part1 ^ key_part3
|
||||
final_key_part2 = key_part2 ^ key_part4
|
||||
|
||||
return f'{final_key_part1:016x}{final_key_part2:016x}'
|
||||
|
||||
def decrypt_at(at_b64, key_bytes):
|
||||
"""Decrypts the 'at' attribute to get file metadata."""
|
||||
at_bytes = b64_to_bytes(at_b64)
|
||||
iv = b'\0' * 16
|
||||
cipher = AES.new(key_bytes, AES.MODE_CBC, iv)
|
||||
decrypted_at = cipher.decrypt(at_bytes)
|
||||
return decrypted_at.decode('utf-8').strip('\0').replace('MEGA', '')
|
||||
|
||||
# --- NEW: Core Logic for Mega Downloads ---
|
||||
|
||||
def get_mega_file_info(file_id, file_key, session, logger_func):
|
||||
"""Fetches file metadata and the temporary download URL from the Mega API."""
|
||||
def _decrypt_mega_attribute(encrypted_attr_b64, key_bytes):
|
||||
try:
|
||||
hex_raw_key = bytes_to_hex(b64_to_bytes(file_key))
|
||||
hex_key = hrk2hk(hex_raw_key)
|
||||
key_bytes = hex_to_bytes(hex_key)
|
||||
attr_bytes = b64_to_bytes(encrypted_attr_b64)
|
||||
padded_len = (len(attr_bytes) + 15) & ~15
|
||||
padded_attr_bytes = attr_bytes.ljust(padded_len, b'\0')
|
||||
iv = b'\0' * 16
|
||||
cipher = AES.new(key_bytes, AES.MODE_CBC, iv)
|
||||
decrypted_attr = cipher.decrypt(padded_attr_bytes)
|
||||
json_str = decrypted_attr.strip(b'\0').decode('utf-8')
|
||||
if json_str.startswith('MEGA'):
|
||||
return json.loads(json_str[4:])
|
||||
return json.loads(json_str)
|
||||
except Exception:
|
||||
return {}
|
||||
|
||||
# Request file attributes
|
||||
payload = [{"a": "g", "p": file_id}]
|
||||
response = session.post(f"{MEGA_API_URL}/cs", json=payload, timeout=20)
|
||||
response.raise_for_status()
|
||||
res_json = response.json()
|
||||
def _decrypt_mega_key(encrypted_key_b64, master_key_bytes):
|
||||
key_bytes = b64_to_bytes(encrypted_key_b64)
|
||||
iv = b'\0' * 16
|
||||
cipher = AES.new(master_key_bytes, AES.MODE_ECB)
|
||||
return cipher.decrypt(key_bytes)
|
||||
|
||||
if isinstance(res_json, list) and isinstance(res_json[0], int) and res_json[0] < 0:
|
||||
logger_func(f" [Mega] ❌ API Error: {res_json[0]}. The link may be invalid or removed.")
|
||||
return None
|
||||
def _parse_mega_key(key_b64):
|
||||
key_bytes = b64_to_bytes(key_b64)
|
||||
key_parts = struct.unpack('>' + 'I' * (len(key_bytes) // 4), key_bytes)
|
||||
if len(key_parts) == 8:
|
||||
final_key = (key_parts[0] ^ key_parts[4], key_parts[1] ^ key_parts[5], key_parts[2] ^ key_parts[6], key_parts[3] ^ key_parts[7])
|
||||
iv = (key_parts[4], key_parts[5], 0, 0)
|
||||
key_bytes = struct.pack('>' + 'I' * 4, *final_key)
|
||||
iv_bytes = struct.pack('>' + 'I' * 4, *iv)
|
||||
return key_bytes, iv_bytes, None
|
||||
elif len(key_parts) == 4:
|
||||
return key_bytes, None, None
|
||||
raise ValueError("Invalid Mega key length")
|
||||
|
||||
file_size = res_json[0]['s']
|
||||
at_b64 = res_json[0]['at']
|
||||
def _process_file_key(file_key_bytes):
|
||||
key_parts = struct.unpack('>' + 'I' * 8, file_key_bytes)
|
||||
final_key_parts = (key_parts[0] ^ key_parts[4], key_parts[1] ^ key_parts[5], key_parts[2] ^ key_parts[6], key_parts[3] ^ key_parts[7])
|
||||
return struct.pack('>' + 'I' * 4, *final_key_parts)
|
||||
|
||||
# Decrypt attributes to get the file name
|
||||
at_dec_json_str = decrypt_at(at_b64, key_bytes)
|
||||
at_dec_json = json.loads(at_dec_json_str)
|
||||
file_name = at_dec_json['n']
|
||||
def _download_and_decrypt_chunk(args):
|
||||
url, temp_path, start_byte, end_byte, key, nonce, part_num, progress_data, progress_callback_func, file_name, cancellation_event, pause_event = args
|
||||
try:
|
||||
headers = {'Range': f'bytes={start_byte}-{end_byte}'}
|
||||
initial_counter = start_byte // 16
|
||||
cipher = AES.new(key, AES.MODE_CTR, nonce=nonce, initial_value=initial_counter)
|
||||
|
||||
with requests.get(url, headers=headers, stream=True, timeout=(15, 300)) as r:
|
||||
r.raise_for_status()
|
||||
with open(temp_path, 'wb') as f:
|
||||
for chunk in r.iter_content(chunk_size=8192):
|
||||
if cancellation_event and cancellation_event.is_set():
|
||||
return False
|
||||
while pause_event and pause_event.is_set():
|
||||
time.sleep(0.5)
|
||||
if cancellation_event and cancellation_event.is_set():
|
||||
return False
|
||||
|
||||
# Request the temporary download URL
|
||||
payload = [{"a": "g", "g": 1, "p": file_id}]
|
||||
response = session.post(f"{MEGA_API_URL}/cs", json=payload, timeout=20)
|
||||
response.raise_for_status()
|
||||
res_json = response.json()
|
||||
dl_temp_url = res_json[0]['g']
|
||||
decrypted_chunk = cipher.decrypt(chunk)
|
||||
f.write(decrypted_chunk)
|
||||
with progress_data['lock']:
|
||||
progress_data['downloaded'] += len(chunk)
|
||||
if progress_callback_func and (time.time() - progress_data['last_update'] > 1):
|
||||
progress_callback_func(file_name, (progress_data['downloaded'], progress_data['total_size']))
|
||||
progress_data['last_update'] = time.time()
|
||||
return True
|
||||
except Exception as e:
|
||||
return False
|
||||
|
||||
return {
|
||||
'file_name': file_name,
|
||||
'file_size': file_size,
|
||||
'dl_url': dl_temp_url,
|
||||
'hex_raw_key': hex_raw_key
|
||||
}
|
||||
except (requests.RequestException, json.JSONDecodeError, KeyError, ValueError) as e:
|
||||
logger_func(f" [Mega] ❌ Failed to get file info: {e}")
|
||||
return None
|
||||
|
||||
def download_and_decrypt_mega_file(info, download_path, logger_func):
|
||||
"""Downloads the file and decrypts it chunk by chunk, reporting progress."""
|
||||
def download_and_decrypt_mega_file(info, download_path, logger_func, progress_callback_func=None, cancellation_event=None, pause_event=None):
|
||||
file_name = info['file_name']
|
||||
file_size = info['file_size']
|
||||
dl_url = info['dl_url']
|
||||
hex_raw_key = info['hex_raw_key']
|
||||
|
||||
final_path = os.path.join(download_path, file_name)
|
||||
|
||||
if os.path.exists(final_path) and os.path.getsize(final_path) == file_size:
|
||||
logger_func(f" [Mega] ℹ️ File '{file_name}' already exists with the correct size. Skipping.")
|
||||
return
|
||||
|
||||
# Prepare for decryption
|
||||
key = hex_to_bytes(hrk2hk(hex_raw_key))
|
||||
iv_hex = hex_raw_key[32:48] + '0000000000000000'
|
||||
iv_bytes = hex_to_bytes(iv_hex)
|
||||
cipher = AES.new(key, AES.MODE_CTR, initial_value=iv_bytes, nonce=b'')
|
||||
os.makedirs(download_path, exist_ok=True)
|
||||
key, iv, _ = _parse_mega_key(urlb64_to_b64(info['file_key']))
|
||||
nonce = iv[:8]
|
||||
|
||||
# Check for cancellation before starting
|
||||
if cancellation_event and cancellation_event.is_set():
|
||||
logger_func(f" [Mega] Download for '{file_name}' cancelled before starting.")
|
||||
return
|
||||
|
||||
try:
|
||||
with requests.get(dl_url, stream=True, timeout=(15, 300)) as r:
|
||||
r.raise_for_status()
|
||||
downloaded_bytes = 0
|
||||
last_log_time = time.time()
|
||||
if file_size < MIN_SIZE_FOR_MULTIPART_MEGA:
|
||||
logger_func(f" [Mega] Downloading '{file_name}' (Single Stream)...")
|
||||
try:
|
||||
cipher = AES.new(key, AES.MODE_CTR, nonce=nonce, initial_value=0)
|
||||
with requests.get(dl_url, stream=True, timeout=(15, 300)) as r:
|
||||
r.raise_for_status()
|
||||
downloaded_bytes = 0
|
||||
last_update_time = time.time()
|
||||
with open(final_path, 'wb') as f:
|
||||
for chunk in r.iter_content(chunk_size=8192):
|
||||
if cancellation_event and cancellation_event.is_set():
|
||||
break
|
||||
while pause_event and pause_event.is_set():
|
||||
time.sleep(0.5)
|
||||
if cancellation_event and cancellation_event.is_set():
|
||||
break
|
||||
if cancellation_event and cancellation_event.is_set():
|
||||
break
|
||||
|
||||
decrypted_chunk = cipher.decrypt(chunk)
|
||||
f.write(decrypted_chunk)
|
||||
downloaded_bytes += len(chunk)
|
||||
current_time = time.time()
|
||||
if current_time - last_update_time > 1:
|
||||
if progress_callback_func:
|
||||
progress_callback_func(file_name, (downloaded_bytes, file_size))
|
||||
last_update_time = time.time()
|
||||
|
||||
with open(final_path, 'wb') as f:
|
||||
for chunk in r.iter_content(chunk_size=8192):
|
||||
if not chunk:
|
||||
continue
|
||||
decrypted_chunk = cipher.decrypt(chunk)
|
||||
f.write(decrypted_chunk)
|
||||
downloaded_bytes += len(chunk)
|
||||
|
||||
# Log progress every second
|
||||
current_time = time.time()
|
||||
if current_time - last_log_time > 1:
|
||||
progress_percent = (downloaded_bytes / file_size) * 100 if file_size > 0 else 0
|
||||
logger_func(f" [Mega] Downloading '{file_name}': {downloaded_bytes/1024/1024:.2f}MB / {file_size/1024/1024:.2f}MB ({progress_percent:.1f}%)")
|
||||
last_log_time = current_time
|
||||
if cancellation_event and cancellation_event.is_set():
|
||||
logger_func(f" [Mega] ❌ Download cancelled for '{file_name}'. Deleting partial file.")
|
||||
if os.path.exists(final_path): os.remove(final_path)
|
||||
else:
|
||||
logger_func(f" [Mega] ✅ Successfully downloaded '{file_name}'")
|
||||
|
||||
except Exception as e:
|
||||
logger_func(f" [Mega] ❌ Download failed for '{file_name}': {e}")
|
||||
if os.path.exists(final_path): os.remove(final_path)
|
||||
else:
|
||||
logger_func(f" [Mega] Downloading '{file_name}' ({NUM_PARTS_FOR_MEGA} Parts)...")
|
||||
chunk_size = file_size // NUM_PARTS_FOR_MEGA
|
||||
chunks = []
|
||||
for i in range(NUM_PARTS_FOR_MEGA):
|
||||
start = i * chunk_size
|
||||
end = start + chunk_size - 1 if i < NUM_PARTS_FOR_MEGA - 1 else file_size - 1
|
||||
chunks.append((start, end))
|
||||
|
||||
progress_data = {'downloaded': 0, 'total_size': file_size, 'lock': Lock(), 'last_update': time.time()}
|
||||
|
||||
logger_func(f" [Mega] ✅ Successfully downloaded '{file_name}' to '{download_path}'")
|
||||
except requests.RequestException as e:
|
||||
logger_func(f" [Mega] ❌ Download failed for '{file_name}': {e}")
|
||||
except IOError as e:
|
||||
logger_func(f" [Mega] ❌ Could not write to file '{final_path}': {e}")
|
||||
tasks = []
|
||||
for i, (start, end) in enumerate(chunks):
|
||||
temp_path = f"{final_path}.part{i}"
|
||||
tasks.append((dl_url, temp_path, start, end, key, nonce, i, progress_data, progress_callback_func, file_name, cancellation_event, pause_event))
|
||||
|
||||
all_parts_successful = True
|
||||
with ThreadPoolExecutor(max_workers=NUM_PARTS_FOR_MEGA) as executor:
|
||||
if cancellation_event and cancellation_event.is_set():
|
||||
executor.shutdown(wait=False, cancel_futures=True)
|
||||
all_parts_successful = False
|
||||
else:
|
||||
results = executor.map(_download_and_decrypt_chunk, tasks)
|
||||
for result in results:
|
||||
if not result:
|
||||
all_parts_successful = False
|
||||
|
||||
# Check for cancellation after threads finish/are cancelled
|
||||
if cancellation_event and cancellation_event.is_set():
|
||||
all_parts_successful = False
|
||||
logger_func(f" [Mega] ❌ Multipart download cancelled for '{file_name}'.")
|
||||
|
||||
if all_parts_successful:
|
||||
logger_func(f" [Mega] All parts for '{file_name}' downloaded. Assembling file...")
|
||||
try:
|
||||
with open(final_path, 'wb') as f_out:
|
||||
for i in range(NUM_PARTS_FOR_MEGA):
|
||||
part_path = f"{final_path}.part{i}"
|
||||
with open(part_path, 'rb') as f_in:
|
||||
f_out.write(f_in.read())
|
||||
os.remove(part_path)
|
||||
logger_func(f" [Mega] ✅ Successfully downloaded and assembled '{file_name}'")
|
||||
except Exception as e:
|
||||
logger_func(f" [Mega] ❌ File assembly failed for '{file_name}': {e}")
|
||||
else:
|
||||
logger_func(f" [Mega] ❌ Multipart download failed or was cancelled for '{file_name}'. Cleaning up partial files.")
|
||||
for i in range(NUM_PARTS_FOR_MEGA):
|
||||
part_path = f"{final_path}.part{i}"
|
||||
if os.path.exists(part_path):
|
||||
os.remove(part_path)
|
||||
|
||||
|
||||
def _process_mega_folder(folder_id, folder_key, session, logger_func):
|
||||
try:
|
||||
master_key_bytes, _, _ = _parse_mega_key(folder_key)
|
||||
payload = [{"a": "f", "c": 1, "r": 1}]
|
||||
params = {'n': folder_id}
|
||||
response = session.post(f"{MEGA_API_URL}/cs", params=params, json=payload, timeout=30)
|
||||
response.raise_for_status()
|
||||
res_json = response.json()
|
||||
|
||||
if isinstance(res_json, int) or (isinstance(res_json, list) and res_json and isinstance(res_json[0], int)):
|
||||
error_code = res_json if isinstance(res_json, int) else res_json[0]
|
||||
logger_func(f" [Mega Folder] ❌ API returned error code: {error_code}. The folder may be invalid or removed.")
|
||||
return None, None
|
||||
if not isinstance(res_json, list) or not res_json or not isinstance(res_json[0], dict) or 'f' not in res_json[0]:
|
||||
logger_func(f" [Mega Folder] ❌ Invalid folder data received: {str(res_json)[:200]}")
|
||||
return None, None
|
||||
|
||||
nodes = res_json[0]['f']
|
||||
decrypted_nodes = {}
|
||||
for node in nodes:
|
||||
try:
|
||||
encrypted_key_b64 = node['k'].split(':')[-1]
|
||||
decrypted_key_raw = _decrypt_mega_key(encrypted_key_b64, master_key_bytes)
|
||||
|
||||
attr_key = _process_file_key(decrypted_key_raw) if node.get('t') == 0 else decrypted_key_raw
|
||||
attributes = _decrypt_mega_attribute(node['a'], attr_key)
|
||||
name = re.sub(r'[<>:"/\\|?*]', '_', attributes.get('n', f"unknown_{node['h']}"))
|
||||
|
||||
decrypted_nodes[node['h']] = {"name": name, "parent": node.get('p'), "type": node.get('t'), "size": node.get('s'), "raw_key_b64": urlb64_to_b64(bytes_to_b64(decrypted_key_raw))}
|
||||
except Exception as e:
|
||||
logger_func(f" [Mega Folder] ⚠️ Could not process node {node.get('h')}: {e}")
|
||||
|
||||
root_name = decrypted_nodes.get(folder_id, {}).get("name", "Mega_Folder")
|
||||
files_to_download = []
|
||||
for handle, node_info in decrypted_nodes.items():
|
||||
if node_info.get("type") == 0:
|
||||
path_parts = [node_info['name']]
|
||||
current_parent_id = node_info.get('parent')
|
||||
while current_parent_id in decrypted_nodes:
|
||||
parent_node = decrypted_nodes[current_parent_id]
|
||||
path_parts.insert(0, parent_node['name'])
|
||||
current_parent_id = parent_node.get('parent')
|
||||
if current_parent_id == folder_id:
|
||||
break
|
||||
files_to_download.append({'h': handle, 's': node_info['size'], 'key': node_info['raw_key_b64'], 'relative_path': os.path.join(*path_parts)})
|
||||
|
||||
return root_name, files_to_download
|
||||
except Exception as e:
|
||||
logger_func(f" [Mega] ❌ An unexpected error occurred during download/decryption: {e}")
|
||||
logger_func(f" [Mega Folder] ❌ Failed to get folder info: {e}")
|
||||
return None, None
|
||||
|
||||
|
||||
# --- REPLACEMENT Main Service Downloader Function for Mega ---
|
||||
|
||||
def download_mega_file(mega_url, download_path, logger_func=print):
|
||||
"""
|
||||
Downloads a file from a Mega.nz URL using direct requests and decryption.
|
||||
This replaces the old mega.py implementation.
|
||||
"""
|
||||
def download_mega_file(mega_url, download_path, logger_func=print, progress_callback_func=None, overall_progress_callback=None, cancellation_event=None, pause_event=None):
|
||||
if not PYCRYPTODOME_AVAILABLE:
|
||||
logger_func("❌ Mega download failed: 'pycryptodome' library is not installed. Please run: pip install pycryptodome")
|
||||
logger_func("❌ Mega download failed: 'pycryptodome' library is not installed.")
|
||||
if overall_progress_callback: overall_progress_callback(1, 1)
|
||||
return
|
||||
|
||||
logger_func(f" [Mega] Initializing download for: {mega_url}")
|
||||
|
||||
# Regex to capture file ID and key from both old and new URL formats
|
||||
match = re.search(r'mega(?:\.co)?\.nz/(?:file/|#!)?([a-zA-Z0-9]+)(?:#|!)([a-zA-Z0-9_.-]+)', mega_url)
|
||||
if not match:
|
||||
logger_func(f" [Mega] ❌ Error: Invalid Mega URL format.")
|
||||
return
|
||||
|
||||
file_id = match.group(1)
|
||||
file_key = match.group(2)
|
||||
|
||||
folder_match = re.search(r'mega(?:\.co)?\.nz/folder/([a-zA-Z0-9]+)#([a-zA-Z0-9_.-]+)', mega_url)
|
||||
file_match = re.search(r'mega(?:\.co)?\.nz/(?:file/|#!)?([a-zA-Z0-9]+)(?:#|!)([a-zA-Z0-9_.-]+)', mega_url)
|
||||
session = requests.Session()
|
||||
session.headers.update({'User-Agent': 'Kemono-Downloader-PyQt/1.0'})
|
||||
|
||||
file_info = get_mega_file_info(file_id, file_key, session, logger_func)
|
||||
if not file_info:
|
||||
logger_func(f" [Mega] ❌ Failed to get file info. The link may be invalid or expired. Aborting.")
|
||||
return
|
||||
|
||||
logger_func(f" [Mega] File found: '{file_info['file_name']}' (Size: {file_info['file_size'] / 1024 / 1024:.2f} MB)")
|
||||
|
||||
download_and_decrypt_mega_file(file_info, download_path, logger_func)
|
||||
if folder_match:
|
||||
folder_id, folder_key = folder_match.groups()
|
||||
logger_func(f" [Mega] Folder link detected. Starting crawl...")
|
||||
root_folder_name, files = _process_mega_folder(folder_id, folder_key, session, logger_func)
|
||||
|
||||
if root_folder_name is None or files is None:
|
||||
logger_func(" [Mega Folder] ❌ Crawling failed. Aborting.")
|
||||
if overall_progress_callback: overall_progress_callback(1, 1)
|
||||
return
|
||||
|
||||
if not files:
|
||||
logger_func(" [Mega Folder] ℹ️ Folder is empty. Nothing to download.")
|
||||
if overall_progress_callback: overall_progress_callback(0, 0)
|
||||
return
|
||||
|
||||
logger_func(" [Mega Folder] Prioritizing largest files first...")
|
||||
files.sort(key=lambda f: f.get('s', 0), reverse=True)
|
||||
|
||||
# --- ORIGINAL Functions for Google Drive and Dropbox (Unchanged) ---
|
||||
total_files = len(files)
|
||||
logger_func(f" [Mega Folder] ✅ Crawl complete. Found {total_files} file(s) in folder '{root_folder_name}'.")
|
||||
|
||||
if overall_progress_callback: overall_progress_callback(total_files, 0)
|
||||
|
||||
def download_gdrive_file(url, download_path, logger_func=print):
|
||||
"""Downloads a file from a Google Drive link."""
|
||||
folder_download_path = os.path.join(download_path, root_folder_name)
|
||||
os.makedirs(folder_download_path, exist_ok=True)
|
||||
|
||||
progress_lock = Lock()
|
||||
processed_count = 0
|
||||
MAX_WORKERS = 3
|
||||
|
||||
logger_func(f" [Mega Folder] Starting concurrent download with up to {MAX_WORKERS} workers...")
|
||||
|
||||
def _download_worker(file_data):
|
||||
nonlocal processed_count
|
||||
try:
|
||||
if cancellation_event and cancellation_event.is_set():
|
||||
return
|
||||
|
||||
params = {'n': folder_id}
|
||||
payload = [{"a": "g", "g": 1, "n": file_data['h']}]
|
||||
response = session.post(f"{MEGA_API_URL}/cs", params=params, json=payload, timeout=20)
|
||||
response.raise_for_status()
|
||||
res_json = response.json()
|
||||
|
||||
if isinstance(res_json, int) or (isinstance(res_json, list) and res_json and isinstance(res_json[0], int)):
|
||||
error_code = res_json if isinstance(res_json, int) else res_json[0]
|
||||
logger_func(f" [Mega Worker] ❌ API Error {error_code} for '{file_data['relative_path']}'. Skipping.")
|
||||
return
|
||||
|
||||
dl_temp_url = res_json[0]['g']
|
||||
file_info = {'file_name': os.path.basename(file_data['relative_path']), 'file_size': file_data['s'], 'dl_url': dl_temp_url, 'file_key': file_data['key']}
|
||||
file_specific_path = os.path.dirname(file_data['relative_path'])
|
||||
final_download_dir = os.path.join(folder_download_path, file_specific_path)
|
||||
|
||||
download_and_decrypt_mega_file(file_info, final_download_dir, logger_func, progress_callback_func, cancellation_event, pause_event)
|
||||
|
||||
except Exception as e:
|
||||
# Don't log error if it was a cancellation
|
||||
if not (cancellation_event and cancellation_event.is_set()):
|
||||
logger_func(f" [Mega Worker] ❌ Failed to process '{file_data['relative_path']}': {e}")
|
||||
finally:
|
||||
with progress_lock:
|
||||
processed_count += 1
|
||||
if overall_progress_callback:
|
||||
overall_progress_callback(total_files, processed_count)
|
||||
|
||||
with ThreadPoolExecutor(max_workers=MAX_WORKERS) as executor:
|
||||
futures = [executor.submit(_download_worker, file_data) for file_data in files]
|
||||
for future in as_completed(futures):
|
||||
if cancellation_event and cancellation_event.is_set():
|
||||
# Attempt to cancel remaining futures
|
||||
for f in futures:
|
||||
if not f.done():
|
||||
f.cancel()
|
||||
break
|
||||
try:
|
||||
future.result()
|
||||
except Exception as e:
|
||||
if not (cancellation_event and cancellation_event.is_set()):
|
||||
logger_func(f" [Mega Folder] A download worker failed with an error: {e}")
|
||||
|
||||
logger_func(" [Mega Folder] ✅ All concurrent downloads complete or cancelled.")
|
||||
|
||||
elif file_match:
|
||||
if overall_progress_callback: overall_progress_callback(1, 0)
|
||||
file_id, file_key = file_match.groups()
|
||||
try:
|
||||
payload = [{"a": "g", "p": file_id}]
|
||||
response = session.post(f"{MEGA_API_URL}/cs", json=payload, timeout=20)
|
||||
res_json = response.json()
|
||||
if isinstance(res_json, list) and res_json and isinstance(res_json[0], int):
|
||||
logger_func(f" [Mega] ❌ API Error {res_json[0]}. Link may be invalid or removed.")
|
||||
if overall_progress_callback: overall_progress_callback(1, 1)
|
||||
return
|
||||
|
||||
file_size = res_json[0]['s']
|
||||
at_b64 = res_json[0]['at']
|
||||
raw_file_key_bytes = b64_to_bytes(file_key)
|
||||
attr_key_bytes = _process_file_key(raw_file_key_bytes)
|
||||
attrs = _decrypt_mega_attribute(at_b64, attr_key_bytes)
|
||||
|
||||
file_name = attrs.get('n', f"unknown_file_{file_id}")
|
||||
payload_dl = [{"a": "g", "g": 1, "p": file_id}]
|
||||
response_dl = session.post(f"{MEGA_API_URL}/cs", json=payload_dl, timeout=20)
|
||||
dl_temp_url = response_dl.json()[0]['g']
|
||||
file_info_obj = {'file_name': file_name, 'file_size': file_size, 'dl_url': dl_temp_url, 'file_key': file_key}
|
||||
|
||||
download_and_decrypt_mega_file(file_info_obj, download_path, logger_func, progress_callback_func, cancellation_event, pause_event)
|
||||
|
||||
if overall_progress_callback: overall_progress_callback(1, 1)
|
||||
except Exception as e:
|
||||
if not (cancellation_event and cancellation_event.is_set()):
|
||||
logger_func(f" [Mega] ❌ Failed to process single file: {e}")
|
||||
if overall_progress_callback: overall_progress_callback(1, 1)
|
||||
else:
|
||||
logger_func(f" [Mega] ❌ Error: Invalid or unsupported Mega URL format.")
|
||||
if '/folder/' in mega_url and '/file/' in mega_url:
|
||||
logger_func(" [Mega] ℹ️ This looks like a link to a file inside a folder. Please use a direct, shareable link to the individual file.")
|
||||
if overall_progress_callback: overall_progress_callback(1, 1)
|
||||
|
||||
def download_gdrive_file(url, download_path, logger_func=print, progress_callback_func=None, overall_progress_callback=None, use_post_subfolder=False, post_title=None):
|
||||
if not GDRIVE_AVAILABLE:
|
||||
logger_func("❌ Google Drive download failed: 'gdown' library is not installed.")
|
||||
return
|
||||
|
||||
# --- Subfolder Logic ---
|
||||
final_download_path = download_path
|
||||
if use_post_subfolder and post_title:
|
||||
subfolder_name = clean_folder_name(post_title)
|
||||
final_download_path = os.path.join(download_path, subfolder_name)
|
||||
logger_func(f" [G-Drive] Using post subfolder: '{subfolder_name}'")
|
||||
os.makedirs(final_download_path, exist_ok=True)
|
||||
# --- End Subfolder Logic ---
|
||||
|
||||
original_stdout = sys.stdout
|
||||
original_stderr = sys.stderr
|
||||
captured_output_buffer = io.StringIO()
|
||||
|
||||
paths = None
|
||||
try:
|
||||
logger_func(f" [G-Drive] Starting download for: {url}")
|
||||
logger_func(" [G-Drive] Download in progress... This may take some time. Please wait.")
|
||||
logger_func(f" [G-Drive] Starting folder download for: {url}")
|
||||
|
||||
output_path = gdown.download(url, output=download_path, quiet=True, fuzzy=True)
|
||||
sys.stdout = captured_output_buffer
|
||||
sys.stderr = captured_output_buffer
|
||||
|
||||
paths = gdown.download_folder(url, output=final_download_path, quiet=False, use_cookies=False, remaining_ok=True)
|
||||
|
||||
if output_path and os.path.exists(output_path):
|
||||
logger_func(f" [G-Drive] ✅ Successfully downloaded to '{output_path}'")
|
||||
else:
|
||||
logger_func(f" [G-Drive] ❌ Download failed. The file may have been moved, deleted, or is otherwise inaccessible.")
|
||||
except Exception as e:
|
||||
logger_func(f" [G-Drive] ❌ An unexpected error occurred: {e}")
|
||||
logger_func(" [G-Drive] ℹ️ This can happen if the folder is private, deleted, or you have been rate-limited by Google.")
|
||||
finally:
|
||||
sys.stdout = original_stdout
|
||||
sys.stderr = original_stderr
|
||||
|
||||
def download_dropbox_file(dropbox_link, download_path=".", logger_func=print):
|
||||
"""
|
||||
Downloads a file from a public Dropbox link by modifying the URL for direct download.
|
||||
"""
|
||||
captured_output = captured_output_buffer.getvalue()
|
||||
if captured_output:
|
||||
processed_files_count = 0
|
||||
current_filename = None
|
||||
|
||||
if overall_progress_callback:
|
||||
overall_progress_callback(-1, 0)
|
||||
|
||||
lines = captured_output.splitlines()
|
||||
for i, line in enumerate(lines):
|
||||
cleaned_line = line.strip('\r').strip()
|
||||
if not cleaned_line:
|
||||
continue
|
||||
|
||||
if cleaned_line.startswith("To: "):
|
||||
try:
|
||||
if current_filename:
|
||||
logger_func(f" [G-Drive] ✅ Saved '{current_filename}'")
|
||||
|
||||
filepath = cleaned_line[4:]
|
||||
current_filename = os.path.basename(filepath)
|
||||
processed_files_count += 1
|
||||
|
||||
logger_func(f" [G-Drive] ({processed_files_count}/?) Downloading '{current_filename}'...")
|
||||
if progress_callback_func:
|
||||
progress_callback_func(current_filename, "In Progress...")
|
||||
if overall_progress_callback:
|
||||
overall_progress_callback(-1, processed_files_count -1)
|
||||
|
||||
except Exception:
|
||||
logger_func(f" [gdown] {cleaned_line}")
|
||||
|
||||
if current_filename:
|
||||
logger_func(f" [G-Drive] ✅ Saved '{current_filename}'")
|
||||
if overall_progress_callback:
|
||||
overall_progress_callback(-1, processed_files_count)
|
||||
|
||||
if paths and all(os.path.exists(p) for p in paths):
|
||||
final_folder_path = os.path.dirname(paths[0]) if paths else final_download_path
|
||||
logger_func(f" [G-Drive] ✅ Finished. Downloaded {len(paths)} file(s) to folder '{final_folder_path}'")
|
||||
else:
|
||||
logger_func(f" [G-Drive] ❌ Download failed or folder was empty. Check the log above for details from gdown.")
|
||||
|
||||
def download_dropbox_file(dropbox_link, download_path=".", logger_func=print, progress_callback_func=None, use_post_subfolder=False, post_title=None):
|
||||
logger_func(f" [Dropbox] Attempting to download: {dropbox_link}")
|
||||
|
||||
|
||||
final_download_path = download_path
|
||||
if use_post_subfolder and post_title:
|
||||
subfolder_name = clean_folder_name(post_title)
|
||||
final_download_path = os.path.join(download_path, subfolder_name)
|
||||
logger_func(f" [Dropbox] Using post subfolder: '{subfolder_name}'")
|
||||
|
||||
parsed_url = urlparse(dropbox_link)
|
||||
query_params = parse_qs(parsed_url.query)
|
||||
query_params['dl'] = ['1']
|
||||
new_query = urlencode(query_params, doseq=True)
|
||||
direct_download_url = urlunparse(parsed_url._replace(query=new_query))
|
||||
|
||||
logger_func(f" [Dropbox] Using direct download URL: {direct_download_url}")
|
||||
|
||||
scraper = cloudscraper.create_scraper()
|
||||
try:
|
||||
if not os.path.exists(download_path):
|
||||
os.makedirs(download_path, exist_ok=True)
|
||||
logger_func(f" [Dropbox] Created download directory: {download_path}")
|
||||
|
||||
with requests.get(direct_download_url, stream=True, allow_redirects=True, timeout=(10, 300)) as r:
|
||||
os.makedirs(final_download_path, exist_ok=True)
|
||||
with scraper.get(direct_download_url, stream=True, allow_redirects=True, timeout=(20, 600)) as r:
|
||||
r.raise_for_status()
|
||||
|
||||
filename = _get_filename_from_headers(r.headers) or os.path.basename(parsed_url.path) or "dropbox_file"
|
||||
full_save_path = os.path.join(download_path, filename)
|
||||
|
||||
filename = _get_filename_from_headers(r.headers) or os.path.basename(parsed_url.path) or "dropbox_download"
|
||||
if not os.path.splitext(filename)[1]:
|
||||
filename += ".zip"
|
||||
full_save_path = os.path.join(final_download_path, filename)
|
||||
logger_func(f" [Dropbox] Starting download of '{filename}'...")
|
||||
|
||||
total_size = int(r.headers.get('content-length', 0))
|
||||
downloaded_bytes = 0
|
||||
last_log_time = time.time()
|
||||
with open(full_save_path, 'wb') as f:
|
||||
for chunk in r.iter_content(chunk_size=8192):
|
||||
f.write(chunk)
|
||||
|
||||
logger_func(f" [Dropbox] ✅ Dropbox file downloaded successfully: {full_save_path}")
|
||||
|
||||
downloaded_bytes += len(chunk)
|
||||
current_time = time.time()
|
||||
if current_time - last_log_time > 1:
|
||||
if progress_callback_func:
|
||||
progress_callback_func(filename, (downloaded_bytes, total_size))
|
||||
last_log_time = current_time
|
||||
logger_func(f" [Dropbox] ✅ Download complete: {full_save_path}")
|
||||
if zipfile.is_zipfile(full_save_path):
|
||||
logger_func(f" [Dropbox] ዚ Detected zip file. Attempting to extract...")
|
||||
extract_folder_name = os.path.splitext(filename)[0]
|
||||
extract_path = os.path.join(final_download_path, extract_folder_name)
|
||||
os.makedirs(extract_path, exist_ok=True)
|
||||
with zipfile.ZipFile(full_save_path, 'r') as zip_ref:
|
||||
zip_ref.extractall(extract_path)
|
||||
logger_func(f" [Dropbox] ✅ Successfully extracted to folder: '{extract_path}'")
|
||||
try:
|
||||
os.remove(full_save_path)
|
||||
logger_func(f" [Dropbox] 🗑️ Removed original zip file.")
|
||||
except OSError as e:
|
||||
logger_func(f" [Dropbox] ⚠️ Could not remove original zip file: {e}")
|
||||
except Exception as e:
|
||||
logger_func(f" [Dropbox] ❌ An error occurred during Dropbox download: {e}")
|
||||
traceback.print_exc(limit=2)
|
||||
raise
|
||||
|
||||
def _get_gofile_api_token(session, logger_func):
|
||||
"""Creates a temporary guest account to get an API token."""
|
||||
try:
|
||||
logger_func(" [Gofile] Creating temporary guest account for API token...")
|
||||
response = session.post("https://api.gofile.io/accounts", timeout=20)
|
||||
response.raise_for_status()
|
||||
data = response.json()
|
||||
if data.get("status") == "ok":
|
||||
token = data["data"]["token"]
|
||||
logger_func(" [Gofile] ✅ Successfully obtained API token.")
|
||||
return token
|
||||
else:
|
||||
logger_func(f" [Gofile] ❌ Failed to get API token, status: {data.get('status')}")
|
||||
return None
|
||||
except Exception as e:
|
||||
logger_func(f" [Gofile] ❌ Error creating guest account: {e}")
|
||||
return None
|
||||
|
||||
def _get_gofile_website_token(session, logger_func):
|
||||
"""Fetches the 'wt' (website token) from Gofile's global JS file."""
|
||||
try:
|
||||
logger_func(" [Gofile] Fetching website token (wt)...")
|
||||
response = session.get("https://gofile.io/dist/js/global.js", timeout=20)
|
||||
response.raise_for_status()
|
||||
match = re.search(r'\.wt = "([^"]+)"', response.text)
|
||||
if match:
|
||||
wt = match.group(1)
|
||||
logger_func(" [Gofile] ✅ Successfully fetched website token.")
|
||||
return wt
|
||||
logger_func(" [Gofile] ❌ Could not find website token in JS file.")
|
||||
return None
|
||||
except Exception as e:
|
||||
logger_func(f" [Gofile] ❌ Error fetching website token: {e}")
|
||||
return None
|
||||
|
||||
def download_gofile_folder(gofile_url, download_path, logger_func=print, progress_callback_func=None, overall_progress_callback=None):
|
||||
"""Downloads all files from a Gofile folder URL."""
|
||||
logger_func(f" [Gofile] Initializing download for: {gofile_url}")
|
||||
|
||||
match = re.search(r"gofile\.io/d/([^/?#]+)", gofile_url)
|
||||
if not match:
|
||||
logger_func(" [Gofile] ❌ Invalid Gofile folder URL format.")
|
||||
if overall_progress_callback: overall_progress_callback(1, 1)
|
||||
return
|
||||
|
||||
content_id = match.group(1)
|
||||
|
||||
scraper = cloudscraper.create_scraper()
|
||||
|
||||
try:
|
||||
retry_strategy = Retry(
|
||||
total=5,
|
||||
backoff_factor=1,
|
||||
status_forcelist=[429, 500, 502, 503, 504],
|
||||
allowed_methods=["HEAD", "GET", "POST"]
|
||||
)
|
||||
adapter = HTTPAdapter(max_retries=retry_strategy)
|
||||
scraper.mount("http://", adapter)
|
||||
scraper.mount("https://", adapter)
|
||||
logger_func(" [Gofile] 🔧 Configured robust retry strategy for network requests.")
|
||||
except Exception as e:
|
||||
logger_func(f" [Gofile] ⚠️ Could not configure retry strategy: {e}")
|
||||
|
||||
api_token = _get_gofile_api_token(scraper, logger_func)
|
||||
if not api_token:
|
||||
if overall_progress_callback: overall_progress_callback(1, 1)
|
||||
return
|
||||
|
||||
website_token = _get_gofile_website_token(scraper, logger_func)
|
||||
if not website_token:
|
||||
if overall_progress_callback: overall_progress_callback(1, 1)
|
||||
return
|
||||
|
||||
try:
|
||||
scraper.cookies.set("accountToken", api_token, domain=".gofile.io")
|
||||
scraper.headers.update({"Authorization": f"Bearer {api_token}"})
|
||||
|
||||
api_url = f"https://api.gofile.io/contents/{content_id}?wt={website_token}"
|
||||
logger_func(f" [Gofile] Fetching folder contents for ID: {content_id}")
|
||||
response = scraper.get(api_url, timeout=30)
|
||||
response.raise_for_status()
|
||||
data = response.json()
|
||||
|
||||
if data.get("status") != "ok":
|
||||
if data.get("status") == "error-passwordRequired":
|
||||
logger_func(" [Gofile] ❌ This folder is password protected. Downloading password-protected folders is not supported.")
|
||||
else:
|
||||
logger_func(f" [Gofile] ❌ API Error: {data.get('status')}. The folder may be expired or invalid.")
|
||||
if overall_progress_callback: overall_progress_callback(1, 1)
|
||||
return
|
||||
|
||||
folder_info = data.get("data", {})
|
||||
folder_name = clean_folder_name(folder_info.get("name", content_id))
|
||||
files_to_download = [item for item in folder_info.get("children", {}).values() if item.get("type") == "file"]
|
||||
|
||||
if not files_to_download:
|
||||
logger_func(" [Gofile] ℹ️ No files found in this Gofile folder.")
|
||||
if overall_progress_callback: overall_progress_callback(0, 0)
|
||||
return
|
||||
|
||||
final_download_path = os.path.join(download_path, folder_name)
|
||||
os.makedirs(final_download_path, exist_ok=True)
|
||||
logger_func(f" [Gofile] Found {len(files_to_download)} file(s). Saving to folder: '{folder_name}'")
|
||||
if overall_progress_callback: overall_progress_callback(len(files_to_download), 0)
|
||||
|
||||
download_session = requests.Session()
|
||||
adapter = HTTPAdapter(max_retries=Retry(
|
||||
total=5, backoff_factor=1, status_forcelist=[429, 500, 502, 503, 504]
|
||||
))
|
||||
download_session.mount("http://", adapter)
|
||||
download_session.mount("https://", adapter)
|
||||
|
||||
for i, file_info in enumerate(files_to_download):
|
||||
filename = file_info.get("name")
|
||||
file_url = file_info.get("link")
|
||||
file_size = file_info.get("size", 0)
|
||||
filepath = os.path.join(final_download_path, filename)
|
||||
|
||||
if os.path.exists(filepath) and os.path.getsize(filepath) == file_size:
|
||||
logger_func(f" [Gofile] ({i+1}/{len(files_to_download)}) ⏩ Skipping existing file: '{filename}'")
|
||||
if overall_progress_callback: overall_progress_callback(len(files_to_download), i + 1)
|
||||
continue
|
||||
|
||||
logger_func(f" [Gofile] ({i+1}/{len(files_to_download)}) 🔽 Downloading: '{filename}'")
|
||||
with download_session.get(file_url, stream=True, timeout=(60, 600)) as r:
|
||||
r.raise_for_status()
|
||||
|
||||
if progress_callback_func:
|
||||
progress_callback_func(filename, (0, file_size))
|
||||
|
||||
downloaded_bytes = 0
|
||||
last_log_time = time.time()
|
||||
with open(filepath, 'wb') as f:
|
||||
for chunk in r.iter_content(chunk_size=8192):
|
||||
f.write(chunk)
|
||||
downloaded_bytes += len(chunk)
|
||||
current_time = time.time()
|
||||
if current_time - last_log_time > 0.5: # Update slightly faster
|
||||
if progress_callback_func:
|
||||
progress_callback_func(filename, (downloaded_bytes, file_size))
|
||||
last_log_time = current_time
|
||||
|
||||
if progress_callback_func:
|
||||
progress_callback_func(filename, (file_size, file_size))
|
||||
|
||||
logger_func(f" [Gofile] ✅ Finished '{filename}'")
|
||||
if overall_progress_callback: overall_progress_callback(len(files_to_download), i + 1)
|
||||
time.sleep(1)
|
||||
|
||||
except Exception as e:
|
||||
logger_func(f" [Gofile] ❌ An error occurred during Gofile download: {e}")
|
||||
if not isinstance(e, requests.exceptions.RequestException):
|
||||
traceback.print_exc()
|
||||
|
||||
@@ -15,7 +15,7 @@ MULTIPART_DOWNLOADER_AVAILABLE = True
|
||||
|
||||
# --- Module Constants ---
|
||||
CHUNK_DOWNLOAD_RETRY_DELAY = 2
|
||||
MAX_CHUNK_DOWNLOAD_RETRIES = 1
|
||||
MAX_CHUNK_DOWNLOAD_RETRIES = 5
|
||||
DOWNLOAD_CHUNK_SIZE_ITER = 1024 * 256 # 256 KB per iteration chunk
|
||||
|
||||
|
||||
|
||||
120
src/services/updater.py
Normal file
120
src/services/updater.py
Normal file
@@ -0,0 +1,120 @@
|
||||
import sys
|
||||
import os
|
||||
import requests
|
||||
import subprocess # Keep this for now, though it's not used in the final command
|
||||
from packaging.version import parse as parse_version
|
||||
from PyQt5.QtCore import QThread, pyqtSignal
|
||||
|
||||
# Constants for the updater
|
||||
GITHUB_REPO_URL = "https://api.github.com/repos/Yuvi63771/Kemono-Downloader/releases/latest"
|
||||
EXE_NAME = "Kemono.Downloader.exe"
|
||||
|
||||
class UpdateChecker(QThread):
|
||||
"""Checks for a new version on GitHub in a background thread."""
|
||||
update_available = pyqtSignal(str, str) # new_version, download_url
|
||||
up_to_date = pyqtSignal(str)
|
||||
update_error = pyqtSignal(str)
|
||||
|
||||
def __init__(self, current_version):
|
||||
super().__init__()
|
||||
self.current_version_str = current_version.lstrip('v')
|
||||
|
||||
def run(self):
|
||||
try:
|
||||
response = requests.get(GITHUB_REPO_URL, timeout=15)
|
||||
response.raise_for_status()
|
||||
data = response.json()
|
||||
|
||||
latest_version_str = data['tag_name'].lstrip('v')
|
||||
current_version = parse_version(self.current_version_str)
|
||||
latest_version = parse_version(latest_version_str)
|
||||
|
||||
if latest_version > current_version:
|
||||
for asset in data.get('assets', []):
|
||||
if asset['name'] == EXE_NAME:
|
||||
self.update_available.emit(latest_version_str, asset['browser_download_url'])
|
||||
return
|
||||
self.update_error.emit(f"Update found, but '{EXE_NAME}' is missing from the release assets.")
|
||||
else:
|
||||
self.up_to_date.emit("You are on the latest version.")
|
||||
|
||||
except requests.exceptions.RequestException as e:
|
||||
self.update_error.emit(f"Network error: {e}")
|
||||
except Exception as e:
|
||||
self.update_error.emit(f"An error occurred: {e}")
|
||||
|
||||
|
||||
class UpdateDownloader(QThread):
|
||||
"""
|
||||
Downloads the new executable and runs an updater script that kills the old process,
|
||||
replaces the file, and displays a message in the terminal.
|
||||
"""
|
||||
download_finished = pyqtSignal()
|
||||
download_error = pyqtSignal(str)
|
||||
|
||||
def __init__(self, download_url, parent_app):
|
||||
super().__init__()
|
||||
self.download_url = download_url
|
||||
self.parent_app = parent_app
|
||||
|
||||
def run(self):
|
||||
try:
|
||||
app_path = sys.executable
|
||||
app_dir = os.path.dirname(app_path)
|
||||
temp_path = os.path.join(app_dir, f"{EXE_NAME}.tmp")
|
||||
old_path = os.path.join(app_dir, f"{EXE_NAME}.old")
|
||||
updater_script_path = os.path.join(app_dir, "updater.bat")
|
||||
|
||||
pid_file_path = os.path.join(app_dir, "updater.pid")
|
||||
|
||||
with requests.get(self.download_url, stream=True, timeout=300) as r:
|
||||
r.raise_for_status()
|
||||
with open(temp_path, 'wb') as f:
|
||||
for chunk in r.iter_content(chunk_size=8192):
|
||||
f.write(chunk)
|
||||
|
||||
with open(pid_file_path, "w") as f:
|
||||
f.write(str(os.getpid()))
|
||||
|
||||
script_content = f"""
|
||||
@echo off
|
||||
SETLOCAL
|
||||
|
||||
echo.
|
||||
echo Reading process information...
|
||||
set /p PID=<{pid_file_path}
|
||||
|
||||
echo Closing the old application (PID: %PID%)...
|
||||
taskkill /F /PID %PID%
|
||||
|
||||
echo Waiting for files to unlock...
|
||||
timeout /t 2 /nobreak > nul
|
||||
|
||||
echo Replacing application files...
|
||||
if exist "{old_path}" del /F /Q "{old_path}"
|
||||
rename "{app_path}" "{os.path.basename(old_path)}"
|
||||
rename "{temp_path}" "{EXE_NAME}"
|
||||
|
||||
echo.
|
||||
echo ============================================================
|
||||
echo Update Complete!
|
||||
echo You can now close this window and run {EXE_NAME}.
|
||||
echo ============================================================
|
||||
echo.
|
||||
pause
|
||||
|
||||
echo Cleaning up helper files...
|
||||
del "{pid_file_path}"
|
||||
del "%~f0"
|
||||
ENDLOCAL
|
||||
"""
|
||||
with open(updater_script_path, "w") as f:
|
||||
f.write(script_content)
|
||||
|
||||
# --- Go back to the os.startfile command that we know works ---
|
||||
os.startfile(updater_script_path)
|
||||
|
||||
self.download_finished.emit()
|
||||
|
||||
except Exception as e:
|
||||
self.download_error.emit(f"Failed to download or run updater: {e}")
|
||||
@@ -16,7 +16,6 @@ class CookieHelpDialog(QDialog):
|
||||
It can be displayed as a simple informational popup or as a modal choice
|
||||
when cookies are required but not found.
|
||||
"""
|
||||
# Constants to define the user's choice from the dialog
|
||||
CHOICE_PROCEED_WITHOUT_COOKIES = 1
|
||||
CHOICE_CANCEL_DOWNLOAD = 2
|
||||
CHOICE_OK_INFO_ONLY = 3
|
||||
@@ -64,7 +63,6 @@ class CookieHelpDialog(QDialog):
|
||||
button_layout.addStretch(1)
|
||||
|
||||
if self.offer_download_without_option:
|
||||
# Add buttons for making a choice
|
||||
self.download_without_button = QPushButton()
|
||||
self.download_without_button.clicked.connect(self._proceed_without_cookies)
|
||||
button_layout.addWidget(self.download_without_button)
|
||||
@@ -73,7 +71,6 @@ class CookieHelpDialog(QDialog):
|
||||
self.cancel_button.clicked.connect(self._cancel_download)
|
||||
button_layout.addWidget(self.cancel_button)
|
||||
else:
|
||||
# Add a simple OK button for informational display
|
||||
self.ok_button = QPushButton()
|
||||
self.ok_button.clicked.connect(self._ok_info_only)
|
||||
button_layout.addWidget(self.ok_button)
|
||||
|
||||
89
src/ui/dialogs/CustomFilenameDialog.py
Normal file
89
src/ui/dialogs/CustomFilenameDialog.py
Normal file
@@ -0,0 +1,89 @@
|
||||
from PyQt5.QtWidgets import (
|
||||
QDialog, QVBoxLayout, QHBoxLayout, QLabel, QLineEdit, QPushButton,
|
||||
QDialogButtonBox, QTextEdit
|
||||
)
|
||||
from PyQt5.QtCore import Qt
|
||||
|
||||
class CustomFilenameDialog(QDialog):
|
||||
"""A dialog for creating a custom filename format string."""
|
||||
|
||||
# --- REPLACE THE 'AVAILABLE_KEYS' LIST WITH THIS DICTIONARY ---
|
||||
DISPLAY_KEY_MAP = {
|
||||
"PostID": "id",
|
||||
"CreatorName": "creator_name",
|
||||
"service": "service",
|
||||
"title": "title",
|
||||
"added": "added",
|
||||
"published": "published",
|
||||
"edited": "edited",
|
||||
"name": "name"
|
||||
}
|
||||
|
||||
def __init__(self, current_format, current_date_format, parent=None):
|
||||
super().__init__(parent)
|
||||
self.setWindowTitle("Custom Filename Format")
|
||||
self.setMinimumWidth(500)
|
||||
|
||||
self.current_format = current_format
|
||||
self.current_date_format = current_date_format
|
||||
|
||||
# --- Main Layout ---
|
||||
layout = QVBoxLayout(self)
|
||||
|
||||
# --- Description ---
|
||||
description_label = QLabel(
|
||||
"Create a filename format using placeholders. The date/time values for 'added', 'published', and 'edited' will be automatically shortened to your specified format."
|
||||
)
|
||||
description_label.setWordWrap(True)
|
||||
layout.addWidget(description_label)
|
||||
|
||||
# --- Format Input ---
|
||||
format_label = QLabel("Filename Format:")
|
||||
layout.addWidget(format_label)
|
||||
self.format_input = QLineEdit(self)
|
||||
self.format_input.setText(self.current_format)
|
||||
self.format_input.setPlaceholderText("e.g., {published} {title} {id}")
|
||||
layout.addWidget(self.format_input)
|
||||
|
||||
# --- Date Format Input ---
|
||||
date_format_label = QLabel("Date Format (for {added}, {published}, {edited}):")
|
||||
layout.addWidget(date_format_label)
|
||||
self.date_format_input = QLineEdit(self)
|
||||
self.date_format_input.setText(self.current_date_format)
|
||||
self.date_format_input.setPlaceholderText("e.g., YYYY-MM-DD or DD-MM-YYYY")
|
||||
layout.addWidget(self.date_format_input)
|
||||
|
||||
# --- Available Keys Display ---
|
||||
keys_label = QLabel("Click to add a placeholder:")
|
||||
layout.addWidget(keys_label)
|
||||
|
||||
keys_layout = QHBoxLayout()
|
||||
keys_layout.setSpacing(5)
|
||||
|
||||
for display_key, internal_key in self.DISPLAY_KEY_MAP.items():
|
||||
key_button = QPushButton(f"{{{display_key}}}")
|
||||
# Use a lambda to pass the correct internal key when the button is clicked
|
||||
key_button.clicked.connect(lambda checked, key=internal_key: self.add_key_to_input(key))
|
||||
keys_layout.addWidget(key_button)
|
||||
keys_layout.addStretch()
|
||||
|
||||
layout.addLayout(keys_layout)
|
||||
|
||||
# --- OK/Cancel Buttons ---
|
||||
button_box = QDialogButtonBox(QDialogButtonBox.Ok | QDialogButtonBox.Cancel)
|
||||
button_box.accepted.connect(self.accept)
|
||||
button_box.rejected.connect(self.reject)
|
||||
layout.addWidget(button_box)
|
||||
|
||||
def add_key_to_input(self, key_to_insert):
|
||||
"""Adds the corresponding internal key placeholder to the input field."""
|
||||
self.format_input.insert(f" {{{key_to_insert}}} ")
|
||||
self.format_input.setFocus()
|
||||
|
||||
def get_format_string(self):
|
||||
"""Returns the final format string from the input field."""
|
||||
return self.format_input.text().strip()
|
||||
|
||||
def get_date_format_string(self):
|
||||
"""Returns the date format string from its input field."""
|
||||
return self.date_format_input.text().strip()
|
||||
@@ -2,77 +2,55 @@
|
||||
from PyQt5.QtCore import pyqtSignal, Qt
|
||||
from PyQt5.QtWidgets import (
|
||||
QApplication, QDialog, QHBoxLayout, QLabel, QListWidget, QListWidgetItem,
|
||||
QMessageBox, QPushButton, QVBoxLayout, QAbstractItemView, QFileDialog
|
||||
QMessageBox, QPushButton, QVBoxLayout, QAbstractItemView, QFileDialog, QCheckBox
|
||||
)
|
||||
|
||||
# --- Local Application Imports ---
|
||||
from ...i18n.translator import get_translation
|
||||
from ..assets import get_app_icon_object
|
||||
# Corrected Import: The filename uses PascalCase.
|
||||
from .ExportOptionsDialog import ExportOptionsDialog
|
||||
from ...utils.resolution import get_dark_theme
|
||||
from ...config.constants import AUTO_RETRY_ON_FINISH_KEY
|
||||
|
||||
class ErrorFilesDialog(QDialog):
|
||||
"""
|
||||
Dialog to display files that were skipped due to errors and
|
||||
allows the user to retry downloading them or export the list of URLs.
|
||||
"""
|
||||
|
||||
# Signal emitted with a list of file info dictionaries to retry
|
||||
retry_selected_signal = pyqtSignal(list)
|
||||
|
||||
def __init__(self, error_files_info_list, parent_app, parent=None):
|
||||
"""
|
||||
Initializes the dialog.
|
||||
|
||||
Args:
|
||||
error_files_info_list (list): A list of dictionaries, each containing
|
||||
info about a failed file.
|
||||
parent_app (DownloaderApp): A reference to the main application window
|
||||
for theming and translations.
|
||||
parent (QWidget, optional): The parent widget. Defaults to None.
|
||||
"""
|
||||
super().__init__(parent)
|
||||
self.parent_app = parent_app
|
||||
self.setModal(True)
|
||||
self.error_files = error_files_info_list
|
||||
|
||||
# --- Basic Window Setup ---
|
||||
app_icon = get_app_icon_object()
|
||||
if app_icon and not app_icon.isNull():
|
||||
self.setWindowIcon(app_icon)
|
||||
|
||||
# --- START OF FIX ---
|
||||
# Get the user-defined scale factor from the parent application.
|
||||
scale_factor = getattr(self.parent_app, 'scale_factor', 1.0)
|
||||
|
||||
# Define base dimensions and apply the correct scale factor.
|
||||
base_width, base_height = 550, 400
|
||||
base_width, base_height = 600, 450
|
||||
self.setMinimumSize(int(base_width * scale_factor), int(base_height * scale_factor))
|
||||
self.resize(int(base_width * scale_factor * 1.1), int(base_height * scale_factor * 1.1))
|
||||
# --- END OF FIX ---
|
||||
|
||||
# --- Initialize UI and Apply Theming ---
|
||||
self._init_ui()
|
||||
self._retranslate_ui()
|
||||
self._apply_theme()
|
||||
|
||||
def _init_ui(self):
|
||||
"""Initializes all UI components and layouts for the dialog."""
|
||||
main_layout = QVBoxLayout(self)
|
||||
|
||||
self.info_label = QLabel()
|
||||
self.info_label.setWordWrap(True)
|
||||
main_layout.addWidget(self.info_label)
|
||||
|
||||
if self.error_files:
|
||||
self.files_list_widget = QListWidget()
|
||||
self.files_list_widget.setSelectionMode(QAbstractItemView.NoSelection)
|
||||
self._populate_list()
|
||||
main_layout.addWidget(self.files_list_widget)
|
||||
self.files_list_widget = QListWidget()
|
||||
self.files_list_widget.setSelectionMode(QAbstractItemView.ExtendedSelection)
|
||||
main_layout.addWidget(self.files_list_widget)
|
||||
self._populate_list()
|
||||
|
||||
# --- Control Buttons ---
|
||||
buttons_layout = QHBoxLayout()
|
||||
|
||||
self.select_all_button = QPushButton()
|
||||
self.select_all_button.clicked.connect(self._select_all_items)
|
||||
buttons_layout.addWidget(self.select_all_button)
|
||||
@@ -81,94 +59,170 @@ class ErrorFilesDialog(QDialog):
|
||||
self.retry_button.clicked.connect(self._handle_retry_selected)
|
||||
buttons_layout.addWidget(self.retry_button)
|
||||
|
||||
self.load_button = QPushButton()
|
||||
self.load_button.clicked.connect(self._handle_load_errors_from_txt)
|
||||
buttons_layout.addWidget(self.load_button)
|
||||
|
||||
self.export_button = QPushButton()
|
||||
self.export_button.clicked.connect(self._handle_export_errors_to_txt)
|
||||
buttons_layout.addWidget(self.export_button)
|
||||
|
||||
# The stretch will push everything added after this point to the right
|
||||
buttons_layout.addStretch(1)
|
||||
|
||||
# --- MOVED: Auto Retry Checkbox ---
|
||||
self.auto_retry_checkbox = QCheckBox()
|
||||
auto_retry_enabled = self.parent_app.settings.value(AUTO_RETRY_ON_FINISH_KEY, False, type=bool)
|
||||
self.auto_retry_checkbox.setChecked(auto_retry_enabled)
|
||||
self.auto_retry_checkbox.toggled.connect(self._save_auto_retry_setting)
|
||||
buttons_layout.addWidget(self.auto_retry_checkbox)
|
||||
# --- END ---
|
||||
|
||||
self.ok_button = QPushButton()
|
||||
self.ok_button.clicked.connect(self.accept)
|
||||
self.ok_button.setDefault(True)
|
||||
buttons_layout.addWidget(self.ok_button)
|
||||
main_layout.addLayout(buttons_layout)
|
||||
|
||||
# Enable/disable buttons based on whether there are errors
|
||||
has_errors = bool(self.error_files)
|
||||
self.select_all_button.setEnabled(has_errors)
|
||||
self.retry_button.setEnabled(has_errors)
|
||||
self.export_button.setEnabled(has_errors)
|
||||
|
||||
def _populate_list(self):
|
||||
"""Populates the list widget with details of the failed files."""
|
||||
self.files_list_widget.clear()
|
||||
for error_info in self.error_files:
|
||||
filename = error_info.get('forced_filename_override',
|
||||
error_info.get('file_info', {}).get('name', 'Unknown Filename'))
|
||||
post_title = error_info.get('post_title', 'Unknown Post')
|
||||
post_id = error_info.get('original_post_id_for_log', 'N/A')
|
||||
self._add_item_to_list(error_info)
|
||||
|
||||
item_text = f"File: {filename}\nFrom Post: '{post_title}' (ID: {post_id})"
|
||||
list_item = QListWidgetItem(item_text)
|
||||
list_item.setData(Qt.UserRole, error_info)
|
||||
list_item.setFlags(list_item.flags() | Qt.ItemIsUserCheckable)
|
||||
list_item.setCheckState(Qt.Unchecked)
|
||||
self.files_list_widget.addItem(list_item)
|
||||
def _handle_load_errors_from_txt(self):
|
||||
"""Opens a file dialog to load URLs from a .txt file."""
|
||||
import re
|
||||
|
||||
filepath, _ = QFileDialog.getOpenFileName(
|
||||
self,
|
||||
self._tr("error_files_load_dialog_title", "Load Error File URLs"),
|
||||
"",
|
||||
"Text Files (*.txt);;All Files (*)"
|
||||
)
|
||||
|
||||
if not filepath:
|
||||
return
|
||||
|
||||
try:
|
||||
detailed_pattern = re.compile(r"^(https?://[^\s]+)\s*\[Post: '(.*?)' \(ID: (.*?)\), File: '(.*?)'\]$")
|
||||
simple_pattern = re.compile(r'^(https?://[^\s]+)')
|
||||
|
||||
with open(filepath, 'r', encoding='utf-8') as f:
|
||||
for line in f:
|
||||
line = line.strip()
|
||||
if not line: continue
|
||||
|
||||
url, post_title, post_id, filename = None, 'Loaded from .txt', 'N/A', None
|
||||
|
||||
detailed_match = detailed_pattern.match(line)
|
||||
if detailed_match:
|
||||
url, post_title, post_id, filename = detailed_match.groups()
|
||||
else:
|
||||
simple_match = simple_pattern.match(line)
|
||||
if simple_match:
|
||||
url = simple_match.group(1)
|
||||
filename = url.split('/')[-1]
|
||||
|
||||
if url:
|
||||
simple_error_info = {
|
||||
'is_loaded_from_txt': True, 'file_info': {'url': url, 'name': filename},
|
||||
'post_title': post_title, 'original_post_id_for_log': post_id,
|
||||
'target_folder_path': self.parent_app.dir_input.text().strip(),
|
||||
'forced_filename_override': filename, 'file_index_in_post': 0,
|
||||
'num_files_in_this_post': 1, 'service': None, 'user_id': None, 'api_url_input': ''
|
||||
}
|
||||
self.error_files.append(simple_error_info)
|
||||
self._add_item_to_list(simple_error_info)
|
||||
|
||||
self.info_label.setText(self._tr("error_files_found_label", "The following {count} file(s)...").format(count=len(self.error_files)))
|
||||
|
||||
has_errors = bool(self.error_files)
|
||||
self.select_all_button.setEnabled(has_errors)
|
||||
self.retry_button.setEnabled(has_errors)
|
||||
self.export_button.setEnabled(has_errors)
|
||||
|
||||
except Exception as e:
|
||||
QMessageBox.critical(self, self._tr("error_files_load_error_title", "Load Error"),
|
||||
self._tr("error_files_load_error_message", "Could not load or parse the file: {error}").format(error=str(e)))
|
||||
|
||||
def _tr(self, key, default_text=""):
|
||||
"""Helper to get translation based on the main application's current language."""
|
||||
if callable(get_translation) and self.parent_app:
|
||||
return get_translation(self.parent_app.current_selected_language, key, default_text)
|
||||
return default_text
|
||||
|
||||
def _retranslate_ui(self):
|
||||
"""Sets the text for all translatable UI elements."""
|
||||
self.setWindowTitle(self._tr("error_files_dialog_title", "Files Skipped Due to Errors"))
|
||||
if not self.error_files:
|
||||
self.info_label.setText(self._tr("error_files_no_errors_label", "No files were recorded as skipped..."))
|
||||
else:
|
||||
self.info_label.setText(self._tr("error_files_found_label", "The following {count} file(s)...").format(count=len(self.error_files)))
|
||||
|
||||
self.select_all_button.setText(self._tr("error_files_select_all_button", "Select All"))
|
||||
self.auto_retry_checkbox.setText(self._tr("error_files_auto_retry_checkbox", "Auto Retry at End"))
|
||||
self.select_all_button.setText(self._tr("error_files_select_all_button", "Select/Deselect All"))
|
||||
self.retry_button.setText(self._tr("error_files_retry_selected_button", "Retry Selected"))
|
||||
self.load_button.setText(self._tr("error_files_load_urls_button", "Load URLs from .txt"))
|
||||
self.export_button.setText(self._tr("error_files_export_urls_button", "Export URLs to .txt"))
|
||||
self.ok_button.setText(self._tr("ok_button", "OK"))
|
||||
|
||||
def _apply_theme(self):
|
||||
"""Applies the current theme from the parent application."""
|
||||
if self.parent_app and self.parent_app.current_theme == "dark":
|
||||
# Get the scale factor from the parent app
|
||||
scale = getattr(self.parent_app, 'scale_factor', 1)
|
||||
# Call the imported function with the correct scale
|
||||
self.setStyleSheet(get_dark_theme(scale))
|
||||
else:
|
||||
# Explicitly set a blank stylesheet for light mode
|
||||
self.setStyleSheet("")
|
||||
|
||||
def _save_auto_retry_setting(self, checked):
|
||||
"""Saves the state of the auto-retry checkbox to QSettings."""
|
||||
self.parent_app.settings.setValue(AUTO_RETRY_ON_FINISH_KEY, checked)
|
||||
|
||||
def _add_item_to_list(self, error_info):
|
||||
"""Creates and adds a single QListWidgetItem based on error_info content."""
|
||||
if error_info.get('is_loaded_from_txt'):
|
||||
filename = error_info.get('file_info', {}).get('name', 'Unknown Filename')
|
||||
post_title = error_info.get('post_title', 'N/A')
|
||||
post_id = error_info.get('original_post_id_for_log', 'N/A')
|
||||
item_text = f"File: {filename}\nPost: '{post_title}' (ID: {post_id}) [Loaded from .txt]"
|
||||
else:
|
||||
filename = error_info.get('forced_filename_override', error_info.get('file_info', {}).get('name', 'Unknown Filename'))
|
||||
post_title = error_info.get('post_title', 'Unknown Post')
|
||||
post_id = error_info.get('original_post_id_for_log', 'N/A')
|
||||
creator_name = "Unknown Creator"
|
||||
service, user_id = error_info.get('service'), error_info.get('user_id')
|
||||
if service and user_id and hasattr(self.parent_app, 'creator_name_cache'):
|
||||
creator_name = self.parent_app.creator_name_cache.get((service.lower(), str(user_id)), user_id)
|
||||
item_text = f"File: {filename}\nCreator: {creator_name} - Post: '{post_title}' (ID: {post_id})"
|
||||
|
||||
list_item = QListWidgetItem(item_text)
|
||||
list_item.setData(Qt.UserRole, error_info)
|
||||
list_item.setFlags(list_item.flags() | Qt.ItemIsUserCheckable)
|
||||
list_item.setCheckState(Qt.Unchecked) # Start as unchecked
|
||||
self.files_list_widget.addItem(list_item)
|
||||
|
||||
def _select_all_items(self):
|
||||
"""Checks all items in the list."""
|
||||
if hasattr(self, 'files_list_widget'):
|
||||
for i in range(self.files_list_widget.count()):
|
||||
self.files_list_widget.item(i).setCheckState(Qt.Checked)
|
||||
"""Toggles checking all items in the list."""
|
||||
# Determine if we should check or uncheck all based on the first item's state
|
||||
is_currently_checked = self.files_list_widget.item(0).checkState() == Qt.Checked if self.files_list_widget.count() > 0 else False
|
||||
new_state = Qt.Unchecked if is_currently_checked else Qt.Checked
|
||||
for i in range(self.files_list_widget.count()):
|
||||
self.files_list_widget.item(i).setCheckState(new_state)
|
||||
|
||||
def _handle_retry_selected(self):
|
||||
"""Gathers selected files and emits the retry signal."""
|
||||
if not hasattr(self, 'files_list_widget'):
|
||||
return
|
||||
|
||||
selected_files_for_retry = [
|
||||
self.files_list_widget.item(i).data(Qt.UserRole)
|
||||
for i in range(self.files_list_widget.count())
|
||||
if self.files_list_widget.item(i).checkState() == Qt.Checked
|
||||
]
|
||||
|
||||
if selected_files_for_retry:
|
||||
self.retry_selected_signal.emit(selected_files_for_retry)
|
||||
self.accept()
|
||||
else:
|
||||
QMessageBox.information(
|
||||
self,
|
||||
self._tr("fav_artists_no_selection_title", "No Selection"),
|
||||
self._tr("error_files_no_selection_retry_message", "Please select at least one file to retry.")
|
||||
)
|
||||
QMessageBox.information(self, self._tr("fav_artists_no_selection_title", "No Selection"),
|
||||
self._tr("error_files_no_selection_retry_message", "Please check the box next to at least one file to retry."))
|
||||
|
||||
def _handle_export_errors_to_txt(self):
|
||||
"""Exports the URLs of failed files to a text file."""
|
||||
@@ -193,10 +247,13 @@ class ErrorFilesDialog(QDialog):
|
||||
|
||||
if url:
|
||||
if export_option == ExportOptionsDialog.EXPORT_MODE_WITH_DETAILS:
|
||||
original_filename = file_info.get('name', 'Unknown Filename')
|
||||
post_title = error_item.get('post_title', 'Unknown Post')
|
||||
post_id = error_item.get('original_post_id_for_log', 'N/A')
|
||||
details_string = f" [Post: '{post_title}' (ID: {post_id}), File: '{original_filename}']"
|
||||
|
||||
# Prioritize the final renamed filename, but fall back to the original from the API
|
||||
filename_to_display = error_item.get('forced_filename_override') or file_info.get('name', 'Unknown Filename')
|
||||
|
||||
details_string = f" [Post: '{post_title}' (ID: {post_id}), File: '{filename_to_display}']"
|
||||
lines_to_export.append(f"{url}{details_string}")
|
||||
else:
|
||||
lines_to_export.append(url)
|
||||
|
||||
226
src/ui/dialogs/ExportLinksDialog.py
Normal file
226
src/ui/dialogs/ExportLinksDialog.py
Normal file
@@ -0,0 +1,226 @@
|
||||
import os
|
||||
import json
|
||||
import re
|
||||
from collections import defaultdict
|
||||
from PyQt5.QtWidgets import (
|
||||
QApplication, QWidget, QLabel, QLineEdit, QTextEdit, QPushButton,
|
||||
QVBoxLayout, QHBoxLayout, QFileDialog, QMessageBox, QListWidget, QRadioButton,
|
||||
QButtonGroup, QCheckBox, QSplitter, QGroupBox, QDialog, QStackedWidget,
|
||||
QScrollArea, QListWidgetItem, QSizePolicy, QProgressBar, QAbstractItemView, QFrame,
|
||||
QMainWindow, QAction, QGridLayout,
|
||||
)
|
||||
from PyQt5.QtCore import Qt
|
||||
|
||||
class ExportLinksDialog(QDialog):
|
||||
"""
|
||||
A dialog for exporting extracted links with various format options, including custom templates.
|
||||
"""
|
||||
def __init__(self, links_data, parent=None):
|
||||
super().__init__(parent)
|
||||
self.links_data = links_data
|
||||
self.setWindowTitle("Export Extracted Links")
|
||||
self.setMinimumWidth(550)
|
||||
self._setup_ui()
|
||||
self._update_options_visibility()
|
||||
|
||||
def _setup_ui(self):
|
||||
"""Initializes the UI components of the dialog."""
|
||||
main_layout = QVBoxLayout(self)
|
||||
|
||||
# Format Selection (Top Level)
|
||||
format_group = QGroupBox("Export Format")
|
||||
format_layout = QHBoxLayout()
|
||||
self.radio_txt = QRadioButton("Plain Text (.txt)")
|
||||
self.radio_json = QRadioButton("JSON (.json)")
|
||||
self.radio_txt.setChecked(True)
|
||||
format_layout.addWidget(self.radio_txt)
|
||||
format_layout.addWidget(self.radio_json)
|
||||
format_group.setLayout(format_layout)
|
||||
main_layout.addWidget(format_group)
|
||||
|
||||
# TXT Options Group
|
||||
self.txt_options_group = QGroupBox("TXT Options")
|
||||
txt_options_layout = QVBoxLayout()
|
||||
|
||||
self.txt_mode_group = QButtonGroup(self)
|
||||
self.radio_simple = QRadioButton("Simple (URL only, one per line)")
|
||||
self.radio_detailed = QRadioButton("Detailed (with checkboxes)")
|
||||
self.radio_custom = QRadioButton("Custom Format Template")
|
||||
|
||||
self.txt_mode_group.addButton(self.radio_simple)
|
||||
self.txt_mode_group.addButton(self.radio_detailed)
|
||||
self.txt_mode_group.addButton(self.radio_custom)
|
||||
|
||||
txt_options_layout.addWidget(self.radio_simple)
|
||||
txt_options_layout.addWidget(self.radio_detailed)
|
||||
|
||||
self.detailed_options_widget = QWidget()
|
||||
detailed_layout = QVBoxLayout(self.detailed_options_widget)
|
||||
detailed_layout.setContentsMargins(20, 5, 0, 5)
|
||||
self.check_include_titles = QCheckBox("Include post titles as separators")
|
||||
self.check_include_link_text = QCheckBox("Include link text/description")
|
||||
self.check_include_platform = QCheckBox("Include platform (e.g., Mega, GDrive)")
|
||||
detailed_layout.addWidget(self.check_include_titles)
|
||||
detailed_layout.addWidget(self.check_include_link_text)
|
||||
detailed_layout.addWidget(self.check_include_platform)
|
||||
txt_options_layout.addWidget(self.detailed_options_widget)
|
||||
|
||||
txt_options_layout.addWidget(self.radio_custom)
|
||||
|
||||
self.custom_format_widget = QWidget()
|
||||
custom_layout = QVBoxLayout(self.custom_format_widget)
|
||||
custom_layout.setContentsMargins(20, 5, 0, 5)
|
||||
placeholders_label = QLabel("Available placeholders: <b>{url} {post_title} {link_text} {platform} {key}</b>")
|
||||
self.custom_format_input = QTextEdit()
|
||||
self.custom_format_input.setAcceptRichText(False)
|
||||
self.custom_format_input.setPlaceholderText("Enter your format, e.g., ({url}) or Title: {post_title}\\nLink: {url}")
|
||||
self.custom_format_input.setText("{url}")
|
||||
self.custom_format_input.setFixedHeight(80)
|
||||
custom_layout.addWidget(placeholders_label)
|
||||
custom_layout.addWidget(self.custom_format_input)
|
||||
txt_options_layout.addWidget(self.custom_format_widget)
|
||||
|
||||
separator = QLabel("-" * 70)
|
||||
txt_options_layout.addWidget(separator)
|
||||
self.check_separate_files = QCheckBox("Save each platform to a separate file (e.g., export_mega.txt)")
|
||||
txt_options_layout.addWidget(self.check_separate_files)
|
||||
|
||||
self.txt_options_group.setLayout(txt_options_layout)
|
||||
main_layout.addWidget(self.txt_options_group)
|
||||
|
||||
# File Path Selection
|
||||
path_layout = QHBoxLayout()
|
||||
self.path_input = QLineEdit()
|
||||
self.browse_button = QPushButton("Browse...")
|
||||
path_layout.addWidget(self.path_input)
|
||||
path_layout.addWidget(self.browse_button)
|
||||
main_layout.addLayout(path_layout)
|
||||
|
||||
# Action Buttons
|
||||
button_layout = QHBoxLayout()
|
||||
button_layout.addStretch(1)
|
||||
self.export_button = QPushButton("Export")
|
||||
self.cancel_button = QPushButton("Cancel")
|
||||
button_layout.addWidget(self.export_button)
|
||||
button_layout.addWidget(self.cancel_button)
|
||||
main_layout.addLayout(button_layout)
|
||||
|
||||
# Connections
|
||||
self.radio_txt.toggled.connect(self._update_options_visibility)
|
||||
self.radio_simple.toggled.connect(self._update_options_visibility)
|
||||
self.radio_detailed.toggled.connect(self._update_options_visibility)
|
||||
self.radio_custom.toggled.connect(self._update_options_visibility)
|
||||
self.browse_button.clicked.connect(self._browse)
|
||||
self.export_button.clicked.connect(self._accept_and_export)
|
||||
self.cancel_button.clicked.connect(self.reject)
|
||||
|
||||
self.radio_simple.setChecked(True)
|
||||
|
||||
def _update_options_visibility(self):
|
||||
is_txt = self.radio_txt.isChecked()
|
||||
self.txt_options_group.setVisible(is_txt)
|
||||
|
||||
self.detailed_options_widget.setVisible(is_txt and self.radio_detailed.isChecked())
|
||||
self.custom_format_widget.setVisible(is_txt and self.radio_custom.isChecked())
|
||||
|
||||
def _browse(self, base_filepath):
|
||||
is_separate_files_mode = self.radio_txt.isChecked() and self.check_separate_files.isChecked()
|
||||
|
||||
if is_separate_files_mode:
|
||||
dir_path = QFileDialog.getExistingDirectory(self, "Select Folder to Save Files")
|
||||
if dir_path:
|
||||
self.path_input.setText(os.path.join(dir_path, "exported_links"))
|
||||
else:
|
||||
default_filename = "exported_links"
|
||||
file_filter = "Text Files (*.txt)"
|
||||
if self.radio_json.isChecked():
|
||||
default_filename += ".json"
|
||||
file_filter = "JSON Files (*.json)"
|
||||
else:
|
||||
default_filename += ".txt"
|
||||
|
||||
filepath, _ = QFileDialog.getSaveFileName(self, "Save Links", default_filename, file_filter)
|
||||
if filepath:
|
||||
self.path_input.setText(filepath)
|
||||
|
||||
def _accept_and_export(self):
|
||||
filepath = self.path_input.text().strip()
|
||||
if not filepath:
|
||||
QMessageBox.warning(self, "Input Error", "Please select a file path or folder.")
|
||||
return
|
||||
|
||||
try:
|
||||
if self.radio_txt.isChecked():
|
||||
self._write_txt_file(filepath)
|
||||
else:
|
||||
self._write_json_file(filepath)
|
||||
|
||||
QMessageBox.information(self, "Export Successful", "Links successfully exported!")
|
||||
self.accept()
|
||||
except OSError as e:
|
||||
QMessageBox.critical(self, "Export Error", f"Could not write to file:\n{e}")
|
||||
|
||||
def _write_txt_file(self, base_filepath):
|
||||
if self.check_separate_files.isChecked():
|
||||
links_by_platform = defaultdict(list)
|
||||
for _, _, link_url, platform, _ in self.links_data:
|
||||
sanitized_platform = re.sub(r'[<>:"/\\|?*]', '_', platform.lower().replace(' ', '_'))
|
||||
links_by_platform[sanitized_platform].append(link_url)
|
||||
|
||||
base, ext = os.path.splitext(base_filepath)
|
||||
if not ext: ext = ".txt"
|
||||
|
||||
for platform_key, links in links_by_platform.items():
|
||||
platform_filepath = f"{base}_{platform_key}{ext}"
|
||||
with open(platform_filepath, 'w', encoding='utf-8') as f:
|
||||
for url in links:
|
||||
f.write(url + "\n")
|
||||
return
|
||||
|
||||
with open(base_filepath, 'w', encoding='utf-8') as f:
|
||||
if self.radio_simple.isChecked():
|
||||
for _, _, link_url, _, _ in self.links_data:
|
||||
f.write(link_url + "\n")
|
||||
|
||||
elif self.radio_detailed.isChecked():
|
||||
include_titles = self.check_include_titles.isChecked()
|
||||
include_text = self.check_include_link_text.isChecked()
|
||||
include_platform = self.check_include_platform.isChecked()
|
||||
current_title = None
|
||||
for post_title, link_text, link_url, platform, _ in self.links_data:
|
||||
if include_titles and post_title != current_title:
|
||||
if current_title is not None: f.write("\n" + "="*60 + "\n\n")
|
||||
f.write(f"# Post: {post_title}\n")
|
||||
current_title = post_title
|
||||
line_parts = [link_url]
|
||||
if include_platform: line_parts.append(f"Platform: {platform}")
|
||||
if include_text and link_text: line_parts.append(f"Description: {link_text}")
|
||||
f.write(" | ".join(line_parts) + "\n")
|
||||
|
||||
elif self.radio_custom.isChecked():
|
||||
template = self.custom_format_input.toPlainText().replace("\\n", "\n")
|
||||
for post_title, link_text, link_url, platform, decryption_key in self.links_data:
|
||||
formatted_line = template.format(
|
||||
url=link_url,
|
||||
post_title=post_title,
|
||||
link_text=link_text,
|
||||
platform=platform,
|
||||
key=decryption_key or ""
|
||||
)
|
||||
f.write(formatted_line)
|
||||
if not template.endswith('\n'):
|
||||
f.write('\n')
|
||||
|
||||
def _write_json_file(self, filepath):
|
||||
output_data = []
|
||||
for post_title, link_text, link_url, platform, decryption_key in self.links_data:
|
||||
output_data.append({
|
||||
"post_title": post_title,
|
||||
"url": link_url,
|
||||
"link_text": link_text,
|
||||
"platform": platform,
|
||||
"key": decryption_key or None
|
||||
})
|
||||
|
||||
with open(filepath, 'w', encoding='utf-8') as f:
|
||||
json.dump(output_data, f, indent=2)
|
||||
@@ -3,7 +3,7 @@ import html
|
||||
import re
|
||||
|
||||
# --- Third-Party Library Imports ---
|
||||
import requests
|
||||
import cloudscraper # MODIFIED: Import cloudscraper
|
||||
from PyQt5.QtCore import QCoreApplication, Qt
|
||||
from PyQt5.QtWidgets import (
|
||||
QApplication, QDialog, QHBoxLayout, QLabel, QLineEdit, QListWidget,
|
||||
@@ -12,7 +12,6 @@ from PyQt5.QtWidgets import (
|
||||
|
||||
# --- Local Application Imports ---
|
||||
from ...i18n.translator import get_translation
|
||||
# Corrected Import: Get the icon from the new assets utility module
|
||||
from ..assets import get_app_icon_object
|
||||
from ...utils.network_utils import prepare_cookies_for_request
|
||||
from .CookieHelpDialog import CookieHelpDialog
|
||||
@@ -41,9 +40,9 @@ class FavoriteArtistsDialog (QDialog ):
|
||||
service_lower = service_name.lower()
|
||||
coomer_primary_services = {'onlyfans', 'fansly', 'manyvids', 'candfans'}
|
||||
if service_lower in coomer_primary_services:
|
||||
return "coomer.st" # Use the new domain
|
||||
return "coomer.st"
|
||||
else:
|
||||
return "kemono.cr" # Use the new domain
|
||||
return "kemono.cr"
|
||||
|
||||
def _tr (self ,key ,default_text =""):
|
||||
"""Helper to get translation based on current app language."""
|
||||
@@ -126,9 +125,11 @@ class FavoriteArtistsDialog (QDialog ):
|
||||
self .artist_list_widget .setVisible (show )
|
||||
|
||||
def _fetch_favorite_artists (self ):
|
||||
# --- FIX: Use cloudscraper and add proper headers ---
|
||||
scraper = cloudscraper.create_scraper()
|
||||
# --- END FIX ---
|
||||
|
||||
if self.cookies_config['use_cookie']:
|
||||
# --- Kemono Check with Fallback ---
|
||||
kemono_cookies = prepare_cookies_for_request(
|
||||
True, self.cookies_config['cookie_text'], self.cookies_config['selected_cookie_file'],
|
||||
self.cookies_config['app_base_dir'], self._logger, target_domain="kemono.cr"
|
||||
@@ -140,7 +141,6 @@ class FavoriteArtistsDialog (QDialog ):
|
||||
self.cookies_config['app_base_dir'], self._logger, target_domain="kemono.su"
|
||||
)
|
||||
|
||||
# --- Coomer Check with Fallback ---
|
||||
coomer_cookies = prepare_cookies_for_request(
|
||||
True, self.cookies_config['cookie_text'], self.cookies_config['selected_cookie_file'],
|
||||
self.cookies_config['app_base_dir'], self._logger, target_domain="coomer.st"
|
||||
@@ -153,28 +153,21 @@ class FavoriteArtistsDialog (QDialog ):
|
||||
)
|
||||
|
||||
if not kemono_cookies and not coomer_cookies:
|
||||
# If cookies are enabled but none could be loaded, show help and stop.
|
||||
self.status_label.setText(self._tr("fav_artists_cookies_required_status", "Error: Cookies enabled but could not be loaded for any source."))
|
||||
self._logger("Error: Cookies enabled but no valid cookies were loaded. Showing help dialog.")
|
||||
cookie_help_dialog = CookieHelpDialog(self.parent_app, self)
|
||||
cookie_help_dialog.exec_()
|
||||
self.download_button.setEnabled(False)
|
||||
return # Stop further execution
|
||||
|
||||
kemono_fav_url ="https://kemono.su/api/v1/account/favorites?type=artist"
|
||||
coomer_fav_url ="https://coomer.su/api/v1/account/favorites?type=artist"
|
||||
return
|
||||
|
||||
self .all_fetched_artists =[]
|
||||
fetched_any_successfully =False
|
||||
errors_occurred =[]
|
||||
any_cookies_loaded_successfully_for_any_source =False
|
||||
|
||||
kemono_cr_fav_url = "https://kemono.cr/api/v1/account/favorites?type=artist"
|
||||
coomer_st_fav_url = "https://coomer.st/api/v1/account/favorites?type=artist"
|
||||
|
||||
api_sources = [
|
||||
{"name": "Kemono.cr", "url": kemono_cr_fav_url, "domain": "kemono.cr"},
|
||||
{"name": "Coomer.st", "url": coomer_st_fav_url, "domain": "coomer.st"}
|
||||
{"name": "Kemono.cr", "url": "https://kemono.cr/api/v1/account/favorites?type=artist", "domain": "kemono.cr"},
|
||||
{"name": "Coomer.st", "url": "https://coomer.st/api/v1/account/favorites?type=artist", "domain": "coomer.st"}
|
||||
]
|
||||
|
||||
for source in api_sources :
|
||||
@@ -185,41 +178,36 @@ class FavoriteArtistsDialog (QDialog ):
|
||||
cookies_dict_for_source = None
|
||||
if self.cookies_config['use_cookie']:
|
||||
primary_domain = source['domain']
|
||||
fallback_domain = None
|
||||
if primary_domain == "kemono.cr":
|
||||
fallback_domain = "kemono.su"
|
||||
elif primary_domain == "coomer.st":
|
||||
fallback_domain = "coomer.su"
|
||||
fallback_domain = "kemono.su" if "kemono" in primary_domain else "coomer.su"
|
||||
|
||||
# First, try the primary domain
|
||||
cookies_dict_for_source = prepare_cookies_for_request(
|
||||
True,
|
||||
self.cookies_config['cookie_text'],
|
||||
self.cookies_config['selected_cookie_file'],
|
||||
self.cookies_config['app_base_dir'],
|
||||
self._logger,
|
||||
target_domain=primary_domain
|
||||
True, self.cookies_config['cookie_text'], self.cookies_config['selected_cookie_file'],
|
||||
self.cookies_config['app_base_dir'], self._logger, target_domain=primary_domain
|
||||
)
|
||||
|
||||
# If no cookies found, try the fallback domain
|
||||
if not cookies_dict_for_source and fallback_domain:
|
||||
self._logger(f"Warning ({source['name']}): No cookies found for '{primary_domain}'. Trying fallback '{fallback_domain}'...")
|
||||
if not cookies_dict_for_source:
|
||||
self._logger(f"Warning ({source['name']}): No cookies for '{primary_domain}'. Trying fallback '{fallback_domain}'...")
|
||||
cookies_dict_for_source = prepare_cookies_for_request(
|
||||
True,
|
||||
self.cookies_config['cookie_text'],
|
||||
self.cookies_config['selected_cookie_file'],
|
||||
self.cookies_config['app_base_dir'],
|
||||
self._logger,
|
||||
target_domain=fallback_domain
|
||||
True, self.cookies_config['cookie_text'], self.cookies_config['selected_cookie_file'],
|
||||
self.cookies_config['app_base_dir'], self._logger, target_domain=fallback_domain
|
||||
)
|
||||
|
||||
if cookies_dict_for_source:
|
||||
any_cookies_loaded_successfully_for_any_source = True
|
||||
else:
|
||||
self._logger(f"Warning ({source['name']}): Cookies enabled but could not be loaded for this source (including fallbacks). Fetch might fail.")
|
||||
self._logger(f"Warning ({source['name']}): Cookies enabled but not loaded for this source. Fetch may fail.")
|
||||
try :
|
||||
headers ={'User-Agent':'Mozilla/5.0'}
|
||||
response =requests .get (source ['url'],headers =headers ,cookies =cookies_dict_for_source ,timeout =20 )
|
||||
# --- FIX: Add Referer and Accept headers ---
|
||||
headers = {
|
||||
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36',
|
||||
'Referer': f"https://{source['domain']}/favorites",
|
||||
'Accept': 'text/css'
|
||||
}
|
||||
# --- END FIX ---
|
||||
|
||||
# --- FIX: Use scraper instead of requests ---
|
||||
response = scraper.get(source['url'], headers=headers, cookies=cookies_dict_for_source, timeout=20)
|
||||
# --- END FIX ---
|
||||
|
||||
response .raise_for_status ()
|
||||
artists_data_from_api =response .json ()
|
||||
|
||||
@@ -254,15 +242,10 @@ class FavoriteArtistsDialog (QDialog ):
|
||||
fetched_any_successfully =True
|
||||
self ._logger (f"Fetched {processed_artists_from_source } artists from {source ['name']}.")
|
||||
|
||||
except requests .exceptions .RequestException as e :
|
||||
except Exception as e :
|
||||
error_msg =f"Error fetching favorites from {source ['name']}: {e }"
|
||||
self ._logger (error_msg )
|
||||
errors_occurred .append (error_msg )
|
||||
except Exception as e :
|
||||
error_msg =f"An unexpected error occurred with {source ['name']}: {e }"
|
||||
self ._logger (error_msg )
|
||||
errors_occurred .append (error_msg )
|
||||
|
||||
|
||||
if self .cookies_config ['use_cookie']and not any_cookies_loaded_successfully_for_any_source :
|
||||
self .status_label .setText (self ._tr ("fav_artists_cookies_required_status","Error: Cookies enabled but could not be loaded for any source."))
|
||||
@@ -288,7 +271,7 @@ class FavoriteArtistsDialog (QDialog ):
|
||||
self ._show_content_elements (True )
|
||||
self .download_button .setEnabled (True )
|
||||
elif not fetched_any_successfully and not errors_occurred :
|
||||
self .status_label .setText (self ._tr ("fav_artists_none_found_status","No favorite artists found on Kemono.su or Coomer.su."))
|
||||
self .status_label .setText (self ._tr ("fav_artists_none_found_status","No favorite artists found on Kemono or Coomer."))
|
||||
self ._show_content_elements (False )
|
||||
self .download_button .setEnabled (False )
|
||||
else :
|
||||
@@ -344,4 +327,4 @@ class FavoriteArtistsDialog (QDialog ):
|
||||
self .accept ()
|
||||
|
||||
def get_selected_artists (self ):
|
||||
return self .selected_artists_data
|
||||
return self .selected_artists_data
|
||||
|
||||
@@ -7,7 +7,7 @@ import traceback
|
||||
import json
|
||||
import re
|
||||
from collections import defaultdict
|
||||
import requests
|
||||
import cloudscraper # MODIFIED: Import cloudscraper
|
||||
from PyQt5.QtCore import QCoreApplication, Qt, pyqtSignal, QThread
|
||||
from PyQt5.QtWidgets import (
|
||||
QApplication, QDialog, QHBoxLayout, QLabel, QLineEdit, QListWidget,
|
||||
@@ -42,10 +42,9 @@ class FavoritePostsFetcherThread (QThread ):
|
||||
self .parent_logger_func (f"[FavPostsFetcherThread] {message }")
|
||||
|
||||
def run(self):
|
||||
kemono_su_fav_posts_url = "https://kemono.su/api/v1/account/favorites?type=post"
|
||||
coomer_su_fav_posts_url = "https://coomer.su/api/v1/account/favorites?type=post"
|
||||
kemono_cr_fav_posts_url = "https://kemono.cr/api/v1/account/favorites?type=post"
|
||||
coomer_st_fav_posts_url = "https://coomer.st/api/v1/account/favorites?type=post"
|
||||
# --- FIX: Use cloudscraper and add proper headers ---
|
||||
scraper = cloudscraper.create_scraper()
|
||||
# --- END FIX ---
|
||||
|
||||
all_fetched_posts_temp = []
|
||||
error_messages_for_summary = []
|
||||
@@ -56,8 +55,8 @@ class FavoritePostsFetcherThread (QThread ):
|
||||
self.progress_bar_update.emit(0, 0)
|
||||
|
||||
api_sources = [
|
||||
{"name": "Kemono.cr", "url": kemono_cr_fav_posts_url, "domain": "kemono.cr"},
|
||||
{"name": "Coomer.st", "url": coomer_st_fav_posts_url, "domain": "coomer.st"}
|
||||
{"name": "Kemono.cr", "url": "https://kemono.cr/api/v1/account/favorites?type=post", "domain": "kemono.cr"},
|
||||
{"name": "Coomer.st", "url": "https://coomer.st/api/v1/account/favorites?type=post", "domain": "coomer.st"}
|
||||
]
|
||||
|
||||
api_sources_to_try =[]
|
||||
@@ -81,32 +80,18 @@ class FavoritePostsFetcherThread (QThread ):
|
||||
cookies_dict_for_source = None
|
||||
if self.cookies_config['use_cookie']:
|
||||
primary_domain = source['domain']
|
||||
fallback_domain = None
|
||||
if primary_domain == "kemono.cr":
|
||||
fallback_domain = "kemono.su"
|
||||
elif primary_domain == "coomer.st":
|
||||
fallback_domain = "coomer.su"
|
||||
fallback_domain = "kemono.su" if "kemono" in primary_domain else "coomer.su"
|
||||
|
||||
# First, try the primary domain
|
||||
cookies_dict_for_source = prepare_cookies_for_request(
|
||||
True,
|
||||
self.cookies_config['cookie_text'],
|
||||
self.cookies_config['selected_cookie_file'],
|
||||
self.cookies_config['app_base_dir'],
|
||||
self._logger,
|
||||
target_domain=primary_domain
|
||||
True, self.cookies_config['cookie_text'], self.cookies_config['selected_cookie_file'],
|
||||
self.cookies_config['app_base_dir'], self._logger, target_domain=primary_domain
|
||||
)
|
||||
|
||||
# If no cookies found, try the fallback domain
|
||||
if not cookies_dict_for_source and fallback_domain:
|
||||
self._logger(f"Warning ({source['name']}): No cookies found for '{primary_domain}'. Trying fallback '{fallback_domain}'...")
|
||||
self._logger(f"Warning ({source['name']}): No cookies for '{primary_domain}'. Trying fallback '{fallback_domain}'...")
|
||||
cookies_dict_for_source = prepare_cookies_for_request(
|
||||
True,
|
||||
self.cookies_config['cookie_text'],
|
||||
self.cookies_config['selected_cookie_file'],
|
||||
self.cookies_config['app_base_dir'],
|
||||
self._logger,
|
||||
target_domain=fallback_domain
|
||||
True, self.cookies_config['cookie_text'], self.cookies_config['selected_cookie_file'],
|
||||
self.cookies_config['app_base_dir'], self._logger, target_domain=fallback_domain
|
||||
)
|
||||
|
||||
if cookies_dict_for_source:
|
||||
@@ -120,8 +105,18 @@ class FavoritePostsFetcherThread (QThread ):
|
||||
QCoreApplication .processEvents ()
|
||||
|
||||
try :
|
||||
headers ={'User-Agent':'Mozilla/5.0'}
|
||||
response =requests .get (source ['url'],headers =headers ,cookies =cookies_dict_for_source ,timeout =20 )
|
||||
# --- FIX: Add Referer and Accept headers ---
|
||||
headers = {
|
||||
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36',
|
||||
'Referer': f"https://{source['domain']}/favorites",
|
||||
'Accept': 'text/css'
|
||||
}
|
||||
# --- END FIX ---
|
||||
|
||||
# --- FIX: Use scraper instead of requests ---
|
||||
response = scraper.get(source['url'], headers=headers, cookies=cookies_dict_for_source, timeout=20)
|
||||
# --- END FIX ---
|
||||
|
||||
response .raise_for_status ()
|
||||
posts_data_from_api =response .json ()
|
||||
|
||||
@@ -153,33 +148,24 @@ class FavoritePostsFetcherThread (QThread ):
|
||||
fetched_any_successfully =True
|
||||
self ._logger (f"Fetched {processed_posts_from_source } posts from {source ['name']}.")
|
||||
|
||||
except requests .exceptions .RequestException as e :
|
||||
except Exception as e :
|
||||
err_detail =f"Error fetching favorite posts from {source ['name']}: {e }"
|
||||
self ._logger (err_detail )
|
||||
error_messages_for_summary .append (err_detail )
|
||||
if e .response is not None and e .response .status_code ==401 :
|
||||
if hasattr(e, 'response') and e.response is not None and e.response.status_code == 401:
|
||||
self .finished .emit ([],"KEY_AUTH_FAILED")
|
||||
self ._logger (f"Authorization failed for {source ['name']}, emitting KEY_AUTH_FAILED.")
|
||||
return
|
||||
except Exception as e :
|
||||
err_detail =f"An unexpected error occurred with {source ['name']}: {e }"
|
||||
self ._logger (err_detail )
|
||||
error_messages_for_summary .append (err_detail )
|
||||
|
||||
if self .cancellation_event .is_set ():
|
||||
self .finished .emit ([],"KEY_FETCH_CANCELLED_AFTER")
|
||||
return
|
||||
|
||||
|
||||
if self .cookies_config ['use_cookie']and not any_cookies_loaded_successfully_for_any_source :
|
||||
|
||||
if self .target_domain_preference and not any_cookies_loaded_successfully_for_any_source :
|
||||
|
||||
domain_key_part =self .error_key_map .get (self .target_domain_preference ,self .target_domain_preference .lower ().replace ('.','_'))
|
||||
self .finished .emit ([],f"KEY_COOKIES_REQUIRED_BUT_NOT_FOUND_FOR_DOMAIN_{domain_key_part }")
|
||||
return
|
||||
|
||||
|
||||
self .finished .emit ([],"KEY_COOKIES_REQUIRED_BUT_NOT_FOUND_GENERIC")
|
||||
return
|
||||
|
||||
@@ -643,4 +629,4 @@ class FavoritePostsDialog (QDialog ):
|
||||
self .accept ()
|
||||
|
||||
def get_selected_posts (self ):
|
||||
return self .selected_posts_data
|
||||
return self .selected_posts_data
|
||||
@@ -1,24 +1,112 @@
|
||||
# --- Standard Library Imports ---
|
||||
import os
|
||||
import json
|
||||
import sys
|
||||
|
||||
# --- PyQt5 Imports ---
|
||||
from PyQt5.QtCore import Qt, QStandardPaths
|
||||
from PyQt5.QtCore import Qt, QStandardPaths, QTimer
|
||||
from PyQt5.QtWidgets import (
|
||||
QApplication, QDialog, QHBoxLayout, QLabel, QPushButton, QVBoxLayout,
|
||||
QGroupBox, QComboBox, QMessageBox, QGridLayout, QCheckBox
|
||||
QGroupBox, QComboBox, QMessageBox, QGridLayout, QCheckBox, QLineEdit
|
||||
)
|
||||
|
||||
# --- Local Application Imports ---
|
||||
from ...i18n.translator import get_translation
|
||||
from ...utils.resolution import get_dark_theme
|
||||
from ..assets import get_app_icon_object
|
||||
|
||||
from ..main_window import get_app_icon_object
|
||||
from ...config.constants import (
|
||||
THEME_KEY, LANGUAGE_KEY, DOWNLOAD_LOCATION_KEY,
|
||||
RESOLUTION_KEY, UI_SCALE_KEY, SAVE_CREATOR_JSON_KEY,
|
||||
COOKIE_TEXT_KEY, USE_COOKIE_KEY
|
||||
DATE_PREFIX_FORMAT_KEY,
|
||||
COOKIE_TEXT_KEY, USE_COOKIE_KEY,
|
||||
FETCH_FIRST_KEY, DISCORD_TOKEN_KEY, POST_DOWNLOAD_ACTION_KEY
|
||||
)
|
||||
from ...services.updater import UpdateChecker, UpdateDownloader
|
||||
|
||||
class CountdownMessageBox(QDialog):
|
||||
"""
|
||||
A custom message box that includes a countdown timer for the 'Yes' button,
|
||||
which automatically accepts the dialog when the timer reaches zero.
|
||||
"""
|
||||
def __init__(self, title, text, countdown_seconds=10, parent_app=None, parent=None):
|
||||
super().__init__(parent)
|
||||
self.parent_app = parent_app
|
||||
self.countdown = countdown_seconds
|
||||
|
||||
# --- Basic Window Setup ---
|
||||
self.setWindowTitle(title)
|
||||
self.setModal(True)
|
||||
app_icon = get_app_icon_object()
|
||||
if app_icon and not app_icon.isNull():
|
||||
self.setWindowIcon(app_icon)
|
||||
|
||||
self._init_ui(text)
|
||||
self._apply_theme()
|
||||
|
||||
# --- Timer Setup ---
|
||||
self.timer = QTimer(self)
|
||||
self.timer.setInterval(1000) # Tick every second
|
||||
self.timer.timeout.connect(self._update_countdown)
|
||||
self.timer.start()
|
||||
|
||||
def _init_ui(self, text):
|
||||
"""Initializes the UI components of the dialog."""
|
||||
main_layout = QVBoxLayout(self)
|
||||
|
||||
self.message_label = QLabel(text)
|
||||
self.message_label.setWordWrap(True)
|
||||
self.message_label.setAlignment(Qt.AlignCenter)
|
||||
main_layout.addWidget(self.message_label)
|
||||
|
||||
buttons_layout = QHBoxLayout()
|
||||
buttons_layout.addStretch(1)
|
||||
|
||||
self.yes_button = QPushButton()
|
||||
self.yes_button.clicked.connect(self.accept)
|
||||
self.yes_button.setDefault(True)
|
||||
|
||||
self.no_button = QPushButton()
|
||||
self.no_button.clicked.connect(self.reject)
|
||||
|
||||
buttons_layout.addWidget(self.yes_button)
|
||||
buttons_layout.addWidget(self.no_button)
|
||||
buttons_layout.addStretch(1)
|
||||
|
||||
main_layout.addLayout(buttons_layout)
|
||||
|
||||
self._retranslate_ui()
|
||||
self._update_countdown() # Initial text setup
|
||||
|
||||
def _tr(self, key, default_text=""):
|
||||
"""Helper for translations."""
|
||||
if self.parent_app and hasattr(self.parent_app, 'current_selected_language'):
|
||||
return get_translation(self.parent_app.current_selected_language, key, default_text)
|
||||
return default_text
|
||||
|
||||
def _retranslate_ui(self):
|
||||
"""Sets translated text for UI elements."""
|
||||
self.no_button.setText(self._tr("no_button_text", "No"))
|
||||
# The 'yes' button text is handled by the countdown
|
||||
|
||||
def _update_countdown(self):
|
||||
"""Updates the countdown and button text each second."""
|
||||
if self.countdown <= 0:
|
||||
self.timer.stop()
|
||||
self.accept() # Automatically accept when countdown finishes
|
||||
return
|
||||
|
||||
yes_text = self._tr("yes_button_text", "Yes")
|
||||
self.yes_button.setText(f"{yes_text} ({self.countdown})")
|
||||
self.countdown -= 1
|
||||
|
||||
def _apply_theme(self):
|
||||
"""Applies the current theme from the parent application."""
|
||||
if self.parent_app and hasattr(self.parent_app, 'current_theme') and self.parent_app.current_theme == "dark":
|
||||
scale = getattr(self.parent_app, 'scale_factor', 1)
|
||||
self.setStyleSheet(get_dark_theme(scale))
|
||||
else:
|
||||
self.setStyleSheet("")
|
||||
|
||||
class FutureSettingsDialog(QDialog):
|
||||
"""
|
||||
@@ -29,6 +117,7 @@ class FutureSettingsDialog(QDialog):
|
||||
super().__init__(parent)
|
||||
self.parent_app = parent_app_ref
|
||||
self.setModal(True)
|
||||
self.update_downloader_thread = None # To keep a reference
|
||||
|
||||
app_icon = get_app_icon_object()
|
||||
if app_icon and not app_icon.isNull():
|
||||
@@ -36,7 +125,7 @@ class FutureSettingsDialog(QDialog):
|
||||
|
||||
screen_height = QApplication.primaryScreen().availableGeometry().height() if QApplication.primaryScreen() else 800
|
||||
scale_factor = screen_height / 800.0
|
||||
base_min_w, base_min_h = 420, 360 # Adjusted height for new layout
|
||||
base_min_w, base_min_h = 420, 520 # Increased height for new options
|
||||
scaled_min_w = int(base_min_w * scale_factor)
|
||||
scaled_min_h = int(base_min_h * scale_factor)
|
||||
self.setMinimumSize(scaled_min_w, scaled_min_h)
|
||||
@@ -49,25 +138,21 @@ class FutureSettingsDialog(QDialog):
|
||||
"""Initializes all UI components and layouts for the dialog."""
|
||||
main_layout = QVBoxLayout(self)
|
||||
|
||||
# --- Group 1: Interface Settings ---
|
||||
self.interface_group_box = QGroupBox()
|
||||
interface_layout = QGridLayout(self.interface_group_box)
|
||||
|
||||
# Theme
|
||||
self.theme_label = QLabel()
|
||||
self.theme_toggle_button = QPushButton()
|
||||
self.theme_toggle_button.clicked.connect(self._toggle_theme)
|
||||
interface_layout.addWidget(self.theme_label, 0, 0)
|
||||
interface_layout.addWidget(self.theme_toggle_button, 0, 1)
|
||||
|
||||
# UI Scale
|
||||
self.ui_scale_label = QLabel()
|
||||
self.ui_scale_combo_box = QComboBox()
|
||||
self.ui_scale_combo_box.currentIndexChanged.connect(self._display_setting_changed)
|
||||
interface_layout.addWidget(self.ui_scale_label, 1, 0)
|
||||
interface_layout.addWidget(self.ui_scale_combo_box, 1, 1)
|
||||
|
||||
# Language
|
||||
|
||||
self.language_label = QLabel()
|
||||
self.language_combo_box = QComboBox()
|
||||
self.language_combo_box.currentIndexChanged.connect(self._language_selection_changed)
|
||||
@@ -76,89 +161,173 @@ class FutureSettingsDialog(QDialog):
|
||||
|
||||
main_layout.addWidget(self.interface_group_box)
|
||||
|
||||
# --- Group 2: Download & Window Settings ---
|
||||
self.download_window_group_box = QGroupBox()
|
||||
download_window_layout = QGridLayout(self.download_window_group_box)
|
||||
|
||||
# Window Size (Resolution)
|
||||
|
||||
self.window_size_label = QLabel()
|
||||
self.resolution_combo_box = QComboBox()
|
||||
self.resolution_combo_box.currentIndexChanged.connect(self._display_setting_changed)
|
||||
download_window_layout.addWidget(self.window_size_label, 0, 0)
|
||||
download_window_layout.addWidget(self.resolution_combo_box, 0, 1)
|
||||
|
||||
# Default Path
|
||||
self.default_path_label = QLabel()
|
||||
self.save_path_button = QPushButton()
|
||||
# --- START: MODIFIED LOGIC ---
|
||||
self.save_path_button.clicked.connect(self._save_cookie_and_path)
|
||||
# --- END: MODIFIED LOGIC ---
|
||||
self.save_path_button.clicked.connect(self._save_settings)
|
||||
download_window_layout.addWidget(self.default_path_label, 1, 0)
|
||||
download_window_layout.addWidget(self.save_path_button, 1, 1)
|
||||
|
||||
# Save Creator.json Checkbox
|
||||
self.date_prefix_format_label = QLabel()
|
||||
self.date_prefix_format_input = QLineEdit()
|
||||
self.date_prefix_format_input.textChanged.connect(self._date_prefix_format_changed)
|
||||
download_window_layout.addWidget(self.date_prefix_format_label, 2, 0)
|
||||
download_window_layout.addWidget(self.date_prefix_format_input, 2, 1)
|
||||
|
||||
self.post_download_action_label = QLabel()
|
||||
self.post_download_action_combo = QComboBox()
|
||||
self.post_download_action_combo.currentIndexChanged.connect(self._post_download_action_changed)
|
||||
download_window_layout.addWidget(self.post_download_action_label, 3, 0)
|
||||
download_window_layout.addWidget(self.post_download_action_combo, 3, 1)
|
||||
|
||||
self.save_creator_json_checkbox = QCheckBox()
|
||||
self.save_creator_json_checkbox.stateChanged.connect(self._creator_json_setting_changed)
|
||||
download_window_layout.addWidget(self.save_creator_json_checkbox, 2, 0, 1, 2)
|
||||
self.save_creator_json_checkbox.stateChanged.connect(self._creator_json_setting_changed)
|
||||
download_window_layout.addWidget(self.save_creator_json_checkbox, 4, 0, 1, 2)
|
||||
|
||||
self.fetch_first_checkbox = QCheckBox()
|
||||
self.fetch_first_checkbox.stateChanged.connect(self._fetch_first_setting_changed)
|
||||
download_window_layout.addWidget(self.fetch_first_checkbox, 5, 0, 1, 2)
|
||||
|
||||
main_layout.addWidget(self.download_window_group_box)
|
||||
|
||||
self.update_group_box = QGroupBox()
|
||||
update_layout = QGridLayout(self.update_group_box)
|
||||
self.version_label = QLabel()
|
||||
self.update_status_label = QLabel()
|
||||
self.check_update_button = QPushButton()
|
||||
self.check_update_button.clicked.connect(self._check_for_updates)
|
||||
update_layout.addWidget(self.version_label, 0, 0)
|
||||
update_layout.addWidget(self.update_status_label, 0, 1)
|
||||
update_layout.addWidget(self.check_update_button, 1, 0, 1, 2)
|
||||
main_layout.addWidget(self.update_group_box)
|
||||
|
||||
main_layout.addStretch(1)
|
||||
|
||||
# --- OK Button ---
|
||||
self.ok_button = QPushButton()
|
||||
self.ok_button.clicked.connect(self.accept)
|
||||
main_layout.addWidget(self.ok_button, 0, Qt.AlignRight | Qt.AlignBottom)
|
||||
|
||||
def _retranslate_ui(self):
|
||||
self.setWindowTitle(self._tr("settings_dialog_title", "Settings"))
|
||||
self.interface_group_box.setTitle(self._tr("interface_group_title", "Interface Settings"))
|
||||
self.download_window_group_box.setTitle(self._tr("download_window_group_title", "Download & Window Settings"))
|
||||
self.theme_label.setText(self._tr("theme_label", "Theme:"))
|
||||
self.ui_scale_label.setText(self._tr("ui_scale_label", "UI Scale:"))
|
||||
self.language_label.setText(self._tr("language_label", "Language:"))
|
||||
|
||||
self.window_size_label.setText(self._tr("window_size_label", "Window Size:"))
|
||||
self.default_path_label.setText(self._tr("default_path_label", "Default Path:"))
|
||||
|
||||
self.date_prefix_format_label.setText(self._tr("date_prefix_format_label", "Post Subfolder Format:"))
|
||||
# Update placeholder to include {post}
|
||||
self.date_prefix_format_input.setPlaceholderText(self._tr("date_prefix_format_placeholder", "e.g., YYYY-MM-DD {post} {postid}"))
|
||||
# Add the tooltip to explain usage
|
||||
self.date_prefix_format_input.setToolTip(self._tr(
|
||||
"date_prefix_format_tooltip",
|
||||
"Create a custom folder name using placeholders:\n"
|
||||
"• YYYY, MM, DD: for the date\n"
|
||||
"• {post}: for the post title\n"
|
||||
"• {postid}: for the post's unique ID\n\n"
|
||||
"Example: {post} [{postid}] [YYYY-MM-DD]"
|
||||
))
|
||||
|
||||
|
||||
self.post_download_action_label.setText(self._tr("post_download_action_label", "Action After Download:"))
|
||||
|
||||
self.post_download_action_label.setText(self._tr("post_download_action_label", "Action After Download:"))
|
||||
self.save_creator_json_checkbox.setText(self._tr("save_creator_json_label", "Save Creator.json file"))
|
||||
self.fetch_first_checkbox.setText(self._tr("fetch_first_label", "Fetch First (Download after all pages are found)"))
|
||||
self.fetch_first_checkbox.setToolTip(self._tr("fetch_first_tooltip", "If checked, the downloader will find all posts from a creator first before starting any downloads.\nThis can be slower to start but provides a more accurate progress bar."))
|
||||
self._update_theme_toggle_button_text()
|
||||
self.save_path_button.setText(self._tr("settings_save_all_button", "Save Path + Cookie + Token"))
|
||||
self.save_path_button.setToolTip(self._tr("settings_save_all_tooltip", "Save the current 'Download Location', Cookie, and Discord Token settings for future sessions."))
|
||||
self.ok_button.setText(self._tr("ok_button", "OK"))
|
||||
|
||||
self.update_group_box.setTitle(self._tr("update_group_title", "Application Updates"))
|
||||
current_version = self.parent_app.windowTitle().split(' v')[-1]
|
||||
self.version_label.setText(self._tr("current_version_label", f"Current Version: v{current_version}"))
|
||||
self.update_status_label.setText(self._tr("update_status_ready", "Ready to check."))
|
||||
self.check_update_button.setText(self._tr("check_for_updates_button", "Check for Updates"))
|
||||
|
||||
self._populate_display_combo_boxes()
|
||||
self._populate_language_combo_box()
|
||||
self._populate_post_download_action_combo()
|
||||
self._load_date_prefix_format()
|
||||
self._load_checkbox_states()
|
||||
|
||||
def _check_for_updates(self):
|
||||
self.check_update_button.setEnabled(False)
|
||||
self.update_status_label.setText(self._tr("update_status_checking", "Checking..."))
|
||||
current_version = self.parent_app.windowTitle().split(' v')[-1]
|
||||
|
||||
self.update_checker_thread = UpdateChecker(current_version)
|
||||
self.update_checker_thread.update_available.connect(self._on_update_available)
|
||||
self.update_checker_thread.up_to_date.connect(self._on_up_to_date)
|
||||
self.update_checker_thread.update_error.connect(self._on_update_error)
|
||||
self.update_checker_thread.start()
|
||||
|
||||
def _on_update_available(self, new_version, download_url):
|
||||
self.update_status_label.setText(self._tr("update_status_found", f"Update found: v{new_version}"))
|
||||
self.check_update_button.setEnabled(True)
|
||||
|
||||
reply = QMessageBox.question(self, self._tr("update_available_title", "Update Available"),
|
||||
self._tr("update_available_message", f"A new version (v{new_version}) is available.\nWould you like to download and install it now?"),
|
||||
QMessageBox.Yes | QMessageBox.No, QMessageBox.Yes)
|
||||
if reply == QMessageBox.Yes:
|
||||
self.ok_button.setEnabled(False)
|
||||
self.check_update_button.setEnabled(False)
|
||||
self.update_status_label.setText(self._tr("update_status_downloading", "Downloading update..."))
|
||||
self.update_downloader_thread = UpdateDownloader(download_url, self.parent_app)
|
||||
self.update_downloader_thread.download_finished.connect(self._on_download_finished)
|
||||
self.update_downloader_thread.download_error.connect(self._on_update_error)
|
||||
self.update_downloader_thread.start()
|
||||
|
||||
def _on_download_finished(self):
|
||||
QApplication.instance().quit()
|
||||
|
||||
def _on_up_to_date(self, message):
|
||||
self.update_status_label.setText(self._tr("update_status_latest", message))
|
||||
self.check_update_button.setEnabled(True)
|
||||
|
||||
def _on_update_error(self, message):
|
||||
self.update_status_label.setText(self._tr("update_status_error", f"Error: {message}"))
|
||||
self.check_update_button.setEnabled(True)
|
||||
self.ok_button.setEnabled(True)
|
||||
|
||||
def _load_checkbox_states(self):
|
||||
"""Loads the initial state for all checkboxes from settings."""
|
||||
self.save_creator_json_checkbox.blockSignals(True)
|
||||
# Default to True so the feature is on by default for users
|
||||
should_save = self.parent_app.settings.value(SAVE_CREATOR_JSON_KEY, True, type=bool)
|
||||
self.save_creator_json_checkbox.setChecked(should_save)
|
||||
self.save_creator_json_checkbox.blockSignals(False)
|
||||
|
||||
self.fetch_first_checkbox.blockSignals(True)
|
||||
should_fetch_first = self.parent_app.settings.value(FETCH_FIRST_KEY, False, type=bool)
|
||||
self.fetch_first_checkbox.setChecked(should_fetch_first)
|
||||
self.fetch_first_checkbox.blockSignals(False)
|
||||
|
||||
def _creator_json_setting_changed(self, state):
|
||||
"""Saves the state of the 'Save Creator.json' checkbox."""
|
||||
is_checked = state == Qt.Checked
|
||||
self.parent_app.settings.setValue(SAVE_CREATOR_JSON_KEY, is_checked)
|
||||
self.parent_app.settings.sync()
|
||||
|
||||
def _fetch_first_setting_changed(self, state):
|
||||
is_checked = state == Qt.Checked
|
||||
self.parent_app.settings.setValue(FETCH_FIRST_KEY, is_checked)
|
||||
self.parent_app.settings.sync()
|
||||
|
||||
def _tr(self, key, default_text=""):
|
||||
if callable(get_translation) and self.parent_app:
|
||||
return get_translation(self.parent_app.current_selected_language, key, default_text)
|
||||
return default_text
|
||||
|
||||
def _retranslate_ui(self):
|
||||
self.setWindowTitle(self._tr("settings_dialog_title", "Settings"))
|
||||
|
||||
# Group Box Titles
|
||||
self.interface_group_box.setTitle(self._tr("interface_group_title", "Interface Settings"))
|
||||
self.download_window_group_box.setTitle(self._tr("download_window_group_title", "Download & Window Settings"))
|
||||
|
||||
# Interface Group Labels
|
||||
self.theme_label.setText(self._tr("theme_label", "Theme:"))
|
||||
self.ui_scale_label.setText(self._tr("ui_scale_label", "UI Scale:"))
|
||||
self.language_label.setText(self._tr("language_label", "Language:"))
|
||||
|
||||
# Download & Window Group Labels
|
||||
self.window_size_label.setText(self._tr("window_size_label", "Window Size:"))
|
||||
self.default_path_label.setText(self._tr("default_path_label", "Default Path:"))
|
||||
self.save_creator_json_checkbox.setText(self._tr("save_creator_json_label", "Save Creator.json file"))
|
||||
|
||||
# --- START: MODIFIED LOGIC ---
|
||||
# Buttons and Controls
|
||||
self._update_theme_toggle_button_text()
|
||||
self.save_path_button.setText(self._tr("settings_save_cookie_path_button", "Save Cookie + Download Path"))
|
||||
self.save_path_button.setToolTip(self._tr("settings_save_cookie_path_tooltip", "Save the current 'Download Location' and Cookie settings for future sessions."))
|
||||
self.ok_button.setText(self._tr("ok_button", "OK"))
|
||||
# --- END: MODIFIED LOGIC ---
|
||||
|
||||
# Populate dropdowns
|
||||
self._populate_display_combo_boxes()
|
||||
self._populate_language_combo_box()
|
||||
self._load_checkbox_states()
|
||||
|
||||
def _apply_theme(self):
|
||||
if self.parent_app and self.parent_app.current_theme == "dark":
|
||||
scale = getattr(self.parent_app, 'scale_factor', 1)
|
||||
@@ -184,14 +353,7 @@ class FutureSettingsDialog(QDialog):
|
||||
def _populate_display_combo_boxes(self):
|
||||
self.resolution_combo_box.blockSignals(True)
|
||||
self.resolution_combo_box.clear()
|
||||
resolutions = [
|
||||
("Auto", self._tr("auto_resolution", "Auto (System Default)")),
|
||||
("1280x720", "1280 x 720"),
|
||||
("1600x900", "1600 x 900"),
|
||||
("1920x1080", "1920 x 1080 (Full HD)"),
|
||||
("2560x1440", "2560 x 1440 (2K)"),
|
||||
("3840x2160", "3840 x 2160 (4K)")
|
||||
]
|
||||
resolutions = [("Auto", "Auto"), ("1280x720", "1280x720"), ("1600x900", "1600x900"), ("1920x1080", "1920x1080")]
|
||||
current_res = self.parent_app.settings.value(RESOLUTION_KEY, "Auto")
|
||||
for res_key, res_name in resolutions:
|
||||
self.resolution_combo_box.addItem(res_name, res_key)
|
||||
@@ -202,43 +364,24 @@ class FutureSettingsDialog(QDialog):
|
||||
self.ui_scale_combo_box.blockSignals(True)
|
||||
self.ui_scale_combo_box.clear()
|
||||
scales = [
|
||||
(0.5, "50%"),
|
||||
(0.7, "70%"),
|
||||
(0.9, "90%"),
|
||||
(1.0, "100% (Default)"),
|
||||
(1.25, "125%"),
|
||||
(1.50, "150%"),
|
||||
(1.75, "175%"),
|
||||
(2.0, "200%")
|
||||
(0.5, "50%"), (0.7, "70%"), (0.9, "90%"), (1.0, "100% (Default)"),
|
||||
(1.25, "125%"), (1.50, "150%"), (1.75, "175%"), (2.0, "200%")
|
||||
]
|
||||
|
||||
current_scale = float(self.parent_app.settings.value(UI_SCALE_KEY, 1.0))
|
||||
current_scale = self.parent_app.settings.value(UI_SCALE_KEY, 1.0)
|
||||
for scale_val, scale_name in scales:
|
||||
self.ui_scale_combo_box.addItem(scale_name, scale_val)
|
||||
if abs(current_scale - scale_val) < 0.01:
|
||||
if abs(float(current_scale) - scale_val) < 0.01:
|
||||
self.ui_scale_combo_box.setCurrentIndex(self.ui_scale_combo_box.count() - 1)
|
||||
self.ui_scale_combo_box.blockSignals(False)
|
||||
|
||||
def _display_setting_changed(self):
|
||||
selected_res = self.resolution_combo_box.currentData()
|
||||
selected_scale = self.ui_scale_combo_box.currentData()
|
||||
|
||||
self.parent_app.settings.setValue(RESOLUTION_KEY, selected_res)
|
||||
self.parent_app.settings.setValue(UI_SCALE_KEY, selected_scale)
|
||||
self.parent_app.settings.sync()
|
||||
|
||||
msg_box = QMessageBox(self)
|
||||
msg_box.setIcon(QMessageBox.Information)
|
||||
msg_box.setWindowTitle(self._tr("display_change_title", "Display Settings Changed"))
|
||||
msg_box.setText(self._tr("language_change_message", "A restart is required for these changes to take effect."))
|
||||
msg_box.setInformativeText(self._tr("language_change_informative", "Would you like to restart now?"))
|
||||
restart_button = msg_box.addButton(self._tr("restart_now_button", "Restart Now"), QMessageBox.ApplyRole)
|
||||
ok_button = msg_box.addButton(self._tr("ok_button", "OK"), QMessageBox.AcceptRole)
|
||||
msg_box.setDefaultButton(ok_button)
|
||||
msg_box.exec_()
|
||||
|
||||
if msg_box.clickedButton() == restart_button:
|
||||
self.parent_app._request_restart_application()
|
||||
QMessageBox.information(self, self._tr("display_change_title", "Display Settings Changed"),
|
||||
self._tr("language_change_message", "A restart is required..."))
|
||||
|
||||
def _populate_language_combo_box(self):
|
||||
self.language_combo_box.blockSignals(True)
|
||||
@@ -262,61 +405,89 @@ class FutureSettingsDialog(QDialog):
|
||||
self.parent_app.settings.setValue(LANGUAGE_KEY, selected_lang_code)
|
||||
self.parent_app.settings.sync()
|
||||
self.parent_app.current_selected_language = selected_lang_code
|
||||
|
||||
self._retranslate_ui()
|
||||
if hasattr(self.parent_app, '_retranslate_main_ui'):
|
||||
self.parent_app._retranslate_main_ui()
|
||||
|
||||
msg_box = QMessageBox(self)
|
||||
msg_box.setIcon(QMessageBox.Information)
|
||||
msg_box.setWindowTitle(self._tr("language_change_title", "Language Changed"))
|
||||
msg_box.setText(self._tr("language_change_message", "A restart is required..."))
|
||||
msg_box.setInformativeText(self._tr("language_change_informative", "Would you like to restart now?"))
|
||||
restart_button = msg_box.addButton(self._tr("restart_now_button", "Restart Now"), QMessageBox.ApplyRole)
|
||||
ok_button = msg_box.addButton(self._tr("ok_button", "OK"), QMessageBox.AcceptRole)
|
||||
msg_box.setDefaultButton(ok_button)
|
||||
msg_box.exec_()
|
||||
self.parent_app._retranslate_main_ui()
|
||||
QMessageBox.information(self, self._tr("language_change_title", "Language Changed"),
|
||||
self._tr("language_change_message", "A restart is required..."))
|
||||
|
||||
if msg_box.clickedButton() == restart_button:
|
||||
self.parent_app._request_restart_application()
|
||||
def _populate_post_download_action_combo(self):
|
||||
"""Populates the action dropdown and sets the current selection from settings."""
|
||||
self.post_download_action_combo.blockSignals(True)
|
||||
self.post_download_action_combo.clear()
|
||||
|
||||
actions = [
|
||||
(self._tr("action_off", "Off"), "off"),
|
||||
(self._tr("action_notify", "Notify with Sound"), "notify"),
|
||||
(self._tr("action_sleep", "Sleep"), "sleep"),
|
||||
(self._tr("action_shutdown", "Shutdown"), "shutdown")
|
||||
]
|
||||
|
||||
current_action = self.parent_app.settings.value(POST_DOWNLOAD_ACTION_KEY, "off")
|
||||
|
||||
for text, key in actions:
|
||||
self.post_download_action_combo.addItem(text, key)
|
||||
if current_action == key:
|
||||
self.post_download_action_combo.setCurrentIndex(self.post_download_action_combo.count() - 1)
|
||||
|
||||
self.post_download_action_combo.blockSignals(False)
|
||||
|
||||
def _save_cookie_and_path(self):
|
||||
"""Saves the current download path and/or cookie settings from the main window."""
|
||||
def _post_download_action_changed(self):
|
||||
"""Saves the selected post-download action to settings."""
|
||||
selected_action = self.post_download_action_combo.currentData()
|
||||
self.parent_app.settings.setValue(POST_DOWNLOAD_ACTION_KEY, selected_action)
|
||||
self.parent_app.settings.sync()
|
||||
|
||||
def _load_date_prefix_format(self):
|
||||
"""Loads the saved date prefix format and sets it in the input field."""
|
||||
self.date_prefix_format_input.blockSignals(True)
|
||||
current_format = self.parent_app.settings.value(DATE_PREFIX_FORMAT_KEY, "YYYY-MM-DD {post}", type=str)
|
||||
self.date_prefix_format_input.setText(current_format)
|
||||
self.date_prefix_format_input.blockSignals(False)
|
||||
|
||||
def _date_prefix_format_changed(self, text):
|
||||
"""Saves the date prefix format whenever it's changed."""
|
||||
self.parent_app.settings.setValue(DATE_PREFIX_FORMAT_KEY, text)
|
||||
self.parent_app.settings.sync()
|
||||
# Also update the live value in the parent app
|
||||
if hasattr(self.parent_app, 'date_prefix_format'):
|
||||
self.parent_app.date_prefix_format = text
|
||||
|
||||
def _save_settings(self):
|
||||
path_saved = False
|
||||
cookie_saved = False
|
||||
|
||||
# --- Save Download Path Logic ---
|
||||
token_saved = False
|
||||
|
||||
if hasattr(self.parent_app, 'dir_input') and self.parent_app.dir_input:
|
||||
current_path = self.parent_app.dir_input.text().strip()
|
||||
if current_path and os.path.isdir(current_path):
|
||||
self.parent_app.settings.setValue(DOWNLOAD_LOCATION_KEY, current_path)
|
||||
path_saved = True
|
||||
|
||||
# --- Save Cookie Logic ---
|
||||
if hasattr(self.parent_app, 'use_cookie_checkbox'):
|
||||
use_cookie = self.parent_app.use_cookie_checkbox.isChecked()
|
||||
cookie_content = self.parent_app.cookie_text_input.text().strip()
|
||||
|
||||
if use_cookie and cookie_content:
|
||||
self.parent_app.settings.setValue(USE_COOKIE_KEY, True)
|
||||
self.parent_app.settings.setValue(COOKIE_TEXT_KEY, cookie_content)
|
||||
cookie_saved = True
|
||||
else: # Also save the 'off' state
|
||||
else:
|
||||
self.parent_app.settings.setValue(USE_COOKIE_KEY, False)
|
||||
self.parent_app.settings.setValue(COOKIE_TEXT_KEY, "")
|
||||
|
||||
|
||||
if (hasattr(self.parent_app, 'remove_from_filename_input') and
|
||||
hasattr(self.parent_app, 'remove_from_filename_label_widget')):
|
||||
|
||||
label_text = self.parent_app.remove_from_filename_label_widget.text()
|
||||
if "Token" in label_text:
|
||||
discord_token = self.parent_app.remove_from_filename_input.text().strip()
|
||||
if discord_token:
|
||||
self.parent_app.settings.setValue(DISCORD_TOKEN_KEY, discord_token)
|
||||
token_saved = True
|
||||
|
||||
self.parent_app.settings.sync()
|
||||
|
||||
# --- User Feedback ---
|
||||
if path_saved and cookie_saved:
|
||||
message = self._tr("settings_save_both_success", "Download location and cookie settings saved.")
|
||||
elif path_saved:
|
||||
message = self._tr("settings_save_path_only_success", "Download location saved. No cookie settings were active to save.")
|
||||
elif cookie_saved:
|
||||
message = self._tr("settings_save_cookie_only_success", "Cookie settings saved. Download location was not set.")
|
||||
if path_saved or cookie_saved or token_saved:
|
||||
QMessageBox.information(self, "Settings Saved", "Settings have been saved successfully.")
|
||||
else:
|
||||
QMessageBox.warning(self, self._tr("settings_save_nothing_title", "Nothing to Save"),
|
||||
self._tr("settings_save_nothing_message", "The download location is not a valid directory and no cookie was active."))
|
||||
return
|
||||
|
||||
QMessageBox.information(self, self._tr("settings_save_success_title", "Settings Saved"), message)
|
||||
QMessageBox.warning(self, "Nothing to Save", "No valid settings were found to save.")
|
||||
@@ -172,7 +172,7 @@ class HelpGuideDialog(QDialog):
|
||||
icon_size = QSize(icon_dim, icon_dim)
|
||||
|
||||
for button, tooltip_key, url in [
|
||||
(self.github_button, "help_guide_github_tooltip", "https://github.com/Yuvi9587"),
|
||||
(self.github_button, "help_guide_github_tooltip", "https://github.com/Yuvi63771/Kemono-Downloader"),
|
||||
(self.instagram_button, "help_guide_instagram_tooltip", "https://www.instagram.com/uvi.arts/"),
|
||||
(self.discord_button, "help_guide_discord_tooltip", "https://discord.gg/BqP64XTdJN")
|
||||
]:
|
||||
|
||||
@@ -66,12 +66,8 @@ def create_single_pdf_from_content(posts_data, output_filename, font_path, logge
|
||||
|
||||
for i, post in enumerate(posts_data):
|
||||
if i > 0:
|
||||
if 'content' in post:
|
||||
pdf.add_page()
|
||||
elif 'comments' in post:
|
||||
pdf.ln(10)
|
||||
pdf.cell(0, 0, '', border='T')
|
||||
pdf.ln(10)
|
||||
# This ensures every post after the first gets its own page.
|
||||
pdf.add_page()
|
||||
|
||||
pdf.set_font(default_font_family, 'B', 16)
|
||||
pdf.multi_cell(w=0, h=10, txt=post.get('title', 'Untitled Post'), align='L')
|
||||
|
||||
@@ -1,71 +1,144 @@
|
||||
# src/ui/dialogs/SupportDialog.py
|
||||
# src/app/dialogs/SupportDialog.py
|
||||
|
||||
# --- Standard Library Imports ---
|
||||
import sys
|
||||
import os
|
||||
|
||||
# --- PyQt5 Imports ---
|
||||
from PyQt5.QtWidgets import (
|
||||
QDialog, QVBoxLayout, QLabel, QFrame, QDialogButtonBox, QGridLayout
|
||||
QDialog, QVBoxLayout, QHBoxLayout, QLabel, QFrame,
|
||||
QPushButton, QSizePolicy
|
||||
)
|
||||
from PyQt5.QtCore import Qt, QSize
|
||||
from PyQt5.QtGui import QFont, QPixmap
|
||||
from PyQt5.QtCore import Qt, QSize, QUrl
|
||||
from PyQt5.QtGui import QPixmap, QDesktopServices
|
||||
|
||||
# --- Local Application Imports ---
|
||||
from ...utils.resolution import get_dark_theme
|
||||
|
||||
# --- Helper function for robust asset loading ---
|
||||
def get_asset_path(filename):
|
||||
"""
|
||||
Gets the absolute path to a file in the assets folder,
|
||||
handling both development and frozen (PyInstaller) environments.
|
||||
"""
|
||||
if getattr(sys, 'frozen', False) and hasattr(sys, '_MEIPASS'):
|
||||
# Running in a PyInstaller bundle
|
||||
base_path = sys._MEIPASS
|
||||
else:
|
||||
# Running in a normal Python environment from src/ui/dialogs/
|
||||
base_path = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
return os.path.join(base_path, 'assets', filename)
|
||||
|
||||
|
||||
class SupportDialog(QDialog):
|
||||
"""
|
||||
A dialog to show support and donation options.
|
||||
A polished dialog showcasing support and community options in a
|
||||
clean, modern card-based layout.
|
||||
"""
|
||||
def __init__(self, parent=None):
|
||||
super().__init__(parent)
|
||||
self.parent_app = parent
|
||||
self.setWindowTitle("❤️ Support the Developer")
|
||||
self.setMinimumWidth(450)
|
||||
|
||||
self.setWindowTitle("❤️ Support & Community")
|
||||
self.setMinimumWidth(560)
|
||||
|
||||
self._init_ui()
|
||||
self._apply_theme()
|
||||
|
||||
def _init_ui(self):
|
||||
"""Initializes all UI components and layouts for the dialog."""
|
||||
# Main layout
|
||||
main_layout = QVBoxLayout(self)
|
||||
main_layout.setSpacing(15)
|
||||
def _create_card_button(
|
||||
self, icon_path, title, subtitle, url,
|
||||
hover_color="#2E2E2E", min_height=110, icon_size=44
|
||||
):
|
||||
"""Reusable clickable card widget with icon, title, and subtitle."""
|
||||
button = QPushButton()
|
||||
button.setCursor(Qt.PointingHandCursor)
|
||||
button.setSizePolicy(QSizePolicy.Expanding, QSizePolicy.Expanding)
|
||||
button.setMinimumHeight(min_height)
|
||||
|
||||
# Title Label
|
||||
title_label = QLabel("Thank You for Your Support!")
|
||||
font = title_label.font()
|
||||
font.setPointSize(14)
|
||||
# Consistent style
|
||||
button.setStyleSheet(f"""
|
||||
QPushButton {{
|
||||
background-color: #3A3A3A;
|
||||
border: 1px solid #555;
|
||||
border-radius: 10px;
|
||||
text-align: center;
|
||||
padding: 12px;
|
||||
}}
|
||||
QPushButton:hover {{
|
||||
background-color: {hover_color};
|
||||
border: 1px solid #777;
|
||||
}}
|
||||
""")
|
||||
|
||||
layout = QVBoxLayout(button)
|
||||
layout.setSpacing(6)
|
||||
|
||||
# Icon
|
||||
icon_label = QLabel()
|
||||
pixmap = QPixmap(icon_path)
|
||||
if not pixmap.isNull():
|
||||
scale = getattr(self.parent_app, 'scale_factor', 1.0)
|
||||
scaled_size = int(icon_size * scale)
|
||||
icon_label.setPixmap(
|
||||
pixmap.scaled(QSize(scaled_size, scaled_size), Qt.KeepAspectRatio, Qt.SmoothTransformation)
|
||||
)
|
||||
icon_label.setAlignment(Qt.AlignCenter)
|
||||
layout.addWidget(icon_label)
|
||||
|
||||
# Title
|
||||
title_label = QLabel(title)
|
||||
font = self.font()
|
||||
font.setPointSize(11)
|
||||
font.setBold(True)
|
||||
title_label.setFont(font)
|
||||
title_label.setAlignment(Qt.AlignCenter)
|
||||
main_layout.addWidget(title_label)
|
||||
title_label.setStyleSheet("background-color: transparent; border: none;")
|
||||
layout.addWidget(title_label)
|
||||
|
||||
# Informational Text
|
||||
info_label = QLabel(
|
||||
"If you find this application useful, please consider supporting its development. "
|
||||
"Your contribution helps cover costs and encourages future updates and features."
|
||||
# Subtitle
|
||||
if subtitle:
|
||||
subtitle_label = QLabel(subtitle)
|
||||
subtitle_label.setStyleSheet("color: #A8A8A8; background-color: transparent; border: none;")
|
||||
subtitle_label.setAlignment(Qt.AlignCenter)
|
||||
layout.addWidget(subtitle_label)
|
||||
|
||||
button.clicked.connect(lambda: QDesktopServices.openUrl(QUrl(url)))
|
||||
return button
|
||||
|
||||
def _create_section_title(self, text):
|
||||
"""Stylized section heading."""
|
||||
label = QLabel(text)
|
||||
font = label.font()
|
||||
font.setPointSize(13)
|
||||
font.setBold(True)
|
||||
label.setFont(font)
|
||||
label.setAlignment(Qt.AlignCenter)
|
||||
label.setStyleSheet("margin-top: 10px; margin-bottom: 5px;")
|
||||
return label
|
||||
|
||||
def _init_ui(self):
|
||||
main_layout = QVBoxLayout(self)
|
||||
main_layout.setSpacing(18)
|
||||
main_layout.setContentsMargins(20, 20, 20, 20)
|
||||
|
||||
# Header
|
||||
header_label = QLabel("Support the Project")
|
||||
font = header_label.font()
|
||||
font.setPointSize(17)
|
||||
font.setBold(True)
|
||||
header_label.setFont(font)
|
||||
header_label.setAlignment(Qt.AlignCenter)
|
||||
main_layout.addWidget(header_label)
|
||||
|
||||
subtext = QLabel(
|
||||
"If you enjoy this application, consider supporting its development. "
|
||||
"Your help keeps the project alive and growing!"
|
||||
)
|
||||
info_label.setWordWrap(True)
|
||||
info_label.setAlignment(Qt.AlignCenter)
|
||||
main_layout.addWidget(info_label)
|
||||
subtext.setWordWrap(True)
|
||||
subtext.setAlignment(Qt.AlignCenter)
|
||||
main_layout.addWidget(subtext)
|
||||
|
||||
# Financial Support
|
||||
main_layout.addWidget(self._create_section_title("Contribute Financially"))
|
||||
donation_layout = QHBoxLayout()
|
||||
donation_layout.setSpacing(15)
|
||||
|
||||
donation_layout.addWidget(self._create_card_button(
|
||||
get_asset_path("ko-fi.png"), "Ko-fi", "One-time ",
|
||||
"https://ko-fi.com/yuvi427183", "#2B2F36"
|
||||
))
|
||||
donation_layout.addWidget(self._create_card_button(
|
||||
get_asset_path("patreon.png"), "Patreon", "Soon ",
|
||||
"https://www.patreon.com/Yuvi102", "#3A2E2B"
|
||||
))
|
||||
donation_layout.addWidget(self._create_card_button(
|
||||
get_asset_path("buymeacoffee.png"), "Buy Me a Coffee", "One-time",
|
||||
"https://buymeacoffee.com/yuvi9587", "#403520"
|
||||
))
|
||||
main_layout.addLayout(donation_layout)
|
||||
|
||||
# Separator
|
||||
line = QFrame()
|
||||
@@ -73,83 +146,62 @@ class SupportDialog(QDialog):
|
||||
line.setFrameShadow(QFrame.Sunken)
|
||||
main_layout.addWidget(line)
|
||||
|
||||
# --- Donation Options Layout (using a grid for icons and text) ---
|
||||
options_layout = QGridLayout()
|
||||
options_layout.setSpacing(18)
|
||||
options_layout.setColumnStretch(0, 1) # Add stretch to center the content horizontally
|
||||
options_layout.setColumnStretch(3, 1)
|
||||
# Community Section
|
||||
main_layout.addWidget(self._create_section_title("Get Help & Connect"))
|
||||
community_layout = QHBoxLayout()
|
||||
community_layout.setSpacing(15)
|
||||
|
||||
link_font = self.font()
|
||||
link_font.setPointSize(12)
|
||||
link_font.setBold(True)
|
||||
|
||||
scale = getattr(self.parent_app, 'scale_factor', 1.0)
|
||||
icon_size = int(32 * scale)
|
||||
|
||||
# --- Ko-fi ---
|
||||
kofi_icon_label = QLabel()
|
||||
kofi_pixmap = QPixmap(get_asset_path("kofi.png"))
|
||||
if not kofi_pixmap.isNull():
|
||||
kofi_icon_label.setPixmap(kofi_pixmap.scaled(QSize(icon_size, icon_size), Qt.KeepAspectRatio, Qt.SmoothTransformation))
|
||||
|
||||
kofi_text_label = QLabel(
|
||||
'<a href="https://ko-fi.com/yuvi427183" style="color: #13C2C2; text-decoration: none;">'
|
||||
'☕ Buy me a Ko-fi'
|
||||
'</a>'
|
||||
)
|
||||
kofi_text_label.setOpenExternalLinks(True)
|
||||
kofi_text_label.setFont(link_font)
|
||||
|
||||
options_layout.addWidget(kofi_icon_label, 0, 1, Qt.AlignRight | Qt.AlignVCenter)
|
||||
options_layout.addWidget(kofi_text_label, 0, 2, Qt.AlignLeft | Qt.AlignVCenter)
|
||||
|
||||
# --- GitHub Sponsors ---
|
||||
github_icon_label = QLabel()
|
||||
github_pixmap = QPixmap(get_asset_path("github_sponsors.png"))
|
||||
if not github_pixmap.isNull():
|
||||
github_icon_label.setPixmap(github_pixmap.scaled(QSize(icon_size, icon_size), Qt.KeepAspectRatio, Qt.SmoothTransformation))
|
||||
|
||||
github_text_label = QLabel(
|
||||
'<a href="https://github.com/sponsors/Yuvi9587" style="color: #EA4AAA; text-decoration: none;">'
|
||||
'💜 Sponsor on GitHub'
|
||||
'</a>'
|
||||
)
|
||||
github_text_label.setOpenExternalLinks(True)
|
||||
github_text_label.setFont(link_font)
|
||||
|
||||
options_layout.addWidget(github_icon_label, 1, 1, Qt.AlignRight | Qt.AlignVCenter)
|
||||
options_layout.addWidget(github_text_label, 1, 2, Qt.AlignLeft | Qt.AlignVCenter)
|
||||
|
||||
# --- Buy Me a Coffee (New) ---
|
||||
bmac_icon_label = QLabel()
|
||||
bmac_pixmap = QPixmap(get_asset_path("bmac.png"))
|
||||
if not bmac_pixmap.isNull():
|
||||
bmac_icon_label.setPixmap(bmac_pixmap.scaled(QSize(icon_size, icon_size), Qt.KeepAspectRatio, Qt.SmoothTransformation))
|
||||
|
||||
bmac_text_label = QLabel(
|
||||
'<a href="https://buymeacoffee.com/yuvi9587" style="color: #FFDD00; text-decoration: none;">'
|
||||
'🍺 Buy Me a Coffee'
|
||||
'</a>'
|
||||
)
|
||||
bmac_text_label.setOpenExternalLinks(True)
|
||||
bmac_text_label.setFont(link_font)
|
||||
|
||||
options_layout.addWidget(bmac_icon_label, 2, 1, Qt.AlignRight | Qt.AlignVCenter)
|
||||
options_layout.addWidget(bmac_text_label, 2, 2, Qt.AlignLeft | Qt.AlignVCenter)
|
||||
|
||||
main_layout.addLayout(options_layout)
|
||||
community_layout.addWidget(self._create_card_button(
|
||||
get_asset_path("github.png"), "GitHub", "Report issues",
|
||||
"https://github.com/Yuvi63771/Kemono-Downloader", "#2E2E2E",
|
||||
min_height=100, icon_size=36
|
||||
))
|
||||
community_layout.addWidget(self._create_card_button(
|
||||
get_asset_path("discord.png"), "Discord", "Join the server",
|
||||
"https://discord.gg/BqP64XTdJN", "#2C2F33",
|
||||
min_height=100, icon_size=36
|
||||
))
|
||||
community_layout.addWidget(self._create_card_button(
|
||||
get_asset_path("instagram.png"), "Instagram", "Follow me",
|
||||
"https://www.instagram.com/uvi.arts/", "#3B2E40",
|
||||
min_height=100, icon_size=36
|
||||
))
|
||||
main_layout.addLayout(community_layout)
|
||||
|
||||
# Close Button
|
||||
self.button_box = QDialogButtonBox(QDialogButtonBox.Close)
|
||||
self.button_box.rejected.connect(self.reject)
|
||||
main_layout.addWidget(self.button_box)
|
||||
close_button = QPushButton("Close")
|
||||
close_button.setMinimumWidth(100)
|
||||
close_button.clicked.connect(self.accept)
|
||||
close_button.setStyleSheet("""
|
||||
QPushButton {
|
||||
padding: 6px 14px;
|
||||
border-radius: 6px;
|
||||
background-color: #444;
|
||||
color: white;
|
||||
}
|
||||
QPushButton:hover {
|
||||
background-color: #555;
|
||||
}
|
||||
""")
|
||||
|
||||
self.setLayout(main_layout)
|
||||
button_layout = QHBoxLayout()
|
||||
button_layout.addStretch()
|
||||
button_layout.addWidget(close_button)
|
||||
button_layout.addStretch()
|
||||
main_layout.addLayout(button_layout)
|
||||
|
||||
def _apply_theme(self):
|
||||
"""Applies the current theme from the parent application."""
|
||||
if self.parent_app and hasattr(self.parent_app, 'current_theme') and self.parent_app.current_theme == "dark":
|
||||
scale = getattr(self.parent_app, 'scale_factor', 1)
|
||||
self.setStyleSheet(get_dark_theme(scale))
|
||||
else:
|
||||
self.setStyleSheet("")
|
||||
self.setStyleSheet("")
|
||||
|
||||
|
||||
def get_asset_path(filename):
|
||||
"""Return the path to an asset, works in both dev and packaged environments."""
|
||||
if getattr(sys, 'frozen', False) and hasattr(sys, '_MEIPASS'):
|
||||
base_path = sys._MEIPASS
|
||||
else:
|
||||
base_path = os.path.abspath(os.path.join(os.path.dirname(__file__), '..', '..', '..'))
|
||||
return os.path.join(base_path, 'assets', filename)
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import os
|
||||
import sys
|
||||
from PyQt5.QtCore import pyqtSignal, Qt, QSettings, QCoreApplication
|
||||
from PyQt5.QtCore import pyqtSignal, Qt, QSettings
|
||||
from PyQt5.QtWidgets import (
|
||||
QApplication, QDialog, QHBoxLayout, QLabel, QPushButton, QVBoxLayout,
|
||||
QStackedWidget, QScrollArea, QFrame, QWidget, QCheckBox
|
||||
@@ -8,89 +8,88 @@ from PyQt5.QtWidgets import (
|
||||
from ...i18n.translator import get_translation
|
||||
from ..main_window import get_app_icon_object
|
||||
from ...utils.resolution import get_dark_theme
|
||||
from ...config.constants import (
|
||||
CONFIG_ORGANIZATION_NAME
|
||||
)
|
||||
from ...config.constants import CONFIG_ORGANIZATION_NAME
|
||||
|
||||
|
||||
class TourStepWidget(QWidget):
|
||||
"""
|
||||
A custom widget representing a single step or page in the feature tour.
|
||||
It neatly formats a title and its corresponding content.
|
||||
A custom widget for a single tour page, with improved styling for titles and content.
|
||||
"""
|
||||
def __init__(self, title_text, content_text, parent=None):
|
||||
super().__init__(parent)
|
||||
layout = QVBoxLayout(self)
|
||||
layout.setContentsMargins(20, 20, 20, 20)
|
||||
layout.setSpacing(10)
|
||||
layout.setContentsMargins(25, 20, 25, 20)
|
||||
layout.setSpacing(15)
|
||||
layout.setAlignment(Qt.AlignHCenter)
|
||||
|
||||
title_label = QLabel(title_text)
|
||||
title_label.setAlignment(Qt.AlignCenter)
|
||||
title_label.setStyleSheet("font-size: 18px; font-weight: bold; color: #E0E0E0; padding-bottom: 15px;")
|
||||
title_label.setWordWrap(True)
|
||||
title_label.setStyleSheet("font-size: 18pt; font-weight: bold; color: #E0E0E0; padding-bottom: 10px;")
|
||||
layout.addWidget(title_label)
|
||||
|
||||
# Frame for the content area to give it a nice border
|
||||
content_frame = QFrame()
|
||||
content_frame.setObjectName("contentFrame")
|
||||
content_layout = QVBoxLayout(content_frame)
|
||||
|
||||
scroll_area = QScrollArea()
|
||||
scroll_area.setWidgetResizable(True)
|
||||
scroll_area.setFrameShape(QFrame.NoFrame)
|
||||
scroll_area.setHorizontalScrollBarPolicy(Qt.ScrollBarAlwaysOff)
|
||||
scroll_area.setVerticalScrollBarPolicy(Qt.ScrollBarAsNeeded)
|
||||
scroll_area.setStyleSheet("background-color: transparent;")
|
||||
|
||||
|
||||
content_label = QLabel(content_text)
|
||||
content_label.setWordWrap(True)
|
||||
content_label.setAlignment(Qt.AlignLeft | Qt.AlignTop)
|
||||
content_label.setTextFormat(Qt.RichText)
|
||||
content_label.setOpenExternalLinks(True)
|
||||
content_label.setStyleSheet("font-size: 11pt; color: #C8C8C8; line-height: 1.8;")
|
||||
# Indent the content slightly for better readability
|
||||
content_label.setStyleSheet("font-size: 11pt; color: #C8C8C8; padding-left: 5px; padding-right: 5px;")
|
||||
|
||||
scroll_area.setWidget(content_label)
|
||||
layout.addWidget(scroll_area, 1)
|
||||
content_layout.addWidget(scroll_area)
|
||||
layout.addWidget(content_frame, 1)
|
||||
|
||||
|
||||
class TourDialog(QDialog):
|
||||
"""
|
||||
A dialog that shows a multi-page tour to the user on first launch.
|
||||
Includes a "Never show again" checkbox and uses QSettings to remember this preference.
|
||||
A redesigned, multi-page tour dialog with a visual progress indicator.
|
||||
"""
|
||||
tour_finished_normally = pyqtSignal()
|
||||
tour_skipped = pyqtSignal()
|
||||
CONFIG_APP_NAME_TOUR = "ApplicationTour"
|
||||
TOUR_SHOWN_KEY = "neverShowTourAgainV19"
|
||||
TOUR_SHOWN_KEY = "neverShowTourAgainV20" # Version bumped to ensure new tour shows once
|
||||
CONFIG_ORGANIZATION_NAME = CONFIG_ORGANIZATION_NAME
|
||||
|
||||
def __init__(self, parent_app, parent=None):
|
||||
"""
|
||||
Initializes the dialog.
|
||||
|
||||
Args:
|
||||
parent_app (DownloaderApp): A reference to the main application window.
|
||||
parent (QWidget, optional): The parent widget. Defaults to None.
|
||||
"""
|
||||
super().__init__(parent)
|
||||
self.settings = QSettings(CONFIG_ORGANIZATION_NAME, self.CONFIG_APP_NAME_TOUR)
|
||||
self.settings = QSettings(self.CONFIG_ORGANIZATION_NAME, self.CONFIG_APP_NAME_TOUR)
|
||||
self.current_step = 0
|
||||
self.parent_app = parent_app
|
||||
self.progress_dots = []
|
||||
|
||||
self.setWindowIcon(get_app_icon_object())
|
||||
self.setModal(True)
|
||||
self.setFixedSize(600, 620)
|
||||
self.setFixedSize(680, 650)
|
||||
|
||||
self._init_ui()
|
||||
self._apply_theme()
|
||||
self._center_on_screen()
|
||||
|
||||
def _tr(self, key, default_text=""):
|
||||
"""Helper for translation."""
|
||||
if callable(get_translation) and self.parent_app:
|
||||
return get_translation(self.parent_app.current_selected_language, key, default_text)
|
||||
return default_text
|
||||
|
||||
def _init_ui(self):
|
||||
"""Initializes all UI components and layouts."""
|
||||
main_layout = QVBoxLayout(self)
|
||||
main_layout.setContentsMargins(0, 0, 0, 0)
|
||||
main_layout.setSpacing(0)
|
||||
|
||||
self.stacked_widget = QStackedWidget()
|
||||
main_layout.addWidget(self.stacked_widget, 1)
|
||||
|
||||
# All 8 steps from your translator.py file
|
||||
steps_content = [
|
||||
("tour_dialog_step1_title", "tour_dialog_step1_content"),
|
||||
("tour_dialog_step2_title", "tour_dialog_step2_content"),
|
||||
@@ -102,52 +101,105 @@ class TourDialog(QDialog):
|
||||
("tour_dialog_step8_title", "tour_dialog_step8_content"),
|
||||
]
|
||||
|
||||
self.tour_steps_widgets = []
|
||||
for title_key, content_key in steps_content:
|
||||
title = self._tr(title_key, title_key)
|
||||
content = self._tr(content_key, "Content not found.")
|
||||
step_widget = TourStepWidget(title, content)
|
||||
self.tour_steps_widgets.append(step_widget)
|
||||
self.stacked_widget.addWidget(step_widget)
|
||||
|
||||
self.setWindowTitle(self._tr("tour_dialog_title", "Welcome to Kemono Downloader!"))
|
||||
bottom_controls_layout = QVBoxLayout()
|
||||
bottom_controls_layout.setContentsMargins(15, 10, 15, 15)
|
||||
bottom_controls_layout.setSpacing(12)
|
||||
|
||||
# --- Bottom Controls Area ---
|
||||
bottom_frame = QFrame()
|
||||
bottom_frame.setObjectName("bottomFrame")
|
||||
main_layout.addWidget(bottom_frame)
|
||||
|
||||
self.never_show_again_checkbox = QCheckBox(self._tr("tour_dialog_never_show_checkbox", "Never show this tour again"))
|
||||
bottom_controls_layout.addWidget(self.never_show_again_checkbox, 0, Qt.AlignLeft)
|
||||
bottom_controls_layout = QVBoxLayout(bottom_frame)
|
||||
bottom_controls_layout.setContentsMargins(20, 15, 20, 20)
|
||||
bottom_controls_layout.setSpacing(15)
|
||||
|
||||
buttons_layout = QHBoxLayout()
|
||||
buttons_layout.setSpacing(10)
|
||||
self.skip_button = QPushButton(self._tr("tour_dialog_skip_button", "Skip Tour"))
|
||||
# --- Progress Indicator ---
|
||||
progress_layout = QHBoxLayout()
|
||||
progress_layout.addStretch()
|
||||
for i in range(len(steps_content)):
|
||||
dot = QLabel()
|
||||
dot.setObjectName("progressDot")
|
||||
dot.setFixedSize(12, 12)
|
||||
self.progress_dots.append(dot)
|
||||
progress_layout.addWidget(dot)
|
||||
progress_layout.addStretch()
|
||||
bottom_controls_layout.addLayout(progress_layout)
|
||||
|
||||
# --- Buttons and Checkbox ---
|
||||
buttons_and_check_layout = QHBoxLayout()
|
||||
self.never_show_again_checkbox = QCheckBox(self._tr("tour_dialog_never_show_checkbox", "Never show this again"))
|
||||
buttons_and_check_layout.addWidget(self.never_show_again_checkbox, 0, Qt.AlignLeft)
|
||||
buttons_and_check_layout.addStretch()
|
||||
|
||||
self.skip_button = QPushButton(self._tr("tour_dialog_skip_button", "Skip"))
|
||||
self.skip_button.clicked.connect(self._skip_tour_action)
|
||||
self.back_button = QPushButton(self._tr("tour_dialog_back_button", "Back"))
|
||||
self.back_button.clicked.connect(self._previous_step)
|
||||
self.next_button = QPushButton(self._tr("tour_dialog_next_button", "Next"))
|
||||
self.next_button.clicked.connect(self._next_step_action)
|
||||
self.next_button.setDefault(True)
|
||||
self.next_button.setObjectName("nextButton") # For special styling
|
||||
|
||||
buttons_layout.addWidget(self.skip_button)
|
||||
buttons_layout.addStretch(1)
|
||||
buttons_layout.addWidget(self.back_button)
|
||||
buttons_layout.addWidget(self.next_button)
|
||||
buttons_and_check_layout.addWidget(self.skip_button)
|
||||
buttons_and_check_layout.addWidget(self.back_button)
|
||||
buttons_and_check_layout.addWidget(self.next_button)
|
||||
bottom_controls_layout.addLayout(buttons_and_check_layout)
|
||||
|
||||
bottom_controls_layout.addLayout(buttons_layout)
|
||||
main_layout.addLayout(bottom_controls_layout)
|
||||
|
||||
self._update_button_states()
|
||||
self._update_ui_states()
|
||||
|
||||
def _apply_theme(self):
|
||||
"""Applies the current theme from the parent application."""
|
||||
if self.parent_app and self.parent_app.current_theme == "dark":
|
||||
scale = getattr(self.parent_app, 'scale_factor', 1)
|
||||
self.setStyleSheet(get_dark_theme(scale))
|
||||
dark_theme_base = get_dark_theme(scale)
|
||||
tour_styles = """
|
||||
QDialog {
|
||||
background-color: #2D2D30;
|
||||
}
|
||||
#bottomFrame {
|
||||
background-color: #252526;
|
||||
border-top: 1px solid #3E3E42;
|
||||
}
|
||||
#contentFrame {
|
||||
border: 1px solid #3E3E42;
|
||||
border-radius: 5px;
|
||||
}
|
||||
QScrollArea {
|
||||
background-color: transparent;
|
||||
border: none;
|
||||
}
|
||||
#progressDot {
|
||||
background-color: #555;
|
||||
border-radius: 6px;
|
||||
border: 1px solid #4F4F4F;
|
||||
}
|
||||
#progressDot[active="true"] {
|
||||
background-color: #007ACC;
|
||||
border: 1px solid #005A9E;
|
||||
}
|
||||
#nextButton {
|
||||
background-color: #007ACC;
|
||||
border: 1px solid #005A9E;
|
||||
padding: 8px 18px;
|
||||
font-weight: bold;
|
||||
}
|
||||
#nextButton:hover {
|
||||
background-color: #1E90FF;
|
||||
}
|
||||
#nextButton:disabled {
|
||||
background-color: #444;
|
||||
border-color: #555;
|
||||
}
|
||||
"""
|
||||
self.setStyleSheet(dark_theme_base + tour_styles)
|
||||
else:
|
||||
self.setStyleSheet("QDialog { background-color: #f0f0f0; }")
|
||||
|
||||
def _center_on_screen(self):
|
||||
"""Centers the dialog on the screen."""
|
||||
try:
|
||||
screen_geo = QApplication.primaryScreen().availableGeometry()
|
||||
self.move(screen_geo.center() - self.rect().center())
|
||||
@@ -155,54 +207,49 @@ class TourDialog(QDialog):
|
||||
print(f"[TourDialog] Error centering dialog: {e}")
|
||||
|
||||
def _next_step_action(self):
|
||||
"""Moves to the next step or finishes the tour."""
|
||||
if self.current_step < len(self.tour_steps_widgets) - 1:
|
||||
if self.current_step < self.stacked_widget.count() - 1:
|
||||
self.current_step += 1
|
||||
self.stacked_widget.setCurrentIndex(self.current_step)
|
||||
else:
|
||||
self._finish_tour_action()
|
||||
self._update_button_states()
|
||||
self._update_ui_states()
|
||||
|
||||
def _previous_step(self):
|
||||
"""Moves to the previous step."""
|
||||
if self.current_step > 0:
|
||||
self.current_step -= 1
|
||||
self.stacked_widget.setCurrentIndex(self.current_step)
|
||||
self._update_button_states()
|
||||
self._update_ui_states()
|
||||
|
||||
def _update_button_states(self):
|
||||
"""Updates the state and text of navigation buttons."""
|
||||
is_last_step = self.current_step == len(self.tour_steps_widgets) - 1
|
||||
def _update_ui_states(self):
|
||||
is_last_step = self.current_step == self.stacked_widget.count() - 1
|
||||
self.next_button.setText(self._tr("tour_dialog_finish_button", "Finish") if is_last_step else self._tr("tour_dialog_next_button", "Next"))
|
||||
self.back_button.setEnabled(self.current_step > 0)
|
||||
self.skip_button.setVisible(not is_last_step)
|
||||
|
||||
for i, dot in enumerate(self.progress_dots):
|
||||
dot.setProperty("active", i == self.current_step)
|
||||
dot.style().polish(dot)
|
||||
|
||||
def _skip_tour_action(self):
|
||||
"""Handles the action when the tour is skipped."""
|
||||
self._save_settings_if_checked()
|
||||
self.tour_skipped.emit()
|
||||
self.reject()
|
||||
|
||||
def _finish_tour_action(self):
|
||||
"""Handles the action when the tour is finished normally."""
|
||||
self._save_settings_if_checked()
|
||||
self.tour_finished_normally.emit()
|
||||
self.accept()
|
||||
|
||||
def _save_settings_if_checked(self):
|
||||
"""Saves the 'never show again' preference to QSettings."""
|
||||
self.settings.setValue(self.TOUR_SHOWN_KEY, self.never_show_again_checkbox.isChecked())
|
||||
self.settings.sync()
|
||||
|
||||
@staticmethod
|
||||
def should_show_tour():
|
||||
"""Checks QSettings to see if the tour should be shown on startup."""
|
||||
settings = QSettings(TourDialog.CONFIG_ORGANIZATION_NAME, TourDialog.CONFIG_APP_NAME_TOUR)
|
||||
never_show = settings.value(TourDialog.TOUR_SHOWN_KEY, False, type=bool)
|
||||
return not never_show
|
||||
|
||||
CONFIG_ORGANIZATION_NAME = CONFIG_ORGANIZATION_NAME
|
||||
|
||||
def closeEvent(self, event):
|
||||
"""Ensures settings are saved if the dialog is closed via the 'X' button."""
|
||||
self._skip_tour_action()
|
||||
super().closeEvent(event)
|
||||
super().closeEvent(event)
|
||||
160
src/ui/dialogs/discord_pdf_generator.py
Normal file
160
src/ui/dialogs/discord_pdf_generator.py
Normal file
@@ -0,0 +1,160 @@
|
||||
import os
|
||||
import re
|
||||
import datetime
|
||||
import time
|
||||
try:
|
||||
from fpdf import FPDF
|
||||
FPDF_AVAILABLE = True
|
||||
|
||||
class PDF(FPDF):
|
||||
"""Custom PDF class for Discord chat logs."""
|
||||
def __init__(self, server_name, channel_name, *args, **kwargs):
|
||||
super().__init__(*args, **kwargs)
|
||||
self.server_name = server_name
|
||||
self.channel_name = channel_name
|
||||
self.default_font_family = 'DejaVu' # Can be changed to Arial if font fails
|
||||
|
||||
def header(self):
|
||||
if self.page_no() == 1:
|
||||
return # No header on the title page
|
||||
self.set_font(self.default_font_family, '', 8)
|
||||
self.cell(0, 10, f'{self.server_name} - #{self.channel_name}', 0, 0, 'L')
|
||||
self.cell(0, 10, 'Page ' + str(self.page_no()), 0, 0, 'R')
|
||||
self.ln(10)
|
||||
|
||||
def footer(self):
|
||||
pass # No footer needed, header has page number
|
||||
|
||||
except ImportError:
|
||||
FPDF_AVAILABLE = False
|
||||
FPDF = None
|
||||
PDF = None
|
||||
|
||||
def create_pdf_from_discord_messages(messages_data, server_name, channel_name, output_filename, font_path, logger=print, cancellation_event=None, pause_event=None):
|
||||
"""
|
||||
Creates a single PDF from a list of Discord message objects, formatted as a chat log.
|
||||
UPDATED to include clickable links for attachments and embeds.
|
||||
"""
|
||||
if not FPDF_AVAILABLE:
|
||||
logger("❌ PDF Creation failed: 'fpdf2' library is not installed.")
|
||||
return False
|
||||
|
||||
if not messages_data:
|
||||
logger(" No messages were found or fetched to create a PDF.")
|
||||
return False
|
||||
|
||||
# --- FIX: This helper function now correctly accepts and checks the event objects ---
|
||||
def check_events(c_event, p_event):
|
||||
"""Helper to safely check for pause and cancel events."""
|
||||
if c_event and hasattr(c_event, 'is_cancelled') and c_event.is_cancelled:
|
||||
return True # Stop
|
||||
if p_event and hasattr(p_event, 'is_paused'):
|
||||
while p_event.is_paused:
|
||||
time.sleep(0.5)
|
||||
if c_event and hasattr(c_event, 'is_cancelled') and c_event.is_cancelled:
|
||||
return True
|
||||
return False
|
||||
|
||||
logger(" Sorting messages by date (oldest first)...")
|
||||
messages_data.sort(key=lambda m: m.get('published', m.get('timestamp', '')))
|
||||
|
||||
pdf = PDF(server_name, channel_name)
|
||||
default_font_family = 'DejaVu'
|
||||
|
||||
try:
|
||||
bold_font_path = font_path.replace("DejaVuSans.ttf", "DejaVuSans-Bold.ttf")
|
||||
if not os.path.exists(font_path) or not os.path.exists(bold_font_path):
|
||||
raise RuntimeError("Font files not found")
|
||||
|
||||
pdf.add_font('DejaVu', '', font_path, uni=True)
|
||||
pdf.add_font('DejaVu', 'B', bold_font_path, uni=True)
|
||||
except Exception as font_error:
|
||||
logger(f" ⚠️ Could not load DejaVu font: {font_error}. Falling back to Arial.")
|
||||
default_font_family = 'Arial'
|
||||
pdf.default_font_family = 'Arial'
|
||||
|
||||
# --- Title Page ---
|
||||
pdf.add_page()
|
||||
pdf.set_font(default_font_family, 'B', 24)
|
||||
pdf.cell(w=0, h=20, text="Discord Chat Log", align='C', new_x="LMARGIN", new_y="NEXT")
|
||||
pdf.ln(10)
|
||||
pdf.set_font(default_font_family, '', 16)
|
||||
pdf.cell(w=0, h=10, text=f"Server: {server_name}", align='C', new_x="LMARGIN", new_y="NEXT")
|
||||
pdf.cell(w=0, h=10, text=f"Channel: #{channel_name}", align='C', new_x="LMARGIN", new_y="NEXT")
|
||||
pdf.ln(5)
|
||||
pdf.set_font(default_font_family, '', 10)
|
||||
pdf.cell(w=0, h=10, text=f"Generated on: {datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')}", align='C', new_x="LMARGIN", new_y="NEXT")
|
||||
pdf.cell(w=0, h=10, text=f"Total Messages: {len(messages_data)}", align='C', new_x="LMARGIN", new_y="NEXT")
|
||||
|
||||
pdf.add_page()
|
||||
|
||||
logger(f" Starting PDF creation with {len(messages_data)} messages...")
|
||||
|
||||
for i, message in enumerate(messages_data):
|
||||
# --- FIX: Pass the event objects to the helper function ---
|
||||
if i % 50 == 0:
|
||||
if check_events(cancellation_event, pause_event):
|
||||
logger(" PDF generation cancelled by user.")
|
||||
return False
|
||||
|
||||
author = message.get('author', {}).get('global_name') or message.get('author', {}).get('username', 'Unknown User')
|
||||
timestamp_str = message.get('published', message.get('timestamp', ''))
|
||||
content = message.get('content', '')
|
||||
attachments = message.get('attachments', [])
|
||||
embeds = message.get('embeds', [])
|
||||
|
||||
try:
|
||||
if timestamp_str.endswith('Z'):
|
||||
timestamp_str = timestamp_str[:-1] + '+00:00'
|
||||
dt_obj = datetime.datetime.fromisoformat(timestamp_str)
|
||||
formatted_timestamp = dt_obj.strftime('%Y-%m-%d %H:%M:%S')
|
||||
except (ValueError, TypeError):
|
||||
formatted_timestamp = timestamp_str
|
||||
|
||||
if i > 0:
|
||||
pdf.ln(2)
|
||||
pdf.set_draw_color(200, 200, 200)
|
||||
pdf.cell(0, 0, '', border='T')
|
||||
pdf.ln(2)
|
||||
|
||||
pdf.set_font(default_font_family, 'B', 11)
|
||||
pdf.write(5, f"{author} ")
|
||||
pdf.set_font(default_font_family, '', 9)
|
||||
pdf.set_text_color(128, 128, 128)
|
||||
pdf.write(5, f"({formatted_timestamp})")
|
||||
pdf.set_text_color(0, 0, 0)
|
||||
pdf.ln(6)
|
||||
|
||||
if content:
|
||||
pdf.set_font(default_font_family, '', 10)
|
||||
pdf.multi_cell(w=0, h=5, text=content)
|
||||
|
||||
if attachments or embeds:
|
||||
pdf.ln(1)
|
||||
pdf.set_font(default_font_family, '', 9)
|
||||
pdf.set_text_color(22, 119, 219)
|
||||
|
||||
for att in attachments:
|
||||
file_name = att.get('filename', 'untitled')
|
||||
full_url = att.get('url', '#')
|
||||
pdf.write(5, text=f"[Attachment: {file_name}]", link=full_url)
|
||||
pdf.ln()
|
||||
|
||||
for embed in embeds:
|
||||
embed_url = embed.get('url', 'no url')
|
||||
pdf.write(5, text=f"[Embed: {embed_url}]", link=embed_url)
|
||||
pdf.ln()
|
||||
|
||||
pdf.set_text_color(0, 0, 0)
|
||||
|
||||
if check_events(cancellation_event, pause_event):
|
||||
logger(" PDF generation cancelled by user before final save.")
|
||||
return False
|
||||
|
||||
try:
|
||||
pdf.output(output_filename)
|
||||
logger(f"✅ Successfully created Discord chat log PDF: '{os.path.basename(output_filename)}'")
|
||||
return True
|
||||
except Exception as e:
|
||||
logger(f"❌ A critical error occurred while saving the final PDF: {e}")
|
||||
return False
|
||||
File diff suppressed because it is too large
Load Diff
49
src/utils/command.py
Normal file
49
src/utils/command.py
Normal file
@@ -0,0 +1,49 @@
|
||||
import re
|
||||
|
||||
# Command constants
|
||||
CMD_ARCHIVE_ONLY = 'ao'
|
||||
CMD_DOMAIN_OVERRIDE_PREFIX = '.'
|
||||
CMD_SFP_PREFIX = 'sfp-'
|
||||
CMD_UNKNOWN = 'unknown' # New command constant
|
||||
|
||||
def parse_commands_from_text(raw_text: str):
|
||||
"""
|
||||
Parses special commands from a text string and returns the cleaned text
|
||||
and a dictionary of found commands.
|
||||
|
||||
Commands are in the format [command].
|
||||
Example: "Tifa, (Cloud, Zack) [.st] [sfp-10] [unknown]"
|
||||
|
||||
Returns:
|
||||
tuple[str, dict]: A tuple containing:
|
||||
- The text string with commands removed.
|
||||
- A dictionary of commands and their values.
|
||||
"""
|
||||
command_pattern = re.compile(r'\[(.*?)\]')
|
||||
commands = {}
|
||||
|
||||
def command_replacer(match):
|
||||
command_str = match.group(1).strip().lower()
|
||||
|
||||
if command_str.startswith(CMD_DOMAIN_OVERRIDE_PREFIX):
|
||||
tld = command_str[len(CMD_DOMAIN_OVERRIDE_PREFIX):]
|
||||
if 'domain_override' not in commands:
|
||||
commands['domain_override'] = tld
|
||||
elif command_str == CMD_ARCHIVE_ONLY:
|
||||
commands['archive_only'] = True
|
||||
elif command_str.startswith(CMD_SFP_PREFIX):
|
||||
try:
|
||||
threshold_str = command_str[len(CMD_SFP_PREFIX):]
|
||||
threshold = int(threshold_str)
|
||||
if 'sfp_threshold' not in commands:
|
||||
commands['sfp_threshold'] = threshold
|
||||
except (ValueError, IndexError):
|
||||
pass
|
||||
elif command_str == CMD_UNKNOWN: # Logic to handle the new command
|
||||
commands['handle_unknown'] = True
|
||||
|
||||
return ''
|
||||
|
||||
text_without_commands = command_pattern.sub(command_replacer, raw_text).strip()
|
||||
|
||||
return text_without_commands, commands
|
||||
@@ -20,7 +20,7 @@ VIDEO_EXTENSIONS = {
|
||||
'.mpg', '.m4v', '.3gp', '.ogv', '.ts', '.vob'
|
||||
}
|
||||
ARCHIVE_EXTENSIONS = {
|
||||
'.zip', '.rar', '.7z', '.tar', '.gz', '.bz2'
|
||||
'.zip', '.rar', '.7z', '.tar', '.gz', '.bz2', '.bin'
|
||||
}
|
||||
AUDIO_EXTENSIONS = {
|
||||
'.mp3', '.wav', '.aac', '.flac', '.ogg', '.wma', '.m4a', '.opus',
|
||||
@@ -140,3 +140,5 @@ def is_audio(filename):
|
||||
if not filename: return False
|
||||
_, ext = os.path.splitext(filename)
|
||||
return ext.lower() in AUDIO_EXTENSIONS
|
||||
|
||||
|
||||
|
||||
@@ -1,14 +1,7 @@
|
||||
# --- Standard Library Imports ---
|
||||
import os
|
||||
import re
|
||||
from urllib.parse import urlparse
|
||||
|
||||
# --- Third-Party Library Imports ---
|
||||
# This module might not require third-party libraries directly,
|
||||
# but 'requests' is a common dependency for network operations.
|
||||
# import requests
|
||||
|
||||
|
||||
def parse_cookie_string(cookie_string):
|
||||
"""
|
||||
Parses a 'name=value; name2=value2' cookie string into a dictionary.
|
||||
@@ -106,13 +99,11 @@ def prepare_cookies_for_request(use_cookie_flag, cookie_text_input, selected_coo
|
||||
if not use_cookie_flag:
|
||||
return None
|
||||
|
||||
# Priority 1: Use the specifically browsed file first
|
||||
if selected_cookie_file_path and os.path.exists(selected_cookie_file_path):
|
||||
cookies = load_cookies_from_netscape_file(selected_cookie_file_path, logger_func, target_domain)
|
||||
if cookies:
|
||||
return cookies
|
||||
|
||||
# Priority 2: Look for a domain-specific cookie file
|
||||
if app_base_dir and target_domain:
|
||||
domain_specific_path = os.path.join(app_base_dir, "data", f"{target_domain}_cookies.txt")
|
||||
if os.path.exists(domain_specific_path):
|
||||
@@ -120,7 +111,6 @@ def prepare_cookies_for_request(use_cookie_flag, cookie_text_input, selected_coo
|
||||
if cookies:
|
||||
return cookies
|
||||
|
||||
# Priority 3: Look for a generic cookies.txt
|
||||
if app_base_dir:
|
||||
default_path = os.path.join(app_base_dir, "appdata", "cookies.txt")
|
||||
if os.path.exists(default_path):
|
||||
@@ -128,7 +118,6 @@ def prepare_cookies_for_request(use_cookie_flag, cookie_text_input, selected_coo
|
||||
if cookies:
|
||||
return cookies
|
||||
|
||||
# Priority 4: Fall back to manually entered text
|
||||
if cookie_text_input:
|
||||
cookies = parse_cookie_string(cookie_text_input)
|
||||
if cookies:
|
||||
@@ -141,28 +130,72 @@ def prepare_cookies_for_request(use_cookie_flag, cookie_text_input, selected_coo
|
||||
def extract_post_info(url_string):
|
||||
"""
|
||||
Parses a URL string to extract the service, user ID, and post ID.
|
||||
|
||||
Args:
|
||||
url_string (str): The URL to parse.
|
||||
|
||||
Returns:
|
||||
tuple: A tuple containing (service, user_id, post_id). Any can be None.
|
||||
UPDATED to support Hentai2Read series and chapters.
|
||||
"""
|
||||
if not isinstance(url_string, str) or not url_string.strip():
|
||||
return None, None, None
|
||||
|
||||
try:
|
||||
parsed_url = urlparse(url_string.strip())
|
||||
path_parts = [part for part in parsed_url.path.strip('/').split('/') if part]
|
||||
|
||||
stripped_url = url_string.strip()
|
||||
|
||||
# --- Danbooru Check ---
|
||||
danbooru_match = re.search(r'danbooru\.donmai\.us|safebooru\.donmai\.us', stripped_url)
|
||||
if danbooru_match:
|
||||
return 'danbooru', None, None
|
||||
|
||||
# Standard format: /<service>/user/<user_id>/post/<post_id>
|
||||
# --- Gelbooru Check ---
|
||||
gelbooru_match = re.search(r'gelbooru\.com', stripped_url)
|
||||
if gelbooru_match:
|
||||
return 'gelbooru', None, None
|
||||
|
||||
# --- Bunkr Check ---
|
||||
bunkr_pattern = re.compile(
|
||||
r"(?:https?://)?(?:[a-zA-Z0-9-]+\.)?bunkr\.(?:si|la|ws|red|black|media|site|is|to|ac|cr|ci|fi|pk|ps|sk|ph|su|ru)|bunkrr\.ru"
|
||||
)
|
||||
if bunkr_pattern.search(stripped_url):
|
||||
return 'bunkr', stripped_url, None
|
||||
|
||||
# --- SimpCity Check (Corrected version) ---
|
||||
simpcity_match = re.search(r'simpcity\.cr/threads/([^/]+)(?:/post-(\d+))?', stripped_url)
|
||||
if simpcity_match:
|
||||
thread_info = simpcity_match.group(1)
|
||||
post_id = simpcity_match.group(2)
|
||||
return 'simpcity', thread_info, post_id
|
||||
|
||||
# --- nhentai Check ---
|
||||
nhentai_match = re.search(r'nhentai\.net/g/(\d+)', stripped_url)
|
||||
if nhentai_match:
|
||||
return 'nhentai', nhentai_match.group(1), None
|
||||
|
||||
# --- Hentai2Read Check (Corrected to match series, chapter, and image URLs) ---
|
||||
hentai2read_match = re.search(r'hentai2read\.com/([^/]+)(?:/(\d+))?/?', stripped_url)
|
||||
if hentai2read_match:
|
||||
manga_slug, chapter_num = hentai2read_match.groups()
|
||||
return 'hentai2read', manga_slug, chapter_num
|
||||
|
||||
# --- Pixeldrain Check ---
|
||||
pixeldrain_match = re.search(r'pixeldrain\.com/[lud]/([^/?#]+)', stripped_url)
|
||||
if pixeldrain_match:
|
||||
return 'pixeldrain', stripped_url, None
|
||||
|
||||
discord_channel_match = re.search(r'discord\.com/channels/(@me|\d+)/(\d+)', stripped_url)
|
||||
if discord_channel_match:
|
||||
server_id, channel_id = discord_channel_match.groups()
|
||||
return 'discord', server_id, channel_id
|
||||
|
||||
# --- Kemono/Coomer/Discord Parsing ---
|
||||
try:
|
||||
parsed_url = urlparse(stripped_url)
|
||||
path_parts = [part for part in parsed_url.path.strip('/').split('/') if part]
|
||||
|
||||
if len(path_parts) >= 3 and path_parts[0].lower() == 'discord' and path_parts[1].lower() == 'server':
|
||||
return 'discord', path_parts[2], path_parts[3] if len(path_parts) >= 4 else None
|
||||
|
||||
if len(path_parts) >= 3 and path_parts[1].lower() == 'user':
|
||||
service = path_parts[0]
|
||||
user_id = path_parts[2]
|
||||
post_id = path_parts[4] if len(path_parts) >= 5 and path_parts[3].lower() == 'post' else None
|
||||
return service, user_id, post_id
|
||||
|
||||
# API format: /api/v1/<service>/user/<user_id>...
|
||||
|
||||
if len(path_parts) >= 5 and path_parts[0:2] == ['api', 'v1'] and path_parts[3].lower() == 'user':
|
||||
service = path_parts[2]
|
||||
user_id = path_parts[4]
|
||||
@@ -173,8 +206,7 @@ def extract_post_info(url_string):
|
||||
print(f"Debug: Exception during URL parsing for '{url_string}': {e}")
|
||||
|
||||
return None, None, None
|
||||
|
||||
|
||||
|
||||
def get_link_platform(url):
|
||||
"""
|
||||
Identifies the platform of a given URL based on its domain.
|
||||
|
||||
@@ -28,19 +28,12 @@ def setup_ui(main_app):
|
||||
main_app.scale_factor = scale
|
||||
|
||||
default_font = QApplication.font()
|
||||
base_font_size = 9 # Use a standard base size
|
||||
base_font_size = 9
|
||||
default_font.setPointSize(int(base_font_size * scale))
|
||||
main_app.setFont(default_font)
|
||||
|
||||
default_font = QApplication.font()
|
||||
base_font_size = 9 # Use a standard base size
|
||||
default_font.setPointSize(int(base_font_size * scale))
|
||||
main_app.setFont(default_font)
|
||||
# --- END: Improved Scaling Logic ---
|
||||
|
||||
main_app.main_splitter = QSplitter(Qt.Horizontal)
|
||||
|
||||
# --- Use a scroll area for the left panel for consistency ---
|
||||
left_scroll_area = QScrollArea()
|
||||
left_scroll_area.setWidgetResizable(True)
|
||||
left_scroll_area.setFrameShape(QFrame.NoFrame)
|
||||
@@ -75,7 +68,7 @@ def setup_ui(main_app):
|
||||
main_app.empty_popup_button.clicked.connect(main_app._show_empty_popup)
|
||||
url_input_layout.addWidget(main_app.empty_popup_button)
|
||||
main_app.page_range_label = QLabel(main_app._tr("page_range_label_text", "Page Range:"))
|
||||
main_app.page_range_label.setStyleSheet("font-weight: bold; padding-left: 10px;")
|
||||
main_app.page_range_label .setStyleSheet("font-weight: bold; padding-left: 10px;")
|
||||
url_input_layout.addWidget(main_app.page_range_label)
|
||||
main_app.start_page_input = QLineEdit()
|
||||
main_app.start_page_input.setPlaceholderText(main_app._tr("start_page_input_placeholder", "Start"))
|
||||
@@ -134,8 +127,6 @@ def setup_ui(main_app):
|
||||
main_app._update_char_filter_scope_button_text()
|
||||
char_input_and_button_layout.addWidget(main_app.char_filter_scope_toggle_button, 1)
|
||||
character_filter_v_layout.addLayout(char_input_and_button_layout)
|
||||
|
||||
# --- Custom Folder Widget Definition ---
|
||||
main_app.custom_folder_widget = QWidget()
|
||||
custom_folder_v_layout = QVBoxLayout(main_app.custom_folder_widget)
|
||||
custom_folder_v_layout.setContentsMargins(0, 0, 0, 0)
|
||||
@@ -146,7 +137,6 @@ def setup_ui(main_app):
|
||||
custom_folder_v_layout.addWidget(main_app.custom_folder_label)
|
||||
custom_folder_v_layout.addWidget(main_app.custom_folder_input)
|
||||
main_app.custom_folder_widget.setVisible(False)
|
||||
|
||||
filters_and_custom_folder_layout.addWidget(main_app.character_filter_widget, 1)
|
||||
filters_and_custom_folder_layout.addWidget(main_app.custom_folder_widget, 1)
|
||||
left_layout.addWidget(main_app.filters_and_custom_folder_container_widget)
|
||||
@@ -199,7 +189,6 @@ def setup_ui(main_app):
|
||||
main_app.radio_only_audio = QRadioButton("🎧 Only Audio")
|
||||
main_app.radio_only_links = QRadioButton("🔗 Only Links")
|
||||
main_app.radio_more = QRadioButton("More")
|
||||
|
||||
main_app.radio_all.setChecked(True)
|
||||
for btn in [main_app.radio_all, main_app.radio_images, main_app.radio_videos, main_app.radio_only_archives, main_app.radio_only_audio, main_app.radio_only_links, main_app.radio_more]:
|
||||
main_app.radio_group.addButton(btn)
|
||||
@@ -211,6 +200,24 @@ def setup_ui(main_app):
|
||||
file_filter_layout.addLayout(radio_button_layout)
|
||||
left_layout.addLayout(file_filter_layout)
|
||||
|
||||
# --- Booru Inputs Container ---
|
||||
main_app.booru_inputs_widget = QWidget()
|
||||
booru_inputs_layout = QHBoxLayout(main_app.booru_inputs_widget)
|
||||
booru_inputs_layout.setContentsMargins(0, 5, 0, 0)
|
||||
main_app.api_key_label = QLabel("API Key:")
|
||||
main_app.api_key_input = QLineEdit()
|
||||
main_app.api_key_input.setPlaceholderText("Danbooru or Gelbooru API Key")
|
||||
main_app.user_id_label = QLabel("User ID:")
|
||||
main_app.user_id_input = QLineEdit()
|
||||
main_app.user_id_input.setPlaceholderText("Danbooru Username or Gelbooru User ID")
|
||||
booru_inputs_layout.addWidget(main_app.api_key_label)
|
||||
booru_inputs_layout.addWidget(main_app.api_key_input, 1)
|
||||
booru_inputs_layout.addSpacing(10)
|
||||
booru_inputs_layout.addWidget(main_app.user_id_label)
|
||||
booru_inputs_layout.addWidget(main_app.user_id_input, 1)
|
||||
left_layout.addWidget(main_app.booru_inputs_widget)
|
||||
main_app.booru_inputs_widget.setVisible(False)
|
||||
|
||||
# --- Checkboxes Group ---
|
||||
checkboxes_group_layout = QVBoxLayout()
|
||||
checkboxes_group_layout.setSpacing(10)
|
||||
@@ -234,40 +241,42 @@ def setup_ui(main_app):
|
||||
row1_layout.addStretch(1)
|
||||
checkboxes_group_layout.addLayout(row1_layout)
|
||||
|
||||
# --- Advanced Settings ---
|
||||
# --- Advanced Settings Container ---
|
||||
main_app.advanced_settings_widget = QWidget()
|
||||
advanced_settings_layout = QVBoxLayout(main_app.advanced_settings_widget)
|
||||
advanced_settings_layout.setContentsMargins(0, 0, 0, 0)
|
||||
advanced_settings_layout.setSpacing(10)
|
||||
advanced_settings_label = QLabel("⚙️ Advanced Settings:")
|
||||
checkboxes_group_layout.addWidget(advanced_settings_label)
|
||||
advanced_row1_layout = QHBoxLayout()
|
||||
advanced_row1_layout.setSpacing(10)
|
||||
advanced_settings_layout.addWidget(advanced_settings_label)
|
||||
|
||||
# --- REORDERED CHECKBOXES ---
|
||||
main_app.advanced_row1_layout = QHBoxLayout()
|
||||
main_app.advanced_row1_layout.setSpacing(10)
|
||||
main_app.use_subfolder_per_post_checkbox = QCheckBox("Subfolder per Post")
|
||||
main_app.use_subfolder_per_post_checkbox.toggled.connect(main_app.update_ui_for_subfolders)
|
||||
main_app.use_subfolder_per_post_checkbox.setChecked(True)
|
||||
advanced_row1_layout.addWidget(main_app.use_subfolder_per_post_checkbox)
|
||||
|
||||
main_app.advanced_row1_layout.addWidget(main_app.use_subfolder_per_post_checkbox)
|
||||
main_app.date_prefix_checkbox = QCheckBox("Date Prefix")
|
||||
main_app.date_prefix_checkbox.setToolTip("When 'Subfolder per Post' is active, prefix the folder name with the post's upload date.")
|
||||
advanced_row1_layout.addWidget(main_app.date_prefix_checkbox)
|
||||
|
||||
main_app.advanced_row1_layout.addWidget(main_app.date_prefix_checkbox)
|
||||
main_app.use_subfolders_checkbox = QCheckBox("Separate Folders by Known.txt")
|
||||
main_app.use_subfolders_checkbox.setChecked(False)
|
||||
main_app.use_subfolders_checkbox.toggled.connect(main_app.update_ui_for_subfolders)
|
||||
advanced_row1_layout.addWidget(main_app.use_subfolders_checkbox)
|
||||
# --- END REORDER ---
|
||||
|
||||
main_app.advanced_row1_layout.addWidget(main_app.use_subfolders_checkbox)
|
||||
|
||||
# --- Original Cookie Controls (for non-SimpCity sites) ---
|
||||
main_app.use_cookie_checkbox = QCheckBox("Use Cookie")
|
||||
main_app.use_cookie_checkbox.setChecked(main_app.use_cookie_setting)
|
||||
main_app.cookie_text_input = QLineEdit()
|
||||
main_app.cookie_text_input.setPlaceholderText("if no Select cookies.txt)")
|
||||
main_app.cookie_text_input.setPlaceholderText("Cookie string or path from Browse...")
|
||||
main_app.cookie_text_input.setText(main_app.cookie_text_setting)
|
||||
advanced_row1_layout.addWidget(main_app.use_cookie_checkbox)
|
||||
advanced_row1_layout.addWidget(main_app.cookie_text_input, 2)
|
||||
main_app.cookie_browse_button = QPushButton("Browse...")
|
||||
main_app.cookie_browse_button.setFixedWidth(int(80 * scale))
|
||||
advanced_row1_layout.addWidget(main_app.cookie_browse_button)
|
||||
advanced_row1_layout.addStretch(1)
|
||||
checkboxes_group_layout.addLayout(advanced_row1_layout)
|
||||
main_app.advanced_row1_layout.addWidget(main_app.use_cookie_checkbox)
|
||||
main_app.advanced_row1_layout.addWidget(main_app.cookie_text_input, 2)
|
||||
main_app.advanced_row1_layout.addWidget(main_app.cookie_browse_button)
|
||||
main_app.advanced_row1_layout.addStretch(1)
|
||||
advanced_settings_layout.addLayout(main_app.advanced_row1_layout)
|
||||
|
||||
advanced_row2_layout = QHBoxLayout()
|
||||
advanced_row2_layout.setSpacing(10)
|
||||
multithreading_layout = QHBoxLayout()
|
||||
@@ -284,13 +293,58 @@ def setup_ui(main_app):
|
||||
advanced_row2_layout.addLayout(multithreading_layout)
|
||||
main_app.external_links_checkbox = QCheckBox("Show External Links in Log")
|
||||
advanced_row2_layout.addWidget(main_app.external_links_checkbox)
|
||||
main_app.manga_mode_checkbox = QCheckBox("Manga/Comic Mode")
|
||||
main_app.manga_mode_checkbox = QCheckBox("Renaming Mode")
|
||||
advanced_row2_layout.addWidget(main_app.manga_mode_checkbox)
|
||||
advanced_row2_layout.addStretch(1)
|
||||
checkboxes_group_layout.addLayout(advanced_row2_layout)
|
||||
advanced_settings_layout.addLayout(advanced_row2_layout)
|
||||
checkboxes_group_layout.addWidget(main_app.advanced_settings_widget)
|
||||
|
||||
# --- SimpCity Settings Container (with its own cookie controls) ---
|
||||
main_app.simpcity_settings_widget = QWidget()
|
||||
simpcity_settings_layout = QVBoxLayout(main_app.simpcity_settings_widget)
|
||||
simpcity_settings_layout.setContentsMargins(0, 0, 0, 0)
|
||||
simpcity_settings_layout.setSpacing(10)
|
||||
simpcity_settings_label = QLabel("⚙️ SimpCity Download Options:")
|
||||
simpcity_settings_layout.addWidget(simpcity_settings_label)
|
||||
|
||||
# Checkbox row
|
||||
simpcity_checkboxes_layout = QHBoxLayout()
|
||||
main_app.simpcity_dl_pixeldrain_cb = QCheckBox("Download Pixeldrain")
|
||||
main_app.simpcity_dl_saint2_cb = QCheckBox("Download Saint2.su")
|
||||
main_app.simpcity_dl_mega_cb = QCheckBox("Download Mega")
|
||||
main_app.simpcity_dl_bunkr_cb = QCheckBox("Download Bunkr")
|
||||
main_app.simpcity_dl_gofile_cb = QCheckBox("Download Gofile")
|
||||
|
||||
simpcity_checkboxes_layout.addWidget(main_app.simpcity_dl_pixeldrain_cb)
|
||||
simpcity_checkboxes_layout.addWidget(main_app.simpcity_dl_saint2_cb)
|
||||
simpcity_checkboxes_layout.addWidget(main_app.simpcity_dl_mega_cb)
|
||||
simpcity_checkboxes_layout.addWidget(main_app.simpcity_dl_bunkr_cb)
|
||||
simpcity_checkboxes_layout.addWidget(main_app.simpcity_dl_gofile_cb)
|
||||
simpcity_checkboxes_layout.addStretch(1)
|
||||
simpcity_settings_layout.addLayout(simpcity_checkboxes_layout)
|
||||
|
||||
# --- START NEW CODE ---
|
||||
# Create the second, dedicated set of cookie controls for SimpCity
|
||||
simpcity_cookie_layout = QHBoxLayout()
|
||||
simpcity_cookie_layout.setContentsMargins(0, 5, 0, 0) # Add some top margin
|
||||
simpcity_cookie_label = QLabel("Cookie:")
|
||||
main_app.simpcity_cookie_text_input = QLineEdit()
|
||||
main_app.simpcity_cookie_text_input.setPlaceholderText("Cookie string or path... (Required)")
|
||||
main_app.simpcity_cookie_browse_button = QPushButton("Browse...")
|
||||
main_app.simpcity_cookie_browse_button.setFixedWidth(int(80 * scale))
|
||||
|
||||
simpcity_cookie_layout.addWidget(simpcity_cookie_label)
|
||||
simpcity_cookie_layout.addWidget(main_app.simpcity_cookie_text_input, 1) # Stretch factor
|
||||
simpcity_cookie_layout.addWidget(main_app.simpcity_cookie_browse_button)
|
||||
|
||||
simpcity_settings_layout.addLayout(simpcity_cookie_layout)
|
||||
checkboxes_group_layout.addWidget(main_app.simpcity_settings_widget)
|
||||
main_app.simpcity_settings_widget.setVisible(False)
|
||||
|
||||
left_layout.addLayout(checkboxes_group_layout)
|
||||
|
||||
# --- Action Buttons ---
|
||||
# --- Action Buttons & Remaining UI ---
|
||||
# ... (The rest of the setup_ui function remains unchanged)
|
||||
main_app.standard_action_buttons_widget = QWidget()
|
||||
btn_layout = QHBoxLayout(main_app.standard_action_buttons_widget)
|
||||
btn_layout.setContentsMargins(0, 10, 0, 0)
|
||||
@@ -326,8 +380,6 @@ def setup_ui(main_app):
|
||||
main_app.bottom_action_buttons_stack.addWidget(main_app.favorite_action_buttons_widget)
|
||||
left_layout.addWidget(main_app.bottom_action_buttons_stack)
|
||||
left_layout.addSpacing(10)
|
||||
|
||||
# --- Known Names Layout ---
|
||||
known_chars_label_layout = QHBoxLayout()
|
||||
known_chars_label_layout.setSpacing(10)
|
||||
main_app.known_chars_label = QLabel("🎭 Known Shows/Characters (for Folder Names):")
|
||||
@@ -376,8 +428,6 @@ def setup_ui(main_app):
|
||||
char_manage_layout.addWidget(main_app.support_button, 0)
|
||||
left_layout.addLayout(char_manage_layout)
|
||||
left_layout.addStretch(0)
|
||||
|
||||
# --- Right Panel (Logs) ---
|
||||
right_panel_widget.setLayout(right_layout)
|
||||
log_title_layout = QHBoxLayout()
|
||||
main_app.progress_log_label = QLabel("📜 Progress Log:")
|
||||
@@ -387,15 +437,31 @@ def setup_ui(main_app):
|
||||
main_app.link_search_input.setPlaceholderText("Search Links...")
|
||||
main_app.link_search_input.setVisible(False)
|
||||
log_title_layout.addWidget(main_app.link_search_input)
|
||||
main_app.link_search_button = QPushButton("<EFBFBD>")
|
||||
main_app.link_search_button = QPushButton("🔎")
|
||||
main_app.link_search_button.setVisible(False)
|
||||
main_app.link_search_button.setFixedWidth(int(30 * scale))
|
||||
log_title_layout.addWidget(main_app.link_search_button)
|
||||
discord_controls_layout = QHBoxLayout()
|
||||
main_app.discord_scope_toggle_button = QPushButton("Scope: Files")
|
||||
main_app.discord_scope_toggle_button.setVisible(False)
|
||||
discord_controls_layout.addWidget(main_app.discord_scope_toggle_button)
|
||||
main_app.discord_message_limit_input = QLineEdit(main_app)
|
||||
main_app.discord_message_limit_input.setPlaceholderText("Msg Limit")
|
||||
main_app.discord_message_limit_input.setToolTip("Optional: Limit the number of recent messages to process.")
|
||||
main_app.discord_message_limit_input.setValidator(QIntValidator(1, 9999999, main_app))
|
||||
main_app.discord_message_limit_input.setFixedWidth(int(80 * scale))
|
||||
main_app.discord_message_limit_input.setVisible(False)
|
||||
discord_controls_layout.addWidget(main_app.discord_message_limit_input)
|
||||
log_title_layout.addLayout(discord_controls_layout)
|
||||
main_app.manga_rename_toggle_button = QPushButton()
|
||||
main_app.manga_rename_toggle_button.setVisible(False)
|
||||
main_app.manga_rename_toggle_button.setFixedWidth(int(140 * scale))
|
||||
main_app._update_manga_filename_style_button_text()
|
||||
log_title_layout.addWidget(main_app.manga_rename_toggle_button)
|
||||
main_app.custom_rename_dialog_button = QPushButton("Open Dialog")
|
||||
main_app.custom_rename_dialog_button.setVisible(False)
|
||||
main_app.custom_rename_dialog_button.clicked.connect(main_app._show_custom_rename_dialog)
|
||||
log_title_layout.addWidget(main_app.custom_rename_dialog_button)
|
||||
main_app.manga_date_prefix_input = QLineEdit()
|
||||
main_app.manga_date_prefix_input.setPlaceholderText("Prefix for Manga Filenames")
|
||||
main_app.manga_date_prefix_input.setVisible(False)
|
||||
@@ -458,26 +524,17 @@ def setup_ui(main_app):
|
||||
main_app.file_progress_label.setWordWrap(True)
|
||||
main_app.file_progress_label.setStyleSheet("padding-top: 2px; font-style: italic; color: #A0A0A0;")
|
||||
right_layout.addWidget(main_app.file_progress_label)
|
||||
|
||||
# --- Final Assembly ---
|
||||
main_app.main_splitter.addWidget(left_scroll_area)
|
||||
main_app.main_splitter.addWidget(right_panel_widget)
|
||||
|
||||
if main_app.width() >= 1920:
|
||||
# For wider resolutions, give more space to the log panel (right).
|
||||
main_app.main_splitter.setStretchFactor(0, 4)
|
||||
main_app.main_splitter.setStretchFactor(1, 6)
|
||||
else:
|
||||
# Default for lower resolutions, giving more space to controls (left).
|
||||
main_app.main_splitter.setStretchFactor(0, 7)
|
||||
main_app.main_splitter.setStretchFactor(1, 3)
|
||||
|
||||
|
||||
top_level_layout = QHBoxLayout(main_app)
|
||||
top_level_layout.setContentsMargins(0, 0, 0, 0)
|
||||
top_level_layout.addWidget(main_app.main_splitter)
|
||||
|
||||
# --- Initial UI State Updates ---
|
||||
main_app.update_ui_for_subfolders(main_app.use_subfolders_checkbox.isChecked())
|
||||
main_app.update_external_links_setting(main_app.external_links_checkbox.isChecked())
|
||||
main_app.update_multithreading_label(main_app.thread_count_input.text())
|
||||
@@ -493,7 +550,6 @@ def setup_ui(main_app):
|
||||
if hasattr(main_app, 'radio_group') and main_app.radio_group.checkedButton():
|
||||
main_app._handle_filter_mode_change(main_app.radio_group.checkedButton(), True)
|
||||
main_app.radio_group.buttonToggled.connect(main_app._handle_more_options_toggled)
|
||||
|
||||
main_app._update_manga_filename_style_button_text()
|
||||
main_app._update_skip_scope_button_text()
|
||||
main_app._update_char_filter_scope_button_text()
|
||||
@@ -535,6 +591,9 @@ def get_dark_theme(scale=1):
|
||||
border-radius: 4px;
|
||||
font-size: {font_size}pt;
|
||||
}}
|
||||
QLineEdit::placeholder {{
|
||||
color: #8A8A8A; /* A muted grey color for placeholder text */
|
||||
}}
|
||||
QTextEdit {{
|
||||
font-family: Consolas, Courier New, monospace;
|
||||
}}
|
||||
|
||||
@@ -168,6 +168,7 @@ def match_folders_from_title(title, names_to_match, unwanted_keywords):
|
||||
def match_folders_from_filename_enhanced(filename, names_to_match, unwanted_keywords):
|
||||
"""
|
||||
Matches folder names from a filename, prioritizing longer and more specific aliases.
|
||||
It returns immediately after finding the first (longest) match.
|
||||
|
||||
Args:
|
||||
filename (str): The filename to check.
|
||||
@@ -175,15 +176,14 @@ def match_folders_from_filename_enhanced(filename, names_to_match, unwanted_keyw
|
||||
unwanted_keywords (set): A set of folder names to ignore.
|
||||
|
||||
Returns:
|
||||
list: A sorted list of matched primary folder names.
|
||||
list: A list containing the single best folder name match, or an empty list.
|
||||
"""
|
||||
if not filename or not names_to_match:
|
||||
return []
|
||||
|
||||
filename_lower = filename.lower()
|
||||
matched_primary_names = set()
|
||||
|
||||
# Create a flat list of (alias, primary_name) tuples to sort by alias length
|
||||
# Create a flat list of (alias, primary_name) tuples
|
||||
alias_map_to_primary = []
|
||||
for name_obj in names_to_match:
|
||||
primary_name = name_obj.get("name")
|
||||
@@ -200,8 +200,11 @@ def match_folders_from_filename_enhanced(filename, names_to_match, unwanted_keyw
|
||||
# Sort by alias length, descending, to match longer aliases first
|
||||
alias_map_to_primary.sort(key=lambda x: len(x[0]), reverse=True)
|
||||
|
||||
# <<< MODIFICATION: Return the FIRST match found, which will be the longest >>>
|
||||
for alias_lower, primary_name_for_alias in alias_map_to_primary:
|
||||
if filename_lower.startswith(alias_lower):
|
||||
matched_primary_names.add(primary_name_for_alias)
|
||||
|
||||
return sorted(list(matched_primary_names))
|
||||
if alias_lower in filename_lower:
|
||||
# Found the longest possible alias that is a substring. Return immediately.
|
||||
return [primary_name_for_alias]
|
||||
|
||||
# If the loop finishes without any matches, return an empty list.
|
||||
return []
|
||||
|
||||
BIN
yt-dlp.exe
Normal file
BIN
yt-dlp.exe
Normal file
Binary file not shown.
Reference in New Issue
Block a user