mirror of
https://github.com/Yuvi9587/Kemono-Downloader.git
synced 2025-12-29 16:14:44 +00:00
Compare commits
6 Commits
v6.5.0
...
510ae5e1d1
| Author | SHA1 | Date | |
|---|---|---|---|
|
510ae5e1d1 | ||
|
|
65b4759bad | ||
|
|
6e993d88de | ||
|
|
cc3565b12b | ||
|
|
f8b150dfdb | ||
|
|
5f7b526852 |
500
features.md
500
features.md
@@ -1,391 +1,159 @@
|
||||
<div>
|
||||
<h1>Kemono Downloader - Comprehensive Feature Guide</h1>
|
||||
<p>This guide provides a detailed overview of all user interface elements, input fields, buttons, popups, and functionalities available in the application.</p>
|
||||
<hr>
|
||||
|
||||
<h2><strong>1. URL Input (🔗)</strong></h2>
|
||||
<p>This is the primary input field where you specify the content you want to download.</p>
|
||||
|
||||
<p><strong>Functionality:</strong></p>
|
||||
<ul>
|
||||
<li><strong>Creator URL:</strong> A link to a creator's main page (e.g., https://kemono.su/patreon/user/12345). Downloads all posts from the creator.</li>
|
||||
<li><strong>Post URL:</strong> A direct link to a specific post (e.g., .../post/98765). Downloads only the specified post.</li>
|
||||
</ul>
|
||||
|
||||
<p><strong>Interaction with Other Features:</strong> The content of this field influences "Manga Mode" and "Page Range". "Page Range" is enabled only with a creator URL.</p>
|
||||
|
||||
<hr>
|
||||
|
||||
<h2><strong>2. Creator Selection & Update (🎨)</strong></h2>
|
||||
<p>The color palette emoji button opens the Creator Selection & Update dialog. This allows managing and downloading from a local creator database.</p>
|
||||
|
||||
<p><strong>Functionality:</strong></p>
|
||||
<ul>
|
||||
<li><strong>Creator Browser:</strong> Loads a list from <code>creators.json</code>. Search by name, service, or paste a URL to find creators.</li>
|
||||
<li><strong>Batch Selection:</strong> Select multiple creators and click "Add Selected" to add them to the batch download session.</li>
|
||||
<li><strong>Update Checker:</strong> Use a saved profile (.json) to download only new content based on previously fetched posts.</li>
|
||||
<li><strong>Post Fetching & Filtering:</strong> "Fetch Posts" loads post titles, allowing you to choose specific posts for download.</li>
|
||||
</ul>
|
||||
|
||||
<hr>
|
||||
|
||||
<h2><strong>3. Download Location Input (📁)</strong></h2>
|
||||
<p>This input defines the destination directory for downloaded files.</p>
|
||||
|
||||
<p><strong>Functionality:</strong></p>
|
||||
<ul>
|
||||
<li><strong>Manual Entry:</strong> Enter or paste the folder path.</li>
|
||||
<li><strong>Browse Button:</strong> Opens a system dialog to choose a folder.</li>
|
||||
<li><strong>Directory Creation:</strong> If the folder doesn't exist, the app can create it after user confirmation.</li>
|
||||
</ul>
|
||||
|
||||
<hr>
|
||||
|
||||
<h2><strong>4. Filter by Character(s) & Scope Button</strong></h2>
|
||||
<p>Used to download content for specific characters or series and organize them into subfolders.</p>
|
||||
|
||||
<p><strong>Input Field (Filter by Character(s)):</strong></p>
|
||||
<ul>
|
||||
<li>Enter comma-separated names (e.g., <code>Tifa, Aerith</code>).</li>
|
||||
<li>Group aliases using parentheses (e.g., <code>(Cloud, Zack)</code>).</li>
|
||||
<li>Names are matched against titles, filenames, or comments.</li>
|
||||
<li>If "Separate Folders by Known.txt" is enabled, the name becomes the subfolder name.</li>
|
||||
</ul>
|
||||
|
||||
<p><strong>Scope Button Modes:</strong></p>
|
||||
<ul>
|
||||
<li><strong>Filter: Title</strong> (default) – Match names in post titles only.</li>
|
||||
<li><strong>Filter: Files</strong> – Match names in filenames only.</li>
|
||||
<li><strong>Filter: Both</strong> – Try title match first, then filenames.</li>
|
||||
<li><strong>Filter: Comments</strong> – Try filenames first, then post comments if no match.</li>
|
||||
</ul>
|
||||
|
||||
<hr>
|
||||
|
||||
<h2><strong>5. Skip with Words & Scope Button</strong></h2>
|
||||
<p>Prevents downloading content based on keywords.</p>
|
||||
|
||||
<p><strong>Input Field (Skip with Words):</strong></p>
|
||||
<ul>
|
||||
<li>Enter comma-separated keywords (e.g., <code>WIP, sketch, preview</code>).</li>
|
||||
<li>Matching is case-insensitive.</li>
|
||||
<li>If a keyword matches, the file or post is skipped.</li>
|
||||
</ul>
|
||||
|
||||
<p><strong>Scope Button Modes:</strong></p>
|
||||
<ul>
|
||||
<li><strong>Scope: Posts</strong> (default) – Skips post if title contains a keyword.</li>
|
||||
<li><strong>Scope: Files</strong> – Skips individual files with keyword matches.</li>
|
||||
<li><strong>Scope: Both</strong> – Skips entire post if title matches, otherwise filters individual files.</li>
|
||||
</ul>
|
||||
</div>
|
||||
<div>
|
||||
<h2><strong>Filter File Section (Radio Buttons)</strong></h2>
|
||||
<p>This section uses a group of radio buttons to control the primary download mode, dictating which types of files are targeted. Only one of these modes can be active at a time.</p>
|
||||
|
||||
<ul>
|
||||
<li>
|
||||
<strong>All:</strong> Default mode. Downloads every file and attachment provided by the API, regardless of type.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Images/GIFs:</strong> Filters for common image formats (<code>.jpg</code>, <code>.png</code>, <code>.gif</code>, <code>.webp</code>), skipping non-image files.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Videos:</strong> Filters for common video formats like <code>.mp4</code>, <code>.webm</code>, and <code>.mov</code>, skipping all others.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Only Archives:</strong> Downloads only archive files (<code>.zip</code>, <code>.rar</code>). Disables "Compress to WebP" and unchecks "Skip Archives".
|
||||
</li>
|
||||
<li>
|
||||
<strong>Only Audio:</strong> Filters for common audio formats like <code>.mp3</code>, <code>.wav</code>, and <code>.flac</code>.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Only Links:</strong> Extracts external hyperlinks from post descriptions (e.g., Mega, Google Drive) and displays them in the log. Disables all download options.
|
||||
</li>
|
||||
<li>
|
||||
<strong>More:</strong> Opens the "More Options" dialog to download text-based content instead of media files.
|
||||
<ul>
|
||||
<li><strong>Scope:</strong> Choose to extract from post description or comments.</li>
|
||||
<li><strong>Export Format:</strong> Save text as PDF, DOCX, or TXT.</li>
|
||||
<li><strong>Single PDF:</strong> Optionally compile all text into one PDF.</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
<hr>
|
||||
|
||||
<h2><strong>Check Box Buttons</strong></h2>
|
||||
<p>These checkboxes provide additional toggles to refine the download behavior and enable special features.</p>
|
||||
|
||||
<ul>
|
||||
<li>
|
||||
<strong>⭐ Favorite Mode:</strong> Changes workflow to download from your personal favorites. Disables the URL input.
|
||||
<ul>
|
||||
<li><strong>Favorite Artists:</strong> Opens a dialog to select from your favorited creators.</li>
|
||||
<li><strong>Favorite Posts:</strong> Opens a dialog to select from your favorited posts on Kemono and Coomer.</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li>
|
||||
<strong>Skip Archives:</strong> When checked, archive files (<code>.zip</code>, <code>.rar</code>) are ignored. Disabled in "Only Archives" mode.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Download Thumbnail Only:</strong> Saves only thumbnail previews, not full-resolution files. Enables "Scan Content for Images".
|
||||
</li>
|
||||
<li>
|
||||
<strong>Scan Content for Images:</strong> Parses post HTML for embedded images not listed in the API. Looks for <code><img></code> tags and direct image links.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Compress to WebP:</strong> Converts large images (over 1.5 MB) to WebP format using the Pillow library for space-saving.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Keep Duplicates:</strong> Provides control over duplicate handling via the "Duplicate Handling Options" dialog.
|
||||
<ul>
|
||||
<li><strong>Skip by Hash:</strong> Default – skip identical files.</li>
|
||||
<li><strong>Keep Everything:</strong> Save all files regardless of duplication.</li>
|
||||
<li><strong>Limit:</strong> Set a limit on how many copies of the same file are saved. A limit of <code>0</code> means no limit.</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
<h2><strong>Folder Organization Checkboxes</strong></h2>
|
||||
<h1>Kemono Downloader - Comprehensive Feature Guide</h1>
|
||||
<p>This guide provides a detailed overview of all user interface elements, input fields, buttons, popups, and functionalities available in the application.</p>
|
||||
<hr>
|
||||
<h2>1. Core Concepts & Supported Sites</h2>
|
||||
<h3>URL Input (🔗)</h3>
|
||||
<p>This is the primary input field where you specify the content you want to download.</p>
|
||||
<p><strong>Supported URL Types:</strong></p>
|
||||
<ul>
|
||||
<li>
|
||||
<strong>Separate folders by Known.txt:</strong> Automatically organizes downloads into folders based on name matches.
|
||||
<ul>
|
||||
<li>Uses "Filter by Character(s)" input first, if available.</li>
|
||||
<li>Then checks names in <code>Known.txt</code>.</li>
|
||||
<li>Falls back to extracting from post title.</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li>
|
||||
<strong>Subfolder per post:</strong> Creates a unique folder per post, using the post’s title.
|
||||
<ul>
|
||||
<li>Prevents mixing files from multiple posts.</li>
|
||||
<li>Can be combined with Known.txt-based folders.</li>
|
||||
<li>Ensures uniqueness (e.g., <code>My Post Title_1</code>).</li>
|
||||
<li>Automatically removes empty folders.</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li>
|
||||
<strong>Date prefix:</strong> Enabled only with "Subfolder per post". Prepends the post date (e.g., <code>2025-08-03 My Post Title</code>) for chronological sorting.
|
||||
</li>
|
||||
<li><strong>Creator URL</strong>: A link to a creator's main page. Downloads all posts from that creator.</li>
|
||||
<li><strong>Post URL</strong>: A direct link to a specific post. Downloads only that single post.</li>
|
||||
<li><strong>Batch Command</strong>: Special keywords to trigger bulk downloading from a text file (see Batch Downloading section).</li>
|
||||
</ul>
|
||||
|
||||
<h2><strong>General Functionality Checkboxes</strong></h2>
|
||||
<p><strong>Supported Websites:</strong></p>
|
||||
<ul>
|
||||
<li>
|
||||
<strong>Use cookie:</strong> Enables login-based access via cookies.
|
||||
<li>Kemono (<code>kemono.su</code>, <code>kemono.party</code>, etc.)</li>
|
||||
<li>Coomer (<code>coomer.su</code>, <code>coomer.party</code>, etc.)</li>
|
||||
<li>Discord (via Kemono/Coomer API)</li>
|
||||
<li>Bunkr</li>
|
||||
<li>Erome</li>
|
||||
<li>Saint2.su</li>
|
||||
<li>nhentai</li>
|
||||
</ul>
|
||||
<hr>
|
||||
<h2>2. Main Download Controls & Inputs</h2>
|
||||
<h3>Download Location (📁)</h3>
|
||||
<p>This input defines the main folder where your files will be saved.</p>
|
||||
<ul>
|
||||
<li><strong>Browse Button</strong>: Opens a system dialog to choose a folder.</li>
|
||||
<li><strong>Directory Creation</strong>: If the folder doesn't exist, the app will ask for confirmation to create it.</li>
|
||||
</ul>
|
||||
<h3>Filter by Character(s) & Scope</h3>
|
||||
<p>Used to download content for specific characters or series and organize them into subfolders.</p>
|
||||
<ul>
|
||||
<li><strong>Input Field</strong>: Enter comma-separated names (e.g., <code>Tifa, Aerith</code>). Group aliases using parentheses for folder naming (e.g., <code>(Cloud, Zack)</code>).</li>
|
||||
<li><strong>Scope Button</strong>: Cycles through where to look for name matches:
|
||||
<ul>
|
||||
<li>Paste cookie string directly, or browse to select a <code>cookies.txt</code> file.</li>
|
||||
<li>Cookies are used in all authenticated API requests.</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li>
|
||||
<strong>Use Multithreading:</strong> Enables parallel downloading of posts.
|
||||
<ul>
|
||||
<li>Specify the number of worker threads (e.g., 10).</li>
|
||||
<li>Disabled for Manga Mode and Only Links mode.</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li>
|
||||
<strong>Show external links in log:</strong> Adds a secondary log that displays links (e.g., Mega, Dropbox) found in post text.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Manga/Comic mode:</strong> Sorts posts chronologically before download.
|
||||
<ul>
|
||||
<li>Ensures correct page order for comics/manga.</li>
|
||||
</ul>
|
||||
<strong>Scope Button (Name: ...):</strong> Controls filename style:
|
||||
<ul>
|
||||
<li><strong>Name: Post Title</strong> — e.g., <code>Chapter-1.jpg</code></li>
|
||||
<li><strong>Name: Date + Original</strong> — e.g., <code>2025-08-03_filename.png</code></li>
|
||||
<li><strong>Name: Date + Title</strong> — e.g., <code>2025-08-03_Chapter-1.jpg</code></li>
|
||||
<li><strong>Name: Title+G.Num</strong> — e.g., <code>Page_001.jpg</code></li>
|
||||
<li><strong>Name: Date Based</strong> — e.g., <code>001.jpg</code>, with optional prefix</li>
|
||||
<li><strong>Name: Post ID</strong> — uses unique post ID as filename</li>
|
||||
<li><strong>Filter: Title</strong>: Matches names in the post title.</li>
|
||||
<li><strong>Filter: Files</strong>: Matches names in the filenames.</li>
|
||||
<li><strong>Filter: Both</strong>: Checks the title first, then filenames.</li>
|
||||
<li><strong>Filter: Comments</strong>: Checks filenames first, then post comments.</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
<h2><strong>Start Download</strong></h2>
|
||||
<h3>Skip with Words & Scope</h3>
|
||||
<p>Prevents downloading content based on keywords or file size.</p>
|
||||
<ul>
|
||||
<li>
|
||||
<strong>Default State ("⬇️ Start Download"):</strong> When idle, this button gathers all current settings (URL, filters, checkboxes, etc.) and begins the download process via the DownloadManager.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Restore State:</strong> If an interrupted session is detected, the tooltip will indicate that starting a new download will discard previous session progress.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Update Mode (Phase 1 - "🔄 Check For Updates"):</strong> If a creator profile is loaded, clicking this button will fetch the creator's posts and compare them against your saved profile to identify new content.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Update Mode (Phase 2 - "⬇️ Start Download (X new)"):</strong> After new posts are found, the button text updates to reflect the number. Clicking it downloads only the new content.
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
<h2><strong>Pause / Resume Download</strong></h2>
|
||||
<ul>
|
||||
<li>
|
||||
<strong>While Downloading:</strong> The button toggles between:
|
||||
<li><strong>Input Field</strong>: Enter comma-separated keywords (e.g., <code>WIP, sketch, preview</code>).</li>
|
||||
<li><strong>Skip by Size</strong>: Enter a number in square brackets to skip any file <strong>smaller than</strong> that size in MB. For example, <code>WIP, [200]</code> skips files with "WIP" in the name AND any file smaller than 200 MB.</li>
|
||||
<li><strong>Scope Button</strong>: Cycles through where to apply keyword filters:
|
||||
<ul>
|
||||
<li><strong>"⏸️ Pause Download":</strong> Sets a <code>pause_event</code>, which tells all worker threads to halt their current task and wait.</li>
|
||||
<li><strong>"▶️ Resume Download":</strong> Clears the <code>pause_event</code>, allowing threads to resume their work.</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li>
|
||||
<strong>While Idle:</strong> The button is disabled.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Restore State:</strong> Changes to "🔄 Restore Download", which resumes the last session from saved data.
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
<h2><strong>Cancel & Reset UI</strong></h2>
|
||||
<ul>
|
||||
<li>
|
||||
<strong>Functionality:</strong> Stops downloads gracefully using a <code>cancellation_event</code>. Threads finish current tasks before shutting down.
|
||||
</li>
|
||||
<li>
|
||||
<strong>The Soft Reset:</strong> After cancellation is confirmed by background threads, the UI resets via the <code>download_finished</code> function. Input fields (URL and Download Location) are preserved for convenience.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Restore State:</strong> Changes to "🗑️ Discard Session", which deletes <code>session.json</code> and resets the UI.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Update State:</strong> Changes to "🗑️ Clear Selection", unloading the selected creator profile and returning to normal UI state.
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
<h2><strong>Error Button</strong></h2>
|
||||
<ul>
|
||||
<li>
|
||||
<strong>Error Counter:</strong> Shows how many files failed to download (e.g., <code>(3) Error</code>). Disabled if there are no errors.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Error Dialog:</strong> Clicking opens the "Files Skipped Due to Errors" dialog (defined in <code>ErrorFilesDialog.py</code>), listing all failed files.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Dialog Features:</strong>
|
||||
<ul>
|
||||
<li><strong>View Failed Files:</strong> Shows filenames and related post info.</li>
|
||||
<li><strong>Select and Retry:</strong> Retry selected failed files in a focused download session.</li>
|
||||
<li><strong>Export URLs:</strong> Save a <code>.txt</code> file of direct download links. Optionally include post metadata with each URL.</li>
|
||||
<li><strong>Scope: Posts</strong>: Skips the entire post if the title matches.</li>
|
||||
<li><strong>Scope: Files</strong>: Skips individual files if the filename matches.</li>
|
||||
<li><strong>Scope: Both</strong>: Checks the post title first, then individual files.</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
<h2><strong>"Known Area" and its Controls</strong></h2>
|
||||
<p>This section, located on the right side of the main window, manages your personal name database (<code>Known.txt</code>), which the app uses to organize downloads into subfolders.</p>
|
||||
|
||||
<h3>Remove Words from Name (✂️)</h3>
|
||||
<p>Enter comma-separated words to remove from final filenames (e.g., <code>patreon, [HD]</code>). This helps clean up file naming.</p>
|
||||
<hr>
|
||||
<h2>3. Primary Download Modes (Filter File Section)</h2>
|
||||
<p>This section uses radio buttons to set the main download mode. Only one can be active at a time.</p>
|
||||
<ul>
|
||||
<li>
|
||||
<strong>Open Known.txt:</strong> Opens the <code>Known.txt</code> file in your system's default text editor for manual editing, such as bulk changes or cleanup.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Search character input:</strong> A live search filter that hides any list items not matching your input text. Useful for quickly locating specific names in large lists.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Known Series/Characters Area:</strong> Displays all names currently stored in your <code>Known.txt</code>. These names are used when "Separate folders by Known.txt" is enabled.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Input at bottom & Add button:</strong> Type a new character or series name into the input field, then click "➕ Add". The app checks for duplicates, updates the list, and saves to <code>Known.txt</code>.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Add to Filter:</strong> Opens a dialog showing all entries from <code>Known.txt</code> with checkboxes. You can select one or more to auto-fill the "Filter by Character(s)" field at the top of the app.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Delete Selected:</strong> Select one or more entries from the list and click "🗑️ Delete Selected" to remove them from the app and update <code>Known.txt</code> accordingly.
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
<h2><strong>Other Buttons</strong></h2>
|
||||
<ul>
|
||||
<li>
|
||||
<strong>(?_?) mark button (Help Guide):</strong> Opens a multi-page help dialog with step-by-step instructions and explanations for all app features. Useful for new users.
|
||||
</li>
|
||||
<li>
|
||||
<strong>History Button:</strong> Opens the Download History dialog (from <code>DownloadHistoryDialog.py</code>), showing:
|
||||
<li><strong>All</strong>: Default mode. Downloads every file and attachment.</li>
|
||||
<li><strong>Images/GIFs</strong>: Downloads only common image formats.</li>
|
||||
<li><strong>Videos</strong>: Downloads only common video formats.</li>
|
||||
<li><strong>Only Archives</strong>: Downloads only <code>.zip</code>, <code>.rar</code>, etc.</li>
|
||||
<li><strong>Only Audio</strong>: Downloads only common audio formats.</li>
|
||||
<li><strong>Only Links</strong>: Extracts external hyperlinks (e.g., Mega, Google Drive) from post descriptions instead of downloading files. <strong>This mode unlocks special features</strong> (see section 6).</li>
|
||||
<li><strong>More</strong>: Opens a dialog to download text-based content.
|
||||
<ul>
|
||||
<li>Recently downloaded files</li>
|
||||
<li>The first few posts processed in the last session</li>
|
||||
</ul>
|
||||
This allows for a quick review of recent activity.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Settings Button:</strong> Opens the Settings dialog (from <code>FutureSettingsDialog.py</code>), where you can change app-wide settings such as theme (light/dark) and language.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Support Button:</strong> Opens the Support dialog (from <code>SupportDialog.py</code>), which includes developer info, source links, and donation platforms like Ko-fi or Patreon.
|
||||
</li>
|
||||
</ul>
|
||||
<h2><strong>Log Area Controls</strong></h2>
|
||||
<p>These controls are located around the main log panel and offer tools for managing downloads, configuring advanced options, and resetting the application.</p>
|
||||
|
||||
<ul>
|
||||
<li>
|
||||
<strong>Multi-part: OFF</strong><br>
|
||||
This button acts as both a status indicator and a configuration panel for multi-part downloading (parallel downloading of large files).
|
||||
<ul>
|
||||
<li><strong>Function:</strong> Opens the <code>Multipart Download Options</code> dialog (defined in <code>MultipartScopeDialog.py</code>).</li>
|
||||
<li><strong>Scope Options:</strong> Choose between "Videos Only", "Archives Only", or "Both".</li>
|
||||
<li><strong>Number of parts:</strong> Set how many simultaneous connections to use (2–16).</li>
|
||||
<li><strong>Minimum file size:</strong> Set a threshold (MB) below which files are downloaded normally.</li>
|
||||
<li><strong>Status:</strong> After applying settings, the button's text updates (e.g., <code>Multi-part: Both</code>); otherwise, it resets to <code>Multi-part: OFF</code>.</li>
|
||||
</ul>
|
||||
</li>
|
||||
|
||||
<li>
|
||||
<strong>👁️ Eye Emoji Button (Log View Toggle)</strong><br>
|
||||
Switches between two views in the log panel:
|
||||
<ul>
|
||||
<li><strong>👁️ Progress Log View:</strong> Shows real-time download progress, status messages, and errors.</li>
|
||||
<li><strong>🚫 Missed Character View:</strong> Displays names detected in posts that didn’t match the current filter — useful for updating <code>Known.txt</code>.</li>
|
||||
</ul>
|
||||
</li>
|
||||
|
||||
<li>
|
||||
<strong>Reset Button</strong><br>
|
||||
Performs a full "soft reset" of the UI when the application is idle.
|
||||
<ul>
|
||||
<li>Clears all inputs (except saved Download Location)</li>
|
||||
<li>Resets checkboxes, buttons, and logs</li>
|
||||
<li>Clears counters, queues, and restores the UI to its default state</li>
|
||||
<li><strong>Note:</strong> This is different from <em>Cancel & Reset UI</em>, which halts active downloads</li>
|
||||
<li><strong>Scope</strong>: Choose to extract text from the post description or comments.</li>
|
||||
<li><strong>Export Format</strong>: Save as PDF, DOCX, or TXT.</li>
|
||||
<li><strong>Single PDF</strong>: Compile all text from the session into one consolidated PDF file.</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
<h3><strong>The Progress Log and "Only Links" Mode Controls</strong></h3>
|
||||
|
||||
<hr>
|
||||
<h2>4. Advanced Features & Toggles (Checkboxes)</h2>
|
||||
<h3>Folder Organization</h3>
|
||||
<ul>
|
||||
<li>
|
||||
<strong>Standard Mode (Progress Log)</strong><br>
|
||||
This is the default behavior. The <code>main_log_output</code> field displays:
|
||||
<li><strong>Separate folders by Known.txt</strong>: Automatically organizes downloads into subfolders based on name matches from your <code>Known.txt</code> list or the "Filter by Character(s)" input.</li>
|
||||
<li><strong>Subfolder per post</strong>: Creates a unique folder for each post, named after the post's title. This prevents files from different posts from mixing.</li>
|
||||
<li><strong>Date prefix</strong>: (Only available with "Subfolder per post") Prepends the post date to the folder name (e.g., <code>2025-08-03 My Post Title</code>) for chronological sorting.</li>
|
||||
</ul>
|
||||
<h3>Special Modes</h3>
|
||||
<ul>
|
||||
<li><strong>⭐ Favorite Mode</strong>: Switches the UI to download from your personal favorites list instead of using the URL input.</li>
|
||||
<li><strong>Manga/Comic mode</strong>: Sorts a creator's posts from oldest to newest before downloading, ensuring correct page order. A scope button appears to control the filename style (e.g., using post title, date, or a global number).</li>
|
||||
</ul>
|
||||
<h3>File Handling</h3>
|
||||
<ul>
|
||||
<li><strong>Skip Archives</strong>: Ignores <code>.zip</code> and <code>.rar</code> files during downloads.</li>
|
||||
<li><strong>Download Thumbnail Only</strong>: Saves only the small preview images instead of full-resolution files.</li>
|
||||
<li><strong>Scan Content for Images</strong>: Parses post HTML to find embedded images that may not be listed in the API data.</li>
|
||||
<li><strong>Compress to WebP</strong>: Converts large images (over 1.5 MB) to the space-saving WebP format.</li>
|
||||
<li><strong>Keep Duplicates</strong>: Opens a dialog to control how duplicate files are handled (skip by default, keep all, or keep a specific number of copies).</li>
|
||||
</ul>
|
||||
<h3>General Functionality</h3>
|
||||
<ul>
|
||||
<li><strong>Use cookie</strong>: Enables login-based access. You can paste a cookie string or browse for a <code>cookies.txt</code> file.</li>
|
||||
<li><strong>Use Multithreading</strong>: Enables parallel processing of posts for faster downloads. You can set the number of concurrent worker threads.</li>
|
||||
<li><strong>Show external links in log</strong>: Opens a secondary log panel that displays external links found in post descriptions.</li>
|
||||
</ul>
|
||||
<hr>
|
||||
<h2>5. Specialized Downloaders & Batch Mode</h2>
|
||||
<h3>Discord Features</h3>
|
||||
<ul>
|
||||
<li>When a Discord URL is entered, a <strong>Scope</strong> button appears.
|
||||
<ul>
|
||||
<li>Post processing steps</li>
|
||||
<li>Download/skipped file notifications</li>
|
||||
<li>Error messages</li>
|
||||
<li>Session summaries</li>
|
||||
<li><strong>Scope: Files</strong>: Downloads all files from the channel/server.</li>
|
||||
<li><strong>Scope: Messages</strong>: Saves the entire message history of the channel/server as a formatted PDF.</li>
|
||||
</ul>
|
||||
</li>
|
||||
|
||||
<li>
|
||||
<strong>"Only Links" Mode</strong><br>
|
||||
When enabled, the log panel switches modes and reveals new controls.
|
||||
<li>A <strong>"Save as PDF"</strong> button also appears as a shortcut for the message saving feature.</li>
|
||||
</ul>
|
||||
<h3>Batch Downloading (<code>nhentai</code> & <code>saint2.su</code>)</h3>
|
||||
<p>This feature allows you to download hundreds of galleries or videos from a simple text file.</p>
|
||||
<ol>
|
||||
<li>In the <code>appdata</code> folder, create <code>nhentai.txt</code> or <code>saint2.su.txt</code>.</li>
|
||||
<li>Add one full URL per line to the corresponding file.</li>
|
||||
<li>In the app's URL input, type either <code>nhentai.net</code> or <code>saint2.su</code> and click "Start Download".</li>
|
||||
<li>The app will read the file and process every URL in the queue.</li>
|
||||
</ol>
|
||||
<hr>
|
||||
<h2>6. "Only Links" Mode: Extraction & Direct Download</h2>
|
||||
<p>When you select the <strong>"Only Links"</strong> radio button, the application's behavior changes significantly.</p>
|
||||
<ul>
|
||||
<li><strong>Link Extraction</strong>: Instead of downloading files, the main log panel will fill with all external links found (Mega, Google Drive, Dropbox, etc.).</li>
|
||||
<li><strong>Export Links</strong>: An "Export Links" button appears, allowing you to save the full list of extracted URLs to a <code>.txt</code> file.</li>
|
||||
<li><strong>Direct Cloud Download</strong>: A <strong>"Download"</strong> button appears next to the export button.
|
||||
<ul>
|
||||
<li><strong>📜 Extracted Links Log:</strong> Replaces progress info with a list of found external links (e.g., Mega, Dropbox).</li>
|
||||
<li><strong>Export Links Button:</strong> Saves the extracted links to a <code>.txt</code> file.</li>
|
||||
<li><strong>Download Button:</strong> Opens the <code>Download Selected External Links</code> dialog (from <code>DownloadExtractedLinksDialog.py</code>), where you can:
|
||||
<ul>
|
||||
<li>View all supported external links</li>
|
||||
<li>Select which ones to download</li>
|
||||
<li>Begin download directly from cloud services</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Links View Button:</strong> Toggles log display between:
|
||||
<ul>
|
||||
<li><strong>🔗 Links View:</strong> Shows all extracted links</li>
|
||||
<li><strong>⬇️ Progress View:</strong> Shows download progress from external services (e.g., Mega)</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li>Clicking this opens a new dialog listing all supported cloud links (Mega, G-Drive, Dropbox).</li>
|
||||
<li>You can select which files you want to download from this list.</li>
|
||||
<li>The application will then download the selected files directly from the cloud service to your chosen download location.</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
<hr>
|
||||
<h2>7. Session & Process Management</h2>
|
||||
<h3>Main Action Buttons</h3>
|
||||
<ul>
|
||||
<li><strong>Start Download</strong>: Begins the download process. This button's text changes contextually (e.g., "Extract Links", "Check for Updates").</li>
|
||||
<li><strong>Pause / Resume</strong>: Pauses or resumes the ongoing download. When paused, you can safely change some settings.</li>
|
||||
<li><strong>Cancel & Reset UI</strong>: Stops the current download and performs a soft reset of the UI, preserving your URL and download location.</li>
|
||||
</ul>
|
||||
<h3>Restore Interrupted Download</h3>
|
||||
<p>If the application is closed unexpectedly during a download, it will save its progress.</p>
|
||||
<ul>
|
||||
<li>On the next launch, the UI will be pre-filled with the settings from the interrupted session.</li>
|
||||
<li>The <strong>Pause</strong> button will change to <strong>"🔄 Restore Download"</strong>. Clicking it will resume the download exactly where it left off, skipping already processed posts.</li>
|
||||
<li>The <strong>Cancel</strong> button will change to <strong>"🗑️ Discard Session"</strong>, allowing you to clear the saved state and start fresh.</li>
|
||||
</ul>
|
||||
<h3>Other UI Controls</h3>
|
||||
<ul>
|
||||
<li><strong>Error Button</strong>: Shows a count of failed files. Clicking it opens a dialog where you can view, export, or retry the failed downloads.</li>
|
||||
<li><strong>History Button</strong>: Shows a log of recently downloaded files and processed posts.</li>
|
||||
<li><strong>Settings Button</strong>: Opens the settings dialog where you can change the theme, language, and <strong>check for application updates</strong>.</li>
|
||||
<li><strong>Support Button</strong>: Opens a dialog with links to the project's source code and developer support pages.</li>
|
||||
</ul>
|
||||
|
||||
202
readme.md
202
readme.md
@@ -1,162 +1,116 @@
|
||||
<h1 align="center">Kemono Downloader </h1>
|
||||
<h1 align="center">Kemono Downloader</h1>
|
||||
|
||||
<div align="center">
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<td align="center">
|
||||
<img src="Read/Read.png" alt="Default Mode" width="400"><br>
|
||||
<strong>Default</strong>
|
||||
</td>
|
||||
<td align="center">
|
||||
<img src="Read/Read1.png" alt="Favorite Mode" width="400"><br>
|
||||
<strong>Favorite Mode</strong>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center">
|
||||
<img src="Read/Read2.png" alt="Single Post" width="400"><br>
|
||||
<strong>Single Post</strong>
|
||||
</td>
|
||||
<td align="center">
|
||||
<img src="Read/Read3.png" alt="Manga/Comic Mode" width="400"><br>
|
||||
<strong>Manga/Comic Mode</strong>
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<table>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td align="center">
|
||||
<img src="Read/Read.png" alt="Default Mode" width="400"><br>
|
||||
<strong>Default Mode</strong>
|
||||
</td>
|
||||
<td align="center">
|
||||
<img src="Read/Read1.png" alt="Favorite Mode" width="400"><br>
|
||||
<strong>Favorite Mode</strong>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center">
|
||||
<img src="Read/Read2.png" alt="Single Post" width="400"><br>
|
||||
<strong>Single Post</strong>
|
||||
</td>
|
||||
<td align="center">
|
||||
<img src="Read/Read3.png" alt="Manga/Comic Mode" width="400"><br>
|
||||
<strong>Manga/Comic Mode</strong>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
---
|
||||
<hr>
|
||||
|
||||
A powerful, feature-rich GUI application for downloading content from **[Kemono.su](https://kemono.su)** (and its mirrors like kemono.party) and **[Coomer.party](https://coomer.party)** (and its mirrors like coomer.su).
|
||||
|
||||
Built with PyQt5, this tool is designed for users who want deep filtering capabilities, customizable folder structures, efficient downloads, and intelligent automation — all within a modern and user-friendly graphical interface.
|
||||
<p>A powerful, feature-rich GUI application for downloading content from a wide array of sites, including <strong>Kemono</strong>, <strong>Coomer</strong>, <strong>Bunkr</strong>, <strong>Erome</strong>, <strong>Saint2.su</strong>, and <strong>nhentai</strong>.</p>
|
||||
<p>Built with PyQt5, this tool is designed for users who want deep filtering capabilities, customizable folder structures, efficient downloads, and intelligent automation — all within a modern and user-friendly graphical interface.</p>
|
||||
|
||||
<div align="center">
|
||||
|
||||
[](features.md)
|
||||
[](LICENSE)
|
||||
[](note.md)
|
||||
|
||||
<a href="features.md"><img src="https://img.shields.io/badge/📚%20Full%20Feature%20List-FFD700?style=for-the-badge&logoColor=black&color=FFD700" alt="Full Feature List"></a>
|
||||
<a href="LICENSE"><img src="https://img.shields.io/badge/📝%20License-90EE90?style=for-the-badge&logoColor=black&color=90EE90" alt="License"></a>
|
||||
<a href="note.md"><img src="https://img.shields.io/badge/⚠️%20Important%20Note-FFCCCB?style=for-the-badge&logoColor=black&color=FFCCCB" alt="Important Note"></a>
|
||||
</div>
|
||||
|
||||
<h2><strong>Core Capabilities Overview</strong></h2>
|
||||
|
||||
<h3><strong>High-Performance Downloading</strong></h3>
|
||||
<h2>Core Capabilities Overview</h2>
|
||||
<h3>High-Performance & Resilient Downloading</h3>
|
||||
<ul>
|
||||
<li><strong>Multi-threading:</strong> Processes multiple posts simultaneously to greatly accelerate downloads from large creator profiles.</li>
|
||||
<li><strong>Multi-part Downloading:</strong> Splits large files into chunks and downloads them in parallel to maximize speed.</li>
|
||||
<li><strong>Resilience:</strong> Supports pausing, resuming, and restoring downloads after crashes or interruptions.</li>
|
||||
<li><strong>Session Management:</strong> Supports pausing, resuming, and <strong>restoring downloads</strong> after crashes or interruptions, so you never lose your progress.</li>
|
||||
</ul>
|
||||
|
||||
<h3><strong>Advanced Filtering & Content Control</strong></h3>
|
||||
<h3>Expanded Site Support</h3>
|
||||
<ul>
|
||||
<li><strong>Direct Downloading:</strong> Full support for Kemono, Coomer, Bunkr, Erome, Saint2.su, and nhentai.</li>
|
||||
<li><strong>Batch Mode:</strong> Download hundreds of URLs at once from <code>nhentai.txt</code> or <code>saint2.su.txt</code> files.</li>
|
||||
<li><strong>Discord Support:</strong> Download files or save entire channel histories as PDFs directly through the API.</li>
|
||||
</ul>
|
||||
<h3>Advanced Filtering & Content Control</h3>
|
||||
<ul>
|
||||
<li><strong>Content Type Filtering:</strong> Select whether to download all files or limit to images, videos, audio, or archives only.</li>
|
||||
<li><strong>Keyword Skipping:</strong> Automatically skips posts or files containing certain keywords (e.g., "WIP", "sketch").</li>
|
||||
<li><strong>Character Filtering:</strong> Restricts downloads to posts that match specific character or series names.</li>
|
||||
<li><strong>Skip by Size:</strong> Avoid small files by setting a minimum size threshold in MB (e.g., <code>[200]</code>).</li>
|
||||
<li><strong>Character Filtering:</strong> Restricts downloads to posts that match specific character or series names, with scope controls for title, filename, or comments.</li>
|
||||
</ul>
|
||||
|
||||
<h3><strong>File Organization & Renaming</strong></h3>
|
||||
<h3>Intelligent File Organization</h3>
|
||||
<ul>
|
||||
<li><strong>Automated Subfolders:</strong> Automatically organizes downloaded files into subdirectories based on character names or per post.</li>
|
||||
<li><strong>Advanced File Renaming:</strong> Flexible renaming options, especially in Manga Mode, including:
|
||||
<ul>
|
||||
<li><strong>Post Title:</strong> Uses the post's title (e.g., <code>Chapter-One.jpg</code>).</li>
|
||||
<li><strong>Date + Original Name:</strong> Prepends the publication date to the original filename.</li>
|
||||
<li><strong>Date + Title:</strong> Combines the date with the post title.</li>
|
||||
<li><strong>Sequential Numbering (Date Based):</strong> Simple sequence numbers (e.g., <code>001.jpg</code>, <code>002.jpg</code>).</li>
|
||||
<li><strong>Title + Global Numbering:</strong> Uses post title with a globally incrementing number across the session.</li>
|
||||
<li><strong>Post ID:</strong> Names files using the post’s unique ID.</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Advanced File Renaming:</strong> Flexible renaming options, especially in Manga Mode, including by post title, date, sequential numbering, or post ID.</li>
|
||||
<li><strong>Filename Cleaning:</strong> Automatically removes unwanted text from filenames.</li>
|
||||
</ul>
|
||||
|
||||
<h3><strong>Specialized Modes</strong></h3>
|
||||
<h3>Specialized Modes</h3>
|
||||
<ul>
|
||||
<li><strong>Manga/Comic Mode:</strong> Sorts posts chronologically before downloading to ensure pages appear in the correct sequence.</li>
|
||||
<li><strong>Favorite Mode:</strong> Connects to your account and downloads from your favorites list (artists or posts).</li>
|
||||
<li><strong>Link Extraction Mode:</strong> Extracts external links from posts for export or targeted downloading.</li>
|
||||
<li><strong>Link Extraction Mode:</strong> Extracts external links (Mega, Google Drive) from posts for export or <strong>direct in-app downloading</strong>.</li>
|
||||
<li><strong>Text Extraction Mode:</strong> Saves post descriptions or comment sections as <code>PDF</code>, <code>DOCX</code>, or <code>TXT</code> files.</li>
|
||||
</ul>
|
||||
|
||||
<h3><strong>Utility & Advanced Features</strong></h3>
|
||||
<h3>Utility & Advanced Features</h3>
|
||||
<ul>
|
||||
<li><strong>In-App Updater:</strong> Check for new versions directly from the settings menu.</li>
|
||||
<li><strong>Cookie Support:</strong> Enables access to subscriber-only content via browser session cookies.</li>
|
||||
<li><strong>Duplicate Detection:</strong> Prevents saving duplicate files using content-based comparison, with configurable limits.</li>
|
||||
<li><strong>Image Compression:</strong> Automatically converts large images to <code>.webp</code> to reduce disk usage.</li>
|
||||
<li><strong>Creator Management:</strong> Built-in creator browser and update checker for downloading only new posts from saved profiles.</li>
|
||||
<li><strong>Error Handling:</strong> Tracks failed downloads and provides a retry dialog with options to export or redownload missing files.</li>
|
||||
</ul>
|
||||
|
||||
## 💻 Installation
|
||||
|
||||
### Requirements
|
||||
|
||||
- Python 3.6 or higher
|
||||
- pip (Python package installer)
|
||||
|
||||
### Install Dependencies
|
||||
|
||||
```bash
|
||||
pip install PyQt5 requests Pillow mega.py fpdf2 python-docx
|
||||
```
|
||||
|
||||
### Running the Application
|
||||
Navigate to the application's directory in your terminal and run:
|
||||
```bash
|
||||
python main.py
|
||||
```
|
||||
|
||||
### Optional Setup
|
||||
- **Main Inputs:**
|
||||
- Place your `cookies.txt` in the root directory (if using cookies).
|
||||
- Prepare your `Known.txt` and `creators.json` in the same directory for advanced filtering and selection features.
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### AttributeError: module 'asyncio' has no attribute 'coroutine'
|
||||
|
||||
If you encounter an error message similar to:
|
||||
```
|
||||
AttributeError: module 'asyncio' has no attribute 'coroutine'. Did you mean: 'coroutines'?
|
||||
```
|
||||
This usually means that a dependency, often `tenacity` (used by `mega.py`), is an older version that's incompatible with your Python version (typically Python 3.10+).
|
||||
|
||||
To fix this, activate your virtual environment and run the following commands to upgrade the libraries:
|
||||
|
||||
```bash
|
||||
pip install --upgrade tenacity
|
||||
pip install --upgrade mega.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Contribution
|
||||
|
||||
Feel free to fork this repo and submit pull requests for bug fixes, new features, or UI improvements!
|
||||
|
||||
---
|
||||
|
||||
## License
|
||||
|
||||
This project is under the MIT Licence
|
||||
|
||||
## Star History
|
||||
|
||||
<h2>💻 Installation</h2>
|
||||
<h3>Requirements</h3>
|
||||
<ul>
|
||||
<li>Python 3.6 or higher</li>
|
||||
<li>pip (Python package installer)</li>
|
||||
</ul>
|
||||
<h3>Install Dependencies</h3>
|
||||
<pre><code>pip install -r requirements.txt
|
||||
</code></pre>
|
||||
<h3>Running the Application</h3>
|
||||
<p>Navigate to the application's directory in your terminal and run:</p>
|
||||
<pre><code>python main.py
|
||||
</code></pre>
|
||||
<h2>Contribution</h2>
|
||||
<p>Feel free to fork this repo and submit pull requests for bug fixes, new features, or UI improvements!</p>
|
||||
<h2>License</h2>
|
||||
<p>This project is under the MIT Licence</p>
|
||||
<h2>Star History</h2>
|
||||
<table align="center" style="border-collapse: collapse; border: none; margin-left: auto; margin-right: auto;">
|
||||
<tr>
|
||||
<td align="center" valign="middle" style="padding: 10px; border: none;">
|
||||
<a href="https://www.star-history.com/#Yuvi9587/Kemono-Downloader&Date">
|
||||
<img src="https://api.star-history.com/svg?repos=Yuvi9587/Kemono-Downloader&type=Date" alt="Star History Chart" width="650">
|
||||
</a>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td align="center" valign="middle" style="padding: 10px; border: none;">
|
||||
<a href="https://www.star-history.com/#Yuvi9587/Kemono-Downloader&Date">
|
||||
<img src="https://api.star-history.com/svg?repos=Yuvi9587/Kemono-Downloader&type=Date" alt="Star History Chart" width="650">
|
||||
</a>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<p align="center">
|
||||
<a href="https://buymeacoffee.com/yuvi9587">
|
||||
<img src="https://img.shields.io/badge/🍺%20Buy%20Me%20a%20Coffee-FFCCCB?style=for-the-badge&logoColor=black&color=FFDD00" alt="Buy Me a Coffee">
|
||||
<img src="https://img.shields.io/badge/🍺%20Buy%20Me%20a%20Coffee-FFCCCB?style=for-the-badge&logoColor=black&color=FFDD00" alt="Buy Me a Coffee">
|
||||
</a>
|
||||
</p>
|
||||
|
||||
|
||||
@@ -3,6 +3,7 @@ import traceback
|
||||
from urllib.parse import urlparse
|
||||
import json
|
||||
import requests
|
||||
import cloudscraper
|
||||
from ..utils.network_utils import extract_post_info, prepare_cookies_for_request
|
||||
from ..config.constants import (
|
||||
STYLE_DATE_POST_TITLE
|
||||
@@ -80,26 +81,26 @@ def fetch_posts_paginated(api_url_base, headers, offset, logger, cancellation_ev
|
||||
|
||||
def fetch_single_post_data(api_domain, service, user_id, post_id, headers, logger, cookies_dict=None):
|
||||
"""
|
||||
--- NEW FUNCTION ---
|
||||
Fetches the full data, including the 'content' field, for a single post.
|
||||
--- MODIFIED FUNCTION ---
|
||||
Fetches the full data, including the 'content' field, for a single post using cloudscraper.
|
||||
"""
|
||||
post_api_url = f"https://{api_domain}/api/v1/{service}/user/{user_id}/post/{post_id}"
|
||||
logger(f" Fetching full content for post ID {post_id}...")
|
||||
try:
|
||||
with requests.get(post_api_url, headers=headers, timeout=(15, 300), cookies=cookies_dict, stream=True) as response:
|
||||
response.raise_for_status()
|
||||
response_body = b""
|
||||
for chunk in response.iter_content(chunk_size=8192):
|
||||
response_body += chunk
|
||||
|
||||
full_post_data = json.loads(response_body)
|
||||
|
||||
if isinstance(full_post_data, list) and full_post_data:
|
||||
return full_post_data[0]
|
||||
if isinstance(full_post_data, dict) and 'post' in full_post_data:
|
||||
return full_post_data['post']
|
||||
return full_post_data
|
||||
|
||||
scraper = cloudscraper.create_scraper()
|
||||
|
||||
try:
|
||||
response = scraper.get(post_api_url, headers=headers, timeout=(15, 300), cookies=cookies_dict)
|
||||
response.raise_for_status()
|
||||
|
||||
full_post_data = response.json()
|
||||
|
||||
if isinstance(full_post_data, list) and full_post_data:
|
||||
return full_post_data[0]
|
||||
if isinstance(full_post_data, dict) and 'post' in full_post_data:
|
||||
return full_post_data['post']
|
||||
return full_post_data
|
||||
|
||||
except Exception as e:
|
||||
logger(f" ❌ Failed to fetch full content for post {post_id}: {e}")
|
||||
return None
|
||||
@@ -138,8 +139,7 @@ def download_from_api(
|
||||
manga_filename_style_for_sort_check=None,
|
||||
processed_post_ids=None,
|
||||
fetch_all_first=False
|
||||
):
|
||||
# FIX: Define api_domain FIRST, before it is used in the headers
|
||||
):
|
||||
parsed_input_url_for_domain = urlparse(api_url_input)
|
||||
api_domain = parsed_input_url_for_domain.netloc
|
||||
|
||||
|
||||
241
src/core/bunkr_client.py
Normal file
241
src/core/bunkr_client.py
Normal file
@@ -0,0 +1,241 @@
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import requests
|
||||
import html
|
||||
import time
|
||||
import datetime
|
||||
import urllib.parse
|
||||
import json
|
||||
import random
|
||||
import binascii
|
||||
import itertools
|
||||
|
||||
class MockMessage:
|
||||
Directory = 1
|
||||
Url = 2
|
||||
Version = 3
|
||||
|
||||
class AlbumException(Exception): pass
|
||||
class ExtractionError(AlbumException): pass
|
||||
class HttpError(ExtractionError):
|
||||
def __init__(self, message="", response=None):
|
||||
self.response = response
|
||||
self.status = response.status_code if response is not None else 0
|
||||
super().__init__(message)
|
||||
class ControlException(AlbumException): pass
|
||||
class AbortExtraction(ExtractionError, ControlException): pass
|
||||
|
||||
try:
|
||||
re_compile = re._compiler.compile
|
||||
except AttributeError:
|
||||
re_compile = re.sre_compile.compile
|
||||
HTML_RE = re_compile(r"<[^>]+>")
|
||||
def extr(txt, begin, end, default=""):
|
||||
try:
|
||||
first = txt.index(begin) + len(begin)
|
||||
return txt[first:txt.index(end, first)]
|
||||
except Exception: return default
|
||||
def extract_iter(txt, begin, end, pos=None):
|
||||
try:
|
||||
index = txt.index
|
||||
lbeg = len(begin)
|
||||
lend = len(end)
|
||||
while True:
|
||||
first = index(begin, pos) + lbeg
|
||||
last = index(end, first)
|
||||
pos = last + lend
|
||||
yield txt[first:last]
|
||||
except Exception: return
|
||||
def split_html(txt):
|
||||
try: return [html.unescape(x).strip() for x in HTML_RE.split(txt) if x and not x.isspace()]
|
||||
except TypeError: return []
|
||||
def parse_datetime(date_string, format="%Y-%m-%dT%H:%M:%S%z", utcoffset=0):
|
||||
try:
|
||||
d = datetime.datetime.strptime(date_string, format)
|
||||
o = d.utcoffset()
|
||||
if o is not None: d = d.replace(tzinfo=None, microsecond=0) - o
|
||||
else:
|
||||
if d.microsecond: d = d.replace(microsecond=0)
|
||||
if utcoffset: d += datetime.timedelta(0, utcoffset * -3600)
|
||||
return d
|
||||
except (TypeError, IndexError, KeyError, ValueError, OverflowError): return None
|
||||
unquote = urllib.parse.unquote
|
||||
unescape = html.unescape
|
||||
|
||||
def decrypt_xor(encrypted, key, base64=True, fromhex=False):
|
||||
if base64: encrypted = binascii.a2b_base64(encrypted)
|
||||
if fromhex: encrypted = bytes.fromhex(encrypted.decode())
|
||||
div = len(key)
|
||||
return bytes([encrypted[i] ^ key[i % div] for i in range(len(encrypted))]).decode()
|
||||
def advance(iterable, num):
|
||||
iterator = iter(iterable)
|
||||
next(itertools.islice(iterator, num, num), None)
|
||||
return iterator
|
||||
def json_loads(s): return json.loads(s)
|
||||
def json_dumps(obj): return json.dumps(obj, separators=(",", ":"))
|
||||
|
||||
class Extractor:
|
||||
def __init__(self, match, logger):
|
||||
self.log = logger
|
||||
self.url = match.string
|
||||
self.match = match
|
||||
self.groups = match.groups()
|
||||
self.session = requests.Session()
|
||||
self.session.headers["User-Agent"] = "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Firefox/102.0"
|
||||
@classmethod
|
||||
def from_url(cls, url, logger):
|
||||
if isinstance(cls.pattern, str): cls.pattern = re.compile(cls.pattern)
|
||||
match = cls.pattern.match(url)
|
||||
return cls(match, logger) if match else None
|
||||
def __iter__(self): return self.items()
|
||||
def items(self): yield MockMessage.Version, 1
|
||||
def request(self, url, method="GET", fatal=True, **kwargs):
|
||||
tries = 1
|
||||
while True:
|
||||
try:
|
||||
response = self.session.request(method, url, **kwargs)
|
||||
if response.status_code < 400: return response
|
||||
msg = f"'{response.status_code} {response.reason}' for '{response.url}'"
|
||||
except requests.exceptions.RequestException as exc:
|
||||
msg = str(exc)
|
||||
|
||||
self.log.info("%s (retrying...)", msg)
|
||||
if tries > 4: break
|
||||
time.sleep(tries)
|
||||
tries += 1
|
||||
if not fatal: return None
|
||||
raise HttpError(msg)
|
||||
def request_json(self, url, **kwargs):
|
||||
response = self.request(url, **kwargs)
|
||||
try: return json_loads(response.text)
|
||||
except Exception as exc:
|
||||
self.log.warning("%s: %s", exc.__class__.__name__, exc)
|
||||
if not kwargs.get("fatal", True): return {}
|
||||
raise
|
||||
|
||||
BASE_PATTERN_BUNKR = r"(?:https?://)?(?:[a-zA-Z0-9-]+\.)?(bunkr\.(?:si|la|ws|red|black|media|site|is|to|ac|cr|ci|fi|pk|ps|sk|ph|su)|bunkrr\.ru)"
|
||||
DOMAINS = ["bunkr.si", "bunkr.ws", "bunkr.la", "bunkr.red", "bunkr.black", "bunkr.media", "bunkr.site"]
|
||||
CF_DOMAINS = set()
|
||||
|
||||
class BunkrAlbumExtractor(Extractor):
|
||||
category = "bunkr"
|
||||
root = "https://bunkr.si"
|
||||
root_dl = "https://get.bunkrr.su"
|
||||
root_api = "https://apidl.bunkr.ru"
|
||||
pattern = re.compile(BASE_PATTERN_BUNKR + r"/a/([^/?#]+)")
|
||||
|
||||
def __init__(self, match, logger):
|
||||
super().__init__(match, logger)
|
||||
domain_match = re.search(BASE_PATTERN_BUNKR, match.string)
|
||||
if domain_match:
|
||||
self.root = "https://" + domain_match.group(1)
|
||||
self.endpoint = self.root_api + "/api/_001_v2"
|
||||
self.album_id = self.groups[-1]
|
||||
|
||||
def items(self):
|
||||
page = self.request(self.url).text
|
||||
title = unescape(unescape(extr(page, 'property="og:title" content="', '"')))
|
||||
items_html = list(extract_iter(page, '<div class="grid-images_box', "</a>"))
|
||||
|
||||
album_data = {
|
||||
"album_id": self.album_id,
|
||||
"album_name": title,
|
||||
"count": len(items_html),
|
||||
}
|
||||
yield MockMessage.Directory, album_data, {}
|
||||
|
||||
for item_html in items_html:
|
||||
try:
|
||||
webpage_url = unescape(extr(item_html, ' href="', '"'))
|
||||
if webpage_url.startswith("/"):
|
||||
webpage_url = self.root + webpage_url
|
||||
|
||||
file_data = self._extract_file(webpage_url)
|
||||
info = split_html(item_html)
|
||||
|
||||
if not file_data.get("name"):
|
||||
file_data["name"] = info[-3]
|
||||
|
||||
yield MockMessage.Url, file_data, {}
|
||||
except Exception as exc:
|
||||
self.log.error("%s: %s", exc.__class__.__name__, exc)
|
||||
|
||||
def _extract_file(self, webpage_url):
|
||||
page = self.request(webpage_url).text
|
||||
data_id = extr(page, 'data-file-id="', '"')
|
||||
referer = self.root_dl + "/file/" + data_id
|
||||
headers = {"Referer": referer, "Origin": self.root_dl}
|
||||
data = self.request_json(self.endpoint, method="POST", headers=headers, json={"id": data_id})
|
||||
|
||||
file_url = decrypt_xor(data["url"], f"SECRET_KEY_{data['timestamp'] // 3600}".encode()) if data.get("encrypted") else data["url"]
|
||||
file_name = extr(page, "<h1", "<").rpartition(">")[2]
|
||||
|
||||
return {
|
||||
"url": file_url,
|
||||
"name": unescape(file_name),
|
||||
"_http_headers": {"Referer": referer}
|
||||
}
|
||||
|
||||
class BunkrMediaExtractor(BunkrAlbumExtractor):
|
||||
pattern = re.compile(BASE_PATTERN_BUNKR + r"(/[fvid]/[^/?#]+)")
|
||||
def items(self):
|
||||
try:
|
||||
media_path = self.groups[-1]
|
||||
file_data = self._extract_file(self.root + media_path)
|
||||
album_data = {"album_name": file_data.get("name", "bunkr_media"), "count": 1}
|
||||
|
||||
yield MockMessage.Directory, album_data, {}
|
||||
yield MockMessage.Url, file_data, {}
|
||||
|
||||
except Exception as exc:
|
||||
self.log.error("%s: %s", exc.__class__.__name__, exc)
|
||||
yield MockMessage.Directory, {"album_name": "error", "count": 0}, {}
|
||||
|
||||
def get_bunkr_extractor(url, logger):
|
||||
"""Selects the correct Bunkr extractor based on the URL pattern."""
|
||||
if BunkrAlbumExtractor.pattern.match(url):
|
||||
logger.info("Bunkr Album URL detected.")
|
||||
return BunkrAlbumExtractor.from_url(url, logger)
|
||||
elif BunkrMediaExtractor.pattern.match(url):
|
||||
logger.info("Bunkr Media URL detected.")
|
||||
return BunkrMediaExtractor.from_url(url, logger)
|
||||
else:
|
||||
logger.error(f"No suitable Bunkr extractor found for URL: {url}")
|
||||
return None
|
||||
|
||||
def fetch_bunkr_data(url, logger):
|
||||
"""
|
||||
Main function to be called from the GUI.
|
||||
It extracts all file information from a Bunkr URL.
|
||||
|
||||
Returns:
|
||||
A tuple of (album_name, list_of_files)
|
||||
- album_name (str): The name of the album.
|
||||
- list_of_files (list): A list of dicts, each containing 'url', 'name', and '_http_headers'.
|
||||
Returns (None, None) on failure.
|
||||
"""
|
||||
extractor = get_bunkr_extractor(url, logger)
|
||||
if not extractor:
|
||||
return None, None
|
||||
|
||||
try:
|
||||
album_name = "default_bunkr_album"
|
||||
files_to_download = []
|
||||
for msg_type, data, metadata in extractor:
|
||||
if msg_type == MockMessage.Directory:
|
||||
raw_album_name = data.get('album_name', 'untitled')
|
||||
album_name = re.sub(r'[<>:"/\\|?*]', '_', raw_album_name).strip() or "untitled"
|
||||
logger.info(f"Processing Bunkr album: {album_name}")
|
||||
elif msg_type == MockMessage.Url:
|
||||
files_to_download.append(data)
|
||||
|
||||
if not files_to_download:
|
||||
logger.warning("No files found to download from the Bunkr URL.")
|
||||
return None, None
|
||||
|
||||
return album_name, files_to_download
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"An error occurred while extracting Bunkr info: {e}", exc_info=True)
|
||||
return None, None
|
||||
@@ -1,63 +1,70 @@
|
||||
import time
|
||||
import requests
|
||||
import cloudscraper
|
||||
import json
|
||||
from urllib.parse import urlparse
|
||||
|
||||
def fetch_server_channels(server_id, logger, cookies=None, cancellation_event=None, pause_event=None):
|
||||
def fetch_server_channels(server_id, logger=print, cookies_dict=None):
|
||||
"""
|
||||
Fetches the list of channels for a given Discord server ID from the Kemono API.
|
||||
UPDATED to be pausable and cancellable.
|
||||
Fetches all channels for a given Discord server ID from the API.
|
||||
Uses cloudscraper to bypass Cloudflare.
|
||||
"""
|
||||
domains_to_try = ["kemono.cr", "kemono.su"]
|
||||
for domain in domains_to_try:
|
||||
if cancellation_event and cancellation_event.is_set():
|
||||
logger(" Channel fetching cancelled by user.")
|
||||
return None
|
||||
while pause_event and pause_event.is_set():
|
||||
if cancellation_event and cancellation_event.is_set(): break
|
||||
time.sleep(0.5)
|
||||
api_url = f"https://kemono.cr/api/v1/discord/server/{server_id}"
|
||||
logger(f" Fetching channels for server: {api_url}")
|
||||
|
||||
lookup_url = f"https://{domain}/api/v1/discord/channel/lookup/{server_id}"
|
||||
logger(f" Attempting to fetch channel list from: {lookup_url}")
|
||||
try:
|
||||
response = requests.get(lookup_url, cookies=cookies, timeout=15)
|
||||
response.raise_for_status()
|
||||
channels = response.json()
|
||||
if isinstance(channels, list):
|
||||
logger(f" ✅ Found {len(channels)} channels for server {server_id}.")
|
||||
return channels
|
||||
except (requests.exceptions.RequestException, json.JSONDecodeError):
|
||||
# This is a silent failure, we'll just try the next domain
|
||||
pass
|
||||
|
||||
logger(f" ❌ Failed to fetch channel list for server {server_id} from all available domains.")
|
||||
return None
|
||||
scraper = cloudscraper.create_scraper()
|
||||
headers = {
|
||||
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36',
|
||||
'Referer': f'https://kemono.cr/discord/server/{server_id}',
|
||||
'Accept': 'text/css'
|
||||
}
|
||||
|
||||
def fetch_channel_messages(channel_id, logger, cancellation_event, pause_event, cookies=None):
|
||||
try:
|
||||
response = scraper.get(api_url, headers=headers, cookies=cookies_dict, timeout=30)
|
||||
response.raise_for_status()
|
||||
channels = response.json()
|
||||
if isinstance(channels, list):
|
||||
logger(f" ✅ Found {len(channels)} channels for server {server_id}.")
|
||||
return channels
|
||||
return None
|
||||
except Exception as e:
|
||||
logger(f" ❌ Error fetching server channels for {server_id}: {e}")
|
||||
return None
|
||||
|
||||
def fetch_channel_messages(channel_id, logger=print, cancellation_event=None, pause_event=None, cookies_dict=None):
|
||||
"""
|
||||
Fetches all messages from a Discord channel by looping through API pages (pagination).
|
||||
Uses a page size of 150 and handles the specific offset logic.
|
||||
A generator that fetches all messages for a specific Discord channel, handling pagination.
|
||||
Uses cloudscraper and proper headers to bypass server protection.
|
||||
"""
|
||||
offset = 0
|
||||
page_size = 150 # Corrected page size based on your findings
|
||||
api_base_url = f"https://kemono.cr/api/v1/discord/channel/{channel_id}"
|
||||
scraper = cloudscraper.create_scraper()
|
||||
base_url = f"https://kemono.cr/api/v1/discord/channel/{channel_id}"
|
||||
headers = {
|
||||
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36',
|
||||
'Referer': f'https://kemono.cr/discord/channel/{channel_id}',
|
||||
'Accept': 'text/css'
|
||||
}
|
||||
|
||||
while not (cancellation_event and cancellation_event.is_set()):
|
||||
if pause_event and pause_event.is_set():
|
||||
logger(" Message fetching paused...")
|
||||
while pause_event.is_set():
|
||||
if cancellation_event and cancellation_event.is_set(): break
|
||||
time.sleep(0.5)
|
||||
logger(" Message fetching resumed.")
|
||||
offset = 0
|
||||
# --- FIX: Corrected the page size for Discord API pagination ---
|
||||
page_size = 150
|
||||
# --- END FIX ---
|
||||
|
||||
while True:
|
||||
if cancellation_event and cancellation_event.is_set():
|
||||
logger(" Discord message fetching cancelled.")
|
||||
break
|
||||
|
||||
paginated_url = f"{api_base_url}?o={offset}"
|
||||
if pause_event and pause_event.is_set():
|
||||
logger(" Discord message fetching paused...")
|
||||
while pause_event.is_set():
|
||||
if cancellation_event and cancellation_event.is_set():
|
||||
break
|
||||
time.sleep(0.5)
|
||||
if not (cancellation_event and cancellation_event.is_set()):
|
||||
logger(" Discord message fetching resumed.")
|
||||
|
||||
paginated_url = f"{base_url}?o={offset}"
|
||||
logger(f" Fetching messages from API: page starting at offset {offset}")
|
||||
|
||||
try:
|
||||
response = requests.get(paginated_url, cookies=cookies, timeout=20)
|
||||
response = scraper.get(paginated_url, headers=headers, cookies=cookies_dict, timeout=30)
|
||||
response.raise_for_status()
|
||||
messages_batch = response.json()
|
||||
|
||||
@@ -73,8 +80,11 @@ def fetch_channel_messages(channel_id, logger, cancellation_event, pause_event,
|
||||
break
|
||||
|
||||
offset += page_size
|
||||
time.sleep(0.5)
|
||||
time.sleep(0.5) # Be respectful to the API
|
||||
|
||||
except (requests.exceptions.RequestException, json.JSONDecodeError) as e:
|
||||
except (cloudscraper.exceptions.CloudflareException, json.JSONDecodeError) as e:
|
||||
logger(f" ❌ Error fetching messages at offset {offset}: {e}")
|
||||
break
|
||||
break
|
||||
except Exception as e:
|
||||
logger(f" ❌ An unexpected error occurred while fetching messages: {e}")
|
||||
break
|
||||
|
||||
131
src/core/erome_client.py
Normal file
131
src/core/erome_client.py
Normal file
@@ -0,0 +1,131 @@
|
||||
|
||||
import os
|
||||
import re
|
||||
import html
|
||||
import time
|
||||
import urllib.parse
|
||||
import requests
|
||||
from datetime import datetime
|
||||
import cloudscraper
|
||||
|
||||
|
||||
def extr(txt, begin, end, default=""):
|
||||
"""Stripped-down version of 'extract()' to find text between two delimiters."""
|
||||
try:
|
||||
first = txt.index(begin) + len(begin)
|
||||
return txt[first:txt.index(end, first)]
|
||||
except (ValueError, IndexError):
|
||||
return default
|
||||
|
||||
def extract_iter(txt, begin, end):
|
||||
"""Yields all occurrences of text between two delimiters."""
|
||||
try:
|
||||
index = txt.index
|
||||
lbeg = len(begin)
|
||||
lend = len(end)
|
||||
pos = 0
|
||||
while True:
|
||||
first = index(begin, pos) + lbeg
|
||||
last = index(end, first)
|
||||
pos = last + lend
|
||||
yield txt[first:last]
|
||||
except (ValueError, IndexError):
|
||||
return
|
||||
|
||||
def nameext_from_url(url):
|
||||
"""Extracts filename and extension from a URL."""
|
||||
data = {}
|
||||
filename = urllib.parse.unquote(url.partition("?")[0].rpartition("/")[2])
|
||||
name, _, ext = filename.rpartition(".")
|
||||
if name and len(ext) <= 16:
|
||||
data["filename"], data["extension"] = name, ext.lower()
|
||||
else:
|
||||
data["filename"], data["extension"] = filename, ""
|
||||
return data
|
||||
|
||||
def parse_timestamp(ts, default=None):
|
||||
"""Creates a datetime object from a Unix timestamp."""
|
||||
try:
|
||||
return datetime.fromtimestamp(int(ts))
|
||||
except (ValueError, TypeError):
|
||||
return default
|
||||
|
||||
|
||||
def fetch_erome_data(url, logger):
|
||||
"""
|
||||
Identifies and extracts all media files from an Erome album URL.
|
||||
|
||||
Args:
|
||||
url (str): The Erome album URL (e.g., https://www.erome.com/a/albumID).
|
||||
logger (function): A function to log progress messages.
|
||||
|
||||
Returns:
|
||||
tuple: A tuple containing (album_folder_name, list_of_file_dicts).
|
||||
Returns (None, []) if data extraction fails.
|
||||
"""
|
||||
album_id_match = re.search(r"/a/(\w+)", url)
|
||||
if not album_id_match:
|
||||
logger(f"Error: The URL '{url}' does not appear to be a valid Erome album link.")
|
||||
return None, []
|
||||
|
||||
album_id = album_id_match.group(1)
|
||||
page_url = f"https://www.erome.com/a/{album_id}"
|
||||
|
||||
session = cloudscraper.create_scraper()
|
||||
|
||||
try:
|
||||
logger(f" Fetching Erome album page: {page_url}")
|
||||
for attempt in range(5):
|
||||
response = session.get(page_url, timeout=30)
|
||||
response.raise_for_status()
|
||||
page_content = response.text
|
||||
if "<title>Please wait a few moments</title>" in page_content:
|
||||
logger(f" Cloudflare check detected. Waiting 5 seconds... (Attempt {attempt + 1}/5)")
|
||||
time.sleep(5)
|
||||
continue
|
||||
break
|
||||
else:
|
||||
logger(" Error: Could not bypass Cloudflare check after several attempts.")
|
||||
return None, []
|
||||
|
||||
title = html.unescape(extr(page_content, 'property="og:title" content="', '"'))
|
||||
user = urllib.parse.unquote(extr(page_content, 'href="https://www.erome.com/', '"', default="unknown_user"))
|
||||
|
||||
sanitized_title = re.sub(r'[<>:"/\\|?*]', '_', title).strip()
|
||||
sanitized_user = re.sub(r'[<>:"/\\|?*]', '_', user).strip()
|
||||
|
||||
album_folder_name = f"Erome - {sanitized_user} - {sanitized_title} [{album_id}]"
|
||||
|
||||
urls = []
|
||||
media_groups = page_content.split('<div class="media-group"')
|
||||
for group in media_groups[1:]:
|
||||
video_url = extr(group, '<source src="', '"') or extr(group, 'data-src="', '"')
|
||||
if video_url:
|
||||
urls.append(video_url)
|
||||
|
||||
if not urls:
|
||||
logger(" Warning: No media URLs found on the album page.")
|
||||
return album_folder_name, []
|
||||
|
||||
logger(f" Found {len(urls)} media files in album '{title}'.")
|
||||
|
||||
file_list = []
|
||||
for i, file_url in enumerate(urls, 1):
|
||||
filename_info = nameext_from_url(file_url)
|
||||
filename = f"{album_id}_{sanitized_title}_{i:03d}.{filename_info.get('extension', 'mp4')}"
|
||||
|
||||
file_data = {
|
||||
"url": file_url,
|
||||
"filename": filename,
|
||||
"headers": {"Referer": page_url},
|
||||
}
|
||||
file_list.append(file_data)
|
||||
|
||||
return album_folder_name, file_list
|
||||
|
||||
except requests.exceptions.RequestException as e:
|
||||
logger(f" Error fetching Erome page: {e}")
|
||||
return None, []
|
||||
except Exception as e:
|
||||
logger(f" An unexpected error occurred during Erome extraction: {e}")
|
||||
return None, []
|
||||
44
src/core/nhentai_client.py
Normal file
44
src/core/nhentai_client.py
Normal file
@@ -0,0 +1,44 @@
|
||||
import requests
|
||||
import cloudscraper
|
||||
import json
|
||||
|
||||
def fetch_nhentai_gallery(gallery_id, logger=print):
|
||||
"""
|
||||
Fetches the metadata for a single nhentai gallery using cloudscraper to bypass Cloudflare.
|
||||
|
||||
Args:
|
||||
gallery_id (str or int): The ID of the nhentai gallery.
|
||||
logger (function): A function to log progress and error messages.
|
||||
|
||||
Returns:
|
||||
dict: A dictionary containing the gallery's metadata if successful, otherwise None.
|
||||
"""
|
||||
api_url = f"https://nhentai.net/api/gallery/{gallery_id}"
|
||||
|
||||
scraper = cloudscraper.create_scraper()
|
||||
|
||||
logger(f" Fetching nhentai gallery metadata from: {api_url}")
|
||||
|
||||
try:
|
||||
# Use the scraper to make the GET request
|
||||
response = scraper.get(api_url, timeout=20)
|
||||
|
||||
if response.status_code == 404:
|
||||
logger(f" ❌ Gallery not found (404): ID {gallery_id}")
|
||||
return None
|
||||
|
||||
response.raise_for_status()
|
||||
|
||||
gallery_data = response.json()
|
||||
|
||||
if "id" in gallery_data and "media_id" in gallery_data and "images" in gallery_data:
|
||||
logger(f" ✅ Successfully fetched metadata for '{gallery_data['title']['english']}'")
|
||||
gallery_data['pages'] = gallery_data.pop('images')['pages']
|
||||
return gallery_data
|
||||
else:
|
||||
logger(" ❌ API response is missing essential keys (id, media_id, or images).")
|
||||
return None
|
||||
|
||||
except Exception as e:
|
||||
logger(f" ❌ An error occurred while fetching gallery {gallery_id}: {e}")
|
||||
return None
|
||||
163
src/core/saint2_client.py
Normal file
163
src/core/saint2_client.py
Normal file
@@ -0,0 +1,163 @@
|
||||
import os
|
||||
import re as re_module
|
||||
import html
|
||||
import urllib.parse
|
||||
import requests
|
||||
|
||||
|
||||
PATTERN_CACHE = {}
|
||||
|
||||
def re(pattern):
|
||||
"""Compile a regular expression pattern and cache it."""
|
||||
try:
|
||||
return PATTERN_CACHE[pattern]
|
||||
except KeyError:
|
||||
p = PATTERN_CACHE[pattern] = re_module.compile(pattern)
|
||||
return p
|
||||
|
||||
def extract_from(txt, pos=None, default=""):
|
||||
"""Returns a function that extracts text between two delimiters from 'txt'."""
|
||||
def extr(begin, end, index=txt.find, txt=txt):
|
||||
nonlocal pos
|
||||
try:
|
||||
start_pos = pos if pos is not None else 0
|
||||
first = index(begin, start_pos) + len(begin)
|
||||
last = index(end, first)
|
||||
if pos is not None:
|
||||
pos = last + len(end)
|
||||
return txt[first:last]
|
||||
except (ValueError, IndexError):
|
||||
return default
|
||||
return extr
|
||||
|
||||
def nameext_from_url(url):
|
||||
"""Extract filename and extension from a URL."""
|
||||
data = {}
|
||||
filename = urllib.parse.unquote(url.partition("?")[0].rpartition("/")[2])
|
||||
name, _, ext = filename.rpartition(".")
|
||||
if name and len(ext) <= 16:
|
||||
data["filename"], data["extension"] = name, ext.lower()
|
||||
else:
|
||||
data["filename"], data["extension"] = filename, ""
|
||||
return data
|
||||
|
||||
class BaseExtractor:
|
||||
"""A simplified base class for extractors."""
|
||||
def __init__(self, match, session, logger):
|
||||
self.match = match
|
||||
self.groups = match.groups()
|
||||
self.session = session
|
||||
self.log = logger
|
||||
|
||||
def request(self, url, **kwargs):
|
||||
"""Makes an HTTP request using the session."""
|
||||
try:
|
||||
response = self.session.get(url, **kwargs)
|
||||
response.raise_for_status()
|
||||
return response
|
||||
except requests.exceptions.RequestException as e:
|
||||
self.log(f"Error making request to {url}: {e}")
|
||||
return None
|
||||
|
||||
class SaintAlbumExtractor(BaseExtractor):
|
||||
"""Extractor for saint.su albums."""
|
||||
root = "https://saint2.su"
|
||||
pattern = re(r"(?:https?://)?saint\d*\.(?:su|pk|cr|to)/a/([^/?#]+)")
|
||||
|
||||
def items(self):
|
||||
"""Generator that yields all files from an album."""
|
||||
album_id = self.groups[0]
|
||||
response = self.request(f"{self.root}/a/{album_id}")
|
||||
if not response:
|
||||
return None, []
|
||||
|
||||
extr = extract_from(response.text)
|
||||
title = extr("<title>", "<").rpartition(" - ")[0]
|
||||
self.log(f"Downloading album: {title}")
|
||||
|
||||
files_html = re_module.findall(r'<a class="image".*?</a>', response.text, re_module.DOTALL)
|
||||
file_list = []
|
||||
for i, file_html in enumerate(files_html, 1):
|
||||
file_extr = extract_from(file_html)
|
||||
file_url = html.unescape(file_extr("onclick=\"play('", "'"))
|
||||
if not file_url:
|
||||
continue
|
||||
|
||||
filename_info = nameext_from_url(file_url)
|
||||
filename = f"{filename_info['filename']}.{filename_info['extension']}"
|
||||
|
||||
file_data = {
|
||||
"url": file_url,
|
||||
"filename": filename,
|
||||
"headers": {"Referer": response.url},
|
||||
}
|
||||
file_list.append(file_data)
|
||||
|
||||
return title, file_list
|
||||
|
||||
class SaintMediaExtractor(BaseExtractor):
|
||||
"""Extractor for single saint.su media links."""
|
||||
root = "https://saint2.su"
|
||||
pattern = re(r"(?:https?://)?saint\d*\.(?:su|pk|cr|to)(/(embe)?d/([^/?#]+))")
|
||||
|
||||
def items(self):
|
||||
"""Generator that yields the single file from a media page."""
|
||||
path, embed, media_id = self.groups
|
||||
url = self.root + path
|
||||
response = self.request(url)
|
||||
if not response:
|
||||
return None, []
|
||||
|
||||
extr = extract_from(response.text)
|
||||
file_url = ""
|
||||
title = extr("<title>", "<").rpartition(" - ")[0] or media_id
|
||||
|
||||
if embed: # /embed/ link
|
||||
file_url = html.unescape(extr('<source src="', '"'))
|
||||
else: # /d/ link
|
||||
file_url = html.unescape(extr('<a href="', '"'))
|
||||
|
||||
if not file_url:
|
||||
self.log("Could not find video URL on the page.")
|
||||
return title, []
|
||||
|
||||
filename_info = nameext_from_url(file_url)
|
||||
filename = f"{filename_info['filename'] or media_id}.{filename_info['extension'] or 'mp4'}"
|
||||
|
||||
file_data = {
|
||||
"url": file_url,
|
||||
"filename": filename,
|
||||
"headers": {"Referer": response.url}
|
||||
}
|
||||
|
||||
return title, [file_data]
|
||||
|
||||
|
||||
def fetch_saint2_data(url, logger):
|
||||
"""
|
||||
Identifies the correct extractor for a saint2.su URL and returns the data.
|
||||
|
||||
Args:
|
||||
url (str): The saint2.su URL.
|
||||
logger (function): A function to log progress messages.
|
||||
|
||||
Returns:
|
||||
tuple: A tuple containing (album_title, list_of_file_dicts).
|
||||
Returns (None, []) if no data could be fetched.
|
||||
"""
|
||||
extractors = [SaintMediaExtractor, SaintAlbumExtractor]
|
||||
session = requests.Session()
|
||||
session.headers.update({
|
||||
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36'
|
||||
})
|
||||
|
||||
for extractor_cls in extractors:
|
||||
match = extractor_cls.pattern.match(url)
|
||||
if match:
|
||||
extractor = extractor_cls(match, session, logger)
|
||||
album_title, files = extractor.items()
|
||||
sanitized_title = re_module.sub(r'[<>:"/\\|?*]', '_', album_title) if album_title else "saint2_download"
|
||||
return sanitized_title, files
|
||||
|
||||
logger(f"Error: The URL '{url}' does not match a known saint2 pattern.")
|
||||
return None, []
|
||||
@@ -15,6 +15,8 @@ from concurrent.futures import ThreadPoolExecutor, as_completed, CancelledError,
|
||||
from io import BytesIO
|
||||
from urllib .parse import urlparse
|
||||
import requests
|
||||
import cloudscraper
|
||||
|
||||
try:
|
||||
from PIL import Image
|
||||
except ImportError:
|
||||
@@ -58,18 +60,13 @@ def robust_clean_name(name):
|
||||
"""A more robust function to remove illegal characters for filenames and folders."""
|
||||
if not name:
|
||||
return ""
|
||||
# Removes illegal characters for Windows, macOS, and Linux: < > : " / \ | ? *
|
||||
# Also removes control characters (ASCII 0-31) which are invisible but invalid.
|
||||
illegal_chars_pattern = r'[\x00-\x1f<>:"/\\|?*]'
|
||||
illegal_chars_pattern = r'[\x00-\x1f<>:"/\\|?*\']'
|
||||
cleaned_name = re.sub(illegal_chars_pattern, '', name)
|
||||
|
||||
# Remove leading/trailing spaces or periods, which can cause issues.
|
||||
|
||||
cleaned_name = cleaned_name.strip(' .')
|
||||
|
||||
# If the name is empty after cleaning (e.g., it was only illegal chars),
|
||||
# provide a safe fallback name.
|
||||
|
||||
if not cleaned_name:
|
||||
return "untitled_folder" # Or "untitled_file" depending on context
|
||||
return "untitled_folder"
|
||||
return cleaned_name
|
||||
|
||||
class PostProcessorSignals (QObject ):
|
||||
@@ -271,7 +268,9 @@ class PostProcessorWorker:
|
||||
|
||||
file_download_headers = {
|
||||
'User-Agent': 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)',
|
||||
'Referer': post_page_url
|
||||
'Referer': post_page_url,
|
||||
'Accept': 'text/css'
|
||||
|
||||
}
|
||||
|
||||
file_url = file_info.get('url')
|
||||
@@ -429,8 +428,26 @@ class PostProcessorWorker:
|
||||
self.logger(f"⚠️ Manga mode: Generated filename was empty. Using generic fallback: '{filename_to_save_in_main_path}'.")
|
||||
was_original_name_kept_flag = False
|
||||
else:
|
||||
filename_to_save_in_main_path = cleaned_original_api_filename
|
||||
was_original_name_kept_flag = True
|
||||
is_url_like = 'http' in api_original_filename.lower()
|
||||
is_too_long = len(cleaned_original_api_filename) > 100
|
||||
|
||||
if is_url_like or is_too_long:
|
||||
self.logger(f" ⚠️ Original filename is a URL or too long. Generating a shorter name.")
|
||||
name_hash = hashlib.md5(api_original_filename.encode()).hexdigest()[:12]
|
||||
_, ext = os.path.splitext(cleaned_original_api_filename)
|
||||
if not ext:
|
||||
try:
|
||||
path = urlparse(api_original_filename).path
|
||||
ext = os.path.splitext(path)[1] or ".file"
|
||||
except Exception:
|
||||
ext = ".file"
|
||||
|
||||
cleaned_post_title = robust_clean_name(post_title.strip() if post_title else "post")[:40]
|
||||
filename_to_save_in_main_path = f"{cleaned_post_title}_{name_hash}{ext}"
|
||||
was_original_name_kept_flag = False
|
||||
else:
|
||||
filename_to_save_in_main_path = cleaned_original_api_filename
|
||||
was_original_name_kept_flag = True
|
||||
|
||||
if self.remove_from_filename_words_list and filename_to_save_in_main_path:
|
||||
base_name_for_removal, ext_for_removal = os.path.splitext(filename_to_save_in_main_path)
|
||||
@@ -854,9 +871,7 @@ class PostProcessorWorker:
|
||||
return 0, 1, filename_to_save_in_main_path, was_original_name_kept_flag, FILE_DOWNLOAD_STATUS_FAILED_RETRYABLE_LATER, details_for_failure
|
||||
|
||||
def process(self):
|
||||
# --- START: REFACTORED PROCESS METHOD ---
|
||||
|
||||
# 1. DATA MAPPING: Map Discord Message or Creator Post fields to a consistent set of variables.
|
||||
if self.service == 'discord':
|
||||
# For Discord, self.post is a MESSAGE object from the API.
|
||||
post_title = self.post.get('content', '') or f"Message {self.post.get('id', 'N/A')}"
|
||||
@@ -885,19 +900,26 @@ class PostProcessorWorker:
|
||||
)
|
||||
|
||||
if content_is_needed and self.post.get('content') is None and self.service != 'discord':
|
||||
|
||||
self.logger(f" Post {post_id} is missing 'content' field, fetching full data...")
|
||||
parsed_url = urlparse(self.api_url_input)
|
||||
api_domain = parsed_url.netloc
|
||||
headers = {'User-Agent': 'Mozilla/5.0'}
|
||||
creator_page_url = f"https://{api_domain}/{self.service}/user/{self.user_id}"
|
||||
headers = {
|
||||
'User-Agent': 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)',
|
||||
'Referer': creator_page_url,
|
||||
'Accept': 'text/css'
|
||||
|
||||
}
|
||||
|
||||
|
||||
cookies = prepare_cookies_for_request(self.use_cookie, self.cookie_text, self.selected_cookie_file, self.app_base_dir, self.logger, target_domain=api_domain)
|
||||
|
||||
full_post_data = fetch_single_post_data(api_domain, self.service, self.user_id, post_id, headers, self.logger, cookies_dict=cookies)
|
||||
|
||||
if full_post_data:
|
||||
self.logger(" ✅ Full post data fetched successfully.")
|
||||
# Update the worker's post object with the complete data
|
||||
self.post = full_post_data
|
||||
# Re-initialize local variables from the new, complete post data
|
||||
post_title = self.post.get('title', '') or 'untitled_post'
|
||||
post_main_file_info = self.post.get('file')
|
||||
post_attachments = self.post.get('attachments', [])
|
||||
@@ -905,9 +927,7 @@ class PostProcessorWorker:
|
||||
post_data = self.post
|
||||
else:
|
||||
self.logger(f" ⚠️ Failed to fetch full content for post {post_id}. Content-dependent features may not work for this post.")
|
||||
# --- END FIX ---
|
||||
|
||||
# 2. SHARED PROCESSING LOGIC: The rest of the function now uses the consistent variables from above.
|
||||
result_tuple = (0, 0, [], [], [], None, None)
|
||||
total_downloaded_this_post = 0
|
||||
total_skipped_this_post = 0
|
||||
@@ -936,7 +956,11 @@ class PostProcessorWorker:
|
||||
else:
|
||||
post_page_url = f"https://{parsed_api_url.netloc}/{self.service}/user/{self.user_id}/post/{post_id}"
|
||||
|
||||
headers = {'User-Agent': 'Mozilla/5.0', 'Referer': post_page_url, 'Accept': '*/*'}
|
||||
headers = {
|
||||
'User-Agent': 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)',
|
||||
'Referer': post_page_url,
|
||||
'Accept': 'text/css'
|
||||
}
|
||||
link_pattern = re.compile(r"""<a\s+.*?href=["'](https?://[^"']+)["'][^>]*>(.*?)</a>""", re.IGNORECASE | re.DOTALL)
|
||||
|
||||
effective_unwanted_keywords_for_folder_naming = self.unwanted_keywords.copy()
|
||||
|
||||
@@ -5,9 +5,12 @@ import traceback
|
||||
import json
|
||||
import base64
|
||||
import time
|
||||
import zipfile
|
||||
from urllib.parse import urlparse, urlunparse, parse_qs, urlencode
|
||||
|
||||
# --- Third-party Library Imports ---
|
||||
import requests
|
||||
import cloudscraper
|
||||
|
||||
try:
|
||||
from Crypto.Cipher import AES
|
||||
@@ -26,12 +29,12 @@ MEGA_API_URL = "https://g.api.mega.co.nz"
|
||||
def _get_filename_from_headers(headers):
|
||||
"""
|
||||
Extracts a filename from the Content-Disposition header.
|
||||
(This is from your original file and is kept for Dropbox downloads)
|
||||
"""
|
||||
cd = headers.get('content-disposition')
|
||||
if not cd:
|
||||
return None
|
||||
|
||||
# Handles both filename="file.zip" and filename*=UTF-8''file%20name.zip
|
||||
fname_match = re.findall('filename="?([^"]+)"?', cd)
|
||||
if fname_match:
|
||||
sanitized_name = re.sub(r'[<>:"/\\|?*]', '_', fname_match[0].strip())
|
||||
@@ -39,28 +42,23 @@ def _get_filename_from_headers(headers):
|
||||
|
||||
return None
|
||||
|
||||
# --- NEW: Helper functions for Mega decryption ---
|
||||
# --- Helper functions for Mega decryption ---
|
||||
|
||||
def urlb64_to_b64(s):
|
||||
"""Converts a URL-safe base64 string to a standard base64 string."""
|
||||
s = s.replace('-', '+').replace('_', '/')
|
||||
s += '=' * (-len(s) % 4)
|
||||
return s
|
||||
|
||||
def b64_to_bytes(s):
|
||||
"""Decodes a URL-safe base64 string to bytes."""
|
||||
return base64.b64decode(urlb64_to_b64(s))
|
||||
|
||||
def bytes_to_hex(b):
|
||||
"""Converts bytes to a hex string."""
|
||||
return b.hex()
|
||||
|
||||
def hex_to_bytes(h):
|
||||
"""Converts a hex string to bytes."""
|
||||
return bytes.fromhex(h)
|
||||
|
||||
def hrk2hk(hex_raw_key):
|
||||
"""Derives the final AES key from the raw key components for Mega."""
|
||||
key_part1 = int(hex_raw_key[0:16], 16)
|
||||
key_part2 = int(hex_raw_key[16:32], 16)
|
||||
key_part3 = int(hex_raw_key[32:48], 16)
|
||||
@@ -72,23 +70,20 @@ def hrk2hk(hex_raw_key):
|
||||
return f'{final_key_part1:016x}{final_key_part2:016x}'
|
||||
|
||||
def decrypt_at(at_b64, key_bytes):
|
||||
"""Decrypts the 'at' attribute to get file metadata."""
|
||||
at_bytes = b64_to_bytes(at_b64)
|
||||
iv = b'\0' * 16
|
||||
cipher = AES.new(key_bytes, AES.MODE_CBC, iv)
|
||||
decrypted_at = cipher.decrypt(at_bytes)
|
||||
return decrypted_at.decode('utf-8').strip('\0').replace('MEGA', '')
|
||||
|
||||
# --- NEW: Core Logic for Mega Downloads ---
|
||||
# --- Core Logic for Mega Downloads ---
|
||||
|
||||
def get_mega_file_info(file_id, file_key, session, logger_func):
|
||||
"""Fetches file metadata and the temporary download URL from the Mega API."""
|
||||
try:
|
||||
hex_raw_key = bytes_to_hex(b64_to_bytes(file_key))
|
||||
hex_key = hrk2hk(hex_raw_key)
|
||||
key_bytes = hex_to_bytes(hex_key)
|
||||
|
||||
# Request file attributes
|
||||
payload = [{"a": "g", "p": file_id}]
|
||||
response = session.post(f"{MEGA_API_URL}/cs", json=payload, timeout=20)
|
||||
response.raise_for_status()
|
||||
@@ -100,13 +95,10 @@ def get_mega_file_info(file_id, file_key, session, logger_func):
|
||||
|
||||
file_size = res_json[0]['s']
|
||||
at_b64 = res_json[0]['at']
|
||||
|
||||
# Decrypt attributes to get the file name
|
||||
at_dec_json_str = decrypt_at(at_b64, key_bytes)
|
||||
at_dec_json = json.loads(at_dec_json_str)
|
||||
file_name = at_dec_json['n']
|
||||
|
||||
# Request the temporary download URL
|
||||
payload = [{"a": "g", "g": 1, "p": file_id}]
|
||||
response = session.post(f"{MEGA_API_URL}/cs", json=payload, timeout=20)
|
||||
response.raise_for_status()
|
||||
@@ -124,19 +116,16 @@ def get_mega_file_info(file_id, file_key, session, logger_func):
|
||||
return None
|
||||
|
||||
def download_and_decrypt_mega_file(info, download_path, logger_func):
|
||||
"""Downloads the file and decrypts it chunk by chunk, reporting progress."""
|
||||
file_name = info['file_name']
|
||||
file_size = info['file_size']
|
||||
dl_url = info['dl_url']
|
||||
hex_raw_key = info['hex_raw_key']
|
||||
|
||||
final_path = os.path.join(download_path, file_name)
|
||||
|
||||
if os.path.exists(final_path) and os.path.getsize(final_path) == file_size:
|
||||
logger_func(f" [Mega] ℹ️ File '{file_name}' already exists with the correct size. Skipping.")
|
||||
return
|
||||
|
||||
# Prepare for decryption
|
||||
key = hex_to_bytes(hrk2hk(hex_raw_key))
|
||||
iv_hex = hex_raw_key[32:48] + '0000000000000000'
|
||||
iv_bytes = hex_to_bytes(iv_hex)
|
||||
@@ -150,13 +139,11 @@ def download_and_decrypt_mega_file(info, download_path, logger_func):
|
||||
|
||||
with open(final_path, 'wb') as f:
|
||||
for chunk in r.iter_content(chunk_size=8192):
|
||||
if not chunk:
|
||||
continue
|
||||
if not chunk: continue
|
||||
decrypted_chunk = cipher.decrypt(chunk)
|
||||
f.write(decrypted_chunk)
|
||||
downloaded_bytes += len(chunk)
|
||||
|
||||
# Log progress every second
|
||||
current_time = time.time()
|
||||
if current_time - last_log_time > 1:
|
||||
progress_percent = (downloaded_bytes / file_size) * 100 if file_size > 0 else 0
|
||||
@@ -164,28 +151,16 @@ def download_and_decrypt_mega_file(info, download_path, logger_func):
|
||||
last_log_time = current_time
|
||||
|
||||
logger_func(f" [Mega] ✅ Successfully downloaded '{file_name}' to '{download_path}'")
|
||||
except requests.RequestException as e:
|
||||
logger_func(f" [Mega] ❌ Download failed for '{file_name}': {e}")
|
||||
except IOError as e:
|
||||
logger_func(f" [Mega] ❌ Could not write to file '{final_path}': {e}")
|
||||
except Exception as e:
|
||||
logger_func(f" [Mega] ❌ An unexpected error occurred during download/decryption: {e}")
|
||||
|
||||
|
||||
# --- REPLACEMENT Main Service Downloader Function for Mega ---
|
||||
|
||||
def download_mega_file(mega_url, download_path, logger_func=print):
|
||||
"""
|
||||
Downloads a file from a Mega.nz URL using direct requests and decryption.
|
||||
This replaces the old mega.py implementation.
|
||||
"""
|
||||
if not PYCRYPTODOME_AVAILABLE:
|
||||
logger_func("❌ Mega download failed: 'pycryptodome' library is not installed. Please run: pip install pycryptodome")
|
||||
return
|
||||
|
||||
logger_func(f" [Mega] Initializing download for: {mega_url}")
|
||||
|
||||
# Regex to capture file ID and key from both old and new URL formats
|
||||
match = re.search(r'mega(?:\.co)?\.nz/(?:file/|#!)?([a-zA-Z0-9]+)(?:#|!)([a-zA-Z0-9_.-]+)', mega_url)
|
||||
if not match:
|
||||
logger_func(f" [Mega] ❌ Error: Invalid Mega URL format.")
|
||||
@@ -199,18 +174,14 @@ def download_mega_file(mega_url, download_path, logger_func=print):
|
||||
|
||||
file_info = get_mega_file_info(file_id, file_key, session, logger_func)
|
||||
if not file_info:
|
||||
logger_func(f" [Mega] ❌ Failed to get file info. The link may be invalid or expired. Aborting.")
|
||||
logger_func(f" [Mega] ❌ Failed to get file info. Aborting.")
|
||||
return
|
||||
|
||||
logger_func(f" [Mega] File found: '{file_info['file_name']}' (Size: {file_info['file_size'] / 1024 / 1024:.2f} MB)")
|
||||
|
||||
download_and_decrypt_mega_file(file_info, download_path, logger_func)
|
||||
|
||||
|
||||
# --- ORIGINAL Functions for Google Drive and Dropbox (Unchanged) ---
|
||||
|
||||
def download_gdrive_file(url, download_path, logger_func=print):
|
||||
"""Downloads a file from a Google Drive link."""
|
||||
if not GDRIVE_AVAILABLE:
|
||||
logger_func("❌ Google Drive download failed: 'gdown' library is not installed.")
|
||||
return
|
||||
@@ -227,12 +198,15 @@ def download_gdrive_file(url, download_path, logger_func=print):
|
||||
except Exception as e:
|
||||
logger_func(f" [G-Drive] ❌ An unexpected error occurred: {e}")
|
||||
|
||||
# --- MODIFIED DROPBOX DOWNLOADER ---
|
||||
def download_dropbox_file(dropbox_link, download_path=".", logger_func=print):
|
||||
"""
|
||||
Downloads a file from a public Dropbox link by modifying the URL for direct download.
|
||||
Downloads a file or a folder (as a zip) from a public Dropbox link.
|
||||
Uses cloudscraper to handle potential browser checks and auto-extracts zip files.
|
||||
"""
|
||||
logger_func(f" [Dropbox] Attempting to download: {dropbox_link}")
|
||||
|
||||
# Modify URL to force download (works for both files and folders)
|
||||
parsed_url = urlparse(dropbox_link)
|
||||
query_params = parse_qs(parsed_url.query)
|
||||
query_params['dl'] = ['1']
|
||||
@@ -241,26 +215,60 @@ def download_dropbox_file(dropbox_link, download_path=".", logger_func=print):
|
||||
|
||||
logger_func(f" [Dropbox] Using direct download URL: {direct_download_url}")
|
||||
|
||||
scraper = cloudscraper.create_scraper()
|
||||
|
||||
try:
|
||||
if not os.path.exists(download_path):
|
||||
os.makedirs(download_path, exist_ok=True)
|
||||
logger_func(f" [Dropbox] Created download directory: {download_path}")
|
||||
|
||||
with requests.get(direct_download_url, stream=True, allow_redirects=True, timeout=(10, 300)) as r:
|
||||
with scraper.get(direct_download_url, stream=True, allow_redirects=True, timeout=(20, 600)) as r:
|
||||
r.raise_for_status()
|
||||
|
||||
filename = _get_filename_from_headers(r.headers) or os.path.basename(parsed_url.path) or "dropbox_file"
|
||||
filename = _get_filename_from_headers(r.headers) or os.path.basename(parsed_url.path) or "dropbox_download"
|
||||
# If it's a folder, Dropbox will name it FolderName.zip
|
||||
if not os.path.splitext(filename)[1]:
|
||||
filename += ".zip"
|
||||
|
||||
full_save_path = os.path.join(download_path, filename)
|
||||
|
||||
logger_func(f" [Dropbox] Starting download of '{filename}'...")
|
||||
|
||||
total_size = int(r.headers.get('content-length', 0))
|
||||
downloaded_bytes = 0
|
||||
last_log_time = time.time()
|
||||
|
||||
with open(full_save_path, 'wb') as f:
|
||||
for chunk in r.iter_content(chunk_size=8192):
|
||||
f.write(chunk)
|
||||
downloaded_bytes += len(chunk)
|
||||
current_time = time.time()
|
||||
if total_size > 0 and current_time - last_log_time > 1:
|
||||
progress = (downloaded_bytes / total_size) * 100
|
||||
logger_func(f" -> Downloading '{filename}'... {downloaded_bytes/1024/1024:.2f}MB / {total_size/1024/1024:.2f}MB ({progress:.1f}%)")
|
||||
last_log_time = current_time
|
||||
|
||||
logger_func(f" [Dropbox] ✅ Dropbox file downloaded successfully: {full_save_path}")
|
||||
logger_func(f" [Dropbox] ✅ Download complete: {full_save_path}")
|
||||
|
||||
# --- NEW: Auto-extraction logic ---
|
||||
if zipfile.is_zipfile(full_save_path):
|
||||
logger_func(f" [Dropbox] ዚ Detected zip file. Attempting to extract...")
|
||||
extract_folder_name = os.path.splitext(filename)[0]
|
||||
extract_path = os.path.join(download_path, extract_folder_name)
|
||||
os.makedirs(extract_path, exist_ok=True)
|
||||
|
||||
with zipfile.ZipFile(full_save_path, 'r') as zip_ref:
|
||||
zip_ref.extractall(extract_path)
|
||||
|
||||
logger_func(f" [Dropbox] ✅ Successfully extracted to folder: '{extract_path}'")
|
||||
|
||||
# Optional: remove the zip file after extraction
|
||||
try:
|
||||
os.remove(full_save_path)
|
||||
logger_func(f" [Dropbox] 🗑️ Removed original zip file.")
|
||||
except OSError as e:
|
||||
logger_func(f" [Dropbox] ⚠️ Could not remove original zip file: {e}")
|
||||
|
||||
except Exception as e:
|
||||
logger_func(f" [Dropbox] ❌ An error occurred during Dropbox download: {e}")
|
||||
traceback.print_exc(limit=2)
|
||||
raise
|
||||
|
||||
125
src/services/updater.py
Normal file
125
src/services/updater.py
Normal file
@@ -0,0 +1,125 @@
|
||||
import sys
|
||||
import os
|
||||
import requests
|
||||
import subprocess # Keep this for now, though it's not used in the final command
|
||||
from packaging.version import parse as parse_version
|
||||
from PyQt5.QtCore import QThread, pyqtSignal
|
||||
|
||||
# Constants for the updater
|
||||
GITHUB_REPO_URL = "https://api.github.com/repos/Yuvi9587/Kemono-Downloader/releases/latest"
|
||||
EXE_NAME = "Kemono.Downloader.exe"
|
||||
|
||||
class UpdateChecker(QThread):
|
||||
"""Checks for a new version on GitHub in a background thread."""
|
||||
update_available = pyqtSignal(str, str) # new_version, download_url
|
||||
up_to_date = pyqtSignal(str)
|
||||
update_error = pyqtSignal(str)
|
||||
|
||||
def __init__(self, current_version):
|
||||
super().__init__()
|
||||
self.current_version_str = current_version.lstrip('v')
|
||||
|
||||
def run(self):
|
||||
try:
|
||||
response = requests.get(GITHUB_REPO_URL, timeout=15)
|
||||
response.raise_for_status()
|
||||
data = response.json()
|
||||
|
||||
latest_version_str = data['tag_name'].lstrip('v')
|
||||
current_version = parse_version(self.current_version_str)
|
||||
latest_version = parse_version(latest_version_str)
|
||||
|
||||
if latest_version > current_version:
|
||||
for asset in data.get('assets', []):
|
||||
if asset['name'] == EXE_NAME:
|
||||
self.update_available.emit(latest_version_str, asset['browser_download_url'])
|
||||
return
|
||||
self.update_error.emit(f"Update found, but '{EXE_NAME}' is missing from the release assets.")
|
||||
else:
|
||||
self.up_to_date.emit("You are on the latest version.")
|
||||
|
||||
except requests.exceptions.RequestException as e:
|
||||
self.update_error.emit(f"Network error: {e}")
|
||||
except Exception as e:
|
||||
self.update_error.emit(f"An error occurred: {e}")
|
||||
|
||||
|
||||
class UpdateDownloader(QThread):
|
||||
"""
|
||||
Downloads the new executable and runs an updater script that kills the old process,
|
||||
replaces the file, and displays a message in the terminal.
|
||||
"""
|
||||
download_finished = pyqtSignal()
|
||||
download_error = pyqtSignal(str)
|
||||
|
||||
def __init__(self, download_url, parent_app):
|
||||
super().__init__()
|
||||
self.download_url = download_url
|
||||
self.parent_app = parent_app
|
||||
|
||||
def run(self):
|
||||
try:
|
||||
app_path = sys.executable
|
||||
app_dir = os.path.dirname(app_path)
|
||||
temp_path = os.path.join(app_dir, f"{EXE_NAME}.tmp")
|
||||
old_path = os.path.join(app_dir, f"{EXE_NAME}.old")
|
||||
updater_script_path = os.path.join(app_dir, "updater.bat")
|
||||
|
||||
# --- NEW: Path for the PID file ---
|
||||
pid_file_path = os.path.join(app_dir, "updater.pid")
|
||||
|
||||
# Download the new executable
|
||||
with requests.get(self.download_url, stream=True, timeout=300) as r:
|
||||
r.raise_for_status()
|
||||
with open(temp_path, 'wb') as f:
|
||||
for chunk in r.iter_content(chunk_size=8192):
|
||||
f.write(chunk)
|
||||
|
||||
# --- NEW: Write the current Process ID to the pid file ---
|
||||
with open(pid_file_path, "w") as f:
|
||||
f.write(str(os.getpid()))
|
||||
|
||||
# --- NEW BATCH SCRIPT ---
|
||||
# This script now reads the PID from the "updater.pid" file.
|
||||
script_content = f"""
|
||||
@echo off
|
||||
SETLOCAL
|
||||
|
||||
echo.
|
||||
echo Reading process information...
|
||||
set /p PID=<{pid_file_path}
|
||||
|
||||
echo Closing the old application (PID: %PID%)...
|
||||
taskkill /F /PID %PID%
|
||||
|
||||
echo Waiting for files to unlock...
|
||||
timeout /t 2 /nobreak > nul
|
||||
|
||||
echo Replacing application files...
|
||||
if exist "{old_path}" del /F /Q "{old_path}"
|
||||
rename "{app_path}" "{os.path.basename(old_path)}"
|
||||
rename "{temp_path}" "{EXE_NAME}"
|
||||
|
||||
echo.
|
||||
echo ============================================================
|
||||
echo Update Complete!
|
||||
echo You can now close this window and run {EXE_NAME}.
|
||||
echo ============================================================
|
||||
echo.
|
||||
pause
|
||||
|
||||
echo Cleaning up helper files...
|
||||
del "{pid_file_path}"
|
||||
del "%~f0"
|
||||
ENDLOCAL
|
||||
"""
|
||||
with open(updater_script_path, "w") as f:
|
||||
f.write(script_content)
|
||||
|
||||
# --- Go back to the os.startfile command that we know works ---
|
||||
os.startfile(updater_script_path)
|
||||
|
||||
self.download_finished.emit()
|
||||
|
||||
except Exception as e:
|
||||
self.download_error.emit(f"Failed to download or run updater: {e}")
|
||||
@@ -3,7 +3,7 @@ import html
|
||||
import re
|
||||
|
||||
# --- Third-Party Library Imports ---
|
||||
import requests
|
||||
import cloudscraper # MODIFIED: Import cloudscraper
|
||||
from PyQt5.QtCore import QCoreApplication, Qt
|
||||
from PyQt5.QtWidgets import (
|
||||
QApplication, QDialog, QHBoxLayout, QLabel, QLineEdit, QListWidget,
|
||||
@@ -12,7 +12,6 @@ from PyQt5.QtWidgets import (
|
||||
|
||||
# --- Local Application Imports ---
|
||||
from ...i18n.translator import get_translation
|
||||
# Corrected Import: Get the icon from the new assets utility module
|
||||
from ..assets import get_app_icon_object
|
||||
from ...utils.network_utils import prepare_cookies_for_request
|
||||
from .CookieHelpDialog import CookieHelpDialog
|
||||
@@ -41,9 +40,9 @@ class FavoriteArtistsDialog (QDialog ):
|
||||
service_lower = service_name.lower()
|
||||
coomer_primary_services = {'onlyfans', 'fansly', 'manyvids', 'candfans'}
|
||||
if service_lower in coomer_primary_services:
|
||||
return "coomer.st" # Use the new domain
|
||||
return "coomer.st"
|
||||
else:
|
||||
return "kemono.cr" # Use the new domain
|
||||
return "kemono.cr"
|
||||
|
||||
def _tr (self ,key ,default_text =""):
|
||||
"""Helper to get translation based on current app language."""
|
||||
@@ -126,9 +125,11 @@ class FavoriteArtistsDialog (QDialog ):
|
||||
self .artist_list_widget .setVisible (show )
|
||||
|
||||
def _fetch_favorite_artists (self ):
|
||||
# --- FIX: Use cloudscraper and add proper headers ---
|
||||
scraper = cloudscraper.create_scraper()
|
||||
# --- END FIX ---
|
||||
|
||||
if self.cookies_config['use_cookie']:
|
||||
# --- Kemono Check with Fallback ---
|
||||
kemono_cookies = prepare_cookies_for_request(
|
||||
True, self.cookies_config['cookie_text'], self.cookies_config['selected_cookie_file'],
|
||||
self.cookies_config['app_base_dir'], self._logger, target_domain="kemono.cr"
|
||||
@@ -140,7 +141,6 @@ class FavoriteArtistsDialog (QDialog ):
|
||||
self.cookies_config['app_base_dir'], self._logger, target_domain="kemono.su"
|
||||
)
|
||||
|
||||
# --- Coomer Check with Fallback ---
|
||||
coomer_cookies = prepare_cookies_for_request(
|
||||
True, self.cookies_config['cookie_text'], self.cookies_config['selected_cookie_file'],
|
||||
self.cookies_config['app_base_dir'], self._logger, target_domain="coomer.st"
|
||||
@@ -153,28 +153,21 @@ class FavoriteArtistsDialog (QDialog ):
|
||||
)
|
||||
|
||||
if not kemono_cookies and not coomer_cookies:
|
||||
# If cookies are enabled but none could be loaded, show help and stop.
|
||||
self.status_label.setText(self._tr("fav_artists_cookies_required_status", "Error: Cookies enabled but could not be loaded for any source."))
|
||||
self._logger("Error: Cookies enabled but no valid cookies were loaded. Showing help dialog.")
|
||||
cookie_help_dialog = CookieHelpDialog(self.parent_app, self)
|
||||
cookie_help_dialog.exec_()
|
||||
self.download_button.setEnabled(False)
|
||||
return # Stop further execution
|
||||
|
||||
kemono_fav_url ="https://kemono.su/api/v1/account/favorites?type=artist"
|
||||
coomer_fav_url ="https://coomer.su/api/v1/account/favorites?type=artist"
|
||||
return
|
||||
|
||||
self .all_fetched_artists =[]
|
||||
fetched_any_successfully =False
|
||||
errors_occurred =[]
|
||||
any_cookies_loaded_successfully_for_any_source =False
|
||||
|
||||
kemono_cr_fav_url = "https://kemono.cr/api/v1/account/favorites?type=artist"
|
||||
coomer_st_fav_url = "https://coomer.st/api/v1/account/favorites?type=artist"
|
||||
|
||||
api_sources = [
|
||||
{"name": "Kemono.cr", "url": kemono_cr_fav_url, "domain": "kemono.cr"},
|
||||
{"name": "Coomer.st", "url": coomer_st_fav_url, "domain": "coomer.st"}
|
||||
{"name": "Kemono.cr", "url": "https://kemono.cr/api/v1/account/favorites?type=artist", "domain": "kemono.cr"},
|
||||
{"name": "Coomer.st", "url": "https://coomer.st/api/v1/account/favorites?type=artist", "domain": "coomer.st"}
|
||||
]
|
||||
|
||||
for source in api_sources :
|
||||
@@ -185,41 +178,36 @@ class FavoriteArtistsDialog (QDialog ):
|
||||
cookies_dict_for_source = None
|
||||
if self.cookies_config['use_cookie']:
|
||||
primary_domain = source['domain']
|
||||
fallback_domain = None
|
||||
if primary_domain == "kemono.cr":
|
||||
fallback_domain = "kemono.su"
|
||||
elif primary_domain == "coomer.st":
|
||||
fallback_domain = "coomer.su"
|
||||
fallback_domain = "kemono.su" if "kemono" in primary_domain else "coomer.su"
|
||||
|
||||
# First, try the primary domain
|
||||
cookies_dict_for_source = prepare_cookies_for_request(
|
||||
True,
|
||||
self.cookies_config['cookie_text'],
|
||||
self.cookies_config['selected_cookie_file'],
|
||||
self.cookies_config['app_base_dir'],
|
||||
self._logger,
|
||||
target_domain=primary_domain
|
||||
True, self.cookies_config['cookie_text'], self.cookies_config['selected_cookie_file'],
|
||||
self.cookies_config['app_base_dir'], self._logger, target_domain=primary_domain
|
||||
)
|
||||
|
||||
# If no cookies found, try the fallback domain
|
||||
if not cookies_dict_for_source and fallback_domain:
|
||||
self._logger(f"Warning ({source['name']}): No cookies found for '{primary_domain}'. Trying fallback '{fallback_domain}'...")
|
||||
if not cookies_dict_for_source:
|
||||
self._logger(f"Warning ({source['name']}): No cookies for '{primary_domain}'. Trying fallback '{fallback_domain}'...")
|
||||
cookies_dict_for_source = prepare_cookies_for_request(
|
||||
True,
|
||||
self.cookies_config['cookie_text'],
|
||||
self.cookies_config['selected_cookie_file'],
|
||||
self.cookies_config['app_base_dir'],
|
||||
self._logger,
|
||||
target_domain=fallback_domain
|
||||
True, self.cookies_config['cookie_text'], self.cookies_config['selected_cookie_file'],
|
||||
self.cookies_config['app_base_dir'], self._logger, target_domain=fallback_domain
|
||||
)
|
||||
|
||||
if cookies_dict_for_source:
|
||||
any_cookies_loaded_successfully_for_any_source = True
|
||||
else:
|
||||
self._logger(f"Warning ({source['name']}): Cookies enabled but could not be loaded for this source (including fallbacks). Fetch might fail.")
|
||||
self._logger(f"Warning ({source['name']}): Cookies enabled but not loaded for this source. Fetch may fail.")
|
||||
try :
|
||||
headers ={'User-Agent':'Mozilla/5.0'}
|
||||
response =requests .get (source ['url'],headers =headers ,cookies =cookies_dict_for_source ,timeout =20 )
|
||||
# --- FIX: Add Referer and Accept headers ---
|
||||
headers = {
|
||||
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36',
|
||||
'Referer': f"https://{source['domain']}/favorites",
|
||||
'Accept': 'text/css'
|
||||
}
|
||||
# --- END FIX ---
|
||||
|
||||
# --- FIX: Use scraper instead of requests ---
|
||||
response = scraper.get(source['url'], headers=headers, cookies=cookies_dict_for_source, timeout=20)
|
||||
# --- END FIX ---
|
||||
|
||||
response .raise_for_status ()
|
||||
artists_data_from_api =response .json ()
|
||||
|
||||
@@ -254,15 +242,10 @@ class FavoriteArtistsDialog (QDialog ):
|
||||
fetched_any_successfully =True
|
||||
self ._logger (f"Fetched {processed_artists_from_source } artists from {source ['name']}.")
|
||||
|
||||
except requests .exceptions .RequestException as e :
|
||||
except Exception as e :
|
||||
error_msg =f"Error fetching favorites from {source ['name']}: {e }"
|
||||
self ._logger (error_msg )
|
||||
errors_occurred .append (error_msg )
|
||||
except Exception as e :
|
||||
error_msg =f"An unexpected error occurred with {source ['name']}: {e }"
|
||||
self ._logger (error_msg )
|
||||
errors_occurred .append (error_msg )
|
||||
|
||||
|
||||
if self .cookies_config ['use_cookie']and not any_cookies_loaded_successfully_for_any_source :
|
||||
self .status_label .setText (self ._tr ("fav_artists_cookies_required_status","Error: Cookies enabled but could not be loaded for any source."))
|
||||
@@ -288,7 +271,7 @@ class FavoriteArtistsDialog (QDialog ):
|
||||
self ._show_content_elements (True )
|
||||
self .download_button .setEnabled (True )
|
||||
elif not fetched_any_successfully and not errors_occurred :
|
||||
self .status_label .setText (self ._tr ("fav_artists_none_found_status","No favorite artists found on Kemono.su or Coomer.su."))
|
||||
self .status_label .setText (self ._tr ("fav_artists_none_found_status","No favorite artists found on Kemono or Coomer."))
|
||||
self ._show_content_elements (False )
|
||||
self .download_button .setEnabled (False )
|
||||
else :
|
||||
@@ -344,4 +327,4 @@ class FavoriteArtistsDialog (QDialog ):
|
||||
self .accept ()
|
||||
|
||||
def get_selected_artists (self ):
|
||||
return self .selected_artists_data
|
||||
return self .selected_artists_data
|
||||
|
||||
@@ -7,7 +7,7 @@ import traceback
|
||||
import json
|
||||
import re
|
||||
from collections import defaultdict
|
||||
import requests
|
||||
import cloudscraper # MODIFIED: Import cloudscraper
|
||||
from PyQt5.QtCore import QCoreApplication, Qt, pyqtSignal, QThread
|
||||
from PyQt5.QtWidgets import (
|
||||
QApplication, QDialog, QHBoxLayout, QLabel, QLineEdit, QListWidget,
|
||||
@@ -42,10 +42,9 @@ class FavoritePostsFetcherThread (QThread ):
|
||||
self .parent_logger_func (f"[FavPostsFetcherThread] {message }")
|
||||
|
||||
def run(self):
|
||||
kemono_su_fav_posts_url = "https://kemono.su/api/v1/account/favorites?type=post"
|
||||
coomer_su_fav_posts_url = "https://coomer.su/api/v1/account/favorites?type=post"
|
||||
kemono_cr_fav_posts_url = "https://kemono.cr/api/v1/account/favorites?type=post"
|
||||
coomer_st_fav_posts_url = "https://coomer.st/api/v1/account/favorites?type=post"
|
||||
# --- FIX: Use cloudscraper and add proper headers ---
|
||||
scraper = cloudscraper.create_scraper()
|
||||
# --- END FIX ---
|
||||
|
||||
all_fetched_posts_temp = []
|
||||
error_messages_for_summary = []
|
||||
@@ -56,8 +55,8 @@ class FavoritePostsFetcherThread (QThread ):
|
||||
self.progress_bar_update.emit(0, 0)
|
||||
|
||||
api_sources = [
|
||||
{"name": "Kemono.cr", "url": kemono_cr_fav_posts_url, "domain": "kemono.cr"},
|
||||
{"name": "Coomer.st", "url": coomer_st_fav_posts_url, "domain": "coomer.st"}
|
||||
{"name": "Kemono.cr", "url": "https://kemono.cr/api/v1/account/favorites?type=post", "domain": "kemono.cr"},
|
||||
{"name": "Coomer.st", "url": "https://coomer.st/api/v1/account/favorites?type=post", "domain": "coomer.st"}
|
||||
]
|
||||
|
||||
api_sources_to_try =[]
|
||||
@@ -81,32 +80,18 @@ class FavoritePostsFetcherThread (QThread ):
|
||||
cookies_dict_for_source = None
|
||||
if self.cookies_config['use_cookie']:
|
||||
primary_domain = source['domain']
|
||||
fallback_domain = None
|
||||
if primary_domain == "kemono.cr":
|
||||
fallback_domain = "kemono.su"
|
||||
elif primary_domain == "coomer.st":
|
||||
fallback_domain = "coomer.su"
|
||||
fallback_domain = "kemono.su" if "kemono" in primary_domain else "coomer.su"
|
||||
|
||||
# First, try the primary domain
|
||||
cookies_dict_for_source = prepare_cookies_for_request(
|
||||
True,
|
||||
self.cookies_config['cookie_text'],
|
||||
self.cookies_config['selected_cookie_file'],
|
||||
self.cookies_config['app_base_dir'],
|
||||
self._logger,
|
||||
target_domain=primary_domain
|
||||
True, self.cookies_config['cookie_text'], self.cookies_config['selected_cookie_file'],
|
||||
self.cookies_config['app_base_dir'], self._logger, target_domain=primary_domain
|
||||
)
|
||||
|
||||
# If no cookies found, try the fallback domain
|
||||
if not cookies_dict_for_source and fallback_domain:
|
||||
self._logger(f"Warning ({source['name']}): No cookies found for '{primary_domain}'. Trying fallback '{fallback_domain}'...")
|
||||
self._logger(f"Warning ({source['name']}): No cookies for '{primary_domain}'. Trying fallback '{fallback_domain}'...")
|
||||
cookies_dict_for_source = prepare_cookies_for_request(
|
||||
True,
|
||||
self.cookies_config['cookie_text'],
|
||||
self.cookies_config['selected_cookie_file'],
|
||||
self.cookies_config['app_base_dir'],
|
||||
self._logger,
|
||||
target_domain=fallback_domain
|
||||
True, self.cookies_config['cookie_text'], self.cookies_config['selected_cookie_file'],
|
||||
self.cookies_config['app_base_dir'], self._logger, target_domain=fallback_domain
|
||||
)
|
||||
|
||||
if cookies_dict_for_source:
|
||||
@@ -120,8 +105,18 @@ class FavoritePostsFetcherThread (QThread ):
|
||||
QCoreApplication .processEvents ()
|
||||
|
||||
try :
|
||||
headers ={'User-Agent':'Mozilla/5.0'}
|
||||
response =requests .get (source ['url'],headers =headers ,cookies =cookies_dict_for_source ,timeout =20 )
|
||||
# --- FIX: Add Referer and Accept headers ---
|
||||
headers = {
|
||||
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36',
|
||||
'Referer': f"https://{source['domain']}/favorites",
|
||||
'Accept': 'text/css'
|
||||
}
|
||||
# --- END FIX ---
|
||||
|
||||
# --- FIX: Use scraper instead of requests ---
|
||||
response = scraper.get(source['url'], headers=headers, cookies=cookies_dict_for_source, timeout=20)
|
||||
# --- END FIX ---
|
||||
|
||||
response .raise_for_status ()
|
||||
posts_data_from_api =response .json ()
|
||||
|
||||
@@ -153,33 +148,24 @@ class FavoritePostsFetcherThread (QThread ):
|
||||
fetched_any_successfully =True
|
||||
self ._logger (f"Fetched {processed_posts_from_source } posts from {source ['name']}.")
|
||||
|
||||
except requests .exceptions .RequestException as e :
|
||||
except Exception as e :
|
||||
err_detail =f"Error fetching favorite posts from {source ['name']}: {e }"
|
||||
self ._logger (err_detail )
|
||||
error_messages_for_summary .append (err_detail )
|
||||
if e .response is not None and e .response .status_code ==401 :
|
||||
if hasattr(e, 'response') and e.response is not None and e.response.status_code == 401:
|
||||
self .finished .emit ([],"KEY_AUTH_FAILED")
|
||||
self ._logger (f"Authorization failed for {source ['name']}, emitting KEY_AUTH_FAILED.")
|
||||
return
|
||||
except Exception as e :
|
||||
err_detail =f"An unexpected error occurred with {source ['name']}: {e }"
|
||||
self ._logger (err_detail )
|
||||
error_messages_for_summary .append (err_detail )
|
||||
|
||||
if self .cancellation_event .is_set ():
|
||||
self .finished .emit ([],"KEY_FETCH_CANCELLED_AFTER")
|
||||
return
|
||||
|
||||
|
||||
if self .cookies_config ['use_cookie']and not any_cookies_loaded_successfully_for_any_source :
|
||||
|
||||
if self .target_domain_preference and not any_cookies_loaded_successfully_for_any_source :
|
||||
|
||||
domain_key_part =self .error_key_map .get (self .target_domain_preference ,self .target_domain_preference .lower ().replace ('.','_'))
|
||||
self .finished .emit ([],f"KEY_COOKIES_REQUIRED_BUT_NOT_FOUND_FOR_DOMAIN_{domain_key_part }")
|
||||
return
|
||||
|
||||
|
||||
self .finished .emit ([],"KEY_COOKIES_REQUIRED_BUT_NOT_FOUND_GENERIC")
|
||||
return
|
||||
|
||||
@@ -643,4 +629,4 @@ class FavoritePostsDialog (QDialog ):
|
||||
self .accept ()
|
||||
|
||||
def get_selected_posts (self ):
|
||||
return self .selected_posts_data
|
||||
return self .selected_posts_data
|
||||
@@ -1,6 +1,7 @@
|
||||
# --- Standard Library Imports ---
|
||||
import os
|
||||
import json
|
||||
import sys
|
||||
|
||||
# --- PyQt5 Imports ---
|
||||
from PyQt5.QtCore import Qt, QStandardPaths
|
||||
@@ -17,9 +18,9 @@ from ...config.constants import (
|
||||
THEME_KEY, LANGUAGE_KEY, DOWNLOAD_LOCATION_KEY,
|
||||
RESOLUTION_KEY, UI_SCALE_KEY, SAVE_CREATOR_JSON_KEY,
|
||||
COOKIE_TEXT_KEY, USE_COOKIE_KEY,
|
||||
FETCH_FIRST_KEY ### ADDED ###
|
||||
FETCH_FIRST_KEY
|
||||
)
|
||||
|
||||
from ...services.updater import UpdateChecker, UpdateDownloader
|
||||
|
||||
class FutureSettingsDialog(QDialog):
|
||||
"""
|
||||
@@ -30,6 +31,7 @@ class FutureSettingsDialog(QDialog):
|
||||
super().__init__(parent)
|
||||
self.parent_app = parent_app_ref
|
||||
self.setModal(True)
|
||||
self.update_downloader_thread = None # To keep a reference
|
||||
|
||||
app_icon = get_app_icon_object()
|
||||
if app_icon and not app_icon.isNull():
|
||||
@@ -37,7 +39,7 @@ class FutureSettingsDialog(QDialog):
|
||||
|
||||
screen_height = QApplication.primaryScreen().availableGeometry().height() if QApplication.primaryScreen() else 800
|
||||
scale_factor = screen_height / 800.0
|
||||
base_min_w, base_min_h = 420, 390
|
||||
base_min_w, base_min_h = 420, 480 # Increased height for update section
|
||||
scaled_min_w = int(base_min_w * scale_factor)
|
||||
scaled_min_h = int(base_min_h * scale_factor)
|
||||
self.setMinimumSize(scaled_min_w, scaled_min_h)
|
||||
@@ -53,21 +55,19 @@ class FutureSettingsDialog(QDialog):
|
||||
self.interface_group_box = QGroupBox()
|
||||
interface_layout = QGridLayout(self.interface_group_box)
|
||||
|
||||
# Theme
|
||||
# Theme, UI Scale, Language (unchanged)...
|
||||
self.theme_label = QLabel()
|
||||
self.theme_toggle_button = QPushButton()
|
||||
self.theme_toggle_button.clicked.connect(self._toggle_theme)
|
||||
interface_layout.addWidget(self.theme_label, 0, 0)
|
||||
interface_layout.addWidget(self.theme_toggle_button, 0, 1)
|
||||
|
||||
# UI Scale
|
||||
self.ui_scale_label = QLabel()
|
||||
self.ui_scale_combo_box = QComboBox()
|
||||
self.ui_scale_combo_box.currentIndexChanged.connect(self._display_setting_changed)
|
||||
interface_layout.addWidget(self.ui_scale_label, 1, 0)
|
||||
interface_layout.addWidget(self.ui_scale_combo_box, 1, 1)
|
||||
|
||||
# Language
|
||||
|
||||
self.language_label = QLabel()
|
||||
self.language_combo_box = QComboBox()
|
||||
self.language_combo_box.currentIndexChanged.connect(self._language_selection_changed)
|
||||
@@ -78,6 +78,7 @@ class FutureSettingsDialog(QDialog):
|
||||
|
||||
self.download_window_group_box = QGroupBox()
|
||||
download_window_layout = QGridLayout(self.download_window_group_box)
|
||||
|
||||
self.window_size_label = QLabel()
|
||||
self.resolution_combo_box = QComboBox()
|
||||
self.resolution_combo_box.currentIndexChanged.connect(self._display_setting_changed)
|
||||
@@ -91,7 +92,7 @@ class FutureSettingsDialog(QDialog):
|
||||
download_window_layout.addWidget(self.save_path_button, 1, 1)
|
||||
|
||||
self.save_creator_json_checkbox = QCheckBox()
|
||||
self.save_creator_json_checkbox.stateChanged.connect(self._creator_json_setting_changed)
|
||||
self.save_creator_json_checkbox.stateChanged.connect(self._creator_json_setting_changed)
|
||||
download_window_layout.addWidget(self.save_creator_json_checkbox, 2, 0, 1, 2)
|
||||
|
||||
self.fetch_first_checkbox = QCheckBox()
|
||||
@@ -100,14 +101,96 @@ class FutureSettingsDialog(QDialog):
|
||||
|
||||
main_layout.addWidget(self.download_window_group_box)
|
||||
|
||||
# --- NEW: Update Section ---
|
||||
self.update_group_box = QGroupBox()
|
||||
update_layout = QGridLayout(self.update_group_box)
|
||||
self.version_label = QLabel()
|
||||
self.update_status_label = QLabel()
|
||||
self.check_update_button = QPushButton()
|
||||
self.check_update_button.clicked.connect(self._check_for_updates)
|
||||
update_layout.addWidget(self.version_label, 0, 0)
|
||||
update_layout.addWidget(self.update_status_label, 0, 1)
|
||||
update_layout.addWidget(self.check_update_button, 1, 0, 1, 2)
|
||||
main_layout.addWidget(self.update_group_box)
|
||||
# --- END: New Section ---
|
||||
|
||||
main_layout.addStretch(1)
|
||||
|
||||
self.ok_button = QPushButton()
|
||||
self.ok_button.clicked.connect(self.accept)
|
||||
main_layout.addWidget(self.ok_button, 0, Qt.AlignRight | Qt.AlignBottom)
|
||||
|
||||
def _retranslate_ui(self):
|
||||
self.setWindowTitle(self._tr("settings_dialog_title", "Settings"))
|
||||
self.interface_group_box.setTitle(self._tr("interface_group_title", "Interface Settings"))
|
||||
self.download_window_group_box.setTitle(self._tr("download_window_group_title", "Download & Window Settings"))
|
||||
self.theme_label.setText(self._tr("theme_label", "Theme:"))
|
||||
self.ui_scale_label.setText(self._tr("ui_scale_label", "UI Scale:"))
|
||||
self.language_label.setText(self._tr("language_label", "Language:"))
|
||||
self.window_size_label.setText(self._tr("window_size_label", "Window Size:"))
|
||||
self.default_path_label.setText(self._tr("default_path_label", "Default Path:"))
|
||||
self.save_creator_json_checkbox.setText(self._tr("save_creator_json_label", "Save Creator.json file"))
|
||||
self.fetch_first_checkbox.setText(self._tr("fetch_first_label", "Fetch First (Download after all pages are found)"))
|
||||
self.fetch_first_checkbox.setToolTip(self._tr("fetch_first_tooltip", "If checked, the downloader will find all posts from a creator first before starting any downloads.\nThis can be slower to start but provides a more accurate progress bar."))
|
||||
self._update_theme_toggle_button_text()
|
||||
self.save_path_button.setText(self._tr("settings_save_cookie_path_button", "Save Cookie + Download Path"))
|
||||
self.save_path_button.setToolTip(self._tr("settings_save_cookie_path_tooltip", "Save the current 'Download Location' and Cookie settings for future sessions."))
|
||||
self.ok_button.setText(self._tr("ok_button", "OK"))
|
||||
|
||||
# --- NEW: Translations for Update Section ---
|
||||
self.update_group_box.setTitle(self._tr("update_group_title", "Application Updates"))
|
||||
current_version = self.parent_app.windowTitle().split(' v')[-1]
|
||||
self.version_label.setText(self._tr("current_version_label", f"Current Version: v{current_version}"))
|
||||
self.update_status_label.setText(self._tr("update_status_ready", "Ready to check."))
|
||||
self.check_update_button.setText(self._tr("check_for_updates_button", "Check for Updates"))
|
||||
# --- END: New Translations ---
|
||||
|
||||
self._populate_display_combo_boxes()
|
||||
self._populate_language_combo_box()
|
||||
self._load_checkbox_states()
|
||||
|
||||
def _check_for_updates(self):
|
||||
"""Starts the update check thread."""
|
||||
self.check_update_button.setEnabled(False)
|
||||
self.update_status_label.setText(self._tr("update_status_checking", "Checking..."))
|
||||
current_version = self.parent_app.windowTitle().split(' v')[-1]
|
||||
|
||||
self.update_checker_thread = UpdateChecker(current_version)
|
||||
self.update_checker_thread.update_available.connect(self._on_update_available)
|
||||
self.update_checker_thread.up_to_date.connect(self._on_up_to_date)
|
||||
self.update_checker_thread.update_error.connect(self._on_update_error)
|
||||
self.update_checker_thread.start()
|
||||
|
||||
def _on_update_available(self, new_version, download_url):
|
||||
self.update_status_label.setText(self._tr("update_status_found", f"Update found: v{new_version}"))
|
||||
self.check_update_button.setEnabled(True)
|
||||
|
||||
reply = QMessageBox.question(self, self._tr("update_available_title", "Update Available"),
|
||||
self._tr("update_available_message", f"A new version (v{new_version}) is available.\nWould you like to download and install it now?"),
|
||||
QMessageBox.Yes | QMessageBox.No, QMessageBox.Yes)
|
||||
if reply == QMessageBox.Yes:
|
||||
self.ok_button.setEnabled(False)
|
||||
self.check_update_button.setEnabled(False)
|
||||
self.update_status_label.setText(self._tr("update_status_downloading", "Downloading update..."))
|
||||
self.update_downloader_thread = UpdateDownloader(download_url, self.parent_app)
|
||||
self.update_downloader_thread.download_finished.connect(self._on_download_finished)
|
||||
self.update_downloader_thread.download_error.connect(self._on_update_error)
|
||||
self.update_downloader_thread.start()
|
||||
|
||||
def _on_download_finished(self):
|
||||
QApplication.instance().quit()
|
||||
|
||||
def _on_up_to_date(self, message):
|
||||
self.update_status_label.setText(self._tr("update_status_latest", message))
|
||||
self.check_update_button.setEnabled(True)
|
||||
|
||||
def _on_update_error(self, message):
|
||||
self.update_status_label.setText(self._tr("update_status_error", f"Error: {message}"))
|
||||
self.check_update_button.setEnabled(True)
|
||||
self.ok_button.setEnabled(True)
|
||||
|
||||
# --- (The rest of the file remains unchanged from your provided code) ---
|
||||
def _load_checkbox_states(self):
|
||||
"""Loads the initial state for all checkboxes from settings."""
|
||||
self.save_creator_json_checkbox.blockSignals(True)
|
||||
should_save = self.parent_app.settings.value(SAVE_CREATOR_JSON_KEY, True, type=bool)
|
||||
self.save_creator_json_checkbox.setChecked(should_save)
|
||||
@@ -119,13 +202,11 @@ class FutureSettingsDialog(QDialog):
|
||||
self.fetch_first_checkbox.blockSignals(False)
|
||||
|
||||
def _creator_json_setting_changed(self, state):
|
||||
"""Saves the state of the 'Save Creator.json' checkbox."""
|
||||
is_checked = state == Qt.Checked
|
||||
self.parent_app.settings.setValue(SAVE_CREATOR_JSON_KEY, is_checked)
|
||||
self.parent_app.settings.sync()
|
||||
|
||||
def _fetch_first_setting_changed(self, state):
|
||||
"""Saves the state of the 'Fetch First' checkbox."""
|
||||
is_checked = state == Qt.Checked
|
||||
self.parent_app.settings.setValue(FETCH_FIRST_KEY, is_checked)
|
||||
self.parent_app.settings.sync()
|
||||
@@ -135,34 +216,6 @@ class FutureSettingsDialog(QDialog):
|
||||
return get_translation(self.parent_app.current_selected_language, key, default_text)
|
||||
return default_text
|
||||
|
||||
def _retranslate_ui(self):
|
||||
self.setWindowTitle(self._tr("settings_dialog_title", "Settings"))
|
||||
|
||||
self.interface_group_box.setTitle(self._tr("interface_group_title", "Interface Settings"))
|
||||
self.download_window_group_box.setTitle(self._tr("download_window_group_title", "Download & Window Settings"))
|
||||
|
||||
self.theme_label.setText(self._tr("theme_label", "Theme:"))
|
||||
self.ui_scale_label.setText(self._tr("ui_scale_label", "UI Scale:"))
|
||||
self.language_label.setText(self._tr("language_label", "Language:"))
|
||||
|
||||
self.window_size_label.setText(self._tr("window_size_label", "Window Size:"))
|
||||
self.default_path_label.setText(self._tr("default_path_label", "Default Path:"))
|
||||
self.save_creator_json_checkbox.setText(self._tr("save_creator_json_label", "Save Creator.json file"))
|
||||
|
||||
self.fetch_first_checkbox.setText(self._tr("fetch_first_label", "Fetch First (Download after all pages are found)"))
|
||||
self.fetch_first_checkbox.setToolTip(self._tr("fetch_first_tooltip", "If checked, the downloader will find all posts from a creator first before starting any downloads.\nThis can be slower to start but provides a more accurate progress bar."))
|
||||
|
||||
self._update_theme_toggle_button_text()
|
||||
self.save_path_button.setText(self._tr("settings_save_cookie_path_button", "Save Cookie + Download Path"))
|
||||
self.save_path_button.setToolTip(self._tr("settings_save_cookie_path_tooltip", "Save the current 'Download Location' and Cookie settings for future sessions."))
|
||||
self.ok_button.setText(self._tr("ok_button", "OK"))
|
||||
|
||||
self._populate_display_combo_boxes()
|
||||
self._populate_language_combo_box()
|
||||
self._load_checkbox_states()
|
||||
|
||||
# --- (The rest of the file remains unchanged) ---
|
||||
|
||||
def _apply_theme(self):
|
||||
if self.parent_app and self.parent_app.current_theme == "dark":
|
||||
scale = getattr(self.parent_app, 'scale_factor', 1)
|
||||
@@ -188,14 +241,7 @@ class FutureSettingsDialog(QDialog):
|
||||
def _populate_display_combo_boxes(self):
|
||||
self.resolution_combo_box.blockSignals(True)
|
||||
self.resolution_combo_box.clear()
|
||||
resolutions = [
|
||||
("Auto", self._tr("auto_resolution", "Auto (System Default)")),
|
||||
("1280x720", "1280 x 720"),
|
||||
("1600x900", "1600 x 900"),
|
||||
("1920x1080", "1920 x 1080 (Full HD)"),
|
||||
("2560x1440", "2560 x 1440 (2K)"),
|
||||
("3840x2160", "3840 x 2160 (4K)")
|
||||
]
|
||||
resolutions = [("Auto", "Auto"), ("1280x720", "1280x720"), ("1600x900", "1600x900"), ("1920x1080", "1920x1080")]
|
||||
current_res = self.parent_app.settings.value(RESOLUTION_KEY, "Auto")
|
||||
for res_key, res_name in resolutions:
|
||||
self.resolution_combo_box.addItem(res_name, res_key)
|
||||
@@ -214,35 +260,22 @@ class FutureSettingsDialog(QDialog):
|
||||
(1.50, "150%"),
|
||||
(1.75, "175%"),
|
||||
(2.0, "200%")
|
||||
]
|
||||
|
||||
current_scale = float(self.parent_app.settings.value(UI_SCALE_KEY, 1.0))
|
||||
]
|
||||
current_scale = self.parent_app.settings.value(UI_SCALE_KEY, 1.0)
|
||||
for scale_val, scale_name in scales:
|
||||
self.ui_scale_combo_box.addItem(scale_name, scale_val)
|
||||
if abs(current_scale - scale_val) < 0.01:
|
||||
if abs(float(current_scale) - scale_val) < 0.01:
|
||||
self.ui_scale_combo_box.setCurrentIndex(self.ui_scale_combo_box.count() - 1)
|
||||
self.ui_scale_combo_box.blockSignals(False)
|
||||
|
||||
def _display_setting_changed(self):
|
||||
selected_res = self.resolution_combo_box.currentData()
|
||||
selected_scale = self.ui_scale_combo_box.currentData()
|
||||
|
||||
self.parent_app.settings.setValue(RESOLUTION_KEY, selected_res)
|
||||
self.parent_app.settings.setValue(UI_SCALE_KEY, selected_scale)
|
||||
self.parent_app.settings.sync()
|
||||
|
||||
msg_box = QMessageBox(self)
|
||||
msg_box.setIcon(QMessageBox.Information)
|
||||
msg_box.setWindowTitle(self._tr("display_change_title", "Display Settings Changed"))
|
||||
msg_box.setText(self._tr("language_change_message", "A restart is required for these changes to take effect."))
|
||||
msg_box.setInformativeText(self._tr("language_change_informative", "Would you like to restart now?"))
|
||||
restart_button = msg_box.addButton(self._tr("restart_now_button", "Restart Now"), QMessageBox.ApplyRole)
|
||||
ok_button = msg_box.addButton(self._tr("ok_button", "OK"), QMessageBox.AcceptRole)
|
||||
msg_box.setDefaultButton(ok_button)
|
||||
msg_box.exec_()
|
||||
|
||||
if msg_box.clickedButton() == restart_button:
|
||||
self.parent_app._request_restart_application()
|
||||
QMessageBox.information(self, self._tr("display_change_title", "Display Settings Changed"),
|
||||
self._tr("language_change_message", "A restart is required..."))
|
||||
|
||||
def _populate_language_combo_box(self):
|
||||
self.language_combo_box.blockSignals(True)
|
||||
@@ -252,7 +285,7 @@ class FutureSettingsDialog(QDialog):
|
||||
("de", "Deutsch (German)"), ("es", "Español (Spanish)"), ("pt", "Português (Portuguese)"),
|
||||
("ru", "Русский (Russian)"), ("zh_CN", "简体中文 (Simplified Chinese)"),
|
||||
("zh_TW", "繁體中文 (Traditional Chinese)"), ("ko", "한국어 (Korean)")
|
||||
]
|
||||
]
|
||||
current_lang = self.parent_app.current_selected_language
|
||||
for lang_code, lang_name in languages:
|
||||
self.language_combo_box.addItem(lang_name, lang_code)
|
||||
@@ -266,59 +299,32 @@ class FutureSettingsDialog(QDialog):
|
||||
self.parent_app.settings.setValue(LANGUAGE_KEY, selected_lang_code)
|
||||
self.parent_app.settings.sync()
|
||||
self.parent_app.current_selected_language = selected_lang_code
|
||||
|
||||
self._retranslate_ui()
|
||||
if hasattr(self.parent_app, '_retranslate_main_ui'):
|
||||
self.parent_app._retranslate_main_ui()
|
||||
|
||||
msg_box = QMessageBox(self)
|
||||
msg_box.setIcon(QMessageBox.Information)
|
||||
msg_box.setWindowTitle(self._tr("language_change_title", "Language Changed"))
|
||||
msg_box.setText(self._tr("language_change_message", "A restart is required..."))
|
||||
msg_box.setInformativeText(self._tr("language_change_informative", "Would you like to restart now?"))
|
||||
restart_button = msg_box.addButton(self._tr("restart_now_button", "Restart Now"), QMessageBox.ApplyRole)
|
||||
ok_button = msg_box.addButton(self._tr("ok_button", "OK"), QMessageBox.AcceptRole)
|
||||
msg_box.setDefaultButton(ok_button)
|
||||
msg_box.exec_()
|
||||
|
||||
if msg_box.clickedButton() == restart_button:
|
||||
self.parent_app._request_restart_application()
|
||||
self.parent_app._retranslate_main_ui()
|
||||
QMessageBox.information(self, self._tr("language_change_title", "Language Changed"),
|
||||
self._tr("language_change_message", "A restart is required..."))
|
||||
|
||||
def _save_cookie_and_path(self):
|
||||
"""Saves the current download path and/or cookie settings from the main window."""
|
||||
path_saved = False
|
||||
cookie_saved = False
|
||||
|
||||
if hasattr(self.parent_app, 'dir_input') and self.parent_app.dir_input:
|
||||
current_path = self.parent_app.dir_input.text().strip()
|
||||
if current_path and os.path.isdir(current_path):
|
||||
self.parent_app.settings.setValue(DOWNLOAD_LOCATION_KEY, current_path)
|
||||
path_saved = True
|
||||
|
||||
if hasattr(self.parent_app, 'use_cookie_checkbox'):
|
||||
use_cookie = self.parent_app.use_cookie_checkbox.isChecked()
|
||||
cookie_content = self.parent_app.cookie_text_input.text().strip()
|
||||
|
||||
if use_cookie and cookie_content:
|
||||
self.parent_app.settings.setValue(USE_COOKIE_KEY, True)
|
||||
self.parent_app.settings.setValue(COOKIE_TEXT_KEY, cookie_content)
|
||||
cookie_saved = True
|
||||
else:
|
||||
else:
|
||||
self.parent_app.settings.setValue(USE_COOKIE_KEY, False)
|
||||
self.parent_app.settings.setValue(COOKIE_TEXT_KEY, "")
|
||||
|
||||
self.parent_app.settings.sync()
|
||||
|
||||
# --- User Feedback ---
|
||||
if path_saved and cookie_saved:
|
||||
message = self._tr("settings_save_both_success", "Download location and cookie settings saved.")
|
||||
elif path_saved:
|
||||
message = self._tr("settings_save_path_only_success", "Download location saved. No cookie settings were active to save.")
|
||||
elif cookie_saved:
|
||||
message = self._tr("settings_save_cookie_only_success", "Cookie settings saved. Download location was not set.")
|
||||
if path_saved or cookie_saved:
|
||||
QMessageBox.information(self, "Settings Saved", "Settings have been saved.")
|
||||
else:
|
||||
QMessageBox.warning(self, self._tr("settings_save_nothing_title", "Nothing to Save"),
|
||||
self._tr("settings_save_nothing_message", "The download location is not a valid directory and no cookie was active."))
|
||||
return
|
||||
|
||||
QMessageBox.information(self, self._tr("settings_save_success_title", "Settings Saved"), message)
|
||||
QMessageBox.warning(self, "Nothing to Save", "No valid settings to save.")
|
||||
@@ -10,6 +10,7 @@ import re
|
||||
import subprocess
|
||||
import datetime
|
||||
import requests
|
||||
import cloudscraper
|
||||
import unicodedata
|
||||
from collections import deque, defaultdict
|
||||
import threading
|
||||
@@ -36,6 +37,10 @@ from ..core.workers import PostProcessorSignals
|
||||
from ..core.api_client import download_from_api
|
||||
from ..core.discord_client import fetch_server_channels, fetch_channel_messages
|
||||
from ..core.manager import DownloadManager
|
||||
from ..core.nhentai_client import fetch_nhentai_gallery
|
||||
from ..core.bunkr_client import fetch_bunkr_data
|
||||
from ..core.saint2_client import fetch_saint2_data
|
||||
from ..core.erome_client import fetch_erome_data
|
||||
from .assets import get_app_icon_object
|
||||
from ..config.constants import *
|
||||
from ..utils.file_utils import KNOWN_NAMES, clean_folder_name
|
||||
@@ -281,19 +286,15 @@ class DownloaderApp (QWidget ):
|
||||
self.download_location_label_widget = None
|
||||
self.remove_from_filename_label_widget = None
|
||||
self.skip_words_label_widget = None
|
||||
self.setWindowTitle("Kemono Downloader v6.4.3")
|
||||
self.setWindowTitle("Kemono Downloader v7.0.0")
|
||||
setup_ui(self)
|
||||
self._connect_signals()
|
||||
self.log_signal.emit("ℹ️ Local API server functionality has been removed.")
|
||||
self.log_signal.emit("ℹ️ 'Skip Current File' button has been removed.")
|
||||
if hasattr(self, 'character_input'):
|
||||
self.character_input.setToolTip(self._tr("character_input_tooltip", "Enter character names (comma-separated)..."))
|
||||
self.log_signal.emit(f"ℹ️ Manga filename style loaded: '{self.manga_filename_style}'")
|
||||
self.log_signal.emit(f"ℹ️ Skip words scope loaded: '{self.skip_words_scope}'")
|
||||
self.log_signal.emit(f"ℹ️ Character filter scope set to default: '{self.char_filter_scope}'")
|
||||
self.log_signal.emit(f"ℹ️ Multi-part download defaults to: {'Enabled' if self.allow_multipart_download_setting else 'Disabled'}")
|
||||
self.log_signal.emit(f"ℹ️ Cookie text defaults to: Empty on launch")
|
||||
self.log_signal.emit(f"ℹ️ 'Use Cookie' setting defaults to: Disabled on launch")
|
||||
self.log_signal.emit(f"ℹ️ Scan post content for images defaults to: {'Enabled' if self.scan_content_images_setting else 'Disabled'}")
|
||||
self.log_signal.emit(f"ℹ️ Application language loaded: '{self.current_selected_language.upper()}' (UI may not reflect this yet).")
|
||||
self._retranslate_main_ui()
|
||||
@@ -302,6 +303,18 @@ class DownloaderApp (QWidget ):
|
||||
self._load_saved_cookie_settings()
|
||||
self._update_button_states_and_connections()
|
||||
self._check_for_interrupted_session()
|
||||
self._cleanup_after_update()
|
||||
|
||||
def _cleanup_after_update(self):
|
||||
"""Deletes the old executable after a successful update."""
|
||||
try:
|
||||
app_path = sys.executable
|
||||
old_app_path = os.path.join(os.path.dirname(app_path), "Kemono.Downloader.exe.old")
|
||||
if os.path.exists(old_app_path):
|
||||
os.remove(old_app_path)
|
||||
self.log_signal.emit("ℹ️ Cleaned up old application file after update.")
|
||||
except Exception as e:
|
||||
self.log_signal.emit(f"⚠️ Could not remove old application file: {e}")
|
||||
|
||||
def _apply_theme_and_restart_prompt(self):
|
||||
"""Applies the theme and prompts the user to restart."""
|
||||
@@ -923,7 +936,7 @@ class DownloaderApp (QWidget ):
|
||||
if hasattr (self ,'use_cookie_checkbox'):
|
||||
self .use_cookie_checkbox .toggled .connect (self ._update_cookie_input_visibility )
|
||||
if hasattr (self ,'link_input'):
|
||||
self .link_input .textChanged .connect (self ._sync_queue_with_link_input )
|
||||
self.link_input.textChanged.connect(self._update_ui_for_url_change)
|
||||
self.link_input.textChanged.connect(self._update_contextual_ui_elements)
|
||||
self.link_input.textChanged.connect(self._update_button_states_and_connections)
|
||||
if hasattr(self, 'discord_scope_toggle_button'):
|
||||
@@ -2215,12 +2228,21 @@ class DownloaderApp (QWidget ):
|
||||
if not button or not checked:
|
||||
return
|
||||
is_only_links = (button == self.radio_only_links)
|
||||
if hasattr(self, 'use_multithreading_checkbox'):
|
||||
|
||||
if hasattr(self, 'use_multithreading_checkbox') and hasattr(self, 'thread_count_input'):
|
||||
if is_only_links:
|
||||
self.use_multithreading_checkbox.setChecked(False)
|
||||
self.use_multithreading_checkbox.setEnabled(False)
|
||||
# When "Only Links" is selected, enable multithreading, set threads to 20, and lock the input.
|
||||
self.use_multithreading_checkbox.setChecked(True)
|
||||
self.thread_count_input.setText("20")
|
||||
self.thread_count_input.setEnabled(False)
|
||||
self.thread_count_label.setEnabled(False)
|
||||
self.update_multithreading_label("20")
|
||||
else:
|
||||
self.use_multithreading_checkbox.setEnabled(True)
|
||||
# When another mode is selected, re-enable the input for user control.
|
||||
is_multithreading_checked = self.use_multithreading_checkbox.isChecked()
|
||||
self.thread_count_input.setEnabled(is_multithreading_checked)
|
||||
self.thread_count_label.setEnabled(is_multithreading_checked)
|
||||
|
||||
if button != self.radio_more and checked:
|
||||
self.radio_more.setText("More")
|
||||
self.more_filter_scope = None
|
||||
@@ -3114,15 +3136,75 @@ class DownloaderApp (QWidget ):
|
||||
if total_posts >0 or processed_posts >0 :
|
||||
self .file_progress_label .setText ("")
|
||||
|
||||
def _set_ui_for_specialized_downloader(self, is_specialized):
|
||||
"""Disables or enables UI elements for non-standard downloaders."""
|
||||
widgets_to_disable = [
|
||||
self.page_range_label, self.start_page_input, self.to_label, self.end_page_input,
|
||||
self.character_filter_widget, self.skip_words_input, self.skip_scope_toggle_button,
|
||||
self.remove_from_filename_input, self.radio_images, self.radio_videos,
|
||||
self.radio_only_archives, self.radio_only_links, self.radio_only_audio, self.radio_more,
|
||||
self.skip_zip_checkbox, self.download_thumbnails_checkbox, self.compress_images_checkbox,
|
||||
self.use_subfolders_checkbox, self.use_subfolder_per_post_checkbox,
|
||||
self.scan_content_images_checkbox, self.external_links_checkbox, self.manga_mode_checkbox,
|
||||
self.use_cookie_checkbox, self.keep_duplicates_checkbox, self.date_prefix_checkbox,
|
||||
self.manga_rename_toggle_button, self.manga_date_prefix_input,
|
||||
self.multipart_toggle_button, self.custom_folder_input, self.custom_folder_label,
|
||||
self.discord_scope_toggle_button, self.save_discord_as_pdf_btn
|
||||
]
|
||||
|
||||
enable_state = not is_specialized
|
||||
|
||||
for widget in widgets_to_disable:
|
||||
if widget:
|
||||
widget.setEnabled(enable_state)
|
||||
|
||||
# When disabling, force 'All' to be checked and disable it too
|
||||
if is_specialized and self.radio_all:
|
||||
self.radio_all.setChecked(True)
|
||||
self.radio_all.setEnabled(False)
|
||||
elif self.radio_all:
|
||||
self.radio_all.setEnabled(True)
|
||||
|
||||
# Re-run standard UI logic when re-enabling to restore correct states
|
||||
if enable_state:
|
||||
self._update_all_ui_states()
|
||||
|
||||
def _update_all_ui_states(self):
|
||||
"""A single function to call all UI update methods to restore state."""
|
||||
is_manga_checked = self.manga_mode_checkbox.isChecked() if self.manga_mode_checkbox else False
|
||||
is_subfolder_checked = self.use_subfolders_checkbox.isChecked() if self.use_subfolders_checkbox else False
|
||||
is_cookie_checked = self.use_cookie_checkbox.isChecked() if self.use_cookie_checkbox else False
|
||||
|
||||
self.update_ui_for_manga_mode(is_manga_checked)
|
||||
self.update_custom_folder_visibility()
|
||||
self.update_page_range_enabled_state()
|
||||
if self.radio_group.checkedButton():
|
||||
self._handle_filter_mode_change(self.radio_group.checkedButton(), True)
|
||||
self.update_ui_for_subfolders(is_subfolder_checked)
|
||||
self._update_cookie_input_visibility(is_cookie_checked)
|
||||
|
||||
def _update_contextual_ui_elements(self, text=""):
|
||||
"""Shows or hides UI elements based on the URL, like the Discord scope button."""
|
||||
if not hasattr(self, 'discord_scope_toggle_button'): return
|
||||
|
||||
url_text = self.link_input.text().strip()
|
||||
service, _, _ = extract_post_info(url_text)
|
||||
|
||||
# Handle specialized downloaders (Bunkr, nhentai)
|
||||
is_saint2 = 'saint2.su' in url_text or 'saint2.pk' in url_text
|
||||
is_erome = 'erome.com' in url_text
|
||||
is_specialized = service in ['bunkr', 'nhentai'] or is_saint2 or is_erome
|
||||
self._set_ui_for_specialized_downloader(is_specialized)
|
||||
|
||||
# Handle Discord UI
|
||||
is_discord = (service == 'discord')
|
||||
self.discord_scope_toggle_button.setVisible(is_discord)
|
||||
if is_discord: self._update_discord_scope_button_text()
|
||||
else: self.download_btn.setText(self._tr("start_download_button_text", "⬇️ Start Download"))
|
||||
self.save_discord_as_pdf_btn.setVisible(is_discord)
|
||||
|
||||
if is_discord:
|
||||
self._update_discord_scope_button_text()
|
||||
elif not is_specialized: # Don't change button text for specialized downloaders
|
||||
self.download_btn.setText(self._tr("start_download_button_text", "⬇️ Start Download"))
|
||||
|
||||
def _update_discord_scope_button_text(self):
|
||||
"""Updates the text of the discord scope button and the main download button."""
|
||||
@@ -3146,8 +3228,8 @@ class DownloaderApp (QWidget ):
|
||||
self._start_download_of_fetched_posts()
|
||||
return True
|
||||
|
||||
self.finish_lock = threading.Lock()
|
||||
self.is_finishing = False
|
||||
self.finish_lock = threading.Lock()
|
||||
self.is_finishing = False
|
||||
if self.active_update_profile:
|
||||
if not self.new_posts_for_update:
|
||||
return self._check_for_updates()
|
||||
@@ -3169,13 +3251,13 @@ class DownloaderApp (QWidget ):
|
||||
|
||||
processed_post_ids_for_restore = []
|
||||
manga_counters_for_restore = None
|
||||
start_offset_for_restore = 0
|
||||
start_offset_for_restore = 0
|
||||
|
||||
if is_restore and self.interrupted_session_data:
|
||||
self.log_signal.emit(" Restoring session state...")
|
||||
download_state = self.interrupted_session_data.get("download_state", {})
|
||||
processed_post_ids_for_restore = download_state.get("processed_post_ids", [])
|
||||
start_offset_for_restore = download_state.get("last_processed_offset", 0)
|
||||
start_offset_for_restore = download_state.get("last_processed_offset", 0)
|
||||
restored_hashes = download_state.get("successfully_downloaded_hashes", [])
|
||||
if restored_hashes:
|
||||
with self.downloaded_file_hashes_lock:
|
||||
@@ -3184,7 +3266,7 @@ class DownloaderApp (QWidget ):
|
||||
manga_counters_for_restore = download_state.get("manga_counters")
|
||||
if processed_post_ids_for_restore:
|
||||
self.log_signal.emit(f" Will skip {len(processed_post_ids_for_restore)} already processed posts.")
|
||||
if start_offset_for_restore > 0:
|
||||
if start_offset_for_restore > 0:
|
||||
self.log_signal.emit(f" Resuming fetch from page offset: {start_offset_for_restore}")
|
||||
if manga_counters_for_restore:
|
||||
self.log_signal.emit(f" Restoring manga counters: {manga_counters_for_restore}")
|
||||
@@ -3207,6 +3289,8 @@ class DownloaderApp (QWidget ):
|
||||
|
||||
api_url = direct_api_url if direct_api_url else self.link_input.text().strip()
|
||||
|
||||
# --- START: MOVED AND CORRECTED LOGIC ---
|
||||
# This block is moved to run before any special URL checks.
|
||||
main_ui_download_dir = self.dir_input.text().strip()
|
||||
extract_links_only = (self.radio_only_links and self.radio_only_links.isChecked())
|
||||
effective_output_dir_for_run = ""
|
||||
@@ -3229,8 +3313,10 @@ class DownloaderApp (QWidget ):
|
||||
return False
|
||||
effective_output_dir_for_run = os.path.normpath(override_output_dir)
|
||||
else:
|
||||
is_special_downloader = 'saint2.su' in api_url or 'saint2.pk' in api_url or 'nhentai.net' in api_url or 'bunkr' in api_url or 'erome.com' in api_url
|
||||
|
||||
if not extract_links_only and not main_ui_download_dir:
|
||||
QMessageBox.critical(self, "Input Error", "Download Directory is required when not in 'Only Links' mode.")
|
||||
QMessageBox.critical(self, "Input Error", "Download Directory is required.")
|
||||
return False
|
||||
|
||||
if not extract_links_only and main_ui_download_dir and not os.path.isdir(main_ui_download_dir):
|
||||
@@ -3247,7 +3333,122 @@ class DownloaderApp (QWidget ):
|
||||
else:
|
||||
self.log_signal.emit("❌ Download cancelled: Output directory does not exist and was not created.")
|
||||
return False
|
||||
effective_output_dir_for_run = os.path.normpath(main_ui_download_dir)
|
||||
effective_output_dir_for_run = os.path.normpath(main_ui_download_dir) if main_ui_download_dir else ""
|
||||
|
||||
if 'erome.com' in api_url:
|
||||
self.log_signal.emit("ℹ️ Erome.com URL detected. Starting dedicated Erome download.")
|
||||
self.set_ui_enabled(False)
|
||||
|
||||
self.download_thread = EromeDownloadThread(api_url, effective_output_dir_for_run, self)
|
||||
self.download_thread.progress_signal.connect(self.handle_main_log)
|
||||
self.download_thread.file_progress_signal.connect(self.update_file_progress_display)
|
||||
self.download_thread.finished_signal.connect(
|
||||
lambda dl, skip, cancelled: self.download_finished(dl, skip, cancelled, [])
|
||||
)
|
||||
self.download_thread.start()
|
||||
self._update_button_states_and_connections()
|
||||
return True
|
||||
|
||||
if 'nhentai.net' in api_url and not re.search(r'/g/(\d+)', api_url):
|
||||
self.log_signal.emit("=" * 40)
|
||||
self.log_signal.emit("🚀 nhentai batch download mode detected.")
|
||||
|
||||
nhentai_txt_path = os.path.join(self.app_base_dir, "appdata", "nhentai.txt")
|
||||
self.log_signal.emit(f" Looking for batch file at: {nhentai_txt_path}")
|
||||
|
||||
if not os.path.exists(nhentai_txt_path):
|
||||
QMessageBox.warning(self, "File Not Found", f"To use batch mode, create a file named 'nhentai.txt' in your 'appdata' folder.\n\nPlace one nhentai URL on each line.")
|
||||
self.log_signal.emit(f" ❌ 'nhentai.txt' not found. Aborting batch download.")
|
||||
return False
|
||||
|
||||
urls_to_download = []
|
||||
try:
|
||||
with open(nhentai_txt_path, 'r', encoding='utf-8') as f:
|
||||
for line in f:
|
||||
found_urls = re.findall(r'https?://nhentai\.net/g/\d+/?', line)
|
||||
if found_urls:
|
||||
urls_to_download.extend(found_urls)
|
||||
except Exception as e:
|
||||
QMessageBox.critical(self, "File Error", f"Could not read 'nhentai.txt':\n{e}")
|
||||
self.log_signal.emit(f" ❌ Error reading 'nhentai.txt': {e}")
|
||||
return False
|
||||
|
||||
if not urls_to_download:
|
||||
QMessageBox.information(self, "Empty File", "No valid nhentai gallery URLs were found in 'nhentai.txt'.")
|
||||
self.log_signal.emit(" 'nhentai.txt' was found but contained no valid URLs.")
|
||||
return False
|
||||
|
||||
self.log_signal.emit(f" Found {len(urls_to_download)} URLs to process.")
|
||||
self.favorite_download_queue.clear()
|
||||
for url in urls_to_download:
|
||||
self.favorite_download_queue.append({
|
||||
'url': url,
|
||||
'name': f"nhentai gallery from batch",
|
||||
'type': 'post'
|
||||
})
|
||||
|
||||
if not self.is_processing_favorites_queue:
|
||||
self._process_next_favorite_download()
|
||||
return True
|
||||
|
||||
is_saint2_url = 'saint2.su' in api_url or 'saint2.pk' in api_url
|
||||
if is_saint2_url:
|
||||
# First, check if it's the batch command. If so, do nothing here and let the next block handle it.
|
||||
if api_url.strip().lower() != 'saint2.su':
|
||||
self.log_signal.emit("ℹ️ Saint2.su URL detected. Starting dedicated Saint2 download.")
|
||||
self.set_ui_enabled(False)
|
||||
self.download_thread = Saint2DownloadThread(api_url, effective_output_dir_for_run, self)
|
||||
self.download_thread.progress_signal.connect(self.handle_main_log)
|
||||
self.download_thread.file_progress_signal.connect(self.update_file_progress_display)
|
||||
self.download_thread.finished_signal.connect(
|
||||
lambda dl, skip, cancelled: self.download_finished(dl, skip, cancelled, [])
|
||||
)
|
||||
self.download_thread.start()
|
||||
self._update_button_states_and_connections()
|
||||
return True
|
||||
|
||||
if api_url.strip().lower() == 'saint2.su':
|
||||
self.log_signal.emit("=" * 40)
|
||||
self.log_signal.emit("🚀 Saint2.su batch download mode detected.")
|
||||
|
||||
saint2_txt_path = os.path.join(self.app_base_dir, "appdata", "saint2.su.txt")
|
||||
self.log_signal.emit(f" Looking for batch file at: {saint2_txt_path}")
|
||||
|
||||
if not os.path.exists(saint2_txt_path):
|
||||
QMessageBox.warning(self, "File Not Found", f"To use batch mode, create a file named 'saint2.su.txt' in your 'appdata' folder.\n\nPlace one saint2.su URL on each line.")
|
||||
self.log_signal.emit(f" ❌ 'saint2.su.txt' not found. Aborting batch download.")
|
||||
return False
|
||||
|
||||
urls_to_download = []
|
||||
try:
|
||||
with open(saint2_txt_path, 'r', encoding='utf-8') as f:
|
||||
for line in f:
|
||||
# Find valid saint2 URLs in the line
|
||||
found_urls = re.findall(r'https?://saint\d*\.(?:su|pk|cr|to)/(?:a|d|embed)/[^/?#\s]+', line)
|
||||
if found_urls:
|
||||
urls_to_download.extend(found_urls)
|
||||
except Exception as e:
|
||||
QMessageBox.critical(self, "File Error", f"Could not read 'saint2.su.txt':\n{e}")
|
||||
self.log_signal.emit(f" ❌ Error reading 'saint2.su.txt': {e}")
|
||||
return False
|
||||
|
||||
if not urls_to_download:
|
||||
QMessageBox.information(self, "Empty File", "No valid saint2.su URLs were found in 'saint2.su.txt'.")
|
||||
self.log_signal.emit(" 'saint2.su.txt' was found but contained no valid URLs.")
|
||||
return False
|
||||
|
||||
self.log_signal.emit(f" Found {len(urls_to_download)} URLs to process.")
|
||||
self.favorite_download_queue.clear()
|
||||
for url in urls_to_download:
|
||||
self.favorite_download_queue.append({
|
||||
'url': url,
|
||||
'name': f"saint2.su link from batch",
|
||||
'type': 'post' # Treat each URL as a single post-like item
|
||||
})
|
||||
|
||||
if not self.is_processing_favorites_queue:
|
||||
self._process_next_favorite_download()
|
||||
return True
|
||||
|
||||
if not is_restore:
|
||||
self._create_initial_session_file(api_url, effective_output_dir_for_run, remaining_queue=self.favorite_download_queue)
|
||||
@@ -3274,6 +3475,44 @@ class DownloaderApp (QWidget ):
|
||||
|
||||
service, id1, id2 = extract_post_info(api_url)
|
||||
|
||||
if service == 'nhentai':
|
||||
gallery_id = id1
|
||||
self.log_signal.emit("=" * 40)
|
||||
self.log_signal.emit(f"🚀 Detected nhentai gallery ID: {gallery_id}")
|
||||
|
||||
if not effective_output_dir_for_run or not os.path.isdir(effective_output_dir_for_run):
|
||||
QMessageBox.critical(self, "Input Error", "A valid Download Location is required.")
|
||||
return False
|
||||
|
||||
gallery_data = fetch_nhentai_gallery(gallery_id, self.log_signal.emit)
|
||||
if not gallery_data:
|
||||
QMessageBox.critical(self, "Error", f"Could not retrieve gallery data for ID {gallery_id}.")
|
||||
return False
|
||||
|
||||
self.set_ui_enabled(False)
|
||||
self.download_thread = NhentaiDownloadThread(gallery_data, effective_output_dir_for_run, self)
|
||||
self.download_thread.progress_signal.connect(self.handle_main_log)
|
||||
self.download_thread.finished_signal.connect(
|
||||
lambda dl, skip, cancelled: self.download_finished(dl, skip, cancelled, [])
|
||||
)
|
||||
self.download_thread.start()
|
||||
self._update_button_states_and_connections()
|
||||
return True
|
||||
|
||||
if service == 'bunkr':
|
||||
self.log_signal.emit("ℹ️ Bunkr URL detected. Starting dedicated Bunkr download.")
|
||||
self.set_ui_enabled(False)
|
||||
|
||||
self.download_thread = BunkrDownloadThread(id1, effective_output_dir_for_run, self)
|
||||
self.download_thread.progress_signal.connect(self.handle_main_log)
|
||||
self.download_thread.file_progress_signal.connect(self.update_file_progress_display)
|
||||
self.download_thread.finished_signal.connect(
|
||||
lambda dl, skip, cancelled: self.download_finished(dl, skip, cancelled, [])
|
||||
)
|
||||
self.download_thread.start()
|
||||
self._update_button_states_and_connections()
|
||||
return True
|
||||
|
||||
if not service or not id1:
|
||||
QMessageBox.critical(self, "Input Error", "Invalid or unsupported URL format.")
|
||||
return False
|
||||
@@ -3282,7 +3521,6 @@ class DownloaderApp (QWidget ):
|
||||
server_id, channel_id = id1, id2
|
||||
|
||||
def discord_processing_task():
|
||||
# --- FIX: Wrap the entire task in a try...finally block ---
|
||||
try:
|
||||
def queue_logger(message):
|
||||
self.worker_to_gui_queue.put({'type': 'progress', 'payload': (message,)})
|
||||
@@ -3295,7 +3533,6 @@ class DownloaderApp (QWidget ):
|
||||
self.selected_cookie_filepath, self.app_base_dir, queue_logger
|
||||
)
|
||||
|
||||
# --- SCOPE: MESSAGES (PDF CREATION) ---
|
||||
if self.discord_download_scope == 'messages':
|
||||
queue_logger("=" * 40)
|
||||
queue_logger(f"🚀 Starting Discord PDF export for: {api_url}")
|
||||
@@ -3307,7 +3544,7 @@ class DownloaderApp (QWidget ):
|
||||
return
|
||||
|
||||
default_filename = f"discord_{server_id}_{channel_id or 'server'}.pdf"
|
||||
output_filepath = os.path.join(output_dir, default_filename) # We'll save with a default name
|
||||
output_filepath = os.path.join(output_dir, default_filename)
|
||||
|
||||
all_messages, channels_to_process = [], []
|
||||
server_name_for_pdf = server_id
|
||||
@@ -3346,7 +3583,6 @@ class DownloaderApp (QWidget ):
|
||||
self.finished_signal.emit(0, len(all_messages), self.cancellation_event.is_set(), [])
|
||||
return
|
||||
|
||||
# --- SCOPE: FILES (DOWNLOAD) ---
|
||||
elif self.discord_download_scope == 'files':
|
||||
worker_args = {
|
||||
'download_root': effective_output_dir_for_run, 'known_names': list(KNOWN_NAMES),
|
||||
@@ -3406,10 +3642,8 @@ class DownloaderApp (QWidget ):
|
||||
|
||||
self.finished_signal.emit(total_dl, total_skip, self.cancellation_event.is_set(), [])
|
||||
finally:
|
||||
# This ensures the flag is reset, allowing the UI to finalize correctly
|
||||
self.is_fetcher_thread_running = False
|
||||
|
||||
# --- FIX: Set the fetcher running flag to prevent premature finalization ---
|
||||
self.is_fetcher_thread_running = True
|
||||
|
||||
self.set_ui_enabled(False)
|
||||
@@ -4057,6 +4291,79 @@ class DownloaderApp (QWidget ):
|
||||
self.is_restore_pending = True
|
||||
self.start_download(direct_api_url=restore_url, override_output_dir=restore_dir, is_restore=True)
|
||||
|
||||
def _update_ui_for_url_change(self, text=""):
|
||||
"""A single, authoritative function to update all UI states based on the URL."""
|
||||
url_text = self.link_input.text().strip()
|
||||
service, _, _ = extract_post_info(url_text)
|
||||
|
||||
# A list of all widgets that are context-dependent
|
||||
widgets_to_manage = [
|
||||
self.page_range_label, self.start_page_input, self.to_label, self.end_page_input,
|
||||
self.character_filter_widget, self.skip_words_input, self.skip_scope_toggle_button,
|
||||
self.remove_from_filename_input, self.radio_all, self.radio_images, self.radio_videos,
|
||||
self.radio_only_archives, self.radio_only_links, self.radio_only_audio, self.radio_more,
|
||||
self.skip_zip_checkbox, self.download_thumbnails_checkbox, self.compress_images_checkbox,
|
||||
self.use_subfolders_checkbox, self.use_subfolder_per_post_checkbox,
|
||||
self.scan_content_images_checkbox, self.external_links_checkbox, self.manga_mode_checkbox,
|
||||
self.use_cookie_checkbox, self.keep_duplicates_checkbox, self.date_prefix_checkbox,
|
||||
self.manga_rename_toggle_button, self.manga_date_prefix_input,
|
||||
self.multipart_toggle_button, self.custom_folder_input, self.custom_folder_label
|
||||
]
|
||||
|
||||
# --- Logic for Specialized Downloaders (Bunkr, nhentai) ---
|
||||
if service in ['bunkr', 'nhentai']:
|
||||
self.progress_log_label.setText("📜 Progress Log:")
|
||||
self.download_btn.setText(self._tr("start_download_button_text", "⬇️ Start Download"))
|
||||
|
||||
# Disable all complex settings
|
||||
for widget in widgets_to_manage:
|
||||
if widget:
|
||||
widget.setEnabled(False)
|
||||
|
||||
# Force 'All' filter and disable it
|
||||
if self.radio_all:
|
||||
self.radio_all.setChecked(True)
|
||||
|
||||
# Ensure Discord UI is hidden
|
||||
if hasattr(self, 'discord_scope_toggle_button'):
|
||||
self.discord_scope_toggle_button.setVisible(False)
|
||||
if hasattr(self, 'save_discord_as_pdf_btn'):
|
||||
self.save_discord_as_pdf_btn.setVisible(False)
|
||||
|
||||
return # CRUCIAL: Stop here for specialized URLs
|
||||
|
||||
# --- Logic for Standard Downloaders (Kemono, Coomer, Discord) ---
|
||||
|
||||
# First, re-enable all managed widgets as a baseline
|
||||
for widget in widgets_to_manage:
|
||||
if widget:
|
||||
widget.setEnabled(True)
|
||||
|
||||
# Now, apply context-specific rules for the standard downloaders
|
||||
is_discord = (service == 'discord')
|
||||
if hasattr(self, 'discord_scope_toggle_button'):
|
||||
self.discord_scope_toggle_button.setVisible(is_discord)
|
||||
if hasattr(self, 'save_discord_as_pdf_btn'):
|
||||
self.save_discord_as_pdf_btn.setVisible(is_discord)
|
||||
|
||||
if is_discord:
|
||||
self._update_discord_scope_button_text()
|
||||
else:
|
||||
self.download_btn.setText(self._tr("start_download_button_text", "⬇️ Start Download"))
|
||||
|
||||
# Re-run all the standard UI state functions to apply the correct logic
|
||||
is_manga_checked = self.manga_mode_checkbox.isChecked() if self.manga_mode_checkbox else False
|
||||
is_subfolder_checked = self.use_subfolders_checkbox.isChecked() if self.use_subfolders_checkbox else False
|
||||
is_cookie_checked = self.use_cookie_checkbox.isChecked() if self.use_cookie_checkbox else False
|
||||
|
||||
self.update_ui_for_manga_mode(is_manga_checked)
|
||||
self.update_custom_folder_visibility()
|
||||
self.update_page_range_enabled_state()
|
||||
if self.radio_group and self.radio_group.checkedButton():
|
||||
self._handle_filter_mode_change(self.radio_group.checkedButton(), True)
|
||||
self.update_ui_for_subfolders(is_subfolder_checked)
|
||||
self._update_cookie_input_visibility(is_cookie_checked)
|
||||
|
||||
def start_single_threaded_download (self ,**kwargs ):
|
||||
global BackendDownloadThread
|
||||
try :
|
||||
@@ -4741,6 +5048,22 @@ class DownloaderApp (QWidget ):
|
||||
self.log_signal.emit(" Cancelling active External Link download thread...")
|
||||
self.external_link_download_thread.cancel()
|
||||
|
||||
if isinstance(self.download_thread, NhentaiDownloadThread):
|
||||
self.log_signal.emit(" Signaling nhentai download thread to cancel.")
|
||||
self.download_thread.cancel()
|
||||
|
||||
if isinstance(self.download_thread, BunkrDownloadThread):
|
||||
self.log_signal.emit(" Signaling Bunkr download thread to cancel.")
|
||||
self.download_thread.cancel()
|
||||
|
||||
if isinstance(self.download_thread, Saint2DownloadThread):
|
||||
self.log_signal.emit(" Signaling Saint2 download thread to cancel.")
|
||||
self.download_thread.cancel()
|
||||
|
||||
if isinstance(self.download_thread, EromeDownloadThread):
|
||||
self.log_signal.emit(" Signaling Erome download thread to cancel.")
|
||||
self.download_thread.cancel()
|
||||
|
||||
def _get_domain_for_service(self, service_name: str) -> str:
|
||||
"""Determines the base domain for a given service."""
|
||||
if not isinstance(service_name, str):
|
||||
@@ -4836,6 +5159,7 @@ class DownloaderApp (QWidget ):
|
||||
if self.download_thread:
|
||||
if isinstance(self.download_thread, QThread):
|
||||
try:
|
||||
# Disconnect signals to prevent any lingering connections
|
||||
if hasattr(self.download_thread, 'progress_signal'): self.download_thread.progress_signal.disconnect(self.handle_main_log)
|
||||
if hasattr(self.download_thread, 'add_character_prompt_signal'): self.download_thread.add_character_prompt_signal.disconnect(self.add_character_prompt_signal)
|
||||
if hasattr(self.download_thread, 'finished_signal'): self.download_thread.finished_signal.disconnect(self.download_finished)
|
||||
@@ -4849,9 +5173,8 @@ class DownloaderApp (QWidget ):
|
||||
except (TypeError, RuntimeError) as e:
|
||||
self.log_signal.emit(f"ℹ️ Note during single-thread signal disconnection: {e}")
|
||||
|
||||
if not self.download_thread.isRunning():
|
||||
self.download_thread.deleteLater()
|
||||
self.download_thread = None
|
||||
self.download_thread.deleteLater()
|
||||
self.download_thread = None
|
||||
else:
|
||||
self.download_thread = None
|
||||
|
||||
@@ -5855,6 +6178,325 @@ class DownloaderApp (QWidget ):
|
||||
# Use a QTimer to avoid deep recursion and correctly move to the next item.
|
||||
QTimer.singleShot(100, self._process_next_favorite_download)
|
||||
|
||||
class Saint2DownloadThread(QThread):
|
||||
"""A dedicated QThread for handling saint2.su downloads."""
|
||||
progress_signal = pyqtSignal(str)
|
||||
file_progress_signal = pyqtSignal(str, object)
|
||||
finished_signal = pyqtSignal(int, int, bool) # dl_count, skip_count, cancelled
|
||||
|
||||
def __init__(self, url, output_dir, parent=None):
|
||||
super().__init__(parent)
|
||||
self.saint2_url = url
|
||||
self.output_dir = output_dir
|
||||
self.is_cancelled = False
|
||||
|
||||
def run(self):
|
||||
download_count = 0
|
||||
skip_count = 0
|
||||
self.progress_signal.emit("=" * 40)
|
||||
self.progress_signal.emit(f"🚀 Starting Saint2.su Download for: {self.saint2_url}")
|
||||
|
||||
# Use the new client to get the download info
|
||||
album_name, files_to_download = fetch_saint2_data(self.saint2_url, self.progress_signal.emit)
|
||||
|
||||
if not files_to_download:
|
||||
self.progress_signal.emit("❌ Failed to extract file information from Saint2. Aborting.")
|
||||
self.finished_signal.emit(0, 0, self.is_cancelled)
|
||||
return
|
||||
|
||||
# For single media, album_name is the title; for albums, it's the album title
|
||||
album_path = os.path.join(self.output_dir, album_name)
|
||||
try:
|
||||
os.makedirs(album_path, exist_ok=True)
|
||||
self.progress_signal.emit(f" Saving to folder: '{album_path}'")
|
||||
except OSError as e:
|
||||
self.progress_signal.emit(f"❌ Critical error creating directory: {e}")
|
||||
self.finished_signal.emit(0, len(files_to_download), self.is_cancelled)
|
||||
return
|
||||
|
||||
total_files = len(files_to_download)
|
||||
session = requests.Session()
|
||||
|
||||
for i, file_data in enumerate(files_to_download):
|
||||
if self.is_cancelled:
|
||||
self.progress_signal.emit(" Download cancelled by user.")
|
||||
skip_count = total_files - download_count
|
||||
break
|
||||
|
||||
filename = file_data.get('filename', f'untitled_{i+1}.mp4')
|
||||
file_url = file_data.get('url')
|
||||
headers = file_data.get('headers')
|
||||
filepath = os.path.join(album_path, filename)
|
||||
|
||||
if os.path.exists(filepath):
|
||||
self.progress_signal.emit(f" -> Skip ({i+1}/{total_files}): '{filename}' already exists.")
|
||||
skip_count += 1
|
||||
continue
|
||||
|
||||
self.progress_signal.emit(f" Downloading ({i+1}/{total_files}): '{filename}'...")
|
||||
|
||||
try:
|
||||
response = session.get(file_url, stream=True, headers=headers, timeout=60)
|
||||
response.raise_for_status()
|
||||
|
||||
total_size = int(response.headers.get('content-length', 0))
|
||||
downloaded_size = 0
|
||||
last_update_time = time.time()
|
||||
|
||||
with open(filepath, 'wb') as f:
|
||||
for chunk in response.iter_content(chunk_size=8192):
|
||||
if self.is_cancelled:
|
||||
break
|
||||
if chunk:
|
||||
f.write(chunk)
|
||||
downloaded_size += len(chunk)
|
||||
current_time = time.time()
|
||||
if total_size > 0 and (current_time - last_update_time) > 0.5:
|
||||
self.file_progress_signal.emit(filename, (downloaded_size, total_size))
|
||||
last_update_time = current_time
|
||||
|
||||
if self.is_cancelled:
|
||||
if os.path.exists(filepath): os.remove(filepath)
|
||||
continue
|
||||
|
||||
if total_size > 0:
|
||||
self.file_progress_signal.emit(filename, (total_size, total_size))
|
||||
|
||||
download_count += 1
|
||||
except requests.exceptions.RequestException as e:
|
||||
self.progress_signal.emit(f" ❌ Failed to download '{filename}'. Error: {e}")
|
||||
if os.path.exists(filepath): os.remove(filepath)
|
||||
skip_count += 1
|
||||
except Exception as e:
|
||||
self.progress_signal.emit(f" ❌ An unexpected error occurred with '{filename}': {e}")
|
||||
if os.path.exists(filepath): os.remove(filepath)
|
||||
skip_count += 1
|
||||
|
||||
self.file_progress_signal.emit("", None)
|
||||
self.finished_signal.emit(download_count, skip_count, self.is_cancelled)
|
||||
|
||||
def cancel(self):
|
||||
self.is_cancelled = True
|
||||
self.progress_signal.emit(" Cancellation signal received by Saint2 thread.")
|
||||
|
||||
class EromeDownloadThread(QThread):
|
||||
"""A dedicated QThread for handling erome.com downloads."""
|
||||
progress_signal = pyqtSignal(str)
|
||||
file_progress_signal = pyqtSignal(str, object)
|
||||
finished_signal = pyqtSignal(int, int, bool) # dl_count, skip_count, cancelled
|
||||
|
||||
def __init__(self, url, output_dir, parent=None):
|
||||
super().__init__(parent)
|
||||
self.erome_url = url
|
||||
self.output_dir = output_dir
|
||||
self.is_cancelled = False
|
||||
|
||||
def run(self):
|
||||
download_count = 0
|
||||
skip_count = 0
|
||||
self.progress_signal.emit("=" * 40)
|
||||
self.progress_signal.emit(f"🚀 Starting Erome.com Download for: {self.erome_url}")
|
||||
|
||||
album_name, files_to_download = fetch_erome_data(self.erome_url, self.progress_signal.emit)
|
||||
|
||||
if not files_to_download:
|
||||
self.progress_signal.emit("❌ Failed to extract file information from Erome. Aborting.")
|
||||
self.finished_signal.emit(0, 0, self.is_cancelled)
|
||||
return
|
||||
|
||||
album_path = os.path.join(self.output_dir, album_name)
|
||||
try:
|
||||
os.makedirs(album_path, exist_ok=True)
|
||||
self.progress_signal.emit(f" Saving to folder: '{album_path}'")
|
||||
except OSError as e:
|
||||
self.progress_signal.emit(f"❌ Critical error creating directory: {e}")
|
||||
self.finished_signal.emit(0, len(files_to_download), self.is_cancelled)
|
||||
return
|
||||
|
||||
total_files = len(files_to_download)
|
||||
session = cloudscraper.create_scraper()
|
||||
|
||||
for i, file_data in enumerate(files_to_download):
|
||||
if self.is_cancelled:
|
||||
self.progress_signal.emit(" Download cancelled by user.")
|
||||
skip_count = total_files - download_count
|
||||
break
|
||||
|
||||
filename = file_data.get('filename', f'untitled_{i+1}.mp4')
|
||||
file_url = file_data.get('url')
|
||||
headers = file_data.get('headers')
|
||||
filepath = os.path.join(album_path, filename)
|
||||
|
||||
if os.path.exists(filepath):
|
||||
self.progress_signal.emit(f" -> Skip ({i+1}/{total_files}): '{filename}' already exists.")
|
||||
skip_count += 1
|
||||
continue
|
||||
|
||||
self.progress_signal.emit(f" Downloading ({i+1}/{total_files}): '{filename}'...")
|
||||
|
||||
try:
|
||||
response = session.get(file_url, stream=True, headers=headers, timeout=60)
|
||||
response.raise_for_status()
|
||||
|
||||
total_size = int(response.headers.get('content-length', 0))
|
||||
downloaded_size = 0
|
||||
last_update_time = time.time()
|
||||
|
||||
with open(filepath, 'wb') as f:
|
||||
for chunk in response.iter_content(chunk_size=8192):
|
||||
if self.is_cancelled:
|
||||
break
|
||||
if chunk:
|
||||
f.write(chunk)
|
||||
downloaded_size += len(chunk)
|
||||
current_time = time.time()
|
||||
if total_size > 0 and (current_time - last_update_time) > 0.5:
|
||||
self.file_progress_signal.emit(filename, (downloaded_size, total_size))
|
||||
last_update_time = current_time
|
||||
|
||||
if self.is_cancelled:
|
||||
if os.path.exists(filepath): os.remove(filepath)
|
||||
continue
|
||||
|
||||
if total_size > 0:
|
||||
self.file_progress_signal.emit(filename, (total_size, total_size))
|
||||
|
||||
download_count += 1
|
||||
except requests.exceptions.RequestException as e:
|
||||
self.progress_signal.emit(f" ❌ Failed to download '{filename}'. Error: {e}")
|
||||
if os.path.exists(filepath): os.remove(filepath)
|
||||
skip_count += 1
|
||||
except Exception as e:
|
||||
self.progress_signal.emit(f" ❌ An unexpected error occurred with '{filename}': {e}")
|
||||
if os.path.exists(filepath): os.remove(filepath)
|
||||
skip_count += 1
|
||||
|
||||
self.file_progress_signal.emit("", None)
|
||||
self.finished_signal.emit(download_count, skip_count, self.is_cancelled)
|
||||
|
||||
def cancel(self):
|
||||
self.is_cancelled = True
|
||||
self.progress_signal.emit(" Cancellation signal received by Erome thread.")
|
||||
|
||||
class BunkrDownloadThread(QThread):
|
||||
"""A dedicated QThread for handling Bunkr downloads."""
|
||||
progress_signal = pyqtSignal(str)
|
||||
# --- ADD THIS SIGNAL for detailed file progress ---
|
||||
file_progress_signal = pyqtSignal(str, object)
|
||||
finished_signal = pyqtSignal(int, int, bool, list)
|
||||
|
||||
def __init__(self, url, output_dir, parent=None):
|
||||
super().__init__(parent)
|
||||
self.bunkr_url = url
|
||||
self.output_dir = output_dir
|
||||
self.is_cancelled = False
|
||||
|
||||
class ThreadLogger:
|
||||
def __init__(self, signal_emitter):
|
||||
self.signal_emitter = signal_emitter
|
||||
def info(self, msg, *args, **kwargs):
|
||||
self.signal_emitter.emit(str(msg))
|
||||
def error(self, msg, *args, **kwargs):
|
||||
self.signal_emitter.emit(f"❌ ERROR: {msg}")
|
||||
def warning(self, msg, *args, **kwargs):
|
||||
self.signal_emitter.emit(f"⚠️ WARNING: {msg}")
|
||||
def debug(self, msg, *args, **kwargs):
|
||||
pass
|
||||
|
||||
self.logger = ThreadLogger(self.progress_signal)
|
||||
|
||||
def run(self):
|
||||
download_count = 0
|
||||
skip_count = 0
|
||||
self.progress_signal.emit("=" * 40)
|
||||
self.progress_signal.emit(f"🚀 Starting Bunkr Download for: {self.bunkr_url}")
|
||||
|
||||
album_name, files_to_download = fetch_bunkr_data(self.bunkr_url, self.logger)
|
||||
|
||||
if not files_to_download:
|
||||
self.progress_signal.emit("❌ Failed to extract file information from Bunkr. Aborting.")
|
||||
self.finished_signal.emit(0, 0, self.is_cancelled, [])
|
||||
return
|
||||
|
||||
album_path = os.path.join(self.output_dir, album_name)
|
||||
try:
|
||||
os.makedirs(album_path, exist_ok=True)
|
||||
self.progress_signal.emit(f" Saving to folder: '{album_path}'")
|
||||
except OSError as e:
|
||||
self.progress_signal.emit(f"❌ Critical error creating directory: {e}")
|
||||
self.finished_signal.emit(0, len(files_to_download), self.is_cancelled, [])
|
||||
return
|
||||
|
||||
total_files = len(files_to_download)
|
||||
for i, file_data in enumerate(files_to_download):
|
||||
if self.is_cancelled:
|
||||
self.progress_signal.emit(" Download cancelled by user.")
|
||||
skip_count = total_files - download_count
|
||||
break
|
||||
|
||||
filename = file_data.get('name', 'untitled_file')
|
||||
file_url = file_data.get('url')
|
||||
headers = file_data.get('_http_headers')
|
||||
|
||||
filename = re.sub(r'[<>:"/\\|?*]', '_', filename).strip()
|
||||
filepath = os.path.join(album_path, filename)
|
||||
|
||||
if os.path.exists(filepath):
|
||||
self.progress_signal.emit(f" -> Skip ({i+1}/{total_files}): '{filename}' already exists.")
|
||||
skip_count += 1
|
||||
continue
|
||||
|
||||
self.progress_signal.emit(f" Downloading ({i+1}/{total_files}): '{filename}'...")
|
||||
|
||||
try:
|
||||
response = requests.get(file_url, stream=True, headers=headers, timeout=60)
|
||||
response.raise_for_status()
|
||||
|
||||
# --- MODIFY THIS BLOCK to calculate and emit progress ---
|
||||
total_size = int(response.headers.get('content-length', 0))
|
||||
downloaded_size = 0
|
||||
last_update_time = time.time()
|
||||
|
||||
with open(filepath, 'wb') as f:
|
||||
for chunk in response.iter_content(chunk_size=8192):
|
||||
if self.is_cancelled:
|
||||
break
|
||||
if chunk:
|
||||
f.write(chunk)
|
||||
downloaded_size += len(chunk)
|
||||
current_time = time.time()
|
||||
if total_size > 0 and (current_time - last_update_time) > 0.5:
|
||||
self.file_progress_signal.emit(filename, (downloaded_size, total_size))
|
||||
last_update_time = current_time
|
||||
|
||||
if self.is_cancelled:
|
||||
if os.path.exists(filepath): os.remove(filepath)
|
||||
continue
|
||||
|
||||
# Emit final progress to show 100%
|
||||
if total_size > 0:
|
||||
self.file_progress_signal.emit(filename, (total_size, total_size))
|
||||
|
||||
download_count += 1
|
||||
# --- END MODIFICATION ---
|
||||
|
||||
except requests.exceptions.RequestException as e:
|
||||
self.progress_signal.emit(f" ❌ Failed to download '{filename}'. Error: {e}")
|
||||
if os.path.exists(filepath): os.remove(filepath)
|
||||
skip_count += 1
|
||||
except Exception as e:
|
||||
self.progress_signal.emit(f" ❌ An unexpected error occurred with '{filename}': {e}")
|
||||
if os.path.exists(filepath): os.remove(filepath)
|
||||
skip_count += 1
|
||||
|
||||
# Clear the progress label when finished
|
||||
self.file_progress_signal.emit("", None)
|
||||
self.finished_signal.emit(download_count, skip_count, self.is_cancelled, [])
|
||||
|
||||
def cancel(self):
|
||||
self.is_cancelled = True
|
||||
self.progress_signal.emit(" Cancellation signal received by Bunkr thread.")
|
||||
|
||||
class ExternalLinkDownloadThread (QThread ):
|
||||
"""A QThread to handle downloading multiple external links sequentially."""
|
||||
progress_signal =pyqtSignal (str )
|
||||
@@ -5911,4 +6553,104 @@ class ExternalLinkDownloadThread (QThread ):
|
||||
self .finished_signal .emit ()
|
||||
|
||||
def cancel (self ):
|
||||
self .is_cancelled =True
|
||||
self .is_cancelled =True
|
||||
|
||||
class NhentaiDownloadThread(QThread):
|
||||
progress_signal = pyqtSignal(str)
|
||||
finished_signal = pyqtSignal(int, int, bool)
|
||||
|
||||
IMAGE_SERVERS = [
|
||||
"https://i.nhentai.net", "https://i2.nhentai.net", "https://i3.nhentai.net",
|
||||
"https://i5.nhentai.net", "https://i7.nhentai.net"
|
||||
]
|
||||
|
||||
EXTENSION_MAP = {'j': 'jpg', 'p': 'png', 'g': 'gif', 'w': 'webp' }
|
||||
|
||||
def __init__(self, gallery_data, output_dir, parent=None):
|
||||
super().__init__(parent)
|
||||
self.gallery_data = gallery_data
|
||||
self.output_dir = output_dir
|
||||
self.is_cancelled = False
|
||||
|
||||
def run(self):
|
||||
title = self.gallery_data.get("title", {}).get("english", f"gallery_{self.gallery_data.get('id')}")
|
||||
gallery_id = self.gallery_data.get("id")
|
||||
media_id = self.gallery_data.get("media_id")
|
||||
pages_info = self.gallery_data.get("pages", [])
|
||||
|
||||
folder_name = clean_folder_name(title)
|
||||
gallery_path = os.path.join(self.output_dir, folder_name)
|
||||
|
||||
try:
|
||||
os.makedirs(gallery_path, exist_ok=True)
|
||||
except OSError as e:
|
||||
self.progress_signal.emit(f"❌ Critical error creating directory: {e}")
|
||||
self.finished_signal.emit(0, len(pages_info), False)
|
||||
return
|
||||
|
||||
self.progress_signal.emit(f"⬇️ Downloading '{title}' to folder '{folder_name}'...")
|
||||
|
||||
# Create a single cloudscraper instance for the entire download
|
||||
scraper = cloudscraper.create_scraper()
|
||||
download_count = 0
|
||||
skip_count = 0
|
||||
|
||||
for i, page_data in enumerate(pages_info):
|
||||
if self.is_cancelled:
|
||||
break
|
||||
|
||||
page_num = i + 1
|
||||
|
||||
ext_char = page_data.get('t', 'j')
|
||||
extension = self.EXTENSION_MAP.get(ext_char, 'jpg')
|
||||
|
||||
relative_path = f"/galleries/{media_id}/{page_num}.{extension}"
|
||||
|
||||
local_filename = f"{page_num:03d}.{extension}"
|
||||
filepath = os.path.join(gallery_path, local_filename)
|
||||
|
||||
if os.path.exists(filepath):
|
||||
self.progress_signal.emit(f" -> Skip (Exists): {local_filename}")
|
||||
skip_count += 1
|
||||
continue
|
||||
|
||||
download_successful = False
|
||||
for server in self.IMAGE_SERVERS:
|
||||
if self.is_cancelled:
|
||||
break
|
||||
|
||||
full_url = f"{server}{relative_path}"
|
||||
try:
|
||||
self.progress_signal.emit(f" Downloading page {page_num}/{len(pages_info)} from {server} ...")
|
||||
|
||||
headers = {
|
||||
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36',
|
||||
'Referer': f'https://nhentai.net/g/{gallery_id}/'
|
||||
}
|
||||
|
||||
# Use the scraper instance to get the image
|
||||
response = scraper.get(full_url, headers=headers, timeout=60, stream=True)
|
||||
|
||||
if response.status_code == 200:
|
||||
with open(filepath, 'wb') as f:
|
||||
for chunk in response.iter_content(chunk_size=8192):
|
||||
f.write(chunk)
|
||||
download_count += 1
|
||||
download_successful = True
|
||||
break
|
||||
else:
|
||||
self.progress_signal.emit(f" -> {server} returned status {response.status_code}. Trying next server...")
|
||||
|
||||
except Exception as e:
|
||||
self.progress_signal.emit(f" -> {server} failed to connect or timed out: {e}. Trying next server...")
|
||||
|
||||
if not download_successful:
|
||||
self.progress_signal.emit(f" ❌ Failed to download {local_filename} from all servers.")
|
||||
skip_count += 1
|
||||
|
||||
time.sleep(0.5)
|
||||
|
||||
self.finished_signal.emit(download_count, skip_count, self.is_cancelled)
|
||||
|
||||
def cancel(self):
|
||||
self.is_cancelled = True
|
||||
|
||||
@@ -138,42 +138,54 @@ def prepare_cookies_for_request(use_cookie_flag, cookie_text_input, selected_coo
|
||||
return None
|
||||
|
||||
|
||||
# In src/utils/network_utils.py
|
||||
|
||||
def extract_post_info(url_string):
|
||||
"""
|
||||
Parses a URL string to extract the service, user ID, and post ID.
|
||||
UPDATED to support Discord server/channel URLs.
|
||||
UPDATED to support Discord, Bunkr, and nhentai URLs.
|
||||
|
||||
Args:
|
||||
url_string (str): The URL to parse.
|
||||
|
||||
Returns:
|
||||
tuple: A tuple containing (service, id1, id2).
|
||||
tuple: A tuple containing (service, id1, id2).
|
||||
For posts: (service, user_id, post_id).
|
||||
For Discord: ('discord', server_id, channel_id).
|
||||
For Bunkr: ('bunkr', full_url, None).
|
||||
For nhentai: ('nhentai', gallery_id, None).
|
||||
"""
|
||||
if not isinstance(url_string, str) or not url_string.strip():
|
||||
return None, None, None
|
||||
|
||||
try:
|
||||
parsed_url = urlparse(url_string.strip())
|
||||
path_parts = [part for part in parsed_url.path.strip('/').split('/') if part]
|
||||
|
||||
# Check for new Discord URL format first
|
||||
# e.g., /discord/server/891670433978531850/1252332668805189723
|
||||
if len(path_parts) >= 3 and path_parts[0].lower() == 'discord' and path_parts[1].lower() == 'server':
|
||||
service = 'discord'
|
||||
server_id = path_parts[2]
|
||||
channel_id = path_parts[3] if len(path_parts) >= 4 else None
|
||||
return service, server_id, channel_id
|
||||
|
||||
# Standard creator/post format: /<service>/user/<user_id>/post/<post_id>
|
||||
stripped_url = url_string.strip()
|
||||
|
||||
# --- Bunkr Check ---
|
||||
bunkr_pattern = re.compile(
|
||||
r"(?:https?://)?(?:[a-zA-Z0-9-]+\.)?bunkr\.(?:si|la|ws|red|black|media|site|is|to|ac|cr|ci|fi|pk|ps|sk|ph|su|ru)|bunkrr\.ru"
|
||||
)
|
||||
if bunkr_pattern.search(stripped_url):
|
||||
return 'bunkr', stripped_url, None
|
||||
|
||||
# --- nhentai Check ---
|
||||
nhentai_match = re.search(r'nhentai\.net/g/(\d+)', stripped_url)
|
||||
if nhentai_match:
|
||||
return 'nhentai', nhentai_match.group(1), None
|
||||
|
||||
# --- Kemono/Coomer/Discord Parsing ---
|
||||
try:
|
||||
parsed_url = urlparse(stripped_url)
|
||||
path_parts = [part for part in parsed_url.path.strip('/').split('/') if part]
|
||||
|
||||
if len(path_parts) >= 3 and path_parts[0].lower() == 'discord' and path_parts[1].lower() == 'server':
|
||||
return 'discord', path_parts[2], path_parts[3] if len(path_parts) >= 4 else None
|
||||
|
||||
if len(path_parts) >= 3 and path_parts[1].lower() == 'user':
|
||||
service = path_parts[0]
|
||||
user_id = path_parts[2]
|
||||
post_id = path_parts[4] if len(path_parts) >= 5 and path_parts[3].lower() == 'post' else None
|
||||
return service, user_id, post_id
|
||||
|
||||
# API format: /api/v1/<service>/user/<user_id>...
|
||||
|
||||
if len(path_parts) >= 5 and path_parts[0:2] == ['api', 'v1'] and path_parts[3].lower() == 'user':
|
||||
service = path_parts[2]
|
||||
user_id = path_parts[4]
|
||||
@@ -184,7 +196,7 @@ def extract_post_info(url_string):
|
||||
print(f"Debug: Exception during URL parsing for '{url_string}': {e}")
|
||||
|
||||
return None, None, None
|
||||
|
||||
|
||||
def get_link_platform(url):
|
||||
"""
|
||||
Identifies the platform of a given URL based on its domain.
|
||||
|
||||
Reference in New Issue
Block a user