1 Commits

Author SHA1 Message Date
Yuvi9587
56a83195b2 Update readme.md 2025-08-11 09:31:53 -07:00
23 changed files with 1058 additions and 3245 deletions

View File

@@ -1,159 +1,391 @@
<h1>Kemono Downloader - Comprehensive Feature Guide</h1> <div>
<p>This guide provides a detailed overview of all user interface elements, input fields, buttons, popups, and functionalities available in the application.</p> <h1>Kemono Downloader - Comprehensive Feature Guide</h1>
<hr> <p>This guide provides a detailed overview of all user interface elements, input fields, buttons, popups, and functionalities available in the application.</p>
<h2>1. Core Concepts &amp; Supported Sites</h2> <hr>
<h3>URL Input (🔗)</h3>
<p>This is the primary input field where you specify the content you want to download.</p> <h2><strong>1. URL Input (🔗)</strong></h2>
<p><strong>Supported URL Types:</strong></p> <p>This is the primary input field where you specify the content you want to download.</p>
<p><strong>Functionality:</strong></p>
<ul>
<li><strong>Creator URL:</strong> A link to a creator's main page (e.g., https://kemono.su/patreon/user/12345). Downloads all posts from the creator.</li>
<li><strong>Post URL:</strong> A direct link to a specific post (e.g., .../post/98765). Downloads only the specified post.</li>
</ul>
<p><strong>Interaction with Other Features:</strong> The content of this field influences "Manga Mode" and "Page Range". "Page Range" is enabled only with a creator URL.</p>
<hr>
<h2><strong>2. Creator Selection & Update (🎨)</strong></h2>
<p>The color palette emoji button opens the Creator Selection & Update dialog. This allows managing and downloading from a local creator database.</p>
<p><strong>Functionality:</strong></p>
<ul>
<li><strong>Creator Browser:</strong> Loads a list from <code>creators.json</code>. Search by name, service, or paste a URL to find creators.</li>
<li><strong>Batch Selection:</strong> Select multiple creators and click "Add Selected" to add them to the batch download session.</li>
<li><strong>Update Checker:</strong> Use a saved profile (.json) to download only new content based on previously fetched posts.</li>
<li><strong>Post Fetching & Filtering:</strong> "Fetch Posts" loads post titles, allowing you to choose specific posts for download.</li>
</ul>
<hr>
<h2><strong>3. Download Location Input (📁)</strong></h2>
<p>This input defines the destination directory for downloaded files.</p>
<p><strong>Functionality:</strong></p>
<ul>
<li><strong>Manual Entry:</strong> Enter or paste the folder path.</li>
<li><strong>Browse Button:</strong> Opens a system dialog to choose a folder.</li>
<li><strong>Directory Creation:</strong> If the folder doesn't exist, the app can create it after user confirmation.</li>
</ul>
<hr>
<h2><strong>4. Filter by Character(s) & Scope Button</strong></h2>
<p>Used to download content for specific characters or series and organize them into subfolders.</p>
<p><strong>Input Field (Filter by Character(s)):</strong></p>
<ul>
<li>Enter comma-separated names (e.g., <code>Tifa, Aerith</code>).</li>
<li>Group aliases using parentheses (e.g., <code>(Cloud, Zack)</code>).</li>
<li>Names are matched against titles, filenames, or comments.</li>
<li>If "Separate Folders by Known.txt" is enabled, the name becomes the subfolder name.</li>
</ul>
<p><strong>Scope Button Modes:</strong></p>
<ul>
<li><strong>Filter: Title</strong> (default) Match names in post titles only.</li>
<li><strong>Filter: Files</strong> Match names in filenames only.</li>
<li><strong>Filter: Both</strong> Try title match first, then filenames.</li>
<li><strong>Filter: Comments</strong> Try filenames first, then post comments if no match.</li>
</ul>
<hr>
<h2><strong>5. Skip with Words & Scope Button</strong></h2>
<p>Prevents downloading content based on keywords.</p>
<p><strong>Input Field (Skip with Words):</strong></p>
<ul>
<li>Enter comma-separated keywords (e.g., <code>WIP, sketch, preview</code>).</li>
<li>Matching is case-insensitive.</li>
<li>If a keyword matches, the file or post is skipped.</li>
</ul>
<p><strong>Scope Button Modes:</strong></p>
<ul>
<li><strong>Scope: Posts</strong> (default) Skips post if title contains a keyword.</li>
<li><strong>Scope: Files</strong> Skips individual files with keyword matches.</li>
<li><strong>Scope: Both</strong> Skips entire post if title matches, otherwise filters individual files.</li>
</ul>
</div>
<div>
<h2><strong>Filter File Section (Radio Buttons)</strong></h2>
<p>This section uses a group of radio buttons to control the primary download mode, dictating which types of files are targeted. Only one of these modes can be active at a time.</p>
<ul>
<li>
<strong>All:</strong> Default mode. Downloads every file and attachment provided by the API, regardless of type.
</li>
<li>
<strong>Images/GIFs:</strong> Filters for common image formats (<code>.jpg</code>, <code>.png</code>, <code>.gif</code>, <code>.webp</code>), skipping non-image files.
</li>
<li>
<strong>Videos:</strong> Filters for common video formats like <code>.mp4</code>, <code>.webm</code>, and <code>.mov</code>, skipping all others.
</li>
<li>
<strong>Only Archives:</strong> Downloads only archive files (<code>.zip</code>, <code>.rar</code>). Disables "Compress to WebP" and unchecks "Skip Archives".
</li>
<li>
<strong>Only Audio:</strong> Filters for common audio formats like <code>.mp3</code>, <code>.wav</code>, and <code>.flac</code>.
</li>
<li>
<strong>Only Links:</strong> Extracts external hyperlinks from post descriptions (e.g., Mega, Google Drive) and displays them in the log. Disables all download options.
</li>
<li>
<strong>More:</strong> Opens the "More Options" dialog to download text-based content instead of media files.
<ul>
<li><strong>Scope:</strong> Choose to extract from post description or comments.</li>
<li><strong>Export Format:</strong> Save text as PDF, DOCX, or TXT.</li>
<li><strong>Single PDF:</strong> Optionally compile all text into one PDF.</li>
</ul>
</li>
</ul>
<hr>
<h2><strong>Check Box Buttons</strong></h2>
<p>These checkboxes provide additional toggles to refine the download behavior and enable special features.</p>
<ul>
<li>
<strong>⭐ Favorite Mode:</strong> Changes workflow to download from your personal favorites. Disables the URL input.
<ul>
<li><strong>Favorite Artists:</strong> Opens a dialog to select from your favorited creators.</li>
<li><strong>Favorite Posts:</strong> Opens a dialog to select from your favorited posts on Kemono and Coomer.</li>
</ul>
</li>
<li>
<strong>Skip Archives:</strong> When checked, archive files (<code>.zip</code>, <code>.rar</code>) are ignored. Disabled in "Only Archives" mode.
</li>
<li>
<strong>Download Thumbnail Only:</strong> Saves only thumbnail previews, not full-resolution files. Enables "Scan Content for Images".
</li>
<li>
<strong>Scan Content for Images:</strong> Parses post HTML for embedded images not listed in the API. Looks for <code>&lt;img&gt;</code> tags and direct image links.
</li>
<li>
<strong>Compress to WebP:</strong> Converts large images (over 1.5 MB) to WebP format using the Pillow library for space-saving.
</li>
<li>
<strong>Keep Duplicates:</strong> Provides control over duplicate handling via the "Duplicate Handling Options" dialog.
<ul>
<li><strong>Skip by Hash:</strong> Default skip identical files.</li>
<li><strong>Keep Everything:</strong> Save all files regardless of duplication.</li>
<li><strong>Limit:</strong> Set a limit on how many copies of the same file are saved. A limit of <code>0</code> means no limit.</li>
</ul>
</li>
</ul>
</div>
<h2><strong>Folder Organization Checkboxes</strong></h2>
<ul> <ul>
<li><strong>Creator URL</strong>: A link to a creator's main page. Downloads all posts from that creator.</li> <li>
<li><strong>Post URL</strong>: A direct link to a specific post. Downloads only that single post.</li> <strong>Separate folders by Known.txt:</strong> Automatically organizes downloads into folders based on name matches.
<li><strong>Batch Command</strong>: Special keywords to trigger bulk downloading from a text file (see Batch Downloading section).</li>
</ul>
<p><strong>Supported Websites:</strong></p>
<ul>
<li>Kemono (<code>kemono.su</code>, <code>kemono.party</code>, etc.)</li>
<li>Coomer (<code>coomer.su</code>, <code>coomer.party</code>, etc.)</li>
<li>Discord (via Kemono/Coomer API)</li>
<li>Bunkr</li>
<li>Erome</li>
<li>Saint2.su</li>
<li>nhentai</li>
</ul>
<hr>
<h2>2. Main Download Controls &amp; Inputs</h2>
<h3>Download Location (📁)</h3>
<p>This input defines the main folder where your files will be saved.</p>
<ul>
<li><strong>Browse Button</strong>: Opens a system dialog to choose a folder.</li>
<li><strong>Directory Creation</strong>: If the folder doesn't exist, the app will ask for confirmation to create it.</li>
</ul>
<h3>Filter by Character(s) &amp; Scope</h3>
<p>Used to download content for specific characters or series and organize them into subfolders.</p>
<ul>
<li><strong>Input Field</strong>: Enter comma-separated names (e.g., <code>Tifa, Aerith</code>). Group aliases using parentheses for folder naming (e.g., <code>(Cloud, Zack)</code>).</li>
<li><strong>Scope Button</strong>: Cycles through where to look for name matches:
<ul> <ul>
<li><strong>Filter: Title</strong>: Matches names in the post title.</li> <li>Uses "Filter by Character(s)" input first, if available.</li>
<li><strong>Filter: Files</strong>: Matches names in the filenames.</li> <li>Then checks names in <code>Known.txt</code>.</li>
<li><strong>Filter: Both</strong>: Checks the title first, then filenames.</li> <li>Falls back to extracting from post title.</li>
<li><strong>Filter: Comments</strong>: Checks filenames first, then post comments.</li> </ul>
</li>
<li>
<strong>Subfolder per post:</strong> Creates a unique folder per post, using the posts title.
<ul>
<li>Prevents mixing files from multiple posts.</li>
<li>Can be combined with Known.txt-based folders.</li>
<li>Ensures uniqueness (e.g., <code>My Post Title_1</code>).</li>
<li>Automatically removes empty folders.</li>
</ul>
</li>
<li>
<strong>Date prefix:</strong> Enabled only with "Subfolder per post". Prepends the post date (e.g., <code>2025-08-03 My Post Title</code>) for chronological sorting.
</li>
</ul>
<h2><strong>General Functionality Checkboxes</strong></h2>
<ul>
<li>
<strong>Use cookie:</strong> Enables login-based access via cookies.
<ul>
<li>Paste cookie string directly, or browse to select a <code>cookies.txt</code> file.</li>
<li>Cookies are used in all authenticated API requests.</li>
</ul>
</li>
<li>
<strong>Use Multithreading:</strong> Enables parallel downloading of posts.
<ul>
<li>Specify the number of worker threads (e.g., 10).</li>
<li>Disabled for Manga Mode and Only Links mode.</li>
</ul>
</li>
<li>
<strong>Show external links in log:</strong> Adds a secondary log that displays links (e.g., Mega, Dropbox) found in post text.
</li>
<li>
<strong>Manga/Comic mode:</strong> Sorts posts chronologically before download.
<ul>
<li>Ensures correct page order for comics/manga.</li>
</ul>
<strong>Scope Button (Name: ...):</strong> Controls filename style:
<ul>
<li><strong>Name: Post Title</strong> — e.g., <code>Chapter-1.jpg</code></li>
<li><strong>Name: Date + Original</strong> — e.g., <code>2025-08-03_filename.png</code></li>
<li><strong>Name: Date + Title</strong> — e.g., <code>2025-08-03_Chapter-1.jpg</code></li>
<li><strong>Name: Title+G.Num</strong> — e.g., <code>Page_001.jpg</code></li>
<li><strong>Name: Date Based</strong> — e.g., <code>001.jpg</code>, with optional prefix</li>
<li><strong>Name: Post ID</strong> — uses unique post ID as filename</li>
</ul> </ul>
</li> </li>
</ul> </ul>
<h3>Skip with Words &amp; Scope</h3> <h2><strong>Start Download</strong></h2>
<p>Prevents downloading content based on keywords or file size.</p>
<ul> <ul>
<li><strong>Input Field</strong>: Enter comma-separated keywords (e.g., <code>WIP, sketch, preview</code>).</li> <li>
<li><strong>Skip by Size</strong>: Enter a number in square brackets to skip any file <strong>smaller than</strong> that size in MB. For example, <code>WIP, [200]</code> skips files with "WIP" in the name AND any file smaller than 200 MB.</li> <strong>Default State ("⬇️ Start Download"):</strong> When idle, this button gathers all current settings (URL, filters, checkboxes, etc.) and begins the download process via the DownloadManager.
<li><strong>Scope Button</strong>: Cycles through where to apply keyword filters: </li>
<li>
<strong>Restore State:</strong> If an interrupted session is detected, the tooltip will indicate that starting a new download will discard previous session progress.
</li>
<li>
<strong>Update Mode (Phase 1 - "🔄 Check For Updates"):</strong> If a creator profile is loaded, clicking this button will fetch the creator's posts and compare them against your saved profile to identify new content.
</li>
<li>
<strong>Update Mode (Phase 2 - "⬇️ Start Download (X new)"):</strong> After new posts are found, the button text updates to reflect the number. Clicking it downloads only the new content.
</li>
</ul>
<h2><strong>Pause / Resume Download</strong></h2>
<ul>
<li>
<strong>While Downloading:</strong> The button toggles between:
<ul> <ul>
<li><strong>Scope: Posts</strong>: Skips the entire post if the title matches.</li> <li><strong>"⏸️ Pause Download":</strong> Sets a <code>pause_event</code>, which tells all worker threads to halt their current task and wait.</li>
<li><strong>Scope: Files</strong>: Skips individual files if the filename matches.</li> <li><strong>"▶️ Resume Download":</strong> Clears the <code>pause_event</code>, allowing threads to resume their work.</li>
<li><strong>Scope: Both</strong>: Checks the post title first, then individual files.</li> </ul>
</li>
<li>
<strong>While Idle:</strong> The button is disabled.
</li>
<li>
<strong>Restore State:</strong> Changes to "🔄 Restore Download", which resumes the last session from saved data.
</li>
</ul>
<h2><strong>Cancel & Reset UI</strong></h2>
<ul>
<li>
<strong>Functionality:</strong> Stops downloads gracefully using a <code>cancellation_event</code>. Threads finish current tasks before shutting down.
</li>
<li>
<strong>The Soft Reset:</strong> After cancellation is confirmed by background threads, the UI resets via the <code>download_finished</code> function. Input fields (URL and Download Location) are preserved for convenience.
</li>
<li>
<strong>Restore State:</strong> Changes to "🗑️ Discard Session", which deletes <code>session.json</code> and resets the UI.
</li>
<li>
<strong>Update State:</strong> Changes to "🗑️ Clear Selection", unloading the selected creator profile and returning to normal UI state.
</li>
</ul>
<h2><strong>Error Button</strong></h2>
<ul>
<li>
<strong>Error Counter:</strong> Shows how many files failed to download (e.g., <code>(3) Error</code>). Disabled if there are no errors.
</li>
<li>
<strong>Error Dialog:</strong> Clicking opens the "Files Skipped Due to Errors" dialog (defined in <code>ErrorFilesDialog.py</code>), listing all failed files.
</li>
<li>
<strong>Dialog Features:</strong>
<ul>
<li><strong>View Failed Files:</strong> Shows filenames and related post info.</li>
<li><strong>Select and Retry:</strong> Retry selected failed files in a focused download session.</li>
<li><strong>Export URLs:</strong> Save a <code>.txt</code> file of direct download links. Optionally include post metadata with each URL.</li>
</ul> </ul>
</li> </li>
</ul> </ul>
<h3>Remove Words from Name (✂️)</h3> <h2><strong>"Known Area" and its Controls</strong></h2>
<p>Enter comma-separated words to remove from final filenames (e.g., <code>patreon, [HD]</code>). This helps clean up file naming.</p> <p>This section, located on the right side of the main window, manages your personal name database (<code>Known.txt</code>), which the app uses to organize downloads into subfolders.</p>
<hr>
<h2>3. Primary Download Modes (Filter File Section)</h2>
<p>This section uses radio buttons to set the main download mode. Only one can be active at a time.</p>
<ul> <ul>
<li><strong>All</strong>: Default mode. Downloads every file and attachment.</li> <li>
<li><strong>Images/GIFs</strong>: Downloads only common image formats.</li> <strong>Open Known.txt:</strong> Opens the <code>Known.txt</code> file in your system's default text editor for manual editing, such as bulk changes or cleanup.
<li><strong>Videos</strong>: Downloads only common video formats.</li> </li>
<li><strong>Only Archives</strong>: Downloads only <code>.zip</code>, <code>.rar</code>, etc.</li> <li>
<li><strong>Only Audio</strong>: Downloads only common audio formats.</li> <strong>Search character input:</strong> A live search filter that hides any list items not matching your input text. Useful for quickly locating specific names in large lists.
<li><strong>Only Links</strong>: Extracts external hyperlinks (e.g., Mega, Google Drive) from post descriptions instead of downloading files. <strong>This mode unlocks special features</strong> (see section 6).</li> </li>
<li><strong>More</strong>: Opens a dialog to download text-based content. <li>
<strong>Known Series/Characters Area:</strong> Displays all names currently stored in your <code>Known.txt</code>. These names are used when "Separate folders by Known.txt" is enabled.
</li>
<li>
<strong>Input at bottom & Add button:</strong> Type a new character or series name into the input field, then click " Add". The app checks for duplicates, updates the list, and saves to <code>Known.txt</code>.
</li>
<li>
<strong>Add to Filter:</strong> Opens a dialog showing all entries from <code>Known.txt</code> with checkboxes. You can select one or more to auto-fill the "Filter by Character(s)" field at the top of the app.
</li>
<li>
<strong>Delete Selected:</strong> Select one or more entries from the list and click "🗑️ Delete Selected" to remove them from the app and update <code>Known.txt</code> accordingly.
</li>
</ul>
<h2><strong>Other Buttons</strong></h2>
<ul>
<li>
<strong>(?_?) mark button (Help Guide):</strong> Opens a multi-page help dialog with step-by-step instructions and explanations for all app features. Useful for new users.
</li>
<li>
<strong>History Button:</strong> Opens the Download History dialog (from <code>DownloadHistoryDialog.py</code>), showing:
<ul> <ul>
<li><strong>Scope</strong>: Choose to extract text from the post description or comments.</li> <li>Recently downloaded files</li>
<li><strong>Export Format</strong>: Save as PDF, DOCX, or TXT.</li> <li>The first few posts processed in the last session</li>
<li><strong>Single PDF</strong>: Compile all text from the session into one consolidated PDF file.</li> </ul>
This allows for a quick review of recent activity.
</li>
<li>
<strong>Settings Button:</strong> Opens the Settings dialog (from <code>FutureSettingsDialog.py</code>), where you can change app-wide settings such as theme (light/dark) and language.
</li>
<li>
<strong>Support Button:</strong> Opens the Support dialog (from <code>SupportDialog.py</code>), which includes developer info, source links, and donation platforms like Ko-fi or Patreon.
</li>
</ul>
<h2><strong>Log Area Controls</strong></h2>
<p>These controls are located around the main log panel and offer tools for managing downloads, configuring advanced options, and resetting the application.</p>
<ul>
<li>
<strong>Multi-part: OFF</strong><br>
This button acts as both a status indicator and a configuration panel for multi-part downloading (parallel downloading of large files).
<ul>
<li><strong>Function:</strong> Opens the <code>Multipart Download Options</code> dialog (defined in <code>MultipartScopeDialog.py</code>).</li>
<li><strong>Scope Options:</strong> Choose between "Videos Only", "Archives Only", or "Both".</li>
<li><strong>Number of parts:</strong> Set how many simultaneous connections to use (216).</li>
<li><strong>Minimum file size:</strong> Set a threshold (MB) below which files are downloaded normally.</li>
<li><strong>Status:</strong> After applying settings, the button's text updates (e.g., <code>Multi-part: Both</code>); otherwise, it resets to <code>Multi-part: OFF</code>.</li>
</ul>
</li>
<li>
<strong>👁️ Eye Emoji Button (Log View Toggle)</strong><br>
Switches between two views in the log panel:
<ul>
<li><strong>👁️ Progress Log View:</strong> Shows real-time download progress, status messages, and errors.</li>
<li><strong>🚫 Missed Character View:</strong> Displays names detected in posts that didnt match the current filter — useful for updating <code>Known.txt</code>.</li>
</ul>
</li>
<li>
<strong>Reset Button</strong><br>
Performs a full "soft reset" of the UI when the application is idle.
<ul>
<li>Clears all inputs (except saved Download Location)</li>
<li>Resets checkboxes, buttons, and logs</li>
<li>Clears counters, queues, and restores the UI to its default state</li>
<li><strong>Note:</strong> This is different from <em>Cancel & Reset UI</em>, which halts active downloads</li>
</ul> </ul>
</li> </li>
</ul> </ul>
<hr>
<h2>4. Advanced Features &amp; Toggles (Checkboxes)</h2> <h3><strong>The Progress Log and "Only Links" Mode Controls</strong></h3>
<h3>Folder Organization</h3>
<ul> <ul>
<li><strong>Separate folders by Known.txt</strong>: Automatically organizes downloads into subfolders based on name matches from your <code>Known.txt</code> list or the "Filter by Character(s)" input.</li> <li>
<li><strong>Subfolder per post</strong>: Creates a unique folder for each post, named after the post's title. This prevents files from different posts from mixing.</li> <strong>Standard Mode (Progress Log)</strong><br>
<li><strong>Date prefix</strong>: (Only available with "Subfolder per post") Prepends the post date to the folder name (e.g., <code>2025-08-03 My Post Title</code>) for chronological sorting.</li> This is the default behavior. The <code>main_log_output</code> field displays:
</ul>
<h3>Special Modes</h3>
<ul>
<li><strong>⭐ Favorite Mode</strong>: Switches the UI to download from your personal favorites list instead of using the URL input.</li>
<li><strong>Manga/Comic mode</strong>: Sorts a creator's posts from oldest to newest before downloading, ensuring correct page order. A scope button appears to control the filename style (e.g., using post title, date, or a global number).</li>
</ul>
<h3>File Handling</h3>
<ul>
<li><strong>Skip Archives</strong>: Ignores <code>.zip</code> and <code>.rar</code> files during downloads.</li>
<li><strong>Download Thumbnail Only</strong>: Saves only the small preview images instead of full-resolution files.</li>
<li><strong>Scan Content for Images</strong>: Parses post HTML to find embedded images that may not be listed in the API data.</li>
<li><strong>Compress to WebP</strong>: Converts large images (over 1.5 MB) to the space-saving WebP format.</li>
<li><strong>Keep Duplicates</strong>: Opens a dialog to control how duplicate files are handled (skip by default, keep all, or keep a specific number of copies).</li>
</ul>
<h3>General Functionality</h3>
<ul>
<li><strong>Use cookie</strong>: Enables login-based access. You can paste a cookie string or browse for a <code>cookies.txt</code> file.</li>
<li><strong>Use Multithreading</strong>: Enables parallel processing of posts for faster downloads. You can set the number of concurrent worker threads.</li>
<li><strong>Show external links in log</strong>: Opens a secondary log panel that displays external links found in post descriptions.</li>
</ul>
<hr>
<h2>5. Specialized Downloaders &amp; Batch Mode</h2>
<h3>Discord Features</h3>
<ul>
<li>When a Discord URL is entered, a <strong>Scope</strong> button appears.
<ul> <ul>
<li><strong>Scope: Files</strong>: Downloads all files from the channel/server.</li> <li>Post processing steps</li>
<li><strong>Scope: Messages</strong>: Saves the entire message history of the channel/server as a formatted PDF.</li> <li>Download/skipped file notifications</li>
<li>Error messages</li>
<li>Session summaries</li>
</ul> </ul>
</li> </li>
<li>A <strong>"Save as PDF"</strong> button also appears as a shortcut for the message saving feature.</li>
</ul> <li>
<h3>Batch Downloading (<code>nhentai</code> &amp; <code>saint2.su</code>)</h3> <strong>"Only Links" Mode</strong><br>
<p>This feature allows you to download hundreds of galleries or videos from a simple text file.</p> When enabled, the log panel switches modes and reveals new controls.
<ol>
<li>In the <code>appdata</code> folder, create <code>nhentai.txt</code> or <code>saint2.su.txt</code>.</li>
<li>Add one full URL per line to the corresponding file.</li>
<li>In the app's URL input, type either <code>nhentai.net</code> or <code>saint2.su</code> and click "Start Download".</li>
<li>The app will read the file and process every URL in the queue.</li>
</ol>
<hr>
<h2>6. "Only Links" Mode: Extraction &amp; Direct Download</h2>
<p>When you select the <strong>"Only Links"</strong> radio button, the application's behavior changes significantly.</p>
<ul>
<li><strong>Link Extraction</strong>: Instead of downloading files, the main log panel will fill with all external links found (Mega, Google Drive, Dropbox, etc.).</li>
<li><strong>Export Links</strong>: An "Export Links" button appears, allowing you to save the full list of extracted URLs to a <code>.txt</code> file.</li>
<li><strong>Direct Cloud Download</strong>: A <strong>"Download"</strong> button appears next to the export button.
<ul> <ul>
<li>Clicking this opens a new dialog listing all supported cloud links (Mega, G-Drive, Dropbox).</li> <li><strong>📜 Extracted Links Log:</strong> Replaces progress info with a list of found external links (e.g., Mega, Dropbox).</li>
<li>You can select which files you want to download from this list.</li> <li><strong>Export Links Button:</strong> Saves the extracted links to a <code>.txt</code> file.</li>
<li>The application will then download the selected files directly from the cloud service to your chosen download location.</li> <li><strong>Download Button:</strong> Opens the <code>Download Selected External Links</code> dialog (from <code>DownloadExtractedLinksDialog.py</code>), where you can:
<ul>
<li>View all supported external links</li>
<li>Select which ones to download</li>
<li>Begin download directly from cloud services</li>
</ul>
</li>
<li><strong>Links View Button:</strong> Toggles log display between:
<ul>
<li><strong>🔗 Links View:</strong> Shows all extracted links</li>
<li><strong>⬇️ Progress View:</strong> Shows download progress from external services (e.g., Mega)</li>
</ul>
</li>
</ul> </ul>
</li> </li>
</ul> </ul>
<hr>
<h2>7. Session &amp; Process Management</h2>
<h3>Main Action Buttons</h3>
<ul>
<li><strong>Start Download</strong>: Begins the download process. This button's text changes contextually (e.g., "Extract Links", "Check for Updates").</li>
<li><strong>Pause / Resume</strong>: Pauses or resumes the ongoing download. When paused, you can safely change some settings.</li>
<li><strong>Cancel &amp; Reset UI</strong>: Stops the current download and performs a soft reset of the UI, preserving your URL and download location.</li>
</ul>
<h3>Restore Interrupted Download</h3>
<p>If the application is closed unexpectedly during a download, it will save its progress.</p>
<ul>
<li>On the next launch, the UI will be pre-filled with the settings from the interrupted session.</li>
<li>The <strong>Pause</strong> button will change to <strong>"🔄 Restore Download"</strong>. Clicking it will resume the download exactly where it left off, skipping already processed posts.</li>
<li>The <strong>Cancel</strong> button will change to <strong>"🗑️ Discard Session"</strong>, allowing you to clear the saved state and start fresh.</li>
</ul>
<h3>Other UI Controls</h3>
<ul>
<li><strong>Error Button</strong>: Shows a count of failed files. Clicking it opens a dialog where you can view, export, or retry the failed downloads.</li>
<li><strong>History Button</strong>: Shows a log of recently downloaded files and processed posts.</li>
<li><strong>Settings Button</strong>: Opens the settings dialog where you can change the theme, language, and <strong>check for application updates</strong>.</li>
<li><strong>Support Button</strong>: Opens a dialog with links to the project's source code and developer support pages.</li>
</ul>

View File

@@ -107,4 +107,4 @@ def main():
if __name__ == '__main__': if __name__ == '__main__':
main() main()

202
readme.md
View File

@@ -1,116 +1,162 @@
<h1 align="center">Kemono Downloader</h1> <h1 align="center">Kemono Downloader </h1>
<div align="center"> <div align="center">
<table>
<tbody> <table>
<tr> <tr>
<td align="center"> <td align="center">
<img src="Read/Read.png" alt="Default Mode" width="400"><br> <img src="Read/Read.png" alt="Default Mode" width="400"><br>
<strong>Default Mode</strong> <strong>Default</strong>
</td> </td>
<td align="center"> <td align="center">
<img src="Read/Read1.png" alt="Favorite Mode" width="400"><br> <img src="Read/Read1.png" alt="Favorite Mode" width="400"><br>
<strong>Favorite Mode</strong> <strong>Favorite Mode</strong>
</td> </td>
</tr> </tr>
<tr> <tr>
<td align="center"> <td align="center">
<img src="Read/Read2.png" alt="Single Post" width="400"><br> <img src="Read/Read2.png" alt="Single Post" width="400"><br>
<strong>Single Post</strong> <strong>Single Post</strong>
</td> </td>
<td align="center"> <td align="center">
<img src="Read/Read3.png" alt="Manga/Comic Mode" width="400"><br> <img src="Read/Read3.png" alt="Manga/Comic Mode" width="400"><br>
<strong>Manga/Comic Mode</strong> <strong>Manga/Comic Mode</strong>
</td> </td>
</tr> </tr>
</tbody> </table>
</table>
</div> </div>
<hr> ---
<p>A powerful, feature-rich GUI application for downloading content from a wide array of sites, including <strong>Kemono</strong>, <strong>Coomer</strong>, <strong>Bunkr</strong>, <strong>Erome</strong>, <strong>Saint2.su</strong>, and <strong>nhentai</strong>.</p> A powerful, feature-rich GUI application for downloading content from **[Kemono.su](https://kemono.su)** (and its mirrors like kemono.party) and **[Coomer.party](https://coomer.party)** (and its mirrors like coomer.su).
<p>Built with PyQt5, this tool is designed for users who want deep filtering capabilities, customizable folder structures, efficient downloads, and intelligent automation — all within a modern and user-friendly graphical interface.</p>
Built with PyQt5, this tool is designed for users who want deep filtering capabilities, customizable folder structures, efficient downloads, and intelligent automation — all within a modern and user-friendly graphical interface.
<div align="center"> <div align="center">
<a href="features.md"><img src="https://img.shields.io/badge/📚%20Full%20Feature%20List-FFD700?style=for-the-badge&logoColor=black&color=FFD700" alt="Full Feature List"></a>
<a href="LICENSE"><img src="https://img.shields.io/badge/📝%20License-90EE90?style=for-the-badge&logoColor=black&color=90EE90" alt="License"></a> [![](https://img.shields.io/badge/📚%20Full%20Feature%20List-FFD700?style=for-the-badge&logoColor=black&color=FFD700)](features.md)
<a href="note.md"><img src="https://img.shields.io/badge/⚠️%20Important%20Note-FFCCCB?style=for-the-badge&logoColor=black&color=FFCCCB" alt="Important Note"></a> [![](https://img.shields.io/badge/📝%20License-90EE90?style=for-the-badge&logoColor=black&color=90EE90)](LICENSE)
[![](https://img.shields.io/badge/⚠️%20Important%20Note-FFCCCB?style=for-the-badge&logoColor=black&color=FFCCCB)](note.md)
</div> </div>
<h2>Core Capabilities Overview</h2> <h2><strong>Core Capabilities Overview</strong></h2>
<h3>High-Performance &amp; Resilient Downloading</h3>
<h3><strong>High-Performance Downloading</strong></h3>
<ul> <ul>
<li><strong>Multi-threading:</strong> Processes multiple posts simultaneously to greatly accelerate downloads from large creator profiles.</li> <li><strong>Multi-threading:</strong> Processes multiple posts simultaneously to greatly accelerate downloads from large creator profiles.</li>
<li><strong>Multi-part Downloading:</strong> Splits large files into chunks and downloads them in parallel to maximize speed.</li> <li><strong>Multi-part Downloading:</strong> Splits large files into chunks and downloads them in parallel to maximize speed.</li>
<li><strong>Session Management:</strong> Supports pausing, resuming, and <strong>restoring downloads</strong> after crashes or interruptions, so you never lose your progress.</li> <li><strong>Resilience:</strong> Supports pausing, resuming, and restoring downloads after crashes or interruptions.</li>
</ul> </ul>
<h3>Expanded Site Support</h3>
<ul> <h3><strong>Advanced Filtering & Content Control</strong></h3>
<li><strong>Direct Downloading:</strong> Full support for Kemono, Coomer, Bunkr, Erome, Saint2.su, and nhentai.</li>
<li><strong>Batch Mode:</strong> Download hundreds of URLs at once from <code>nhentai.txt</code> or <code>saint2.su.txt</code> files.</li>
<li><strong>Discord Support:</strong> Download files or save entire channel histories as PDFs directly through the API.</li>
</ul>
<h3>Advanced Filtering &amp; Content Control</h3>
<ul> <ul>
<li><strong>Content Type Filtering:</strong> Select whether to download all files or limit to images, videos, audio, or archives only.</li> <li><strong>Content Type Filtering:</strong> Select whether to download all files or limit to images, videos, audio, or archives only.</li>
<li><strong>Keyword Skipping:</strong> Automatically skips posts or files containing certain keywords (e.g., "WIP", "sketch").</li> <li><strong>Keyword Skipping:</strong> Automatically skips posts or files containing certain keywords (e.g., "WIP", "sketch").</li>
<li><strong>Skip by Size:</strong> Avoid small files by setting a minimum size threshold in MB (e.g., <code>[200]</code>).</li> <li><strong>Character Filtering:</strong> Restricts downloads to posts that match specific character or series names.</li>
<li><strong>Character Filtering:</strong> Restricts downloads to posts that match specific character or series names, with scope controls for title, filename, or comments.</li>
</ul> </ul>
<h3>Intelligent File Organization</h3>
<h3><strong>File Organization & Renaming</strong></h3>
<ul> <ul>
<li><strong>Automated Subfolders:</strong> Automatically organizes downloaded files into subdirectories based on character names or per post.</li> <li><strong>Automated Subfolders:</strong> Automatically organizes downloaded files into subdirectories based on character names or per post.</li>
<li><strong>Advanced File Renaming:</strong> Flexible renaming options, especially in Manga Mode, including by post title, date, sequential numbering, or post ID.</li> <li><strong>Advanced File Renaming:</strong> Flexible renaming options, especially in Manga Mode, including:
<li><strong>Filename Cleaning:</strong> Automatically removes unwanted text from filenames.</li> <ul>
<li><strong>Post Title:</strong> Uses the post's title (e.g., <code>Chapter-One.jpg</code>).</li>
<li><strong>Date + Original Name:</strong> Prepends the publication date to the original filename.</li>
<li><strong>Date + Title:</strong> Combines the date with the post title.</li>
<li><strong>Sequential Numbering (Date Based):</strong> Simple sequence numbers (e.g., <code>001.jpg</code>, <code>002.jpg</code>).</li>
<li><strong>Title + Global Numbering:</strong> Uses post title with a globally incrementing number across the session.</li>
<li><strong>Post ID:</strong> Names files using the posts unique ID.</li>
</ul>
</li>
</ul> </ul>
<h3>Specialized Modes</h3>
<h3><strong>Specialized Modes</strong></h3>
<ul> <ul>
<li><strong>Manga/Comic Mode:</strong> Sorts posts chronologically before downloading to ensure pages appear in the correct sequence.</li> <li><strong>Manga/Comic Mode:</strong> Sorts posts chronologically before downloading to ensure pages appear in the correct sequence.</li>
<li><strong>Favorite Mode:</strong> Connects to your account and downloads from your favorites list (artists or posts).</li> <li><strong>Favorite Mode:</strong> Connects to your account and downloads from your favorites list (artists or posts).</li>
<li><strong>Link Extraction Mode:</strong> Extracts external links (Mega, Google Drive) from posts for export or <strong>direct in-app downloading</strong>.</li> <li><strong>Link Extraction Mode:</strong> Extracts external links from posts for export or targeted downloading.</li>
<li><strong>Text Extraction Mode:</strong> Saves post descriptions or comment sections as <code>PDF</code>, <code>DOCX</code>, or <code>TXT</code> files.</li> <li><strong>Text Extraction Mode:</strong> Saves post descriptions or comment sections as <code>PDF</code>, <code>DOCX</code>, or <code>TXT</code> files.</li>
</ul> </ul>
<h3>Utility &amp; Advanced Features</h3>
<h3><strong>Utility & Advanced Features</strong></h3>
<ul> <ul>
<li><strong>In-App Updater:</strong> Check for new versions directly from the settings menu.</li>
<li><strong>Cookie Support:</strong> Enables access to subscriber-only content via browser session cookies.</li> <li><strong>Cookie Support:</strong> Enables access to subscriber-only content via browser session cookies.</li>
<li><strong>Duplicate Detection:</strong> Prevents saving duplicate files using content-based comparison, with configurable limits.</li> <li><strong>Duplicate Detection:</strong> Prevents saving duplicate files using content-based comparison, with configurable limits.</li>
<li><strong>Image Compression:</strong> Automatically converts large images to <code>.webp</code> to reduce disk usage.</li> <li><strong>Image Compression:</strong> Automatically converts large images to <code>.webp</code> to reduce disk usage.</li>
<li><strong>Creator Management:</strong> Built-in creator browser and update checker for downloading only new posts from saved profiles.</li> <li><strong>Creator Management:</strong> Built-in creator browser and update checker for downloading only new posts from saved profiles.</li>
<li><strong>Error Handling:</strong> Tracks failed downloads and provides a retry dialog with options to export or redownload missing files.</li> <li><strong>Error Handling:</strong> Tracks failed downloads and provides a retry dialog with options to export or redownload missing files.</li>
</ul> </ul>
<h2>💻 Installation</h2>
<h3>Requirements</h3> ## 💻 Installation
<ul>
<li>Python 3.6 or higher</li> ### Requirements
<li>pip (Python package installer)</li>
</ul> - Python 3.6 or higher
<h3>Install Dependencies</h3> - pip (Python package installer)
<pre><code>pip install PyQt5 requests cloudscraper Pillow fpdf2 python-docx
</code></pre> ### Install Dependencies
<h3>Running the Application</h3>
<p>Navigate to the application's directory in your terminal and run:</p> ```bash
<pre><code>python main.py pip install PyQt5 requests Pillow mega.py fpdf python-docx
</code></pre> ```
<h2>Contribution</h2>
<p>Feel free to fork this repo and submit pull requests for bug fixes, new features, or UI improvements!</p> ### Running the Application
<h2>License</h2> Navigate to the application's directory in your terminal and run:
<p>This project is under the MIT Licence</p> ```bash
<h2>Star History</h2> python main.py
```
### Optional Setup
- **Main Inputs:**
- Place your `cookies.txt` in the root directory (if using cookies).
- Prepare your `Known.txt` and `creators.json` in the same directory for advanced filtering and selection features.
---
## Troubleshooting
### AttributeError: module 'asyncio' has no attribute 'coroutine'
If you encounter an error message similar to:
```
AttributeError: module 'asyncio' has no attribute 'coroutine'. Did you mean: 'coroutines'?
```
This usually means that a dependency, often `tenacity` (used by `mega.py`), is an older version that's incompatible with your Python version (typically Python 3.10+).
To fix this, activate your virtual environment and run the following commands to upgrade the libraries:
```bash
pip install --upgrade tenacity
pip install --upgrade mega.py
```
---
## Contribution
Feel free to fork this repo and submit pull requests for bug fixes, new features, or UI improvements!
---
## License
This project is under the MIT Licence
## Star History
<table align="center" style="border-collapse: collapse; border: none; margin-left: auto; margin-right: auto;"> <table align="center" style="border-collapse: collapse; border: none; margin-left: auto; margin-right: auto;">
<tbody> <tr>
<tr> <td align="center" valign="middle" style="padding: 10px; border: none;">
<td align="center" valign="middle" style="padding: 10px; border: none;"> <a href="https://www.star-history.com/#Yuvi9587/Kemono-Downloader&Date">
<a href="https://www.star-history.com/#Yuvi9587/Kemono-Downloader&amp;Date"> <img src="https://api.star-history.com/svg?repos=Yuvi9587/Kemono-Downloader&type=Date" alt="Star History Chart" width="650">
<img src="https://api.star-history.com/svg?repos=Yuvi9587/Kemono-Downloader&amp;type=Date" alt="Star History Chart" width="650"> </a>
</a>
</td>
</tr>
</tbody>
</table> </table>
<p align="center"> <p align="center">
<a href="https://buymeacoffee.com/yuvi9587"> <a href="https://buymeacoffee.com/yuvi9587">
<img src="https://img.shields.io/badge/🍺%20Buy%20Me%20a%20Coffee-FFCCCB?style=for-the-badge&amp;logoColor=black&amp;color=FFDD00" alt="Buy Me a Coffee"> <img src="https://img.shields.io/badge/🍺%20Buy%20Me%20a%20Coffee-FFCCCB?style=for-the-badge&logoColor=black&color=FFDD00" alt="Buy Me a Coffee">
</a> </a>
</p> </p>

View File

@@ -1,3 +1,4 @@
# --- Application Metadata ---
CONFIG_ORGANIZATION_NAME = "KemonoDownloader" CONFIG_ORGANIZATION_NAME = "KemonoDownloader"
CONFIG_APP_NAME_MAIN = "ApplicationSettings" CONFIG_APP_NAME_MAIN = "ApplicationSettings"
CONFIG_APP_NAME_TOUR = "ApplicationTour" CONFIG_APP_NAME_TOUR = "ApplicationTour"
@@ -8,7 +9,7 @@ STYLE_ORIGINAL_NAME = "original_name"
STYLE_DATE_BASED = "date_based" STYLE_DATE_BASED = "date_based"
STYLE_DATE_POST_TITLE = "date_post_title" STYLE_DATE_POST_TITLE = "date_post_title"
STYLE_POST_TITLE_GLOBAL_NUMBERING = "post_title_global_numbering" STYLE_POST_TITLE_GLOBAL_NUMBERING = "post_title_global_numbering"
STYLE_POST_ID = "post_id" STYLE_POST_ID = "post_id" # Add this line
MANGA_DATE_PREFIX_DEFAULT = "" MANGA_DATE_PREFIX_DEFAULT = ""
# --- Download Scopes --- # --- Download Scopes ---
@@ -59,10 +60,7 @@ DOWNLOAD_LOCATION_KEY = "downloadLocationV1"
RESOLUTION_KEY = "window_resolution" RESOLUTION_KEY = "window_resolution"
UI_SCALE_KEY = "ui_scale_factor" UI_SCALE_KEY = "ui_scale_factor"
SAVE_CREATOR_JSON_KEY = "saveCreatorJsonProfile" SAVE_CREATOR_JSON_KEY = "saveCreatorJsonProfile"
FETCH_FIRST_KEY = "fetchAllPostsFirst" FETCH_FIRST_KEY = "fetchAllPostsFirst"
DISCORD_TOKEN_KEY = "discord/token"
POST_DOWNLOAD_ACTION_KEY = "postDownloadAction"
# --- UI Constants and Identifiers --- # --- UI Constants and Identifiers ---
HTML_PREFIX = "<!HTML!>" HTML_PREFIX = "<!HTML!>"
@@ -122,4 +120,4 @@ CREATOR_DOWNLOAD_DEFAULT_FOLDER_IGNORE_WORDS = {
# --- Duplicate Handling Modes --- # --- Duplicate Handling Modes ---
DUPLICATE_HANDLING_HASH = "hash" DUPLICATE_HANDLING_HASH = "hash"
DUPLICATE_HANDLING_KEEP_ALL = "keep_all" DUPLICATE_HANDLING_KEEP_ALL = "keep_all"

View File

@@ -1,72 +0,0 @@
# src/core/Hentai2read_client.py
import re
import os
import json
import requests
import cloudscraper
from bs4 import BeautifulSoup
def fetch_hentai2read_data(url, logger, session):
"""
Scrapes a SINGLE Hentai2Read chapter page using a provided session.
"""
logger(f"Attempting to fetch chapter data from: {url}")
try:
response = session.get(url, timeout=30)
response.raise_for_status()
page_content_text = response.text
soup = BeautifulSoup(page_content_text, 'html.parser')
album_title = ""
title_tags = soup.select('span[itemprop="name"]')
if title_tags:
album_title = title_tags[-1].text.strip()
if not album_title:
title_tag = soup.select_one('h1.title')
if title_tag:
album_title = title_tag.text.strip()
if not album_title:
logger("❌ Could not find album title on page.")
return None, None
image_urls = []
try:
start_index = page_content_text.index("'images' : ") + len("'images' : ")
end_index = page_content_text.index(",\n", start_index)
images_json_str = page_content_text[start_index:end_index]
image_paths = json.loads(images_json_str)
image_urls = ["https://hentaicdn.com/hentai" + part for part in image_paths]
except (ValueError, json.JSONDecodeError):
logger("❌ Could not find or parse image JSON data for this chapter.")
return None, None
if not image_urls:
logger("❌ No image URLs found for this chapter.")
return None, None
logger(f" Found {len(image_urls)} images for album '{album_title}'.")
files_to_download = []
for i, img_url in enumerate(image_urls):
page_num = i + 1
extension = os.path.splitext(img_url)[1].split('?')[0]
if not extension: extension = ".jpg"
filename = f"{page_num:03d}{extension}"
files_to_download.append({'url': img_url, 'filename': filename})
return album_title, files_to_download
except requests.exceptions.HTTPError as e:
if e.response.status_code == 404:
logger(f" Chapter not found (404 Error). This likely marks the end of the series.")
else:
logger(f"❌ An HTTP error occurred: {e}")
return None, None
except Exception as e:
logger(f"❌ An unexpected error occurred while fetching data: {e}")
return None, None

View File

@@ -3,7 +3,6 @@ import traceback
from urllib.parse import urlparse from urllib.parse import urlparse
import json import json
import requests import requests
import cloudscraper
from ..utils.network_utils import extract_post_info, prepare_cookies_for_request from ..utils.network_utils import extract_post_info, prepare_cookies_for_request
from ..config.constants import ( from ..config.constants import (
STYLE_DATE_POST_TITLE STYLE_DATE_POST_TITLE
@@ -13,6 +12,7 @@ from ..config.constants import (
def fetch_posts_paginated(api_url_base, headers, offset, logger, cancellation_event=None, pause_event=None, cookies_dict=None): def fetch_posts_paginated(api_url_base, headers, offset, logger, cancellation_event=None, pause_event=None, cookies_dict=None):
""" """
Fetches a single page of posts from the API with robust retry logic. Fetches a single page of posts from the API with robust retry logic.
NEW: Requests only essential fields to keep the response size small and reliable.
""" """
if cancellation_event and cancellation_event.is_set(): if cancellation_event and cancellation_event.is_set():
raise RuntimeError("Fetch operation cancelled by user.") raise RuntimeError("Fetch operation cancelled by user.")
@@ -33,7 +33,7 @@ def fetch_posts_paginated(api_url_base, headers, offset, logger, cancellation_ev
if cancellation_event and cancellation_event.is_set(): if cancellation_event and cancellation_event.is_set():
raise RuntimeError("Fetch operation cancelled by user during retry loop.") raise RuntimeError("Fetch operation cancelled by user during retry loop.")
log_message = f" Fetching post list: {paginated_url} (Page approx. {offset // 50 + 1})" log_message = f" Fetching post list: {api_url_base}?o={offset} (Page approx. {offset // 50 + 1})"
if attempt > 0: if attempt > 0:
log_message += f" (Attempt {attempt + 1}/{max_retries})" log_message += f" (Attempt {attempt + 1}/{max_retries})"
logger(log_message) logger(log_message)
@@ -41,23 +41,9 @@ def fetch_posts_paginated(api_url_base, headers, offset, logger, cancellation_ev
try: try:
response = requests.get(paginated_url, headers=headers, timeout=(15, 60), cookies=cookies_dict) response = requests.get(paginated_url, headers=headers, timeout=(15, 60), cookies=cookies_dict)
response.raise_for_status() response.raise_for_status()
response.encoding = 'utf-8'
return response.json() return response.json()
except requests.exceptions.RequestException as e: except requests.exceptions.RequestException as e:
# Handle 403 error on the FIRST page as a rate limit/block
if e.response is not None and e.response.status_code == 403 and offset == 0:
logger(" ❌ Access Denied (403 Forbidden) on the first page.")
logger(" This is likely a rate limit or a Cloudflare block.")
logger(" 💡 SOLUTION: Wait a while, use a VPN, or provide a valid session cookie.")
return [] # Stop the process gracefully
# Handle 400 error as the end of pages
if e.response is not None and e.response.status_code == 400:
logger(f" ✅ Reached end of posts (API returned 400 Bad Request for offset {offset}).")
return []
# Handle all other network errors with a retry
logger(f" ⚠️ Retryable network error on page fetch (Attempt {attempt + 1}): {e}") logger(f" ⚠️ Retryable network error on page fetch (Attempt {attempt + 1}): {e}")
if attempt < max_retries - 1: if attempt < max_retries - 1:
delay = retry_delay * (2 ** attempt) delay = retry_delay * (2 ** attempt)
@@ -79,28 +65,26 @@ def fetch_posts_paginated(api_url_base, headers, offset, logger, cancellation_ev
raise RuntimeError(f"Failed to fetch page {paginated_url} after all attempts.") raise RuntimeError(f"Failed to fetch page {paginated_url} after all attempts.")
def fetch_single_post_data(api_domain, service, user_id, post_id, headers, logger, cookies_dict=None): def fetch_single_post_data(api_domain, service, user_id, post_id, headers, logger, cookies_dict=None):
""" """
--- MODIFIED FUNCTION --- --- NEW FUNCTION ---
Fetches the full data, including the 'content' field, for a single post using cloudscraper. Fetches the full data, including the 'content' field, for a single post.
""" """
post_api_url = f"https://{api_domain}/api/v1/{service}/user/{user_id}/post/{post_id}" post_api_url = f"https://{api_domain}/api/v1/{service}/user/{user_id}/post/{post_id}"
logger(f" Fetching full content for post ID {post_id}...") logger(f" Fetching full content for post ID {post_id}...")
scraper = cloudscraper.create_scraper()
try: try:
response = scraper.get(post_api_url, headers=headers, timeout=(15, 300), cookies=cookies_dict) with requests.get(post_api_url, headers=headers, timeout=(15, 300), cookies=cookies_dict, stream=True) as response:
response.raise_for_status() response.raise_for_status()
response_body = b""
full_post_data = response.json() for chunk in response.iter_content(chunk_size=8192):
response_body += chunk
if isinstance(full_post_data, list) and full_post_data:
return full_post_data[0] full_post_data = json.loads(response_body)
if isinstance(full_post_data, dict) and 'post' in full_post_data: if isinstance(full_post_data, list) and full_post_data:
return full_post_data['post'] return full_post_data[0]
return full_post_data return full_post_data
except Exception as e: except Exception as e:
logger(f" ❌ Failed to fetch full content for post {post_id}: {e}") logger(f" ❌ Failed to fetch full content for post {post_id}: {e}")
return None return None
@@ -117,7 +101,6 @@ def fetch_post_comments(api_domain, service, user_id, post_id, headers, logger,
try: try:
response = requests.get(comments_api_url, headers=headers, timeout=(10, 30), cookies=cookies_dict) response = requests.get(comments_api_url, headers=headers, timeout=(10, 30), cookies=cookies_dict)
response.raise_for_status() response.raise_for_status()
response.encoding = 'utf-8'
return response.json() return response.json()
except requests.exceptions.RequestException as e: except requests.exceptions.RequestException as e:
raise RuntimeError(f"Error fetching comments for post {post_id}: {e}") raise RuntimeError(f"Error fetching comments for post {post_id}: {e}")
@@ -139,16 +122,11 @@ def download_from_api(
manga_filename_style_for_sort_check=None, manga_filename_style_for_sort_check=None,
processed_post_ids=None, processed_post_ids=None,
fetch_all_first=False fetch_all_first=False
): ):
parsed_input_url_for_domain = urlparse(api_url_input)
api_domain = parsed_input_url_for_domain.netloc
headers = { headers = {
'User-Agent': 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)', 'User-Agent': 'Mozilla/5.0',
'Referer': f'https://{api_domain}/', 'Accept': 'application/json'
'Accept': 'text/css'
} }
if processed_post_ids is None: if processed_post_ids is None:
processed_post_ids = set() processed_post_ids = set()
else: else:
@@ -160,11 +138,15 @@ def download_from_api(
logger(" Download_from_api cancelled at start.") logger(" Download_from_api cancelled at start.")
return return
# The code that defined api_domain was moved from here to the top of the function parsed_input_url_for_domain = urlparse(api_url_input)
api_domain = parsed_input_url_for_domain.netloc
# --- START: MODIFIED LOGIC ---
# This list is updated to include the new .cr and .st mirrors for validation.
if not any(d in api_domain.lower() for d in ['kemono.su', 'kemono.party', 'kemono.cr', 'coomer.su', 'coomer.party', 'coomer.st']): if not any(d in api_domain.lower() for d in ['kemono.su', 'kemono.party', 'kemono.cr', 'coomer.su', 'coomer.party', 'coomer.st']):
logger(f"⚠️ Unrecognized domain '{api_domain}' from input URL. Defaulting to kemono.su for API calls.") logger(f"⚠️ Unrecognized domain '{api_domain}' from input URL. Defaulting to kemono.su for API calls.")
api_domain = "kemono.su" api_domain = "kemono.su"
# --- END: MODIFIED LOGIC ---
cookies_for_api = None cookies_for_api = None
if use_cookie and app_base_dir: if use_cookie and app_base_dir:
@@ -178,7 +160,6 @@ def download_from_api(
try: try:
direct_response = requests.get(direct_post_api_url, headers=headers, timeout=(10, 30), cookies=cookies_for_api) direct_response = requests.get(direct_post_api_url, headers=headers, timeout=(10, 30), cookies=cookies_for_api)
direct_response.raise_for_status() direct_response.raise_for_status()
direct_response.encoding = 'utf-8'
direct_post_data = direct_response.json() direct_post_data = direct_response.json()
if isinstance(direct_post_data, list) and direct_post_data: if isinstance(direct_post_data, list) and direct_post_data:
direct_post_data = direct_post_data[0] direct_post_data = direct_post_data[0]
@@ -204,7 +185,7 @@ def download_from_api(
is_manga_mode_fetch_all_and_sort_oldest_first = manga_mode and (manga_filename_style_for_sort_check != STYLE_DATE_POST_TITLE) and not target_post_id is_manga_mode_fetch_all_and_sort_oldest_first = manga_mode and (manga_filename_style_for_sort_check != STYLE_DATE_POST_TITLE) and not target_post_id
should_fetch_all = fetch_all_first or is_manga_mode_fetch_all_and_sort_oldest_first should_fetch_all = fetch_all_first or is_manga_mode_fetch_all_and_sort_oldest_first
api_base_url = f"https://{api_domain}/api/v1/{service}/user/{user_id}/posts" api_base_url = f"https://{api_domain}/api/v1/{service}/user/{user_id}"
page_size = 50 page_size = 50
if is_manga_mode_fetch_all_and_sort_oldest_first: if is_manga_mode_fetch_all_and_sort_oldest_first:
logger(f" Manga Mode (Style: {manga_filename_style_for_sort_check if manga_filename_style_for_sort_check else 'Default'} - Oldest First Sort Active): Fetching all posts to sort by date...") logger(f" Manga Mode (Style: {manga_filename_style_for_sort_check if manga_filename_style_for_sort_check else 'Default'} - Oldest First Sort Active): Fetching all posts to sort by date...")
@@ -375,4 +356,3 @@ def download_from_api(
time.sleep(0.6) time.sleep(0.6)
if target_post_id and not processed_target_post_flag and not (cancellation_event and cancellation_event.is_set()): if target_post_id and not processed_target_post_flag and not (cancellation_event and cancellation_event.is_set()):
logger(f"❌ Target post {target_post_id} could not be found after checking all relevant pages (final check after loop).") logger(f"❌ Target post {target_post_id} could not be found after checking all relevant pages (final check after loop).")

View File

@@ -1,241 +0,0 @@
import logging
import os
import re
import requests
import html
import time
import datetime
import urllib.parse
import json
import random
import binascii
import itertools
class MockMessage:
Directory = 1
Url = 2
Version = 3
class AlbumException(Exception): pass
class ExtractionError(AlbumException): pass
class HttpError(ExtractionError):
def __init__(self, message="", response=None):
self.response = response
self.status = response.status_code if response is not None else 0
super().__init__(message)
class ControlException(AlbumException): pass
class AbortExtraction(ExtractionError, ControlException): pass
try:
re_compile = re._compiler.compile
except AttributeError:
re_compile = re.sre_compile.compile
HTML_RE = re_compile(r"<[^>]+>")
def extr(txt, begin, end, default=""):
try:
first = txt.index(begin) + len(begin)
return txt[first:txt.index(end, first)]
except Exception: return default
def extract_iter(txt, begin, end, pos=None):
try:
index = txt.index
lbeg = len(begin)
lend = len(end)
while True:
first = index(begin, pos) + lbeg
last = index(end, first)
pos = last + lend
yield txt[first:last]
except Exception: return
def split_html(txt):
try: return [html.unescape(x).strip() for x in HTML_RE.split(txt) if x and not x.isspace()]
except TypeError: return []
def parse_datetime(date_string, format="%Y-%m-%dT%H:%M:%S%z", utcoffset=0):
try:
d = datetime.datetime.strptime(date_string, format)
o = d.utcoffset()
if o is not None: d = d.replace(tzinfo=None, microsecond=0) - o
else:
if d.microsecond: d = d.replace(microsecond=0)
if utcoffset: d += datetime.timedelta(0, utcoffset * -3600)
return d
except (TypeError, IndexError, KeyError, ValueError, OverflowError): return None
unquote = urllib.parse.unquote
unescape = html.unescape
def decrypt_xor(encrypted, key, base64=True, fromhex=False):
if base64: encrypted = binascii.a2b_base64(encrypted)
if fromhex: encrypted = bytes.fromhex(encrypted.decode())
div = len(key)
return bytes([encrypted[i] ^ key[i % div] for i in range(len(encrypted))]).decode()
def advance(iterable, num):
iterator = iter(iterable)
next(itertools.islice(iterator, num, num), None)
return iterator
def json_loads(s): return json.loads(s)
def json_dumps(obj): return json.dumps(obj, separators=(",", ":"))
class Extractor:
def __init__(self, match, logger):
self.log = logger
self.url = match.string
self.match = match
self.groups = match.groups()
self.session = requests.Session()
self.session.headers["User-Agent"] = "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Firefox/102.0"
@classmethod
def from_url(cls, url, logger):
if isinstance(cls.pattern, str): cls.pattern = re.compile(cls.pattern)
match = cls.pattern.match(url)
return cls(match, logger) if match else None
def __iter__(self): return self.items()
def items(self): yield MockMessage.Version, 1
def request(self, url, method="GET", fatal=True, **kwargs):
tries = 1
while True:
try:
response = self.session.request(method, url, **kwargs)
if response.status_code < 400: return response
msg = f"'{response.status_code} {response.reason}' for '{response.url}'"
except requests.exceptions.RequestException as exc:
msg = str(exc)
self.log.info("%s (retrying...)", msg)
if tries > 4: break
time.sleep(tries)
tries += 1
if not fatal: return None
raise HttpError(msg)
def request_json(self, url, **kwargs):
response = self.request(url, **kwargs)
try: return json_loads(response.text)
except Exception as exc:
self.log.warning("%s: %s", exc.__class__.__name__, exc)
if not kwargs.get("fatal", True): return {}
raise
BASE_PATTERN_BUNKR = r"(?:https?://)?(?:[a-zA-Z0-9-]+\.)?(bunkr\.(?:si|la|ws|red|black|media|site|is|to|ac|cr|ci|fi|pk|ps|sk|ph|su)|bunkrr\.ru)"
DOMAINS = ["bunkr.si", "bunkr.ws", "bunkr.la", "bunkr.red", "bunkr.black", "bunkr.media", "bunkr.site"]
CF_DOMAINS = set()
class BunkrAlbumExtractor(Extractor):
category = "bunkr"
root = "https://bunkr.si"
root_dl = "https://get.bunkrr.su"
root_api = "https://apidl.bunkr.ru"
pattern = re.compile(BASE_PATTERN_BUNKR + r"/a/([^/?#]+)")
def __init__(self, match, logger):
super().__init__(match, logger)
domain_match = re.search(BASE_PATTERN_BUNKR, match.string)
if domain_match:
self.root = "https://" + domain_match.group(1)
self.endpoint = self.root_api + "/api/_001_v2"
self.album_id = self.groups[-1]
def items(self):
page = self.request(self.url).text
title = unescape(unescape(extr(page, 'property="og:title" content="', '"')))
items_html = list(extract_iter(page, '<div class="grid-images_box', "</a>"))
album_data = {
"album_id": self.album_id,
"album_name": title,
"count": len(items_html),
}
yield MockMessage.Directory, album_data, {}
for item_html in items_html:
try:
webpage_url = unescape(extr(item_html, ' href="', '"'))
if webpage_url.startswith("/"):
webpage_url = self.root + webpage_url
file_data = self._extract_file(webpage_url)
info = split_html(item_html)
if not file_data.get("name"):
file_data["name"] = info[-3]
yield MockMessage.Url, file_data, {}
except Exception as exc:
self.log.error("%s: %s", exc.__class__.__name__, exc)
def _extract_file(self, webpage_url):
page = self.request(webpage_url).text
data_id = extr(page, 'data-file-id="', '"')
referer = self.root_dl + "/file/" + data_id
headers = {"Referer": referer, "Origin": self.root_dl}
data = self.request_json(self.endpoint, method="POST", headers=headers, json={"id": data_id})
file_url = decrypt_xor(data["url"], f"SECRET_KEY_{data['timestamp'] // 3600}".encode()) if data.get("encrypted") else data["url"]
file_name = extr(page, "<h1", "<").rpartition(">")[2]
return {
"url": file_url,
"name": unescape(file_name),
"_http_headers": {"Referer": referer}
}
class BunkrMediaExtractor(BunkrAlbumExtractor):
pattern = re.compile(BASE_PATTERN_BUNKR + r"(/[fvid]/[^/?#]+)")
def items(self):
try:
media_path = self.groups[-1]
file_data = self._extract_file(self.root + media_path)
album_data = {"album_name": file_data.get("name", "bunkr_media"), "count": 1}
yield MockMessage.Directory, album_data, {}
yield MockMessage.Url, file_data, {}
except Exception as exc:
self.log.error("%s: %s", exc.__class__.__name__, exc)
yield MockMessage.Directory, {"album_name": "error", "count": 0}, {}
def get_bunkr_extractor(url, logger):
"""Selects the correct Bunkr extractor based on the URL pattern."""
if BunkrAlbumExtractor.pattern.match(url):
logger.info("Bunkr Album URL detected.")
return BunkrAlbumExtractor.from_url(url, logger)
elif BunkrMediaExtractor.pattern.match(url):
logger.info("Bunkr Media URL detected.")
return BunkrMediaExtractor.from_url(url, logger)
else:
logger.error(f"No suitable Bunkr extractor found for URL: {url}")
return None
def fetch_bunkr_data(url, logger):
"""
Main function to be called from the GUI.
It extracts all file information from a Bunkr URL.
Returns:
A tuple of (album_name, list_of_files)
- album_name (str): The name of the album.
- list_of_files (list): A list of dicts, each containing 'url', 'name', and '_http_headers'.
Returns (None, None) on failure.
"""
extractor = get_bunkr_extractor(url, logger)
if not extractor:
return None, None
try:
album_name = "default_bunkr_album"
files_to_download = []
for msg_type, data, metadata in extractor:
if msg_type == MockMessage.Directory:
raw_album_name = data.get('album_name', 'untitled')
album_name = re.sub(r'[<>:"/\\|?*]', '_', raw_album_name).strip() or "untitled"
logger.info(f"Processing Bunkr album: {album_name}")
elif msg_type == MockMessage.Url:
files_to_download.append(data)
if not files_to_download:
logger.warning("No files found to download from the Bunkr URL.")
return None, None
return album_name, files_to_download
except Exception as e:
logger.error(f"An error occurred while extracting Bunkr info: {e}", exc_info=True)
return None, None

View File

@@ -1,68 +1,63 @@
import time import time
import cloudscraper import requests
import json import json
from urllib.parse import urlparse
def fetch_server_channels(server_id, logger=print, cookies_dict=None): def fetch_server_channels(server_id, logger, cookies=None, cancellation_event=None, pause_event=None):
""" """
Fetches all channels for a given Discord server ID from the API. Fetches the list of channels for a given Discord server ID from the Kemono API.
Uses cloudscraper to bypass Cloudflare. UPDATED to be pausable and cancellable.
""" """
api_url = f"https://kemono.cr/api/v1/discord/server/{server_id}" domains_to_try = ["kemono.cr", "kemono.su"]
logger(f" Fetching channels for server: {api_url}") for domain in domains_to_try:
scraper = cloudscraper.create_scraper()
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36',
'Referer': f'https://kemono.cr/discord/server/{server_id}',
'Accept': 'text/css'
}
try:
response = scraper.get(api_url, headers=headers, cookies=cookies_dict, timeout=30)
response.raise_for_status()
channels = response.json()
if isinstance(channels, list):
logger(f" ✅ Found {len(channels)} channels for server {server_id}.")
return channels
return None
except Exception as e:
logger(f" ❌ Error fetching server channels for {server_id}: {e}")
return None
def fetch_channel_messages(channel_id, logger=print, cancellation_event=None, pause_event=None, cookies_dict=None):
"""
A generator that fetches all messages for a specific Discord channel, handling pagination.
Uses cloudscraper and proper headers to bypass server protection.
"""
scraper = cloudscraper.create_scraper()
base_url = f"https://kemono.cr/api/v1/discord/channel/{channel_id}"
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36',
'Referer': f'https://kemono.cr/discord/channel/{channel_id}',
'Accept': 'text/css'
}
offset = 0
page_size = 150
while True:
if cancellation_event and cancellation_event.is_set(): if cancellation_event and cancellation_event.is_set():
logger(" Discord message fetching cancelled.") logger(" Channel fetching cancelled by user.")
break return None
if pause_event and pause_event.is_set(): while pause_event and pause_event.is_set():
logger(" Discord message fetching paused...") if cancellation_event and cancellation_event.is_set(): break
while pause_event.is_set(): time.sleep(0.5)
if cancellation_event and cancellation_event.is_set():
break
time.sleep(0.5)
if not (cancellation_event and cancellation_event.is_set()):
logger(" Discord message fetching resumed.")
paginated_url = f"{base_url}?o={offset}" lookup_url = f"https://{domain}/api/v1/discord/channel/lookup/{server_id}"
logger(f" Attempting to fetch channel list from: {lookup_url}")
try:
response = requests.get(lookup_url, cookies=cookies, timeout=15)
response.raise_for_status()
channels = response.json()
if isinstance(channels, list):
logger(f" ✅ Found {len(channels)} channels for server {server_id}.")
return channels
except (requests.exceptions.RequestException, json.JSONDecodeError):
# This is a silent failure, we'll just try the next domain
pass
logger(f" ❌ Failed to fetch channel list for server {server_id} from all available domains.")
return None
def fetch_channel_messages(channel_id, logger, cancellation_event, pause_event, cookies=None):
"""
Fetches all messages from a Discord channel by looping through API pages (pagination).
Uses a page size of 150 and handles the specific offset logic.
"""
offset = 0
page_size = 150 # Corrected page size based on your findings
api_base_url = f"https://kemono.cr/api/v1/discord/channel/{channel_id}"
while not (cancellation_event and cancellation_event.is_set()):
if pause_event and pause_event.is_set():
logger(" Message fetching paused...")
while pause_event.is_set():
if cancellation_event and cancellation_event.is_set(): break
time.sleep(0.5)
logger(" Message fetching resumed.")
if cancellation_event and cancellation_event.is_set():
break
paginated_url = f"{api_base_url}?o={offset}"
logger(f" Fetching messages from API: page starting at offset {offset}") logger(f" Fetching messages from API: page starting at offset {offset}")
try: try:
response = scraper.get(paginated_url, headers=headers, cookies=cookies_dict, timeout=30) response = requests.get(paginated_url, cookies=cookies, timeout=20)
response.raise_for_status() response.raise_for_status()
messages_batch = response.json() messages_batch = response.json()
@@ -78,11 +73,8 @@ def fetch_channel_messages(channel_id, logger=print, cancellation_event=None, pa
break break
offset += page_size offset += page_size
time.sleep(0.5) # Be respectful to the API time.sleep(0.5)
except (cloudscraper.exceptions.CloudflareException, json.JSONDecodeError) as e: except (requests.exceptions.RequestException, json.JSONDecodeError) as e:
logger(f" ❌ Error fetching messages at offset {offset}: {e}") logger(f" ❌ Error fetching messages at offset {offset}: {e}")
break break
except Exception as e:
logger(f" ❌ An unexpected error occurred while fetching messages: {e}")
break

View File

@@ -1,131 +0,0 @@
import os
import re
import html
import time
import urllib.parse
import requests
from datetime import datetime
import cloudscraper
def extr(txt, begin, end, default=""):
"""Stripped-down version of 'extract()' to find text between two delimiters."""
try:
first = txt.index(begin) + len(begin)
return txt[first:txt.index(end, first)]
except (ValueError, IndexError):
return default
def extract_iter(txt, begin, end):
"""Yields all occurrences of text between two delimiters."""
try:
index = txt.index
lbeg = len(begin)
lend = len(end)
pos = 0
while True:
first = index(begin, pos) + lbeg
last = index(end, first)
pos = last + lend
yield txt[first:last]
except (ValueError, IndexError):
return
def nameext_from_url(url):
"""Extracts filename and extension from a URL."""
data = {}
filename = urllib.parse.unquote(url.partition("?")[0].rpartition("/")[2])
name, _, ext = filename.rpartition(".")
if name and len(ext) <= 16:
data["filename"], data["extension"] = name, ext.lower()
else:
data["filename"], data["extension"] = filename, ""
return data
def parse_timestamp(ts, default=None):
"""Creates a datetime object from a Unix timestamp."""
try:
return datetime.fromtimestamp(int(ts))
except (ValueError, TypeError):
return default
def fetch_erome_data(url, logger):
"""
Identifies and extracts all media files from an Erome album URL.
Args:
url (str): The Erome album URL (e.g., https://www.erome.com/a/albumID).
logger (function): A function to log progress messages.
Returns:
tuple: A tuple containing (album_folder_name, list_of_file_dicts).
Returns (None, []) if data extraction fails.
"""
album_id_match = re.search(r"/a/(\w+)", url)
if not album_id_match:
logger(f"Error: The URL '{url}' does not appear to be a valid Erome album link.")
return None, []
album_id = album_id_match.group(1)
page_url = f"https://www.erome.com/a/{album_id}"
session = cloudscraper.create_scraper()
try:
logger(f" Fetching Erome album page: {page_url}")
for attempt in range(5):
response = session.get(page_url, timeout=30)
response.raise_for_status()
page_content = response.text
if "<title>Please wait a few moments</title>" in page_content:
logger(f" Cloudflare check detected. Waiting 5 seconds... (Attempt {attempt + 1}/5)")
time.sleep(5)
continue
break
else:
logger(" Error: Could not bypass Cloudflare check after several attempts.")
return None, []
title = html.unescape(extr(page_content, 'property="og:title" content="', '"'))
user = urllib.parse.unquote(extr(page_content, 'href="https://www.erome.com/', '"', default="unknown_user"))
sanitized_title = re.sub(r'[<>:"/\\|?*]', '_', title).strip()
sanitized_user = re.sub(r'[<>:"/\\|?*]', '_', user).strip()
album_folder_name = f"Erome - {sanitized_user} - {sanitized_title} [{album_id}]"
urls = []
media_groups = page_content.split('<div class="media-group"')
for group in media_groups[1:]:
video_url = extr(group, '<source src="', '"') or extr(group, 'data-src="', '"')
if video_url:
urls.append(video_url)
if not urls:
logger(" Warning: No media URLs found on the album page.")
return album_folder_name, []
logger(f" Found {len(urls)} media files in album '{title}'.")
file_list = []
for i, file_url in enumerate(urls, 1):
filename_info = nameext_from_url(file_url)
filename = f"{album_id}_{sanitized_title}_{i:03d}.{filename_info.get('extension', 'mp4')}"
file_data = {
"url": file_url,
"filename": filename,
"headers": {"Referer": page_url},
}
file_list.append(file_data)
return album_folder_name, file_list
except requests.exceptions.RequestException as e:
logger(f" Error fetching Erome page: {e}")
return None, []
except Exception as e:
logger(f" An unexpected error occurred during Erome extraction: {e}")
return None, []

View File

@@ -1,44 +0,0 @@
import requests
import cloudscraper
import json
def fetch_nhentai_gallery(gallery_id, logger=print):
"""
Fetches the metadata for a single nhentai gallery using cloudscraper to bypass Cloudflare.
Args:
gallery_id (str or int): The ID of the nhentai gallery.
logger (function): A function to log progress and error messages.
Returns:
dict: A dictionary containing the gallery's metadata if successful, otherwise None.
"""
api_url = f"https://nhentai.net/api/gallery/{gallery_id}"
scraper = cloudscraper.create_scraper()
logger(f" Fetching nhentai gallery metadata from: {api_url}")
try:
# Use the scraper to make the GET request
response = scraper.get(api_url, timeout=20)
if response.status_code == 404:
logger(f" ❌ Gallery not found (404): ID {gallery_id}")
return None
response.raise_for_status()
gallery_data = response.json()
if "id" in gallery_data and "media_id" in gallery_data and "images" in gallery_data:
logger(f" ✅ Successfully fetched metadata for '{gallery_data['title']['english']}'")
gallery_data['pages'] = gallery_data.pop('images')['pages']
return gallery_data
else:
logger(" ❌ API response is missing essential keys (id, media_id, or images).")
return None
except Exception as e:
logger(f" ❌ An error occurred while fetching gallery {gallery_id}: {e}")
return None

View File

@@ -1,163 +0,0 @@
import os
import re as re_module
import html
import urllib.parse
import requests
PATTERN_CACHE = {}
def re(pattern):
"""Compile a regular expression pattern and cache it."""
try:
return PATTERN_CACHE[pattern]
except KeyError:
p = PATTERN_CACHE[pattern] = re_module.compile(pattern)
return p
def extract_from(txt, pos=None, default=""):
"""Returns a function that extracts text between two delimiters from 'txt'."""
def extr(begin, end, index=txt.find, txt=txt):
nonlocal pos
try:
start_pos = pos if pos is not None else 0
first = index(begin, start_pos) + len(begin)
last = index(end, first)
if pos is not None:
pos = last + len(end)
return txt[first:last]
except (ValueError, IndexError):
return default
return extr
def nameext_from_url(url):
"""Extract filename and extension from a URL."""
data = {}
filename = urllib.parse.unquote(url.partition("?")[0].rpartition("/")[2])
name, _, ext = filename.rpartition(".")
if name and len(ext) <= 16:
data["filename"], data["extension"] = name, ext.lower()
else:
data["filename"], data["extension"] = filename, ""
return data
class BaseExtractor:
"""A simplified base class for extractors."""
def __init__(self, match, session, logger):
self.match = match
self.groups = match.groups()
self.session = session
self.log = logger
def request(self, url, **kwargs):
"""Makes an HTTP request using the session."""
try:
response = self.session.get(url, **kwargs)
response.raise_for_status()
return response
except requests.exceptions.RequestException as e:
self.log(f"Error making request to {url}: {e}")
return None
class SaintAlbumExtractor(BaseExtractor):
"""Extractor for saint.su albums."""
root = "https://saint2.su"
pattern = re(r"(?:https?://)?saint\d*\.(?:su|pk|cr|to)/a/([^/?#]+)")
def items(self):
"""Generator that yields all files from an album."""
album_id = self.groups[0]
response = self.request(f"{self.root}/a/{album_id}")
if not response:
return None, []
extr = extract_from(response.text)
title = extr("<title>", "<").rpartition(" - ")[0]
self.log(f"Downloading album: {title}")
files_html = re_module.findall(r'<a class="image".*?</a>', response.text, re_module.DOTALL)
file_list = []
for i, file_html in enumerate(files_html, 1):
file_extr = extract_from(file_html)
file_url = html.unescape(file_extr("onclick=\"play('", "'"))
if not file_url:
continue
filename_info = nameext_from_url(file_url)
filename = f"{filename_info['filename']}.{filename_info['extension']}"
file_data = {
"url": file_url,
"filename": filename,
"headers": {"Referer": response.url},
}
file_list.append(file_data)
return title, file_list
class SaintMediaExtractor(BaseExtractor):
"""Extractor for single saint.su media links."""
root = "https://saint2.su"
pattern = re(r"(?:https?://)?saint\d*\.(?:su|pk|cr|to)(/(embe)?d/([^/?#]+))")
def items(self):
"""Generator that yields the single file from a media page."""
path, embed, media_id = self.groups
url = self.root + path
response = self.request(url)
if not response:
return None, []
extr = extract_from(response.text)
file_url = ""
title = extr("<title>", "<").rpartition(" - ")[0] or media_id
if embed: # /embed/ link
file_url = html.unescape(extr('<source src="', '"'))
else: # /d/ link
file_url = html.unescape(extr('<a href="', '"'))
if not file_url:
self.log("Could not find video URL on the page.")
return title, []
filename_info = nameext_from_url(file_url)
filename = f"{filename_info['filename'] or media_id}.{filename_info['extension'] or 'mp4'}"
file_data = {
"url": file_url,
"filename": filename,
"headers": {"Referer": response.url}
}
return title, [file_data]
def fetch_saint2_data(url, logger):
"""
Identifies the correct extractor for a saint2.su URL and returns the data.
Args:
url (str): The saint2.su URL.
logger (function): A function to log progress messages.
Returns:
tuple: A tuple containing (album_title, list_of_file_dicts).
Returns (None, []) if no data could be fetched.
"""
extractors = [SaintMediaExtractor, SaintAlbumExtractor]
session = requests.Session()
session.headers.update({
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36'
})
for extractor_cls in extractors:
match = extractor_cls.pattern.match(url)
if match:
extractor = extractor_cls(match, session, logger)
album_title, files = extractor.items()
sanitized_title = re_module.sub(r'[<>:"/\\|?*]', '_', album_title) if album_title else "saint2_download"
return sanitized_title, files
logger(f"Error: The URL '{url}' does not match a known saint2 pattern.")
return None, []

View File

@@ -15,8 +15,6 @@ from concurrent.futures import ThreadPoolExecutor, as_completed, CancelledError,
from io import BytesIO from io import BytesIO
from urllib .parse import urlparse from urllib .parse import urlparse
import requests import requests
import cloudscraper
try: try:
from PIL import Image from PIL import Image
except ImportError: except ImportError:
@@ -39,7 +37,7 @@ try:
except ImportError: except ImportError:
Document = None Document = None
from PyQt5 .QtCore import Qt ,QThread ,pyqtSignal ,QMutex ,QMutexLocker ,QObject ,QTimer ,QSettings ,QStandardPaths ,QCoreApplication ,QUrl ,QSize ,QProcess from PyQt5 .QtCore import Qt ,QThread ,pyqtSignal ,QMutex ,QMutexLocker ,QObject ,QTimer ,QSettings ,QStandardPaths ,QCoreApplication ,QUrl ,QSize ,QProcess
from .api_client import download_from_api, fetch_post_comments, fetch_single_post_data from .api_client import download_from_api, fetch_post_comments
from ..services.multipart_downloader import download_file_in_parts, MULTIPART_DOWNLOADER_AVAILABLE from ..services.multipart_downloader import download_file_in_parts, MULTIPART_DOWNLOADER_AVAILABLE
from ..services.drive_downloader import ( from ..services.drive_downloader import (
download_mega_file, download_gdrive_file, download_dropbox_file download_mega_file, download_gdrive_file, download_dropbox_file
@@ -60,13 +58,18 @@ def robust_clean_name(name):
"""A more robust function to remove illegal characters for filenames and folders.""" """A more robust function to remove illegal characters for filenames and folders."""
if not name: if not name:
return "" return ""
illegal_chars_pattern = r'[\x00-\x1f<>:"/\\|?*\']' # Removes illegal characters for Windows, macOS, and Linux: < > : " / \ | ? *
# Also removes control characters (ASCII 0-31) which are invisible but invalid.
illegal_chars_pattern = r'[\x00-\x1f<>:"/\\|?*]'
cleaned_name = re.sub(illegal_chars_pattern, '', name) cleaned_name = re.sub(illegal_chars_pattern, '', name)
# Remove leading/trailing spaces or periods, which can cause issues.
cleaned_name = cleaned_name.strip(' .') cleaned_name = cleaned_name.strip(' .')
# If the name is empty after cleaning (e.g., it was only illegal chars),
# provide a safe fallback name.
if not cleaned_name: if not cleaned_name:
return "untitled_folder" return "untitled_folder" # Or "untitled_file" depending on context
return cleaned_name return cleaned_name
class PostProcessorSignals (QObject ): class PostProcessorSignals (QObject ):
@@ -121,8 +124,7 @@ class PostProcessorWorker:
processed_post_ids=None, processed_post_ids=None,
multipart_scope='both', multipart_scope='both',
multipart_parts_count=4, multipart_parts_count=4,
multipart_min_size_mb=100, multipart_min_size_mb=100
skip_file_size_mb=None
): ):
self.post = post_data self.post = post_data
self.download_root = download_root self.download_root = download_root
@@ -187,7 +189,6 @@ class PostProcessorWorker:
self.multipart_scope = multipart_scope self.multipart_scope = multipart_scope
self.multipart_parts_count = multipart_parts_count self.multipart_parts_count = multipart_parts_count
self.multipart_min_size_mb = multipart_min_size_mb self.multipart_min_size_mb = multipart_min_size_mb
self.skip_file_size_mb = skip_file_size_mb
if self.compress_images and Image is None: if self.compress_images and Image is None:
self.logger("⚠️ Image compression disabled: Pillow library not found.") self.logger("⚠️ Image compression disabled: Pillow library not found.")
self.compress_images = False self.compress_images = False
@@ -267,35 +268,15 @@ class PostProcessorWorker:
return 0, 1, "", False, FILE_DOWNLOAD_STATUS_SKIPPED, None return 0, 1, "", False, FILE_DOWNLOAD_STATUS_SKIPPED, None
file_download_headers = { file_download_headers = {
'User-Agent': 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)', 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36',
'Referer': post_page_url, 'Referer': post_page_url
'Accept': 'text/css'
} }
file_url = file_info.get('url') file_url = file_info.get('url')
cookies_to_use_for_file = None cookies_to_use_for_file = None
if self.use_cookie: if self.use_cookie:
cookies_to_use_for_file = prepare_cookies_for_request(self.use_cookie, self.cookie_text, self.selected_cookie_file, self.app_base_dir, self.logger) cookies_to_use_for_file = prepare_cookies_for_request(self.use_cookie, self.cookie_text, self.selected_cookie_file, self.app_base_dir, self.logger)
if self.skip_file_size_mb is not None:
api_original_filename_for_size_check = file_info.get('_original_name_for_log', file_info.get('name'))
try:
# Use a stream=True HEAD request to get headers without downloading the body
with requests.head(file_url, headers=file_download_headers, timeout=15, cookies=cookies_to_use_for_file, allow_redirects=True) as head_response:
head_response.raise_for_status()
content_length = head_response.headers.get('Content-Length')
if content_length:
file_size_bytes = int(content_length)
file_size_mb = file_size_bytes / (1024 * 1024)
if file_size_mb < self.skip_file_size_mb:
self.logger(f" -> Skip File (Size): '{api_original_filename_for_size_check}' is {file_size_mb:.2f} MB, which is smaller than the {self.skip_file_size_mb} MB limit.")
return 0, 1, api_original_filename_for_size_check, False, FILE_DOWNLOAD_STATUS_SKIPPED, None
else:
self.logger(f" ⚠️ Could not determine file size for '{api_original_filename_for_size_check}' to check against size limit. Proceeding with download.")
except requests.RequestException as e:
self.logger(f" ⚠️ Could not fetch file headers to check size for '{api_original_filename_for_size_check}': {e}. Proceeding with download.")
api_original_filename = file_info.get('_original_name_for_log', file_info.get('name')) api_original_filename = file_info.get('_original_name_for_log', file_info.get('name'))
filename_to_save_in_main_path = "" filename_to_save_in_main_path = ""
if forced_filename_override: if forced_filename_override:
@@ -428,26 +409,8 @@ class PostProcessorWorker:
self.logger(f"⚠️ Manga mode: Generated filename was empty. Using generic fallback: '{filename_to_save_in_main_path}'.") self.logger(f"⚠️ Manga mode: Generated filename was empty. Using generic fallback: '{filename_to_save_in_main_path}'.")
was_original_name_kept_flag = False was_original_name_kept_flag = False
else: else:
is_url_like = 'http' in api_original_filename.lower() filename_to_save_in_main_path = cleaned_original_api_filename
is_too_long = len(cleaned_original_api_filename) > 100 was_original_name_kept_flag = True
if is_url_like or is_too_long:
self.logger(f" ⚠️ Original filename is a URL or too long. Generating a shorter name.")
name_hash = hashlib.md5(api_original_filename.encode()).hexdigest()[:12]
_, ext = os.path.splitext(cleaned_original_api_filename)
if not ext:
try:
path = urlparse(api_original_filename).path
ext = os.path.splitext(path)[1] or ".file"
except Exception:
ext = ".file"
cleaned_post_title = robust_clean_name(post_title.strip() if post_title else "post")[:40]
filename_to_save_in_main_path = f"{cleaned_post_title}_{name_hash}{ext}"
was_original_name_kept_flag = False
else:
filename_to_save_in_main_path = cleaned_original_api_filename
was_original_name_kept_flag = True
if self.remove_from_filename_words_list and filename_to_save_in_main_path: if self.remove_from_filename_words_list and filename_to_save_in_main_path:
base_name_for_removal, ext_for_removal = os.path.splitext(filename_to_save_in_main_path) base_name_for_removal, ext_for_removal = os.path.splitext(filename_to_save_in_main_path)
@@ -525,18 +488,19 @@ class PostProcessorWorker:
except requests.RequestException as e: except requests.RequestException as e:
self.logger(f" ⚠️ Could not verify size of existing file '{filename_to_save_in_main_path}': {e}. Proceeding with download.") self.logger(f" ⚠️ Could not verify size of existing file '{filename_to_save_in_main_path}': {e}. Proceeding with download.")
max_retries = 3
retry_delay = 5 retry_delay = 5
downloaded_size_bytes = 0 downloaded_size_bytes = 0
calculated_file_hash = None calculated_file_hash = None
downloaded_part_file_path = None downloaded_part_file_path = None
total_size_bytes = 0
download_successful_flag = False download_successful_flag = False
last_exception_for_retry_later = None last_exception_for_retry_later = None
is_permanent_error = False is_permanent_error = False
data_to_write_io = None data_to_write_io = None
response_for_this_attempt = None
for attempt_num_single_stream in range(max_retries + 1): for attempt_num_single_stream in range(max_retries + 1):
response = None response_for_this_attempt = None
if self._check_pause(f"File download attempt for '{api_original_filename}'"): break if self._check_pause(f"File download attempt for '{api_original_filename}'"): break
if self.check_cancel() or (skip_event and skip_event.is_set()): break if self.check_cancel() or (skip_event and skip_event.is_set()): break
try: try:
@@ -555,24 +519,12 @@ class PostProcessorWorker:
new_url = self._find_valid_subdomain(current_url_to_try) new_url = self._find_valid_subdomain(current_url_to_try)
if new_url != current_url_to_try: if new_url != current_url_to_try:
self.logger(f" Retrying with new URL: {new_url}") self.logger(f" Retrying with new URL: {new_url}")
file_url = new_url file_url = new_url # Update the main file_url for subsequent retries
response.close() # Close the old response
response = requests.get(new_url, headers=file_download_headers, timeout=(30, 300), stream=True, cookies=cookies_to_use_for_file) response = requests.get(new_url, headers=file_download_headers, timeout=(30, 300), stream=True, cookies=cookies_to_use_for_file)
response.raise_for_status() response.raise_for_status()
# --- REVISED AND MOVED SIZE CHECK LOGIC ---
total_size_bytes = int(response.headers.get('Content-Length', 0)) total_size_bytes = int(response.headers.get('Content-Length', 0))
if self.skip_file_size_mb is not None:
if total_size_bytes > 0:
file_size_mb = total_size_bytes / (1024 * 1024)
if file_size_mb < self.skip_file_size_mb:
self.logger(f" -> Skip File (Size): '{api_original_filename}' is {file_size_mb:.2f} MB, which is smaller than the {self.skip_file_size_mb} MB limit.")
return 0, 1, api_original_filename, False, FILE_DOWNLOAD_STATUS_SKIPPED, None
# If Content-Length is missing, we can't check, so we no longer log a warning here and just proceed.
# --- END OF REVISED LOGIC ---
num_parts_for_file = min(self.multipart_parts_count, MAX_PARTS_FOR_MULTIPART_DOWNLOAD) num_parts_for_file = min(self.multipart_parts_count, MAX_PARTS_FOR_MULTIPART_DOWNLOAD)
file_is_eligible_by_scope = False file_is_eligible_by_scope = False
@@ -596,7 +548,9 @@ class PostProcessorWorker:
if self._check_pause(f"Multipart decision for '{api_original_filename}'"): break if self._check_pause(f"Multipart decision for '{api_original_filename}'"): break
if attempt_multipart: if attempt_multipart:
response.close() # Close the initial connection before starting multipart if response_for_this_attempt:
response_for_this_attempt.close()
response_for_this_attempt = None
mp_save_path_for_unique_part_stem_arg = os.path.join(target_folder_path, f"{unique_part_file_stem_on_disk}{temp_file_ext_for_unique_part}") mp_save_path_for_unique_part_stem_arg = os.path.join(target_folder_path, f"{unique_part_file_stem_on_disk}{temp_file_ext_for_unique_part}")
mp_success, mp_bytes, mp_hash, mp_file_handle = download_file_in_parts( mp_success, mp_bytes, mp_hash, mp_file_handle = download_file_in_parts(
file_url, mp_save_path_for_unique_part_stem_arg, total_size_bytes, num_parts_for_file, file_download_headers, api_original_filename, file_url, mp_save_path_for_unique_part_stem_arg, total_size_bytes, num_parts_for_file, file_download_headers, api_original_filename,
@@ -622,6 +576,7 @@ class PostProcessorWorker:
current_attempt_downloaded_bytes = 0 current_attempt_downloaded_bytes = 0
md5_hasher = hashlib.md5() md5_hasher = hashlib.md5()
last_progress_time = time.time() last_progress_time = time.time()
single_stream_exception = None
try: try:
with open(current_single_stream_part_path, 'wb') as f_part: with open(current_single_stream_part_path, 'wb') as f_part:
for chunk in response.iter_content(chunk_size=1 * 1024 * 1024): for chunk in response.iter_content(chunk_size=1 * 1024 * 1024):
@@ -688,8 +643,8 @@ class PostProcessorWorker:
is_permanent_error = True is_permanent_error = True
break break
finally: finally:
if response: if response_for_this_attempt:
response.close() response_for_this_attempt.close()
self._emit_signal('file_download_status', False) self._emit_signal('file_download_status', False)
final_total_for_progress = total_size_bytes if download_successful_flag and total_size_bytes > 0 else downloaded_size_bytes final_total_for_progress = total_size_bytes if download_successful_flag and total_size_bytes > 0 else downloaded_size_bytes
@@ -848,8 +803,6 @@ class PostProcessorWorker:
'original_post_id_for_log': original_post_id_for_log, 'post_title': post_title, 'original_post_id_for_log': original_post_id_for_log, 'post_title': post_title,
'file_index_in_post': file_index_in_post, 'num_files_in_this_post': num_files_in_this_post, 'file_index_in_post': file_index_in_post, 'num_files_in_this_post': num_files_in_this_post,
'forced_filename_override': filename_to_save_in_main_path, 'forced_filename_override': filename_to_save_in_main_path,
'service': self.service,
'user_id': self.user_id
} }
return 0, 1, final_filename_saved_for_return, was_original_name_kept_flag, FILE_DOWNLOAD_STATUS_FAILED_PERMANENTLY_THIS_SESSION, permanent_failure_details return 0, 1, final_filename_saved_for_return, was_original_name_kept_flag, FILE_DOWNLOAD_STATUS_FAILED_PERMANENTLY_THIS_SESSION, permanent_failure_details
finally: finally:
@@ -873,60 +826,29 @@ class PostProcessorWorker:
return 0, 1, filename_to_save_in_main_path, was_original_name_kept_flag, FILE_DOWNLOAD_STATUS_FAILED_RETRYABLE_LATER, details_for_failure return 0, 1, filename_to_save_in_main_path, was_original_name_kept_flag, FILE_DOWNLOAD_STATUS_FAILED_RETRYABLE_LATER, details_for_failure
def process(self): def process(self):
# --- START: REFACTORED PROCESS METHOD ---
# 1. DATA MAPPING: Map Discord Message or Creator Post fields to a consistent set of variables.
if self.service == 'discord': if self.service == 'discord':
# For Discord, self.post is a MESSAGE object from the API.
post_title = self.post.get('content', '') or f"Message {self.post.get('id', 'N/A')}" post_title = self.post.get('content', '') or f"Message {self.post.get('id', 'N/A')}"
post_id = self.post.get('id', 'unknown_id') post_id = self.post.get('id', 'unknown_id')
post_main_file_info = {} post_main_file_info = {} # Discord messages don't have a single main file
post_attachments = self.post.get('attachments', []) post_attachments = self.post.get('attachments', [])
post_content_html = self.post.get('content', '') post_content_html = self.post.get('content', '')
post_data = self.post post_data = self.post # Keep a reference to the original message object
log_prefix = "Message" log_prefix = "Message"
else: else:
# Existing logic for standard creator posts
post_title = self.post.get('title', '') or 'untitled_post' post_title = self.post.get('title', '') or 'untitled_post'
post_id = self.post.get('id', 'unknown_id') post_id = self.post.get('id', 'unknown_id')
post_main_file_info = self.post.get('file') post_main_file_info = self.post.get('file')
post_attachments = self.post.get('attachments', []) post_attachments = self.post.get('attachments', [])
post_content_html = self.post.get('content', '') post_content_html = self.post.get('content', '')
post_data = self.post post_data = self.post # Reference to the post object
log_prefix = "Post" log_prefix = "Post"
content_is_needed = ( # 2. SHARED PROCESSING LOGIC: The rest of the function now uses the consistent variables from above.
self.show_external_links or
self.extract_links_only or
self.scan_content_for_images or
(self.filter_mode == 'text_only' and self.text_only_scope == 'content')
)
if content_is_needed and self.post.get('content') is None and self.service != 'discord':
self.logger(f" Post {post_id} is missing 'content' field, fetching full data...")
parsed_url = urlparse(self.api_url_input)
api_domain = parsed_url.netloc
creator_page_url = f"https://{api_domain}/{self.service}/user/{self.user_id}"
headers = {
'User-Agent': 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)',
'Referer': creator_page_url,
'Accept': 'text/css'
}
cookies = prepare_cookies_for_request(self.use_cookie, self.cookie_text, self.selected_cookie_file, self.app_base_dir, self.logger, target_domain=api_domain)
full_post_data = fetch_single_post_data(api_domain, self.service, self.user_id, post_id, headers, self.logger, cookies_dict=cookies)
if full_post_data:
self.logger(" ✅ Full post data fetched successfully.")
self.post = full_post_data
post_title = self.post.get('title', '') or 'untitled_post'
post_main_file_info = self.post.get('file')
post_attachments = self.post.get('attachments', [])
post_content_html = self.post.get('content', '')
post_data = self.post
else:
self.logger(f" ⚠️ Failed to fetch full content for post {post_id}. Content-dependent features may not work for this post.")
result_tuple = (0, 0, [], [], [], None, None) result_tuple = (0, 0, [], [], [], None, None)
total_downloaded_this_post = 0 total_downloaded_this_post = 0
total_skipped_this_post = 0 total_skipped_this_post = 0
@@ -955,11 +877,7 @@ class PostProcessorWorker:
else: else:
post_page_url = f"https://{parsed_api_url.netloc}/{self.service}/user/{self.user_id}/post/{post_id}" post_page_url = f"https://{parsed_api_url.netloc}/{self.service}/user/{self.user_id}/post/{post_id}"
headers = { headers = {'User-Agent': 'Mozilla/5.0', 'Referer': post_page_url, 'Accept': '*/*'}
'User-Agent': 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)',
'Referer': post_page_url,
'Accept': 'text/css'
}
link_pattern = re.compile(r"""<a\s+.*?href=["'](https?://[^"']+)["'][^>]*>(.*?)</a>""", re.IGNORECASE | re.DOTALL) link_pattern = re.compile(r"""<a\s+.*?href=["'](https?://[^"']+)["'][^>]*>(.*?)</a>""", re.IGNORECASE | re.DOTALL)
effective_unwanted_keywords_for_folder_naming = self.unwanted_keywords.copy() effective_unwanted_keywords_for_folder_naming = self.unwanted_keywords.copy()
@@ -1340,6 +1258,7 @@ class PostProcessorWorker:
parsed_url = urlparse(self.api_url_input) parsed_url = urlparse(self.api_url_input)
api_domain = parsed_url.netloc api_domain = parsed_url.netloc
cookies = prepare_cookies_for_request(self.use_cookie, self.cookie_text, self.selected_cookie_file, self.app_base_dir, self.logger, target_domain=api_domain) cookies = prepare_cookies_for_request(self.use_cookie, self.cookie_text, self.selected_cookie_file, self.app_base_dir, self.logger, target_domain=api_domain)
from .api_client import fetch_single_post_data
full_data = fetch_single_post_data(api_domain, self.service, self.user_id, post_id, headers, self.logger, cookies_dict=cookies) full_data = fetch_single_post_data(api_domain, self.service, self.user_id, post_id, headers, self.logger, cookies_dict=cookies)
if full_data: if full_data:
final_post_data = full_data final_post_data = full_data
@@ -2016,9 +1935,7 @@ class DownloadThread(QThread):
project_root_dir=None, project_root_dir=None,
processed_post_ids=None, processed_post_ids=None,
start_offset=0, start_offset=0,
fetch_first=False, fetch_first=False):
skip_file_size_mb=None
):
super().__init__() super().__init__()
self.api_url_input = api_url_input self.api_url_input = api_url_input
self.output_dir = output_dir self.output_dir = output_dir
@@ -2085,7 +2002,6 @@ class DownloadThread(QThread):
self.processed_post_ids_set = set(processed_post_ids) if processed_post_ids is not None else set() self.processed_post_ids_set = set(processed_post_ids) if processed_post_ids is not None else set()
self.start_offset = start_offset self.start_offset = start_offset
self.fetch_first = fetch_first self.fetch_first = fetch_first
self.skip_file_size_mb = skip_file_size_mb
if self.compress_images and Image is None: if self.compress_images and Image is None:
self.logger("⚠️ Image compression disabled: Pillow library not found (DownloadThread).") self.logger("⚠️ Image compression disabled: Pillow library not found (DownloadThread).")
@@ -2206,7 +2122,6 @@ class DownloadThread(QThread):
'single_pdf_mode': self.single_pdf_mode, 'single_pdf_mode': self.single_pdf_mode,
'multipart_parts_count': self.multipart_parts_count, 'multipart_parts_count': self.multipart_parts_count,
'multipart_min_size_mb': self.multipart_min_size_mb, 'multipart_min_size_mb': self.multipart_min_size_mb,
'skip_file_size_mb': self.skip_file_size_mb,
'project_root_dir': self.project_root_dir, 'project_root_dir': self.project_root_dir,
} }

File diff suppressed because one or more lines are too long

View File

@@ -5,12 +5,9 @@ import traceback
import json import json
import base64 import base64
import time import time
import zipfile
from urllib.parse import urlparse, urlunparse, parse_qs, urlencode from urllib.parse import urlparse, urlunparse, parse_qs, urlencode
# --- Third-party Library Imports ---
import requests import requests
import cloudscraper
try: try:
from Crypto.Cipher import AES from Crypto.Cipher import AES
@@ -29,12 +26,12 @@ MEGA_API_URL = "https://g.api.mega.co.nz"
def _get_filename_from_headers(headers): def _get_filename_from_headers(headers):
""" """
Extracts a filename from the Content-Disposition header. Extracts a filename from the Content-Disposition header.
(This is from your original file and is kept for Dropbox downloads)
""" """
cd = headers.get('content-disposition') cd = headers.get('content-disposition')
if not cd: if not cd:
return None return None
# Handles both filename="file.zip" and filename*=UTF-8''file%20name.zip
fname_match = re.findall('filename="?([^"]+)"?', cd) fname_match = re.findall('filename="?([^"]+)"?', cd)
if fname_match: if fname_match:
sanitized_name = re.sub(r'[<>:"/\\|?*]', '_', fname_match[0].strip()) sanitized_name = re.sub(r'[<>:"/\\|?*]', '_', fname_match[0].strip())
@@ -42,23 +39,28 @@ def _get_filename_from_headers(headers):
return None return None
# --- Helper functions for Mega decryption --- # --- NEW: Helper functions for Mega decryption ---
def urlb64_to_b64(s): def urlb64_to_b64(s):
"""Converts a URL-safe base64 string to a standard base64 string."""
s = s.replace('-', '+').replace('_', '/') s = s.replace('-', '+').replace('_', '/')
s += '=' * (-len(s) % 4) s += '=' * (-len(s) % 4)
return s return s
def b64_to_bytes(s): def b64_to_bytes(s):
"""Decodes a URL-safe base64 string to bytes."""
return base64.b64decode(urlb64_to_b64(s)) return base64.b64decode(urlb64_to_b64(s))
def bytes_to_hex(b): def bytes_to_hex(b):
"""Converts bytes to a hex string."""
return b.hex() return b.hex()
def hex_to_bytes(h): def hex_to_bytes(h):
"""Converts a hex string to bytes."""
return bytes.fromhex(h) return bytes.fromhex(h)
def hrk2hk(hex_raw_key): def hrk2hk(hex_raw_key):
"""Derives the final AES key from the raw key components for Mega."""
key_part1 = int(hex_raw_key[0:16], 16) key_part1 = int(hex_raw_key[0:16], 16)
key_part2 = int(hex_raw_key[16:32], 16) key_part2 = int(hex_raw_key[16:32], 16)
key_part3 = int(hex_raw_key[32:48], 16) key_part3 = int(hex_raw_key[32:48], 16)
@@ -70,20 +72,23 @@ def hrk2hk(hex_raw_key):
return f'{final_key_part1:016x}{final_key_part2:016x}' return f'{final_key_part1:016x}{final_key_part2:016x}'
def decrypt_at(at_b64, key_bytes): def decrypt_at(at_b64, key_bytes):
"""Decrypts the 'at' attribute to get file metadata."""
at_bytes = b64_to_bytes(at_b64) at_bytes = b64_to_bytes(at_b64)
iv = b'\0' * 16 iv = b'\0' * 16
cipher = AES.new(key_bytes, AES.MODE_CBC, iv) cipher = AES.new(key_bytes, AES.MODE_CBC, iv)
decrypted_at = cipher.decrypt(at_bytes) decrypted_at = cipher.decrypt(at_bytes)
return decrypted_at.decode('utf-8').strip('\0').replace('MEGA', '') return decrypted_at.decode('utf-8').strip('\0').replace('MEGA', '')
# --- Core Logic for Mega Downloads --- # --- NEW: Core Logic for Mega Downloads ---
def get_mega_file_info(file_id, file_key, session, logger_func): def get_mega_file_info(file_id, file_key, session, logger_func):
"""Fetches file metadata and the temporary download URL from the Mega API."""
try: try:
hex_raw_key = bytes_to_hex(b64_to_bytes(file_key)) hex_raw_key = bytes_to_hex(b64_to_bytes(file_key))
hex_key = hrk2hk(hex_raw_key) hex_key = hrk2hk(hex_raw_key)
key_bytes = hex_to_bytes(hex_key) key_bytes = hex_to_bytes(hex_key)
# Request file attributes
payload = [{"a": "g", "p": file_id}] payload = [{"a": "g", "p": file_id}]
response = session.post(f"{MEGA_API_URL}/cs", json=payload, timeout=20) response = session.post(f"{MEGA_API_URL}/cs", json=payload, timeout=20)
response.raise_for_status() response.raise_for_status()
@@ -95,10 +100,13 @@ def get_mega_file_info(file_id, file_key, session, logger_func):
file_size = res_json[0]['s'] file_size = res_json[0]['s']
at_b64 = res_json[0]['at'] at_b64 = res_json[0]['at']
# Decrypt attributes to get the file name
at_dec_json_str = decrypt_at(at_b64, key_bytes) at_dec_json_str = decrypt_at(at_b64, key_bytes)
at_dec_json = json.loads(at_dec_json_str) at_dec_json = json.loads(at_dec_json_str)
file_name = at_dec_json['n'] file_name = at_dec_json['n']
# Request the temporary download URL
payload = [{"a": "g", "g": 1, "p": file_id}] payload = [{"a": "g", "g": 1, "p": file_id}]
response = session.post(f"{MEGA_API_URL}/cs", json=payload, timeout=20) response = session.post(f"{MEGA_API_URL}/cs", json=payload, timeout=20)
response.raise_for_status() response.raise_for_status()
@@ -116,16 +124,19 @@ def get_mega_file_info(file_id, file_key, session, logger_func):
return None return None
def download_and_decrypt_mega_file(info, download_path, logger_func): def download_and_decrypt_mega_file(info, download_path, logger_func):
"""Downloads the file and decrypts it chunk by chunk, reporting progress."""
file_name = info['file_name'] file_name = info['file_name']
file_size = info['file_size'] file_size = info['file_size']
dl_url = info['dl_url'] dl_url = info['dl_url']
hex_raw_key = info['hex_raw_key'] hex_raw_key = info['hex_raw_key']
final_path = os.path.join(download_path, file_name) final_path = os.path.join(download_path, file_name)
if os.path.exists(final_path) and os.path.getsize(final_path) == file_size: if os.path.exists(final_path) and os.path.getsize(final_path) == file_size:
logger_func(f" [Mega] File '{file_name}' already exists with the correct size. Skipping.") logger_func(f" [Mega] File '{file_name}' already exists with the correct size. Skipping.")
return return
# Prepare for decryption
key = hex_to_bytes(hrk2hk(hex_raw_key)) key = hex_to_bytes(hrk2hk(hex_raw_key))
iv_hex = hex_raw_key[32:48] + '0000000000000000' iv_hex = hex_raw_key[32:48] + '0000000000000000'
iv_bytes = hex_to_bytes(iv_hex) iv_bytes = hex_to_bytes(iv_hex)
@@ -139,11 +150,13 @@ def download_and_decrypt_mega_file(info, download_path, logger_func):
with open(final_path, 'wb') as f: with open(final_path, 'wb') as f:
for chunk in r.iter_content(chunk_size=8192): for chunk in r.iter_content(chunk_size=8192):
if not chunk: continue if not chunk:
continue
decrypted_chunk = cipher.decrypt(chunk) decrypted_chunk = cipher.decrypt(chunk)
f.write(decrypted_chunk) f.write(decrypted_chunk)
downloaded_bytes += len(chunk) downloaded_bytes += len(chunk)
# Log progress every second
current_time = time.time() current_time = time.time()
if current_time - last_log_time > 1: if current_time - last_log_time > 1:
progress_percent = (downloaded_bytes / file_size) * 100 if file_size > 0 else 0 progress_percent = (downloaded_bytes / file_size) * 100 if file_size > 0 else 0
@@ -151,16 +164,28 @@ def download_and_decrypt_mega_file(info, download_path, logger_func):
last_log_time = current_time last_log_time = current_time
logger_func(f" [Mega] ✅ Successfully downloaded '{file_name}' to '{download_path}'") logger_func(f" [Mega] ✅ Successfully downloaded '{file_name}' to '{download_path}'")
except requests.RequestException as e:
logger_func(f" [Mega] ❌ Download failed for '{file_name}': {e}")
except IOError as e:
logger_func(f" [Mega] ❌ Could not write to file '{final_path}': {e}")
except Exception as e: except Exception as e:
logger_func(f" [Mega] ❌ An unexpected error occurred during download/decryption: {e}") logger_func(f" [Mega] ❌ An unexpected error occurred during download/decryption: {e}")
# --- REPLACEMENT Main Service Downloader Function for Mega ---
def download_mega_file(mega_url, download_path, logger_func=print): def download_mega_file(mega_url, download_path, logger_func=print):
"""
Downloads a file from a Mega.nz URL using direct requests and decryption.
This replaces the old mega.py implementation.
"""
if not PYCRYPTODOME_AVAILABLE: if not PYCRYPTODOME_AVAILABLE:
logger_func("❌ Mega download failed: 'pycryptodome' library is not installed. Please run: pip install pycryptodome") logger_func("❌ Mega download failed: 'pycryptodome' library is not installed. Please run: pip install pycryptodome")
return return
logger_func(f" [Mega] Initializing download for: {mega_url}") logger_func(f" [Mega] Initializing download for: {mega_url}")
# Regex to capture file ID and key from both old and new URL formats
match = re.search(r'mega(?:\.co)?\.nz/(?:file/|#!)?([a-zA-Z0-9]+)(?:#|!)([a-zA-Z0-9_.-]+)', mega_url) match = re.search(r'mega(?:\.co)?\.nz/(?:file/|#!)?([a-zA-Z0-9]+)(?:#|!)([a-zA-Z0-9_.-]+)', mega_url)
if not match: if not match:
logger_func(f" [Mega] ❌ Error: Invalid Mega URL format.") logger_func(f" [Mega] ❌ Error: Invalid Mega URL format.")
@@ -174,14 +199,18 @@ def download_mega_file(mega_url, download_path, logger_func=print):
file_info = get_mega_file_info(file_id, file_key, session, logger_func) file_info = get_mega_file_info(file_id, file_key, session, logger_func)
if not file_info: if not file_info:
logger_func(f" [Mega] ❌ Failed to get file info. Aborting.") logger_func(f" [Mega] ❌ Failed to get file info. The link may be invalid or expired. Aborting.")
return return
logger_func(f" [Mega] File found: '{file_info['file_name']}' (Size: {file_info['file_size'] / 1024 / 1024:.2f} MB)") logger_func(f" [Mega] File found: '{file_info['file_name']}' (Size: {file_info['file_size'] / 1024 / 1024:.2f} MB)")
download_and_decrypt_mega_file(file_info, download_path, logger_func) download_and_decrypt_mega_file(file_info, download_path, logger_func)
# --- ORIGINAL Functions for Google Drive and Dropbox (Unchanged) ---
def download_gdrive_file(url, download_path, logger_func=print): def download_gdrive_file(url, download_path, logger_func=print):
"""Downloads a file from a Google Drive link."""
if not GDRIVE_AVAILABLE: if not GDRIVE_AVAILABLE:
logger_func("❌ Google Drive download failed: 'gdown' library is not installed.") logger_func("❌ Google Drive download failed: 'gdown' library is not installed.")
return return
@@ -198,15 +227,12 @@ def download_gdrive_file(url, download_path, logger_func=print):
except Exception as e: except Exception as e:
logger_func(f" [G-Drive] ❌ An unexpected error occurred: {e}") logger_func(f" [G-Drive] ❌ An unexpected error occurred: {e}")
# --- MODIFIED DROPBOX DOWNLOADER ---
def download_dropbox_file(dropbox_link, download_path=".", logger_func=print): def download_dropbox_file(dropbox_link, download_path=".", logger_func=print):
""" """
Downloads a file or a folder (as a zip) from a public Dropbox link. Downloads a file from a public Dropbox link by modifying the URL for direct download.
Uses cloudscraper to handle potential browser checks and auto-extracts zip files.
""" """
logger_func(f" [Dropbox] Attempting to download: {dropbox_link}") logger_func(f" [Dropbox] Attempting to download: {dropbox_link}")
# Modify URL to force download (works for both files and folders)
parsed_url = urlparse(dropbox_link) parsed_url = urlparse(dropbox_link)
query_params = parse_qs(parsed_url.query) query_params = parse_qs(parsed_url.query)
query_params['dl'] = ['1'] query_params['dl'] = ['1']
@@ -215,60 +241,26 @@ def download_dropbox_file(dropbox_link, download_path=".", logger_func=print):
logger_func(f" [Dropbox] Using direct download URL: {direct_download_url}") logger_func(f" [Dropbox] Using direct download URL: {direct_download_url}")
scraper = cloudscraper.create_scraper()
try: try:
if not os.path.exists(download_path): if not os.path.exists(download_path):
os.makedirs(download_path, exist_ok=True) os.makedirs(download_path, exist_ok=True)
logger_func(f" [Dropbox] Created download directory: {download_path}") logger_func(f" [Dropbox] Created download directory: {download_path}")
with scraper.get(direct_download_url, stream=True, allow_redirects=True, timeout=(20, 600)) as r: with requests.get(direct_download_url, stream=True, allow_redirects=True, timeout=(10, 300)) as r:
r.raise_for_status() r.raise_for_status()
filename = _get_filename_from_headers(r.headers) or os.path.basename(parsed_url.path) or "dropbox_download" filename = _get_filename_from_headers(r.headers) or os.path.basename(parsed_url.path) or "dropbox_file"
# If it's a folder, Dropbox will name it FolderName.zip
if not os.path.splitext(filename)[1]:
filename += ".zip"
full_save_path = os.path.join(download_path, filename) full_save_path = os.path.join(download_path, filename)
logger_func(f" [Dropbox] Starting download of '{filename}'...") logger_func(f" [Dropbox] Starting download of '{filename}'...")
total_size = int(r.headers.get('content-length', 0))
downloaded_bytes = 0
last_log_time = time.time()
with open(full_save_path, 'wb') as f: with open(full_save_path, 'wb') as f:
for chunk in r.iter_content(chunk_size=8192): for chunk in r.iter_content(chunk_size=8192):
f.write(chunk) f.write(chunk)
downloaded_bytes += len(chunk)
current_time = time.time()
if total_size > 0 and current_time - last_log_time > 1:
progress = (downloaded_bytes / total_size) * 100
logger_func(f" -> Downloading '{filename}'... {downloaded_bytes/1024/1024:.2f}MB / {total_size/1024/1024:.2f}MB ({progress:.1f}%)")
last_log_time = current_time
logger_func(f" [Dropbox] ✅ Download complete: {full_save_path}") logger_func(f" [Dropbox] ✅ Dropbox file downloaded successfully: {full_save_path}")
# --- NEW: Auto-extraction logic ---
if zipfile.is_zipfile(full_save_path):
logger_func(f" [Dropbox] ዚ Detected zip file. Attempting to extract...")
extract_folder_name = os.path.splitext(filename)[0]
extract_path = os.path.join(download_path, extract_folder_name)
os.makedirs(extract_path, exist_ok=True)
with zipfile.ZipFile(full_save_path, 'r') as zip_ref:
zip_ref.extractall(extract_path)
logger_func(f" [Dropbox] ✅ Successfully extracted to folder: '{extract_path}'")
# Optional: remove the zip file after extraction
try:
os.remove(full_save_path)
logger_func(f" [Dropbox] 🗑️ Removed original zip file.")
except OSError as e:
logger_func(f" [Dropbox] ⚠️ Could not remove original zip file: {e}")
except Exception as e: except Exception as e:
logger_func(f" [Dropbox] ❌ An error occurred during Dropbox download: {e}") logger_func(f" [Dropbox] ❌ An error occurred during Dropbox download: {e}")
traceback.print_exc(limit=2) traceback.print_exc(limit=2)
raise

View File

@@ -1,125 +0,0 @@
import sys
import os
import requests
import subprocess # Keep this for now, though it's not used in the final command
from packaging.version import parse as parse_version
from PyQt5.QtCore import QThread, pyqtSignal
# Constants for the updater
GITHUB_REPO_URL = "https://api.github.com/repos/Yuvi9587/Kemono-Downloader/releases/latest"
EXE_NAME = "Kemono.Downloader.exe"
class UpdateChecker(QThread):
"""Checks for a new version on GitHub in a background thread."""
update_available = pyqtSignal(str, str) # new_version, download_url
up_to_date = pyqtSignal(str)
update_error = pyqtSignal(str)
def __init__(self, current_version):
super().__init__()
self.current_version_str = current_version.lstrip('v')
def run(self):
try:
response = requests.get(GITHUB_REPO_URL, timeout=15)
response.raise_for_status()
data = response.json()
latest_version_str = data['tag_name'].lstrip('v')
current_version = parse_version(self.current_version_str)
latest_version = parse_version(latest_version_str)
if latest_version > current_version:
for asset in data.get('assets', []):
if asset['name'] == EXE_NAME:
self.update_available.emit(latest_version_str, asset['browser_download_url'])
return
self.update_error.emit(f"Update found, but '{EXE_NAME}' is missing from the release assets.")
else:
self.up_to_date.emit("You are on the latest version.")
except requests.exceptions.RequestException as e:
self.update_error.emit(f"Network error: {e}")
except Exception as e:
self.update_error.emit(f"An error occurred: {e}")
class UpdateDownloader(QThread):
"""
Downloads the new executable and runs an updater script that kills the old process,
replaces the file, and displays a message in the terminal.
"""
download_finished = pyqtSignal()
download_error = pyqtSignal(str)
def __init__(self, download_url, parent_app):
super().__init__()
self.download_url = download_url
self.parent_app = parent_app
def run(self):
try:
app_path = sys.executable
app_dir = os.path.dirname(app_path)
temp_path = os.path.join(app_dir, f"{EXE_NAME}.tmp")
old_path = os.path.join(app_dir, f"{EXE_NAME}.old")
updater_script_path = os.path.join(app_dir, "updater.bat")
# --- NEW: Path for the PID file ---
pid_file_path = os.path.join(app_dir, "updater.pid")
# Download the new executable
with requests.get(self.download_url, stream=True, timeout=300) as r:
r.raise_for_status()
with open(temp_path, 'wb') as f:
for chunk in r.iter_content(chunk_size=8192):
f.write(chunk)
# --- NEW: Write the current Process ID to the pid file ---
with open(pid_file_path, "w") as f:
f.write(str(os.getpid()))
# --- NEW BATCH SCRIPT ---
# This script now reads the PID from the "updater.pid" file.
script_content = f"""
@echo off
SETLOCAL
echo.
echo Reading process information...
set /p PID=<{pid_file_path}
echo Closing the old application (PID: %PID%)...
taskkill /F /PID %PID%
echo Waiting for files to unlock...
timeout /t 2 /nobreak > nul
echo Replacing application files...
if exist "{old_path}" del /F /Q "{old_path}"
rename "{app_path}" "{os.path.basename(old_path)}"
rename "{temp_path}" "{EXE_NAME}"
echo.
echo ============================================================
echo Update Complete!
echo You can now close this window and run {EXE_NAME}.
echo ============================================================
echo.
pause
echo Cleaning up helper files...
del "{pid_file_path}"
del "%~f0"
ENDLOCAL
"""
with open(updater_script_path, "w") as f:
f.write(script_content)
# --- Go back to the os.startfile command that we know works ---
os.startfile(updater_script_path)
self.download_finished.emit()
except Exception as e:
self.download_error.emit(f"Failed to download or run updater: {e}")

View File

@@ -42,12 +42,17 @@ class ErrorFilesDialog(QDialog):
if app_icon and not app_icon.isNull(): if app_icon and not app_icon.isNull():
self.setWindowIcon(app_icon) self.setWindowIcon(app_icon)
# --- START OF FIX ---
# Get the user-defined scale factor from the parent application.
scale_factor = getattr(self.parent_app, 'scale_factor', 1.0) scale_factor = getattr(self.parent_app, 'scale_factor', 1.0)
# Define base dimensions and apply the correct scale factor.
base_width, base_height = 550, 400 base_width, base_height = 550, 400
self.setMinimumSize(int(base_width * scale_factor), int(base_height * scale_factor)) self.setMinimumSize(int(base_width * scale_factor), int(base_height * scale_factor))
self.resize(int(base_width * scale_factor * 1.1), int(base_height * scale_factor * 1.1)) self.resize(int(base_width * scale_factor * 1.1), int(base_height * scale_factor * 1.1))
# --- END OF FIX ---
# --- Initialize UI and Apply Theming ---
self._init_ui() self._init_ui()
self._retranslate_ui() self._retranslate_ui()
self._apply_theme() self._apply_theme()
@@ -101,17 +106,7 @@ class ErrorFilesDialog(QDialog):
post_title = error_info.get('post_title', 'Unknown Post') post_title = error_info.get('post_title', 'Unknown Post')
post_id = error_info.get('original_post_id_for_log', 'N/A') post_id = error_info.get('original_post_id_for_log', 'N/A')
creator_name = "Unknown Creator" item_text = f"File: {filename}\nFrom Post: '{post_title}' (ID: {post_id})"
service = error_info.get('service')
user_id = error_info.get('user_id')
# Check if we have the necessary info and access to the cache
if service and user_id and hasattr(self.parent_app, 'creator_name_cache'):
creator_key = (service.lower(), str(user_id))
# Look up the name, fall back to the user_id if not found
creator_name = self.parent_app.creator_name_cache.get(creator_key, user_id)
item_text = f"File: {filename}\nCreator: {creator_name} - Post: '{post_title}' (ID: {post_id})"
list_item = QListWidgetItem(item_text) list_item = QListWidgetItem(item_text)
list_item.setData(Qt.UserRole, error_info) list_item.setData(Qt.UserRole, error_info)
list_item.setFlags(list_item.flags() | Qt.ItemIsUserCheckable) list_item.setFlags(list_item.flags() | Qt.ItemIsUserCheckable)

View File

@@ -3,7 +3,7 @@ import html
import re import re
# --- Third-Party Library Imports --- # --- Third-Party Library Imports ---
import cloudscraper # MODIFIED: Import cloudscraper import requests
from PyQt5.QtCore import QCoreApplication, Qt from PyQt5.QtCore import QCoreApplication, Qt
from PyQt5.QtWidgets import ( from PyQt5.QtWidgets import (
QApplication, QDialog, QHBoxLayout, QLabel, QLineEdit, QListWidget, QApplication, QDialog, QHBoxLayout, QLabel, QLineEdit, QListWidget,
@@ -12,6 +12,7 @@ from PyQt5.QtWidgets import (
# --- Local Application Imports --- # --- Local Application Imports ---
from ...i18n.translator import get_translation from ...i18n.translator import get_translation
# Corrected Import: Get the icon from the new assets utility module
from ..assets import get_app_icon_object from ..assets import get_app_icon_object
from ...utils.network_utils import prepare_cookies_for_request from ...utils.network_utils import prepare_cookies_for_request
from .CookieHelpDialog import CookieHelpDialog from .CookieHelpDialog import CookieHelpDialog
@@ -40,9 +41,9 @@ class FavoriteArtistsDialog (QDialog ):
service_lower = service_name.lower() service_lower = service_name.lower()
coomer_primary_services = {'onlyfans', 'fansly', 'manyvids', 'candfans'} coomer_primary_services = {'onlyfans', 'fansly', 'manyvids', 'candfans'}
if service_lower in coomer_primary_services: if service_lower in coomer_primary_services:
return "coomer.st" return "coomer.st" # Use the new domain
else: else:
return "kemono.cr" return "kemono.cr" # Use the new domain
def _tr (self ,key ,default_text =""): def _tr (self ,key ,default_text =""):
"""Helper to get translation based on current app language.""" """Helper to get translation based on current app language."""
@@ -125,11 +126,9 @@ class FavoriteArtistsDialog (QDialog ):
self .artist_list_widget .setVisible (show ) self .artist_list_widget .setVisible (show )
def _fetch_favorite_artists (self ): def _fetch_favorite_artists (self ):
# --- FIX: Use cloudscraper and add proper headers ---
scraper = cloudscraper.create_scraper()
# --- END FIX ---
if self.cookies_config['use_cookie']: if self.cookies_config['use_cookie']:
# --- Kemono Check with Fallback ---
kemono_cookies = prepare_cookies_for_request( kemono_cookies = prepare_cookies_for_request(
True, self.cookies_config['cookie_text'], self.cookies_config['selected_cookie_file'], True, self.cookies_config['cookie_text'], self.cookies_config['selected_cookie_file'],
self.cookies_config['app_base_dir'], self._logger, target_domain="kemono.cr" self.cookies_config['app_base_dir'], self._logger, target_domain="kemono.cr"
@@ -141,6 +140,7 @@ class FavoriteArtistsDialog (QDialog ):
self.cookies_config['app_base_dir'], self._logger, target_domain="kemono.su" self.cookies_config['app_base_dir'], self._logger, target_domain="kemono.su"
) )
# --- Coomer Check with Fallback ---
coomer_cookies = prepare_cookies_for_request( coomer_cookies = prepare_cookies_for_request(
True, self.cookies_config['cookie_text'], self.cookies_config['selected_cookie_file'], True, self.cookies_config['cookie_text'], self.cookies_config['selected_cookie_file'],
self.cookies_config['app_base_dir'], self._logger, target_domain="coomer.st" self.cookies_config['app_base_dir'], self._logger, target_domain="coomer.st"
@@ -153,21 +153,28 @@ class FavoriteArtistsDialog (QDialog ):
) )
if not kemono_cookies and not coomer_cookies: if not kemono_cookies and not coomer_cookies:
# If cookies are enabled but none could be loaded, show help and stop.
self.status_label.setText(self._tr("fav_artists_cookies_required_status", "Error: Cookies enabled but could not be loaded for any source.")) self.status_label.setText(self._tr("fav_artists_cookies_required_status", "Error: Cookies enabled but could not be loaded for any source."))
self._logger("Error: Cookies enabled but no valid cookies were loaded. Showing help dialog.") self._logger("Error: Cookies enabled but no valid cookies were loaded. Showing help dialog.")
cookie_help_dialog = CookieHelpDialog(self.parent_app, self) cookie_help_dialog = CookieHelpDialog(self.parent_app, self)
cookie_help_dialog.exec_() cookie_help_dialog.exec_()
self.download_button.setEnabled(False) self.download_button.setEnabled(False)
return return # Stop further execution
kemono_fav_url ="https://kemono.su/api/v1/account/favorites?type=artist"
coomer_fav_url ="https://coomer.su/api/v1/account/favorites?type=artist"
self .all_fetched_artists =[] self .all_fetched_artists =[]
fetched_any_successfully =False fetched_any_successfully =False
errors_occurred =[] errors_occurred =[]
any_cookies_loaded_successfully_for_any_source =False any_cookies_loaded_successfully_for_any_source =False
kemono_cr_fav_url = "https://kemono.cr/api/v1/account/favorites?type=artist"
coomer_st_fav_url = "https://coomer.st/api/v1/account/favorites?type=artist"
api_sources = [ api_sources = [
{"name": "Kemono.cr", "url": "https://kemono.cr/api/v1/account/favorites?type=artist", "domain": "kemono.cr"}, {"name": "Kemono.cr", "url": kemono_cr_fav_url, "domain": "kemono.cr"},
{"name": "Coomer.st", "url": "https://coomer.st/api/v1/account/favorites?type=artist", "domain": "coomer.st"} {"name": "Coomer.st", "url": coomer_st_fav_url, "domain": "coomer.st"}
] ]
for source in api_sources : for source in api_sources :
@@ -178,36 +185,41 @@ class FavoriteArtistsDialog (QDialog ):
cookies_dict_for_source = None cookies_dict_for_source = None
if self.cookies_config['use_cookie']: if self.cookies_config['use_cookie']:
primary_domain = source['domain'] primary_domain = source['domain']
fallback_domain = "kemono.su" if "kemono" in primary_domain else "coomer.su" fallback_domain = None
if primary_domain == "kemono.cr":
fallback_domain = "kemono.su"
elif primary_domain == "coomer.st":
fallback_domain = "coomer.su"
# First, try the primary domain
cookies_dict_for_source = prepare_cookies_for_request( cookies_dict_for_source = prepare_cookies_for_request(
True, self.cookies_config['cookie_text'], self.cookies_config['selected_cookie_file'], True,
self.cookies_config['app_base_dir'], self._logger, target_domain=primary_domain self.cookies_config['cookie_text'],
self.cookies_config['selected_cookie_file'],
self.cookies_config['app_base_dir'],
self._logger,
target_domain=primary_domain
) )
if not cookies_dict_for_source:
self._logger(f"Warning ({source['name']}): No cookies for '{primary_domain}'. Trying fallback '{fallback_domain}'...") # If no cookies found, try the fallback domain
if not cookies_dict_for_source and fallback_domain:
self._logger(f"Warning ({source['name']}): No cookies found for '{primary_domain}'. Trying fallback '{fallback_domain}'...")
cookies_dict_for_source = prepare_cookies_for_request( cookies_dict_for_source = prepare_cookies_for_request(
True, self.cookies_config['cookie_text'], self.cookies_config['selected_cookie_file'], True,
self.cookies_config['app_base_dir'], self._logger, target_domain=fallback_domain self.cookies_config['cookie_text'],
self.cookies_config['selected_cookie_file'],
self.cookies_config['app_base_dir'],
self._logger,
target_domain=fallback_domain
) )
if cookies_dict_for_source: if cookies_dict_for_source:
any_cookies_loaded_successfully_for_any_source = True any_cookies_loaded_successfully_for_any_source = True
else: else:
self._logger(f"Warning ({source['name']}): Cookies enabled but not loaded for this source. Fetch may fail.") self._logger(f"Warning ({source['name']}): Cookies enabled but could not be loaded for this source (including fallbacks). Fetch might fail.")
try : try :
# --- FIX: Add Referer and Accept headers --- headers ={'User-Agent':'Mozilla/5.0'}
headers = { response =requests .get (source ['url'],headers =headers ,cookies =cookies_dict_for_source ,timeout =20 )
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36',
'Referer': f"https://{source['domain']}/favorites",
'Accept': 'text/css'
}
# --- END FIX ---
# --- FIX: Use scraper instead of requests ---
response = scraper.get(source['url'], headers=headers, cookies=cookies_dict_for_source, timeout=20)
# --- END FIX ---
response .raise_for_status () response .raise_for_status ()
artists_data_from_api =response .json () artists_data_from_api =response .json ()
@@ -242,10 +254,15 @@ class FavoriteArtistsDialog (QDialog ):
fetched_any_successfully =True fetched_any_successfully =True
self ._logger (f"Fetched {processed_artists_from_source } artists from {source ['name']}.") self ._logger (f"Fetched {processed_artists_from_source } artists from {source ['name']}.")
except Exception as e : except requests .exceptions .RequestException as e :
error_msg =f"Error fetching favorites from {source ['name']}: {e }" error_msg =f"Error fetching favorites from {source ['name']}: {e }"
self ._logger (error_msg ) self ._logger (error_msg )
errors_occurred .append (error_msg ) errors_occurred .append (error_msg )
except Exception as e :
error_msg =f"An unexpected error occurred with {source ['name']}: {e }"
self ._logger (error_msg )
errors_occurred .append (error_msg )
if self .cookies_config ['use_cookie']and not any_cookies_loaded_successfully_for_any_source : if self .cookies_config ['use_cookie']and not any_cookies_loaded_successfully_for_any_source :
self .status_label .setText (self ._tr ("fav_artists_cookies_required_status","Error: Cookies enabled but could not be loaded for any source.")) self .status_label .setText (self ._tr ("fav_artists_cookies_required_status","Error: Cookies enabled but could not be loaded for any source."))
@@ -271,7 +288,7 @@ class FavoriteArtistsDialog (QDialog ):
self ._show_content_elements (True ) self ._show_content_elements (True )
self .download_button .setEnabled (True ) self .download_button .setEnabled (True )
elif not fetched_any_successfully and not errors_occurred : elif not fetched_any_successfully and not errors_occurred :
self .status_label .setText (self ._tr ("fav_artists_none_found_status","No favorite artists found on Kemono or Coomer.")) self .status_label .setText (self ._tr ("fav_artists_none_found_status","No favorite artists found on Kemono.su or Coomer.su."))
self ._show_content_elements (False ) self ._show_content_elements (False )
self .download_button .setEnabled (False ) self .download_button .setEnabled (False )
else : else :
@@ -327,4 +344,4 @@ class FavoriteArtistsDialog (QDialog ):
self .accept () self .accept ()
def get_selected_artists (self ): def get_selected_artists (self ):
return self .selected_artists_data return self .selected_artists_data

View File

@@ -7,7 +7,7 @@ import traceback
import json import json
import re import re
from collections import defaultdict from collections import defaultdict
import cloudscraper # MODIFIED: Import cloudscraper import requests
from PyQt5.QtCore import QCoreApplication, Qt, pyqtSignal, QThread from PyQt5.QtCore import QCoreApplication, Qt, pyqtSignal, QThread
from PyQt5.QtWidgets import ( from PyQt5.QtWidgets import (
QApplication, QDialog, QHBoxLayout, QLabel, QLineEdit, QListWidget, QApplication, QDialog, QHBoxLayout, QLabel, QLineEdit, QListWidget,
@@ -42,9 +42,10 @@ class FavoritePostsFetcherThread (QThread ):
self .parent_logger_func (f"[FavPostsFetcherThread] {message }") self .parent_logger_func (f"[FavPostsFetcherThread] {message }")
def run(self): def run(self):
# --- FIX: Use cloudscraper and add proper headers --- kemono_su_fav_posts_url = "https://kemono.su/api/v1/account/favorites?type=post"
scraper = cloudscraper.create_scraper() coomer_su_fav_posts_url = "https://coomer.su/api/v1/account/favorites?type=post"
# --- END FIX --- kemono_cr_fav_posts_url = "https://kemono.cr/api/v1/account/favorites?type=post"
coomer_st_fav_posts_url = "https://coomer.st/api/v1/account/favorites?type=post"
all_fetched_posts_temp = [] all_fetched_posts_temp = []
error_messages_for_summary = [] error_messages_for_summary = []
@@ -55,8 +56,8 @@ class FavoritePostsFetcherThread (QThread ):
self.progress_bar_update.emit(0, 0) self.progress_bar_update.emit(0, 0)
api_sources = [ api_sources = [
{"name": "Kemono.cr", "url": "https://kemono.cr/api/v1/account/favorites?type=post", "domain": "kemono.cr"}, {"name": "Kemono.cr", "url": kemono_cr_fav_posts_url, "domain": "kemono.cr"},
{"name": "Coomer.st", "url": "https://coomer.st/api/v1/account/favorites?type=post", "domain": "coomer.st"} {"name": "Coomer.st", "url": coomer_st_fav_posts_url, "domain": "coomer.st"}
] ]
api_sources_to_try =[] api_sources_to_try =[]
@@ -80,18 +81,32 @@ class FavoritePostsFetcherThread (QThread ):
cookies_dict_for_source = None cookies_dict_for_source = None
if self.cookies_config['use_cookie']: if self.cookies_config['use_cookie']:
primary_domain = source['domain'] primary_domain = source['domain']
fallback_domain = "kemono.su" if "kemono" in primary_domain else "coomer.su" fallback_domain = None
if primary_domain == "kemono.cr":
fallback_domain = "kemono.su"
elif primary_domain == "coomer.st":
fallback_domain = "coomer.su"
# First, try the primary domain
cookies_dict_for_source = prepare_cookies_for_request( cookies_dict_for_source = prepare_cookies_for_request(
True, self.cookies_config['cookie_text'], self.cookies_config['selected_cookie_file'], True,
self.cookies_config['app_base_dir'], self._logger, target_domain=primary_domain self.cookies_config['cookie_text'],
self.cookies_config['selected_cookie_file'],
self.cookies_config['app_base_dir'],
self._logger,
target_domain=primary_domain
) )
# If no cookies found, try the fallback domain
if not cookies_dict_for_source and fallback_domain: if not cookies_dict_for_source and fallback_domain:
self._logger(f"Warning ({source['name']}): No cookies for '{primary_domain}'. Trying fallback '{fallback_domain}'...") self._logger(f"Warning ({source['name']}): No cookies found for '{primary_domain}'. Trying fallback '{fallback_domain}'...")
cookies_dict_for_source = prepare_cookies_for_request( cookies_dict_for_source = prepare_cookies_for_request(
True, self.cookies_config['cookie_text'], self.cookies_config['selected_cookie_file'], True,
self.cookies_config['app_base_dir'], self._logger, target_domain=fallback_domain self.cookies_config['cookie_text'],
self.cookies_config['selected_cookie_file'],
self.cookies_config['app_base_dir'],
self._logger,
target_domain=fallback_domain
) )
if cookies_dict_for_source: if cookies_dict_for_source:
@@ -105,18 +120,8 @@ class FavoritePostsFetcherThread (QThread ):
QCoreApplication .processEvents () QCoreApplication .processEvents ()
try : try :
# --- FIX: Add Referer and Accept headers --- headers ={'User-Agent':'Mozilla/5.0'}
headers = { response =requests .get (source ['url'],headers =headers ,cookies =cookies_dict_for_source ,timeout =20 )
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36',
'Referer': f"https://{source['domain']}/favorites",
'Accept': 'text/css'
}
# --- END FIX ---
# --- FIX: Use scraper instead of requests ---
response = scraper.get(source['url'], headers=headers, cookies=cookies_dict_for_source, timeout=20)
# --- END FIX ---
response .raise_for_status () response .raise_for_status ()
posts_data_from_api =response .json () posts_data_from_api =response .json ()
@@ -148,24 +153,33 @@ class FavoritePostsFetcherThread (QThread ):
fetched_any_successfully =True fetched_any_successfully =True
self ._logger (f"Fetched {processed_posts_from_source } posts from {source ['name']}.") self ._logger (f"Fetched {processed_posts_from_source } posts from {source ['name']}.")
except Exception as e : except requests .exceptions .RequestException as e :
err_detail =f"Error fetching favorite posts from {source ['name']}: {e }" err_detail =f"Error fetching favorite posts from {source ['name']}: {e }"
self ._logger (err_detail ) self ._logger (err_detail )
error_messages_for_summary .append (err_detail ) error_messages_for_summary .append (err_detail )
if hasattr(e, 'response') and e.response is not None and e.response.status_code == 401: if e .response is not None and e .response .status_code ==401 :
self .finished .emit ([],"KEY_AUTH_FAILED") self .finished .emit ([],"KEY_AUTH_FAILED")
self ._logger (f"Authorization failed for {source ['name']}, emitting KEY_AUTH_FAILED.") self ._logger (f"Authorization failed for {source ['name']}, emitting KEY_AUTH_FAILED.")
return return
except Exception as e :
err_detail =f"An unexpected error occurred with {source ['name']}: {e }"
self ._logger (err_detail )
error_messages_for_summary .append (err_detail )
if self .cancellation_event .is_set (): if self .cancellation_event .is_set ():
self .finished .emit ([],"KEY_FETCH_CANCELLED_AFTER") self .finished .emit ([],"KEY_FETCH_CANCELLED_AFTER")
return return
if self .cookies_config ['use_cookie']and not any_cookies_loaded_successfully_for_any_source : if self .cookies_config ['use_cookie']and not any_cookies_loaded_successfully_for_any_source :
if self .target_domain_preference and not any_cookies_loaded_successfully_for_any_source : if self .target_domain_preference and not any_cookies_loaded_successfully_for_any_source :
domain_key_part =self .error_key_map .get (self .target_domain_preference ,self .target_domain_preference .lower ().replace ('.','_')) domain_key_part =self .error_key_map .get (self .target_domain_preference ,self .target_domain_preference .lower ().replace ('.','_'))
self .finished .emit ([],f"KEY_COOKIES_REQUIRED_BUT_NOT_FOUND_FOR_DOMAIN_{domain_key_part }") self .finished .emit ([],f"KEY_COOKIES_REQUIRED_BUT_NOT_FOUND_FOR_DOMAIN_{domain_key_part }")
return return
self .finished .emit ([],"KEY_COOKIES_REQUIRED_BUT_NOT_FOUND_GENERIC") self .finished .emit ([],"KEY_COOKIES_REQUIRED_BUT_NOT_FOUND_GENERIC")
return return
@@ -629,4 +643,4 @@ class FavoritePostsDialog (QDialog ):
self .accept () self .accept ()
def get_selected_posts (self ): def get_selected_posts (self ):
return self .selected_posts_data return self .selected_posts_data

View File

@@ -1,111 +1,25 @@
# --- Standard Library Imports --- # --- Standard Library Imports ---
import os import os
import json import json
import sys
# --- PyQt5 Imports --- # --- PyQt5 Imports ---
from PyQt5.QtCore import Qt, QStandardPaths, QTimer from PyQt5.QtCore import Qt, QStandardPaths
from PyQt5.QtWidgets import ( from PyQt5.QtWidgets import (
QApplication, QDialog, QHBoxLayout, QLabel, QPushButton, QVBoxLayout, QApplication, QDialog, QHBoxLayout, QLabel, QPushButton, QVBoxLayout,
QGroupBox, QComboBox, QMessageBox, QGridLayout, QCheckBox QGroupBox, QComboBox, QMessageBox, QGridLayout, QCheckBox
) )
# --- Local Application Imports --- # --- Local Application Imports ---
from ...i18n.translator import get_translation from ...i18n.translator import get_translation
from ...utils.resolution import get_dark_theme from ...utils.resolution import get_dark_theme
from ..assets import get_app_icon_object
from ..main_window import get_app_icon_object from ..main_window import get_app_icon_object
from ...config.constants import ( from ...config.constants import (
THEME_KEY, LANGUAGE_KEY, DOWNLOAD_LOCATION_KEY, THEME_KEY, LANGUAGE_KEY, DOWNLOAD_LOCATION_KEY,
RESOLUTION_KEY, UI_SCALE_KEY, SAVE_CREATOR_JSON_KEY, RESOLUTION_KEY, UI_SCALE_KEY, SAVE_CREATOR_JSON_KEY,
COOKIE_TEXT_KEY, USE_COOKIE_KEY, COOKIE_TEXT_KEY, USE_COOKIE_KEY,
FETCH_FIRST_KEY, DISCORD_TOKEN_KEY, POST_DOWNLOAD_ACTION_KEY FETCH_FIRST_KEY ### ADDED ###
) )
from ...services.updater import UpdateChecker, UpdateDownloader
class CountdownMessageBox(QDialog):
"""
A custom message box that includes a countdown timer for the 'Yes' button,
which automatically accepts the dialog when the timer reaches zero.
"""
def __init__(self, title, text, countdown_seconds=10, parent_app=None, parent=None):
super().__init__(parent)
self.parent_app = parent_app
self.countdown = countdown_seconds
# --- Basic Window Setup ---
self.setWindowTitle(title)
self.setModal(True)
app_icon = get_app_icon_object()
if app_icon and not app_icon.isNull():
self.setWindowIcon(app_icon)
self._init_ui(text)
self._apply_theme()
# --- Timer Setup ---
self.timer = QTimer(self)
self.timer.setInterval(1000) # Tick every second
self.timer.timeout.connect(self._update_countdown)
self.timer.start()
def _init_ui(self, text):
"""Initializes the UI components of the dialog."""
main_layout = QVBoxLayout(self)
self.message_label = QLabel(text)
self.message_label.setWordWrap(True)
self.message_label.setAlignment(Qt.AlignCenter)
main_layout.addWidget(self.message_label)
buttons_layout = QHBoxLayout()
buttons_layout.addStretch(1)
self.yes_button = QPushButton()
self.yes_button.clicked.connect(self.accept)
self.yes_button.setDefault(True)
self.no_button = QPushButton()
self.no_button.clicked.connect(self.reject)
buttons_layout.addWidget(self.yes_button)
buttons_layout.addWidget(self.no_button)
buttons_layout.addStretch(1)
main_layout.addLayout(buttons_layout)
self._retranslate_ui()
self._update_countdown() # Initial text setup
def _tr(self, key, default_text=""):
"""Helper for translations."""
if self.parent_app and hasattr(self.parent_app, 'current_selected_language'):
return get_translation(self.parent_app.current_selected_language, key, default_text)
return default_text
def _retranslate_ui(self):
"""Sets translated text for UI elements."""
self.no_button.setText(self._tr("no_button_text", "No"))
# The 'yes' button text is handled by the countdown
def _update_countdown(self):
"""Updates the countdown and button text each second."""
if self.countdown <= 0:
self.timer.stop()
self.accept() # Automatically accept when countdown finishes
return
yes_text = self._tr("yes_button_text", "Yes")
self.yes_button.setText(f"{yes_text} ({self.countdown})")
self.countdown -= 1
def _apply_theme(self):
"""Applies the current theme from the parent application."""
if self.parent_app and hasattr(self.parent_app, 'current_theme') and self.parent_app.current_theme == "dark":
scale = getattr(self.parent_app, 'scale_factor', 1)
self.setStyleSheet(get_dark_theme(scale))
else:
self.setStyleSheet("")
class FutureSettingsDialog(QDialog): class FutureSettingsDialog(QDialog):
""" """
@@ -116,7 +30,6 @@ class FutureSettingsDialog(QDialog):
super().__init__(parent) super().__init__(parent)
self.parent_app = parent_app_ref self.parent_app = parent_app_ref
self.setModal(True) self.setModal(True)
self.update_downloader_thread = None # To keep a reference
app_icon = get_app_icon_object() app_icon = get_app_icon_object()
if app_icon and not app_icon.isNull(): if app_icon and not app_icon.isNull():
@@ -124,7 +37,7 @@ class FutureSettingsDialog(QDialog):
screen_height = QApplication.primaryScreen().availableGeometry().height() if QApplication.primaryScreen() else 800 screen_height = QApplication.primaryScreen().availableGeometry().height() if QApplication.primaryScreen() else 800
scale_factor = screen_height / 800.0 scale_factor = screen_height / 800.0
base_min_w, base_min_h = 420, 520 # Increased height for new options base_min_w, base_min_h = 420, 390
scaled_min_w = int(base_min_w * scale_factor) scaled_min_w = int(base_min_w * scale_factor)
scaled_min_h = int(base_min_h * scale_factor) scaled_min_h = int(base_min_h * scale_factor)
self.setMinimumSize(scaled_min_w, scaled_min_h) self.setMinimumSize(scaled_min_w, scaled_min_h)
@@ -140,18 +53,21 @@ class FutureSettingsDialog(QDialog):
self.interface_group_box = QGroupBox() self.interface_group_box = QGroupBox()
interface_layout = QGridLayout(self.interface_group_box) interface_layout = QGridLayout(self.interface_group_box)
# Theme
self.theme_label = QLabel() self.theme_label = QLabel()
self.theme_toggle_button = QPushButton() self.theme_toggle_button = QPushButton()
self.theme_toggle_button.clicked.connect(self._toggle_theme) self.theme_toggle_button.clicked.connect(self._toggle_theme)
interface_layout.addWidget(self.theme_label, 0, 0) interface_layout.addWidget(self.theme_label, 0, 0)
interface_layout.addWidget(self.theme_toggle_button, 0, 1) interface_layout.addWidget(self.theme_toggle_button, 0, 1)
# UI Scale
self.ui_scale_label = QLabel() self.ui_scale_label = QLabel()
self.ui_scale_combo_box = QComboBox() self.ui_scale_combo_box = QComboBox()
self.ui_scale_combo_box.currentIndexChanged.connect(self._display_setting_changed) self.ui_scale_combo_box.currentIndexChanged.connect(self._display_setting_changed)
interface_layout.addWidget(self.ui_scale_label, 1, 0) interface_layout.addWidget(self.ui_scale_label, 1, 0)
interface_layout.addWidget(self.ui_scale_combo_box, 1, 1) interface_layout.addWidget(self.ui_scale_combo_box, 1, 1)
# Language
self.language_label = QLabel() self.language_label = QLabel()
self.language_combo_box = QComboBox() self.language_combo_box = QComboBox()
self.language_combo_box.currentIndexChanged.connect(self._language_selection_changed) self.language_combo_box.currentIndexChanged.connect(self._language_selection_changed)
@@ -162,7 +78,6 @@ class FutureSettingsDialog(QDialog):
self.download_window_group_box = QGroupBox() self.download_window_group_box = QGroupBox()
download_window_layout = QGridLayout(self.download_window_group_box) download_window_layout = QGridLayout(self.download_window_group_box)
self.window_size_label = QLabel() self.window_size_label = QLabel()
self.resolution_combo_box = QComboBox() self.resolution_combo_box = QComboBox()
self.resolution_combo_box.currentIndexChanged.connect(self._display_setting_changed) self.resolution_combo_box.currentIndexChanged.connect(self._display_setting_changed)
@@ -171,112 +86,28 @@ class FutureSettingsDialog(QDialog):
self.default_path_label = QLabel() self.default_path_label = QLabel()
self.save_path_button = QPushButton() self.save_path_button = QPushButton()
self.save_path_button.clicked.connect(self._save_settings) self.save_path_button.clicked.connect(self._save_cookie_and_path)
download_window_layout.addWidget(self.default_path_label, 1, 0) download_window_layout.addWidget(self.default_path_label, 1, 0)
download_window_layout.addWidget(self.save_path_button, 1, 1) download_window_layout.addWidget(self.save_path_button, 1, 1)
self.post_download_action_label = QLabel()
self.post_download_action_combo = QComboBox()
self.post_download_action_combo.currentIndexChanged.connect(self._post_download_action_changed)
download_window_layout.addWidget(self.post_download_action_label, 2, 0)
download_window_layout.addWidget(self.post_download_action_combo, 2, 1)
self.save_creator_json_checkbox = QCheckBox() self.save_creator_json_checkbox = QCheckBox()
self.save_creator_json_checkbox.stateChanged.connect(self._creator_json_setting_changed) self.save_creator_json_checkbox.stateChanged.connect(self._creator_json_setting_changed)
download_window_layout.addWidget(self.save_creator_json_checkbox, 3, 0, 1, 2) download_window_layout.addWidget(self.save_creator_json_checkbox, 2, 0, 1, 2)
self.fetch_first_checkbox = QCheckBox() self.fetch_first_checkbox = QCheckBox()
self.fetch_first_checkbox.stateChanged.connect(self._fetch_first_setting_changed) self.fetch_first_checkbox.stateChanged.connect(self._fetch_first_setting_changed)
download_window_layout.addWidget(self.fetch_first_checkbox, 4, 0, 1, 2) download_window_layout.addWidget(self.fetch_first_checkbox, 3, 0, 1, 2)
main_layout.addWidget(self.download_window_group_box) main_layout.addWidget(self.download_window_group_box)
self.update_group_box = QGroupBox()
update_layout = QGridLayout(self.update_group_box)
self.version_label = QLabel()
self.update_status_label = QLabel()
self.check_update_button = QPushButton()
self.check_update_button.clicked.connect(self._check_for_updates)
update_layout.addWidget(self.version_label, 0, 0)
update_layout.addWidget(self.update_status_label, 0, 1)
update_layout.addWidget(self.check_update_button, 1, 0, 1, 2)
main_layout.addWidget(self.update_group_box)
main_layout.addStretch(1) main_layout.addStretch(1)
self.ok_button = QPushButton() self.ok_button = QPushButton()
self.ok_button.clicked.connect(self.accept) self.ok_button.clicked.connect(self.accept)
main_layout.addWidget(self.ok_button, 0, Qt.AlignRight | Qt.AlignBottom) main_layout.addWidget(self.ok_button, 0, Qt.AlignRight | Qt.AlignBottom)
def _retranslate_ui(self):
self.setWindowTitle(self._tr("settings_dialog_title", "Settings"))
self.interface_group_box.setTitle(self._tr("interface_group_title", "Interface Settings"))
self.download_window_group_box.setTitle(self._tr("download_window_group_title", "Download & Window Settings"))
self.theme_label.setText(self._tr("theme_label", "Theme:"))
self.ui_scale_label.setText(self._tr("ui_scale_label", "UI Scale:"))
self.language_label.setText(self._tr("language_label", "Language:"))
self.window_size_label.setText(self._tr("window_size_label", "Window Size:"))
self.default_path_label.setText(self._tr("default_path_label", "Default Path:"))
self.post_download_action_label.setText(self._tr("post_download_action_label", "Action After Download:"))
self.save_creator_json_checkbox.setText(self._tr("save_creator_json_label", "Save Creator.json file"))
self.fetch_first_checkbox.setText(self._tr("fetch_first_label", "Fetch First (Download after all pages are found)"))
self.fetch_first_checkbox.setToolTip(self._tr("fetch_first_tooltip", "If checked, the downloader will find all posts from a creator first before starting any downloads.\nThis can be slower to start but provides a more accurate progress bar."))
self._update_theme_toggle_button_text()
self.save_path_button.setText(self._tr("settings_save_all_button", "Save Path + Cookie + Token"))
self.save_path_button.setToolTip(self._tr("settings_save_all_tooltip", "Save the current 'Download Location', Cookie, and Discord Token settings for future sessions."))
self.ok_button.setText(self._tr("ok_button", "OK"))
self.update_group_box.setTitle(self._tr("update_group_title", "Application Updates"))
current_version = self.parent_app.windowTitle().split(' v')[-1]
self.version_label.setText(self._tr("current_version_label", f"Current Version: v{current_version}"))
self.update_status_label.setText(self._tr("update_status_ready", "Ready to check."))
self.check_update_button.setText(self._tr("check_for_updates_button", "Check for Updates"))
self._populate_display_combo_boxes()
self._populate_language_combo_box()
self._populate_post_download_action_combo()
self._load_checkbox_states()
def _check_for_updates(self):
self.check_update_button.setEnabled(False)
self.update_status_label.setText(self._tr("update_status_checking", "Checking..."))
current_version = self.parent_app.windowTitle().split(' v')[-1]
self.update_checker_thread = UpdateChecker(current_version)
self.update_checker_thread.update_available.connect(self._on_update_available)
self.update_checker_thread.up_to_date.connect(self._on_up_to_date)
self.update_checker_thread.update_error.connect(self._on_update_error)
self.update_checker_thread.start()
def _on_update_available(self, new_version, download_url):
self.update_status_label.setText(self._tr("update_status_found", f"Update found: v{new_version}"))
self.check_update_button.setEnabled(True)
reply = QMessageBox.question(self, self._tr("update_available_title", "Update Available"),
self._tr("update_available_message", f"A new version (v{new_version}) is available.\nWould you like to download and install it now?"),
QMessageBox.Yes | QMessageBox.No, QMessageBox.Yes)
if reply == QMessageBox.Yes:
self.ok_button.setEnabled(False)
self.check_update_button.setEnabled(False)
self.update_status_label.setText(self._tr("update_status_downloading", "Downloading update..."))
self.update_downloader_thread = UpdateDownloader(download_url, self.parent_app)
self.update_downloader_thread.download_finished.connect(self._on_download_finished)
self.update_downloader_thread.download_error.connect(self._on_update_error)
self.update_downloader_thread.start()
def _on_download_finished(self):
QApplication.instance().quit()
def _on_up_to_date(self, message):
self.update_status_label.setText(self._tr("update_status_latest", message))
self.check_update_button.setEnabled(True)
def _on_update_error(self, message):
self.update_status_label.setText(self._tr("update_status_error", f"Error: {message}"))
self.check_update_button.setEnabled(True)
self.ok_button.setEnabled(True)
def _load_checkbox_states(self): def _load_checkbox_states(self):
"""Loads the initial state for all checkboxes from settings."""
self.save_creator_json_checkbox.blockSignals(True) self.save_creator_json_checkbox.blockSignals(True)
should_save = self.parent_app.settings.value(SAVE_CREATOR_JSON_KEY, True, type=bool) should_save = self.parent_app.settings.value(SAVE_CREATOR_JSON_KEY, True, type=bool)
self.save_creator_json_checkbox.setChecked(should_save) self.save_creator_json_checkbox.setChecked(should_save)
@@ -288,11 +119,13 @@ class FutureSettingsDialog(QDialog):
self.fetch_first_checkbox.blockSignals(False) self.fetch_first_checkbox.blockSignals(False)
def _creator_json_setting_changed(self, state): def _creator_json_setting_changed(self, state):
"""Saves the state of the 'Save Creator.json' checkbox."""
is_checked = state == Qt.Checked is_checked = state == Qt.Checked
self.parent_app.settings.setValue(SAVE_CREATOR_JSON_KEY, is_checked) self.parent_app.settings.setValue(SAVE_CREATOR_JSON_KEY, is_checked)
self.parent_app.settings.sync() self.parent_app.settings.sync()
def _fetch_first_setting_changed(self, state): def _fetch_first_setting_changed(self, state):
"""Saves the state of the 'Fetch First' checkbox."""
is_checked = state == Qt.Checked is_checked = state == Qt.Checked
self.parent_app.settings.setValue(FETCH_FIRST_KEY, is_checked) self.parent_app.settings.setValue(FETCH_FIRST_KEY, is_checked)
self.parent_app.settings.sync() self.parent_app.settings.sync()
@@ -302,6 +135,34 @@ class FutureSettingsDialog(QDialog):
return get_translation(self.parent_app.current_selected_language, key, default_text) return get_translation(self.parent_app.current_selected_language, key, default_text)
return default_text return default_text
def _retranslate_ui(self):
self.setWindowTitle(self._tr("settings_dialog_title", "Settings"))
self.interface_group_box.setTitle(self._tr("interface_group_title", "Interface Settings"))
self.download_window_group_box.setTitle(self._tr("download_window_group_title", "Download & Window Settings"))
self.theme_label.setText(self._tr("theme_label", "Theme:"))
self.ui_scale_label.setText(self._tr("ui_scale_label", "UI Scale:"))
self.language_label.setText(self._tr("language_label", "Language:"))
self.window_size_label.setText(self._tr("window_size_label", "Window Size:"))
self.default_path_label.setText(self._tr("default_path_label", "Default Path:"))
self.save_creator_json_checkbox.setText(self._tr("save_creator_json_label", "Save Creator.json file"))
self.fetch_first_checkbox.setText(self._tr("fetch_first_label", "Fetch First (Download after all pages are found)"))
self.fetch_first_checkbox.setToolTip(self._tr("fetch_first_tooltip", "If checked, the downloader will find all posts from a creator first before starting any downloads.\nThis can be slower to start but provides a more accurate progress bar."))
self._update_theme_toggle_button_text()
self.save_path_button.setText(self._tr("settings_save_cookie_path_button", "Save Cookie + Download Path"))
self.save_path_button.setToolTip(self._tr("settings_save_cookie_path_tooltip", "Save the current 'Download Location' and Cookie settings for future sessions."))
self.ok_button.setText(self._tr("ok_button", "OK"))
self._populate_display_combo_boxes()
self._populate_language_combo_box()
self._load_checkbox_states()
# --- (The rest of the file remains unchanged) ---
def _apply_theme(self): def _apply_theme(self):
if self.parent_app and self.parent_app.current_theme == "dark": if self.parent_app and self.parent_app.current_theme == "dark":
scale = getattr(self.parent_app, 'scale_factor', 1) scale = getattr(self.parent_app, 'scale_factor', 1)
@@ -327,7 +188,14 @@ class FutureSettingsDialog(QDialog):
def _populate_display_combo_boxes(self): def _populate_display_combo_boxes(self):
self.resolution_combo_box.blockSignals(True) self.resolution_combo_box.blockSignals(True)
self.resolution_combo_box.clear() self.resolution_combo_box.clear()
resolutions = [("Auto", "Auto"), ("1280x720", "1280x720"), ("1600x900", "1600x900"), ("1920x1080", "1920x1080")] resolutions = [
("Auto", self._tr("auto_resolution", "Auto (System Default)")),
("1280x720", "1280 x 720"),
("1600x900", "1600 x 900"),
("1920x1080", "1920 x 1080 (Full HD)"),
("2560x1440", "2560 x 1440 (2K)"),
("3840x2160", "3840 x 2160 (4K)")
]
current_res = self.parent_app.settings.value(RESOLUTION_KEY, "Auto") current_res = self.parent_app.settings.value(RESOLUTION_KEY, "Auto")
for res_key, res_name in resolutions: for res_key, res_name in resolutions:
self.resolution_combo_box.addItem(res_name, res_key) self.resolution_combo_box.addItem(res_name, res_key)
@@ -338,24 +206,43 @@ class FutureSettingsDialog(QDialog):
self.ui_scale_combo_box.blockSignals(True) self.ui_scale_combo_box.blockSignals(True)
self.ui_scale_combo_box.clear() self.ui_scale_combo_box.clear()
scales = [ scales = [
(0.5, "50%"), (0.7, "70%"), (0.9, "90%"), (1.0, "100% (Default)"), (0.5, "50%"),
(1.25, "125%"), (1.50, "150%"), (1.75, "175%"), (2.0, "200%") (0.7, "70%"),
(0.9, "90%"),
(1.0, "100% (Default)"),
(1.25, "125%"),
(1.50, "150%"),
(1.75, "175%"),
(2.0, "200%")
] ]
current_scale = self.parent_app.settings.value(UI_SCALE_KEY, 1.0)
current_scale = float(self.parent_app.settings.value(UI_SCALE_KEY, 1.0))
for scale_val, scale_name in scales: for scale_val, scale_name in scales:
self.ui_scale_combo_box.addItem(scale_name, scale_val) self.ui_scale_combo_box.addItem(scale_name, scale_val)
if abs(float(current_scale) - scale_val) < 0.01: if abs(current_scale - scale_val) < 0.01:
self.ui_scale_combo_box.setCurrentIndex(self.ui_scale_combo_box.count() - 1) self.ui_scale_combo_box.setCurrentIndex(self.ui_scale_combo_box.count() - 1)
self.ui_scale_combo_box.blockSignals(False) self.ui_scale_combo_box.blockSignals(False)
def _display_setting_changed(self): def _display_setting_changed(self):
selected_res = self.resolution_combo_box.currentData() selected_res = self.resolution_combo_box.currentData()
selected_scale = self.ui_scale_combo_box.currentData() selected_scale = self.ui_scale_combo_box.currentData()
self.parent_app.settings.setValue(RESOLUTION_KEY, selected_res) self.parent_app.settings.setValue(RESOLUTION_KEY, selected_res)
self.parent_app.settings.setValue(UI_SCALE_KEY, selected_scale) self.parent_app.settings.setValue(UI_SCALE_KEY, selected_scale)
self.parent_app.settings.sync() self.parent_app.settings.sync()
QMessageBox.information(self, self._tr("display_change_title", "Display Settings Changed"),
self._tr("language_change_message", "A restart is required...")) msg_box = QMessageBox(self)
msg_box.setIcon(QMessageBox.Information)
msg_box.setWindowTitle(self._tr("display_change_title", "Display Settings Changed"))
msg_box.setText(self._tr("language_change_message", "A restart is required for these changes to take effect."))
msg_box.setInformativeText(self._tr("language_change_informative", "Would you like to restart now?"))
restart_button = msg_box.addButton(self._tr("restart_now_button", "Restart Now"), QMessageBox.ApplyRole)
ok_button = msg_box.addButton(self._tr("ok_button", "OK"), QMessageBox.AcceptRole)
msg_box.setDefaultButton(ok_button)
msg_box.exec_()
if msg_box.clickedButton() == restart_button:
self.parent_app._request_restart_application()
def _populate_language_combo_box(self): def _populate_language_combo_box(self):
self.language_combo_box.blockSignals(True) self.language_combo_box.blockSignals(True)
@@ -379,44 +266,29 @@ class FutureSettingsDialog(QDialog):
self.parent_app.settings.setValue(LANGUAGE_KEY, selected_lang_code) self.parent_app.settings.setValue(LANGUAGE_KEY, selected_lang_code)
self.parent_app.settings.sync() self.parent_app.settings.sync()
self.parent_app.current_selected_language = selected_lang_code self.parent_app.current_selected_language = selected_lang_code
self._retranslate_ui() self._retranslate_ui()
if hasattr(self.parent_app, '_retranslate_main_ui'): if hasattr(self.parent_app, '_retranslate_main_ui'):
self.parent_app._retranslate_main_ui() self.parent_app._retranslate_main_ui()
QMessageBox.information(self, self._tr("language_change_title", "Language Changed"),
self._tr("language_change_message", "A restart is required...")) msg_box = QMessageBox(self)
msg_box.setIcon(QMessageBox.Information)
msg_box.setWindowTitle(self._tr("language_change_title", "Language Changed"))
msg_box.setText(self._tr("language_change_message", "A restart is required..."))
msg_box.setInformativeText(self._tr("language_change_informative", "Would you like to restart now?"))
restart_button = msg_box.addButton(self._tr("restart_now_button", "Restart Now"), QMessageBox.ApplyRole)
ok_button = msg_box.addButton(self._tr("ok_button", "OK"), QMessageBox.AcceptRole)
msg_box.setDefaultButton(ok_button)
msg_box.exec_()
def _populate_post_download_action_combo(self): if msg_box.clickedButton() == restart_button:
"""Populates the action dropdown and sets the current selection from settings.""" self.parent_app._request_restart_application()
self.post_download_action_combo.blockSignals(True)
self.post_download_action_combo.clear()
actions = [
(self._tr("action_off", "Off"), "off"),
(self._tr("action_notify", "Notify with Sound"), "notify"),
(self._tr("action_sleep", "Sleep"), "sleep"),
(self._tr("action_shutdown", "Shutdown"), "shutdown")
]
current_action = self.parent_app.settings.value(POST_DOWNLOAD_ACTION_KEY, "off")
for text, key in actions:
self.post_download_action_combo.addItem(text, key)
if current_action == key:
self.post_download_action_combo.setCurrentIndex(self.post_download_action_combo.count() - 1)
self.post_download_action_combo.blockSignals(False)
def _post_download_action_changed(self): def _save_cookie_and_path(self):
"""Saves the selected post-download action to settings.""" """Saves the current download path and/or cookie settings from the main window."""
selected_action = self.post_download_action_combo.currentData()
self.parent_app.settings.setValue(POST_DOWNLOAD_ACTION_KEY, selected_action)
self.parent_app.settings.sync()
def _save_settings(self):
path_saved = False path_saved = False
cookie_saved = False cookie_saved = False
token_saved = False
if hasattr(self.parent_app, 'dir_input') and self.parent_app.dir_input: if hasattr(self.parent_app, 'dir_input') and self.parent_app.dir_input:
current_path = self.parent_app.dir_input.text().strip() current_path = self.parent_app.dir_input.text().strip()
if current_path and os.path.isdir(current_path): if current_path and os.path.isdir(current_path):
@@ -426,27 +298,27 @@ class FutureSettingsDialog(QDialog):
if hasattr(self.parent_app, 'use_cookie_checkbox'): if hasattr(self.parent_app, 'use_cookie_checkbox'):
use_cookie = self.parent_app.use_cookie_checkbox.isChecked() use_cookie = self.parent_app.use_cookie_checkbox.isChecked()
cookie_content = self.parent_app.cookie_text_input.text().strip() cookie_content = self.parent_app.cookie_text_input.text().strip()
if use_cookie and cookie_content: if use_cookie and cookie_content:
self.parent_app.settings.setValue(USE_COOKIE_KEY, True) self.parent_app.settings.setValue(USE_COOKIE_KEY, True)
self.parent_app.settings.setValue(COOKIE_TEXT_KEY, cookie_content) self.parent_app.settings.setValue(COOKIE_TEXT_KEY, cookie_content)
cookie_saved = True cookie_saved = True
else: else:
self.parent_app.settings.setValue(USE_COOKIE_KEY, False) self.parent_app.settings.setValue(USE_COOKIE_KEY, False)
self.parent_app.settings.setValue(COOKIE_TEXT_KEY, "") self.parent_app.settings.setValue(COOKIE_TEXT_KEY, "")
if (hasattr(self.parent_app, 'remove_from_filename_input') and
hasattr(self.parent_app, 'remove_from_filename_label_widget')):
label_text = self.parent_app.remove_from_filename_label_widget.text()
if "Token" in label_text:
discord_token = self.parent_app.remove_from_filename_input.text().strip()
if discord_token:
self.parent_app.settings.setValue(DISCORD_TOKEN_KEY, discord_token)
token_saved = True
self.parent_app.settings.sync() self.parent_app.settings.sync()
if path_saved or cookie_saved or token_saved: # --- User Feedback ---
QMessageBox.information(self, "Settings Saved", "Settings have been saved successfully.") if path_saved and cookie_saved:
message = self._tr("settings_save_both_success", "Download location and cookie settings saved.")
elif path_saved:
message = self._tr("settings_save_path_only_success", "Download location saved. No cookie settings were active to save.")
elif cookie_saved:
message = self._tr("settings_save_cookie_only_success", "Cookie settings saved. Download location was not set.")
else: else:
QMessageBox.warning(self, "Nothing to Save", "No valid settings were found to save.") QMessageBox.warning(self, self._tr("settings_save_nothing_title", "Nothing to Save"),
self._tr("settings_save_nothing_message", "The download location is not a valid directory and no cookie was active."))
return
QMessageBox.information(self, self._tr("settings_save_success_title", "Settings Saved"), message)

View File

@@ -1,7 +1,6 @@
import os import os
import re import re
import datetime import datetime
import time
try: try:
from fpdf import FPDF from fpdf import FPDF
FPDF_AVAILABLE = True FPDF_AVAILABLE = True
@@ -30,7 +29,7 @@ except ImportError:
FPDF = None FPDF = None
PDF = None PDF = None
def create_pdf_from_discord_messages(messages_data, server_name, channel_name, output_filename, font_path, logger=print, cancellation_event=None, pause_event=None): def create_pdf_from_discord_messages(messages_data, server_name, channel_name, output_filename, font_path, logger=print):
""" """
Creates a single PDF from a list of Discord message objects, formatted as a chat log. Creates a single PDF from a list of Discord message objects, formatted as a chat log.
UPDATED to include clickable links for attachments and embeds. UPDATED to include clickable links for attachments and embeds.
@@ -43,20 +42,8 @@ def create_pdf_from_discord_messages(messages_data, server_name, channel_name, o
logger(" No messages were found or fetched to create a PDF.") logger(" No messages were found or fetched to create a PDF.")
return False return False
# --- FIX: This helper function now correctly accepts and checks the event objects ---
def check_events(c_event, p_event):
"""Helper to safely check for pause and cancel events."""
if c_event and hasattr(c_event, 'is_cancelled') and c_event.is_cancelled:
return True # Stop
if p_event and hasattr(p_event, 'is_paused'):
while p_event.is_paused:
time.sleep(0.5)
if c_event and hasattr(c_event, 'is_cancelled') and c_event.is_cancelled:
return True
return False
logger(" Sorting messages by date (oldest first)...") logger(" Sorting messages by date (oldest first)...")
messages_data.sort(key=lambda m: m.get('published', m.get('timestamp', ''))) messages_data.sort(key=lambda m: m.get('published', ''))
pdf = PDF(server_name, channel_name) pdf = PDF(server_name, channel_name)
default_font_family = 'DejaVu' default_font_family = 'DejaVu'
@@ -91,19 +78,14 @@ def create_pdf_from_discord_messages(messages_data, server_name, channel_name, o
logger(f" Starting PDF creation with {len(messages_data)} messages...") logger(f" Starting PDF creation with {len(messages_data)} messages...")
for i, message in enumerate(messages_data): for i, message in enumerate(messages_data):
# --- FIX: Pass the event objects to the helper function ---
if i % 50 == 0:
if check_events(cancellation_event, pause_event):
logger(" PDF generation cancelled by user.")
return False
author = message.get('author', {}).get('global_name') or message.get('author', {}).get('username', 'Unknown User') author = message.get('author', {}).get('global_name') or message.get('author', {}).get('username', 'Unknown User')
timestamp_str = message.get('published', message.get('timestamp', '')) timestamp_str = message.get('published', '')
content = message.get('content', '') content = message.get('content', '')
attachments = message.get('attachments', []) attachments = message.get('attachments', [])
embeds = message.get('embeds', []) embeds = message.get('embeds', [])
try: try:
# Handle timezone information correctly
if timestamp_str.endswith('Z'): if timestamp_str.endswith('Z'):
timestamp_str = timestamp_str[:-1] + '+00:00' timestamp_str = timestamp_str[:-1] + '+00:00'
dt_obj = datetime.datetime.fromisoformat(timestamp_str) dt_obj = datetime.datetime.fromisoformat(timestamp_str)
@@ -111,12 +93,14 @@ def create_pdf_from_discord_messages(messages_data, server_name, channel_name, o
except (ValueError, TypeError): except (ValueError, TypeError):
formatted_timestamp = timestamp_str formatted_timestamp = timestamp_str
# Draw a separator line
if i > 0: if i > 0:
pdf.ln(2) pdf.ln(2)
pdf.set_draw_color(200, 200, 200) pdf.set_draw_color(200, 200, 200) # Light grey line
pdf.cell(0, 0, '', border='T') pdf.cell(0, 0, '', border='T')
pdf.ln(2) pdf.ln(2)
# Message Header
pdf.set_font(default_font_family, 'B', 11) pdf.set_font(default_font_family, 'B', 11)
pdf.write(5, f"{author} ") pdf.write(5, f"{author} ")
pdf.set_font(default_font_family, '', 9) pdf.set_font(default_font_family, '', 9)
@@ -125,31 +109,33 @@ def create_pdf_from_discord_messages(messages_data, server_name, channel_name, o
pdf.set_text_color(0, 0, 0) pdf.set_text_color(0, 0, 0)
pdf.ln(6) pdf.ln(6)
# Message Content
if content: if content:
pdf.set_font(default_font_family, '', 10) pdf.set_font(default_font_family, '', 10)
pdf.multi_cell(w=0, h=5, text=content) pdf.multi_cell(w=0, h=5, text=content)
# --- START: MODIFIED ATTACHMENT AND EMBED LOGIC ---
if attachments or embeds: if attachments or embeds:
pdf.ln(1) pdf.ln(1)
pdf.set_font(default_font_family, '', 9) pdf.set_font(default_font_family, '', 9)
pdf.set_text_color(22, 119, 219) pdf.set_text_color(22, 119, 219) # A nice blue for links
for att in attachments: for att in attachments:
file_name = att.get('filename', 'untitled') file_name = att.get('name', 'untitled')
full_url = att.get('url', '#') file_path = att.get('path', '')
# Construct the full, clickable URL for the attachment
full_url = f"https://kemono.cr/data{file_path}"
pdf.write(5, text=f"[Attachment: {file_name}]", link=full_url) pdf.write(5, text=f"[Attachment: {file_name}]", link=full_url)
pdf.ln() pdf.ln() # New line after each attachment
for embed in embeds: for embed in embeds:
embed_url = embed.get('url', 'no url') embed_url = embed.get('url', 'no url')
# The embed URL is already a full URL
pdf.write(5, text=f"[Embed: {embed_url}]", link=embed_url) pdf.write(5, text=f"[Embed: {embed_url}]", link=embed_url)
pdf.ln() pdf.ln() # New line after each embed
pdf.set_text_color(0, 0, 0) pdf.set_text_color(0, 0, 0) # Reset color to black
# --- END: MODIFIED ATTACHMENT AND EMBED LOGIC ---
if check_events(cancellation_event, pause_event):
logger(" PDF generation cancelled by user before final save.")
return False
try: try:
pdf.output(output_filename) pdf.output(output_filename)

File diff suppressed because it is too large Load Diff

View File

@@ -141,51 +141,39 @@ def prepare_cookies_for_request(use_cookie_flag, cookie_text_input, selected_coo
def extract_post_info(url_string): def extract_post_info(url_string):
""" """
Parses a URL string to extract the service, user ID, and post ID. Parses a URL string to extract the service, user ID, and post ID.
UPDATED to support Hentai2Read series and chapters. UPDATED to support Discord server/channel URLs.
Args:
url_string (str): The URL to parse.
Returns:
tuple: A tuple containing (service, id1, id2).
For posts: (service, user_id, post_id).
For Discord: ('discord', server_id, channel_id).
""" """
if not isinstance(url_string, str) or not url_string.strip(): if not isinstance(url_string, str) or not url_string.strip():
return None, None, None return None, None, None
stripped_url = url_string.strip()
# --- Bunkr Check ---
bunkr_pattern = re.compile(
r"(?:https?://)?(?:[a-zA-Z0-9-]+\.)?bunkr\.(?:si|la|ws|red|black|media|site|is|to|ac|cr|ci|fi|pk|ps|sk|ph|su|ru)|bunkrr\.ru"
)
if bunkr_pattern.search(stripped_url):
return 'bunkr', stripped_url, None
# --- nhentai Check ---
nhentai_match = re.search(r'nhentai\.net/g/(\d+)', stripped_url)
if nhentai_match:
return 'nhentai', nhentai_match.group(1), None
# --- Hentai2Read Check (Updated) ---
# This regex now captures the manga slug (id1) and optionally the chapter number (id2)
hentai2read_match = re.search(r'hentai2read\.com/([^/]+)(?:/(\d+))?/?', stripped_url)
if hentai2read_match:
manga_slug, chapter_num = hentai2read_match.groups()
return 'hentai2read', manga_slug, chapter_num # chapter_num will be None for series URLs
discord_channel_match = re.search(r'discord\.com/channels/(@me|\d+)/(\d+)', stripped_url)
if discord_channel_match:
server_id, channel_id = discord_channel_match.groups()
return 'discord', server_id, channel_id
# --- Kemono/Coomer/Discord Parsing ---
try: try:
parsed_url = urlparse(stripped_url) parsed_url = urlparse(url_string.strip())
path_parts = [part for part in parsed_url.path.strip('/').split('/') if part] path_parts = [part for part in parsed_url.path.strip('/').split('/') if part]
# Check for new Discord URL format first
# e.g., /discord/server/891670433978531850/1252332668805189723
if len(path_parts) >= 3 and path_parts[0].lower() == 'discord' and path_parts[1].lower() == 'server': if len(path_parts) >= 3 and path_parts[0].lower() == 'discord' and path_parts[1].lower() == 'server':
return 'discord', path_parts[2], path_parts[3] if len(path_parts) >= 4 else None service = 'discord'
server_id = path_parts[2]
channel_id = path_parts[3] if len(path_parts) >= 4 else None
return service, server_id, channel_id
# Standard creator/post format: /<service>/user/<user_id>/post/<post_id>
if len(path_parts) >= 3 and path_parts[1].lower() == 'user': if len(path_parts) >= 3 and path_parts[1].lower() == 'user':
service = path_parts[0] service = path_parts[0]
user_id = path_parts[2] user_id = path_parts[2]
post_id = path_parts[4] if len(path_parts) >= 5 and path_parts[3].lower() == 'post' else None post_id = path_parts[4] if len(path_parts) >= 5 and path_parts[3].lower() == 'post' else None
return service, user_id, post_id return service, user_id, post_id
# API format: /api/v1/<service>/user/<user_id>...
if len(path_parts) >= 5 and path_parts[0:2] == ['api', 'v1'] and path_parts[3].lower() == 'user': if len(path_parts) >= 5 and path_parts[0:2] == ['api', 'v1'] and path_parts[3].lower() == 'user':
service = path_parts[2] service = path_parts[2]
user_id = path_parts[4] user_id = path_parts[4]
@@ -196,7 +184,7 @@ def extract_post_info(url_string):
print(f"Debug: Exception during URL parsing for '{url_string}': {e}") print(f"Debug: Exception during URL parsing for '{url_string}': {e}")
return None, None, None return None, None, None
def get_link_platform(url): def get_link_platform(url):
""" """
Identifies the platform of a given URL based on its domain. Identifies the platform of a given URL based on its domain.

View File

@@ -284,7 +284,7 @@ def setup_ui(main_app):
advanced_row2_layout.addLayout(multithreading_layout) advanced_row2_layout.addLayout(multithreading_layout)
main_app.external_links_checkbox = QCheckBox("Show External Links in Log") main_app.external_links_checkbox = QCheckBox("Show External Links in Log")
advanced_row2_layout.addWidget(main_app.external_links_checkbox) advanced_row2_layout.addWidget(main_app.external_links_checkbox)
main_app.manga_mode_checkbox = QCheckBox("Renaming Mode") main_app.manga_mode_checkbox = QCheckBox("Manga/Comic Mode")
advanced_row2_layout.addWidget(main_app.manga_mode_checkbox) advanced_row2_layout.addWidget(main_app.manga_mode_checkbox)
advanced_row2_layout.addStretch(1) advanced_row2_layout.addStretch(1)
checkboxes_group_layout.addLayout(advanced_row2_layout) checkboxes_group_layout.addLayout(advanced_row2_layout)
@@ -391,23 +391,10 @@ def setup_ui(main_app):
main_app.link_search_button.setVisible(False) main_app.link_search_button.setVisible(False)
main_app.link_search_button.setFixedWidth(int(30 * scale)) main_app.link_search_button.setFixedWidth(int(30 * scale))
log_title_layout.addWidget(main_app.link_search_button) log_title_layout.addWidget(main_app.link_search_button)
discord_controls_layout = QHBoxLayout()
main_app.discord_scope_toggle_button = QPushButton("Scope: Files") main_app.discord_scope_toggle_button = QPushButton("Scope: Files")
main_app.discord_scope_toggle_button.setVisible(False) # Hidden by default main_app.discord_scope_toggle_button.setVisible(False) # Hidden by default
discord_controls_layout.addWidget(main_app.discord_scope_toggle_button) main_app.discord_scope_toggle_button.setFixedWidth(int(140 * scale))
log_title_layout.addWidget(main_app.discord_scope_toggle_button)
main_app.discord_message_limit_input = QLineEdit(main_app)
main_app.discord_message_limit_input.setPlaceholderText("Msg Limit")
main_app.discord_message_limit_input.setToolTip("Optional: Limit the number of recent messages to process.")
main_app.discord_message_limit_input.setValidator(QIntValidator(1, 9999999, main_app))
main_app.discord_message_limit_input.setFixedWidth(int(80 * scale))
main_app.discord_message_limit_input.setVisible(False) # Hide it by default
discord_controls_layout.addWidget(main_app.discord_message_limit_input)
log_title_layout.addLayout(discord_controls_layout)
main_app.manga_rename_toggle_button = QPushButton() main_app.manga_rename_toggle_button = QPushButton()
main_app.manga_rename_toggle_button.setVisible(False) main_app.manga_rename_toggle_button.setVisible(False)
main_app.manga_rename_toggle_button.setFixedWidth(int(140 * scale)) main_app.manga_rename_toggle_button.setFixedWidth(int(140 * scale))