48 Commits

Author SHA1 Message Date
Yuvi9587
a9b210b2ba Commit 2025-09-07 08:16:43 -07:00
Yuvi9587
ec94417569 Update main.py 2025-09-07 05:24:54 -07:00
Yuvi9587
0a902895a8 Update main_window.py 2025-09-07 05:23:44 -07:00
Yuvi9587
7217bfdb39 Commit 2025-09-07 04:56:08 -07:00
Yuvi9587
24880b5042 Update readme.md 2025-08-28 04:34:37 -07:00
Yuvi9587
510ae5e1d1 Update readme.md 2025-08-27 19:59:11 -07:00
Yuvi9587
65b4759bad Commit 2025-08-27 19:51:42 -07:00
Yuvi9587
6e993d88de Commit 2025-08-27 19:50:13 -07:00
Yuvi9587
cc3565b12b Commit 2025-08-27 07:21:30 -07:00
Yuvi9587
f8b150dfdb commit 2025-08-17 08:43:27 -07:00
Yuvi9587
5f7b526852 Commit 2025-08-17 05:51:25 -07:00
Yuvi9587
b0a6c264e1 Commit 2025-08-15 20:22:40 -07:00
Yuvi9587
d9364f4f91 commit 2025-08-14 09:48:55 -07:00
Yuvi9587
9cd48bb63a Update main_window.py 2025-08-13 19:49:10 -07:00
Yuvi9587
d0f11c4a06 Commit 2025-08-13 19:38:33 -07:00
Yuvi9587
26fa3b9bc1 Commit 2025-08-10 09:16:31 -07:00
Yuvi9587
f7c4d892a8 commit 2025-08-07 21:42:04 -07:00
Yuvi9587
661b97aa16 Commit 2025-08-06 06:56:49 -07:00
Yuvi9587
3704fece2b Update main_window.py 2025-08-04 04:53:52 -07:00
Yuvi9587
bdb7ac93c4 Update readme.md 2025-08-03 09:16:25 -07:00
Yuvi9587
76d4a3ea8a Update main_window.py 2025-08-03 09:15:01 -07:00
Yuvi9587
ccc7804505 Update readme.md 2025-08-03 09:13:47 -07:00
Yuvi9587
4ee750c5d4 Update drive_downloader.py 2025-08-03 09:11:27 -07:00
Yuvi9587
e9be13c4e3 Update readme.md 2025-08-03 09:07:29 -07:00
Yuvi9587
a5cb04ea6f Update features.md 2025-08-03 06:46:30 -07:00
Yuvi9587
842f18d70d Update features.md 2025-08-03 06:32:32 -07:00
Yuvi9587
fb3f0e8913 Update features.md 2025-08-03 06:11:05 -07:00
Yuvi9587
0758887154 Update features.md 2025-08-03 06:07:05 -07:00
Yuvi9587
e752d881e7 Update features.md 2025-08-03 06:01:32 -07:00
Yuvi9587
a776d1abe9 Update features.md 2025-08-03 06:01:15 -07:00
Yuvi9587
21d1ce4fa9 Commit 2025-08-03 05:46:51 -07:00
Yuvi9587
d5112a25ee Commit 2025-08-01 09:42:10 -07:00
Yuvi9587
791ce503ff Update main_window.py 2025-08-01 07:57:32 -07:00
Yuvi9587
e5b519d5ce Commit 2025-08-01 06:33:36 -07:00
Yuvi9587
9888ed0862 Update multipart_downloader.py 2025-07-30 22:28:05 -07:00
Yuvi9587
9e996bf682 Commit 2025-07-30 21:31:02 -07:00
Yuvi9587
e7a6a91542 commit 2025-07-30 19:30:50 -07:00
Yuvi9587
d7faccce18 Commit 2025-07-29 06:37:28 -07:00
Yuvi9587
a78c01c4f6 Update workers.py 2025-07-27 07:44:14 -07:00
Yuvi9587
6de9967e0b Commit 2025-07-27 07:18:08 -07:00
Yuvi9587
e3dd0e70b6 commit 2025-07-27 06:32:15 -07:00
Yuvi9587
9db89cfad0 Commit 2025-07-25 11:00:33 -07:00
Yuvi9587
0a6034a632 Update features.md 2025-07-25 10:49:51 -07:00
Yuvi9587
2da69e7017 Update features.md 2025-07-25 10:45:50 -07:00
Yuvi9587
3209770d00 Update LICENSE 2025-07-23 20:14:01 -07:00
Yuvi9587
337cdd342c Commit 2025-07-23 20:08:44 -07:00
Yuvi9587
d54b013bbc commit 2025-07-22 07:00:34 -07:00
Yuvi9587
2785fc1121 Update EmptyPopupDialog.py 2025-07-19 20:27:55 -07:00
31 changed files with 6237 additions and 2010 deletions

24
LICENSE
View File

@@ -1,11 +1,21 @@
Custom License - No Commercial Use
MIT License
Copyright [Yuvi9587] [2025]
Copyright (c) [2025] [Yuvi9587]
Permission is hereby granted to any person obtaining a copy of this software and associated documentation files (the "Software"), to use, copy, modify, and distribute the Software for **non-commercial purposes only**, subject to the following conditions:
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
1. The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
2. Proper credit must be given to the original author in any public use, distribution, or derivative works.
3. Commercial use, resale, or sublicensing of the Software or any derivative works is strictly prohibited without explicit written permission.
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND...
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

View File

@@ -1,192 +1,159 @@
# Kemono Downloader - Feature Guide
This guide provides a comprehensive overview of all user interface elements, input fields, buttons, popups, and functionalities available in the Kemono Downloader.
## 1. Main Interface & Workflow
These are the primary controls you'll interact with to initiate and manage downloads.
### 1.1. Core Inputs
**🔗 Creator/Post URL Input Field**  
- **Purpose**: Paste the URL of the content you want to download.  
- **Supported Sites**: Kemono.su, Coomer.party, Simpcity.su.  
- **Supported URL Types**:  
  - Creator pages (e.g., `https://kemono.su/patreon/user/12345`).  
  - Individual posts (e.g., `https://kemono.su/patreon/user/12345/post/98765`).  
- **Note**: When ⭐ Favorite Mode is active, this field is disabled. For Simpcity.su URLs, the "Use Cookie" option is mandatory and auto-enabled.
**🎨 Creator Selection Button**  
- **Icon**: 🎨 (Artist Palette)  
- **Purpose**: Opens the "Creator Selection" dialog to browse and queue downloads from known creators.  
- **Dialog Features**:  
  - Loads creators from `creators.json`.  
  - **Search Bar**: Filter creators by name.  
  - **Creator List**: Displays creators with their service (e.g., Patreon, Fanbox).  
  - **Selection**: Checkboxes to select one or more creators.  
  - **Download Scope**: Organize downloads by Characters or Creators.  
  - **Add to Queue**: Adds selected creators or their posts to the download queue.
**Page Range (Start to End) Input Fields**  
- **Purpose**: Specify a range of pages to fetch for creator URLs.  
- **Usage**: Enter the starting and ending page numbers.  
- **Behavior**:  
  - If blank, all pages are processed.  
  - Disabled for single post URLs.
**📁 Download Location Input Field & Browse Button**  
- **Purpose**: Specify the main directory for downloaded files.  
- **Usage**: Type the path or click "Browse..." to select a folder.  
- **Requirement**: Mandatory for all download operations.
### 1.2. Action Buttons
**⬇️ Start Download / 🔗 Extract Links Button**  
- **Purpose**: Initiates downloading or link extraction.  
- **Behavior**:  
  - Shows "🔗 Extract Links" if "Only Links" is selected.  
  - Otherwise, shows "⬇️ Start Download".  
  - Supports single-threaded or multi-threaded downloads based on settings.
**🔄 Restore Download Button**  
- **Visibility**: Appears if an incomplete session is detected on startup.  
- **Purpose**: Resumes a previously interrupted download session.
**⏸️ Pause / ▶️ Resume Download Button**  
- **Purpose**: Pause or resume the ongoing download.  
- **Behavior**: Toggles between "Pause" and "Resume". Some UI settings can be changed while paused.
**❌ Cancel & Reset UI Button**  
- **Purpose**: Stops the current operation and performs a "soft" reset.  
- **Behavior**: Halts background threads, preserves URL and Download Location inputs, resets other settings.
**🔄 Reset Button (in the log area)**  
- **Purpose**: Performs a "hard" reset when no operation is active.  
- **Behavior**: Clears all inputs, resets options to default, and clears logs.
## 2. Filtering & Content Selection
These options allow precise control over downloaded content.
### 2.1. Content Filtering
**🎯 Filter by Character(s) Input Field**  
- **Purpose**: Download content related to specific characters or series.  
- **Usage**: Enter comma-separated character names.  
- **Advanced Syntax**:  
  - `Nami`: Simple filter.  
  - `(Vivi, Ulti)`: Grouped filter. Matches posts with "Vivi" OR "Ulti". Creates a shared folder like `Vivi Ulti` if subfolders are enabled.  
  - `(Boa, Hancock)~`: Aliased filter. Treats "Boa" and "Hancock" as the same entity.
**Filter: [Type] Button (Character Filter Scope)**  
- **Purpose**: Defines where the character filter is applied. Cycles on click.  
- **Options**:  
  - **Filter: Title** (Default): Matches post titles.  
  - **Filter: Files**: Matches filenames.  
  - **Filter: Both**: Checks title first, then filenames.  
  - **Filter: Comments (Beta)**: Checks filenames, then post comments.
**🚫 Skip with Words Input Field**  
- **Purpose**: Exclude posts/files with specified keywords (e.g., `WIP`, `sketch`).
**Scope: [Type] Button (Skip Words Scope)**  
- **Purpose**: Defines where skip words are applied. Cycles on click.  
- **Options**:  
  - **Scope: Posts** (Default): Skips posts if the title contains a skip word.  
  - **Scope: Files**: Skips files if the filename contains a skip word.  
  - **Scope: Both**: Applies both rules.
**✂️ Remove Words from Name Input Field**  
- **Purpose**: Remove unwanted text from filenames (e.g., `patreon`, `[HD]`).
### 2.2. File Type Filtering
**Filter Files (Radio Buttons)**  
- **Purpose**: Select file types to download.  
- **Options**:  
  - **All**: All file types.  
  - **Images/GIFs**: Common image formats.  
  - **Videos**: Common video formats.  
  - **🎧 Only Audio**: Common audio formats.  
  - **📦 Only Archives**: Only `.zip` and `.rar` files.  
  - **🔗 Only Links**: Extracts external links without downloading files.
**Skip .zip / Skip .rar Checkboxes**  
- **Purpose**: Skip downloading `.zip` or `.rar` files.  
- **Behavior**: Disabled when "📦 Only Archives" is active.
## 3. Download Customization
Options to refine the download process and output.
- **Download Thumbnails Only**: Downloads small preview images instead of full-resolution files.  
- **Scan Content for Images**: Scans post HTML for `<img>` tags, crucial for images in descriptions.  
- **Compress to WebP**: Converts images to WebP format (requires Pillow library).
- **Keep Duplicates**: Normally, if a post contains multiple files with the same name, only the first is downloaded. Checking this option will download all of them, renaming subsequent unique files with a numeric suffix (e.g., `image_1.jpg`).
- **🗄️ Custom Folder Name (Single Post Only)**: Specify a custom folder name for a single post's content (appears if subfolders are enabled).
## 4. 📖 Manga/Comic Mode
A mode for downloading creator feeds in chronological order, ideal for sequential content.
- **Activation**: Active when downloading a creator's entire feed (not a single post).  
- **Core Behavior**: Fetches all posts, processing from oldest to newest.  
- **Filename Style Toggle Button (in the log area)**:  
  - **Purpose**: Controls file naming in Manga Mode. Cycles on click.  
  - **Options**:  
    - **Name: Post Title**: First file named after post title; others keep original names.  
    - **Name: Original File**: Files keep server-provided names, with optional prefix.  
    - **Name: Title+G.Num**: Global numbering with post title prefix (e.g., `Chapter 1_001.jpg`).  
    - **Name: Date Based**: Sequential naming by post date (e.g., `001.jpg`), with optional prefix.  
    - **Name: Post ID**: Files named after post ID to avoid clashes.  
    - **Name: Date + Title**: Combines post date and title for filenames.
## 5. Folder Organization & Known.txt
Controls for structuring downloaded content.
- **Separate Folders by Name/Title Checkbox**: Enables automatic subfolder creation.  
- **Subfolder per Post Checkbox**: Creates subfolders for each post, named after the post title.  
- **Date Prefix for Post Subfolders Checkbox**: When used with "Subfolder per Post," this option prefixes the folder name with the post's upload date (e.g., `2025-07-11 Post Title`), allowing for chronological sorting.
- **Known.txt Management UI (Bottom Left)**:  
  - **Purpose**: Manages a local `Known.txt` file for series, characters, or terms used in folder creation.  
  - **List Display**: Shows primary names from `Known.txt`.  
  - ** Add Button**: Adds names or groups (e.g., `(Character A, Alias B)~`).  
  - **⤵️ Add to Filter Button**: Select names from `Known.txt` for the character filter.  
  - **🗑️ Delete Selected Button**: Removes selected names from `Known.txt`.  
  - **Open Known.txt Button**: Opens the file in the default text editor.  
  - **❓ Help Button**: Opens this feature guide.  
  - **📜 History Button**: Views recent download history.
## 6. ⭐ Favorite Mode (Kemono.su Only)
Download from favorited artists/posts on Kemono.su.
- **Enable Checkbox ("⭐ Favorite Mode")**:  
  - Switches to Favorite Mode.  
  - Disables the main URL input.  
  - Changes action buttons to "Favorite Artists" and "Favorite Posts".  
  - Requires cookies.  
- **🖼️ Favorite Artists Button**: Select and download from favorited artists.  
- **📄 Favorite Posts Button**: Select and download specific favorited posts.  
- **Favorite Download Scope Button**:  
  - **Scope: Selected Location**: Downloads favorites to the main directory.  
  - **Scope: Artist Folders**: Creates subfolders per artist.
## 7. Advanced Settings & Performance
- **🍪 Cookie Management**:  
  - **Use Cookie Checkbox**: Enables cookies for restricted content.  
  - **Cookie Text Field**: Paste cookie string.  
  - **Browse... Button**: Select a `cookies.txt` file (Netscape format).  
- **Use Multithreading Checkbox & Threads Input**:  
  - **Purpose**: Configures simultaneous operations.  
  - **Behavior**: Sets concurrent post processing (creator feeds) or file downloads (single posts).  
- **Multi-part Download Toggle Button**:  
  - **Purpose**: Enables/disables multi-segment downloading for large files.  
  - **Note**: Best for large files; less efficient for small files.
## 8. Logging, Monitoring & Error Handling
- **📜 Progress Log Area**: Displays messages, progress, and errors.  
- **👁️ / 🙈 Log View Toggle Button**: Switches between Progress Log and Missed Character Log (skipped posts).  
- **Show External Links in Log**: Displays external links (e.g., Mega, Google Drive) in a secondary panel.  
- **Export Links Button**: Saves extracted links to a `.txt` file in "Only Links" mode.  
- **Download Extracted Links Button**: Downloads files from supported external links in "Only Links" mode.  
- **🆘 Error Button & Dialog**:  
  - **Purpose**: Active if files fail to download. The button will display a live count of failed files (e.g., **(3) Error**).  
  - **Dialog Features**:  
    - Lists failed files.  
    - Retry failed downloads.  
    - Export failed URLs to a text file.
## 9. Application Settings (⚙️)
- **Appearance**: Switch between Light and Dark themes.  
- **Language**: Change UI language (restart required).
<h1>Kemono Downloader - Comprehensive Feature Guide</h1>
<p>This guide provides a detailed overview of all user interface elements, input fields, buttons, popups, and functionalities available in the application.</p>
<hr>
<h2>1. Core Concepts &amp; Supported Sites</h2>
<h3>URL Input (🔗)</h3>
<p>This is the primary input field where you specify the content you want to download.</p>
<p><strong>Supported URL Types:</strong></p>
<ul>
<li><strong>Creator URL</strong>: A link to a creator's main page. Downloads all posts from that creator.</li>
<li><strong>Post URL</strong>: A direct link to a specific post. Downloads only that single post.</li>
<li><strong>Batch Command</strong>: Special keywords to trigger bulk downloading from a text file (see Batch Downloading section).</li>
</ul>
<p><strong>Supported Websites:</strong></p>
<ul>
<li>Kemono (<code>kemono.su</code>, <code>kemono.party</code>, etc.)</li>
<li>Coomer (<code>coomer.su</code>, <code>coomer.party</code>, etc.)</li>
<li>Discord (via Kemono/Coomer API)</li>
<li>Bunkr</li>
<li>Erome</li>
<li>Saint2.su</li>
<li>nhentai</li>
</ul>
<hr>
<h2>2. Main Download Controls &amp; Inputs</h2>
<h3>Download Location (📁)</h3>
<p>This input defines the main folder where your files will be saved.</p>
<ul>
<li><strong>Browse Button</strong>: Opens a system dialog to choose a folder.</li>
<li><strong>Directory Creation</strong>: If the folder doesn't exist, the app will ask for confirmation to create it.</li>
</ul>
<h3>Filter by Character(s) &amp; Scope</h3>
<p>Used to download content for specific characters or series and organize them into subfolders.</p>
<ul>
<li><strong>Input Field</strong>: Enter comma-separated names (e.g., <code>Tifa, Aerith</code>). Group aliases using parentheses for folder naming (e.g., <code>(Cloud, Zack)</code>).</li>
<li><strong>Scope Button</strong>: Cycles through where to look for name matches:
<ul>
<li><strong>Filter: Title</strong>: Matches names in the post title.</li>
<li><strong>Filter: Files</strong>: Matches names in the filenames.</li>
<li><strong>Filter: Both</strong>: Checks the title first, then filenames.</li>
<li><strong>Filter: Comments</strong>: Checks filenames first, then post comments.</li>
</ul>
</li>
</ul>
<h3>Skip with Words &amp; Scope</h3>
<p>Prevents downloading content based on keywords or file size.</p>
<ul>
<li><strong>Input Field</strong>: Enter comma-separated keywords (e.g., <code>WIP, sketch, preview</code>).</li>
<li><strong>Skip by Size</strong>: Enter a number in square brackets to skip any file <strong>smaller than</strong> that size in MB. For example, <code>WIP, [200]</code> skips files with "WIP" in the name AND any file smaller than 200 MB.</li>
<li><strong>Scope Button</strong>: Cycles through where to apply keyword filters:
<ul>
<li><strong>Scope: Posts</strong>: Skips the entire post if the title matches.</li>
<li><strong>Scope: Files</strong>: Skips individual files if the filename matches.</li>
<li><strong>Scope: Both</strong>: Checks the post title first, then individual files.</li>
</ul>
</li>
</ul>
<h3>Remove Words from Name (✂️)</h3>
<p>Enter comma-separated words to remove from final filenames (e.g., <code>patreon, [HD]</code>). This helps clean up file naming.</p>
<hr>
<h2>3. Primary Download Modes (Filter File Section)</h2>
<p>This section uses radio buttons to set the main download mode. Only one can be active at a time.</p>
<ul>
<li><strong>All</strong>: Default mode. Downloads every file and attachment.</li>
<li><strong>Images/GIFs</strong>: Downloads only common image formats.</li>
<li><strong>Videos</strong>: Downloads only common video formats.</li>
<li><strong>Only Archives</strong>: Downloads only <code>.zip</code>, <code>.rar</code>, etc.</li>
<li><strong>Only Audio</strong>: Downloads only common audio formats.</li>
<li><strong>Only Links</strong>: Extracts external hyperlinks (e.g., Mega, Google Drive) from post descriptions instead of downloading files. <strong>This mode unlocks special features</strong> (see section 6).</li>
<li><strong>More</strong>: Opens a dialog to download text-based content.
<ul>
<li><strong>Scope</strong>: Choose to extract text from the post description or comments.</li>
<li><strong>Export Format</strong>: Save as PDF, DOCX, or TXT.</li>
<li><strong>Single PDF</strong>: Compile all text from the session into one consolidated PDF file.</li>
</ul>
</li>
</ul>
<hr>
<h2>4. Advanced Features &amp; Toggles (Checkboxes)</h2>
<h3>Folder Organization</h3>
<ul>
<li><strong>Separate folders by Known.txt</strong>: Automatically organizes downloads into subfolders based on name matches from your <code>Known.txt</code> list or the "Filter by Character(s)" input.</li>
<li><strong>Subfolder per post</strong>: Creates a unique folder for each post, named after the post's title. This prevents files from different posts from mixing.</li>
<li><strong>Date prefix</strong>: (Only available with "Subfolder per post") Prepends the post date to the folder name (e.g., <code>2025-08-03 My Post Title</code>) for chronological sorting.</li>
</ul>
<h3>Special Modes</h3>
<ul>
<li><strong>⭐ Favorite Mode</strong>: Switches the UI to download from your personal favorites list instead of using the URL input.</li>
<li><strong>Manga/Comic mode</strong>: Sorts a creator's posts from oldest to newest before downloading, ensuring correct page order. A scope button appears to control the filename style (e.g., using post title, date, or a global number).</li>
</ul>
<h3>File Handling</h3>
<ul>
<li><strong>Skip Archives</strong>: Ignores <code>.zip</code> and <code>.rar</code> files during downloads.</li>
<li><strong>Download Thumbnail Only</strong>: Saves only the small preview images instead of full-resolution files.</li>
<li><strong>Scan Content for Images</strong>: Parses post HTML to find embedded images that may not be listed in the API data.</li>
<li><strong>Compress to WebP</strong>: Converts large images (over 1.5 MB) to the space-saving WebP format.</li>
<li><strong>Keep Duplicates</strong>: Opens a dialog to control how duplicate files are handled (skip by default, keep all, or keep a specific number of copies).</li>
</ul>
<h3>General Functionality</h3>
<ul>
<li><strong>Use cookie</strong>: Enables login-based access. You can paste a cookie string or browse for a <code>cookies.txt</code> file.</li>
<li><strong>Use Multithreading</strong>: Enables parallel processing of posts for faster downloads. You can set the number of concurrent worker threads.</li>
<li><strong>Show external links in log</strong>: Opens a secondary log panel that displays external links found in post descriptions.</li>
</ul>
<hr>
<h2>5. Specialized Downloaders &amp; Batch Mode</h2>
<h3>Discord Features</h3>
<ul>
<li>When a Discord URL is entered, a <strong>Scope</strong> button appears.
<ul>
<li><strong>Scope: Files</strong>: Downloads all files from the channel/server.</li>
<li><strong>Scope: Messages</strong>: Saves the entire message history of the channel/server as a formatted PDF.</li>
</ul>
</li>
<li>A <strong>"Save as PDF"</strong> button also appears as a shortcut for the message saving feature.</li>
</ul>
<h3>Batch Downloading (<code>nhentai</code> &amp; <code>saint2.su</code>)</h3>
<p>This feature allows you to download hundreds of galleries or videos from a simple text file.</p>
<ol>
<li>In the <code>appdata</code> folder, create <code>nhentai.txt</code> or <code>saint2.su.txt</code>.</li>
<li>Add one full URL per line to the corresponding file.</li>
<li>In the app's URL input, type either <code>nhentai.net</code> or <code>saint2.su</code> and click "Start Download".</li>
<li>The app will read the file and process every URL in the queue.</li>
</ol>
<hr>
<h2>6. "Only Links" Mode: Extraction &amp; Direct Download</h2>
<p>When you select the <strong>"Only Links"</strong> radio button, the application's behavior changes significantly.</p>
<ul>
<li><strong>Link Extraction</strong>: Instead of downloading files, the main log panel will fill with all external links found (Mega, Google Drive, Dropbox, etc.).</li>
<li><strong>Export Links</strong>: An "Export Links" button appears, allowing you to save the full list of extracted URLs to a <code>.txt</code> file.</li>
<li><strong>Direct Cloud Download</strong>: A <strong>"Download"</strong> button appears next to the export button.
<ul>
<li>Clicking this opens a new dialog listing all supported cloud links (Mega, G-Drive, Dropbox).</li>
<li>You can select which files you want to download from this list.</li>
<li>The application will then download the selected files directly from the cloud service to your chosen download location.</li>
</ul>
</li>
</ul>
<hr>
<h2>7. Session &amp; Process Management</h2>
<h3>Main Action Buttons</h3>
<ul>
<li><strong>Start Download</strong>: Begins the download process. This button's text changes contextually (e.g., "Extract Links", "Check for Updates").</li>
<li><strong>Pause / Resume</strong>: Pauses or resumes the ongoing download. When paused, you can safely change some settings.</li>
<li><strong>Cancel &amp; Reset UI</strong>: Stops the current download and performs a soft reset of the UI, preserving your URL and download location.</li>
</ul>
<h3>Restore Interrupted Download</h3>
<p>If the application is closed unexpectedly during a download, it will save its progress.</p>
<ul>
<li>On the next launch, the UI will be pre-filled with the settings from the interrupted session.</li>
<li>The <strong>Pause</strong> button will change to <strong>"🔄 Restore Download"</strong>. Clicking it will resume the download exactly where it left off, skipping already processed posts.</li>
<li>The <strong>Cancel</strong> button will change to <strong>"🗑️ Discard Session"</strong>, allowing you to clear the saved state and start fresh.</li>
</ul>
<h3>Other UI Controls</h3>
<ul>
<li><strong>Error Button</strong>: Shows a count of failed files. Clicking it opens a dialog where you can view, export, or retry the failed downloads.</li>
<li><strong>History Button</strong>: Shows a log of recently downloaded files and processed posts.</li>
<li><strong>Settings Button</strong>: Opens the settings dialog where you can change the theme, language, and <strong>check for application updates</strong>.</li>
<li><strong>Support Button</strong>: Opens a dialog with links to the project's source code and developer support pages.</li>
</ul>

View File

@@ -107,4 +107,4 @@ def main():
if __name__ == '__main__':
main()
main()

303
readme.md
View File

@@ -1,217 +1,116 @@
<h1 align="center">Kemono Downloader v6.0.0</h1>
<h1 align="center">Kemono Downloader</h1>
<div align="center">
<table>
<tr>
<td align="center">
<img src="Read/Read.png" alt="Default Mode" width="400"><br>
<strong>Default</strong>
</td>
<td align="center">
<img src="Read/Read1.png" alt="Favorite Mode" width="400"><br>
<strong>Favorite Mode</strong>
</td>
</tr>
<tr>
<td align="center">
<img src="Read/Read2.png" alt="Single Post" width="400"><br>
<strong>Single Post</strong>
</td>
<td align="center">
<img src="Read/Read3.png" alt="Manga/Comic Mode" width="400"><br>
<strong>Manga/Comic Mode</strong>
</td>
</tr>
</table>
<table>
<tbody>
<tr>
<td align="center">
<img src="Read/Read.png" alt="Default Mode" width="400"><br>
<strong>Default Mode</strong>
</td>
<td align="center">
<img src="Read/Read1.png" alt="Favorite Mode" width="400"><br>
<strong>Favorite Mode</strong>
</td>
</tr>
<tr>
<td align="center">
<img src="Read/Read2.png" alt="Single Post" width="400"><br>
<strong>Single Post</strong>
</td>
<td align="center">
<img src="Read/Read3.png" alt="Manga/Comic Mode" width="400"><br>
<strong>Manga/Comic Mode</strong>
</td>
</tr>
</tbody>
</table>
</div>
---
<hr>
A powerful, feature-rich GUI application for downloading content from **[Kemono.su](https://kemono.su)** (and its mirrors like kemono.party) and **[Coomer.party](https://coomer.party)** (and its mirrors like coomer.su).
Built with PyQt5, this tool is designed for users who want deep filtering capabilities, customizable folder structures, efficient downloads, and intelligent automation — all within a modern and user-friendly graphical interface.
<p>A powerful, feature-rich GUI application for downloading content from a wide array of sites, including <strong>Kemono</strong>, <strong>Coomer</strong>, <strong>Bunkr</strong>, <strong>Erome</strong>, <strong>Saint2.su</strong>, and <strong>nhentai</strong>.</p>
<p>Built with PyQt5, this tool is designed for users who want deep filtering capabilities, customizable folder structures, efficient downloads, and intelligent automation — all within a modern and user-friendly graphical interface.</p>
<div align="center">
[![](https://img.shields.io/badge/📚%20Full%20Feature%20List-FFD700?style=for-the-badge&logoColor=black&color=FFD700)](features.md)
[![](https://img.shields.io/badge/📝%20License-90EE90?style=for-the-badge&logoColor=black&color=90EE90)](LICENSE)
[![](https://img.shields.io/badge/⚠️%20Important%20Note-FFCCCB?style=for-the-badge&logoColor=black&color=FFCCCB)](note.md)
<a href="features.md"><img src="https://img.shields.io/badge/📚%20Full%20Feature%20List-FFD700?style=for-the-badge&logoColor=black&color=FFD700" alt="Full Feature List"></a>
<a href="LICENSE"><img src="https://img.shields.io/badge/📝%20License-90EE90?style=for-the-badge&logoColor=black&color=90EE90" alt="License"></a>
<a href="note.md"><img src="https://img.shields.io/badge/⚠️%20Important%20Note-FFCCCB?style=for-the-badge&logoColor=black&color=FFCCCB" alt="Important Note"></a>
</div>
---
## Feature Overview
Kemono Downloader offers a range of features to streamline your content downloading experience:
- **User-Friendly Interface:** A modern PyQt5 GUI for easy navigation and operation.
- **Flexible Downloading:**
- Download content from Kemono.su (and mirrors) and Coomer.party (and mirrors).
- Supports creator pages (with page range selection) and individual post URLs.
- Standard download controls: Start, Pause, Resume, and Cancel.
- **Powerful Filtering:**
- **Character Filtering:** Filter content by character names. Supports simple comma-separated names and grouped names for shared folders.
- **Keyword Skipping:** Skip posts or files based on specified keywords.
- **Filename Cleaning:** Remove unwanted words or phrases from downloaded filenames.
- **File Type Selection:** Choose to download all files, or limit to images/GIFs, videos, audio, or archives. Can also extract external links only.
- **Customizable Downloads:**
- **Thumbnails Only:** Option to download only small preview images.
- **Content Scanning:** Scan post HTML for `<img>` tags and direct image links, useful for images embedded in descriptions.
- **WebP Conversion:** Convert images to WebP format for smaller file sizes (requires Pillow library).
- **Organized Output:**
- **Automatic Subfolders:** Create subfolders based on character names (from filters or `Known.txt`) or post titles.
- **Per-Post Subfolders:** Option to create an additional subfolder for each individual post.
- **Manga/Comic Mode:**
- Downloads posts from a creator's feed in chronological order (oldest to newest).
- Offers various filename styling options for sequential reading (e.g., post title, original name, global numbering).
- **⭐ Favorite Mode:**
- Directly download from your favorited artists and posts on Kemono.su.
- Requires a valid cookie and adapts the UI for easy selection from your favorites.
- Supports downloading into a single location or artist-specific subfolders.
- **Performance & Advanced Options:**
- **Cookie Support:** Use cookies (paste string or load from `cookies.txt`) to access restricted content.
- **Multithreading:** Configure the number of simultaneous downloads/post processing threads for improved speed.
- **Logging:**
- A detailed progress log displays download activity, errors, and summaries.
- **Multi-language Interface:** Choose from several languages for the UI (English, Japanese, French, Spanish, German, Russian, Korean, Chinese Simplified).
- **Theme Customization:** Selectable Light and Dark themes for user comfort.
---
## ✨ What's New in v6.0.0
This release focuses on providing more granular control over file organization and improving at-a-glance status monitoring.
### New Features
- **Live Error Count on Button**
The **"Error" button** now dynamically displays the number of failed files during a download. Instead of opening the dialog, you can quickly see a live count like `(3) Error`, helping you track issues at a glance.
- **Date Prefix for Post Subfolders**
A new checkbox labeled **"Date Prefix"** is now available in the advanced settings.
When enabled alongside **"Subfolder per Post"**, it prepends the post's upload date to the folder name (e.g., `2025-07-11 Post Title`).
This makes your downloads sortable and easier to browse chronologically.
- **Keep Duplicates Within a Post**
A **"Keep Duplicates"** option has been added to preserve all files from a post — even if some have the same name.
Instead of skipping or overwriting, the downloader will save duplicates with numbered suffixes (e.g., `image.jpg`, `image_1.jpg`, etc.), which is especially useful when the same file name points to different media.
### Bug Fixes
- The downloader now correctly renames large `.part` files when completed, avoiding leftover temp files.
- The list of failed files shown in the Error Dialog is now saved and restored with your session — so no errors get lost if you close the app.
- Your selected download location is remembered, even after pressing the **Reset** button.
- The **Cancel** button is now enabled when restoring a pending session, so you can abort stuck jobs more easily.
- Internal cleanup logs (like "Deleting post cache") are now excluded from the final download summary for clarity.
---
## 📅 Next Update Plans
### 🔖 Post Tag Filtering (Planned for v6.1.0)
A powerful new **"Filter by Post Tags"** feature is planned:
- Filter and download content based on specific post tags.
- Combine tag filtering with current filters (character, file type, etc.).
- Use tag presets to automate frequent downloads.
This will provide **much greater control** over what gets downloaded, especially for creators who use tags consistently.
### 📁 Creator Download History (.json Save)
To streamline incremental downloads, a new system will allow the app to:
- Save a `.json` file with metadata about already-downloaded posts.
- Compare that file on future runs, so only **new** posts are downloaded.
- Avoids duplication and makes regular syncs fast and efficient.
Ideal for users managing large collections or syncing favorites regularly.
---
## 💻 Installation
### Requirements
- Python 3.6 or higher
- pip (Python package installer)
### Install Dependencies
```bash
pip install PyQt5 requests Pillow mega.py
```
### Running the Application
Navigate to the application's directory in your terminal and run:
```bash
python main.py
```
### Optional Setup
- **Main Inputs:**
- Place your `cookies.txt` in the root directory (if using cookies).
- Prepare your `Known.txt` and `creators.json` in the same directory for advanced filtering and selection features.
---
## Troubleshooting
### AttributeError: module 'asyncio' has no attribute 'coroutine'
If you encounter an error message similar to:
```
AttributeError: module 'asyncio' has no attribute 'coroutine'. Did you mean: 'coroutines'?
```
This usually means that a dependency, often `tenacity` (used by `mega.py`), is an older version that's incompatible with your Python version (typically Python 3.10+).
To fix this, activate your virtual environment and run the following commands to upgrade the libraries:
```bash
pip install --upgrade tenacity
pip install --upgrade mega.py
```
---
## Contribution
Feel free to fork this repo and submit pull requests for bug fixes, new features, or UI improvements!
---
## License
This project is under the Custom Licence
## Star History
<h2>Core Capabilities Overview</h2>
<h3>High-Performance &amp; Resilient Downloading</h3>
<ul>
<li><strong>Multi-threading:</strong> Processes multiple posts simultaneously to greatly accelerate downloads from large creator profiles.</li>
<li><strong>Multi-part Downloading:</strong> Splits large files into chunks and downloads them in parallel to maximize speed.</li>
<li><strong>Session Management:</strong> Supports pausing, resuming, and <strong>restoring downloads</strong> after crashes or interruptions, so you never lose your progress.</li>
</ul>
<h3>Expanded Site Support</h3>
<ul>
<li><strong>Direct Downloading:</strong> Full support for Kemono, Coomer, Bunkr, Erome, Saint2.su, and nhentai.</li>
<li><strong>Batch Mode:</strong> Download hundreds of URLs at once from <code>nhentai.txt</code> or <code>saint2.su.txt</code> files.</li>
<li><strong>Discord Support:</strong> Download files or save entire channel histories as PDFs directly through the API.</li>
</ul>
<h3>Advanced Filtering &amp; Content Control</h3>
<ul>
<li><strong>Content Type Filtering:</strong> Select whether to download all files or limit to images, videos, audio, or archives only.</li>
<li><strong>Keyword Skipping:</strong> Automatically skips posts or files containing certain keywords (e.g., "WIP", "sketch").</li>
<li><strong>Skip by Size:</strong> Avoid small files by setting a minimum size threshold in MB (e.g., <code>[200]</code>).</li>
<li><strong>Character Filtering:</strong> Restricts downloads to posts that match specific character or series names, with scope controls for title, filename, or comments.</li>
</ul>
<h3>Intelligent File Organization</h3>
<ul>
<li><strong>Automated Subfolders:</strong> Automatically organizes downloaded files into subdirectories based on character names or per post.</li>
<li><strong>Advanced File Renaming:</strong> Flexible renaming options, especially in Manga Mode, including by post title, date, sequential numbering, or post ID.</li>
<li><strong>Filename Cleaning:</strong> Automatically removes unwanted text from filenames.</li>
</ul>
<h3>Specialized Modes</h3>
<ul>
<li><strong>Manga/Comic Mode:</strong> Sorts posts chronologically before downloading to ensure pages appear in the correct sequence.</li>
<li><strong>Favorite Mode:</strong> Connects to your account and downloads from your favorites list (artists or posts).</li>
<li><strong>Link Extraction Mode:</strong> Extracts external links (Mega, Google Drive) from posts for export or <strong>direct in-app downloading</strong>.</li>
<li><strong>Text Extraction Mode:</strong> Saves post descriptions or comment sections as <code>PDF</code>, <code>DOCX</code>, or <code>TXT</code> files.</li>
</ul>
<h3>Utility &amp; Advanced Features</h3>
<ul>
<li><strong>In-App Updater:</strong> Check for new versions directly from the settings menu.</li>
<li><strong>Cookie Support:</strong> Enables access to subscriber-only content via browser session cookies.</li>
<li><strong>Duplicate Detection:</strong> Prevents saving duplicate files using content-based comparison, with configurable limits.</li>
<li><strong>Image Compression:</strong> Automatically converts large images to <code>.webp</code> to reduce disk usage.</li>
<li><strong>Creator Management:</strong> Built-in creator browser and update checker for downloading only new posts from saved profiles.</li>
<li><strong>Error Handling:</strong> Tracks failed downloads and provides a retry dialog with options to export or redownload missing files.</li>
</ul>
<h2>💻 Installation</h2>
<h3>Requirements</h3>
<ul>
<li>Python 3.6 or higher</li>
<li>pip (Python package installer)</li>
</ul>
<h3>Install Dependencies</h3>
<pre><code>pip install PyQt5 requests cloudscraper Pillow fpdf2 python-docx
</code></pre>
<h3>Running the Application</h3>
<p>Navigate to the application's directory in your terminal and run:</p>
<pre><code>python main.py
</code></pre>
<h2>Contribution</h2>
<p>Feel free to fork this repo and submit pull requests for bug fixes, new features, or UI improvements!</p>
<h2>License</h2>
<p>This project is under the MIT Licence</p>
<h2>Star History</h2>
<table align="center" style="border-collapse: collapse; border: none; margin-left: auto; margin-right: auto;">
<tr>
<td align="center" valign="middle" style="padding: 10px; border: none;">
<a href="https://www.star-history.com/#Yuvi9587/Kemono-Downloader&Date">
<img src="https://api.star-history.com/svg?repos=Yuvi9587/Kemono-Downloader&type=Date" alt="Star History Chart" width="650">
</a>
<tbody>
<tr>
<td align="center" valign="middle" style="padding: 10px; border: none;">
<a href="https://www.star-history.com/#Yuvi9587/Kemono-Downloader&amp;Date">
<img src="https://api.star-history.com/svg?repos=Yuvi9587/Kemono-Downloader&amp;type=Date" alt="Star History Chart" width="650">
</a>
</td>
</tr>
</tbody>
</table>
<p align="center">
<a href="https://buymeacoffee.com/yuvi9587">
<img src="https://img.shields.io/badge/🍺%20Buy%20Me%20a%20Coffee-FFCCCB?style=for-the-badge&logoColor=black&color=FFDD00" alt="Buy Me a Coffee">
<img src="https://img.shields.io/badge/🍺%20Buy%20Me%20a%20Coffee-FFCCCB?style=for-the-badge&amp;logoColor=black&amp;color=FFDD00" alt="Buy Me a Coffee">
</a>
</p>

View File

@@ -1,4 +1,3 @@
# --- Application Metadata ---
CONFIG_ORGANIZATION_NAME = "KemonoDownloader"
CONFIG_APP_NAME_MAIN = "ApplicationSettings"
CONFIG_APP_NAME_TOUR = "ApplicationTour"
@@ -9,7 +8,7 @@ STYLE_ORIGINAL_NAME = "original_name"
STYLE_DATE_BASED = "date_based"
STYLE_DATE_POST_TITLE = "date_post_title"
STYLE_POST_TITLE_GLOBAL_NUMBERING = "post_title_global_numbering"
STYLE_POST_ID = "post_id" # Add this line
STYLE_POST_ID = "post_id"
MANGA_DATE_PREFIX_DEFAULT = ""
# --- Download Scopes ---
@@ -59,6 +58,11 @@ LANGUAGE_KEY = "currentLanguageV1"
DOWNLOAD_LOCATION_KEY = "downloadLocationV1"
RESOLUTION_KEY = "window_resolution"
UI_SCALE_KEY = "ui_scale_factor"
SAVE_CREATOR_JSON_KEY = "saveCreatorJsonProfile"
FETCH_FIRST_KEY = "fetchAllPostsFirst"
DISCORD_TOKEN_KEY = "discord/token"
POST_DOWNLOAD_ACTION_KEY = "postDownloadAction"
# --- UI Constants and Identifiers ---
HTML_PREFIX = "<!HTML!>"
@@ -96,7 +100,7 @@ FOLDER_NAME_STOP_WORDS = {
"for", "he", "her", "his", "i", "im", "in", "is", "it", "its",
"me", "my", "net", "not", "of", "on", "or", "org", "our",
"s", "she", "so", "the", "their", "they", "this",
"to", "ve", "was", "we", "were", "with", "www", "you", "your",
"to", "ve", "was", "we", "were", "with", "www", "you", "your", "nsfw", "sfw",
# add more according to need
}
@@ -110,10 +114,12 @@ CREATOR_DOWNLOAD_DEFAULT_FOLDER_IGNORE_WORDS = {
"may", "jun", "june", "jul", "july", "aug", "august", "sep", "september",
"oct", "october", "nov", "november", "dec", "december",
"mon", "monday", "tue", "tuesday", "wed", "wednesday", "thu", "thursday",
"fri", "friday", "sat", "saturday", "sun", "sunday"
"fri", "friday", "sat", "saturday", "sun", "sunday", "Pack", "tier", "spoiler",
# add more according to need
}
# --- Duplicate Handling Modes ---
DUPLICATE_HANDLING_HASH = "hash"
DUPLICATE_HANDLING_KEEP_ALL = "keep_all"
DUPLICATE_HANDLING_KEEP_ALL = "keep_all"

View File

@@ -0,0 +1,72 @@
# src/core/Hentai2read_client.py
import re
import os
import json
import requests
import cloudscraper
from bs4 import BeautifulSoup
def fetch_hentai2read_data(url, logger, session):
"""
Scrapes a SINGLE Hentai2Read chapter page using a provided session.
"""
logger(f"Attempting to fetch chapter data from: {url}")
try:
response = session.get(url, timeout=30)
response.raise_for_status()
page_content_text = response.text
soup = BeautifulSoup(page_content_text, 'html.parser')
album_title = ""
title_tags = soup.select('span[itemprop="name"]')
if title_tags:
album_title = title_tags[-1].text.strip()
if not album_title:
title_tag = soup.select_one('h1.title')
if title_tag:
album_title = title_tag.text.strip()
if not album_title:
logger("❌ Could not find album title on page.")
return None, None
image_urls = []
try:
start_index = page_content_text.index("'images' : ") + len("'images' : ")
end_index = page_content_text.index(",\n", start_index)
images_json_str = page_content_text[start_index:end_index]
image_paths = json.loads(images_json_str)
image_urls = ["https://hentaicdn.com/hentai" + part for part in image_paths]
except (ValueError, json.JSONDecodeError):
logger("❌ Could not find or parse image JSON data for this chapter.")
return None, None
if not image_urls:
logger("❌ No image URLs found for this chapter.")
return None, None
logger(f" Found {len(image_urls)} images for album '{album_title}'.")
files_to_download = []
for i, img_url in enumerate(image_urls):
page_num = i + 1
extension = os.path.splitext(img_url)[1].split('?')[0]
if not extension: extension = ".jpg"
filename = f"{page_num:03d}{extension}"
files_to_download.append({'url': img_url, 'filename': filename})
return album_title, files_to_download
except requests.exceptions.HTTPError as e:
if e.response.status_code == 404:
logger(f" Chapter not found (404 Error). This likely marks the end of the series.")
else:
logger(f"❌ An HTTP error occurred: {e}")
return None, None
except Exception as e:
logger(f"❌ An unexpected error occurred while fetching data: {e}")
return None, None

View File

@@ -1,8 +1,9 @@
import time
import traceback
from urllib.parse import urlparse
import json # Ensure json is imported
import json
import requests
import cloudscraper
from ..utils.network_utils import extract_post_info, prepare_cookies_for_request
from ..config.constants import (
STYLE_DATE_POST_TITLE
@@ -12,7 +13,6 @@ from ..config.constants import (
def fetch_posts_paginated(api_url_base, headers, offset, logger, cancellation_event=None, pause_event=None, cookies_dict=None):
"""
Fetches a single page of posts from the API with robust retry logic.
NEW: Requests only essential fields to keep the response size small and reliable.
"""
if cancellation_event and cancellation_event.is_set():
raise RuntimeError("Fetch operation cancelled by user.")
@@ -33,7 +33,7 @@ def fetch_posts_paginated(api_url_base, headers, offset, logger, cancellation_ev
if cancellation_event and cancellation_event.is_set():
raise RuntimeError("Fetch operation cancelled by user during retry loop.")
log_message = f" Fetching post list: {api_url_base}?o={offset} (Page approx. {offset // 50 + 1})"
log_message = f" Fetching post list: {paginated_url} (Page approx. {offset // 50 + 1})"
if attempt > 0:
log_message += f" (Attempt {attempt + 1}/{max_retries})"
logger(log_message)
@@ -41,9 +41,23 @@ def fetch_posts_paginated(api_url_base, headers, offset, logger, cancellation_ev
try:
response = requests.get(paginated_url, headers=headers, timeout=(15, 60), cookies=cookies_dict)
response.raise_for_status()
response.encoding = 'utf-8'
return response.json()
except requests.exceptions.RequestException as e:
# Handle 403 error on the FIRST page as a rate limit/block
if e.response is not None and e.response.status_code == 403 and offset == 0:
logger(" ❌ Access Denied (403 Forbidden) on the first page.")
logger(" This is likely a rate limit or a Cloudflare block.")
logger(" 💡 SOLUTION: Wait a while, use a VPN, or provide a valid session cookie.")
return [] # Stop the process gracefully
# Handle 400 error as the end of pages
if e.response is not None and e.response.status_code == 400:
logger(f" ✅ Reached end of posts (API returned 400 Bad Request for offset {offset}).")
return []
# Handle all other network errors with a retry
logger(f" ⚠️ Retryable network error on page fetch (Attempt {attempt + 1}): {e}")
if attempt < max_retries - 1:
delay = retry_delay * (2 ** attempt)
@@ -65,26 +79,28 @@ def fetch_posts_paginated(api_url_base, headers, offset, logger, cancellation_ev
raise RuntimeError(f"Failed to fetch page {paginated_url} after all attempts.")
def fetch_single_post_data(api_domain, service, user_id, post_id, headers, logger, cookies_dict=None):
"""
--- NEW FUNCTION ---
Fetches the full data, including the 'content' field, for a single post.
--- MODIFIED FUNCTION ---
Fetches the full data, including the 'content' field, for a single post using cloudscraper.
"""
post_api_url = f"https://{api_domain}/api/v1/{service}/user/{user_id}/post/{post_id}"
logger(f" Fetching full content for post ID {post_id}...")
scraper = cloudscraper.create_scraper()
try:
with requests.get(post_api_url, headers=headers, timeout=(15, 300), cookies=cookies_dict, stream=True) as response:
response.raise_for_status()
response_body = b""
for chunk in response.iter_content(chunk_size=8192):
response_body += chunk
full_post_data = json.loads(response_body)
if isinstance(full_post_data, list) and full_post_data:
return full_post_data[0]
return full_post_data
response = scraper.get(post_api_url, headers=headers, timeout=(15, 300), cookies=cookies_dict)
response.raise_for_status()
full_post_data = response.json()
if isinstance(full_post_data, list) and full_post_data:
return full_post_data[0]
if isinstance(full_post_data, dict) and 'post' in full_post_data:
return full_post_data['post']
return full_post_data
except Exception as e:
logger(f" ❌ Failed to fetch full content for post {post_id}: {e}")
return None
@@ -101,6 +117,7 @@ def fetch_post_comments(api_domain, service, user_id, post_id, headers, logger,
try:
response = requests.get(comments_api_url, headers=headers, timeout=(10, 30), cookies=cookies_dict)
response.raise_for_status()
response.encoding = 'utf-8'
return response.json()
except requests.exceptions.RequestException as e:
raise RuntimeError(f"Error fetching comments for post {post_id}: {e}")
@@ -120,12 +137,18 @@ def download_from_api(
selected_cookie_file=None,
app_base_dir=None,
manga_filename_style_for_sort_check=None,
processed_post_ids=None # --- ADD THIS ARGUMENT ---
):
processed_post_ids=None,
fetch_all_first=False
):
parsed_input_url_for_domain = urlparse(api_url_input)
api_domain = parsed_input_url_for_domain.netloc
headers = {
'User-Agent': 'Mozilla/5.0',
'Accept': 'application/json'
'User-Agent': 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)',
'Referer': f'https://{api_domain}/',
'Accept': 'text/css'
}
if processed_post_ids is None:
processed_post_ids = set()
else:
@@ -137,11 +160,12 @@ def download_from_api(
logger(" Download_from_api cancelled at start.")
return
parsed_input_url_for_domain = urlparse(api_url_input)
api_domain = parsed_input_url_for_domain.netloc
if not any(d in api_domain.lower() for d in ['kemono.su', 'kemono.party', 'coomer.su', 'coomer.party']):
# The code that defined api_domain was moved from here to the top of the function
if not any(d in api_domain.lower() for d in ['kemono.su', 'kemono.party', 'kemono.cr', 'coomer.su', 'coomer.party', 'coomer.st']):
logger(f"⚠️ Unrecognized domain '{api_domain}' from input URL. Defaulting to kemono.su for API calls.")
api_domain = "kemono.su"
cookies_for_api = None
if use_cookie and app_base_dir:
cookies_for_api = prepare_cookies_for_request(use_cookie, cookie_text, selected_cookie_file, app_base_dir, logger, target_domain=api_domain)
@@ -154,6 +178,7 @@ def download_from_api(
try:
direct_response = requests.get(direct_post_api_url, headers=headers, timeout=(10, 30), cookies=cookies_for_api)
direct_response.raise_for_status()
direct_response.encoding = 'utf-8'
direct_post_data = direct_response.json()
if isinstance(direct_post_data, list) and direct_post_data:
direct_post_data = direct_post_data[0]
@@ -178,7 +203,8 @@ def download_from_api(
logger("⚠️ Page range (start/end page) is ignored when a specific post URL is provided (searching all pages for the post).")
is_manga_mode_fetch_all_and_sort_oldest_first = manga_mode and (manga_filename_style_for_sort_check != STYLE_DATE_POST_TITLE) and not target_post_id
api_base_url = f"https://{api_domain}/api/v1/{service}/user/{user_id}"
should_fetch_all = fetch_all_first or is_manga_mode_fetch_all_and_sort_oldest_first
api_base_url = f"https://{api_domain}/api/v1/{service}/user/{user_id}/posts"
page_size = 50
if is_manga_mode_fetch_all_and_sort_oldest_first:
logger(f" Manga Mode (Style: {manga_filename_style_for_sort_check if manga_filename_style_for_sort_check else 'Default'} - Oldest First Sort Active): Fetching all posts to sort by date...")
@@ -220,6 +246,9 @@ def download_from_api(
logger(f" Manga Mode: No posts found within the specified page range ({start_page or 1}-{end_page}).")
break
all_posts_for_manga_mode.extend(posts_batch_manga)
logger(f"MANGA_FETCH_PROGRESS:{len(all_posts_for_manga_mode)}:{current_page_num_manga}")
current_offset_manga += page_size
time.sleep(0.6)
except RuntimeError as e:
@@ -232,7 +261,12 @@ def download_from_api(
logger(f"❌ Unexpected error during manga mode fetch: {e}")
traceback.print_exc()
break
if cancellation_event and cancellation_event.is_set(): return
if all_posts_for_manga_mode:
logger(f"MANGA_FETCH_COMPLETE:{len(all_posts_for_manga_mode)}")
if all_posts_for_manga_mode:
if processed_post_ids:
original_count = len(all_posts_for_manga_mode)
@@ -341,3 +375,4 @@ def download_from_api(
time.sleep(0.6)
if target_post_id and not processed_target_post_flag and not (cancellation_event and cancellation_event.is_set()):
logger(f"❌ Target post {target_post_id} could not be found after checking all relevant pages (final check after loop).")

241
src/core/bunkr_client.py Normal file
View File

@@ -0,0 +1,241 @@
import logging
import os
import re
import requests
import html
import time
import datetime
import urllib.parse
import json
import random
import binascii
import itertools
class MockMessage:
Directory = 1
Url = 2
Version = 3
class AlbumException(Exception): pass
class ExtractionError(AlbumException): pass
class HttpError(ExtractionError):
def __init__(self, message="", response=None):
self.response = response
self.status = response.status_code if response is not None else 0
super().__init__(message)
class ControlException(AlbumException): pass
class AbortExtraction(ExtractionError, ControlException): pass
try:
re_compile = re._compiler.compile
except AttributeError:
re_compile = re.sre_compile.compile
HTML_RE = re_compile(r"<[^>]+>")
def extr(txt, begin, end, default=""):
try:
first = txt.index(begin) + len(begin)
return txt[first:txt.index(end, first)]
except Exception: return default
def extract_iter(txt, begin, end, pos=None):
try:
index = txt.index
lbeg = len(begin)
lend = len(end)
while True:
first = index(begin, pos) + lbeg
last = index(end, first)
pos = last + lend
yield txt[first:last]
except Exception: return
def split_html(txt):
try: return [html.unescape(x).strip() for x in HTML_RE.split(txt) if x and not x.isspace()]
except TypeError: return []
def parse_datetime(date_string, format="%Y-%m-%dT%H:%M:%S%z", utcoffset=0):
try:
d = datetime.datetime.strptime(date_string, format)
o = d.utcoffset()
if o is not None: d = d.replace(tzinfo=None, microsecond=0) - o
else:
if d.microsecond: d = d.replace(microsecond=0)
if utcoffset: d += datetime.timedelta(0, utcoffset * -3600)
return d
except (TypeError, IndexError, KeyError, ValueError, OverflowError): return None
unquote = urllib.parse.unquote
unescape = html.unescape
def decrypt_xor(encrypted, key, base64=True, fromhex=False):
if base64: encrypted = binascii.a2b_base64(encrypted)
if fromhex: encrypted = bytes.fromhex(encrypted.decode())
div = len(key)
return bytes([encrypted[i] ^ key[i % div] for i in range(len(encrypted))]).decode()
def advance(iterable, num):
iterator = iter(iterable)
next(itertools.islice(iterator, num, num), None)
return iterator
def json_loads(s): return json.loads(s)
def json_dumps(obj): return json.dumps(obj, separators=(",", ":"))
class Extractor:
def __init__(self, match, logger):
self.log = logger
self.url = match.string
self.match = match
self.groups = match.groups()
self.session = requests.Session()
self.session.headers["User-Agent"] = "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Firefox/102.0"
@classmethod
def from_url(cls, url, logger):
if isinstance(cls.pattern, str): cls.pattern = re.compile(cls.pattern)
match = cls.pattern.match(url)
return cls(match, logger) if match else None
def __iter__(self): return self.items()
def items(self): yield MockMessage.Version, 1
def request(self, url, method="GET", fatal=True, **kwargs):
tries = 1
while True:
try:
response = self.session.request(method, url, **kwargs)
if response.status_code < 400: return response
msg = f"'{response.status_code} {response.reason}' for '{response.url}'"
except requests.exceptions.RequestException as exc:
msg = str(exc)
self.log.info("%s (retrying...)", msg)
if tries > 4: break
time.sleep(tries)
tries += 1
if not fatal: return None
raise HttpError(msg)
def request_json(self, url, **kwargs):
response = self.request(url, **kwargs)
try: return json_loads(response.text)
except Exception as exc:
self.log.warning("%s: %s", exc.__class__.__name__, exc)
if not kwargs.get("fatal", True): return {}
raise
BASE_PATTERN_BUNKR = r"(?:https?://)?(?:[a-zA-Z0-9-]+\.)?(bunkr\.(?:si|la|ws|red|black|media|site|is|to|ac|cr|ci|fi|pk|ps|sk|ph|su)|bunkrr\.ru)"
DOMAINS = ["bunkr.si", "bunkr.ws", "bunkr.la", "bunkr.red", "bunkr.black", "bunkr.media", "bunkr.site"]
CF_DOMAINS = set()
class BunkrAlbumExtractor(Extractor):
category = "bunkr"
root = "https://bunkr.si"
root_dl = "https://get.bunkrr.su"
root_api = "https://apidl.bunkr.ru"
pattern = re.compile(BASE_PATTERN_BUNKR + r"/a/([^/?#]+)")
def __init__(self, match, logger):
super().__init__(match, logger)
domain_match = re.search(BASE_PATTERN_BUNKR, match.string)
if domain_match:
self.root = "https://" + domain_match.group(1)
self.endpoint = self.root_api + "/api/_001_v2"
self.album_id = self.groups[-1]
def items(self):
page = self.request(self.url).text
title = unescape(unescape(extr(page, 'property="og:title" content="', '"')))
items_html = list(extract_iter(page, '<div class="grid-images_box', "</a>"))
album_data = {
"album_id": self.album_id,
"album_name": title,
"count": len(items_html),
}
yield MockMessage.Directory, album_data, {}
for item_html in items_html:
try:
webpage_url = unescape(extr(item_html, ' href="', '"'))
if webpage_url.startswith("/"):
webpage_url = self.root + webpage_url
file_data = self._extract_file(webpage_url)
info = split_html(item_html)
if not file_data.get("name"):
file_data["name"] = info[-3]
yield MockMessage.Url, file_data, {}
except Exception as exc:
self.log.error("%s: %s", exc.__class__.__name__, exc)
def _extract_file(self, webpage_url):
page = self.request(webpage_url).text
data_id = extr(page, 'data-file-id="', '"')
referer = self.root_dl + "/file/" + data_id
headers = {"Referer": referer, "Origin": self.root_dl}
data = self.request_json(self.endpoint, method="POST", headers=headers, json={"id": data_id})
file_url = decrypt_xor(data["url"], f"SECRET_KEY_{data['timestamp'] // 3600}".encode()) if data.get("encrypted") else data["url"]
file_name = extr(page, "<h1", "<").rpartition(">")[2]
return {
"url": file_url,
"name": unescape(file_name),
"_http_headers": {"Referer": referer}
}
class BunkrMediaExtractor(BunkrAlbumExtractor):
pattern = re.compile(BASE_PATTERN_BUNKR + r"(/[fvid]/[^/?#]+)")
def items(self):
try:
media_path = self.groups[-1]
file_data = self._extract_file(self.root + media_path)
album_data = {"album_name": file_data.get("name", "bunkr_media"), "count": 1}
yield MockMessage.Directory, album_data, {}
yield MockMessage.Url, file_data, {}
except Exception as exc:
self.log.error("%s: %s", exc.__class__.__name__, exc)
yield MockMessage.Directory, {"album_name": "error", "count": 0}, {}
def get_bunkr_extractor(url, logger):
"""Selects the correct Bunkr extractor based on the URL pattern."""
if BunkrAlbumExtractor.pattern.match(url):
logger.info("Bunkr Album URL detected.")
return BunkrAlbumExtractor.from_url(url, logger)
elif BunkrMediaExtractor.pattern.match(url):
logger.info("Bunkr Media URL detected.")
return BunkrMediaExtractor.from_url(url, logger)
else:
logger.error(f"No suitable Bunkr extractor found for URL: {url}")
return None
def fetch_bunkr_data(url, logger):
"""
Main function to be called from the GUI.
It extracts all file information from a Bunkr URL.
Returns:
A tuple of (album_name, list_of_files)
- album_name (str): The name of the album.
- list_of_files (list): A list of dicts, each containing 'url', 'name', and '_http_headers'.
Returns (None, None) on failure.
"""
extractor = get_bunkr_extractor(url, logger)
if not extractor:
return None, None
try:
album_name = "default_bunkr_album"
files_to_download = []
for msg_type, data, metadata in extractor:
if msg_type == MockMessage.Directory:
raw_album_name = data.get('album_name', 'untitled')
album_name = re.sub(r'[<>:"/\\|?*]', '_', raw_album_name).strip() or "untitled"
logger.info(f"Processing Bunkr album: {album_name}")
elif msg_type == MockMessage.Url:
files_to_download.append(data)
if not files_to_download:
logger.warning("No files found to download from the Bunkr URL.")
return None, None
return album_name, files_to_download
except Exception as e:
logger.error(f"An error occurred while extracting Bunkr info: {e}", exc_info=True)
return None, None

View File

@@ -0,0 +1,88 @@
import time
import cloudscraper
import json
def fetch_server_channels(server_id, logger=print, cookies_dict=None):
"""
Fetches all channels for a given Discord server ID from the API.
Uses cloudscraper to bypass Cloudflare.
"""
api_url = f"https://kemono.cr/api/v1/discord/server/{server_id}"
logger(f" Fetching channels for server: {api_url}")
scraper = cloudscraper.create_scraper()
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36',
'Referer': f'https://kemono.cr/discord/server/{server_id}',
'Accept': 'text/css'
}
try:
response = scraper.get(api_url, headers=headers, cookies=cookies_dict, timeout=30)
response.raise_for_status()
channels = response.json()
if isinstance(channels, list):
logger(f" ✅ Found {len(channels)} channels for server {server_id}.")
return channels
return None
except Exception as e:
logger(f" ❌ Error fetching server channels for {server_id}: {e}")
return None
def fetch_channel_messages(channel_id, logger=print, cancellation_event=None, pause_event=None, cookies_dict=None):
"""
A generator that fetches all messages for a specific Discord channel, handling pagination.
Uses cloudscraper and proper headers to bypass server protection.
"""
scraper = cloudscraper.create_scraper()
base_url = f"https://kemono.cr/api/v1/discord/channel/{channel_id}"
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36',
'Referer': f'https://kemono.cr/discord/channel/{channel_id}',
'Accept': 'text/css'
}
offset = 0
page_size = 150
while True:
if cancellation_event and cancellation_event.is_set():
logger(" Discord message fetching cancelled.")
break
if pause_event and pause_event.is_set():
logger(" Discord message fetching paused...")
while pause_event.is_set():
if cancellation_event and cancellation_event.is_set():
break
time.sleep(0.5)
if not (cancellation_event and cancellation_event.is_set()):
logger(" Discord message fetching resumed.")
paginated_url = f"{base_url}?o={offset}"
logger(f" Fetching messages from API: page starting at offset {offset}")
try:
response = scraper.get(paginated_url, headers=headers, cookies=cookies_dict, timeout=30)
response.raise_for_status()
messages_batch = response.json()
if not messages_batch:
logger(f" ✅ Reached end of messages for channel {channel_id}.")
break
logger(f" Fetched {len(messages_batch)} messages...")
yield messages_batch
if len(messages_batch) < page_size:
logger(f" ✅ Last page of messages received for channel {channel_id}.")
break
offset += page_size
time.sleep(0.5) # Be respectful to the API
except (cloudscraper.exceptions.CloudflareException, json.JSONDecodeError) as e:
logger(f" ❌ Error fetching messages at offset {offset}: {e}")
break
except Exception as e:
logger(f" ❌ An unexpected error occurred while fetching messages: {e}")
break

131
src/core/erome_client.py Normal file
View File

@@ -0,0 +1,131 @@
import os
import re
import html
import time
import urllib.parse
import requests
from datetime import datetime
import cloudscraper
def extr(txt, begin, end, default=""):
"""Stripped-down version of 'extract()' to find text between two delimiters."""
try:
first = txt.index(begin) + len(begin)
return txt[first:txt.index(end, first)]
except (ValueError, IndexError):
return default
def extract_iter(txt, begin, end):
"""Yields all occurrences of text between two delimiters."""
try:
index = txt.index
lbeg = len(begin)
lend = len(end)
pos = 0
while True:
first = index(begin, pos) + lbeg
last = index(end, first)
pos = last + lend
yield txt[first:last]
except (ValueError, IndexError):
return
def nameext_from_url(url):
"""Extracts filename and extension from a URL."""
data = {}
filename = urllib.parse.unquote(url.partition("?")[0].rpartition("/")[2])
name, _, ext = filename.rpartition(".")
if name and len(ext) <= 16:
data["filename"], data["extension"] = name, ext.lower()
else:
data["filename"], data["extension"] = filename, ""
return data
def parse_timestamp(ts, default=None):
"""Creates a datetime object from a Unix timestamp."""
try:
return datetime.fromtimestamp(int(ts))
except (ValueError, TypeError):
return default
def fetch_erome_data(url, logger):
"""
Identifies and extracts all media files from an Erome album URL.
Args:
url (str): The Erome album URL (e.g., https://www.erome.com/a/albumID).
logger (function): A function to log progress messages.
Returns:
tuple: A tuple containing (album_folder_name, list_of_file_dicts).
Returns (None, []) if data extraction fails.
"""
album_id_match = re.search(r"/a/(\w+)", url)
if not album_id_match:
logger(f"Error: The URL '{url}' does not appear to be a valid Erome album link.")
return None, []
album_id = album_id_match.group(1)
page_url = f"https://www.erome.com/a/{album_id}"
session = cloudscraper.create_scraper()
try:
logger(f" Fetching Erome album page: {page_url}")
for attempt in range(5):
response = session.get(page_url, timeout=30)
response.raise_for_status()
page_content = response.text
if "<title>Please wait a few moments</title>" in page_content:
logger(f" Cloudflare check detected. Waiting 5 seconds... (Attempt {attempt + 1}/5)")
time.sleep(5)
continue
break
else:
logger(" Error: Could not bypass Cloudflare check after several attempts.")
return None, []
title = html.unescape(extr(page_content, 'property="og:title" content="', '"'))
user = urllib.parse.unquote(extr(page_content, 'href="https://www.erome.com/', '"', default="unknown_user"))
sanitized_title = re.sub(r'[<>:"/\\|?*]', '_', title).strip()
sanitized_user = re.sub(r'[<>:"/\\|?*]', '_', user).strip()
album_folder_name = f"Erome - {sanitized_user} - {sanitized_title} [{album_id}]"
urls = []
media_groups = page_content.split('<div class="media-group"')
for group in media_groups[1:]:
video_url = extr(group, '<source src="', '"') or extr(group, 'data-src="', '"')
if video_url:
urls.append(video_url)
if not urls:
logger(" Warning: No media URLs found on the album page.")
return album_folder_name, []
logger(f" Found {len(urls)} media files in album '{title}'.")
file_list = []
for i, file_url in enumerate(urls, 1):
filename_info = nameext_from_url(file_url)
filename = f"{album_id}_{sanitized_title}_{i:03d}.{filename_info.get('extension', 'mp4')}"
file_data = {
"url": file_url,
"filename": filename,
"headers": {"Referer": page_url},
}
file_list.append(file_data)
return album_folder_name, file_list
except requests.exceptions.RequestException as e:
logger(f" Error fetching Erome page: {e}")
return None, []
except Exception as e:
logger(f" An unexpected error occurred during Erome extraction: {e}")
return None, []

View File

@@ -5,11 +5,10 @@ import json
import traceback
from concurrent.futures import ThreadPoolExecutor, as_completed, Future
from .api_client import download_from_api
from .workers import PostProcessorWorker, DownloadThread
from .workers import PostProcessorWorker
from ..config.constants import (
STYLE_DATE_BASED, STYLE_POST_TITLE_GLOBAL_NUMBERING,
MAX_THREADS, POST_WORKER_BATCH_THRESHOLD, POST_WORKER_NUM_BATCHES,
POST_WORKER_BATCH_DELAY_SECONDS
MAX_THREADS
)
from ..utils.file_utils import clean_folder_name
@@ -41,6 +40,10 @@ class DownloadManager:
self.total_downloads = 0
self.total_skips = 0
self.all_kept_original_filenames = []
self.creator_profiles_dir = None
self.current_creator_name_for_profile = None
self.current_creator_profile_path = None
self.session_file_path = None
def _log(self, message):
"""Puts a progress message into the queue for the UI."""
@@ -58,6 +61,17 @@ class DownloadManager:
if self.is_running:
self._log("❌ Cannot start a new session: A session is already in progress.")
return
self.session_file_path = config.get('session_file_path')
creator_profile_data = self._setup_creator_profile(config)
# Save settings to profile at the start of the session
if self.current_creator_profile_path:
creator_profile_data['settings'] = config
creator_profile_data.setdefault('processed_post_ids', [])
self._save_creator_profile(creator_profile_data)
self._log(f"✅ Loaded/created profile for '{self.current_creator_name_for_profile}'. Settings saved.")
self.is_running = True
self.cancellation_event.clear()
self.pause_event.clear()
@@ -67,76 +81,109 @@ class DownloadManager:
self.total_downloads = 0
self.total_skips = 0
self.all_kept_original_filenames = []
is_single_post = bool(config.get('target_post_id_from_initial_url'))
use_multithreading = config.get('use_multithreading', True)
is_manga_sequential = config.get('manga_mode_active') and config.get('manga_filename_style') in [STYLE_DATE_BASED, STYLE_POST_TITLE_GLOBAL_NUMBERING]
should_use_multithreading_for_posts = use_multithreading and not is_single_post and not is_manga_sequential
if should_use_multithreading_for_posts:
fetcher_thread = threading.Thread(
target=self._fetch_and_queue_posts_for_pool,
args=(config, restore_data),
args=(config, restore_data, creator_profile_data),
daemon=True
)
fetcher_thread.start()
else:
self._start_single_threaded_session(config)
# Single-threaded mode does not use the manager's complex logic
self._log(" Manager is handing off to a single-threaded worker...")
# The single-threaded worker will manage its own lifecycle and signals.
# The manager's role for this session is effectively over.
self.is_running = False # Allow another session to start if needed
self.progress_queue.put({'type': 'handoff_to_single_thread', 'payload': (config,)})
def _start_single_threaded_session(self, config):
"""Handles downloads that are best processed by a single worker thread."""
self._log(" Initializing single-threaded download process...")
self.worker_thread = threading.Thread(
target=self._run_single_worker,
args=(config,),
daemon=True
)
self.worker_thread.start()
def _run_single_worker(self, config):
"""Target function for the single-worker thread."""
try:
worker = DownloadThread(config, self.progress_queue)
worker.run() # This is the main blocking call for this thread
except Exception as e:
self._log(f"❌ CRITICAL ERROR in single-worker thread: {e}")
self._log(traceback.format_exc())
finally:
self.is_running = False
def _fetch_and_queue_posts_for_pool(self, config, restore_data):
def _fetch_and_queue_posts_for_pool(self, config, restore_data, creator_profile_data):
"""
Fetches all posts from the API and submits them as tasks to a thread pool.
This method runs in its own dedicated thread to avoid blocking.
Fetches posts from the API in batches and submits them as tasks to a thread pool.
This method runs in its own dedicated thread to avoid blocking the UI.
It provides immediate feedback as soon as the first batch of posts is found.
"""
try:
num_workers = min(config.get('num_threads', 4), MAX_THREADS)
self.thread_pool = ThreadPoolExecutor(max_workers=num_workers, thread_name_prefix='PostWorker_')
if restore_data:
session_processed_ids = set(restore_data.get('processed_post_ids', [])) if restore_data else set()
profile_processed_ids = set(creator_profile_data.get('processed_post_ids', []))
processed_ids = session_processed_ids.union(profile_processed_ids)
if restore_data and 'all_posts_data' in restore_data:
# This logic for session restore remains as it relies on a pre-fetched list
all_posts = restore_data['all_posts_data']
processed_ids = set(restore_data['processed_post_ids'])
posts_to_process = [p for p in all_posts if p.get('id') not in processed_ids]
self.total_posts = len(all_posts)
self.processed_posts = len(processed_ids)
self._log(f"🔄 Restoring session. {len(posts_to_process)} posts remaining.")
self.progress_queue.put({'type': 'overall_progress', 'payload': (self.total_posts, self.processed_posts)})
if not posts_to_process:
self._log("✅ No new posts to process from restored session.")
return
for post_data in posts_to_process:
if self.cancellation_event.is_set(): break
worker = PostProcessorWorker(post_data, config, self.progress_queue)
future = self.thread_pool.submit(worker.process)
future.add_done_callback(self._handle_future_result)
self.active_futures.append(future)
else:
posts_to_process = self._get_all_posts(config)
self.total_posts = len(posts_to_process)
# --- START: REFACTORED STREAMING LOGIC ---
post_generator = download_from_api(
api_url_input=config['api_url'],
logger=self._log,
start_page=config.get('start_page'),
end_page=config.get('end_page'),
manga_mode=config.get('manga_mode_active', False),
cancellation_event=self.cancellation_event,
pause_event=self.pause_event,
use_cookie=config.get('use_cookie', False),
cookie_text=config.get('cookie_text', ''),
selected_cookie_file=config.get('selected_cookie_file'),
app_base_dir=config.get('app_base_dir'),
manga_filename_style_for_sort_check=config.get('manga_filename_style'),
processed_post_ids=list(processed_ids)
)
self.total_posts = 0
self.processed_posts = 0
self.progress_queue.put({'type': 'overall_progress', 'payload': (self.total_posts, self.processed_posts)})
if not posts_to_process:
self._log("✅ No new posts to process.")
return
for post_data in posts_to_process:
if self.cancellation_event.is_set():
break
worker = PostProcessorWorker(post_data, config, self.progress_queue)
future = self.thread_pool.submit(worker.process)
future.add_done_callback(self._handle_future_result)
self.active_futures.append(future)
# Process posts in batches as they are yielded by the API client
for batch in post_generator:
if self.cancellation_event.is_set():
self._log(" Post fetching cancelled.")
break
# Filter out any posts that might have been processed since the start
posts_in_batch_to_process = [p for p in batch if p.get('id') not in processed_ids]
if not posts_in_batch_to_process:
continue
# Update total count and immediately inform the UI
self.total_posts += len(posts_in_batch_to_process)
self.progress_queue.put({'type': 'overall_progress', 'payload': (self.total_posts, self.processed_posts)})
for post_data in posts_in_batch_to_process:
if self.cancellation_event.is_set(): break
worker = PostProcessorWorker(post_data, config, self.progress_queue)
future = self.thread_pool.submit(worker.process)
future.add_done_callback(self._handle_future_result)
self.active_futures.append(future)
if self.total_posts == 0 and not self.cancellation_event.is_set():
self._log("✅ No new posts found to process.")
except Exception as e:
self._log(f"❌ CRITICAL ERROR in post fetcher thread: {e}")
self._log(traceback.format_exc())
@@ -144,33 +191,11 @@ class DownloadManager:
if self.thread_pool:
self.thread_pool.shutdown(wait=True)
self.is_running = False
self._log("🏁 All processing tasks have completed.")
self._log("🏁 All processing tasks have completed or been cancelled.")
self.progress_queue.put({
'type': 'finished',
'payload': (self.total_downloads, self.total_skips, self.cancellation_event.is_set(), self.all_kept_original_filenames)
})
def _get_all_posts(self, config):
"""Helper to fetch all posts using the API client."""
all_posts = []
post_generator = download_from_api(
api_url_input=config['api_url'],
logger=self._log,
start_page=config.get('start_page'),
end_page=config.get('end_page'),
manga_mode=config.get('manga_mode_active', False),
cancellation_event=self.cancellation_event,
pause_event=self.pause_event,
use_cookie=config.get('use_cookie', False),
cookie_text=config.get('cookie_text', ''),
selected_cookie_file=config.get('selected_cookie_file'),
app_base_dir=config.get('app_base_dir'),
manga_filename_style_for_sort_check=config.get('manga_filename_style'),
processed_post_ids=config.get('processed_post_ids', [])
)
for batch in post_generator:
all_posts.extend(batch)
return all_posts
def _handle_future_result(self, future: Future):
"""Callback executed when a worker task completes."""
@@ -196,19 +221,65 @@ class DownloadManager:
self.progress_queue.put({'type': 'permanent_failure', 'payload': (permanent,)})
if history:
self.progress_queue.put({'type': 'post_processed_history', 'payload': (history,)})
post_id = history.get('post_id')
if post_id and self.current_creator_profile_path:
profile_data = self._setup_creator_profile({'creator_name_for_profile': self.current_creator_name_for_profile, 'session_file_path': self.session_file_path})
if post_id not in profile_data.get('processed_post_ids', []):
profile_data.setdefault('processed_post_ids', []).append(post_id)
self._save_creator_profile(profile_data)
except Exception as e:
self._log(f"❌ Worker task resulted in an exception: {e}")
self.total_skips += 1 # Count errored posts as skipped
self.progress_queue.put({'type': 'overall_progress', 'payload': (self.total_posts, self.processed_posts)})
def _setup_creator_profile(self, config):
"""Prepares the path and loads data for the current creator's profile."""
self.current_creator_name_for_profile = config.get('creator_name_for_profile')
if not self.current_creator_name_for_profile:
self._log("⚠️ Cannot create creator profile: Name not provided in config.")
return {}
appdata_dir = os.path.dirname(config.get('session_file_path', '.'))
self.creator_profiles_dir = os.path.join(appdata_dir, "creator_profiles")
os.makedirs(self.creator_profiles_dir, exist_ok=True)
safe_filename = clean_folder_name(self.current_creator_name_for_profile) + ".json"
self.current_creator_profile_path = os.path.join(self.creator_profiles_dir, safe_filename)
if os.path.exists(self.current_creator_profile_path):
try:
with open(self.current_creator_profile_path, 'r', encoding='utf-8') as f:
return json.load(f)
except (json.JSONDecodeError, OSError) as e:
self._log(f"❌ Error loading creator profile '{safe_filename}': {e}. Starting fresh.")
return {}
def _save_creator_profile(self, data):
"""Saves the provided data to the current creator's profile file."""
if not self.current_creator_profile_path:
return
try:
temp_path = self.current_creator_profile_path + ".tmp"
with open(temp_path, 'w', encoding='utf-8') as f:
json.dump(data, f, indent=2)
os.replace(temp_path, self.current_creator_profile_path)
except OSError as e:
self._log(f"❌ Error saving creator profile to '{self.current_creator_profile_path}': {e}")
def cancel_session(self):
"""Cancels the current running session."""
if not self.is_running:
return
if self.cancellation_event.is_set():
self._log(" Cancellation already in progress.")
return
self._log("⚠️ Cancellation requested by user...")
self.cancellation_event.set()
if self.thread_pool:
self.thread_pool.shutdown(wait=False, cancel_futures=True)
self.is_running = False
self._log(" Signaling all worker threads to stop and shutting down pool...")
self.thread_pool.shutdown(wait=False)

View File

@@ -0,0 +1,44 @@
import requests
import cloudscraper
import json
def fetch_nhentai_gallery(gallery_id, logger=print):
"""
Fetches the metadata for a single nhentai gallery using cloudscraper to bypass Cloudflare.
Args:
gallery_id (str or int): The ID of the nhentai gallery.
logger (function): A function to log progress and error messages.
Returns:
dict: A dictionary containing the gallery's metadata if successful, otherwise None.
"""
api_url = f"https://nhentai.net/api/gallery/{gallery_id}"
scraper = cloudscraper.create_scraper()
logger(f" Fetching nhentai gallery metadata from: {api_url}")
try:
# Use the scraper to make the GET request
response = scraper.get(api_url, timeout=20)
if response.status_code == 404:
logger(f" ❌ Gallery not found (404): ID {gallery_id}")
return None
response.raise_for_status()
gallery_data = response.json()
if "id" in gallery_data and "media_id" in gallery_data and "images" in gallery_data:
logger(f" ✅ Successfully fetched metadata for '{gallery_data['title']['english']}'")
gallery_data['pages'] = gallery_data.pop('images')['pages']
return gallery_data
else:
logger(" ❌ API response is missing essential keys (id, media_id, or images).")
return None
except Exception as e:
logger(f" ❌ An error occurred while fetching gallery {gallery_id}: {e}")
return None

163
src/core/saint2_client.py Normal file
View File

@@ -0,0 +1,163 @@
import os
import re as re_module
import html
import urllib.parse
import requests
PATTERN_CACHE = {}
def re(pattern):
"""Compile a regular expression pattern and cache it."""
try:
return PATTERN_CACHE[pattern]
except KeyError:
p = PATTERN_CACHE[pattern] = re_module.compile(pattern)
return p
def extract_from(txt, pos=None, default=""):
"""Returns a function that extracts text between two delimiters from 'txt'."""
def extr(begin, end, index=txt.find, txt=txt):
nonlocal pos
try:
start_pos = pos if pos is not None else 0
first = index(begin, start_pos) + len(begin)
last = index(end, first)
if pos is not None:
pos = last + len(end)
return txt[first:last]
except (ValueError, IndexError):
return default
return extr
def nameext_from_url(url):
"""Extract filename and extension from a URL."""
data = {}
filename = urllib.parse.unquote(url.partition("?")[0].rpartition("/")[2])
name, _, ext = filename.rpartition(".")
if name and len(ext) <= 16:
data["filename"], data["extension"] = name, ext.lower()
else:
data["filename"], data["extension"] = filename, ""
return data
class BaseExtractor:
"""A simplified base class for extractors."""
def __init__(self, match, session, logger):
self.match = match
self.groups = match.groups()
self.session = session
self.log = logger
def request(self, url, **kwargs):
"""Makes an HTTP request using the session."""
try:
response = self.session.get(url, **kwargs)
response.raise_for_status()
return response
except requests.exceptions.RequestException as e:
self.log(f"Error making request to {url}: {e}")
return None
class SaintAlbumExtractor(BaseExtractor):
"""Extractor for saint.su albums."""
root = "https://saint2.su"
pattern = re(r"(?:https?://)?saint\d*\.(?:su|pk|cr|to)/a/([^/?#]+)")
def items(self):
"""Generator that yields all files from an album."""
album_id = self.groups[0]
response = self.request(f"{self.root}/a/{album_id}")
if not response:
return None, []
extr = extract_from(response.text)
title = extr("<title>", "<").rpartition(" - ")[0]
self.log(f"Downloading album: {title}")
files_html = re_module.findall(r'<a class="image".*?</a>', response.text, re_module.DOTALL)
file_list = []
for i, file_html in enumerate(files_html, 1):
file_extr = extract_from(file_html)
file_url = html.unescape(file_extr("onclick=\"play('", "'"))
if not file_url:
continue
filename_info = nameext_from_url(file_url)
filename = f"{filename_info['filename']}.{filename_info['extension']}"
file_data = {
"url": file_url,
"filename": filename,
"headers": {"Referer": response.url},
}
file_list.append(file_data)
return title, file_list
class SaintMediaExtractor(BaseExtractor):
"""Extractor for single saint.su media links."""
root = "https://saint2.su"
pattern = re(r"(?:https?://)?saint\d*\.(?:su|pk|cr|to)(/(embe)?d/([^/?#]+))")
def items(self):
"""Generator that yields the single file from a media page."""
path, embed, media_id = self.groups
url = self.root + path
response = self.request(url)
if not response:
return None, []
extr = extract_from(response.text)
file_url = ""
title = extr("<title>", "<").rpartition(" - ")[0] or media_id
if embed: # /embed/ link
file_url = html.unescape(extr('<source src="', '"'))
else: # /d/ link
file_url = html.unescape(extr('<a href="', '"'))
if not file_url:
self.log("Could not find video URL on the page.")
return title, []
filename_info = nameext_from_url(file_url)
filename = f"{filename_info['filename'] or media_id}.{filename_info['extension'] or 'mp4'}"
file_data = {
"url": file_url,
"filename": filename,
"headers": {"Referer": response.url}
}
return title, [file_data]
def fetch_saint2_data(url, logger):
"""
Identifies the correct extractor for a saint2.su URL and returns the data.
Args:
url (str): The saint2.su URL.
logger (function): A function to log progress messages.
Returns:
tuple: A tuple containing (album_title, list_of_file_dicts).
Returns (None, []) if no data could be fetched.
"""
extractors = [SaintMediaExtractor, SaintAlbumExtractor]
session = requests.Session()
session.headers.update({
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36'
})
for extractor_cls in extractors:
match = extractor_cls.pattern.match(url)
if match:
extractor = extractor_cls(match, session, logger)
album_title, files = extractor.items()
sanitized_title = re_module.sub(r'[<>:"/\\|?*]', '_', album_title) if album_title else "saint2_download"
return sanitized_title, files
logger(f"Error: The URL '{url}' does not match a known saint2 pattern.")
return None, []

View File

@@ -1,4 +1,5 @@
import os
import sys
import queue
import re
import threading
@@ -14,6 +15,8 @@ from concurrent.futures import ThreadPoolExecutor, as_completed, CancelledError,
from io import BytesIO
from urllib .parse import urlparse
import requests
import cloudscraper
try:
from PIL import Image
except ImportError:
@@ -36,7 +39,7 @@ try:
except ImportError:
Document = None
from PyQt5 .QtCore import Qt ,QThread ,pyqtSignal ,QMutex ,QMutexLocker ,QObject ,QTimer ,QSettings ,QStandardPaths ,QCoreApplication ,QUrl ,QSize ,QProcess
from .api_client import download_from_api, fetch_post_comments
from .api_client import download_from_api, fetch_post_comments, fetch_single_post_data
from ..services.multipart_downloader import download_file_in_parts, MULTIPART_DOWNLOADER_AVAILABLE
from ..services.drive_downloader import (
download_mega_file, download_gdrive_file, download_dropbox_file
@@ -53,6 +56,19 @@ from ..utils.text_utils import (
)
from ..config.constants import *
def robust_clean_name(name):
"""A more robust function to remove illegal characters for filenames and folders."""
if not name:
return ""
illegal_chars_pattern = r'[\x00-\x1f<>:"/\\|?*\']'
cleaned_name = re.sub(illegal_chars_pattern, '', name)
cleaned_name = cleaned_name.strip(' .')
if not cleaned_name:
return "untitled_folder"
return cleaned_name
class PostProcessorSignals (QObject ):
progress_signal =pyqtSignal (str )
file_download_status_signal =pyqtSignal (bool )
@@ -63,7 +79,6 @@ class PostProcessorSignals (QObject ):
worker_finished_signal = pyqtSignal(tuple)
class PostProcessorWorker:
def __init__(self, post_data, download_root, known_names,
filter_character_list, emitter,
unwanted_keywords, filter_mode, skip_zip,
@@ -103,7 +118,11 @@ class PostProcessorWorker:
text_export_format='txt',
single_pdf_mode=False,
project_root_dir=None,
processed_post_ids=None
processed_post_ids=None,
multipart_scope='both',
multipart_parts_count=4,
multipart_min_size_mb=100,
skip_file_size_mb=None
):
self.post = post_data
self.download_root = download_root
@@ -165,7 +184,10 @@ class PostProcessorWorker:
self.single_pdf_mode = single_pdf_mode
self.project_root_dir = project_root_dir
self.processed_post_ids = processed_post_ids if processed_post_ids is not None else []
self.multipart_scope = multipart_scope
self.multipart_parts_count = multipart_parts_count
self.multipart_min_size_mb = multipart_min_size_mb
self.skip_file_size_mb = skip_file_size_mb
if self.compress_images and Image is None:
self.logger("⚠️ Image compression disabled: Pillow library not found.")
self.compress_images = False
@@ -199,8 +221,38 @@ class PostProcessorWorker:
if self .dynamic_filter_holder :
return self .dynamic_filter_holder .get_filters ()
return self .filter_character_list_objects_initial
def _download_single_file(self, file_info, target_folder_path, headers, original_post_id_for_log, skip_event,
def _find_valid_subdomain(self, url: str, max_subdomains: int = 4) -> str:
"""
Attempts to find a working subdomain for a Kemono/Coomer URL that returned a 403 error.
Returns the original URL if no other valid subdomain is found.
"""
self.logger(f" probing for a valid subdomain...")
parsed_url = urlparse(url)
original_domain = parsed_url.netloc
for i in range(1, max_subdomains + 1):
domain_parts = original_domain.split('.')
if len(domain_parts) > 1:
base_domain = ".".join(domain_parts[-2:])
new_domain = f"n{i}.{base_domain}"
else:
continue
new_url = parsed_url._replace(netloc=new_domain).geturl()
try:
with requests.head(new_url, headers={'User-Agent': 'Mozilla/5.0'}, timeout=5, allow_redirects=True) as resp:
if resp.status_code == 200:
self.logger(f" ✅ Valid subdomain found: {new_domain}")
return new_url
except requests.RequestException:
continue
self.logger(f" ⚠️ No other valid subdomain found. Sticking with the original.")
return url
def _download_single_file(self, file_info, target_folder_path, post_page_url, original_post_id_for_log, skip_event,
post_title="", file_index_in_post=0, num_files_in_this_post=1,
manga_date_file_counter_ref=None,
forced_filename_override=None,
@@ -214,11 +266,36 @@ class PostProcessorWorker:
if self.check_cancel() or (skip_event and skip_event.is_set()):
return 0, 1, "", False, FILE_DOWNLOAD_STATUS_SKIPPED, None
file_download_headers = {
'User-Agent': 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)',
'Referer': post_page_url,
'Accept': 'text/css'
}
file_url = file_info.get('url')
cookies_to_use_for_file = None
if self.use_cookie:
cookies_to_use_for_file = prepare_cookies_for_request(self.use_cookie, self.cookie_text, self.selected_cookie_file, self.app_base_dir, self.logger)
if self.skip_file_size_mb is not None:
api_original_filename_for_size_check = file_info.get('_original_name_for_log', file_info.get('name'))
try:
# Use a stream=True HEAD request to get headers without downloading the body
with requests.head(file_url, headers=file_download_headers, timeout=15, cookies=cookies_to_use_for_file, allow_redirects=True) as head_response:
head_response.raise_for_status()
content_length = head_response.headers.get('Content-Length')
if content_length:
file_size_bytes = int(content_length)
file_size_mb = file_size_bytes / (1024 * 1024)
if file_size_mb < self.skip_file_size_mb:
self.logger(f" -> Skip File (Size): '{api_original_filename_for_size_check}' is {file_size_mb:.2f} MB, which is smaller than the {self.skip_file_size_mb} MB limit.")
return 0, 1, api_original_filename_for_size_check, False, FILE_DOWNLOAD_STATUS_SKIPPED, None
else:
self.logger(f" ⚠️ Could not determine file size for '{api_original_filename_for_size_check}' to check against size limit. Proceeding with download.")
except requests.RequestException as e:
self.logger(f" ⚠️ Could not fetch file headers to check size for '{api_original_filename_for_size_check}': {e}. Proceeding with download.")
api_original_filename = file_info.get('_original_name_for_log', file_info.get('name'))
filename_to_save_in_main_path = ""
if forced_filename_override:
@@ -232,23 +309,28 @@ class PostProcessorWorker:
self.logger(f" -> Skip File (Keyword in Original Name '{skip_word}'): '{api_original_filename}'. Scope: {self.skip_words_scope}")
return 0, 1, api_original_filename, False, FILE_DOWNLOAD_STATUS_SKIPPED, None
cleaned_original_api_filename = clean_filename(api_original_filename)
cleaned_original_api_filename = robust_clean_name(api_original_filename)
original_filename_cleaned_base, original_ext = os.path.splitext(cleaned_original_api_filename)
if not original_ext.startswith('.'): original_ext = '.' + original_ext if original_ext else ''
if self.manga_mode_active:
if self.manga_filename_style == STYLE_ORIGINAL_NAME:
filename_to_save_in_main_path = cleaned_original_api_filename
if self.manga_date_prefix and self.manga_date_prefix.strip():
cleaned_prefix = clean_filename(self.manga_date_prefix.strip())
if cleaned_prefix:
filename_to_save_in_main_path = f"{cleaned_prefix} {filename_to_save_in_main_path}"
else:
self.logger(f"⚠️ Manga Original Name Mode: Provided prefix '{self.manga_date_prefix}' was empty after cleaning. Using original name only.")
published_date_str = self.post.get('published')
added_date_str = self.post.get('added')
formatted_date_str = "nodate"
date_to_use_str = published_date_str or added_date_str
if date_to_use_str:
try:
formatted_date_str = date_to_use_str.split('T')[0]
except Exception:
self.logger(f" ⚠️ Could not parse date '{date_to_use_str}'. Using 'nodate' prefix.")
else:
self.logger(f" ⚠️ Post ID {original_post_id_for_log} has no date. Using 'nodate' prefix.")
filename_to_save_in_main_path = f"{formatted_date_str}_{cleaned_original_api_filename}"
was_original_name_kept_flag = True
elif self.manga_filename_style == STYLE_POST_TITLE:
if post_title and post_title.strip():
cleaned_post_title_base = clean_filename(post_title.strip())
cleaned_post_title_base = robust_clean_name(post_title.strip())
if num_files_in_this_post > 1:
if file_index_in_post == 0:
filename_to_save_in_main_path = f"{cleaned_post_title_base}{original_ext}"
@@ -269,7 +351,7 @@ class PostProcessorWorker:
manga_date_file_counter_ref[0] += 1
base_numbered_name = f"{counter_val_for_filename:03d}"
if self.manga_date_prefix and self.manga_date_prefix.strip():
cleaned_prefix = clean_filename(self.manga_date_prefix.strip())
cleaned_prefix = robust_clean_name(self.manga_date_prefix.strip())
if cleaned_prefix:
filename_to_save_in_main_path = f"{cleaned_prefix} {base_numbered_name}{original_ext}"
else:
@@ -286,7 +368,7 @@ class PostProcessorWorker:
with counter_lock:
counter_val_for_filename = manga_global_file_counter_ref[0]
manga_global_file_counter_ref[0] += 1
cleaned_post_title_base_for_global = clean_filename(post_title.strip() if post_title and post_title.strip() else "post")
cleaned_post_title_base_for_global = robust_clean_name(post_title.strip() if post_title and post_title.strip() else "post")
filename_to_save_in_main_path = f"{cleaned_post_title_base_for_global}_{counter_val_for_filename:03d}{original_ext}"
else:
self.logger(f"⚠️ Manga Title+GlobalNum Mode: Counter ref not provided or malformed for '{api_original_filename}'. Using original. Ref: {manga_global_file_counter_ref}")
@@ -318,8 +400,8 @@ class PostProcessorWorker:
self.logger(f" ⚠️ Post ID {original_post_id_for_log} missing both 'published' and 'added' dates for STYLE_DATE_POST_TITLE. Using 'nodate'.")
if post_title and post_title.strip():
temp_cleaned_title = clean_filename(post_title.strip())
if not temp_cleaned_title or temp_cleaned_title.startswith("untitled_file"):
temp_cleaned_title = robust_clean_name(post_title.strip())
if not temp_cleaned_title or temp_cleaned_title.startswith("untitled_folder"):
self.logger(f"⚠️ Manga mode (Date+PostTitle Style): Post title for post {original_post_id_for_log} ('{post_title}') was empty or generic after cleaning. Using 'post' as title part.")
cleaned_post_title_for_filename = "post"
else:
@@ -346,8 +428,26 @@ class PostProcessorWorker:
self.logger(f"⚠️ Manga mode: Generated filename was empty. Using generic fallback: '{filename_to_save_in_main_path}'.")
was_original_name_kept_flag = False
else:
filename_to_save_in_main_path = cleaned_original_api_filename
was_original_name_kept_flag = True
is_url_like = 'http' in api_original_filename.lower()
is_too_long = len(cleaned_original_api_filename) > 100
if is_url_like or is_too_long:
self.logger(f" ⚠️ Original filename is a URL or too long. Generating a shorter name.")
name_hash = hashlib.md5(api_original_filename.encode()).hexdigest()[:12]
_, ext = os.path.splitext(cleaned_original_api_filename)
if not ext:
try:
path = urlparse(api_original_filename).path
ext = os.path.splitext(path)[1] or ".file"
except Exception:
ext = ".file"
cleaned_post_title = robust_clean_name(post_title.strip() if post_title else "post")[:40]
filename_to_save_in_main_path = f"{cleaned_post_title}_{name_hash}{ext}"
was_original_name_kept_flag = False
else:
filename_to_save_in_main_path = cleaned_original_api_filename
was_original_name_kept_flag = True
if self.remove_from_filename_words_list and filename_to_save_in_main_path:
base_name_for_removal, ext_for_removal = os.path.splitext(filename_to_save_in_main_path)
@@ -397,43 +497,109 @@ class PostProcessorWorker:
temp_file_base_for_unique_part, temp_file_ext_for_unique_part = os.path.splitext(filename_to_save_in_main_path if filename_to_save_in_main_path else api_original_filename)
unique_id_for_part_file = uuid.uuid4().hex[:8]
unique_part_file_stem_on_disk = f"{temp_file_base_for_unique_part}_{unique_id_for_part_file}"
max_retries = 3
if not self.keep_in_post_duplicates:
final_save_path_check = os.path.join(target_folder_path, filename_to_save_in_main_path)
if os.path.exists(final_save_path_check):
try:
with requests.head(file_url, headers=file_download_headers, timeout=15, cookies=cookies_to_use_for_file, allow_redirects=True) as head_response:
head_response.raise_for_status()
expected_size = int(head_response.headers.get('Content-Length', -1))
actual_size = os.path.getsize(final_save_path_check)
if expected_size != -1 and actual_size == expected_size:
self.logger(f" -> Skip (File Exists & Complete): '{filename_to_save_in_main_path}' is already on disk with the correct size.")
try:
md5_hasher = hashlib.md5()
with open(final_save_path_check, 'rb') as f_verify:
for chunk in iter(lambda: f_verify.read(8192), b""):
md5_hasher.update(chunk)
with self.downloaded_hash_counts_lock:
self.downloaded_hash_counts[md5_hasher.hexdigest()] += 1
except Exception as hash_exc:
self.logger(f" ⚠️ Could not hash existing file '{filename_to_save_in_main_path}' for session: {hash_exc}")
return 0, 1, filename_to_save_in_main_path, was_original_name_kept_flag, FILE_DOWNLOAD_STATUS_SKIPPED, None
else:
self.logger(f" ⚠️ File '{filename_to_save_in_main_path}' exists but is incomplete (Expected: {expected_size}, Actual: {actual_size}). Re-downloading.")
except requests.RequestException as e:
self.logger(f" ⚠️ Could not verify size of existing file '{filename_to_save_in_main_path}': {e}. Proceeding with download.")
max_retries = 3
retry_delay = 5
downloaded_size_bytes = 0
calculated_file_hash = None
downloaded_part_file_path = None
total_size_bytes = 0
download_successful_flag = False
last_exception_for_retry_later = None
is_permanent_error = False
data_to_write_io = None
response_for_this_attempt = None
for attempt_num_single_stream in range(max_retries + 1):
response_for_this_attempt = None
response = None
if self._check_pause(f"File download attempt for '{api_original_filename}'"): break
if self.check_cancel() or (skip_event and skip_event.is_set()): break
try:
if attempt_num_single_stream > 0:
self.logger(f" Retrying download for '{api_original_filename}' (Overall Attempt {attempt_num_single_stream + 1}/{max_retries + 1})...")
time.sleep(retry_delay * (2 ** (attempt_num_single_stream - 1)))
self._emit_signal('file_download_status', True)
response = requests.get(file_url, headers=headers, timeout=(15, 300), stream=True, cookies=cookies_to_use_for_file)
current_url_to_try = file_url
response = requests.get(current_url_to_try, headers=file_download_headers, timeout=(30, 300), stream=True, cookies=cookies_to_use_for_file)
if response.status_code == 403 and ('kemono.cr' in current_url_to_try or 'coomer.st' in current_url_to_try):
self.logger(f" ⚠️ Got 403 Forbidden for '{api_original_filename}'. Attempting subdomain rotation...")
new_url = self._find_valid_subdomain(current_url_to_try)
if new_url != current_url_to_try:
self.logger(f" Retrying with new URL: {new_url}")
file_url = new_url
response.close() # Close the old response
response = requests.get(new_url, headers=file_download_headers, timeout=(30, 300), stream=True, cookies=cookies_to_use_for_file)
response.raise_for_status()
# --- REVISED AND MOVED SIZE CHECK LOGIC ---
total_size_bytes = int(response.headers.get('Content-Length', 0))
num_parts_for_file = min(self.num_file_threads, MAX_PARTS_FOR_MULTIPART_DOWNLOAD)
if self.skip_file_size_mb is not None:
if total_size_bytes > 0:
file_size_mb = total_size_bytes / (1024 * 1024)
if file_size_mb < self.skip_file_size_mb:
self.logger(f" -> Skip File (Size): '{api_original_filename}' is {file_size_mb:.2f} MB, which is smaller than the {self.skip_file_size_mb} MB limit.")
return 0, 1, api_original_filename, False, FILE_DOWNLOAD_STATUS_SKIPPED, None
# If Content-Length is missing, we can't check, so we no longer log a warning here and just proceed.
# --- END OF REVISED LOGIC ---
num_parts_for_file = min(self.multipart_parts_count, MAX_PARTS_FOR_MULTIPART_DOWNLOAD)
file_is_eligible_by_scope = False
if self.multipart_scope == 'videos':
if is_video(api_original_filename):
file_is_eligible_by_scope = True
elif self.multipart_scope == 'archives':
if is_archive(api_original_filename):
file_is_eligible_by_scope = True
elif self.multipart_scope == 'both':
if is_video(api_original_filename) or is_archive(api_original_filename):
file_is_eligible_by_scope = True
min_size_in_bytes = self.multipart_min_size_mb * 1024 * 1024
attempt_multipart = (self.allow_multipart_download and MULTIPART_DOWNLOADER_AVAILABLE and
num_parts_for_file > 1 and total_size_bytes > MIN_SIZE_FOR_MULTIPART_DOWNLOAD and
file_is_eligible_by_scope and
num_parts_for_file > 1 and total_size_bytes > min_size_in_bytes and
'bytes' in response.headers.get('Accept-Ranges', '').lower())
if self._check_pause(f"Multipart decision for '{api_original_filename}'"): break
if attempt_multipart:
if response_for_this_attempt:
response_for_this_attempt.close()
response_for_this_attempt = None
response.close() # Close the initial connection before starting multipart
mp_save_path_for_unique_part_stem_arg = os.path.join(target_folder_path, f"{unique_part_file_stem_on_disk}{temp_file_ext_for_unique_part}")
mp_success, mp_bytes, mp_hash, mp_file_handle = download_file_in_parts(
file_url, mp_save_path_for_unique_part_stem_arg, total_size_bytes, num_parts_for_file, headers, api_original_filename,
file_url, mp_save_path_for_unique_part_stem_arg, total_size_bytes, num_parts_for_file, file_download_headers, api_original_filename,
emitter_for_multipart=self.emitter, cookies_for_chunk_session=cookies_to_use_for_file,
cancellation_event=self.cancellation_event, skip_event=skip_event, logger_func=self.logger,
pause_event=self.pause_event
@@ -442,7 +608,7 @@ class PostProcessorWorker:
download_successful_flag = True
downloaded_size_bytes = mp_bytes
calculated_file_hash = mp_hash
downloaded_part_file_path = mp_save_path_for_unique_part_stem_arg + ".part"
downloaded_part_file_path = mp_save_path_for_unique_part_stem_arg
if mp_file_handle: mp_file_handle.close()
break
else:
@@ -456,7 +622,6 @@ class PostProcessorWorker:
current_attempt_downloaded_bytes = 0
md5_hasher = hashlib.md5()
last_progress_time = time.time()
single_stream_exception = None
try:
with open(current_single_stream_part_path, 'wb') as f_part:
for chunk in response.iter_content(chunk_size=1 * 1024 * 1024):
@@ -508,20 +673,23 @@ class PostProcessorWorker:
if isinstance(e, requests.exceptions.ConnectionError) and ("Failed to resolve" in str(e) or "NameResolutionError" in str(e)):
self.logger(" 💡 This looks like a DNS resolution problem. Please check your internet connection, DNS settings, or VPN.")
except requests.exceptions.RequestException as e:
self.logger(f" ❌ Download Error (Non-Retryable): {api_original_filename}. Error: {e}")
last_exception_for_retry_later = e
is_permanent_error = True
if ("Failed to resolve" in str(e) or "NameResolutionError" in str(e)):
self.logger(" 💡 This looks like a DNS resolution problem. Please check your internet connection, DNS settings, or VPN.")
break
if e.response is not None and e.response.status_code == 403:
self.logger(f" ⚠️ Download Error (403 Forbidden): {api_original_filename}. This often requires valid cookies.")
self.logger(f" Will retry... Check your 'Use Cookie' settings if this persists.")
last_exception_for_retry_later = e
else:
self.logger(f" ❌ Download Error (Non-Retryable): {api_original_filename}. Error: {e}")
last_exception_for_retry_later = e
is_permanent_error = True
break
except Exception as e:
self.logger(f" ❌ Unexpected Download Error: {api_original_filename}: {e}\n{traceback.format_exc(limit=2)}")
last_exception_for_retry_later = e
is_permanent_error = True
break
finally:
if response_for_this_attempt:
response_for_this_attempt.close()
if response:
response.close()
self._emit_signal('file_download_status', False)
final_total_for_progress = total_size_bytes if download_successful_flag and total_size_bytes > 0 else downloaded_size_bytes
@@ -582,7 +750,30 @@ class PostProcessorWorker:
os.remove(downloaded_part_file_path)
except OSError: pass
return 0, 1, filename_to_save_in_main_path, was_original_name_kept_flag, FILE_DOWNLOAD_STATUS_SKIPPED, None
if (self.compress_images and downloaded_part_file_path and
is_image(api_original_filename) and
os.path.getsize(downloaded_part_file_path) > 1.5 * 1024 * 1024):
self.logger(f" 🔄 Compressing '{api_original_filename}' to WebP...")
try:
with Image.open(downloaded_part_file_path) as img:
if img.mode not in ('RGB', 'RGBA'):
img = img.convert('RGBA')
output_buffer = BytesIO()
img.save(output_buffer, format='WebP', quality=85)
data_to_write_io = output_buffer
base, _ = os.path.splitext(filename_to_save_in_main_path)
filename_to_save_in_main_path = f"{base}.webp"
self.logger(f" ✅ Compression successful. New size: {len(data_to_write_io.getvalue()) / (1024*1024):.2f} MB")
except Exception as e_compress:
self.logger(f" ⚠️ Failed to compress '{api_original_filename}': {e_compress}. Saving original file instead.")
data_to_write_io = None
effective_save_folder = target_folder_path
base_name, extension = os.path.splitext(filename_to_save_in_main_path)
counter = 1
@@ -600,7 +791,6 @@ class PostProcessorWorker:
try:
if data_to_write_io:
with open(final_save_path, 'wb') as f_out:
time.sleep(0.05)
f_out.write(data_to_write_io.getvalue())
if downloaded_part_file_path and os.path.exists(downloaded_part_file_path):
try:
@@ -654,10 +844,12 @@ class PostProcessorWorker:
self.logger(f" -> Failed to remove partially saved file: {final_save_path}")
permanent_failure_details = {
'file_info': file_info, 'target_folder_path': target_folder_path, 'headers': headers,
'file_info': file_info, 'target_folder_path': target_folder_path, 'headers': file_download_headers,
'original_post_id_for_log': original_post_id_for_log, 'post_title': post_title,
'file_index_in_post': file_index_in_post, 'num_files_in_this_post': num_files_in_this_post,
'forced_filename_override': filename_to_save_in_main_path,
'service': self.service,
'user_id': self.user_id
}
return 0, 1, final_filename_saved_for_return, was_original_name_kept_flag, FILE_DOWNLOAD_STATUS_FAILED_PERMANENTLY_THIS_SESSION, permanent_failure_details
finally:
@@ -668,7 +860,7 @@ class PostProcessorWorker:
details_for_failure = {
'file_info': file_info,
'target_folder_path': target_folder_path,
'headers': headers,
'headers': file_download_headers,
'original_post_id_for_log': original_post_id_for_log,
'post_title': post_title,
'file_index_in_post': file_index_in_post,
@@ -680,45 +872,116 @@ class PostProcessorWorker:
else:
return 0, 1, filename_to_save_in_main_path, was_original_name_kept_flag, FILE_DOWNLOAD_STATUS_FAILED_RETRYABLE_LATER, details_for_failure
def process(self):
if self.service == 'discord':
post_title = self.post.get('content', '') or f"Message {self.post.get('id', 'N/A')}"
post_id = self.post.get('id', 'unknown_id')
post_main_file_info = {}
post_attachments = self.post.get('attachments', [])
post_content_html = self.post.get('content', '')
post_data = self.post
log_prefix = "Message"
else:
post_title = self.post.get('title', '') or 'untitled_post'
post_id = self.post.get('id', 'unknown_id')
post_main_file_info = self.post.get('file')
post_attachments = self.post.get('attachments', [])
post_content_html = self.post.get('content', '')
post_data = self.post
log_prefix = "Post"
content_is_needed = (
self.show_external_links or
self.extract_links_only or
self.scan_content_for_images or
(self.filter_mode == 'text_only' and self.text_only_scope == 'content')
)
if content_is_needed and self.post.get('content') is None and self.service != 'discord':
self.logger(f" Post {post_id} is missing 'content' field, fetching full data...")
parsed_url = urlparse(self.api_url_input)
api_domain = parsed_url.netloc
creator_page_url = f"https://{api_domain}/{self.service}/user/{self.user_id}"
headers = {
'User-Agent': 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)',
'Referer': creator_page_url,
'Accept': 'text/css'
}
cookies = prepare_cookies_for_request(self.use_cookie, self.cookie_text, self.selected_cookie_file, self.app_base_dir, self.logger, target_domain=api_domain)
full_post_data = fetch_single_post_data(api_domain, self.service, self.user_id, post_id, headers, self.logger, cookies_dict=cookies)
if full_post_data:
self.logger(" ✅ Full post data fetched successfully.")
self.post = full_post_data
post_title = self.post.get('title', '') or 'untitled_post'
post_main_file_info = self.post.get('file')
post_attachments = self.post.get('attachments', [])
post_content_html = self.post.get('content', '')
post_data = self.post
else:
self.logger(f" ⚠️ Failed to fetch full content for post {post_id}. Content-dependent features may not work for this post.")
result_tuple = (0, 0, [], [], [], None, None)
total_downloaded_this_post = 0
total_skipped_this_post = 0
determined_post_save_path_for_history = self.override_output_dir if self.override_output_dir else self.download_root
try:
if self._check_pause(f"Post processing for ID {self.post.get('id', 'N/A')}"):
result_tuple = (0, 0, [], [], [], None, None)
return result_tuple
if self._check_pause(f"{log_prefix} processing for ID {post_id}"):
return (0, 0, [], [], [], None, None)
if self.check_cancel():
result_tuple = (0, 0, [], [], [], None, None)
return result_tuple
return (0, 0, [], [], [], None, None)
current_character_filters = self._get_current_character_filters()
kept_original_filenames_for_log = []
retryable_failures_this_post = []
permanent_failures_this_post = []
total_downloaded_this_post = 0
total_skipped_this_post = 0
history_data_for_this_post = None
parsed_api_url = urlparse(self.api_url_input)
referer_url = f"https://{parsed_api_url.netloc}/"
headers = {'User-Agent': 'Mozilla/5.0', 'Referer': referer_url, 'Accept': '*/*'}
link_pattern = re.compile(r"""<a\s+.*?href=["'](https?://[^"']+)["'][^>]*>(.*?)</a>""", re.IGNORECASE | re.DOTALL)
post_data = self.post
post_title = post_data.get('title', '') or 'untitled_post'
post_id = post_data.get('id', 'unknown_id')
post_main_file_info = post_data.get('file')
post_attachments = post_data.get('attachments', [])
# CONTEXT-AWARE URL for Referer Header
if self.service == 'discord':
server_id = self.user_id
channel_id = self.post.get('channel', 'unknown_channel')
post_page_url = f"https://{parsed_api_url.netloc}/discord/server/{server_id}/{channel_id}"
else:
post_page_url = f"https://{parsed_api_url.netloc}/{self.service}/user/{self.user_id}/post/{post_id}"
headers = {
'User-Agent': 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)',
'Referer': post_page_url,
'Accept': 'text/css'
}
link_pattern = re.compile(r"""<a\s+.*?href=["'](https?://[^"']+)["'][^>]*>(.*?)</a>""", re.IGNORECASE | re.DOTALL)
effective_unwanted_keywords_for_folder_naming = self.unwanted_keywords.copy()
is_full_creator_download_no_char_filter = not self.target_post_id_from_initial_url and not current_character_filters
if (self.show_external_links or self.extract_links_only):
embed_data = post_data.get('embed')
if isinstance(embed_data, dict) and embed_data.get('url'):
embed_url = embed_data['url']
embed_subject = embed_data.get('subject', embed_url) # Use subject as link text, fallback to URL
platform = get_link_platform(embed_url)
self.logger(f" 🔗 Found embed link: {embed_url}")
self._emit_signal('external_link', post_title, embed_subject, embed_url, platform, "")
if is_full_creator_download_no_char_filter and self.creator_download_folder_ignore_words:
self.logger(f" Applying creator download specific folder ignore words ({len(self.creator_download_folder_ignore_words)} words).")
effective_unwanted_keywords_for_folder_naming.update(self.creator_download_folder_ignore_words)
post_content_html = post_data.get('content', '')
if not self.extract_links_only:
self.logger(f"\n--- Processing Post {post_id} ('{post_title[:50]}...') (Thread: {threading.current_thread().name}) ---")
self.logger(f"\n--- Processing {log_prefix} {post_id} ('{post_title[:50]}...') (Thread: {threading.current_thread().name}) ---")
num_potential_files_in_post = len(post_attachments or []) + (1 if post_main_file_info and post_main_file_info.get('path') else 0)
post_is_candidate_by_title_char_match = False
@@ -750,8 +1013,8 @@ class PostProcessorWorker:
all_files_from_post_api_for_char_check = []
api_file_domain_for_char_check = urlparse(self.api_url_input).netloc
if not api_file_domain_for_char_check or not any(d in api_file_domain_for_char_check.lower() for d in ['kemono.su', 'kemono.party', 'coomer.su', 'coomer.party']):
api_file_domain_for_char_check = "kemono.su" if "kemono" in self.service.lower() else "coomer.party"
if not api_file_domain_for_char_check or not any(d in api_file_domain_for_char_check.lower() for d in ['kemono.su', 'kemono.party', 'kemono.cr', 'coomer.su', 'coomer.party', 'coomer.st']):
api_file_domain_for_char_check = "kemono.cr" if "kemono" in self.service.lower() else "coomer.st"
if post_main_file_info and isinstance(post_main_file_info, dict) and post_main_file_info.get('path'):
original_api_name = post_main_file_info.get('name') or os.path.basename(post_main_file_info['path'].lstrip('/'))
if original_api_name:
@@ -762,7 +1025,7 @@ class PostProcessorWorker:
if original_api_att_name:
all_files_from_post_api_for_char_check.append({'_original_name_for_log': original_api_att_name})
if current_character_filters and self.char_filter_scope == CHAR_SCOPE_COMMENTS:
if current_character_filters and self.char_filter_scope == CHAR_SCOPE_COMMENTS and self.service != 'discord':
self.logger(f" [Char Scope: Comments] Phase 1: Checking post files for matches before comments for post ID '{post_id}'.")
if self._check_pause(f"File check (comments scope) for post {post_id}"):
result_tuple = (0, num_potential_files_in_post, [], [], [], None, None)
@@ -785,7 +1048,7 @@ class PostProcessorWorker:
if post_is_candidate_by_file_char_match_in_comment_scope: break
self.logger(f" [Char Scope: Comments] Phase 1 Result: post_is_candidate_by_file_char_match_in_comment_scope = {post_is_candidate_by_file_char_match_in_comment_scope}")
if current_character_filters and self.char_filter_scope == CHAR_SCOPE_COMMENTS:
if current_character_filters and self.char_filter_scope == CHAR_SCOPE_COMMENTS and self.service != 'discord':
if not post_is_candidate_by_file_char_match_in_comment_scope:
if self._check_pause(f"Comment check for post {post_id}"):
result_tuple = (0, num_potential_files_in_post, [], [], [], None, None)
@@ -794,9 +1057,9 @@ class PostProcessorWorker:
try:
parsed_input_url_for_comments = urlparse(self.api_url_input)
api_domain_for_comments = parsed_input_url_for_comments.netloc
if not any(d in api_domain_for_comments.lower() for d in ['kemono.su', 'kemono.party', 'coomer.su', 'coomer.party']):
if not any(d in api_domain_for_comments.lower() for d in ['kemono.su', 'kemono.party', 'kemono.cr', 'coomer.su', 'coomer.party', 'coomer.st']):
self.logger(f"⚠️ Unrecognized domain '{api_domain_for_comments}' for comment API. Defaulting based on service.")
api_domain_for_comments = "kemono.su" if "kemono" in self.service.lower() else "coomer.party"
api_domain_for_comments = "kemono.cr" if "kemono" in self.service.lower() else "coomer.st"
comments_data = fetch_post_comments(
api_domain_for_comments, self.service, self.user_id, post_id,
headers, self.logger, self.cancellation_event, self.pause_event,
@@ -848,27 +1111,17 @@ class PostProcessorWorker:
result_tuple = (0, num_potential_files_in_post, [], [], [], None, None)
return result_tuple
if self.skip_words_list and (self.skip_words_scope == SKIP_SCOPE_POSTS or self.skip_words_scope == SKIP_SCOPE_BOTH):
if self._check_pause(f"Skip words (post title) for post {post_id}"):
result_tuple = (0, num_potential_files_in_post, [], [], [], None, None)
return result_tuple
post_title_lower = post_title.lower()
for skip_word in self.skip_words_list:
if skip_word.lower() in post_title_lower:
self.logger(f" -> Skip Post (Keyword in Title '{skip_word}'): '{post_title[:50]}...'. Scope: {self.skip_words_scope}")
result_tuple = (0, num_potential_files_in_post, [], [], [], None, None)
return result_tuple
if not self.extract_links_only and self.manga_mode_active and current_character_filters and (self.char_filter_scope == CHAR_SCOPE_TITLE or self.char_filter_scope == CHAR_SCOPE_BOTH) and not post_is_candidate_by_title_char_match:
self.logger(f" -> Skip Post (Manga Mode with Title/Both Scope - No Title Char Match): Title '{post_title[:50]}' doesn't match filters.")
self._emit_signal('missed_character_post', post_title, "Manga Mode: No title match for character filter (Title/Both scope)")
result_tuple = (0, num_potential_files_in_post, [], [], [], None, None)
return result_tuple
self.logger(f" -> Skip Post (Manga Mode with Title/Both Scope - No Title Char Match): Title '{post_title[:50]}' doesn't match filters.")
self._emit_signal('missed_character_post', post_title, "Manga Mode: No title match for character filter (Title/Both scope)")
result_tuple = (0, num_potential_files_in_post, [], [], [], None, None)
return result_tuple
if not isinstance(post_attachments, list):
self.logger(f"⚠️ Corrupt attachment data for post {post_id} (expected list, got {type(post_attachments)}). Skipping attachments.")
post_attachments = []
# CORRECTED LOGIC: Determine folder path BEFORE skip checks
base_folder_names_for_post_content = []
determined_post_save_path_for_history = self.override_output_dir if self.override_output_dir else self.download_root
if not self.extract_links_only and self.use_subfolders:
@@ -967,7 +1220,10 @@ class PostProcessorWorker:
determined_post_save_path_for_history = os.path.join(determined_post_save_path_for_history, base_folder_names_for_post_content[0])
if not self.extract_links_only and self.use_post_subfolders:
cleaned_post_title_for_sub = clean_folder_name(post_title)
cleaned_post_title_for_sub = robust_clean_name(post_title)
max_folder_len = 100
if len(cleaned_post_title_for_sub) > max_folder_len:
cleaned_post_title_for_sub = cleaned_post_title_for_sub[:max_folder_len].strip()
post_id_for_fallback = self.post.get('id', 'unknown_id')
if not cleaned_post_title_for_sub or cleaned_post_title_for_sub == "untitled_folder":
@@ -992,31 +1248,74 @@ class PostProcessorWorker:
suffix_counter = 0
final_post_subfolder_name = ""
while True:
suffix_counter = 0
folder_creation_successful = False
final_post_subfolder_name = ""
post_id_for_folder = str(self.post.get('id', 'unknown_id'))
while not folder_creation_successful:
if suffix_counter == 0:
name_candidate = original_cleaned_post_title_for_sub
else:
name_candidate = f"{original_cleaned_post_title_for_sub}_{suffix_counter}"
potential_post_subfolder_path = os.path.join(base_path_for_post_subfolder, name_candidate)
try:
os.makedirs(potential_post_subfolder_path, exist_ok=False)
final_post_subfolder_name = name_candidate
if suffix_counter > 0:
self.logger(f" Post subfolder name conflict: Using '{final_post_subfolder_name}' instead of '{original_cleaned_post_title_for_sub}' to avoid mixing posts.")
break
except FileExistsError:
suffix_counter += 1
if suffix_counter > 100:
self.logger(f" ⚠️ Exceeded 100 attempts to find unique subfolder name for '{original_cleaned_post_title_for_sub}'. Using UUID.")
final_post_subfolder_name = f"{original_cleaned_post_title_for_sub}_{uuid.uuid4().hex[:8]}"
os.makedirs(os.path.join(base_path_for_post_subfolder, final_post_subfolder_name), exist_ok=True)
id_file_path = os.path.join(potential_post_subfolder_path, f".postid_{post_id_for_folder}")
if not os.path.isdir(potential_post_subfolder_path):
# Folder does not exist, create it and its ID file
try:
os.makedirs(potential_post_subfolder_path)
with open(id_file_path, 'w') as f:
f.write(post_id_for_folder)
final_post_subfolder_name = name_candidate
folder_creation_successful = True
if suffix_counter > 0:
self.logger(f" Post subfolder name conflict: Using '{final_post_subfolder_name}' to avoid mixing posts.")
except OSError as e_mkdir:
self.logger(f" ❌ Error creating directory '{potential_post_subfolder_path}': {e_mkdir}.")
final_post_subfolder_name = original_cleaned_post_title_for_sub
break
except OSError as e_mkdir:
self.logger(f" ❌ Error creating directory '{potential_post_subfolder_path}': {e_mkdir}. Files for this post might be saved in parent or fail.")
final_post_subfolder_name = original_cleaned_post_title_for_sub
break
else:
# Folder exists, check if it's for this post or a different one
if os.path.exists(id_file_path):
# ID file matches! This is a restore scenario. Reuse the folder.
self.logger(f" Re-using existing post subfolder: '{name_candidate}'")
final_post_subfolder_name = name_candidate
folder_creation_successful = True
else:
# Folder exists but ID file does not match (or is missing). This is a normal name collision.
suffix_counter += 1
if suffix_counter > 100: # Safety break
self.logger(f" ⚠️ Exceeded 100 attempts to find unique subfolder for '{original_cleaned_post_title_for_sub}'.")
final_post_subfolder_name = f"{original_cleaned_post_title_for_sub}_{uuid.uuid4().hex[:8]}"
os.makedirs(os.path.join(base_path_for_post_subfolder, final_post_subfolder_name), exist_ok=True)
break
determined_post_save_path_for_history = os.path.join(base_path_for_post_subfolder, final_post_subfolder_name)
if self.skip_words_list and (self.skip_words_scope == SKIP_SCOPE_POSTS or self.skip_words_scope == SKIP_SCOPE_BOTH):
if self._check_pause(f"Skip words (post title) for post {post_id}"):
result_tuple = (0, num_potential_files_in_post, [], [], [], None, None)
return result_tuple
post_title_lower = post_title.lower()
for skip_word in self.skip_words_list:
if skip_word.lower() in post_title_lower:
self.logger(f" -> Skip Post (Keyword in Title '{skip_word}'): '{post_title[:50]}...'. Scope: {self.skip_words_scope}")
# Create a history object for the skipped post to record its ID
history_data_for_skipped_post = {
'post_id': post_id,
'service': self.service,
'user_id': self.user_id,
'post_title': post_title,
'top_file_name': "N/A (Post Skipped)",
'num_files': num_potential_files_in_post,
'upload_date_str': post_data.get('published') or post_data.get('added') or "Unknown",
'download_location': determined_post_save_path_for_history
}
result_tuple = (0, num_potential_files_in_post, [], [], [], history_data_for_skipped_post, None)
return result_tuple
if self.filter_mode == 'text_only' and not self.extract_links_only:
self.logger(f" Mode: Text Only (Scope: {self.text_only_scope})")
post_title_lower = post_title.lower()
@@ -1041,7 +1340,6 @@ class PostProcessorWorker:
parsed_url = urlparse(self.api_url_input)
api_domain = parsed_url.netloc
cookies = prepare_cookies_for_request(self.use_cookie, self.cookie_text, self.selected_cookie_file, self.app_base_dir, self.logger, target_domain=api_domain)
from .api_client import fetch_single_post_data
full_data = fetch_single_post_data(api_domain, self.service, self.user_id, post_id, headers, self.logger, cookies_dict=cookies)
if full_data:
final_post_data = full_data
@@ -1124,11 +1422,18 @@ class PostProcessorWorker:
if FPDF:
self.logger(f" Creating formatted PDF for {'comments' if self.text_only_scope == 'comments' else 'content'}...")
pdf = PDF()
if getattr(sys, 'frozen', False) and hasattr(sys, '_MEIPASS'):
# If the application is run as a bundled exe, _MEIPASS is the temp folder
base_path = sys._MEIPASS
else:
# If running as a normal .py script, use the project_root_dir
base_path = self.project_root_dir
font_path = ""
bold_font_path = ""
if self.project_root_dir:
font_path = os.path.join(self.project_root_dir, 'data', 'dejavu-sans', 'DejaVuSans.ttf')
bold_font_path = os.path.join(self.project_root_dir, 'data', 'dejavu-sans', 'DejaVuSans-Bold.ttf')
if base_path:
font_path = os.path.join(base_path, 'data', 'dejavu-sans', 'DejaVuSans.ttf')
bold_font_path = os.path.join(base_path, 'data', 'dejavu-sans', 'DejaVuSans-Bold.ttf')
try:
if not os.path.exists(font_path): raise RuntimeError(f"Font file not found: {font_path}")
@@ -1261,9 +1566,8 @@ class PostProcessorWorker:
all_files_from_post_api = []
api_file_domain = urlparse(self.api_url_input).netloc
if not api_file_domain or not any(d in api_file_domain.lower() for d in ['kemono.su', 'kemono.party', 'coomer.su', 'coomer.party']):
api_file_domain = "kemono.su" if "kemono" in self.service.lower() else "coomer.party"
if not api_file_domain or not any(d in api_file_domain.lower() for d in ['kemono.su', 'kemono.party', 'kemono.cr', 'coomer.su', 'coomer.party', 'coomer.st']):
api_file_domain = "kemono.cr" if "kemono" in self.service.lower() else "coomer.st"
if post_main_file_info and isinstance(post_main_file_info, dict) and post_main_file_info.get('path'):
file_path = post_main_file_info['path'].lstrip('/')
original_api_name = post_main_file_info.get('name') or os.path.basename(file_path)
@@ -1385,7 +1689,17 @@ class PostProcessorWorker:
if not all_files_from_post_api:
self.logger(f" No files found to download for post {post_id}.")
result_tuple = (0, 0, [], [], [], None, None)
history_data_for_no_files_post = {
'post_title': post_title,
'post_id': post_id,
'service': self.service,
'user_id': self.user_id,
'top_file_name': "N/A (No Files)",
'num_files': 0,
'upload_date_str': post_data.get('published') or post_data.get('added') or "Unknown",
'download_location': determined_post_save_path_for_history
}
result_tuple = (0, 0, [], [], [], history_data_for_no_files_post, None)
return result_tuple
files_to_download_info_list = []
@@ -1511,7 +1825,7 @@ class PostProcessorWorker:
self._download_single_file,
file_info=file_info_to_dl,
target_folder_path=current_path_for_file_instance,
headers=headers, original_post_id_for_log=post_id, skip_event=self.skip_current_file_flag,
post_page_url=post_page_url, original_post_id_for_log=post_id, skip_event=self.skip_current_file_flag,
post_title=post_title, manga_date_file_counter_ref=manga_date_counter_to_pass,
manga_global_file_counter_ref=manga_global_counter_to_pass, folder_context_name_for_history=folder_context_for_file,
file_index_in_post=file_idx, num_files_in_this_post=len(files_to_download_info_list)
@@ -1605,10 +1919,12 @@ class PostProcessorWorker:
if not self.extract_links_only and self.use_post_subfolders and total_downloaded_this_post == 0:
path_to_check_for_emptiness = determined_post_save_path_for_history
try:
# Check if the path is a directory and if it's empty
if os.path.isdir(path_to_check_for_emptiness) and not os.listdir(path_to_check_for_emptiness):
self.logger(f" 🗑️ Removing empty post-specific subfolder: '{path_to_check_for_emptiness}'")
os.rmdir(path_to_check_for_emptiness)
except OSError as e_rmdir:
# Log if removal fails for any reason (e.g., permissions)
self.logger(f" ⚠️ Could not remove empty post-specific subfolder '{path_to_check_for_emptiness}': {e_rmdir}")
result_tuple = (total_downloaded_this_post, total_skipped_this_post,
@@ -1616,7 +1932,25 @@ class PostProcessorWorker:
permanent_failures_this_post, history_data_for_this_post,
None)
except Exception as main_thread_err:
self.logger(f"\n❌ Critical error within Worker process for {log_prefix} {post_id}: {main_thread_err}")
self.logger(traceback.format_exc())
# Ensure we still return a valid tuple to prevent the app from stalling
result_tuple = (0, 1, [], [], [{'error': str(main_thread_err)}], None, None)
finally:
# This block ALWAYS executes, ensuring that every task signals its completion.
# This is critical for the main thread to know when all work is done.
if not self.extract_links_only and self.use_post_subfolders and total_downloaded_this_post == 0:
path_to_check_for_emptiness = determined_post_save_path_for_history
try:
# Check if the path is a directory and if it's empty
if os.path.isdir(path_to_check_for_emptiness) and not os.listdir(path_to_check_for_emptiness):
self.logger(f" 🗑️ Removing empty post-specific subfolder: '{path_to_check_for_emptiness}'")
os.rmdir(path_to_check_for_emptiness)
except OSError as e_rmdir:
# Log if removal fails for any reason (e.g., permissions)
self.logger(f" ⚠️ Could not remove potentially empty subfolder '{path_to_check_for_emptiness}': {e_rmdir}")
self._emit_signal('worker_finished', result_tuple)
return result_tuple
@@ -1657,6 +1991,8 @@ class DownloadThread(QThread):
remove_from_filename_words_list=None,
manga_date_prefix='',
allow_multipart_download=True,
multipart_parts_count=4,
multipart_min_size_mb=100,
selected_cookie_file=None,
override_output_dir=None,
app_base_dir=None,
@@ -1679,7 +2015,10 @@ class DownloadThread(QThread):
single_pdf_mode=False,
project_root_dir=None,
processed_post_ids=None,
start_offset=0):
start_offset=0,
fetch_first=False,
skip_file_size_mb=None
):
super().__init__()
self.api_url_input = api_url_input
self.output_dir = output_dir
@@ -1719,6 +2058,8 @@ class DownloadThread(QThread):
self.remove_from_filename_words_list = remove_from_filename_words_list
self.manga_date_prefix = manga_date_prefix
self.allow_multipart_download = allow_multipart_download
self.multipart_parts_count = multipart_parts_count
self.multipart_min_size_mb = multipart_min_size_mb
self.selected_cookie_file = selected_cookie_file
self.app_base_dir = app_base_dir
self.cookie_text = cookie_text
@@ -1743,6 +2084,8 @@ class DownloadThread(QThread):
self.project_root_dir = project_root_dir
self.processed_post_ids_set = set(processed_post_ids) if processed_post_ids is not None else set()
self.start_offset = start_offset
self.fetch_first = fetch_first
self.skip_file_size_mb = skip_file_size_mb
if self.compress_images and Image is None:
self.logger("⚠️ Image compression disabled: Pillow library not found (DownloadThread).")
@@ -1789,7 +2132,8 @@ class DownloadThread(QThread):
selected_cookie_file=self.selected_cookie_file,
app_base_dir=self.app_base_dir,
manga_filename_style_for_sort_check=self.manga_filename_style if self.manga_mode_active else None,
processed_post_ids=self.processed_post_ids_set
processed_post_ids=self.processed_post_ids_set,
fetch_all_first=self.fetch_first
)
for posts_batch_data in post_generator:
@@ -1860,6 +2204,9 @@ class DownloadThread(QThread):
'text_only_scope': self.text_only_scope,
'text_export_format': self.text_export_format,
'single_pdf_mode': self.single_pdf_mode,
'multipart_parts_count': self.multipart_parts_count,
'multipart_min_size_mb': self.multipart_min_size_mb,
'skip_file_size_mb': self.skip_file_size_mb,
'project_root_dir': self.project_root_dir,
}

File diff suppressed because one or more lines are too long

View File

@@ -3,131 +3,210 @@ import os
import re
import traceback
import json
import base64
import time
import zipfile
from urllib.parse import urlparse, urlunparse, parse_qs, urlencode
# --- Third-Party Library Imports ---
# --- Third-party Library Imports ---
import requests
import cloudscraper
try:
from mega import Mega
MEGA_AVAILABLE = True
from Crypto.Cipher import AES
PYCRYPTODOME_AVAILABLE = True
except ImportError:
MEGA_AVAILABLE = False
PYCRYPTODOME_AVAILABLE = False
try:
import gdown
GDOWN_AVAILABLE = True
GDRIVE_AVAILABLE = True
except ImportError:
GDOWN_AVAILABLE = False
GDRIVE_AVAILABLE = False
# --- Helper Functions ---
MEGA_API_URL = "https://g.api.mega.co.nz"
def _get_filename_from_headers(headers):
"""
Extracts a filename from the Content-Disposition header.
Args:
headers (dict): A dictionary of HTTP response headers.
Returns:
str or None: The extracted filename, or None if not found.
"""
cd = headers.get('content-disposition')
if not cd:
return None
# Handles both filename="file.zip" and filename*=UTF-8''file%20name.zip
fname_match = re.findall('filename="?([^"]+)"?', cd)
if fname_match:
# Sanitize the filename to prevent directory traversal issues
# and remove invalid characters for most filesystems.
sanitized_name = re.sub(r'[<>:"/\\|?*]', '_', fname_match[0].strip())
return sanitized_name
return None
# --- Main Service Downloader Functions ---
# --- Helper functions for Mega decryption ---
def download_mega_file(mega_link, download_path=".", logger_func=print):
"""
Downloads a file from a public Mega.nz link.
def urlb64_to_b64(s):
s = s.replace('-', '+').replace('_', '/')
s += '=' * (-len(s) % 4)
return s
Args:
mega_link (str): The public Mega.nz link to the file.
download_path (str): The directory to save the downloaded file.
logger_func (callable): Function to use for logging.
"""
if not MEGA_AVAILABLE:
logger_func("❌ Error: mega.py library is not installed. Cannot download from Mega.")
logger_func(" Please install it: pip install mega.py")
raise ImportError("mega.py library not found.")
def b64_to_bytes(s):
return base64.b64decode(urlb64_to_b64(s))
logger_func(f" [Mega] Initializing Mega client...")
def bytes_to_hex(b):
return b.hex()
def hex_to_bytes(h):
return bytes.fromhex(h)
def hrk2hk(hex_raw_key):
key_part1 = int(hex_raw_key[0:16], 16)
key_part2 = int(hex_raw_key[16:32], 16)
key_part3 = int(hex_raw_key[32:48], 16)
key_part4 = int(hex_raw_key[48:64], 16)
final_key_part1 = key_part1 ^ key_part3
final_key_part2 = key_part2 ^ key_part4
return f'{final_key_part1:016x}{final_key_part2:016x}'
def decrypt_at(at_b64, key_bytes):
at_bytes = b64_to_bytes(at_b64)
iv = b'\0' * 16
cipher = AES.new(key_bytes, AES.MODE_CBC, iv)
decrypted_at = cipher.decrypt(at_bytes)
return decrypted_at.decode('utf-8').strip('\0').replace('MEGA', '')
# --- Core Logic for Mega Downloads ---
def get_mega_file_info(file_id, file_key, session, logger_func):
try:
mega_client = Mega()
m = mega_client.login()
logger_func(f" [Mega] Attempting to download from: {mega_link}")
if not os.path.exists(download_path):
os.makedirs(download_path, exist_ok=True)
logger_func(f" [Mega] Created download directory: {download_path}")
hex_raw_key = bytes_to_hex(b64_to_bytes(file_key))
hex_key = hrk2hk(hex_raw_key)
key_bytes = hex_to_bytes(hex_key)
payload = [{"a": "g", "p": file_id}]
response = session.post(f"{MEGA_API_URL}/cs", json=payload, timeout=20)
response.raise_for_status()
res_json = response.json()
if isinstance(res_json, list) and isinstance(res_json[0], int) and res_json[0] < 0:
logger_func(f" [Mega] ❌ API Error: {res_json[0]}. The link may be invalid or removed.")
return None
file_size = res_json[0]['s']
at_b64 = res_json[0]['at']
at_dec_json_str = decrypt_at(at_b64, key_bytes)
at_dec_json = json.loads(at_dec_json_str)
file_name = at_dec_json['n']
payload = [{"a": "g", "g": 1, "p": file_id}]
response = session.post(f"{MEGA_API_URL}/cs", json=payload, timeout=20)
response.raise_for_status()
res_json = response.json()
dl_temp_url = res_json[0]['g']
return {
'file_name': file_name,
'file_size': file_size,
'dl_url': dl_temp_url,
'hex_raw_key': hex_raw_key
}
except (requests.RequestException, json.JSONDecodeError, KeyError, ValueError) as e:
logger_func(f" [Mega] ❌ Failed to get file info: {e}")
return None
def download_and_decrypt_mega_file(info, download_path, logger_func):
file_name = info['file_name']
file_size = info['file_size']
dl_url = info['dl_url']
hex_raw_key = info['hex_raw_key']
final_path = os.path.join(download_path, file_name)
if os.path.exists(final_path) and os.path.getsize(final_path) == file_size:
logger_func(f" [Mega] File '{file_name}' already exists with the correct size. Skipping.")
return
key = hex_to_bytes(hrk2hk(hex_raw_key))
iv_hex = hex_raw_key[32:48] + '0000000000000000'
iv_bytes = hex_to_bytes(iv_hex)
cipher = AES.new(key, AES.MODE_CTR, initial_value=iv_bytes, nonce=b'')
try:
with requests.get(dl_url, stream=True, timeout=(15, 300)) as r:
r.raise_for_status()
downloaded_bytes = 0
last_log_time = time.time()
# The download_url method handles file info fetching and saving internally.
downloaded_file_path = m.download_url(mega_link, dest_path=download_path)
if downloaded_file_path and os.path.exists(downloaded_file_path):
logger_func(f" [Mega] ✅ File downloaded successfully! Saved as: {downloaded_file_path}")
else:
raise Exception(f"Mega download failed or file not found. Returned: {downloaded_file_path}")
with open(final_path, 'wb') as f:
for chunk in r.iter_content(chunk_size=8192):
if not chunk: continue
decrypted_chunk = cipher.decrypt(chunk)
f.write(decrypted_chunk)
downloaded_bytes += len(chunk)
current_time = time.time()
if current_time - last_log_time > 1:
progress_percent = (downloaded_bytes / file_size) * 100 if file_size > 0 else 0
logger_func(f" [Mega] Downloading '{file_name}': {downloaded_bytes/1024/1024:.2f}MB / {file_size/1024/1024:.2f}MB ({progress_percent:.1f}%)")
last_log_time = current_time
logger_func(f" [Mega] ✅ Successfully downloaded '{file_name}' to '{download_path}'")
except Exception as e:
logger_func(f" [Mega] ❌ An unexpected error occurred during Mega download: {e}")
traceback.print_exc(limit=2)
raise # Re-raise the exception to be handled by the calling worker
logger_func(f" [Mega] ❌ An unexpected error occurred during download/decryption: {e}")
def download_gdrive_file(gdrive_link, download_path=".", logger_func=print):
"""
Downloads a file from a public Google Drive link using the gdown library.
def download_mega_file(mega_url, download_path, logger_func=print):
if not PYCRYPTODOME_AVAILABLE:
logger_func("❌ Mega download failed: 'pycryptodome' library is not installed. Please run: pip install pycryptodome")
return
Args:
gdrive_link (str): The public Google Drive link to the file.
download_path (str): The directory to save the downloaded file.
logger_func (callable): Function to use for logging.
"""
if not GDOWN_AVAILABLE:
logger_func("❌ Error: gdown library is not installed. Cannot download from Google Drive.")
logger_func(" Please install it: pip install gdown")
raise ImportError("gdown library not found.")
logger_func(f" [Mega] Initializing download for: {mega_url}")
match = re.search(r'mega(?:\.co)?\.nz/(?:file/|#!)?([a-zA-Z0-9]+)(?:#|!)([a-zA-Z0-9_.-]+)', mega_url)
if not match:
logger_func(f" [Mega] ❌ Error: Invalid Mega URL format.")
return
file_id = match.group(1)
file_key = match.group(2)
logger_func(f" [GDrive] Attempting to download: {gdrive_link}")
session = requests.Session()
session.headers.update({'User-Agent': 'Kemono-Downloader-PyQt/1.0'})
file_info = get_mega_file_info(file_id, file_key, session, logger_func)
if not file_info:
logger_func(f" [Mega] ❌ Failed to get file info. Aborting.")
return
logger_func(f" [Mega] File found: '{file_info['file_name']}' (Size: {file_info['file_size'] / 1024 / 1024:.2f} MB)")
download_and_decrypt_mega_file(file_info, download_path, logger_func)
def download_gdrive_file(url, download_path, logger_func=print):
if not GDRIVE_AVAILABLE:
logger_func("❌ Google Drive download failed: 'gdown' library is not installed.")
return
try:
if not os.path.exists(download_path):
os.makedirs(download_path, exist_ok=True)
logger_func(f" [GDrive] Created download directory: {download_path}")
# gdown handles finding the file ID and downloading. 'fuzzy=True' helps with various URL formats.
output_file_path = gdown.download(gdrive_link, output=download_path, quiet=False, fuzzy=True)
if output_file_path and os.path.exists(output_file_path):
logger_func(f" [GDrive] ✅ Google Drive file downloaded successfully: {output_file_path}")
logger_func(f" [G-Drive] Starting download for: {url}")
logger_func(" [G-Drive] Download in progress... This may take some time. Please wait.")
output_path = gdown.download(url, output=download_path, quiet=True, fuzzy=True)
if output_path and os.path.exists(output_path):
logger_func(f" [G-Drive] ✅ Successfully downloaded to '{output_path}'")
else:
raise Exception(f"gdown download failed or file not found. Returned: {output_file_path}")
logger_func(f" [G-Drive] ❌ Download failed. The file may have been moved, deleted, or is otherwise inaccessible.")
except Exception as e:
logger_func(f" [GDrive] ❌ An error occurred during Google Drive download: {e}")
traceback.print_exc(limit=2)
raise
logger_func(f" [G-Drive] ❌ An unexpected error occurred: {e}")
# --- MODIFIED DROPBOX DOWNLOADER ---
def download_dropbox_file(dropbox_link, download_path=".", logger_func=print):
"""
Downloads a file from a public Dropbox link by modifying the URL for direct download.
Args:
dropbox_link (str): The public Dropbox link to the file.
download_path (str): The directory to save the downloaded file.
logger_func (callable): Function to use for logging.
Downloads a file or a folder (as a zip) from a public Dropbox link.
Uses cloudscraper to handle potential browser checks and auto-extracts zip files.
"""
logger_func(f" [Dropbox] Attempting to download: {dropbox_link}")
# Modify the Dropbox URL to force a direct download instead of showing the preview page.
# Modify URL to force download (works for both files and folders)
parsed_url = urlparse(dropbox_link)
query_params = parse_qs(parsed_url.query)
query_params['dl'] = ['1']
@@ -136,28 +215,60 @@ def download_dropbox_file(dropbox_link, download_path=".", logger_func=print):
logger_func(f" [Dropbox] Using direct download URL: {direct_download_url}")
scraper = cloudscraper.create_scraper()
try:
if not os.path.exists(download_path):
os.makedirs(download_path, exist_ok=True)
logger_func(f" [Dropbox] Created download directory: {download_path}")
with requests.get(direct_download_url, stream=True, allow_redirects=True, timeout=(10, 300)) as r:
with scraper.get(direct_download_url, stream=True, allow_redirects=True, timeout=(20, 600)) as r:
r.raise_for_status()
# Determine filename from headers or URL
filename = _get_filename_from_headers(r.headers) or os.path.basename(parsed_url.path) or "dropbox_file"
filename = _get_filename_from_headers(r.headers) or os.path.basename(parsed_url.path) or "dropbox_download"
# If it's a folder, Dropbox will name it FolderName.zip
if not os.path.splitext(filename)[1]:
filename += ".zip"
full_save_path = os.path.join(download_path, filename)
logger_func(f" [Dropbox] Starting download of '{filename}'...")
# Write file to disk in chunks
total_size = int(r.headers.get('content-length', 0))
downloaded_bytes = 0
last_log_time = time.time()
with open(full_save_path, 'wb') as f:
for chunk in r.iter_content(chunk_size=8192):
f.write(chunk)
downloaded_bytes += len(chunk)
current_time = time.time()
if total_size > 0 and current_time - last_log_time > 1:
progress = (downloaded_bytes / total_size) * 100
logger_func(f" -> Downloading '{filename}'... {downloaded_bytes/1024/1024:.2f}MB / {total_size/1024/1024:.2f}MB ({progress:.1f}%)")
last_log_time = current_time
logger_func(f" [Dropbox] ✅ Dropbox file downloaded successfully: {full_save_path}")
logger_func(f" [Dropbox] ✅ Download complete: {full_save_path}")
# --- NEW: Auto-extraction logic ---
if zipfile.is_zipfile(full_save_path):
logger_func(f" [Dropbox] ዚ Detected zip file. Attempting to extract...")
extract_folder_name = os.path.splitext(filename)[0]
extract_path = os.path.join(download_path, extract_folder_name)
os.makedirs(extract_path, exist_ok=True)
with zipfile.ZipFile(full_save_path, 'r') as zip_ref:
zip_ref.extractall(extract_path)
logger_func(f" [Dropbox] ✅ Successfully extracted to folder: '{extract_path}'")
# Optional: remove the zip file after extraction
try:
os.remove(full_save_path)
logger_func(f" [Dropbox] 🗑️ Removed original zip file.")
except OSError as e:
logger_func(f" [Dropbox] ⚠️ Could not remove original zip file: {e}")
except Exception as e:
logger_func(f" [Dropbox] ❌ An error occurred during Dropbox download: {e}")
traceback.print_exc(limit=2)
raise
traceback.print_exc(limit=2)

View File

@@ -1,4 +1,5 @@
# --- Standard Library Imports ---
# --- Standard Library Imports ---
import os
import time
import hashlib
@@ -10,28 +11,49 @@ from concurrent.futures import ThreadPoolExecutor, as_completed
# --- Third-Party Library Imports ---
import requests
MULTIPART_DOWNLOADER_AVAILABLE = True
# --- Module Constants ---
CHUNK_DOWNLOAD_RETRY_DELAY = 2
MAX_CHUNK_DOWNLOAD_RETRIES = 1
DOWNLOAD_CHUNK_SIZE_ITER = 1024 * 256 # 256 KB per iteration chunk
# Flag to indicate if this module and its dependencies are available.
# This was missing and caused the ImportError.
MULTIPART_DOWNLOADER_AVAILABLE = True
def _download_individual_chunk(
chunk_url, temp_file_path, start_byte, end_byte, headers,
chunk_url, chunk_temp_file_path, start_byte, end_byte, headers,
part_num, total_parts, progress_data, cancellation_event,
skip_event, pause_event, global_emit_time_ref, cookies_for_chunk,
logger_func, emitter=None, api_original_filename=None
):
"""
Downloads a single segment (chunk) of a larger file. This function is
intended to be run in a separate thread by a ThreadPoolExecutor.
Downloads a single segment (chunk) of a larger file to its own unique part file.
This function is intended to be run in a separate thread by a ThreadPoolExecutor.
It handles retries, pauses, and cancellations for its specific chunk.
It handles retries, pauses, and cancellations for its specific chunk. If a
download fails, the partial chunk file is removed, allowing a clean retry later.
Args:
chunk_url (str): The URL to download the file from.
chunk_temp_file_path (str): The unique path to save this specific chunk
(e.g., 'my_video.mp4.part0').
start_byte (int): The starting byte for the Range header.
end_byte (int): The ending byte for the Range header.
headers (dict): The HTTP headers to use for the request.
part_num (int): The index of this chunk (e.g., 0 for the first part).
total_parts (int): The total number of chunks for the entire file.
progress_data (dict): A thread-safe dictionary for sharing progress.
cancellation_event (threading.Event): Event to signal cancellation.
skip_event (threading.Event): Event to signal skipping the file.
pause_event (threading.Event): Event to signal pausing the download.
global_emit_time_ref (list): A mutable list with one element (a timestamp)
to rate-limit UI updates.
cookies_for_chunk (dict): Cookies to use for the request.
logger_func (function): A function to log messages.
emitter (queue.Queue or QObject): Emitter for sending progress to the UI.
api_original_filename (str): The original filename for UI display.
Returns:
tuple: A tuple containing (bytes_downloaded, success_flag).
"""
# --- Pre-download checks for control events ---
if cancellation_event and cancellation_event.is_set():
@@ -49,103 +71,135 @@ def _download_individual_chunk(
time.sleep(0.2)
logger_func(f" [Chunk {part_num + 1}/{total_parts}] Download resumed.")
# Prepare headers for the specific byte range of this chunk
chunk_headers = headers.copy()
if end_byte != -1:
chunk_headers['Range'] = f"bytes={start_byte}-{end_byte}"
bytes_this_chunk = 0
last_speed_calc_time = time.time()
bytes_at_last_speed_calc = 0
# Set this chunk's status to 'active' before starting the download.
with progress_data['lock']:
progress_data['chunks_status'][part_num]['active'] = True
# --- Retry Loop ---
for attempt in range(MAX_CHUNK_DOWNLOAD_RETRIES + 1):
if cancellation_event and cancellation_event.is_set():
return bytes_this_chunk, False
try:
# Prepare headers for the specific byte range of this chunk
chunk_headers = headers.copy()
if end_byte != -1:
chunk_headers['Range'] = f"bytes={start_byte}-{end_byte}"
try:
if attempt > 0:
logger_func(f" [Chunk {part_num + 1}/{total_parts}] Retrying (Attempt {attempt + 1}/{MAX_CHUNK_DOWNLOAD_RETRIES + 1})...")
time.sleep(CHUNK_DOWNLOAD_RETRY_DELAY * (2 ** (attempt - 1)))
last_speed_calc_time = time.time()
bytes_at_last_speed_calc = bytes_this_chunk
bytes_this_chunk = 0
last_speed_calc_time = time.time()
bytes_at_last_speed_calc = 0
logger_func(f" 🚀 [Chunk {part_num + 1}/{total_parts}] Starting download: bytes {start_byte}-{end_byte if end_byte != -1 else 'EOF'}")
response = requests.get(chunk_url, headers=chunk_headers, timeout=(10, 120), stream=True, cookies=cookies_for_chunk)
response.raise_for_status()
# --- Retry Loop ---
for attempt in range(MAX_CHUNK_DOWNLOAD_RETRIES + 1):
if cancellation_event and cancellation_event.is_set():
return bytes_this_chunk, False
# --- Data Writing Loop ---
with open(temp_file_path, 'r+b') as f:
f.seek(start_byte)
for data_segment in response.iter_content(chunk_size=DOWNLOAD_CHUNK_SIZE_ITER):
if cancellation_event and cancellation_event.is_set():
return bytes_this_chunk, False
if pause_event and pause_event.is_set():
# Handle pausing during the download stream
logger_func(f" [Chunk {part_num + 1}/{total_parts}] Paused...")
while pause_event.is_set():
if cancellation_event and cancellation_event.is_set(): return bytes_this_chunk, False
time.sleep(0.2)
logger_func(f" [Chunk {part_num + 1}/{total_parts}] Resumed.")
try:
if attempt > 0:
logger_func(f" [Chunk {part_num + 1}/{total_parts}] Retrying (Attempt {attempt + 1}/{MAX_CHUNK_DOWNLOAD_RETRIES + 1})...")
time.sleep(CHUNK_DOWNLOAD_RETRY_DELAY * (2 ** (attempt - 1)))
last_speed_calc_time = time.time()
bytes_at_last_speed_calc = bytes_this_chunk
if data_segment:
f.write(data_segment)
bytes_this_chunk += len(data_segment)
# Update shared progress data structure
with progress_data['lock']:
progress_data['total_downloaded_so_far'] += len(data_segment)
progress_data['chunks_status'][part_num]['downloaded'] = bytes_this_chunk
# Calculate and update speed for this chunk
current_time = time.time()
time_delta = current_time - last_speed_calc_time
if time_delta > 0.5:
bytes_delta = bytes_this_chunk - bytes_at_last_speed_calc
current_speed_bps = (bytes_delta * 8) / time_delta if time_delta > 0 else 0
progress_data['chunks_status'][part_num]['speed_bps'] = current_speed_bps
last_speed_calc_time = current_time
bytes_at_last_speed_calc = bytes_this_chunk
# Emit progress signal to the UI via the queue
if emitter and (current_time - global_emit_time_ref[0] > 0.25):
global_emit_time_ref[0] = current_time
status_list_copy = [dict(s) for s in progress_data['chunks_status']]
if isinstance(emitter, queue.Queue):
emitter.put({'type': 'file_progress', 'payload': (api_original_filename, status_list_copy)})
elif hasattr(emitter, 'file_progress_signal'):
emitter.file_progress_signal.emit(api_original_filename, status_list_copy)
# If we reach here, the download for this chunk was successful
return bytes_this_chunk, True
logger_func(f" 🚀 [Chunk {part_num + 1}/{total_parts}] Starting download: bytes {start_byte}-{end_byte if end_byte != -1 else 'EOF'}")
except (requests.exceptions.ConnectionError, requests.exceptions.Timeout, http.client.IncompleteRead) as e:
logger_func(f" ❌ [Chunk {part_num + 1}/{total_parts}] Retryable error: {e}")
except requests.exceptions.RequestException as e:
logger_func(f" ❌ [Chunk {part_num + 1}/{total_parts}] Non-retryable error: {e}")
return bytes_this_chunk, False # Break loop on non-retryable errors
except Exception as e:
logger_func(f" ❌ [Chunk {part_num + 1}/{total_parts}] Unexpected error: {e}\n{traceback.format_exc(limit=1)}")
return bytes_this_chunk, False
response = requests.get(chunk_url, headers=chunk_headers, timeout=(10, 120), stream=True, cookies=cookies_for_chunk)
response.raise_for_status()
return bytes_this_chunk, False
# --- Data Writing Loop ---
# We open the unique chunk file in write-binary ('wb') mode.
# No more seeking is required.
with open(chunk_temp_file_path, 'wb') as f:
for data_segment in response.iter_content(chunk_size=DOWNLOAD_CHUNK_SIZE_ITER):
if cancellation_event and cancellation_event.is_set():
return bytes_this_chunk, False
if pause_event and pause_event.is_set():
# Handle pausing during the download stream
logger_func(f" [Chunk {part_num + 1}/{total_parts}] Paused...")
while pause_event.is_set():
if cancellation_event and cancellation_event.is_set(): return bytes_this_chunk, False
time.sleep(0.2)
logger_func(f" [Chunk {part_num + 1}/{total_parts}] Resumed.")
if data_segment:
f.write(data_segment)
bytes_this_chunk += len(data_segment)
# Update shared progress data structure
with progress_data['lock']:
progress_data['total_downloaded_so_far'] += len(data_segment)
progress_data['chunks_status'][part_num]['downloaded'] = bytes_this_chunk
# Calculate and update speed for this chunk
current_time = time.time()
time_delta = current_time - last_speed_calc_time
if time_delta > 0.5:
bytes_delta = bytes_this_chunk - bytes_at_last_speed_calc
current_speed_bps = (bytes_delta * 8) / time_delta if time_delta > 0 else 0
progress_data['chunks_status'][part_num]['speed_bps'] = current_speed_bps
last_speed_calc_time = current_time
bytes_at_last_speed_calc = bytes_this_chunk
# Emit progress signal to the UI via the queue
if emitter and (current_time - global_emit_time_ref[0] > 0.25):
global_emit_time_ref[0] = current_time
status_list_copy = [dict(s) for s in progress_data['chunks_status']]
if isinstance(emitter, queue.Queue):
emitter.put({'type': 'file_progress', 'payload': (api_original_filename, status_list_copy)})
elif hasattr(emitter, 'file_progress_signal'):
emitter.file_progress_signal.emit(api_original_filename, status_list_copy)
# If we get here, the download for this chunk is successful
return bytes_this_chunk, True
except (requests.exceptions.ConnectionError, requests.exceptions.Timeout, http.client.IncompleteRead) as e:
logger_func(f" ❌ [Chunk {part_num + 1}/{total_parts}] Retryable error: {e}")
except requests.exceptions.RequestException as e:
logger_func(f" ❌ [Chunk {part_num + 1}/{total_parts}] Non-retryable error: {e}")
return bytes_this_chunk, False # Break loop on non-retryable errors
except Exception as e:
logger_func(f" ❌ [Chunk {part_num + 1}/{total_parts}] Unexpected error: {e}\n{traceback.format_exc(limit=1)}")
return bytes_this_chunk, False
# If the retry loop finishes without a successful download
return bytes_this_chunk, False
finally:
# This block runs whether the download succeeded or failed
with progress_data['lock']:
progress_data['chunks_status'][part_num]['active'] = False
progress_data['chunks_status'][part_num]['speed_bps'] = 0.0
def download_file_in_parts(file_url, save_path, total_size, num_parts, headers, api_original_filename,
emitter_for_multipart, cookies_for_chunk_session,
cancellation_event, skip_event, logger_func, pause_event):
logger_func(f"⬇️ Initializing Multi-part Download ({num_parts} parts) for: '{api_original_filename}' (Size: {total_size / (1024*1024):.2f} MB)")
temp_file_path = save_path + ".part"
"""
Manages a resilient, multipart file download by saving each chunk to a separate file.
try:
with open(temp_file_path, 'wb') as f_temp:
if total_size > 0:
f_temp.truncate(total_size)
except IOError as e:
logger_func(f" ❌ Error creating/truncating temp file '{temp_file_path}': {e}")
return False, 0, None, None
This function orchestrates the download process by:
1. Checking for already completed chunk files to resume a previous download.
2. Submitting only the missing chunks to a thread pool for parallel download.
3. Assembling the final file from the individual chunks upon successful completion.
4. Cleaning up temporary chunk files after assembly.
5. Leaving completed chunks on disk if the download fails, allowing for a future resume.
Args:
file_url (str): The URL of the file to download.
save_path (str): The final desired path for the downloaded file (e.g., 'my_video.mp4').
total_size (int): The total size of the file in bytes.
num_parts (int): The number of parts to split the download into.
headers (dict): HTTP headers for the download requests.
api_original_filename (str): The original filename for UI progress display.
emitter_for_multipart (queue.Queue or QObject): Emitter for UI signals.
cookies_for_chunk_session (dict): Cookies for the download requests.
cancellation_event (threading.Event): Event to signal cancellation.
skip_event (threading.Event): Event to signal skipping the file.
logger_func (function): A function for logging messages.
pause_event (threading.Event): Event to signal pausing the download.
Returns:
tuple: A tuple containing (success_flag, total_bytes_downloaded, md5_hash, file_handle).
The file_handle will be for the final assembled file if successful, otherwise None.
"""
logger_func(f"⬇️ Initializing Resumable Multi-part Download ({num_parts} parts) for: '{api_original_filename}' (Size: {total_size / (1024*1024):.2f} MB)")
# Calculate the byte range for each chunk
chunk_size_calc = total_size // num_parts
chunks_ranges = []
for i in range(num_parts):
@@ -153,76 +207,119 @@ def download_file_in_parts(file_url, save_path, total_size, num_parts, headers,
end = start + chunk_size_calc - 1 if i < num_parts - 1 else total_size - 1
if start <= end:
chunks_ranges.append((start, end))
elif total_size == 0 and i == 0:
elif total_size == 0 and i == 0: # Handle zero-byte files
chunks_ranges.append((0, -1))
# Calculate the expected size of each chunk
chunk_actual_sizes = []
for start, end in chunks_ranges:
if end == -1 and start == 0:
chunk_actual_sizes.append(0)
else:
chunk_actual_sizes.append(end - start + 1)
chunk_actual_sizes.append(end - start + 1 if end != -1 else 0)
if not chunks_ranges and total_size > 0:
logger_func(f" ⚠️ No valid chunk ranges for multipart download of '{api_original_filename}'. Aborting multipart.")
if os.path.exists(temp_file_path): os.remove(temp_file_path)
logger_func(f" ⚠️ No valid chunk ranges for multipart download of '{api_original_filename}'. Aborting.")
return False, 0, None, None
# --- Resumption Logic: Check for existing complete chunks ---
chunks_to_download = []
total_bytes_resumed = 0
for i, (start, end) in enumerate(chunks_ranges):
chunk_part_path = f"{save_path}.part{i}"
expected_chunk_size = chunk_actual_sizes[i]
if os.path.exists(chunk_part_path) and os.path.getsize(chunk_part_path) == expected_chunk_size:
logger_func(f" [Chunk {i + 1}/{num_parts}] Resuming with existing complete chunk file.")
total_bytes_resumed += expected_chunk_size
else:
chunks_to_download.append({'index': i, 'start': start, 'end': end})
# Setup the shared progress data structure
progress_data = {
'total_file_size': total_size,
'total_downloaded_so_far': 0,
'chunks_status': [
{'id': i, 'downloaded': 0, 'total': chunk_actual_sizes[i] if i < len(chunk_actual_sizes) else 0, 'active': False, 'speed_bps': 0.0}
for i in range(num_parts)
],
'total_downloaded_so_far': total_bytes_resumed,
'chunks_status': [],
'lock': threading.Lock(),
'last_global_emit_time': [time.time()]
}
for i in range(num_parts):
is_resumed = not any(c['index'] == i for c in chunks_to_download)
progress_data['chunks_status'].append({
'id': i,
'downloaded': chunk_actual_sizes[i] if is_resumed else 0,
'total': chunk_actual_sizes[i],
'active': False,
'speed_bps': 0.0
})
# --- Download Phase ---
chunk_futures = []
all_chunks_successful = True
total_bytes_from_chunks = 0
total_bytes_from_threads = 0
with ThreadPoolExecutor(max_workers=num_parts, thread_name_prefix=f"MPChunk_{api_original_filename[:10]}_") as chunk_pool:
for i, (start, end) in enumerate(chunks_ranges):
if cancellation_event and cancellation_event.is_set(): all_chunks_successful = False; break
chunk_futures.append(chunk_pool.submit(
_download_individual_chunk, chunk_url=file_url, temp_file_path=temp_file_path,
for chunk_info in chunks_to_download:
if cancellation_event and cancellation_event.is_set():
all_chunks_successful = False
break
i, start, end = chunk_info['index'], chunk_info['start'], chunk_info['end']
chunk_part_path = f"{save_path}.part{i}"
future = chunk_pool.submit(
_download_individual_chunk,
chunk_url=file_url,
chunk_temp_file_path=chunk_part_path,
start_byte=start, end_byte=end, headers=headers, part_num=i, total_parts=num_parts,
progress_data=progress_data, cancellation_event=cancellation_event, skip_event=skip_event, global_emit_time_ref=progress_data['last_global_emit_time'],
pause_event=pause_event, cookies_for_chunk=cookies_for_chunk_session, logger_func=logger_func, emitter=emitter_for_multipart,
progress_data=progress_data, cancellation_event=cancellation_event,
skip_event=skip_event, global_emit_time_ref=progress_data['last_global_emit_time'],
pause_event=pause_event, cookies_for_chunk=cookies_for_chunk_session,
logger_func=logger_func, emitter=emitter_for_multipart,
api_original_filename=api_original_filename
))
)
chunk_futures.append(future)
for future in as_completed(chunk_futures):
if cancellation_event and cancellation_event.is_set(): all_chunks_successful = False; break
bytes_downloaded_this_chunk, success_this_chunk = future.result()
total_bytes_from_chunks += bytes_downloaded_this_chunk
if not success_this_chunk:
if cancellation_event and cancellation_event.is_set():
all_chunks_successful = False
bytes_downloaded, success = future.result()
total_bytes_from_threads += bytes_downloaded
if not success:
all_chunks_successful = False
total_bytes_final = total_bytes_resumed + total_bytes_from_threads
if cancellation_event and cancellation_event.is_set():
logger_func(f" Multi-part download for '{api_original_filename}' cancelled by main event.")
all_chunks_successful = False
if emitter_for_multipart:
with progress_data['lock']:
status_list_copy = [dict(s) for s in progress_data['chunks_status']]
if isinstance(emitter_for_multipart, queue.Queue):
emitter_for_multipart.put({'type': 'file_progress', 'payload': (api_original_filename, status_list_copy)})
elif hasattr(emitter_for_multipart, 'file_progress_signal'):
emitter_for_multipart.file_progress_signal.emit(api_original_filename, status_list_copy)
if all_chunks_successful and (total_bytes_from_chunks == total_size or total_size == 0):
logger_func(f" ✅ Multi-part download successful for '{api_original_filename}'. Total bytes: {total_bytes_from_chunks}")
# --- Assembly and Cleanup Phase ---
if all_chunks_successful and (total_bytes_final == total_size or total_size == 0):
logger_func(f" ✅ All {num_parts} chunks complete. Assembling final file...")
md5_hasher = hashlib.md5()
with open(temp_file_path, 'rb') as f_hash:
for buf in iter(lambda: f_hash.read(4096*10), b''):
md5_hasher.update(buf)
calculated_hash = md5_hasher.hexdigest()
return True, total_bytes_from_chunks, calculated_hash, open(temp_file_path, 'rb')
try:
with open(save_path, 'wb') as final_file:
for i in range(num_parts):
chunk_part_path = f"{save_path}.part{i}"
with open(chunk_part_path, 'rb') as chunk_file:
content = chunk_file.read()
final_file.write(content)
md5_hasher.update(content)
calculated_hash = md5_hasher.hexdigest()
logger_func(f" ✅ Assembly successful for '{api_original_filename}'. Total bytes: {total_bytes_final}")
return True, total_bytes_final, calculated_hash, open(save_path, 'rb')
except Exception as e:
logger_func(f" ❌ Critical error during file assembly: {e}. Cleaning up.")
return False, total_bytes_final, None, None
finally:
# Cleanup all individual chunk files after successful assembly
for i in range(num_parts):
chunk_part_path = f"{save_path}.part{i}"
if os.path.exists(chunk_part_path):
try:
os.remove(chunk_part_path)
except OSError as e:
logger_func(f" ⚠️ Failed to remove temp part file '{chunk_part_path}': {e}")
else:
logger_func(f" ❌ Multi-part download failed for '{api_original_filename}'. Success: {all_chunks_successful}, Bytes: {total_bytes_from_chunks}/{total_size}. Cleaning up.")
if os.path.exists(temp_file_path):
try: os.remove(temp_file_path)
except OSError as e: logger_func(f" Failed to remove temp part file '{temp_file_path}': {e}")
return False, total_bytes_from_chunks, None, None
# If download failed, we do NOT clean up, allowing for resumption later
logger_func(f" ❌ Multi-part download failed for '{api_original_filename}'. Success: {all_chunks_successful}, Bytes: {total_bytes_final}/{total_size}. Partial chunks saved for future resumption.")
return False, total_bytes_final, None, None

125
src/services/updater.py Normal file
View File

@@ -0,0 +1,125 @@
import sys
import os
import requests
import subprocess # Keep this for now, though it's not used in the final command
from packaging.version import parse as parse_version
from PyQt5.QtCore import QThread, pyqtSignal
# Constants for the updater
GITHUB_REPO_URL = "https://api.github.com/repos/Yuvi9587/Kemono-Downloader/releases/latest"
EXE_NAME = "Kemono.Downloader.exe"
class UpdateChecker(QThread):
"""Checks for a new version on GitHub in a background thread."""
update_available = pyqtSignal(str, str) # new_version, download_url
up_to_date = pyqtSignal(str)
update_error = pyqtSignal(str)
def __init__(self, current_version):
super().__init__()
self.current_version_str = current_version.lstrip('v')
def run(self):
try:
response = requests.get(GITHUB_REPO_URL, timeout=15)
response.raise_for_status()
data = response.json()
latest_version_str = data['tag_name'].lstrip('v')
current_version = parse_version(self.current_version_str)
latest_version = parse_version(latest_version_str)
if latest_version > current_version:
for asset in data.get('assets', []):
if asset['name'] == EXE_NAME:
self.update_available.emit(latest_version_str, asset['browser_download_url'])
return
self.update_error.emit(f"Update found, but '{EXE_NAME}' is missing from the release assets.")
else:
self.up_to_date.emit("You are on the latest version.")
except requests.exceptions.RequestException as e:
self.update_error.emit(f"Network error: {e}")
except Exception as e:
self.update_error.emit(f"An error occurred: {e}")
class UpdateDownloader(QThread):
"""
Downloads the new executable and runs an updater script that kills the old process,
replaces the file, and displays a message in the terminal.
"""
download_finished = pyqtSignal()
download_error = pyqtSignal(str)
def __init__(self, download_url, parent_app):
super().__init__()
self.download_url = download_url
self.parent_app = parent_app
def run(self):
try:
app_path = sys.executable
app_dir = os.path.dirname(app_path)
temp_path = os.path.join(app_dir, f"{EXE_NAME}.tmp")
old_path = os.path.join(app_dir, f"{EXE_NAME}.old")
updater_script_path = os.path.join(app_dir, "updater.bat")
# --- NEW: Path for the PID file ---
pid_file_path = os.path.join(app_dir, "updater.pid")
# Download the new executable
with requests.get(self.download_url, stream=True, timeout=300) as r:
r.raise_for_status()
with open(temp_path, 'wb') as f:
for chunk in r.iter_content(chunk_size=8192):
f.write(chunk)
# --- NEW: Write the current Process ID to the pid file ---
with open(pid_file_path, "w") as f:
f.write(str(os.getpid()))
# --- NEW BATCH SCRIPT ---
# This script now reads the PID from the "updater.pid" file.
script_content = f"""
@echo off
SETLOCAL
echo.
echo Reading process information...
set /p PID=<{pid_file_path}
echo Closing the old application (PID: %PID%)...
taskkill /F /PID %PID%
echo Waiting for files to unlock...
timeout /t 2 /nobreak > nul
echo Replacing application files...
if exist "{old_path}" del /F /Q "{old_path}"
rename "{app_path}" "{os.path.basename(old_path)}"
rename "{temp_path}" "{EXE_NAME}"
echo.
echo ============================================================
echo Update Complete!
echo You can now close this window and run {EXE_NAME}.
echo ============================================================
echo.
pause
echo Cleaning up helper files...
del "{pid_file_path}"
del "%~f0"
ENDLOCAL
"""
with open(updater_script_path, "w") as f:
f.write(script_content)
# --- Go back to the os.startfile command that we know works ---
os.startfile(updater_script_path)
self.download_finished.emit()
except Exception as e:
self.download_error.emit(f"Failed to download or run updater: {e}")

View File

@@ -13,7 +13,7 @@ from PyQt5.QtCore import pyqtSignal, QCoreApplication, QSize, QThread, QTimer, Q
from PyQt5.QtWidgets import (
QApplication, QDialog, QHBoxLayout, QLabel, QLineEdit, QListWidget,
QListWidgetItem, QMessageBox, QPushButton, QVBoxLayout, QAbstractItemView,
QSplitter, QProgressBar, QWidget
QSplitter, QProgressBar, QWidget, QFileDialog
)
# --- Local Application Imports ---
@@ -151,6 +151,8 @@ class EmptyPopupDialog (QDialog ):
app_icon =get_app_icon_object ()
if app_icon and not app_icon .isNull ():
self .setWindowIcon (app_icon )
self.update_profile_data = None
self.update_creator_name = None
self .selected_creators_for_queue =[]
self .globally_selected_creators ={}
self .fetched_posts_data ={}
@@ -205,6 +207,9 @@ class EmptyPopupDialog (QDialog ):
self .scope_button .clicked .connect (self ._toggle_scope_mode )
left_bottom_buttons_layout .addWidget (self .scope_button )
left_pane_layout .addLayout (left_bottom_buttons_layout )
self.update_button = QPushButton()
self.update_button.clicked.connect(self._handle_update_check)
left_bottom_buttons_layout.addWidget(self.update_button)
self .right_pane_widget =QWidget ()
@@ -315,6 +320,31 @@ class EmptyPopupDialog (QDialog ):
except AttributeError :
pass
def _handle_update_check(self):
"""Opens a dialog to select a creator profile and loads it for an update session."""
appdata_dir = os.path.join(self.app_base_dir, "appdata")
profiles_dir = os.path.join(appdata_dir, "creator_profiles")
if not os.path.isdir(profiles_dir):
QMessageBox.warning(self, "Directory Not Found", f"The creator profiles directory does not exist yet.\n\nPath: {profiles_dir}")
return
filepath, _ = QFileDialog.getOpenFileName(self, "Select Creator Profile for Update", profiles_dir, "JSON Files (*.json)")
if filepath:
try:
with open(filepath, 'r', encoding='utf-8') as f:
data = json.load(f)
if 'creator_url' not in data or 'processed_post_ids' not in data:
raise ValueError("Invalid profile format.")
self.update_profile_data = data
self.update_creator_name = os.path.basename(filepath).replace('.json', '')
self.accept() # Close the dialog and signal success
except Exception as e:
QMessageBox.critical(self, "Error Loading Profile", f"Could not load or parse the selected profile file:\n\n{e}")
def _handle_fetch_posts_click (self ):
selected_creators =list (self .globally_selected_creators .values ())
print(f"[DEBUG] Selected creators for fetch: {selected_creators}")
@@ -370,6 +400,7 @@ class EmptyPopupDialog (QDialog ):
self .add_selected_button .setText (self ._tr ("creator_popup_add_selected_button","Add Selected"))
self .fetch_posts_button .setText (self ._tr ("fetch_posts_button_text","Fetch Posts"))
self ._update_scope_button_text_and_tooltip ()
self.update_button.setText(self._tr("check_for_updates_button", "Check for Updates"))
self .posts_search_input .setPlaceholderText (self ._tr ("creator_popup_posts_search_placeholder","Search fetched posts by title..."))
@@ -929,15 +960,19 @@ class EmptyPopupDialog (QDialog ):
self .parent_app .log_signal .emit (f" Added {num_just_added_posts } selected posts to the download queue. Total in queue: {total_in_queue }.")
# --- START: MODIFIED LOGIC ---
# Removed the blockSignals(True/False) calls to allow the main window's UI to update correctly.
if self .parent_app .link_input :
self .parent_app .link_input .blockSignals (True )
self .parent_app .link_input .setText (
self .parent_app ._tr ("popup_posts_selected_text","Posts - {count} selected").format (count =num_just_added_posts )
)
self .parent_app .link_input .blockSignals (False )
self .parent_app .link_input .setPlaceholderText (
self .parent_app ._tr ("items_in_queue_placeholder","{count} items in queue from popup.").format (count =total_in_queue )
)
# --- END: MODIFIED LOGIC ---
self.selected_creators_for_queue.clear()
self .accept ()
else :
QMessageBox .information (self ,self ._tr ("no_selection_title","No Selection"),
@@ -955,9 +990,6 @@ class EmptyPopupDialog (QDialog ):
self .add_selected_button .setEnabled (True )
self .setWindowTitle (self ._tr ("creator_popup_title","Creator Selection"))
def _get_domain_for_service (self ,service_name ):
"""Determines the base domain for a given service."""
service_lower =service_name .lower ()
@@ -1003,4 +1035,4 @@ class EmptyPopupDialog (QDialog ):
else :
if unique_key in self .globally_selected_creators :
del self .globally_selected_creators [unique_key ]
self .fetch_posts_button .setEnabled (bool (self .globally_selected_creators ))
self .fetch_posts_button .setEnabled (bool (self .globally_selected_creators ))

View File

@@ -42,17 +42,12 @@ class ErrorFilesDialog(QDialog):
if app_icon and not app_icon.isNull():
self.setWindowIcon(app_icon)
# --- START OF FIX ---
# Get the user-defined scale factor from the parent application.
scale_factor = getattr(self.parent_app, 'scale_factor', 1.0)
# Define base dimensions and apply the correct scale factor.
base_width, base_height = 550, 400
self.setMinimumSize(int(base_width * scale_factor), int(base_height * scale_factor))
self.resize(int(base_width * scale_factor * 1.1), int(base_height * scale_factor * 1.1))
# --- END OF FIX ---
# --- Initialize UI and Apply Theming ---
self._init_ui()
self._retranslate_ui()
self._apply_theme()
@@ -106,7 +101,17 @@ class ErrorFilesDialog(QDialog):
post_title = error_info.get('post_title', 'Unknown Post')
post_id = error_info.get('original_post_id_for_log', 'N/A')
item_text = f"File: {filename}\nFrom Post: '{post_title}' (ID: {post_id})"
creator_name = "Unknown Creator"
service = error_info.get('service')
user_id = error_info.get('user_id')
# Check if we have the necessary info and access to the cache
if service and user_id and hasattr(self.parent_app, 'creator_name_cache'):
creator_key = (service.lower(), str(user_id))
# Look up the name, fall back to the user_id if not found
creator_name = self.parent_app.creator_name_cache.get(creator_key, user_id)
item_text = f"File: {filename}\nCreator: {creator_name} - Post: '{post_title}' (ID: {post_id})"
list_item = QListWidgetItem(item_text)
list_item.setData(Qt.UserRole, error_info)
list_item.setFlags(list_item.flags() | Qt.ItemIsUserCheckable)

View File

@@ -3,7 +3,7 @@ import html
import re
# --- Third-Party Library Imports ---
import requests
import cloudscraper # MODIFIED: Import cloudscraper
from PyQt5.QtCore import QCoreApplication, Qt
from PyQt5.QtWidgets import (
QApplication, QDialog, QHBoxLayout, QLabel, QLineEdit, QListWidget,
@@ -12,7 +12,6 @@ from PyQt5.QtWidgets import (
# --- Local Application Imports ---
from ...i18n.translator import get_translation
# Corrected Import: Get the icon from the new assets utility module
from ..assets import get_app_icon_object
from ...utils.network_utils import prepare_cookies_for_request
from .CookieHelpDialog import CookieHelpDialog
@@ -37,13 +36,13 @@ class FavoriteArtistsDialog (QDialog ):
self ._init_ui ()
self ._fetch_favorite_artists ()
def _get_domain_for_service (self ,service_name ):
service_lower =service_name .lower ()
coomer_primary_services ={'onlyfans','fansly','manyvids','candfans'}
if service_lower in coomer_primary_services :
return "coomer.su"
else :
return "kemono.su"
def _get_domain_for_service(self, service_name):
service_lower = service_name.lower()
coomer_primary_services = {'onlyfans', 'fansly', 'manyvids', 'candfans'}
if service_lower in coomer_primary_services:
return "coomer.st"
else:
return "kemono.cr"
def _tr (self ,key ,default_text =""):
"""Helper to get translation based on current app language."""
@@ -126,32 +125,49 @@ class FavoriteArtistsDialog (QDialog ):
self .artist_list_widget .setVisible (show )
def _fetch_favorite_artists (self ):
# --- FIX: Use cloudscraper and add proper headers ---
scraper = cloudscraper.create_scraper()
# --- END FIX ---
if self.cookies_config['use_cookie']:
# Check if we can load cookies for at least one of the services.
kemono_cookies = prepare_cookies_for_request(True, self.cookies_config['cookie_text'], self.cookies_config['selected_cookie_file'], self.cookies_config['app_base_dir'], self._logger, target_domain="kemono.su")
coomer_cookies = prepare_cookies_for_request(True, self.cookies_config['cookie_text'], self.cookies_config['selected_cookie_file'], self.cookies_config['app_base_dir'], self._logger, target_domain="coomer.su")
kemono_cookies = prepare_cookies_for_request(
True, self.cookies_config['cookie_text'], self.cookies_config['selected_cookie_file'],
self.cookies_config['app_base_dir'], self._logger, target_domain="kemono.cr"
)
if not kemono_cookies:
self._logger("No cookies for kemono.cr, trying fallback kemono.su...")
kemono_cookies = prepare_cookies_for_request(
True, self.cookies_config['cookie_text'], self.cookies_config['selected_cookie_file'],
self.cookies_config['app_base_dir'], self._logger, target_domain="kemono.su"
)
coomer_cookies = prepare_cookies_for_request(
True, self.cookies_config['cookie_text'], self.cookies_config['selected_cookie_file'],
self.cookies_config['app_base_dir'], self._logger, target_domain="coomer.st"
)
if not coomer_cookies:
self._logger("No cookies for coomer.st, trying fallback coomer.su...")
coomer_cookies = prepare_cookies_for_request(
True, self.cookies_config['cookie_text'], self.cookies_config['selected_cookie_file'],
self.cookies_config['app_base_dir'], self._logger, target_domain="coomer.su"
)
if not kemono_cookies and not coomer_cookies:
# If cookies are enabled but none could be loaded, show help and stop.
self.status_label.setText(self._tr("fav_artists_cookies_required_status", "Error: Cookies enabled but could not be loaded for any source."))
self._logger("Error: Cookies enabled but no valid cookies were loaded. Showing help dialog.")
cookie_help_dialog = CookieHelpDialog(self.parent_app, self)
cookie_help_dialog.exec_()
self.download_button.setEnabled(False)
return # Stop further execution
kemono_fav_url ="https://kemono.su/api/v1/account/favorites?type=artist"
coomer_fav_url ="https://coomer.su/api/v1/account/favorites?type=artist"
return
self .all_fetched_artists =[]
fetched_any_successfully =False
errors_occurred =[]
any_cookies_loaded_successfully_for_any_source =False
api_sources =[
{"name":"Kemono.su","url":kemono_fav_url ,"domain":"kemono.su"},
{"name":"Coomer.su","url":coomer_fav_url ,"domain":"coomer.su"}
api_sources = [
{"name": "Kemono.cr", "url": "https://kemono.cr/api/v1/account/favorites?type=artist", "domain": "kemono.cr"},
{"name": "Coomer.st", "url": "https://coomer.st/api/v1/account/favorites?type=artist", "domain": "coomer.st"}
]
for source in api_sources :
@@ -159,23 +175,39 @@ class FavoriteArtistsDialog (QDialog ):
self .status_label .setText (self ._tr ("fav_artists_loading_from_source_status","⏳ Loading favorites from {source_name}...").format (source_name =source ['name']))
QCoreApplication .processEvents ()
cookies_dict_for_source =None
if self .cookies_config ['use_cookie']:
cookies_dict_for_source =prepare_cookies_for_request (
True ,
self .cookies_config ['cookie_text'],
self .cookies_config ['selected_cookie_file'],
self .cookies_config ['app_base_dir'],
self ._logger ,
target_domain =source ['domain']
cookies_dict_for_source = None
if self.cookies_config['use_cookie']:
primary_domain = source['domain']
fallback_domain = "kemono.su" if "kemono" in primary_domain else "coomer.su"
cookies_dict_for_source = prepare_cookies_for_request(
True, self.cookies_config['cookie_text'], self.cookies_config['selected_cookie_file'],
self.cookies_config['app_base_dir'], self._logger, target_domain=primary_domain
)
if cookies_dict_for_source :
any_cookies_loaded_successfully_for_any_source =True
else :
self ._logger (f"Warning ({source ['name']}): Cookies enabled but could not be loaded for this domain. Fetch might fail if cookies are required.")
if not cookies_dict_for_source:
self._logger(f"Warning ({source['name']}): No cookies for '{primary_domain}'. Trying fallback '{fallback_domain}'...")
cookies_dict_for_source = prepare_cookies_for_request(
True, self.cookies_config['cookie_text'], self.cookies_config['selected_cookie_file'],
self.cookies_config['app_base_dir'], self._logger, target_domain=fallback_domain
)
if cookies_dict_for_source:
any_cookies_loaded_successfully_for_any_source = True
else:
self._logger(f"Warning ({source['name']}): Cookies enabled but not loaded for this source. Fetch may fail.")
try :
headers ={'User-Agent':'Mozilla/5.0'}
response =requests .get (source ['url'],headers =headers ,cookies =cookies_dict_for_source ,timeout =20 )
# --- FIX: Add Referer and Accept headers ---
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36',
'Referer': f"https://{source['domain']}/favorites",
'Accept': 'text/css'
}
# --- END FIX ---
# --- FIX: Use scraper instead of requests ---
response = scraper.get(source['url'], headers=headers, cookies=cookies_dict_for_source, timeout=20)
# --- END FIX ---
response .raise_for_status ()
artists_data_from_api =response .json ()
@@ -210,20 +242,15 @@ class FavoriteArtistsDialog (QDialog ):
fetched_any_successfully =True
self ._logger (f"Fetched {processed_artists_from_source } artists from {source ['name']}.")
except requests .exceptions .RequestException as e :
except Exception as e :
error_msg =f"Error fetching favorites from {source ['name']}: {e }"
self ._logger (error_msg )
errors_occurred .append (error_msg )
except Exception as e :
error_msg =f"An unexpected error occurred with {source ['name']}: {e }"
self ._logger (error_msg )
errors_occurred .append (error_msg )
if self .cookies_config ['use_cookie']and not any_cookies_loaded_successfully_for_any_source :
self .status_label .setText (self ._tr ("fav_artists_cookies_required_status","Error: Cookies enabled but could not be loaded for any source."))
self ._logger ("Error: Cookies enabled but no cookies loaded for any source. Showing help dialog.")
cookie_help_dialog =CookieHelpDialog (self )
cookie_help_dialog = CookieHelpDialog(self.parent_app, self)
cookie_help_dialog .exec_ ()
self .download_button .setEnabled (False )
if not fetched_any_successfully :
@@ -244,7 +271,7 @@ class FavoriteArtistsDialog (QDialog ):
self ._show_content_elements (True )
self .download_button .setEnabled (True )
elif not fetched_any_successfully and not errors_occurred :
self .status_label .setText (self ._tr ("fav_artists_none_found_status","No favorite artists found on Kemono.su or Coomer.su."))
self .status_label .setText (self ._tr ("fav_artists_none_found_status","No favorite artists found on Kemono or Coomer."))
self ._show_content_elements (False )
self .download_button .setEnabled (False )
else :
@@ -300,4 +327,4 @@ class FavoriteArtistsDialog (QDialog ):
self .accept ()
def get_selected_artists (self ):
return self .selected_artists_data
return self .selected_artists_data

View File

@@ -7,7 +7,7 @@ import traceback
import json
import re
from collections import defaultdict
import requests
import cloudscraper # MODIFIED: Import cloudscraper
from PyQt5.QtCore import QCoreApplication, Qt, pyqtSignal, QThread
from PyQt5.QtWidgets import (
QApplication, QDialog, QHBoxLayout, QLabel, QLineEdit, QListWidget,
@@ -34,28 +34,29 @@ class FavoritePostsFetcherThread (QThread ):
self .target_domain_preference =target_domain_preference
self .cancellation_event =threading .Event ()
self .error_key_map ={
"Kemono.su":"kemono_su",
"Coomer.su":"coomer_su"
"kemono.cr":"kemono_su",
"coomer.st":"coomer_su"
}
def _logger (self ,message ):
self .parent_logger_func (f"[FavPostsFetcherThread] {message }")
def run (self ):
kemono_fav_posts_url ="https://kemono.su/api/v1/account/favorites?type=post"
coomer_fav_posts_url ="https://coomer.su/api/v1/account/favorites?type=post"
def run(self):
# --- FIX: Use cloudscraper and add proper headers ---
scraper = cloudscraper.create_scraper()
# --- END FIX ---
all_fetched_posts_temp =[]
error_messages_for_summary =[]
fetched_any_successfully =False
any_cookies_loaded_successfully_for_any_source =False
all_fetched_posts_temp = []
error_messages_for_summary = []
fetched_any_successfully = False
any_cookies_loaded_successfully_for_any_source = False
self .status_update .emit ("key_fetching_fav_post_list_init")
self .progress_bar_update .emit (0 ,0 )
self.status_update.emit("key_fetching_fav_post_list_init")
self.progress_bar_update.emit(0, 0)
api_sources =[
{"name":"Kemono.su","url":kemono_fav_posts_url ,"domain":"kemono.su"},
{"name":"Coomer.su","url":coomer_fav_posts_url ,"domain":"coomer.su"}
api_sources = [
{"name": "Kemono.cr", "url": "https://kemono.cr/api/v1/account/favorites?type=post", "domain": "kemono.cr"},
{"name": "Coomer.st", "url": "https://coomer.st/api/v1/account/favorites?type=post", "domain": "coomer.st"}
]
api_sources_to_try =[]
@@ -76,20 +77,27 @@ class FavoritePostsFetcherThread (QThread ):
if self .cancellation_event .is_set ():
self .finished .emit ([],"KEY_FETCH_CANCELLED_DURING")
return
cookies_dict_for_source =None
if self .cookies_config ['use_cookie']:
cookies_dict_for_source =prepare_cookies_for_request (
True ,
self .cookies_config ['cookie_text'],
self .cookies_config ['selected_cookie_file'],
self .cookies_config ['app_base_dir'],
self ._logger ,
target_domain =source ['domain']
cookies_dict_for_source = None
if self.cookies_config['use_cookie']:
primary_domain = source['domain']
fallback_domain = "kemono.su" if "kemono" in primary_domain else "coomer.su"
cookies_dict_for_source = prepare_cookies_for_request(
True, self.cookies_config['cookie_text'], self.cookies_config['selected_cookie_file'],
self.cookies_config['app_base_dir'], self._logger, target_domain=primary_domain
)
if cookies_dict_for_source :
any_cookies_loaded_successfully_for_any_source =True
else :
self ._logger (f"Warning ({source ['name']}): Cookies enabled but could not be loaded for this domain. Fetch might fail if cookies are required.")
if not cookies_dict_for_source and fallback_domain:
self._logger(f"Warning ({source['name']}): No cookies for '{primary_domain}'. Trying fallback '{fallback_domain}'...")
cookies_dict_for_source = prepare_cookies_for_request(
True, self.cookies_config['cookie_text'], self.cookies_config['selected_cookie_file'],
self.cookies_config['app_base_dir'], self._logger, target_domain=fallback_domain
)
if cookies_dict_for_source:
any_cookies_loaded_successfully_for_any_source = True
else:
self._logger(f"Warning ({source['name']}): Cookies enabled but could not be loaded for this domain. Fetch might fail if cookies are required.")
self ._logger (f"Attempting to fetch favorite posts from: {source ['name']} ({source ['url']})")
source_key_part =self .error_key_map .get (source ['name'],source ['name'].lower ().replace ('.','_'))
@@ -97,8 +105,18 @@ class FavoritePostsFetcherThread (QThread ):
QCoreApplication .processEvents ()
try :
headers ={'User-Agent':'Mozilla/5.0'}
response =requests .get (source ['url'],headers =headers ,cookies =cookies_dict_for_source ,timeout =20 )
# --- FIX: Add Referer and Accept headers ---
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36',
'Referer': f"https://{source['domain']}/favorites",
'Accept': 'text/css'
}
# --- END FIX ---
# --- FIX: Use scraper instead of requests ---
response = scraper.get(source['url'], headers=headers, cookies=cookies_dict_for_source, timeout=20)
# --- END FIX ---
response .raise_for_status ()
posts_data_from_api =response .json ()
@@ -130,33 +148,24 @@ class FavoritePostsFetcherThread (QThread ):
fetched_any_successfully =True
self ._logger (f"Fetched {processed_posts_from_source } posts from {source ['name']}.")
except requests .exceptions .RequestException as e :
except Exception as e :
err_detail =f"Error fetching favorite posts from {source ['name']}: {e }"
self ._logger (err_detail )
error_messages_for_summary .append (err_detail )
if e .response is not None and e .response .status_code ==401 :
if hasattr(e, 'response') and e.response is not None and e.response.status_code == 401:
self .finished .emit ([],"KEY_AUTH_FAILED")
self ._logger (f"Authorization failed for {source ['name']}, emitting KEY_AUTH_FAILED.")
return
except Exception as e :
err_detail =f"An unexpected error occurred with {source ['name']}: {e }"
self ._logger (err_detail )
error_messages_for_summary .append (err_detail )
if self .cancellation_event .is_set ():
self .finished .emit ([],"KEY_FETCH_CANCELLED_AFTER")
return
if self .cookies_config ['use_cookie']and not any_cookies_loaded_successfully_for_any_source :
if self .target_domain_preference and not any_cookies_loaded_successfully_for_any_source :
domain_key_part =self .error_key_map .get (self .target_domain_preference ,self .target_domain_preference .lower ().replace ('.','_'))
self .finished .emit ([],f"KEY_COOKIES_REQUIRED_BUT_NOT_FOUND_FOR_DOMAIN_{domain_key_part }")
return
self .finished .emit ([],"KEY_COOKIES_REQUIRED_BUT_NOT_FOUND_GENERIC")
return
@@ -409,14 +418,14 @@ class FavoritePostsDialog (QDialog ):
if status_key .startswith ("KEY_COOKIES_REQUIRED_BUT_NOT_FOUND_FOR_DOMAIN_")or status_key =="KEY_COOKIES_REQUIRED_BUT_NOT_FOUND_GENERIC":
status_label_text_key ="fav_posts_cookies_required_error"
self ._logger (f"Cookie error: {status_key }. Showing help dialog.")
cookie_help_dialog =CookieHelpDialog (self )
cookie_help_dialog = CookieHelpDialog(self.parent_app, self)
cookie_help_dialog .exec_ ()
elif status_key =="KEY_AUTH_FAILED":
status_label_text_key ="fav_posts_auth_failed_title"
self ._logger (f"Auth error: {status_key }. Showing help dialog.")
QMessageBox .warning (self ,self ._tr ("fav_posts_auth_failed_title","Authorization Failed (Posts)"),
self ._tr ("fav_posts_auth_failed_message_generic","...").format (domain_specific_part =specific_domain_msg_part ))
cookie_help_dialog =CookieHelpDialog (self )
cookie_help_dialog = CookieHelpDialog(self.parent_app, self)
cookie_help_dialog .exec_ ()
elif status_key =="KEY_NO_FAVORITES_FOUND_ALL_PLATFORMS":
status_label_text_key ="fav_posts_no_posts_found_status"
@@ -620,4 +629,4 @@ class FavoritePostsDialog (QDialog ):
self .accept ()
def get_selected_posts (self ):
return self .selected_posts_data
return self .selected_posts_data

View File

@@ -1,23 +1,111 @@
# --- Standard Library Imports ---
import os
import json
import sys
# --- PyQt5 Imports ---
from PyQt5.QtCore import Qt, QStandardPaths
from PyQt5.QtCore import Qt, QStandardPaths, QTimer
from PyQt5.QtWidgets import (
QApplication, QDialog, QHBoxLayout, QLabel, QPushButton, QVBoxLayout,
QGroupBox, QComboBox, QMessageBox, QGridLayout
QGroupBox, QComboBox, QMessageBox, QGridLayout, QCheckBox
)
# --- Local Application Imports ---
from ...i18n.translator import get_translation
from ...utils.resolution import get_dark_theme
from ..assets import get_app_icon_object
from ..main_window import get_app_icon_object
from ...config.constants import (
THEME_KEY, LANGUAGE_KEY, DOWNLOAD_LOCATION_KEY,
RESOLUTION_KEY, UI_SCALE_KEY
RESOLUTION_KEY, UI_SCALE_KEY, SAVE_CREATOR_JSON_KEY,
COOKIE_TEXT_KEY, USE_COOKIE_KEY,
FETCH_FIRST_KEY, DISCORD_TOKEN_KEY, POST_DOWNLOAD_ACTION_KEY
)
from ...services.updater import UpdateChecker, UpdateDownloader
class CountdownMessageBox(QDialog):
"""
A custom message box that includes a countdown timer for the 'Yes' button,
which automatically accepts the dialog when the timer reaches zero.
"""
def __init__(self, title, text, countdown_seconds=10, parent_app=None, parent=None):
super().__init__(parent)
self.parent_app = parent_app
self.countdown = countdown_seconds
# --- Basic Window Setup ---
self.setWindowTitle(title)
self.setModal(True)
app_icon = get_app_icon_object()
if app_icon and not app_icon.isNull():
self.setWindowIcon(app_icon)
self._init_ui(text)
self._apply_theme()
# --- Timer Setup ---
self.timer = QTimer(self)
self.timer.setInterval(1000) # Tick every second
self.timer.timeout.connect(self._update_countdown)
self.timer.start()
def _init_ui(self, text):
"""Initializes the UI components of the dialog."""
main_layout = QVBoxLayout(self)
self.message_label = QLabel(text)
self.message_label.setWordWrap(True)
self.message_label.setAlignment(Qt.AlignCenter)
main_layout.addWidget(self.message_label)
buttons_layout = QHBoxLayout()
buttons_layout.addStretch(1)
self.yes_button = QPushButton()
self.yes_button.clicked.connect(self.accept)
self.yes_button.setDefault(True)
self.no_button = QPushButton()
self.no_button.clicked.connect(self.reject)
buttons_layout.addWidget(self.yes_button)
buttons_layout.addWidget(self.no_button)
buttons_layout.addStretch(1)
main_layout.addLayout(buttons_layout)
self._retranslate_ui()
self._update_countdown() # Initial text setup
def _tr(self, key, default_text=""):
"""Helper for translations."""
if self.parent_app and hasattr(self.parent_app, 'current_selected_language'):
return get_translation(self.parent_app.current_selected_language, key, default_text)
return default_text
def _retranslate_ui(self):
"""Sets translated text for UI elements."""
self.no_button.setText(self._tr("no_button_text", "No"))
# The 'yes' button text is handled by the countdown
def _update_countdown(self):
"""Updates the countdown and button text each second."""
if self.countdown <= 0:
self.timer.stop()
self.accept() # Automatically accept when countdown finishes
return
yes_text = self._tr("yes_button_text", "Yes")
self.yes_button.setText(f"{yes_text} ({self.countdown})")
self.countdown -= 1
def _apply_theme(self):
"""Applies the current theme from the parent application."""
if self.parent_app and hasattr(self.parent_app, 'current_theme') and self.parent_app.current_theme == "dark":
scale = getattr(self.parent_app, 'scale_factor', 1)
self.setStyleSheet(get_dark_theme(scale))
else:
self.setStyleSheet("")
class FutureSettingsDialog(QDialog):
"""
@@ -28,6 +116,7 @@ class FutureSettingsDialog(QDialog):
super().__init__(parent)
self.parent_app = parent_app_ref
self.setModal(True)
self.update_downloader_thread = None # To keep a reference
app_icon = get_app_icon_object()
if app_icon and not app_icon.isNull():
@@ -35,7 +124,7 @@ class FutureSettingsDialog(QDialog):
screen_height = QApplication.primaryScreen().availableGeometry().height() if QApplication.primaryScreen() else 800
scale_factor = screen_height / 800.0
base_min_w, base_min_h = 420, 320 # Adjusted height for new layout
base_min_w, base_min_h = 420, 520 # Increased height for new options
scaled_min_w = int(base_min_w * scale_factor)
scaled_min_h = int(base_min_h * scale_factor)
self.setMinimumSize(scaled_min_w, scaled_min_h)
@@ -48,25 +137,21 @@ class FutureSettingsDialog(QDialog):
"""Initializes all UI components and layouts for the dialog."""
main_layout = QVBoxLayout(self)
# --- Group 1: Interface Settings ---
self.interface_group_box = QGroupBox()
interface_layout = QGridLayout(self.interface_group_box)
# Theme
self.theme_label = QLabel()
self.theme_toggle_button = QPushButton()
self.theme_toggle_button.clicked.connect(self._toggle_theme)
interface_layout.addWidget(self.theme_label, 0, 0)
interface_layout.addWidget(self.theme_toggle_button, 0, 1)
# UI Scale
self.ui_scale_label = QLabel()
self.ui_scale_combo_box = QComboBox()
self.ui_scale_combo_box.currentIndexChanged.connect(self._display_setting_changed)
interface_layout.addWidget(self.ui_scale_label, 1, 0)
interface_layout.addWidget(self.ui_scale_combo_box, 1, 1)
# Language
self.language_label = QLabel()
self.language_combo_box = QComboBox()
self.language_combo_box.currentIndexChanged.connect(self._language_selection_changed)
@@ -75,64 +160,148 @@ class FutureSettingsDialog(QDialog):
main_layout.addWidget(self.interface_group_box)
# --- Group 2: Download & Window Settings ---
self.download_window_group_box = QGroupBox()
download_window_layout = QGridLayout(self.download_window_group_box)
# Window Size (Resolution)
self.window_size_label = QLabel()
self.resolution_combo_box = QComboBox()
self.resolution_combo_box.currentIndexChanged.connect(self._display_setting_changed)
download_window_layout.addWidget(self.window_size_label, 0, 0)
download_window_layout.addWidget(self.resolution_combo_box, 0, 1)
# Default Path
self.default_path_label = QLabel()
self.save_path_button = QPushButton()
self.save_path_button.clicked.connect(self._save_download_path)
self.save_path_button.clicked.connect(self._save_settings)
download_window_layout.addWidget(self.default_path_label, 1, 0)
download_window_layout.addWidget(self.save_path_button, 1, 1)
self.post_download_action_label = QLabel()
self.post_download_action_combo = QComboBox()
self.post_download_action_combo.currentIndexChanged.connect(self._post_download_action_changed)
download_window_layout.addWidget(self.post_download_action_label, 2, 0)
download_window_layout.addWidget(self.post_download_action_combo, 2, 1)
self.save_creator_json_checkbox = QCheckBox()
self.save_creator_json_checkbox.stateChanged.connect(self._creator_json_setting_changed)
download_window_layout.addWidget(self.save_creator_json_checkbox, 3, 0, 1, 2)
self.fetch_first_checkbox = QCheckBox()
self.fetch_first_checkbox.stateChanged.connect(self._fetch_first_setting_changed)
download_window_layout.addWidget(self.fetch_first_checkbox, 4, 0, 1, 2)
main_layout.addWidget(self.download_window_group_box)
self.update_group_box = QGroupBox()
update_layout = QGridLayout(self.update_group_box)
self.version_label = QLabel()
self.update_status_label = QLabel()
self.check_update_button = QPushButton()
self.check_update_button.clicked.connect(self._check_for_updates)
update_layout.addWidget(self.version_label, 0, 0)
update_layout.addWidget(self.update_status_label, 0, 1)
update_layout.addWidget(self.check_update_button, 1, 0, 1, 2)
main_layout.addWidget(self.update_group_box)
main_layout.addStretch(1)
# --- OK Button ---
self.ok_button = QPushButton()
self.ok_button.clicked.connect(self.accept)
main_layout.addWidget(self.ok_button, 0, Qt.AlignRight | Qt.AlignBottom)
def _retranslate_ui(self):
self.setWindowTitle(self._tr("settings_dialog_title", "Settings"))
self.interface_group_box.setTitle(self._tr("interface_group_title", "Interface Settings"))
self.download_window_group_box.setTitle(self._tr("download_window_group_title", "Download & Window Settings"))
self.theme_label.setText(self._tr("theme_label", "Theme:"))
self.ui_scale_label.setText(self._tr("ui_scale_label", "UI Scale:"))
self.language_label.setText(self._tr("language_label", "Language:"))
self.window_size_label.setText(self._tr("window_size_label", "Window Size:"))
self.default_path_label.setText(self._tr("default_path_label", "Default Path:"))
self.post_download_action_label.setText(self._tr("post_download_action_label", "Action After Download:"))
self.save_creator_json_checkbox.setText(self._tr("save_creator_json_label", "Save Creator.json file"))
self.fetch_first_checkbox.setText(self._tr("fetch_first_label", "Fetch First (Download after all pages are found)"))
self.fetch_first_checkbox.setToolTip(self._tr("fetch_first_tooltip", "If checked, the downloader will find all posts from a creator first before starting any downloads.\nThis can be slower to start but provides a more accurate progress bar."))
self._update_theme_toggle_button_text()
self.save_path_button.setText(self._tr("settings_save_all_button", "Save Path + Cookie + Token"))
self.save_path_button.setToolTip(self._tr("settings_save_all_tooltip", "Save the current 'Download Location', Cookie, and Discord Token settings for future sessions."))
self.ok_button.setText(self._tr("ok_button", "OK"))
self.update_group_box.setTitle(self._tr("update_group_title", "Application Updates"))
current_version = self.parent_app.windowTitle().split(' v')[-1]
self.version_label.setText(self._tr("current_version_label", f"Current Version: v{current_version}"))
self.update_status_label.setText(self._tr("update_status_ready", "Ready to check."))
self.check_update_button.setText(self._tr("check_for_updates_button", "Check for Updates"))
self._populate_display_combo_boxes()
self._populate_language_combo_box()
self._populate_post_download_action_combo()
self._load_checkbox_states()
def _check_for_updates(self):
self.check_update_button.setEnabled(False)
self.update_status_label.setText(self._tr("update_status_checking", "Checking..."))
current_version = self.parent_app.windowTitle().split(' v')[-1]
self.update_checker_thread = UpdateChecker(current_version)
self.update_checker_thread.update_available.connect(self._on_update_available)
self.update_checker_thread.up_to_date.connect(self._on_up_to_date)
self.update_checker_thread.update_error.connect(self._on_update_error)
self.update_checker_thread.start()
def _on_update_available(self, new_version, download_url):
self.update_status_label.setText(self._tr("update_status_found", f"Update found: v{new_version}"))
self.check_update_button.setEnabled(True)
reply = QMessageBox.question(self, self._tr("update_available_title", "Update Available"),
self._tr("update_available_message", f"A new version (v{new_version}) is available.\nWould you like to download and install it now?"),
QMessageBox.Yes | QMessageBox.No, QMessageBox.Yes)
if reply == QMessageBox.Yes:
self.ok_button.setEnabled(False)
self.check_update_button.setEnabled(False)
self.update_status_label.setText(self._tr("update_status_downloading", "Downloading update..."))
self.update_downloader_thread = UpdateDownloader(download_url, self.parent_app)
self.update_downloader_thread.download_finished.connect(self._on_download_finished)
self.update_downloader_thread.download_error.connect(self._on_update_error)
self.update_downloader_thread.start()
def _on_download_finished(self):
QApplication.instance().quit()
def _on_up_to_date(self, message):
self.update_status_label.setText(self._tr("update_status_latest", message))
self.check_update_button.setEnabled(True)
def _on_update_error(self, message):
self.update_status_label.setText(self._tr("update_status_error", f"Error: {message}"))
self.check_update_button.setEnabled(True)
self.ok_button.setEnabled(True)
def _load_checkbox_states(self):
self.save_creator_json_checkbox.blockSignals(True)
should_save = self.parent_app.settings.value(SAVE_CREATOR_JSON_KEY, True, type=bool)
self.save_creator_json_checkbox.setChecked(should_save)
self.save_creator_json_checkbox.blockSignals(False)
self.fetch_first_checkbox.blockSignals(True)
should_fetch_first = self.parent_app.settings.value(FETCH_FIRST_KEY, False, type=bool)
self.fetch_first_checkbox.setChecked(should_fetch_first)
self.fetch_first_checkbox.blockSignals(False)
def _creator_json_setting_changed(self, state):
is_checked = state == Qt.Checked
self.parent_app.settings.setValue(SAVE_CREATOR_JSON_KEY, is_checked)
self.parent_app.settings.sync()
def _fetch_first_setting_changed(self, state):
is_checked = state == Qt.Checked
self.parent_app.settings.setValue(FETCH_FIRST_KEY, is_checked)
self.parent_app.settings.sync()
def _tr(self, key, default_text=""):
if callable(get_translation) and self.parent_app:
return get_translation(self.parent_app.current_selected_language, key, default_text)
return default_text
def _retranslate_ui(self):
self.setWindowTitle(self._tr("settings_dialog_title", "Settings"))
# Group Box Titles
self.interface_group_box.setTitle(self._tr("interface_group_title", "Interface Settings"))
self.download_window_group_box.setTitle(self._tr("download_window_group_title", "Download & Window Settings"))
# Interface Group Labels
self.theme_label.setText(self._tr("theme_label", "Theme:"))
self.ui_scale_label.setText(self._tr("ui_scale_label", "UI Scale:"))
self.language_label.setText(self._tr("language_label", "Language:"))
# Download & Window Group Labels
self.window_size_label.setText(self._tr("window_size_label", "Window Size:"))
self.default_path_label.setText(self._tr("default_path_label", "Default Path:"))
# Buttons and Controls
self._update_theme_toggle_button_text()
self.save_path_button.setText(self._tr("settings_save_path_button", "Save Current Download Path"))
self.save_path_button.setToolTip(self._tr("settings_save_path_tooltip", "Save the current 'Download Location' for future sessions."))
self.ok_button.setText(self._tr("ok_button", "OK"))
# Populate dropdowns
self._populate_display_combo_boxes()
self._populate_language_combo_box()
def _apply_theme(self):
if self.parent_app and self.parent_app.current_theme == "dark":
scale = getattr(self.parent_app, 'scale_factor', 1)
@@ -158,14 +327,7 @@ class FutureSettingsDialog(QDialog):
def _populate_display_combo_boxes(self):
self.resolution_combo_box.blockSignals(True)
self.resolution_combo_box.clear()
resolutions = [
("Auto", self._tr("auto_resolution", "Auto (System Default)")),
("1280x720", "1280 x 720"),
("1600x900", "1600 x 900"),
("1920x1080", "1920 x 1080 (Full HD)"),
("2560x1440", "2560 x 1440 (2K)"),
("3840x2160", "3840 x 2160 (4K)")
]
resolutions = [("Auto", "Auto"), ("1280x720", "1280x720"), ("1600x900", "1600x900"), ("1920x1080", "1920x1080")]
current_res = self.parent_app.settings.value(RESOLUTION_KEY, "Auto")
for res_key, res_name in resolutions:
self.resolution_combo_box.addItem(res_name, res_key)
@@ -176,43 +338,24 @@ class FutureSettingsDialog(QDialog):
self.ui_scale_combo_box.blockSignals(True)
self.ui_scale_combo_box.clear()
scales = [
(0.5, "50%"),
(0.7, "70%"),
(0.9, "90%"),
(1.0, "100% (Default)"),
(1.25, "125%"),
(1.50, "150%"),
(1.75, "175%"),
(2.0, "200%")
(0.5, "50%"), (0.7, "70%"), (0.9, "90%"), (1.0, "100% (Default)"),
(1.25, "125%"), (1.50, "150%"), (1.75, "175%"), (2.0, "200%")
]
current_scale = float(self.parent_app.settings.value(UI_SCALE_KEY, 1.0))
current_scale = self.parent_app.settings.value(UI_SCALE_KEY, 1.0)
for scale_val, scale_name in scales:
self.ui_scale_combo_box.addItem(scale_name, scale_val)
if abs(current_scale - scale_val) < 0.01:
if abs(float(current_scale) - scale_val) < 0.01:
self.ui_scale_combo_box.setCurrentIndex(self.ui_scale_combo_box.count() - 1)
self.ui_scale_combo_box.blockSignals(False)
def _display_setting_changed(self):
selected_res = self.resolution_combo_box.currentData()
selected_scale = self.ui_scale_combo_box.currentData()
self.parent_app.settings.setValue(RESOLUTION_KEY, selected_res)
self.parent_app.settings.setValue(UI_SCALE_KEY, selected_scale)
self.parent_app.settings.sync()
msg_box = QMessageBox(self)
msg_box.setIcon(QMessageBox.Information)
msg_box.setWindowTitle(self._tr("display_change_title", "Display Settings Changed"))
msg_box.setText(self._tr("language_change_message", "A restart is required for these changes to take effect."))
msg_box.setInformativeText(self._tr("language_change_informative", "Would you like to restart now?"))
restart_button = msg_box.addButton(self._tr("restart_now_button", "Restart Now"), QMessageBox.ApplyRole)
ok_button = msg_box.addButton(self._tr("ok_button", "OK"), QMessageBox.AcceptRole)
msg_box.setDefaultButton(ok_button)
msg_box.exec_()
if msg_box.clickedButton() == restart_button:
self.parent_app._request_restart_application()
QMessageBox.information(self, self._tr("display_change_title", "Display Settings Changed"),
self._tr("language_change_message", "A restart is required..."))
def _populate_language_combo_box(self):
self.language_combo_box.blockSignals(True)
@@ -236,40 +379,74 @@ class FutureSettingsDialog(QDialog):
self.parent_app.settings.setValue(LANGUAGE_KEY, selected_lang_code)
self.parent_app.settings.sync()
self.parent_app.current_selected_language = selected_lang_code
self._retranslate_ui()
if hasattr(self.parent_app, '_retranslate_main_ui'):
self.parent_app._retranslate_main_ui()
msg_box = QMessageBox(self)
msg_box.setIcon(QMessageBox.Information)
msg_box.setWindowTitle(self._tr("language_change_title", "Language Changed"))
msg_box.setText(self._tr("language_change_message", "A restart is required..."))
msg_box.setInformativeText(self._tr("language_change_informative", "Would you like to restart now?"))
restart_button = msg_box.addButton(self._tr("restart_now_button", "Restart Now"), QMessageBox.ApplyRole)
ok_button = msg_box.addButton(self._tr("ok_button", "OK"), QMessageBox.AcceptRole)
msg_box.setDefaultButton(ok_button)
msg_box.exec_()
self.parent_app._retranslate_main_ui()
QMessageBox.information(self, self._tr("language_change_title", "Language Changed"),
self._tr("language_change_message", "A restart is required..."))
if msg_box.clickedButton() == restart_button:
self.parent_app._request_restart_application()
def _populate_post_download_action_combo(self):
"""Populates the action dropdown and sets the current selection from settings."""
self.post_download_action_combo.blockSignals(True)
self.post_download_action_combo.clear()
actions = [
(self._tr("action_off", "Off"), "off"),
(self._tr("action_notify", "Notify with Sound"), "notify"),
(self._tr("action_sleep", "Sleep"), "sleep"),
(self._tr("action_shutdown", "Shutdown"), "shutdown")
]
current_action = self.parent_app.settings.value(POST_DOWNLOAD_ACTION_KEY, "off")
for text, key in actions:
self.post_download_action_combo.addItem(text, key)
if current_action == key:
self.post_download_action_combo.setCurrentIndex(self.post_download_action_combo.count() - 1)
self.post_download_action_combo.blockSignals(False)
def _post_download_action_changed(self):
"""Saves the selected post-download action to settings."""
selected_action = self.post_download_action_combo.currentData()
self.parent_app.settings.setValue(POST_DOWNLOAD_ACTION_KEY, selected_action)
self.parent_app.settings.sync()
def _save_settings(self):
path_saved = False
cookie_saved = False
token_saved = False
def _save_download_path(self):
if hasattr(self.parent_app, 'dir_input') and self.parent_app.dir_input:
current_path = self.parent_app.dir_input.text().strip()
if current_path and os.path.isdir(current_path):
self.parent_app.settings.setValue(DOWNLOAD_LOCATION_KEY, current_path)
self.parent_app.settings.sync()
QMessageBox.information(self,
self._tr("settings_save_path_success_title", "Path Saved"),
self._tr("settings_save_path_success_message", "Download location '{path}' saved.").format(path=current_path))
elif not current_path:
QMessageBox.warning(self,
self._tr("settings_save_path_empty_title", "Empty Path"),
self._tr("settings_save_path_empty_message", "Download location cannot be empty."))
path_saved = True
if hasattr(self.parent_app, 'use_cookie_checkbox'):
use_cookie = self.parent_app.use_cookie_checkbox.isChecked()
cookie_content = self.parent_app.cookie_text_input.text().strip()
if use_cookie and cookie_content:
self.parent_app.settings.setValue(USE_COOKIE_KEY, True)
self.parent_app.settings.setValue(COOKIE_TEXT_KEY, cookie_content)
cookie_saved = True
else:
QMessageBox.warning(self,
self._tr("settings_save_path_invalid_title", "Invalid Path"),
self._tr("settings_save_path_invalid_message", "The path '{path}' is not a valid directory.").format(path=current_path))
self.parent_app.settings.setValue(USE_COOKIE_KEY, False)
self.parent_app.settings.setValue(COOKIE_TEXT_KEY, "")
if (hasattr(self.parent_app, 'remove_from_filename_input') and
hasattr(self.parent_app, 'remove_from_filename_label_widget')):
label_text = self.parent_app.remove_from_filename_label_widget.text()
if "Token" in label_text:
discord_token = self.parent_app.remove_from_filename_input.text().strip()
if discord_token:
self.parent_app.settings.setValue(DISCORD_TOKEN_KEY, discord_token)
token_saved = True
self.parent_app.settings.sync()
if path_saved or cookie_saved or token_saved:
QMessageBox.information(self, "Settings Saved", "Settings have been saved successfully.")
else:
QMessageBox.critical(self, "Error", "Could not access download path input from main application.")
QMessageBox.warning(self, "Nothing to Save", "No valid settings were found to save.")

View File

@@ -4,7 +4,7 @@ from PyQt5.QtCore import QUrl, QSize, Qt
from PyQt5.QtGui import QIcon, QDesktopServices
from PyQt5.QtWidgets import (
QApplication, QDialog, QHBoxLayout, QLabel, QPushButton, QVBoxLayout,
QStackedWidget, QScrollArea, QFrame, QWidget
QStackedWidget, QListWidget, QFrame, QWidget, QScrollArea
)
from ...i18n.translator import get_translation
from ..main_window import get_app_icon_object
@@ -46,13 +46,12 @@ class TourStepWidget(QWidget):
layout.addWidget(scroll_area, 1)
class HelpGuideDialog (QDialog ):
"""A multi-page dialog for displaying the feature guide."""
def __init__ (self ,steps_data ,parent_app ,parent =None ):
super ().__init__ (parent )
self .current_step =0
self .steps_data =steps_data
self .parent_app =parent_app
class HelpGuideDialog(QDialog):
"""A multi-page dialog for displaying the feature guide with a navigation list."""
def __init__(self, steps_data, parent_app, parent=None):
super().__init__(parent)
self.steps_data = steps_data
self.parent_app = parent_app
scale = self.parent_app.scale_factor if hasattr(self.parent_app, 'scale_factor') else 1.0
@@ -61,7 +60,7 @@ class HelpGuideDialog (QDialog ):
self.setWindowIcon(app_icon)
self.setModal(True)
self.resize(int(650 * scale), int(600 * scale))
self.resize(int(800 * scale), int(650 * scale))
dialog_font_size = int(11 * scale)
@@ -69,6 +68,7 @@ class HelpGuideDialog (QDialog ):
if hasattr(self.parent_app, 'current_theme') and self.parent_app.current_theme == "dark":
current_theme_style = get_dark_theme(scale)
else:
# Basic light theme fallback
current_theme_style = f"""
QDialog {{ background-color: #F0F0F0; border: 1px solid #B0B0B0; }}
QLabel {{ color: #1E1E1E; }}
@@ -86,118 +86,107 @@ class HelpGuideDialog (QDialog ):
"""
self.setStyleSheet(current_theme_style)
self ._init_ui ()
if self .parent_app :
self .move (self .parent_app .geometry ().center ()-self .rect ().center ())
self._init_ui()
if self.parent_app:
self.move(self.parent_app.geometry().center() - self.rect().center())
def _tr (self ,key ,default_text =""):
def _tr(self, key, default_text=""):
"""Helper to get translation based on current app language."""
if callable (get_translation )and self .parent_app :
return get_translation (self .parent_app .current_selected_language ,key ,default_text )
return default_text
if callable(get_translation) and self.parent_app:
return get_translation(self.parent_app.current_selected_language, key, default_text)
return default_text
def _init_ui(self):
main_layout = QVBoxLayout(self)
main_layout.setContentsMargins(15, 15, 15, 15)
main_layout.setSpacing(10)
def _init_ui (self ):
main_layout =QVBoxLayout (self )
main_layout .setContentsMargins (0 ,0 ,0 ,0 )
main_layout .setSpacing (0 )
# Title
title_label = QLabel(self._tr("help_guide_dialog_title", "Kemono Downloader - Feature Guide"))
scale = getattr(self.parent_app, 'scale_factor', 1.0)
title_font_size = int(16 * scale)
title_label.setStyleSheet(f"font-size: {title_font_size}pt; font-weight: bold; color: #E0E0E0;")
title_label.setAlignment(Qt.AlignCenter)
main_layout.addWidget(title_label)
self .stacked_widget =QStackedWidget ()
main_layout .addWidget (self .stacked_widget ,1 )
# Content Layout (Navigation + Stacked Pages)
content_layout = QHBoxLayout()
main_layout.addLayout(content_layout, 1)
self .tour_steps_widgets =[]
scale = self.parent_app.scale_factor if hasattr(self.parent_app, 'scale_factor') else 1.0
for title, content in self.steps_data:
step_widget = TourStepWidget(title, content, scale=scale)
self.tour_steps_widgets.append(step_widget)
self.nav_list = QListWidget()
self.nav_list.setFixedWidth(int(220 * scale))
self.nav_list.setStyleSheet(f"""
QListWidget {{
background-color: #2E2E2E;
border: 1px solid #4A4A4A;
border-radius: 4px;
font-size: {int(11 * scale)}pt;
}}
QListWidget::item {{
padding: 10px;
border-bottom: 1px solid #4A4A4A;
}}
QListWidget::item:selected {{
background-color: #87CEEB;
color: #2E2E2E;
font-weight: bold;
}}
""")
content_layout.addWidget(self.nav_list)
self.stacked_widget = QStackedWidget()
content_layout.addWidget(self.stacked_widget)
for title_key, content_key in self.steps_data:
title = self._tr(title_key, title_key)
content = self._tr(content_key, f"Content for {content_key} not found.")
self.nav_list.addItem(title)
step_widget = TourStepWidget(title, content, scale=scale)
self.stacked_widget.addWidget(step_widget)
self .setWindowTitle (self ._tr ("help_guide_dialog_title","Kemono Downloader - Feature Guide"))
self.nav_list.currentRowChanged.connect(self.stacked_widget.setCurrentIndex)
if self.nav_list.count() > 0:
self.nav_list.setCurrentRow(0)
buttons_layout =QHBoxLayout ()
buttons_layout .setContentsMargins (15 ,10 ,15 ,15 )
buttons_layout .setSpacing (10 )
# Footer Layout (Social links and Close button)
footer_layout = QHBoxLayout()
footer_layout.setContentsMargins(0, 10, 0, 0)
# Social Media Icons
if getattr(sys, 'frozen', False) and hasattr(sys, '_MEIPASS'):
assets_base_dir = sys._MEIPASS
else:
assets_base_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), '..', '..', '..'))
self .back_button =QPushButton (self ._tr ("tour_dialog_back_button","Back"))
self .back_button .clicked .connect (self ._previous_step )
self .back_button .setEnabled (False )
github_icon_path = os.path.join(assets_base_dir, "assets", "github.png")
instagram_icon_path = os.path.join(assets_base_dir, "assets", "instagram.png")
discord_icon_path = os.path.join(assets_base_dir, "assets", "discord.png")
if getattr (sys ,'frozen',False )and hasattr (sys ,'_MEIPASS'):
assets_base_dir =sys ._MEIPASS
else :
assets_base_dir =os.path.abspath(os.path.join(os.path.dirname(__file__), '..', '..', '..'))
self.github_button = QPushButton(QIcon(github_icon_path), "")
self.instagram_button = QPushButton(QIcon(instagram_icon_path), "")
self.discord_button = QPushButton(QIcon(discord_icon_path), "")
github_icon_path =os .path .join (assets_base_dir ,"assets","github.png")
instagram_icon_path =os .path .join (assets_base_dir ,"assets","instagram.png")
discord_icon_path =os .path .join (assets_base_dir ,"assets","discord.png")
self .github_button =QPushButton (QIcon (github_icon_path ),"")
self .instagram_button =QPushButton (QIcon (instagram_icon_path ),"")
self .Discord_button =QPushButton (QIcon (discord_icon_path ),"")
scale = self.parent_app.scale_factor if hasattr(self.parent_app, 'scale_factor') else 1.0
icon_dim = int(24 * scale)
icon_size = QSize(icon_dim, icon_dim)
self .github_button .setIconSize (icon_size )
self .instagram_button .setIconSize (icon_size )
self .Discord_button .setIconSize (icon_size )
for button, tooltip_key, url in [
(self.github_button, "help_guide_github_tooltip", "https://github.com/Yuvi9587"),
(self.instagram_button, "help_guide_instagram_tooltip", "https://www.instagram.com/uvi.arts/"),
(self.discord_button, "help_guide_discord_tooltip", "https://discord.gg/BqP64XTdJN")
]:
button.setIconSize(icon_size)
button.setToolTip(self._tr(tooltip_key))
button.setFixedSize(icon_size.width() + 8, icon_size.height() + 8)
button.setStyleSheet("background-color: transparent; border: none;")
button.clicked.connect(lambda _, u=url: QDesktopServices.openUrl(QUrl(u)))
footer_layout.addWidget(button)
self .next_button =QPushButton (self ._tr ("tour_dialog_next_button","Next"))
self .next_button .clicked .connect (self ._next_step_action )
self .next_button .setDefault (True )
self .github_button .clicked .connect (self ._open_github_link )
self .instagram_button .clicked .connect (self ._open_instagram_link )
self .Discord_button .clicked .connect (self ._open_Discord_link )
self .github_button .setToolTip (self ._tr ("help_guide_github_tooltip","Visit project's GitHub page (Opens in browser)"))
self .instagram_button .setToolTip (self ._tr ("help_guide_instagram_tooltip","Visit our Instagram page (Opens in browser)"))
self .Discord_button .setToolTip (self ._tr ("help_guide_discord_tooltip","Visit our Discord community (Opens in browser)"))
footer_layout.addStretch(1)
self.finish_button = QPushButton(self._tr("tour_dialog_finish_button", "Finish"))
self.finish_button.clicked.connect(self.accept)
footer_layout.addWidget(self.finish_button)
social_layout =QHBoxLayout ()
social_layout .setSpacing (10 )
social_layout .addWidget (self .github_button )
social_layout .addWidget (self .instagram_button )
social_layout .addWidget (self .Discord_button )
while buttons_layout .count ():
item =buttons_layout .takeAt (0 )
if item .widget ():
item .widget ().setParent (None )
elif item .layout ():
pass
buttons_layout .addLayout (social_layout )
buttons_layout .addStretch (1 )
buttons_layout .addWidget (self .back_button )
buttons_layout .addWidget (self .next_button )
main_layout .addLayout (buttons_layout )
self ._update_button_states ()
def _next_step_action (self ):
if self .current_step <len (self .tour_steps_widgets )-1 :
self .current_step +=1
self .stacked_widget .setCurrentIndex (self .current_step )
else :
self .accept ()
self ._update_button_states ()
def _previous_step (self ):
if self .current_step >0 :
self .current_step -=1
self .stacked_widget .setCurrentIndex (self .current_step )
self ._update_button_states ()
def _update_button_states (self ):
if self .current_step ==len (self .tour_steps_widgets )-1 :
self .next_button .setText (self ._tr ("tour_dialog_finish_button","Finish"))
else :
self .next_button .setText (self ._tr ("tour_dialog_next_button","Next"))
self .back_button .setEnabled (self .current_step >0 )
def _open_github_link (self ):
QDesktopServices .openUrl (QUrl ("https://github.com/Yuvi9587"))
def _open_instagram_link (self ):
QDesktopServices .openUrl (QUrl ("https://www.instagram.com/uvi.arts/"))
def _open_Discord_link (self ):
QDesktopServices .openUrl (QUrl ("https://discord.gg/BqP64XTdJN"))
main_layout.addLayout(footer_layout)

View File

@@ -24,7 +24,7 @@ class MoreOptionsDialog(QDialog):
layout.addWidget(self.description_label)
self.radio_button_group = QButtonGroup(self)
self.radio_content = QRadioButton("Description/Content")
self.radio_comments = QRadioButton("Comments (Not Working)")
self.radio_comments = QRadioButton("Comments")
self.radio_button_group.addButton(self.radio_content)
self.radio_button_group.addButton(self.radio_comments)
layout.addWidget(self.radio_content)

View File

@@ -0,0 +1,118 @@
# multipart_scope_dialog.py
from PyQt5.QtWidgets import (
QDialog, QVBoxLayout, QGroupBox, QRadioButton, QDialogButtonBox, QButtonGroup,
QLabel, QLineEdit, QHBoxLayout, QFrame
)
from PyQt5.QtGui import QIntValidator
from PyQt5.QtCore import Qt
# It's good practice to get this constant from the source
# but for this example, we will define it here.
MAX_PARTS = 16
class MultipartScopeDialog(QDialog):
"""
A dialog to let the user select the scope, number of parts, and minimum size for multipart downloads.
"""
SCOPE_VIDEOS = 'videos'
SCOPE_ARCHIVES = 'archives'
SCOPE_BOTH = 'both'
def __init__(self, current_scope='both', current_parts=4, current_min_size_mb=100, parent=None):
super().__init__(parent)
self.setWindowTitle("Multipart Download Options")
self.setWindowFlags(self.windowFlags() & ~Qt.WindowContextHelpButtonHint)
self.setMinimumWidth(350)
# Main Layout
layout = QVBoxLayout(self)
# --- Options Group for Scope ---
self.options_group_box = QGroupBox("Apply multipart downloads to:")
options_layout = QVBoxLayout()
# ... (Radio buttons and button group code remains unchanged) ...
self.radio_videos = QRadioButton("Videos Only")
self.radio_archives = QRadioButton("Archives Only (.zip, .rar, etc.)")
self.radio_both = QRadioButton("Both Videos and Archives")
if current_scope == self.SCOPE_VIDEOS:
self.radio_videos.setChecked(True)
elif current_scope == self.SCOPE_ARCHIVES:
self.radio_archives.setChecked(True)
else:
self.radio_both.setChecked(True)
self.button_group = QButtonGroup(self)
self.button_group.addButton(self.radio_videos)
self.button_group.addButton(self.radio_archives)
self.button_group.addButton(self.radio_both)
options_layout.addWidget(self.radio_videos)
options_layout.addWidget(self.radio_archives)
options_layout.addWidget(self.radio_both)
self.options_group_box.setLayout(options_layout)
layout.addWidget(self.options_group_box)
# --- START: MODIFIED Download Settings Group ---
self.settings_group_box = QGroupBox("Download settings:")
settings_layout = QVBoxLayout()
# Layout for Parts count
parts_layout = QHBoxLayout()
self.parts_label = QLabel("Number of download parts per file:")
self.parts_input = QLineEdit(str(current_parts))
self.parts_input.setValidator(QIntValidator(2, MAX_PARTS, self))
self.parts_input.setFixedWidth(40)
self.parts_input.setToolTip(f"Set the number of concurrent connections per file (2-{MAX_PARTS}).")
parts_layout.addWidget(self.parts_label)
parts_layout.addStretch()
parts_layout.addWidget(self.parts_input)
settings_layout.addLayout(parts_layout)
# Layout for Minimum Size
size_layout = QHBoxLayout()
self.size_label = QLabel("Minimum file size for multipart (MB):")
self.size_input = QLineEdit(str(current_min_size_mb))
self.size_input.setValidator(QIntValidator(10, 10000, self)) # Min 10MB, Max ~10GB
self.size_input.setFixedWidth(40)
self.size_input.setToolTip("Files smaller than this will use a normal, single-part download.")
size_layout.addWidget(self.size_label)
size_layout.addStretch()
size_layout.addWidget(self.size_input)
settings_layout.addLayout(size_layout)
self.settings_group_box.setLayout(settings_layout)
layout.addWidget(self.settings_group_box)
# --- END: MODIFIED Download Settings Group ---
# OK and Cancel Buttons
self.button_box = QDialogButtonBox(QDialogButtonBox.Ok | QDialogButtonBox.Cancel)
self.button_box.accepted.connect(self.accept)
self.button_box.rejected.connect(self.reject)
layout.addWidget(self.button_box)
self.setLayout(layout)
def get_selected_scope(self):
# ... (This method remains unchanged) ...
if self.radio_videos.isChecked():
return self.SCOPE_VIDEOS
if self.radio_archives.isChecked():
return self.SCOPE_ARCHIVES
return self.SCOPE_BOTH
def get_selected_parts(self):
# ... (This method remains unchanged) ...
try:
parts = int(self.parts_input.text())
return max(2, min(parts, MAX_PARTS))
except (ValueError, TypeError):
return 4
def get_selected_min_size(self):
"""Returns the selected minimum size in MB as an integer."""
try:
size = int(self.size_input.text())
return max(10, min(size, 10000)) # Enforce valid range
except (ValueError, TypeError):
return 100 # Return a safe default

View File

@@ -3,8 +3,27 @@ import re
try:
from fpdf import FPDF
FPDF_AVAILABLE = True
# --- FIX: Move the class definition inside the try block ---
class PDF(FPDF):
"""Custom PDF class to handle headers and footers."""
def header(self):
pass
def footer(self):
self.set_y(-15)
if self.font_family:
self.set_font(self.font_family, '', 8)
else:
self.set_font('Arial', '', 8)
self.cell(0, 10, 'Page ' + str(self.page_no()), 0, 0, 'C')
except ImportError:
FPDF_AVAILABLE = False
# If the import fails, FPDF and PDF will not be defined,
# but the program won't crash here.
FPDF = None
PDF = None
def strip_html_tags(text):
if not text:
@@ -12,19 +31,6 @@ def strip_html_tags(text):
clean = re.compile('<.*?>')
return re.sub(clean, '', text)
class PDF(FPDF):
"""Custom PDF class to handle headers and footers."""
def header(self):
pass
def footer(self):
self.set_y(-15)
if self.font_family:
self.set_font(self.font_family, '', 8)
else:
self.set_font('Arial', '', 8)
self.cell(0, 10, 'Page ' + str(self.page_no()), 0, 0, 'C')
def create_single_pdf_from_content(posts_data, output_filename, font_path, logger=print):
"""
Creates a single, continuous PDF, correctly formatting both descriptions and comments.
@@ -68,7 +74,7 @@ def create_single_pdf_from_content(posts_data, output_filename, font_path, logge
pdf.ln(10)
pdf.set_font(default_font_family, 'B', 16)
pdf.multi_cell(w=0, h=10, text=post.get('title', 'Untitled Post'), align='L')
pdf.multi_cell(w=0, h=10, txt=post.get('title', 'Untitled Post'), align='L')
pdf.ln(5)
if 'comments' in post and post['comments']:
@@ -89,7 +95,7 @@ def create_single_pdf_from_content(posts_data, output_filename, font_path, logge
pdf.ln(10)
pdf.set_font(default_font_family, '', 11)
pdf.multi_cell(0, 7, body)
pdf.multi_cell(w=0, h=7, txt=body)
if comment_index < len(comments_list) - 1:
pdf.ln(3)
@@ -97,7 +103,7 @@ def create_single_pdf_from_content(posts_data, output_filename, font_path, logge
pdf.ln(3)
elif 'content' in post:
pdf.set_font(default_font_family, '', 12)
pdf.multi_cell(w=0, h=7, text=post.get('content', 'No Content'))
pdf.multi_cell(w=0, h=7, txt=post.get('content', 'No Content'))
try:
pdf.output(output_filename)
@@ -105,4 +111,4 @@ def create_single_pdf_from_content(posts_data, output_filename, font_path, logge
return True
except Exception as e:
logger(f"❌ A critical error occurred while saving the final PDF: {e}")
return False
return False

View File

@@ -0,0 +1,160 @@
import os
import re
import datetime
import time
try:
from fpdf import FPDF
FPDF_AVAILABLE = True
class PDF(FPDF):
"""Custom PDF class for Discord chat logs."""
def __init__(self, server_name, channel_name, *args, **kwargs):
super().__init__(*args, **kwargs)
self.server_name = server_name
self.channel_name = channel_name
self.default_font_family = 'DejaVu' # Can be changed to Arial if font fails
def header(self):
if self.page_no() == 1:
return # No header on the title page
self.set_font(self.default_font_family, '', 8)
self.cell(0, 10, f'{self.server_name} - #{self.channel_name}', 0, 0, 'L')
self.cell(0, 10, 'Page ' + str(self.page_no()), 0, 0, 'R')
self.ln(10)
def footer(self):
pass # No footer needed, header has page number
except ImportError:
FPDF_AVAILABLE = False
FPDF = None
PDF = None
def create_pdf_from_discord_messages(messages_data, server_name, channel_name, output_filename, font_path, logger=print, cancellation_event=None, pause_event=None):
"""
Creates a single PDF from a list of Discord message objects, formatted as a chat log.
UPDATED to include clickable links for attachments and embeds.
"""
if not FPDF_AVAILABLE:
logger("❌ PDF Creation failed: 'fpdf2' library is not installed.")
return False
if not messages_data:
logger(" No messages were found or fetched to create a PDF.")
return False
# --- FIX: This helper function now correctly accepts and checks the event objects ---
def check_events(c_event, p_event):
"""Helper to safely check for pause and cancel events."""
if c_event and hasattr(c_event, 'is_cancelled') and c_event.is_cancelled:
return True # Stop
if p_event and hasattr(p_event, 'is_paused'):
while p_event.is_paused:
time.sleep(0.5)
if c_event and hasattr(c_event, 'is_cancelled') and c_event.is_cancelled:
return True
return False
logger(" Sorting messages by date (oldest first)...")
messages_data.sort(key=lambda m: m.get('published', m.get('timestamp', '')))
pdf = PDF(server_name, channel_name)
default_font_family = 'DejaVu'
try:
bold_font_path = font_path.replace("DejaVuSans.ttf", "DejaVuSans-Bold.ttf")
if not os.path.exists(font_path) or not os.path.exists(bold_font_path):
raise RuntimeError("Font files not found")
pdf.add_font('DejaVu', '', font_path, uni=True)
pdf.add_font('DejaVu', 'B', bold_font_path, uni=True)
except Exception as font_error:
logger(f" ⚠️ Could not load DejaVu font: {font_error}. Falling back to Arial.")
default_font_family = 'Arial'
pdf.default_font_family = 'Arial'
# --- Title Page ---
pdf.add_page()
pdf.set_font(default_font_family, 'B', 24)
pdf.cell(w=0, h=20, text="Discord Chat Log", align='C', new_x="LMARGIN", new_y="NEXT")
pdf.ln(10)
pdf.set_font(default_font_family, '', 16)
pdf.cell(w=0, h=10, text=f"Server: {server_name}", align='C', new_x="LMARGIN", new_y="NEXT")
pdf.cell(w=0, h=10, text=f"Channel: #{channel_name}", align='C', new_x="LMARGIN", new_y="NEXT")
pdf.ln(5)
pdf.set_font(default_font_family, '', 10)
pdf.cell(w=0, h=10, text=f"Generated on: {datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')}", align='C', new_x="LMARGIN", new_y="NEXT")
pdf.cell(w=0, h=10, text=f"Total Messages: {len(messages_data)}", align='C', new_x="LMARGIN", new_y="NEXT")
pdf.add_page()
logger(f" Starting PDF creation with {len(messages_data)} messages...")
for i, message in enumerate(messages_data):
# --- FIX: Pass the event objects to the helper function ---
if i % 50 == 0:
if check_events(cancellation_event, pause_event):
logger(" PDF generation cancelled by user.")
return False
author = message.get('author', {}).get('global_name') or message.get('author', {}).get('username', 'Unknown User')
timestamp_str = message.get('published', message.get('timestamp', ''))
content = message.get('content', '')
attachments = message.get('attachments', [])
embeds = message.get('embeds', [])
try:
if timestamp_str.endswith('Z'):
timestamp_str = timestamp_str[:-1] + '+00:00'
dt_obj = datetime.datetime.fromisoformat(timestamp_str)
formatted_timestamp = dt_obj.strftime('%Y-%m-%d %H:%M:%S')
except (ValueError, TypeError):
formatted_timestamp = timestamp_str
if i > 0:
pdf.ln(2)
pdf.set_draw_color(200, 200, 200)
pdf.cell(0, 0, '', border='T')
pdf.ln(2)
pdf.set_font(default_font_family, 'B', 11)
pdf.write(5, f"{author} ")
pdf.set_font(default_font_family, '', 9)
pdf.set_text_color(128, 128, 128)
pdf.write(5, f"({formatted_timestamp})")
pdf.set_text_color(0, 0, 0)
pdf.ln(6)
if content:
pdf.set_font(default_font_family, '', 10)
pdf.multi_cell(w=0, h=5, text=content)
if attachments or embeds:
pdf.ln(1)
pdf.set_font(default_font_family, '', 9)
pdf.set_text_color(22, 119, 219)
for att in attachments:
file_name = att.get('filename', 'untitled')
full_url = att.get('url', '#')
pdf.write(5, text=f"[Attachment: {file_name}]", link=full_url)
pdf.ln()
for embed in embeds:
embed_url = embed.get('url', 'no url')
pdf.write(5, text=f"[Embed: {embed_url}]", link=embed_url)
pdf.ln()
pdf.set_text_color(0, 0, 0)
if check_events(cancellation_event, pause_event):
logger(" PDF generation cancelled by user before final save.")
return False
try:
pdf.output(output_filename)
logger(f"✅ Successfully created Discord chat log PDF: '{os.path.basename(output_filename)}'")
return True
except Exception as e:
logger(f"❌ A critical error occurred while saving the final PDF: {e}")
return False

File diff suppressed because it is too large Load Diff

View File

@@ -141,28 +141,51 @@ def prepare_cookies_for_request(use_cookie_flag, cookie_text_input, selected_coo
def extract_post_info(url_string):
"""
Parses a URL string to extract the service, user ID, and post ID.
Args:
url_string (str): The URL to parse.
Returns:
tuple: A tuple containing (service, user_id, post_id). Any can be None.
UPDATED to support Hentai2Read series and chapters.
"""
if not isinstance(url_string, str) or not url_string.strip():
return None, None, None
stripped_url = url_string.strip()
# --- Bunkr Check ---
bunkr_pattern = re.compile(
r"(?:https?://)?(?:[a-zA-Z0-9-]+\.)?bunkr\.(?:si|la|ws|red|black|media|site|is|to|ac|cr|ci|fi|pk|ps|sk|ph|su|ru)|bunkrr\.ru"
)
if bunkr_pattern.search(stripped_url):
return 'bunkr', stripped_url, None
# --- nhentai Check ---
nhentai_match = re.search(r'nhentai\.net/g/(\d+)', stripped_url)
if nhentai_match:
return 'nhentai', nhentai_match.group(1), None
# --- Hentai2Read Check (Updated) ---
# This regex now captures the manga slug (id1) and optionally the chapter number (id2)
hentai2read_match = re.search(r'hentai2read\.com/([^/]+)(?:/(\d+))?/?', stripped_url)
if hentai2read_match:
manga_slug, chapter_num = hentai2read_match.groups()
return 'hentai2read', manga_slug, chapter_num # chapter_num will be None for series URLs
discord_channel_match = re.search(r'discord\.com/channels/(@me|\d+)/(\d+)', stripped_url)
if discord_channel_match:
server_id, channel_id = discord_channel_match.groups()
return 'discord', server_id, channel_id
# --- Kemono/Coomer/Discord Parsing ---
try:
parsed_url = urlparse(url_string.strip())
parsed_url = urlparse(stripped_url)
path_parts = [part for part in parsed_url.path.strip('/').split('/') if part]
# Standard format: /<service>/user/<user_id>/post/<post_id>
if len(path_parts) >= 3 and path_parts[0].lower() == 'discord' and path_parts[1].lower() == 'server':
return 'discord', path_parts[2], path_parts[3] if len(path_parts) >= 4 else None
if len(path_parts) >= 3 and path_parts[1].lower() == 'user':
service = path_parts[0]
user_id = path_parts[2]
post_id = path_parts[4] if len(path_parts) >= 5 and path_parts[3].lower() == 'post' else None
return service, user_id, post_id
# API format: /api/v1/<service>/user/<user_id>...
if len(path_parts) >= 5 and path_parts[0:2] == ['api', 'v1'] and path_parts[3].lower() == 'user':
service = path_parts[2]
user_id = path_parts[4]
@@ -173,8 +196,7 @@ def extract_post_info(url_string):
print(f"Debug: Exception during URL parsing for '{url_string}': {e}")
return None, None, None
def get_link_platform(url):
"""
Identifies the platform of a given URL based on its domain.
@@ -196,10 +218,9 @@ def get_link_platform(url):
if 'twitter.com' in domain or 'x.com' in domain: return 'twitter/x'
if 'discord.gg' in domain or 'discord.com/invite' in domain: return 'discord invite'
if 'pixiv.net' in domain: return 'pixiv'
if 'kemono.su' in domain or 'kemono.party' in domain: return 'kemono'
if 'coomer.su' in domain or 'coomer.party' in domain: return 'coomer'
if 'kemono.su' in domain or 'kemono.party' in domain or 'kemono.cr' in domain: return 'kemono'
if 'coomer.su' in domain or 'coomer.party' in domain or 'coomer.st' in domain: return 'coomer'
# Fallback to a generic name for other domains
parts = domain.split('.')
if len(parts) >= 2:
return parts[-2]

View File

@@ -239,16 +239,23 @@ def setup_ui(main_app):
checkboxes_group_layout.addWidget(advanced_settings_label)
advanced_row1_layout = QHBoxLayout()
advanced_row1_layout.setSpacing(10)
main_app.use_subfolders_checkbox = QCheckBox("Separate Folders by Known.txt")
main_app.use_subfolders_checkbox.setChecked(True)
main_app.use_subfolders_checkbox.toggled.connect(main_app.update_ui_for_subfolders)
advanced_row1_layout.addWidget(main_app.use_subfolders_checkbox)
# --- REORDERED CHECKBOXES ---
main_app.use_subfolder_per_post_checkbox = QCheckBox("Subfolder per Post")
main_app.use_subfolder_per_post_checkbox.toggled.connect(main_app.update_ui_for_subfolders)
main_app.use_subfolder_per_post_checkbox.setChecked(True)
advanced_row1_layout.addWidget(main_app.use_subfolder_per_post_checkbox)
main_app.date_prefix_checkbox = QCheckBox("Date Prefix")
main_app.date_prefix_checkbox.setToolTip("When 'Subfolder per Post' is active, prefix the folder name with the post's upload date.")
advanced_row1_layout.addWidget(main_app.date_prefix_checkbox)
main_app.use_subfolders_checkbox = QCheckBox("Separate Folders by Known.txt")
main_app.use_subfolders_checkbox.setChecked(False)
main_app.use_subfolders_checkbox.toggled.connect(main_app.update_ui_for_subfolders)
advanced_row1_layout.addWidget(main_app.use_subfolders_checkbox)
# --- END REORDER ---
main_app.use_cookie_checkbox = QCheckBox("Use Cookie")
main_app.use_cookie_checkbox.setChecked(main_app.use_cookie_setting)
main_app.cookie_text_input = QLineEdit()
@@ -277,7 +284,7 @@ def setup_ui(main_app):
advanced_row2_layout.addLayout(multithreading_layout)
main_app.external_links_checkbox = QCheckBox("Show External Links in Log")
advanced_row2_layout.addWidget(main_app.external_links_checkbox)
main_app.manga_mode_checkbox = QCheckBox("Manga/Comic Mode")
main_app.manga_mode_checkbox = QCheckBox("Renaming Mode")
advanced_row2_layout.addWidget(main_app.manga_mode_checkbox)
advanced_row2_layout.addStretch(1)
checkboxes_group_layout.addLayout(advanced_row2_layout)
@@ -380,10 +387,27 @@ def setup_ui(main_app):
main_app.link_search_input.setPlaceholderText("Search Links...")
main_app.link_search_input.setVisible(False)
log_title_layout.addWidget(main_app.link_search_input)
main_app.link_search_button = QPushButton("🔍")
main_app.link_search_button = QPushButton("<EFBFBD>")
main_app.link_search_button.setVisible(False)
main_app.link_search_button.setFixedWidth(int(30 * scale))
log_title_layout.addWidget(main_app.link_search_button)
discord_controls_layout = QHBoxLayout()
main_app.discord_scope_toggle_button = QPushButton("Scope: Files")
main_app.discord_scope_toggle_button.setVisible(False) # Hidden by default
discord_controls_layout.addWidget(main_app.discord_scope_toggle_button)
main_app.discord_message_limit_input = QLineEdit(main_app)
main_app.discord_message_limit_input.setPlaceholderText("Msg Limit")
main_app.discord_message_limit_input.setToolTip("Optional: Limit the number of recent messages to process.")
main_app.discord_message_limit_input.setValidator(QIntValidator(1, 9999999, main_app))
main_app.discord_message_limit_input.setFixedWidth(int(80 * scale))
main_app.discord_message_limit_input.setVisible(False) # Hide it by default
discord_controls_layout.addWidget(main_app.discord_message_limit_input)
log_title_layout.addLayout(discord_controls_layout)
main_app.manga_rename_toggle_button = QPushButton()
main_app.manga_rename_toggle_button.setVisible(False)
main_app.manga_rename_toggle_button.setFixedWidth(int(140 * scale))