v3.2.3

Add settings for maximum attempts
Enhance posts rename logic
Dependencies update
Switch reactive from rx-java to reactor

.gitignore

@@ -3,4 +3,5 @@
 /**/*/target
 /**/*/build-dir
 /**/*/java-runtime
-.idea
+.idea
+.vripper

CHANGELOG.md

@@ -1,5 +1,143 @@
 # Changelog
 
+## [3.2.3] - 2021-01-13
+### Changed
+- Finished posts are not cleared from the UI
+
+## [3.2.2] - 2021-01-13
+### Changed
+- Set total download to max value
+
+## [3.2.1] - 2021-01-12
+### Added
+- Add settings for connection timeout
+- Add settings for maximum attempts
+### Changed
+- Enhance posts rename logic
+- Dependencies update
+- Switch reactive from rx-java to reactor
+
+## [3.2.0] - 2020-11-16
+### Changed
+- Fix build issue
+
+## [3.1.0] - 2020-11-16
+### Changed
+- UI revamp
+- Fix bugs
+
+## [3.0.11] - 2020-10-31
+### Added
+- Support for vipr.im
+
+## [3.0.10] - 2020-10-04
+### Changed
+- Fix bug on settings update
+
+## [3.0.9] - 2020-09-27
+### Changed
+- Revert to appending post id to download folders
+- Fix a bug that lead to adding duplicate posts
+- Remove old unused settings
+ 
+## [3.0.8] - 2020-08-22
+### Changed
+- Bug fix which breaks concurrent downloads
+
+## [3.0.7] - 2020-08-22
+### Changed
+- Fix MacOs build
+
+## [3.0.6] - 2020-08-18
+### Changed
+- Fix TurboImageHost
+
+## [3.0.5] - 2020-08-17
+### Changed
+- Update packages
+- Host class refactoring
+
+## [3.0.4] - 2020-08-16
+### Changed
+- Improve rename UI
+
+## [3.0.3] - 2020-08-16
+### Changed
+- Fix bugs with electron app
+
+## [3.0.2] - 2020-08-15
+### Changed
+- Fix bugs with download queue
+
+## [3.0.1] - 2020-08-03
+### Changed
+- Fix bugs with download queue
+
+## [3.0.0] - 2020-08-03
+### Changed
+- Major rewrites
+- Introducing a proper database for persistence
+
+## [2.12.1] - 2020-05-22
+### Changed
+- Fix alternative name bug
+
+## [2.12.0] - 2020-05-21
+### Changed
+- Use new user hash from the api
+
+## [2.11.7] - 2020-05-08
+### Changed
+- Fix bug with paths creation
+- Move Open Link button
+- Rename rename all title
+
+## [2.11.6] - 2020-05-04
+### Changed
+- PNG filenames are now correctly named
+- Fix issue with special characters in post titles
+
+## [2.11.5] - 2020-05-03
+### Changed
+- Reduce rows buffer size
+
+## [2.11.4] - 2020-05-03
+### Changed
+- Code cleanup
+- Dependencies update
+
+## [2.11.3] - 2020-04-30
+### Changed
+- Source code restructure
+- UI fixes and enhancements
+### Added
+- Add menu entry to rename a gallery to first alternative title
+- Add menu on toolbar for massive rename to first alternative title
+
+## [2.11.2] - 2020-04-26
+### Changed
+- Bug fixes
+
+## [2.11.1] - 2020-04-25
+### Changed
+- Code source maintenance
+### Added
+- Added option to rename gallery
+
+## [2.11.0] - 2020-03-29
+### Changed
+- Upgrade Electron to v8
+- Upgrade Angular to v9
+- App settings are now portable in a json file
+- General code source maintenance
+### Added
+- Added Support for pixroute.com
+- Added grab post title from thread option
+
+## [2.10.8] - 2020-01-25
+### Changed
+- Fix app url
+
 ## [2.10.7] - 2020-01-25
 ### Changed
 - Drop SockJs and switch to WebSocket
@@ -313,4 +451,4 @@
 - Add real time communication for progress updates via websocket (SockJS)
 - Add settings for download path, max concurrent download, enable/disable integration with vipergirls
 - Add integration with vipergitls.to (possibility to auto leave thanks after grab post)
-- Add support for acidimg.cc, imagezilla.net, imgbox.com, imx.to, pixhost.to, turboimagehost.com
+- Add support for acidimg.cc, imagezilla.net, imgbox.com, imx.to, pixhost.to, turboimagehost.com

README.md

@@ -1,26 +1,22 @@
 # Viper Ripper!
 
-This is my spin for a gallery ripper app for [vipergirls](https://vipergirls.to) website.
+This is my spin for a cross platform gallery ripper app for [vipergirls](https://vipergirls.to) website.
 The purpose of this project is to build a robust and clean application to conveniently download photo galleries using modern web technologies, Java + Spring boot for the back end and angular + electron for the front end.
 
 ## How to build
 You need a recent version of maven 3.6.1+.
 
 The application support 2 build modes:
- - Desktop app (*Partially cross platform): Self contained app, built with electron, depends on Java runtimes.
- - Server app (Fully cross platform): Depends on Java runtimes, you need a web browser to access the app UI.
+ - Desktop app: Self contained app, built with electron includes Java runtimes.
+ - Server app: Depends on Java runtimes, you need a web browser to access the app UI.
 
 Most people will be interested only on the desktop app, however if you have a nas or a server, the server app is there for you.
 
 To build the server app:
 
-    mvn clean install
+    mvn clean install -DskipTests
 To build the desktop app:
 
-    mvn clean install -Pelectron
+    mvn clean install -Pelectron -DskipTests
 
 Maven will automatically handle front end compilation. However, for development, you will need to install a recent version of nodejs on your system.
-
-*Partially cross platform:
-Technically cross platform, but the Desktop app relies on electron builder to make appropriate binaries for each platform. mac binaries can only be generated on a mac, and i don't own one. So i can only provide binaries for windows and linux.
-The Server app however depends only on Java, so it will work just fine on a mac, you only need a browser to access it.

pom.xml

@@ -4,12 +4,12 @@
     <modelVersion>4.0.0</modelVersion>
     <groupId>tn.mnlr</groupId>
     <artifactId>vripper</artifactId>
-    <version>2.10.7</version>
+    <version>3.2.3</version>
     <packaging>pom</packaging>
     <parent>
         <groupId>org.springframework.boot</groupId>
         <artifactId>spring-boot-starter-parent</artifactId>
-        <version>2.1.1.RELEASE</version>
+        <version>2.4.1</version>
         <relativePath/> <!-- lookup parent from repository -->
     </parent>
     <modules>
@@ -17,4 +17,4 @@
         <module>vripper-ui</module>
         <module>vripper-electron</module>
     </modules>
-</project>
+</project>

vripper-electron/main.js

@@ -1,12 +1,16 @@
-const { app, BrowserWindow } = require("electron");
+require('v8-compile-cache');
+const {app, BrowserWindow} = require("electron");
 const path = require("path");
 const url = require("url");
 const getPort = require("get-port");
-const { spawn } = require("child_process");
-const { ipcMain } = require("electron");
-const { dialog } = require("electron");
+const {spawn} = require("child_process");
+const {ipcMain} = require("electron");
+const {dialog} = require("electron");
 const axios = require('axios');
-const appDir = process.env.APPDIR;
+
+// non null value when it is an AppImage
+const appImageDir = process.env.APPDIR;
+const appImagePath = process.env.APPIMAGE;
 
 let win;
 let vripperServer;
@@ -18,139 +22,150 @@ let terminationInteval;
 let terminated = false;
 
 process.on("uncaughtException", err => {
-  dialog.showErrorBox(err.message, err.stack);
-  process.exit(1);
+    dialog.showErrorBox(err.message, err.stack);
+    process.exit(1);
 });
 
-function createWindow() {
+createWindow = () => {
 
-  if (process.platform === 'win32') {
-    app.setAppUserModelId("tn.mnlr.vripper");
-  }
+    if (process.platform === 'win32') {
+        app.setAppUserModelId("tn.mnlr.vripper");
+    }
 
-  let icon;
-  if(process.platform === "win32") {
-    icon = __dirname + '/icon.ico';
-  } else {
-    icon = __dirname + '/icon.png';
-  }
-  win = new BrowserWindow({
-    width: 1024,
-    height: 800,
-    minWidth: 800,
-    minHeight: 600,
-    webPreferences: {
-      nodeIntegration: true
-    },
-    icon: icon
-  });
+    let icon;
+    if (process.platform === "win32") {
+        icon = __dirname + '/icon.ico';
+    } else {
+        icon = __dirname + '/icon.png';
+    }
+    win = new BrowserWindow({
+        width: 1024,
+        height: 800,
+        minWidth: 800,
+        minHeight: 600,
+        webPreferences: {
+            nodeIntegration: true,
+            enableRemoteModule: true
+        },
+        icon: icon
+    });
 
-  win.removeMenu();
-  win.setMenu(null);
-  win.setMenuBarVisibility(false);
+    win.removeMenu();
+    win.setMenu(null);
+    win.setMenuBarVisibility(false);
 
-  win.loadURL(
-    url.format({
-      pathname: path.join(__dirname, `vripper-ui/index.html`),
-      protocol: "file:",
-      slashes: true
-    })
-  );
+    win.loadURL(
+        url.format({
+            pathname: path.join(__dirname, `vripper-ui/index.html`),
+            protocol: "file:",
+            slashes: true
+        })
+    );
 
-  // win.webContents.openDevTools();
+    // win.webContents.openDevTools();
 
-  win.on("closed", () => {
-    win = null;
-  });
+    win.on("closed", () => {
+        win = null;
+    });
 }
 
-const gotTheLock = app.requestSingleInstanceLock();
-
-if (!gotTheLock) {
-  app.quit();
-} else {
-  getPort().then(port => {
-    serverPort = port;
-    ipcMain.on("get-port", event => {
-      event.reply("port", port);
-    });
-    let javaBinPath;
-    if(appDir !== undefined) {
-      javaBinPath = path.join(appDir, "java-runtime/bin/java");
-    } else {
-      if(process.platform === 'darwin') {
-        javaBinPath = path.join(app.getPath('exe'), "../../java-runtime/bin/java");
-      } else {
-        javaBinPath = path.join(app.getPath('exe'), "../java-runtime/bin/java");
-      }
-    }
-    let jarPath;
-    if(appDir !== undefined) {
-      jarPath = path.join(appDir, "bin/vripper-server.jar");
-    } else {
-      if(process.platform === 'darwin') {
-        jarPath = path.join(app.getPath('exe'), "../../bin/vripper-server.jar");
-      } else {
-        jarPath = path.join(app.getPath('exe'), "../bin/vripper-server.jar");
-      }
-    }
-    vripperServer = spawn(javaBinPath, [
-      "-Xms256m",
-      "-Dvripper.server.port=" + port,
-      "-jar",
-      jarPath
-    ], {
-      stdio: 'ignore'
-    });
-    vripperServer.on('exit', (code, signal) => {
-      console.log(`vripper server terminated, code = ${code}, signal = ${signal}`);
-      terminated = true;
-      app.quit();
-    });
-  });
-  
-  app.on("second-instance", (event, commandLine, workingDirectory) => {
-    if (win) {
-      if (win.isMinimized()) win.restore();
-      win.focus();
-    }
-  });
-
-  app.on("ready", createWindow);
-
-  app.on("window-all-closed", () => {
-    if (process.platform !== "darwin") {
-      axios.post('http://localhost:' + serverPort + '/actuator/shutdown', {}, {
-        headers: { 'content-type': 'application/json' },
-      }).then((response) => {
-        terminationInteval=  setInterval(() => {
-          terminationAttemps++;
-          if(terminated) {
-            console.log('viper server terminated');
-            clearInterval(terminationInteval);
-            app.quit();
-          } else if(terminationAttemps > maxTerminationAttemps) {
-            console.log('viper server is not terminated');
-            console.log('Proceed to kill');
-            vripperServer.kill('SIGKILL');
-            clearInterval(terminationInteval);
-            app.quit();
-          }
+shutdownServer = () => {
+    axios.post('http://localhost:' + serverPort + '/actuator/shutdown', {}, {
+        headers: {'content-type': 'application/json'},
+    }).then((response) => {
+        terminationInteval = setInterval(() => {
+            terminationAttemps++;
+            if (terminated) {
+                console.log('viper server terminated');
+                clearInterval(terminationInteval);
+                app.quit();
+            } else if (terminationAttemps > maxTerminationAttemps) {
+                console.log('viper server is not terminated');
+                console.log('Proceed to kill');
+                vripperServer.kill('SIGKILL');
+                clearInterval(terminationInteval);
+                app.quit();
+            }
         }, 1000);
-      })
-      .catch((error) => {
+    }).catch((error) => {
         // Terminate immediately
         console.log(error);
         vripperServer.kill('SIGKILL');
         terminated = true;
         app.quit();
-      });
-    }
-  });
-
-  app.on("activate", () => {
-    if (win === null) {
-      createWindow();
-    }
-  });
+    });
+}
+
+const gotTheLock = app.requestSingleInstanceLock();
+
+if (!gotTheLock) {
+    app.quit();
+} else {
+    getPort().then(port => {
+        serverPort = port;
+        ipcMain.on("get-port", event => {
+            event.reply("port", port);
+        });
+
+        const appPath = path.join(app.getAppPath(), '../../');
+        let javaBinPath, jarPath, baseDir;
+        if (appImageDir !== undefined && process.platform === 'linux') {
+            javaBinPath = path.join(appImageDir, "java-runtime/bin/java");
+            jarPath = path.join(appImageDir, "bin/vripper-server.jar");
+            baseDir = path.join(appImagePath, '..');
+        } else if (process.platform === 'darwin') {
+            javaBinPath = path.join(appPath, "java-runtime/bin/java");
+            jarPath = path.join(appPath, "bin/vripper-server.jar");
+            baseDir = app.getPath('appData');
+        } else if (process.platform === 'win32') {
+            javaBinPath = path.join(appPath, "java-runtime/bin/java");
+            jarPath = path.join(appPath, "bin/vripper-server.jar");
+            baseDir = appPath;
+        } else {
+            console.error(`Unknown platform ${process.platform}`);
+            app.quit();
+        }
+
+        vripperServer = spawn(javaBinPath, [
+            "-Xms256m",
+            "-Dvripper.server.port=" + port,
+            "-Dbase.dir=" + baseDir,
+            "-jar",
+            jarPath
+        ], {
+            stdio: 'ignore'
+        });
+        vripperServer.on('exit', (code, signal) => {
+            console.log(`vripper server terminated, code = ${code}, signal = ${signal}`);
+            terminated = true;
+            app.quit();
+        });
+    });
+
+    app.on("second-instance", (event, commandLine, workingDirectory) => {
+        if (win) {
+            if (win.isMinimized()) win.restore();
+            win.focus();
+        }
+    });
+
+    app.on("ready", createWindow);
+
+    app.on("window-all-closed", () => {
+        if (process.platform !== "darwin") {
+            shutdownServer();
+        }
+    });
+
+    app.on("will-quit", () => {
+        if (process.platform === "darwin") {
+            shutdownServer();
+        }
+    });
+
+    app.on("activate", () => {
+        if (win === null) {
+            createWindow();
+        }
+    });
 }

vripper-electron/package.json

@@ -1,6 +1,6 @@
 {
   "name": "vripper-electron",
-  "version": "2.10.7",
+  "version": "3.2.3",
   "description": "A ripper for vipergirls.to built using web technolgies",
   "main": "main.js",
   "author": "death-claw <53543762+death-claw@users.noreply.github.com>",
@@ -40,14 +40,9 @@
     "win": {
       "icon": "icon.ico",
       "target": [
-        "nsis"
+        "dir"
       ]
     },
-    "nsis": {
-      "createDesktopShortcut": "always",
-      "oneClick": false,
-      "perMachine": false
-    },
     "linux": {
       "synopsis": "vipergirls.to ripper",
       "category": "Utility",
@@ -70,15 +65,16 @@
     "dist": "node pre-build.js && electron-builder"
   },
   "devDependencies": {
-    "electron": "^7.1.9",
-    "electron-builder": "^22.2.0"
+    "electron": "11.1.1",
+    "electron-builder": "22.9.1"
   },
   "dependencies": {
-    "axios": "^0.19.0",
-    "cheerio": "^1.0.0-rc.3",
-    "copy-dir": "^1.1.0",
-    "electron-context-menu": "^0.13.0",
-    "get-port": "^5.0.0",
-    "rimraf": "^2.6.3"
+    "axios": "0.21.1",
+    "cheerio": "1.0.0-rc.3",
+    "copy-dir": "1.3.0",
+    "electron-context-menu": "2.1.0",
+    "get-port": "5.1.1",
+    "rimraf": "3.0.2",
+    "v8-compile-cache": "2.1.1"
   }
 }

vripper-electron/pom.xml

@@ -1,77 +1,77 @@
 <?xml version="1.0"?>
-<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"
-         xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
-  <modelVersion>4.0.0</modelVersion>
-  <parent>
-    <groupId>tn.mnlr</groupId>
-    <artifactId>vripper</artifactId>
-    <version>2.10.7</version>
-  </parent>
-  <artifactId>vripper-electron</artifactId>
-  <name>vripper-electron</name>
-  <build>
-    <plugins>
-      <plugin>
-        <artifactId>maven-jar-plugin</artifactId>
-        <configuration>
-          <classifier>${buildClassifier}</classifier>
-        </configuration>
-      </plugin>
-    </plugins>
-  </build>
-  <profiles>
-    <profile>
-      <id>electron</id>
-      <properties>
-        <buildClassifier>electron</buildClassifier>
-      </properties>
-      <dependencies>
-        <dependency>
-          <groupId>tn.mnlr</groupId>
-          <artifactId>vripper-server</artifactId>
-          <version>${project.version}</version>
-          <scope>runtime</scope>
-          <classifier>${buildClassifier}</classifier>
-        </dependency>
-      </dependencies>
-      <build>
+<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0">
+    <modelVersion>4.0.0</modelVersion>
+    <parent>
+        <groupId>tn.mnlr</groupId>
+        <artifactId>vripper</artifactId>
+        <version>3.2.3</version>
+    </parent>
+    <artifactId>vripper-electron</artifactId>
+    <name>vripper-electron</name>
+    <build>
         <plugins>
-          <plugin>
-            <groupId>com.github.eirslett</groupId>
-            <artifactId>frontend-maven-plugin</artifactId>
-            <version>1.3</version>
-            <configuration>
-              <nodeVersion>v12.2.0</nodeVersion>
-              <npmVersion>6.9.0</npmVersion>
-              <workingDirectory>build-dir</workingDirectory>
-            </configuration>
-            <executions>
-              <execution>
-                <id>install node and npm</id>
-                <goals>
-                  <goal>install-node-and-npm</goal>
-                </goals>
-              </execution>
-              <execution>
-                <id>npm install</id>
-                <goals>
-                  <goal>npm</goal>
-                </goals>
-              </execution>
-              <execution>
-                <id>npm run build</id>
-                <goals>
-                  <goal>npm</goal>
-                </goals>
+            <plugin>
+                <artifactId>maven-jar-plugin</artifactId>
                 <configuration>
-                  <arguments>run dist</arguments>
+                    <classifier>${buildClassifier}</classifier>
                 </configuration>
-                <phase>generate-resources</phase>
-              </execution>
-            </executions>
-          </plugin>
+            </plugin>
         </plugins>
-      </build>
-    </profile>
-  </profiles>
+    </build>
+    <profiles>
+        <profile>
+            <id>electron</id>
+            <properties>
+                <buildClassifier>electron</buildClassifier>
+            </properties>
+            <dependencies>
+                <dependency>
+                    <groupId>tn.mnlr</groupId>
+                    <artifactId>vripper-server</artifactId>
+                    <version>${project.version}</version>
+                    <scope>runtime</scope>
+                    <classifier>${buildClassifier}</classifier>
+                </dependency>
+            </dependencies>
+            <build>
+                <plugins>
+                    <plugin>
+                        <groupId>com.github.eirslett</groupId>
+                        <artifactId>frontend-maven-plugin</artifactId>
+                        <version>1.3</version>
+                        <configuration>
+                            <nodeVersion>v12.2.0</nodeVersion>
+                            <npmVersion>6.9.0</npmVersion>
+                            <workingDirectory>build-dir</workingDirectory>
+                        </configuration>
+                        <executions>
+                            <execution>
+                                <id>install node and npm</id>
+                                <goals>
+                                    <goal>install-node-and-npm</goal>
+                                </goals>
+                            </execution>
+                            <execution>
+                                <id>npm install</id>
+                                <goals>
+                                    <goal>npm</goal>
+                                </goals>
+                            </execution>
+                            <execution>
+                                <id>npm run build</id>
+                                <goals>
+                                    <goal>npm</goal>
+                                </goals>
+                                <configuration>
+                                    <arguments>run dist</arguments>
+                                </configuration>
+                                <phase>generate-resources</phase>
+                            </execution>
+                        </executions>
+                    </plugin>
+                </plugins>
+            </build>
+        </profile>
+    </profiles>
 </project>

vripper-electron/pre-build.js

@@ -2,7 +2,7 @@ const cheerio = require('cheerio');
 const fs = require('fs');
 const rimraf = require("rimraf");
 const copydir = require("copy-dir");
-const { execSync } = require("child_process");
+const {execSync} = require("child_process");
 
 rimraf.sync('./build/vripper-ui');
 copydir.sync('../vripper-ui/dist/vripper-ui', './build/vripper-ui', {});
@@ -19,7 +19,7 @@ fs.writeFileSync('./build/vripper-ui/index.html', $.html());
 console.log('Building runtime environment');
 rimraf.sync('java-runtime');
 execSync('jlink --no-header-files --no-man-pages --compress=2 --strip-debug --add-modules java.base,java.desktop,java.instrument,java.management,java.naming,java.prefs,java.rmi,java.scripting,java.security.jgss,java.sql,jdk.httpserver,jdk.unsupported,jdk.crypto.ec --output java-runtime');
-if(process.platform === 'linux') {
+if (process.platform === 'linux') {
     console.log('Stripping libjvm.so');
     execSync('strip -p --strip-unneeded java-runtime/lib/server/libjvm.so');
-}
+}

vripper-electron/vripper-electron.iml

@@ -0,0 +1,12 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<module org.jetbrains.idea.maven.project.MavenProjectsManager.isMavenModule="true" type="JAVA_MODULE" version="4">
+  <component name="NewModuleRootManager" LANGUAGE_LEVEL="JDK_1_8">
+    <output url="file://$MODULE_DIR$/target/classes" />
+    <output-test url="file://$MODULE_DIR$/target/test-classes" />
+    <content url="file://$MODULE_DIR$">
+      <excludeFolder url="file://$MODULE_DIR$/target" />
+    </content>
+    <orderEntry type="inheritedJdk" />
+    <orderEntry type="sourceFolder" forTests="false" />
+  </component>
+</module>

vripper-server/pom.xml

@@ -5,7 +5,7 @@
     <parent>
         <groupId>tn.mnlr</groupId>
         <artifactId>vripper</artifactId>
-        <version>2.10.7</version>
+        <version>3.2.3</version>
     </parent>
     <artifactId>vripper-server</artifactId>
     <name>vripper-server</name>
@@ -32,6 +32,20 @@
             <groupId>org.springframework.boot</groupId>
             <artifactId>spring-boot-starter-actuator</artifactId>
         </dependency>
+        <dependency>
+            <groupId>org.springframework.boot</groupId>
+            <artifactId>spring-boot-starter-data-jdbc</artifactId>
+        </dependency>
+        <dependency>
+            <groupId>org.liquibase</groupId>
+            <artifactId>liquibase-core</artifactId>
+        </dependency>
+        <dependency>
+            <groupId>org.hsqldb</groupId>
+            <artifactId>hsqldb</artifactId>
+            <version>2.5.1</version>
+            <scope>runtime</scope>
+        </dependency>
         <dependency>
             <groupId>org.projectlombok</groupId>
             <artifactId>lombok</artifactId>
@@ -58,9 +72,8 @@
             <version>2.22</version>
         </dependency>
         <dependency>
-            <groupId>io.reactivex.rxjava2</groupId>
-            <artifactId>rxjava</artifactId>
-            <version>2.2.6</version>
+            <groupId>io.projectreactor</groupId>
+            <artifactId>reactor-core</artifactId>
         </dependency>
         <dependency>
             <groupId>net.jodah</groupId>
@@ -72,11 +85,6 @@
             <artifactId>guava</artifactId>
             <version>28.1-jre</version>
         </dependency>
-        <dependency>
-            <groupId>org.imgscalr</groupId>
-            <artifactId>imgscalr-lib</artifactId>
-            <version>4.2</version>
-        </dependency>
         <dependency>
             <groupId>tn.mnlr</groupId>
             <artifactId>vripper-ui</artifactId>

vripper-server/src/main/java/tn/mnlr/vripper/EventListenerBean.java

@@ -5,20 +5,32 @@ import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.context.event.ContextRefreshedEvent;
 import org.springframework.context.event.EventListener;
 import org.springframework.stereotype.Component;
-import tn.mnlr.vripper.services.PersistenceService;
+import tn.mnlr.vripper.jpa.repositories.IRepository;
+import tn.mnlr.vripper.services.DataService;
+
+import java.util.Set;
 
 @Component
 public class EventListenerBean {
 
-    @Autowired
-    private PersistenceService persistenceService;
-
     @Getter
     private static boolean init = false;
 
+    private final DataService dataService;
+    private final Set<IRepository> repositorySet;
+
+    @Autowired
+    public EventListenerBean(DataService dataService, Set<IRepository> repositorySet) {
+        this.dataService = dataService;
+        this.repositorySet = repositorySet;
+    }
+
     @EventListener
     public void onApplicationEvent(ContextRefreshedEvent event) {
+        repositorySet.forEach(IRepository::init);
+        dataService.setDownloadingToStopped();
         init = true;
-        persistenceService.restore();
     }
+
+
 }

vripper-server/src/main/java/tn/mnlr/vripper/SpringContext.java

@@ -1,7 +1,6 @@
 package tn.mnlr.vripper;
 
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
+import lombok.extern.slf4j.Slf4j;
 import org.springframework.beans.BeansException;
 import org.springframework.context.ApplicationContext;
 import org.springframework.context.ApplicationContextAware;
@@ -11,10 +10,9 @@ import org.springframework.stereotype.Component;
 import java.util.Map;
 
 @Component
+@Slf4j
 public class SpringContext implements ApplicationContextAware {
 
-    private static final Logger logger = LoggerFactory.getLogger(SpringContext.class);
-
     private static ConfigurableApplicationContext context;
 
     public static <T> T getBean(Class<T> beanClass) {
@@ -26,7 +24,7 @@ public class SpringContext implements ApplicationContextAware {
     }
 
     public static void close() {
-        logger.info("Application terminating...");
+        log.info("Application terminating...");
         if (context != null) {
             context.close();
         }

vripper-server/src/main/java/tn/mnlr/vripper/VripperApplication.java

@@ -1,25 +1,19 @@
 package tn.mnlr.vripper;
 
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
+import lombok.extern.slf4j.Slf4j;
 import org.springframework.boot.SpringApplication;
 import org.springframework.boot.autoconfigure.SpringBootApplication;
 
-import java.util.concurrent.ExecutorService;
-import java.util.concurrent.Executors;
-
 @SpringBootApplication
+@Slf4j
 public class VripperApplication {
 
-    private static final Logger logger = LoggerFactory.getLogger(VripperApplication.class);
-    public static final ExecutorService commonExecutor = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());
-
     public static void main(String[] args) {
         try {
             Runtime.getRuntime().addShutdownHook(new Thread(SpringContext::close));
             SpringApplication.run(VripperApplication.class, args);
         } catch (Exception e) {
-            logger.error("Failed to run the application", e);
+            log.error("Failed to run the application", e);
         }
     }
 }

vripper-server/src/main/java/tn/mnlr/vripper/download/DownloadJob.java

@@ -0,0 +1,277 @@
+package tn.mnlr.vripper.download;
+
+import lombok.Getter;
+import lombok.extern.slf4j.Slf4j;
+import net.jodah.failsafe.function.CheckedRunnable;
+import org.apache.http.client.HttpClient;
+import org.apache.http.client.methods.AbstractExecutionAwareRequest;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpGet;
+import org.apache.http.client.protocol.HttpClientContext;
+import org.apache.http.impl.client.BasicCookieStore;
+import org.apache.http.util.EntityUtils;
+import tn.mnlr.vripper.SpringContext;
+import tn.mnlr.vripper.exception.DownloadException;
+import tn.mnlr.vripper.exception.HostException;
+import tn.mnlr.vripper.jpa.domain.Image;
+import tn.mnlr.vripper.jpa.domain.Post;
+import tn.mnlr.vripper.jpa.domain.enums.Status;
+import tn.mnlr.vripper.services.*;
+
+import javax.imageio.ImageIO;
+import javax.imageio.ImageReader;
+import javax.imageio.stream.ImageInputStream;
+import java.io.File;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.file.Files;
+import java.nio.file.StandardCopyOption;
+import java.util.*;
+
+@Slf4j
+public class DownloadJob implements CheckedRunnable {
+
+    private static final Object LOCK = new Object();
+    private static final int READ_BUFFER_SIZE = 8192;
+    private final DataService dataService;
+    private final PathService pathService;
+    private final ConnectionService cm;
+    private final VGAuthService authService;
+    private final DownloadSpeedService downloadSpeedService;
+    private final SettingsService settingsService;
+    private final HttpClientContext context;
+    @Getter
+    private final Image image;
+    @Getter
+    private final Post post;
+    private boolean stopped = false;
+    @Getter
+    private boolean finished = false;
+
+    DownloadJob(Post post, Image image) {
+        this.image = image;
+        this.post = post;
+        dataService = SpringContext.getBean(DataService.class);
+        pathService = SpringContext.getBean(PathService.class);
+        cm = SpringContext.getBean(ConnectionService.class);
+        authService = SpringContext.getBean(VGAuthService.class);
+        downloadSpeedService = SpringContext.getBean(DownloadSpeedService.class);
+        settingsService = SpringContext.getBean(SettingsService.class);
+        context = HttpClientContext.create();
+        context.setCookieStore(new BasicCookieStore());
+        context.setAttribute(ContextAttributes.OPEN_CONNECTION.toString(), Collections.synchronizedList(new ArrayList<AbstractExecutionAwareRequest>()));
+    }
+
+    public void download(final Post post, final Image image) throws DownloadException {
+
+        try {
+
+            image.setStatus(Status.DOWNLOADING);
+            image.setCurrent(0);
+            dataService.updateImageStatus(image.getStatus(), image.getId());
+            dataService.updateImageCurrent(image.getCurrent(), image.getId());
+
+            synchronized (LOCK) {
+                if (!post.getStatus().equals(Status.DOWNLOADING) && !post.getStatus().equals(Status.PARTIAL)) {
+                    post.setStatus(Status.DOWNLOADING);
+                    dataService.updatePostStatus(post.getStatus(), post.getId());
+                }
+
+                // The post may be updated and the download directory might be set by another thread
+                Post updatedPost = dataService.findById(post.getId()).orElseThrow();
+                if (updatedPost.getDownloadDirectory() == null) {
+                    pathService.createDefaultPostFolder(updatedPost);
+                }
+                if (settingsService.getSettings().getLeaveThanksOnStart()) {
+                    authService.leaveThanks(updatedPost);
+                }
+            }
+
+            if (stopped) {
+                return;
+            }
+            /*
+             * HOST SPECIFIC
+             */
+            log.debug(String.format("Getting image url and name from %s using %s", image.getUrl(), image.getHost()));
+            HostService.NameUrl nameAndUrl = image.getHost().getNameAndUrl(image.getUrl(), context);
+            log.debug(String.format("Resolved name for %s: %s", image.getUrl(), nameAndUrl.getName()));
+            log.debug(String.format("Resolved image url for %s: %s", image.getUrl(), nameAndUrl.getUrl()));
+            /*
+             * END HOST SPECIFIC
+             */
+
+            if (stopped) {
+                return;
+            }
+            String formatImageFileName = pathService.formatImageFileName(nameAndUrl.getName());
+            log.debug(String.format("Sanitizing image name from %s to %s", nameAndUrl.getName(), formatImageFileName));
+            nameAndUrl = new HostService.NameUrl(formatImageFileName, nameAndUrl.getUrl());
+
+            HttpClient client = cm.getClient().build();
+
+            log.debug(String.format("Downloading %s", nameAndUrl.getUrl()));
+            HttpGet httpGet = cm.buildHttpGet(nameAndUrl.getUrl(), context);
+            httpGet.addHeader("Referer", image.getUrl());
+            try (CloseableHttpResponse response = (CloseableHttpResponse) client.execute(httpGet, context)) {
+
+                if (response.getStatusLine().getStatusCode() / 100 != 2) {
+                    EntityUtils.consumeQuietly(response.getEntity());
+                    throw new DownloadException(String.format("Server returned code %d", response.getStatusLine().getStatusCode()));
+                }
+                if (stopped) {
+                    return;
+                }
+                File outputFile = Files.createTempFile("vripper", "tmp").toFile();
+                try (InputStream downloadStream = response.getEntity().getContent(); FileOutputStream fos = new FileOutputStream(outputFile)) {
+
+                    if (stopped) {
+                        return;
+                    }
+
+                    image.setTotal(response.getEntity().getContentLength());
+                    dataService.updateImageTotal(image.getTotal(), image.getId());
+
+                    log.debug(String.format("%s length is %d", nameAndUrl.getUrl(), image.getTotal()));
+                    log.debug(String.format("Starting data transfer for %s", nameAndUrl.getUrl()));
+
+                    byte[] buffer = new byte[READ_BUFFER_SIZE];
+                    int read;
+                    while ((read = downloadStream.read(buffer, 0, READ_BUFFER_SIZE)) != -1 && !stopped) {
+                        fos.write(buffer, 0, read);
+                        image.increase(read);
+                        downloadSpeedService.increase(read);
+                        dataService.updateImageCurrent(image.getCurrent(), image.getId());
+                    }
+                    fos.flush();
+                    EntityUtils.consumeQuietly(response.getEntity());
+                    if (stopped) {
+                        return;
+                    }
+                }
+                checkImageTypeAndRename(dataService.findById(post.getId()).orElseThrow(), outputFile, nameAndUrl.getName(), image.getIndex());
+            }
+        } catch (Exception e) {
+            if (stopped) {
+                return;
+            }
+            throw new DownloadException(e);
+        } finally {
+            if (image.getCurrent() == image.getTotal() && image.getTotal() > 0) {
+                image.setStatus(Status.COMPLETE);
+            } else if (stopped) {
+                image.setStatus(Status.STOPPED);
+            } else {
+                image.setStatus(Status.ERROR);
+            }
+            dataService.updateImageStatus(image.getStatus(), image.getId());
+            finished = true;
+        }
+    }
+
+    private void checkImageTypeAndRename(Post post, File outputFile, String imageName, int index) throws HostException {
+        try (ImageInputStream iis = ImageIO.createImageInputStream(outputFile)) {
+            Iterator<ImageReader> it = ImageIO.getImageReaders(iis);
+            if (!it.hasNext()) {
+                throw new HostException("Image file is not recognized!");
+            }
+            ImageReader reader = it.next();
+            if (reader.getFormatName().equalsIgnoreCase("JPEG")) {
+                String imageNameLC = imageName.toLowerCase();
+                if (!imageNameLC.endsWith("_jpg") && !imageNameLC.endsWith("_jpeg")) {
+                    imageName += ".jpg";
+                } else {
+                    String toReplace = null;
+                    if (imageNameLC.endsWith("_jpg")) {
+                        toReplace = "_jpg";
+                    } else if (imageNameLC.endsWith("_jpeg")) {
+                        toReplace = "_jpeg";
+                    }
+                    if (toReplace != null) {
+                        imageName = imageName.substring(0, imageName.length() - toReplace.length()) + ".jpg";
+                    }
+                }
+            } else if (reader.getFormatName().equalsIgnoreCase("PNG")) {
+                String imageNameLC = imageName.toLowerCase();
+                if (!imageNameLC.endsWith("_png")) {
+                    imageName += ".png";
+                } else {
+                    String toReplace = null;
+                    if (imageNameLC.endsWith("_png")) {
+                        toReplace = "_png";
+                    }
+                    if (toReplace != null) {
+                        imageName = imageName.substring(0, imageName.length() - toReplace.length()) + ".png";
+                    }
+                }
+            }
+        } catch (Exception e) {
+            throw new HostException("Failed to guess image format", e);
+        }
+        try {
+            pathService.getDirectoryAccess().lock();
+            File downloadDestinationFolder = pathService.calcDownloadDirectory(post);
+            File outImage = new File(downloadDestinationFolder, (settingsService.getSettings().getForceOrder() ? String.format("%03d_", index) : "") + imageName);
+            Files.copy(outputFile.toPath(), outImage.toPath(), StandardCopyOption.REPLACE_EXISTING);
+        } catch (Exception e) {
+            throw new HostException("Failed to rename the image", e);
+        } finally {
+            pathService.getDirectoryAccess().unlock();
+            try {
+                Files.delete(outputFile.toPath());
+            } catch (IOException e) {
+                log.warn(String.format("Failed to delete temporary file %s", outputFile.getAbsolutePath()));
+            }
+        }
+    }
+
+    @Override
+    public void run() throws Exception {
+        if (stopped) {
+            finished = true;
+            return;
+        }
+        log.debug(String.format("Starting downloading %s", image.getUrl()));
+        download(post, image);
+    }
+
+    @Override
+    public boolean equals(Object o) {
+        if (this == o) return true;
+        if (o == null || getClass() != o.getClass()) return false;
+        DownloadJob that = (DownloadJob) o;
+        return Objects.equals(image, that.image) &&
+                Objects.equals(post, that.post);
+    }
+
+    @Override
+    public int hashCode() {
+        return Objects.hash(image, post);
+    }
+
+    public void stop() {
+        List<AbstractExecutionAwareRequest> requests = (List<AbstractExecutionAwareRequest>) this.context.getAttribute(ContextAttributes.OPEN_CONNECTION.toString());
+        if (requests != null) {
+            for (AbstractExecutionAwareRequest request : requests) {
+                request.abort();
+            }
+        }
+        this.stopped = true;
+    }
+
+    public enum ContextAttributes {
+        OPEN_CONNECTION("OPEN_CONNECTION");
+
+        private final String value;
+
+        ContextAttributes(String value) {
+            this.value = value;
+        }
+
+        @Override
+        public String toString() {
+            return value;
+        }
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/download/DownloadRunnable.java

@@ -0,0 +1,40 @@
+package tn.mnlr.vripper.download;
+
+import lombok.extern.slf4j.Slf4j;
+import net.jodah.failsafe.Failsafe;
+import tn.mnlr.vripper.SpringContext;
+import tn.mnlr.vripper.jpa.domain.enums.Status;
+import tn.mnlr.vripper.services.ConnectionService;
+import tn.mnlr.vripper.services.DataService;
+
+@Slf4j
+public class DownloadRunnable implements Runnable {
+
+    private final DownloadService downloadService;
+    private final DataService dataService;
+    private final ConnectionService connectionService;
+
+    private final DownloadJob downloadJob;
+
+    public DownloadRunnable(final DownloadJob downloadJob) {
+        downloadService = SpringContext.getBean(DownloadService.class);
+        dataService = SpringContext.getBean(DataService.class);
+        connectionService = SpringContext.getBean(ConnectionService.class);
+        this.downloadJob = downloadJob;
+    }
+
+    @Override
+    public void run() {
+        Failsafe.with(connectionService.getRetryPolicy())
+                .onFailure(e -> {
+                    log.error(String.format("Failed to download %s after %d tries", downloadJob.getImage().getUrl(), e.getAttemptCount()), e.getFailure());
+                    downloadJob.getImage().setStatus(Status.ERROR);
+                    dataService.updateImageStatus(downloadJob.getImage().getStatus(), downloadJob.getImage().getId());
+                })
+                .onComplete(e -> {
+                    dataService.afterJobFinish(downloadJob.getImage(), downloadJob.getPost());
+                    downloadService.afterJobFinish(downloadJob);
+                    log.debug(String.format("Finished downloading %s", downloadJob.getImage().getUrl()));
+                }).run(downloadJob);
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/download/DownloadService.java

@@ -0,0 +1,223 @@
+package tn.mnlr.vripper.download;
+
+import lombok.NonNull;
+import lombok.extern.slf4j.Slf4j;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Service;
+import tn.mnlr.vripper.host.Host;
+import tn.mnlr.vripper.jpa.domain.Image;
+import tn.mnlr.vripper.jpa.domain.Post;
+import tn.mnlr.vripper.jpa.domain.enums.Status;
+import tn.mnlr.vripper.services.DataService;
+import tn.mnlr.vripper.services.PostService;
+import tn.mnlr.vripper.services.SettingsService;
+
+import javax.annotation.PostConstruct;
+import java.util.*;
+import java.util.concurrent.*;
+import java.util.concurrent.atomic.AtomicInteger;
+
+@Service
+@Slf4j
+public class DownloadService {
+
+    private static final List<Status> FINISHED = Arrays.asList(Status.ERROR, Status.COMPLETE, Status.STOPPED);
+    private final int MAX_POOL_SIZE = 12;
+
+    private final ConcurrentHashMap<Host, AtomicInteger> threadCount = new ConcurrentHashMap<>();
+    private final ExecutorService executor = Executors.newFixedThreadPool(MAX_POOL_SIZE);
+    private final BlockingQueue<DownloadJob> executionQueue = new LinkedBlockingQueue<>();
+    private final List<DownloadJob> executing = Collections.synchronizedList(new ArrayList<>());
+
+    private final PendingQueue pendingQueue;
+    private final SettingsService settings;
+    private final DataService dataService;
+    private final PostService postService;
+
+    private boolean pauseQ = false;
+    private Thread executionThread;
+    private Thread pollThread;
+
+    @Autowired
+    public DownloadService(PendingQueue pendingQueue, SettingsService settings, DataService dataService, PostService postService) {
+        this.pendingQueue = pendingQueue;
+        this.settings = settings;
+        this.dataService = dataService;
+        this.postService = postService;
+    }
+
+    @PostConstruct
+    private void init() {
+        executionThread = new Thread(this::start, "Executor thread");
+        pollThread = new Thread(this::poll, "Polling thread");
+        pollThread.start();
+        executionThread.start();
+    }
+
+    public void destroy() throws Exception {
+        log.info("Shutting down ExecutionService");
+        executionThread.interrupt();
+        pollThread.interrupt();
+        executor.shutdown();
+        dataService.findAllPosts().forEach(p -> {
+            log.debug(String.format("Stopping download jobs for %s", p));
+            this.stopRunning(p.getPostId());
+        });
+        executor.awaitTermination(5, TimeUnit.SECONDS);
+    }
+
+    private void stopRunning(@NonNull String postId) {
+        List<DownloadJob> stopping = new ArrayList<>();
+        Iterator<DownloadJob> iterator = executing.iterator();
+        while (iterator.hasNext()) {
+            DownloadJob downloadJob = iterator.next();
+            if (postId.equals(downloadJob.getPost().getPostId())) {
+                downloadJob.stop();
+                iterator.remove();
+                stopping.add(downloadJob);
+            }
+        }
+
+        while (!stopping.isEmpty()) {
+            stopping.removeIf(DownloadJob::isFinished);
+            try {
+                Thread.sleep(500);
+            } catch (InterruptedException e) {
+                Thread.currentThread().interrupt();
+            }
+        }
+    }
+
+    public void stopAll(List<String> posIds) {
+        if (posIds != null) {
+            posIds.forEach(this::stop);
+        } else {
+            dataService.findAllPosts().forEach(p -> this.stop(p.getPostId()));
+        }
+    }
+
+    public void restartAll(List<String> posIds) {
+        if (posIds != null) {
+            posIds.forEach(this::restart);
+        } else {
+            dataService.findAllPosts().forEach(p -> this.restart(p.getPostId()));
+        }
+    }
+
+    private void restart(@NonNull String postId) {
+        if (isPending(postId)) {
+            log.warn(String.format("Cannot restart, jobs are currently running for post id %s", postId));
+            return;
+        }
+        List<Image> images = dataService.findByPostIdAndIsNotCompleted(postId);
+        if (images.isEmpty()) {
+            return;
+        }
+        Post post = dataService.findPostByPostId(postId).orElseThrow();
+        post.setStatus(Status.PENDING);
+        dataService.updatePostStatus(post.getStatus(), post.getId());
+        log.debug(String.format("Restarting %d jobs for post id %s", images.size(), postId));
+        for (Image image : images) {
+            try {
+                pendingQueue.put(post, image);
+            } catch (InterruptedException e) {
+                log.warn("Thread was interrupted", e);
+                Thread.currentThread().interrupt();
+            }
+        }
+    }
+
+    private boolean isPending(String postId) {
+        return pendingQueue.isPending(postId);
+    }
+
+    private void stop(String postId) {
+        try {
+            pauseQ = true;
+            final Post post = dataService.findPostByPostId(postId).orElseThrow();
+            if (post == null) {
+                return;
+            }
+            if (FINISHED.contains(post.getStatus())) {
+                return;
+            }
+            pendingQueue.stop(post);
+            stopRunning(postId);
+            dataService.stopImagesByPostIdAndIsNotCompleted(postId);
+            dataService.finishPost(post);
+            postService.stopFetchingMetadata(post);
+        } finally {
+            pauseQ = false;
+        }
+    }
+
+    private boolean canRun(Host host) {
+        boolean canRun;
+        AtomicInteger count = threadCount.get(host);
+        if (count == null) {
+            threadCount.put(host, new AtomicInteger(0));
+        }
+        canRun = threadCount.get(host).get() < settings.getSettings().getMaxThreads() && (settings.getSettings().getMaxTotalThreads() == 0 ? threadCount.values().stream().mapToInt(AtomicInteger::get).sum() < MAX_POOL_SIZE : threadCount.values().stream().mapToInt(AtomicInteger::get).sum() < settings.getSettings().getMaxTotalThreads());
+        if (canRun && !pauseQ) {
+            threadCount.get(host).incrementAndGet();
+            return true;
+        }
+        return false;
+    }
+
+    private void poll() {
+        while (!Thread.interrupted()) {
+            try {
+                List<DownloadJob> peek = pendingQueue.peek();
+                for (DownloadJob downloadJob : peek) {
+                    if (canRun(downloadJob.getImage().getHost())) {
+                        executionQueue.offer(downloadJob);
+                        pendingQueue.remove(downloadJob);
+                    }
+                }
+                synchronized (threadCount) {
+                    threadCount.wait(2_000);
+                }
+            } catch (InterruptedException e) {
+                Thread.currentThread().interrupt();
+                break;
+            }
+        }
+    }
+
+    private void start() {
+        while (!Thread.interrupted()) {
+            try {
+                push(executionQueue.take());
+            } catch (InterruptedException e) {
+                Thread.currentThread().interrupt();
+                break;
+            } catch (Exception e) {
+                log.error("Execution Service failed", e);
+                break;
+            }
+        }
+    }
+
+    private void push(DownloadJob downloadJob) {
+        log.debug(String.format("Scheduling a job for %s", downloadJob.getImage().getUrl()));
+        executor.execute(new DownloadRunnable(downloadJob));
+        executing.add(downloadJob);
+    }
+
+    public synchronized void afterJobFinish(DownloadJob downloadJob) {
+        int count = pendingQueue.decrement(downloadJob.getPost().getPostId());
+        if (count == 0) {
+            dataService.finishPost(downloadJob.getPost());
+        }
+        threadCount.get(downloadJob.getImage().getHost()).decrementAndGet();
+        executing.remove(downloadJob);
+        synchronized (threadCount) {
+            threadCount.notify();
+        }
+    }
+
+    public int runningCount() {
+        return executing.size();
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/download/PendingQueue.java

@@ -0,0 +1,115 @@
+package tn.mnlr.vripper.download;
+
+import lombok.extern.slf4j.Slf4j;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Service;
+import tn.mnlr.vripper.host.Host;
+import tn.mnlr.vripper.jpa.domain.Image;
+import tn.mnlr.vripper.jpa.domain.Post;
+import tn.mnlr.vripper.services.DataService;
+import tn.mnlr.vripper.services.SettingsService;
+
+import javax.annotation.PostConstruct;
+import java.util.*;
+import java.util.concurrent.BlockingDeque;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.LinkedBlockingDeque;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.function.Predicate;
+
+@Service
+@Slf4j
+public class PendingQueue {
+
+    private final DataService dataService;
+    private final SettingsService settingsService;
+    private final List<Host> hosts;
+
+    private final ConcurrentHashMap<Host, BlockingDeque<DownloadJob>> pendingQ = new ConcurrentHashMap<>();
+    private final ConcurrentHashMap<String, AtomicInteger> toBeExecuted = new ConcurrentHashMap<>();
+
+    @Autowired
+    public PendingQueue(DataService dataService, SettingsService settingsService, List<Host> hosts) {
+        this.dataService = dataService;
+        this.settingsService = settingsService;
+        this.hosts = hosts;
+    }
+
+    @PostConstruct
+    private void init() {
+        hosts.forEach(host -> pendingQ.put(host, new LinkedBlockingDeque<>()));
+    }
+
+    public void put(Post post, Image image) throws InterruptedException {
+        log.debug(String.format("Enqueuing a job for %s", image.getUrl()));
+        image.init();
+        dataService.updateImageStatus(image.getStatus(), image.getId());
+        dataService.updateImageCurrent(image.getCurrent(), image.getId());
+        DownloadJob downloadJob = new DownloadJob(post, image);
+        pendingQ.get(downloadJob.getImage().getHost()).putLast(downloadJob);
+        checkKey(post.getPostId());
+        toBeExecuted.get(post.getPostId()).incrementAndGet();
+    }
+
+    private synchronized void checkKey(String postId) {
+        if (!toBeExecuted.containsKey(postId)) {
+            toBeExecuted.put(postId, new AtomicInteger(0));
+        }
+    }
+
+    public void remove(final DownloadJob downloadJob) {
+        pendingQ.get(downloadJob.getImage().getHost()).remove(downloadJob);
+    }
+
+    public List<DownloadJob> peek() {
+        List<DownloadJob> downloadJobs = new ArrayList<>();
+        if (hosts.size() == 0) {
+            return downloadJobs;
+        }
+        for (Host host : hosts) {
+            Iterator<DownloadJob> it = pendingQ.get(host).iterator();
+            for (int i = 0; i < settingsService.getSettings().getMaxThreads(); i++) {
+                DownloadJob downloadJob = it.hasNext() ? it.next() : null;
+                if (downloadJob != null) {
+                    downloadJobs.add(downloadJob);
+                }
+            }
+        }
+        return downloadJobs;
+    }
+
+    public void enqueue(Post post, Set<Image> images) throws InterruptedException {
+        for (Image image : images) {
+            put(post, image);
+        }
+    }
+
+    public int size() {
+        return toBeExecuted.values().stream().mapToInt(AtomicInteger::get).sum();
+    }
+
+    public void stop(Post post) {
+        Predicate<DownloadJob> predicate = next -> next.getImage().getPostId().equals(post.getPostId());
+        for (Map.Entry<Host, BlockingDeque<DownloadJob>> entry : pendingQ.entrySet()) {
+            entry.getValue().stream().filter(predicate).forEach(e -> decrement(post.getPostId()));
+            entry.getValue().removeIf(predicate);
+            decrement(post.getPostId());
+        }
+    }
+
+    public boolean isPending(String postId) {
+        return toBeExecuted.containsKey(postId);
+    }
+
+    public synchronized int decrement(String postId) {
+        AtomicInteger counter = toBeExecuted.get(postId);
+        if (counter == null) {
+            return 0;
+        }
+        int count = counter.decrementAndGet();
+        if (count == 0) {
+            toBeExecuted.remove(postId);
+        }
+        return count;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/entities/Image.java

@@ -1,134 +0,0 @@
-package tn.mnlr.vripper.entities;
-
-import io.reactivex.disposables.Disposable;
-import io.reactivex.processors.BehaviorProcessor;
-import lombok.Getter;
-import lombok.NoArgsConstructor;
-import lombok.Setter;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import tn.mnlr.vripper.host.Host;
-import tn.mnlr.vripper.services.AppStateService;
-
-import java.util.Objects;
-import java.util.concurrent.atomic.AtomicLong;
-
-@Getter
-@NoArgsConstructor
-public class Image {
-
-    private static final Logger logger = LoggerFactory.getLogger(Image.class);
-
-    private AppStateService appStateService;
-
-    private Disposable subscription;
-
-    private final String type = "img";
-
-    private String postId;
-
-    private String postName;
-
-    private Host host;
-
-    private String url;
-
-    private int index;
-
-    private AtomicLong current = new AtomicLong(0);
-    private Status status;
-    private BehaviorProcessor<Image> imageStateProcessor;
-
-    @Setter
-    private long total = 0;
-
-    public Image(String url, String postId, String postName, Host host, int index) {
-        this.url = url;
-        this.postId = postId;
-        this.postName = postName;
-        this.host = host;
-        this.index = index;
-        status = Status.STOPPED;
-        imageStateProcessor = BehaviorProcessor.create();
-    }
-
-    public void setAppStateService(AppStateService appStateService) {
-        this.appStateService = appStateService;
-        appStateService.getCurrentImages().put(this.url, this);
-        appStateService.getLiveImageUpdates().onNext(this);
-    }
-
-    public void setStatus(Status status) {
-        this.status = status;
-        update();
-    }
-
-    public boolean isCompleted() {
-        return status.equals(Status.COMPLETE);
-    }
-
-    public void init() {
-        if (appStateService == null) {
-            logger.warn("Attempting to init the image whilst the App State is null, unexpected behaviour could occur from this");
-            return;
-        }
-        cleanup();
-        imageStateProcessor = BehaviorProcessor.create();
-        subscription = imageStateProcessor
-                .onBackpressureLatest()
-                .doOnNext(appStateService::onImageUpdate)
-                .subscribe();
-
-        current.set(0);
-        status = Status.STOPPED;
-        imageStateProcessor.onNext(this);
-    }
-
-    public void cleanup() {
-        if (imageStateProcessor != null) {
-            imageStateProcessor.onComplete();
-        }
-        if (subscription != null) {
-            subscription.dispose();
-        }
-    }
-
-    public void setCurrent(int current) {
-        this.current.set(current);
-        update();
-    }
-
-    public void increase(int read) {
-        current.addAndGet(read);
-        update();
-    }
-
-    private void update() {
-        if(imageStateProcessor == null) {
-            return;
-        }
-        imageStateProcessor.onNext(this);
-        if (isCompleted()) {
-            imageStateProcessor.onComplete();
-            subscription.dispose();
-            imageStateProcessor = null;
-        }
-    }
-
-    public enum Status {
-        PENDING, DOWNLOADING, COMPLETE, ERROR, STOPPED
-    }
-
-    @Override
-    public boolean equals(Object o) {
-        if (this == o) return true;
-        if (o == null || getClass() != o.getClass()) return false;
-        Image image = (Image) o;
-        return url.equals(image.url);
-    }
-
-    @Override
-    public int hashCode() {
-        return Objects.hash(url);
-    }
-}

vripper-server/src/main/java/tn/mnlr/vripper/entities/Post.java

@@ -1,112 +0,0 @@
-package tn.mnlr.vripper.entities;
-
-import lombok.Getter;
-import lombok.NoArgsConstructor;
-import lombok.ToString;
-import tn.mnlr.vripper.host.Host;
-import tn.mnlr.vripper.services.AppStateService;
-
-import java.util.List;
-import java.util.Map;
-import java.util.Objects;
-import java.util.Set;
-import java.util.concurrent.atomic.AtomicInteger;
-import java.util.stream.Collectors;
-
-@Getter
-@ToString
-@NoArgsConstructor
-public class Post {
-
-    public enum METADATA {
-        PREVIEWS, RESOLVED_NAME, POSTED_BY, THANKED
-    }
-
-    private AppStateService appStateService;
-
-    private Status status;
-
-    private final String type = "post";
-
-    private String postId;
-
-    private String threadTitle;
-
-    private String threadId;
-
-    private String title;
-
-    private String url;
-
-    private List<Image> images;
-
-    private Map<String, Object> metadata;
-
-    private AtomicInteger done = new AtomicInteger(0);
-
-    private int total;
-
-    private Set<String> hosts;
-
-    private boolean removed = false;
-
-    private String forum;
-
-    public Post(String title, String url, List<Image> images, Map<String, Object> metadata, String postId, String threadId, String threadTitle, String forum) {
-        this.title = title;
-        this.url = url;
-        this.images = images;
-        this.metadata = metadata;
-        this.postId = postId;
-        this.forum = forum;
-        this.threadId = threadId;
-        this.threadTitle = threadTitle;
-        this.hosts = this.images.stream().map(Image::getHost).map(Host::getHost).collect(Collectors.toSet());
-        total = images.size();
-        status = Status.STOPPED;
-    }
-
-    public void setAppStateService(AppStateService appStateService) {
-        this.appStateService = appStateService;
-        this.appStateService.getCurrentPosts().put(postId, this);
-        this.appStateService.getLivePostsState().onNext(this);
-    }
-
-    public void setRemoved(boolean removed) {
-        this.removed = removed;
-        updateNotification();
-    }
-
-    public void increase() {
-        done.incrementAndGet();
-        updateNotification();
-    }
-
-    public void setStatus(Status status) {
-        this.status = status;
-        updateNotification();
-    }
-
-    private void updateNotification() {
-        if (appStateService != null) {
-            appStateService.getLivePostsState().onNext(this);
-        }
-    }
-
-    public enum Status {
-        PENDING, DOWNLOADING, COMPLETE, ERROR, PARTIAL, STOPPED
-    }
-
-    @Override
-    public boolean equals(Object o) {
-        if (this == o) return true;
-        if (o == null || getClass() != o.getClass()) return false;
-        Post post = (Post) o;
-        return postId.equals(post.postId);
-    }
-
-    @Override
-    public int hashCode() {
-        return Objects.hash(postId);
-    }
-}

vripper-server/src/main/java/tn/mnlr/vripper/entities/mixin/persistance/HostToString.java

@@ -1,12 +0,0 @@
-package tn.mnlr.vripper.entities.mixin.persistance;
-
-import com.fasterxml.jackson.databind.util.StdConverter;
-import tn.mnlr.vripper.host.Host;
-
-public class HostToString extends StdConverter<Host, String> {
-
-    @Override
-    public String convert(Host value) {
-        return value.getClass().getSimpleName();
-    }
-}

vripper-server/src/main/java/tn/mnlr/vripper/entities/mixin/persistance/ImagePersistanceMixin.java

@@ -1,36 +0,0 @@
-package tn.mnlr.vripper.entities.mixin.persistance;
-
-import com.fasterxml.jackson.annotation.JsonIgnore;
-import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
-import com.fasterxml.jackson.databind.annotation.JsonSerialize;
-import io.reactivex.disposables.Disposable;
-import io.reactivex.processors.BehaviorProcessor;
-import lombok.Getter;
-import lombok.Setter;
-import tn.mnlr.vripper.entities.Image;
-import tn.mnlr.vripper.host.Host;
-import tn.mnlr.vripper.services.AppStateService;
-
-@Getter
-@Setter
-public abstract class ImagePersistanceMixin {
-
-    @JsonSerialize(converter = HostToString.class)
-    @JsonDeserialize(converter = StringToHost.class)
-    private Host host;
-
-    @JsonIgnore
-    private BehaviorProcessor<Image> imageStateProcessor;
-
-    @JsonIgnore
-    private Disposable subscription;
-
-    @JsonIgnore
-    private String type;
-
-    @JsonIgnore
-    public abstract boolean isCompleted();
-
-    @JsonIgnore
-    private AppStateService appStateService;
-}

vripper-server/src/main/java/tn/mnlr/vripper/entities/mixin/persistance/PostPersistanceMixin.java

@@ -1,21 +0,0 @@
-package tn.mnlr.vripper.entities.mixin.persistance;
-
-import com.fasterxml.jackson.annotation.JsonIgnore;
-import lombok.Getter;
-import lombok.Setter;
-import tn.mnlr.vripper.services.AppStateService;
-
-@Getter
-@Setter
-public abstract class PostPersistanceMixin {
-
-    @JsonIgnore
-    private String type;
-
-    @JsonIgnore
-    private AppStateService appStateService;
-
-
-    @JsonIgnore
-    private boolean removed;
-}

vripper-server/src/main/java/tn/mnlr/vripper/entities/mixin/persistance/StringToHost.java

@@ -1,17 +0,0 @@
-package tn.mnlr.vripper.entities.mixin.persistance;
-
-import com.fasterxml.jackson.databind.util.StdConverter;
-import tn.mnlr.vripper.SpringContext;
-import tn.mnlr.vripper.host.Host;
-
-import java.util.Collection;
-
-public class StringToHost extends StdConverter<String, Host> {
-
-    private Collection<Host> hosts = SpringContext.getBeansOfType(Host.class).values();
-
-    @Override
-    public Host convert(String value) {
-        return hosts.stream().filter(e -> e.getClass().getSimpleName().equals(value)).findAny().orElse(null);
-    }
-}

vripper-server/src/main/java/tn/mnlr/vripper/entities/mixin/ui/ImageUIMixin.java

@@ -1,27 +0,0 @@
-package tn.mnlr.vripper.entities.mixin.ui;
-
-import com.fasterxml.jackson.annotation.JsonIgnore;
-import io.reactivex.disposables.Disposable;
-import io.reactivex.processors.BehaviorProcessor;
-import lombok.Getter;
-import lombok.Setter;
-import tn.mnlr.vripper.entities.Image;
-import tn.mnlr.vripper.host.Host;
-import tn.mnlr.vripper.services.AppStateService;
-
-@Getter
-@Setter
-public abstract class ImageUIMixin {
-
-    @JsonIgnore
-    private Host host;
-
-    @JsonIgnore
-    private BehaviorProcessor<Image> imageStateProcessor;
-
-    @JsonIgnore
-    private Disposable subscription;
-
-    @JsonIgnore
-    private AppStateService appStateService;
-}

vripper-server/src/main/java/tn/mnlr/vripper/entities/mixin/ui/PostUIMixin.java

@@ -1,20 +0,0 @@
-package tn.mnlr.vripper.entities.mixin.ui;
-
-import com.fasterxml.jackson.annotation.JsonIgnore;
-import lombok.Getter;
-import lombok.Setter;
-import tn.mnlr.vripper.entities.Image;
-import tn.mnlr.vripper.services.AppStateService;
-
-import java.util.List;
-
-@Getter
-@Setter
-public abstract class PostUIMixin {
-
-    @JsonIgnore
-    private List<Image> images;
-
-    @JsonIgnore
-    private AppStateService appStateService;
-}

vripper-server/src/main/java/tn/mnlr/vripper/event/ImageUpdateEvent.java

@@ -0,0 +1,15 @@
+package tn.mnlr.vripper.event;
+
+import lombok.Getter;
+import org.springframework.context.ApplicationEvent;
+
+@Getter
+public class ImageUpdateEvent extends ApplicationEvent {
+
+    private final Long id;
+
+    public ImageUpdateEvent(Object source, Long id) {
+        super(source);
+        this.id = id;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/event/MetadataUpdateEvent.java

@@ -0,0 +1,17 @@
+package tn.mnlr.vripper.event;
+
+import lombok.Getter;
+import org.springframework.context.ApplicationEvent;
+
+@Getter
+public class MetadataUpdateEvent extends ApplicationEvent {
+
+    private final Long id;
+    private final Long postIdRef;
+
+    public MetadataUpdateEvent(Object source, Long id, Long postIdRef) {
+        super(source);
+        this.id = id;
+        this.postIdRef = postIdRef;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/event/PostRemoveEvent.java

@@ -0,0 +1,15 @@
+package tn.mnlr.vripper.event;
+
+import lombok.Getter;
+import org.springframework.context.ApplicationEvent;
+
+@Getter
+public class PostRemoveEvent extends ApplicationEvent {
+
+    private final String postId;
+
+    public PostRemoveEvent(Object source, String postId) {
+        super(source);
+        this.postId = postId;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/event/PostUpdateEvent.java

@@ -0,0 +1,15 @@
+package tn.mnlr.vripper.event;
+
+import lombok.Getter;
+import org.springframework.context.ApplicationEvent;
+
+@Getter
+public class PostUpdateEvent extends ApplicationEvent {
+
+    private final Long id;
+
+    public PostUpdateEvent(Object source, Long id) {
+        super(source);
+        this.id = id;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/event/QueuedRemoveEvent.java

@@ -0,0 +1,15 @@
+package tn.mnlr.vripper.event;
+
+import lombok.Getter;
+import org.springframework.context.ApplicationEvent;
+
+@Getter
+public class QueuedRemoveEvent extends ApplicationEvent {
+
+    private final String threadId;
+
+    public QueuedRemoveEvent(Object source, String threadId) {
+        super(source);
+        this.threadId = threadId;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/event/QueuedUpdateEvent.java

@@ -0,0 +1,15 @@
+package tn.mnlr.vripper.event;
+
+import lombok.Getter;
+import org.springframework.context.ApplicationEvent;
+
+@Getter
+public class QueuedUpdateEvent extends ApplicationEvent {
+
+    private final Long id;
+
+    public QueuedUpdateEvent(Object source, Long id) {
+        super(source);
+        this.id = id;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/exception/HostException.java

@@ -10,6 +10,6 @@ public class HostException extends Exception {
     }
 
     public HostException(String message, Throwable e) {
-        super(message,e);
+        super(message, e);
     }
 }

vripper-server/src/main/java/tn/mnlr/vripper/exception/QueueException.java

@@ -0,0 +1,16 @@
+package tn.mnlr.vripper.exception;
+
+public class QueueException extends Exception {
+
+    public QueueException(String message) {
+        super(message);
+    }
+
+    public QueueException(String message, Throwable e) {
+        super(message, e);
+    }
+
+    public QueueException(Throwable e) {
+        super(e);
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/exception/RenameException.java

@@ -0,0 +1,10 @@
+package tn.mnlr.vripper.exception;
+
+import java.io.IOException;
+
+public class RenameException extends Exception {
+
+    public RenameException(String message, IOException e) {
+        super(message, e);
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/host/AcidimgHost.java

@@ -1,5 +1,6 @@
 package tn.mnlr.vripper.host;
 
+import lombok.extern.slf4j.Slf4j;
 import org.apache.http.NameValuePair;
 import org.apache.http.client.HttpClient;
 import org.apache.http.client.entity.UrlEncodedFormEntity;
@@ -8,31 +9,41 @@ import org.apache.http.client.methods.HttpPost;
 import org.apache.http.client.protocol.HttpClientContext;
 import org.apache.http.message.BasicNameValuePair;
 import org.apache.http.util.EntityUtils;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Service;
 import org.w3c.dom.Document;
 import org.w3c.dom.Node;
 import tn.mnlr.vripper.exception.HostException;
 import tn.mnlr.vripper.exception.XpathException;
-import tn.mnlr.vripper.q.ImageFileData;
-import tn.mnlr.vripper.services.ConnectionManager;
+import tn.mnlr.vripper.services.ConnectionService;
+import tn.mnlr.vripper.services.HostService;
+import tn.mnlr.vripper.services.HtmlProcessorService;
+import tn.mnlr.vripper.services.XpathService;
 
 import java.util.ArrayList;
 import java.util.List;
 
 @Service
+@Slf4j
 public class AcidimgHost extends Host {
 
-    private static final Logger logger = LoggerFactory.getLogger(AcidimgHost.class);
-
     private static final String host = "acidimg.cc";
     private static final String CONTINUE_BUTTON_XPATH = "//input[@id='continuebutton']";
     private static final String IMG_XPATH = "//img[@class='centred']";
 
+    private final ConnectionService cm;
+    private final HostService hostService;
+    private final XpathService xpathService;
+    private final HtmlProcessorService htmlProcessorService;
+
+
     @Autowired
-    private ConnectionManager cm;
+    public AcidimgHost(ConnectionService cm, HostService hostService, XpathService xpathService, HtmlProcessorService htmlProcessorService) {
+        this.cm = cm;
+        this.hostService = hostService;
+        this.xpathService = xpathService;
+        this.htmlProcessorService = htmlProcessorService;
+    }
 
     @Override
     public String getHost() {
@@ -45,22 +56,22 @@ public class AcidimgHost extends Host {
     }
 
     @Override
-    protected void setNameAndUrl(final String url, final ImageFileData imageFileData, final HttpClientContext context) throws HostException {
+    public HostService.NameUrl getNameAndUrl(final String url, final HttpClientContext context) throws HostException {
 
-        Document doc = getResponse(url, context).getDocument();
+        Document doc = hostService.getResponse(url, context).getDocument();
 
         Node contDiv;
         try {
-            logger.debug(String.format("Looking for xpath expression %s in %s", CONTINUE_BUTTON_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", CONTINUE_BUTTON_XPATH, url));
             contDiv = xpathService.getAsNode(doc, CONTINUE_BUTTON_XPATH);
         } catch (XpathException e) {
             throw new HostException(e);
         }
 
         if (contDiv != null) {
-            logger.debug(String.format("Click button found for %s", url));
+            log.debug(String.format("Click button found for %s", url));
             HttpClient client = cm.getClient().build();
-            HttpPost httpPost = cm.buildHttpPost(url);
+            HttpPost httpPost = cm.buildHttpPost(url, context);
             httpPost.addHeader("Referer", url);
             List<NameValuePair> params = new ArrayList<>();
             params.add(new BasicNameValuePair("imgContinue", "Continue to your image"));
@@ -70,9 +81,9 @@ public class AcidimgHost extends Host {
                 throw new HostException(e);
             }
 
-            logger.debug(String.format("Requesting %s", httpPost));
+            log.debug(String.format("Requesting %s", httpPost));
             try (CloseableHttpResponse response = (CloseableHttpResponse) client.execute(httpPost, context)) {
-                logger.debug(String.format("Cleaning response for %s", httpPost));
+                log.debug(String.format("Cleaning response for %s", httpPost));
                 doc = htmlProcessorService.clean(EntityUtils.toString(response.getEntity()));
                 EntityUtils.consumeQuietly(response.getEntity());
             } catch (Exception e) {
@@ -82,23 +93,22 @@ public class AcidimgHost extends Host {
 
         Node imgNode;
         try {
-            logger.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
             imgNode = xpathService.getAsNode(doc, IMG_XPATH);
         } catch (XpathException e) {
             throw new HostException(e);
         }
 
-        if(imgNode == null) {
+        if (imgNode == null) {
             throw new HostException("Cannot find the image node");
         }
 
         try {
-            logger.debug(String.format("Resolving name and image url for %s", url));
+            log.debug(String.format("Resolving name and image url for %s", url));
             String imgTitle = imgNode.getAttributes().getNamedItem("alt").getTextContent().trim();
             String imgUrl = imgNode.getAttributes().getNamedItem("src").getTextContent().trim();
 
-            imageFileData.setImageUrl(imgUrl);
-            imageFileData.setImageName(imgTitle.isEmpty() ? getDefaultImageName(imgUrl) : imgTitle);
+            return new HostService.NameUrl(imgTitle.isEmpty() ? hostService.getDefaultImageName(imgUrl) : imgTitle, imgUrl);
         } catch (Exception e) {
             throw new HostException("Unexpected error occurred", e);
         }

vripper-server/src/main/java/tn/mnlr/vripper/host/Host.java

@@ -1,243 +1,26 @@
 package tn.mnlr.vripper.host;
 
-import lombok.Getter;
-import org.apache.http.Header;
-import org.apache.http.client.HttpClient;
-import org.apache.http.client.methods.CloseableHttpResponse;
-import org.apache.http.client.methods.HttpGet;
+import lombok.extern.slf4j.Slf4j;
 import org.apache.http.client.protocol.HttpClientContext;
-import org.apache.http.impl.client.BasicCookieStore;
-import org.apache.http.util.EntityUtils;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Service;
-import org.w3c.dom.Document;
-import tn.mnlr.vripper.entities.Image;
-import tn.mnlr.vripper.exception.DownloadException;
 import tn.mnlr.vripper.exception.HostException;
-import tn.mnlr.vripper.exception.HtmlProcessorException;
-import tn.mnlr.vripper.q.ImageFileData;
-import tn.mnlr.vripper.services.*;
+import tn.mnlr.vripper.services.HostService;
 
-import javax.imageio.ImageIO;
-import javax.imageio.ImageReader;
-import javax.imageio.stream.ImageInputStream;
-import java.io.File;
-import java.io.FileOutputStream;
-import java.io.IOException;
-import java.io.InputStream;
-import java.net.URI;
-import java.net.URISyntaxException;
-import java.nio.file.Files;
-import java.nio.file.StandardCopyOption;
-import java.util.Iterator;
 import java.util.Objects;
 
 @Service
+@Slf4j
 abstract public class Host {
 
-    private static final Logger logger = LoggerFactory.getLogger(Host.class);
-
-    private static final int READ_BUFFER_SIZE = 8192;
-
-    @Autowired
-    protected HtmlProcessorService htmlProcessorService;
-
-    @Autowired
-    protected XpathService xpathService;
-
-    @Autowired
-    private AppSettingsService appSettingsService;
-
-    @Autowired
-    private AppStateService appStateService;
-
-    @Autowired
-    private ConnectionManager cm;
-
-    @Autowired
-    private DownloadSpeedService downloadSpeedService;
-
-    @Autowired
-    private PathService pathService;
-
     abstract public String getHost();
 
     abstract public String getLookup();
 
-    @Override
-    public int hashCode() {
-        return Objects.hash(pathService);
-    }
-
     public boolean isSupported(String url) {
         return url.contains(getLookup());
     }
 
-    public void download(Image image, ImageFileData imageFileData) throws DownloadException, InterruptedException {
-
-        HttpClientContext context = HttpClientContext.create();
-        context.setCookieStore(new BasicCookieStore());
-        try {
-
-            appStateService.postDownloadingUpdate(image.getPostId());
-
-            imageFileData.setPageUrl(image.getUrl());
-
-            /*
-             * HOST SPECIFIC
-             */
-            logger.debug(String.format("Getting image url and name from %s using %s", image.getUrl(), this.getHost()));
-            setNameAndUrl(image.getUrl(), imageFileData, context);
-            logger.debug(String.format("Resolved name for %s: %s", image.getUrl(), imageFileData.getImageName()));
-            logger.debug(String.format("Resolved image url for %s: %s", image.getUrl(), imageFileData.getImageUrl()));
-
-            logger.debug(String.format("Building image request for %s", image.getUrl()));
-            setImageRequest(imageFileData);
-            /*
-             * END HOST SPECIFIC
-             */
-
-            String formatImageFileName = pathService.formatImageFileName(imageFileData.getImageName());
-            logger.debug(String.format("Sanitizing image name from %s to %s", imageFileData.getImageName(), formatImageFileName));
-            imageFileData.setImageName(formatImageFileName);
-            File destinationFolder = pathService.getDownloadDestinationFolder(image.getPostId());
-            logger.debug(String.format("Saving to %s", destinationFolder.getPath()));
-            logger.debug(String.format("Creating %s", destinationFolder.getPath()));
-            destinationFolder.mkdirs();
-            if (!destinationFolder.exists()) {
-                logger.debug(String.format("Folder %s is created", destinationFolder.toString()));
-            }
-
-            HttpClient client = cm.getClient().build();
-
-            logger.debug(String.format("Downloading %s", imageFileData.getImageUrl()));
-            try (CloseableHttpResponse response = (CloseableHttpResponse) client.execute(imageFileData.getImageRequest(), context)) {
-
-                if (response.getStatusLine().getStatusCode() / 100 != 2) {
-                    EntityUtils.consumeQuietly(response.getEntity());
-                    throw new DownloadException(String.format("Server returned code %d", response.getStatusLine().getStatusCode()));
-                }
-
-                File outputFile = new File(destinationFolder.getPath() + File.separator + String.format("%03d_", image.getIndex()) + imageFileData.getImageName() + ".tmp");
-                try (InputStream downloadStream = response.getEntity().getContent(); FileOutputStream fos = new FileOutputStream(outputFile)) {
-                    image.setTotal(response.getEntity().getContentLength());
-                    logger.debug(String.format("%s length is %d", imageFileData.getImageUrl(), image.getTotal()));
-                    logger.debug(String.format("Starting data transfer for %s", imageFileData.getImageUrl()));
-
-                    byte[] buffer = new byte[READ_BUFFER_SIZE];
-                    int read;
-                    while ((read = downloadStream.read(buffer, 0, READ_BUFFER_SIZE)) != -1) {
-                        fos.write(buffer, 0, read);
-                        image.increase(read);
-                        downloadSpeedService.increase(read);
-                    }
-                    fos.flush();
-                    EntityUtils.consumeQuietly(response.getEntity());
-                }
-                File finalName = checkImageTypeAndRename(outputFile, imageFileData.getImageName(), image.getIndex());
-                imageFileData.setFileName(finalName.getName());
-            }
-        } catch (Exception e) {
-            if (Thread.interrupted()) {
-                throw new InterruptedException("Download was interrupted");
-            }
-            throw new DownloadException(e);
-        }
-    }
-
-    private File checkImageTypeAndRename(File outputFile, String imageName, int index) throws HostException {
-        try (ImageInputStream iis = ImageIO.createImageInputStream(outputFile)) {
-            Iterator<ImageReader> it = ImageIO.getImageReaders(iis);
-            if (!it.hasNext()) {
-                throw new HostException("Image file is not recognized!");
-            }
-            ImageReader reader = it.next();
-            if (reader.getFormatName().toUpperCase().equals("JPEG")) {
-                String imageNameLC = imageName.toLowerCase();
-                if (!imageNameLC.endsWith("_jpg") && !imageNameLC.endsWith("_jpeg")) {
-                    imageName += ".jpg";
-                } else {
-                    String toReplace = null;
-                    if (imageNameLC.endsWith("_jpg")) {
-                        toReplace = "_jpg";
-                    } else if (imageNameLC.endsWith("_jpeg")) {
-                        toReplace = "_jpeg";
-                    }
-                    if (toReplace != null) {
-                        imageName = imageName.substring(0, imageName.length() - toReplace.length()) + ".jpg";
-                    }
-                }
-            }
-        } catch (Exception e) {
-            throw new HostException("Failed to guess image format", e);
-        }
-        try {
-            File outImage = new File(outputFile.getParent(), (appSettingsService.isForceOrder() ? String.format("%03d_", index) : "") + imageName);
-            if (outImage.exists() && outImage.delete()) {
-                logger.debug(String.format("%s is deleted", outImage.toString()));
-            }
-            return Files.move(outputFile.toPath(), outImage.toPath(), StandardCopyOption.ATOMIC_MOVE).toFile();
-        } catch (Exception e) {
-            throw new HostException("Failed to rename the image", e);
-        }
-    }
-
-    final String getDefaultImageName(final String imgUrl) {
-        String imageTitle = imgUrl.substring(imgUrl.lastIndexOf('/') + 1);
-        logger.debug(String.format("Extracting name from url %s: %s", imgUrl, imageTitle));
-        return imgUrl;
-    }
-
-    final Response getResponse(final String url, final HttpClientContext context) throws HostException {
-        String basePage;
-
-        HttpClient client = cm.getClient().build();
-        HttpGet httpGet = cm.buildHttpGet(url);
-        Header[] headers;
-        logger.debug(String.format("Requesting %s", url));
-        try (CloseableHttpResponse response = (CloseableHttpResponse) client.execute(httpGet, context)) {
-            if (response.getStatusLine().getStatusCode() / 100 != 2) {
-                throw new HostException(String.format("Unexpected response code: %d", response.getStatusLine().getStatusCode()));
-            }
-            headers = response.getAllHeaders();
-            basePage = EntityUtils.toString(response.getEntity());
-            logger.debug(String.format("%s response: %n%s", url, basePage));
-            EntityUtils.consumeQuietly(response.getEntity());
-        } catch (IOException e) {
-            throw new HostException(e);
-        }
-
-        try {
-            logger.debug(String.format("Cleaning %s response", url));
-            return new Response(htmlProcessorService.clean(basePage), headers);
-        } catch (HtmlProcessorException e) {
-            throw new HostException(e);
-        }
-    }
-
-    protected String appendUri(String uri, String appendQuery) throws URISyntaxException {
-        URI oldUri = new URI(uri);
-
-        String newQuery = oldUri.getQuery();
-        if (newQuery == null) {
-            newQuery = appendQuery;
-        } else {
-            newQuery += "&" + appendQuery;
-        }
-
-        return new URI(oldUri.getScheme(), oldUri.getAuthority(),
-                oldUri.getPath(), newQuery, oldUri.getFragment()).toString();
-    }
-
-    private void setImageRequest(final ImageFileData imageFileData) {
-        HttpGet httpGet = cm.buildHttpGet(imageFileData.getImageUrl());
-        httpGet.addHeader("Referer", imageFileData.getPageUrl());
-        imageFileData.setImageRequest(httpGet);
-    }
-
-    protected abstract void setNameAndUrl(final String url, final ImageFileData imageFileData, final HttpClientContext context) throws HostException;
+    public abstract HostService.NameUrl getNameAndUrl(final String url, final HttpClientContext context) throws HostException;
 
     @Override
     public boolean equals(Object o) {
@@ -247,16 +30,9 @@ abstract public class Host {
         return Objects.equals(getHost(), host.getHost());
     }
 
-    @Getter
-    public static class Response {
-
-        protected Response(Document document, Header[] headers) {
-            this.document = document;
-            this.headers = headers;
-        }
-
-        private Document document;
-        private Header[] headers;
+    @Override
+    public int hashCode() {
+        return Objects.hash(getHost());
     }
 
     @Override

vripper-server/src/main/java/tn/mnlr/vripper/host/ImageBamHost.java

@@ -1,12 +1,11 @@
 package tn.mnlr.vripper.host;
 
+import lombok.extern.slf4j.Slf4j;
 import org.apache.http.client.HttpClient;
 import org.apache.http.client.methods.CloseableHttpResponse;
 import org.apache.http.client.methods.HttpGet;
 import org.apache.http.client.protocol.HttpClientContext;
 import org.apache.http.util.EntityUtils;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Service;
 import org.w3c.dom.Document;
@@ -14,22 +13,33 @@ import org.w3c.dom.Node;
 import tn.mnlr.vripper.exception.HostException;
 import tn.mnlr.vripper.exception.HtmlProcessorException;
 import tn.mnlr.vripper.exception.XpathException;
-import tn.mnlr.vripper.q.ImageFileData;
-import tn.mnlr.vripper.services.ConnectionManager;
+import tn.mnlr.vripper.services.ConnectionService;
+import tn.mnlr.vripper.services.HostService;
+import tn.mnlr.vripper.services.HtmlProcessorService;
+import tn.mnlr.vripper.services.XpathService;
 
 import java.io.IOException;
 
 @Service
+@Slf4j
 public class ImageBamHost extends Host {
 
-    private static final Logger logger = LoggerFactory.getLogger(ImageBamHost.class);
-
     private static final String host = "imagebam.com";
     private static final String CONTINUE_BUTTON_XPATH = "//a[@title='Continue to your image']";
     private static final String IMG_XPATH = "//img[@class='image']";
 
+    private final HostService hostService;
+    private final ConnectionService cm;
+    private final XpathService xpathService;
+    private final HtmlProcessorService htmlProcessorService;
+
     @Autowired
-    private ConnectionManager cm;
+    public ImageBamHost(HostService hostService, ConnectionService cm, XpathService xpathService, HtmlProcessorService htmlProcessorService) {
+        this.hostService = hostService;
+        this.cm = cm;
+        this.xpathService = xpathService;
+        this.htmlProcessorService = htmlProcessorService;
+    }
 
     @Override
     public String getHost() {
@@ -42,29 +52,29 @@ public class ImageBamHost extends Host {
     }
 
     @Override
-    protected void setNameAndUrl(final String url, final ImageFileData imageFileData, final HttpClientContext context) throws HostException {
+    public HostService.NameUrl getNameAndUrl(final String url, final HttpClientContext context) throws HostException {
 
-        Response response = getResponse(url, context);
+        HostService.Response response = hostService.getResponse(url, context);
         Document doc = response.getDocument();
 
         Node contDiv;
         try {
-            logger.debug(String.format("Looking for xpath expression %s in %s", CONTINUE_BUTTON_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", CONTINUE_BUTTON_XPATH, url));
             contDiv = xpathService.getAsNode(doc, CONTINUE_BUTTON_XPATH);
         } catch (XpathException e) {
             throw new HostException(e);
         }
 
         if (contDiv != null) {
-            logger.debug(String.format("Click button found for %s", url));
+            log.debug(String.format("Click button found for %s", url));
             HttpClient client = cm.getClient().build();
-            HttpGet httpGet = cm.buildHttpGet(url);
+            HttpGet httpGet = cm.buildHttpGet(url, context);
             httpGet.addHeader("Referer", url);
-            logger.debug(String.format("Requesting %s", httpGet));
+            log.debug(String.format("Requesting %s", httpGet));
             try (CloseableHttpResponse res = (CloseableHttpResponse) client.execute(httpGet, context)) {
                 String s = EntityUtils.toString(res.getEntity());
-                logger.debug(String.format("%s response is:%n%s", httpGet, s));
-                logger.debug(String.format("Cleaning response for %s", httpGet));
+                log.debug(String.format("%s response is:%n%s", httpGet, s));
+                log.debug(String.format("Cleaning response for %s", httpGet));
                 doc = htmlProcessorService.clean(s);
                 EntityUtils.consumeQuietly(res.getEntity());
             } catch (IOException | HtmlProcessorException e) {
@@ -74,20 +84,19 @@ public class ImageBamHost extends Host {
 
         Node imgNode;
         try {
-            logger.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
             imgNode = xpathService.getAsNode(doc, IMG_XPATH);
         } catch (XpathException e) {
             throw new HostException(e);
         }
 
         try {
-            logger.debug(String.format("Resolving name and image url for %s", url));
+            log.debug(String.format("Resolving name and image url for %s", url));
             String imgTitle = imgNode.getAttributes().getNamedItem("id").getTextContent().trim();
             String imgUrl = imgNode.getAttributes().getNamedItem("src").getTextContent().trim();
 
-            imageFileData.setImageUrl(imgUrl);
-            imageFileData.setImageName(imgTitle.isEmpty() ? imgUrl.substring(imgUrl.lastIndexOf('/') + 1) : imgTitle);
-        } catch(Exception e) {
+            return new HostService.NameUrl(imgTitle.isEmpty() ? imgUrl.substring(imgUrl.lastIndexOf('/') + 1) : imgTitle, imgUrl);
+        } catch (Exception e) {
             throw new HostException("Unexpected error occurred", e);
         }
     }

vripper-server/src/main/java/tn/mnlr/vripper/host/ImageTwistHost.java

@@ -1,27 +1,33 @@
 package tn.mnlr.vripper.host;
 
+import lombok.extern.slf4j.Slf4j;
 import org.apache.http.client.protocol.HttpClientContext;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Service;
 import org.w3c.dom.Document;
 import org.w3c.dom.Node;
 import tn.mnlr.vripper.exception.HostException;
 import tn.mnlr.vripper.exception.XpathException;
-import tn.mnlr.vripper.q.ImageFileData;
-import tn.mnlr.vripper.services.ConnectionManager;
+import tn.mnlr.vripper.services.HostService;
+import tn.mnlr.vripper.services.XpathService;
 
 import java.util.Optional;
 
 @Service
+@Slf4j
 public class ImageTwistHost extends Host {
 
     private static final String IMG_XPATH = "//img[contains(@class, 'img')]";
-    private static final Logger logger = LoggerFactory.getLogger(ImageTwistHost.class);
     private static final String host = "imagetwist.com";
+
+    private final HostService hostService;
+    private final XpathService xpathService;
+
     @Autowired
-    private ConnectionManager cm;
+    public ImageTwistHost(HostService hostService, XpathService xpathService) {
+        this.hostService = hostService;
+        this.xpathService = xpathService;
+    }
 
     @Override
     public String getHost() {
@@ -34,25 +40,24 @@ public class ImageTwistHost extends Host {
     }
 
     @Override
-    protected void setNameAndUrl(final String url, final ImageFileData imageFileData, final HttpClientContext context) throws HostException {
+    public HostService.NameUrl getNameAndUrl(final String url, final HttpClientContext context) throws HostException {
 
-        Document doc = getResponse(url, context).getDocument();
+        Document doc = hostService.getResponse(url, context).getDocument();
 
         Node imgNode;
         try {
-            logger.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
             imgNode = xpathService.getAsNode(doc, IMG_XPATH);
         } catch (XpathException e) {
             throw new HostException(e);
         }
 
         try {
-            logger.debug(String.format("Resolving name and image url for %s", url));
+            log.debug(String.format("Resolving name and image url for %s", url));
             String imgTitle = Optional.ofNullable(imgNode.getAttributes().getNamedItem("alt")).map(Node::getTextContent).map(String::trim).orElse(null);
             String imgUrl = imgNode.getAttributes().getNamedItem("src").getTextContent().trim();
 
-            imageFileData.setImageUrl(imgUrl);
-            imageFileData.setImageName(imgTitle);
+            return new HostService.NameUrl(imgTitle, imgUrl);
         } catch (Exception e) {
             throw new HostException("Unexpected error occurred", e);
         }

vripper-server/src/main/java/tn/mnlr/vripper/host/ImageVenueHost.java

@@ -1,25 +1,32 @@
 package tn.mnlr.vripper.host;
 
+import lombok.extern.slf4j.Slf4j;
 import org.apache.http.client.protocol.HttpClientContext;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
+import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Service;
 import org.w3c.dom.Document;
 import org.w3c.dom.Node;
 import tn.mnlr.vripper.exception.HostException;
 import tn.mnlr.vripper.exception.XpathException;
-import tn.mnlr.vripper.q.ImageFileData;
-
-import java.net.URI;
+import tn.mnlr.vripper.services.HostService;
+import tn.mnlr.vripper.services.XpathService;
 
 @Service
+@Slf4j
 public class ImageVenueHost extends Host {
 
-    private static final Logger logger = LoggerFactory.getLogger(ImageVenueHost.class);
-
     private static final String host = "imagevenue.com";
-    private static final String CONTINUE_BUTTON_XPATH = "//a[@title='Continue to your image']";
-    private static final String IMG_XPATH = "//img[@id='thepic']";
+    private static final String CONTINUE_BUTTON_XPATH = "//a[@title='Continue to ImageVenue']";
+    private static final String IMG_XPATH = "//a[@data-toggle='full']/img";
+
+    private final HostService hostService;
+    private final XpathService xpathService;
+
+    @Autowired
+    public ImageVenueHost(HostService hostService, XpathService xpathService) {
+        this.hostService = hostService;
+        this.xpathService = xpathService;
+    }
 
     @Override
     public String getHost() {
@@ -32,20 +39,18 @@ public class ImageVenueHost extends Host {
     }
 
     @Override
-    protected void setNameAndUrl(final String url, final ImageFileData imageFileData, final HttpClientContext context) throws HostException {
+    public HostService.NameUrl getNameAndUrl(final String _url, final HttpClientContext context) throws HostException {
 
-        //Sadly, they do not support https.
-        //If they add such support in the future,
-        //then we should automatically adapt the URL here as done elsewhere.
+        String url = _url.replace("http://", "https://");
 
-        Response resp = getResponse(url, context);
+        HostService.Response resp = hostService.getResponse(url, context);
         Document doc = resp.getDocument();
 
         try {
-            logger.debug(String.format("Looking for xpath expression %s in %s", CONTINUE_BUTTON_XPATH, url));
-            if(xpathService.getAsNode(doc, CONTINUE_BUTTON_XPATH) != null) {
+            log.debug(String.format("Looking for xpath expression %s in %s", CONTINUE_BUTTON_XPATH, url));
+            if (xpathService.getAsNode(doc, CONTINUE_BUTTON_XPATH) != null) {
                 //Button detected. No need to actually click it, just make the call again.
-                resp = getResponse(url, context);
+                resp = hostService.getResponse(url, context);
                 doc = resp.getDocument();
             }
         } catch (XpathException e) {
@@ -54,7 +59,7 @@ public class ImageVenueHost extends Host {
 
         Node imgNode;
         try {
-            logger.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
             imgNode = xpathService.getAsNode(doc, IMG_XPATH);
         } catch (XpathException e) {
             throw new HostException(e);
@@ -65,14 +70,12 @@ public class ImageVenueHost extends Host {
         }
 
         try {
-            logger.debug(String.format("Resolving name and image url for %s", url));
+            log.debug(String.format("Resolving name and image url for %s", url));
             String imgTitle = imgNode.getAttributes().getNamedItem("alt").getTextContent().trim();
             String imgUrl = imgNode.getAttributes().getNamedItem("src").getTextContent().trim();
 
-            URI baseUri = new URI(url);
-            imageFileData.setImageUrl(new URI(baseUri.getScheme(), baseUri.getHost(), '/'+imgUrl, null).toString());
-            imageFileData.setImageName(imgTitle.isEmpty() ? imgUrl.substring(imgUrl.lastIndexOf('/') + 1) : imgTitle);
-        } catch(Exception e) {
+            return new HostService.NameUrl(imgTitle.isEmpty() ? imgUrl.substring(imgUrl.lastIndexOf('/') + 1) : imgTitle, imgUrl);
+        } catch (Exception e) {
             throw new HostException("Unexpected error occurred", e);
         }
     }

vripper-server/src/main/java/tn/mnlr/vripper/host/ImageZillaHost.java

@@ -1,28 +1,32 @@
 package tn.mnlr.vripper.host;
 
+import lombok.extern.slf4j.Slf4j;
 import org.apache.http.client.protocol.HttpClientContext;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Service;
 import org.w3c.dom.Document;
 import org.w3c.dom.Node;
 import tn.mnlr.vripper.exception.HostException;
 import tn.mnlr.vripper.exception.XpathException;
-import tn.mnlr.vripper.q.ImageFileData;
-import tn.mnlr.vripper.services.ConnectionManager;
+import tn.mnlr.vripper.services.HostService;
+import tn.mnlr.vripper.services.XpathService;
 
 @Service
+@Slf4j
 public class ImageZillaHost extends Host {
 
-    private static final Logger logger = LoggerFactory.getLogger(ImageZillaHost.class);
-
     private static final String host = "imagezilla.net";
     private static final String lookup = "imagezilla.net/show";
     private static final String IMG_XPATH = "//img[@id='photo']";
 
+    private final HostService hostService;
+    private final XpathService xpathService;
+
     @Autowired
-    private ConnectionManager cm;
+    public ImageZillaHost(HostService hostService, XpathService xpathService) {
+        this.hostService = hostService;
+        this.xpathService = xpathService;
+    }
 
     @Override
     public String getHost() {
@@ -35,16 +39,16 @@ public class ImageZillaHost extends Host {
     }
 
     @Override
-    protected void setNameAndUrl(final String url, final ImageFileData imageFileData, final HttpClientContext context) throws HostException {
+    public HostService.NameUrl getNameAndUrl(final String url, final HttpClientContext context) throws HostException {
 
-        Document doc = getResponse(url, context).getDocument();
+        Document doc = hostService.getResponse(url, context).getDocument();
 
         String title;
         try {
-            logger.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
             Node titleNode = xpathService.getAsNode(doc, IMG_XPATH).getAttributes().getNamedItem("title");
-            logger.debug(String.format("Resolving name for %s", url));
-            if(titleNode != null) {
+            log.debug(String.format("Resolving name for %s", url));
+            if (titleNode != null) {
                 title = titleNode.getTextContent().trim();
             } else {
                 title = null;
@@ -53,13 +57,12 @@ public class ImageZillaHost extends Host {
             throw new HostException(e);
         }
 
-        if(title == null || title.isEmpty()) {
-            title = getDefaultImageName(url);
+        if (title == null || title.isEmpty()) {
+            title = hostService.getDefaultImageName(url);
         }
 
         try {
-            imageFileData.setImageUrl(url.replace("show", "images"));
-            imageFileData.setImageName(title);
+            return new HostService.NameUrl(title, url.replace("show", "images"));
         } catch (Exception e) {
             throw new HostException("Unexpected error occurred", e);
         }

vripper-server/src/main/java/tn/mnlr/vripper/host/ImgSpiceHost.java

@@ -1,27 +1,31 @@
 package tn.mnlr.vripper.host;
 
+import lombok.extern.slf4j.Slf4j;
 import org.apache.http.client.protocol.HttpClientContext;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Service;
 import org.w3c.dom.Document;
 import org.w3c.dom.Node;
 import tn.mnlr.vripper.exception.HostException;
 import tn.mnlr.vripper.exception.XpathException;
-import tn.mnlr.vripper.q.ImageFileData;
-import tn.mnlr.vripper.services.ConnectionManager;
+import tn.mnlr.vripper.services.HostService;
+import tn.mnlr.vripper.services.XpathService;
 
 @Service
+@Slf4j
 public class ImgSpiceHost extends Host {
 
-    private static final Logger logger = LoggerFactory.getLogger(ImgSpiceHost.class);
-
     private static final String host = "imgspice.com";
     private static final String IMG_XPATH = "//img[@id='imgpreview']";
 
+    private final HostService hostService;
+    private final XpathService xpathService;
+
     @Autowired
-    private ConnectionManager cm;
+    public ImgSpiceHost(HostService hostService, XpathService xpathService) {
+        this.hostService = hostService;
+        this.xpathService = xpathService;
+    }
 
     @Override
     public String getHost() {
@@ -34,27 +38,26 @@ public class ImgSpiceHost extends Host {
     }
 
     @Override
-    protected void setNameAndUrl(final String _url, final ImageFileData imageFileData, final HttpClientContext context) throws HostException {
+    public HostService.NameUrl getNameAndUrl(final String _url, final HttpClientContext context) throws HostException {
 
         String url = _url.replace("http://", "https://");
-        Response resp = getResponse(url, context);
+        HostService.Response resp = hostService.getResponse(url, context);
         Document doc = resp.getDocument();
 
         Node imgNode;
         try {
-            logger.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
             imgNode = xpathService.getAsNode(doc, IMG_XPATH);
         } catch (XpathException e) {
             throw new HostException(e);
         }
 
         try {
-            logger.debug(String.format("Resolving name and image url for %s", url));
+            log.debug(String.format("Resolving name and image url for %s", url));
             String imgTitle = imgNode.getAttributes().getNamedItem("alt").getTextContent().trim();
             String imgUrl = imgNode.getAttributes().getNamedItem("src").getTextContent().trim();
 
-            imageFileData.setImageUrl(imgUrl);
-            imageFileData.setImageName(imgTitle.isEmpty() ? imgUrl.substring(imgUrl.lastIndexOf('/') + 1) : imgTitle);
+            return new HostService.NameUrl(imgTitle.isEmpty() ? imgUrl.substring(imgUrl.lastIndexOf('/') + 1) : imgTitle, imgUrl);
         } catch (Exception e) {
             throw new HostException("Unexpected error occurred", e);
         }

vripper-server/src/main/java/tn/mnlr/vripper/host/ImgboxHost.java

@@ -1,27 +1,32 @@
 package tn.mnlr.vripper.host;
 
+import lombok.extern.slf4j.Slf4j;
 import org.apache.http.client.protocol.HttpClientContext;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Service;
 import org.w3c.dom.Document;
 import org.w3c.dom.Node;
 import tn.mnlr.vripper.exception.HostException;
 import tn.mnlr.vripper.exception.XpathException;
-import tn.mnlr.vripper.q.ImageFileData;
-import tn.mnlr.vripper.services.ConnectionManager;
+import tn.mnlr.vripper.services.HostService;
+import tn.mnlr.vripper.services.XpathService;
 
 @Service
+@Slf4j
 public class ImgboxHost extends Host {
 
-    private static final Logger logger = LoggerFactory.getLogger(ImgboxHost.class);
-
     private static final String host = "imgbox.com";
     private static final String IMG_XPATH = "//img[@id='img']";
 
+    private final HostService hostService;
+    private final XpathService xpathService;
+
+
     @Autowired
-    private ConnectionManager cm;
+    public ImgboxHost(HostService hostService, XpathService xpathService) {
+        this.hostService = hostService;
+        this.xpathService = xpathService;
+    }
 
     @Override
     public String getHost() {
@@ -34,25 +39,24 @@ public class ImgboxHost extends Host {
     }
 
     @Override
-    protected void setNameAndUrl(final String url, final ImageFileData imageFileData, final HttpClientContext context) throws HostException {
+    public HostService.NameUrl getNameAndUrl(final String url, final HttpClientContext context) throws HostException {
 
-        Document doc = getResponse(url, context).getDocument();
+        Document doc = hostService.getResponse(url, context).getDocument();
 
         Node imgNode;
         try {
-            logger.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
             imgNode = xpathService.getAsNode(doc, IMG_XPATH);
         } catch (XpathException e) {
             throw new HostException(e);
         }
 
         try {
-            logger.debug(String.format("Resolving name and image url for %s", url));
+            log.debug(String.format("Resolving name and image url for %s", url));
             String imgTitle = imgNode.getAttributes().getNamedItem("title").getTextContent().trim();
             String imgUrl = imgNode.getAttributes().getNamedItem("src").getTextContent().trim();
 
-            imageFileData.setImageUrl(imgUrl);
-            imageFileData.setImageName(imgTitle);
+            return new HostService.NameUrl(imgTitle, imgUrl);
         } catch (Exception e) {
             throw new HostException("Unexpected error occurred", e);
         }

vripper-server/src/main/java/tn/mnlr/vripper/host/ImxHost.java

@@ -1,5 +1,6 @@
 package tn.mnlr.vripper.host;
 
+import lombok.extern.slf4j.Slf4j;
 import org.apache.http.NameValuePair;
 import org.apache.http.client.HttpClient;
 import org.apache.http.client.entity.UrlEncodedFormEntity;
@@ -8,8 +9,6 @@ import org.apache.http.client.methods.HttpPost;
 import org.apache.http.client.protocol.HttpClientContext;
 import org.apache.http.message.BasicNameValuePair;
 import org.apache.http.util.EntityUtils;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Service;
 import org.w3c.dom.Document;
@@ -17,24 +16,35 @@ import org.w3c.dom.Node;
 import tn.mnlr.vripper.exception.HostException;
 import tn.mnlr.vripper.exception.HtmlProcessorException;
 import tn.mnlr.vripper.exception.XpathException;
-import tn.mnlr.vripper.q.ImageFileData;
-import tn.mnlr.vripper.services.ConnectionManager;
+import tn.mnlr.vripper.services.ConnectionService;
+import tn.mnlr.vripper.services.HostService;
+import tn.mnlr.vripper.services.HtmlProcessorService;
+import tn.mnlr.vripper.services.XpathService;
 
 import java.io.IOException;
 import java.util.ArrayList;
 import java.util.List;
 
 @Service
+@Slf4j
 public class ImxHost extends Host {
 
-    private static final Logger logger = LoggerFactory.getLogger(ImxHost.class);
-
     private static final String host = "imx.to";
     private static final String CONTINUE_BUTTON_XPATH = "//*[@name='imgContinue']";
     private static final String IMG_XPATH = "//img[@class='centred']";
 
+    private final HostService hostService;
+    private final XpathService xpathService;
+    private final ConnectionService cm;
+    private final HtmlProcessorService htmlProcessorService;
+
     @Autowired
-    private ConnectionManager cm;
+    public ImxHost(HostService hostService, XpathService xpathService, ConnectionService cm, HtmlProcessorService htmlProcessorService) {
+        this.hostService = hostService;
+        this.xpathService = xpathService;
+        this.cm = cm;
+        this.htmlProcessorService = htmlProcessorService;
+    }
 
     @Override
     public String getHost() {
@@ -47,16 +57,16 @@ public class ImxHost extends Host {
     }
 
     @Override
-    protected void setNameAndUrl(final String _url, final ImageFileData imageFileData, final HttpClientContext context) throws HostException {
+    public HostService.NameUrl getNameAndUrl(final String _url, final HttpClientContext context) throws HostException {
 
         String url = _url.replace("http://", "https://");
-        Response resp = getResponse(url, context);
+        HostService.Response resp = hostService.getResponse(url, context);
         Document doc = resp.getDocument();
 
         Node contDiv;
         String value = null;
         try {
-            logger.debug(String.format("Looking for xpath expression %s in %s", CONTINUE_BUTTON_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", CONTINUE_BUTTON_XPATH, url));
             contDiv = xpathService.getAsNode(doc, CONTINUE_BUTTON_XPATH);
             Node node = contDiv.getAttributes().getNamedItem("value");
             if (node != null) {
@@ -69,9 +79,9 @@ public class ImxHost extends Host {
         if (value == null) {
             throw new HostException("Failed to obtain value attribute from continue input");
         }
-        logger.debug(String.format("Click button found for %s", url));
+        log.debug(String.format("Click button found for %s", url));
         HttpClient client = cm.getClient().build();
-        HttpPost httpPost = cm.buildHttpPost(url);
+        HttpPost httpPost = cm.buildHttpPost(url, context);
         List<NameValuePair> params = new ArrayList<>();
         params.add(new BasicNameValuePair("imgContinue", value));
         try {
@@ -79,9 +89,9 @@ public class ImxHost extends Host {
         } catch (Exception e) {
             throw new HostException(e);
         }
-        logger.debug(String.format("Requesting %s", httpPost));
+        log.debug(String.format("Requesting %s", httpPost));
         try (CloseableHttpResponse response = (CloseableHttpResponse) client.execute(httpPost, context)) {
-            logger.debug(String.format("Cleaning response for %s", httpPost));
+            log.debug(String.format("Cleaning response for %s", httpPost));
             doc = htmlProcessorService.clean(EntityUtils.toString(response.getEntity()));
             EntityUtils.consumeQuietly(response.getEntity());
         } catch (IOException | HtmlProcessorException e) {
@@ -90,20 +100,19 @@ public class ImxHost extends Host {
 
         Node imgNode;
         try {
-            logger.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
             imgNode = xpathService.getAsNode(doc, IMG_XPATH);
         } catch (XpathException e) {
             throw new HostException(e);
         }
 
         try {
-            logger.debug(String.format("Resolving name and image url for %s", url));
+            log.debug(String.format("Resolving name and image url for %s", url));
             String imgTitle = imgNode.getAttributes().getNamedItem("alt").getTextContent().trim();
             String imgUrl = imgNode.getAttributes().getNamedItem("src").getTextContent().trim();
 
-            imageFileData.setImageUrl(imgUrl);
-            imageFileData.setImageName(imgTitle.isEmpty() ? imgUrl.substring(imgUrl.lastIndexOf('/') + 1) : imgTitle);
-        } catch(Exception e) {
+            return new HostService.NameUrl(imgTitle.isEmpty() ? imgUrl.substring(imgUrl.lastIndexOf('/') + 1) : imgTitle, imgUrl);
+        } catch (Exception e) {
             throw new HostException("Unexpected error occurred", e);
         }
     }

vripper-server/src/main/java/tn/mnlr/vripper/host/PimpandhostHost.java

@@ -1,27 +1,32 @@
 package tn.mnlr.vripper.host;
 
+import lombok.extern.slf4j.Slf4j;
 import org.apache.http.client.protocol.HttpClientContext;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Service;
 import org.w3c.dom.Document;
 import org.w3c.dom.Node;
 import tn.mnlr.vripper.exception.HostException;
 import tn.mnlr.vripper.exception.XpathException;
-import tn.mnlr.vripper.q.ImageFileData;
-import tn.mnlr.vripper.services.ConnectionManager;
+import tn.mnlr.vripper.services.HostService;
+import tn.mnlr.vripper.services.XpathService;
 
 @Service
+@Slf4j
 public class PimpandhostHost extends Host {
 
-    private static final Logger logger = LoggerFactory.getLogger(PimpandhostHost.class);
-
     private static final String host = "pimpandhost.com";
     private static final String IMG_XPATH = "//img[contains(@class, 'original')]";
 
+    private final HostService hostService;
+    private final XpathService xpathService;
+
     @Autowired
-    private ConnectionManager cm;
+    public PimpandhostHost(HostService hostService, XpathService xpathService) {
+        super();
+        this.hostService = hostService;
+        this.xpathService = xpathService;
+    }
 
     @Override
     public String getHost() {
@@ -34,33 +39,35 @@ public class PimpandhostHost extends Host {
     }
 
     @Override
-    protected void setNameAndUrl(final String _url, final ImageFileData imageFileData, final HttpClientContext context) throws HostException {
+    public HostService.NameUrl getNameAndUrl(final String _url, final HttpClientContext context) throws HostException {
 
         String url;
         try {
-            url = appendUri(_url.replace("http://", "https://"), "size=original");
+            url = hostService.appendUri(_url.replace("http://", "https://"), "size=original");
         } catch (Exception e) {
             throw new HostException(e);
         }
 
-        Response resp = getResponse(url, context);
+        HostService.Response resp = hostService.getResponse(url, context);
         Document doc = resp.getDocument();
 
         Node imgNode;
         try {
-            logger.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
             imgNode = xpathService.getAsNode(doc, IMG_XPATH);
         } catch (XpathException e) {
             throw new HostException(e);
         }
 
         try {
-            logger.debug(String.format("Resolving name and image url for %s", url));
+            log.debug(String.format("Resolving name and image url for %s", url));
             String imgTitle = imgNode.getAttributes().getNamedItem("alt").getTextContent().trim();
             String imgUrl = "https:" + imgNode.getAttributes().getNamedItem("src").getTextContent().trim();
 
-            imageFileData.setImageUrl(imgUrl);
-            imageFileData.setImageName(imgTitle.isEmpty() ? imgUrl.substring(imgUrl.lastIndexOf('/') + 1) : imgTitle);
+            return new HostService.NameUrl(
+                    imgTitle.isEmpty() ? imgUrl.substring(imgUrl.lastIndexOf('/') + 1) : imgTitle,
+                    imgUrl
+            );
         } catch (Exception e) {
             throw new HostException("Unexpected error occurred", e);
         }

vripper-server/src/main/java/tn/mnlr/vripper/host/PixRouteHost.java

@@ -0,0 +1,69 @@
+package tn.mnlr.vripper.host;
+
+import lombok.extern.slf4j.Slf4j;
+import org.apache.http.client.protocol.HttpClientContext;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Service;
+import org.w3c.dom.Document;
+import org.w3c.dom.Node;
+import tn.mnlr.vripper.exception.HostException;
+import tn.mnlr.vripper.exception.XpathException;
+import tn.mnlr.vripper.services.HostService;
+import tn.mnlr.vripper.services.XpathService;
+
+@Service
+@Slf4j
+public class PixRouteHost extends Host {
+
+    private static final String host = "pixroute.com";
+    private static final String IMG_XPATH = "//img[@id='imgpreview']";
+
+    private final HostService hostService;
+    private final XpathService xpathService;
+
+    @Autowired
+    public PixRouteHost(HostService hostService, XpathService xpathService) {
+        this.hostService = hostService;
+        this.xpathService = xpathService;
+    }
+
+    @Override
+    public String getHost() {
+        return host;
+    }
+
+    @Override
+    public String getLookup() {
+        return host;
+    }
+
+    @Override
+    public HostService.NameUrl getNameAndUrl(final String _url, final HttpClientContext context) throws HostException {
+
+        String url = _url.replace("http://", "https://");
+        Document doc = hostService.getResponse(url, context).getDocument();
+
+        Node imgNode;
+        try {
+            log.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
+            imgNode = xpathService.getAsNode(doc, IMG_XPATH);
+        } catch (XpathException e) {
+            throw new HostException(e);
+        }
+
+        if (imgNode == null) {
+            throw new HostException("Failed to locate image");
+        }
+
+        try {
+            log.debug(String.format("Resolving name and image url for %s", url));
+
+            return new HostService.NameUrl(
+                    imgNode.getAttributes().getNamedItem("alt").getTextContent().trim(),
+                    imgNode.getAttributes().getNamedItem("src").getTextContent().trim()
+            );
+        } catch (Exception e) {
+            throw new HostException("Unexpected error occurred", e);
+        }
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/host/PixhostHost.java

@@ -1,28 +1,32 @@
 package tn.mnlr.vripper.host;
 
+import lombok.extern.slf4j.Slf4j;
 import org.apache.http.client.protocol.HttpClientContext;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Service;
 import org.w3c.dom.Document;
 import org.w3c.dom.Node;
 import tn.mnlr.vripper.exception.HostException;
 import tn.mnlr.vripper.exception.XpathException;
-import tn.mnlr.vripper.q.ImageFileData;
-import tn.mnlr.vripper.services.ConnectionManager;
+import tn.mnlr.vripper.services.HostService;
+import tn.mnlr.vripper.services.XpathService;
 
 @Service
+@Slf4j
 public class PixhostHost extends Host {
 
-    private static final Logger logger = LoggerFactory.getLogger(PixhostHost.class);
-
     private static final String host = "pixhost.to";
     private static final String lookup = "pixhost.to/show";
     private static final String IMG_XPATH = "//img[@id='image']";
 
+    private final HostService hostService;
+    private final XpathService xpathService;
+
     @Autowired
-    private ConnectionManager cm;
+    public PixhostHost(HostService hostService, XpathService xpathService) {
+        this.hostService = hostService;
+        this.xpathService = xpathService;
+    }
 
     @Override
     public String getHost() {
@@ -35,25 +39,24 @@ public class PixhostHost extends Host {
     }
 
     @Override
-    protected void setNameAndUrl(final String url, final ImageFileData imageFileData, final HttpClientContext context) throws HostException {
+    public HostService.NameUrl getNameAndUrl(final String url, final HttpClientContext context) throws HostException {
 
-        Document doc = getResponse(url, context).getDocument();
+        Document doc = hostService.getResponse(url, context).getDocument();
 
         Node imgNode;
         try {
-            logger.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
             imgNode = xpathService.getAsNode(doc, IMG_XPATH);
         } catch (XpathException e) {
             throw new HostException(e);
         }
 
         try {
-            logger.debug(String.format("Resolving name and image url for %s", url));
+            log.debug(String.format("Resolving name and image url for %s", url));
             String imgTitle = imgNode.getAttributes().getNamedItem("alt").getTextContent().trim();
             String imgUrl = imgNode.getAttributes().getNamedItem("src").getTextContent().trim();
 
-            imageFileData.setImageUrl(imgUrl);
-            imageFileData.setImageName(imgTitle.substring(imgTitle.indexOf('_') + 1));
+            return new HostService.NameUrl(imgTitle.substring(imgTitle.indexOf('_') + 1), imgUrl);
         } catch (Exception e) {
             throw new HostException("Unexpected error occurred", e);
         }

vripper-server/src/main/java/tn/mnlr/vripper/host/PixxxelsHost.java

@@ -1,28 +1,32 @@
 package tn.mnlr.vripper.host;
 
+import lombok.extern.slf4j.Slf4j;
 import org.apache.http.client.protocol.HttpClientContext;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Service;
 import org.w3c.dom.Document;
 import org.w3c.dom.Node;
 import tn.mnlr.vripper.exception.HostException;
 import tn.mnlr.vripper.exception.XpathException;
-import tn.mnlr.vripper.q.ImageFileData;
-import tn.mnlr.vripper.services.ConnectionManager;
+import tn.mnlr.vripper.services.HostService;
+import tn.mnlr.vripper.services.XpathService;
 
 @Service
+@Slf4j
 public class PixxxelsHost extends Host {
 
-    private static final Logger logger = LoggerFactory.getLogger(PixxxelsHost.class);
-
     private static final String host = "pixxxels.cc";
     private static final String IMG_XPATH = "//*[@id='download']";
     private static final String TITLE_XPATH = "//*[contains(@class,'imagename')]";
 
+    private final HostService hostService;
+    private final XpathService xpathService;
+
     @Autowired
-    private ConnectionManager cm;
+    public PixxxelsHost(HostService hostService, XpathService xpathService) {
+        this.hostService = hostService;
+        this.xpathService = xpathService;
+    }
 
     @Override
     public String getHost() {
@@ -35,18 +39,18 @@ public class PixxxelsHost extends Host {
     }
 
     @Override
-    protected void setNameAndUrl(final String _url, final ImageFileData imageFileData, final HttpClientContext context) throws HostException {
+    public HostService.NameUrl getNameAndUrl(final String _url, final HttpClientContext context) throws HostException {
 
         String url = _url.replace("http://", "https://");
-        Response resp = getResponse(url, context);
+        HostService.Response resp = hostService.getResponse(url, context);
         Document doc = resp.getDocument();
 
         Node imgNode, titleNode;
         try {
-            logger.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
             imgNode = xpathService.getAsNode(doc, IMG_XPATH);
 
-            logger.debug(String.format("Looking for xpath expression %s in %s", TITLE_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", TITLE_XPATH, url));
             titleNode = xpathService.getAsNode(doc, TITLE_XPATH);
 
         } catch (XpathException e) {
@@ -54,12 +58,11 @@ public class PixxxelsHost extends Host {
         }
 
         try {
-            logger.debug(String.format("Resolving name and image url for %s", url));
+            log.debug(String.format("Resolving name and image url for %s", url));
             String imgTitle = titleNode.getTextContent().trim();
             String imgUrl = imgNode.getAttributes().getNamedItem("href").getTextContent().trim();
 
-            imageFileData.setImageUrl(imgUrl);
-            imageFileData.setImageName(imgTitle.isEmpty() ? imgUrl.substring(imgUrl.lastIndexOf('/') + 1) : imgTitle);
+            return new HostService.NameUrl(imgTitle.isEmpty() ? imgUrl.substring(imgUrl.lastIndexOf('/') + 1) : imgTitle, imgUrl);
         } catch (Exception e) {
             throw new HostException("Unexpected error occurred", e);
         }

vripper-server/src/main/java/tn/mnlr/vripper/host/PostImgHost.java

@@ -1,26 +1,35 @@
 package tn.mnlr.vripper.host;
 
+import lombok.extern.slf4j.Slf4j;
 import org.apache.http.client.protocol.HttpClientContext;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
+import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Service;
 import org.w3c.dom.Document;
 import org.w3c.dom.Node;
 import tn.mnlr.vripper.exception.HostException;
 import tn.mnlr.vripper.exception.XpathException;
-import tn.mnlr.vripper.q.ImageFileData;
+import tn.mnlr.vripper.services.HostService;
+import tn.mnlr.vripper.services.XpathService;
 
 import java.util.Optional;
 
 @Service
+@Slf4j
 public class PostImgHost extends Host {
 
-    private static final Logger logger = LoggerFactory.getLogger(PostImgHost.class);
-
     private static final String host = "postimg.cc";
     private static final String TITLE_XPATH = "//span[contains(@class,'imagename')]";
     private static final String IMG_XPATH = "//a[@id='download']";
 
+    private final HostService hostService;
+    private final XpathService xpathService;
+
+    @Autowired
+    public PostImgHost(HostService hostService, XpathService xpathService) {
+        this.hostService = hostService;
+        this.xpathService = xpathService;
+    }
+
     @Override
     public String getHost() {
         return host;
@@ -32,28 +41,27 @@ public class PostImgHost extends Host {
     }
 
     @Override
-    protected void setNameAndUrl(final String _url, final ImageFileData imageFileData, final HttpClientContext context) throws HostException {
+    public HostService.NameUrl getNameAndUrl(final String _url, final HttpClientContext context) throws HostException {
 
         String url = _url.replace("http://", "https://");
-        Document doc = getResponse(url, context).getDocument();
+        Document doc = hostService.getResponse(url, context).getDocument();
 
         Node urlNode, titleNode;
         try {
-            logger.debug(String.format("Looking for xpath expression %s in %s", TITLE_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", TITLE_XPATH, url));
             titleNode = xpathService.getAsNode(doc, TITLE_XPATH);
 
-            logger.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
             urlNode = xpathService.getAsNode(doc, IMG_XPATH);
         } catch (XpathException e) {
             throw new HostException(e);
         }
 
         try {
-            logger.debug(String.format("Resolving name and image url for %s", url));
-            String imgTitle = Optional.ofNullable(titleNode).map(node -> node.getTextContent().trim()).orElseGet(() -> getDefaultImageName(url));
+            log.debug(String.format("Resolving name and image url for %s", url));
+            String imgTitle = Optional.ofNullable(titleNode).map(node -> node.getTextContent().trim()).orElseGet(() -> hostService.getDefaultImageName(url));
 
-            imageFileData.setImageUrl(urlNode.getAttributes().getNamedItem("href").getTextContent().trim());
-            imageFileData.setImageName(imgTitle);
+            return new HostService.NameUrl(imgTitle, urlNode.getAttributes().getNamedItem("href").getTextContent().trim());
         } catch (Exception e) {
             throw new HostException("Unexpected error occurred", e);
         }

vripper-server/src/main/java/tn/mnlr/vripper/host/TurboImageHost.java

@@ -1,28 +1,32 @@
 package tn.mnlr.vripper.host;
 
+import lombok.extern.slf4j.Slf4j;
 import org.apache.http.client.protocol.HttpClientContext;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Service;
 import org.w3c.dom.Document;
 import org.w3c.dom.Node;
 import tn.mnlr.vripper.exception.HostException;
 import tn.mnlr.vripper.exception.XpathException;
-import tn.mnlr.vripper.q.ImageFileData;
-import tn.mnlr.vripper.services.ConnectionManager;
+import tn.mnlr.vripper.services.HostService;
+import tn.mnlr.vripper.services.XpathService;
 
 @Service
+@Slf4j
 public class TurboImageHost extends Host {
 
-    private static final Logger logger = LoggerFactory.getLogger(TurboImageHost.class);
-
     private static final String host = "turboimagehost.com";
     private static final String TITLE_XPATH = "//div[contains(@class,'titleFullS')]/h1";
-    private static final String IMG_XPATH = "//img[@id='uImage']";
+    private static final String IMG_XPATH = "//img[@id='imageid']";
+
+    private final HostService hostService;
+    private final XpathService xpathService;
 
     @Autowired
-    private ConnectionManager cm;
+    public TurboImageHost(HostService hostService, XpathService xpathService) {
+        this.hostService = hostService;
+        this.xpathService = xpathService;
+    }
 
     @Override
     public String getHost() {
@@ -35,16 +39,16 @@ public class TurboImageHost extends Host {
     }
 
     @Override
-    protected void setNameAndUrl(final String url, final ImageFileData imageFileData, final HttpClientContext context) throws HostException {
+    public HostService.NameUrl getNameAndUrl(final String url, final HttpClientContext context) throws HostException {
 
-        Document doc = getResponse(url, context).getDocument();
+        Document doc = hostService.getResponse(url, context).getDocument();
 
         String title;
         try {
-            logger.debug(String.format("Looking for xpath expression %s in %s", TITLE_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", TITLE_XPATH, url));
             Node titleNode = xpathService.getAsNode(doc, TITLE_XPATH);
-            logger.debug(String.format("Resolving name for %s", url));
-            if(titleNode != null) {
+            log.debug(String.format("Resolving name for %s", url));
+            if (titleNode != null) {
                 title = titleNode.getTextContent().trim();
             } else {
                 title = null;
@@ -53,14 +57,13 @@ public class TurboImageHost extends Host {
             throw new HostException(e);
         }
 
-        if(title == null || title.isEmpty()) {
-            title = getDefaultImageName(url);
+        if (title == null || title.isEmpty()) {
+            title = hostService.getDefaultImageName(url);
         }
 
         try {
             Node urlNode = xpathService.getAsNode(doc, IMG_XPATH);
-            imageFileData.setImageUrl(urlNode.getAttributes().getNamedItem("src").getTextContent().trim());
-            imageFileData.setImageName(title);
+            return new HostService.NameUrl(title, urlNode.getAttributes().getNamedItem("src").getTextContent().trim());
         } catch (Exception e) {
             throw new HostException("Unexpected error occurred", e);
         }

vripper-server/src/main/java/tn/mnlr/vripper/host/ViprImHost.java

@@ -0,0 +1,65 @@
+package tn.mnlr.vripper.host;
+
+import lombok.extern.slf4j.Slf4j;
+import org.apache.http.client.protocol.HttpClientContext;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Service;
+import org.w3c.dom.Document;
+import org.w3c.dom.Node;
+import tn.mnlr.vripper.exception.HostException;
+import tn.mnlr.vripper.exception.XpathException;
+import tn.mnlr.vripper.services.HostService;
+import tn.mnlr.vripper.services.XpathService;
+
+import java.util.Optional;
+
+@Service
+@Slf4j
+public class ViprImHost extends Host {
+
+    private static final String IMG_XPATH = "//img[contains(@class, 'img')]";
+    private static final String host = "vipr.im";
+
+    private final HostService hostService;
+    private final XpathService xpathService;
+
+    @Autowired
+    public ViprImHost(HostService hostService, XpathService xpathService) {
+        this.hostService = hostService;
+        this.xpathService = xpathService;
+    }
+
+    @Override
+    public String getHost() {
+        return host;
+    }
+
+    @Override
+    public String getLookup() {
+        return host;
+    }
+
+    @Override
+    public HostService.NameUrl getNameAndUrl(final String url, final HttpClientContext context) throws HostException {
+
+        Document doc = hostService.getResponse(url, context).getDocument();
+
+        Node imgNode;
+        try {
+            log.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
+            imgNode = xpathService.getAsNode(doc, IMG_XPATH);
+        } catch (XpathException e) {
+            throw new HostException(e);
+        }
+
+        try {
+            log.debug(String.format("Resolving name and image url for %s", url));
+            String imgTitle = Optional.ofNullable(imgNode.getAttributes().getNamedItem("alt")).map(Node::getTextContent).map(String::trim).orElse(null);
+            String imgUrl = imgNode.getAttributes().getNamedItem("src").getTextContent().trim();
+
+            return new HostService.NameUrl(imgTitle, imgUrl);
+        } catch (Exception e) {
+            throw new HostException("Unexpected error occurred", e);
+        }
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/Management.java

@@ -0,0 +1,70 @@
+package tn.mnlr.vripper.jpa;
+
+import org.springframework.beans.factory.annotation.Value;
+import org.springframework.jdbc.core.JdbcTemplate;
+import org.springframework.scheduling.annotation.EnableScheduling;
+import org.springframework.scheduling.annotation.Scheduled;
+import org.springframework.stereotype.Component;
+import tn.mnlr.vripper.download.DownloadService;
+import tn.mnlr.vripper.services.ThreadPoolService;
+
+import javax.annotation.PostConstruct;
+import javax.annotation.PreDestroy;
+import java.io.File;
+import java.time.LocalDate;
+import java.time.LocalDateTime;
+import java.time.format.DateTimeFormatter;
+import java.util.Optional;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+@Component
+@EnableScheduling
+public class Management {
+
+    public static final DateTimeFormatter FORMATTER = DateTimeFormatter.ofPattern("yyyy-MM-dd");
+    private static final Pattern BACKUP_FILE_PATTERN = Pattern.compile("^db_(\\d{4}-\\d{2}-\\d{2})\\.tar\\.gz$");
+
+    private final JdbcTemplate jdbcTemplate;
+    private final String backupFolder;
+    private final ThreadPoolService threadPoolService;
+    private final DownloadService downloadService;
+
+    public Management(JdbcTemplate jdbcTemplate, @Value("${base.dir}") String baseDir, @Value("${base.dir.name}") String baseDirName, ThreadPoolService threadPoolService, DownloadService downloadService) {
+        this.jdbcTemplate = jdbcTemplate;
+        this.threadPoolService = threadPoolService;
+        this.downloadService = downloadService;
+        this.backupFolder = baseDir + File.separator + baseDirName + File.separator + "backup";
+    }
+
+    @PreDestroy
+    public void destroy() {
+
+        try {
+            threadPoolService.destroy();
+            downloadService.destroy();
+            this.jdbcTemplate.execute("SHUTDOWN");
+        } catch (Exception ignored) {
+            Thread.currentThread().interrupt();
+        }
+    }
+
+    @PostConstruct
+    @Scheduled(cron = "0 0 0 ? * *")
+    private void backup() {
+
+        for (File file : Optional.ofNullable(new File(backupFolder).listFiles()).orElse(new File[]{})) {
+            Matcher matcher = BACKUP_FILE_PATTERN.matcher(file.getName());
+            if (matcher.find()) {
+                LocalDate localDate = LocalDate.parse(matcher.group(1), FORMATTER);
+                if (localDate.isEqual(LocalDate.now())) {
+                    return;
+                }
+            }
+        }
+
+        // create a backup file
+        File backupFile = new File(backupFolder, String.format("db_%s.tar.gz", LocalDateTime.now().format(FORMATTER)));
+        jdbcTemplate.execute(String.format("BACKUP DATABASE TO '%s' BLOCKING", backupFile.getPath()));
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/domain/Image.java

@@ -0,0 +1,70 @@
+package tn.mnlr.vripper.jpa.domain;
+
+import com.fasterxml.jackson.annotation.JsonIgnore;
+import lombok.Getter;
+import lombok.NoArgsConstructor;
+import lombok.Setter;
+import lombok.ToString;
+import tn.mnlr.vripper.host.Host;
+import tn.mnlr.vripper.jpa.domain.enums.Status;
+
+import java.util.Objects;
+
+@Getter
+@Setter
+@ToString
+@NoArgsConstructor
+public class Image {
+
+    @JsonIgnore
+    protected Long id;
+
+    @JsonIgnore
+    private Host host;
+
+    private String url;
+
+    private int index;
+
+    private long current = 0;
+
+    private long total = 0;
+
+    private Status status;
+
+    private String postId;
+
+    @JsonIgnore
+    private Long postIdRef;
+
+    public Image(String postId, String url, Host host, int index) {
+        this.postId = postId;
+        this.url = url;
+        this.host = host;
+        this.index = index;
+        status = Status.STOPPED;
+    }
+
+
+    public void increase(int read) {
+        current += read;
+    }
+
+    public void init() {
+        current = 0;
+        status = Status.STOPPED;
+    }
+
+    @Override
+    public boolean equals(Object o) {
+        if (this == o) return true;
+        if (o == null || getClass() != o.getClass()) return false;
+        Image image = (Image) o;
+        return Objects.equals(url, image.url);
+    }
+
+    @Override
+    public int hashCode() {
+        return Objects.hash(url);
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/domain/Metadata.java

@@ -0,0 +1,24 @@
+package tn.mnlr.vripper.jpa.domain;
+
+import com.fasterxml.jackson.annotation.JsonIgnore;
+import lombok.Getter;
+import lombok.NoArgsConstructor;
+import lombok.Setter;
+
+import java.util.Collections;
+import java.util.List;
+
+@Getter
+@Setter
+@NoArgsConstructor
+public class Metadata {
+
+    @JsonIgnore
+    private Long id;
+
+    private List<String> resolvedNames = Collections.emptyList();
+
+    private String postedBy;
+
+    private Long postIdRef;
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/domain/Post.java

@@ -0,0 +1,78 @@
+package tn.mnlr.vripper.jpa.domain;
+
+import com.fasterxml.jackson.annotation.JsonIgnore;
+import lombok.Getter;
+import lombok.NoArgsConstructor;
+import lombok.Setter;
+import lombok.ToString;
+import tn.mnlr.vripper.jpa.domain.enums.Status;
+
+import java.util.Collections;
+import java.util.Objects;
+import java.util.Set;
+
+@Getter
+@Setter
+@ToString
+@NoArgsConstructor
+public class Post {
+
+    @JsonIgnore
+    private Long id;
+
+    private Status status;
+
+    private String postId;
+
+    private String threadTitle;
+
+    private String threadId;
+
+    private String title;
+
+    private String url;
+
+    private int done;
+
+    private int total;
+
+    private Set<String> hosts;
+
+    private String forum;
+
+    @JsonIgnore
+    private String securityToken;
+
+    @JsonIgnore
+    private String downloadDirectory;
+
+    private boolean thanked;
+
+    private Set<String> previews = Collections.emptySet();
+
+    private Metadata metadata;
+
+    public Post(String title, String url, String postId, String threadId, String threadTitle, String forum, String securityToken) {
+        this.title = title;
+        this.url = url;
+        this.postId = postId;
+        this.forum = forum;
+        this.threadId = threadId;
+        this.threadTitle = threadTitle;
+        this.securityToken = securityToken;
+        status = Status.STOPPED;
+    }
+
+    @Override
+    public boolean equals(Object o) {
+        if (this == o) return true;
+        if (o == null || getClass() != o.getClass()) return false;
+        Post post = (Post) o;
+        return Objects.equals(postId, post.postId);
+    }
+
+    @Override
+    public int hashCode() {
+        return Objects.hash(postId);
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/domain/Queued.java

@@ -0,0 +1,57 @@
+package tn.mnlr.vripper.jpa.domain;
+
+import com.fasterxml.jackson.annotation.JsonIgnore;
+import lombok.Getter;
+import lombok.NoArgsConstructor;
+import lombok.Setter;
+import lombok.ToString;
+
+import java.util.Objects;
+
+@Getter
+@Setter
+@NoArgsConstructor
+@ToString
+public class Queued {
+
+    @JsonIgnore
+    private Long id;
+
+    private String link;
+
+    private String threadId;
+
+    private String postId;
+
+    private int total = 0;
+
+    private boolean loading = true;
+
+    public Queued(String link, String threadId, String postId) {
+        this();
+        this.link = link;
+        this.threadId = threadId;
+        this.postId = postId;
+    }
+
+    public void done() {
+        this.loading = false;
+    }
+
+    public void increment() {
+        this.total++;
+    }
+
+    @Override
+    public boolean equals(Object o) {
+        if (this == o) return true;
+        if (o == null || getClass() != o.getClass()) return false;
+        Queued that = (Queued) o;
+        return Objects.equals(threadId, that.threadId);
+    }
+
+    @Override
+    public int hashCode() {
+        return Objects.hash(threadId);
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/domain/enums/Status.java

@@ -0,0 +1,5 @@
+package tn.mnlr.vripper.jpa.domain.enums;
+
+public enum Status {
+    PENDING, DOWNLOADING, COMPLETE, ERROR, STOPPED, PARTIAL
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/repositories/IImageRepository.java

@@ -0,0 +1,32 @@
+package tn.mnlr.vripper.jpa.repositories;
+
+import tn.mnlr.vripper.jpa.domain.Image;
+import tn.mnlr.vripper.jpa.domain.enums.Status;
+
+import java.util.List;
+import java.util.Optional;
+
+public interface IImageRepository extends IRepository {
+
+    Image save(Image image);
+
+    void deleteAllByPostId(String postId);
+
+    List<Image> findByPostId(String postId);
+
+    Integer countError();
+
+    List<Image> findByPostIdAndIsNotCompleted(String postId);
+
+    int stopByPostIdAndIsNotCompleted(String postId);
+
+    List<Image> findByPostIdAndIsError(String postId);
+
+    Optional<Image> findById(Long id);
+
+    int updateStatus(Status status, Long id);
+
+    int updateCurrent(long current, Long id);
+
+    int updateTotal(long total, Long id);
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/repositories/IMetadataRepository.java

@@ -0,0 +1,16 @@
+package tn.mnlr.vripper.jpa.repositories;
+
+import tn.mnlr.vripper.jpa.domain.Metadata;
+
+import java.util.Optional;
+
+public interface IMetadataRepository extends IRepository {
+
+    Metadata save(Metadata metadata);
+
+    Optional<Metadata> findById(Long id);
+
+    Optional<Metadata> findByPostId(String postId);
+
+    int deleteByPostId(String postId);
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/repositories/IPostRepository.java

@@ -0,0 +1,36 @@
+package tn.mnlr.vripper.jpa.repositories;
+
+import tn.mnlr.vripper.jpa.domain.Post;
+import tn.mnlr.vripper.jpa.domain.enums.Status;
+
+import java.util.List;
+import java.util.Optional;
+
+public interface IPostRepository extends IRepository {
+
+    Post save(Post post);
+
+    Optional<Post> findByPostId(String postId);
+
+    Optional<Post> findById(Long id);
+
+    List<String> findCompleted();
+
+    List<Post> findAll();
+
+    boolean existByPostId(String postId);
+
+    int setDownloadingToStopped();
+
+    int deleteByPostId(String postId);
+
+    int updateStatus(Status status, Long id);
+
+    int updateDone(int done, Long id);
+
+    int updateFolderName(String postFolderName, Long id);
+
+    int updateTitle(String title, Long id);
+
+    int updateThanked(boolean thanked, Long id);
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/repositories/IQueuedRepository.java

@@ -0,0 +1,18 @@
+package tn.mnlr.vripper.jpa.repositories;
+
+import tn.mnlr.vripper.jpa.domain.Queued;
+
+import java.util.List;
+import java.util.Optional;
+
+public interface IQueuedRepository extends IRepository {
+    Queued save(Queued queued);
+
+    Optional<Queued> findByThreadId(String threadId);
+
+    List<Queued> findAll();
+
+    Optional<Queued> findById(Long id);
+
+    int deleteByThreadId(String threadId);
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/repositories/IRepository.java

@@ -0,0 +1,6 @@
+package tn.mnlr.vripper.jpa.repositories;
+
+public interface IRepository {
+
+    void init();
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/repositories/impl/ImageRepository.java

@@ -0,0 +1,153 @@
+package tn.mnlr.vripper.jpa.repositories.impl;
+
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.context.ApplicationEventPublisher;
+import org.springframework.context.ApplicationEventPublisherAware;
+import org.springframework.jdbc.core.JdbcTemplate;
+import org.springframework.stereotype.Service;
+import tn.mnlr.vripper.event.ImageUpdateEvent;
+import tn.mnlr.vripper.jpa.domain.Image;
+import tn.mnlr.vripper.jpa.domain.enums.Status;
+import tn.mnlr.vripper.jpa.repositories.IImageRepository;
+
+import java.util.List;
+import java.util.Optional;
+import java.util.concurrent.atomic.AtomicLong;
+
+@Service
+public class ImageRepository implements IImageRepository, ApplicationEventPublisherAware {
+
+    private final JdbcTemplate jdbcTemplate;
+    private final AtomicLong counter = new AtomicLong(0);
+    private ApplicationEventPublisher applicationEventPublisher;
+
+    @Autowired
+    public ImageRepository(JdbcTemplate jdbcTemplate) {
+        this.jdbcTemplate = jdbcTemplate;
+    }
+
+    @Override
+    public void init() {
+        Long maxId = jdbcTemplate.queryForObject(
+                "SELECT MAX(ID) FROM IMAGE",
+                Long.class
+        );
+        if (maxId == null) {
+            maxId = 0L;
+        }
+        counter.set(maxId);
+    }
+
+    @Override
+    public Image save(Image image) {
+        long id = counter.incrementAndGet();
+        jdbcTemplate.update(
+                "INSERT INTO IMAGE (ID, CURRENT, HOST, INDEX, POST_ID, STATUS, TOTAL, URL, POST_ID_REF) VALUES (?,?,?,?,?,?,?,?,?)",
+                id,
+                image.getCurrent(),
+                image.getHost().getHost(),
+                image.getIndex(),
+                image.getPostId(),
+                image.getStatus().name(),
+                image.getTotal(),
+                image.getUrl(),
+                image.getPostIdRef()
+        );
+        image.setId(id);
+        applicationEventPublisher.publishEvent(new ImageUpdateEvent(ImageRepository.class, id));
+        return image;
+    }
+
+    @Override
+    public void deleteAllByPostId(String postId) {
+        jdbcTemplate.update("DELETE FROM IMAGE WHERE POST_ID = ?", postId);
+    }
+
+    @Override
+    public List<Image> findByPostId(String postId) {
+        return jdbcTemplate.query(
+                "SELECT * FROM IMAGE WHERE POST_ID = ?",
+                new ImageRowMapper(),
+                postId);
+    }
+
+    @Override
+    public Integer countError() {
+        return jdbcTemplate.queryForObject(
+                "SELECT COUNT(*) FROM IMAGE AS image WHERE image.STATUS = 'ERROR'",
+                Integer.class
+        );
+    }
+
+    @Override
+    public List<Image> findByPostIdAndIsNotCompleted(String postId) {
+        return jdbcTemplate.query(
+                "SELECT * FROM IMAGE AS image WHERE image.POST_ID = ? AND image.STATUS <> 'COMPLETE'",
+                new ImageRowMapper(),
+                postId);
+    }
+
+    @Override
+    public int stopByPostIdAndIsNotCompleted(String postId) {
+        return jdbcTemplate.update(
+                "UPDATE IMAGE AS image SET image.STATUS = 'STOPPED' WHERE image.POST_ID = ? AND image.STATUS <> 'COMPLETE'",
+                postId
+        );
+    }
+
+    @Override
+    public List<Image> findByPostIdAndIsError(String postId) {
+        return jdbcTemplate.query(
+                "SELECT * FROM IMAGE AS image WHERE image.POST_ID = ? AND image.STATUS = 'ERROR'",
+                new ImageRowMapper(),
+                postId);
+    }
+
+    @Override
+    public Optional<Image> findById(Long id) {
+        List<Image> images = jdbcTemplate.query(
+                "SELECT * FROM IMAGE AS image WHERE image.ID = ?",
+                new ImageRowMapper(),
+                id);
+        if (images.isEmpty()) {
+            return Optional.empty();
+        } else {
+            return Optional.of(images.get(0));
+        }
+    }
+
+    @Override
+    public int updateStatus(Status status, Long id) {
+        int mutationCount = jdbcTemplate.update(
+                "UPDATE IMAGE AS image SET image.STATUS = ? WHERE image.ID = ?",
+                status.name(), id
+        );
+        applicationEventPublisher.publishEvent(new ImageUpdateEvent(ImageRepository.class, id));
+        return mutationCount;
+    }
+
+    @Override
+    public int updateCurrent(long current, Long id) {
+        int mutationCount = jdbcTemplate.update(
+                "UPDATE IMAGE AS image SET image.CURRENT = ? WHERE image.ID = ?",
+                current, id
+        );
+        applicationEventPublisher.publishEvent(new ImageUpdateEvent(ImageRepository.class, id));
+        return mutationCount;
+    }
+
+    @Override
+    public int updateTotal(long total, Long id) {
+        int mutationCount = jdbcTemplate.update(
+                "UPDATE IMAGE AS image SET image.TOTAL = ? WHERE image.ID = ?",
+                total, id
+        );
+        applicationEventPublisher.publishEvent(new ImageUpdateEvent(ImageRepository.class, id));
+        return mutationCount;
+    }
+
+    @Override
+    public void setApplicationEventPublisher(ApplicationEventPublisher applicationEventPublisher) {
+        this.applicationEventPublisher = applicationEventPublisher;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/repositories/impl/ImageRowMapper.java

@@ -0,0 +1,29 @@
+package tn.mnlr.vripper.jpa.repositories.impl;
+
+import org.springframework.jdbc.core.RowMapper;
+import tn.mnlr.vripper.SpringContext;
+import tn.mnlr.vripper.host.Host;
+import tn.mnlr.vripper.jpa.domain.Image;
+import tn.mnlr.vripper.jpa.domain.enums.Status;
+
+import java.sql.ResultSet;
+import java.sql.SQLException;
+
+public class ImageRowMapper implements RowMapper<Image> {
+
+    @Override
+    public Image mapRow(ResultSet rs, int rowNum) throws SQLException {
+        Image image = new Image();
+        image.setId(rs.getLong("ID"));
+        String host = rs.getString("HOST");
+        image.setHost(SpringContext.getBeansOfType(Host.class).values().stream().filter(e -> e.getHost().equals(host)).findAny().orElse(null));
+        image.setUrl(rs.getString("URL"));
+        image.setIndex(rs.getInt("INDEX"));
+        image.setCurrent(rs.getLong("CURRENT"));
+        image.setTotal(rs.getLong("TOTAL"));
+        image.setStatus(Status.valueOf(rs.getString("STATUS")));
+        image.setPostId(rs.getString("POST_ID"));
+        image.setPostIdRef(rs.getLong("POST_ID_REF"));
+        return image;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/repositories/impl/MetadataRepository.java

@@ -0,0 +1,95 @@
+package tn.mnlr.vripper.jpa.repositories.impl;
+
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.context.ApplicationEventPublisher;
+import org.springframework.context.ApplicationEventPublisherAware;
+import org.springframework.jdbc.core.JdbcTemplate;
+import org.springframework.stereotype.Service;
+import tn.mnlr.vripper.event.MetadataUpdateEvent;
+import tn.mnlr.vripper.jpa.domain.Metadata;
+import tn.mnlr.vripper.jpa.repositories.IMetadataRepository;
+
+import java.util.List;
+import java.util.Optional;
+import java.util.concurrent.atomic.AtomicLong;
+
+@Service
+public class MetadataRepository implements IMetadataRepository, ApplicationEventPublisherAware {
+
+    private final JdbcTemplate jdbcTemplate;
+    private final AtomicLong counter = new AtomicLong(0);
+    private ApplicationEventPublisher applicationEventPublisher;
+
+    @Autowired
+    public MetadataRepository(JdbcTemplate jdbcTemplate) {
+        this.jdbcTemplate = jdbcTemplate;
+    }
+
+    @Override
+    public void init() {
+        Long maxId = jdbcTemplate.queryForObject(
+                "SELECT MAX(ID) FROM IMAGE",
+                Long.class
+        );
+        if (maxId == null) {
+            maxId = 0L;
+        }
+        counter.set(maxId);
+    }
+
+    @Override
+    public Metadata save(Metadata metadata) {
+        long id = counter.incrementAndGet();
+        jdbcTemplate.update(
+                "INSERT INTO METADATA (ID, POSTED_BY, RESOLVED_NAMES, POST_ID_REF) VALUES (?,?,?,?)",
+                id,
+                metadata.getPostedBy(),
+                String.join("%sep%", metadata.getResolvedNames()),
+                metadata.getPostIdRef()
+        );
+        metadata.setId(id);
+        applicationEventPublisher.publishEvent(new MetadataUpdateEvent(MetadataRepository.class, id, metadata.getPostIdRef()));
+        return metadata;
+    }
+
+    @Override
+    public Optional<Metadata> findById(Long id) {
+        List<Metadata> metadata = jdbcTemplate.query(
+                "SELECT * FROM METADATA AS metadata WHERE metadata.ID = ?",
+                new Object[]{id},
+                new MetadataRowMapper()
+        );
+        if (metadata.isEmpty()) {
+            return Optional.empty();
+        } else {
+            return Optional.of(metadata.get(0));
+        }
+    }
+
+    @Override
+    public Optional<Metadata> findByPostId(String postId) {
+        List<Metadata> metadata = jdbcTemplate.query(
+                "SELECT metadata.* FROM METADATA AS metadata INNER JOIN POST post ON post.ID = metadata.POST_ID_REF WHERE post.POST_ID = ?",
+                new Object[]{postId},
+                new MetadataRowMapper()
+        );
+        if (metadata.isEmpty()) {
+            return Optional.empty();
+        } else {
+            return Optional.of(metadata.get(0));
+        }
+    }
+
+    @Override
+    public int deleteByPostId(String postId) {
+        return jdbcTemplate.update(
+                "DELETE FROM METADATA AS metadata WHERE metadata.ID = (SELECT inner_metadata.ID FROM POST AS post INNER JOIN METADATA inner_metadata ON post.ID = inner_metadata.POST_ID_REF WHERE post.POST_ID = ?)",
+                postId
+        );
+    }
+
+    @Override
+    public void setApplicationEventPublisher(ApplicationEventPublisher applicationEventPublisher) {
+        this.applicationEventPublisher = applicationEventPublisher;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/repositories/impl/MetadataRowMapper.java

@@ -0,0 +1,24 @@
+package tn.mnlr.vripper.jpa.repositories.impl;
+
+import org.springframework.jdbc.core.RowMapper;
+import tn.mnlr.vripper.jpa.domain.Metadata;
+
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.util.List;
+
+public class MetadataRowMapper implements RowMapper<Metadata> {
+
+    @Override
+    public Metadata mapRow(ResultSet rs, int rowNum) throws SQLException {
+        Metadata metadata = new Metadata();
+        metadata.setId(rs.getLong("ID"));
+        metadata.setPostedBy(rs.getString("POSTED_BY"));
+        String resolvedNames = rs.getString("RESOLVED_NAMES");
+        if (resolvedNames != null && !resolvedNames.isBlank()) {
+            metadata.setResolvedNames(List.of(resolvedNames.split("%sep%")));
+        }
+        metadata.setPostIdRef(rs.getLong("POST_ID_REF"));
+        return metadata;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/repositories/impl/PostRepository.java

@@ -0,0 +1,195 @@
+package tn.mnlr.vripper.jpa.repositories.impl;
+
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.context.ApplicationEventPublisher;
+import org.springframework.context.ApplicationEventPublisherAware;
+import org.springframework.jdbc.core.JdbcTemplate;
+import org.springframework.stereotype.Service;
+import tn.mnlr.vripper.event.PostRemoveEvent;
+import tn.mnlr.vripper.event.PostUpdateEvent;
+import tn.mnlr.vripper.jpa.domain.Post;
+import tn.mnlr.vripper.jpa.domain.enums.Status;
+import tn.mnlr.vripper.jpa.repositories.IPostRepository;
+
+import java.util.List;
+import java.util.Optional;
+import java.util.concurrent.atomic.AtomicLong;
+
+@Service
+public class PostRepository implements IPostRepository, ApplicationEventPublisherAware {
+
+    private final JdbcTemplate jdbcTemplate;
+    private final AtomicLong counter = new AtomicLong(0);
+    private ApplicationEventPublisher applicationEventPublisher;
+
+    @Autowired
+    public PostRepository(JdbcTemplate jdbcTemplate) {
+        this.jdbcTemplate = jdbcTemplate;
+
+    }
+
+    @Override
+    public void init() {
+        Long maxId = jdbcTemplate.queryForObject(
+                "SELECT MAX(ID) FROM POST",
+                Long.class
+        );
+        if (maxId == null) {
+            maxId = 0L;
+        }
+        counter.set(maxId);
+    }
+
+    @Override
+    public Post save(Post post) {
+        long id = counter.incrementAndGet();
+        jdbcTemplate.update(
+                "INSERT INTO POST (ID, DONE, FORUM, HOSTS, POST_FOLDER_NAME, POST_ID, PREVIEWS, SECURITY_TOKEN, STATUS, THANKED, THREAD_ID, THREAD_TITLE, TITLE, TOTAL, URL) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)",
+                id,
+                post.getDone(),
+                post.getForum(),
+                String.join(";", post.getHosts()),
+                post.getDownloadDirectory(),
+                post.getPostId(),
+                String.join(";", post.getPreviews()),
+                post.getSecurityToken(),
+                post.getStatus().name(),
+                post.isThanked(),
+                post.getThreadId(),
+                post.getThreadTitle(),
+                post.getTitle(),
+                post.getTotal(),
+                post.getUrl()
+        );
+        post.setId(id);
+        applicationEventPublisher.publishEvent(new PostUpdateEvent(PostRepository.class, id));
+        return post;
+    }
+
+    @Override
+    public Optional<Post> findByPostId(String postId) {
+        List<Post> posts = jdbcTemplate.query(
+                "SELECT metadata.*,post.* FROM METADATA metadata FULL JOIN POST post ON metadata.POST_ID_REF = post.ID WHERE POST_ID = ?",
+                new PostRowMapper(),
+                postId);
+        if (posts.isEmpty()) {
+            return Optional.empty();
+        } else {
+            return Optional.of(posts.get(0));
+        }
+    }
+
+    @Override
+    public List<String> findCompleted() {
+        return jdbcTemplate.query(
+                "SELECT POST_ID FROM POST WHERE status = 'COMPLETE' AND done >= total",
+                ((rs, rowNum) -> rs.getString("POST_ID"))
+        );
+    }
+
+    @Override
+    public Optional<Post> findById(Long id) {
+        List<Post> posts = jdbcTemplate.query(
+                "SELECT metadata.*,post.* FROM METADATA metadata FULL JOIN POST post ON metadata.POST_ID_REF = post.ID WHERE post.ID = ?",
+                new PostRowMapper(),
+                id);
+        if (posts.isEmpty()) {
+            return Optional.empty();
+        } else {
+            return Optional.of(posts.get(0));
+        }
+    }
+
+    @Override
+    public List<Post> findAll() {
+        return jdbcTemplate.query(
+                "SELECT metadata.*,post.* FROM METADATA metadata FULL JOIN POST post ON metadata.POST_ID_REF = post.ID",
+                new PostRowMapper()
+        );
+    }
+
+    @Override
+    public boolean existByPostId(String postId) {
+        Integer count = jdbcTemplate.queryForObject(
+                "SELECT COUNT(*) FROM POST AS post WHERE post.POST_ID = ?",
+                Integer.class,
+                postId);
+        if (count == null) {
+            return false;
+        } else {
+            return count > 0;
+        }
+    }
+
+    @Override
+    public int setDownloadingToStopped() {
+        return jdbcTemplate.update(
+                "UPDATE POST AS post SET post.STATUS = 'STOPPED' WHERE post.STATUS = 'DOWNLOADING' OR post.STATUS = 'PARTIAL' OR post.STATUS = 'PENDING'"
+        );
+    }
+
+    @Override
+    public int deleteByPostId(String postId) {
+        int mutationCount = jdbcTemplate.update(
+                "DELETE FROM POST AS post WHERE post.POST_ID = ?",
+                postId
+        );
+        applicationEventPublisher.publishEvent(new PostRemoveEvent(PostRepository.class, postId));
+        return mutationCount;
+    }
+
+    @Override
+    public int updateStatus(Status status, Long id) {
+        int mutationCount = jdbcTemplate.update(
+                "UPDATE POST AS post SET post.STATUS = ? WHERE post.ID = ?",
+                status.name(), id
+        );
+        applicationEventPublisher.publishEvent(new PostUpdateEvent(PostRepository.class, id));
+        return mutationCount;
+    }
+
+    @Override
+    public int updateDone(int done, Long id) {
+        int mutationCount = jdbcTemplate.update(
+                "UPDATE POST AS post SET post.DONE = ? WHERE post.ID = ?",
+                done, id
+        );
+        applicationEventPublisher.publishEvent(new PostUpdateEvent(PostRepository.class, id));
+        return mutationCount;
+    }
+
+    @Override
+    public int updateFolderName(String postFolderName, Long id) {
+        int mutationCount = jdbcTemplate.update(
+                "UPDATE POST AS post SET post.POST_FOLDER_NAME = ? WHERE post.ID = ?",
+                postFolderName, id
+        );
+        applicationEventPublisher.publishEvent(new PostUpdateEvent(PostRepository.class, id));
+        return mutationCount;
+    }
+
+    @Override
+    public int updateTitle(String title, Long id) {
+        int mutationCount = jdbcTemplate.update(
+                "UPDATE POST AS post SET post.TITLE = ? WHERE post.ID = ?",
+                title, id
+        );
+        applicationEventPublisher.publishEvent(new PostUpdateEvent(PostRepository.class, id));
+        return mutationCount;
+    }
+
+    @Override
+    public int updateThanked(boolean thanked, Long id) {
+        int mutationCount = jdbcTemplate.update(
+                "UPDATE POST AS post SET post.THANKED = ? WHERE post.ID = ?",
+                thanked, id
+        );
+        applicationEventPublisher.publishEvent(new PostUpdateEvent(PostRepository.class, id));
+        return mutationCount;
+    }
+
+    @Override
+    public void setApplicationEventPublisher(ApplicationEventPublisher applicationEventPublisher) {
+        this.applicationEventPublisher = applicationEventPublisher;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/repositories/impl/PostRowMapper.java

@@ -0,0 +1,55 @@
+package tn.mnlr.vripper.jpa.repositories.impl;
+
+import org.springframework.jdbc.core.RowMapper;
+import tn.mnlr.vripper.jpa.domain.Metadata;
+import tn.mnlr.vripper.jpa.domain.Post;
+import tn.mnlr.vripper.jpa.domain.enums.Status;
+
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.util.List;
+import java.util.Set;
+
+public class PostRowMapper implements RowMapper<Post> {
+
+    private static final String DELIMITER = ";";
+
+    @Override
+    public Post mapRow(ResultSet rs, int rowNum) throws SQLException {
+
+        Post post = new Post();
+        post.setId(rs.getLong("post.ID"));
+        post.setStatus(Status.valueOf(rs.getString("STATUS")));
+        post.setPostId(rs.getString("POST_ID"));
+        post.setThreadTitle(rs.getString("THREAD_TITLE"));
+        post.setThreadId(rs.getString("THREAD_ID"));
+        post.setTitle(rs.getString("TITLE"));
+        post.setUrl(rs.getString("URL"));
+        post.setDone(rs.getInt("DONE"));
+        post.setTotal(rs.getInt("TOTAL"));
+        post.setHosts(Set.of(rs.getString("HOSTS").split(DELIMITER)));
+        post.setForum(rs.getString("FORUM"));
+        post.setSecurityToken(rs.getString("SECURITY_TOKEN"));
+        post.setDownloadDirectory(rs.getString("POST_FOLDER_NAME"));
+        post.setThanked(rs.getBoolean("THANKED"));
+        String previews;
+        if ((previews = rs.getString("PREVIEWS")) != null) {
+            post.setPreviews(Set.of(previews.split(DELIMITER)));
+        }
+
+        Long metadataId = rs.getLong("metadata.ID");
+        if (!rs.wasNull()) {
+            Metadata metadata = new Metadata();
+            metadata.setId(metadataId);
+            metadata.setPostIdRef(rs.getLong("POST_ID_REF"));
+            String resolvedNames = rs.getString("RESOLVED_NAMES");
+            if (resolvedNames != null && !resolvedNames.isBlank()) {
+                metadata.setResolvedNames(List.of(resolvedNames.split("%sep%")));
+            }
+            metadata.setPostedBy(rs.getString("POSTED_BY"));
+            post.setMetadata(metadata);
+        }
+
+        return post;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/repositories/impl/QueuedRepository.java

@@ -0,0 +1,108 @@
+package tn.mnlr.vripper.jpa.repositories.impl;
+
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.context.ApplicationEventPublisher;
+import org.springframework.context.ApplicationEventPublisherAware;
+import org.springframework.jdbc.core.JdbcTemplate;
+import org.springframework.stereotype.Service;
+import tn.mnlr.vripper.event.QueuedRemoveEvent;
+import tn.mnlr.vripper.event.QueuedUpdateEvent;
+import tn.mnlr.vripper.jpa.domain.Queued;
+import tn.mnlr.vripper.jpa.repositories.IQueuedRepository;
+
+import java.util.List;
+import java.util.Optional;
+import java.util.concurrent.atomic.AtomicLong;
+
+@Service
+public class QueuedRepository implements IQueuedRepository, ApplicationEventPublisherAware {
+
+    private final JdbcTemplate jdbcTemplate;
+    private final AtomicLong counter = new AtomicLong(0);
+    private ApplicationEventPublisher applicationEventPublisher;
+
+    @Autowired
+    public QueuedRepository(JdbcTemplate jdbcTemplate) {
+        this.jdbcTemplate = jdbcTemplate;
+    }
+
+    @Override
+    public void init() {
+        Long maxId = jdbcTemplate.queryForObject(
+                "SELECT MAX(ID) FROM QUEUED",
+                Long.class
+        );
+        if (maxId == null) {
+            maxId = 0L;
+        }
+        counter.set(maxId);
+    }
+
+    @Override
+    public Queued save(Queued queued) {
+        long id = counter.incrementAndGet();
+        jdbcTemplate.update(
+                "INSERT INTO QUEUED (ID, TOTAL, LINK, LOADING, POST_ID, THREAD_ID) values (?,?,?,?,?,?)",
+                id,
+                queued.getTotal(),
+                queued.getLink(),
+                queued.isLoading(),
+                queued.getPostId(),
+                queued.getThreadId()
+        );
+        queued.setId(id);
+        applicationEventPublisher.publishEvent(new QueuedUpdateEvent(QueuedRepository.class, id));
+        return queued;
+    }
+
+    @Override
+    public Optional<Queued> findByThreadId(String threadId) {
+        List<Queued> queuedList = jdbcTemplate.query(
+                "SELECT * FROM QUEUED AS queued WHERE queued.THREAD_ID = ?",
+                new Object[]{threadId},
+                new QueuedRowMapper()
+        );
+        if (queuedList.isEmpty()) {
+            return Optional.empty();
+        } else {
+            return Optional.of(queuedList.get(0));
+        }
+    }
+
+    @Override
+    public List<Queued> findAll() {
+        return jdbcTemplate.query(
+                "SELECT * FROM QUEUED",
+                new QueuedRowMapper()
+        );
+    }
+
+    @Override
+    public Optional<Queued> findById(Long id) {
+        List<Queued> queuedList = jdbcTemplate.query(
+                "SELECT * FROM QUEUED AS queued WHERE queued.ID = ?",
+                new Object[]{id},
+                new QueuedRowMapper()
+        );
+        if (queuedList.isEmpty()) {
+            return Optional.empty();
+        } else {
+            return Optional.of(queuedList.get(0));
+        }
+    }
+
+    @Override
+    public int deleteByThreadId(String threadId) {
+        int mutationCount = jdbcTemplate.update(
+                "DELETE FROM QUEUED AS queued WHERE THREAD_ID = ?",
+                threadId
+        );
+        applicationEventPublisher.publishEvent(new QueuedRemoveEvent(QueuedRepository.class, threadId));
+        return mutationCount;
+    }
+
+    @Override
+    public void setApplicationEventPublisher(ApplicationEventPublisher applicationEventPublisher) {
+        this.applicationEventPublisher = applicationEventPublisher;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/repositories/impl/QueuedRowMapper.java

@@ -0,0 +1,22 @@
+package tn.mnlr.vripper.jpa.repositories.impl;
+
+import org.springframework.jdbc.core.RowMapper;
+import tn.mnlr.vripper.jpa.domain.Queued;
+
+import java.sql.ResultSet;
+import java.sql.SQLException;
+
+public class QueuedRowMapper implements RowMapper<Queued> {
+
+    @Override
+    public Queued mapRow(ResultSet rs, int rowNum) throws SQLException {
+        Queued queued = new Queued();
+        queued.setId(rs.getLong("ID"));
+        queued.setLink(rs.getString("LINK"));
+        queued.setThreadId(rs.getString("THREAD_ID"));
+        queued.setPostId(rs.getString("POST_ID"));
+        queued.setTotal(rs.getInt("TOTAL"));
+        queued.setLoading(rs.getBoolean("LOADING"));
+        return queued;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/listener/DataEventListener.java

@@ -0,0 +1,7 @@
+package tn.mnlr.vripper.listener;
+
+import reactor.core.publisher.Flux;
+
+public interface DataEventListener<T> {
+    Flux<T> getDataFlux();
+}

vripper-server/src/main/java/tn/mnlr/vripper/listener/EmitHandler.java

@@ -0,0 +1,7 @@
+package tn.mnlr.vripper.listener;
+
+import reactor.core.publisher.Sinks.EmitFailureHandler;
+
+public class EmitHandler {
+    public static final EmitFailureHandler RETRY = (signalType, emitResult) -> true;
+}

vripper-server/src/main/java/tn/mnlr/vripper/listener/ImageUpdateEventListener.java

@@ -0,0 +1,32 @@
+package tn.mnlr.vripper.listener;
+
+import lombok.extern.slf4j.Slf4j;
+import org.springframework.context.ApplicationListener;
+import org.springframework.stereotype.Component;
+import reactor.core.publisher.Flux;
+import reactor.core.publisher.Sinks;
+import tn.mnlr.vripper.event.ImageUpdateEvent;
+
+import javax.annotation.PreDestroy;
+
+@Component
+@Slf4j
+public class ImageUpdateEventListener implements ApplicationListener<ImageUpdateEvent>, DataEventListener<ImageUpdateEvent> {
+
+    private final Sinks.Many<ImageUpdateEvent> sink = Sinks.many().multicast().onBackpressureBuffer();
+
+    @Override
+    public void onApplicationEvent(ImageUpdateEvent event) {
+        sink.emitNext(event, EmitHandler.RETRY);
+    }
+
+    @Override
+    public Flux<ImageUpdateEvent> getDataFlux() {
+        return sink.asFlux();
+    }
+
+    @PreDestroy
+    private void destroy() {
+        sink.emitComplete(Sinks.EmitFailureHandler.FAIL_FAST);
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/listener/MetadataUpdateEventListener.java

@@ -0,0 +1,32 @@
+package tn.mnlr.vripper.listener;
+
+import lombok.extern.slf4j.Slf4j;
+import org.springframework.context.ApplicationListener;
+import org.springframework.stereotype.Component;
+import reactor.core.publisher.Flux;
+import reactor.core.publisher.Sinks;
+import tn.mnlr.vripper.event.MetadataUpdateEvent;
+
+import javax.annotation.PreDestroy;
+
+@Component
+@Slf4j
+public class MetadataUpdateEventListener implements ApplicationListener<MetadataUpdateEvent>, DataEventListener<MetadataUpdateEvent> {
+
+    private final Sinks.Many<MetadataUpdateEvent> sink = Sinks.many().multicast().onBackpressureBuffer();
+
+    @Override
+    public void onApplicationEvent(MetadataUpdateEvent event) {
+        sink.emitNext(event, (signalType, emitResult) -> true);
+    }
+
+    @Override
+    public Flux<MetadataUpdateEvent> getDataFlux() {
+        return sink.asFlux();
+    }
+
+    @PreDestroy
+    private void destroy() {
+        sink.emitComplete(Sinks.EmitFailureHandler.FAIL_FAST);
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/listener/PostRemoveEventListener.java

@@ -0,0 +1,32 @@
+package tn.mnlr.vripper.listener;
+
+import lombok.extern.slf4j.Slf4j;
+import org.springframework.context.ApplicationListener;
+import org.springframework.stereotype.Component;
+import reactor.core.publisher.Flux;
+import reactor.core.publisher.Sinks;
+import tn.mnlr.vripper.event.PostRemoveEvent;
+
+import javax.annotation.PreDestroy;
+
+@Component
+@Slf4j
+public class PostRemoveEventListener implements ApplicationListener<PostRemoveEvent>, DataEventListener<PostRemoveEvent> {
+
+    private final Sinks.Many<PostRemoveEvent> sink = Sinks.many().multicast().onBackpressureBuffer();
+
+    @Override
+    public void onApplicationEvent(PostRemoveEvent event) {
+        sink.emitNext(event, EmitHandler.RETRY);
+    }
+
+    @Override
+    public Flux<PostRemoveEvent> getDataFlux() {
+        return sink.asFlux();
+    }
+
+    @PreDestroy
+    private void destroy() {
+        sink.emitComplete(Sinks.EmitFailureHandler.FAIL_FAST);
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/listener/PostUpdateEventListener.java

@@ -0,0 +1,32 @@
+package tn.mnlr.vripper.listener;
+
+import lombok.extern.slf4j.Slf4j;
+import org.springframework.context.ApplicationListener;
+import org.springframework.stereotype.Component;
+import reactor.core.publisher.Flux;
+import reactor.core.publisher.Sinks;
+import tn.mnlr.vripper.event.PostUpdateEvent;
+
+import javax.annotation.PreDestroy;
+
+@Component
+@Slf4j
+public class PostUpdateEventListener implements ApplicationListener<PostUpdateEvent>, DataEventListener<PostUpdateEvent> {
+
+    private final Sinks.Many<PostUpdateEvent> sink = Sinks.many().multicast().onBackpressureBuffer();
+
+    @Override
+    public void onApplicationEvent(PostUpdateEvent event) {
+        sink.emitNext(event, EmitHandler.RETRY);
+    }
+
+    @Override
+    public Flux<PostUpdateEvent> getDataFlux() {
+        return sink.asFlux();
+    }
+
+    @PreDestroy
+    private void destroy() {
+        sink.emitComplete(Sinks.EmitFailureHandler.FAIL_FAST);
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/listener/QueuedRemoveEventListener.java

@@ -0,0 +1,32 @@
+package tn.mnlr.vripper.listener;
+
+import lombok.extern.slf4j.Slf4j;
+import org.springframework.context.ApplicationListener;
+import org.springframework.stereotype.Component;
+import reactor.core.publisher.Flux;
+import reactor.core.publisher.Sinks;
+import tn.mnlr.vripper.event.QueuedRemoveEvent;
+
+import javax.annotation.PreDestroy;
+
+@Component
+@Slf4j
+public class QueuedRemoveEventListener implements ApplicationListener<QueuedRemoveEvent>, DataEventListener<QueuedRemoveEvent> {
+
+    private final Sinks.Many<QueuedRemoveEvent> sink = Sinks.many().multicast().onBackpressureBuffer();
+
+    @Override
+    public void onApplicationEvent(QueuedRemoveEvent event) {
+        sink.emitNext(event, (signalType, emitResult) -> true);
+    }
+
+    @Override
+    public Flux<QueuedRemoveEvent> getDataFlux() {
+        return sink.asFlux();
+    }
+
+    @PreDestroy
+    private void destroy() {
+        sink.emitComplete(Sinks.EmitFailureHandler.FAIL_FAST);
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/listener/QueuedUpdateEventListener.java

@@ -0,0 +1,32 @@
+package tn.mnlr.vripper.listener;
+
+import lombok.extern.slf4j.Slf4j;
+import org.springframework.context.ApplicationListener;
+import org.springframework.stereotype.Component;
+import reactor.core.publisher.Flux;
+import reactor.core.publisher.Sinks;
+import tn.mnlr.vripper.event.QueuedUpdateEvent;
+
+import javax.annotation.PreDestroy;
+
+@Component
+@Slf4j
+public class QueuedUpdateEventListener implements ApplicationListener<QueuedUpdateEvent>, DataEventListener<QueuedUpdateEvent> {
+
+    private final Sinks.Many<QueuedUpdateEvent> sink = Sinks.many().multicast().onBackpressureBuffer();
+
+    @Override
+    public void onApplicationEvent(QueuedUpdateEvent event) {
+        sink.emitNext(event, (signalType, emitResult) -> true);
+    }
+
+    @Override
+    public Flux<QueuedUpdateEvent> getDataFlux() {
+        return sink.asFlux();
+    }
+
+    @PreDestroy
+    private void destroy() {
+        sink.emitComplete(Sinks.EmitFailureHandler.FAIL_FAST);
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/q/DownloadJob.java

@@ -1,48 +0,0 @@
-package tn.mnlr.vripper.q;
-
-import lombok.Getter;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import tn.mnlr.vripper.entities.Image;
-
-import java.util.Objects;
-import java.util.concurrent.Callable;
-
-public class DownloadJob implements Callable<Image> {
-
-    private static final Logger logger = LoggerFactory.getLogger(DownloadJob.class);
-
-    @Getter
-    private final Image image;
-
-    @Getter
-    private final ImageFileData imageFileData = new ImageFileData();
-
-    DownloadJob(Image image) {
-        this.image = image;
-    }
-
-    @Override
-    public Image call() throws Exception {
-
-        logger.debug(String.format("Starting downloading %s", image.getUrl()));
-        image.setStatus(Image.Status.DOWNLOADING);
-        image.setCurrent(0);
-        image.getHost().download(image, imageFileData);
-        image.setStatus(Image.Status.COMPLETE);
-        return image;
-    }
-
-    @Override
-    public boolean equals(Object o) {
-        if (this == o) return true;
-        if (o == null || getClass() != o.getClass()) return false;
-        DownloadJob that = (DownloadJob) o;
-        return image.equals(that.image);
-    }
-
-    @Override
-    public int hashCode() {
-        return Objects.hash(image);
-    }
-}

vripper-server/src/main/java/tn/mnlr/vripper/q/DownloadQ.java

@@ -1,83 +0,0 @@
-package tn.mnlr.vripper.q;
-
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import org.springframework.beans.factory.annotation.Autowired;
-import org.springframework.stereotype.Service;
-import tn.mnlr.vripper.entities.Image;
-import tn.mnlr.vripper.entities.Post;
-import tn.mnlr.vripper.host.Host;
-import tn.mnlr.vripper.services.AppSettingsService;
-import tn.mnlr.vripper.services.AppStateService;
-
-import javax.annotation.PostConstruct;
-import java.util.ArrayList;
-import java.util.Iterator;
-import java.util.List;
-import java.util.Map;
-import java.util.concurrent.BlockingDeque;
-import java.util.concurrent.ConcurrentHashMap;
-import java.util.concurrent.LinkedBlockingDeque;
-
-@Service
-public class DownloadQ {
-
-    private static final Logger logger = LoggerFactory.getLogger(DownloadQ.class);
-
-    @Autowired
-    private AppStateService appStateService;
-
-    private final ConcurrentHashMap<Host, BlockingDeque<DownloadJob>> downloadQ = new ConcurrentHashMap<>();
-    @Autowired
-    private AppSettingsService appSettingsService;
-    @Autowired
-    private List<Host> hosts;
-
-    @PostConstruct
-    private void init() {
-        hosts.forEach(host -> downloadQ.put(host, new LinkedBlockingDeque<>()));
-    }
-
-    public void put(Image image) throws InterruptedException {
-        logger.debug(String.format("Enqueuing a job for %s", image.getUrl()));
-        image.init();
-        DownloadJob downloadJob = new DownloadJob(image);
-        downloadQ.get(downloadJob.getImage().getHost()).putLast(downloadJob);
-        appStateService.newDownloadJob(downloadJob);
-    }
-
-    public void remove(final DownloadJob downloadJob) {
-        downloadQ.get(downloadJob.getImage().getHost()).remove(downloadJob);
-    }
-
-    public List<DownloadJob> peek() {
-        List<DownloadJob> downloadJobs = new ArrayList<>();
-        if (hosts.size() == 0) {
-            return downloadJobs;
-        }
-        for (Host host : hosts) {
-            Iterator<DownloadJob> it = downloadQ.get(host).iterator();
-            for (int i = 0; i < appSettingsService.getMaxThreads(); i++) {
-                DownloadJob downloadJob = it.hasNext() ? it.next() : null;
-                if (downloadJob != null) {
-                    downloadJobs.add(downloadJob);
-                }
-            }
-        }
-        return downloadJobs;
-    }
-
-    public void enqueue(Post post) throws InterruptedException {
-        for (Image image : post.getImages()) {
-            put(image);
-        }
-    }
-
-    public int size() {
-        return downloadQ.values().stream().mapToInt(BlockingDeque::size).sum();
-    }
-
-    public Iterable<? extends Map.Entry<Host, BlockingDeque<DownloadJob>>> entries() {
-        return downloadQ.entrySet();
-    }
-}

vripper-server/src/main/java/tn/mnlr/vripper/q/ExecutionService.java

@@ -1,281 +0,0 @@
-package tn.mnlr.vripper.q;
-
-import net.jodah.failsafe.Failsafe;
-import net.jodah.failsafe.FailsafeException;
-import net.jodah.failsafe.RetryPolicy;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import org.springframework.beans.factory.annotation.Autowired;
-import org.springframework.stereotype.Service;
-import tn.mnlr.vripper.VripperApplication;
-import tn.mnlr.vripper.entities.Image;
-import tn.mnlr.vripper.entities.Post;
-import tn.mnlr.vripper.host.Host;
-import tn.mnlr.vripper.services.AppSettingsService;
-import tn.mnlr.vripper.services.AppStateService;
-import tn.mnlr.vripper.services.ThumbnailGenerator;
-
-import javax.annotation.PostConstruct;
-import javax.annotation.PreDestroy;
-import java.time.Duration;
-import java.time.temporal.ChronoUnit;
-import java.util.*;
-import java.util.concurrent.*;
-import java.util.concurrent.atomic.AtomicInteger;
-import java.util.stream.Collectors;
-
-@Service
-public class ExecutionService {
-
-    private static final Logger logger = LoggerFactory.getLogger(ExecutionService.class);
-
-    private static final List<Post.Status> FINISHED = Arrays.asList(Post.Status.ERROR, Post.Status.COMPLETE, Post.Status.STOPPED);
-
-    @Autowired
-    private DownloadQ downloadQ;
-
-    @Autowired
-    private AppSettingsService settings;
-
-    @Autowired
-    private AppStateService appStateService;
-
-    @Autowired
-    private ThumbnailGenerator thumbnailGenerator;
-
-    @Autowired
-    private AppSettingsService appSettingsService;
-    private final ConcurrentHashMap<Host, AtomicInteger> threadCount = new ConcurrentHashMap<>();
-    private boolean notPauseQ = true;
-    private ExecutorService executor = Executors.newFixedThreadPool(20);
-
-    private Thread executionThread;
-
-    BlockingQueue<DownloadJob> queue = new LinkedBlockingQueue<>();
-
-    private RetryPolicy<Object> retryPolicy;
-
-    private List<DownloadJob> running = Collections.synchronizedList(new ArrayList<>());
-
-    private Map<String, Future<?>> futures = new ConcurrentHashMap<>();
-    private Thread pollThread;
-
-    @PostConstruct
-    private void init() {
-
-        retryPolicy = new RetryPolicy<>()
-                .handleIf(e -> !(e instanceof InterruptedException))
-                .withBackoff(1, 10, ChronoUnit.SECONDS)
-                .withMaxDuration(Duration.of(10, ChronoUnit.SECONDS))
-                .withMaxAttempts(3)
-                .abortOn(InterruptedException.class)
-                .onFailedAttempt(e -> logger.warn(String.format("#%d tries failed", e.getAttemptCount()), e.getLastFailure()));
-
-        executionThread = new Thread(this::start, "Executor thread");
-        pollThread = new Thread(this::poll, "Polling thread");
-        pollThread.start();
-        executionThread.start();
-    }
-
-    @PreDestroy
-    private void destroy() throws InterruptedException {
-        logger.info("Shutting down ExecutionService");
-        executionThread.interrupt();
-        executor.shutdown();
-        appStateService.getCurrentPosts().keySet().forEach(p -> {
-            logger.debug(String.format("Stopping download jobs for %s", p));
-            this.stopRunning(p);
-        });
-        executor.awaitTermination(10, TimeUnit.SECONDS);
-    }
-
-    public void stopRunning(String postId) {
-        List<DownloadJob> data = running
-                .stream()
-                .filter(e -> e.getImage().getPostId().equals(postId))
-                .peek(e -> e.getImage().setStatus(Image.Status.STOPPED))
-                .collect(Collectors.toList());
-        logger.debug(String.format("Interrupting %d jobs for post id %s", data.size(), postId));
-
-        data.forEach(e -> {
-            futures.get(e.getImage().getUrl()).cancel(true);
-            if (e.getImageFileData().getImageRequest() != null) {
-                e.getImageFileData().getImageRequest().abort();
-            }
-            e.getImage().cleanup();
-        });
-    }
-
-
-    public synchronized void stopAll() {
-        appStateService.getCurrentPosts().values().stream().map(Post::getPostId).forEach(this::stop);
-    }
-
-    public synchronized void restartAll() throws InterruptedException {
-        for (Post post : appStateService.getCurrentPosts().values()) {
-            String postId = post.getPostId();
-            restart(postId);
-        }
-    }
-
-    public synchronized void restart(String postId) throws InterruptedException {
-        if (appStateService.getRunningPosts().get(postId) != null && appStateService.getRunningPosts().get(postId).get() > 0) {
-            logger.warn(String.format("Cannot restart, jobs are currently running for post id %s", postId));
-            return;
-        }
-        List<Image> images = appStateService.getPost(postId)
-                .getImages()
-                .stream()
-                .filter(e -> !e.getStatus().equals(Image.Status.COMPLETE))
-                .collect(Collectors.toList());
-        if (images.isEmpty()) {
-            return;
-        }
-        appStateService.getPost(postId).setStatus(Post.Status.PENDING);
-        logger.debug(String.format("Restarting %d jobs for post id %s", images.size(), postId));
-        for (Image image : images) {
-            downloadQ.put(image);
-        }
-    }
-
-    private synchronized void removeScheduled(Image image) {
-        image.setStatus(Image.Status.STOPPED);
-        logger.debug(String.format("Removing scheduled job for %s", image.getUrl()));
-
-        boolean removed = false;
-        main:
-        for (Map.Entry<Host, BlockingDeque<DownloadJob>> entry : downloadQ.entries()) {
-            Iterator<DownloadJob> iterator = entry.getValue().iterator();
-            while (iterator.hasNext()) {
-                DownloadJob next = iterator.next();
-                if (next.getImage().getPostId().equals(image.getPostId())) {
-                    iterator.remove();
-                    appStateService.doneDownloadJob(image);
-                    logger.debug(String.format("Scheduled job for %s is removed", image.getUrl()));
-                    removed = true;
-                    break main;
-                }
-            }
-        }
-
-        if (!removed) {
-            logger.debug(String.format("Job for %s does not exist", image.getUrl()));
-        }
-
-        image.cleanup();
-    }
-
-    private synchronized void removeRunning(String postId) {
-        logger.debug(String.format("Interrupting running jobs for post id %s", postId));
-        stopRunning(postId);
-    }
-
-    public synchronized void stop(String postId) {
-        try {
-            if (FINISHED.contains(appStateService.getPost(postId).getStatus())) {
-                return;
-            }
-            notPauseQ = false;
-            appStateService.getPost(postId).setStatus(Post.Status.STOPPED);
-            List<Image> images = appStateService.getPost(postId)
-                    .getImages()
-                    .stream()
-                    .filter(e -> !e.getStatus().equals(Image.Status.COMPLETE))
-                    .collect(Collectors.toList());
-            if (images.isEmpty()) {
-                return;
-            }
-            logger.debug(String.format("Stopping %d jobs for post id %s", images.size(), postId));
-            images.forEach(this::removeScheduled);
-            removeRunning(postId);
-        } finally {
-            notPauseQ = true;
-        }
-    }
-
-    private boolean canRun(Host host) {
-        boolean canRun;
-        AtomicInteger count = threadCount.get(host);
-        if (count == null) {
-            threadCount.put(host, new AtomicInteger(0));
-        }
-        canRun = threadCount.get(host).get() < settings.getMaxThreads() && (settings.getMaxTotalThreads() == 0 || threadCount.values().stream().mapToInt(AtomicInteger::get).sum() < settings.getMaxTotalThreads());
-        if (canRun && notPauseQ) {
-            threadCount.get(host).incrementAndGet();
-            return true;
-        }
-        return false;
-    }
-
-    public void poll() {
-        while (!Thread.interrupted()) {
-            try {
-                List<DownloadJob> peek = downloadQ.peek();
-                for (DownloadJob downloadJob : peek) {
-                    if (canRun(downloadJob.getImage().getHost())) {
-                        queue.offer(downloadJob);
-                        downloadQ.remove(downloadJob);
-                    }
-                }
-                synchronized (threadCount) {
-                    threadCount.wait(2_000);
-                }
-            } catch (InterruptedException e) {
-                Thread.currentThread().interrupt();
-                break;
-            }
-        }
-    }
-
-    public void start() {
-        while (!Thread.interrupted()) {
-            try {
-                push(queue.take());
-            } catch (InterruptedException e) {
-                Thread.currentThread().interrupt();
-                break;
-            } catch (Exception e) {
-                logger.error("Execution Service failed", e);
-                break;
-            }
-        }
-    }
-
-    private void push(DownloadJob take) {
-        Runnable task = () -> {
-            running.add(take);
-
-            Failsafe.with(retryPolicy)
-                    .onFailure(e -> {
-                        if (e.getFailure() instanceof InterruptedException || (e.getFailure() instanceof FailsafeException && e.getFailure().getCause() instanceof InterruptedException)) {
-                            logger.debug("Job successfully interrupted");
-                            return;
-                        }
-                        logger.error(String.format("Failed to download %s after %d tries", take.getImage().getUrl(), e.getAttemptCount()), e.getFailure());
-                        take.getImage().setStatus(Image.Status.ERROR);
-                    })
-                    .onComplete(e -> {
-                        appStateService.doneDownloadJob(take.getImage());
-                        logger.debug(String.format("Finished downloading %s", take.getImage().getUrl()));
-                        if (appSettingsService.isViewPhotos()) {
-                            VripperApplication.commonExecutor.submit(
-                                    () -> thumbnailGenerator.getThumbnails()
-                                            .get(new ThumbnailGenerator.CacheKey(take.getImage().getPostId(), take.getImageFileData().getFileName())));
-                        }
-                        threadCount.get(take.getImage().getHost()).decrementAndGet();
-                        running.remove(take);
-                        futures.remove(take.getImage().getUrl());
-                        synchronized (threadCount) {
-                            threadCount.notify();
-                        }
-                    })
-                    .get(take::call);
-        };
-        logger.debug(String.format("Scheduling a job for %s", take.getImage().getUrl()));
-        futures.put(take.getImage().getUrl(), executor.submit(task));
-    }
-
-    public int runningCount() {
-        return running.size();
-    }
-}

vripper-server/src/main/java/tn/mnlr/vripper/q/ImageFileData.java

@@ -1,16 +0,0 @@
-package tn.mnlr.vripper.q;
-
-import lombok.Getter;
-import lombok.Setter;
-import org.apache.http.client.methods.HttpUriRequest;
-
-@Getter
-@Setter
-public class ImageFileData {
-
-    private String pageUrl;
-    private String imageName;
-    private String fileName;
-    private String imageUrl;
-    private HttpUriRequest imageRequest;
-}

vripper-server/src/main/java/tn/mnlr/vripper/services/AppSettingsService.java

@@ -1,226 +0,0 @@
-package tn.mnlr.vripper.services;
-
-import com.fasterxml.jackson.annotation.JsonProperty;
-import lombok.Getter;
-import lombok.NoArgsConstructor;
-import lombok.Setter;
-import org.apache.commons.codec.digest.DigestUtils;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import org.springframework.beans.factory.annotation.Value;
-import org.springframework.stereotype.Service;
-import tn.mnlr.vripper.exception.ValidationException;
-
-import javax.annotation.PostConstruct;
-import javax.annotation.PreDestroy;
-import java.nio.file.Files;
-import java.nio.file.InvalidPathException;
-import java.nio.file.Path;
-import java.nio.file.Paths;
-import java.util.prefs.BackingStoreException;
-import java.util.prefs.Preferences;
-
-@Service
-@Getter
-@Setter
-public class AppSettingsService {
-
-    private final String MAX_TOTAL_THREADS = "MAX_TOTAL_THREADS";
-
-    @Value("${base.dir}")
-    private String defaultDownloadPath;
-
-    private static final Logger logger = LoggerFactory.getLogger(AppSettingsService.class);
-
-    private Preferences prefs = Preferences.userNodeForPackage(AppSettingsService.class);
-
-    private final String DOWNLOAD_PATH = "DOWNLOAD_PATH";
-    private final String MAX_THREADS = "MAX_THREADS";
-    private int maxTotalThreads;
-    private final String AUTO_START = "AUTO_START";
-    private final String V_LOGIN = "VLOGIN";
-    private final String V_USERNAME = "VUSERNAME";
-    private final String V_PASSWORD = "VPASSWORD";
-    private final String V_THANKS = "VTHANKS";
-    private final String DESKTOP_CLIPBOARD = "DESKTOP_CLIPBOARD";
-    private final String FORCE_ORDER = "FORCE_ORDER";
-    private final String SUBFOLDER = "SUBFOLDER";
-    private final String THREADSUBFOLDER = "THREADSUBFOLDER";
-    private final String CLEAR = "CLEAR";
-    private final String DARK_THEME = "DARK_THEME";
-    private final String VIEW_PHOTOS = "VIEW_PHOTOS";
-    private final String NOTIFICATION = "NOTIFICATION";
-
-    @PostConstruct
-    private void init() {
-        restore();
-    }
-
-    private String downloadPath;
-    private int maxThreads;
-    private boolean autoStart;
-    private boolean vLogin;
-    private String vUsername;
-    private String vPassword;
-    private boolean vThanks;
-    private boolean desktopClipboard;
-    private boolean subLocation;
-    private boolean threadSubLocation;
-    private boolean forceOrder;
-    private boolean clearCompleted;
-    private boolean darkTheme;
-    private boolean viewPhotos;
-    private boolean notificationEnabled;
-
-    public void setVPassword(String vPassword) {
-        if(vPassword.isEmpty()) {
-            this.vPassword = "";
-        } else {
-            this.vPassword = DigestUtils.md5Hex(vPassword);
-        }
-    }
-
-    public void restore() {
-
-        downloadPath = prefs.get(DOWNLOAD_PATH, defaultDownloadPath);
-        maxThreads = prefs.getInt(MAX_THREADS, 4);
-        maxTotalThreads = prefs.getInt(MAX_TOTAL_THREADS, 0);
-        autoStart = prefs.getBoolean(AUTO_START, true);
-        vLogin = prefs.getBoolean(V_LOGIN, false);
-        vUsername = prefs.get(V_USERNAME, "");
-        vPassword = prefs.get(V_PASSWORD, "");
-        vThanks = prefs.getBoolean(V_THANKS, false);
-        desktopClipboard = prefs.getBoolean(DESKTOP_CLIPBOARD, false);
-        forceOrder = prefs.getBoolean(FORCE_ORDER, false);
-        subLocation = prefs.getBoolean(SUBFOLDER, false);
-        threadSubLocation = prefs.getBoolean(THREADSUBFOLDER, false);
-        clearCompleted = prefs.getBoolean(CLEAR, false);
-        darkTheme = prefs.getBoolean(DARK_THEME, false);
-        viewPhotos = prefs.getBoolean(VIEW_PHOTOS, false);
-        notificationEnabled = prefs.getBoolean(NOTIFICATION, false);
-    }
-
-    @PreDestroy
-    public void save() {
-
-        prefs.put(DOWNLOAD_PATH, downloadPath);
-        prefs.putInt(MAX_THREADS, maxThreads);
-        prefs.putInt(MAX_TOTAL_THREADS, maxTotalThreads);
-        prefs.putBoolean(AUTO_START, autoStart);
-        prefs.putBoolean(V_LOGIN, vLogin);
-        prefs.put(V_USERNAME, vUsername);
-        prefs.put(V_PASSWORD, vPassword);
-        prefs.putBoolean(V_THANKS, vThanks);
-        prefs.putBoolean(DESKTOP_CLIPBOARD, desktopClipboard);
-        prefs.putBoolean(FORCE_ORDER, forceOrder);
-        prefs.putBoolean(SUBFOLDER, subLocation);
-        prefs.putBoolean(THREADSUBFOLDER, threadSubLocation);
-        prefs.putBoolean(CLEAR, clearCompleted);
-        prefs.putBoolean(DARK_THEME, darkTheme);
-        prefs.putBoolean(VIEW_PHOTOS, viewPhotos);
-        prefs.putBoolean(NOTIFICATION, notificationEnabled);
-
-        try {
-            prefs.sync();
-        } catch (BackingStoreException e) {
-            logger.error("Failed to store user settings", e);
-        }
-    }
-
-    public void check(Settings settings) throws ValidationException {
-
-        Path path;
-        try {
-            path = Paths.get(settings.getDownloadPath());
-        } catch (InvalidPathException e) {
-            throw new ValidationException(String.format("%s is invalid", settings.getDownloadPath()));
-        }
-        if (!Files.exists(path)) {
-            throw new ValidationException(String.format("%s does not exist", settings.getDownloadPath()));
-        } else if (!Files.isDirectory(path)) {
-            throw new ValidationException(String.format("%s is not a directory", settings.getDownloadPath()));
-        }
-
-        if (settings.getMaxTotalThreads() < 0) {
-            throw new ValidationException(String.format("Invalid max global concurrent download settings, values must be in greater than %d", 1));
-        }
-
-        if (settings.getMaxThreads() < 1 || settings.getMaxThreads() > 4) {
-            throw new ValidationException(String.format("Invalid max concurrent download settings, values must be in [%d,%d]", 1, 4));
-        }
-    }
-
-    public Theme getTheme() {
-        return new Theme(this.darkTheme);
-    }
-
-    public void setTheme(Theme theme) {
-        this.darkTheme = theme.darkTheme;
-        save();
-    }
-
-    @Getter
-    @NoArgsConstructor
-    public static class Theme {
-
-        @JsonProperty("darkTheme")
-        private boolean darkTheme;
-
-        Theme(boolean darkTheme) {
-            this.darkTheme = darkTheme;
-        }
-    }
-
-    @Getter
-    public static class Settings {
-
-        @JsonProperty("downloadPath")
-        private String downloadPath;
-        @JsonProperty("maxThreads")
-        private int maxThreads;
-        @JsonProperty("maxTotalThreads")
-        private int maxTotalThreads;
-        @JsonProperty("autoStart")
-        private boolean autoStart;
-        @JsonProperty("vLogin")
-        private boolean vLogin;
-        @JsonProperty("vUsername")
-        private String vUsername;
-        @JsonProperty("vPassword")
-        private String vPassword;
-        @JsonProperty("vThanks")
-        private boolean vThanks;
-        @JsonProperty("desktopClipboard")
-        private boolean desktopClipboard;
-        @JsonProperty("forceOrder")
-        private boolean forceOrder;
-        @JsonProperty("subLocation")
-        private boolean subLocation;
-        @JsonProperty("threadSubLocation")
-        private boolean threadSubLocation;
-        @JsonProperty("clearCompleted")
-        private boolean clearCompleted;
-        @JsonProperty("viewPhotos")
-        private boolean viewPhotos;
-        @JsonProperty("notification")
-        private boolean notification;
-
-        public Settings(String downloadPath, int maxThreads, int maxTotalThreads, boolean autoStart, boolean vLogin, String vUsername, String vPassword, boolean vThanks, boolean desktopClipboard, boolean forceOrder, boolean subLocation, boolean threadSubLocation, boolean clearCompleted, boolean viewPhotos, boolean notification) {
-            this.downloadPath = downloadPath;
-            this.maxThreads = maxThreads;
-            this.maxTotalThreads = maxTotalThreads;
-            this.autoStart = autoStart;
-            this.vLogin = vLogin;
-            this.vUsername = vUsername;
-            this.vPassword = vPassword;
-            this.vThanks = vThanks;
-            this.desktopClipboard = desktopClipboard;
-            this.forceOrder = forceOrder;
-            this.subLocation = subLocation;
-            this.threadSubLocation = threadSubLocation;
-            this.clearCompleted = clearCompleted;
-            this.viewPhotos = viewPhotos;
-            this.notification = notification;
-        }
-    }
-}

vripper-server/src/main/java/tn/mnlr/vripper/services/AppStateService.java

@@ -1,142 +0,0 @@
-package tn.mnlr.vripper.services;
-
-import io.reactivex.processors.PublishProcessor;
-import lombok.Getter;
-import lombok.Setter;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import org.springframework.beans.factory.annotation.Autowired;
-import org.springframework.stereotype.Service;
-import tn.mnlr.vripper.entities.Image;
-import tn.mnlr.vripper.entities.Post;
-import tn.mnlr.vripper.q.DownloadJob;
-
-import java.util.List;
-import java.util.Map;
-import java.util.concurrent.ConcurrentHashMap;
-import java.util.concurrent.atomic.AtomicInteger;
-import java.util.stream.Collectors;
-
-@Service
-@Getter
-public class AppStateService {
-
-    private static final Logger logger = LoggerFactory.getLogger(AppStateService.class);
-
-    private Map<String, Image> currentImages = new ConcurrentHashMap<>();
-
-    private Map<String, Post> currentPosts = new ConcurrentHashMap<>();
-
-    private Map<String, AtomicInteger> runningPosts = new ConcurrentHashMap<>();
-
-    private Map<String, QueuedVGLink> grabQueue = new ConcurrentHashMap<>();
-
-    private PublishProcessor<QueuedVGLink> liveGrabQueue = PublishProcessor.create();
-
-    private PublishProcessor<Image> liveImageUpdates = PublishProcessor.create();
-
-    private PublishProcessor<Post> livePostsState = PublishProcessor.create();
-
-    @Autowired
-    private PersistenceService persistenceService;
-
-    @Autowired
-    private AppSettingsService appSettingsService;
-
-    public void onImageUpdate(Image imageState) {
-
-        persistenceService.getProcessor().onNext(currentPosts);
-
-        liveImageUpdates.onNext(imageState);
-        if (imageState.isCompleted()) {
-
-            Post postState = currentPosts.get(imageState.getPostId());
-            postState.increase();
-        }
-    }
-
-    public Post getPost(String postId) {
-        return currentPosts.get(postId);
-    }
-
-    public void newDownloadJob(DownloadJob downloadJob) {
-        String postId = downloadJob.getImage().getPostId();
-        checkKeyRunningPosts(postId);
-        runningPosts.get(postId).incrementAndGet();
-    }
-
-    public synchronized void postDownloadingUpdate(String postId) {
-        Post post = currentPosts.get(postId);
-        if (!post.getStatus().equals(Post.Status.DOWNLOADING) && !post.getStatus().equals(Post.Status.PARTIAL)) {
-            post.setStatus(Post.Status.DOWNLOADING);
-            livePostsState.onNext(post);
-        }
-    }
-
-    public void doneDownloadJob(Image image) {
-        String postId = image.getPostId();
-        int i = runningPosts.get(postId).decrementAndGet();
-        Post post = currentPosts.get(postId);
-        if(image.getStatus().equals(Image.Status.ERROR)) {
-            post.setStatus(Post.Status.PARTIAL);
-        }
-        if (i == 0) {
-            if (post.getImages().stream().map(Image::getStatus).anyMatch(e -> e.equals(Image.Status.ERROR))) {
-                post.setStatus(Post.Status.ERROR);
-            } else {
-                if (!Post.Status.STOPPED.equals(post.getStatus())) {
-                    post.setStatus(Post.Status.COMPLETE);
-                    if (appSettingsService.isClearCompleted()) {
-                        remove(image.getPostId());
-                    }
-                }
-            }
-        }
-    }
-
-    private synchronized void checkKeyRunningPosts(String key) {
-        if (!runningPosts.containsKey(key)) {
-            runningPosts.put(key, new AtomicInteger(0));
-        }
-    }
-
-    public synchronized void remove(String postId) {
-        currentPosts.get(postId).setRemoved(true);
-        runningPosts.remove(postId);
-        currentPosts.remove(postId);
-        currentImages.entrySet().removeIf(entry -> entry.getValue().getPostId().equals(postId));
-        persistenceService.getProcessor().onNext(currentPosts);
-    }
-
-    public synchronized List<String> clearAll() {
-        List<String> toRemove = this.currentPosts
-                .values()
-                .stream()
-                .filter(e -> e.getStatus().equals(Post.Status.COMPLETE) && e.getDone().get() >= e.getTotal())
-                .map(Post::getPostId)
-                .collect(Collectors.toList());
-        toRemove.forEach(this::remove);
-        return toRemove;
-    }
-
-    public synchronized List<String> removeAll() {
-        List<String> toRemove = this.currentPosts
-                .values()
-                .stream()
-                .map(Post::getPostId)
-                .collect(Collectors.toList());
-        toRemove.forEach(this::remove);
-        return toRemove;
-    }
-
-
-    @Getter
-    public static class CachedThread {
-
-        private Map<String, Post> posts = new ConcurrentHashMap<>();
-        private AtomicInteger parsed = new AtomicInteger(0);
-
-        @Setter
-        private int total = 0;
-    }
-}

vripper-server/src/main/java/tn/mnlr/vripper/services/ConnectionManager.java

@@ -1,73 +0,0 @@
-package tn.mnlr.vripper.services;
-
-import org.apache.http.client.config.CookieSpecs;
-import org.apache.http.client.config.RequestConfig;
-import org.apache.http.client.methods.HttpGet;
-import org.apache.http.client.methods.HttpPost;
-import org.apache.http.impl.client.HttpClientBuilder;
-import org.apache.http.impl.client.HttpClients;
-import org.apache.http.impl.client.LaxRedirectStrategy;
-import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;
-import org.springframework.scheduling.annotation.EnableScheduling;
-import org.springframework.scheduling.annotation.Scheduled;
-import org.springframework.stereotype.Service;
-
-import java.net.URI;
-import java.util.concurrent.TimeUnit;
-
-@Service
-@EnableScheduling
-public class ConnectionManager {
-
-    private ConnectionManager() {
-        buildConnectionPool();
-    }
-
-    private PoolingHttpClientConnectionManager pcm;
-
-    private RequestConfig rc = RequestConfig.custom()
-            .setConnectionRequestTimeout(5_000)
-            .setConnectTimeout(5_000)
-            .setSocketTimeout(5_000)
-            .setCookieSpec(CookieSpecs.STANDARD)
-            .build();
-
-    private void buildConnectionPool() {
-
-        pcm = new PoolingHttpClientConnectionManager();
-        pcm.setMaxTotal(50);
-        pcm.setDefaultMaxPerRoute(4);
-    }
-
-    @Scheduled(fixedDelay = 1000)
-    private void idleConnectionMonitoring() {
-        pcm.closeExpiredConnections();
-        pcm.closeIdleConnections(30, TimeUnit.SECONDS);
-    }
-
-    public HttpClientBuilder getClient() {
-        return HttpClients.custom()
-                .setConnectionManager(pcm)
-                .setRedirectStrategy(new LaxRedirectStrategy())
-                .disableAutomaticRetries()
-                .setDefaultRequestConfig(rc);
-    }
-
-    public HttpGet buildHttpGet(String url) {
-        HttpGet httpGet = new HttpGet(url.replace(" ", "+"));
-        httpGet.addHeader("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36");
-        return httpGet;
-    }
-
-    public HttpPost buildHttpPost(String url) {
-        HttpPost httpPost = new HttpPost(url.replace(" ", "+"));
-        httpPost.addHeader("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36");
-        return httpPost;
-    }
-
-    public HttpGet buildHttpGet(URI uri) {
-        HttpGet httpGet = new HttpGet(uri);
-        httpGet.addHeader("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36");
-        return httpGet;
-    }
-}

vripper-server/src/main/java/tn/mnlr/vripper/services/ConnectionService.java

@@ -0,0 +1,134 @@
+package tn.mnlr.vripper.services;
+
+import lombok.Getter;
+import lombok.extern.slf4j.Slf4j;
+import net.jodah.failsafe.RetryPolicy;
+import org.apache.http.client.config.CookieSpecs;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.AbstractExecutionAwareRequest;
+import org.apache.http.client.methods.HttpGet;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.protocol.HttpClientContext;
+import org.apache.http.impl.client.HttpClientBuilder;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.impl.client.LaxRedirectStrategy;
+import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;
+import org.springframework.scheduling.annotation.EnableScheduling;
+import org.springframework.scheduling.annotation.Scheduled;
+import org.springframework.stereotype.Service;
+import reactor.core.Disposable;
+import reactor.core.publisher.Flux;
+import tn.mnlr.vripper.download.DownloadJob;
+
+import javax.annotation.PreDestroy;
+import java.net.URI;
+import java.time.temporal.ChronoUnit;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+
+@Service
+@EnableScheduling
+@Slf4j
+public class ConnectionService {
+
+    private final Disposable disposable;
+    private PoolingHttpClientConnectionManager pcm;
+    private RequestConfig rc;
+
+    @Getter
+    private RetryPolicy<Object> retryPolicy;
+
+    private int connectionTimeout;
+    private int maxAttempts;
+
+    public ConnectionService(SettingsService settingsService) {
+        connectionTimeout = settingsService.getSettings().getConnectionTimeout();
+        maxAttempts = settingsService.getSettings().getMaxAttempts();
+        Flux<SettingsService.Settings> settingsFlux = settingsService.getSettingsFlux();
+        disposable = settingsFlux.subscribe(settings -> {
+            if (connectionTimeout != settings.getConnectionTimeout()) {
+                connectionTimeout = settings.getConnectionTimeout();
+                buildRequestConfig();
+            }
+            if (maxAttempts != settings.getMaxAttempts()) {
+                maxAttempts = settings.getMaxAttempts();
+                buildRetryPolicy();
+            }
+        });
+
+        buildRequestConfig();
+        buildRetryPolicy();
+        buildConnectionPool();
+    }
+
+    @PreDestroy
+    private void destroy() {
+        disposable.dispose();
+    }
+
+    private void buildRetryPolicy() {
+        retryPolicy = new RetryPolicy<>()
+                .withDelay(2, 5, ChronoUnit.SECONDS)
+                .withMaxAttempts(maxAttempts)
+                .onFailedAttempt(e -> log.warn(String.format("#%d tries failed", e.getAttemptCount()), e.getLastFailure()));
+    }
+
+    private void buildConnectionPool() {
+        pcm = new PoolingHttpClientConnectionManager();
+        pcm.setMaxTotal(Integer.MAX_VALUE);
+        pcm.setDefaultMaxPerRoute(Integer.MAX_VALUE);
+    }
+
+    private void buildRequestConfig() {
+        rc = RequestConfig.custom()
+                .setConnectionRequestTimeout(connectionTimeout * 1000)
+                .setConnectTimeout(connectionTimeout * 1000)
+                .setSocketTimeout(connectionTimeout * 1000)
+                .setCookieSpec(CookieSpecs.STANDARD)
+                .build();
+    }
+
+    @Scheduled(fixedDelay = 5_000)
+    private void idleConnectionMonitoring() {
+        pcm.closeExpiredConnections();
+        pcm.closeIdleConnections(30, TimeUnit.SECONDS);
+    }
+
+    public HttpClientBuilder getClient() {
+        return HttpClients.custom()
+                .setConnectionManager(pcm)
+                .setRedirectStrategy(new LaxRedirectStrategy())
+                .disableAutomaticRetries()
+                .setDefaultRequestConfig(rc);
+    }
+
+    public HttpGet buildHttpGet(String url, final HttpClientContext context) {
+        HttpGet httpGet = new HttpGet(url.replace(" ", "+"));
+        httpGet.addHeader("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36");
+        addToContext(context, httpGet);
+        return httpGet;
+    }
+
+    public HttpPost buildHttpPost(String url, final HttpClientContext context) {
+        HttpPost httpPost = new HttpPost(url.replace(" ", "+"));
+        httpPost.addHeader("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36");
+        addToContext(context, httpPost);
+        return httpPost;
+    }
+
+    public HttpGet buildHttpGet(URI uri, final HttpClientContext context) {
+        HttpGet httpGet = new HttpGet(uri);
+        httpGet.addHeader("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36");
+        addToContext(context, httpGet);
+        return httpGet;
+    }
+
+    public void addToContext(HttpClientContext context, AbstractExecutionAwareRequest request) {
+        if (context != null) {
+            List<AbstractExecutionAwareRequest> requests = (List<AbstractExecutionAwareRequest>) context.getAttribute(DownloadJob.ContextAttributes.OPEN_CONNECTION.toString());
+            if (requests != null) {
+                requests.add(request);
+            }
+        }
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/services/DataService.java

@@ -0,0 +1,213 @@
+package tn.mnlr.vripper.services;
+
+import lombok.NonNull;
+import lombok.extern.slf4j.Slf4j;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Service;
+import org.springframework.transaction.annotation.Transactional;
+import tn.mnlr.vripper.jpa.domain.Image;
+import tn.mnlr.vripper.jpa.domain.Metadata;
+import tn.mnlr.vripper.jpa.domain.Post;
+import tn.mnlr.vripper.jpa.domain.Queued;
+import tn.mnlr.vripper.jpa.domain.enums.Status;
+import tn.mnlr.vripper.jpa.repositories.IImageRepository;
+import tn.mnlr.vripper.jpa.repositories.IMetadataRepository;
+import tn.mnlr.vripper.jpa.repositories.IPostRepository;
+import tn.mnlr.vripper.jpa.repositories.IQueuedRepository;
+
+import java.util.Collection;
+import java.util.List;
+import java.util.Optional;
+
+@Service
+@Transactional
+@Slf4j
+public class DataService {
+
+    private final IPostRepository postRepository;
+    private final IImageRepository imageRepository;
+    private final IQueuedRepository queuedRepository;
+    private final IMetadataRepository metadataRepository;
+    private final SettingsService settingsService;
+
+    @Autowired
+    public DataService(IPostRepository postRepository, IImageRepository imageRepository, IQueuedRepository queuedRepository, IMetadataRepository metadataRepository, SettingsService settingsService) {
+        this.postRepository = postRepository;
+        this.imageRepository = imageRepository;
+        this.queuedRepository = queuedRepository;
+        this.metadataRepository = metadataRepository;
+        this.settingsService = settingsService;
+    }
+
+    private void save(Post post) {
+        postRepository.save(post);
+    }
+
+    private void save(Queued queued) {
+        queuedRepository.save(queued);
+    }
+
+    private void save(Image image) {
+        imageRepository.save(image);
+    }
+
+    public boolean exists(String postId) {
+        return postRepository.existByPostId(postId);
+    }
+
+    public void newPost(Post post, Collection<Image> images) {
+        save(post);
+        images.forEach(image -> {
+            image.setPostIdRef(post.getId());
+            save(image);
+        });
+    }
+
+    public void setDownloadingToStopped() {
+        postRepository.setDownloadingToStopped();
+    }
+
+    public synchronized void afterJobFinish(Image image, Post post) {
+        if (image.getStatus().equals(Status.COMPLETE)) {
+            post.setDone(post.getDone() + 1);
+            updatePostDone(post.getDone(), post.getId());
+        } else if (image.getStatus().equals(Status.ERROR)) {
+            post.setStatus(Status.PARTIAL);
+            updatePostStatus(post.getStatus(), post.getId());
+        }
+    }
+
+    private void updatePostDone(int done, Long id) {
+        postRepository.updateDone(done, id);
+    }
+
+    public void finishPost(@NonNull Post post) {
+        if (!imageRepository.findByPostIdAndIsError(post.getPostId()).isEmpty()) {
+            post.setStatus(Status.ERROR);
+            updatePostStatus(post.getStatus(), post.getId());
+        } else {
+            if (post.getDone() < post.getTotal()) {
+                post.setStatus(Status.STOPPED);
+                updatePostStatus(post.getStatus(), post.getId());
+            } else {
+                post.setStatus(Status.COMPLETE);
+                updatePostStatus(post.getStatus(), post.getId());
+                if (settingsService.getSettings().getClearCompleted()) {
+                    remove(post.getPostId());
+                }
+            }
+        }
+    }
+
+    private void remove(@NonNull final String postId) {
+        imageRepository.deleteAllByPostId(postId);
+        metadataRepository.deleteByPostId(postId);
+        postRepository.deleteByPostId(postId);
+    }
+
+    public void newQueueLink(@NonNull final Queued queued) {
+        save(queued);
+    }
+
+    public void removeQueueLink(@NonNull final String threadId) {
+        queuedRepository.deleteByThreadId(threadId);
+    }
+
+    public List<String> clearCompleted() {
+        List<String> completed = postRepository.findCompleted();
+        completed.forEach(this::remove);
+        return completed;
+    }
+
+    public void removeAll(final List<String> postIds) {
+        if (postIds != null && !postIds.isEmpty()) {
+            for (String postId : postIds) {
+                remove(postId);
+            }
+        } else {
+            postRepository.findAll().forEach(p -> remove(p.getPostId()));
+        }
+    }
+
+    public List<Image> findByPostIdAndIsNotCompleted(@NonNull String postId) {
+        return imageRepository.findByPostIdAndIsNotCompleted(postId);
+
+    }
+
+    public long countErrorImages() {
+        return imageRepository.countError();
+    }
+
+    public List<Image> findImagesByPostId(String postId) {
+        return imageRepository.findByPostId(postId);
+    }
+
+    public Iterable<Post> findAllPosts() {
+        return postRepository.findAll();
+    }
+
+    public Optional<Post> findPostByPostId(String postId) {
+        return postRepository.findByPostId(postId);
+    }
+
+    public void stopImagesByPostIdAndIsNotCompleted(String postId) {
+        imageRepository.stopByPostIdAndIsNotCompleted(postId);
+    }
+
+    public Optional<Queued> findQueuedByThreadId(String threadId) {
+        return queuedRepository.findByThreadId(threadId);
+    }
+
+    public Iterable<Queued> findAllQueued() {
+        return queuedRepository.findAll();
+    }
+
+    public Optional<Post> findById(Long aLong) {
+        return postRepository.findById(aLong);
+    }
+
+    public Optional<Image> findImageById(Long aLong) {
+        return imageRepository.findById(aLong);
+    }
+
+    public Optional<Queued> findQueuedById(Long aLong) {
+        return queuedRepository.findById(aLong);
+    }
+
+    public void setMetadata(Post post, Metadata metadata) {
+        metadata.setPostIdRef(post.getId());
+        metadataRepository.save(metadata);
+    }
+
+    public Optional<Metadata> findMetadataByPostId(String postId) {
+        return metadataRepository.findByPostId(postId);
+    }
+
+    public void updateImageStatus(Status status, Long id) {
+        imageRepository.updateStatus(status, id);
+    }
+
+    public void updateImageCurrent(long current, Long id) {
+        imageRepository.updateCurrent(current, id);
+    }
+
+    public void updateImageTotal(long total, Long id) {
+        imageRepository.updateTotal(total, id);
+    }
+
+    public void updatePostStatus(Status status, Long id) {
+        postRepository.updateStatus(status, id);
+    }
+
+    public void updateDownloadDirectory(String postFolderName, Long id) {
+        postRepository.updateFolderName(postFolderName, id);
+    }
+
+    public void updatePostTitle(String title, Long id) {
+        postRepository.updateTitle(title, id);
+    }
+
+    public void updatePostThanked(boolean thanked, Long id) {
+        postRepository.updateThanked(thanked, id);
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/services/DownloadSpeedService.java

@@ -1,26 +1,30 @@
 package tn.mnlr.vripper.services;
 
-import io.reactivex.processors.PublishProcessor;
 import lombok.Getter;
 import org.springframework.scheduling.annotation.EnableScheduling;
 import org.springframework.scheduling.annotation.Scheduled;
 import org.springframework.stereotype.Service;
+import reactor.core.publisher.Flux;
+import reactor.core.publisher.Sinks;
+import tn.mnlr.vripper.listener.EmitHandler;
 
+import javax.annotation.PreDestroy;
 import java.util.concurrent.atomic.AtomicLong;
 
 @Service
 @EnableScheduling
 public class DownloadSpeedService {
 
-    private AtomicLong read = new AtomicLong(0);
-
-    private long currentValue;
-
+    private final AtomicLong read = new AtomicLong(0);
+    private final Sinks.Many<Long> sink = Sinks.many().multicast().onBackpressureBuffer();
     @Getter
-    private PublishProcessor<Long> readBytesPerSecond = PublishProcessor.create();
-
+    private long currentValue;
     private boolean allowWrite = false;
 
+    public Flux<Long> getReadBytesPerSecond() {
+        return sink.asFlux();
+    }
+
     public void increase(long read) {
         if (allowWrite) {
             this.read.addAndGet(read);
@@ -33,8 +37,13 @@ public class DownloadSpeedService {
         long newValue = read.getAndSet(0);
         if (newValue != currentValue) {
             currentValue = newValue;
-            readBytesPerSecond.onNext(currentValue);
+            sink.emitNext(currentValue, EmitHandler.RETRY);
         }
         allowWrite = true;
     }
+
+    @PreDestroy
+    private void destroy() {
+        sink.emitComplete(EmitHandler.RETRY);
+    }
 }

vripper-server/src/main/java/tn/mnlr/vripper/services/GlobalState.java

@@ -1,35 +0,0 @@
-package tn.mnlr.vripper.services;
-
-import lombok.Getter;
-
-import java.util.Objects;
-
-@Getter
-public class GlobalState {
-
-    private final String type = "globalState";
-    private long running;
-    private long queued;
-    private long remaining;
-    private long error;
-
-    GlobalState(long running, long queued, long remaining, long error) {
-        this.running = running;
-        this.queued = queued;
-        this.remaining = remaining;
-        this.error = error;
-    }
-
-    @Override
-    public boolean equals(Object o) {
-        if (this == o) return true;
-        if (o == null || getClass() != o.getClass()) return false;
-        GlobalState that = (GlobalState) o;
-        return running == that.running && queued == that.queued && remaining == that.remaining && error == that.error;
-    }
-
-    @Override
-    public int hashCode() {
-        return Objects.hash(type, running, queued, remaining, error);
-    }
-}

vripper-server/src/main/java/tn/mnlr/vripper/services/GlobalStateService.java

@@ -1,55 +1,55 @@
 package tn.mnlr.vripper.services;
 
-import io.reactivex.processors.PublishProcessor;
 import lombok.Getter;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.scheduling.annotation.EnableScheduling;
 import org.springframework.scheduling.annotation.Scheduled;
 import org.springframework.stereotype.Service;
-import tn.mnlr.vripper.entities.Image;
-import tn.mnlr.vripper.q.DownloadQ;
-import tn.mnlr.vripper.q.ExecutionService;
+import reactor.core.publisher.Flux;
+import reactor.core.publisher.Sinks;
+import tn.mnlr.vripper.download.DownloadService;
+import tn.mnlr.vripper.download.PendingQueue;
+import tn.mnlr.vripper.listener.EmitHandler;
+import tn.mnlr.vripper.services.domain.GlobalState;
+
+import javax.annotation.PreDestroy;
 
 @Service
 @EnableScheduling
 public class GlobalStateService {
 
-    @Autowired
-    private DownloadQ downloadQ;
-
-    @Autowired
-    private ExecutionService executionService;
-
-    @Autowired
-    private AppStateService appStateService;
-
+    private final PendingQueue pendingQueue;
+    private final DownloadService downloadService;
+    private final DataService dataService;
+    private final Sinks.Many<GlobalState> sink = Sinks.many().multicast().onBackpressureBuffer();
     @Getter
     private GlobalState currentState;
 
-    @Getter
-    private PublishProcessor<GlobalState> liveGlobalState = PublishProcessor.create();
+    @Autowired
+    public GlobalStateService(PendingQueue pendingQueue, DownloadService downloadService, DataService dataService) {
+        this.pendingQueue = pendingQueue;
+        this.downloadService = downloadService;
+        this.dataService = dataService;
+    }
 
-    public GlobalStateService() {
+    public Flux<GlobalState> getGlobalState() {
+        return sink.asFlux();
     }
 
     @Scheduled(fixedDelay = 3000)
     private void interval() {
         GlobalState newGlobalState = new GlobalState(
-                executionService.runningCount(),
-                downloadQ.size(),
-                appStateService.getCurrentImages()
-                        .values()
-                        .stream()
-                        .filter(e -> e.getTotal() == 0 || e.getTotal() != e.getCurrent().get())
-                        .count(),
-                appStateService.getCurrentImages()
-                        .values()
-                        .stream()
-                        .filter(e -> e.getStatus().equals(Image.Status.ERROR))
-                        .count());
+                downloadService.runningCount(),
+                pendingQueue.size(),
+                dataService.countErrorImages());
         if (!newGlobalState.equals(currentState)) {
             currentState = newGlobalState;
-            liveGlobalState.onNext(currentState);
+            sink.emitNext(currentState, EmitHandler.RETRY);
         }
     }
+
+    @PreDestroy
+    private void destroy() {
+        sink.emitComplete(EmitHandler.RETRY);
+    }
 }

vripper-server/src/main/java/tn/mnlr/vripper/services/HostService.java

@@ -0,0 +1,104 @@
+package tn.mnlr.vripper.services;
+
+import lombok.Getter;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.http.Header;
+import org.apache.http.client.HttpClient;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpGet;
+import org.apache.http.client.protocol.HttpClientContext;
+import org.apache.http.util.EntityUtils;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Service;
+import org.w3c.dom.Document;
+import tn.mnlr.vripper.exception.HostException;
+import tn.mnlr.vripper.exception.HtmlProcessorException;
+
+import java.io.IOException;
+import java.net.URI;
+import java.net.URISyntaxException;
+
+@Service
+@Slf4j
+public class HostService {
+
+    private final ConnectionService cm;
+    private final HtmlProcessorService htmlProcessorService;
+
+    @Autowired
+    public HostService(ConnectionService cm, HtmlProcessorService htmlProcessorService) {
+        this.cm = cm;
+        this.htmlProcessorService = htmlProcessorService;
+    }
+
+    public Response getResponse(final String url, final HttpClientContext context) throws HostException {
+        String basePage;
+
+        HttpClient client = cm.getClient().build();
+        HttpGet httpGet = cm.buildHttpGet(url, context);
+        Header[] headers;
+        log.debug(String.format("Requesting %s", url));
+        try (CloseableHttpResponse response = (CloseableHttpResponse) client.execute(httpGet, context)) {
+            if (response.getStatusLine().getStatusCode() / 100 != 2) {
+                throw new HostException(String.format("Unexpected response code: %d", response.getStatusLine().getStatusCode()));
+            }
+            headers = response.getAllHeaders();
+            basePage = EntityUtils.toString(response.getEntity());
+            log.debug(String.format("%s response: %n%s", url, basePage));
+            EntityUtils.consumeQuietly(response.getEntity());
+        } catch (IOException e) {
+            throw new HostException(e);
+        }
+
+        try {
+            log.debug(String.format("Cleaning %s response", url));
+            return new Response(htmlProcessorService.clean(basePage), headers);
+        } catch (HtmlProcessorException e) {
+            throw new HostException(e);
+        }
+    }
+
+    public String appendUri(String uri, String appendQuery) throws URISyntaxException {
+        URI oldUri = new URI(uri);
+
+        String newQuery = oldUri.getQuery();
+        if (newQuery == null) {
+            newQuery = appendQuery;
+        } else {
+            newQuery += "&" + appendQuery;
+        }
+
+        return new URI(oldUri.getScheme(), oldUri.getAuthority(),
+                oldUri.getPath(), newQuery, oldUri.getFragment()).toString();
+    }
+
+    public String getDefaultImageName(final String imgUrl) {
+        String imageTitle = imgUrl.substring(imgUrl.lastIndexOf('/') + 1);
+        log.debug(String.format("Extracting name from url %s: %s", imgUrl, imageTitle));
+        return imgUrl;
+    }
+
+    @Getter
+    public static class Response {
+
+        private final Document document;
+        private final Header[] headers;
+
+        protected Response(Document document, Header[] headers) {
+            this.document = document;
+            this.headers = headers;
+        }
+    }
+
+    @Getter
+    public static class NameUrl {
+
+        private final String name;
+        private final String url;
+
+        public NameUrl(String name, String url) {
+            this.name = name;
+            this.url = url;
+        }
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/services/MetadataService.java

@@ -0,0 +1,162 @@
+package tn.mnlr.vripper.services;
+
+import com.google.common.cache.CacheBuilder;
+import com.google.common.cache.CacheLoader;
+import com.google.common.cache.LoadingCache;
+import lombok.Getter;
+import lombok.NonNull;
+import lombok.extern.slf4j.Slf4j;
+import net.jodah.failsafe.Failsafe;
+import org.apache.http.client.HttpClient;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpGet;
+import org.apache.http.util.EntityUtils;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Service;
+import org.w3c.dom.Document;
+import org.w3c.dom.Node;
+import tn.mnlr.vripper.exception.DownloadException;
+import tn.mnlr.vripper.exception.PostParseException;
+import tn.mnlr.vripper.jpa.domain.Metadata;
+import tn.mnlr.vripper.jpa.domain.Post;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Objects;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+
+@Slf4j
+@Service
+public class MetadataService {
+
+    private static final List<String> dictionary = Arrays.asList("download", "link", "rapidgator", "filefactory", "filefox");
+    private final LoadingCache<Key, Metadata> cache;
+    private final ConnectionService cm;
+    private final VGAuthService VGAuthService;
+    private final HtmlProcessorService htmlProcessorService;
+    private final XpathService xpathService;
+    @Autowired
+    public MetadataService(ConnectionService cm, VGAuthService VGAuthService, HtmlProcessorService htmlProcessorService, XpathService xpathService) {
+        this.cm = cm;
+        this.VGAuthService = VGAuthService;
+        this.htmlProcessorService = htmlProcessorService;
+        this.xpathService = xpathService;
+
+        CacheLoader<Key, Metadata> loader = new CacheLoader<>() {
+            @Override
+            public Metadata load(@NonNull Key key) {
+                return fetchMetadata(key);
+            }
+        };
+        cache = CacheBuilder.newBuilder()
+                .expireAfterWrite(30, TimeUnit.MINUTES)
+                .build(loader);
+    }
+
+    public Metadata get(Post post) throws ExecutionException {
+        Metadata metadata = new Metadata();
+        Metadata cachedMetadata = cache.get(new Key(post.getPostId(), post.getThreadId(), post.getUrl()));
+        metadata.setPostedBy(cachedMetadata.getPostedBy());
+        metadata.setResolvedNames(List.copyOf(cachedMetadata.getResolvedNames()));
+        return metadata;
+    }
+
+    private Metadata fetchMetadata(Key key) {
+        HttpGet httpGet = cm.buildHttpGet(key.getUrl(), null);
+        Metadata metadata = new Metadata();
+        Failsafe.with(cm.getRetryPolicy())
+                .onFailure(e -> {
+                    if (e.getFailure() instanceof InterruptedException || e.getFailure().getCause() instanceof InterruptedException) {
+                        log.debug("Fetching interrupted");
+                        return;
+                    }
+                    log.error(String.format("Error occurred when getting post metadata, postId %s", key.getPostId()), e.getFailure());
+                })
+                .run(() -> {
+                    HttpClient connection = cm.getClient().build();
+                    try (CloseableHttpResponse response = (CloseableHttpResponse) connection.execute(httpGet, VGAuthService.getContext())) {
+                        if (response.getStatusLine().getStatusCode() / 100 != 2) {
+                            throw new DownloadException(String.format("Unexpected response code '%d' for %s", response.getStatusLine().getStatusCode(), httpGet));
+                        }
+                        try {
+                            if (Thread.interrupted()) {
+                                return;
+                            }
+                            Document document = htmlProcessorService.clean(EntityUtils.toString(response.getEntity()));
+                            Node postNode = xpathService.getAsNode(document, String.format("//li[@id='post_%s']/div[contains(@class, 'postdetails')]", key.getPostId()));
+                            String postedBy = xpathService.getAsNode(postNode, "./div[contains(@class, 'userinfo')]//a[contains(@class, 'username')]//font").getTextContent().trim();
+                            metadata.setPostedBy(postedBy);
+
+                            Node node = xpathService.getAsNode(document, String.format("//div[@id='post_message_%s']", key.getPostId()));
+                            metadata.setResolvedNames(findTitleInContent(node));
+                        } catch (Exception e) {
+                            throw new PostParseException(String.format("Failed to parse thread %s, post %s", key.getThreadId(), key.getPostId()), e);
+                        } finally {
+                            EntityUtils.consumeQuietly(response.getEntity());
+                        }
+                    }
+                });
+        return metadata;
+    }
+
+    private List<String> findTitleInContent(Node node) {
+        List<String> altTitle = new ArrayList<>();
+        findTitle(node, altTitle, new AtomicBoolean(true));
+        return altTitle.stream().distinct().collect(Collectors.toList());
+    }
+
+    private void findTitle(Node node, List<String> altTitle, AtomicBoolean keepGoing) {
+        if (!keepGoing.get()) {
+            return;
+        }
+        if (node.getNodeName().equals("a") || node.getNodeName().equals("img")) {
+            keepGoing.set(false);
+            return;
+        }
+        if (node.getNodeType() == Node.ELEMENT_NODE) {
+            for (int i = 0; i < node.getChildNodes().getLength(); i++) {
+                Node item = node.getChildNodes().item(i);
+                findTitle(item, altTitle, keepGoing);
+                if (!keepGoing.get()) {
+                    return;
+                }
+            }
+
+        } else if (node.getNodeType() == Node.TEXT_NODE) {
+            String text = node.getTextContent().trim();
+            if (!text.isBlank() && dictionary.stream().noneMatch(e -> text.toLowerCase().contains(e.toLowerCase()))) {
+                altTitle.add(text);
+            }
+        }
+    }
+
+    @Getter
+    static class Key {
+        private final String postId;
+        private final String threadId;
+        private final String url;
+
+        Key(String postId, String threadId, String url) {
+            this.postId = postId;
+            this.threadId = threadId;
+            this.url = url;
+        }
+
+        @Override
+        public boolean equals(Object o) {
+            if (this == o) return true;
+            if (o == null || getClass() != o.getClass()) return false;
+            Key key = (Key) o;
+            return Objects.equals(postId, key.postId);
+        }
+
+        @Override
+        public int hashCode() {
+            return Objects.hash(postId);
+        }
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/services/PathService.java

@@ -1,36 +1,99 @@
 package tn.mnlr.vripper.services;
 
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import org.springframework.beans.factory.annotation.Autowired;
+import lombok.Getter;
+import lombok.NonNull;
+import lombok.extern.slf4j.Slf4j;
 import org.springframework.stereotype.Service;
-import tn.mnlr.vripper.entities.Post;
+import tn.mnlr.vripper.exception.RenameException;
+import tn.mnlr.vripper.jpa.domain.Post;
 
 import java.io.File;
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.StandardCopyOption;
+import java.util.concurrent.locks.ReentrantLock;
 
 @Service
+@Slf4j
 public class PathService {
 
-    private static final Logger logger = LoggerFactory.getLogger(PathService.class);
+    private final SettingsService settingsService;
+    private final DataService dataService;
 
-    @Autowired
-    private AppStateService appStateService;
+    @Getter
+    private final ReentrantLock directoryAccess = new ReentrantLock();
 
-    @Autowired
-    private AppSettingsService appSettingsService;
-
-    public final File getDownloadDestinationFolder(String postId) {
-        Post post = appStateService.getCurrentPosts().get(postId);
-        String postTitle = post.getTitle();
-        String threadTitle = post.getThreadTitle();
-        File sourceFolder = appSettingsService.isSubLocation() ? new File(appSettingsService.getDownloadPath(), sanitize(post.getForum())) : new File(appSettingsService.getDownloadPath());
-        sourceFolder = appSettingsService.isThreadSubLocation() ? new File(sourceFolder, threadTitle) : sourceFolder;
-        return new File(sourceFolder, sanitize(postTitle + "_" + postId));
+    public PathService(SettingsService settingsService, DataService dataService) {
+        this.settingsService = settingsService;
+        this.dataService = dataService;
     }
 
-    public final String sanitize(final String folderName) {
+    public final File calcDownloadDirectory(Post post) {
+        return new File(settingsService.getSettings().getDownloadPath(), post.getDownloadDirectory());
+    }
+
+    private File getRootFolder(@NonNull String forum, @NonNull String threadTitle) {
+        File sourceFolder = settingsService.getSettings().getSubLocation() ? new File(settingsService.getSettings().getDownloadPath(), sanitize(forum)) : new File(settingsService.getSettings().getDownloadPath());
+        return settingsService.getSettings().getThreadSubLocation() ? new File(sourceFolder, threadTitle) : sourceFolder;
+    }
+
+    public final void createDefaultPostFolder(Post post) {
+        File downloadDirectory = getRootFolder(post.getForum(), post.getThreadTitle());
+        downloadDirectory = new File(downloadDirectory, settingsService.getSettings().getAppendPostId() ? sanitize(post.getTitle()) + "_" + post.getPostId() : sanitize(post.getTitle()));
+        downloadDirectory = makeDir(downloadDirectory);
+        post.setDownloadDirectory(downloadDirectory.getAbsolutePath().replace(settingsService.getSettings().getDownloadPath(), ""));
+        dataService.updateDownloadDirectory(post.getDownloadDirectory(), post.getId());
+    }
+
+    public final void rename(@NonNull String postId, @NonNull String altName) throws RenameException {
+        Post post = dataService.findPostByPostId(postId).orElseThrow();
+        if (altName.equals(post.getTitle())) {
+            return;
+        }
+
+        // Download have not started yet
+        if (post.getDownloadDirectory() == null) {
+            post.setTitle(altName);
+            dataService.updatePostTitle(post.getTitle(), post.getId());
+            return;
+        }
+
+        File newDownloadDirectory = getRootFolder(post.getForum(), post.getThreadTitle());
+        newDownloadDirectory = new File(newDownloadDirectory, sanitize(altName));
+        File currentDownloadDirectory = calcDownloadDirectory(post);
+        try {
+            directoryAccess.lock();
+            Files.move(currentDownloadDirectory.toPath(), newDownloadDirectory.toPath(), StandardCopyOption.ATOMIC_MOVE);
+            post.setDownloadDirectory(newDownloadDirectory.getAbsolutePath().replace(settingsService.getSettings().getDownloadPath(), ""));
+            dataService.updateDownloadDirectory(post.getDownloadDirectory(), post.getId());
+
+            post.setTitle(altName);
+            dataService.updatePostTitle(post.getTitle(), post.getId());
+        } catch (IOException e) {
+            throw new RenameException(String.format("Failed to move files from %s to %s", currentDownloadDirectory.toString(), newDownloadDirectory.toString()), e);
+        } finally {
+            directoryAccess.unlock();
+        }
+    }
+
+    private File makeDir(@NonNull final File sourceFolder) {
+        int counter = 1;
+        File folder = sourceFolder;
+
+        while (folder.exists()) {
+            folder = new File(sourceFolder.toString() + '.' + counter++);
+        }
+
+        if (!folder.mkdirs()) {
+            throw new RuntimeException(String.format("Failed to create the folder %s", sourceFolder.toString()));
+        }
+
+        return folder;
+    }
+
+    private String sanitize(final String folderName) {
         String sanitizedFolderName = folderName.replaceAll("\\.|\\\\|/|\\||:|\\?|\\*|\"|<|>|\\p{Cntrl}", "_");
-        logger.debug(String.format("%s sanitized to %s", folderName, sanitizedFolderName));
+        log.debug(String.format("%s sanitized to %s", folderName, sanitizedFolderName));
         return sanitizedFolderName;
     }
 

vripper-server/src/main/java/tn/mnlr/vripper/services/PersistenceService.java

@@ -1,161 +0,0 @@
-package tn.mnlr.vripper.services;
-
-import com.fasterxml.jackson.databind.ObjectMapper;
-import io.reactivex.disposables.Disposable;
-import io.reactivex.processors.PublishProcessor;
-import lombok.Getter;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import org.springframework.beans.factory.annotation.Autowired;
-import org.springframework.beans.factory.annotation.Value;
-import org.springframework.stereotype.Service;
-import tn.mnlr.vripper.EventListenerBean;
-import tn.mnlr.vripper.SpringContext;
-import tn.mnlr.vripper.entities.Image;
-import tn.mnlr.vripper.entities.Post;
-import tn.mnlr.vripper.entities.mixin.persistance.ImagePersistanceMixin;
-import tn.mnlr.vripper.entities.mixin.persistance.PostPersistanceMixin;
-
-import javax.annotation.PostConstruct;
-import javax.annotation.PreDestroy;
-import java.io.File;
-import java.io.FileWriter;
-import java.io.IOException;
-import java.io.PrintWriter;
-import java.nio.charset.StandardCharsets;
-import java.nio.file.Files;
-import java.nio.file.Paths;
-import java.util.Arrays;
-import java.util.Date;
-import java.util.HashMap;
-import java.util.Map;
-import java.util.concurrent.TimeUnit;
-
-@Service
-public class PersistenceService {
-
-    private static final Logger logger = LoggerFactory.getLogger(PersistenceService.class);
-
-    @Autowired
-    private AppStateService stateService;
-
-    @Value("${base.dir}")
-    private String baseDir;
-
-    @Getter
-    private String dataPath;
-
-    private ObjectMapper om;
-
-    private Disposable subscription;
-
-    @Getter
-    private PublishProcessor<Map<String, Post>> processor = PublishProcessor.create();
-
-    private PersistenceService() {
-        om = new ObjectMapper();
-        om.addMixIn(Image.class, ImagePersistanceMixin.class);
-        om.addMixIn(Post.class, PostPersistanceMixin.class);
-        subscription = processor
-                .onBackpressureLatest()
-                .throttleLatest(1, TimeUnit.SECONDS)
-                .doOnNext(this::persist)
-                .subscribe();
-    }
-
-    @PostConstruct
-    public void init() {
-        dataPath = baseDir + File.separator + ".vripper" + File.separator + "data.json";
-        File dataFile = new File(dataPath);
-        if (!dataFile.exists()) {
-            try {
-                if (dataFile.getParentFile().mkdirs()) {
-                    logger.debug(String.format("%s is created", dataFile.getParentFile().toString()));
-                }
-                if (!dataFile.getParentFile().isDirectory() || !dataFile.getParentFile().canWrite()) {
-                    logger.error(String.format("Unable to write in %s", dataFile.getParent()));
-                    SpringContext.close();
-                }
-
-                if (dataFile.createNewFile()) {
-                    logger.debug("Data file successfully created");
-                    try (FileWriter fw = new FileWriter(dataFile)) {
-                        fw.write("{}");
-                    }
-                } else {
-                    logger.warn("Data file already exists");
-                }
-            } catch (IOException e) {
-                logger.error("Unable to create data file", e);
-                SpringContext.close();
-            }
-        }
-    }
-
-    @PreDestroy
-    public void preDestroy() {
-        this.subscription.dispose();
-        logger.info(String.format("Destroying %s", PersistenceService.class.getSimpleName()));
-        logger.info("Persisting data before destroying");
-        if (EventListenerBean.isInit()) {
-            this.persist(stateService.getCurrentPosts());
-        }
-    }
-
-    private void persist(Map<String, Post> currentPosts) {
-
-        try (PrintWriter out = new PrintWriter(dataPath, StandardCharsets.UTF_8)) {
-            out.print(om.writeValueAsString(currentPosts));
-        } catch (IOException e) {
-            logger.error("Failed to persist app state", e);
-        }
-    }
-
-    private Map<String, Post> read(String content) {
-
-        try {
-            return om.readValue(content, om.getTypeFactory().constructMapType(HashMap.class, String.class, Post.class));
-        } catch (IOException e) {
-            logger.error("Failed to read app state", e);
-            long timestamp = new Date().getTime();
-            logger.warn(String.format("trying to rename old data file from %s to %s", dataPath, dataPath + "." + timestamp + ".old"));
-            try {
-                Files.move(new File(dataPath).toPath(), new File(dataPath + "." + timestamp + ".old").toPath());
-            } catch (IOException ex) {
-                logger.error(String.format("Failed to rename %s to %s", dataPath, dataPath + ".old"));
-                SpringContext.close();
-            }
-        }
-
-        return new HashMap<>();
-    }
-
-    public void restore() {
-
-        String jsonContent = null;
-        try {
-            jsonContent = String.join("", Files.readAllLines(Paths.get(dataPath), StandardCharsets.UTF_8));
-        } catch (Exception e) {
-            logger.error("data file cannot be read, previous state cannot be restored", e);
-            SpringContext.close();
-        }
-
-        Map<String, Post> read = read(jsonContent);
-
-        stateService.getCurrentPosts().clear();
-        stateService.getCurrentPosts().putAll(read);
-        stateService.getCurrentPosts().values().forEach(p -> {
-            if (Arrays.asList(Post.Status.DOWNLOADING, Post.Status.PARTIAL, Post.Status.PENDING).contains(p.getStatus())) {
-                p.setStatus(Post.Status.STOPPED);
-            }
-        });
-
-        stateService.getCurrentImages().clear();
-        read.values().stream().flatMap(e -> e.getImages().stream()).forEach(e -> {
-            e.setAppStateService(stateService);
-            stateService.getCurrentImages().put(e.getUrl(), e);
-        });
-
-        read.values().forEach(e -> e.setAppStateService(stateService));
-    }
-}

vripper-server/src/main/java/tn/mnlr/vripper/services/PostParser.java

@@ -1,88 +0,0 @@
-package tn.mnlr.vripper.services;
-
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import org.springframework.beans.factory.annotation.Autowired;
-import org.springframework.stereotype.Service;
-import tn.mnlr.vripper.entities.Post;
-import tn.mnlr.vripper.exception.PostParseException;
-import tn.mnlr.vripper.host.Host;
-import tn.mnlr.vripper.q.DownloadQ;
-
-import javax.annotation.PostConstruct;
-import java.util.List;
-
-@Service
-public class PostParser {
-
-    private static final Logger logger = LoggerFactory.getLogger(PostParser.class);
-
-    static final String VR_API = "https://vipergirls.to/vr.php";
-
-    @Autowired
-    private ConnectionManager cm;
-
-    @Autowired
-    private AppStateService appStateService;
-
-    @Autowired
-    private VipergirlsAuthService authService;
-
-    @Autowired
-    private AppSettingsService appSettingsService;
-
-    @Autowired
-    private DownloadQ downloadQ;
-
-    @Autowired
-    private VipergirlsAuthService vipergirlsAuthService;
-
-    @Autowired
-    private List<Host> supportedHosts;
-
-    @Autowired
-    private HtmlProcessorService htmlProcessorService;
-
-    @Autowired
-    private XpathService xpathService;
-
-    @PostConstruct
-    private void init() {
-
-    }
-
-    public synchronized void addPost(String postId, String threadId) throws PostParseException {
-
-        if (appStateService.getCurrentPosts().containsKey(postId)) {
-            logger.warn(String.format("skipping %s, already loaded", postId));
-            return;
-        }
-
-        VRPostParser vrPostParser = new VRPostParser(threadId, postId, cm, vipergirlsAuthService, supportedHosts, htmlProcessorService, xpathService);
-        Post post = vrPostParser.parse();
-
-        post.setAppStateService(appStateService);
-        post.getImages().forEach(e -> e.setAppStateService(appStateService));
-        authService.leaveThanks(post.getUrl(), post.getPostId());
-        if (appSettingsService.isAutoStart()) {
-            logger.debug("Auto start downloads option is enabled");
-            logger.debug(String.format("Starting to enqueue %d jobs for %s", post.getImages().size(), post.getUrl()));
-            post.setStatus(Post.Status.PENDING);
-            try {
-                downloadQ.enqueue(post);
-            } catch (InterruptedException e) {
-                logger.warn("Interruption was caught");
-                Thread.currentThread().interrupt();
-                return;
-            }
-            logger.debug(String.format("Done enqueuing jobs for %s", post.getUrl()));
-        } else {
-            post.setStatus(Post.Status.STOPPED);
-            logger.debug("Auto start downloads option is disabled");
-        }
-    }
-
-    public VRThreadParser createVRThreadParser(String threadId) {
-        return new VRThreadParser(threadId, cm, vipergirlsAuthService, supportedHosts);
-    }
-}

vripper-server/src/main/java/tn/mnlr/vripper/services/PostService.java

@@ -0,0 +1,156 @@
+package tn.mnlr.vripper.services;
+
+import com.google.common.cache.CacheBuilder;
+import com.google.common.cache.CacheLoader;
+import com.google.common.cache.LoadingCache;
+import lombok.Getter;
+import lombok.NonNull;
+import lombok.extern.slf4j.Slf4j;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Service;
+import tn.mnlr.vripper.download.PendingQueue;
+import tn.mnlr.vripper.exception.PostParseException;
+import tn.mnlr.vripper.jpa.domain.Image;
+import tn.mnlr.vripper.jpa.domain.Post;
+import tn.mnlr.vripper.jpa.domain.Queued;
+import tn.mnlr.vripper.jpa.domain.enums.Status;
+import tn.mnlr.vripper.services.domain.*;
+
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Future;
+import java.util.concurrent.TimeUnit;
+
+@Service
+@Slf4j
+public class PostService {
+
+    private final SettingsService settingsService;
+    private final PendingQueue pendingQueue;
+    private final DataService dataService;
+    private final ThreadPoolService threadPoolService;
+    private final Map<String, Future<?>> fetchingMetadata = new ConcurrentHashMap<>();
+    private final VGAuthService VGAuthService;
+
+    @Getter
+    private final LoadingCache<Queued, List<MultiPostItem>> cache;
+
+    @Autowired
+    public PostService(SettingsService settingsService, PendingQueue pendingQueue, DataService dataService, ThreadPoolService threadPoolService, VGAuthService VGAuthService) {
+        this.settingsService = settingsService;
+        this.pendingQueue = pendingQueue;
+        this.dataService = dataService;
+        this.threadPoolService = threadPoolService;
+        this.VGAuthService = VGAuthService;
+
+        CacheLoader<Queued, List<MultiPostItem>> loader = new CacheLoader<>() {
+            @Override
+            public List<MultiPostItem> load(@NonNull Queued multiPostItem) throws Exception {
+                ApiThreadParser apiThreadParser = new ApiThreadParser(multiPostItem);
+                return apiThreadParser.parse();
+            }
+        };
+
+        cache = CacheBuilder.newBuilder()
+                .expireAfterWrite(30, TimeUnit.MINUTES)
+                .build(loader);
+    }
+
+    public void addPost(String postId, String threadId) throws PostParseException {
+
+        if (dataService.exists(postId)) {
+            log.warn(String.format("skipping %s, already loaded", postId));
+            return;
+        }
+
+        ApiPostParser apiPostParser = new ApiPostParser(threadId, postId);
+        ApiPost apiPost = apiPostParser.parse();
+        if (apiPost.getPost().isEmpty()) {
+            throw new PostParseException(String.format("parsing failed for thread %s, post %s", threadId, postId));
+        }
+
+        Post post = apiPost.getPost().get();
+        Set<Image> images = apiPost.getImages();
+
+        dataService.newPost(post, images);
+
+        // Metadata thread
+        fetchingMetadata.put(post.getPostId(), threadPoolService.getGeneralExecutor().submit(new MetadataRunnable(post)));
+
+        if (settingsService.getSettings().getAutoStart()) {
+            log.debug("Auto start downloads option is enabled");
+            post.setStatus(Status.PENDING);
+            try {
+                pendingQueue.enqueue(post, images);
+            } catch (InterruptedException e) {
+                log.warn("Interruption was caught");
+                Thread.currentThread().interrupt();
+                return;
+            }
+            log.debug(String.format("Done enqueuing jobs for %s", post.getUrl()));
+        } else {
+            post.setStatus(Status.STOPPED);
+            log.debug("Auto start downloads option is disabled");
+        }
+        if (!settingsService.getSettings().getLeaveThanksOnStart()) {
+            VGAuthService.leaveThanks(post);
+        }
+        dataService.updatePostStatus(post.getStatus(), post.getId());
+    }
+
+    public void stopFetchingMetadata(Post post) {
+        this.fetchingMetadata.forEach((k, v) -> {
+            if (k.equals(post.getPostId())) {
+                v.cancel(true);
+            }
+        });
+        fetchingMetadata.remove(post.getPostId());
+    }
+
+    public void processMultiPost(List<Queued> queuedList) throws Exception {
+        for (Queued queued : queuedList) {
+            if (queued.getPostId() != null) {
+                addPost(queued.getPostId(), queued.getThreadId());
+            } else {
+                threadPoolService.getGeneralExecutor().submit(() -> this.multiPost(queued));
+            }
+        }
+    }
+
+    private void multiPost(Queued queued) {
+
+        List<MultiPostItem> multiPostItems;
+        try {
+            multiPostItems = cache.get(queued);
+        } catch (ExecutionException e) {
+            log.error(String.format("Failed to add post with thread id %s, postId %s", queued.getThreadId(), queued.getPostId()), e);
+            return;
+        }
+        queued.setTotal(multiPostItems.size());
+        queued.done();
+        log.debug(String.format("%d found for %s", multiPostItems.size(), queued.getLink()));
+        if (multiPostItems.size() == 1) {
+            try {
+                addPost(multiPostItems.get(0).getPostId(), multiPostItems.get(0).getThreadId());
+            } catch (PostParseException e) {
+                log.error(String.format("Failed to add post with postId %s", multiPostItems.get(0).getPostId()), e);
+                return;
+            }
+            log.debug(String.format("threadId %s, postId %s is added automatically for download", queued.getThreadId(), queued.getPostId()));
+        } else {
+            if (dataService.findQueuedByThreadId(queued.getThreadId()).isEmpty()) {
+                dataService.newQueueLink(queued);
+            } else {
+                log.info(String.format("Thread with id = %s is already loaded", queued.getThreadId()));
+            }
+        }
+
+    }
+
+    public void remove(String threadId) {
+        dataService.removeQueueLink(threadId);
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/services/QueuedVGLink.java

@@ -1,38 +0,0 @@
-package tn.mnlr.vripper.services;
-
-import lombok.Getter;
-import lombok.Setter;
-import lombok.ToString;
-
-import java.util.Objects;
-
-@Getter
-@ToString
-public class QueuedVGLink {
-
-    private final String type = "grabQueue";
-    private final String link;
-    private final String threadId;
-    private final String postId;
-    @Setter
-    boolean removed;
-
-    public QueuedVGLink(String link, String threadId, String postId) {
-        this.link = link;
-        this.threadId = threadId;
-        this.postId = postId;
-    }
-
-    @Override
-    public boolean equals(Object o) {
-        if (this == o) return true;
-        if (o == null || getClass() != o.getClass()) return false;
-        QueuedVGLink that = (QueuedVGLink) o;
-        return Objects.equals(link, that.link);
-    }
-
-    @Override
-    public int hashCode() {
-        return Objects.hash(link);
-    }
-}

vripper-server/src/main/java/tn/mnlr/vripper/services/SettingsService.java

@@ -0,0 +1,293 @@
+package tn.mnlr.vripper.services;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import lombok.Getter;
+import lombok.NoArgsConstructor;
+import lombok.Setter;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.commons.codec.digest.DigestUtils;
+import org.springframework.beans.factory.annotation.Value;
+import org.springframework.stereotype.Service;
+import reactor.core.publisher.Flux;
+import reactor.core.publisher.Sinks;
+import tn.mnlr.vripper.SpringContext;
+import tn.mnlr.vripper.exception.ValidationException;
+import tn.mnlr.vripper.listener.EmitHandler;
+
+import javax.annotation.PostConstruct;
+import javax.annotation.PreDestroy;
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.InvalidPathException;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+
+import static java.nio.file.StandardOpenOption.*;
+
+@Service
+@Setter
+@Slf4j
+public class SettingsService {
+
+    private final Path configPath;
+    private final ObjectMapper om = new ObjectMapper();
+
+    private Sinks.Many<Settings> sink = Sinks.many().multicast().onBackpressureBuffer();
+
+    @Getter
+    private Settings settings = new Settings();
+
+    public SettingsService(@Value("${base.dir}") String baseDir, @Value("${base.dir.name}") String baseDirName) {
+        this.configPath = Paths.get(baseDir, baseDirName, "config.json");
+        om.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
+    }
+
+    @PostConstruct
+    private void init() {
+        restore();
+        sink.emitNext(settings, EmitHandler.RETRY);
+    }
+
+    Flux<Settings> getSettingsFlux() {
+        return sink.asFlux();
+    }
+
+    public void newSettings(Settings settings) {
+
+        if (settings.getVLogin() != null && settings.getVLogin()) {
+            if (!this.settings.getVPassword().equals(settings.getVPassword())) {
+                settings.setVPassword(DigestUtils.md5Hex(settings.getVPassword()));
+            }
+        } else {
+            settings.setVUsername("");
+            settings.setVPassword("");
+            settings.setVThanks(false);
+            settings.setVLogin(false);
+        }
+        this.settings = settings;
+
+        save();
+        sink.emitNext(settings, EmitHandler.RETRY);
+    }
+
+    public void restore() {
+        try {
+            settings = om.readValue(configPath.toFile(), Settings.class);
+        } catch (IOException e) {
+            log.error("Failed restore user settings", e);
+            settings = new Settings();
+        }
+
+        if (settings.getDownloadPath() == null) {
+            settings.setDownloadPath(System.getProperty("user.home"));
+        }
+
+        if (settings.getMaxThreads() == null) {
+            settings.setMaxThreads(4);
+        }
+
+        if (settings.getMaxTotalThreads() == null) {
+            settings.setMaxTotalThreads(0);
+        }
+
+        if (settings.getAutoStart() == null) {
+            settings.setAutoStart(true);
+        }
+
+        if (settings.getVLogin() == null) {
+            settings.setVLogin(false);
+        }
+
+        if (settings.getVUsername() == null) {
+            settings.setVUsername("");
+        }
+
+        if (settings.getVPassword() == null) {
+            settings.setVPassword("");
+        }
+
+        if (settings.getVThanks() == null) {
+            settings.setVThanks(false);
+        }
+
+        if (settings.getDesktopClipboard() == null) {
+            settings.setDesktopClipboard(false);
+        }
+
+        if (settings.getForceOrder() == null) {
+            settings.setForceOrder(false);
+        }
+
+        if (settings.getSubLocation() == null) {
+            settings.setSubLocation(false);
+        }
+
+        if (settings.getThreadSubLocation() == null) {
+            settings.setThreadSubLocation(false);
+        }
+
+        if (settings.getClearCompleted() == null) {
+            settings.setClearCompleted(false);
+        }
+
+        if (settings.getDarkTheme() == null) {
+            settings.setDarkTheme(false);
+        }
+
+        if (settings.getAppendPostId() == null) {
+            settings.setAppendPostId(true);
+        }
+
+        if (settings.getLeaveThanksOnStart() == null) {
+            settings.setLeaveThanksOnStart(false);
+        }
+
+        if (settings.getConnectionTimeout() == null) {
+            settings.setConnectionTimeout(30);
+        }
+
+        if (settings.getMaxAttempts() == null) {
+            settings.setMaxAttempts(5);
+        }
+
+        try {
+            check(this.settings);
+        } catch (ValidationException e) {
+            log.error(String.format("Your settings are invalid, either remove %s, or fix it", configPath.toString()), e);
+            SpringContext.close();
+        }
+
+        save();
+    }
+
+
+    public void save() {
+        try {
+            Files.write(configPath, om.writeValueAsBytes(settings), CREATE, WRITE, TRUNCATE_EXISTING, SYNC);
+        } catch (IOException e) {
+            log.error("Failed to store user settings", e);
+        }
+    }
+
+    @PreDestroy
+    private void destroy() {
+        save();
+        sink.emitComplete(EmitHandler.RETRY);
+    }
+
+    public void check(Settings settings) throws ValidationException {
+
+        Path path;
+        try {
+            path = Paths.get(settings.getDownloadPath());
+        } catch (InvalidPathException e) {
+            throw new ValidationException(String.format("%s is invalid", settings.getDownloadPath()));
+        }
+        if (!Files.exists(path)) {
+            throw new ValidationException(String.format("%s does not exist", settings.getDownloadPath()));
+        } else if (!Files.isDirectory(path)) {
+            throw new ValidationException(String.format("%s is not a directory", settings.getDownloadPath()));
+        }
+
+        if (settings.getMaxTotalThreads() < 0 || settings.getMaxTotalThreads() > 12) {
+            throw new ValidationException(String.format("Invalid max global concurrent download settings, values must be in [%d,%d]", 0, 12));
+        }
+
+        if (settings.getMaxThreads() < 1 || settings.getMaxThreads() > 4) {
+            throw new ValidationException(String.format("Invalid max concurrent download settings, values must be in [%d,%d]", 1, 4));
+        }
+
+        if (settings.getConnectionTimeout() < 1 || settings.getConnectionTimeout() > 300) {
+            throw new ValidationException(String.format("Invalid connection timeout settings, values must be in [%d,%d]", 1, 300));
+        }
+
+        if (settings.getMaxAttempts() < 1 || settings.getMaxAttempts() > 10) {
+            throw new ValidationException(String.format("Invalid maximum attempts settings, values must be in [%d,%d]", 1, 10));
+        }
+    }
+
+    public Theme getTheme() {
+        return new Theme(this.settings.getDarkTheme());
+    }
+
+    public void setTheme(Theme theme) {
+        this.settings.setDarkTheme(theme.darkTheme);
+        save();
+    }
+
+    @Getter
+    @NoArgsConstructor
+    public static class Theme {
+
+        @JsonProperty("darkTheme")
+        private boolean darkTheme;
+
+        Theme(boolean darkTheme) {
+            this.darkTheme = darkTheme;
+        }
+    }
+
+    @Getter
+    @Setter
+    @NoArgsConstructor
+    @JsonInclude(JsonInclude.Include.NON_NULL)
+    public static class Settings {
+
+        @JsonProperty("downloadPath")
+        private String downloadPath;
+
+        @JsonProperty("maxThreads")
+        private Integer maxThreads;
+
+        @JsonProperty("maxTotalThreads")
+        private Integer maxTotalThreads;
+
+        @JsonProperty("autoStart")
+        private Boolean autoStart;
+
+        @JsonProperty("vLogin")
+        private Boolean vLogin;
+
+        @JsonProperty("vUsername")
+        private String vUsername;
+
+        @JsonProperty("vPassword")
+        private String vPassword;
+
+        @JsonProperty("vThanks")
+        private Boolean vThanks;
+
+        @JsonProperty("desktopClipboard")
+        private Boolean desktopClipboard;
+
+        @JsonProperty("forceOrder")
+        private Boolean forceOrder;
+
+        @JsonProperty("subLocation")
+        private Boolean subLocation;
+
+        @JsonProperty("threadSubLocation")
+        private Boolean threadSubLocation;
+
+        @JsonProperty("clearCompleted")
+        private Boolean clearCompleted;
+
+        @JsonProperty("darkTheme")
+        private Boolean darkTheme;
+
+        @JsonProperty("appendPostId")
+        private Boolean appendPostId;
+
+        @JsonProperty("leaveThanksOnStart")
+        private Boolean leaveThanksOnStart;
+
+        @JsonProperty("connectionTimeout")
+        private Integer connectionTimeout;
+
+        @JsonProperty("maxAttempts")
+        private Integer maxAttempts;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/services/ThreadPoolService.java

@@ -0,0 +1,19 @@
+package tn.mnlr.vripper.services;
+
+import lombok.Getter;
+import org.springframework.stereotype.Service;
+
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.TimeUnit;
+
+@Service
+public class ThreadPoolService {
+    @Getter
+    private final ExecutorService generalExecutor = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());
+
+    public void destroy() throws Exception {
+        generalExecutor.shutdown();
+        generalExecutor.awaitTermination(5, TimeUnit.SECONDS);
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/services/ThumbnailGenerator.java

@@ -1,165 +0,0 @@
-package tn.mnlr.vripper.services;
-
-import com.google.common.cache.CacheBuilder;
-import com.google.common.cache.CacheLoader;
-import com.google.common.cache.LoadingCache;
-import com.google.common.cache.Weigher;
-import lombok.Getter;
-import lombok.NoArgsConstructor;
-import lombok.Setter;
-import org.imgscalr.Scalr;
-import org.springframework.beans.factory.annotation.Autowired;
-import org.springframework.beans.factory.annotation.Value;
-import org.springframework.stereotype.Service;
-import org.springframework.util.FileSystemUtils;
-
-import javax.annotation.PostConstruct;
-import javax.imageio.ImageIO;
-import java.awt.image.BufferedImage;
-import java.io.File;
-import java.io.IOException;
-import java.nio.file.FileVisitResult;
-import java.nio.file.Files;
-import java.nio.file.Path;
-import java.nio.file.SimpleFileVisitor;
-import java.nio.file.attribute.BasicFileAttributes;
-import java.time.Duration;
-import java.util.Objects;
-import java.util.Optional;
-import java.util.concurrent.atomic.AtomicLong;
-
-@Service
-public class ThumbnailGenerator {
-
-    @Getter
-    private LoadingCache<CacheKey, byte[]> thumbnails;
-
-    @Value("${base.dir}")
-    private String baseDir;
-
-    @Autowired
-    private PathService pathService;
-
-    @Getter
-    private File cacheFolder;
-    CacheLoader<CacheKey, byte[]> loader = new CacheLoader<>() {
-        @Override
-        public byte[] load(CacheKey key) throws Exception {
-            File destinationFolder = pathService.getDownloadDestinationFolder(key.getPostId());
-            if (!destinationFolder.exists() || !destinationFolder.isDirectory()) {
-                return null;
-            }
-            return Files.readAllBytes(generateThumbnail(new File(destinationFolder, key.getImgName()), key.getPostId()).toPath());
-        }
-    };
-
-    @PostConstruct
-    private void init() throws Exception {
-        cacheFolder = new File(baseDir + File.separator + ".vripper" + File.separator + "cache");
-        cacheFolder.mkdirs();
-        if (!cacheFolder.exists()) {
-            throw new Exception(String.format("%s could not be created", cacheFolder.toString()));
-        }
-        thumbnails = CacheBuilder.newBuilder()
-                .expireAfterAccess(Duration.ofMinutes(30))
-                .weigher((Weigher<CacheKey, byte[]>) (k, v) -> v.length)
-                .maximumWeight(104_857_600)
-                .build(loader);
-    }
-
-    public void clearCache() {
-        thumbnails.invalidateAll();
-        for (File file : Optional.ofNullable(cacheFolder.listFiles()).orElse(new File[]{})) {
-            FileSystemUtils.deleteRecursively(file);
-        }
-    }
-
-    public long cacheSize() {
-
-        final AtomicLong size = new AtomicLong(0);
-
-        try {
-            Files.walkFileTree(cacheFolder.toPath(), new SimpleFileVisitor<>() {
-                @Override
-                public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {
-
-                    size.addAndGet(attrs.size());
-                    return FileVisitResult.CONTINUE;
-                }
-
-                @Override
-                public FileVisitResult visitFileFailed(Path file, IOException exc) {
-
-                    System.out.println("skipped: " + file + " (" + exc + ")");
-                    // Skip folders that can't be traversed
-                    return FileVisitResult.CONTINUE;
-                }
-
-                @Override
-                public FileVisitResult postVisitDirectory(Path dir, IOException exc) {
-
-                    if (exc != null)
-                        System.out.println("had trouble traversing: " + dir + " (" + exc + ")");
-                    else
-                        size.addAndGet(dir.toFile().length());
-                    // Ignore errors traversing a folder
-                    return FileVisitResult.CONTINUE;
-                }
-            });
-        } catch (IOException e) {
-            throw new AssertionError("walkFileTree will not throw IOException if the FileVisitor does not");
-        }
-
-        return size.get();
-    }
-
-    private File generateThumbnail(File inputFile, String postId) throws Exception {
-        if (!inputFile.exists()) {
-            throw new Exception(String.format("Input file %s does not exist", inputFile.toString()));
-        }
-        File postsCacheFolder = new File(cacheFolder, postId);
-        postsCacheFolder.mkdirs();
-        if (!postsCacheFolder.exists()) {
-            throw new Exception(String.format("%s could not be created", postsCacheFolder.toString()));
-        }
-
-        File thumbFile = new File(postsCacheFolder, inputFile.getName());
-        if (thumbFile.exists()) {
-            return thumbFile;
-        }
-
-        BufferedImage image = ImageIO.read(inputFile);
-        BufferedImage resize = Scalr.resize(image, 350, 350);
-        ImageIO.write(resize, "jpg", thumbFile);
-        image.flush();
-        resize.flush();
-        return thumbFile;
-    }
-
-    @Getter
-    @Setter
-    @NoArgsConstructor
-    public static class CacheKey {
-        private String postId;
-        private String imgName;
-
-        public CacheKey(String postId, String imgName) {
-            this.postId = postId;
-            this.imgName = imgName;
-        }
-
-        @Override
-        public boolean equals(Object o) {
-            if (this == o) return true;
-            if (o == null || getClass() != o.getClass()) return false;
-            CacheKey cacheKey = (CacheKey) o;
-            return Objects.equals(postId, cacheKey.postId) &&
-                    Objects.equals(imgName, cacheKey.imgName);
-        }
-
-        @Override
-        public int hashCode() {
-            return Objects.hash(postId, imgName);
-        }
-    }
-}