v3.3.5

Change event names and add some icons

.gitignore

@@ -3,4 +3,5 @@
 /**/*/target
 /**/*/build-dir
 /**/*/java-runtime
-.idea
+.idea
+.vripper

CHANGELOG.md

@@ -1,5 +1,199 @@
 # Changelog
 
+## [3.3.5] - 2021-03-18
+
+### Changed
+
+- Bug fixe
+
+## [3.3.4] - 2021-03-18
+
+### Changed
+
+- Bug fixes
+
+## [3.3.3] - 2021-03-13
+
+### Changed
+
+- Remove unnecessary event logs
+
+## [3.3.2] - 2021-03-11
+
+### Changed
+
+- Fix a bug with event logs
+- Change event names and add some icons
+
+## [3.3.1] - 2021-02-18
+
+### Added
+
+- Add about menu
+
+### Changed
+
+- Fix README.md
+
+## [3.3.0] - 2021-02-16
+
+### Changed
+
+- Add support for proxies
+- Add a Event Logs
+- Fix bugs with the database
+
+## [3.2.3] - 2021-01-13
+
+### Changed
+
+- Finished posts are not cleared from the UI
+
+## [3.2.2] - 2021-01-13
+
+### Changed
+
+- Set total download to max value
+
+## [3.2.1] - 2021-01-12
+### Added
+- Add settings for connection timeout
+- Add settings for maximum attempts
+### Changed
+- Enhance posts rename logic
+- Dependencies update
+- Switch reactive from rx-java to reactor
+
+## [3.2.0] - 2020-11-16
+### Changed
+- Fix build issue
+
+## [3.1.0] - 2020-11-16
+### Changed
+- UI revamp
+- Fix bugs
+
+## [3.0.11] - 2020-10-31
+### Added
+- Support for vipr.im
+
+## [3.0.10] - 2020-10-04
+### Changed
+- Fix bug on settings update
+
+## [3.0.9] - 2020-09-27
+### Changed
+- Revert to appending post id to download folders
+- Fix a bug that lead to adding duplicate posts
+- Remove old unused settings
+ 
+## [3.0.8] - 2020-08-22
+### Changed
+- Bug fix which breaks concurrent downloads
+
+## [3.0.7] - 2020-08-22
+### Changed
+- Fix MacOs build
+
+## [3.0.6] - 2020-08-18
+### Changed
+- Fix TurboImageHost
+
+## [3.0.5] - 2020-08-17
+### Changed
+- Update packages
+- Host class refactoring
+
+## [3.0.4] - 2020-08-16
+### Changed
+- Improve rename UI
+
+## [3.0.3] - 2020-08-16
+### Changed
+- Fix bugs with electron app
+
+## [3.0.2] - 2020-08-15
+### Changed
+- Fix bugs with download queue
+
+## [3.0.1] - 2020-08-03
+### Changed
+- Fix bugs with download queue
+
+## [3.0.0] - 2020-08-03
+### Changed
+- Major rewrites
+- Introducing a proper database for persistence
+
+## [2.12.1] - 2020-05-22
+### Changed
+- Fix alternative name bug
+
+## [2.12.0] - 2020-05-21
+### Changed
+- Use new user hash from the api
+
+## [2.11.7] - 2020-05-08
+### Changed
+- Fix bug with paths creation
+- Move Open Link button
+- Rename rename all title
+
+## [2.11.6] - 2020-05-04
+### Changed
+- PNG filenames are now correctly named
+- Fix issue with special characters in post titles
+
+## [2.11.5] - 2020-05-03
+### Changed
+- Reduce rows buffer size
+
+## [2.11.4] - 2020-05-03
+### Changed
+- Code cleanup
+- Dependencies update
+
+## [2.11.3] - 2020-04-30
+### Changed
+- Source code restructure
+- UI fixes and enhancements
+### Added
+- Add menu entry to rename a gallery to first alternative title
+- Add menu on toolbar for massive rename to first alternative title
+
+## [2.11.2] - 2020-04-26
+### Changed
+- Bug fixes
+
+## [2.11.1] - 2020-04-25
+### Changed
+- Code source maintenance
+### Added
+- Added option to rename gallery
+
+## [2.11.0] - 2020-03-29
+### Changed
+- Upgrade Electron to v8
+- Upgrade Angular to v9
+- App settings are now portable in a json file
+- General code source maintenance
+### Added
+- Added Support for pixroute.com
+- Added grab post title from thread option
+
+## [2.10.8] - 2020-01-25
+### Changed
+- Fix app url
+
+## [2.10.7] - 2020-01-25
+### Changed
+- Drop SockJs and switch to WebSocket
+- Autoreconnect front to back
+
+## [2.10.6] - 2020-01-19
+### Changed
+- Limit cache size
+
 ## [2.10.5] - 2020-01-18
 ### Changed
 - Bug fixes
@@ -304,4 +498,4 @@
 - Add real time communication for progress updates via websocket (SockJS)
 - Add settings for download path, max concurrent download, enable/disable integration with vipergirls
 - Add integration with vipergitls.to (possibility to auto leave thanks after grab post)
-- Add support for acidimg.cc, imagezilla.net, imgbox.com, imx.to, pixhost.to, turboimagehost.com
+- Add support for acidimg.cc, imagezilla.net, imgbox.com, imx.to, pixhost.to, turboimagehost.com

README.md

@@ -1,26 +1,30 @@
-# Viper Ripper!
+# VRipper!
 
-This is my spin for a gallery ripper app for [vipergirls](https://vipergirls.to) website.
+This is my spin for a cross platform gallery ripper app for [vipergirls](https://vipergirls.to) website.
 The purpose of this project is to build a robust and clean application to conveniently download photo galleries using modern web technologies, Java + Spring boot for the back end and angular + electron for the front end.
 
 ## How to build
 You need a recent version of maven 3.6.1+.
 
 The application support 2 build modes:
- - Desktop app (*Partially cross platform): Self contained app, built with electron, depends on Java runtimes.
- - Server app (Fully cross platform): Depends on Java runtimes, you need a web browser to access the app UI.
+ - Desktop app: Self contained app, built with electron includes Java runtimes.
+ - Server app: Depends on Java runtimes, you need a web browser to access the app UI.
 
 Most people will be interested only on the desktop app, however if you have a nas or a server, the server app is there for you.
 
 To build the server app:
 
-    mvn clean install
+    mvn clean install -DskipTests
+
 To build the desktop app:
 
-    mvn clean install -Pelectron
+    mvn clean install -Pelectron -DskipTests
 
-Maven will automatically handle front end compilation. However, for development, you will need to install a recent version of nodejs on your system.
+Maven will automatically handle front end compilation. However, for development, you will need to install a recent
+version of nodejs on your system.
 
-*Partially cross platform:
-Technically cross platform, but the Desktop app relies on electron builder to make appropriate binaries for each platform. mac binaries can only be generated on a mac, and i don't own one. So i can only provide binaries for windows and linux.
-The Server app however depends only on Java, so it will work just fine on a mac, you only need a browser to access it.
+If you feel generous and want to support me, you can donate some coins, and I would be very grateful 🙏
+
+I accept Bitcoin (BTC) for donations, following is my address
+
+    bc1q9jzzw3st49r9vmxgtylq4vrd67cl7zfclf3lsq

pom.xml

@@ -4,12 +4,12 @@
     <modelVersion>4.0.0</modelVersion>
     <groupId>tn.mnlr</groupId>
     <artifactId>vripper</artifactId>
-    <version>2.10.5</version>
+    <version>3.3.5</version>
     <packaging>pom</packaging>
     <parent>
         <groupId>org.springframework.boot</groupId>
         <artifactId>spring-boot-starter-parent</artifactId>
-        <version>2.1.1.RELEASE</version>
+        <version>2.4.1</version>
         <relativePath/> <!-- lookup parent from repository -->
     </parent>
     <modules>
@@ -17,4 +17,4 @@
         <module>vripper-ui</module>
         <module>vripper-electron</module>
     </modules>
-</project>
+</project>

vripper-electron/main.js

@@ -1,12 +1,16 @@
-const { app, BrowserWindow } = require("electron");
+require('v8-compile-cache');
+const {app, BrowserWindow} = require("electron");
 const path = require("path");
 const url = require("url");
 const getPort = require("get-port");
-const { spawn } = require("child_process");
-const { ipcMain } = require("electron");
-const { dialog } = require("electron");
+const {spawn} = require("child_process");
+const {ipcMain} = require("electron");
+const {dialog} = require("electron");
 const axios = require('axios');
-const appDir = process.env.APPDIR;
+
+// non null value when it is an AppImage
+const appImageDir = process.env.APPDIR;
+const appImagePath = process.env.APPIMAGE;
 
 let win;
 let vripperServer;
@@ -18,139 +22,150 @@ let terminationInteval;
 let terminated = false;
 
 process.on("uncaughtException", err => {
-  dialog.showErrorBox(err.message, err.stack);
-  process.exit(1);
+    dialog.showErrorBox(err.message, err.stack);
+    process.exit(1);
 });
 
-function createWindow() {
+createWindow = () => {
 
-  if (process.platform === 'win32') {
-    app.setAppUserModelId("tn.mnlr.vripper");
-  }
+    if (process.platform === 'win32') {
+        app.setAppUserModelId("tn.mnlr.vripper");
+    }
 
-  let icon;
-  if(process.platform === "win32") {
-    icon = __dirname + '/icon.ico';
-  } else {
-    icon = __dirname + '/icon.png';
-  }
-  win = new BrowserWindow({
-    width: 1024,
-    height: 800,
-    minWidth: 800,
-    minHeight: 600,
-    webPreferences: {
-      nodeIntegration: true
-    },
-    icon: icon
-  });
+    let icon;
+    if (process.platform === "win32") {
+        icon = __dirname + '/icon.ico';
+    } else {
+        icon = __dirname + '/icon.png';
+    }
+    win = new BrowserWindow({
+        width: 1024,
+        height: 800,
+        minWidth: 800,
+        minHeight: 600,
+        webPreferences: {
+            nodeIntegration: true,
+            enableRemoteModule: true
+        },
+        icon: icon
+    });
 
-  win.removeMenu();
-  win.setMenu(null);
-  win.setMenuBarVisibility(false);
+    win.removeMenu();
+    win.setMenu(null);
+    win.setMenuBarVisibility(false);
 
-  win.loadURL(
-    url.format({
-      pathname: path.join(__dirname, `vripper-ui/index.html`),
-      protocol: "file:",
-      slashes: true
-    })
-  );
+    win.loadURL(
+        url.format({
+            pathname: path.join(__dirname, `vripper-ui/index.html`),
+            protocol: "file:",
+            slashes: true
+        })
+    );
 
-  // win.webContents.openDevTools();
+    // win.webContents.openDevTools();
 
-  win.on("closed", () => {
-    win = null;
-  });
+    win.on("closed", () => {
+        win = null;
+    });
 }
 
-const gotTheLock = app.requestSingleInstanceLock();
-
-if (!gotTheLock) {
-  app.quit();
-} else {
-  getPort().then(port => {
-    serverPort = port;
-    ipcMain.on("get-port", event => {
-      event.reply("port", port);
-    });
-    let javaBinPath;
-    if(appDir !== undefined) {
-      javaBinPath = path.join(appDir, "java-runtime/bin/java");
-    } else {
-      if(process.platform === 'darwin') {
-        javaBinPath = path.join(app.getPath('exe'), "../../java-runtime/bin/java");
-      } else {
-        javaBinPath = path.join(app.getPath('exe'), "../java-runtime/bin/java");
-      }
-    }
-    let jarPath;
-    if(appDir !== undefined) {
-      jarPath = path.join(appDir, "bin/vripper-server.jar");
-    } else {
-      if(process.platform === 'darwin') {
-        jarPath = path.join(app.getPath('exe'), "../../bin/vripper-server.jar");
-      } else {
-        jarPath = path.join(app.getPath('exe'), "../bin/vripper-server.jar");
-      }
-    }
-    vripperServer = spawn(javaBinPath, [
-      "-Xms256m",
-      "-Dvripper.server.port=" + port,
-      "-jar",
-      jarPath
-    ], {
-      stdio: 'ignore'
-    });
-    vripperServer.on('exit', (code, signal) => {
-      console.log(`vripper server terminated, code = ${code}, signal = ${signal}`);
-      terminated = true;
-      app.quit();
-    });
-  });
-  
-  app.on("second-instance", (event, commandLine, workingDirectory) => {
-    if (win) {
-      if (win.isMinimized()) win.restore();
-      win.focus();
-    }
-  });
-
-  app.on("ready", createWindow);
-
-  app.on("window-all-closed", () => {
-    if (process.platform !== "darwin") {
-      axios.post('http://localhost:' + serverPort + '/actuator/shutdown', {}, {
-        headers: { 'content-type': 'application/json' },
-      }).then((response) => {
-        terminationInteval=  setInterval(() => {
-          terminationAttemps++;
-          if(terminated) {
-            console.log('viper server terminated');
-            clearInterval(terminationInteval);
-            app.quit();
-          } else if(terminationAttemps > maxTerminationAttemps) {
-            console.log('viper server is not terminated');
-            console.log('Proceed to kill');
-            vripperServer.kill('SIGKILL');
-            clearInterval(terminationInteval);
-            app.quit();
-          }
+shutdownServer = () => {
+    axios.post('http://localhost:' + serverPort + '/actuator/shutdown', {}, {
+        headers: {'content-type': 'application/json'},
+    }).then((response) => {
+        terminationInteval = setInterval(() => {
+            terminationAttemps++;
+            if (terminated) {
+                console.log('viper server terminated');
+                clearInterval(terminationInteval);
+                app.quit();
+            } else if (terminationAttemps > maxTerminationAttemps) {
+                console.log('viper server is not terminated');
+                console.log('Proceed to kill');
+                vripperServer.kill('SIGKILL');
+                clearInterval(terminationInteval);
+                app.quit();
+            }
         }, 1000);
-      })
-      .catch((error) => {
+    }).catch((error) => {
         // Terminate immediately
         console.log(error);
         vripperServer.kill('SIGKILL');
         terminated = true;
         app.quit();
-      });
-    }
-  });
-
-  app.on("activate", () => {
-    if (win === null) {
-      createWindow();
-    }
-  });
+    });
+}
+
+const gotTheLock = app.requestSingleInstanceLock();
+
+if (!gotTheLock) {
+    app.quit();
+} else {
+    getPort().then(port => {
+        serverPort = port;
+        ipcMain.on("get-port", event => {
+            event.reply("port", port);
+        });
+
+        const appPath = path.join(app.getAppPath(), '../../');
+        let javaBinPath, jarPath, baseDir;
+        if (appImageDir !== undefined && process.platform === 'linux') {
+            javaBinPath = path.join(appImageDir, "java-runtime/bin/java");
+            jarPath = path.join(appImageDir, "bin/vripper-server.jar");
+            baseDir = path.join(appImagePath, '..');
+        } else if (process.platform === 'darwin') {
+            javaBinPath = path.join(appPath, "java-runtime/bin/java");
+            jarPath = path.join(appPath, "bin/vripper-server.jar");
+            baseDir = app.getPath('appData');
+        } else if (process.platform === 'win32') {
+            javaBinPath = path.join(appPath, "java-runtime/bin/java");
+            jarPath = path.join(appPath, "bin/vripper-server.jar");
+            baseDir = appPath;
+        } else {
+            console.error(`Unknown platform ${process.platform}`);
+            app.quit();
+        }
+
+        vripperServer = spawn(javaBinPath, [
+            "-Xms256m",
+            "-Dvripper.server.port=" + port,
+            "-Dbase.dir=" + baseDir,
+            "-jar",
+            jarPath
+        ], {
+            stdio: 'ignore'
+        });
+        vripperServer.on('exit', (code, signal) => {
+            console.log(`vripper server terminated, code = ${code}, signal = ${signal}`);
+            terminated = true;
+            app.quit();
+        });
+    });
+
+    app.on("second-instance", (event, commandLine, workingDirectory) => {
+        if (win) {
+            if (win.isMinimized()) win.restore();
+            win.focus();
+        }
+    });
+
+    app.on("ready", createWindow);
+
+    app.on("window-all-closed", () => {
+        if (process.platform !== "darwin") {
+            shutdownServer();
+        }
+    });
+
+    app.on("will-quit", () => {
+        if (process.platform === "darwin") {
+            shutdownServer();
+        }
+    });
+
+    app.on("activate", () => {
+        if (win === null) {
+            createWindow();
+        }
+    });
 }

vripper-electron/package.json

@@ -1,6 +1,6 @@
 {
   "name": "vripper-electron",
-  "version": "2.10.5",
+  "version": "3.3.5",
   "description": "A ripper for vipergirls.to built using web technolgies",
   "main": "main.js",
   "author": "death-claw <53543762+death-claw@users.noreply.github.com>",
@@ -40,14 +40,9 @@
     "win": {
       "icon": "icon.ico",
       "target": [
-        "nsis"
+        "dir"
       ]
     },
-    "nsis": {
-      "createDesktopShortcut": "always",
-      "oneClick": false,
-      "perMachine": false
-    },
     "linux": {
       "synopsis": "vipergirls.to ripper",
       "category": "Utility",
@@ -70,15 +65,16 @@
     "dist": "node pre-build.js && electron-builder"
   },
   "devDependencies": {
-    "electron": "^7.1.9",
-    "electron-builder": "^22.2.0"
+    "electron": "11.1.1",
+    "electron-builder": "22.9.1"
   },
   "dependencies": {
-    "axios": "^0.19.0",
-    "cheerio": "^1.0.0-rc.3",
-    "copy-dir": "^1.1.0",
-    "electron-context-menu": "^0.13.0",
-    "get-port": "^5.0.0",
-    "rimraf": "^2.6.3"
+    "axios": "0.21.1",
+    "cheerio": "1.0.0-rc.3",
+    "copy-dir": "1.3.0",
+    "electron-context-menu": "2.1.0",
+    "get-port": "5.1.1",
+    "rimraf": "3.0.2",
+    "v8-compile-cache": "2.1.1"
   }
 }

vripper-electron/pom.xml

@@ -1,77 +1,76 @@
 <?xml version="1.0"?>
-<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"
-         xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
-  <modelVersion>4.0.0</modelVersion>
-  <parent>
-    <groupId>tn.mnlr</groupId>
-    <artifactId>vripper</artifactId>
-    <version>2.10.5</version>
-  </parent>
-  <artifactId>vripper-electron</artifactId>
-  <name>vripper-electron</name>
-  <build>
-    <plugins>
-      <plugin>
-        <artifactId>maven-jar-plugin</artifactId>
-        <configuration>
-          <classifier>${buildClassifier}</classifier>
-        </configuration>
-      </plugin>
-    </plugins>
-  </build>
-  <profiles>
-    <profile>
-      <id>electron</id>
-      <properties>
-        <buildClassifier>electron</buildClassifier>
-      </properties>
-      <dependencies>
-        <dependency>
-          <groupId>tn.mnlr</groupId>
-          <artifactId>vripper-server</artifactId>
-          <version>${project.version}</version>
-          <scope>runtime</scope>
-          <classifier>${buildClassifier}</classifier>
-        </dependency>
-      </dependencies>
-      <build>
+<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0">
+    <modelVersion>4.0.0</modelVersion>
+    <parent>
+        <groupId>tn.mnlr</groupId>
+        <artifactId>vripper</artifactId>
+        <version>3.3.5</version>
+    </parent>
+    <artifactId>vripper-electron</artifactId>
+    <name>vripper-electron</name>
+    <build>
         <plugins>
-          <plugin>
-            <groupId>com.github.eirslett</groupId>
-            <artifactId>frontend-maven-plugin</artifactId>
-            <version>1.3</version>
-            <configuration>
-              <nodeVersion>v12.2.0</nodeVersion>
-              <npmVersion>6.9.0</npmVersion>
-              <workingDirectory>build-dir</workingDirectory>
-            </configuration>
-            <executions>
-              <execution>
-                <id>install node and npm</id>
-                <goals>
-                  <goal>install-node-and-npm</goal>
-                </goals>
-              </execution>
-              <execution>
-                <id>npm install</id>
-                <goals>
-                  <goal>npm</goal>
-                </goals>
-              </execution>
-              <execution>
-                <id>npm run build</id>
-                <goals>
-                  <goal>npm</goal>
-                </goals>
+            <plugin>
+                <artifactId>maven-jar-plugin</artifactId>
                 <configuration>
-                  <arguments>run dist</arguments>
+                    <classifier>${buildClassifier}</classifier>
                 </configuration>
-                <phase>generate-resources</phase>
-              </execution>
-            </executions>
-          </plugin>
+            </plugin>
         </plugins>
-      </build>
-    </profile>
-  </profiles>
+    </build>
+    <profiles>
+        <profile>
+            <id>electron</id>
+            <properties>
+                <buildClassifier>electron</buildClassifier>
+            </properties>
+            <dependencies>
+                <dependency>
+                    <groupId>tn.mnlr</groupId>
+                    <artifactId>vripper-server</artifactId>
+                    <version>${project.version}</version>
+                    <scope>runtime</scope>
+                    <classifier>${buildClassifier}</classifier>
+                </dependency>
+            </dependencies>
+            <build>
+                <plugins>
+                    <plugin>
+                        <groupId>com.github.eirslett</groupId>
+                        <artifactId>frontend-maven-plugin</artifactId>
+                        <version>1.11.2</version>
+                        <configuration>
+                            <nodeVersion>v14.16.0</nodeVersion>
+                            <yarnVersion>v1.22.10</yarnVersion>
+                            <workingDirectory>build-dir</workingDirectory>
+                        </configuration>
+                        <executions>
+                            <execution>
+                                <id>install node and yarn</id>
+                                <goals>
+                                    <goal>install-node-and-yarn</goal>
+                                </goals>
+                            </execution>
+                            <execution>
+                                <id>yarn install</id>
+                                <goals>
+                                    <goal>yarn</goal>
+                                </goals>
+                            </execution>
+                            <execution>
+                                <id>yarn run build</id>
+                                <goals>
+                                    <goal>yarn</goal>
+                                </goals>
+                                <configuration>
+                                    <arguments>run dist</arguments>
+                                </configuration>
+                            </execution>
+                        </executions>
+                    </plugin>
+                </plugins>
+            </build>
+        </profile>
+    </profiles>
 </project>

vripper-electron/pre-build.js

@@ -2,7 +2,7 @@ const cheerio = require('cheerio');
 const fs = require('fs');
 const rimraf = require("rimraf");
 const copydir = require("copy-dir");
-const { execSync } = require("child_process");
+const {execSync} = require("child_process");
 
 rimraf.sync('./build/vripper-ui');
 copydir.sync('../vripper-ui/dist/vripper-ui', './build/vripper-ui', {});
@@ -19,7 +19,7 @@ fs.writeFileSync('./build/vripper-ui/index.html', $.html());
 console.log('Building runtime environment');
 rimraf.sync('java-runtime');
 execSync('jlink --no-header-files --no-man-pages --compress=2 --strip-debug --add-modules java.base,java.desktop,java.instrument,java.management,java.naming,java.prefs,java.rmi,java.scripting,java.security.jgss,java.sql,jdk.httpserver,jdk.unsupported,jdk.crypto.ec --output java-runtime');
-if(process.platform === 'linux') {
+if (process.platform === 'linux') {
     console.log('Stripping libjvm.so');
     execSync('strip -p --strip-unneeded java-runtime/lib/server/libjvm.so');
-}
+}

vripper-electron/vripper-electron.iml

@@ -0,0 +1,12 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<module org.jetbrains.idea.maven.project.MavenProjectsManager.isMavenModule="true" type="JAVA_MODULE" version="4">
+  <component name="NewModuleRootManager" LANGUAGE_LEVEL="JDK_1_8">
+    <output url="file://$MODULE_DIR$/target/classes" />
+    <output-test url="file://$MODULE_DIR$/target/test-classes" />
+    <content url="file://$MODULE_DIR$">
+      <excludeFolder url="file://$MODULE_DIR$/target" />
+    </content>
+    <orderEntry type="inheritedJdk" />
+    <orderEntry type="sourceFolder" forTests="false" />
+  </component>
+</module>

vripper-server/pom.xml

@@ -5,7 +5,7 @@
     <parent>
         <groupId>tn.mnlr</groupId>
         <artifactId>vripper</artifactId>
-        <version>2.10.5</version>
+        <version>3.3.5</version>
     </parent>
     <artifactId>vripper-server</artifactId>
     <name>vripper-server</name>
@@ -32,6 +32,20 @@
             <groupId>org.springframework.boot</groupId>
             <artifactId>spring-boot-starter-actuator</artifactId>
         </dependency>
+        <dependency>
+            <groupId>org.springframework.boot</groupId>
+            <artifactId>spring-boot-starter-data-jdbc</artifactId>
+        </dependency>
+        <dependency>
+            <groupId>org.liquibase</groupId>
+            <artifactId>liquibase-core</artifactId>
+        </dependency>
+        <dependency>
+            <groupId>org.hsqldb</groupId>
+            <artifactId>hsqldb</artifactId>
+            <version>2.5.1</version>
+            <scope>runtime</scope>
+        </dependency>
         <dependency>
             <groupId>org.projectlombok</groupId>
             <artifactId>lombok</artifactId>
@@ -58,9 +72,8 @@
             <version>2.22</version>
         </dependency>
         <dependency>
-            <groupId>io.reactivex.rxjava2</groupId>
-            <artifactId>rxjava</artifactId>
-            <version>2.2.6</version>
+            <groupId>io.projectreactor</groupId>
+            <artifactId>reactor-core</artifactId>
         </dependency>
         <dependency>
             <groupId>net.jodah</groupId>
@@ -72,11 +85,6 @@
             <artifactId>guava</artifactId>
             <version>28.1-jre</version>
         </dependency>
-        <dependency>
-            <groupId>org.imgscalr</groupId>
-            <artifactId>imgscalr-lib</artifactId>
-            <version>4.2</version>
-        </dependency>
         <dependency>
             <groupId>tn.mnlr</groupId>
             <artifactId>vripper-ui</artifactId>

vripper-server/src/main/java/tn/mnlr/vripper/EventListenerBean.java

@@ -5,20 +5,29 @@ import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.context.event.ContextRefreshedEvent;
 import org.springframework.context.event.EventListener;
 import org.springframework.stereotype.Component;
-import tn.mnlr.vripper.services.PersistenceService;
+import tn.mnlr.vripper.jpa.repositories.IRepository;
+import tn.mnlr.vripper.services.DataService;
+
+import java.util.Set;
 
 @Component
 public class EventListenerBean {
 
-    @Autowired
-    private PersistenceService persistenceService;
-
     @Getter
     private static boolean init = false;
 
+    private final DataService dataService;
+    private final Set<IRepository> repositorySet;
+
+    @Autowired
+    public EventListenerBean(DataService dataService, Set<IRepository> repositorySet) {
+        this.dataService = dataService;
+        this.repositorySet = repositorySet;
+    }
+
     @EventListener
     public void onApplicationEvent(ContextRefreshedEvent event) {
+        dataService.setDownloadingToStopped();
         init = true;
-        persistenceService.restore();
     }
-}
+}

vripper-server/src/main/java/tn/mnlr/vripper/SpringContext.java

@@ -1,7 +1,6 @@
 package tn.mnlr.vripper;
 
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
+import lombok.extern.slf4j.Slf4j;
 import org.springframework.beans.BeansException;
 import org.springframework.context.ApplicationContext;
 import org.springframework.context.ApplicationContextAware;
@@ -11,10 +10,9 @@ import org.springframework.stereotype.Component;
 import java.util.Map;
 
 @Component
+@Slf4j
 public class SpringContext implements ApplicationContextAware {
 
-    private static final Logger logger = LoggerFactory.getLogger(SpringContext.class);
-
     private static ConfigurableApplicationContext context;
 
     public static <T> T getBean(Class<T> beanClass) {
@@ -26,7 +24,7 @@ public class SpringContext implements ApplicationContextAware {
     }
 
     public static void close() {
-        logger.info("Application terminating...");
+        log.info("Application terminating...");
         if (context != null) {
             context.close();
         }

vripper-server/src/main/java/tn/mnlr/vripper/Utils.java

@@ -0,0 +1,23 @@
+package tn.mnlr.vripper;
+
+import lombok.extern.slf4j.Slf4j;
+
+import java.io.IOException;
+import java.io.PrintWriter;
+import java.io.StringWriter;
+
+@Slf4j
+public class Utils {
+
+    public static String throwableToString(Throwable th) {
+        try (StringWriter stringWriter = new StringWriter(); PrintWriter printWriter = new PrintWriter(stringWriter)) {
+            th.printStackTrace(printWriter);
+            printWriter.flush();
+            stringWriter.flush();
+            return stringWriter.toString();
+        } catch (IOException e) {
+            log.error("Failed to convert throwable to string", e);
+        }
+        return "Cannot display error, please check the log file for details";
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/VripperApplication.java

@@ -1,25 +1,19 @@
 package tn.mnlr.vripper;
 
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
+import lombok.extern.slf4j.Slf4j;
 import org.springframework.boot.SpringApplication;
 import org.springframework.boot.autoconfigure.SpringBootApplication;
 
-import java.util.concurrent.ExecutorService;
-import java.util.concurrent.Executors;
-
 @SpringBootApplication
+@Slf4j
 public class VripperApplication {
 
-    private static final Logger logger = LoggerFactory.getLogger(VripperApplication.class);
-    public static final ExecutorService commonExecutor = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());
-
     public static void main(String[] args) {
         try {
             Runtime.getRuntime().addShutdownHook(new Thread(SpringContext::close));
             SpringApplication.run(VripperApplication.class, args);
         } catch (Exception e) {
-            logger.error("Failed to run the application", e);
+            log.error("Failed to run the application", e);
         }
     }
 }

vripper-server/src/main/java/tn/mnlr/vripper/download/DownloadJob.java

@@ -0,0 +1,277 @@
+package tn.mnlr.vripper.download;
+
+import lombok.Getter;
+import lombok.extern.slf4j.Slf4j;
+import net.jodah.failsafe.function.CheckedRunnable;
+import org.apache.http.client.HttpClient;
+import org.apache.http.client.methods.AbstractExecutionAwareRequest;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpGet;
+import org.apache.http.client.protocol.HttpClientContext;
+import org.apache.http.impl.client.BasicCookieStore;
+import org.apache.http.util.EntityUtils;
+import tn.mnlr.vripper.SpringContext;
+import tn.mnlr.vripper.exception.DownloadException;
+import tn.mnlr.vripper.exception.HostException;
+import tn.mnlr.vripper.jpa.domain.Image;
+import tn.mnlr.vripper.jpa.domain.Post;
+import tn.mnlr.vripper.jpa.domain.enums.Status;
+import tn.mnlr.vripper.services.*;
+
+import javax.imageio.ImageIO;
+import javax.imageio.ImageReader;
+import javax.imageio.stream.ImageInputStream;
+import java.io.File;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.file.Files;
+import java.nio.file.StandardCopyOption;
+import java.util.*;
+
+@Slf4j
+public class DownloadJob implements CheckedRunnable {
+
+    private static final Object LOCK = new Object();
+    private static final int READ_BUFFER_SIZE = 8192;
+    private final DataService dataService;
+    private final PathService pathService;
+    private final ConnectionService cm;
+    private final VGAuthService authService;
+    private final DownloadSpeedService downloadSpeedService;
+    private final SettingsService settingsService;
+    private final HttpClientContext context;
+    @Getter
+    private final Image image;
+    @Getter
+    private final Post post;
+    private volatile boolean stopped = false;
+    @Getter
+    private boolean finished = false;
+
+    DownloadJob(Post post, Image image) {
+        this.image = image;
+        this.post = post;
+        dataService = SpringContext.getBean(DataService.class);
+        pathService = SpringContext.getBean(PathService.class);
+        cm = SpringContext.getBean(ConnectionService.class);
+        authService = SpringContext.getBean(VGAuthService.class);
+        downloadSpeedService = SpringContext.getBean(DownloadSpeedService.class);
+        settingsService = SpringContext.getBean(SettingsService.class);
+        context = HttpClientContext.create();
+        context.setCookieStore(new BasicCookieStore());
+        context.setAttribute(ContextAttributes.OPEN_CONNECTION.toString(), Collections.synchronizedList(new ArrayList<AbstractExecutionAwareRequest>()));
+    }
+
+    public void download(final Post post, final Image image) throws DownloadException {
+
+        try {
+
+            image.setStatus(Status.DOWNLOADING);
+            image.setCurrent(0);
+            dataService.updateImageStatus(image.getStatus(), image.getId());
+            dataService.updateImageCurrent(image.getCurrent(), image.getId());
+
+            synchronized (LOCK) {
+                if (!post.getStatus().equals(Status.DOWNLOADING) && !post.getStatus().equals(Status.PARTIAL)) {
+                    post.setStatus(Status.DOWNLOADING);
+                    dataService.updatePostStatus(post.getStatus(), post.getId());
+                }
+
+                // The post may be updated and the download directory might be set by another thread
+                Post updatedPost = dataService.findById(post.getId()).orElseThrow();
+                if (updatedPost.getDownloadDirectory() == null) {
+                    pathService.createDefaultPostFolder(updatedPost);
+                }
+                if (settingsService.getSettings().getLeaveThanksOnStart() != null && settingsService.getSettings().getLeaveThanksOnStart()) {
+                    authService.leaveThanks(updatedPost);
+                }
+            }
+
+            if (stopped) {
+                return;
+            }
+            /*
+             * HOST SPECIFIC
+             */
+            log.debug(String.format("Getting image url and name from %s using %s", image.getUrl(), image.getHost()));
+            HostService.NameUrl nameAndUrl = image.getHost().getNameAndUrl(image.getUrl(), context);
+            log.debug(String.format("Resolved name for %s: %s", image.getUrl(), nameAndUrl.getName()));
+            log.debug(String.format("Resolved image url for %s: %s", image.getUrl(), nameAndUrl.getUrl()));
+            /*
+             * END HOST SPECIFIC
+             */
+
+            if (stopped) {
+                return;
+            }
+            String formatImageFileName = pathService.formatImageFileName(nameAndUrl.getName());
+            log.debug(String.format("Sanitizing image name from %s to %s", nameAndUrl.getName(), formatImageFileName));
+            nameAndUrl = new HostService.NameUrl(formatImageFileName, nameAndUrl.getUrl());
+
+            HttpClient client = cm.getClient().build();
+
+            log.debug(String.format("Downloading %s", nameAndUrl.getUrl()));
+            HttpGet httpGet = cm.buildHttpGet(nameAndUrl.getUrl(), context);
+            httpGet.addHeader("Referer", image.getUrl());
+            try (CloseableHttpResponse response = (CloseableHttpResponse) client.execute(httpGet, context)) {
+
+                if (response.getStatusLine().getStatusCode() / 100 != 2) {
+                    EntityUtils.consumeQuietly(response.getEntity());
+                    throw new DownloadException(String.format("Server returned code %d", response.getStatusLine().getStatusCode()));
+                }
+                if (stopped) {
+                    return;
+                }
+                File outputFile = Files.createTempFile("vripper", "tmp").toFile();
+                try (InputStream downloadStream = response.getEntity().getContent(); FileOutputStream fos = new FileOutputStream(outputFile)) {
+
+                    if (stopped) {
+                        return;
+                    }
+
+                    image.setTotal(response.getEntity().getContentLength());
+                    dataService.updateImageTotal(image.getTotal(), image.getId());
+
+                    log.debug(String.format("%s length is %d", nameAndUrl.getUrl(), image.getTotal()));
+                    log.debug(String.format("Starting data transfer for %s", nameAndUrl.getUrl()));
+
+                    byte[] buffer = new byte[READ_BUFFER_SIZE];
+                    int read;
+                    while ((read = downloadStream.read(buffer, 0, READ_BUFFER_SIZE)) != -1 && !stopped) {
+                        fos.write(buffer, 0, read);
+                        image.increase(read);
+                        downloadSpeedService.increase(read);
+                        dataService.updateImageCurrent(image.getCurrent(), image.getId());
+                    }
+                    fos.flush();
+                    EntityUtils.consumeQuietly(response.getEntity());
+                    if (stopped) {
+                        return;
+                    }
+                }
+                checkImageTypeAndRename(dataService.findById(post.getId()).orElseThrow(), outputFile, nameAndUrl.getName(), image.getIndex());
+            }
+        } catch (Exception e) {
+            if (stopped) {
+                return;
+            }
+            throw new DownloadException(e);
+        } finally {
+            if (image.getCurrent() == image.getTotal() && image.getTotal() > 0) {
+                image.setStatus(Status.COMPLETE);
+            } else if (stopped) {
+                image.setStatus(Status.STOPPED);
+            } else {
+                image.setStatus(Status.ERROR);
+            }
+            dataService.updateImageStatus(image.getStatus(), image.getId());
+            finished = true;
+        }
+    }
+
+    private void checkImageTypeAndRename(Post post, File outputFile, String imageName, int index) throws HostException {
+        try (ImageInputStream iis = ImageIO.createImageInputStream(outputFile)) {
+            Iterator<ImageReader> it = ImageIO.getImageReaders(iis);
+            if (!it.hasNext()) {
+                throw new HostException("Image file is not recognized!");
+            }
+            ImageReader reader = it.next();
+            if (reader.getFormatName().equalsIgnoreCase("JPEG")) {
+                String imageNameLC = imageName.toLowerCase();
+                if (!imageNameLC.endsWith("_jpg") && !imageNameLC.endsWith("_jpeg")) {
+                    imageName += ".jpg";
+                } else {
+                    String toReplace = null;
+                    if (imageNameLC.endsWith("_jpg")) {
+                        toReplace = "_jpg";
+                    } else if (imageNameLC.endsWith("_jpeg")) {
+                        toReplace = "_jpeg";
+                    }
+                    if (toReplace != null) {
+                        imageName = imageName.substring(0, imageName.length() - toReplace.length()) + ".jpg";
+                    }
+                }
+            } else if (reader.getFormatName().equalsIgnoreCase("PNG")) {
+                String imageNameLC = imageName.toLowerCase();
+                if (!imageNameLC.endsWith("_png")) {
+                    imageName += ".png";
+                } else {
+                    String toReplace = null;
+                    if (imageNameLC.endsWith("_png")) {
+                        toReplace = "_png";
+                    }
+                    if (toReplace != null) {
+                        imageName = imageName.substring(0, imageName.length() - toReplace.length()) + ".png";
+                    }
+                }
+            }
+        } catch (Exception e) {
+            throw new HostException("Failed to guess image format", e);
+        }
+        try {
+            pathService.getDirectoryAccess().lock();
+            File downloadDestinationFolder = pathService.calcDownloadDirectory(post);
+            File outImage = new File(downloadDestinationFolder, (settingsService.getSettings().getForceOrder() ? String.format("%03d_", index) : "") + imageName);
+            Files.copy(outputFile.toPath(), outImage.toPath(), StandardCopyOption.REPLACE_EXISTING);
+        } catch (Exception e) {
+            throw new HostException("Failed to rename the image", e);
+        } finally {
+            pathService.getDirectoryAccess().unlock();
+            try {
+                Files.delete(outputFile.toPath());
+            } catch (IOException e) {
+                log.warn(String.format("Failed to delete temporary file %s", outputFile.getAbsolutePath()));
+            }
+        }
+    }
+
+    @Override
+    public void run() throws Exception {
+        if (stopped) {
+            finished = true;
+            return;
+        }
+        log.debug(String.format("Starting downloading %s", image.getUrl()));
+        download(post, image);
+    }
+
+    @Override
+    public boolean equals(Object o) {
+        if (this == o) return true;
+        if (o == null || getClass() != o.getClass()) return false;
+        DownloadJob that = (DownloadJob) o;
+        return Objects.equals(image, that.image) &&
+                Objects.equals(post, that.post);
+    }
+
+    @Override
+    public int hashCode() {
+        return Objects.hash(image, post);
+    }
+
+    public void stop() {
+        List<AbstractExecutionAwareRequest> requests = (List<AbstractExecutionAwareRequest>) this.context.getAttribute(ContextAttributes.OPEN_CONNECTION.toString());
+        if (requests != null) {
+            for (AbstractExecutionAwareRequest request : requests) {
+                request.abort();
+            }
+        }
+        this.stopped = true;
+    }
+
+    public enum ContextAttributes {
+        OPEN_CONNECTION("OPEN_CONNECTION");
+
+        private final String value;
+
+        ContextAttributes(String value) {
+            this.value = value;
+        }
+
+        @Override
+        public String toString() {
+            return value;
+        }
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/download/DownloadRunnable.java

@@ -0,0 +1,54 @@
+package tn.mnlr.vripper.download;
+
+import lombok.extern.slf4j.Slf4j;
+import net.jodah.failsafe.Failsafe;
+import tn.mnlr.vripper.SpringContext;
+import tn.mnlr.vripper.Utils;
+import tn.mnlr.vripper.jpa.domain.Event;
+import tn.mnlr.vripper.jpa.domain.enums.Status;
+import tn.mnlr.vripper.jpa.repositories.IEventRepository;
+import tn.mnlr.vripper.services.ConnectionService;
+import tn.mnlr.vripper.services.DataService;
+
+import java.time.LocalDateTime;
+
+@Slf4j
+public class DownloadRunnable implements Runnable {
+
+    private final DownloadService downloadService;
+    private final DataService dataService;
+    private final ConnectionService connectionService;
+    private final IEventRepository eventRepository;
+
+    private final DownloadJob downloadJob;
+
+    public DownloadRunnable(final DownloadJob downloadJob) {
+        downloadService = SpringContext.getBean(DownloadService.class);
+        dataService = SpringContext.getBean(DataService.class);
+        connectionService = SpringContext.getBean(ConnectionService.class);
+        eventRepository = SpringContext.getBean(IEventRepository.class);
+
+        this.downloadJob = downloadJob;
+    }
+
+    @Override
+    public void run() {
+        Failsafe.with(connectionService.getRetryPolicy())
+                .onFailure(e -> {
+                    try {
+                        Event event = new Event(Event.Type.DOWNLOAD, Event.Status.ERROR, LocalDateTime.now(), String.format("Failed to download %s\n %s", downloadJob.getImage().getUrl(), Utils.throwableToString(e.getFailure())));
+                        eventRepository.save(event);
+                    } catch (Exception exp) {
+                        log.error("Failed to save event", exp);
+                    }
+                    log.error(String.format("Failed to download %s after %d tries", downloadJob.getImage().getUrl(), e.getAttemptCount()), e.getFailure());
+                    downloadJob.getImage().setStatus(Status.ERROR);
+                    dataService.updateImageStatus(downloadJob.getImage().getStatus(), downloadJob.getImage().getId());
+                })
+                .onComplete(e -> {
+                    dataService.afterJobFinish(downloadJob.getImage(), downloadJob.getPost());
+                    downloadService.afterJobFinish(downloadJob);
+                    log.debug(String.format("Finished downloading %s", downloadJob.getImage().getUrl()));
+                }).run(downloadJob);
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/download/DownloadService.java

@@ -0,0 +1,223 @@
+package tn.mnlr.vripper.download;
+
+import lombok.NonNull;
+import lombok.extern.slf4j.Slf4j;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Service;
+import tn.mnlr.vripper.host.Host;
+import tn.mnlr.vripper.jpa.domain.Image;
+import tn.mnlr.vripper.jpa.domain.Post;
+import tn.mnlr.vripper.jpa.domain.enums.Status;
+import tn.mnlr.vripper.services.DataService;
+import tn.mnlr.vripper.services.PostService;
+import tn.mnlr.vripper.services.SettingsService;
+
+import javax.annotation.PostConstruct;
+import java.util.*;
+import java.util.concurrent.*;
+import java.util.concurrent.atomic.AtomicInteger;
+
+@Service
+@Slf4j
+public class DownloadService {
+
+    private static final List<Status> FINISHED = Arrays.asList(Status.ERROR, Status.COMPLETE, Status.STOPPED);
+    private final int MAX_POOL_SIZE = 12;
+
+    private final ConcurrentHashMap<Host, AtomicInteger> threadCount = new ConcurrentHashMap<>();
+    private final ExecutorService executor = Executors.newFixedThreadPool(MAX_POOL_SIZE);
+    private final BlockingQueue<DownloadJob> executionQueue = new LinkedBlockingQueue<>();
+    private final List<DownloadJob> executing = Collections.synchronizedList(new ArrayList<>());
+
+    private final PendingQueue pendingQueue;
+    private final SettingsService settings;
+    private final DataService dataService;
+    private final PostService postService;
+
+    private boolean pauseQ = false;
+    private Thread executionThread;
+    private Thread pollThread;
+
+    @Autowired
+    public DownloadService(PendingQueue pendingQueue, SettingsService settings, DataService dataService, PostService postService) {
+        this.pendingQueue = pendingQueue;
+        this.settings = settings;
+        this.dataService = dataService;
+        this.postService = postService;
+    }
+
+    @PostConstruct
+    private void init() {
+        executionThread = new Thread(this::start, "Executor thread");
+        pollThread = new Thread(this::poll, "Polling thread");
+        pollThread.start();
+        executionThread.start();
+    }
+
+    public void destroy() throws Exception {
+        log.info("Shutting down ExecutionService");
+        executionThread.interrupt();
+        pollThread.interrupt();
+        executor.shutdown();
+        dataService.findAllPosts().forEach(p -> {
+            log.debug(String.format("Stopping download jobs for %s", p));
+            this.stopRunning(p.getPostId());
+        });
+        executor.awaitTermination(5, TimeUnit.SECONDS);
+    }
+
+    private void stopRunning(@NonNull String postId) {
+        List<DownloadJob> stopping = new ArrayList<>();
+        Iterator<DownloadJob> iterator = executing.iterator();
+        while (iterator.hasNext()) {
+            DownloadJob downloadJob = iterator.next();
+            if (postId.equals(downloadJob.getPost().getPostId())) {
+                downloadJob.stop();
+                iterator.remove();
+                stopping.add(downloadJob);
+            }
+        }
+
+        while (!stopping.isEmpty()) {
+            stopping.removeIf(DownloadJob::isFinished);
+            try {
+                Thread.sleep(500);
+            } catch (InterruptedException e) {
+                Thread.currentThread().interrupt();
+            }
+        }
+    }
+
+    public void stopAll(List<String> posIds) {
+        if (posIds != null) {
+            posIds.forEach(this::stop);
+        } else {
+            dataService.findAllPosts().forEach(p -> this.stop(p.getPostId()));
+        }
+    }
+
+    public void restartAll(List<String> posIds) {
+        if (posIds != null) {
+            posIds.forEach(this::restart);
+        } else {
+            dataService.findAllPosts().forEach(p -> this.restart(p.getPostId()));
+        }
+    }
+
+    private void restart(@NonNull String postId) {
+        if (isPending(postId)) {
+            log.warn(String.format("Cannot restart, jobs are currently running for post id %s", postId));
+            return;
+        }
+        List<Image> images = dataService.findByPostIdAndIsNotCompleted(postId);
+        if (images.isEmpty()) {
+            return;
+        }
+        Post post = dataService.findPostByPostId(postId).orElseThrow();
+        post.setStatus(Status.PENDING);
+        dataService.updatePostStatus(post.getStatus(), post.getId());
+        log.debug(String.format("Restarting %d jobs for post id %s", images.size(), postId));
+        for (Image image : images) {
+            try {
+                pendingQueue.put(post, image);
+            } catch (InterruptedException e) {
+                log.warn("Thread was interrupted", e);
+                Thread.currentThread().interrupt();
+            }
+        }
+    }
+
+    private boolean isPending(String postId) {
+        return pendingQueue.isPending(postId);
+    }
+
+    private void stop(String postId) {
+        try {
+            pauseQ = true;
+            final Post post = dataService.findPostByPostId(postId).orElseThrow();
+            if (post == null) {
+                return;
+            }
+            if (FINISHED.contains(post.getStatus())) {
+                return;
+            }
+            pendingQueue.stop(post);
+            stopRunning(postId);
+            dataService.stopImagesByPostIdAndIsNotCompleted(postId);
+            dataService.finishPost(post);
+            postService.stopFetchingMetadata(post);
+        } finally {
+            pauseQ = false;
+        }
+    }
+
+    private boolean canRun(Host host) {
+        boolean canRun;
+        AtomicInteger count = threadCount.get(host);
+        if (count == null) {
+            threadCount.put(host, new AtomicInteger(0));
+        }
+        canRun = threadCount.get(host).get() < settings.getSettings().getMaxThreads() && (settings.getSettings().getMaxTotalThreads() == 0 ? threadCount.values().stream().mapToInt(AtomicInteger::get).sum() < MAX_POOL_SIZE : threadCount.values().stream().mapToInt(AtomicInteger::get).sum() < settings.getSettings().getMaxTotalThreads());
+        if (canRun && !pauseQ) {
+            threadCount.get(host).incrementAndGet();
+            return true;
+        }
+        return false;
+    }
+
+    private void poll() {
+        while (!Thread.interrupted()) {
+            try {
+                List<DownloadJob> peek = pendingQueue.peek();
+                for (DownloadJob downloadJob : peek) {
+                    if (canRun(downloadJob.getImage().getHost())) {
+                        executionQueue.offer(downloadJob);
+                        pendingQueue.remove(downloadJob);
+                    }
+                }
+                synchronized (threadCount) {
+                    threadCount.wait(2_000);
+                }
+            } catch (InterruptedException e) {
+                Thread.currentThread().interrupt();
+                break;
+            }
+        }
+    }
+
+    private void start() {
+        while (!Thread.interrupted()) {
+            try {
+                push(executionQueue.take());
+            } catch (InterruptedException e) {
+                Thread.currentThread().interrupt();
+                break;
+            } catch (Exception e) {
+                log.error("Execution Service failed", e);
+                break;
+            }
+        }
+    }
+
+    private void push(DownloadJob downloadJob) {
+        log.debug(String.format("Scheduling a job for %s", downloadJob.getImage().getUrl()));
+        executor.execute(new DownloadRunnable(downloadJob));
+        executing.add(downloadJob);
+    }
+
+    public synchronized void afterJobFinish(DownloadJob downloadJob) {
+        int count = pendingQueue.decrement(downloadJob.getPost().getPostId());
+        if (count == 0) {
+            dataService.finishPost(downloadJob.getPost());
+        }
+        threadCount.get(downloadJob.getImage().getHost()).decrementAndGet();
+        executing.remove(downloadJob);
+        synchronized (threadCount) {
+            threadCount.notify();
+        }
+    }
+
+    public int runningCount() {
+        return executing.size();
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/download/PendingQueue.java

@@ -0,0 +1,115 @@
+package tn.mnlr.vripper.download;
+
+import lombok.extern.slf4j.Slf4j;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Service;
+import tn.mnlr.vripper.host.Host;
+import tn.mnlr.vripper.jpa.domain.Image;
+import tn.mnlr.vripper.jpa.domain.Post;
+import tn.mnlr.vripper.services.DataService;
+import tn.mnlr.vripper.services.SettingsService;
+
+import javax.annotation.PostConstruct;
+import java.util.*;
+import java.util.concurrent.BlockingDeque;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.LinkedBlockingDeque;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.function.Predicate;
+
+@Service
+@Slf4j
+public class PendingQueue {
+
+    private final DataService dataService;
+    private final SettingsService settingsService;
+    private final List<Host> hosts;
+
+    private final ConcurrentHashMap<Host, BlockingDeque<DownloadJob>> pendingQ = new ConcurrentHashMap<>();
+    private final ConcurrentHashMap<String, AtomicInteger> toBeExecuted = new ConcurrentHashMap<>();
+
+    @Autowired
+    public PendingQueue(DataService dataService, SettingsService settingsService, List<Host> hosts) {
+        this.dataService = dataService;
+        this.settingsService = settingsService;
+        this.hosts = hosts;
+    }
+
+    @PostConstruct
+    private void init() {
+        hosts.forEach(host -> pendingQ.put(host, new LinkedBlockingDeque<>()));
+    }
+
+    public void put(Post post, Image image) throws InterruptedException {
+        log.debug(String.format("Enqueuing a job for %s", image.getUrl()));
+        image.init();
+        dataService.updateImageStatus(image.getStatus(), image.getId());
+        dataService.updateImageCurrent(image.getCurrent(), image.getId());
+        DownloadJob downloadJob = new DownloadJob(post, image);
+        pendingQ.get(downloadJob.getImage().getHost()).putLast(downloadJob);
+        checkKey(post.getPostId());
+        toBeExecuted.get(post.getPostId()).incrementAndGet();
+    }
+
+    private synchronized void checkKey(String postId) {
+        if (!toBeExecuted.containsKey(postId)) {
+            toBeExecuted.put(postId, new AtomicInteger(0));
+        }
+    }
+
+    public void remove(final DownloadJob downloadJob) {
+        pendingQ.get(downloadJob.getImage().getHost()).remove(downloadJob);
+    }
+
+    public List<DownloadJob> peek() {
+        List<DownloadJob> downloadJobs = new ArrayList<>();
+        if (hosts.size() == 0) {
+            return downloadJobs;
+        }
+        for (Host host : hosts) {
+            Iterator<DownloadJob> it = pendingQ.get(host).iterator();
+            for (int i = 0; i < settingsService.getSettings().getMaxThreads(); i++) {
+                DownloadJob downloadJob = it.hasNext() ? it.next() : null;
+                if (downloadJob != null) {
+                    downloadJobs.add(downloadJob);
+                }
+            }
+        }
+        return downloadJobs;
+    }
+
+    public void enqueue(Post post, Set<Image> images) throws InterruptedException {
+        for (Image image : images) {
+            put(post, image);
+        }
+    }
+
+    public int size() {
+        return toBeExecuted.values().stream().mapToInt(AtomicInteger::get).sum();
+    }
+
+    public void stop(Post post) {
+        Predicate<DownloadJob> predicate = next -> next.getImage().getPostId().equals(post.getPostId());
+        for (Map.Entry<Host, BlockingDeque<DownloadJob>> entry : pendingQ.entrySet()) {
+            entry.getValue().stream().filter(predicate).forEach(e -> decrement(post.getPostId()));
+            entry.getValue().removeIf(predicate);
+            decrement(post.getPostId());
+        }
+    }
+
+    public boolean isPending(String postId) {
+        return toBeExecuted.containsKey(postId);
+    }
+
+    public synchronized int decrement(String postId) {
+        AtomicInteger counter = toBeExecuted.get(postId);
+        if (counter == null) {
+            return 0;
+        }
+        int count = counter.decrementAndGet();
+        if (count == 0) {
+            toBeExecuted.remove(postId);
+        }
+        return count;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/entities/Image.java

@@ -1,134 +0,0 @@
-package tn.mnlr.vripper.entities;
-
-import io.reactivex.disposables.Disposable;
-import io.reactivex.processors.BehaviorProcessor;
-import lombok.Getter;
-import lombok.NoArgsConstructor;
-import lombok.Setter;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import tn.mnlr.vripper.host.Host;
-import tn.mnlr.vripper.services.AppStateService;
-
-import java.util.Objects;
-import java.util.concurrent.atomic.AtomicLong;
-
-@Getter
-@NoArgsConstructor
-public class Image {
-
-    private static final Logger logger = LoggerFactory.getLogger(Image.class);
-
-    private AppStateService appStateService;
-
-    private Disposable subscription;
-
-    private final String type = "img";
-
-    private String postId;
-
-    private String postName;
-
-    private Host host;
-
-    private String url;
-
-    private int index;
-
-    private AtomicLong current = new AtomicLong(0);
-    private Status status;
-    private BehaviorProcessor<Image> imageStateProcessor;
-
-    @Setter
-    private long total = 0;
-
-    public Image(String url, String postId, String postName, Host host, int index) {
-        this.url = url;
-        this.postId = postId;
-        this.postName = postName;
-        this.host = host;
-        this.index = index;
-        status = Status.STOPPED;
-        imageStateProcessor = BehaviorProcessor.create();
-    }
-
-    public void setAppStateService(AppStateService appStateService) {
-        this.appStateService = appStateService;
-        appStateService.getCurrentImages().put(this.url, this);
-        appStateService.getLiveImageUpdates().onNext(this);
-    }
-
-    public void setStatus(Status status) {
-        this.status = status;
-        update();
-    }
-
-    public boolean isCompleted() {
-        return status.equals(Status.COMPLETE);
-    }
-
-    public void init() {
-        if (appStateService == null) {
-            logger.warn("Attempting to init the image whilst the App State is null, unexpected behaviour could occur from this");
-            return;
-        }
-        cleanup();
-        imageStateProcessor = BehaviorProcessor.create();
-        subscription = imageStateProcessor
-                .onBackpressureLatest()
-                .doOnNext(appStateService::onImageUpdate)
-                .subscribe();
-
-        current.set(0);
-        status = Status.STOPPED;
-        imageStateProcessor.onNext(this);
-    }
-
-    public void cleanup() {
-        if (imageStateProcessor != null) {
-            imageStateProcessor.onComplete();
-        }
-        if (subscription != null) {
-            subscription.dispose();
-        }
-    }
-
-    public void setCurrent(int current) {
-        this.current.set(current);
-        update();
-    }
-
-    public void increase(int read) {
-        current.addAndGet(read);
-        update();
-    }
-
-    private void update() {
-        if(imageStateProcessor == null) {
-            return;
-        }
-        imageStateProcessor.onNext(this);
-        if (isCompleted()) {
-            imageStateProcessor.onComplete();
-            subscription.dispose();
-            imageStateProcessor = null;
-        }
-    }
-
-    public enum Status {
-        PENDING, DOWNLOADING, COMPLETE, ERROR, STOPPED
-    }
-
-    @Override
-    public boolean equals(Object o) {
-        if (this == o) return true;
-        if (o == null || getClass() != o.getClass()) return false;
-        Image image = (Image) o;
-        return url.equals(image.url);
-    }
-
-    @Override
-    public int hashCode() {
-        return Objects.hash(url);
-    }
-}

vripper-server/src/main/java/tn/mnlr/vripper/entities/Post.java

@@ -1,108 +0,0 @@
-package tn.mnlr.vripper.entities;
-
-import lombok.Getter;
-import lombok.NoArgsConstructor;
-import lombok.ToString;
-import tn.mnlr.vripper.host.Host;
-import tn.mnlr.vripper.services.AppStateService;
-
-import java.util.List;
-import java.util.Map;
-import java.util.Objects;
-import java.util.Set;
-import java.util.concurrent.atomic.AtomicInteger;
-import java.util.stream.Collectors;
-
-@Getter
-@ToString
-@NoArgsConstructor
-public class Post {
-
-    private AppStateService appStateService;
-
-    private Status status;
-
-    private final String type = "post";
-
-    private String postId;
-
-    private String threadTitle;
-
-    private String threadId;
-
-    private String title;
-
-    private String url;
-
-    private List<Image> images;
-
-    private Map<String, Object> metadata;
-
-    private AtomicInteger done = new AtomicInteger(0);
-
-    private int total;
-
-    private Set<String> hosts;
-
-    private boolean removed = false;
-
-    private String forum;
-
-    public Post(String title, String url, List<Image> images, Map<String, Object> metadata, String postId, String threadId, String threadTitle, String forum) {
-        this.title = title;
-        this.url = url;
-        this.images = images;
-        this.metadata = metadata;
-        this.postId = postId;
-        this.forum = forum;
-        this.threadId = threadId;
-        this.threadTitle = threadTitle;
-        this.hosts = this.images.stream().map(Image::getHost).map(Host::getHost).collect(Collectors.toSet());
-        total = images.size();
-        status = Status.STOPPED;
-    }
-
-    public void setAppStateService(AppStateService appStateService) {
-        this.appStateService = appStateService;
-        this.appStateService.getCurrentPosts().put(postId, this);
-        this.appStateService.getLivePostsState().onNext(this);
-    }
-
-    public void setRemoved(boolean removed) {
-        this.removed = removed;
-        updateNotification();
-    }
-
-    public void increase() {
-        done.incrementAndGet();
-        updateNotification();
-    }
-
-    public void setStatus(Status status) {
-        this.status = status;
-        updateNotification();
-    }
-
-    private void updateNotification() {
-        if (appStateService != null) {
-            appStateService.getLivePostsState().onNext(this);
-        }
-    }
-
-    public enum Status {
-        PENDING, DOWNLOADING, COMPLETE, ERROR, PARTIAL, STOPPED
-    }
-
-    @Override
-    public boolean equals(Object o) {
-        if (this == o) return true;
-        if (o == null || getClass() != o.getClass()) return false;
-        Post post = (Post) o;
-        return postId.equals(post.postId);
-    }
-
-    @Override
-    public int hashCode() {
-        return Objects.hash(postId);
-    }
-}

vripper-server/src/main/java/tn/mnlr/vripper/entities/mixin/persistance/HostToString.java

@@ -1,12 +0,0 @@
-package tn.mnlr.vripper.entities.mixin.persistance;
-
-import com.fasterxml.jackson.databind.util.StdConverter;
-import tn.mnlr.vripper.host.Host;
-
-public class HostToString extends StdConverter<Host, String> {
-
-    @Override
-    public String convert(Host value) {
-        return value.getClass().getSimpleName();
-    }
-}

vripper-server/src/main/java/tn/mnlr/vripper/entities/mixin/persistance/ImagePersistanceMixin.java

@@ -1,36 +0,0 @@
-package tn.mnlr.vripper.entities.mixin.persistance;
-
-import com.fasterxml.jackson.annotation.JsonIgnore;
-import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
-import com.fasterxml.jackson.databind.annotation.JsonSerialize;
-import io.reactivex.disposables.Disposable;
-import io.reactivex.processors.BehaviorProcessor;
-import lombok.Getter;
-import lombok.Setter;
-import tn.mnlr.vripper.entities.Image;
-import tn.mnlr.vripper.host.Host;
-import tn.mnlr.vripper.services.AppStateService;
-
-@Getter
-@Setter
-public abstract class ImagePersistanceMixin {
-
-    @JsonSerialize(converter = HostToString.class)
-    @JsonDeserialize(converter = StringToHost.class)
-    private Host host;
-
-    @JsonIgnore
-    private BehaviorProcessor<Image> imageStateProcessor;
-
-    @JsonIgnore
-    private Disposable subscription;
-
-    @JsonIgnore
-    private String type;
-
-    @JsonIgnore
-    public abstract boolean isCompleted();
-
-    @JsonIgnore
-    private AppStateService appStateService;
-}

vripper-server/src/main/java/tn/mnlr/vripper/entities/mixin/persistance/PostPersistanceMixin.java

@@ -1,21 +0,0 @@
-package tn.mnlr.vripper.entities.mixin.persistance;
-
-import com.fasterxml.jackson.annotation.JsonIgnore;
-import lombok.Getter;
-import lombok.Setter;
-import tn.mnlr.vripper.services.AppStateService;
-
-@Getter
-@Setter
-public abstract class PostPersistanceMixin {
-
-    @JsonIgnore
-    private String type;
-
-    @JsonIgnore
-    private AppStateService appStateService;
-
-
-    @JsonIgnore
-    private boolean removed;
-}

vripper-server/src/main/java/tn/mnlr/vripper/entities/mixin/persistance/StringToHost.java

@@ -1,17 +0,0 @@
-package tn.mnlr.vripper.entities.mixin.persistance;
-
-import com.fasterxml.jackson.databind.util.StdConverter;
-import tn.mnlr.vripper.SpringContext;
-import tn.mnlr.vripper.host.Host;
-
-import java.util.Collection;
-
-public class StringToHost extends StdConverter<String, Host> {
-
-    private Collection<Host> hosts = SpringContext.getBeansOfType(Host.class).values();
-
-    @Override
-    public Host convert(String value) {
-        return hosts.stream().filter(e -> e.getClass().getSimpleName().equals(value)).findAny().orElse(null);
-    }
-}

vripper-server/src/main/java/tn/mnlr/vripper/entities/mixin/ui/ImageUIMixin.java

@@ -1,27 +0,0 @@
-package tn.mnlr.vripper.entities.mixin.ui;
-
-import com.fasterxml.jackson.annotation.JsonIgnore;
-import io.reactivex.disposables.Disposable;
-import io.reactivex.processors.BehaviorProcessor;
-import lombok.Getter;
-import lombok.Setter;
-import tn.mnlr.vripper.entities.Image;
-import tn.mnlr.vripper.host.Host;
-import tn.mnlr.vripper.services.AppStateService;
-
-@Getter
-@Setter
-public abstract class ImageUIMixin {
-
-    @JsonIgnore
-    private Host host;
-
-    @JsonIgnore
-    private BehaviorProcessor<Image> imageStateProcessor;
-
-    @JsonIgnore
-    private Disposable subscription;
-
-    @JsonIgnore
-    private AppStateService appStateService;
-}

vripper-server/src/main/java/tn/mnlr/vripper/entities/mixin/ui/PostUIMixin.java

@@ -1,20 +0,0 @@
-package tn.mnlr.vripper.entities.mixin.ui;
-
-import com.fasterxml.jackson.annotation.JsonIgnore;
-import lombok.Getter;
-import lombok.Setter;
-import tn.mnlr.vripper.entities.Image;
-import tn.mnlr.vripper.services.AppStateService;
-
-import java.util.List;
-
-@Getter
-@Setter
-public abstract class PostUIMixin {
-
-    @JsonIgnore
-    private List<Image> images;
-
-    @JsonIgnore
-    private AppStateService appStateService;
-}

vripper-server/src/main/java/tn/mnlr/vripper/event/EventRemoveEvent.java

@@ -0,0 +1,15 @@
+package tn.mnlr.vripper.event;
+
+import lombok.Getter;
+import org.springframework.context.ApplicationEvent;
+
+@Getter
+public class EventRemoveEvent extends ApplicationEvent {
+
+    private final Long id;
+
+    public EventRemoveEvent(Object source, Long id) {
+        super(source);
+        this.id = id;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/event/EventUpdateEvent.java

@@ -0,0 +1,15 @@
+package tn.mnlr.vripper.event;
+
+import lombok.Getter;
+import org.springframework.context.ApplicationEvent;
+
+@Getter
+public class EventUpdateEvent extends ApplicationEvent {
+
+    private final Long id;
+
+    public EventUpdateEvent(Object source, Long id) {
+        super(source);
+        this.id = id;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/event/ImageUpdateEvent.java

@@ -0,0 +1,15 @@
+package tn.mnlr.vripper.event;
+
+import lombok.Getter;
+import org.springframework.context.ApplicationEvent;
+
+@Getter
+public class ImageUpdateEvent extends ApplicationEvent {
+
+    private final Long id;
+
+    public ImageUpdateEvent(Object source, Long id) {
+        super(source);
+        this.id = id;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/event/MetadataUpdateEvent.java

@@ -0,0 +1,15 @@
+package tn.mnlr.vripper.event;
+
+import lombok.Getter;
+import org.springframework.context.ApplicationEvent;
+
+@Getter
+public class MetadataUpdateEvent extends ApplicationEvent {
+
+    private final Long postIdRef;
+
+    public MetadataUpdateEvent(Object source, Long postIdRef) {
+        super(source);
+        this.postIdRef = postIdRef;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/event/PostRemoveEvent.java

@@ -0,0 +1,15 @@
+package tn.mnlr.vripper.event;
+
+import lombok.Getter;
+import org.springframework.context.ApplicationEvent;
+
+@Getter
+public class PostRemoveEvent extends ApplicationEvent {
+
+    private final String postId;
+
+    public PostRemoveEvent(Object source, String postId) {
+        super(source);
+        this.postId = postId;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/event/PostUpdateEvent.java

@@ -0,0 +1,15 @@
+package tn.mnlr.vripper.event;
+
+import lombok.Getter;
+import org.springframework.context.ApplicationEvent;
+
+@Getter
+public class PostUpdateEvent extends ApplicationEvent {
+
+    private final Long id;
+
+    public PostUpdateEvent(Object source, Long id) {
+        super(source);
+        this.id = id;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/event/QueuedRemoveEvent.java

@@ -0,0 +1,15 @@
+package tn.mnlr.vripper.event;
+
+import lombok.Getter;
+import org.springframework.context.ApplicationEvent;
+
+@Getter
+public class QueuedRemoveEvent extends ApplicationEvent {
+
+    private final String threadId;
+
+    public QueuedRemoveEvent(Object source, String threadId) {
+        super(source);
+        this.threadId = threadId;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/event/QueuedUpdateEvent.java

@@ -0,0 +1,15 @@
+package tn.mnlr.vripper.event;
+
+import lombok.Getter;
+import org.springframework.context.ApplicationEvent;
+
+@Getter
+public class QueuedUpdateEvent extends ApplicationEvent {
+
+    private final Long id;
+
+    public QueuedUpdateEvent(Object source, Long id) {
+        super(source);
+        this.id = id;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/exception/HostException.java

@@ -10,6 +10,6 @@ public class HostException extends Exception {
     }
 
     public HostException(String message, Throwable e) {
-        super(message,e);
+        super(message, e);
     }
 }

vripper-server/src/main/java/tn/mnlr/vripper/exception/QueueException.java

@@ -0,0 +1,16 @@
+package tn.mnlr.vripper.exception;
+
+public class QueueException extends Exception {
+
+    public QueueException(String message) {
+        super(message);
+    }
+
+    public QueueException(String message, Throwable e) {
+        super(message, e);
+    }
+
+    public QueueException(Throwable e) {
+        super(e);
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/exception/RenameException.java

@@ -0,0 +1,10 @@
+package tn.mnlr.vripper.exception;
+
+import java.io.IOException;
+
+public class RenameException extends Exception {
+
+    public RenameException(String message, IOException e) {
+        super(message, e);
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/host/AcidimgHost.java

@@ -1,5 +1,6 @@
 package tn.mnlr.vripper.host;
 
+import lombok.extern.slf4j.Slf4j;
 import org.apache.http.NameValuePair;
 import org.apache.http.client.HttpClient;
 import org.apache.http.client.entity.UrlEncodedFormEntity;
@@ -8,31 +9,41 @@ import org.apache.http.client.methods.HttpPost;
 import org.apache.http.client.protocol.HttpClientContext;
 import org.apache.http.message.BasicNameValuePair;
 import org.apache.http.util.EntityUtils;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Service;
 import org.w3c.dom.Document;
 import org.w3c.dom.Node;
 import tn.mnlr.vripper.exception.HostException;
 import tn.mnlr.vripper.exception.XpathException;
-import tn.mnlr.vripper.q.ImageFileData;
-import tn.mnlr.vripper.services.ConnectionManager;
+import tn.mnlr.vripper.services.ConnectionService;
+import tn.mnlr.vripper.services.HostService;
+import tn.mnlr.vripper.services.HtmlProcessorService;
+import tn.mnlr.vripper.services.XpathService;
 
 import java.util.ArrayList;
 import java.util.List;
 
 @Service
+@Slf4j
 public class AcidimgHost extends Host {
 
-    private static final Logger logger = LoggerFactory.getLogger(AcidimgHost.class);
-
     private static final String host = "acidimg.cc";
     private static final String CONTINUE_BUTTON_XPATH = "//input[@id='continuebutton']";
     private static final String IMG_XPATH = "//img[@class='centred']";
 
+    private final ConnectionService cm;
+    private final HostService hostService;
+    private final XpathService xpathService;
+    private final HtmlProcessorService htmlProcessorService;
+
+
     @Autowired
-    private ConnectionManager cm;
+    public AcidimgHost(ConnectionService cm, HostService hostService, XpathService xpathService, HtmlProcessorService htmlProcessorService) {
+        this.cm = cm;
+        this.hostService = hostService;
+        this.xpathService = xpathService;
+        this.htmlProcessorService = htmlProcessorService;
+    }
 
     @Override
     public String getHost() {
@@ -45,22 +56,22 @@ public class AcidimgHost extends Host {
     }
 
     @Override
-    protected void setNameAndUrl(final String url, final ImageFileData imageFileData, final HttpClientContext context) throws HostException {
+    public HostService.NameUrl getNameAndUrl(final String url, final HttpClientContext context) throws HostException {
 
-        Document doc = getResponse(url, context).getDocument();
+        Document doc = hostService.getResponse(url, context).getDocument();
 
         Node contDiv;
         try {
-            logger.debug(String.format("Looking for xpath expression %s in %s", CONTINUE_BUTTON_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", CONTINUE_BUTTON_XPATH, url));
             contDiv = xpathService.getAsNode(doc, CONTINUE_BUTTON_XPATH);
         } catch (XpathException e) {
             throw new HostException(e);
         }
 
         if (contDiv != null) {
-            logger.debug(String.format("Click button found for %s", url));
+            log.debug(String.format("Click button found for %s", url));
             HttpClient client = cm.getClient().build();
-            HttpPost httpPost = cm.buildHttpPost(url);
+            HttpPost httpPost = cm.buildHttpPost(url, context);
             httpPost.addHeader("Referer", url);
             List<NameValuePair> params = new ArrayList<>();
             params.add(new BasicNameValuePair("imgContinue", "Continue to your image"));
@@ -70,9 +81,9 @@ public class AcidimgHost extends Host {
                 throw new HostException(e);
             }
 
-            logger.debug(String.format("Requesting %s", httpPost));
+            log.debug(String.format("Requesting %s", httpPost));
             try (CloseableHttpResponse response = (CloseableHttpResponse) client.execute(httpPost, context)) {
-                logger.debug(String.format("Cleaning response for %s", httpPost));
+                log.debug(String.format("Cleaning response for %s", httpPost));
                 doc = htmlProcessorService.clean(EntityUtils.toString(response.getEntity()));
                 EntityUtils.consumeQuietly(response.getEntity());
             } catch (Exception e) {
@@ -82,23 +93,22 @@ public class AcidimgHost extends Host {
 
         Node imgNode;
         try {
-            logger.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
             imgNode = xpathService.getAsNode(doc, IMG_XPATH);
         } catch (XpathException e) {
             throw new HostException(e);
         }
 
-        if(imgNode == null) {
+        if (imgNode == null) {
             throw new HostException("Cannot find the image node");
         }
 
         try {
-            logger.debug(String.format("Resolving name and image url for %s", url));
+            log.debug(String.format("Resolving name and image url for %s", url));
             String imgTitle = imgNode.getAttributes().getNamedItem("alt").getTextContent().trim();
             String imgUrl = imgNode.getAttributes().getNamedItem("src").getTextContent().trim();
 
-            imageFileData.setImageUrl(imgUrl);
-            imageFileData.setImageName(imgTitle.isEmpty() ? getDefaultImageName(imgUrl) : imgTitle);
+            return new HostService.NameUrl(imgTitle.isEmpty() ? hostService.getDefaultImageName(imgUrl) : imgTitle, imgUrl);
         } catch (Exception e) {
             throw new HostException("Unexpected error occurred", e);
         }

vripper-server/src/main/java/tn/mnlr/vripper/host/Host.java

@@ -1,243 +1,26 @@
 package tn.mnlr.vripper.host;
 
-import lombok.Getter;
-import org.apache.http.Header;
-import org.apache.http.client.HttpClient;
-import org.apache.http.client.methods.CloseableHttpResponse;
-import org.apache.http.client.methods.HttpGet;
+import lombok.extern.slf4j.Slf4j;
 import org.apache.http.client.protocol.HttpClientContext;
-import org.apache.http.impl.client.BasicCookieStore;
-import org.apache.http.util.EntityUtils;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Service;
-import org.w3c.dom.Document;
-import tn.mnlr.vripper.entities.Image;
-import tn.mnlr.vripper.exception.DownloadException;
 import tn.mnlr.vripper.exception.HostException;
-import tn.mnlr.vripper.exception.HtmlProcessorException;
-import tn.mnlr.vripper.q.ImageFileData;
-import tn.mnlr.vripper.services.*;
+import tn.mnlr.vripper.services.HostService;
 
-import javax.imageio.ImageIO;
-import javax.imageio.ImageReader;
-import javax.imageio.stream.ImageInputStream;
-import java.io.File;
-import java.io.FileOutputStream;
-import java.io.IOException;
-import java.io.InputStream;
-import java.net.URI;
-import java.net.URISyntaxException;
-import java.nio.file.Files;
-import java.nio.file.StandardCopyOption;
-import java.util.Iterator;
 import java.util.Objects;
 
 @Service
+@Slf4j
 abstract public class Host {
 
-    private static final Logger logger = LoggerFactory.getLogger(Host.class);
-
-    private static final int READ_BUFFER_SIZE = 8192;
-
-    @Autowired
-    protected HtmlProcessorService htmlProcessorService;
-
-    @Autowired
-    protected XpathService xpathService;
-
-    @Autowired
-    private AppSettingsService appSettingsService;
-
-    @Autowired
-    private AppStateService appStateService;
-
-    @Autowired
-    private ConnectionManager cm;
-
-    @Autowired
-    private DownloadSpeedService downloadSpeedService;
-
-    @Autowired
-    private PathService pathService;
-
     abstract public String getHost();
 
     abstract public String getLookup();
 
-    @Override
-    public int hashCode() {
-        return Objects.hash(pathService);
-    }
-
     public boolean isSupported(String url) {
         return url.contains(getLookup());
     }
 
-    public void download(Image image, ImageFileData imageFileData) throws DownloadException, InterruptedException {
-
-        HttpClientContext context = HttpClientContext.create();
-        context.setCookieStore(new BasicCookieStore());
-        try {
-
-            appStateService.postDownloadingUpdate(image.getPostId());
-
-            imageFileData.setPageUrl(image.getUrl());
-
-            /*
-             * HOST SPECIFIC
-             */
-            logger.debug(String.format("Getting image url and name from %s using %s", image.getUrl(), this.getHost()));
-            setNameAndUrl(image.getUrl(), imageFileData, context);
-            logger.debug(String.format("Resolved name for %s: %s", image.getUrl(), imageFileData.getImageName()));
-            logger.debug(String.format("Resolved image url for %s: %s", image.getUrl(), imageFileData.getImageUrl()));
-
-            logger.debug(String.format("Building image request for %s", image.getUrl()));
-            setImageRequest(imageFileData);
-            /*
-             * END HOST SPECIFIC
-             */
-
-            String formatImageFileName = pathService.formatImageFileName(imageFileData.getImageName());
-            logger.debug(String.format("Sanitizing image name from %s to %s", imageFileData.getImageName(), formatImageFileName));
-            imageFileData.setImageName(formatImageFileName);
-            File destinationFolder = pathService.getDownloadDestinationFolder(image.getPostId());
-            logger.debug(String.format("Saving to %s", destinationFolder.getPath()));
-            logger.debug(String.format("Creating %s", destinationFolder.getPath()));
-            destinationFolder.mkdirs();
-            if (!destinationFolder.exists()) {
-                logger.debug(String.format("Folder %s is created", destinationFolder.toString()));
-            }
-
-            HttpClient client = cm.getClient().build();
-
-            logger.debug(String.format("Downloading %s", imageFileData.getImageUrl()));
-            try (CloseableHttpResponse response = (CloseableHttpResponse) client.execute(imageFileData.getImageRequest(), context)) {
-
-                if (response.getStatusLine().getStatusCode() / 100 != 2) {
-                    EntityUtils.consumeQuietly(response.getEntity());
-                    throw new DownloadException(String.format("Server returned code %d", response.getStatusLine().getStatusCode()));
-                }
-
-                File outputFile = new File(destinationFolder.getPath() + File.separator + String.format("%03d_", image.getIndex()) + imageFileData.getImageName() + ".tmp");
-                try (InputStream downloadStream = response.getEntity().getContent(); FileOutputStream fos = new FileOutputStream(outputFile)) {
-                    image.setTotal(response.getEntity().getContentLength());
-                    logger.debug(String.format("%s length is %d", imageFileData.getImageUrl(), image.getTotal()));
-                    logger.debug(String.format("Starting data transfer for %s", imageFileData.getImageUrl()));
-
-                    byte[] buffer = new byte[READ_BUFFER_SIZE];
-                    int read;
-                    while ((read = downloadStream.read(buffer, 0, READ_BUFFER_SIZE)) != -1) {
-                        fos.write(buffer, 0, read);
-                        image.increase(read);
-                        downloadSpeedService.increase(read);
-                    }
-                    fos.flush();
-                    EntityUtils.consumeQuietly(response.getEntity());
-                }
-                File finalName = checkImageTypeAndRename(outputFile, imageFileData.getImageName(), image.getIndex());
-                imageFileData.setFileName(finalName.getName());
-            }
-        } catch (Exception e) {
-            if (Thread.interrupted()) {
-                throw new InterruptedException("Download was interrupted");
-            }
-            throw new DownloadException(e);
-        }
-    }
-
-    private File checkImageTypeAndRename(File outputFile, String imageName, int index) throws HostException {
-        try (ImageInputStream iis = ImageIO.createImageInputStream(outputFile)) {
-            Iterator<ImageReader> it = ImageIO.getImageReaders(iis);
-            if (!it.hasNext()) {
-                throw new HostException("Image file is not recognized!");
-            }
-            ImageReader reader = it.next();
-            if (reader.getFormatName().toUpperCase().equals("JPEG")) {
-                String imageNameLC = imageName.toLowerCase();
-                if (!imageNameLC.endsWith("_jpg") && !imageNameLC.endsWith("_jpeg")) {
-                    imageName += ".jpg";
-                } else {
-                    String toReplace = null;
-                    if (imageNameLC.endsWith("_jpg")) {
-                        toReplace = "_jpg";
-                    } else if (imageNameLC.endsWith("_jpeg")) {
-                        toReplace = "_jpeg";
-                    }
-                    if (toReplace != null) {
-                        imageName = imageName.substring(0, imageName.length() - toReplace.length()) + ".jpg";
-                    }
-                }
-            }
-        } catch (Exception e) {
-            throw new HostException("Failed to guess image format", e);
-        }
-        try {
-            File outImage = new File(outputFile.getParent(), (appSettingsService.isForceOrder() ? String.format("%03d_", index) : "") + imageName);
-            if (outImage.exists() && outImage.delete()) {
-                logger.debug(String.format("%s is deleted", outImage.toString()));
-            }
-            return Files.move(outputFile.toPath(), outImage.toPath(), StandardCopyOption.ATOMIC_MOVE).toFile();
-        } catch (Exception e) {
-            throw new HostException("Failed to rename the image", e);
-        }
-    }
-
-    final String getDefaultImageName(final String imgUrl) {
-        String imageTitle = imgUrl.substring(imgUrl.lastIndexOf('/') + 1);
-        logger.debug(String.format("Extracting name from url %s: %s", imgUrl, imageTitle));
-        return imgUrl;
-    }
-
-    final Response getResponse(final String url, final HttpClientContext context) throws HostException {
-        String basePage;
-
-        HttpClient client = cm.getClient().build();
-        HttpGet httpGet = cm.buildHttpGet(url);
-        Header[] headers;
-        logger.debug(String.format("Requesting %s", url));
-        try (CloseableHttpResponse response = (CloseableHttpResponse) client.execute(httpGet, context)) {
-            if (response.getStatusLine().getStatusCode() / 100 != 2) {
-                throw new HostException(String.format("Unexpected response code: %d", response.getStatusLine().getStatusCode()));
-            }
-            headers = response.getAllHeaders();
-            basePage = EntityUtils.toString(response.getEntity());
-            logger.debug(String.format("%s response: %n%s", url, basePage));
-            EntityUtils.consumeQuietly(response.getEntity());
-        } catch (IOException e) {
-            throw new HostException(e);
-        }
-
-        try {
-            logger.debug(String.format("Cleaning %s response", url));
-            return new Response(htmlProcessorService.clean(basePage), headers);
-        } catch (HtmlProcessorException e) {
-            throw new HostException(e);
-        }
-    }
-
-    protected String appendUri(String uri, String appendQuery) throws URISyntaxException {
-        URI oldUri = new URI(uri);
-
-        String newQuery = oldUri.getQuery();
-        if (newQuery == null) {
-            newQuery = appendQuery;
-        } else {
-            newQuery += "&" + appendQuery;
-        }
-
-        return new URI(oldUri.getScheme(), oldUri.getAuthority(),
-                oldUri.getPath(), newQuery, oldUri.getFragment()).toString();
-    }
-
-    private void setImageRequest(final ImageFileData imageFileData) {
-        HttpGet httpGet = cm.buildHttpGet(imageFileData.getImageUrl());
-        httpGet.addHeader("Referer", imageFileData.getPageUrl());
-        imageFileData.setImageRequest(httpGet);
-    }
-
-    protected abstract void setNameAndUrl(final String url, final ImageFileData imageFileData, final HttpClientContext context) throws HostException;
+    public abstract HostService.NameUrl getNameAndUrl(final String url, final HttpClientContext context) throws HostException;
 
     @Override
     public boolean equals(Object o) {
@@ -247,16 +30,9 @@ abstract public class Host {
         return Objects.equals(getHost(), host.getHost());
     }
 
-    @Getter
-    public static class Response {
-
-        protected Response(Document document, Header[] headers) {
-            this.document = document;
-            this.headers = headers;
-        }
-
-        private Document document;
-        private Header[] headers;
+    @Override
+    public int hashCode() {
+        return Objects.hash(getHost());
     }
 
     @Override

vripper-server/src/main/java/tn/mnlr/vripper/host/ImageBamHost.java

@@ -1,12 +1,11 @@
 package tn.mnlr.vripper.host;
 
+import lombok.extern.slf4j.Slf4j;
 import org.apache.http.client.HttpClient;
 import org.apache.http.client.methods.CloseableHttpResponse;
 import org.apache.http.client.methods.HttpGet;
 import org.apache.http.client.protocol.HttpClientContext;
 import org.apache.http.util.EntityUtils;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Service;
 import org.w3c.dom.Document;
@@ -14,22 +13,33 @@ import org.w3c.dom.Node;
 import tn.mnlr.vripper.exception.HostException;
 import tn.mnlr.vripper.exception.HtmlProcessorException;
 import tn.mnlr.vripper.exception.XpathException;
-import tn.mnlr.vripper.q.ImageFileData;
-import tn.mnlr.vripper.services.ConnectionManager;
+import tn.mnlr.vripper.services.ConnectionService;
+import tn.mnlr.vripper.services.HostService;
+import tn.mnlr.vripper.services.HtmlProcessorService;
+import tn.mnlr.vripper.services.XpathService;
 
 import java.io.IOException;
 
 @Service
+@Slf4j
 public class ImageBamHost extends Host {
 
-    private static final Logger logger = LoggerFactory.getLogger(ImageBamHost.class);
-
     private static final String host = "imagebam.com";
     private static final String CONTINUE_BUTTON_XPATH = "//a[@title='Continue to your image']";
     private static final String IMG_XPATH = "//img[@class='image']";
 
+    private final HostService hostService;
+    private final ConnectionService cm;
+    private final XpathService xpathService;
+    private final HtmlProcessorService htmlProcessorService;
+
     @Autowired
-    private ConnectionManager cm;
+    public ImageBamHost(HostService hostService, ConnectionService cm, XpathService xpathService, HtmlProcessorService htmlProcessorService) {
+        this.hostService = hostService;
+        this.cm = cm;
+        this.xpathService = xpathService;
+        this.htmlProcessorService = htmlProcessorService;
+    }
 
     @Override
     public String getHost() {
@@ -42,29 +52,29 @@ public class ImageBamHost extends Host {
     }
 
     @Override
-    protected void setNameAndUrl(final String url, final ImageFileData imageFileData, final HttpClientContext context) throws HostException {
+    public HostService.NameUrl getNameAndUrl(final String url, final HttpClientContext context) throws HostException {
 
-        Response response = getResponse(url, context);
+        HostService.Response response = hostService.getResponse(url, context);
         Document doc = response.getDocument();
 
         Node contDiv;
         try {
-            logger.debug(String.format("Looking for xpath expression %s in %s", CONTINUE_BUTTON_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", CONTINUE_BUTTON_XPATH, url));
             contDiv = xpathService.getAsNode(doc, CONTINUE_BUTTON_XPATH);
         } catch (XpathException e) {
             throw new HostException(e);
         }
 
         if (contDiv != null) {
-            logger.debug(String.format("Click button found for %s", url));
+            log.debug(String.format("Click button found for %s", url));
             HttpClient client = cm.getClient().build();
-            HttpGet httpGet = cm.buildHttpGet(url);
+            HttpGet httpGet = cm.buildHttpGet(url, context);
             httpGet.addHeader("Referer", url);
-            logger.debug(String.format("Requesting %s", httpGet));
+            log.debug(String.format("Requesting %s", httpGet));
             try (CloseableHttpResponse res = (CloseableHttpResponse) client.execute(httpGet, context)) {
                 String s = EntityUtils.toString(res.getEntity());
-                logger.debug(String.format("%s response is:%n%s", httpGet, s));
-                logger.debug(String.format("Cleaning response for %s", httpGet));
+                log.debug(String.format("%s response is:%n%s", httpGet, s));
+                log.debug(String.format("Cleaning response for %s", httpGet));
                 doc = htmlProcessorService.clean(s);
                 EntityUtils.consumeQuietly(res.getEntity());
             } catch (IOException | HtmlProcessorException e) {
@@ -74,20 +84,19 @@ public class ImageBamHost extends Host {
 
         Node imgNode;
         try {
-            logger.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
             imgNode = xpathService.getAsNode(doc, IMG_XPATH);
         } catch (XpathException e) {
             throw new HostException(e);
         }
 
         try {
-            logger.debug(String.format("Resolving name and image url for %s", url));
+            log.debug(String.format("Resolving name and image url for %s", url));
             String imgTitle = imgNode.getAttributes().getNamedItem("id").getTextContent().trim();
             String imgUrl = imgNode.getAttributes().getNamedItem("src").getTextContent().trim();
 
-            imageFileData.setImageUrl(imgUrl);
-            imageFileData.setImageName(imgTitle.isEmpty() ? imgUrl.substring(imgUrl.lastIndexOf('/') + 1) : imgTitle);
-        } catch(Exception e) {
+            return new HostService.NameUrl(imgTitle.isEmpty() ? imgUrl.substring(imgUrl.lastIndexOf('/') + 1) : imgTitle, imgUrl);
+        } catch (Exception e) {
             throw new HostException("Unexpected error occurred", e);
         }
     }

vripper-server/src/main/java/tn/mnlr/vripper/host/ImageTwistHost.java

@@ -1,27 +1,33 @@
 package tn.mnlr.vripper.host;
 
+import lombok.extern.slf4j.Slf4j;
 import org.apache.http.client.protocol.HttpClientContext;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Service;
 import org.w3c.dom.Document;
 import org.w3c.dom.Node;
 import tn.mnlr.vripper.exception.HostException;
 import tn.mnlr.vripper.exception.XpathException;
-import tn.mnlr.vripper.q.ImageFileData;
-import tn.mnlr.vripper.services.ConnectionManager;
+import tn.mnlr.vripper.services.HostService;
+import tn.mnlr.vripper.services.XpathService;
 
 import java.util.Optional;
 
 @Service
+@Slf4j
 public class ImageTwistHost extends Host {
 
     private static final String IMG_XPATH = "//img[contains(@class, 'img')]";
-    private static final Logger logger = LoggerFactory.getLogger(ImageTwistHost.class);
     private static final String host = "imagetwist.com";
+
+    private final HostService hostService;
+    private final XpathService xpathService;
+
     @Autowired
-    private ConnectionManager cm;
+    public ImageTwistHost(HostService hostService, XpathService xpathService) {
+        this.hostService = hostService;
+        this.xpathService = xpathService;
+    }
 
     @Override
     public String getHost() {
@@ -34,25 +40,24 @@ public class ImageTwistHost extends Host {
     }
 
     @Override
-    protected void setNameAndUrl(final String url, final ImageFileData imageFileData, final HttpClientContext context) throws HostException {
+    public HostService.NameUrl getNameAndUrl(final String url, final HttpClientContext context) throws HostException {
 
-        Document doc = getResponse(url, context).getDocument();
+        Document doc = hostService.getResponse(url, context).getDocument();
 
         Node imgNode;
         try {
-            logger.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
             imgNode = xpathService.getAsNode(doc, IMG_XPATH);
         } catch (XpathException e) {
             throw new HostException(e);
         }
 
         try {
-            logger.debug(String.format("Resolving name and image url for %s", url));
+            log.debug(String.format("Resolving name and image url for %s", url));
             String imgTitle = Optional.ofNullable(imgNode.getAttributes().getNamedItem("alt")).map(Node::getTextContent).map(String::trim).orElse(null);
             String imgUrl = imgNode.getAttributes().getNamedItem("src").getTextContent().trim();
 
-            imageFileData.setImageUrl(imgUrl);
-            imageFileData.setImageName(imgTitle);
+            return new HostService.NameUrl(imgTitle, imgUrl);
         } catch (Exception e) {
             throw new HostException("Unexpected error occurred", e);
         }

vripper-server/src/main/java/tn/mnlr/vripper/host/ImageVenueHost.java

@@ -1,25 +1,32 @@
 package tn.mnlr.vripper.host;
 
+import lombok.extern.slf4j.Slf4j;
 import org.apache.http.client.protocol.HttpClientContext;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
+import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Service;
 import org.w3c.dom.Document;
 import org.w3c.dom.Node;
 import tn.mnlr.vripper.exception.HostException;
 import tn.mnlr.vripper.exception.XpathException;
-import tn.mnlr.vripper.q.ImageFileData;
-
-import java.net.URI;
+import tn.mnlr.vripper.services.HostService;
+import tn.mnlr.vripper.services.XpathService;
 
 @Service
+@Slf4j
 public class ImageVenueHost extends Host {
 
-    private static final Logger logger = LoggerFactory.getLogger(ImageVenueHost.class);
-
     private static final String host = "imagevenue.com";
-    private static final String CONTINUE_BUTTON_XPATH = "//a[@title='Continue to your image']";
-    private static final String IMG_XPATH = "//img[@id='thepic']";
+    private static final String CONTINUE_BUTTON_XPATH = "//a[@title='Continue to ImageVenue']";
+    private static final String IMG_XPATH = "//a[@data-toggle='full']/img";
+
+    private final HostService hostService;
+    private final XpathService xpathService;
+
+    @Autowired
+    public ImageVenueHost(HostService hostService, XpathService xpathService) {
+        this.hostService = hostService;
+        this.xpathService = xpathService;
+    }
 
     @Override
     public String getHost() {
@@ -32,20 +39,18 @@ public class ImageVenueHost extends Host {
     }
 
     @Override
-    protected void setNameAndUrl(final String url, final ImageFileData imageFileData, final HttpClientContext context) throws HostException {
+    public HostService.NameUrl getNameAndUrl(final String _url, final HttpClientContext context) throws HostException {
 
-        //Sadly, they do not support https.
-        //If they add such support in the future,
-        //then we should automatically adapt the URL here as done elsewhere.
+        String url = _url.replace("http://", "https://");
 
-        Response resp = getResponse(url, context);
+        HostService.Response resp = hostService.getResponse(url, context);
         Document doc = resp.getDocument();
 
         try {
-            logger.debug(String.format("Looking for xpath expression %s in %s", CONTINUE_BUTTON_XPATH, url));
-            if(xpathService.getAsNode(doc, CONTINUE_BUTTON_XPATH) != null) {
+            log.debug(String.format("Looking for xpath expression %s in %s", CONTINUE_BUTTON_XPATH, url));
+            if (xpathService.getAsNode(doc, CONTINUE_BUTTON_XPATH) != null) {
                 //Button detected. No need to actually click it, just make the call again.
-                resp = getResponse(url, context);
+                resp = hostService.getResponse(url, context);
                 doc = resp.getDocument();
             }
         } catch (XpathException e) {
@@ -54,7 +59,7 @@ public class ImageVenueHost extends Host {
 
         Node imgNode;
         try {
-            logger.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
             imgNode = xpathService.getAsNode(doc, IMG_XPATH);
         } catch (XpathException e) {
             throw new HostException(e);
@@ -65,14 +70,12 @@ public class ImageVenueHost extends Host {
         }
 
         try {
-            logger.debug(String.format("Resolving name and image url for %s", url));
+            log.debug(String.format("Resolving name and image url for %s", url));
             String imgTitle = imgNode.getAttributes().getNamedItem("alt").getTextContent().trim();
             String imgUrl = imgNode.getAttributes().getNamedItem("src").getTextContent().trim();
 
-            URI baseUri = new URI(url);
-            imageFileData.setImageUrl(new URI(baseUri.getScheme(), baseUri.getHost(), '/'+imgUrl, null).toString());
-            imageFileData.setImageName(imgTitle.isEmpty() ? imgUrl.substring(imgUrl.lastIndexOf('/') + 1) : imgTitle);
-        } catch(Exception e) {
+            return new HostService.NameUrl(imgTitle.isEmpty() ? imgUrl.substring(imgUrl.lastIndexOf('/') + 1) : imgTitle, imgUrl);
+        } catch (Exception e) {
             throw new HostException("Unexpected error occurred", e);
         }
     }

vripper-server/src/main/java/tn/mnlr/vripper/host/ImageZillaHost.java

@@ -1,28 +1,32 @@
 package tn.mnlr.vripper.host;
 
+import lombok.extern.slf4j.Slf4j;
 import org.apache.http.client.protocol.HttpClientContext;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Service;
 import org.w3c.dom.Document;
 import org.w3c.dom.Node;
 import tn.mnlr.vripper.exception.HostException;
 import tn.mnlr.vripper.exception.XpathException;
-import tn.mnlr.vripper.q.ImageFileData;
-import tn.mnlr.vripper.services.ConnectionManager;
+import tn.mnlr.vripper.services.HostService;
+import tn.mnlr.vripper.services.XpathService;
 
 @Service
+@Slf4j
 public class ImageZillaHost extends Host {
 
-    private static final Logger logger = LoggerFactory.getLogger(ImageZillaHost.class);
-
     private static final String host = "imagezilla.net";
     private static final String lookup = "imagezilla.net/show";
     private static final String IMG_XPATH = "//img[@id='photo']";
 
+    private final HostService hostService;
+    private final XpathService xpathService;
+
     @Autowired
-    private ConnectionManager cm;
+    public ImageZillaHost(HostService hostService, XpathService xpathService) {
+        this.hostService = hostService;
+        this.xpathService = xpathService;
+    }
 
     @Override
     public String getHost() {
@@ -35,16 +39,16 @@ public class ImageZillaHost extends Host {
     }
 
     @Override
-    protected void setNameAndUrl(final String url, final ImageFileData imageFileData, final HttpClientContext context) throws HostException {
+    public HostService.NameUrl getNameAndUrl(final String url, final HttpClientContext context) throws HostException {
 
-        Document doc = getResponse(url, context).getDocument();
+        Document doc = hostService.getResponse(url, context).getDocument();
 
         String title;
         try {
-            logger.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
             Node titleNode = xpathService.getAsNode(doc, IMG_XPATH).getAttributes().getNamedItem("title");
-            logger.debug(String.format("Resolving name for %s", url));
-            if(titleNode != null) {
+            log.debug(String.format("Resolving name for %s", url));
+            if (titleNode != null) {
                 title = titleNode.getTextContent().trim();
             } else {
                 title = null;
@@ -53,13 +57,12 @@ public class ImageZillaHost extends Host {
             throw new HostException(e);
         }
 
-        if(title == null || title.isEmpty()) {
-            title = getDefaultImageName(url);
+        if (title == null || title.isEmpty()) {
+            title = hostService.getDefaultImageName(url);
         }
 
         try {
-            imageFileData.setImageUrl(url.replace("show", "images"));
-            imageFileData.setImageName(title);
+            return new HostService.NameUrl(title, url.replace("show", "images"));
         } catch (Exception e) {
             throw new HostException("Unexpected error occurred", e);
         }

vripper-server/src/main/java/tn/mnlr/vripper/host/ImgSpiceHost.java

@@ -1,27 +1,31 @@
 package tn.mnlr.vripper.host;
 
+import lombok.extern.slf4j.Slf4j;
 import org.apache.http.client.protocol.HttpClientContext;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Service;
 import org.w3c.dom.Document;
 import org.w3c.dom.Node;
 import tn.mnlr.vripper.exception.HostException;
 import tn.mnlr.vripper.exception.XpathException;
-import tn.mnlr.vripper.q.ImageFileData;
-import tn.mnlr.vripper.services.ConnectionManager;
+import tn.mnlr.vripper.services.HostService;
+import tn.mnlr.vripper.services.XpathService;
 
 @Service
+@Slf4j
 public class ImgSpiceHost extends Host {
 
-    private static final Logger logger = LoggerFactory.getLogger(ImgSpiceHost.class);
-
     private static final String host = "imgspice.com";
     private static final String IMG_XPATH = "//img[@id='imgpreview']";
 
+    private final HostService hostService;
+    private final XpathService xpathService;
+
     @Autowired
-    private ConnectionManager cm;
+    public ImgSpiceHost(HostService hostService, XpathService xpathService) {
+        this.hostService = hostService;
+        this.xpathService = xpathService;
+    }
 
     @Override
     public String getHost() {
@@ -34,27 +38,26 @@ public class ImgSpiceHost extends Host {
     }
 
     @Override
-    protected void setNameAndUrl(final String _url, final ImageFileData imageFileData, final HttpClientContext context) throws HostException {
+    public HostService.NameUrl getNameAndUrl(final String _url, final HttpClientContext context) throws HostException {
 
         String url = _url.replace("http://", "https://");
-        Response resp = getResponse(url, context);
+        HostService.Response resp = hostService.getResponse(url, context);
         Document doc = resp.getDocument();
 
         Node imgNode;
         try {
-            logger.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
             imgNode = xpathService.getAsNode(doc, IMG_XPATH);
         } catch (XpathException e) {
             throw new HostException(e);
         }
 
         try {
-            logger.debug(String.format("Resolving name and image url for %s", url));
+            log.debug(String.format("Resolving name and image url for %s", url));
             String imgTitle = imgNode.getAttributes().getNamedItem("alt").getTextContent().trim();
             String imgUrl = imgNode.getAttributes().getNamedItem("src").getTextContent().trim();
 
-            imageFileData.setImageUrl(imgUrl);
-            imageFileData.setImageName(imgTitle.isEmpty() ? imgUrl.substring(imgUrl.lastIndexOf('/') + 1) : imgTitle);
+            return new HostService.NameUrl(imgTitle.isEmpty() ? imgUrl.substring(imgUrl.lastIndexOf('/') + 1) : imgTitle, imgUrl);
         } catch (Exception e) {
             throw new HostException("Unexpected error occurred", e);
         }

vripper-server/src/main/java/tn/mnlr/vripper/host/ImgboxHost.java

@@ -1,27 +1,32 @@
 package tn.mnlr.vripper.host;
 
+import lombok.extern.slf4j.Slf4j;
 import org.apache.http.client.protocol.HttpClientContext;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Service;
 import org.w3c.dom.Document;
 import org.w3c.dom.Node;
 import tn.mnlr.vripper.exception.HostException;
 import tn.mnlr.vripper.exception.XpathException;
-import tn.mnlr.vripper.q.ImageFileData;
-import tn.mnlr.vripper.services.ConnectionManager;
+import tn.mnlr.vripper.services.HostService;
+import tn.mnlr.vripper.services.XpathService;
 
 @Service
+@Slf4j
 public class ImgboxHost extends Host {
 
-    private static final Logger logger = LoggerFactory.getLogger(ImgboxHost.class);
-
     private static final String host = "imgbox.com";
     private static final String IMG_XPATH = "//img[@id='img']";
 
+    private final HostService hostService;
+    private final XpathService xpathService;
+
+
     @Autowired
-    private ConnectionManager cm;
+    public ImgboxHost(HostService hostService, XpathService xpathService) {
+        this.hostService = hostService;
+        this.xpathService = xpathService;
+    }
 
     @Override
     public String getHost() {
@@ -34,25 +39,24 @@ public class ImgboxHost extends Host {
     }
 
     @Override
-    protected void setNameAndUrl(final String url, final ImageFileData imageFileData, final HttpClientContext context) throws HostException {
+    public HostService.NameUrl getNameAndUrl(final String url, final HttpClientContext context) throws HostException {
 
-        Document doc = getResponse(url, context).getDocument();
+        Document doc = hostService.getResponse(url, context).getDocument();
 
         Node imgNode;
         try {
-            logger.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
             imgNode = xpathService.getAsNode(doc, IMG_XPATH);
         } catch (XpathException e) {
             throw new HostException(e);
         }
 
         try {
-            logger.debug(String.format("Resolving name and image url for %s", url));
+            log.debug(String.format("Resolving name and image url for %s", url));
             String imgTitle = imgNode.getAttributes().getNamedItem("title").getTextContent().trim();
             String imgUrl = imgNode.getAttributes().getNamedItem("src").getTextContent().trim();
 
-            imageFileData.setImageUrl(imgUrl);
-            imageFileData.setImageName(imgTitle);
+            return new HostService.NameUrl(imgTitle, imgUrl);
         } catch (Exception e) {
             throw new HostException("Unexpected error occurred", e);
         }

vripper-server/src/main/java/tn/mnlr/vripper/host/ImxHost.java

@@ -1,5 +1,6 @@
 package tn.mnlr.vripper.host;
 
+import lombok.extern.slf4j.Slf4j;
 import org.apache.http.NameValuePair;
 import org.apache.http.client.HttpClient;
 import org.apache.http.client.entity.UrlEncodedFormEntity;
@@ -8,8 +9,6 @@ import org.apache.http.client.methods.HttpPost;
 import org.apache.http.client.protocol.HttpClientContext;
 import org.apache.http.message.BasicNameValuePair;
 import org.apache.http.util.EntityUtils;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Service;
 import org.w3c.dom.Document;
@@ -17,24 +16,35 @@ import org.w3c.dom.Node;
 import tn.mnlr.vripper.exception.HostException;
 import tn.mnlr.vripper.exception.HtmlProcessorException;
 import tn.mnlr.vripper.exception.XpathException;
-import tn.mnlr.vripper.q.ImageFileData;
-import tn.mnlr.vripper.services.ConnectionManager;
+import tn.mnlr.vripper.services.ConnectionService;
+import tn.mnlr.vripper.services.HostService;
+import tn.mnlr.vripper.services.HtmlProcessorService;
+import tn.mnlr.vripper.services.XpathService;
 
 import java.io.IOException;
 import java.util.ArrayList;
 import java.util.List;
 
 @Service
+@Slf4j
 public class ImxHost extends Host {
 
-    private static final Logger logger = LoggerFactory.getLogger(ImxHost.class);
-
     private static final String host = "imx.to";
     private static final String CONTINUE_BUTTON_XPATH = "//*[@name='imgContinue']";
     private static final String IMG_XPATH = "//img[@class='centred']";
 
+    private final HostService hostService;
+    private final XpathService xpathService;
+    private final ConnectionService cm;
+    private final HtmlProcessorService htmlProcessorService;
+
     @Autowired
-    private ConnectionManager cm;
+    public ImxHost(HostService hostService, XpathService xpathService, ConnectionService cm, HtmlProcessorService htmlProcessorService) {
+        this.hostService = hostService;
+        this.xpathService = xpathService;
+        this.cm = cm;
+        this.htmlProcessorService = htmlProcessorService;
+    }
 
     @Override
     public String getHost() {
@@ -47,17 +57,20 @@ public class ImxHost extends Host {
     }
 
     @Override
-    protected void setNameAndUrl(final String _url, final ImageFileData imageFileData, final HttpClientContext context) throws HostException {
+    public HostService.NameUrl getNameAndUrl(final String _url, final HttpClientContext context) throws HostException {
 
         String url = _url.replace("http://", "https://");
-        Response resp = getResponse(url, context);
+        HostService.Response resp = hostService.getResponse(url, context);
         Document doc = resp.getDocument();
 
         Node contDiv;
         String value = null;
         try {
-            logger.debug(String.format("Looking for xpath expression %s in %s", CONTINUE_BUTTON_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", CONTINUE_BUTTON_XPATH, url));
             contDiv = xpathService.getAsNode(doc, CONTINUE_BUTTON_XPATH);
+            if (contDiv == null) {
+                throw new HostException(CONTINUE_BUTTON_XPATH + " cannot be found");
+            }
             Node node = contDiv.getAttributes().getNamedItem("value");
             if (node != null) {
                 value = node.getTextContent();
@@ -69,9 +82,9 @@ public class ImxHost extends Host {
         if (value == null) {
             throw new HostException("Failed to obtain value attribute from continue input");
         }
-        logger.debug(String.format("Click button found for %s", url));
+        log.debug(String.format("Click button found for %s", url));
         HttpClient client = cm.getClient().build();
-        HttpPost httpPost = cm.buildHttpPost(url);
+        HttpPost httpPost = cm.buildHttpPost(url, context);
         List<NameValuePair> params = new ArrayList<>();
         params.add(new BasicNameValuePair("imgContinue", value));
         try {
@@ -79,9 +92,9 @@ public class ImxHost extends Host {
         } catch (Exception e) {
             throw new HostException(e);
         }
-        logger.debug(String.format("Requesting %s", httpPost));
+        log.debug(String.format("Requesting %s", httpPost));
         try (CloseableHttpResponse response = (CloseableHttpResponse) client.execute(httpPost, context)) {
-            logger.debug(String.format("Cleaning response for %s", httpPost));
+            log.debug(String.format("Cleaning response for %s", httpPost));
             doc = htmlProcessorService.clean(EntityUtils.toString(response.getEntity()));
             EntityUtils.consumeQuietly(response.getEntity());
         } catch (IOException | HtmlProcessorException e) {
@@ -90,20 +103,19 @@ public class ImxHost extends Host {
 
         Node imgNode;
         try {
-            logger.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
             imgNode = xpathService.getAsNode(doc, IMG_XPATH);
         } catch (XpathException e) {
             throw new HostException(e);
         }
 
         try {
-            logger.debug(String.format("Resolving name and image url for %s", url));
+            log.debug(String.format("Resolving name and image url for %s", url));
             String imgTitle = imgNode.getAttributes().getNamedItem("alt").getTextContent().trim();
             String imgUrl = imgNode.getAttributes().getNamedItem("src").getTextContent().trim();
 
-            imageFileData.setImageUrl(imgUrl);
-            imageFileData.setImageName(imgTitle.isEmpty() ? imgUrl.substring(imgUrl.lastIndexOf('/') + 1) : imgTitle);
-        } catch(Exception e) {
+            return new HostService.NameUrl(imgTitle.isEmpty() ? imgUrl.substring(imgUrl.lastIndexOf('/') + 1) : imgTitle, imgUrl);
+        } catch (Exception e) {
             throw new HostException("Unexpected error occurred", e);
         }
     }

vripper-server/src/main/java/tn/mnlr/vripper/host/PimpandhostHost.java

@@ -1,27 +1,32 @@
 package tn.mnlr.vripper.host;
 
+import lombok.extern.slf4j.Slf4j;
 import org.apache.http.client.protocol.HttpClientContext;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Service;
 import org.w3c.dom.Document;
 import org.w3c.dom.Node;
 import tn.mnlr.vripper.exception.HostException;
 import tn.mnlr.vripper.exception.XpathException;
-import tn.mnlr.vripper.q.ImageFileData;
-import tn.mnlr.vripper.services.ConnectionManager;
+import tn.mnlr.vripper.services.HostService;
+import tn.mnlr.vripper.services.XpathService;
 
 @Service
+@Slf4j
 public class PimpandhostHost extends Host {
 
-    private static final Logger logger = LoggerFactory.getLogger(PimpandhostHost.class);
-
     private static final String host = "pimpandhost.com";
     private static final String IMG_XPATH = "//img[contains(@class, 'original')]";
 
+    private final HostService hostService;
+    private final XpathService xpathService;
+
     @Autowired
-    private ConnectionManager cm;
+    public PimpandhostHost(HostService hostService, XpathService xpathService) {
+        super();
+        this.hostService = hostService;
+        this.xpathService = xpathService;
+    }
 
     @Override
     public String getHost() {
@@ -34,33 +39,35 @@ public class PimpandhostHost extends Host {
     }
 
     @Override
-    protected void setNameAndUrl(final String _url, final ImageFileData imageFileData, final HttpClientContext context) throws HostException {
+    public HostService.NameUrl getNameAndUrl(final String _url, final HttpClientContext context) throws HostException {
 
         String url;
         try {
-            url = appendUri(_url.replace("http://", "https://"), "size=original");
+            url = hostService.appendUri(_url.replace("http://", "https://"), "size=original");
         } catch (Exception e) {
             throw new HostException(e);
         }
 
-        Response resp = getResponse(url, context);
+        HostService.Response resp = hostService.getResponse(url, context);
         Document doc = resp.getDocument();
 
         Node imgNode;
         try {
-            logger.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
             imgNode = xpathService.getAsNode(doc, IMG_XPATH);
         } catch (XpathException e) {
             throw new HostException(e);
         }
 
         try {
-            logger.debug(String.format("Resolving name and image url for %s", url));
+            log.debug(String.format("Resolving name and image url for %s", url));
             String imgTitle = imgNode.getAttributes().getNamedItem("alt").getTextContent().trim();
             String imgUrl = "https:" + imgNode.getAttributes().getNamedItem("src").getTextContent().trim();
 
-            imageFileData.setImageUrl(imgUrl);
-            imageFileData.setImageName(imgTitle.isEmpty() ? imgUrl.substring(imgUrl.lastIndexOf('/') + 1) : imgTitle);
+            return new HostService.NameUrl(
+                    imgTitle.isEmpty() ? imgUrl.substring(imgUrl.lastIndexOf('/') + 1) : imgTitle,
+                    imgUrl
+            );
         } catch (Exception e) {
             throw new HostException("Unexpected error occurred", e);
         }

vripper-server/src/main/java/tn/mnlr/vripper/host/PixRouteHost.java

@@ -0,0 +1,69 @@
+package tn.mnlr.vripper.host;
+
+import lombok.extern.slf4j.Slf4j;
+import org.apache.http.client.protocol.HttpClientContext;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Service;
+import org.w3c.dom.Document;
+import org.w3c.dom.Node;
+import tn.mnlr.vripper.exception.HostException;
+import tn.mnlr.vripper.exception.XpathException;
+import tn.mnlr.vripper.services.HostService;
+import tn.mnlr.vripper.services.XpathService;
+
+@Service
+@Slf4j
+public class PixRouteHost extends Host {
+
+    private static final String host = "pixroute.com";
+    private static final String IMG_XPATH = "//img[@id='imgpreview']";
+
+    private final HostService hostService;
+    private final XpathService xpathService;
+
+    @Autowired
+    public PixRouteHost(HostService hostService, XpathService xpathService) {
+        this.hostService = hostService;
+        this.xpathService = xpathService;
+    }
+
+    @Override
+    public String getHost() {
+        return host;
+    }
+
+    @Override
+    public String getLookup() {
+        return host;
+    }
+
+    @Override
+    public HostService.NameUrl getNameAndUrl(final String _url, final HttpClientContext context) throws HostException {
+
+        String url = _url.replace("http://", "https://");
+        Document doc = hostService.getResponse(url, context).getDocument();
+
+        Node imgNode;
+        try {
+            log.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
+            imgNode = xpathService.getAsNode(doc, IMG_XPATH);
+        } catch (XpathException e) {
+            throw new HostException(e);
+        }
+
+        if (imgNode == null) {
+            throw new HostException("Failed to locate image");
+        }
+
+        try {
+            log.debug(String.format("Resolving name and image url for %s", url));
+
+            return new HostService.NameUrl(
+                    imgNode.getAttributes().getNamedItem("alt").getTextContent().trim(),
+                    imgNode.getAttributes().getNamedItem("src").getTextContent().trim()
+            );
+        } catch (Exception e) {
+            throw new HostException("Unexpected error occurred", e);
+        }
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/host/PixhostHost.java

@@ -1,28 +1,32 @@
 package tn.mnlr.vripper.host;
 
+import lombok.extern.slf4j.Slf4j;
 import org.apache.http.client.protocol.HttpClientContext;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Service;
 import org.w3c.dom.Document;
 import org.w3c.dom.Node;
 import tn.mnlr.vripper.exception.HostException;
 import tn.mnlr.vripper.exception.XpathException;
-import tn.mnlr.vripper.q.ImageFileData;
-import tn.mnlr.vripper.services.ConnectionManager;
+import tn.mnlr.vripper.services.HostService;
+import tn.mnlr.vripper.services.XpathService;
 
 @Service
+@Slf4j
 public class PixhostHost extends Host {
 
-    private static final Logger logger = LoggerFactory.getLogger(PixhostHost.class);
-
     private static final String host = "pixhost.to";
     private static final String lookup = "pixhost.to/show";
     private static final String IMG_XPATH = "//img[@id='image']";
 
+    private final HostService hostService;
+    private final XpathService xpathService;
+
     @Autowired
-    private ConnectionManager cm;
+    public PixhostHost(HostService hostService, XpathService xpathService) {
+        this.hostService = hostService;
+        this.xpathService = xpathService;
+    }
 
     @Override
     public String getHost() {
@@ -35,25 +39,24 @@ public class PixhostHost extends Host {
     }
 
     @Override
-    protected void setNameAndUrl(final String url, final ImageFileData imageFileData, final HttpClientContext context) throws HostException {
+    public HostService.NameUrl getNameAndUrl(final String url, final HttpClientContext context) throws HostException {
 
-        Document doc = getResponse(url, context).getDocument();
+        Document doc = hostService.getResponse(url, context).getDocument();
 
         Node imgNode;
         try {
-            logger.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
             imgNode = xpathService.getAsNode(doc, IMG_XPATH);
         } catch (XpathException e) {
             throw new HostException(e);
         }
 
         try {
-            logger.debug(String.format("Resolving name and image url for %s", url));
+            log.debug(String.format("Resolving name and image url for %s", url));
             String imgTitle = imgNode.getAttributes().getNamedItem("alt").getTextContent().trim();
             String imgUrl = imgNode.getAttributes().getNamedItem("src").getTextContent().trim();
 
-            imageFileData.setImageUrl(imgUrl);
-            imageFileData.setImageName(imgTitle.substring(imgTitle.indexOf('_') + 1));
+            return new HostService.NameUrl(imgTitle.substring(imgTitle.indexOf('_') + 1), imgUrl);
         } catch (Exception e) {
             throw new HostException("Unexpected error occurred", e);
         }

vripper-server/src/main/java/tn/mnlr/vripper/host/PixxxelsHost.java

@@ -1,28 +1,32 @@
 package tn.mnlr.vripper.host;
 
+import lombok.extern.slf4j.Slf4j;
 import org.apache.http.client.protocol.HttpClientContext;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Service;
 import org.w3c.dom.Document;
 import org.w3c.dom.Node;
 import tn.mnlr.vripper.exception.HostException;
 import tn.mnlr.vripper.exception.XpathException;
-import tn.mnlr.vripper.q.ImageFileData;
-import tn.mnlr.vripper.services.ConnectionManager;
+import tn.mnlr.vripper.services.HostService;
+import tn.mnlr.vripper.services.XpathService;
 
 @Service
+@Slf4j
 public class PixxxelsHost extends Host {
 
-    private static final Logger logger = LoggerFactory.getLogger(PixxxelsHost.class);
-
     private static final String host = "pixxxels.cc";
     private static final String IMG_XPATH = "//*[@id='download']";
     private static final String TITLE_XPATH = "//*[contains(@class,'imagename')]";
 
+    private final HostService hostService;
+    private final XpathService xpathService;
+
     @Autowired
-    private ConnectionManager cm;
+    public PixxxelsHost(HostService hostService, XpathService xpathService) {
+        this.hostService = hostService;
+        this.xpathService = xpathService;
+    }
 
     @Override
     public String getHost() {
@@ -35,18 +39,18 @@ public class PixxxelsHost extends Host {
     }
 
     @Override
-    protected void setNameAndUrl(final String _url, final ImageFileData imageFileData, final HttpClientContext context) throws HostException {
+    public HostService.NameUrl getNameAndUrl(final String _url, final HttpClientContext context) throws HostException {
 
         String url = _url.replace("http://", "https://");
-        Response resp = getResponse(url, context);
+        HostService.Response resp = hostService.getResponse(url, context);
         Document doc = resp.getDocument();
 
         Node imgNode, titleNode;
         try {
-            logger.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
             imgNode = xpathService.getAsNode(doc, IMG_XPATH);
 
-            logger.debug(String.format("Looking for xpath expression %s in %s", TITLE_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", TITLE_XPATH, url));
             titleNode = xpathService.getAsNode(doc, TITLE_XPATH);
 
         } catch (XpathException e) {
@@ -54,12 +58,11 @@ public class PixxxelsHost extends Host {
         }
 
         try {
-            logger.debug(String.format("Resolving name and image url for %s", url));
+            log.debug(String.format("Resolving name and image url for %s", url));
             String imgTitle = titleNode.getTextContent().trim();
             String imgUrl = imgNode.getAttributes().getNamedItem("href").getTextContent().trim();
 
-            imageFileData.setImageUrl(imgUrl);
-            imageFileData.setImageName(imgTitle.isEmpty() ? imgUrl.substring(imgUrl.lastIndexOf('/') + 1) : imgTitle);
+            return new HostService.NameUrl(imgTitle.isEmpty() ? imgUrl.substring(imgUrl.lastIndexOf('/') + 1) : imgTitle, imgUrl);
         } catch (Exception e) {
             throw new HostException("Unexpected error occurred", e);
         }

vripper-server/src/main/java/tn/mnlr/vripper/host/PostImgHost.java

@@ -1,26 +1,35 @@
 package tn.mnlr.vripper.host;
 
+import lombok.extern.slf4j.Slf4j;
 import org.apache.http.client.protocol.HttpClientContext;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
+import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Service;
 import org.w3c.dom.Document;
 import org.w3c.dom.Node;
 import tn.mnlr.vripper.exception.HostException;
 import tn.mnlr.vripper.exception.XpathException;
-import tn.mnlr.vripper.q.ImageFileData;
+import tn.mnlr.vripper.services.HostService;
+import tn.mnlr.vripper.services.XpathService;
 
 import java.util.Optional;
 
 @Service
+@Slf4j
 public class PostImgHost extends Host {
 
-    private static final Logger logger = LoggerFactory.getLogger(PostImgHost.class);
-
     private static final String host = "postimg.cc";
     private static final String TITLE_XPATH = "//span[contains(@class,'imagename')]";
     private static final String IMG_XPATH = "//a[@id='download']";
 
+    private final HostService hostService;
+    private final XpathService xpathService;
+
+    @Autowired
+    public PostImgHost(HostService hostService, XpathService xpathService) {
+        this.hostService = hostService;
+        this.xpathService = xpathService;
+    }
+
     @Override
     public String getHost() {
         return host;
@@ -32,28 +41,27 @@ public class PostImgHost extends Host {
     }
 
     @Override
-    protected void setNameAndUrl(final String _url, final ImageFileData imageFileData, final HttpClientContext context) throws HostException {
+    public HostService.NameUrl getNameAndUrl(final String _url, final HttpClientContext context) throws HostException {
 
         String url = _url.replace("http://", "https://");
-        Document doc = getResponse(url, context).getDocument();
+        Document doc = hostService.getResponse(url, context).getDocument();
 
         Node urlNode, titleNode;
         try {
-            logger.debug(String.format("Looking for xpath expression %s in %s", TITLE_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", TITLE_XPATH, url));
             titleNode = xpathService.getAsNode(doc, TITLE_XPATH);
 
-            logger.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
             urlNode = xpathService.getAsNode(doc, IMG_XPATH);
         } catch (XpathException e) {
             throw new HostException(e);
         }
 
         try {
-            logger.debug(String.format("Resolving name and image url for %s", url));
-            String imgTitle = Optional.ofNullable(titleNode).map(node -> node.getTextContent().trim()).orElseGet(() -> getDefaultImageName(url));
+            log.debug(String.format("Resolving name and image url for %s", url));
+            String imgTitle = Optional.ofNullable(titleNode).map(node -> node.getTextContent().trim()).orElseGet(() -> hostService.getDefaultImageName(url));
 
-            imageFileData.setImageUrl(urlNode.getAttributes().getNamedItem("href").getTextContent().trim());
-            imageFileData.setImageName(imgTitle);
+            return new HostService.NameUrl(imgTitle, urlNode.getAttributes().getNamedItem("href").getTextContent().trim());
         } catch (Exception e) {
             throw new HostException("Unexpected error occurred", e);
         }

vripper-server/src/main/java/tn/mnlr/vripper/host/TurboImageHost.java

@@ -1,28 +1,32 @@
 package tn.mnlr.vripper.host;
 
+import lombok.extern.slf4j.Slf4j;
 import org.apache.http.client.protocol.HttpClientContext;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Service;
 import org.w3c.dom.Document;
 import org.w3c.dom.Node;
 import tn.mnlr.vripper.exception.HostException;
 import tn.mnlr.vripper.exception.XpathException;
-import tn.mnlr.vripper.q.ImageFileData;
-import tn.mnlr.vripper.services.ConnectionManager;
+import tn.mnlr.vripper.services.HostService;
+import tn.mnlr.vripper.services.XpathService;
 
 @Service
+@Slf4j
 public class TurboImageHost extends Host {
 
-    private static final Logger logger = LoggerFactory.getLogger(TurboImageHost.class);
-
     private static final String host = "turboimagehost.com";
     private static final String TITLE_XPATH = "//div[contains(@class,'titleFullS')]/h1";
-    private static final String IMG_XPATH = "//img[@id='uImage']";
+    private static final String IMG_XPATH = "//img[@id='imageid']";
+
+    private final HostService hostService;
+    private final XpathService xpathService;
 
     @Autowired
-    private ConnectionManager cm;
+    public TurboImageHost(HostService hostService, XpathService xpathService) {
+        this.hostService = hostService;
+        this.xpathService = xpathService;
+    }
 
     @Override
     public String getHost() {
@@ -35,16 +39,16 @@ public class TurboImageHost extends Host {
     }
 
     @Override
-    protected void setNameAndUrl(final String url, final ImageFileData imageFileData, final HttpClientContext context) throws HostException {
+    public HostService.NameUrl getNameAndUrl(final String url, final HttpClientContext context) throws HostException {
 
-        Document doc = getResponse(url, context).getDocument();
+        Document doc = hostService.getResponse(url, context).getDocument();
 
         String title;
         try {
-            logger.debug(String.format("Looking for xpath expression %s in %s", TITLE_XPATH, url));
+            log.debug(String.format("Looking for xpath expression %s in %s", TITLE_XPATH, url));
             Node titleNode = xpathService.getAsNode(doc, TITLE_XPATH);
-            logger.debug(String.format("Resolving name for %s", url));
-            if(titleNode != null) {
+            log.debug(String.format("Resolving name for %s", url));
+            if (titleNode != null) {
                 title = titleNode.getTextContent().trim();
             } else {
                 title = null;
@@ -53,14 +57,13 @@ public class TurboImageHost extends Host {
             throw new HostException(e);
         }
 
-        if(title == null || title.isEmpty()) {
-            title = getDefaultImageName(url);
+        if (title == null || title.isEmpty()) {
+            title = hostService.getDefaultImageName(url);
         }
 
         try {
             Node urlNode = xpathService.getAsNode(doc, IMG_XPATH);
-            imageFileData.setImageUrl(urlNode.getAttributes().getNamedItem("src").getTextContent().trim());
-            imageFileData.setImageName(title);
+            return new HostService.NameUrl(title, urlNode.getAttributes().getNamedItem("src").getTextContent().trim());
         } catch (Exception e) {
             throw new HostException("Unexpected error occurred", e);
         }

vripper-server/src/main/java/tn/mnlr/vripper/host/ViprImHost.java

@@ -0,0 +1,65 @@
+package tn.mnlr.vripper.host;
+
+import lombok.extern.slf4j.Slf4j;
+import org.apache.http.client.protocol.HttpClientContext;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Service;
+import org.w3c.dom.Document;
+import org.w3c.dom.Node;
+import tn.mnlr.vripper.exception.HostException;
+import tn.mnlr.vripper.exception.XpathException;
+import tn.mnlr.vripper.services.HostService;
+import tn.mnlr.vripper.services.XpathService;
+
+import java.util.Optional;
+
+@Service
+@Slf4j
+public class ViprImHost extends Host {
+
+    private static final String IMG_XPATH = "//img[contains(@class, 'img')]";
+    private static final String host = "vipr.im";
+
+    private final HostService hostService;
+    private final XpathService xpathService;
+
+    @Autowired
+    public ViprImHost(HostService hostService, XpathService xpathService) {
+        this.hostService = hostService;
+        this.xpathService = xpathService;
+    }
+
+    @Override
+    public String getHost() {
+        return host;
+    }
+
+    @Override
+    public String getLookup() {
+        return host;
+    }
+
+    @Override
+    public HostService.NameUrl getNameAndUrl(final String url, final HttpClientContext context) throws HostException {
+
+        Document doc = hostService.getResponse(url, context).getDocument();
+
+        Node imgNode;
+        try {
+            log.debug(String.format("Looking for xpath expression %s in %s", IMG_XPATH, url));
+            imgNode = xpathService.getAsNode(doc, IMG_XPATH);
+        } catch (XpathException e) {
+            throw new HostException(e);
+        }
+
+        try {
+            log.debug(String.format("Resolving name and image url for %s", url));
+            String imgTitle = Optional.ofNullable(imgNode.getAttributes().getNamedItem("alt")).map(Node::getTextContent).map(String::trim).orElse(null);
+            String imgUrl = imgNode.getAttributes().getNamedItem("src").getTextContent().trim();
+
+            return new HostService.NameUrl(imgTitle, imgUrl);
+        } catch (Exception e) {
+            throw new HostException("Unexpected error occurred", e);
+        }
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/Management.java

@@ -0,0 +1,70 @@
+package tn.mnlr.vripper.jpa;
+
+import org.springframework.beans.factory.annotation.Value;
+import org.springframework.jdbc.core.JdbcTemplate;
+import org.springframework.scheduling.annotation.EnableScheduling;
+import org.springframework.scheduling.annotation.Scheduled;
+import org.springframework.stereotype.Component;
+import tn.mnlr.vripper.download.DownloadService;
+import tn.mnlr.vripper.services.ThreadPoolService;
+
+import javax.annotation.PostConstruct;
+import javax.annotation.PreDestroy;
+import java.io.File;
+import java.time.LocalDate;
+import java.time.LocalDateTime;
+import java.time.format.DateTimeFormatter;
+import java.util.Optional;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+@Component
+@EnableScheduling
+public class Management {
+
+    public static final DateTimeFormatter FORMATTER = DateTimeFormatter.ofPattern("yyyy-MM-dd");
+    private static final Pattern BACKUP_FILE_PATTERN = Pattern.compile("^db_(\\d{4}-\\d{2}-\\d{2})\\.tar\\.gz$");
+
+    private final JdbcTemplate jdbcTemplate;
+    private final String backupFolder;
+    private final ThreadPoolService threadPoolService;
+    private final DownloadService downloadService;
+
+    public Management(JdbcTemplate jdbcTemplate, @Value("${base.dir}") String baseDir, @Value("${base.dir.name}") String baseDirName, ThreadPoolService threadPoolService, DownloadService downloadService) {
+        this.jdbcTemplate = jdbcTemplate;
+        this.threadPoolService = threadPoolService;
+        this.downloadService = downloadService;
+        this.backupFolder = baseDir + File.separator + baseDirName + File.separator + "backup";
+    }
+
+    @PreDestroy
+    public void destroy() {
+
+        try {
+            threadPoolService.destroy();
+            downloadService.destroy();
+            this.jdbcTemplate.execute("SHUTDOWN");
+        } catch (Exception ignored) {
+            Thread.currentThread().interrupt();
+        }
+    }
+
+    @PostConstruct
+    @Scheduled(cron = "0 0 0 ? * *")
+    private void backup() {
+
+        for (File file : Optional.ofNullable(new File(backupFolder).listFiles()).orElse(new File[]{})) {
+            Matcher matcher = BACKUP_FILE_PATTERN.matcher(file.getName());
+            if (matcher.find()) {
+                LocalDate localDate = LocalDate.parse(matcher.group(1), FORMATTER);
+                if (localDate.isEqual(LocalDate.now())) {
+                    return;
+                }
+            }
+        }
+
+        // create a backup file
+        File backupFile = new File(backupFolder, String.format("db_%s.tar.gz", LocalDateTime.now().format(FORMATTER)));
+        jdbcTemplate.execute(String.format("BACKUP DATABASE TO '%s' BLOCKING", backupFile.getPath()));
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/domain/Event.java

@@ -0,0 +1,66 @@
+package tn.mnlr.vripper.jpa.domain;
+
+import com.fasterxml.jackson.core.JsonGenerator;
+import com.fasterxml.jackson.databind.SerializerProvider;
+import com.fasterxml.jackson.databind.annotation.JsonSerialize;
+import com.fasterxml.jackson.databind.ser.std.StdSerializer;
+import lombok.Getter;
+import lombok.NoArgsConstructor;
+import lombok.Setter;
+import lombok.ToString;
+
+import java.io.IOException;
+import java.time.LocalDateTime;
+import java.time.format.DateTimeFormatter;
+
+@Getter
+@Setter
+@ToString
+@NoArgsConstructor
+public class Event {
+
+    private Long id;
+    private Type type;
+    private Status status;
+    @JsonSerialize(using = DateTimeSerializer.class)
+    private LocalDateTime time;
+    private String message;
+
+    public Event(Type type, Status status, LocalDateTime time, String message) {
+        this.type = type;
+        this.status = status;
+        this.time = time;
+        this.message = message;
+    }
+
+    public enum Type {
+        POST,
+        QUEUED,
+        THANKS,
+        METADATA,
+        SCAN,
+        DOWNLOAD,
+
+        // Below are deprecated events
+        METADATA_CACHE_MISS,
+        QUEUED_CACHE_MISS
+    }
+
+    public enum Status {
+        PENDING, PROCESSING, DONE, ERROR
+    }
+}
+
+class DateTimeSerializer extends StdSerializer<LocalDateTime> {
+
+    private final static DateTimeFormatter DATE_TIME_FORMATTER = DateTimeFormatter.ofPattern("yyyy/MM/dd HH:mm:ss.SSS");
+
+    protected DateTimeSerializer() {
+        super(LocalDateTime.class);
+    }
+
+    @Override
+    public void serialize(LocalDateTime value, JsonGenerator gen, SerializerProvider provider) throws IOException {
+        gen.writeString(value.format(DATE_TIME_FORMATTER));
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/domain/Image.java

@@ -0,0 +1,70 @@
+package tn.mnlr.vripper.jpa.domain;
+
+import com.fasterxml.jackson.annotation.JsonIgnore;
+import lombok.Getter;
+import lombok.NoArgsConstructor;
+import lombok.Setter;
+import lombok.ToString;
+import tn.mnlr.vripper.host.Host;
+import tn.mnlr.vripper.jpa.domain.enums.Status;
+
+import java.util.Objects;
+
+@Getter
+@Setter
+@ToString
+@NoArgsConstructor
+public class Image {
+
+    @JsonIgnore
+    protected Long id;
+
+    @JsonIgnore
+    private Host host;
+
+    private String url;
+
+    private int index;
+
+    private long current = 0;
+
+    private long total = 0;
+
+    private Status status;
+
+    private String postId;
+
+    @JsonIgnore
+    private Long postIdRef;
+
+    public Image(String postId, String url, Host host, int index) {
+        this.postId = postId;
+        this.url = url;
+        this.host = host;
+        this.index = index;
+        status = Status.STOPPED;
+    }
+
+
+    public void increase(int read) {
+        current += read;
+    }
+
+    public void init() {
+        current = 0;
+        status = Status.STOPPED;
+    }
+
+    @Override
+    public boolean equals(Object o) {
+        if (this == o) return true;
+        if (o == null || getClass() != o.getClass()) return false;
+        Image image = (Image) o;
+        return Objects.equals(url, image.url);
+    }
+
+    @Override
+    public int hashCode() {
+        return Objects.hash(url);
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/domain/Metadata.java

@@ -0,0 +1,31 @@
+package tn.mnlr.vripper.jpa.domain;
+
+import lombok.Getter;
+import lombok.NoArgsConstructor;
+import lombok.Setter;
+
+import java.util.Collections;
+import java.util.List;
+
+@Getter
+@Setter
+@NoArgsConstructor
+public class Metadata {
+
+    private Long postIdRef;
+
+    private String postId;
+
+    private String postedBy;
+
+    private List<String> resolvedNames = Collections.emptyList();
+
+    public static Metadata from(Metadata metadata) {
+        Metadata copy = new Metadata();
+        copy.postIdRef = metadata.postIdRef;
+        copy.postId = metadata.postId;
+        copy.postedBy = metadata.postedBy;
+        copy.resolvedNames = List.copyOf(metadata.resolvedNames);
+        return copy;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/domain/Post.java

@@ -0,0 +1,78 @@
+package tn.mnlr.vripper.jpa.domain;
+
+import com.fasterxml.jackson.annotation.JsonIgnore;
+import lombok.Getter;
+import lombok.NoArgsConstructor;
+import lombok.Setter;
+import lombok.ToString;
+import tn.mnlr.vripper.jpa.domain.enums.Status;
+
+import java.util.Collections;
+import java.util.Objects;
+import java.util.Set;
+
+@Getter
+@Setter
+@ToString
+@NoArgsConstructor
+public class Post {
+
+    @JsonIgnore
+    private Long id;
+
+    private Status status;
+
+    private String postId;
+
+    private String threadTitle;
+
+    private String threadId;
+
+    private String title;
+
+    private String url;
+
+    private int done;
+
+    private int total;
+
+    private Set<String> hosts;
+
+    private String forum;
+
+    @JsonIgnore
+    private String securityToken;
+
+    @JsonIgnore
+    private String downloadDirectory;
+
+    private boolean thanked;
+
+    private Set<String> previews = Collections.emptySet();
+
+    private Metadata metadata;
+
+    public Post(String title, String url, String postId, String threadId, String threadTitle, String forum, String securityToken) {
+        this.title = title;
+        this.url = url;
+        this.postId = postId;
+        this.forum = forum;
+        this.threadId = threadId;
+        this.threadTitle = threadTitle;
+        this.securityToken = securityToken;
+        status = Status.STOPPED;
+    }
+
+    @Override
+    public boolean equals(Object o) {
+        if (this == o) return true;
+        if (o == null || getClass() != o.getClass()) return false;
+        Post post = (Post) o;
+        return Objects.equals(postId, post.postId);
+    }
+
+    @Override
+    public int hashCode() {
+        return Objects.hash(postId);
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/domain/Queued.java

@@ -0,0 +1,57 @@
+package tn.mnlr.vripper.jpa.domain;
+
+import com.fasterxml.jackson.annotation.JsonIgnore;
+import lombok.Getter;
+import lombok.NoArgsConstructor;
+import lombok.Setter;
+import lombok.ToString;
+
+import java.util.Objects;
+
+@Getter
+@Setter
+@NoArgsConstructor
+@ToString
+public class Queued {
+
+    @JsonIgnore
+    private Long id;
+
+    private String link;
+
+    private String threadId;
+
+    private String postId;
+
+    private int total = 0;
+
+    private boolean loading = true;
+
+    public Queued(String link, String threadId, String postId) {
+        this();
+        this.link = link;
+        this.threadId = threadId;
+        this.postId = postId;
+    }
+
+    public void done() {
+        this.loading = false;
+    }
+
+    public void increment() {
+        this.total++;
+    }
+
+    @Override
+    public boolean equals(Object o) {
+        if (this == o) return true;
+        if (o == null || getClass() != o.getClass()) return false;
+        Queued that = (Queued) o;
+        return Objects.equals(threadId, that.threadId);
+    }
+
+    @Override
+    public int hashCode() {
+        return Objects.hash(threadId);
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/domain/enums/Status.java

@@ -0,0 +1,5 @@
+package tn.mnlr.vripper.jpa.domain.enums;
+
+public enum Status {
+    PENDING, DOWNLOADING, COMPLETE, ERROR, STOPPED, PARTIAL
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/repositories/IEventRepository.java

@@ -0,0 +1,21 @@
+package tn.mnlr.vripper.jpa.repositories;
+
+import tn.mnlr.vripper.jpa.domain.Event;
+
+import java.util.List;
+import java.util.Optional;
+
+public interface IEventRepository extends IRepository {
+
+    Event save(Event event);
+
+    Event update(Event event);
+
+    List<Event> findAll();
+
+    Optional<Event> findById(Long id);
+
+    void delete(Long id);
+
+    void deleteAll();
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/repositories/IImageRepository.java

@@ -0,0 +1,32 @@
+package tn.mnlr.vripper.jpa.repositories;
+
+import tn.mnlr.vripper.jpa.domain.Image;
+import tn.mnlr.vripper.jpa.domain.enums.Status;
+
+import java.util.List;
+import java.util.Optional;
+
+public interface IImageRepository extends IRepository {
+
+    Image save(Image image);
+
+    void deleteAllByPostId(String postId);
+
+    List<Image> findByPostId(String postId);
+
+    Integer countError();
+
+    List<Image> findByPostIdAndIsNotCompleted(String postId);
+
+    int stopByPostIdAndIsNotCompleted(String postId);
+
+    List<Image> findByPostIdAndIsError(String postId);
+
+    Optional<Image> findById(Long id);
+
+    int updateStatus(Status status, Long id);
+
+    int updateCurrent(long current, Long id);
+
+    int updateTotal(long total, Long id);
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/repositories/IMetadataRepository.java

@@ -0,0 +1,14 @@
+package tn.mnlr.vripper.jpa.repositories;
+
+import tn.mnlr.vripper.jpa.domain.Metadata;
+
+import java.util.Optional;
+
+public interface IMetadataRepository extends IRepository {
+
+    Metadata save(Metadata metadata);
+
+    Optional<Metadata> findByPostId(String postId);
+
+    int deleteByPostId(String postId);
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/repositories/IPostRepository.java

@@ -0,0 +1,36 @@
+package tn.mnlr.vripper.jpa.repositories;
+
+import tn.mnlr.vripper.jpa.domain.Post;
+import tn.mnlr.vripper.jpa.domain.enums.Status;
+
+import java.util.List;
+import java.util.Optional;
+
+public interface IPostRepository extends IRepository {
+
+    Post save(Post post);
+
+    Optional<Post> findByPostId(String postId);
+
+    Optional<Post> findById(Long id);
+
+    List<String> findCompleted();
+
+    List<Post> findAll();
+
+    boolean existByPostId(String postId);
+
+    int setDownloadingToStopped();
+
+    int deleteByPostId(String postId);
+
+    int updateStatus(Status status, Long id);
+
+    int updateDone(int done, Long id);
+
+    int updateFolderName(String postFolderName, Long id);
+
+    int updateTitle(String title, Long id);
+
+    int updateThanked(boolean thanked, Long id);
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/repositories/IQueuedRepository.java

@@ -0,0 +1,20 @@
+package tn.mnlr.vripper.jpa.repositories;
+
+import tn.mnlr.vripper.jpa.domain.Queued;
+
+import java.util.List;
+import java.util.Optional;
+
+public interface IQueuedRepository extends IRepository {
+    Queued save(Queued queued);
+
+    Optional<Queued> findByThreadId(String threadId);
+
+    List<Queued> findAll();
+
+    Optional<Queued> findById(Long id);
+
+    int deleteByThreadId(String threadId);
+
+    void deleteAll();
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/repositories/IRepository.java

@@ -0,0 +1,4 @@
+package tn.mnlr.vripper.jpa.repositories;
+
+public interface IRepository {
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/repositories/impl/EventRepository.java

@@ -0,0 +1,117 @@
+package tn.mnlr.vripper.jpa.repositories.impl;
+
+import lombok.extern.slf4j.Slf4j;
+import org.springframework.context.ApplicationEventPublisher;
+import org.springframework.context.ApplicationEventPublisherAware;
+import org.springframework.jdbc.core.JdbcTemplate;
+import org.springframework.lang.NonNull;
+import org.springframework.stereotype.Service;
+import tn.mnlr.vripper.event.EventRemoveEvent;
+import tn.mnlr.vripper.event.EventUpdateEvent;
+import tn.mnlr.vripper.jpa.domain.Event;
+import tn.mnlr.vripper.jpa.repositories.IEventRepository;
+import tn.mnlr.vripper.services.SettingsService;
+
+import java.time.format.DateTimeFormatter;
+import java.util.List;
+import java.util.Optional;
+
+@Service
+@Slf4j
+public class EventRepository implements IEventRepository, ApplicationEventPublisherAware {
+
+    private final JdbcTemplate jdbcTemplate;
+    private final SettingsService settingsService;
+    private ApplicationEventPublisher applicationEventPublisher;
+
+    public EventRepository(JdbcTemplate jdbcTemplate, SettingsService settingsService) {
+        this.jdbcTemplate = jdbcTemplate;
+        this.settingsService = settingsService;
+    }
+
+    private synchronized Long nextId() {
+        return jdbcTemplate.queryForObject(
+                "CALL NEXT VALUE FOR SEQ_EVENT",
+                Long.class);
+    }
+
+    @Override
+    public synchronized Event save(@NonNull Event event) {
+
+        int maxRecords = settingsService.getSettings().getMaxEventLog() - 1;
+
+        Long count = jdbcTemplate.queryForObject("SELECT COUNT(*) FROM EVENT", Long.class);
+        if (count > maxRecords) {
+            List<Long> idList = jdbcTemplate.queryForList("SELECT ID FROM EVENT ORDER BY TIME ASC LIMIT ?", Long.class, count - maxRecords);
+            idList.forEach(this::delete);
+        }
+
+        long id = nextId();
+        jdbcTemplate.update(
+                "INSERT INTO EVENT (ID, TYPE, STATUS, TIME, MESSAGE) VALUES (?,?,?,?,?)",
+                id,
+                event.getType().name(),
+                event.getStatus().name(),
+                event.getTime().format(DateTimeFormatter.ISO_LOCAL_DATE_TIME),
+                event.getMessage()
+        );
+        event.setId(id);
+        applicationEventPublisher.publishEvent(new EventUpdateEvent(EventRepository.class, id));
+        return event;
+    }
+
+    @Override
+    public Event update(@NonNull Event event) {
+        if (event.getId() == null) {
+            log.warn("Cannot update entity with null id");
+            return event;
+        }
+
+        jdbcTemplate.update(
+                "UPDATE EVENT SET TYPE = ?, STATUS = ?, TIME = ?, MESSAGE = ? WHERE ID = ?",
+                event.getType().name(),
+                event.getStatus().name(),
+                event.getTime().format(DateTimeFormatter.ISO_LOCAL_DATE_TIME),
+                event.getMessage(),
+                event.getId()
+        );
+        applicationEventPublisher.publishEvent(new EventUpdateEvent(EventRepository.class, event.getId()));
+        return event;
+    }
+
+    @Override
+    public Optional<Event> findById(Long id) {
+        List<Event> events = jdbcTemplate.query(
+                "SELECT * FROM EVENT WHERE ID = ?",
+                new EventRowMapper(),
+                id);
+        if (events.isEmpty()) {
+            return Optional.empty();
+        } else {
+            return Optional.of(events.get(0));
+        }
+    }
+
+    @Override
+    public List<Event> findAll() {
+        return jdbcTemplate.query(
+                "SELECT * FROM EVENT",
+                new EventRowMapper());
+    }
+
+    @Override
+    public void delete(Long id) {
+        jdbcTemplate.update("DELETE FROM EVENT WHERE ID = ?", id);
+        applicationEventPublisher.publishEvent(new EventRemoveEvent(EventRepository.class, id));
+    }
+
+    @Override
+    public void deleteAll() {
+        jdbcTemplate.update("DELETE FROM EVENT");
+    }
+
+    @Override
+    public void setApplicationEventPublisher(@NonNull ApplicationEventPublisher applicationEventPublisher) {
+        this.applicationEventPublisher = applicationEventPublisher;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/repositories/impl/EventRowMapper.java

@@ -0,0 +1,23 @@
+package tn.mnlr.vripper.jpa.repositories.impl;
+
+import org.springframework.jdbc.core.RowMapper;
+import tn.mnlr.vripper.jpa.domain.Event;
+
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.time.LocalDateTime;
+import java.time.format.DateTimeFormatter;
+
+public class EventRowMapper implements RowMapper<Event> {
+
+    @Override
+    public Event mapRow(ResultSet rs, int rowNum) throws SQLException {
+        Event event = new Event();
+        event.setId(rs.getLong("ID"));
+        event.setType(Event.Type.valueOf(rs.getString("TYPE")));
+        event.setStatus(Event.Status.valueOf(rs.getString("STATUS")));
+        event.setTime(LocalDateTime.parse(rs.getString("TIME"), DateTimeFormatter.ISO_LOCAL_DATE_TIME));
+        event.setMessage(rs.getString("MESSAGE"));
+        return event;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/repositories/impl/ImageRepository.java

@@ -0,0 +1,146 @@
+package tn.mnlr.vripper.jpa.repositories.impl;
+
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.context.ApplicationEventPublisher;
+import org.springframework.context.ApplicationEventPublisherAware;
+import org.springframework.jdbc.core.JdbcTemplate;
+import org.springframework.lang.NonNull;
+import org.springframework.stereotype.Service;
+import tn.mnlr.vripper.event.ImageUpdateEvent;
+import tn.mnlr.vripper.jpa.domain.Image;
+import tn.mnlr.vripper.jpa.domain.enums.Status;
+import tn.mnlr.vripper.jpa.repositories.IImageRepository;
+
+import java.util.List;
+import java.util.Optional;
+
+@Service
+public class ImageRepository implements IImageRepository, ApplicationEventPublisherAware {
+
+    private final JdbcTemplate jdbcTemplate;
+    private ApplicationEventPublisher applicationEventPublisher;
+
+    @Autowired
+    public ImageRepository(JdbcTemplate jdbcTemplate) {
+        this.jdbcTemplate = jdbcTemplate;
+    }
+
+    private synchronized Long nextId() {
+        return jdbcTemplate.queryForObject(
+                "CALL NEXT VALUE FOR SEQ_IMAGE",
+                Long.class);
+    }
+
+    @Override
+    public Image save(Image image) {
+        long id = nextId();
+        jdbcTemplate.update(
+                "INSERT INTO IMAGE (ID, CURRENT, HOST, INDEX, POST_ID, STATUS, TOTAL, URL, POST_ID_REF) VALUES (?,?,?,?,?,?,?,?,?)",
+                id,
+                image.getCurrent(),
+                image.getHost().getHost(),
+                image.getIndex(),
+                image.getPostId(),
+                image.getStatus().name(),
+                image.getTotal(),
+                image.getUrl(),
+                image.getPostIdRef()
+        );
+        image.setId(id);
+        applicationEventPublisher.publishEvent(new ImageUpdateEvent(ImageRepository.class, id));
+        return image;
+    }
+
+    @Override
+    public void deleteAllByPostId(String postId) {
+        jdbcTemplate.update("DELETE FROM IMAGE WHERE POST_ID = ?", postId);
+    }
+
+    @Override
+    public List<Image> findByPostId(String postId) {
+        return jdbcTemplate.query(
+                "SELECT * FROM IMAGE WHERE POST_ID = ?",
+                new ImageRowMapper(),
+                postId);
+    }
+
+    @Override
+    public Integer countError() {
+        return jdbcTemplate.queryForObject(
+                "SELECT COUNT(*) FROM IMAGE AS image WHERE image.STATUS = 'ERROR'",
+                Integer.class
+        );
+    }
+
+    @Override
+    public List<Image> findByPostIdAndIsNotCompleted(String postId) {
+        return jdbcTemplate.query(
+                "SELECT * FROM IMAGE AS image WHERE image.POST_ID = ? AND image.STATUS <> 'COMPLETE'",
+                new ImageRowMapper(),
+                postId);
+    }
+
+    @Override
+    public int stopByPostIdAndIsNotCompleted(String postId) {
+        return jdbcTemplate.update(
+                "UPDATE IMAGE AS image SET image.STATUS = 'STOPPED' WHERE image.POST_ID = ? AND image.STATUS <> 'COMPLETE'",
+                postId
+        );
+    }
+
+    @Override
+    public List<Image> findByPostIdAndIsError(String postId) {
+        return jdbcTemplate.query(
+                "SELECT * FROM IMAGE AS image WHERE image.POST_ID = ? AND image.STATUS = 'ERROR'",
+                new ImageRowMapper(),
+                postId);
+    }
+
+    @Override
+    public Optional<Image> findById(Long id) {
+        List<Image> images = jdbcTemplate.query(
+                "SELECT * FROM IMAGE AS image WHERE image.ID = ?",
+                new ImageRowMapper(),
+                id);
+        if (images.isEmpty()) {
+            return Optional.empty();
+        } else {
+            return Optional.of(images.get(0));
+        }
+    }
+
+    @Override
+    public int updateStatus(Status status, Long id) {
+        int mutationCount = jdbcTemplate.update(
+                "UPDATE IMAGE AS image SET image.STATUS = ? WHERE image.ID = ?",
+                status.name(), id
+        );
+        applicationEventPublisher.publishEvent(new ImageUpdateEvent(ImageRepository.class, id));
+        return mutationCount;
+    }
+
+    @Override
+    public int updateCurrent(long current, Long id) {
+        int mutationCount = jdbcTemplate.update(
+                "UPDATE IMAGE AS image SET image.CURRENT = ? WHERE image.ID = ?",
+                current, id
+        );
+        applicationEventPublisher.publishEvent(new ImageUpdateEvent(ImageRepository.class, id));
+        return mutationCount;
+    }
+
+    @Override
+    public int updateTotal(long total, Long id) {
+        int mutationCount = jdbcTemplate.update(
+                "UPDATE IMAGE AS image SET image.TOTAL = ? WHERE image.ID = ?",
+                total, id
+        );
+        applicationEventPublisher.publishEvent(new ImageUpdateEvent(ImageRepository.class, id));
+        return mutationCount;
+    }
+
+    @Override
+    public void setApplicationEventPublisher(@NonNull ApplicationEventPublisher applicationEventPublisher) {
+        this.applicationEventPublisher = applicationEventPublisher;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/repositories/impl/ImageRowMapper.java

@@ -0,0 +1,29 @@
+package tn.mnlr.vripper.jpa.repositories.impl;
+
+import org.springframework.jdbc.core.RowMapper;
+import tn.mnlr.vripper.SpringContext;
+import tn.mnlr.vripper.host.Host;
+import tn.mnlr.vripper.jpa.domain.Image;
+import tn.mnlr.vripper.jpa.domain.enums.Status;
+
+import java.sql.ResultSet;
+import java.sql.SQLException;
+
+public class ImageRowMapper implements RowMapper<Image> {
+
+    @Override
+    public Image mapRow(ResultSet rs, int rowNum) throws SQLException {
+        Image image = new Image();
+        image.setId(rs.getLong("ID"));
+        String host = rs.getString("HOST");
+        image.setHost(SpringContext.getBeansOfType(Host.class).values().stream().filter(e -> e.getHost().equals(host)).findAny().orElse(null));
+        image.setUrl(rs.getString("URL"));
+        image.setIndex(rs.getInt("INDEX"));
+        image.setCurrent(rs.getLong("CURRENT"));
+        image.setTotal(rs.getLong("TOTAL"));
+        image.setStatus(Status.valueOf(rs.getString("STATUS")));
+        image.setPostId(rs.getString("POST_ID"));
+        image.setPostIdRef(rs.getLong("POST_ID_REF"));
+        return image;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/repositories/impl/MetadataRepository.java

@@ -0,0 +1,65 @@
+package tn.mnlr.vripper.jpa.repositories.impl;
+
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.context.ApplicationEventPublisher;
+import org.springframework.context.ApplicationEventPublisherAware;
+import org.springframework.jdbc.core.JdbcTemplate;
+import org.springframework.stereotype.Service;
+import tn.mnlr.vripper.event.MetadataUpdateEvent;
+import tn.mnlr.vripper.jpa.domain.Metadata;
+import tn.mnlr.vripper.jpa.repositories.IMetadataRepository;
+
+import java.util.List;
+import java.util.Optional;
+
+@Service
+public class MetadataRepository implements IMetadataRepository, ApplicationEventPublisherAware {
+
+    private final JdbcTemplate jdbcTemplate;
+    private ApplicationEventPublisher applicationEventPublisher;
+
+    @Autowired
+    public MetadataRepository(JdbcTemplate jdbcTemplate) {
+        this.jdbcTemplate = jdbcTemplate;
+    }
+
+    @Override
+    public Metadata save(Metadata metadata) {
+        jdbcTemplate.update(
+                "INSERT INTO METADATA (POST_ID_REF, POST_ID, POSTED_BY, RESOLVED_NAMES) VALUES (?,?,?,?)",
+                metadata.getPostIdRef(),
+                metadata.getPostId(),
+                metadata.getPostedBy(),
+                String.join("%sep%", metadata.getResolvedNames())
+        );
+        applicationEventPublisher.publishEvent(new MetadataUpdateEvent(MetadataRepository.class, metadata.getPostIdRef()));
+        return metadata;
+    }
+
+    @Override
+    public Optional<Metadata> findByPostId(String postId) {
+        List<Metadata> metadata = jdbcTemplate.query(
+                "SELECT metadata.* FROM METADATA AS metadata WHERE metadata.POST_ID = ?",
+                new MetadataRowMapper(),
+                postId
+        );
+        if (metadata.isEmpty()) {
+            return Optional.empty();
+        } else {
+            return Optional.of(metadata.get(0));
+        }
+    }
+
+    @Override
+    public int deleteByPostId(String postId) {
+        return jdbcTemplate.update(
+                "DELETE FROM METADATA AS metadata WHERE metadata.POST_ID = ?",
+                postId
+        );
+    }
+
+    @Override
+    public void setApplicationEventPublisher(ApplicationEventPublisher applicationEventPublisher) {
+        this.applicationEventPublisher = applicationEventPublisher;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/repositories/impl/MetadataRowMapper.java

@@ -0,0 +1,24 @@
+package tn.mnlr.vripper.jpa.repositories.impl;
+
+import org.springframework.jdbc.core.RowMapper;
+import tn.mnlr.vripper.jpa.domain.Metadata;
+
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.util.List;
+
+public class MetadataRowMapper implements RowMapper<Metadata> {
+
+    @Override
+    public Metadata mapRow(ResultSet rs, int rowNum) throws SQLException {
+        Metadata metadata = new Metadata();
+        metadata.setPostIdRef(rs.getLong("POST_ID_REF"));
+        metadata.setPostId(rs.getString("POST_ID"));
+        metadata.setPostedBy(rs.getString("POSTED_BY"));
+        String resolvedNames = rs.getString("RESOLVED_NAMES");
+        if (resolvedNames != null && !resolvedNames.isBlank()) {
+            metadata.setResolvedNames(List.of(resolvedNames.split("%sep%")));
+        }
+        return metadata;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/repositories/impl/PostRepository.java

@@ -0,0 +1,188 @@
+package tn.mnlr.vripper.jpa.repositories.impl;
+
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.context.ApplicationEventPublisher;
+import org.springframework.context.ApplicationEventPublisherAware;
+import org.springframework.jdbc.core.JdbcTemplate;
+import org.springframework.lang.NonNull;
+import org.springframework.stereotype.Service;
+import tn.mnlr.vripper.event.PostRemoveEvent;
+import tn.mnlr.vripper.event.PostUpdateEvent;
+import tn.mnlr.vripper.jpa.domain.Post;
+import tn.mnlr.vripper.jpa.domain.enums.Status;
+import tn.mnlr.vripper.jpa.repositories.IPostRepository;
+
+import java.util.List;
+import java.util.Optional;
+
+@Service
+public class PostRepository implements IPostRepository, ApplicationEventPublisherAware {
+
+    private final JdbcTemplate jdbcTemplate;
+    private ApplicationEventPublisher applicationEventPublisher;
+
+    @Autowired
+    public PostRepository(JdbcTemplate jdbcTemplate) {
+        this.jdbcTemplate = jdbcTemplate;
+
+    }
+
+    private synchronized Long nextId() {
+        return jdbcTemplate.queryForObject(
+                "CALL NEXT VALUE FOR SEQ_POST",
+                Long.class);
+    }
+
+    @Override
+    public Post save(Post post) {
+        long id = nextId();
+        jdbcTemplate.update(
+                "INSERT INTO POST (ID, DONE, FORUM, HOSTS, POST_FOLDER_NAME, POST_ID, PREVIEWS, SECURITY_TOKEN, STATUS, THANKED, THREAD_ID, THREAD_TITLE, TITLE, TOTAL, URL) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)",
+                id,
+                post.getDone(),
+                post.getForum(),
+                String.join(";", post.getHosts()),
+                post.getDownloadDirectory(),
+                post.getPostId(),
+                String.join(";", post.getPreviews()),
+                post.getSecurityToken(),
+                post.getStatus().name(),
+                post.isThanked(),
+                post.getThreadId(),
+                post.getThreadTitle(),
+                post.getTitle(),
+                post.getTotal(),
+                post.getUrl()
+        );
+        post.setId(id);
+        applicationEventPublisher.publishEvent(new PostUpdateEvent(PostRepository.class, id));
+        return post;
+    }
+
+    @Override
+    public Optional<Post> findByPostId(String postId) {
+        List<Post> posts = jdbcTemplate.query(
+                "SELECT metadata.*,post.* FROM METADATA metadata FULL JOIN POST post ON metadata.POST_ID_REF = post.ID WHERE post.POST_ID = ?",
+                new PostRowMapper(),
+                postId);
+        if (posts.isEmpty()) {
+            return Optional.empty();
+        } else {
+            return Optional.of(posts.get(0));
+        }
+    }
+
+    @Override
+    public List<String> findCompleted() {
+        return jdbcTemplate.query(
+                "SELECT POST_ID FROM POST AS post WHERE status = 'COMPLETE' AND done >= total",
+                ((rs, rowNum) -> rs.getString("POST_ID"))
+        );
+    }
+
+    @Override
+    public Optional<Post> findById(Long id) {
+        List<Post> posts = jdbcTemplate.query(
+                "SELECT metadata.*,post.* FROM METADATA metadata FULL JOIN POST post ON metadata.POST_ID_REF = post.ID WHERE post.ID = ?",
+                new PostRowMapper(),
+                id);
+        if (posts.isEmpty()) {
+            return Optional.empty();
+        } else {
+            return Optional.of(posts.get(0));
+        }
+    }
+
+    @Override
+    public List<Post> findAll() {
+        return jdbcTemplate.query(
+                "SELECT metadata.*,post.* FROM METADATA metadata FULL JOIN POST post ON metadata.POST_ID_REF = post.ID",
+                new PostRowMapper()
+        );
+    }
+
+    @Override
+    public boolean existByPostId(String postId) {
+        Integer count = jdbcTemplate.queryForObject(
+                "SELECT COUNT(*) FROM POST AS post WHERE post.POST_ID = ?",
+                Integer.class,
+                postId);
+        if (count == null) {
+            return false;
+        } else {
+            return count > 0;
+        }
+    }
+
+    @Override
+    public int setDownloadingToStopped() {
+        return jdbcTemplate.update(
+                "UPDATE POST AS post SET post.STATUS = 'STOPPED' WHERE post.STATUS = 'DOWNLOADING' OR post.STATUS = 'PARTIAL' OR post.STATUS = 'PENDING'"
+        );
+    }
+
+    @Override
+    public int deleteByPostId(String postId) {
+        int mutationCount = jdbcTemplate.update(
+                "DELETE FROM POST AS post WHERE post.POST_ID = ?",
+                postId
+        );
+        applicationEventPublisher.publishEvent(new PostRemoveEvent(PostRepository.class, postId));
+        return mutationCount;
+    }
+
+    @Override
+    public int updateStatus(Status status, Long id) {
+        int mutationCount = jdbcTemplate.update(
+                "UPDATE POST AS post SET post.STATUS = ? WHERE post.ID = ?",
+                status.name(), id
+        );
+        applicationEventPublisher.publishEvent(new PostUpdateEvent(PostRepository.class, id));
+        return mutationCount;
+    }
+
+    @Override
+    public int updateDone(int done, Long id) {
+        int mutationCount = jdbcTemplate.update(
+                "UPDATE POST AS post SET post.DONE = ? WHERE post.ID = ?",
+                done, id
+        );
+        applicationEventPublisher.publishEvent(new PostUpdateEvent(PostRepository.class, id));
+        return mutationCount;
+    }
+
+    @Override
+    public int updateFolderName(String postFolderName, Long id) {
+        int mutationCount = jdbcTemplate.update(
+                "UPDATE POST AS post SET post.POST_FOLDER_NAME = ? WHERE post.ID = ?",
+                postFolderName, id
+        );
+        applicationEventPublisher.publishEvent(new PostUpdateEvent(PostRepository.class, id));
+        return mutationCount;
+    }
+
+    @Override
+    public int updateTitle(String title, Long id) {
+        int mutationCount = jdbcTemplate.update(
+                "UPDATE POST AS post SET post.TITLE = ? WHERE post.ID = ?",
+                title, id
+        );
+        applicationEventPublisher.publishEvent(new PostUpdateEvent(PostRepository.class, id));
+        return mutationCount;
+    }
+
+    @Override
+    public int updateThanked(boolean thanked, Long id) {
+        int mutationCount = jdbcTemplate.update(
+                "UPDATE POST AS post SET post.THANKED = ? WHERE post.ID = ?",
+                thanked, id
+        );
+        applicationEventPublisher.publishEvent(new PostUpdateEvent(PostRepository.class, id));
+        return mutationCount;
+    }
+
+    @Override
+    public void setApplicationEventPublisher(@NonNull ApplicationEventPublisher applicationEventPublisher) {
+        this.applicationEventPublisher = applicationEventPublisher;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/repositories/impl/PostRowMapper.java

@@ -0,0 +1,55 @@
+package tn.mnlr.vripper.jpa.repositories.impl;
+
+import org.springframework.jdbc.core.RowMapper;
+import tn.mnlr.vripper.jpa.domain.Metadata;
+import tn.mnlr.vripper.jpa.domain.Post;
+import tn.mnlr.vripper.jpa.domain.enums.Status;
+
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.util.List;
+import java.util.Set;
+
+public class PostRowMapper implements RowMapper<Post> {
+
+    private static final String DELIMITER = ";";
+
+    @Override
+    public Post mapRow(ResultSet rs, int rowNum) throws SQLException {
+
+        Post post = new Post();
+        post.setId(rs.getLong("post.ID"));
+        post.setStatus(Status.valueOf(rs.getString("post.STATUS")));
+        post.setPostId(rs.getString("post.POST_ID"));
+        post.setThreadTitle(rs.getString("post.THREAD_TITLE"));
+        post.setThreadId(rs.getString("post.THREAD_ID"));
+        post.setTitle(rs.getString("post.TITLE"));
+        post.setUrl(rs.getString("post.URL"));
+        post.setDone(rs.getInt("post.DONE"));
+        post.setTotal(rs.getInt("post.TOTAL"));
+        post.setHosts(Set.of(rs.getString("post.HOSTS").split(DELIMITER)));
+        post.setForum(rs.getString("post.FORUM"));
+        post.setSecurityToken(rs.getString("post.SECURITY_TOKEN"));
+        post.setDownloadDirectory(rs.getString("post.POST_FOLDER_NAME"));
+        post.setThanked(rs.getBoolean("post.THANKED"));
+        String previews;
+        if ((previews = rs.getString("post.PREVIEWS")) != null) {
+            post.setPreviews(Set.of(previews.split(DELIMITER)));
+        }
+
+        Long metadataId = rs.getLong("metadata.POST_ID_REF");
+        if (!rs.wasNull()) {
+            Metadata metadata = new Metadata();
+            metadata.setPostIdRef(metadataId);
+            metadata.setPostId(rs.getString("metadata.POST_ID"));
+            String resolvedNames = rs.getString("metadata.RESOLVED_NAMES");
+            if (resolvedNames != null && !resolvedNames.isBlank()) {
+                metadata.setResolvedNames(List.of(resolvedNames.split("%sep%")));
+            }
+            metadata.setPostedBy(rs.getString("metadata.POSTED_BY"));
+            post.setMetadata(metadata);
+        }
+
+        return post;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/repositories/impl/QueuedRepository.java

@@ -0,0 +1,105 @@
+package tn.mnlr.vripper.jpa.repositories.impl;
+
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.context.ApplicationEventPublisher;
+import org.springframework.context.ApplicationEventPublisherAware;
+import org.springframework.jdbc.core.JdbcTemplate;
+import org.springframework.stereotype.Service;
+import tn.mnlr.vripper.event.QueuedRemoveEvent;
+import tn.mnlr.vripper.event.QueuedUpdateEvent;
+import tn.mnlr.vripper.jpa.domain.Queued;
+import tn.mnlr.vripper.jpa.repositories.IQueuedRepository;
+
+import java.util.List;
+import java.util.Optional;
+
+@Service
+public class QueuedRepository implements IQueuedRepository, ApplicationEventPublisherAware {
+
+    private final JdbcTemplate jdbcTemplate;
+    private ApplicationEventPublisher applicationEventPublisher;
+
+    @Autowired
+    public QueuedRepository(JdbcTemplate jdbcTemplate) {
+        this.jdbcTemplate = jdbcTemplate;
+    }
+
+    private synchronized Long nextId() {
+        return jdbcTemplate.queryForObject(
+                "CALL NEXT VALUE FOR SEQ_QUEUED",
+                Long.class);
+    }
+
+    @Override
+    public Queued save(Queued queued) {
+        long id = nextId();
+        jdbcTemplate.update(
+                "INSERT INTO QUEUED (ID, TOTAL, LINK, LOADING, POST_ID, THREAD_ID) values (?,?,?,?,?,?)",
+                id,
+                queued.getTotal(),
+                queued.getLink(),
+                queued.isLoading(),
+                queued.getPostId(),
+                queued.getThreadId()
+        );
+        queued.setId(id);
+        applicationEventPublisher.publishEvent(new QueuedUpdateEvent(QueuedRepository.class, id));
+        return queued;
+    }
+
+    @Override
+    public Optional<Queued> findByThreadId(String threadId) {
+        List<Queued> queuedList = jdbcTemplate.query(
+                "SELECT * FROM QUEUED AS queued WHERE queued.THREAD_ID = ?",
+                new Object[]{threadId},
+                new QueuedRowMapper()
+        );
+        if (queuedList.isEmpty()) {
+            return Optional.empty();
+        } else {
+            return Optional.of(queuedList.get(0));
+        }
+    }
+
+    @Override
+    public List<Queued> findAll() {
+        return jdbcTemplate.query(
+                "SELECT * FROM QUEUED",
+                new QueuedRowMapper()
+        );
+    }
+
+    @Override
+    public Optional<Queued> findById(Long id) {
+        List<Queued> queuedList = jdbcTemplate.query(
+                "SELECT * FROM QUEUED AS queued WHERE queued.ID = ?",
+                new Object[]{id},
+                new QueuedRowMapper()
+        );
+        if (queuedList.isEmpty()) {
+            return Optional.empty();
+        } else {
+            return Optional.of(queuedList.get(0));
+        }
+    }
+
+    @Override
+    public int deleteByThreadId(String threadId) {
+        int mutationCount = jdbcTemplate.update(
+                "DELETE FROM QUEUED AS queued WHERE THREAD_ID = ?",
+                threadId
+        );
+        applicationEventPublisher.publishEvent(new QueuedRemoveEvent(QueuedRepository.class, threadId));
+        return mutationCount;
+    }
+
+    @Override
+    public void deleteAll() {
+        jdbcTemplate.update("DELETE FROM QUEUED");
+    }
+
+    @Override
+    public void setApplicationEventPublisher(ApplicationEventPublisher applicationEventPublisher) {
+        this.applicationEventPublisher = applicationEventPublisher;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/jpa/repositories/impl/QueuedRowMapper.java

@@ -0,0 +1,22 @@
+package tn.mnlr.vripper.jpa.repositories.impl;
+
+import org.springframework.jdbc.core.RowMapper;
+import tn.mnlr.vripper.jpa.domain.Queued;
+
+import java.sql.ResultSet;
+import java.sql.SQLException;
+
+public class QueuedRowMapper implements RowMapper<Queued> {
+
+    @Override
+    public Queued mapRow(ResultSet rs, int rowNum) throws SQLException {
+        Queued queued = new Queued();
+        queued.setId(rs.getLong("ID"));
+        queued.setLink(rs.getString("LINK"));
+        queued.setThreadId(rs.getString("THREAD_ID"));
+        queued.setPostId(rs.getString("POST_ID"));
+        queued.setTotal(rs.getInt("TOTAL"));
+        queued.setLoading(rs.getBoolean("LOADING"));
+        return queued;
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/listener/DataEventListener.java

@@ -0,0 +1,7 @@
+package tn.mnlr.vripper.listener;
+
+import reactor.core.publisher.Flux;
+
+public interface DataEventListener<T> {
+    Flux<T> getDataFlux();
+}

vripper-server/src/main/java/tn/mnlr/vripper/listener/EmitHandler.java

@@ -0,0 +1,7 @@
+package tn.mnlr.vripper.listener;
+
+import reactor.core.publisher.Sinks.EmitFailureHandler;
+
+public class EmitHandler {
+    public static final EmitFailureHandler RETRY = (signalType, emitResult) -> true;
+}

vripper-server/src/main/java/tn/mnlr/vripper/listener/EventRemoveEventListener.java

@@ -0,0 +1,32 @@
+package tn.mnlr.vripper.listener;
+
+import lombok.extern.slf4j.Slf4j;
+import org.springframework.context.ApplicationListener;
+import org.springframework.stereotype.Component;
+import reactor.core.publisher.Flux;
+import reactor.core.publisher.Sinks;
+import tn.mnlr.vripper.event.EventRemoveEvent;
+
+import javax.annotation.PreDestroy;
+
+@Component
+@Slf4j
+public class EventRemoveEventListener implements ApplicationListener<EventRemoveEvent>, DataEventListener<EventRemoveEvent> {
+
+    private final Sinks.Many<EventRemoveEvent> sink = Sinks.many().multicast().onBackpressureBuffer();
+
+    @Override
+    public void onApplicationEvent(EventRemoveEvent event) {
+        sink.emitNext(event, EmitHandler.RETRY);
+    }
+
+    @Override
+    public Flux<EventRemoveEvent> getDataFlux() {
+        return sink.asFlux();
+    }
+
+    @PreDestroy
+    private void destroy() {
+        sink.emitComplete(Sinks.EmitFailureHandler.FAIL_FAST);
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/listener/EventUpdateEventListener.java

@@ -0,0 +1,33 @@
+package tn.mnlr.vripper.listener;
+
+import lombok.extern.slf4j.Slf4j;
+import org.springframework.context.ApplicationListener;
+import org.springframework.lang.NonNull;
+import org.springframework.stereotype.Component;
+import reactor.core.publisher.Flux;
+import reactor.core.publisher.Sinks;
+import tn.mnlr.vripper.event.EventUpdateEvent;
+
+import javax.annotation.PreDestroy;
+
+@Component
+@Slf4j
+public class EventUpdateEventListener implements ApplicationListener<EventUpdateEvent>, DataEventListener<EventUpdateEvent> {
+
+    private final Sinks.Many<EventUpdateEvent> sink = Sinks.many().multicast().onBackpressureBuffer();
+
+    @Override
+    public void onApplicationEvent(@NonNull EventUpdateEvent event) {
+        sink.emitNext(event, EmitHandler.RETRY);
+    }
+
+    @Override
+    public Flux<EventUpdateEvent> getDataFlux() {
+        return sink.asFlux();
+    }
+
+    @PreDestroy
+    private void destroy() {
+        sink.emitComplete(Sinks.EmitFailureHandler.FAIL_FAST);
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/listener/ImageUpdateEventListener.java

@@ -0,0 +1,32 @@
+package tn.mnlr.vripper.listener;
+
+import lombok.extern.slf4j.Slf4j;
+import org.springframework.context.ApplicationListener;
+import org.springframework.stereotype.Component;
+import reactor.core.publisher.Flux;
+import reactor.core.publisher.Sinks;
+import tn.mnlr.vripper.event.ImageUpdateEvent;
+
+import javax.annotation.PreDestroy;
+
+@Component
+@Slf4j
+public class ImageUpdateEventListener implements ApplicationListener<ImageUpdateEvent>, DataEventListener<ImageUpdateEvent> {
+
+    private final Sinks.Many<ImageUpdateEvent> sink = Sinks.many().multicast().onBackpressureBuffer();
+
+    @Override
+    public void onApplicationEvent(ImageUpdateEvent event) {
+        sink.emitNext(event, EmitHandler.RETRY);
+    }
+
+    @Override
+    public Flux<ImageUpdateEvent> getDataFlux() {
+        return sink.asFlux();
+    }
+
+    @PreDestroy
+    private void destroy() {
+        sink.emitComplete(Sinks.EmitFailureHandler.FAIL_FAST);
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/listener/MetadataUpdateEventListener.java

@@ -0,0 +1,32 @@
+package tn.mnlr.vripper.listener;
+
+import lombok.extern.slf4j.Slf4j;
+import org.springframework.context.ApplicationListener;
+import org.springframework.stereotype.Component;
+import reactor.core.publisher.Flux;
+import reactor.core.publisher.Sinks;
+import tn.mnlr.vripper.event.MetadataUpdateEvent;
+
+import javax.annotation.PreDestroy;
+
+@Component
+@Slf4j
+public class MetadataUpdateEventListener implements ApplicationListener<MetadataUpdateEvent>, DataEventListener<MetadataUpdateEvent> {
+
+    private final Sinks.Many<MetadataUpdateEvent> sink = Sinks.many().multicast().onBackpressureBuffer();
+
+    @Override
+    public void onApplicationEvent(MetadataUpdateEvent event) {
+        sink.emitNext(event, (signalType, emitResult) -> true);
+    }
+
+    @Override
+    public Flux<MetadataUpdateEvent> getDataFlux() {
+        return sink.asFlux();
+    }
+
+    @PreDestroy
+    private void destroy() {
+        sink.emitComplete(Sinks.EmitFailureHandler.FAIL_FAST);
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/listener/PostRemoveEventListener.java

@@ -0,0 +1,32 @@
+package tn.mnlr.vripper.listener;
+
+import lombok.extern.slf4j.Slf4j;
+import org.springframework.context.ApplicationListener;
+import org.springframework.stereotype.Component;
+import reactor.core.publisher.Flux;
+import reactor.core.publisher.Sinks;
+import tn.mnlr.vripper.event.PostRemoveEvent;
+
+import javax.annotation.PreDestroy;
+
+@Component
+@Slf4j
+public class PostRemoveEventListener implements ApplicationListener<PostRemoveEvent>, DataEventListener<PostRemoveEvent> {
+
+    private final Sinks.Many<PostRemoveEvent> sink = Sinks.many().multicast().onBackpressureBuffer();
+
+    @Override
+    public void onApplicationEvent(PostRemoveEvent event) {
+        sink.emitNext(event, EmitHandler.RETRY);
+    }
+
+    @Override
+    public Flux<PostRemoveEvent> getDataFlux() {
+        return sink.asFlux();
+    }
+
+    @PreDestroy
+    private void destroy() {
+        sink.emitComplete(Sinks.EmitFailureHandler.FAIL_FAST);
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/listener/PostUpdateEventListener.java

@@ -0,0 +1,32 @@
+package tn.mnlr.vripper.listener;
+
+import lombok.extern.slf4j.Slf4j;
+import org.springframework.context.ApplicationListener;
+import org.springframework.stereotype.Component;
+import reactor.core.publisher.Flux;
+import reactor.core.publisher.Sinks;
+import tn.mnlr.vripper.event.PostUpdateEvent;
+
+import javax.annotation.PreDestroy;
+
+@Component
+@Slf4j
+public class PostUpdateEventListener implements ApplicationListener<PostUpdateEvent>, DataEventListener<PostUpdateEvent> {
+
+    private final Sinks.Many<PostUpdateEvent> sink = Sinks.many().multicast().onBackpressureBuffer();
+
+    @Override
+    public void onApplicationEvent(PostUpdateEvent event) {
+        sink.emitNext(event, EmitHandler.RETRY);
+    }
+
+    @Override
+    public Flux<PostUpdateEvent> getDataFlux() {
+        return sink.asFlux();
+    }
+
+    @PreDestroy
+    private void destroy() {
+        sink.emitComplete(Sinks.EmitFailureHandler.FAIL_FAST);
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/listener/QueuedRemoveEventListener.java

@@ -0,0 +1,32 @@
+package tn.mnlr.vripper.listener;
+
+import lombok.extern.slf4j.Slf4j;
+import org.springframework.context.ApplicationListener;
+import org.springframework.stereotype.Component;
+import reactor.core.publisher.Flux;
+import reactor.core.publisher.Sinks;
+import tn.mnlr.vripper.event.QueuedRemoveEvent;
+
+import javax.annotation.PreDestroy;
+
+@Component
+@Slf4j
+public class QueuedRemoveEventListener implements ApplicationListener<QueuedRemoveEvent>, DataEventListener<QueuedRemoveEvent> {
+
+    private final Sinks.Many<QueuedRemoveEvent> sink = Sinks.many().multicast().onBackpressureBuffer();
+
+    @Override
+    public void onApplicationEvent(QueuedRemoveEvent event) {
+        sink.emitNext(event, (signalType, emitResult) -> true);
+    }
+
+    @Override
+    public Flux<QueuedRemoveEvent> getDataFlux() {
+        return sink.asFlux();
+    }
+
+    @PreDestroy
+    private void destroy() {
+        sink.emitComplete(Sinks.EmitFailureHandler.FAIL_FAST);
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/listener/QueuedUpdateEventListener.java

@@ -0,0 +1,32 @@
+package tn.mnlr.vripper.listener;
+
+import lombok.extern.slf4j.Slf4j;
+import org.springframework.context.ApplicationListener;
+import org.springframework.stereotype.Component;
+import reactor.core.publisher.Flux;
+import reactor.core.publisher.Sinks;
+import tn.mnlr.vripper.event.QueuedUpdateEvent;
+
+import javax.annotation.PreDestroy;
+
+@Component
+@Slf4j
+public class QueuedUpdateEventListener implements ApplicationListener<QueuedUpdateEvent>, DataEventListener<QueuedUpdateEvent> {
+
+    private final Sinks.Many<QueuedUpdateEvent> sink = Sinks.many().multicast().onBackpressureBuffer();
+
+    @Override
+    public void onApplicationEvent(QueuedUpdateEvent event) {
+        sink.emitNext(event, (signalType, emitResult) -> true);
+    }
+
+    @Override
+    public Flux<QueuedUpdateEvent> getDataFlux() {
+        return sink.asFlux();
+    }
+
+    @PreDestroy
+    private void destroy() {
+        sink.emitComplete(Sinks.EmitFailureHandler.FAIL_FAST);
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/q/DownloadJob.java

@@ -1,48 +0,0 @@
-package tn.mnlr.vripper.q;
-
-import lombok.Getter;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import tn.mnlr.vripper.entities.Image;
-
-import java.util.Objects;
-import java.util.concurrent.Callable;
-
-public class DownloadJob implements Callable<Image> {
-
-    private static final Logger logger = LoggerFactory.getLogger(DownloadJob.class);
-
-    @Getter
-    private final Image image;
-
-    @Getter
-    private final ImageFileData imageFileData = new ImageFileData();
-
-    DownloadJob(Image image) {
-        this.image = image;
-    }
-
-    @Override
-    public Image call() throws Exception {
-
-        logger.debug(String.format("Starting downloading %s", image.getUrl()));
-        image.setStatus(Image.Status.DOWNLOADING);
-        image.setCurrent(0);
-        image.getHost().download(image, imageFileData);
-        image.setStatus(Image.Status.COMPLETE);
-        return image;
-    }
-
-    @Override
-    public boolean equals(Object o) {
-        if (this == o) return true;
-        if (o == null || getClass() != o.getClass()) return false;
-        DownloadJob that = (DownloadJob) o;
-        return image.equals(that.image);
-    }
-
-    @Override
-    public int hashCode() {
-        return Objects.hash(image);
-    }
-}

vripper-server/src/main/java/tn/mnlr/vripper/q/DownloadQ.java

@@ -1,83 +0,0 @@
-package tn.mnlr.vripper.q;
-
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import org.springframework.beans.factory.annotation.Autowired;
-import org.springframework.stereotype.Service;
-import tn.mnlr.vripper.entities.Image;
-import tn.mnlr.vripper.entities.Post;
-import tn.mnlr.vripper.host.Host;
-import tn.mnlr.vripper.services.AppSettingsService;
-import tn.mnlr.vripper.services.AppStateService;
-
-import javax.annotation.PostConstruct;
-import java.util.ArrayList;
-import java.util.Iterator;
-import java.util.List;
-import java.util.Map;
-import java.util.concurrent.BlockingDeque;
-import java.util.concurrent.ConcurrentHashMap;
-import java.util.concurrent.LinkedBlockingDeque;
-
-@Service
-public class DownloadQ {
-
-    private static final Logger logger = LoggerFactory.getLogger(DownloadQ.class);
-
-    @Autowired
-    private AppStateService appStateService;
-
-    private final ConcurrentHashMap<Host, BlockingDeque<DownloadJob>> downloadQ = new ConcurrentHashMap<>();
-    @Autowired
-    private AppSettingsService appSettingsService;
-    @Autowired
-    private List<Host> hosts;
-
-    @PostConstruct
-    private void init() {
-        hosts.forEach(host -> downloadQ.put(host, new LinkedBlockingDeque<>()));
-    }
-
-    public void put(Image image) throws InterruptedException {
-        logger.debug(String.format("Enqueuing a job for %s", image.getUrl()));
-        image.init();
-        DownloadJob downloadJob = new DownloadJob(image);
-        downloadQ.get(downloadJob.getImage().getHost()).putLast(downloadJob);
-        appStateService.newDownloadJob(downloadJob);
-    }
-
-    public void remove(final DownloadJob downloadJob) {
-        downloadQ.get(downloadJob.getImage().getHost()).remove(downloadJob);
-    }
-
-    public List<DownloadJob> peek() {
-        List<DownloadJob> downloadJobs = new ArrayList<>();
-        if (hosts.size() == 0) {
-            return downloadJobs;
-        }
-        for (Host host : hosts) {
-            Iterator<DownloadJob> it = downloadQ.get(host).iterator();
-            for (int i = 0; i < appSettingsService.getMaxThreads(); i++) {
-                DownloadJob downloadJob = it.hasNext() ? it.next() : null;
-                if (downloadJob != null) {
-                    downloadJobs.add(downloadJob);
-                }
-            }
-        }
-        return downloadJobs;
-    }
-
-    public void enqueue(Post post) throws InterruptedException {
-        for (Image image : post.getImages()) {
-            put(image);
-        }
-    }
-
-    public int size() {
-        return downloadQ.values().stream().mapToInt(BlockingDeque::size).sum();
-    }
-
-    public Iterable<? extends Map.Entry<Host, BlockingDeque<DownloadJob>>> entries() {
-        return downloadQ.entrySet();
-    }
-}

vripper-server/src/main/java/tn/mnlr/vripper/q/ExecutionService.java

@@ -1,281 +0,0 @@
-package tn.mnlr.vripper.q;
-
-import net.jodah.failsafe.Failsafe;
-import net.jodah.failsafe.FailsafeException;
-import net.jodah.failsafe.RetryPolicy;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import org.springframework.beans.factory.annotation.Autowired;
-import org.springframework.stereotype.Service;
-import tn.mnlr.vripper.VripperApplication;
-import tn.mnlr.vripper.entities.Image;
-import tn.mnlr.vripper.entities.Post;
-import tn.mnlr.vripper.host.Host;
-import tn.mnlr.vripper.services.AppSettingsService;
-import tn.mnlr.vripper.services.AppStateService;
-import tn.mnlr.vripper.services.ThumbnailGenerator;
-
-import javax.annotation.PostConstruct;
-import javax.annotation.PreDestroy;
-import java.time.Duration;
-import java.time.temporal.ChronoUnit;
-import java.util.*;
-import java.util.concurrent.*;
-import java.util.concurrent.atomic.AtomicInteger;
-import java.util.stream.Collectors;
-
-@Service
-public class ExecutionService {
-
-    private static final Logger logger = LoggerFactory.getLogger(ExecutionService.class);
-
-    private static final List<Post.Status> FINISHED = Arrays.asList(Post.Status.ERROR, Post.Status.COMPLETE, Post.Status.STOPPED);
-
-    @Autowired
-    private DownloadQ downloadQ;
-
-    @Autowired
-    private AppSettingsService settings;
-
-    @Autowired
-    private AppStateService appStateService;
-
-    @Autowired
-    private ThumbnailGenerator thumbnailGenerator;
-
-    @Autowired
-    private AppSettingsService appSettingsService;
-    private final ConcurrentHashMap<Host, AtomicInteger> threadCount = new ConcurrentHashMap<>();
-    private boolean notPauseQ = true;
-    private ExecutorService executor = Executors.newFixedThreadPool(20);
-
-    private Thread executionThread;
-
-    BlockingQueue<DownloadJob> queue = new LinkedBlockingQueue<>();
-
-    private RetryPolicy<Object> retryPolicy;
-
-    private List<DownloadJob> running = Collections.synchronizedList(new ArrayList<>());
-
-    private Map<String, Future<?>> futures = new ConcurrentHashMap<>();
-    private Thread pollThread;
-
-    @PostConstruct
-    private void init() {
-
-        retryPolicy = new RetryPolicy<>()
-                .handleIf(e -> !(e instanceof InterruptedException))
-                .withBackoff(1, 10, ChronoUnit.SECONDS)
-                .withMaxDuration(Duration.of(10, ChronoUnit.SECONDS))
-                .withMaxAttempts(3)
-                .abortOn(InterruptedException.class)
-                .onFailedAttempt(e -> logger.warn(String.format("#%d tries failed", e.getAttemptCount()), e.getLastFailure()));
-
-        executionThread = new Thread(this::start, "Executor thread");
-        pollThread = new Thread(this::poll, "Polling thread");
-        pollThread.start();
-        executionThread.start();
-    }
-
-    @PreDestroy
-    private void destroy() throws InterruptedException {
-        logger.info("Shutting down ExecutionService");
-        executionThread.interrupt();
-        executor.shutdown();
-        appStateService.getCurrentPosts().keySet().forEach(p -> {
-            logger.debug(String.format("Stopping download jobs for %s", p));
-            this.stopRunning(p);
-        });
-        executor.awaitTermination(10, TimeUnit.SECONDS);
-    }
-
-    public void stopRunning(String postId) {
-        List<DownloadJob> data = running
-                .stream()
-                .filter(e -> e.getImage().getPostId().equals(postId))
-                .peek(e -> e.getImage().setStatus(Image.Status.STOPPED))
-                .collect(Collectors.toList());
-        logger.debug(String.format("Interrupting %d jobs for post id %s", data.size(), postId));
-
-        data.forEach(e -> {
-            futures.get(e.getImage().getUrl()).cancel(true);
-            if (e.getImageFileData().getImageRequest() != null) {
-                e.getImageFileData().getImageRequest().abort();
-            }
-            e.getImage().cleanup();
-        });
-    }
-
-
-    public synchronized void stopAll() {
-        appStateService.getCurrentPosts().values().stream().map(Post::getPostId).forEach(this::stop);
-    }
-
-    public synchronized void restartAll() throws InterruptedException {
-        for (Post post : appStateService.getCurrentPosts().values()) {
-            String postId = post.getPostId();
-            restart(postId);
-        }
-    }
-
-    public synchronized void restart(String postId) throws InterruptedException {
-        if (appStateService.getRunningPosts().get(postId) != null && appStateService.getRunningPosts().get(postId).get() > 0) {
-            logger.warn(String.format("Cannot restart, jobs are currently running for post id %s", postId));
-            return;
-        }
-        List<Image> images = appStateService.getPost(postId)
-                .getImages()
-                .stream()
-                .filter(e -> !e.getStatus().equals(Image.Status.COMPLETE))
-                .collect(Collectors.toList());
-        if (images.isEmpty()) {
-            return;
-        }
-        appStateService.getPost(postId).setStatus(Post.Status.PENDING);
-        logger.debug(String.format("Restarting %d jobs for post id %s", images.size(), postId));
-        for (Image image : images) {
-            downloadQ.put(image);
-        }
-    }
-
-    private synchronized void removeScheduled(Image image) {
-        image.setStatus(Image.Status.STOPPED);
-        logger.debug(String.format("Removing scheduled job for %s", image.getUrl()));
-
-        boolean removed = false;
-        main:
-        for (Map.Entry<Host, BlockingDeque<DownloadJob>> entry : downloadQ.entries()) {
-            Iterator<DownloadJob> iterator = entry.getValue().iterator();
-            while (iterator.hasNext()) {
-                DownloadJob next = iterator.next();
-                if (next.getImage().getPostId().equals(image.getPostId())) {
-                    iterator.remove();
-                    appStateService.doneDownloadJob(image);
-                    logger.debug(String.format("Scheduled job for %s is removed", image.getUrl()));
-                    removed = true;
-                    break main;
-                }
-            }
-        }
-
-        if (!removed) {
-            logger.debug(String.format("Job for %s does not exist", image.getUrl()));
-        }
-
-        image.cleanup();
-    }
-
-    private synchronized void removeRunning(String postId) {
-        logger.debug(String.format("Interrupting running jobs for post id %s", postId));
-        stopRunning(postId);
-    }
-
-    public synchronized void stop(String postId) {
-        try {
-            if (FINISHED.contains(appStateService.getPost(postId).getStatus())) {
-                return;
-            }
-            notPauseQ = false;
-            appStateService.getPost(postId).setStatus(Post.Status.STOPPED);
-            List<Image> images = appStateService.getPost(postId)
-                    .getImages()
-                    .stream()
-                    .filter(e -> !e.getStatus().equals(Image.Status.COMPLETE))
-                    .collect(Collectors.toList());
-            if (images.isEmpty()) {
-                return;
-            }
-            logger.debug(String.format("Stopping %d jobs for post id %s", images.size(), postId));
-            images.forEach(this::removeScheduled);
-            removeRunning(postId);
-        } finally {
-            notPauseQ = true;
-        }
-    }
-
-    private boolean canRun(Host host) {
-        boolean canRun;
-        AtomicInteger count = threadCount.get(host);
-        if (count == null) {
-            threadCount.put(host, new AtomicInteger(0));
-        }
-        canRun = threadCount.get(host).get() < settings.getMaxThreads() && (settings.getMaxTotalThreads() == 0 || threadCount.values().stream().mapToInt(AtomicInteger::get).sum() < settings.getMaxTotalThreads());
-        if (canRun && notPauseQ) {
-            threadCount.get(host).incrementAndGet();
-            return true;
-        }
-        return false;
-    }
-
-    public void poll() {
-        while (!Thread.interrupted()) {
-            try {
-                List<DownloadJob> peek = downloadQ.peek();
-                for (DownloadJob downloadJob : peek) {
-                    if (canRun(downloadJob.getImage().getHost())) {
-                        queue.offer(downloadJob);
-                        downloadQ.remove(downloadJob);
-                    }
-                }
-                synchronized (threadCount) {
-                    threadCount.wait(2_000);
-                }
-            } catch (InterruptedException e) {
-                Thread.currentThread().interrupt();
-                break;
-            }
-        }
-    }
-
-    public void start() {
-        while (!Thread.interrupted()) {
-            try {
-                push(queue.take());
-            } catch (InterruptedException e) {
-                Thread.currentThread().interrupt();
-                break;
-            } catch (Exception e) {
-                logger.error("Execution Service failed", e);
-                break;
-            }
-        }
-    }
-
-    private void push(DownloadJob take) {
-        Runnable task = () -> {
-            running.add(take);
-
-            Failsafe.with(retryPolicy)
-                    .onFailure(e -> {
-                        if (e.getFailure() instanceof InterruptedException || (e.getFailure() instanceof FailsafeException && e.getFailure().getCause() instanceof InterruptedException)) {
-                            logger.debug("Job successfully interrupted");
-                            return;
-                        }
-                        logger.error(String.format("Failed to download %s after %d tries", take.getImage().getUrl(), e.getAttemptCount()), e.getFailure());
-                        take.getImage().setStatus(Image.Status.ERROR);
-                    })
-                    .onComplete(e -> {
-                        appStateService.doneDownloadJob(take.getImage());
-                        logger.debug(String.format("Finished downloading %s", take.getImage().getUrl()));
-                        if (appSettingsService.isViewPhotos()) {
-                            VripperApplication.commonExecutor.submit(
-                                    () -> thumbnailGenerator.getThumbnails()
-                                            .get(new ThumbnailGenerator.CacheKey(take.getImage().getPostId(), take.getImageFileData().getFileName())));
-                        }
-                        threadCount.get(take.getImage().getHost()).decrementAndGet();
-                        running.remove(take);
-                        futures.remove(take.getImage().getUrl());
-                        synchronized (threadCount) {
-                            threadCount.notify();
-                        }
-                    })
-                    .get(take::call);
-        };
-        logger.debug(String.format("Scheduling a job for %s", take.getImage().getUrl()));
-        futures.put(take.getImage().getUrl(), executor.submit(task));
-    }
-
-    public int runningCount() {
-        return running.size();
-    }
-}

vripper-server/src/main/java/tn/mnlr/vripper/q/ImageFileData.java

@@ -1,16 +0,0 @@
-package tn.mnlr.vripper.q;
-
-import lombok.Getter;
-import lombok.Setter;
-import org.apache.http.client.methods.HttpUriRequest;
-
-@Getter
-@Setter
-public class ImageFileData {
-
-    private String pageUrl;
-    private String imageName;
-    private String fileName;
-    private String imageUrl;
-    private HttpUriRequest imageRequest;
-}

vripper-server/src/main/java/tn/mnlr/vripper/services/AppSettingsService.java

@@ -1,226 +0,0 @@
-package tn.mnlr.vripper.services;
-
-import com.fasterxml.jackson.annotation.JsonProperty;
-import lombok.Getter;
-import lombok.NoArgsConstructor;
-import lombok.Setter;
-import org.apache.commons.codec.digest.DigestUtils;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import org.springframework.beans.factory.annotation.Value;
-import org.springframework.stereotype.Service;
-import tn.mnlr.vripper.exception.ValidationException;
-
-import javax.annotation.PostConstruct;
-import javax.annotation.PreDestroy;
-import java.nio.file.Files;
-import java.nio.file.InvalidPathException;
-import java.nio.file.Path;
-import java.nio.file.Paths;
-import java.util.prefs.BackingStoreException;
-import java.util.prefs.Preferences;
-
-@Service
-@Getter
-@Setter
-public class AppSettingsService {
-
-    private final String MAX_TOTAL_THREADS = "MAX_TOTAL_THREADS";
-
-    @Value("${base.dir}")
-    private String defaultDownloadPath;
-
-    private static final Logger logger = LoggerFactory.getLogger(AppSettingsService.class);
-
-    private Preferences prefs = Preferences.userNodeForPackage(AppSettingsService.class);
-
-    private final String DOWNLOAD_PATH = "DOWNLOAD_PATH";
-    private final String MAX_THREADS = "MAX_THREADS";
-    private int maxTotalThreads;
-    private final String AUTO_START = "AUTO_START";
-    private final String V_LOGIN = "VLOGIN";
-    private final String V_USERNAME = "VUSERNAME";
-    private final String V_PASSWORD = "VPASSWORD";
-    private final String V_THANKS = "VTHANKS";
-    private final String DESKTOP_CLIPBOARD = "DESKTOP_CLIPBOARD";
-    private final String FORCE_ORDER = "FORCE_ORDER";
-    private final String SUBFOLDER = "SUBFOLDER";
-    private final String THREADSUBFOLDER = "THREADSUBFOLDER";
-    private final String CLEAR = "CLEAR";
-    private final String DARK_THEME = "DARK_THEME";
-    private final String VIEW_PHOTOS = "VIEW_PHOTOS";
-    private final String NOTIFICATION = "NOTIFICATION";
-
-    @PostConstruct
-    private void init() {
-        restore();
-    }
-
-    private String downloadPath;
-    private int maxThreads;
-    private boolean autoStart;
-    private boolean vLogin;
-    private String vUsername;
-    private String vPassword;
-    private boolean vThanks;
-    private boolean desktopClipboard;
-    private boolean subLocation;
-    private boolean threadSubLocation;
-    private boolean forceOrder;
-    private boolean clearCompleted;
-    private boolean darkTheme;
-    private boolean viewPhotos;
-    private boolean notificationEnabled;
-
-    public void setVPassword(String vPassword) {
-        if(vPassword.isEmpty()) {
-            this.vPassword = "";
-        } else {
-            this.vPassword = DigestUtils.md5Hex(vPassword);
-        }
-    }
-
-    public void restore() {
-
-        downloadPath = prefs.get(DOWNLOAD_PATH, defaultDownloadPath);
-        maxThreads = prefs.getInt(MAX_THREADS, 4);
-        maxTotalThreads = prefs.getInt(MAX_TOTAL_THREADS, 0);
-        autoStart = prefs.getBoolean(AUTO_START, true);
-        vLogin = prefs.getBoolean(V_LOGIN, false);
-        vUsername = prefs.get(V_USERNAME, "");
-        vPassword = prefs.get(V_PASSWORD, "");
-        vThanks = prefs.getBoolean(V_THANKS, false);
-        desktopClipboard = prefs.getBoolean(DESKTOP_CLIPBOARD, false);
-        forceOrder = prefs.getBoolean(FORCE_ORDER, false);
-        subLocation = prefs.getBoolean(SUBFOLDER, false);
-        threadSubLocation = prefs.getBoolean(THREADSUBFOLDER, false);
-        clearCompleted = prefs.getBoolean(CLEAR, false);
-        darkTheme = prefs.getBoolean(DARK_THEME, false);
-        viewPhotos = prefs.getBoolean(VIEW_PHOTOS, false);
-        notificationEnabled = prefs.getBoolean(NOTIFICATION, false);
-    }
-
-    @PreDestroy
-    public void save() {
-
-        prefs.put(DOWNLOAD_PATH, downloadPath);
-        prefs.putInt(MAX_THREADS, maxThreads);
-        prefs.putInt(MAX_TOTAL_THREADS, maxTotalThreads);
-        prefs.putBoolean(AUTO_START, autoStart);
-        prefs.putBoolean(V_LOGIN, vLogin);
-        prefs.put(V_USERNAME, vUsername);
-        prefs.put(V_PASSWORD, vPassword);
-        prefs.putBoolean(V_THANKS, vThanks);
-        prefs.putBoolean(DESKTOP_CLIPBOARD, desktopClipboard);
-        prefs.putBoolean(FORCE_ORDER, forceOrder);
-        prefs.putBoolean(SUBFOLDER, subLocation);
-        prefs.putBoolean(THREADSUBFOLDER, threadSubLocation);
-        prefs.putBoolean(CLEAR, clearCompleted);
-        prefs.putBoolean(DARK_THEME, darkTheme);
-        prefs.putBoolean(VIEW_PHOTOS, viewPhotos);
-        prefs.putBoolean(NOTIFICATION, notificationEnabled);
-
-        try {
-            prefs.sync();
-        } catch (BackingStoreException e) {
-            logger.error("Failed to store user settings", e);
-        }
-    }
-
-    public void check(Settings settings) throws ValidationException {
-
-        Path path;
-        try {
-            path = Paths.get(settings.getDownloadPath());
-        } catch (InvalidPathException e) {
-            throw new ValidationException(String.format("%s is invalid", settings.getDownloadPath()));
-        }
-        if (!Files.exists(path)) {
-            throw new ValidationException(String.format("%s does not exist", settings.getDownloadPath()));
-        } else if (!Files.isDirectory(path)) {
-            throw new ValidationException(String.format("%s is not a directory", settings.getDownloadPath()));
-        }
-
-        if (settings.getMaxTotalThreads() < 0) {
-            throw new ValidationException(String.format("Invalid max global concurrent download settings, values must be in greater than %d", 1));
-        }
-
-        if (settings.getMaxThreads() < 1 || settings.getMaxThreads() > 4) {
-            throw new ValidationException(String.format("Invalid max concurrent download settings, values must be in [%d,%d]", 1, 4));
-        }
-    }
-
-    public Theme getTheme() {
-        return new Theme(this.darkTheme);
-    }
-
-    public void setTheme(Theme theme) {
-        this.darkTheme = theme.darkTheme;
-        save();
-    }
-
-    @Getter
-    @NoArgsConstructor
-    public static class Theme {
-
-        @JsonProperty("darkTheme")
-        private boolean darkTheme;
-
-        Theme(boolean darkTheme) {
-            this.darkTheme = darkTheme;
-        }
-    }
-
-    @Getter
-    public static class Settings {
-
-        @JsonProperty("downloadPath")
-        private String downloadPath;
-        @JsonProperty("maxThreads")
-        private int maxThreads;
-        @JsonProperty("maxTotalThreads")
-        private int maxTotalThreads;
-        @JsonProperty("autoStart")
-        private boolean autoStart;
-        @JsonProperty("vLogin")
-        private boolean vLogin;
-        @JsonProperty("vUsername")
-        private String vUsername;
-        @JsonProperty("vPassword")
-        private String vPassword;
-        @JsonProperty("vThanks")
-        private boolean vThanks;
-        @JsonProperty("desktopClipboard")
-        private boolean desktopClipboard;
-        @JsonProperty("forceOrder")
-        private boolean forceOrder;
-        @JsonProperty("subLocation")
-        private boolean subLocation;
-        @JsonProperty("threadSubLocation")
-        private boolean threadSubLocation;
-        @JsonProperty("clearCompleted")
-        private boolean clearCompleted;
-        @JsonProperty("viewPhotos")
-        private boolean viewPhotos;
-        @JsonProperty("notification")
-        private boolean notification;
-
-        public Settings(String downloadPath, int maxThreads, int maxTotalThreads, boolean autoStart, boolean vLogin, String vUsername, String vPassword, boolean vThanks, boolean desktopClipboard, boolean forceOrder, boolean subLocation, boolean threadSubLocation, boolean clearCompleted, boolean viewPhotos, boolean notification) {
-            this.downloadPath = downloadPath;
-            this.maxThreads = maxThreads;
-            this.maxTotalThreads = maxTotalThreads;
-            this.autoStart = autoStart;
-            this.vLogin = vLogin;
-            this.vUsername = vUsername;
-            this.vPassword = vPassword;
-            this.vThanks = vThanks;
-            this.desktopClipboard = desktopClipboard;
-            this.forceOrder = forceOrder;
-            this.subLocation = subLocation;
-            this.threadSubLocation = threadSubLocation;
-            this.clearCompleted = clearCompleted;
-            this.viewPhotos = viewPhotos;
-            this.notification = notification;
-        }
-    }
-}

vripper-server/src/main/java/tn/mnlr/vripper/services/AppStateService.java

@@ -1,142 +0,0 @@
-package tn.mnlr.vripper.services;
-
-import io.reactivex.processors.PublishProcessor;
-import lombok.Getter;
-import lombok.Setter;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import org.springframework.beans.factory.annotation.Autowired;
-import org.springframework.stereotype.Service;
-import tn.mnlr.vripper.entities.Image;
-import tn.mnlr.vripper.entities.Post;
-import tn.mnlr.vripper.q.DownloadJob;
-
-import java.util.List;
-import java.util.Map;
-import java.util.concurrent.ConcurrentHashMap;
-import java.util.concurrent.atomic.AtomicInteger;
-import java.util.stream.Collectors;
-
-@Service
-@Getter
-public class AppStateService {
-
-    private static final Logger logger = LoggerFactory.getLogger(AppStateService.class);
-
-    private Map<String, Image> currentImages = new ConcurrentHashMap<>();
-
-    private Map<String, Post> currentPosts = new ConcurrentHashMap<>();
-
-    private Map<String, AtomicInteger> runningPosts = new ConcurrentHashMap<>();
-
-    private Map<String, QueuedVGLink> grabQueue = new ConcurrentHashMap<>();
-
-    private PublishProcessor<QueuedVGLink> liveGrabQueue = PublishProcessor.create();
-
-    private PublishProcessor<Image> liveImageUpdates = PublishProcessor.create();
-
-    private PublishProcessor<Post> livePostsState = PublishProcessor.create();
-
-    @Autowired
-    private PersistenceService persistenceService;
-
-    @Autowired
-    private AppSettingsService appSettingsService;
-
-    public void onImageUpdate(Image imageState) {
-
-        persistenceService.getProcessor().onNext(currentPosts);
-
-        liveImageUpdates.onNext(imageState);
-        if (imageState.isCompleted()) {
-
-            Post postState = currentPosts.get(imageState.getPostId());
-            postState.increase();
-        }
-    }
-
-    public Post getPost(String postId) {
-        return currentPosts.get(postId);
-    }
-
-    public void newDownloadJob(DownloadJob downloadJob) {
-        String postId = downloadJob.getImage().getPostId();
-        checkKeyRunningPosts(postId);
-        runningPosts.get(postId).incrementAndGet();
-    }
-
-    public synchronized void postDownloadingUpdate(String postId) {
-        Post post = currentPosts.get(postId);
-        if (!post.getStatus().equals(Post.Status.DOWNLOADING) && !post.getStatus().equals(Post.Status.PARTIAL)) {
-            post.setStatus(Post.Status.DOWNLOADING);
-            livePostsState.onNext(post);
-        }
-    }
-
-    public void doneDownloadJob(Image image) {
-        String postId = image.getPostId();
-        int i = runningPosts.get(postId).decrementAndGet();
-        Post post = currentPosts.get(postId);
-        if(image.getStatus().equals(Image.Status.ERROR)) {
-            post.setStatus(Post.Status.PARTIAL);
-        }
-        if (i == 0) {
-            if (post.getImages().stream().map(Image::getStatus).anyMatch(e -> e.equals(Image.Status.ERROR))) {
-                post.setStatus(Post.Status.ERROR);
-            } else {
-                if (!Post.Status.STOPPED.equals(post.getStatus())) {
-                    post.setStatus(Post.Status.COMPLETE);
-                    if (appSettingsService.isClearCompleted()) {
-                        remove(image.getPostId());
-                    }
-                }
-            }
-        }
-    }
-
-    private synchronized void checkKeyRunningPosts(String key) {
-        if (!runningPosts.containsKey(key)) {
-            runningPosts.put(key, new AtomicInteger(0));
-        }
-    }
-
-    public synchronized void remove(String postId) {
-        currentPosts.get(postId).setRemoved(true);
-        runningPosts.remove(postId);
-        currentPosts.remove(postId);
-        currentImages.entrySet().removeIf(entry -> entry.getValue().getPostId().equals(postId));
-        persistenceService.getProcessor().onNext(currentPosts);
-    }
-
-    public synchronized List<String> clearAll() {
-        List<String> toRemove = this.currentPosts
-                .values()
-                .stream()
-                .filter(e -> e.getStatus().equals(Post.Status.COMPLETE) && e.getDone().get() >= e.getTotal())
-                .map(Post::getPostId)
-                .collect(Collectors.toList());
-        toRemove.forEach(this::remove);
-        return toRemove;
-    }
-
-    public synchronized List<String> removeAll() {
-        List<String> toRemove = this.currentPosts
-                .values()
-                .stream()
-                .map(Post::getPostId)
-                .collect(Collectors.toList());
-        toRemove.forEach(this::remove);
-        return toRemove;
-    }
-
-
-    @Getter
-    public static class CachedThread {
-
-        private Map<String, Post> posts = new ConcurrentHashMap<>();
-        private AtomicInteger parsed = new AtomicInteger(0);
-
-        @Setter
-        private int total = 0;
-    }
-}

vripper-server/src/main/java/tn/mnlr/vripper/services/ConnectionManager.java

@@ -1,73 +0,0 @@
-package tn.mnlr.vripper.services;
-
-import org.apache.http.client.config.CookieSpecs;
-import org.apache.http.client.config.RequestConfig;
-import org.apache.http.client.methods.HttpGet;
-import org.apache.http.client.methods.HttpPost;
-import org.apache.http.impl.client.HttpClientBuilder;
-import org.apache.http.impl.client.HttpClients;
-import org.apache.http.impl.client.LaxRedirectStrategy;
-import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;
-import org.springframework.scheduling.annotation.EnableScheduling;
-import org.springframework.scheduling.annotation.Scheduled;
-import org.springframework.stereotype.Service;
-
-import java.net.URI;
-import java.util.concurrent.TimeUnit;
-
-@Service
-@EnableScheduling
-public class ConnectionManager {
-
-    private ConnectionManager() {
-        buildConnectionPool();
-    }
-
-    private PoolingHttpClientConnectionManager pcm;
-
-    private RequestConfig rc = RequestConfig.custom()
-            .setConnectionRequestTimeout(5_000)
-            .setConnectTimeout(5_000)
-            .setSocketTimeout(5_000)
-            .setCookieSpec(CookieSpecs.STANDARD)
-            .build();
-
-    private void buildConnectionPool() {
-
-        pcm = new PoolingHttpClientConnectionManager();
-        pcm.setMaxTotal(50);
-        pcm.setDefaultMaxPerRoute(4);
-    }
-
-    @Scheduled(fixedDelay = 1000)
-    private void idleConnectionMonitoring() {
-        pcm.closeExpiredConnections();
-        pcm.closeIdleConnections(30, TimeUnit.SECONDS);
-    }
-
-    public HttpClientBuilder getClient() {
-        return HttpClients.custom()
-                .setConnectionManager(pcm)
-                .setRedirectStrategy(new LaxRedirectStrategy())
-                .disableAutomaticRetries()
-                .setDefaultRequestConfig(rc);
-    }
-
-    public HttpGet buildHttpGet(String url) {
-        HttpGet httpGet = new HttpGet(url.replace(" ", "+"));
-        httpGet.addHeader("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36");
-        return httpGet;
-    }
-
-    public HttpPost buildHttpPost(String url) {
-        HttpPost httpPost = new HttpPost(url.replace(" ", "+"));
-        httpPost.addHeader("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36");
-        return httpPost;
-    }
-
-    public HttpGet buildHttpGet(URI uri) {
-        HttpGet httpGet = new HttpGet(uri);
-        httpGet.addHeader("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36");
-        return httpGet;
-    }
-}

vripper-server/src/main/java/tn/mnlr/vripper/services/ConnectionService.java

@@ -0,0 +1,134 @@
+package tn.mnlr.vripper.services;
+
+import lombok.Getter;
+import lombok.extern.slf4j.Slf4j;
+import net.jodah.failsafe.RetryPolicy;
+import org.apache.http.client.config.CookieSpecs;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.AbstractExecutionAwareRequest;
+import org.apache.http.client.methods.HttpGet;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.protocol.HttpClientContext;
+import org.apache.http.impl.client.HttpClientBuilder;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.impl.client.LaxRedirectStrategy;
+import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;
+import org.springframework.scheduling.annotation.EnableScheduling;
+import org.springframework.scheduling.annotation.Scheduled;
+import org.springframework.stereotype.Service;
+import reactor.core.Disposable;
+import reactor.core.publisher.Flux;
+import tn.mnlr.vripper.download.DownloadJob;
+
+import javax.annotation.PreDestroy;
+import java.net.URI;
+import java.time.temporal.ChronoUnit;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+
+@Service
+@EnableScheduling
+@Slf4j
+public class ConnectionService {
+
+    private final Disposable disposable;
+    private PoolingHttpClientConnectionManager pcm;
+    private RequestConfig rc;
+
+    @Getter
+    private RetryPolicy<Object> retryPolicy;
+
+    private int connectionTimeout;
+    private int maxAttempts;
+
+    public ConnectionService(SettingsService settingsService) {
+        connectionTimeout = settingsService.getSettings().getConnectionTimeout();
+        maxAttempts = settingsService.getSettings().getMaxAttempts();
+        Flux<SettingsService.Settings> settingsFlux = settingsService.getSettingsFlux();
+        disposable = settingsFlux.subscribe(settings -> {
+            if (connectionTimeout != settings.getConnectionTimeout()) {
+                connectionTimeout = settings.getConnectionTimeout();
+                buildRequestConfig();
+            }
+            if (maxAttempts != settings.getMaxAttempts()) {
+                maxAttempts = settings.getMaxAttempts();
+                buildRetryPolicy();
+            }
+        });
+
+        buildRequestConfig();
+        buildRetryPolicy();
+        buildConnectionPool();
+    }
+
+    @PreDestroy
+    private void destroy() {
+        disposable.dispose();
+    }
+
+    private void buildRetryPolicy() {
+        retryPolicy = new RetryPolicy<>()
+                .withDelay(2, 5, ChronoUnit.SECONDS)
+                .withMaxAttempts(maxAttempts)
+                .onFailedAttempt(e -> log.warn(String.format("#%d tries failed", e.getAttemptCount()), e.getLastFailure()));
+    }
+
+    private void buildConnectionPool() {
+        pcm = new PoolingHttpClientConnectionManager();
+        pcm.setMaxTotal(Integer.MAX_VALUE);
+        pcm.setDefaultMaxPerRoute(Integer.MAX_VALUE);
+    }
+
+    private void buildRequestConfig() {
+        rc = RequestConfig.custom()
+                .setConnectionRequestTimeout(connectionTimeout * 1000)
+                .setConnectTimeout(connectionTimeout * 1000)
+                .setSocketTimeout(connectionTimeout * 1000)
+                .setCookieSpec(CookieSpecs.STANDARD)
+                .build();
+    }
+
+    @Scheduled(fixedDelay = 5_000)
+    private void idleConnectionMonitoring() {
+        pcm.closeExpiredConnections();
+        pcm.closeIdleConnections(30, TimeUnit.SECONDS);
+    }
+
+    public HttpClientBuilder getClient() {
+        return HttpClients.custom()
+                .setConnectionManager(pcm)
+                .setRedirectStrategy(new LaxRedirectStrategy())
+                .disableAutomaticRetries()
+                .setDefaultRequestConfig(rc);
+    }
+
+    public HttpGet buildHttpGet(String url, final HttpClientContext context) {
+        HttpGet httpGet = new HttpGet(url.replace(" ", "+"));
+        httpGet.addHeader("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36");
+        addToContext(context, httpGet);
+        return httpGet;
+    }
+
+    public HttpPost buildHttpPost(String url, final HttpClientContext context) {
+        HttpPost httpPost = new HttpPost(url.replace(" ", "+"));
+        httpPost.addHeader("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36");
+        addToContext(context, httpPost);
+        return httpPost;
+    }
+
+    public HttpGet buildHttpGet(URI uri, final HttpClientContext context) {
+        HttpGet httpGet = new HttpGet(uri);
+        httpGet.addHeader("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36");
+        addToContext(context, httpGet);
+        return httpGet;
+    }
+
+    public void addToContext(HttpClientContext context, AbstractExecutionAwareRequest request) {
+        if (context != null) {
+            List<AbstractExecutionAwareRequest> requests = (List<AbstractExecutionAwareRequest>) context.getAttribute(DownloadJob.ContextAttributes.OPEN_CONNECTION.toString());
+            if (requests != null) {
+                requests.add(request);
+            }
+        }
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/services/DataService.java

@@ -0,0 +1,227 @@
+package tn.mnlr.vripper.services;
+
+import lombok.NonNull;
+import lombok.extern.slf4j.Slf4j;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Service;
+import org.springframework.transaction.annotation.Transactional;
+import tn.mnlr.vripper.jpa.domain.*;
+import tn.mnlr.vripper.jpa.domain.enums.Status;
+import tn.mnlr.vripper.jpa.repositories.IImageRepository;
+import tn.mnlr.vripper.jpa.repositories.IMetadataRepository;
+import tn.mnlr.vripper.jpa.repositories.IPostRepository;
+import tn.mnlr.vripper.jpa.repositories.IQueuedRepository;
+import tn.mnlr.vripper.jpa.repositories.impl.EventRepository;
+
+import java.util.Collection;
+import java.util.List;
+import java.util.Optional;
+
+@Service
+@Transactional
+@Slf4j
+public class DataService {
+
+    private final IPostRepository postRepository;
+    private final IImageRepository imageRepository;
+    private final IQueuedRepository queuedRepository;
+    private final IMetadataRepository metadataRepository;
+    private final SettingsService settingsService;
+    private final EventRepository eventRepository;
+
+    @Autowired
+    public DataService(IPostRepository postRepository, IImageRepository imageRepository, IQueuedRepository queuedRepository, IMetadataRepository metadataRepository, SettingsService settingsService, EventRepository eventRepository) {
+        this.postRepository = postRepository;
+        this.imageRepository = imageRepository;
+        this.queuedRepository = queuedRepository;
+        this.metadataRepository = metadataRepository;
+        this.settingsService = settingsService;
+        this.eventRepository = eventRepository;
+    }
+
+    private void save(Post post) {
+        postRepository.save(post);
+    }
+
+    private void save(Queued queued) {
+        queuedRepository.save(queued);
+    }
+
+    private void save(Image image) {
+        imageRepository.save(image);
+    }
+
+    public boolean exists(String postId) {
+        return postRepository.existByPostId(postId);
+    }
+
+    public void newPost(Post post, Collection<Image> images) {
+        save(post);
+        images.forEach(image -> {
+            image.setPostIdRef(post.getId());
+            save(image);
+        });
+    }
+
+    public void setDownloadingToStopped() {
+        postRepository.setDownloadingToStopped();
+    }
+
+    public synchronized void afterJobFinish(Image image, Post post) {
+        if (image.getStatus().equals(Status.COMPLETE)) {
+            post.setDone(post.getDone() + 1);
+            updatePostDone(post.getDone(), post.getId());
+        } else if (image.getStatus().equals(Status.ERROR)) {
+            post.setStatus(Status.PARTIAL);
+            updatePostStatus(post.getStatus(), post.getId());
+        }
+    }
+
+    private void updatePostDone(int done, Long id) {
+        postRepository.updateDone(done, id);
+    }
+
+    public void finishPost(@NonNull Post post) {
+        if (!imageRepository.findByPostIdAndIsError(post.getPostId()).isEmpty()) {
+            post.setStatus(Status.ERROR);
+            updatePostStatus(post.getStatus(), post.getId());
+        } else {
+            if (post.getDone() < post.getTotal()) {
+                post.setStatus(Status.STOPPED);
+                updatePostStatus(post.getStatus(), post.getId());
+            } else {
+                post.setStatus(Status.COMPLETE);
+                updatePostStatus(post.getStatus(), post.getId());
+                if (settingsService.getSettings().getClearCompleted()) {
+                    remove(post.getPostId());
+                }
+            }
+        }
+    }
+
+    private void remove(@NonNull final String postId) {
+        imageRepository.deleteAllByPostId(postId);
+        metadataRepository.deleteByPostId(postId);
+        postRepository.deleteByPostId(postId);
+    }
+
+    public void newQueueLink(@NonNull final Queued queued) {
+        save(queued);
+    }
+
+    public void removeQueueLink(@NonNull final String threadId) {
+        queuedRepository.deleteByThreadId(threadId);
+    }
+
+    public List<String> clearCompleted() {
+        List<String> completed = postRepository.findCompleted();
+        completed.forEach(this::remove);
+        return completed;
+    }
+
+    public void removeAll(final List<String> postIds) {
+        if (postIds != null && !postIds.isEmpty()) {
+            for (String postId : postIds) {
+                remove(postId);
+            }
+        } else {
+            postRepository.findAll().forEach(p -> remove(p.getPostId()));
+        }
+    }
+
+    public List<Image> findByPostIdAndIsNotCompleted(@NonNull String postId) {
+        return imageRepository.findByPostIdAndIsNotCompleted(postId);
+
+    }
+
+    public long countErrorImages() {
+        return imageRepository.countError();
+    }
+
+    public List<Image> findImagesByPostId(String postId) {
+        return imageRepository.findByPostId(postId);
+    }
+
+    public List<Post> findAllPosts() {
+        return postRepository.findAll();
+    }
+
+    public Optional<Post> findPostByPostId(String postId) {
+        return postRepository.findByPostId(postId);
+    }
+
+    public void stopImagesByPostIdAndIsNotCompleted(String postId) {
+        imageRepository.stopByPostIdAndIsNotCompleted(postId);
+    }
+
+    public Optional<Queued> findQueuedByThreadId(String threadId) {
+        return queuedRepository.findByThreadId(threadId);
+    }
+
+    public List<Queued> findAllQueued() {
+        return queuedRepository.findAll();
+    }
+
+    public Optional<Post> findById(Long aLong) {
+        return postRepository.findById(aLong);
+    }
+
+    public Optional<Image> findImageById(Long aLong) {
+        return imageRepository.findById(aLong);
+    }
+
+    public Optional<Queued> findQueuedById(Long aLong) {
+        return queuedRepository.findById(aLong);
+    }
+
+    public synchronized void setMetadata(Post post, Metadata metadata) {
+        if (metadataRepository.findByPostId(post.getPostId()).isEmpty()) {
+            metadata.setPostIdRef(post.getId());
+            metadataRepository.save(metadata);
+        }
+    }
+
+    public Optional<Metadata> findMetadataByPostId(String postId) {
+        return metadataRepository.findByPostId(postId);
+    }
+
+    public void updateImageStatus(Status status, Long id) {
+        imageRepository.updateStatus(status, id);
+    }
+
+    public void updateImageCurrent(long current, Long id) {
+        imageRepository.updateCurrent(current, id);
+    }
+
+    public void updateImageTotal(long total, Long id) {
+        imageRepository.updateTotal(total, id);
+    }
+
+    public void updatePostStatus(Status status, Long id) {
+        postRepository.updateStatus(status, id);
+    }
+
+    public void updateDownloadDirectory(String postFolderName, Long id) {
+        postRepository.updateFolderName(postFolderName, id);
+    }
+
+    public void updatePostTitle(String title, Long id) {
+        postRepository.updateTitle(title, id);
+    }
+
+    public void updatePostThanked(boolean thanked, Long id) {
+        postRepository.updateThanked(thanked, id);
+    }
+
+    public Optional<Event> findEventById(Long id) {
+        return eventRepository.findById(id);
+    }
+
+    public List<Event> findAllEvents() {
+        return eventRepository.findAll();
+    }
+
+    public void clearQueueLinks() {
+        queuedRepository.deleteAll();
+    }
+}

vripper-server/src/main/java/tn/mnlr/vripper/services/DownloadSpeedService.java

@@ -1,26 +1,30 @@
 package tn.mnlr.vripper.services;
 
-import io.reactivex.processors.PublishProcessor;
 import lombok.Getter;
 import org.springframework.scheduling.annotation.EnableScheduling;
 import org.springframework.scheduling.annotation.Scheduled;
 import org.springframework.stereotype.Service;
+import reactor.core.publisher.Flux;
+import reactor.core.publisher.Sinks;
+import tn.mnlr.vripper.listener.EmitHandler;
 
+import javax.annotation.PreDestroy;
 import java.util.concurrent.atomic.AtomicLong;
 
 @Service
 @EnableScheduling
 public class DownloadSpeedService {
 
-    private AtomicLong read = new AtomicLong(0);
-
-    private long currentValue;
-
+    private final AtomicLong read = new AtomicLong(0);
+    private final Sinks.Many<Long> sink = Sinks.many().multicast().onBackpressureBuffer();
     @Getter
-    private PublishProcessor<Long> readBytesPerSecond = PublishProcessor.create();
-
+    private long currentValue;
     private boolean allowWrite = false;
 
+    public Flux<Long> getReadBytesPerSecond() {
+        return sink.asFlux();
+    }
+
     public void increase(long read) {
         if (allowWrite) {
             this.read.addAndGet(read);
@@ -33,8 +37,13 @@ public class DownloadSpeedService {
         long newValue = read.getAndSet(0);
         if (newValue != currentValue) {
             currentValue = newValue;
-            readBytesPerSecond.onNext(currentValue);
+            sink.emitNext(currentValue, EmitHandler.RETRY);
         }
         allowWrite = true;
     }
+
+    @PreDestroy
+    private void destroy() {
+        sink.emitComplete(EmitHandler.RETRY);
+    }
 }

vripper-server/src/main/java/tn/mnlr/vripper/services/GlobalState.java

@@ -1,35 +0,0 @@
-package tn.mnlr.vripper.services;
-
-import lombok.Getter;
-
-import java.util.Objects;
-
-@Getter
-public class GlobalState {
-
-    private final String type = "globalState";
-    private long running;
-    private long queued;
-    private long remaining;
-    private long error;
-
-    GlobalState(long running, long queued, long remaining, long error) {
-        this.running = running;
-        this.queued = queued;
-        this.remaining = remaining;
-        this.error = error;
-    }
-
-    @Override
-    public boolean equals(Object o) {
-        if (this == o) return true;
-        if (o == null || getClass() != o.getClass()) return false;
-        GlobalState that = (GlobalState) o;
-        return running == that.running && queued == that.queued && remaining == that.remaining && error == that.error;
-    }
-
-    @Override
-    public int hashCode() {
-        return Objects.hash(type, running, queued, remaining, error);
-    }
-}

vripper-server/src/main/java/tn/mnlr/vripper/services/GlobalStateService.java

@@ -1,55 +1,55 @@
 package tn.mnlr.vripper.services;
 
-import io.reactivex.processors.PublishProcessor;
 import lombok.Getter;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.scheduling.annotation.EnableScheduling;
 import org.springframework.scheduling.annotation.Scheduled;
 import org.springframework.stereotype.Service;
-import tn.mnlr.vripper.entities.Image;
-import tn.mnlr.vripper.q.DownloadQ;
-import tn.mnlr.vripper.q.ExecutionService;
+import reactor.core.publisher.Flux;
+import reactor.core.publisher.Sinks;
+import tn.mnlr.vripper.download.DownloadService;
+import tn.mnlr.vripper.download.PendingQueue;
+import tn.mnlr.vripper.listener.EmitHandler;
+import tn.mnlr.vripper.services.domain.GlobalState;
+
+import javax.annotation.PreDestroy;
 
 @Service
 @EnableScheduling
 public class GlobalStateService {
 
-    @Autowired
-    private DownloadQ downloadQ;
-
-    @Autowired
-    private ExecutionService executionService;
-
-    @Autowired
-    private AppStateService appStateService;
-
+    private final PendingQueue pendingQueue;
+    private final DownloadService downloadService;
+    private final DataService dataService;
+    private final Sinks.Many<GlobalState> sink = Sinks.many().multicast().onBackpressureBuffer();
     @Getter
     private GlobalState currentState;
 
-    @Getter
-    private PublishProcessor<GlobalState> liveGlobalState = PublishProcessor.create();
+    @Autowired
+    public GlobalStateService(PendingQueue pendingQueue, DownloadService downloadService, DataService dataService) {
+        this.pendingQueue = pendingQueue;
+        this.downloadService = downloadService;
+        this.dataService = dataService;
+    }
 
-    public GlobalStateService() {
+    public Flux<GlobalState> getGlobalState() {
+        return sink.asFlux();
     }
 
     @Scheduled(fixedDelay = 3000)
     private void interval() {
         GlobalState newGlobalState = new GlobalState(
-                executionService.runningCount(),
-                downloadQ.size(),
-                appStateService.getCurrentImages()
-                        .values()
-                        .stream()
-                        .filter(e -> e.getTotal() == 0 || e.getTotal() != e.getCurrent().get())
-                        .count(),
-                appStateService.getCurrentImages()
-                        .values()
-                        .stream()
-                        .filter(e -> e.getStatus().equals(Image.Status.ERROR))
-                        .count());
+                downloadService.runningCount(),
+                pendingQueue.size(),
+                dataService.countErrorImages());
         if (!newGlobalState.equals(currentState)) {
             currentState = newGlobalState;
-            liveGlobalState.onNext(currentState);
+            sink.emitNext(currentState, EmitHandler.RETRY);
         }
     }
+
+    @PreDestroy
+    private void destroy() {
+        sink.emitComplete(EmitHandler.RETRY);
+    }
 }