Commit Graph
7 Commits
Author SHA1 Message Date
Jochen SchalandaandGitHub b6b8d6814c Make requestDriver more resilient to errors (#46)
If the async request performed in `requestDriver.makeDriver()` fails, it would call the `callback` function with empty parameters but then continue the execution which can lead to the following error and crash of Fredy:
```
Error while trying to scrape data. Received error: Request failed with status code 504
/fredy/lib/services/requestDriver.js:25
    if (typeof result.data === 'object' && url.toLowerCase().indexOf('scrapingant') !== -1) {
                      ^

TypeError: Cannot read properties of undefined (reading 'data')
    at driver (/fredy/lib/services/requestDriver.js:25:23)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
```
2022-01-31 10:48:11 +01:00
Jochen SchalandaandGitHub 2f5cc31ae3 Add support for Immo Südwest Presse (immo.swp.de) (#45) 2022-01-26 14:41:44 +01:00
Jochen SchalandaandGitHub 70e78492ec Telegram: Use job name instead of ID and link in title (#44) 2022-01-26 14:39:53 +01:00
Jochen SchalandaandGitHub 47adb88cb5 Fix race condition if user ID is in session but not in user store (#43) 2022-01-25 15:11:21 +01:00
Jochen SchalandaandGitHub e5627e1d02 Allow visiting the original provider URL (#42)
Instead of truncating the original URL of each provider in the job configuration to 60 characters and losing a lot of context information, put a link to the original URL in the provider table which can be opened directly to verify what is being scraped by Fredy.
2022-01-25 14:20:42 +01:00
Jochen SchalandaandGitHub 2764b2b776 Refactor Docker build and automatically push images (#41)
Refactor Docker build / Add GitHub workflow for creating and publishing Docker image
2022-01-24 16:41:23 +01:00
Jochen SchalandaandGitHub 0de6d3df04 Add GitHub Actions workflow to test changes (#40) 2022-01-24 16:34:31 +01:00