Christian Kellner
3bda88a075
upgrade dependencies
2023-05-11 11:51:23 +02:00
weakmap@gmail.com
4ba098e0b6
bringing back immonet by using scrapingant
2023-04-15 18:24:51 +02:00
weakmap@gmail.com
3ec9ed3b2a
ignoring expired ssl certificate o0
2023-04-07 19:44:59 +02:00
Christian Kellner
2c5eceb0c1
Making Fredy an ESM project ( #70 )
...
Making Fredy an ESM project
2023-03-13 13:42:43 +01:00
weakmap@gmail.com
8a5fbcdf71
moving to node-fetch coz axios is causing issues with scrapingAnt
2022-12-20 10:21:15 +01:00
Christian Kellner
2062aa11a3
Scrapingant proxies ( #59 )
...
* preparing scraping ant proxies
* adding general settings for scraping ant proxy
* retrying with new ui settings
2022-06-13 08:10:30 +02:00
Christian Kellner
676d48807a
scraping ant retries
2022-04-29 13:22:39 +02:00
orangecoding
5347d0014d
adding scraping ant infos as we now need to use residental proxies
2022-03-25 11:19:17 +01:00
Jochen Schalanda
b6b8d6814c
Make requestDriver more resilient to errors ( #46 )
...
If the async request performed in `requestDriver.makeDriver()` fails, it would call the `callback` function with empty parameters but then continue the execution which can lead to the following error and crash of Fredy:
```
Error while trying to scrape data. Received error: Request failed with status code 504
/fredy/lib/services/requestDriver.js:25
if (typeof result.data === 'object' && url.toLowerCase().indexOf('scrapingant') !== -1) {
^
TypeError: Cannot read properties of undefined (reading 'data')
at driver (/fredy/lib/services/requestDriver.js:25:23)
at runMicrotasks (<anonymous>)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
```
2022-01-31 10:48:11 +01:00
orangecoding
c1c4d55ede
adding or replacing sort params for provider urls when necessary
2021-11-26 21:02:09 +01:00
Christian Kellner
c97b323b35
fix: node would crash if axios throw an error
2021-11-11 15:31:53 +01:00
Christian Kellner
59e6d287fc
adding similarity check ( #29 )
...
* adding similarity check
* adding paging
* fixing tests
* docu
* better error handling
* fixing tests
* adjusting page limit
* fixing login screen
* cleanup
* upgrade browser list
* prevent spamming the log
* fixing tests
* removing job listings when removing a job or the user
2021-06-28 08:52:09 +02:00
orangecoding
1b3a95b325
making sure fredy is not crashing if scrapingant has issues
2021-05-13 20:51:15 +02:00
Christian Kellner
70cab66651
Bringing back immoscout ( #21 )
...
Bringing back immoscout support 🎉
2021-05-11 11:25:14 +02:00
Christian Kellner
b2847f6834
UI ( #15 )
...
Adding new Admin UI. Updating Fredy to V3.0.0 as it has been a large rewrite. Thanks for all contributions and help on the way!
2021-01-21 16:09:23 +01:00
Christian Kellner
c91471a0b2
remove user agent from config
2020-02-28 14:03:11 +01:00
Christian Kellner
770d30af95
Concurrent-Jobs ( #7 )
...
Fredy 2.0.0 introduces the concept of search jobs. You can now configure multiple of these jobs running in the same instance of Fredy. Also a new API has been created 🎉
2020-02-26 09:05:20 +01:00
Christian Kellner
9106144d9c
fixing neubaukompass && updating dependencies
2019-10-24 21:25:19 +02:00
Christian Kellner
97d7ba7f9a
IMPROVEMENT: improve stats to not fill up memory over time 🎉
2018-02-05 11:26:19 +01:00
Christian Kellner
ddce90a047
ADDED: nvmrc, CHANGED: rename stats output
2018-01-22 13:05:20 +01:00
Christian Kellner
c6cffe029d
Release v1.0.0 🎉
2018-01-20 20:23:27 +01:00
