Christian Kellner
f9c6b10976
fixing tests
2025-05-26 10:43:13 +02:00
Leon C.
1f54bcfd3f
ImmoScout: Allow web paths with SEO optimization to be filtered to query params ( #128 )
2025-05-20 12:44:43 +02:00
Christian Kellner
25dfad4f5d
run cleanup once at start
2025-05-16 13:26:39 +02:00
Christian Kellner
b7a3823049
console log when removing demo jobs
2025-05-16 13:25:55 +02:00
Christian Kellner
6964998695
fixing removing demo jobs
2025-05-16 13:20:54 +02:00
Christian Kellner
030e0ca169
starting docu on reverse engineering immoscout api ( #127 )
...
* starting docu on reverse engineering immoscout api
* improving immoscout reverse engineering and adding support for most other types
2025-05-14 13:58:58 +02:00
Christian Kellner
1ecbbdd774
better logging
2025-01-07 13:34:43 +01:00
Christian Kellner
e1db3840f6
adding puppeteer timeout and fixing waitForSelector
2025-01-07 12:37:50 +01:00
Christian Kellner
90a4ee5dcf
better logging, fixing code smells
2025-01-07 12:25:19 +01:00
weakmap@gmail.com
3b8279c714
adding fredy version
2024-12-17 13:07:25 +01:00
Christian Kellner
214e714c03
Puppeteer rewrite ( #119 )
...
* Moving to puppeteer | removing scrapingAnt
2024-12-17 12:38:28 +01:00
Christian Kellner
324b14da50
improving tracking
2024-12-03 14:23:09 +01:00
Christian Kellner
f8f911aa00
improving tracking
2024-12-03 14:05:00 +01:00
Christian Kellner
337ee922a6
Demo Mode ( #117 )
...
* Adding Demo Mode to Fredy
2024-11-22 09:11:10 +01:00
Christian Kellner
8f91267b5d
sending tracking information ( #116 )
...
* Ability to send tracking information
2024-11-20 22:22:16 +01:00
Christian Kellner
92db8219b4
building multi platform docker images ( #101 )
...
* building multi platform docker images
* upgrading dependencies | using scraping ant for neubaukompass
2024-07-24 09:32:21 +02:00
Vladislav
e7db4e23f5
update error handling ( #100 )
2024-07-22 10:41:30 +02:00
Christian Kellner
3bda88a075
upgrade dependencies
2023-05-11 11:51:23 +02:00
weakmap@gmail.com
4ba098e0b6
bringing back immonet by using scrapingant
2023-04-15 18:24:51 +02:00
weakmap@gmail.com
3ec9ed3b2a
ignoring expired ssl certificate o0
2023-04-07 19:44:59 +02:00
Christian Kellner
2c5eceb0c1
Making Fredy an ESM project ( #70 )
...
Making Fredy an ESM project
2023-03-13 13:42:43 +01:00
weakmap@gmail.com
8a5fbcdf71
moving to node-fetch coz axios is causing issues with scrapingAnt
2022-12-20 10:21:15 +01:00
Christian Kellner
2062aa11a3
Scrapingant proxies ( #59 )
...
* preparing scraping ant proxies
* adding general settings for scraping ant proxy
* retrying with new ui settings
2022-06-13 08:10:30 +02:00
Christian Kellner
676d48807a
scraping ant retries
2022-04-29 13:22:39 +02:00
orangecoding
5347d0014d
adding scraping ant infos as we now need to use residental proxies
2022-03-25 11:19:17 +01:00
Jochen Schalanda
b6b8d6814c
Make requestDriver more resilient to errors ( #46 )
...
If the async request performed in `requestDriver.makeDriver()` fails, it would call the `callback` function with empty parameters but then continue the execution which can lead to the following error and crash of Fredy:
```
Error while trying to scrape data. Received error: Request failed with status code 504
/fredy/lib/services/requestDriver.js:25
if (typeof result.data === 'object' && url.toLowerCase().indexOf('scrapingant') !== -1) {
^
TypeError: Cannot read properties of undefined (reading 'data')
at driver (/fredy/lib/services/requestDriver.js:25:23)
at runMicrotasks (<anonymous>)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
```
2022-01-31 10:48:11 +01:00
orangecoding
c1c4d55ede
adding or replacing sort params for provider urls when necessary
2021-11-26 21:02:09 +01:00
Christian Kellner
c97b323b35
fix: node would crash if axios throw an error
2021-11-11 15:31:53 +01:00
Christian Kellner
59e6d287fc
adding similarity check ( #29 )
...
* adding similarity check
* adding paging
* fixing tests
* docu
* better error handling
* fixing tests
* adjusting page limit
* fixing login screen
* cleanup
* upgrade browser list
* prevent spamming the log
* fixing tests
* removing job listings when removing a job or the user
2021-06-28 08:52:09 +02:00
orangecoding
1b3a95b325
making sure fredy is not crashing if scrapingant has issues
2021-05-13 20:51:15 +02:00
Christian Kellner
70cab66651
Bringing back immoscout ( #21 )
...
Bringing back immoscout support 🎉
2021-05-11 11:25:14 +02:00
Christian Kellner
b2847f6834
UI ( #15 )
...
Adding new Admin UI. Updating Fredy to V3.0.0 as it has been a large rewrite. Thanks for all contributions and help on the way!
2021-01-21 16:09:23 +01:00
Christian Kellner
c91471a0b2
remove user agent from config
2020-02-28 14:03:11 +01:00
Christian Kellner
770d30af95
Concurrent-Jobs ( #7 )
...
Fredy 2.0.0 introduces the concept of search jobs. You can now configure multiple of these jobs running in the same instance of Fredy. Also a new API has been created 🎉
2020-02-26 09:05:20 +01:00
Christian Kellner
9106144d9c
fixing neubaukompass && updating dependencies
2019-10-24 21:25:19 +02:00
Christian Kellner
97d7ba7f9a
IMPROVEMENT: improve stats to not fill up memory over time 🎉
2018-02-05 11:26:19 +01:00
Christian Kellner
ddce90a047
ADDED: nvmrc, CHANGED: rename stats output
2018-01-22 13:05:20 +01:00
Christian Kellner
c6cffe029d
Release v1.0.0 🎉
2018-01-20 20:23:27 +01:00
