Commit Graph

75 Commits

Author SHA1 Message Date
orangecoding
7fd8be07a2 adding wohnungsboerse provider 2026-01-09 11:37:03 +01:00
orangecoding
5dc976c7e3 ability to start jobs individually 2025-12-18 19:16:28 +01:00
orangecoding
05f1bc61c9 fixing tests 2025-12-17 16:35:24 +01:00
orangecoding
6e8a35a836 adding backup/restore ability 2025-12-17 15:48:56 +01:00
orangecoding
790c559316 foced to move to Apache 2.0 license 2025-12-11 10:40:55 +01:00
orangecoding
5bd4219743 upgrading dependencies | adding ohneMakler provider 2025-12-08 20:31:28 +01:00
orangecoding
22df683969 more efficient bot protection 2025-11-27 10:30:47 +01:00
orangecoding
79a8420dfb improving similarity cache 2025-10-29 09:36:05 +01:00
orangecoding
0436c7f7d7 upgrading dependencies / FredyRuntime >> FredyPipeline 2025-10-12 16:43:56 +02:00
Thomas Brockmöller
7ebd73c9cf Add new provider McMakler (#201) 2025-09-28 14:16:28 +02:00
Thomas Brockmöller
4d37e890ab Add provider for Regionalimmobilien24 (#197) 2025-09-27 14:19:37 +02:00
Thomas Brockmöller
7589f20a18 Add sparkasse immobilien (#199) 2025-09-27 09:43:24 +02:00
Thomas Brockmöller
702ffabc1a Fix and improve immowelt/immonet provider (#194)
* Fix and improve immowelt provider

* Add description to immonet provider

* Fix tests and improve readability
2025-09-27 09:42:08 +02:00
Christian Kellner
8324357edb Improvements (#193)
* improving release banner

* renaming general to settings

* fixing working hours if they go to next day

* fixing comparing versions

* upgrade dependencies
2025-09-26 10:45:55 +02:00
Thomas Brockmöller
dd5c5b29d9 Fix address value in similarity filtering (#191)
* Fix address field in similarity filter
2025-09-25 15:02:00 +02:00
Christian Kellner
c839f3abc9 Check if a listing is still active (#184)
* check if a listing is still active

* upgrade dependencies
2025-09-22 09:57:50 +02:00
orangecoding
da8fd13973 fixing immoscout 2025-09-19 21:11:28 +02:00
orangecoding
28f0a167e6 fixing docker migration path 2025-09-18 17:28:30 +02:00
Christian Kellner
8d95f052c6 Migrate to SQLite (#174)
* Migrating Fredy from LowDb to SqLite 🎉

* adding new sql migration system for future sql migrations

* adding setting to change  sqlite path for db files

* create migration plan for graceful migration lowdb -> sqlite

* Improving Documentation

* adding test for sqliteconnection

* upgrading dependencies

* making nodejs 22 as min version

* improve scraper

* adding overwrite ability for db migra
2025-09-18 15:38:23 +02:00
orangecoding
09c6ce1d0b improve similarity cache. It now checks for similarities independend from jobs 2025-09-07 22:15:14 +02:00
Christian Kellner
1854b421af avoid warnings on test 2025-09-03 14:47:56 +02:00
Christian Kellner
f0b146fd7f Adding images to scraping data (#157)
* Fredy now supports pulling the main Image from the listing and send it together with the usual information
2025-08-30 21:21:34 +02:00
Nic
3a54ab0e31 Fix typo in test import (#154)
* Fix typo in test import

* Rename test file
2025-08-25 20:42:40 +02:00
Alexander Roidl
2b36f868e7 Project-wide linting and formatting (#150)
* chore: configure project-wide linting and formatting

* chore: run lint autofix and formatter
2025-07-26 20:42:58 +02:00
Alexander Roidl
cca1463a68 chore: run formatter (#145) 2025-07-23 08:47:26 +02:00
Christian Kellner
f032e6a724 test: verify unrelated text yields no similarity (#130) 2025-06-04 09:15:53 +02:00
Christian Kellner
f9c6b10976 fixing tests 2025-05-26 10:43:13 +02:00
Leon C.
1f54bcfd3f ImmoScout: Allow web paths with SEO optimization to be filtered to query params (#128) 2025-05-20 12:44:43 +02:00
Christian Kellner
030e0ca169 starting docu on reverse engineering immoscout api (#127)
* starting docu on reverse engineering immoscout api

* improving immoscout reverse engineering and adding support for most other types
2025-05-14 13:58:58 +02:00
Christian Kellner
cd3631f910 fixing new immoscout url handling 2025-05-09 10:05:30 +02:00
Christian Kellner
48e2ca942f fixing tests, renaming immoscout-mobile to immoscout 2025-05-09 09:26:24 +02:00
Patrick Klein
b9e4bca244 Add immoscout mobile API provider to avoid failing bot checks (#125)
* Add provider that uses the immoscout mobile API to avoid failing bot checks.
2025-05-09 09:13:52 +02:00
weakmap@gmail.com
a3471a091a upgrade dependencies, fixing tests 2025-02-23 17:13:08 +01:00
Christian Kellner
214e714c03 Puppeteer rewrite (#119)
* Moving to puppeteer | removing scrapingAnt
2024-12-17 12:38:28 +01:00
weakmap@gmail.com
a4c5bfcbf7 fixing tests 2024-10-03 16:09:19 +02:00
weakmap@gmail.com
6d2ab5f958 making sure immowelt does not include suggested ranges 2024-10-03 16:03:47 +02:00
Christian Kellner
111ef8be43 fixing kleinanzeigen test 2024-09-05 13:36:02 +02:00
Christian Kellner
35feb772d7 upgrading dependencies, fixing immowelt, using hash of price and id as unique identifier for listings 2024-09-05 13:34:14 +02:00
Christian Kellner
92db8219b4 building multi platform docker images (#101)
* building multi platform docker images

* upgrading dependencies | using scraping ant for neubaukompass
2024-07-24 09:32:21 +02:00
Christian Kellner
06c4ebb975 fixing immoswp 2024-06-12 14:15:21 +02:00
weakmap@gmail.com
27e098c244 upgrading dependencies, dropping support for node < 18. Happy new Year 2024-01-01 16:14:25 +01:00
weakmap@gmail.com
88c6641485 fixing wgGesucht test 2023-09-07 20:40:07 +02:00
Christian Kellner
d947dad488 fixing ebay kleinanzeigen, now becoming kleinanzeigen 2023-05-08 09:32:07 +02:00
weakmap@gmail.com
4ba098e0b6 bringing back immonet by using scrapingant 2023-04-15 18:24:51 +02:00
weakmap@gmail.com
1feb5bfda1 running all tests 2023-04-07 19:45:24 +02:00
weakmap@gmail.com
3ec9ed3b2a ignoring expired ssl certificate o0 2023-04-07 19:44:59 +02:00
Christian Kellner
2c5eceb0c1 Making Fredy an ESM project (#70)
Making Fredy an ESM project
2023-03-13 13:42:43 +01:00
Quoc Duong Bui
47e4230b39 Provider for immobilien.de (#65)
* Add provider for immobilien.de
2022-12-19 19:29:13 +01:00
weakmap@gmail.com
ab5ee59d72 ugrading dependencies | fixing tests | supporting multiple provider of the same type 2022-12-11 20:07:18 +01:00
Christian Kellner
bc01806421 fixing telegram provider not respecting rate limits 2022-06-10 14:19:20 +02:00