Commit Graph

62 Commits

Author SHA1 Message Date
Christian Kellner
eabade9ba7 fixing working hours if they go to next day 2025-09-25 15:57:13 +02:00
Thomas Brockmöller
dd5c5b29d9 Fix address value in similarity filtering (#191)
* Fix address field in similarity filter
2025-09-25 15:02:00 +02:00
Christian Kellner
c839f3abc9 Check if a listing is still active (#184)
* check if a listing is still active

* upgrade dependencies
2025-09-22 09:57:50 +02:00
orangecoding
da8fd13973 fixing immoscout 2025-09-19 21:11:28 +02:00
orangecoding
28f0a167e6 fixing docker migration path 2025-09-18 17:28:30 +02:00
Christian Kellner
8d95f052c6 Migrate to SQLite (#174)
* Migrating Fredy from LowDb to SqLite 🎉

* adding new sql migration system for future sql migrations

* adding setting to change  sqlite path for db files

* create migration plan for graceful migration lowdb -> sqlite

* Improving Documentation

* adding test for sqliteconnection

* upgrading dependencies

* making nodejs 22 as min version

* improve scraper

* adding overwrite ability for db migra
2025-09-18 15:38:23 +02:00
orangecoding
09c6ce1d0b improve similarity cache. It now checks for similarities independend from jobs 2025-09-07 22:15:14 +02:00
Christian Kellner
1854b421af avoid warnings on test 2025-09-03 14:47:56 +02:00
Christian Kellner
f0b146fd7f Adding images to scraping data (#157)
* Fredy now supports pulling the main Image from the listing and send it together with the usual information
2025-08-30 21:21:34 +02:00
Nic
3a54ab0e31 Fix typo in test import (#154)
* Fix typo in test import

* Rename test file
2025-08-25 20:42:40 +02:00
Alexander Roidl
2b36f868e7 Project-wide linting and formatting (#150)
* chore: configure project-wide linting and formatting

* chore: run lint autofix and formatter
2025-07-26 20:42:58 +02:00
Alexander Roidl
cca1463a68 chore: run formatter (#145) 2025-07-23 08:47:26 +02:00
Christian Kellner
f032e6a724 test: verify unrelated text yields no similarity (#130) 2025-06-04 09:15:53 +02:00
Christian Kellner
f9c6b10976 fixing tests 2025-05-26 10:43:13 +02:00
Leon C.
1f54bcfd3f ImmoScout: Allow web paths with SEO optimization to be filtered to query params (#128) 2025-05-20 12:44:43 +02:00
Christian Kellner
030e0ca169 starting docu on reverse engineering immoscout api (#127)
* starting docu on reverse engineering immoscout api

* improving immoscout reverse engineering and adding support for most other types
2025-05-14 13:58:58 +02:00
Christian Kellner
cd3631f910 fixing new immoscout url handling 2025-05-09 10:05:30 +02:00
Christian Kellner
48e2ca942f fixing tests, renaming immoscout-mobile to immoscout 2025-05-09 09:26:24 +02:00
Patrick Klein
b9e4bca244 Add immoscout mobile API provider to avoid failing bot checks (#125)
* Add provider that uses the immoscout mobile API to avoid failing bot checks.
2025-05-09 09:13:52 +02:00
weakmap@gmail.com
a3471a091a upgrade dependencies, fixing tests 2025-02-23 17:13:08 +01:00
Christian Kellner
214e714c03 Puppeteer rewrite (#119)
* Moving to puppeteer | removing scrapingAnt
2024-12-17 12:38:28 +01:00
weakmap@gmail.com
a4c5bfcbf7 fixing tests 2024-10-03 16:09:19 +02:00
weakmap@gmail.com
6d2ab5f958 making sure immowelt does not include suggested ranges 2024-10-03 16:03:47 +02:00
Christian Kellner
111ef8be43 fixing kleinanzeigen test 2024-09-05 13:36:02 +02:00
Christian Kellner
35feb772d7 upgrading dependencies, fixing immowelt, using hash of price and id as unique identifier for listings 2024-09-05 13:34:14 +02:00
Christian Kellner
92db8219b4 building multi platform docker images (#101)
* building multi platform docker images

* upgrading dependencies | using scraping ant for neubaukompass
2024-07-24 09:32:21 +02:00
Christian Kellner
06c4ebb975 fixing immoswp 2024-06-12 14:15:21 +02:00
weakmap@gmail.com
27e098c244 upgrading dependencies, dropping support for node < 18. Happy new Year 2024-01-01 16:14:25 +01:00
weakmap@gmail.com
88c6641485 fixing wgGesucht test 2023-09-07 20:40:07 +02:00
Christian Kellner
d947dad488 fixing ebay kleinanzeigen, now becoming kleinanzeigen 2023-05-08 09:32:07 +02:00
weakmap@gmail.com
4ba098e0b6 bringing back immonet by using scrapingant 2023-04-15 18:24:51 +02:00
weakmap@gmail.com
1feb5bfda1 running all tests 2023-04-07 19:45:24 +02:00
weakmap@gmail.com
3ec9ed3b2a ignoring expired ssl certificate o0 2023-04-07 19:44:59 +02:00
Christian Kellner
2c5eceb0c1 Making Fredy an ESM project (#70)
Making Fredy an ESM project
2023-03-13 13:42:43 +01:00
Quoc Duong Bui
47e4230b39 Provider for immobilien.de (#65)
* Add provider for immobilien.de
2022-12-19 19:29:13 +01:00
weakmap@gmail.com
ab5ee59d72 ugrading dependencies | fixing tests | supporting multiple provider of the same type 2022-12-11 20:07:18 +01:00
Christian Kellner
bc01806421 fixing telegram provider not respecting rate limits 2022-06-10 14:19:20 +02:00
orangecoding
6e6144e02f fixing tests 2022-02-15 13:14:19 +01:00
Jochen Schalanda
2f5cc31ae3 Add support for Immo Südwest Presse (immo.swp.de) (#45) 2022-01-26 14:41:44 +01:00
orangecoding
0f7a07e7fb fixing immowelt test 2021-12-14 20:36:00 +01:00
orangecoding
c1c4d55ede adding or replacing sort params for provider urls when necessary 2021-11-26 21:02:09 +01:00
Christian Kellner
ec986e4b18 adding comma separation for MailJet & Sendgrid | fixing immowelt | upgrading dependencies (#34) 2021-10-06 10:45:13 +02:00
Christian Kellner
59e6d287fc adding similarity check (#29)
* adding similarity check

* adding paging

* fixing tests

* docu

* better error handling

* fixing tests

* adjusting page limit

* fixing login screen

* cleanup

* upgrade browser list

* prevent spamming the log

* fixing tests

* removing job listings when removing a job or the user
2021-06-28 08:52:09 +02:00
Christian Kellner
97858b7539 General settings (#28)
adding ui for general settings | adding 'working hours' as new feature
2021-05-30 09:37:45 +02:00
Christian Kellner
70cab66651 Bringing back immoscout (#21)
Bringing back immoscout support 🎉
2021-05-11 11:25:14 +02:00
orangecoding
a3420ab4ef showin price on ebay listings 2021-03-06 21:26:39 +01:00
orangecoding
bb3e62fa03 fixing ebay kleinanzeigen test 2021-03-01 17:59:33 +01:00
Christian Kellner
b2847f6834 UI (#15)
Adding new Admin UI. Updating Fredy to V3.0.0 as it has been a large rewrite. Thanks for all contributions and help on the way!
2021-01-21 16:09:23 +01:00
Christian Kellner
d67fd87155 enable all tests 🔥 2020-03-15 19:16:01 +01:00
Christian Kellner
200e3217a4 fixing kalaydo provider 2020-03-15 16:10:49 +01:00