90 Commits

Author SHA1 Message Date
orangecoding
921057252d adding immoscout shape search 2026-05-07 19:11:47 +02:00
orangecoding
3d10dc6042 moving from restana to fastify 2026-04-27 16:56:04 +02:00
orangecoding
c78472bd19 adding 'open in fredy' 2026-04-21 19:42:39 +02:00
orangecoding
8c5607e20b adding test fixtures so that we can run tests 'offline' 2026-04-21 13:37:00 +02:00
orangecoding
e2d10d179e next release version 2026-04-12 09:21:08 +02:00
Stephan
10c94eea0a Feature/spec filter (#276)
* feat(): create map component, add area filtering to the job config

* feat(): filter listings by area filter

* chore(): cleanup

* feat(): solve feedback

* feat(): solve most providers

* feat(): solve maybe other providers

* feat(): add specFilter config, also add rooms to listing

* feat(): change tests

* feat(): fix kleinanzeigen parser

* feat(): add spec filter switch for listing overviiews

* feat(): add rooms and size to the overview and detail of a listing

* feat(): rem label

* feat(): add types, update providers, they now return specs as numbers

* feat(): add jsonconfig to enable type checks

* feat: add type for prividerConfig, add fieldNames per provider

* feat: fix tests, provider, add formatListing

* chore: remov duplicates

* feat(): fix tests

* feat: fix immoscout

* chore: geojson typing

* feat: solve requested changes
2026-04-12 09:17:23 +02:00
Christian Kellner
cdc0cbda2f Feature/kleinanzeigen new (#292)
* Feature/Kleinanzeigen addresses (#289)

* upgrade dependencies

* immoscout_details -> provider_details

* fetching details more generic

* removing claude action

* fixing sparkassen selector

* improvements

* fixing immobilienDE test

* upgrading dependencies

* settings for many provider

---------

Co-authored-by: Adrian Bach <65734063+realDayaa@users.noreply.github.com>
2026-04-07 19:53:40 +02:00
orangecoding
cbf2766783 cleanup 2026-03-16 14:48:01 +01:00
orangecoding
1b39e345b6 moving from jest to vitest 2026-03-16 14:26:58 +01:00
orangecoding
4596442f64 upgrading dependencies | mark listings as 'manually_removed' when filtered 2026-03-08 09:55:46 +01:00
Stephan
0bcfa1d4ad feat(): map area filter (#273)
* feat(): create map component, add area filtering to the job config

* feat(): filter listings by area filter

* chore(): cleanup

* feat(): solve feedback

* feat(): solve most providers

* feat(): solve maybe other providers
2026-03-08 09:44:18 +01:00
Christian Kellner
00d6a12b30 Puppeteer improvements (#270)
* improve puppeteer handling. Now only 1 puppeteer instance is being used which is WAY more efficient

* removing package-lock

* reduce logging

* removing problematic docker command

* Remove Immonet. They now belong to immowelt
2026-02-18 20:05:02 +01:00
orangecoding
3117044139 fixing immoscout scraper 2026-01-26 19:52:37 +01:00
Christian Kellner
4dd0370ec1 Calculating the distance (#255)
* migra for distance

* adding distance calculator

* adding ability to store home address

* improve distance calculation

* calculating distance

* show distance in grid view

* upgrading dependencies

* moving to react 19

* ability to clone a job

* fixing tests

* polishing
2026-01-22 16:09:36 +01:00
Christian Kellner
d43c5b3f97 Map View in Fredy :D (#253)
* init map view

* switching off 3d buildings when sattelite view is on

* rename menu items

* upgrading dependencies, adding provider to popups

* adding screenshot for map view

* fixing readme

* next release version
2026-01-12 15:00:36 +01:00
orangecoding
7fd8be07a2 adding wohnungsboerse provider 2026-01-09 11:37:03 +01:00
orangecoding
5dc976c7e3 ability to start jobs individually 2025-12-18 19:16:28 +01:00
orangecoding
05f1bc61c9 fixing tests 2025-12-17 16:35:24 +01:00
orangecoding
6e8a35a836 adding backup/restore ability 2025-12-17 15:48:56 +01:00
orangecoding
790c559316 foced to move to Apache 2.0 license 2025-12-11 10:40:55 +01:00
orangecoding
5bd4219743 upgrading dependencies | adding ohneMakler provider 2025-12-08 20:31:28 +01:00
orangecoding
22df683969 more efficient bot protection 2025-11-27 10:30:47 +01:00
orangecoding
79a8420dfb improving similarity cache 2025-10-29 09:36:05 +01:00
orangecoding
0436c7f7d7 upgrading dependencies / FredyRuntime >> FredyPipeline 2025-10-12 16:43:56 +02:00
Thomas Brockmöller
7ebd73c9cf Add new provider McMakler (#201) 2025-09-28 14:16:28 +02:00
Thomas Brockmöller
4d37e890ab Add provider for Regionalimmobilien24 (#197) 2025-09-27 14:19:37 +02:00
Thomas Brockmöller
7589f20a18 Add sparkasse immobilien (#199) 2025-09-27 09:43:24 +02:00
Thomas Brockmöller
702ffabc1a Fix and improve immowelt/immonet provider (#194)
* Fix and improve immowelt provider

* Add description to immonet provider

* Fix tests and improve readability
2025-09-27 09:42:08 +02:00
Christian Kellner
8324357edb Improvements (#193)
* improving release banner

* renaming general to settings

* fixing working hours if they go to next day

* fixing comparing versions

* upgrade dependencies
2025-09-26 10:45:55 +02:00
Thomas Brockmöller
dd5c5b29d9 Fix address value in similarity filtering (#191)
* Fix address field in similarity filter
2025-09-25 15:02:00 +02:00
Christian Kellner
c839f3abc9 Check if a listing is still active (#184)
* check if a listing is still active

* upgrade dependencies
2025-09-22 09:57:50 +02:00
orangecoding
da8fd13973 fixing immoscout 2025-09-19 21:11:28 +02:00
orangecoding
28f0a167e6 fixing docker migration path 2025-09-18 17:28:30 +02:00
Christian Kellner
8d95f052c6 Migrate to SQLite (#174)
* Migrating Fredy from LowDb to SqLite 🎉

* adding new sql migration system for future sql migrations

* adding setting to change  sqlite path for db files

* create migration plan for graceful migration lowdb -> sqlite

* Improving Documentation

* adding test for sqliteconnection

* upgrading dependencies

* making nodejs 22 as min version

* improve scraper

* adding overwrite ability for db migra
2025-09-18 15:38:23 +02:00
orangecoding
09c6ce1d0b improve similarity cache. It now checks for similarities independend from jobs 2025-09-07 22:15:14 +02:00
Christian Kellner
1854b421af avoid warnings on test 2025-09-03 14:47:56 +02:00
Christian Kellner
f0b146fd7f Adding images to scraping data (#157)
* Fredy now supports pulling the main Image from the listing and send it together with the usual information
2025-08-30 21:21:34 +02:00
Nic
3a54ab0e31 Fix typo in test import (#154)
* Fix typo in test import

* Rename test file
2025-08-25 20:42:40 +02:00
Alexander Roidl
2b36f868e7 Project-wide linting and formatting (#150)
* chore: configure project-wide linting and formatting

* chore: run lint autofix and formatter
2025-07-26 20:42:58 +02:00
Alexander Roidl
cca1463a68 chore: run formatter (#145) 2025-07-23 08:47:26 +02:00
Christian Kellner
f032e6a724 test: verify unrelated text yields no similarity (#130) 2025-06-04 09:15:53 +02:00
Christian Kellner
f9c6b10976 fixing tests 2025-05-26 10:43:13 +02:00
Leon C.
1f54bcfd3f ImmoScout: Allow web paths with SEO optimization to be filtered to query params (#128) 2025-05-20 12:44:43 +02:00
Christian Kellner
030e0ca169 starting docu on reverse engineering immoscout api (#127)
* starting docu on reverse engineering immoscout api

* improving immoscout reverse engineering and adding support for most other types
2025-05-14 13:58:58 +02:00
Christian Kellner
cd3631f910 fixing new immoscout url handling 2025-05-09 10:05:30 +02:00
Christian Kellner
48e2ca942f fixing tests, renaming immoscout-mobile to immoscout 2025-05-09 09:26:24 +02:00
Patrick Klein
b9e4bca244 Add immoscout mobile API provider to avoid failing bot checks (#125)
* Add provider that uses the immoscout mobile API to avoid failing bot checks.
2025-05-09 09:13:52 +02:00
weakmap@gmail.com
a3471a091a upgrade dependencies, fixing tests 2025-02-23 17:13:08 +01:00
Christian Kellner
214e714c03 Puppeteer rewrite (#119)
* Moving to puppeteer | removing scrapingAnt
2024-12-17 12:38:28 +01:00
weakmap@gmail.com
a4c5bfcbf7 fixing tests 2024-10-03 16:09:19 +02:00