Commit Graph

82 Commits

Author SHA1 Message Date
orangecoding
e2d10d179e next release version 2026-04-12 09:21:08 +02:00
Stephan
10c94eea0a Feature/spec filter (#276)
* feat(): create map component, add area filtering to the job config

* feat(): filter listings by area filter

* chore(): cleanup

* feat(): solve feedback

* feat(): solve most providers

* feat(): solve maybe other providers

* feat(): add specFilter config, also add rooms to listing

* feat(): change tests

* feat(): fix kleinanzeigen parser

* feat(): add spec filter switch for listing overviiews

* feat(): add rooms and size to the overview and detail of a listing

* feat(): rem label

* feat(): add types, update providers, they now return specs as numbers

* feat(): add jsonconfig to enable type checks

* feat: add type for prividerConfig, add fieldNames per provider

* feat: fix tests, provider, add formatListing

* chore: remov duplicates

* feat(): fix tests

* feat: fix immoscout

* chore: geojson typing

* feat: solve requested changes
2026-04-12 09:17:23 +02:00
Christian Kellner
cdc0cbda2f Feature/kleinanzeigen new (#292)
* Feature/Kleinanzeigen addresses (#289)

* upgrade dependencies

* immoscout_details -> provider_details

* fetching details more generic

* removing claude action

* fixing sparkassen selector

* improvements

* fixing immobilienDE test

* upgrading dependencies

* settings for many provider

---------

Co-authored-by: Adrian Bach <65734063+realDayaa@users.noreply.github.com>
2026-04-07 19:53:40 +02:00
orangecoding
0cad05124a fixing immowelt provider 2026-03-08 09:29:40 +01:00
Noah Elijah Till
eb53b68d45 🕵️ More immoscout details (#258)
* 🕵️ More immoscout details

- Added more details to immoscout api - description is now populated with a lot of data from the expose using app API
- You can ignore certificates, if deploying locally and using the http notification adapter
- More details for the test call/example for easier testing + placeholder image + actual values + address (famous Erika Mustermans address see https://de.wikipedia.org/wiki/Mustermann)
- Grater timeout for geocode since the api is sometimes slow in germany
- uiElement, type boolean, now has a label as well

* 👀 Requested changes + some extra

Req:
- using logger
- using node-fetch

Extra:
- boolean input fields will trigger the validate check, because they are set undefined at first - setting them to false if they are undefined now
- added more data to the description (phone number and name of the agent)

*  Fixed import

* ️ Toggle immoscout detail fetching

* ️ Requested change
2026-03-08 09:08:40 +01:00
Christian Kellner
00d6a12b30 Puppeteer improvements (#270)
* improve puppeteer handling. Now only 1 puppeteer instance is being used which is WAY more efficient

* removing package-lock

* reduce logging

* removing problematic docker command

* Remove Immonet. They now belong to immowelt
2026-02-18 20:05:02 +01:00
orangecoding
3117044139 fixing immoscout scraper 2026-01-26 19:52:37 +01:00
orangecoding
afd1048c9e hardening the check if a listing is active 2026-01-26 12:34:49 +01:00
orangecoding
51b4e51f3f fixing setting kleinanzeigen listings to inactive if not available anymore 2026-01-16 11:36:51 +01:00
Christian Kellner
d43c5b3f97 Map View in Fredy :D (#253)
* init map view

* switching off 3d buildings when sattelite view is on

* rename menu items

* upgrading dependencies, adding provider to popups

* adding screenshot for map view

* fixing readme

* next release version
2026-01-12 15:00:36 +01:00
orangecoding
7fd8be07a2 adding wohnungsboerse provider 2026-01-09 11:37:03 +01:00
Timur
ad46500d4e Fix: correct baseUrl for ohne-makler provider - Fixes #251 (#252) 2026-01-02 08:36:39 +01:00
orangecoding
790c559316 foced to move to Apache 2.0 license 2025-12-11 10:40:55 +01:00
orangecoding
5bd4219743 upgrading dependencies | adding ohneMakler provider 2025-12-08 20:31:28 +01:00
orangecoding
53d5098cec fixing wrong number extraction 2025-11-03 20:01:55 +01:00
Thomas Brockmöller
7ebd73c9cf Add new provider McMakler (#201) 2025-09-28 14:16:28 +02:00
Thomas Brockmöller
4d37e890ab Add provider for Regionalimmobilien24 (#197) 2025-09-27 14:19:37 +02:00
Thomas Brockmöller
7589f20a18 Add sparkasse immobilien (#199) 2025-09-27 09:43:24 +02:00
Thomas Brockmöller
702ffabc1a Fix and improve immowelt/immonet provider (#194)
* Fix and improve immowelt provider

* Add description to immonet provider

* Fix tests and improve readability
2025-09-27 09:42:08 +02:00
Christian Kellner
11fd18e76a Puppeteer improvements (#186)
* improving puppeteer handling

* upgrade dependencies

* reduce logging

* upgrade nanoid
2025-09-22 20:53:00 +02:00
Christian Kellner
c839f3abc9 Check if a listing is still active (#184)
* check if a listing is still active

* upgrade dependencies
2025-09-22 09:57:50 +02:00
orangecoding
da8fd13973 fixing immoscout 2025-09-19 21:11:28 +02:00
Christian Kellner
8d95f052c6 Migrate to SQLite (#174)
* Migrating Fredy from LowDb to SqLite 🎉

* adding new sql migration system for future sql migrations

* adding setting to change  sqlite path for db files

* create migration plan for graceful migration lowdb -> sqlite

* Improving Documentation

* adding test for sqliteconnection

* upgrading dependencies

* making nodejs 22 as min version

* improve scraper

* adding overwrite ability for db migra
2025-09-18 15:38:23 +02:00
orangecoding
21415dcff3 using winston logger 2025-09-13 18:57:56 +02:00
weakmap@gmail.com
9db1ffd8eb making immonet null safe 2025-08-31 20:25:52 +02:00
weakmap@gmail.com
1cb79d1287 making immoscout image scraping null safe 2025-08-31 20:19:32 +02:00
Christian Kellner
f0b146fd7f Adding images to scraping data (#157)
* Fredy now supports pulling the main Image from the listing and send it together with the usual information
2025-08-30 21:21:34 +02:00
Alexander Roidl
2302f69ff3 Rename NPM startup scripts (#144)
* feat: rename npm start scripts
2025-07-25 13:13:04 +02:00
Alexander Roidl
cca1463a68 chore: run formatter (#145) 2025-07-23 08:47:26 +02:00
Christian Kellner
f9c6b10976 fixing tests 2025-05-26 10:43:13 +02:00
Christian Kellner
030e0ca169 starting docu on reverse engineering immoscout api (#127)
* starting docu on reverse engineering immoscout api

* improving immoscout reverse engineering and adding support for most other types
2025-05-14 13:58:58 +02:00
Christian Kellner
f1effe941f fixing immoscout url and description 2025-05-09 11:00:35 +02:00
Christian Kellner
cd3631f910 fixing new immoscout url handling 2025-05-09 10:05:30 +02:00
Christian Kellner
48e2ca942f fixing tests, renaming immoscout-mobile to immoscout 2025-05-09 09:26:24 +02:00
Patrick Klein
b9e4bca244 Add immoscout mobile API provider to avoid failing bot checks (#125)
* Add provider that uses the immoscout mobile API to avoid failing bot checks.
2025-05-09 09:13:52 +02:00
Christian Kellner
a138dafc31 fixing immoweltsp title 2025-03-31 18:38:18 +02:00
Stefan
3903ab59cf fix normalized wggesucht link (#123) 2025-01-17 22:05:34 +01:00
Christian Kellner
e1db3840f6 adding puppeteer timeout and fixing waitForSelector 2025-01-07 12:37:50 +01:00
weakmap@gmail.com
b473cf7fb4 fixing kleinanzeigen test 2024-12-26 19:18:30 +01:00
Christian Kellner
214e714c03 Puppeteer rewrite (#119)
* Moving to puppeteer | removing scrapingAnt
2024-12-17 12:38:28 +01:00
weakmap@gmail.com
3c0e9e56c6 fixing immowelt 2024-12-10 09:08:25 +01:00
weakmap@gmail.com
6d2ab5f958 making sure immowelt does not include suggested ranges 2024-10-03 16:03:47 +02:00
weakmap@gmail.com
d3cb3a5881 regex for einsAImmobilien price normalization | filter listings that does not have all required keys 2024-09-29 16:58:01 +02:00
Christian Kellner
35feb772d7 upgrading dependencies, fixing immowelt, using hash of price and id as unique identifier for listings 2024-09-05 13:34:14 +02:00
Christian Kellner
92db8219b4 building multi platform docker images (#101)
* building multi platform docker images

* upgrading dependencies | using scraping ant for neubaukompass
2024-07-24 09:32:21 +02:00
Christian Kellner
06c4ebb975 fixing immoswp 2024-06-12 14:15:21 +02:00
Christian Kellner
a85400d570 fixing immoscout 2024-02-08 10:36:47 +01:00
Farasath Ahamed
4ab1fd9294 Update immoscout.js (#88)
Fixes https://github.com/orangecoding/fredy/issues/87
2024-01-26 19:33:45 +01:00
jstnw
72993312c7 fix: kleinanzeigen price (#82) 2023-10-05 18:33:55 +02:00
Christian Kellner
d947dad488 fixing ebay kleinanzeigen, now becoming kleinanzeigen 2023-05-08 09:32:07 +02:00