Compare commits

..

358 Commits

Author SHA1 Message Date
orangecoding
1c9d7c9d92 storing filter settings in url 2026-03-31 11:46:22 +02:00
orangecoding
bc73de6703 upgrade dependencies 2026-03-31 10:38:50 +02:00
orangecoding
568e0abfa1 fixing login not showing if username or password is incorrect 2026-03-31 09:18:49 +02:00
Stephan
3992a9c81c fix: maplibre-gl runtime errors in production build by isoliting it into a chunk (#288) 2026-03-31 09:14:49 +02:00
Christian Kellner
7346075b9d Add Claude Code GitHub Workflow (#285)
* "Claude PR Assistant workflow"

* "Claude Code Review workflow"
2026-03-24 08:40:04 +01:00
Christian Kellner
8c039f0026 UI improvements (#283)
* ui-improvements

* improving dashboard and settings

* improve job overview

* improving job card

* improving grid view of listings+

* restructuring settings

* next release version
2026-03-23 13:22:34 +01:00
orangecoding
a1289acf15 fixing some docker issues 2026-03-22 09:41:20 +01:00
orangecoding
8501fc7266 upgrading dependencies 2026-03-21 08:09:15 +01:00
orangecoding
4960846cd7 fixing docker run 2026-03-21 08:08:52 +01:00
orangecoding
3ed17f4442 fixing broken puppeteer providers in docker caused by alpine chromium 146 crashing / switched to debian slim with puppeteer's own chrome for testing / dropped 2-stage build / run as non-root / purge build tools after install, improve docker-test.sh to verify it all works. That's it. ;) 2026-03-20 19:19:20 +01:00
orangecoding
b531a7b77a fixing mcp issue, adding claude example 2026-03-20 13:45:42 +01:00
Adrian Bach
3523057221 feat: add smtp adapter (#279) 2026-03-20 11:37:28 +01:00
orangecoding
77311cf39d next release version 2026-03-17 11:26:39 +01:00
orangecoding
556c0aff35 fixing duplicate migration 2026-03-17 11:26:23 +01:00
orangecoding
c40d275e52 cleanup 2026-03-16 14:48:41 +01:00
orangecoding
cbf2766783 cleanup 2026-03-16 14:48:01 +01:00
orangecoding
1b39e345b6 moving from jest to vitest 2026-03-16 14:26:58 +01:00
orangecoding
6ccbdd8afc upgrading dependencies 2026-03-16 10:41:53 +01:00
orangecoding
2a30c89eb2 improving version banner 2026-03-16 10:37:36 +01:00
orangecoding
4878dc98e3 Merge branch 'master' of github.com:orangecoding/fredy 2026-03-11 15:26:56 +01:00
orangecoding
dc2704997d upgrading dependencies 2026-03-11 15:26:25 +01:00
orangecoding
e107b0fb00 next release version 2026-03-11 15:25:20 +01:00
Promises
6c08675fee Add new properties to real estate translation mappings (#275)
Added few more properties for buying a house
2026-03-11 14:49:21 +01:00
orangecoding
34c4de7267 fixing stdin for mcp 2026-03-10 09:27:04 +01:00
orangecoding
b64a118a18 moving mcp into lib to make it available in docker setup 2026-03-09 16:26:53 +01:00
orangecoding
03cb4d18cb fix formatting 2026-03-09 15:40:29 +01:00
orangecoding
be5c4af3cf adding an MCP Server 🎉 2026-03-09 15:35:29 +01:00
orangecoding
a460b813c1 adding news info 2026-03-08 10:07:51 +01:00
orangecoding
4596442f64 upgrading dependencies | mark listings as 'manually_removed' when filtered 2026-03-08 09:55:46 +01:00
Stephan
0bcfa1d4ad feat(): map area filter (#273)
* feat(): create map component, add area filtering to the job config

* feat(): filter listings by area filter

* chore(): cleanup

* feat(): solve feedback

* feat(): solve most providers

* feat(): solve maybe other providers
2026-03-08 09:44:18 +01:00
orangecoding
0cad05124a fixing immowelt provider 2026-03-08 09:29:40 +01:00
Noah Elijah Till
eb53b68d45 🕵️ More immoscout details (#258)
* 🕵️ More immoscout details

- Added more details to immoscout api - description is now populated with a lot of data from the expose using app API
- You can ignore certificates, if deploying locally and using the http notification adapter
- More details for the test call/example for easier testing + placeholder image + actual values + address (famous Erika Mustermans address see https://de.wikipedia.org/wiki/Mustermann)
- Grater timeout for geocode since the api is sometimes slow in germany
- uiElement, type boolean, now has a label as well

* 👀 Requested changes + some extra

Req:
- using logger
- using node-fetch

Extra:
- boolean input fields will trigger the validate check, because they are set undefined at first - setting them to false if they are undefined now
- added more data to the description (phone number and name of the agent)

*  Fixed import

* ️ Toggle immoscout detail fetching

* ️ Requested change
2026-03-08 09:08:40 +01:00
Tom Dohrmann
ba0732e1f6 add support for fulltext parameter for immoscout (#274) 2026-02-24 09:53:14 +01:00
orangecoding
aa67647bbb adding resend as net notification adapter 2026-02-20 17:08:38 +01:00
orangecoding
7a9d49899b improve reusing of puppeteer by adding a safeguard for broken chrome 2026-02-18 20:16:55 +01:00
orangecoding
9a87c58d3e next release version 2026-02-18 20:06:40 +01:00
orangecoding
fdd7e835e8 improve default puppeteer timeout 2026-02-18 20:06:22 +01:00
Christian Kellner
00d6a12b30 Puppeteer improvements (#270)
* improve puppeteer handling. Now only 1 puppeteer instance is being used which is WAY more efficient

* removing package-lock

* reduce logging

* removing problematic docker command

* Remove Immonet. They now belong to immowelt
2026-02-18 20:05:02 +01:00
orangecoding
05218800d2 fixing app init 2026-02-17 14:28:08 +01:00
orangecoding
19d4721f9f improve welcome screen 2026-02-17 14:03:15 +01:00
orangecoding
a794645393 fixing login route 2026-02-17 12:50:21 +01:00
orangecoding
fd7e228972 adding welcome screen 2026-02-17 12:35:39 +01:00
orangecoding
b86e351007 fixing lint even harder 2026-02-16 13:50:50 +01:00
orangecoding
19c4860da7 fixing eslint harder 2026-02-16 12:59:34 +01:00
orangecoding
d98e06cfdf fixing eslint 2026-02-16 12:40:41 +01:00
orangecoding
6ae0c9749b update dependencies 2026-02-16 12:30:59 +01:00
orangecoding
10e40e038e adding check if fredy is running in docker 2026-02-16 12:29:02 +01:00
orangecoding
4ba6828939 adding release tool 2026-02-05 12:02:18 +01:00
orangecoding
d09770dae2 fancy, almost impossible to see animation on dashboard 2026-02-05 09:54:42 +01:00
orangecoding
248e4d2562 improve tracking 2026-02-04 14:41:55 +01:00
orangecoding
7b8e961b49 adding confirmation dialog if to remove listing entirely from db or just hide it 2026-02-03 14:04:40 +01:00
orangecoding
f66ceccbb4 next release version 2026-01-29 13:01:39 +01:00
orangecoding
a3db725af6 fixing image rendering 2026-01-29 13:01:07 +01:00
orangecoding
0663bd945f smaller demo improvements 2026-01-29 09:46:23 +01:00
orangecoding
bc355fb5fe fixing some bugs the wife found ;) 2026-01-28 21:25:48 +01:00
orangecoding
797421f0d5 hardening demo handling 2026-01-28 16:29:59 +01:00
orangecoding
0b2b42fc75 improve geocoding 2026-01-28 15:55:23 +01:00
Christian Kellner
472169693f Improvements 01 28 (#264)
* improving footer

* improve ui

* upgrading dependencies

* adding glow to all boxes on dashboard

* introducing single listing view

* next release version

* improve screenshots and login page
2026-01-28 14:27:03 +01:00
orangecoding
3117044139 fixing immoscout scraper 2026-01-26 19:52:37 +01:00
orangecoding
7879d0e94a next release version 2026-01-26 12:35:57 +01:00
orangecoding
afd1048c9e hardening the check if a listing is active 2026-01-26 12:34:49 +01:00
orangecoding
acbaab05ed next release version 2026-01-26 12:07:43 +01:00
orangecoding
72fffc526b deleting a listing now sets it to deleted in the db, preventing it from reappearing when scraping happens 2026-01-26 12:07:21 +01:00
orangecoding
9e5989ece3 zoom into map where most markers are 2026-01-26 11:54:47 +01:00
orangecoding
afc200c9e1 improved tooltip in map, improved user-settings handling 2026-01-26 11:50:16 +01:00
orangecoding
59226491f2 improved tooltip in map, improved user-settings handling 2026-01-26 11:20:02 +01:00
orangecoding
28f7760120 adapt link to listing in grid view to behave like a real link 2026-01-26 10:43:38 +01:00
orangecoding
2465514b7a fixing immoscout translator, allowing balcony and garden for purchases 2026-01-26 10:20:21 +01:00
Christian Kellner
9dde377fe6 possibility to display distance (#262) 2026-01-25 13:52:56 +01:00
Katrin Leinweber
28a3a7f372 Use EUR-symbol to match Map.jsx (see d43c5b3) (#261)
Co-authored-by: Katrin Leinweber <katrinleinweber@noreply.github.com>
2026-01-25 12:32:11 +01:00
orangecoding
e859250545 next release version 2026-01-22 15:10:31 +00:00
Christian Kellner
4dd0370ec1 Calculating the distance (#255)
* migra for distance

* adding distance calculator

* adding ability to store home address

* improve distance calculation

* calculating distance

* show distance in grid view

* upgrading dependencies

* moving to react 19

* ability to clone a job

* fixing tests

* polishing
2026-01-22 16:09:36 +01:00
orangecoding
51b4e51f3f fixing setting kleinanzeigen listings to inactive if not available anymore 2026-01-16 11:36:51 +01:00
orangecoding
fa1899765c fixing some rendering issues in map 2026-01-16 10:46:50 +01:00
Christian Kellner
d43c5b3f97 Map View in Fredy :D (#253)
* init map view

* switching off 3d buildings when sattelite view is on

* rename menu items

* upgrading dependencies, adding provider to popups

* adding screenshot for map view

* fixing readme

* next release version
2026-01-12 15:00:36 +01:00
orangecoding
7fd8be07a2 adding wohnungsboerse provider 2026-01-09 11:37:03 +01:00
orangecoding
2926ee7e08 upgrading dependencies 2026-01-06 09:51:04 +01:00
Christian Kellner
9506d1a9db next release version 2026-01-06 08:13:39 +01:00
Christian Kellner
feaa06c132 Update LICENSE to 2026 2026-01-04 06:46:32 +01:00
Timur
ad46500d4e Fix: correct baseUrl for ohne-makler provider - Fixes #251 (#252) 2026-01-02 08:36:39 +01:00
Christian Kellner
3c209a8f97 Redesigning listing table (#248)
* redesigning listing table

* getting rid of old listing table view

* improving listing grid
2025-12-23 08:47:51 +01:00
orangecoding
398259ff20 next release version 2025-12-18 19:25:33 +01:00
orangecoding
cf030bfa39 next release version / fixing valuers not being shown when editing a notification adapter 2025-12-18 19:24:48 +01:00
orangecoding
5dc976c7e3 ability to start jobs individually 2025-12-18 19:16:28 +01:00
orangecoding
05f1bc61c9 fixing tests 2025-12-17 16:35:24 +01:00
orangecoding
6e8a35a836 adding backup/restore ability 2025-12-17 15:48:56 +01:00
orangecoding
87771655a8 adding new dashboard view. Muchas wow 2025-12-14 12:23:59 +01:00
Christian Kellner
87b5673bf0 Update package.json 2025-12-12 22:22:50 +01:00
lorem-ipsum-dolor-sit
9291155cc2 fix: catch error (#246) 2025-12-12 22:21:49 +01:00
orangecoding
ac90d4122b next release version 2025-12-11 10:42:41 +01:00
orangecoding
790c559316 foced to move to Apache 2.0 license 2025-12-11 10:40:55 +01:00
orangecoding
2a815c92e6 reduce docker image size 2025-12-10 13:23:17 +01:00
orangecoding
cef9b5c8fc fixing default configs 2025-12-10 09:25:09 +01:00
Christian Kellner
1e2476a375 Update README.md 2025-12-10 09:09:20 +01:00
orangecoding
78b762bd9e fixing analytics popup 2025-12-09 14:57:29 +01:00
Christian Kellner
3e5cd97400 Listing management (#223)
* upgrading dependencies, fixing image placeholder

* improving processing times label and hide when screen width is too low

* aligning run now button

* renaming settings -> general settings

* smaller security and memory improvements

* improving footer

* preparing listing management

* improve filtering for listings

* preparing new settings page

* preparing new settings page

* storing settings in db

* next release version
2025-12-09 13:56:46 +01:00
orangecoding
5cfa674d7f adding unraid logo 2025-12-09 09:17:21 +01:00
orangecoding
5bd4219743 upgrading dependencies | adding ohneMakler provider 2025-12-08 20:31:28 +01:00
orangecoding
ea24eb4374 upgrading dependencies 2025-12-04 09:58:58 +01:00
orangecoding
9f67e30ff4 upgrade version 2025-11-27 16:09:44 +01:00
orangecoding
20d44b60ad upgrading dependencies 2025-11-27 15:54:54 +01:00
orangecoding
22df683969 more efficient bot protection 2025-11-27 10:30:47 +01:00
Robin Fuchs
4aab850b4f feat: updated the UI to enable editing of provider URLs (#234)
* feat: updated the UI to enable editing of provider URLs


---------

Co-authored-by: foxx-tech <robin.foxx.tech@gmail.com>
2025-11-26 17:10:42 +01:00
Efe
3eb3f6ee66 fix: notification adapter modal improvements (#230)
* fix: fix notification modal
2025-11-18 12:24:27 +01:00
Efe
1b2fc79536 feat: add http adapter (#231)
* feat: add http adapter
2025-11-18 12:23:50 +01:00
orangecoding
0606122736 improving bot detection prevention 2025-11-16 19:59:08 +01:00
orangecoding
53d5098cec fixing wrong number extraction 2025-11-03 20:01:55 +01:00
Christian Kellner
32c7518454 Listing improvements (#222)
* upgrading dependencies, fixing image placeholder

* improving processing times label and hide when screen width is too low

* aligning run now button

* renaming settings -> general settings

* smaller security and memory improvements

* improving footer
2025-11-01 10:46:55 +01:00
orangecoding
db3702ed33 improve markdown readme's & and adding ability to send telegram messages to a topic in a supergroup 2025-10-30 12:42:03 +01:00
orangecoding
e3c62d4696 fixing test runner 2025-10-29 10:35:07 +01:00
orangecoding
79a8420dfb improving similarity cache 2025-10-29 09:36:05 +01:00
orangecoding
d433b13db6 next release version 2025-10-12 16:47:46 +02:00
orangecoding
41d9274dfd reducing logging 2025-10-12 16:47:28 +02:00
orangecoding
0436c7f7d7 upgrading dependencies / FredyRuntime >> FredyPipeline 2025-10-12 16:43:56 +02:00
Christian Kellner
a1cb57318e Update README.md 2025-10-11 17:37:51 +02:00
orangecoding
2566db9805 improve index 2025-10-08 15:00:28 +02:00
orangecoding
b48f786fd3 improve docu 2025-10-08 12:16:10 +02:00
orangecoding
9c74129489 fixing listings 2025-10-07 21:22:29 +02:00
orangecoding
33120ebeca ability to share jobs with users 2025-10-07 21:06:59 +02:00
orangecoding
de2dd05c70 reverting docker file change 2025-10-07 07:18:45 +02:00
orangecoding
e4784e5960 reverting docker file change 2025-10-06 20:21:26 +02:00
orangecoding
2e537ce0be improving ntfy error handling 2025-10-06 20:19:53 +02:00
orangecoding
f0f1244baa using docker without root 2025-10-06 19:55:37 +02:00
orangecoding
b858529f06 next release version 2025-10-05 18:57:52 +02:00
orangecoding
c9bd5dc161 fixing delete listings 2025-10-05 18:57:27 +02:00
orangecoding
daa4a7b8f1 refine telegram adapter 2025-10-05 18:53:17 +02:00
Thomas Brockmöller
035f0e9f83 Check Telegram response (#205) (#211)
* Add error handling and logging to Telegram message sending

* Add debug logging for new listings
2025-10-05 17:06:57 +02:00
Christian Kellner
a5efd9af32 New Feature: Watch Listings (#215)
* adding new feature: watch listings for changes

* adding todo for watch feature

* sort by watch
2025-10-05 14:23:32 +02:00
orangecoding
9f1e27d011 check if fredy config exists and is accessible 2025-10-03 17:23:46 +02:00
orangecoding
ebc57702dc next release version 2025-10-03 13:28:09 +02:00
orangecoding
3aa30bc1e2 remove listings from listingstable when clicked 2025-10-03 13:27:44 +02:00
orangecoding
f97fb48e51 Merge branch 'master' of github.com:orangecoding/fredy 2025-10-03 13:04:56 +02:00
orangecoding
4b15894603 adding buttons to remove listings from a given job 2025-10-03 13:04:35 +02:00
orangecoding
31a14a0352 improve footer and upgrade dependencies 2025-10-03 12:45:48 +02:00
Christian Kellner
eecbe91dbd Update README.md 2025-10-02 22:05:49 +02:00
orangecoding
9dd3947cb7 reverting docker file changes, adding script to test things locally 2025-10-02 09:37:01 +02:00
Iaroslav Postovalov
c151f4f76e Use non-root user in Dockerfile (#214) 2025-10-01 20:04:08 +02:00
Christian Kellner
b6755497e4 Ui-Redesign (#203)
* new ui design

* improving ui design

* adding new screenshots

* upgrade dependencies
2025-09-29 20:36:56 +02:00
rugk
412e24b1e3 Add VOLUME to Dockerfile (#208)
Notes/exposes the intended volumes as per best practices.

See https://docs.docker.com/build/building/best-practices/#volume
2025-09-29 12:31:32 +02:00
rugk
0a5785fa1a Specify GitHub image in docker-compose directly (#204)
It's recommend to specify the full "URL" and this aligns with the Readme and default docker would search on Docker Hub, where this is not available: https://hub.docker.com/search?q=fredy%2Ffredy
2025-09-29 12:31:08 +02:00
Thomas Brockmöller
7ebd73c9cf Add new provider McMakler (#201) 2025-09-28 14:16:28 +02:00
orangecoding
95cd4028d7 next release version 2025-09-28 08:13:03 +02:00
orangecoding
eb01c2107c fixing default header 2025-09-28 08:12:51 +02:00
orangecoding
42cd4fa0ae next release version 2025-09-27 18:15:58 +02:00
orangecoding
6d96fd2bf8 Merge branch 'master' of github.com:orangecoding/fredy 2025-09-27 18:15:42 +02:00
orangecoding
ff1d2317a1 improve default puppeteer header 2025-09-27 18:15:28 +02:00
orangecoding
a47fa41278 fixing smaller problems in apprise and mattermost 2025-09-27 18:07:48 +02:00
orangecoding
9654e56846 improving some labels 2025-09-27 18:01:42 +02:00
Christian Kellner
43094640a8 Update README.md 2025-09-27 14:27:25 +02:00
orangecoding
fa234d2d78 fixing code style issues in new discord adapter 2025-09-27 14:24:05 +02:00
orangecoding
7cb0d6e382 next release version 2025-09-27 14:22:09 +02:00
mari
d79f8d2664 Add Discord webhook adapter (#196)
* Add Discord webhook adapter
2025-09-27 14:20:43 +02:00
Thomas Brockmöller
4d37e890ab Add provider for Regionalimmobilien24 (#197) 2025-09-27 14:19:37 +02:00
Thomas Brockmöller
7589f20a18 Add sparkasse immobilien (#199) 2025-09-27 09:43:24 +02:00
Thomas Brockmöller
702ffabc1a Fix and improve immowelt/immonet provider (#194)
* Fix and improve immowelt provider

* Add description to immonet provider

* Fix tests and improve readability
2025-09-27 09:42:08 +02:00
orangecoding
9387de1cd9 next version 2025-09-26 13:09:22 +02:00
orangecoding
facd683d45 santizing ntfy header 2025-09-26 13:07:54 +02:00
Christian Kellner
8324357edb Improvements (#193)
* improving release banner

* renaming general to settings

* fixing working hours if they go to next day

* fixing comparing versions

* upgrade dependencies
2025-09-26 10:45:55 +02:00
Christian Kellner
67af7c7dc5 next version 2025-09-25 15:06:38 +02:00
Christian Kellner
6f5b52f3ad Merge branch 'master' of github.com:orangecoding/fredy 2025-09-25 15:06:25 +02:00
Christian Kellner
89d239c360 New Listings view (#192)
* completing found listings

---------

Co-authored-by: Christian Kellner <Christian.Kellner1@ibm.com>
2025-09-25 15:03:47 +02:00
Thomas Brockmöller
dd5c5b29d9 Fix address value in similarity filtering (#191)
* Fix address field in similarity filter
2025-09-25 15:02:00 +02:00
Christian Kellner
0cb2f48645 Merge branch 'master' of github.com:orangecoding/fredy 2025-09-25 09:47:16 +02:00
orangecoding
3f294b8099 next release version 2025-09-22 20:56:42 +02:00
Christian Kellner
11fd18e76a Puppeteer improvements (#186)
* improving puppeteer handling

* upgrade dependencies

* reduce logging

* upgrade nanoid
2025-09-22 20:53:00 +02:00
Christian Kellner
c839f3abc9 Check if a listing is still active (#184)
* check if a listing is still active

* upgrade dependencies
2025-09-22 09:57:50 +02:00
orangecoding
28eddc5d7f next release version 2025-09-20 19:49:32 +02:00
Iaroslav Postovalov
0ca9c5ae02 Add health check for Docker container (#179)
- Introduced `HealthCheck` in `docker-compose.yml` to monitor container status.
- Added a test step to validate container's health using Docker Compose in the GitHub workflow.
- Updated `Dockerfile` to include `curl` for health check commands.
2025-09-20 19:39:48 +02:00
orangecoding
a7d0037edd next release version 2025-09-20 19:37:47 +02:00
orangecoding
f339a2e2cf adding version banner to check if a new version of fredy is available 2025-09-20 19:37:27 +02:00
orangecoding
da8fd13973 fixing immoscout 2025-09-19 21:11:28 +02:00
orangecoding
7deffc64af next release version 2025-09-18 20:48:49 +02:00
orangecoding
d1dad7fd3b adding new unique index, adding button to start now 2025-09-18 20:48:25 +02:00
Christian Kellner
4f79c5cba2 replacing rematch with zustand (#180)
* replacing rematch with zustand

* upgrading dependencies

* next release version
2025-09-18 20:09:11 +02:00
orangecoding
28e885f6c7 fixing migration checksum 2025-09-18 18:42:19 +02:00
orangecoding
1d99fc95f7 using cron to run demo cleanup every day at midnight 2025-09-18 18:04:49 +02:00
orangecoding
28f0a167e6 fixing docker migration path 2025-09-18 17:28:30 +02:00
Christian Kellner
8d95f052c6 Migrate to SQLite (#174)
* Migrating Fredy from LowDb to SqLite 🎉

* adding new sql migration system for future sql migrations

* adding setting to change  sqlite path for db files

* create migration plan for graceful migration lowdb -> sqlite

* Improving Documentation

* adding test for sqliteconnection

* upgrading dependencies

* making nodejs 22 as min version

* improve scraper

* adding overwrite ability for db migra
2025-09-18 15:38:23 +02:00
orangecoding
18fdbd761a next release version 2025-09-17 09:12:45 +02:00
Iaroslav Postovalov
027e7d70ed Update SQLite adapter: configurable database path (#169) 2025-09-17 09:12:04 +02:00
Christian Kellner
de119c9199 Update logger.js 2025-09-14 15:46:31 +02:00
orangecoding
ce7f0bca9f next release version 2025-09-14 10:40:41 +02:00
orangecoding
ae1c4d936b do not log debug on production 2025-09-14 10:40:18 +02:00
orangecoding
d01a1a94d0 Merge branch 'master' of github.com:orangecoding/fredy 2025-09-14 10:32:52 +02:00
orangecoding
bda4212249 improve logging 2025-09-14 10:32:39 +02:00
Christian Kellner
694809fedf Using white fredy logo on dark background 2025-09-13 22:20:50 +02:00
Christian Kellner
3cd1893b51 Update Jetbrains logo to use the correct one on dark background 2025-09-13 22:16:16 +02:00
orangecoding
21415dcff3 using winston logger 2025-09-13 18:57:56 +02:00
orangecoding
e868cdce86 Merge branch 'master' of github.com:orangecoding/fredy 2025-09-13 17:06:30 +02:00
orangecoding
d66dc2cd93 improve tracking 2025-09-13 17:06:18 +02:00
Christian Kellner
5e0405f1ec Update README.md 2025-09-12 18:47:10 +02:00
orangecoding
251de1e42d next release version 2025-09-12 13:48:05 +02:00
orangecoding
edc91291b6 fixing telegram 2025-09-12 13:45:54 +02:00
orangecoding
ac0ea64c07 remove unnecessary logging 2025-09-12 13:41:08 +02:00
orangecoding
9f7506a1b3 Merge branch 'master' of github.com:orangecoding/fredy 2025-09-12 13:39:15 +02:00
orangecoding
85cea66051 improving tracking. now using internal tracking 2025-09-12 13:38:53 +02:00
Christian Kellner
05c2df917c Adding link to fredy demo 2025-09-12 13:00:43 +02:00
Christian Kellner
4ad2895eec Update docker command 2025-09-10 11:31:49 +02:00
orangecoding
7372e5313f creating config automagically if missing 2025-09-09 18:41:14 +02:00
orangecoding
637a54e01e upgrading dependencies 2025-09-09 15:17:36 +02:00
orangecoding
04265eaec7 making sure scan interval does not go under 5 2025-09-08 08:30:45 +02:00
orangecoding
fa76821f7d next release version 2025-09-07 22:15:45 +02:00
orangecoding
09c6ce1d0b improve similarity cache. It now checks for similarities independend from jobs 2025-09-07 22:15:14 +02:00
Christian Kellner
7fa9a265ef Fixing docker command 2025-09-07 16:46:43 +02:00
Christian Kellner
f201090b56 Update README.md 2025-09-05 12:35:20 +02:00
Christian Kellner
dda5b5fbcb Update README.md 2025-09-05 12:34:03 +02:00
Christian Kellner
a93c7ffee5 Update README.md 2025-09-05 12:33:28 +02:00
Christian Kellner
79a2d967e8 Update README.md 2025-09-05 12:33:12 +02:00
Christian Kellner
c264e11c26 Update README.md 2025-09-05 12:32:50 +02:00
Christian Kellner
9f8d189f47 Update README.md 2025-09-05 12:24:16 +02:00
orangecoding
bed0843f30 next release version 2025-09-05 12:07:35 +02:00
orangecoding
947e895de6 upgrading puppeteer / updating config 2025-09-05 12:07:08 +02:00
Christian Kellner
0d2b21c789 improve security by shortn the cookie ttl 2025-09-04 12:52:18 +02:00
Christian Kellner
da848fcca1 Update README.md 2025-09-04 09:08:25 +02:00
Christian Kellner
74c3edd635 Update README.md 2025-09-03 15:09:35 +02:00
Christian Kellner
154043bed1 Update README.md 2025-09-03 14:52:58 +02:00
Christian Kellner
1854b421af avoid warnings on test 2025-09-03 14:47:56 +02:00
Christian Kellner
b29fc4b183 avoid warnings on test 2025-09-03 14:47:43 +02:00
Christian Kellner
47afa5659e Merge branch 'master' of github.com:orangecoding/fredy 2025-09-03 14:47:39 +02:00
Christian Kellner
3d87aeb5f9 new chart system (#158)
* new chart system
2025-09-03 14:22:32 +02:00
Christian Kellner
f8e0376ddd adding new chart system 2025-09-03 14:22:04 +02:00
weakmap@gmail.com
29026ccad8 new chart system 2025-09-03 09:45:09 +02:00
weakmap@gmail.com
9774989eeb upgrade to react router 7 2025-09-02 20:18:37 +02:00
weakmap@gmail.com
9db1ffd8eb making immonet null safe 2025-08-31 20:25:52 +02:00
weakmap@gmail.com
1cb79d1287 making immoscout image scraping null safe 2025-08-31 20:19:32 +02:00
weakmap@gmail.com
212d6e0367 next release version 2025-08-31 20:15:57 +02:00
weakmap@gmail.com
97cb6fa5eb upgrading lowdb 2025-08-31 20:15:23 +02:00
weakmap@gmail.com
8d2cc7f3e0 upgrade sqlite and vite 2025-08-31 20:12:46 +02:00
weakmap@gmail.com
3de81903a1 using eslint 9 2025-08-31 20:09:38 +02:00
weakmap@gmail.com
1ad79230c2 upgrading chai 2025-08-31 18:46:23 +02:00
weakmap@gmail.com
fb19c52b0f upgrading mocha and reduce timeout for tests 2025-08-31 18:45:02 +02:00
weakmap@gmail.com
db12d33910 upgrading dependencies 2025-08-31 18:41:46 +02:00
weakmap@gmail.com
f1c3106ae4 improve mailjet 2025-08-30 21:27:43 +02:00
weakmap@gmail.com
dd8d88404a next release version 2025-08-30 21:22:09 +02:00
Christian Kellner
f0b146fd7f Adding images to scraping data (#157)
* Fredy now supports pulling the main Image from the listing and send it together with the usual information
2025-08-30 21:21:34 +02:00
Christian Kellner
da743c8279 only check js files 2025-08-26 08:00:33 +02:00
Christian Kellner
aeffddc5a4 upgrade dependencies 2025-08-25 21:14:45 +02:00
Christian Kellner
3f92b5b099 next version 2025-08-25 21:12:23 +02:00
Christian Kellner
34317107be improve feature and bug templates 2025-08-25 21:11:30 +02:00
Christian Kellner
0bf211cb93 improve feature and bug templates 2025-08-25 21:10:01 +02:00
Christian Kellner
44a84cc3f2 improve feature and bug templates 2025-08-25 21:07:56 +02:00
Christian Kellner
d1566cf689 improve feature and bug templates 2025-08-25 20:56:10 +02:00
Christian Kellner
36f1bddedd deny blank issues 2025-08-25 20:51:07 +02:00
Christian Kellner
220df3f11a improve bug/feature templates 2025-08-25 20:47:16 +02:00
Nic
3a54ab0e31 Fix typo in test import (#154)
* Fix typo in test import

* Rename test file
2025-08-25 20:42:40 +02:00
Alexander Roidl
963a309889 Inline architecture diagram in README (#152) 2025-08-02 07:18:19 +02:00
Alexander Roidl
b66f873a91 Telegram request throttling per chat ID (#147)
* feat: telegram request throttling per chat id

* feat: telegram chat throttle cleanup

* feat: telegram throttled chats cleanup
This reverts commit 6c1786dcc2.
2025-08-01 10:03:40 +02:00
Alexander Roidl
ae4b6d1f40 Mobile view and wording (#151)
* feat(ui): simplified titles and adjusted some wording

* style(ui): simplified some views for mobile

* style(ui): make job table responsive for mobile

* style(ui): login button gap

* style(ui): dont hide mobile columns

* fix: method return type
2025-08-01 09:51:42 +02:00
Alexander Roidl
2b36f868e7 Project-wide linting and formatting (#150)
* chore: configure project-wide linting and formatting

* chore: run lint autofix and formatter
2025-07-26 20:42:58 +02:00
Christian Kellner
206f768b41 next version 2025-07-25 13:21:12 +02:00
Alexander Roidl
2302f69ff3 Rename NPM startup scripts (#144)
* feat: rename npm start scripts
2025-07-25 13:13:04 +02:00
Alexander Roidl
9bb33e723a Workflow to check sourcecode's linting and formatting (#146)
* ci: workflow to check sourcecode

* fix: make workflow to check source fail for incorrect linting/formatting

* ci: change step name for workflow to check sourcecode
2025-07-23 08:58:43 +02:00
Alexander Roidl
cca1463a68 chore: run formatter (#145) 2025-07-23 08:47:26 +02:00
Alexander Roidl
314b1818d7 Formatting and linting pre-commit hook (#143) 2025-07-22 21:39:52 +02:00
Christian Kellner
25cc7fb650 next release version 2025-07-22 20:01:01 +02:00
Alexander Roidl
78df4b21a6 Remove leading commas from listings in Telegram messages (#142) 2025-07-22 19:58:16 +02:00
weakmap@gmail.com
d89b078237 lol 2025-07-19 22:41:30 +02:00
weakmap@gmail.com
395199a4a2 fixing duplicate provider removal / ugrade dependencies 2025-07-19 20:10:19 +02:00
weakmap@gmail.com
c2680fe49f next release version 2025-06-14 19:26:17 +02:00
weakmap@gmail.com
2b862b2d98 fixing blacklist 2025-06-14 19:25:52 +02:00
weakmap@gmail.com
9065448b6b upgrade dependencies 2025-06-14 19:12:55 +02:00
weakmap@gmail.com
b9f49cb5b2 upgrade dependencies 2025-06-14 19:06:27 +02:00
weakmap@gmail.com
53121742c2 improving error message 2025-06-14 19:03:23 +02:00
Christian Kellner
1a3eae0390 next version 2025-06-04 09:47:42 +02:00
Christian Kellner
a42905d63f fixing docker ignore issue 2025-06-04 09:46:07 +02:00
Christian Kellner
9917491728 Merge branch 'master' of github.com:orangecoding/fredy 2025-06-04 09:29:50 +02:00
Christian Kellner
f032e6a724 test: verify unrelated text yields no similarity (#130) 2025-06-04 09:15:53 +02:00
Christian Kellner
111c154ae3 Fix job ownership verification (#132) 2025-06-04 09:15:36 +02:00
Christian Kellner
2194ffe0f4 Fix typo in README (#133) 2025-06-04 09:15:15 +02:00
Christian Kellner
cfa25fc0e0 docs: fix adapter sentence (#131) 2025-06-04 09:14:57 +02:00
Christian Kellner
d50dd61f3e Merge branch 'master' of github.com:orangecoding/fredy 2025-06-04 09:12:00 +02:00
Christian Kellner
31e7f77bde uprade restana & vite 2025-05-27 12:01:26 +02:00
Christian Kellner
a418d64f1a uprade dependencies 2025-05-27 11:51:57 +02:00
Christian Kellner
d099872950 Update README.md 2025-05-26 13:23:36 +02:00
Christian Kellner
2fd03bce79 improve docker build 2025-05-26 13:20:12 +02:00
Christian Kellner
78a122b3ea improve docker build 2025-05-26 12:07:22 +02:00
Christian Kellner
918c6ade36 next version 2025-05-26 11:57:54 +02:00
Christian Kellner
9fac1aee06 adding forgotten yarn.lock 2025-05-26 11:34:05 +02:00
Christian Kellner
f9c6b10976 fixing tests 2025-05-26 10:43:13 +02:00
Christian Kellner
d8ccccb82a Next version of fredy 2025-05-20 12:45:12 +02:00
Leon C.
1f54bcfd3f ImmoScout: Allow web paths with SEO optimization to be filtered to query params (#128) 2025-05-20 12:44:43 +02:00
Christian Kellner
f4c2130829 Update README.md 2025-05-17 09:09:42 +02:00
Christian Kellner
d624e70732 adding lock 2025-05-16 15:10:06 +02:00
Christian Kellner
0cbfaaf092 revert to use yarn 2025-05-16 15:03:28 +02:00
Christian Kellner
c6fb856cb6 fix docker build 2025-05-16 14:26:39 +02:00
Christian Kellner
6fe0a9dc3c fix pnpm version 2025-05-16 14:23:01 +02:00
Christian Kellner
5d52e4152d fix pnpm version 2025-05-16 14:21:29 +02:00
Christian Kellner
a8e5f8b524 improve test and docker runner 2025-05-16 14:19:20 +02:00
Christian Kellner
4b45ff4430 improve readme 2025-05-16 14:06:02 +02:00
Christian Kellner
db6211777b improve test and docker runner 2025-05-16 14:04:55 +02:00
Christian Kellner
21dd48527c fixing test runner 2025-05-16 14:00:17 +02:00
Christian Kellner
b0d494eed6 ading lock 2025-05-16 13:58:45 +02:00
Christian Kellner
9efb3e4b94 tagging new version, switching to node 22 2025-05-16 13:45:29 +02:00
Christian Kellner
683c47f61c tagging new version, switching to node 22 2025-05-16 13:44:45 +02:00
Christian Kellner
b3c11320d4 switching to pnpm for faster build 2025-05-16 13:38:25 +02:00
Christian Kellner
25dfad4f5d run cleanup once at start 2025-05-16 13:26:39 +02:00
Christian Kellner
b7a3823049 console log when removing demo jobs 2025-05-16 13:25:55 +02:00
Christian Kellner
6964998695 fixing removing demo jobs 2025-05-16 13:20:54 +02:00
Christian Kellner
ef689cf97e fix docker build harder 2025-05-15 10:37:21 +02:00
Christian Kellner
bd6a572ab0 fix docker build 2025-05-15 10:23:56 +02:00
Christian Kellner
d96c1ee3fe Merge branch 'master' of github.com:orangecoding/fredy 2025-05-15 10:16:55 +02:00
Christian Kellner
9a09548a07 fix docker build 2025-05-15 10:16:43 +02:00
Christian Kellner
00eabecd08 Update README.md 2025-05-15 09:00:17 +02:00
Christian Kellner
c07dc6220e Update README.md 2025-05-14 15:05:50 +02:00
Christian Kellner
4bab3bd9da fix docker build 2025-05-14 14:28:07 +02:00
Christian Kellner
b113621202 next version 2025-05-14 14:00:03 +02:00
Christian Kellner
030e0ca169 starting docu on reverse engineering immoscout api (#127)
* starting docu on reverse engineering immoscout api

* improving immoscout reverse engineering and adding support for most other types
2025-05-14 13:58:58 +02:00
Christian Kellner
3aae81ca19 next version 2025-05-09 11:02:23 +02:00
Christian Kellner
f1effe941f fixing immoscout url and description 2025-05-09 11:00:35 +02:00
Christian Kellner
cd3631f910 fixing new immoscout url handling 2025-05-09 10:05:30 +02:00
Christian Kellner
8f490f2426 improve test runner 2025-05-09 09:46:33 +02:00
Christian Kellner
48e2ca942f fixing tests, renaming immoscout-mobile to immoscout 2025-05-09 09:26:24 +02:00
Patrick Klein
b9e4bca244 Add immoscout mobile API provider to avoid failing bot checks (#125)
* Add provider that uses the immoscout mobile API to avoid failing bot checks.
2025-05-09 09:13:52 +02:00
Christian Kellner
a138dafc31 fixing immoweltsp title 2025-03-31 18:38:18 +02:00
weakmap@gmail.com
c6bb3c44d4 upgrade dependencies, fixing tests 2025-02-23 17:14:39 +01:00
weakmap@gmail.com
a3471a091a upgrade dependencies, fixing tests 2025-02-23 17:13:08 +01:00
Christian Kellner
b5a96afcc8 upgrading dependencies 2025-01-17 22:08:04 +01:00
Stefan
3903ab59cf fix normalized wggesucht link (#123) 2025-01-17 22:05:34 +01:00
weakmap@gmail.com
8fe7cec2a1 improve pushover notification service 2025-01-10 19:51:14 +01:00
Christian Kellner
97deea6f5b Update README.md 2025-01-09 17:31:46 +01:00
Christian Kellner
1ecbbdd774 better logging 2025-01-07 13:34:43 +01:00
Christian Kellner
e1db3840f6 adding puppeteer timeout and fixing waitForSelector 2025-01-07 12:37:50 +01:00
Christian Kellner
26127eeac1 updating dependencies 2025-01-07 12:27:16 +01:00
Christian Kellner
90a4ee5dcf better logging, fixing code smells 2025-01-07 12:25:19 +01:00
Christian Kellner
2aaf63c253 Happy New Year 2025-01-05 06:53:07 +01:00
Christian Kellner
f52e3e9fd8 Update package.json 2025-01-04 21:52:06 +01:00
Fabian Pfaff
0d69232395 install chrome via apt instead of bundled (#122) 2025-01-04 21:50:59 +01:00
weakmap@gmail.com
b473cf7fb4 fixing kleinanzeigen test 2024-12-26 19:18:30 +01:00
weakmap@gmail.com
3b8279c714 adding fredy version 2024-12-17 13:07:25 +01:00
Christian Kellner
214e714c03 Puppeteer rewrite (#119)
* Moving to puppeteer | removing scrapingAnt
2024-12-17 12:38:28 +01:00
Christian Kellner
58965a6f1b Running tests at least once a day 2024-12-16 14:06:34 +01:00
weakmap@gmail.com
3c0e9e56c6 fixing immowelt 2024-12-10 09:08:25 +01:00
Christian Kellner
f5d56a6bda version update 2024-12-03 14:25:02 +01:00
Christian Kellner
324b14da50 improving tracking 2024-12-03 14:23:09 +01:00
Christian Kellner
f8f911aa00 improving tracking 2024-12-03 14:05:00 +01:00
Christian Kellner
13b8701447 Update CONTRIBUTING.md 2024-12-02 15:02:36 +01:00
Christian Kellner
e25b956eda Update config.json 2024-11-22 12:32:37 +01:00
weakmap@gmail.com
a2c769f786 Merge branch 'master' of https://github.com/orangecoding/fredy 2024-11-22 11:37:51 +01:00
weakmap@gmail.com
1825a25eaa fixing typo 2024-11-22 11:37:44 +01:00
Christian Kellner
0f20b85f38 Update README.md 2024-11-22 09:38:50 +01:00
weakmap@gmail.com
d17ef9ef1e update fredy version 2024-11-22 09:11:43 +01:00
Christian Kellner
337ee922a6 Demo Mode (#117)
* Adding Demo Mode to Fredy
2024-11-22 09:11:10 +01:00
Christian Kellner
b3ae5f640c Update README.md 2024-11-20 22:23:05 +01:00
Christian Kellner
8f91267b5d sending tracking information (#116)
* Ability to send tracking information
2024-11-20 22:22:16 +01:00
Christian Kellner
3d59c0096d reverting config changes. accidentally pushed 2024-11-20 08:19:16 +01:00
Christian Kellner
dab6e4edf3 upgrading husky 2024-11-19 13:45:07 +01:00
Christian Kellner
e1c45f18e0 adding action for stale pr's 2024-11-06 16:16:16 +01:00
weakmap@gmail.com
5cceae11cc upgrading dependencies | adding sqlite for later analysis 2024-11-01 17:03:43 +01:00
weakmap@gmail.com
a4c5bfcbf7 fixing tests 2024-10-03 16:09:19 +02:00
weakmap@gmail.com
6d2ab5f958 making sure immowelt does not include suggested ranges 2024-10-03 16:03:47 +02:00
weakmap@gmail.com
d3cb3a5881 regex for einsAImmobilien price normalization | filter listings that does not have all required keys 2024-09-29 16:58:01 +02:00
Christian Kellner
111ef8be43 fixing kleinanzeigen test 2024-09-05 13:36:02 +02:00
Christian Kellner
35feb772d7 upgrading dependencies, fixing immowelt, using hash of price and id as unique identifier for listings 2024-09-05 13:34:14 +02:00
Christian Kellner
1bf012f13e next fredy version 2024-07-24 09:44:13 +02:00
Christian Kellner
933dc3fc64 using node 20 in tests as well 2024-07-24 09:43:11 +02:00
Christian Kellner
42c48fdceb using only 64 bit 2024-07-24 09:41:34 +02:00
Christian Kellner
f07aa0a06d using node 20 2024-07-24 09:39:27 +02:00
Christian Kellner
92db8219b4 building multi platform docker images (#101)
* building multi platform docker images

* upgrading dependencies | using scraping ant for neubaukompass
2024-07-24 09:32:21 +02:00
304 changed files with 24186 additions and 8449 deletions

View File

@@ -3,9 +3,7 @@
[
"@babel/preset-env",
{
"exclude": [
"transform-regenerator"
]
"exclude": ["transform-regenerator"]
}
],
[
@@ -15,4 +13,4 @@
}
]
]
}
}

View File

@@ -1,7 +1,47 @@
# Dependencies (will be installed fresh in container)
node_modules/
npm-debug.log
test/
conf/
# Database and config (mounted as volumes)
db/
conf/
# Git
.git/
.github/
.gitignore
# IDE and editor
.idea/
.vscode/
*.swp
*.swo
.DS_Store
# Testing
test/
# Documentation
doc/
*.md
!README.md
# Development config files
.babelrc
.husky/
.nvmrc
.prettierrc
.prettierignore
eslint.config.js
# Docker files (not needed inside container)
Dockerfile
docker-compose.yml
docker-test.sh
.dockerignore
# Logs
*.log
npm-debug.log
# Build artifacts (built fresh in container)
dist/

View File

@@ -1,282 +0,0 @@
module.exports = {
env: {
es2021: true,
node: true,
browser: true,
mocha: true,
},
parser: '@babel/eslint-parser',
extends: ['eslint:recommended', 'prettier'],
plugins: ['react'],
globals: {
Promise: false,
describe: true,
after: true,
it: true,
fetch: true,
},
parserOptions: {
sourceType: 'module',
},
rules: {
eqeqeq: [2, 'allow-null'],
// ###########################################################
// ### Semantics / Performance impacting
// ###########################################################
// babel inserts `'use strict';` for us
strict: 0,
'no-redeclare': [2, { builtinGlobals: false }],
// If a class method does not use this, it can safely be made a static function.
// http://eslint.org/docs/rules/class-methods-use-this
'class-methods-use-this': ['off'],
// ###########################################################
// ### Style
// ###########################################################
indent: ['off', 2],
'linebreak-style': ['error', 'unix'],
quotes: ['error', 'single', { avoidEscape: true, allowTemplateLiterals: true }],
semi: ['error', 'always'],
'no-console': ['error', { allow: ['warn', 'error'] }],
// ###########################################################
// ### React
// ###########################################################
// Specify whether double or single quotes should be used in JSX attributes
// http://eslint.org/docs/rules/jsx-quotes
'jsx-quotes': ['error', 'prefer-double'],
// Prevent missing displayName in a React component definition
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/display-name.md
'react/display-name': ['off'],
// Forbid certain propTypes (any, array, object)
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/forbid-prop-types.md
'react/forbid-prop-types': 'off',
// Validate closing bracket location in JSX
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/jsx-closing-bracket-location.md
'react/jsx-closing-bracket-location': ['off'],
// Enforce or disallow spaces inside of curly braces in JSX attributes
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/jsx-curly-spacing.md
'react/jsx-curly-spacing': ['off'],
// Enforce event handler naming conventions in JSX
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/jsx-handler-names.md
'react/jsx-handler-names': [
'off',
{
eventHandlerPrefix: 'handle',
eventHandlerPropPrefix: 'on',
},
],
// Validate props indentation in JSX
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/jsx-indent-props.md
'react/jsx-indent-props': 'off',
// Validate JSX has key prop when in array or iterator
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/jsx-key.md
'react/jsx-key': 'off',
// Limit maximum of props on a single line in JSX
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/jsx-max-props-per-line.md
'react/jsx-max-props-per-line': ['off'],
// Prevent usage of .bind() in JSX props
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/jsx-no-bind.md
'react/jsx-no-bind': [
'error',
{
ignoreRefs: true,
allowArrowFunctions: true,
allowBind: false,
},
],
// Prevent duplicate props in JSX
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/jsx-no-duplicate-props.md
'react/jsx-no-duplicate-props': ['error', { ignoreCase: true }],
// Prevent usage of unwrapped JSX strings
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/jsx-no-literals.md
'react/jsx-no-literals': 'off',
// Disallow undeclared variables in JSX
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/jsx-no-undef.md
'react/jsx-no-undef': 'error',
// Enforce PascalCase for user-defined JSX components
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/jsx-pascal-case.md
'react/jsx-pascal-case': [
'error',
{
allowAllCaps: true,
ignore: [],
},
],
// Enforce propTypes declarations alphabetical sorting
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/sort-prop-types.md
'react/sort-prop-types': [
'off',
{
ignoreCase: true,
callbacksLast: false,
requiredFirst: false,
},
],
// Deprecated in favor of react/jsx-sort-props
'react/jsx-sort-prop-types': 'off',
// Enforce props alphabetical sorting
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/jsx-sort-props.md
'react/jsx-sort-props': 'off',
// Prevent React to be incorrectly marked as unused
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/jsx-uses-react.md
'react/jsx-uses-react': 'error',
// Prevent variables used in JSX to be incorrectly marked as unused
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/jsx-uses-vars.md
'react/jsx-uses-vars': 'error',
// Prevent usage of dangerous JSX properties
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/no-danger.md
'react/no-danger': 'warn',
// Prevent usage of deprecated methods
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/no-deprecated.md
'react/no-deprecated': ['error'],
// Prevent usage of setState in componentDidMount
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/no-did-mount-set-state.md
'react/no-did-mount-set-state': ['error'],
// Prevent usage of setState in componentDidUpdate
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/no-did-update-set-state.md
'react/no-did-update-set-state': ['warn'],
// Prevent direct mutation of this.state
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/no-direct-mutation-state.md
'react/no-direct-mutation-state': 'off',
// Prevent usage of isMounted
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/no-is-mounted.md
'react/no-is-mounted': 'error',
// Prevent usage of setState
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/no-set-state.md
'react/no-set-state': 'off',
// Prevent using string references
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/no-string-refs.md
'react/no-string-refs': 'warn',
// Prevent usage of unknown DOM property
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/no-unknown-property.md
'react/no-unknown-property': 'error',
// Prevent missing props validation in a React component definition
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/prop-types.md
'react/prop-types': ['error', { ignore: [], customValidators: [], skipUndeclared: true }],
// Prevent missing React when using JSX
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/react-in-jsx-scope.md
'react/react-in-jsx-scope': 'error',
// Restrict file extensions that may be required
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/require-extension.md
// deprecated in favor of import/extensions
'react/require-extension': ['off', { extensions: ['.jsx', '.js'] }],
// Require render() methods to return something
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/require-render-return.md
'react/require-render-return': 'error',
// Prevent extra closing tags for components without children
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/self-closing-comp.md
'react/self-closing-comp': 'warn',
// Enforce component methods order
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/sort-comp.md
'react/sort-comp': 'off',
// Prevent missing parentheses around multilines JSX
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/jsx-wrap-multilines.md
'react/jsx-wrap-multilines': [
'warn',
{
declaration: true,
assignment: true,
return: true,
},
],
'react/wrap-multilines': 'off', // deprecated version
// Require that the first prop in a JSX element be on a new line when the element is multiline
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/jsx-first-prop-new-line.md
'react/jsx-first-prop-new-line': ['off'],
// Enforce spacing around jsx equals signs
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/jsx-equals-spacing.md
'react/jsx-equals-spacing': ['warn', 'never'],
// Disallow target="_blank" on links
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/jsx-no-target-blank.md
'react/jsx-no-target-blank': 'error',
// only .jsx files may have JSX
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/jsx-filename-extension.md
'react/jsx-filename-extension': ['error', { extensions: ['.jsx'] }],
// prevent accidental JS comments from being injected into JSX as text
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/jsx-no-comment-textnodes.md
'react/jsx-no-comment-textnodes': 'error',
'react/no-comment-textnodes': 'off', // deprecated version
// disallow using React.render/ReactDOM.render's return value
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/no-render-return-value.md
'react/no-render-return-value': 'error',
// require a shouldComponentUpdate method, or PureRenderMixin
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/require-optimization.md
'react/require-optimization': ['off', { allowDecorators: [] }],
// warn against using findDOMNode()
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/no-find-dom-node.md
'react/no-find-dom-node': 'warn',
// Forbid certain props on Components
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/forbid-component-props.md
'react/forbid-component-props': ['off', { forbid: [] }],
// Prevent problem with children and props.dangerouslySetInnerHTML
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/no-danger-with-children.md
'react/no-danger-with-children': 'error',
// Prevent unused propType definitions
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/no-unused-prop-types.md
'react/no-unused-prop-types': [
'warn',
{
customValidators: [],
skipShapeProps: true,
},
],
// Require style prop value be an object or var
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/style-prop-object.md
'react/style-prop-object': 'error',
// Prevent passing of children as props
// https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/no-children-prop.md
'react/no-children-prop': 'warn',
},
};

73
.github/ISSUE_TEMPLATE/bug.yml vendored Normal file
View File

@@ -0,0 +1,73 @@
name: Bug Report
description: Help us improve Fredy by reporting a bug
title: "[Bug]: "
labels: [bug]
assignees: []
body:
- type: textarea
id: description
attributes:
label: Bug Description
description: Provide a clear and concise description of the bug.
placeholder: e.g. "Fredy crashes when I click on Save."
validations:
required: true
- type: textarea
id: steps
attributes:
label: Steps to Reproduce
description: List the steps to reproduce the issue.
placeholder: |
1. Go to '...'
2. Click on '...'
3. Scroll down to '...'
4. See error
validations:
required: true
- type: textarea
id: expected
attributes:
label: Expected Behavior
description: What did you expect to happen?
placeholder: "It should save without errors."
validations:
required: true
- type: textarea
id: actual
attributes:
label: Actual Behavior
description: What actually happened?
placeholder: "Fredy crashed with error XYZ."
validations:
required: true
- type: textarea
id: screenshots
attributes:
label: Screenshots / Logs
description: Add screenshots or paste log output to help explain the problem.
placeholder: "Drag and drop screenshots here, or paste logs."
validations:
required: false
- type: input
id: environment
attributes:
label: Environment
description: Provide details about your environment.
placeholder: "OS: macOS 15, Browser: Chrome 124, App version: 1.2.3"
validations:
required: true
- type: textarea
id: context
attributes:
label: Additional Context
description: Add any other context about the problem here.
placeholder: "Any other information that might help..."
validations:
required: false

View File

@@ -1,24 +0,0 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: ''
assignees: ''
---
**Describe the bug**
A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
**Expected behavior**
A clear and concise description of what you expected to happen.
**Screenshots**
If applicable, add screenshots to help explain your problem.

1
.github/ISSUE_TEMPLATE/config.yml vendored Normal file
View File

@@ -0,0 +1 @@
blank_issues_enabled: false

51
.github/ISSUE_TEMPLATE/feature.yml vendored Normal file
View File

@@ -0,0 +1,51 @@
name: Feature Request
description: Suggest an improvement or new idea for Fredy
title: "[Feature]: "
labels: [enhancement]
assignees: []
body:
- type: textarea
id: problem
attributes:
label: Related Problem
description: Is your feature request related to a problem? Describe it clearly.
placeholder: "Example: Its difficult to do X when Y happens..."
validations:
required: false
- type: textarea
id: solution
attributes:
label: Proposed Feature
description: Describe the feature you would like to see.
placeholder: "I would like Fredy to automatically..."
validations:
required: true
- type: textarea
id: alternatives
attributes:
label: Alternatives Considered
description: List any alternative solutions or workarounds youve tried or thought about.
placeholder: "Instead of this, I also considered..."
validations:
required: false
- type: textarea
id: benefits
attributes:
label: Benefits
description: Explain how this feature would improve Fredy or it's user experience.
placeholder: "This would save users time by..."
validations:
required: true
- type: textarea
id: context
attributes:
label: Additional Context
description: Add any other context, examples, or screenshots that might help clarify your idea.
placeholder: "Any other relevant information..."
validations:
required: false

View File

@@ -1,20 +0,0 @@
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: ''
assignees: ''
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.

26
.github/workflows/check_source.yml vendored Normal file
View File

@@ -0,0 +1,26 @@
name: Check the source code
on:
push:
branches: [master]
pull_request:
branches: [master]
jobs:
check_source_code:
name: Check the source code
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 22
cache: 'yarn'
- name: Install dependencies
run: yarn install
- name: Check formatting
run: yarn format:check
- name: Lint
run: yarn lint

View File

@@ -0,0 +1,44 @@
name: Claude Code Review
on:
pull_request:
types: [opened, synchronize, ready_for_review, reopened]
# Optional: Only run on specific file changes
# paths:
# - "src/**/*.ts"
# - "src/**/*.tsx"
# - "src/**/*.js"
# - "src/**/*.jsx"
jobs:
claude-review:
# Optional: Filter by PR author
# if: |
# github.event.pull_request.user.login == 'external-contributor' ||
# github.event.pull_request.user.login == 'new-developer' ||
# github.event.pull_request.author_association == 'FIRST_TIME_CONTRIBUTOR'
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: read
issues: read
id-token: write
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Run Claude Code Review
id: claude-review
uses: anthropics/claude-code-action@v1
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
plugin_marketplaces: 'https://github.com/anthropics/claude-code.git'
plugins: 'code-review@claude-code-plugins'
prompt: '/code-review:code-review ${{ github.repository }}/pull/${{ github.event.pull_request.number }}'
# See https://github.com/anthropics/claude-code-action/blob/main/docs/usage.md
# or https://code.claude.com/docs/en/cli-reference for available options

50
.github/workflows/claude.yml vendored Normal file
View File

@@ -0,0 +1,50 @@
name: Claude Code
on:
issue_comment:
types: [created]
pull_request_review_comment:
types: [created]
issues:
types: [opened, assigned]
pull_request_review:
types: [submitted]
jobs:
claude:
if: |
(github.event_name == 'issue_comment' && contains(github.event.comment.body, '@claude')) ||
(github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '@claude')) ||
(github.event_name == 'pull_request_review' && contains(github.event.review.body, '@claude')) ||
(github.event_name == 'issues' && (contains(github.event.issue.body, '@claude') || contains(github.event.issue.title, '@claude')))
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
issues: read
id-token: write
actions: read # Required for Claude to read CI results on PRs
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Run Claude Code
id: claude
uses: anthropics/claude-code-action@v1
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
# This is an optional setting that allows Claude to read CI results on PRs
additional_permissions: |
actions: read
# Optional: Give a custom prompt to Claude. If this is not specified, Claude will perform the instructions specified in the comment that tagged it.
# prompt: 'Update the pull request description to include a summary of changes.'
# Optional: Add claude_args to customize behavior and configuration
# See https://github.com/anthropics/claude-code-action/blob/main/docs/usage.md
# or https://code.claude.com/docs/en/cli-reference for available options
# claude_args: '--allowed-tools Bash(gh pr:*)'

View File

@@ -1,4 +1,5 @@
name: Create and publish Docker image
on:
push:
branches:
@@ -17,15 +18,24 @@ jobs:
contents: read
packages: write
steps:
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
steps:
- name: Checkout repository
uses: actions/checkout@v2
uses: actions/checkout@v3
- name: Set up QEMU
uses: docker/setup-qemu-action@v2
with:
platforms: linux/amd64,linux/arm64
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
- name: Log in to the Container registry
uses: docker/login-action@v1
uses: docker/login-action@v2
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
@@ -33,14 +43,56 @@ jobs:
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v3
uses: docker/metadata-action@v4
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
- name: Build and push Docker image
uses: docker/build-push-action@v2
uses: docker/build-push-action@v3
with:
context: .
push: true
platforms: linux/amd64,linux/arm64
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
# Test container health with docker compose
- name: Test container with docker compose
run: |
echo "Starting container with docker compose..."
mkdir -p ./db ./conf && chmod 777 ./db ./conf
docker compose up --build -d
echo "Waiting for container to be ready (60 seconds for start_period)..."
sleep 60
echo "Monitoring container health for 30 seconds..."
SECONDS_ELAPSED=0
HEALTH_CHECK_INTERVAL=5
TOTAL_DURATION=30
while [ $SECONDS_ELAPSED -lt $TOTAL_DURATION ]; do
HEALTH_STATUS=$(docker inspect --format='{{.State.Health.Status}}' fredy 2>/dev/null || echo "not_found")
CONTAINER_STATUS=$(docker inspect --format='{{.State.Status}}' fredy 2>/dev/null || echo "not_found")
echo "[$SECONDS_ELAPSED/$TOTAL_DURATION sec] Container: $CONTAINER_STATUS, Health: $HEALTH_STATUS"
# Check if container is not running or unhealthy
if [ "$CONTAINER_STATUS" != "running" ]; then
echo "Container stopped running! Status: $CONTAINER_STATUS"
docker compose logs fredy
exit 1
fi
if [ "$HEALTH_STATUS" = "unhealthy" ]; then
echo "Container is unhealthy!"
docker compose logs fredy
docker inspect --format='{{json .State.Health}}' fredy | jq
exit 1
fi
sleep $HEALTH_CHECK_INTERVAL
SECONDS_ELAPSED=$((SECONDS_ELAPSED + HEALTH_CHECK_INTERVAL))
done
docker compose down

21
.github/workflows/stales.yml vendored Normal file
View File

@@ -0,0 +1,21 @@
name: Close stale issues and PRs
on:
schedule:
- cron: '0 0 * * *' # Daily
jobs:
stale:
runs-on: ubuntu-latest
steps:
- uses: actions/stale@v7
with:
days-before-stale: 30
days-before-close: 7
stale-issue-message: 'This issue has been automatically marked as stale due to inactivity.'
stale-pr-message: 'This PR has been automatically marked as stale due to inactivity.'
close-issue-message: 'Closing this issue due to prolonged inactivity.'
close-pr-message: 'Closing this PR due to prolonged inactivity.'
exempt-issue-labels: 'keep-open'
exempt-pr-labels: 'keep-open'
only: 'pulls'

View File

@@ -1,21 +1,22 @@
name: Test
on:
push:
branches:
- master
branches: [master]
pull_request:
branches:
- master
branches: [master]
schedule:
- cron: '0 12 * * *'
jobs:
test:
name: Test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- name: Setup node
uses: actions/setup-node@v2.5.1
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 18
node-version: 22
cache: 'yarn'
- run: yarn install
- run: yarn run test
- run: yarn testGH

5
.gitignore vendored
View File

@@ -1,6 +1,9 @@
node_modules/
ui/public/
db/
db/*.json
db/*.db*
npm-debug.log
.DS_Store
.idea
.vscode
tools/release/config.json

1
.husky/pre-commit Executable file
View File

@@ -0,0 +1 @@
npx lint-staged

6
.prettierignore Normal file
View File

@@ -0,0 +1,6 @@
/ui/public
/db/
/conf/
# TODO re-write from scratch or fix all html structure issues
/lib/notification/emailTemplate/template.hbs

4
.prettierrc Normal file
View File

@@ -0,0 +1,4 @@
{
"singleQuote": true,
"printWidth": 120
}

View File

@@ -1,2 +0,0 @@
sudo: false
language: node_js

View File

@@ -1,78 +0,0 @@
Newer release changelog see https://github.com/orangecoding/fredy/releases
------------
###### [V5.5.0]
- Upgrading dependencies
- fixing provider
- allow multiple instances of 1 provider
- __BREAKING__: Minimum node version is now 16
###### [V5.4.6]
- Adding Instana node.js monitoring
-
###### [V5.4.5]
- Adding Instana node.js monitoring
###### [V5.4.4]
- Add support for Immo Südwest Presse (immo.swp.de)
- Telegram: Use job name instead of ID and link in title
- Fix race condition if user ID is in session but not in user store
- Allow visiting the original provider URL
###### [V5.4.3]
- re-writing readme
- improving docker build
- using github's actions to build docker and test automatically
###### [V5.4.2]
- Fixing prod build
###### [V5.4.1]
- Upgrading dependencies
- Provider urls are now automagically been changed to include the correct sort order for search results
```
Note: It has been an point of confusion since the very beginning of Fredy, that people simply copied the url, but
did not take care of sorting the search results by date. If this is not done, Fredy will most likely not see the latest
results, thus cannot report them. This release fixes it by adding the necessary params (or replaces them).
```
###### [V5.3.0]
- Upgrading dependencies
- It's now possible to send mails to multiple receiver using comma separation for MailJet & Sendgrid
- Fixing Immowelt scraping
###### [V5.2.0]
- Upgrading dependencies
- Adding new similarity check layer (Duplicates are being removed now)
- Adding paging for search results
###### [V5.1.0]
- Upgrading dependencies
- NodeJS 12.13 is now the minimum supported version
- Adding general settings as new configuration page to ui
- Adding new feature working hours
###### [V5.0.0]
- Upgrading dependencies
- NodeJS 12 is now the minimum supported version
###### [V4.0.0]
Bringing back Immoscout :tada:
###### [V3.0.0]
This is basically a re-write, your old config file will not be compatible anymore. Please re-created your search jobs
on the new ui and use the values from your previous config file if needed.
```
- We're getting rid of manual config changes, Fredy, now ships with a UI so that it's easy for you to create and edit jobs
```
###### [V2.0.0]
```
- Fredy can now run multiple search job on one instance
- Changed lot's of the structure of Fredy to make this happen
[BREAKING CHANGES]
- The config has been changed, the config of V1.x will not work any longer
- Sources have been renamed to provider
```

View File

@@ -2,8 +2,8 @@
If you want to contribute, please make sure you've executed the tests.
### How to write new provider?
- create the provider filer under `/lib/provider`
- create a test under /test and make sure it is running successfully
@@ -13,7 +13,7 @@ let appliedBlackList = [];
//normalize incoming values
function normalize(o) {
const id = parseInt(o.id.substring(o.id.indexOf('_') + 1, o.id.length));
return Object.assign(o, { id });
}
@@ -27,7 +27,7 @@ function applyBlacklist(o) {
const config = {
url: null,
//this is the container wrapping the search listings
//this is the container wrapping the search listings
crawlContainer: '#result-list-stage .item',
crawlFields: {
id: '@id',
@@ -49,7 +49,7 @@ exports.init = (sourceConfig, blacklist) => {
appliedBlackList = blacklist || [];
};
//ths
//ths
exports.metaInformation = {
name: 'your provider name',
baseUrl: 'https://www.yourprovider.de/',
@@ -57,11 +57,10 @@ exports.metaInformation = {
};
exports.config = config;
```
### How to write new notification adapter?
- create the provider filer under `/lib/notification/adapter`
- create a description of the provider under `/lib/notification/adapter/*.md`. Make sure the name of the md file is equal to the notification adapter
@@ -72,50 +71,48 @@ const Slack = require('slack');
const msg = Slack.chat.postMessage;
const { markdown2Html } = require('../../services/markdown');
//as a parameter, you will always get the serviceName, newListings and all the values, that
//you have defined exports.config.fields. (This is being used for rendering in the frontend)
exports.send = ({ serviceName, newListings, notificationConfig, jobKey }) => {
const { token, channel } = notificationConfig.find((adapter) => adapter.id === 'slack').fields;
return newListings.map((payload) => {
//tho whatever needs to be done to send the data to the receiver, make sure the format is human readable
});
const { token, channel } = notificationConfig.find((adapter) => adapter.id === 'slack').fields;
return newListings.map((payload) => {
//tho whatever needs to be done to send the data to the receiver, make sure the format is human readable
});
};
exports.config = {
id: __filename.slice(__dirname.length + 1, -3),
name: 'someUniqueName, used in the frontend',
//this readme is rendered in the frontend to explain how to use this
readme: markdown2Html('lib/notification/adapter/slack.md'),
description: 'Some description text rendered on the notification page',
fields: {
token: {
//type can be text/number/boolean
type: 'text',
label: 'Token',
description: 'The token needed to send notifications to slack.',
},
channel: {
type: 'channel',
label: 'Channel',
description: 'The channel where fredy should send notifications to.',
},
id: __filename.slice(__dirname.length + 1, -3),
name: 'someUniqueName, used in the frontend',
//this readme is rendered in the frontend to explain how to use this
readme: markdown2Html('lib/notification/adapter/slack.md'),
description: 'Some description text rendered on the notification page',
fields: {
token: {
//type can be text/number/boolean
type: 'text',
label: 'Token',
description: 'The token needed to send notifications to slack.',
},
channel: {
type: 'channel',
label: 'Channel',
description: 'The channel where fredy should send notifications to.',
},
},
};
```
#### Running Tests
If you've written a new provider you are an awesome person. You know it and I do. If you now write tests for it, you are even more awesome. And who doesn't want to be more awesome right?
To write tests for provider, you need to use Node 8 as the tests are using `async / await`
If you've written a new provider you are an awesome person. If you now write tests for it, you are even more awesome. And who doesn't want to be more awesome, right?
#### Codestyle
I'm using Eslint to maintain quote style and quality. Do not skip it...
##### To do before merging:
I'm using ESLint to maintain quote style and quality. Do not skip it...
- executed tests? (`yarn run test`)
- sure the changes are useful for everybody? Or is it maybe a custom modification just for your case?
##### To-do before merging:
- Have you executed the tests? (`yarn test`)
- Are you sure the changes are useful for everybody? Or is it maybe a custom modification just for your case?
_Thanks!_ :heart:

View File

@@ -1,20 +1,55 @@
FROM node:18
FROM node:22-slim
WORKDIR /fredy
ARG TARGETARCH
COPY . /fredy
# System deps for Chrome for Testing + build tools for native modules (better-sqlite3)
# On ARM64 we also install system Chromium (Chrome for Testing has no ARM64 binary)
RUN apt-get update && apt-get install -y --no-install-recommends \
curl ca-certificates fonts-liberation libasound2 \
libatk-bridge2.0-0 libatk1.0-0 libcups2 libdbus-1-3 \
libdrm2 libgbm1 libgtk-3-0 libnspr4 libnss3 \
libx11-xcb1 libxcomposite1 libxdamage1 libxrandr2 xdg-utils \
python3 make g++ \
&& if [ "$TARGETARCH" = "arm64" ]; then apt-get install -y --no-install-recommends chromium; fi \
&& rm -rf /var/lib/apt/lists/* \
&& mkdir -p /db /conf /fredy
RUN yarn install
WORKDIR /fredy
RUN yarn global add pm2
ENV NODE_ENV=production \
IS_DOCKER=true
RUN yarn run prod
COPY package.json yarn.lock ./
RUN mkdir /db /conf && \
chown 1000:1000 /db /conf && \
chmod 777 -R /db/ && \
ln -s /db /fredy/db && ln -s /conf /fredy/conf
# Install dependencies and purge build tools (only needed to compile better-sqlite3)
RUN yarn config set network-timeout 600000 \
&& yarn --frozen-lockfile \
&& yarn cache clean
# on arm64 use the system Chromium installed above
RUN if [ "$TARGETARCH" != "arm64" ]; then npx puppeteer browsers install chrome; fi
# Purge build tools now that native modules are compiled
RUN apt-get purge -y python3 make g++ \
&& apt-get autoremove -y \
&& rm -rf /var/lib/apt/lists/*
COPY index.html vite.config.js ./
COPY ui ./ui
COPY lib ./lib
RUN yarn build:frontend
COPY index.js ./
RUN ln -s /db /fredy/db \
&& ln -s /conf /fredy/conf
EXPOSE 9998
VOLUME /db
VOLUME /conf
CMD pm2-runtime index.js
HEALTHCHECK --interval=30s --timeout=10s --start-period=30s --retries=3 \
CMD curl -f http://localhost:9998/ || exit 1
CMD ["node", "index.js"]

227
LICENSE
View File

@@ -1,21 +1,214 @@
MIT License
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
Copyright (c) 2024 Christian Kellner
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
1. Definitions.
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor
be liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
Additional License Condition Commons Clause
The Licensed Work is provided under the terms of this license and is also
subject to the following additional condition ("Commons Clause"):
"License Condition v1.0":
The Licensed Work and its derivative works may not be used by any person or
organization to Sell the Licensed Work (as defined below).
"Sell" or "Selling" means practicing any or all of the rights granted to you
under the License to provide to third parties, for a fee or other consideration
(including without limitation fees for hosting or consulting/support services
related to the Software), a product or service whose value derives, entirely or
substantially, from the functionality of the Licensed Work.
A non-exhaustive list of activities considered "Selling" includes:
- Using the Licensed Work to provide paid hosted services or managed services
- Distributing the Licensed Work as part of a commercial product or service
for which a fee is charged primarily for the value of the Licensed Work
This restriction does not apply to the use of the Licensed Work for internal
business purposes or non-commercial use.
Attribution and Naming Clause
Any derivative work based on this software must include clear and visible
attribution to the original project "Fredy" and its author(s).
Derivative works may not be distributed, published, or presented under a
different name or branding without the explicit written permission of the
original copyright holder.
Copyright (c) 2026 Christian Kellner
Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause

307
README.md
View File

@@ -1,119 +1,244 @@
<img src="https://github.com/orangecoding/fredy/blob/master/doc/logo.png" width="400">
![Build Status](https://github.com/orangecoding/fredy/actions/workflows/test.yml/badge.svg)
Searching an apartment in Germany can be a frustrating task. Not any longer though, as _Fredy_ will take over and will only notify you once new listings have been found that match your requirements.
_Fredy_ scrapes multiple services (Immonet, Immowelt etc.) and send new listings to you once they become available. The list of available services can easily be extended. For your convenience, _Fredy_ has a UI to help you configure your search jobs.
If _Fredy_ finds matching results, it will send them to you via Slack, Email, Telegram etc. (More adapters can be configured.) As _Fredy_ stores the listings it has found, new results will not be sent to you twice (and as a side-effect, _Fredy_ can show some statistics). Furthermore, _Fredy_ checks duplicates per scraping so that the same listings are not being sent twice or more when posted on various platforms (which happens more often than one might think).
# Sponsorship [![](https://img.shields.io/static/v1?label=Sponsor&message=%E2%9D%A4&logo=GitHub&color=%23fe8e86)](https://github.com/sponsors/orangecoding)
If you like my work, consider becoming a sponsor. I'm not expecting anybody to pay for _Fredy_ or any other Open Source Project I'm maintaining, however keep in mind, I'm doing all of this in my spare time :) Thanks.
<img src="https://github.com/orangecoding/fredy/blob/master/doc/jetbrains.png" width="200">
_Fredy_ is supported by JetBrains under Open Source Support Program
## Usage
- Make sure to use Node.js 18 or above
- Run the following commands:
```ssh
yarn (or npm install)
yarn run prod
yarn run start
```
_Fredy_ will start with the default port, set to `9998`. You can access _Fredy_ by opening your browser at `http://localhost:9998`. The default login is `admin`, both for username and password. You should change the password as soon as possible when you plan to run Fredy on a server.
<p align="center">
<img alt="Job Configuration" src="https://github.com/orangecoding/fredy/blob/master/doc/screenshot1.png" width="30%">
&nbsp; &nbsp; &nbsp; &nbsp;
<img alt="Job Analytics" src="https://github.com/orangecoding/fredy/blob/master/doc/screenshot_2.png" width="30%">
&nbsp; &nbsp; &nbsp; &nbsp;
<img alt="Job Overview" width="30%" src="https://github.com/orangecoding/fredy/blob/master/doc/screenshot_3.png">
<a href="https://fredy.orange-coding.net/">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://github.com/orangecoding/fredy/blob/master/doc/logo_white.png" width="400">
<source media="(prefers-color-scheme: light)" srcset="https://github.com/orangecoding/fredy/blob/master/doc/logo.png" width="400">
<img alt="Jetbrains Open Source" src="https://github.com/orangecoding/fredy/blob/master/doc/logo.png">
</picture>
</a>
</p>
## Understanding the fundamentals
There are 3 important parts in Fredy, that you need to understand to leverage the full power of _Fredy_.
<p align="center">
<a href="https://fredy.orange-coding.net/" target="_blank">Website</a>&nbsp;&nbsp;|&nbsp;&nbsp;
<a href="https://fredy-demo.orange-coding.net/" target="_blank">Demo</a>
</p>
#### Provider
_Fredy_ supports multiple services. Immonet, Immowelt and Ebay are just a few examples. Those services are called providers within _Fredy_. When creating a new job, you can choose one or more providers.
A provider contains the URL that points to the search results for the respective service. If you go to immonet.de and search for something, the displayed URL in the browser is what the provider needs to do its magic.
**It is important that you order the search results by date, so that _Fredy_ always picks the latest results first!**
<p align="center">
<img src="https://github.com/orangecoding/fredy/actions/workflows/test.yml/badge.svg" alt="Tests" />
<img src="https://github.com/orangecoding/fredy/actions/workflows/docker.yml/badge.svg" alt="Docker" />
<img src="https://github.com/orangecoding/fredy/actions/workflows/check_source.yml/badge.svg" alt="Source" />
<img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fghcr-badge.elias.eu.org%2Fapi%2Forangecoding%2Ffredy%2Ffredy&query=%24.downloadCount&label=Docker%20Pulls" alt="Docker Pulls" />
</p>
#### Adapter
_Fredy_ supports multiple adapters, such as Slack, SendGrid, Telegram etc. A search job can have as many adapters as supported by _Fredy_. Each adapter needs different configuration values, which you have to provide when using them. A adapter dictactes how the frontend renders by telling the frontend what information it needs in order to send listings to the user.
#### Jobs
A Job wraps adapters and providers. _Fredy_ runs the configured jobs in a specific interval (can be configured in `/conf/config.json`).
## Creating your first job
To create your first job, click on the button "Create New Job" on the job table. The job creation dialog should be self-explanatory, however there is one important thing.
When configuring providers, before copying the URL from your browser, make sure that you have sorted the results by date to make sure _Fredy_ always picks the latest results first.
# Fredy 🏡 Your Self-Hosted Real Estate Finder for Germany
## User management
As an administrator, you can create, edit and remove users from _Fredy_. Be careful, each job is connected to the user that has created the job. If you remove the user, their jobs will also be removed.
Finding an apartment or house in Germany can be stressful and
time-consuming.\
**Fredy** makes it easier: it automatically scrapes **ImmoScout24,
Immowelt, Immonet, eBay Kleinanzeigen, and WG-Gesucht** and notifies you
instantly via **Slack, Telegram, Email, ntfy, discord and more** when new
listings appear.
# Development
With a modern architecture, Fredy provides a **clean Web UI**, removes
duplicates across platforms, and stores results so you never see the
same listing twice.
### Running Fredy in development mode
To run _Fredy_ in development mode, you need to run the backend & frontend separately.
Start the backend with:
```shell
yarn run start
------------------------------------------------------------------------
## ✨ Key Features
- 🏠 Scrapes **ImmoScout24, Immowelt, Immonet, eBay Kleinanzeigen,
WG-Gesucht**
- ⚡ Instant notifications: Slack, Telegram, Email (SendGrid,
Mailjet), ntfy, discord
- 🔎 Uses the **ImmoScout Mobile API** (reverse engineered)
- 🌍 Runs anywhere: Docker, Node.js, self-hosted
- 🖥️ Intuitive **Web UI** to manage searches
- 🎯 Easy to use thanks to a user-friendly Web UI
- 🔄 Deduplication across platforms
- ⏱️ Customizable search intervals
------------------------------------------------------------------------
## 🤝 Sponsorship [![](https://img.shields.io/static/v1?label=Sponsor&message=❤&logo=GitHub&color=%23fe8e86)](https://github.com/sponsors/orangecoding)
I maintain Fredy and other open-source projects in my free time.\
If you find it useful, consider supporting the project 💙
Fredy is proudly backed by the **JetBrains Open Source Support Program**.
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://www.jetbrains.com/company/brand/img/logo_jb_dos_3.svg">
<source media="(prefers-color-scheme: light)" srcset="https://resources.jetbrains.com/storage/products/company/brand/logos/jetbrains.svg">
<img alt="Jetbrains Open Source" src="https://resources.jetbrains.com/storage/products/company/brand/logos/jetbrains.svg">
</picture>
------------------------------------------------------------------------
## 👨‍🏫 Demo
You can try out Fredy here: [Fredy Demo](https://fredy-demo.orange-coding.net/)
------------------------------------------------------------------------
## 🚀 Quick Start
### With Docker
> [!NOTE]
> In order to start Fredy, you must provide a config.json. As a start, use the one in this repo: https://github.com/orangecoding/fredy/blob/master/conf/config.json
``` bash
docker run -d --name fredy \
-v fredy_conf:/conf \
-v fredy_db:/db \
-p 9998:9998 \
ghcr.io/orangecoding/fredy:master
```
For the frontend, run:
```shell
yarn run dev
Logs:
``` bash
docker logs fredy -f
```
### Manual (Node.js)
- Requirement: **Node.js 22 or higher**
- Install dependencies and start:
``` bash
yarn
yarn run start:backend # in one terminal
yarn run start:frontend # in another terminal
```
👉 Open <http://localhost:9998>
### With Unraid
Should you use [Unraid](https://unraid.net/), you can now install Fredy from the community store :)
**Default Login:**
- Username: `admin`
- Password: `admin`
------------------------------------------------------------------------
## 📸 Screenshots
| Fredy Maps View | Dashboard | Found Listings |
|--------------------------------------------------|-----------------------------------------------------------------------|-----------------------------------------------------------------------------|
| ![Screenshot showing Fredy](doc/screenshot1.png) | ![Screenshot showing job configuration in Fredy](doc/screenshot3.png) | ![Screenshot showing found listings in Fredy](doc/screenshot2.png) |
------------------------------------------------------------------------
## 🧩 Core Concepts
Fredy is built around three simple concepts:
### Provider 🌐
A **provider** is a real-estate platform (e.g. ImmoScout24, Immowelt,
Immonet, eBay Kleinanzeigen, WG-Gesucht).\
When you create a job, you paste the search URL from the platform into
Fredy.\
⚠️ Always make sure the search results are sorted by **date**, so Fredy
picks up the newest listings first.
### Adapter 📡
An **adapter** is the channel through which Fredy notifies you (Slack,
Telegram, Email, ntfy, discord ...).\
Each adapter has its own configuration (e.g. API keys, webhook URLs).\
You can use multiple adapters at once --- Fredy will send new listings
through all of them.
### Job 📅
A **job** combines providers and adapters.\
Example: "Search apartments on ImmoScout24 + Immowelt and send results
to Slack + Telegram."\
Jobs run automatically at the interval you configure (see
`/conf/config.json`).
### MCP Server 🤖
Starting with **V20**, Fredy ships with a built-in **MCP Server **. This allows you to connect Fredy to LLMs (like Claude, ChatGPT, or local models via LM Studio) and query your real estate data using natural language.
The local LLM can even enrich existing listings by checking the listing online.
For more information on how to set it up and use it, please refer to the [MCP Readme](lib/mcp/README.md).
------------------------------------------------------------------------
## Immoscout
Immoscout has implemented advanced bot detection. In order to work around this, we are using a reversed engineered version of their mobile api. See [Immoscout Reverse Engineering Documentation](https://github.com/orangecoding/fredy/blob/master/reverse-engineered-immoscout.md)
## Analytics
Fredy is completely free (and will always remain free). However, it would be a huge help if youd allow me to collect some analytical data.
Before you freak out, let me explain...
If you agree, Fredy will send a ping once every 6 hours to my internal tracking project (Will be open sourced soon).
The data includes: names of active adapters/providers, OS, architecture, Node version, and language. The information is entirely anonymous and helps me understand which adapters/providers are most frequently used.</p>
**Thanks**🤘
## 🛠️ Development
### Development Mode
``` bash
yarn run start:backend:dev
yarn run start:frontend:dev
```
You should now be able to access _Fredy_ from your browser. Check your Terminal to see what port the frontend is running on.
### Running Tests
To run the tests, run
```shell
### Run Tests
``` bash
yarn run test
```
# Architecture
![Architecture](/doc/architecture.jpg "Architecture")
------------------------------------------------------------------------
### Immoscout / Immonet
I have added **experimental** support for Immoscout and Immonet. They both are somewhat special, because they have decided to secure their service from bots using Re-Capture. Finding a way around this is barely possible. For _Fredy_ to be able to bypass this check, I'm using a service called [ScrapingAnt](https://scrapingant.com/). The trick is to use a headless browser, rotating proxies and (once successfully validated) to re-send the cookies each time.
## 📐 Architecture
To be able to use Immoscout / Immonet, you need to create an account at ScrapingAnt. Configure the API key in the "General Settings" tab (visible when logged in as administrator).
The rest will be handled by _Fredy_. Keep in mind, the support is experimental. There might be bugs and you might not always pass the re-capture check, but most of the time it works rather well :)
``` mermaid
flowchart TD
subgraph Jobs["Jobs"]
A1["Job 1"]
A2["Job 2"]
A3["Job 3"]
end
subgraph Providers["Providers"]
C1["Provider 1"]
C2["Provider 2"]
C3["Provider 3"]
end
subgraph NotificationAdapters["Notification Adapters"]
F1["Adapter 1"]
F2["Adapter 2"]
end
If you need more than the 1000 API calls allowed per month, I'd suggest opting for a paid account... ScrapingAnt loves OpenSource, therefore they have decided to give all _Fredy_ users a 10% discount by using the code **FREDY10** (Disclaimer: I do not earn any money for recommending their service).
A1 --> B["FredyPipelineExecutioner"]
A2 --> B
A3 --> B
B --> C1 & C2 & C3
C1 --> D["Similarity Check"]
C2 --> D
C3 --> D
D --> E{"Duplicate?"}
E -- No --> F1
F1 --> F2
```
### 👐 Contributing
Thanks to all the people who already contributed!
------------------------------------------------------------------------
<a href="https://github.com/orangecoding/fredy/graphs/contributors">
<img src="https://contrib.rocks/image?repo=orangecoding/fredy" />
</a>
## 👐 Contributing
See [Contributing](https://github.com/orangecoding/fredy/blob/master/CONTRIBUTING.md)
Thanks to everyone who has contributed!
# Docker
Use the Dockerfile in this repository to build an image.
<a href="https://github.com/orangecoding/fredy/graphs/contributors"><img src="https://contrib.rocks/image?repo=orangecoding/fredy" /></a>
Example: `docker build -t fredy/fredy /path/to/your/Dockerfile`
See the [Contributing
Guide](https://github.com/orangecoding/fredy/blob/master/CONTRIBUTING.md).
Or use docker-compose:
------------------------------------------------------------------------
Example `docker-compose build`
## ⭐ Star History
Or use the container that will be built automatically.
`docker pull ghcr.io/orangecoding/fredy:master`
## Create & run a container
Put your config.json into a path of your choice, such as `/path/to/your/conf/`.
Example: `docker create --name fredy -v /path/to/your/conf/:/conf -p 9998:9998 fredy/fredy`
## Logs
You can browse the logs with `docker logs fredy -f`.
[![Star History
Chart](https://api.star-history.com/svg?repos=orangecoding/fredy&type=Date)](https://www.star-history.com/#orangecoding/fredy&Date)

2
conf/config.json Executable file → Normal file
View File

@@ -1 +1 @@
{"interval":"60","port":9998,"scrapingAnt":{"apiKey":"","proxy":"datacenter"},"workingHours":{"from":"","to":""}}
{"sqlitepath":"/db"}

52
copyright.js Normal file
View File

@@ -0,0 +1,52 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import fs from 'fs/promises';
import path from 'path';
const COPYRIGHT = `/*
* Copyright (c) ${new Date().getFullYear()} by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
`;
async function getAllFiles(dir = '.') {
const entries = await fs.readdir(dir, { withFileTypes: true });
let files = [];
for (let entry of entries) {
const fullPath = path.join(dir, entry.name);
if (entry.isDirectory()) {
if (entry.name === 'node_modules' || entry.name.startsWith('.')) continue;
files = files.concat(await getAllFiles(fullPath));
} else if (fullPath.endsWith('.js') || fullPath.endsWith('.jsx')) {
files.push(fullPath);
}
}
return files;
}
/* eslint-disable no-console */
async function addCopyright(files) {
const oldCopyrightRegex =
/^(\/\*\n \* Copyright \(c\) \d{4} by Christian Kellner\.\n \* Licensed under Apache-2.0 with Commons Clause and Attribution\/Naming Clause\n \*\/\n\n)+/;
for (let file of files) {
try {
let content = await fs.readFile(file, 'utf8');
const strippedContent = content.replace(oldCopyrightRegex, '');
const newContent = COPYRIGHT + strippedContent;
if (content !== newContent) {
await fs.writeFile(file, newContent);
console.log(`Added/Updated copyright in ${file}`);
}
} catch (err) {
console.error(`Error processing ${file}: ${err}`);
}
}
}
/* eslint-enable no-console */
const filesToProcess = process.argv.length > 2 ? process.argv.slice(2) : await getAllFiles();
await addCopyright(filesToProcess);

0
db/.gitkeep Normal file
View File

View File

@@ -1,84 +0,0 @@
<mxfile host="app.diagrams.net" modified="2022-01-29T18:34:51.211Z" agent="5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36" etag="W0jmvptvMSkuHq89hwUy" version="16.5.2" type="github">
<diagram id="C5RBs43oDa-KdzZeNtuy" name="Page-1">
<mxGraphModel dx="850" dy="907" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="827" pageHeight="1169" math="0" shadow="0">
<root>
<mxCell id="WIyWlLk6GJQsqaUBKTNV-0" />
<mxCell id="WIyWlLk6GJQsqaUBKTNV-1" parent="WIyWlLk6GJQsqaUBKTNV-0" />
<mxCell id="4kAlOAlRylSy7JMoHAEd-5" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" edge="1" parent="WIyWlLk6GJQsqaUBKTNV-1" source="WIyWlLk6GJQsqaUBKTNV-3" target="WIyWlLk6GJQsqaUBKTNV-7">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="WIyWlLk6GJQsqaUBKTNV-3" value="Job1" style="rounded=1;whiteSpace=wrap;html=1;fontSize=12;glass=0;strokeWidth=1;shadow=0;fillColor=#dae8fc;strokeColor=#6c8ebf;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
<mxGeometry x="100" y="50" width="120" height="30" as="geometry" />
</mxCell>
<mxCell id="4kAlOAlRylSy7JMoHAEd-8" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" edge="1" parent="WIyWlLk6GJQsqaUBKTNV-1" source="WIyWlLk6GJQsqaUBKTNV-7" target="4kAlOAlRylSy7JMoHAEd-2">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="4kAlOAlRylSy7JMoHAEd-10" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" edge="1" parent="WIyWlLk6GJQsqaUBKTNV-1" source="WIyWlLk6GJQsqaUBKTNV-7" target="4kAlOAlRylSy7JMoHAEd-3">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="4kAlOAlRylSy7JMoHAEd-11" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" edge="1" parent="WIyWlLk6GJQsqaUBKTNV-1" source="WIyWlLk6GJQsqaUBKTNV-7" target="4kAlOAlRylSy7JMoHAEd-4">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="WIyWlLk6GJQsqaUBKTNV-7" value="FredyRuntime" style="rounded=1;whiteSpace=wrap;html=1;fontSize=12;glass=0;strokeWidth=1;shadow=0;fillColor=#fff2cc;strokeColor=#d6b656;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
<mxGeometry x="110" y="120" width="360" height="40" as="geometry" />
</mxCell>
<mxCell id="4kAlOAlRylSy7JMoHAEd-6" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" edge="1" parent="WIyWlLk6GJQsqaUBKTNV-1" source="4kAlOAlRylSy7JMoHAEd-0" target="WIyWlLk6GJQsqaUBKTNV-7">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="4kAlOAlRylSy7JMoHAEd-0" value="Job2" style="rounded=1;whiteSpace=wrap;html=1;fontSize=12;glass=0;strokeWidth=1;shadow=0;fillColor=#dae8fc;strokeColor=#6c8ebf;" vertex="1" parent="WIyWlLk6GJQsqaUBKTNV-1">
<mxGeometry x="230" y="50" width="120" height="30" as="geometry" />
</mxCell>
<mxCell id="4kAlOAlRylSy7JMoHAEd-7" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" edge="1" parent="WIyWlLk6GJQsqaUBKTNV-1" source="4kAlOAlRylSy7JMoHAEd-1" target="WIyWlLk6GJQsqaUBKTNV-7">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="4kAlOAlRylSy7JMoHAEd-1" value="Job3" style="rounded=1;whiteSpace=wrap;html=1;fontSize=12;glass=0;strokeWidth=1;shadow=0;fillColor=#dae8fc;strokeColor=#6c8ebf;" vertex="1" parent="WIyWlLk6GJQsqaUBKTNV-1">
<mxGeometry x="360" y="50" width="120" height="30" as="geometry" />
</mxCell>
<mxCell id="4kAlOAlRylSy7JMoHAEd-13" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;" edge="1" parent="WIyWlLk6GJQsqaUBKTNV-1" source="4kAlOAlRylSy7JMoHAEd-2" target="4kAlOAlRylSy7JMoHAEd-12">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="4kAlOAlRylSy7JMoHAEd-2" value="Provider1" style="rounded=1;whiteSpace=wrap;html=1;fontSize=12;glass=0;strokeWidth=1;shadow=0;fillColor=#d5e8d4;strokeColor=#82b366;" vertex="1" parent="WIyWlLk6GJQsqaUBKTNV-1">
<mxGeometry x="100" y="210" width="120" height="40" as="geometry" />
</mxCell>
<mxCell id="4kAlOAlRylSy7JMoHAEd-14" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;" edge="1" parent="WIyWlLk6GJQsqaUBKTNV-1" source="4kAlOAlRylSy7JMoHAEd-3">
<mxGeometry relative="1" as="geometry">
<mxPoint x="290" y="290" as="targetPoint" />
</mxGeometry>
</mxCell>
<mxCell id="4kAlOAlRylSy7JMoHAEd-3" value="Provider2" style="rounded=1;whiteSpace=wrap;html=1;fontSize=12;glass=0;strokeWidth=1;shadow=0;fillColor=#d5e8d4;strokeColor=#82b366;" vertex="1" parent="WIyWlLk6GJQsqaUBKTNV-1">
<mxGeometry x="230" y="210" width="120" height="40" as="geometry" />
</mxCell>
<mxCell id="4kAlOAlRylSy7JMoHAEd-15" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" edge="1" parent="WIyWlLk6GJQsqaUBKTNV-1" source="4kAlOAlRylSy7JMoHAEd-4" target="4kAlOAlRylSy7JMoHAEd-12">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="4kAlOAlRylSy7JMoHAEd-4" value="Provider3" style="rounded=1;whiteSpace=wrap;html=1;fontSize=12;glass=0;strokeWidth=1;shadow=0;fillColor=#d5e8d4;strokeColor=#82b366;" vertex="1" parent="WIyWlLk6GJQsqaUBKTNV-1">
<mxGeometry x="360" y="210" width="120" height="40" as="geometry" />
</mxCell>
<mxCell id="4kAlOAlRylSy7JMoHAEd-17" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;" edge="1" parent="WIyWlLk6GJQsqaUBKTNV-1" source="4kAlOAlRylSy7JMoHAEd-12" target="4kAlOAlRylSy7JMoHAEd-16">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="4kAlOAlRylSy7JMoHAEd-12" value="Similarity check" style="rounded=1;whiteSpace=wrap;html=1;fontSize=12;glass=0;strokeWidth=1;shadow=0;fillColor=#e1d5e7;strokeColor=#9673a6;" vertex="1" parent="WIyWlLk6GJQsqaUBKTNV-1">
<mxGeometry x="110" y="290" width="360" height="40" as="geometry" />
</mxCell>
<mxCell id="4kAlOAlRylSy7JMoHAEd-20" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" edge="1" parent="WIyWlLk6GJQsqaUBKTNV-1" source="4kAlOAlRylSy7JMoHAEd-16" target="4kAlOAlRylSy7JMoHAEd-18">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="4kAlOAlRylSy7JMoHAEd-16" value="Found similarity" style="rhombus;whiteSpace=wrap;html=1;" vertex="1" parent="WIyWlLk6GJQsqaUBKTNV-1">
<mxGeometry x="250" y="360" width="80" height="80" as="geometry" />
</mxCell>
<mxCell id="4kAlOAlRylSy7JMoHAEd-21" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;" edge="1" parent="WIyWlLk6GJQsqaUBKTNV-1" source="4kAlOAlRylSy7JMoHAEd-18" target="4kAlOAlRylSy7JMoHAEd-19">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="4kAlOAlRylSy7JMoHAEd-18" value="Notification Adapter1" style="rounded=1;whiteSpace=wrap;html=1;fontSize=12;glass=0;strokeWidth=1;shadow=0;fillColor=#f8cecc;strokeColor=#b85450;" vertex="1" parent="WIyWlLk6GJQsqaUBKTNV-1">
<mxGeometry x="230" y="460" width="120" height="40" as="geometry" />
</mxCell>
<mxCell id="4kAlOAlRylSy7JMoHAEd-19" value="Notification Adapter2" style="rounded=1;whiteSpace=wrap;html=1;fontSize=12;glass=0;strokeWidth=1;shadow=0;fillColor=#f8cecc;strokeColor=#b85450;" vertex="1" parent="WIyWlLk6GJQsqaUBKTNV-1">
<mxGeometry x="230" y="520" width="120" height="40" as="geometry" />
</mxCell>
<mxCell id="4kAlOAlRylSy7JMoHAEd-22" value="No" style="text;html=1;resizable=0;autosize=1;align=center;verticalAlign=middle;points=[];fillColor=none;strokeColor=none;rounded=0;" vertex="1" parent="WIyWlLk6GJQsqaUBKTNV-1">
<mxGeometry x="300" y="440" width="30" height="20" as="geometry" />
</mxCell>
</root>
</mxGraphModel>
</diagram>
</mxfile>

Binary file not shown.

Before

Width:  |  Height:  |  Size: 189 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 121 KiB

After

Width:  |  Height:  |  Size: 3.7 MiB

BIN
doc/screenshot2.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.8 MiB

BIN
doc/screenshot3.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 531 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 323 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 93 KiB

BIN
doc/unraid_fredy_logo.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 417 KiB

View File

@@ -1,16 +1,26 @@
version: '3.8'
services:
fredy:
container_name: fredy
# build from empty build folder to reduce size of image
build:
context: .
dockerfile: Dockerfile
image: fredy/fredy
# map existing config and database
image: ghcr.io/orangecoding/fredy
environment:
- NODE_ENV=production
volumes:
- ./conf:/conf
- ./db:/db
ports:
- 9998:9998
- "9998:9998"
restart: unless-stopped
# Resource limits to prevent runaway memory usage from Chromium
deploy:
resources:
limits:
memory: 1G
healthcheck:
test: ["CMD", "curl", "--fail", "--silent", "--show-error", "--max-time", "5", "http://localhost:9998/"]
interval: 120s
timeout: 10s
retries: 3
start_period: 30s

78
docker-test.sh Executable file
View File

@@ -0,0 +1,78 @@
#!/bin/sh
set -e
# Stop and remove old container if it exists
if [ "$(docker ps -aq -f name=fredy)" ]; then
docker stop fredy || true
docker rm fredy || true
fi
# On Apple Silicon, force linux/amd64 to match production CI and avoid arm64/x86_64
# Chrome mismatch under Rosetta. On native Linux (amd64 or arm64) let Docker pick naturally. That took me fucking 1 hour to figure out.
PLATFORM=""
if [ "$(uname -m)" = "arm64" ] && [ "$(uname -s)" = "Darwin" ]; then
PLATFORM="linux/amd64"
fi
# Build image from local Dockerfile, forcing a fresh build without cache
if [ -n "$PLATFORM" ]; then
docker build --no-cache --platform "$PLATFORM" -t fredy:local .
else
docker build --no-cache -t fredy:local .
fi
# Run container with volumes and port mapping
if [ -n "$PLATFORM" ]; then
docker run -d --name fredy --platform "$PLATFORM" -v fredy_conf:/conf -v fredy_db:/db -p 9998:9998 fredy:local
else
docker run -d --name fredy -v fredy_conf:/conf -v fredy_db:/db -p 9998:9998 fredy:local
fi
echo "Waiting for app to be ready..."
for i in $(seq 1 30); do
if docker exec fredy curl -sf http://localhost:9998/ > /dev/null 2>&1; then
echo "App is up"
break
fi
if [ "$i" = "30" ]; then
echo "App did not come up in time"
docker logs fredy
exit 1
fi
sleep 2
done
# Verify the DB is readable/writable via the API.
# /api/demo is unauthenticated and reads the settings table — if SQLite is broken this returns an error.
echo "Testing DB via API (/api/demo)..."
DEMO_RESPONSE=$(docker exec fredy curl -sf http://localhost:9998/api/demo 2>&1)
if echo "$DEMO_RESPONSE" | grep -q "demoMode"; then
echo "DB is readable (got demoMode from /api/demo)"
else
echo "DB check failed — unexpected response from /api/demo: $DEMO_RESPONSE"
docker logs fredy
exit 1
fi
# Verify Chrome launches without crashing.
# On amd64: Chrome for Testing lives in the puppeteer cache.
# On arm64: system Chromium is used instead.
echo "Testing Chrome..."
CHROME=$(docker exec fredy find /root/.cache/puppeteer /home -name chrome -type f 2>/dev/null | head -1)
if [ -z "$CHROME" ]; then
CHROME=$(docker exec fredy which chromium 2>/dev/null || true)
fi
if [ -z "$CHROME" ]; then
echo "Chrome/Chromium binary not found"
exit 1
fi
if docker exec fredy "$CHROME" --headless --no-sandbox --disable-gpu --dump-dom https://example.com 2>&1 | grep -q "<html"; then
echo "Chrome works"
else
echo "Chrome failed to render a page"
docker exec fredy "$CHROME" --headless --no-sandbox --disable-gpu --dump-dom https://example.com 2>&1 | head -20
exit 1
fi
echo ""
echo "All checks passed."

48
eslint.config.js Normal file
View File

@@ -0,0 +1,48 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
// eslint.config.js
import js from '@eslint/js';
import prettier from 'eslint-config-prettier';
import globals from 'globals';
import react from 'eslint-plugin-react';
export default [
{
ignores: ['**/node_modules/**', '**/dist/**', '**/build/**', '**/public/**', 'db/**', 'conf/**'],
},
{
files: ['**/*.{js,jsx}'],
languageOptions: {
ecmaVersion: 'latest',
sourceType: 'module',
parserOptions: {
ecmaFeatures: { jsx: true },
},
globals: {
...globals.browser,
...globals.node,
...globals.jest,
Promise: 'readonly',
fetch: 'readonly',
describe: 'readonly',
after: 'readonly',
it: 'readonly',
beforeEach: 'readonly',
afterEach: 'readonly',
vi: 'readonly',
},
},
plugins: { react },
settings: { react: { version: 'detect' } },
rules: {
...js.configs.recommended.rules,
'no-console': ['error', { allow: ['warn', 'error'] }],
},
},
prettier,
];

View File

@@ -1,16 +1,20 @@
<!DOCTYPE html>
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8"
name="viewport"
content="user-scalable=no, width=device-width, initial-scale=1, maximum-scale=1">
<meta name="google" content="notranslate">
<head>
<meta
charset="UTF-8"
name="viewport"
content="user-scalable=no, width=device-width, initial-scale=1, maximum-scale=1"
/>
<meta name="google" content="notranslate" />
<meta name="mobile-web-app-capable" content="yes">
<meta name="apple-mobile-web-app-status-bar-style" content="black-translucent" />
<title>Fredy</title>
</head>
<body theme-mode="dark">
<title>Fredy || Real Estate Finder</title>
</head>
<body theme-mode="dark">
<div id="fredy" style="position: absolute; top: 0; left: 0; right: 0; bottom: 0"></div>
</body>
<script type="module" src="/ui/src/Index.jsx"></script>
</html>
<div id="fredy" style="position: absolute;top: 0;left: 0;right: 0;bottom: 0;"></div>
</body>
<script type="module" src="/ui/src/Index.jsx"></script>
</html>

116
index.js
View File

@@ -1,50 +1,68 @@
import fs from 'fs';
import { config } from './lib/utils.js';
import * as similarityCache from './lib/services/similarity-check/similarityCache.js';
import { setLastJobExecution } from './lib/services/storage/listingsStorage.js';
import * as jobStorage from './lib/services/storage/jobStorage.js';
import FredyRuntime from './lib/FredyRuntime.js';
import { duringWorkingHoursOrNotSet } from './lib/utils.js';
import './lib/api/api.js';
//if db folder does not exist, ensure to create it before loading anything else
if (!fs.existsSync('./db')) {
fs.mkdirSync('./db');
}
const path = './lib/provider';
const provider = fs.readdirSync(path).filter((file) => file.endsWith('.js'));
//assuming interval is always in minutes
const INTERVAL = config.interval * 60 * 1000;
/* eslint-disable no-console */
console.log(`Started Fredy successfully. Ui can be accessed via http://localhost:${config.port}`);
/* eslint-enable no-console */
const fetchedProvider = await Promise.all(
provider.filter((provider) => provider.endsWith('.js')).map(async (pro) => import(`${path}/${pro}`))
);
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
setInterval(
(function exec() {
const isDuringWorkingHoursOrNotSet = duringWorkingHoursOrNotSet(config, Date.now());
if (isDuringWorkingHoursOrNotSet) {
config.lastRun = Date.now();
jobStorage
.getJobs()
.filter((job) => job.enabled)
.forEach((job) => {
job.provider
.filter((p) => fetchedProvider.find((fp) => fp.metaInformation.id === p.id) != null)
.forEach(async (prov) => {
const pro = fetchedProvider.find((fp) => fp.metaInformation.id === prov.id);
pro.init(prov, job.blacklist);
await new FredyRuntime(pro.config, job.notificationAdapter, prov.id, job.id, similarityCache).execute();
setLastJobExecution(job.id);
});
});
} else {
/* eslint-disable no-console */
console.debug('Working hours set. Skipping as outside of working hours.');
/* eslint-enable no-console */
}
return exec;
})(),
INTERVAL
);
import fs from 'fs';
import { checkIfConfigIsAccessible, getProviders, refreshConfig } from './lib/utils.js';
import * as similarityCache from './lib/services/similarity-check/similarityCache.js';
import { runMigrations } from './lib/services/storage/migrations/migrate.js';
import { ensureDemoUserExists, ensureAdminUserExists } from './lib/services/storage/userStorage.js';
import { initTrackerCron } from './lib/services/crons/tracker-cron.js';
import logger from './lib/services/logger.js';
import { initActiveCheckerCron } from './lib/services/crons/listing-alive-cron.js';
import { initGeocodingCron } from './lib/services/crons/geocoding-cron.js';
import { getSettings } from './lib/services/storage/settingsStorage.js';
import SqliteConnection, { computeDbPath } from './lib/services/storage/SqliteConnection.js';
import { initJobExecutionService } from './lib/services/jobs/jobExecutionService.js';
//in the config, we store the path of the sqlite file, thus we must check if it is available
const isConfigAccessible = await checkIfConfigIsAccessible();
await SqliteConnection.init();
// Load configuration before any other startup steps
await refreshConfig();
if (!isConfigAccessible) {
logger.error('Configuration exists, but is not accessible. Please check the file permission');
process.exit(1);
}
// Run DB migrations once at startup and block until finished
await runMigrations();
const settings = await getSettings();
// Ensure the sqlite directory exists before loading anything else (based on config.sqlitepath)
const { dir: sqliteDir } = await computeDbPath();
if (!fs.existsSync(sqliteDir)) {
fs.mkdirSync(sqliteDir, { recursive: true });
}
// Load provider modules once at startup
const providers = await getProviders();
similarityCache.initSimilarityCache();
similarityCache.startSimilarityCacheReloader();
//assuming interval is always in minutes
const INTERVAL = settings.interval * 60 * 1000;
// Initialize API only after migrations completed
await import('./lib/api/api.js');
if (settings.demoMode) {
logger.info('Running in demo mode');
}
ensureAdminUserExists();
ensureDemoUserExists();
await initTrackerCron();
//do not wait for this to finish, let it run in the background
initActiveCheckerCron();
initGeocodingCron();
logger.info(`Started Fredy successfully. Ui can be accessed via http://localhost:${settings.port}`);
// Initialize the lean Job Execution Service (schedules and bus listeners)
initJobExecutionService({ providers, settings, intervalMs: INTERVAL });

341
lib/FredyPipelineExecutioner.js Executable file
View File

@@ -0,0 +1,341 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import { NoNewListingsWarning } from './errors.js';
import {
storeListings,
getKnownListingHashesForJobAndProvider,
deleteListingsById,
} from './services/storage/listingsStorage.js';
import { getJob } from './services/storage/jobStorage.js';
import * as notify from './notification/notify.js';
import Extractor from './services/extractor/extractor.js';
import urlModifier from './services/queryStringMutator.js';
import logger from './services/logger.js';
import { geocodeAddress } from './services/geocoding/geoCodingService.js';
import { distanceMeters } from './services/listings/distanceCalculator.js';
import { getUserSettings } from './services/storage/settingsStorage.js';
import { updateListingDistance } from './services/storage/listingsStorage.js';
import booleanPointInPolygon from '@turf/boolean-point-in-polygon';
/**
* @typedef {Object} Listing
* @property {string} id Stable unique identifier (hash) of the listing.
* @property {string} title Title or headline of the listing.
* @property {string} [address] Optional address/location text.
* @property {string} [price] Optional price text/value.
* @property {string} [url] Link to the listing detail page.
* @property {any} [meta] Provider-specific additional metadata.
*/
/**
* @typedef {Object} SimilarityCache
* @property {(title:string, address?:string)=>boolean} hasSimilarEntries Returns true if a similar entry is known.
* @property {(title:string, address?:string)=>void} addCacheEntry Adds a new entry to the similarity cache.
*/
/**
* Runtime orchestrator for fetching, normalizing, filtering, deduplicating, storing,
* and notifying about new listings from a configured provider.
*
* The execution flow is:
* 1) Prepare provider URL (sorting, etc.)
* 2) Extract raw listings from the provider
* 3) Normalize listings to the provider schema
* 4) Filter out incomplete/blacklisted listings
* 5) Identify new listings (vs. previously stored hashes)
* 6) Persist new listings
* 7) Filter out entries similar to already seen ones
* 8) Dispatch notifications
*/
class FredyPipelineExecutioner {
/**
* Create a new runtime instance for a single provider/job execution.
*
* @param {Object} providerConfig Provider configuration.
* @param {string} providerConfig.url Base URL to crawl.
* @param {string} [providerConfig.sortByDateParam] Query parameter used to enforce sorting by date (provider-specific).
* @param {string} [providerConfig.waitForSelector] CSS selector to wait for before parsing content.
* @param {Object.<string, string>} providerConfig.crawlFields Mapping of field names to selectors/paths to extract.
* @param {string} providerConfig.crawlContainer CSS selector for the container holding listing items.
* @param {(raw:any)=>Listing} providerConfig.normalize Function to convert raw scraped data into a Listing shape.
* @param {(listing:Listing)=>boolean} providerConfig.filter Function to filter out unwanted listings.
* @param {(url:string, waitForSelector?:string)=>Promise<void>|Promise<Listing[]>} [providerConfig.getListings] Optional override to fetch listings.
* @param {Object} notificationConfig Notification configuration passed to notification adapters.
* @param {Object} spatialFilter Optional spatial filter configuration.
* @param {string} providerId The ID of the provider currently in use.
* @param {string} jobKey Key of the job that is currently running (from within the config).
* @param {SimilarityCache} similarityCache Cache instance for checking similar entries.
* @param browser
*/
constructor(providerConfig, notificationConfig, spatialFilter, providerId, jobKey, similarityCache, browser) {
this._providerConfig = providerConfig;
this._notificationConfig = notificationConfig;
this._spatialFilter = spatialFilter;
this._providerId = providerId;
this._jobKey = jobKey;
this._similarityCache = similarityCache;
this._browser = browser;
}
/**
* Execute the end-to-end pipeline for a single provider run.
*
* @returns {Promise<Listing[]|void>} Resolves to the list of new (and similarity-filtered) listings
* after notifications have been sent; resolves to void when there are no new listings.
*/
execute() {
return Promise.resolve(urlModifier(this._providerConfig.url, this._providerConfig.sortByDateParam))
.then(this._providerConfig.getListings?.bind(this) ?? this._getListings.bind(this))
.then(this._normalize.bind(this))
.then(this._filter.bind(this))
.then(this._findNew.bind(this))
.then(this._geocode.bind(this))
.then(this._save.bind(this))
.then(this._calculateDistance.bind(this))
.then(this._filterBySimilarListings.bind(this))
.then(this._filterByArea.bind(this))
.then(this._notify.bind(this))
.catch(this._handleError.bind(this));
}
/**
* Geocode new listings.
*
* @param {Listing[]} newListings New listings to geocode.
* @returns {Promise<Listing[]>} Resolves with the listings (potentially with added coordinates).
*/
async _geocode(newListings) {
for (const listing of newListings) {
if (listing.address) {
const coords = await geocodeAddress(listing.address);
if (coords) {
listing.latitude = coords.lat;
listing.longitude = coords.lng;
}
}
}
return newListings;
}
/**
* Filter listings by area using the provider's area filter if available.
* Only filters if areaFilter is set on the provider AND the listing has coordinates.
*
* @param {Listing[]} newListings New listings to filter by area.
* @returns {Promise<Listing[]>} Resolves with listings that are within the area (or not filtered if no area is set).
*/
_filterByArea(newListings) {
const polygonFeatures = this._spatialFilter?.features?.filter((f) => f.geometry?.type === 'Polygon');
// If no area filter is set, return all listings
if (!polygonFeatures?.length) {
return newListings;
}
const filteredIds = [];
// Filter listings by area - keep only those within the polygon
const keptListings = newListings.filter((listing) => {
// If listing doesn't have coordinates, keep it (don't filter out)
if (listing.latitude == null || listing.longitude == null) {
return true;
}
// Check if the point is inside the polygons
const point = [listing.longitude, listing.latitude]; // GeoJSON format: [lon, lat]
const isInPolygon = polygonFeatures.some((feature) => booleanPointInPolygon(point, feature));
if (!isInPolygon) {
filteredIds.push(listing.id);
}
return isInPolygon;
});
if (filteredIds.length > 0) {
deleteListingsById(filteredIds);
}
return keptListings;
}
/**
* Fetch listings from the provider, using the default Extractor flow unless
* a provider-specific getListings override is supplied.
*
* @param {string} url The provider URL to fetch from.
* @returns {Promise<Listing[]>} Resolves with an array of listings (empty when none found).
*/
_getListings(url) {
const extractor = new Extractor({ ...this._providerConfig.puppeteerOptions, browser: this._browser });
return new Promise((resolve, reject) => {
extractor
.execute(url, this._providerConfig.waitForSelector)
.then(() => {
const listings = extractor.parseResponseText(
this._providerConfig.crawlContainer,
this._providerConfig.crawlFields,
url,
);
resolve(listings == null ? [] : listings);
})
.catch((err) => {
reject(err);
logger.error(err);
});
});
}
/**
* Normalize raw listings into the provider-specific Listing shape.
*
* @param {any[]} listings Raw listing entries from the extractor or override.
* @returns {Listing[]} Normalized listings.
*/
_normalize(listings) {
return listings.map(this._providerConfig.normalize);
}
/**
* Filter out listings that are missing required fields and those rejected by the
* provider's blacklist/filter function.
*
* @param {Listing[]} listings Listings to filter.
* @returns {Listing[]} Filtered listings that pass validation and provider filter.
*/
_filter(listings) {
const keys = Object.keys(this._providerConfig.crawlFields);
const filteredListings = listings.filter((item) => keys.every((key) => key in item));
return filteredListings.filter(this._providerConfig.filter);
}
/**
* Determine which listings are new by comparing their IDs against stored hashes.
*
* @param {Listing[]} listings Listings to evaluate for novelty.
* @returns {Listing[]} New listings not seen before.
* @throws {NoNewListingsWarning} When no new listings are found.
*/
_findNew(listings) {
logger.debug(`Checking ${listings.length} listings for new entries (Provider: '${this._providerId}')`);
const hashes = getKnownListingHashesForJobAndProvider(this._jobKey, this._providerId) || [];
const newListings = listings.filter((o) => !hashes.includes(o.id));
if (newListings.length === 0) {
throw new NoNewListingsWarning();
}
return newListings;
}
/**
* Send notifications for new listings using the configured notification adapter(s).
*
* @param {Listing[]} newListings New listings to notify about.
* @returns {Promise<Listing[]>} Resolves to the provided listings after notifications complete.
* @throws {NoNewListingsWarning} When there are no listings to notify about.
*/
_notify(newListings) {
if (newListings.length === 0) {
throw new NoNewListingsWarning();
}
const sendNotifications = notify.send(this._providerId, newListings, this._notificationConfig, this._jobKey);
return Promise.all(sendNotifications).then(() => newListings);
}
/**
* Persist new listings and pass them through.
*
* @param {Listing[]} newListings Listings to store.
* @returns {Listing[]} The same listings, unchanged.
*/
_save(newListings) {
logger.debug(`Storing ${newListings.length} new listings (Provider: '${this._providerId}')`);
storeListings(this._jobKey, this._providerId, newListings);
return newListings;
}
/**
* Calculate distance for new listings.
*
* @param {Listing[]} listings
* @returns {Listing[]}
* @private
*/
_calculateDistance(listings) {
if (listings.length === 0) return [];
const job = getJob(this._jobKey);
const userId = job?.userId;
if (userId == null || typeof userId !== 'string') {
logger.debug('Skipping distance calculation: userId is missing or invalid');
return listings;
}
const userSettings = getUserSettings(userId);
const homeAddress = userSettings?.home_address;
if (!homeAddress || !homeAddress.coords) {
return listings;
}
const { lat, lng } = homeAddress.coords;
for (const listing of listings) {
if (listing.latitude != null && listing.longitude != null) {
const dist = distanceMeters(lat, lng, listing.latitude, listing.longitude);
updateListingDistance(listing.id, dist);
listing.distance_to_destination = dist;
}
}
return listings;
}
/**
* Remove listings that are similar to already known entries according to the similarity cache.
* Adds the remaining listings to the cache.
*
* @param {Listing[]} listings Listings to filter by similarity.
* @returns {Listing[]} Listings considered unique enough to keep.
*/
_filterBySimilarListings(listings) {
const filteredIds = [];
const keptListings = listings.filter((listing) => {
const similar = this._similarityCache.checkAndAddEntry({
title: listing.title,
address: listing.address,
price: listing.price,
});
if (similar) {
logger.debug(
`Filtering similar entry for title '${listing.title}' and address '${listing.address}' (Provider: '${this._providerId}')`,
);
filteredIds.push(listing.id);
}
return !similar;
});
if (filteredIds.length > 0) {
deleteListingsById(filteredIds);
}
return keptListings;
}
/**
* Handle errors occurring in the pipeline, logging levels depending on type.
*
* @param {Error} err Error instance thrown by previous steps.
* @returns {void}
*/
_handleError(err) {
if (err.name === 'NoNewListingsWarning') {
logger.debug(`No new listings found (Provider: '${this._providerId}').`);
} else {
logger.error(err);
}
}
}
export default FredyPipelineExecutioner;

View File

@@ -1,131 +0,0 @@
import { NoNewListingsWarning } from './errors.js';
import { setKnownListings, getKnownListings } from './services/storage/listingsStorage.js';
import * as notify from './notification/notify.js';
import xray from './services/scraper.js';
import * as scrapingAnt from './services/scrapingAnt.js';
import urlModifier from './services/queryStringMutator.js';
class FredyRuntime {
/**
*
* @param providerConfig the config for the specific provider, we're going to query at the moment
* @param notificationConfig the config for all notifications
* @param providerId the id of the provider currently in use
* @param jobKey key of the job that is currently running (from within the config)
* @param similarityCache cache instance holding values to check for similarity of entries
*/
constructor(providerConfig, notificationConfig, providerId, jobKey, similarityCache) {
this._providerConfig = providerConfig;
this._notificationConfig = notificationConfig;
this._providerId = providerId;
this._jobKey = jobKey;
this._similarityCache = similarityCache;
}
execute() {
return (
//modify the url to make sure search order is correctly set
Promise.resolve(urlModifier(this._providerConfig.url, this._providerConfig.sortByDateParam))
//scraping the site and try finding new listings
.then(this._getListings.bind(this))
//bring them in a proper form (dictated by the provider)
.then(this._normalize.bind(this))
//filter listings with stuff tagged by the blacklist of the provider
.then(this._filter.bind(this))
//check if new listings available. if so proceed
.then(this._findNew.bind(this))
//store everything in db
.then(this._save.bind(this))
//check for similar listings. if found, remove them before notifying
.then(this._filterBySimilarListings.bind(this))
//notify the user using the configured notification adapter
.then(this._notify.bind(this))
//if an error occurred on the way, handle it here.
.catch(this._handleError.bind(this))
);
}
_getListings(url) {
return new Promise((resolve, reject) => {
const id = this._providerId;
if (scrapingAnt.needScrapingAnt(id) && !scrapingAnt.isScrapingAntApiKeySet()) {
const error = 'Immoscout or Immonet can only be used with if you have set an apikey for scrapingAnt.';
/* eslint-disable no-console */
console.log(error);
/* eslint-enable no-console */
reject(error);
return;
}
const u = scrapingAnt.needScrapingAnt(id) ? scrapingAnt.transformUrlForScrapingAnt(url, id) : url;
try {
if (this._providerConfig.paginate != null) {
xray(u, this._providerConfig.crawlContainer, [this._providerConfig.crawlFields])
//the first 2 pages should be enough here
.limit(2)
.paginate(this._providerConfig.paginate)
.then((listings) => {
resolve(listings == null ? [] : listings);
})
.catch((err) => {
reject(err);
console.error(err);
});
} else {
xray(u, this._providerConfig.crawlContainer, [this._providerConfig.crawlFields])
.then((listings) => {
resolve(listings == null ? [] : listings);
})
.catch((err) => {
reject(err);
console.error(err);
});
}
} catch (error) {
reject(error);
console.error(error);
}
});
}
_normalize(listings) {
return listings.map(this._providerConfig.normalize);
}
_filter(listings) {
return listings.filter(this._providerConfig.filter);
}
_findNew(listings) {
const newListings = listings.filter((o) => getKnownListings(this._jobKey, this._providerId)[o.id] == null);
if (newListings.length === 0) {
throw new NoNewListingsWarning();
}
return newListings;
}
_notify(newListings) {
if (newListings.length === 0) {
throw new NoNewListingsWarning();
}
const sendNotifications = notify.send(this._providerId, newListings, this._notificationConfig, this._jobKey);
return Promise.all(sendNotifications).then(() => newListings);
}
_save(newListings) {
const currentListings = getKnownListings(this._jobKey, this._providerId) || {};
newListings.forEach((listing) => {
currentListings[listing.id] = Date.now();
});
setKnownListings(this._jobKey, this._providerId, currentListings);
return newListings;
}
_filterBySimilarListings(listings) {
const filteredList = listings.filter((listing) => {
const similar = this._similarityCache.hasSimilarEntries(this._jobKey, listing.title);
if (similar) {
/* eslint-disable no-console */
console.debug(`Filtering similar entry for job with id ${this._jobKey} with title: `, listing.title);
/* eslint-enable no-console */
}
return !similar;
});
filteredList.forEach((filter) => this._similarityCache.addCacheEntry(this._jobKey, filter.title));
return filteredList;
}
_handleError(err) {
if (err.name !== 'NoNewListingsWarning') console.error(err);
}
}
export default FredyRuntime;

10
lib/TRACKING_POIS.js Normal file
View File

@@ -0,0 +1,10 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
export const TRACKING_POIS = {
DISTANCE_ADDRESS_ENTERED: 'DISTANCE_ADDRESS_ENTERED',
WELCOME_FINISHED: 'WELCOME_FINISHED',
WELCOME_SKIPPED: 'WELCOME_SKIPPED',
};

View File

@@ -1,36 +1,65 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import { notificationAdapterRouter } from './routes/notificationAdapterRouter.js';
import { authInterceptor, cookieSession, adminInterceptor } from './security.js';
import { generalSettingsRouter } from './routes/generalSettingsRoute.js';
import { analyticsRouter } from './routes/analyticsRouter.js';
import { providerRouter } from './routes/providerRouter.js';
import { versionRouter } from './routes/versionRouter.js';
import { loginRouter } from './routes/loginRoute.js';
import { config } from '../utils.js';
import { userRouter } from './routes/userRoute.js';
import { userSettingsRouter } from './routes/userSettingsRoute.js';
import { jobRouter } from './routes/jobRouter.js';
import bodyParser from 'body-parser';
import restana from 'restana';
import files from 'serve-static';
import path from 'path';
import { getDirName } from '../utils.js';
import { demoRouter } from './routes/demoRouter.js';
import logger from '../services/logger.js';
import { listingsRouter } from './routes/listingsRouter.js';
import { getSettings } from '../services/storage/settingsStorage.js';
import { dashboardRouter } from './routes/dashboardRouter.js';
import { backupRouter } from './routes/backupRouter.js';
import { trackingRouter } from './routes/trackingRoute.js';
import { registerMcpRoutes } from '../mcp/mcpHttpRoute.js';
const service = restana();
const staticService = files(path.join(getDirName(), '../ui/public'));
const PORT = config.port || 9998;
const PORT = (await getSettings()).port || 9998;
service.use(bodyParser.json());
service.use(cookieSession());
service.use(staticService);
service.use('/api/admin', authInterceptor());
service.use('/api/jobs', authInterceptor());
service.use('/api/version', authInterceptor());
service.use('/api/listings', authInterceptor());
service.use('/api/dashboard', authInterceptor());
service.use('/api/user/settings', authInterceptor());
service.use('/api/tracking', authInterceptor());
// /admin can only be accessed when user is having admin permissions
service.use('/api/admin', adminInterceptor());
service.use('/api/jobs/notificationAdapter', notificationAdapterRouter);
service.use('/api/admin/generalSettings', generalSettingsRouter);
service.use('/api/admin/backup', backupRouter);
service.use('/api/jobs/provider', providerRouter);
service.use('/api/jobs/insights', analyticsRouter);
service.use('/api/admin/users', userRouter);
service.use('/api/user/settings', userSettingsRouter);
service.use('/api/version', versionRouter);
service.use('/api/jobs', jobRouter);
service.use('/api/login', loginRouter);
/* eslint-disable no-console */
service.use('/api/listings', listingsRouter);
service.use('/api/dashboard', dashboardRouter);
service.use('/api/tracking', trackingRouter);
//this route is unsecured intentionally as it is being queried from the login page
service.use('/api/demo', demoRouter);
// MCP Streamable HTTP endpoint (secured via Bearer token, not cookie-session)
registerMcpRoutes(service);
service.start(PORT).then(() => {
console.info(`Started API service on port ${PORT}`);
logger.debug(`Started API service on port ${PORT}`);
});

View File

@@ -1,10 +0,0 @@
import restana from 'restana';
import * as listingStorage from '../../services/storage/listingsStorage.js';
const service = restana();
const analyticsRouter = service.newRouter();
analyticsRouter.get('/:jobId', async (req, res) => {
const { jobId } = req.params;
res.body = listingStorage.getListingProviderDataForAnalytics(jobId) || {};
res.send();
});
export { analyticsRouter };

View File

@@ -0,0 +1,75 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import restana from 'restana';
import {
buildBackupFileName,
createBackupZip,
precheckRestore,
restoreFromZip,
} from '../../services/storage/backupRestoreService.js';
/**
* Backup & Restore Admin Router
*
* Endpoints:
* - GET /api/admin/backup
* Returns the current database as a zip download. Content-Type: application/zip
* - POST /api/admin/backup/restore?dryRun=true
* Accepts a zip file (raw body). Returns a compatibility report, does not restore.
* - POST /api/admin/backup/restore?force=true|false
* Accepts a zip file (raw body). Restores the database; when incompatible and force=false, returns 400.
*/
const service = restana();
const backupRouter = service.newRouter();
backupRouter.get('/', async (req, res) => {
const zipBuffer = await createBackupZip();
const fileName = await buildBackupFileName();
res.setHeader('Content-Type', 'application/zip');
res.setHeader('Content-Disposition', `attachment; filename="${fileName}"`);
res.send(zipBuffer);
});
/**
* Read the full request body as a Buffer. Used for raw zip uploads.
* @param {import('http').IncomingMessage} req
* @returns {Promise<Buffer>}
*/
function readBody(req) {
return new Promise((resolve, reject) => {
const chunks = [];
req.on('data', (c) => chunks.push(c));
req.on('end', () => resolve(Buffer.concat(chunks)));
req.on('error', (e) => reject(e));
});
}
// Upload endpoint. Accepts raw zip (Content-Type: application/zip or application/octet-stream)
// Query parameters:
// - dryRun=true => only validate and return compatibility info
// - force=true => proceed even if incompatible
backupRouter.post('/restore', async (req, res) => {
const { dryRun = 'false', force = 'false' } = req.query || {};
const doDryRun = String(dryRun) === 'true';
const doForce = String(force) === 'true';
const body = await readBody(req);
if (doDryRun) {
res.body = await precheckRestore(body);
return res.send();
}
try {
res.body = await restoreFromZip(body, { force: doForce });
return res.send();
} catch (e) {
res.statusCode = 400;
res.body = { message: e?.message || 'Restore failed', details: e?.payload || null };
return res.send();
}
});
export { backupRouter };

View File

@@ -0,0 +1,71 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import restana from 'restana';
import * as jobStorage from '../../services/storage/jobStorage.js';
import * as userStorage from '../../services/storage/userStorage.js';
import { getListingsKpisForJobIds, getProviderDistributionForJobIds } from '../../services/storage/listingsStorage.js';
import { getSettings } from '../../services/storage/settingsStorage.js';
const service = restana();
export const dashboardRouter = service.newRouter();
function isAdmin(req) {
const user = req.session?.currentUser ? userStorage.getUser(req.session.currentUser) : null;
return !!user?.isAdmin;
}
function getAccessibleJobs(req) {
const currentUser = req.session.currentUser;
const admin = isAdmin(req);
return jobStorage
.getJobs()
.filter((job) => admin || job.userId === currentUser || job.shared_with_user.includes(currentUser));
}
function cap(val) {
return String(val).charAt(0).toUpperCase() + String(val).slice(1);
}
dashboardRouter.get('/', async (req, res) => {
const jobs = getAccessibleJobs(req);
const settings = await getSettings();
// KPIs
const totalJobs = jobs.length;
const totalListings = jobs.reduce((sum, j) => sum + (j.numberOfFoundListings || 0), 0);
const jobIds = jobs.map((j) => j.id);
const { numberOfActiveListings, avgPriceOfListings } = getListingsKpisForJobIds(jobIds);
// Build Pie data in a simple shape the frontend can consume directly
// Shape: { labels: string[], values: number[] } with values as percentages
const providerPieRaw = getProviderDistributionForJobIds(jobIds);
const providerPie = Array.isArray(providerPieRaw)
? {
labels: providerPieRaw.map((p) => cap(p.type)),
values: providerPieRaw.map((p) => Number(p.value) || 0),
}
: providerPieRaw && typeof providerPieRaw === 'object'
? {
labels: Array.isArray(providerPieRaw.labels) ? providerPieRaw.labels : [],
values: Array.isArray(providerPieRaw.values) ? providerPieRaw.values : [],
}
: { labels: [], values: [] };
res.body = {
general: {
interval: settings.interval,
lastRun: settings.lastRun || null,
nextRun: settings.lastRun == null ? 0 : settings.lastRun + settings.interval * 60000,
},
kpis: {
totalJobs,
totalListings,
numberOfActiveListings,
avgPriceOfListings,
},
pie: providerPie,
};
res.send();
});

View File

@@ -0,0 +1,17 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import restana from 'restana';
import { getSettings } from '../../services/storage/settingsStorage.js';
const service = restana();
const demoRouter = service.newRouter();
demoRouter.get('/', async (req, res) => {
const settings = await getSettings();
res.body = Object.assign({}, { demoMode: settings.demoMode });
res.send();
});
export { demoRouter };

View File

@@ -1,18 +1,38 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import restana from 'restana';
import { config, getDirName } from '../../utils.js';
import { getDirName } from '../../utils.js';
import fs from 'fs';
import { ensureDemoUserExists } from '../../services/storage/userStorage.js';
import logger from '../../services/logger.js';
import { getSettings, upsertSettings } from '../../services/storage/settingsStorage.js';
const service = restana();
const generalSettingsRouter = service.newRouter();
generalSettingsRouter.get('/', async (req, res) => {
res.body = Object.assign({}, config);
res.body = Object.assign({}, await getSettings());
res.send();
});
generalSettingsRouter.post('/', async (req, res) => {
const settings = req.body;
const { sqlitepath, ...appSettings } = req.body || {};
const localSettings = await getSettings();
if (localSettings.demoMode) {
res.send(new Error('In demo mode, it is not allowed to change these settings.'));
return;
}
try {
fs.writeFileSync(`${getDirName()}/../conf/config.json`, JSON.stringify(settings));
if (typeof sqlitepath !== 'undefined') {
fs.writeFileSync(`${getDirName()}/../conf/config.json`, JSON.stringify({ sqlitepath }));
}
upsertSettings(appSettings);
ensureDemoUserExists();
} catch (err) {
console.error(err);
logger.error(err);
res.send(new Error('Error while trying to write settings.'));
return;
}

View File

@@ -1,12 +1,23 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import restana from 'restana';
import fetch from 'node-fetch';
import * as jobStorage from '../../services/storage/jobStorage.js';
import * as userStorage from '../../services/storage/userStorage.js';
import * as immoscoutProvider from '../../provider/immoscout.js';
import { config } from '../../utils.js';
import { isAdmin } from '../security.js';
import logger from '../../services/logger.js';
import { bus } from '../../services/events/event-bus.js';
import { isRunning as isJobRunning } from '../../services/jobs/run-state.js';
import { addClient as addSseClient, removeClient } from '../../services/sse/sse-broker.js';
import { getSettings } from '../../services/storage/settingsStorage.js';
const service = restana();
const jobRouter = service.newRouter();
const DEMO_JOB_NAME = 'Demo-Job';
function doesJobBelongsToUser(job, req) {
const userId = req.session.currentUser;
if (userId == null) {
@@ -16,43 +27,166 @@ function doesJobBelongsToUser(job, req) {
if (user == null) {
return false;
}
return user.isAdmin || job.userId === job.userId;
return user.isAdmin || job.userId === user.id;
}
jobRouter.get('/', async (req, res) => {
const isUserAdmin = isAdmin(req);
//show only the jobs which belongs to the user (or all of the user is an admin)
res.body = jobStorage.getJobs().filter((job) => isUserAdmin || job.userId === req.session.currentUser);
res.body = jobStorage
.getJobs()
.filter(
(job) =>
isUserAdmin || job.userId === req.session.currentUser || job.shared_with_user.includes(req.session.currentUser),
)
.map((job) => {
return {
...job,
running: isJobRunning(job.id),
isOnlyShared:
!isUserAdmin &&
job.userId !== req.session.currentUser &&
job.shared_with_user.includes(req.session.currentUser),
};
});
res.send();
});
jobRouter.get('/processingTimes', async (req, res) => {
let scrapingAntData = null;
if (config.scrapingAnt.apiKey != null && config.scrapingAnt.apiKey.length > 0) {
try {
const response = await fetch(`https://api.scrapingant.com/v2/usage?x-api-key=${config.scrapingAnt.apiKey}`);
scrapingAntData = await response.json();
} catch (Exception) {
console.error('Could not query plan data from scraping ant.', Exception);
}
}
res.body = {
interval: config.interval,
lastRun: config.lastRun || null,
scrapingAntData,
jobRouter.get('/data', async (req, res) => {
const { page, pageSize = 50, activityFilter, sortfield = null, sortdir = 'asc', freeTextFilter } = req.query || {};
// normalize booleans
const toBool = (v) => {
if (v === true || v === 'true' || v === 1 || v === '1') return true;
if (v === false || v === 'false' || v === 0 || v === '0') return false;
return null;
};
const normalizedActivity = toBool(activityFilter);
const queryResult = jobStorage.queryJobs({
page: page ? parseInt(page, 10) : 1,
pageSize: pageSize ? parseInt(pageSize, 10) : 50,
freeTextFilter: freeTextFilter || null,
activityFilter: normalizedActivity,
sortField: sortfield || null,
sortDir: sortdir === 'desc' ? 'desc' : 'asc',
userId: req.session.currentUser,
isAdmin: isAdmin(req),
});
const isUserAdmin = isAdmin(req);
// Map result to include runtime status
queryResult.result = queryResult.result.map((job) => {
return {
...job,
running: isJobRunning(job.id),
isOnlyShared:
!isUserAdmin &&
job.userId !== req.session.currentUser &&
job.shared_with_user.includes(req.session.currentUser),
};
});
res.body = queryResult;
res.send();
});
jobRouter.post('/', async (req, res) => {
const { provider, notificationAdapter, name, blacklist = [], jobId, enabled } = req.body;
if (
provider.find((p) => p.id === immoscoutProvider.metaInformation.id) != null &&
(config.scrapingAnt.apiKey == null || config.scrapingAnt.apiKey.length === 0)
) {
res.send(
new Error('To use Immoscout as provider, you need to configure ScrapingAnt first. Please check the readme.')
);
// Server-Sent Events for job status updates
jobRouter.get('/events', async (req, res) => {
const userId = req.session.currentUser;
if (userId == null) {
res.send({ message: 'Unauthorized' }, 401);
return;
}
// SSE headers
res.setHeader('Content-Type', 'text/event-stream');
res.setHeader('Cache-Control', 'no-cache');
res.setHeader('Connection', 'keep-alive');
try {
// Initial comment to establish stream
res.write(': connected\n\n');
addSseClient(userId, res);
// Cleanup on close/aborted
const onClose = () => removeClient(userId, res);
// restana exposes original req/res; use both close and finish
req.on('close', onClose);
req.on('aborted', onClose);
res.on('close', onClose);
} catch (e) {
logger.error('Error establishing SSE connection', e);
try {
res.end();
} catch {
//noop
}
}
});
jobRouter.post('/startAll', async (req, res) => {
try {
const userId = req.session.currentUser;
// Emit only the userId; handler will decide based on admin/ownership
bus.emit('jobs:runAll', { userId });
res.send({ message: 'Run all accepted' }, 202);
} catch (err) {
logger.error('Failed to trigger startAll', err);
res.send({ message: 'Unexpected error' }, 500);
}
});
// Trigger a single job run
jobRouter.post('/:jobId/run', async (req, res) => {
const { jobId } = req.params;
try {
const job = jobStorage.getJob(jobId);
if (!job) {
res.send({ message: 'Job not found' }, 404);
return;
}
if (!doesJobBelongsToUser(job, req)) {
res.send({ message: 'You are trying to run a job that is not associated to your user' }, 403);
return;
}
if (isJobRunning(jobId)) {
res.send({ message: 'Job is already running' }, 409);
return;
}
// fire and forget; actual execution handled by index.js listener
bus.emit('jobs:runOne', { jobId });
res.send({ message: 'Job run accepted' }, 202);
} catch (error) {
logger.error(error);
res.send({ message: 'Unexpected error triggering job' }, 500);
}
});
jobRouter.post('/', async (req, res) => {
const {
provider,
notificationAdapter,
name,
blacklist = [],
jobId,
enabled,
shareWithUsers = [],
spatialFilter = null,
} = req.body;
const settings = await getSettings();
try {
let jobFromDb = jobStorage.getJob(jobId);
if (jobFromDb && !doesJobBelongsToUser(jobFromDb, req)) {
res.send(new Error('You are trying to change a job that is not associated to your user.'));
return;
}
if (settings.demoMode && jobFromDb && jobFromDb.name === DEMO_JOB_NAME) {
res.send(new Error('Sorry, but you cannot change the Status of our Demo Job ;)'));
return;
}
jobStorage.upsertJob({
userId: req.session.currentUser,
jobId,
@@ -61,17 +195,26 @@ jobRouter.post('/', async (req, res) => {
blacklist,
provider,
notificationAdapter,
shareWithUsers,
spatialFilter,
});
} catch (error) {
res.send(new Error(error));
console.error(error);
logger.error(error);
}
res.send();
});
jobRouter.delete('', async (req, res) => {
const { jobId } = req.body;
const settings = await getSettings();
try {
const job = jobStorage.getJob(jobId);
if (settings.demoMode && job.name === DEMO_JOB_NAME) {
res.send(new Error('Sorry, but you cannot remove the Demo Job ;)'));
return;
}
if (!doesJobBelongsToUser(job, req)) {
res.send(new Error('You are trying to remove a job that is not associated to your user'));
} else {
@@ -79,15 +222,22 @@ jobRouter.delete('', async (req, res) => {
}
} catch (error) {
res.send(new Error(error));
console.error(error);
logger.error(error);
}
res.send();
});
jobRouter.put('/:jobId/status', async (req, res) => {
const { status } = req.body;
const { jobId } = req.params;
const settings = await getSettings();
try {
const job = jobStorage.getJob(jobId);
if (settings.demoMode && job.name === DEMO_JOB_NAME) {
res.send(new Error('Sorry, but you cannot change the Status of our Demo Job ;)'));
return;
}
if (!doesJobBelongsToUser(job, req)) {
res.send(new Error('You are trying change a job that is not associated to your user'));
} else {
@@ -98,8 +248,20 @@ jobRouter.put('/:jobId/status', async (req, res) => {
}
} catch (error) {
res.send(new Error(error));
console.error(error);
logger.error(error);
}
res.send();
});
jobRouter.get('/shareableUserList', async (req, res) => {
const currentUser = req.session.currentUser;
const users = userStorage.getUsers(false);
res.body = users
.filter((user) => !user.isAdmin && user.id !== currentUser)
.map((user) => ({
id: user.id,
name: user.username,
}));
res.send();
});
export { jobRouter };

View File

@@ -0,0 +1,139 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import restana from 'restana';
import * as listingStorage from '../../services/storage/listingsStorage.js';
import * as watchListStorage from '../../services/storage/watchListStorage.js';
import { isAdmin as isAdminFn } from '../security.js';
import logger from '../../services/logger.js';
import { nullOrEmpty } from '../../utils.js';
import { getJobs } from '../../services/storage/jobStorage.js';
import { getSettings } from '../../services/storage/settingsStorage.js';
const service = restana();
const listingsRouter = service.newRouter();
listingsRouter.get('/table', async (req, res) => {
const {
page,
pageSize = 50,
activityFilter,
jobNameFilter,
providerFilter,
watchListFilter,
sortfield = null,
sortdir = 'asc',
freeTextFilter,
} = req.query || {};
// normalize booleans (accept true, 'true', 1, '1' for true; false, 'false', 0, '0' for false)
const toBool = (v) => {
if (v === true || v === 'true' || v === 1 || v === '1') return true;
if (v === false || v === 'false' || v === 0 || v === '0') return false;
return null;
};
const normalizedActivity = toBool(activityFilter);
const normalizedWatch = toBool(watchListFilter);
let jobFilter = null;
let jobIdFilter = null;
const jobs = getJobs();
if (!nullOrEmpty(jobNameFilter)) {
const job = jobs.find((j) => j.id === jobNameFilter);
jobFilter = job != null ? job.name : null;
jobIdFilter = job != null ? job.id : null;
}
res.body = listingStorage.queryListings({
page: page ? parseInt(page, 10) : 1,
pageSize: pageSize ? parseInt(pageSize, 10) : 50,
freeTextFilter: freeTextFilter || null,
activityFilter: normalizedActivity,
jobNameFilter: jobFilter,
jobIdFilter: jobIdFilter,
providerFilter,
watchListFilter: normalizedWatch,
sortField: sortfield || null,
sortDir: sortdir === 'desc' ? 'desc' : 'asc',
userId: req.session.currentUser,
isAdmin: isAdminFn(req),
});
res.send();
});
listingsRouter.get('/map', async (req, res) => {
const { jobId } = req.query || {};
res.body = listingStorage.getListingsForMap({
jobId: nullOrEmpty(jobId) ? null : jobId,
userId: req.session.currentUser,
isAdmin: isAdminFn(req),
});
res.send();
});
listingsRouter.get('/:listingId', async (req, res) => {
const { listingId } = req.params;
const listing = listingStorage.getListingById(listingId, req.session.currentUser, isAdminFn(req));
if (!listing) {
res.statusCode = 404;
res.body = { message: 'Listing not found' };
return res.send();
}
res.body = listing;
res.send();
});
// Toggle watch state for the current user on a listing
listingsRouter.post('/watch', async (req, res) => {
try {
const { listingId } = req.body || {};
const userId = req.session?.currentUser;
if (!listingId || !userId) {
res.statusCode = 400;
res.body = { message: 'listingId or user not provided' };
return res.send();
}
watchListStorage.toggleWatch(listingId, userId);
} catch (error) {
logger.error(error);
res.statusCode = 500;
res.body = { message: 'Failed to toggle watch' };
}
res.send();
});
listingsRouter.delete('/job', async (req, res) => {
const { jobId, hardDelete = false } = req.body;
const settings = await getSettings();
try {
if (settings.demoMode) {
res.send(new Error('Sorry, but you cannot remove listings in demo mode ;)'));
return;
}
listingStorage.deleteListingsByJobId(jobId, hardDelete);
} catch (error) {
res.send(new Error(error));
logger.error(error);
}
res.send();
});
listingsRouter.delete('/', async (req, res) => {
const { ids, hardDelete = false } = req.body;
try {
if (Array.isArray(ids) && ids.length > 0) {
listingStorage.deleteListingsById(ids, hardDelete);
}
} catch (error) {
res.send(new Error(error));
logger.error(error);
}
res.send();
});
export { listingsRouter };

View File

@@ -1,6 +1,14 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import restana from 'restana';
import * as userStorage from '../../services/storage/userStorage.js';
import * as hasher from '../../services/security/hash.js';
import { trackDemoAccessed } from '../../services/tracking/Tracker.js';
import logger from '../../services/logger.js';
import { getSettings } from '../../services/storage/settingsStorage.js';
const service = restana();
const loginRouter = service.newRouter();
loginRouter.get('/user', async (req, res) => {
@@ -17,6 +25,7 @@ loginRouter.get('/user', async (req, res) => {
res.send();
});
loginRouter.post('/', async (req, res) => {
const settings = await getSettings();
const { username, password } = req.body;
const user = userStorage.getUsers(true).find((user) => user.username === username);
if (user == null) {
@@ -24,12 +33,16 @@ loginRouter.post('/', async (req, res) => {
return;
}
if (user.password === hasher.hash(password)) {
if (settings.demoMode) {
await trackDemoAccessed();
}
req.session.currentUser = user.id;
userStorage.setLastLoginToNow({ userId: user.id });
res.send(200);
return;
} else {
console.error(`User ${username} tried to login, but password was wrong.`);
logger.error(`User ${username} tried to login, but password was wrong.`);
}
res.send(401);
});

View File

@@ -1,12 +1,19 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import fs from 'fs';
import restana from 'restana';
import logger from '../../services/logger.js';
const service = restana();
const notificationAdapterRouter = service.newRouter();
const notificationAdapterList = fs.readdirSync('./lib//notification/adapter').filter((file) => file.endsWith('.js'));
const notificationAdapter = await Promise.all(
notificationAdapterList.map(async (pro) => {
return await import(`../../notification/adapter/${pro}`);
})
}),
);
notificationAdapterRouter.post('/try', async (req, res) => {
const { id, fields } = req.body;
@@ -29,11 +36,14 @@ notificationAdapterRouter.post('/try', async (req, res) => {
serviceName: 'TestCall',
newListings: [
{
price: '42 €',
title: 'This is a test listing',
address: 'some address',
size: '666 2m',
link: 'https://www.orange-coding.net',
address: 'Heidestrasse 17, 51147 Köln',
description: exampleDescription,
id: '1',
imageUrl: 'https://placehold.co/600x400/png',
price: '1.000 €',
size: '76 m²',
title: 'Stilvolle gepflegte 3-Raum-Wohnung mit gehobener Innenausstattung',
url: 'https://www.orange-coding.net',
},
],
notificationConfig,
@@ -41,6 +51,7 @@ notificationAdapterRouter.post('/try', async (req, res) => {
});
res.send();
} catch (Exception) {
logger.error('Error during notification adapter test:', Exception);
res.send(new Error(Exception));
}
});
@@ -49,3 +60,51 @@ notificationAdapterRouter.get('/', async (req, res) => {
res.send();
});
export { notificationAdapterRouter };
const exampleDescription = `
Wohnungstyp: Etagenwohnung
Nutzfläche: 76 m²
Etage: 2 von 3
Schlafzimmer: 1
Badezimmer: 1
Bezugsfrei ab: 1.4.2026
Haustiere: Nein
Garage/Stellplatz: Tiefgarage
Anzahl Garage/Stellplatz: 1
Kaltmiete (zzgl. Nebenkosten): 1.000 €
Preis/m²: 13,16 €/m²
Nebenkosten: 230 €
Heizkosten in Nebenkosten enthalten: Ja
Gesamtmiete: 1.230 €
Kaution: 3.000,00
Preis pro Parkfläche: 60 €
Baujahr: 2000
Objektzustand: Modernisiert
Qualität der Ausstattung: Gehoben
Heizungsart: Fernwärme
Energieausweistyp: Verbrauchsausweis
Energieausweis: liegt vor
Endenergieverbrauch: 72 kWh/(m²∙a)
Baujahr laut Energieausweis: 2000
Diese moderne 3-Zimmer-Wohnung liegt direkt neben einem Park und nur wenige Minuten von der S-Bahn-Haltestelle entfernt. Das Stadtzentrum sowie Freizeiteinrichtungen sind 1,5 km entfernt.
Die Wohnung ist ideal für Paare oder kleine Familien geeignet.
Ausstattung:
- neuer hochwertiger Bodenbelag (Holzoptik) in allen Räumen außer Bad/Küche
- sonniger Balkon (Süd)
- Tiefgaragenstellplatz
- Kellerabteil
- gepflegtes Mehrfamilienhaus
Die Küche ist vom Mieter nach eigenen Wünschen einzurichten.
Vermietung direkt vom Eigentümer - provisionsfrei!
Lage:
• Park: 1 Minute zu Fuß
• S-Bahn Station: 2 Minuten zu Fuß
• Supermärkte, Restaurants, täglicher Bedarf in der Nähe
• Gute Anbindung Richtung Großstadt und Flughafen
`;

View File

@@ -1,3 +1,8 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import fs from 'fs';
import restana from 'restana';
const service = restana();
@@ -6,7 +11,7 @@ const providerList = fs.readdirSync('./lib/provider').filter((file) => file.ends
const provider = await Promise.all(
providerList.map(async (pro) => {
return await import(`../../provider/${pro}`);
})
}),
);
providerRouter.get('/', async (req, res) => {
res.body = provider.map((p) => p.metaInformation);

View File

@@ -0,0 +1,37 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import restana from 'restana';
import { trackPoi } from '../../services/tracking/Tracker.js';
import { TRACKING_POIS } from '../../TRACKING_POIS.js';
import logger from '../../services/logger.js';
const service = restana();
const trackingRouter = service.newRouter();
trackingRouter.get('/trackingPois', async (req, res) => {
res.body = TRACKING_POIS;
res.send();
});
trackingRouter.post('/poi', async (req, res) => {
const { poi } = req.body;
if (!poi) {
res.statusCode = 400;
res.send({ error: 'Feature name is required' });
return;
}
try {
await trackPoi(poi);
res.send({ success: true });
} catch (error) {
logger.error('Error tracking feature', error);
res.statusCode = 500;
res.send({ error: error.message });
}
});
export { trackingRouter };

View File

@@ -1,6 +1,12 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import restana from 'restana';
import * as userStorage from '../../services/storage/userStorage.js';
import * as jobStorage from '../../services/storage/jobStorage.js';
import { getSettings } from '../../services/storage/settingsStorage.js';
const service = restana();
const userRouter = service.newRouter();
function checkIfAnyAdminAfterRemovingUser(userIdToBeRemoved, allUser) {
@@ -10,16 +16,24 @@ function checkIfUserToBeRemovedIsLoggedIn(userIdToBeRemoved, req) {
return req.session.currentUser === userIdToBeRemoved;
}
const nullOrEmpty = (str) => str == null || str.length === 0;
userRouter.get('/', async (req, res) => {
res.body = userStorage.getUsers(false);
res.send();
});
userRouter.get('/:userId', async (req, res) => {
const { userId } = req.params;
res.body = userStorage.getUser(userId);
res.send();
});
userRouter.delete('/', async (req, res) => {
const settings = await getSettings();
if (settings.demoMode) {
res.send(new Error('In demo mode, it is not allowed to remove user.'));
return;
}
const { userId } = req.body;
const allUser = userStorage.getUsers(false);
if (!checkIfAnyAdminAfterRemovingUser(userId, allUser)) {
@@ -36,6 +50,12 @@ userRouter.delete('/', async (req, res) => {
res.send();
});
userRouter.post('/', async (req, res) => {
const settings = await getSettings();
if (settings.demoMode) {
res.send(new Error('In demo mode, it is not allowed to change or add user.'));
return;
}
const { username, password, password2, isAdmin, userId } = req.body;
if (password !== password2) {
res.send(new Error('Passwords does not match'));
@@ -48,7 +68,7 @@ userRouter.post('/', async (req, res) => {
const allUser = userStorage.getUsers(false);
if (!isAdmin && !checkIfAnyAdminAfterRemovingUser(userId, allUser)) {
res.send(
new Error('You cannot change the admin flag for this user as otherwise, there is no other user in the system')
new Error('You cannot change the admin flag for this user as otherwise, there is no other user in the system'),
);
return;
}

View File

@@ -0,0 +1,121 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import restana from 'restana';
import SqliteConnection from '../../services/storage/SqliteConnection.js';
import { getSettings, upsertSettings } from '../../services/storage/settingsStorage.js';
import { resetGeocoordinatesAndDistanceForUser } from '../../services/storage/listingsStorage.js';
import { geocodeAddress } from '../../services/geocoding/geoCodingService.js';
import { autocompleteAddress } from '../../services/geocoding/autocompleteService.js';
import { fromJson } from '../../utils.js';
import { trackPoi } from '../../services/tracking/Tracker.js';
import { TRACKING_POIS } from '../../TRACKING_POIS.js';
import logger from '../../services/logger.js';
import { runGeoCordTask } from '../../services/crons/geocoding-cron.js';
const service = restana();
const userSettingsRouter = service.newRouter();
userSettingsRouter.get('/', async (req, res) => {
const userId = req.session.currentUser;
const rows = SqliteConnection.query('SELECT name, value FROM settings WHERE user_id = @userId', { userId });
const settings = {};
for (const r of rows) {
settings[r.name] = fromJson(r.value, null);
}
res.body = settings;
res.send();
});
userSettingsRouter.get('/autocomplete', async (req, res) => {
const { q } = req.query;
try {
const results = await autocompleteAddress(q);
res.body = results;
res.send();
} catch (error) {
res.statusCode = 500;
res.send({ error: error.message });
}
});
userSettingsRouter.post('/home-address', async (req, res) => {
const userId = req.session.currentUser;
const { home_address } = req.body;
const settings = await getSettings();
if (settings.demoMode) {
res.send(new Error('In demo mode, it is not allowed to change the home address.'));
return;
}
try {
if (home_address) {
await trackPoi(TRACKING_POIS.DISTANCE_ADDRESS_ENTERED);
const coords = await geocodeAddress(home_address);
if (coords && coords.lat !== -1) {
upsertSettings({ home_address: { address: home_address, coords } }, userId);
resetGeocoordinatesAndDistanceForUser(userId);
//we do NOT wait for this to finish, as we don't want to block the response
runGeoCordTask();
res.send({ success: true, coords });
} else {
res.statusCode = 400;
res.send({ error: 'Could not geocode address' });
}
} else {
upsertSettings({ home_address: null }, userId);
res.send({ success: true });
}
} catch (error) {
logger.error('Error updating home address settings', error);
res.statusCode = 500;
res.send({ error: error.message });
}
});
userSettingsRouter.post('/news-hash', async (req, res) => {
const userId = req.session.currentUser;
const { news_hash } = req.body;
const globalSettings = await getSettings();
if (globalSettings.demoMode) {
res.statusCode = 403;
res.send({ error: 'In demo mode, it is not allowed to change settings.' });
return;
}
try {
upsertSettings({ news_hash }, userId);
res.send({ success: true });
} catch (error) {
logger.error('Error updating news hash', error);
res.statusCode = 500;
res.send({ error: error.message });
}
});
userSettingsRouter.post('/immoscout-details', async (req, res) => {
const userId = req.session.currentUser;
const { immoscout_details } = req.body;
const globalSettings = await getSettings();
if (globalSettings.demoMode) {
res.statusCode = 403;
res.send({ error: 'In demo mode, it is not allowed to change settings.' });
return;
}
try {
upsertSettings({ immoscout_details: !!immoscout_details }, userId);
res.send({ success: true });
} catch (error) {
logger.error('Error updating immoscout details setting', error);
res.statusCode = 500;
res.send({ error: error.message });
}
});
export { userSettingsRouter };

View File

@@ -0,0 +1,43 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import restana from 'restana';
import fetch from 'node-fetch';
import { getPackageVersion } from '../../utils.js';
import semver from 'semver';
const service = restana();
const versionRouter = service.newRouter();
versionRouter.get('/', async (req, res) => {
const versionPayload = await getCurrentVersionFromGithub();
const localFredyVersion = await getPackageVersion();
res.body =
versionPayload == null
? {
newVersion: false,
localFredyVersion,
}
: versionPayload;
res.send();
});
async function getCurrentVersionFromGithub() {
const raw = await fetch('https://api.github.com/repos/orangecoding/fredy/releases/latest');
const data = await raw.json();
const localFredyVersion = await getPackageVersion();
if (data.tag_name == null || semver.gte(localFredyVersion, data.tag_name)) {
return null;
}
return {
newVersion: true,
version: data.tag_name,
url: data.html_url,
body: data.body,
localFredyVersion,
};
}
export { versionRouter };

View File

@@ -1,3 +1,8 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import * as userStorage from '../services/storage/userStorage.js';
import cookieSession from 'cookie-session';
import { nanoid } from 'nanoid';
@@ -37,7 +42,7 @@ const cookieSession$0 = (userId) => {
name: 'fredy-admin-session',
keys: ['fredy', 'super', 'fancy', 'key', nanoid()],
userId,
maxAge: 8 * 60 * 60 * 1000, // 8 hours
maxAge: 2 * 60 * 60 * 1000, // 2 hours
});
};
export { cookieSession$0 as cookieSession };

9
lib/defaultConfig.js Normal file
View File

@@ -0,0 +1,9 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
export const DEFAULT_CONFIG = {
// Default path for sqlite storage directory. Interpreted relative to project root.
sqlitepath: '/db',
};

View File

@@ -1,3 +1,8 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
class ExtendableError extends Error {
constructor(message) {
super(message);

323
lib/mcp/README.md Normal file
View File

@@ -0,0 +1,323 @@
# Fredy MCP Server
The Fredy MCP Server exposes your real estate jobs and listings data to LLM clients. It supports two transports:
- **Stdio**: for local LLM clients (Claude Desktop, LM Studio, llm-cli, mcp-cli, etc.)
- **Streamable HTTP**: for remote LLM clients (ChatGPT, cloud-hosted agents, etc.)
## Authentication
All MCP access is **token-based** based. Every Fredy user is automatically assigned a **permanent, non-expiring MCP token** when their account is created. This token is a secret and should be treated like a password.
### Where to find your token
MCP tokens are displayed in the **User Management** list (Admin → Users). Each user's token is shown in the **"MCP Token"** column.
> **Important:** MCP tokens never expire. They are permanent secrets tied to each user account. If a token is compromised, you must change the token! If you chose to use a token from an admin account, the LLM can query information from ALL jobs/listings.
## Available Tools
| Tool | Description |
|------|--------------------------------------------------------------------------------|
| `list_jobs` | List real estate search jobs with pagination and text filtering |
| `get_job` | Get detailed information about a specific job |
| `list_listings` | Search and list real estate listings with pagination, text search, and filters |
| `get_listing` | Get full details of a single listing |
| `get_current_date_time` | Gets the current date/time for the llm to be used |
### Tool Details
#### list_jobs
- `page` (number, optional) Page number (default: 1)
- `pageSize` (number, optional) Results per page (default: 50, max: 1000). Use pagination to fetch more.
- `filter` (string, optional) Free-text filter on job name
Response: markdown table with columns ID, Name, Enabled, Active Listings. Includes summary and pagination info.
#### get_job
- `jobId` (string, required) The job ID to retrieve
#### list_listings
- `page` (number, optional) Page number (default: 1)
- `pageSize` (number, optional) Results per page (default: 50, max: 1000). Use pagination to fetch more.
- `filter` (string, optional) Free-text search across title, address, provider, link
- `jobId` (string, optional) Filter listings by job ID
- `activeOnly` (boolean, optional) When true, only show active listings
- `provider` (string, optional) Filter by provider name
- `createdAfter` (number, optional) Only include listings created at or after this unix timestamp in milliseconds (e.g. `1772008362564`). Useful for queries like "give me all listings from today".
- `createdBefore` (number, optional) Only include listings created at or before this unix timestamp in milliseconds (e.g. `1772008362564`).
- `minPrice` (number, optional) Only include listings with price >= this value (e.g. `500`). Numeric, no currency symbol.
- `maxPrice` (number, optional) Only include listings with price <= this value (e.g. `1500`). Numeric, no currency symbol.
- `sortField` (string, optional) Sort by: created_at, price, size, provider, title, is_active
- `sortDir` (string, optional) Sort direction: asc or desc
Response: markdown table with columns ID, Title, Address, Price, Size, Provider, Active, Created, Job. Includes summary and pagination info. Use `get_listing` for full details.
> **Note:** All timestamps are **unix timestamps in milliseconds** (e.g. `1772008362564`), not seconds.
#### get_listing
- `listingId` (string, required) The listing ID to retrieve
## Usage with Local LLM (stdio transport)
The stdio transport communicates over stdin/stdout and is ideal for local LLM tools.
### Quick Start
```bash
MCP_TOKEN=fredy_<your-token> node mcp/stdio.js
# or
MCP_TOKEN=fredy_<your-token> yarn mcp:stdio
```
### Testing with MCP Inspector
The [MCP Inspector](https://github.com/modelcontextprotocol/inspector) lets you interactively test your MCP server in a browser UI.
```bash
npx @modelcontextprotocol/inspector -e MCP_TOKEN=fredy_<your-token> -- node mcp/stdio.js
```
Once the inspector is running, open the URL shown in your terminal (usually `http://localhost:6274`). You can then:
1. Click **Connect** to establish the stdio connection
2. Go to the **Tools** tab to see all available tools
3. Select a tool, fill in parameters, and click **Run** to test it
### LM Studio Configuration
[LM Studio](https://lmstudio.ai/) supports MCP servers natively, allowing your local LLM to access Fredy's jobs and listings data.
#### Setup
1. Open **LM Studio** and load a model that supports tool use (e.g., Qwen 2.5, Llama 3.1, Mistral, etc.)
2. In the right side under **Integrations** click on "# install" and "edit mcp.json"
3. Edit the LM Studio MCP config file directly (`~/.lmstudio/config/mcp.json` or via the UI export):
```json
{
"mcpServers": {
"fredy": {
"command": "node",
"args": ["/absolute/path/to/fredy/mcp/stdio.js"],
"env": {
"MCP_TOKEN": "fredy_<your-token>"
}
}
}
}
```
4. Toggle the server **on**: LM Studio will spawn the stdio process and connect
5. You should see the Fredy tools appear as available tools
#### Suggestion on LLM
After testing numerous LLM's, I got the best results with Qwen 3.5 or Qwen 2.5.. E.g. `Qwen2.5-14B-Instruct-1M-8bit`.
#### Usage
Once connected, simply ask your LLM about your real estate data in natural language:
- *"Show me all my active search jobs"*
- *"List the latest listings from my Berlin apartment search"*
- *"Get details for listing XYZ"*
- *"What are the cheapest listings across all my jobs?"*
The LLM will automatically call the appropriate Fredy MCP tools and present the results.
> **Tip:** Make sure Fredy is running and the database is accessible before starting the MCP server in LM Studio. The stdio transport initializes its own database connection, so Fredy's main process does not need to be running, but the database file must exist and be up-to-date (migrations applied).
### Claude Desktop Configuration
[Claude Desktop](https://claude.ai/download) supports MCP servers natively via its developer settings.
#### Setup
1. Open **Claude Desktop**
2. Go to **Settings → Developer → Edit Config** — this opens the `claude_desktop_config.json` file
3. Add the `fredy` server to the `mcpServers` object:
```json
{
"mcpServers": {
"fredy": {
"command": "/opt/homebrew/opt/node@22/bin/node",
"args": ["/absolute/path/to/fredy/lib/mcp/stdio.js"],
"env": {
"MCP_TOKEN": "fredy_<your-token>"
}
}
}
}
```
Replace `/absolute/path/to/fredy` with the actual path on your machine (e.g. `/Users/you/dev/fredy`).
> **Important:** Claude Desktop launches with a restricted `PATH` and often cannot find `node` by name. Always use the **full absolute path** to the node binary. Find yours by running `which node` in a terminal. Common locations:
> - Homebrew (default): `/opt/homebrew/bin/node`
> - Homebrew (versioned, e.g. node@22): `/opt/homebrew/opt/node@22/bin/node`
> - nvm: `/Users/<you>/.nvm/versions/node/<version>/bin/node`
4. Save the file and **restart Claude Desktop**
5. You should see a hammer icon (🔨) in the chat input — click it to confirm the Fredy tools are listed
#### Usage
Once connected, simply ask Claude about your real estate data:
- *"Show me all my active search jobs"*
- *"List the latest listings from my Berlin apartment search"*
- *"What are the cheapest apartments added this week?"*
Claude will automatically call the appropriate Fredy MCP tools.
> **Note:** Fredy's main web process does not need to be running — the stdio transport opens its own database connection directly. But the SQLite database file must exist and migrations must have been applied.
---
## Usage with Remote LLM (Streamable HTTP transport)
The HTTP transport is automatically available when Fredy is running. It uses the MCP Streamable HTTP protocol at:
```
POST /api/mcp JSON-RPC messages (initialize, tool calls)
GET /api/mcp SSE stream for server-initiated notifications
DELETE /api/mcp Terminate session
```
### Authentication
All requests must include the token as a Bearer token:
```
Authorization: Bearer fredy_<your-token>
```
### Example: Initialize a session
```bash
curl -X POST http://localhost:9998/api/mcp \
-H "Content-Type: application/json" \
-H "Authorization: Bearer fredy_<your-token>" \
-d '{
"jsonrpc": "2.0",
"id": 1,
"method": "initialize",
"params": {
"protocolVersion": "2025-03-26",
"capabilities": {},
"clientInfo": { "name": "test-client", "version": "1.0.0" }
}
}'
```
### Example: Call a tool
```bash
curl -X POST http://localhost:9998/api/mcp \
-H "Content-Type: application/json" \
-H "Authorization: Bearer fredy_<your-token>" \
-H "Mcp-Session-Id: <session-id-from-init-response>" \
-d '{
"jsonrpc": "2.0",
"id": 2,
"method": "tools/call",
"params": {
"name": "list_jobs",
"arguments": { "page": 1, "pageSize": 10 }
}
}'
```
## Security
- Every user is automatically assigned a permanent MCP token at account creation **tokens never expire**
- Tokens are cryptographically random (256-bit) and prefixed with `fredy_`
- Each token is scoped to a single user the LLM can only access that user's data
- Non-admin users only see their own jobs and jobs shared with them
- Tokens are stored in the `mcp_token` column of the `users` table
- Tokens are deleted automatically when the owning user is removed
- The `/api/mcp` endpoint uses Bearer token auth (independent of cookie-session)
- Treat MCP tokens like passwords do not share them publicly
## Response Format
All tool responses use **markdown** instead of JSON for maximum LLM readability and token efficiency:
- **List responses** (list_jobs, list_listings) use markdown tables with a summary line and pagination footer
- **Detail responses** (get_job, get_listing) use markdown key-value lists
- **Error responses** include the tool name and error message
Example list response:
```
**Tool:** list_listings | **Status:** OK
Found **85** listing(s). Showing page 1 of 2 (50 on this page). More pages available — use page=2 to continue.
| ID | Title | Address | Price | Size | Provider | Active | Created | Job |
|----|-------|---------|-------|------|----------|--------|---------|-----|
| abc123 | Nice flat | Berlin | 1200 | 70 | immoscout | yes | 2026-02-25 10:30:00 | My Search |
| ... | ... | ... | ... | ... | ... | ... | ... | ... |
Use **get_listing** with an ID for full details (description, link, image).
**Page:** 1/2 | **Has more:** yes
```
Example detail response:
```
**Tool:** get_listing | **Status:** OK
### Listing: Nice flat
- **ID:** abc123
- **Title:** Nice flat
- **Address:** Berlin
- **Price:** 1200
- **Size:** 70
- **Provider:** immoscout
- **Link:** https://...
- **Active:** yes
- **Created:** 2026-02-25 10:30:00
```
Markdown is used because it is significantly more token-efficient than JSON (~40-60% fewer tokens for tabular data) and natively understood by all LLMs.
## Architecture
```
┌─────────────────┐ stdio ┌──────────────┐
│ Local LLM │◄──────────────►│ mcp/stdio.js│
│ (LM Studio, │ │ (transport) │
│ Claude, etc.) │ │ │
└─────────────────┘ └──────┬───────┘
┌─────────────────┐ Streamable HTTP ┌────┴────────┐
│ Remote LLM │◄───────────────►│ /api/mcp │
│ │ (Bearer token) │ (transport) │
└─────────────────┘ └──────┬───────┘
┌──────────┴──────────┐
│ mcpAuthentication │
│ (token validation, │
│ access control) │
└──────────┬──────────┘
┌────────┴────────┐
│ mcpAdapter.js │
│ (tool routing │
│ + data fetch) │
└────────┬────────┘
┌────────┴────────┐
│ mcpNormalizer.js│
│ (markdown │
│ formatting) │
└────────┬────────┘
┌──────┴───────┐
│ Fredy DB │
│ (SQLite) │
└──────────────┘
```

231
lib/mcp/mcpAdapter.js Normal file
View File

@@ -0,0 +1,231 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
import { z } from 'zod';
import { queryJobs, getJob } from '../services/storage/jobStorage.js';
import { queryListings, getListingById } from '../services/storage/listingsStorage.js';
import { authenticateToolCall, checkJobAccess } from './mcpAuthentication.js';
import {
normalizeListJobs,
normalizeGetJob,
normalizeListListings,
normalizeGetListing,
normalizeError,
} from './mcpNormalizer.js';
/**
* Create a configured MCP server instance with all Fredy tools registered.
*
* The adapter fetches raw data from storage and delegates response formatting
* to the normalizer layer (mcpNormalizer.js) which produces a consistent
* { ok, summary, data, meta } envelope for every tool response.
*
* Each tool call requires a userId (resolved from the MCP token before invocation).
* Tools respect user scoping: non-admin users only see their own jobs/listings.
*
* @returns {McpServer}
*/
export function createMcpServer() {
const server = new McpServer(
{
name: 'fredy-mcp',
version: '1.0.0',
},
{
capabilities: {
tools: {},
},
instructions:
'Fredy MCP Server query real estate jobs and listings. ' +
'All timestamps are unix timestamps in milliseconds (e.g. 1772008362564). ' +
'Use list_jobs to browse jobs, get_job for details, ' +
'list_listings to search listings (supports time filters like createdAfter/createdBefore), ' +
'and get_listing for full details of a single listing. ' +
'Responses are formatted as markdown with a summary, data (tables for lists, key-value for details), and pagination info. ' +
'Always present results to the user as soon as you have them — do NOT call the tool again unless you need additional pages or different data.',
},
);
// ── list_jobs ───────────────────────────────────────────────────────
server.tool(
'list_jobs',
'List real estate search jobs for the authenticated user. ' +
'Returns up to 50 jobs per page by default. Use pagination (page parameter) to fetch more. ' +
'Check meta.hasMore to know if there are additional pages.',
{
page: z.number().optional().describe('Page number (default: 1)'),
pageSize: z
.number()
.optional()
.describe('Results per page (default: 50, max: 1000). Start with the default and paginate if needed.'),
filter: z.string().optional().describe('Free-text filter on job name'),
},
async ({ page, pageSize, filter }, extra) => {
const { user, error } = authenticateToolCall(extra, 'list_jobs');
if (error) return normalizeError(error, 'list_jobs');
const safePage = page ?? 1;
const safePageSize = pageSize ?? 50;
const result = queryJobs({
page: safePage,
pageSize: safePageSize,
freeTextFilter: filter,
userId: user.id,
isAdmin: user.isAdmin,
});
return normalizeListJobs(result, { page: safePage, pageSize: safePageSize });
},
);
// ── get_job ─────────────────────────────────────────────────────────
server.tool(
'get_job',
'Get detailed information about a specific job by its ID.',
{
jobId: z.string().describe('The job ID to retrieve'),
},
async ({ jobId }, extra) => {
const { user, error } = authenticateToolCall(extra, 'get_job');
if (error) return normalizeError(error, 'get_job');
const job = getJob(jobId);
if (!job) {
return normalizeError('Job not found.', 'get_job');
}
if (!checkJobAccess(user, job)) {
return normalizeError('Access denied.', 'get_job');
}
return normalizeGetJob(job);
},
);
// ── list_listings ───────────────────────────────────────────────────
server.tool(
'list_listings',
'Search and list real estate listings. Returns up to 50 listings per page by default. ' +
'Use pagination (page parameter) to fetch more. Check meta.hasMore in the response. ' +
'Supports text search, time filtering, and various filters. ' +
'All timestamps are unix timestamps in milliseconds (e.g. 1772008362564). ' +
'Use createdAfter/createdBefore to filter by time, e.g. "give me all listings from today". ' +
'Use get_listing to get full details (description, link, image) for a specific listing.',
{
page: z.number().optional().describe('Page number (default: 1)'),
pageSize: z
.number()
.optional()
.describe('Results per page (default: 50, max: 1000). Start with the default and paginate if needed.'),
filter: z.string().optional().describe('Free-text search across title, address, provider, link'),
jobId: z.string().optional().describe('Filter listings by job ID'),
activeOnly: z.boolean().optional().describe('When true, only show active listings'),
provider: z.string().optional().describe('Filter by provider name'),
createdAfter: z
.number()
.optional()
.describe(
'Only include listings created at or after this unix timestamp in milliseconds (e.g. 1772008362564). Useful for queries like "listings from today".',
),
createdBefore: z
.number()
.optional()
.describe(
'Only include listings created at or before this unix timestamp in milliseconds (e.g. 1772008362564).',
),
minPrice: z
.number()
.optional()
.describe(
'Only include listings with price >= this value (e.g. 500). Price is a numeric value (no currency symbol).',
),
maxPrice: z
.number()
.optional()
.describe(
'Only include listings with price <= this value (e.g. 1500). Price is a numeric value (no currency symbol).',
),
sortField: z.string().optional().describe('Sort by: created_at, price, size, provider, title, is_active'),
sortDir: z.string().optional().describe('Sort direction: asc or desc'),
},
async (
{
page,
pageSize,
filter,
jobId,
activeOnly,
provider,
createdAfter,
createdBefore,
minPrice,
maxPrice,
sortField,
sortDir,
},
extra,
) => {
const { user, error } = authenticateToolCall(extra, 'list_listings');
if (error) return normalizeError(error, 'list_listings');
const safePage = page ?? 1;
const safePageSize = pageSize ?? 50;
const result = queryListings({
page: safePage,
pageSize: safePageSize,
freeTextFilter: filter,
jobIdFilter: jobId,
activityFilter: activeOnly === true ? true : activeOnly === false ? false : undefined,
providerFilter: provider,
createdAfter: createdAfter ?? null,
createdBefore: createdBefore ?? null,
minPrice: minPrice ?? null,
maxPrice: maxPrice ?? null,
sortField: sortField ?? null,
sortDir: sortDir ?? 'desc',
userId: user.id,
isAdmin: user.isAdmin,
});
return normalizeListListings(result, { page: safePage, pageSize: safePageSize });
},
);
// ── get_listing ─────────────────────────────────────────────────────
server.tool(
'get_listing',
'Get full details of a single listing by its ID.',
{
listingId: z.string().describe('The listing ID to retrieve'),
},
async ({ listingId }, extra) => {
const { user, error } = authenticateToolCall(extra, 'get_listing');
if (error) return normalizeError(error, 'get_listing');
const listing = getListingById(listingId, user.id, user.isAdmin);
if (!listing) {
return normalizeError('Listing not found or access denied.', 'get_listing');
}
return normalizeGetListing(listing);
},
);
// ── get_current_date_ime ─────────────────────────────────────────────────────
server.tool('get_current_date_time', 'Returns the current date and time.', {}, () => {
return {
content: [{ type: 'text', text: `Timestring: ${new Date().toLocaleString()}, MS since 1970: ${Date.now()}` }],
};
});
return server;
}

View File

@@ -0,0 +1,66 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
/**
* MCP Authentication Layer
*
* Centralizes all authentication and authorization logic for MCP tool calls
* and HTTP requests. Ensures consistent access control across all transports.
*/
import { getUser, validateMcpToken } from '../services/storage/userStorage.js';
/**
* Authenticate an MCP tool call by extracting and validating the user from authInfo.
*
* @param {{ authInfo?: { userId?: string } }} extra - The extra context passed by the MCP SDK.
* @returns {{ user: object|null, error: string|null }}
* - On success: { user: <userObject>, error: null }
* - On failure: { user: null, error: <errorMessage> }
*/
export function authenticateToolCall(extra) {
const userId = extra?.authInfo?.userId;
if (!userId) {
return { user: null, error: 'Authentication required. Please provide a valid MCP API token.' };
}
const user = getUser(userId);
if (!user) {
return { user: null, error: 'Authentication required. Please provide a valid MCP API token.' };
}
return { user, error: null };
}
/**
* Check whether a user has access to a specific job.
* Admins have access to all jobs. Non-admins can only access their own jobs
* or jobs explicitly shared with them.
*
* @param {object} user - The authenticated user object.
* @param {object} job - The job object from storage.
* @returns {boolean} True if the user is allowed to access this job.
*/
export function checkJobAccess(user, job) {
if (user.isAdmin) return true;
if (job.userId === user.id) return true;
if (Array.isArray(job.shared_with_user) && job.shared_with_user.includes(user.id)) return true;
return false;
}
/**
* Authenticate an HTTP request by extracting and validating the Bearer token
* from the Authorization header.
*
* @param {import('http').IncomingMessage} req
* @returns {{ userId: string } | null} The authenticated user info, or null if invalid.
*/
export function authenticateRequest(req) {
const authHeader = req.headers['authorization'] || '';
if (!authHeader.startsWith('Bearer ')) return null;
const token = authHeader.slice(7).trim();
if (!token) return null;
return validateMcpToken(token);
}

131
lib/mcp/mcpHttpRoute.js Normal file
View File

@@ -0,0 +1,131 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import { StreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/streamableHttp.js';
import { createMcpServer } from './mcpAdapter.js';
import { authenticateRequest } from './mcpAuthentication.js';
import logger from '../services/logger.js';
import crypto from 'crypto';
/**
* Active transports keyed by session id.
* Each session gets its own McpServer + StreamableHTTPServerTransport pair.
* @type {Map<string, { server: McpServer, transport: StreamableHTTPServerTransport }>}
*/
const sessions = new Map();
/**
* Get or create a session for the given session id with authentication.
* @param {string|undefined} sessionId
* @param {{ userId: string }} auth
* @returns {{ server: McpServer, transport: StreamableHTTPServerTransport }}
*/
function getOrCreateSession(sessionId, auth) {
if (sessionId && sessions.has(sessionId)) {
return sessions.get(sessionId);
}
const transport = new StreamableHTTPServerTransport({
sessionIdGenerator: () => crypto.randomUUID(),
onsessioninitialized: (sid) => {
sessions.set(sid, entry);
logger.debug(`MCP session created: ${sid}`);
},
});
const server = createMcpServer();
const entry = { server, transport, userId: auth.userId };
transport.onclose = () => {
const sid = transport.sessionId;
if (sid) {
sessions.delete(sid);
logger.debug(`MCP session closed: ${sid}`);
}
};
return entry;
}
/**
* Register MCP Streamable HTTP routes on a restana service.
*
* Mounts handlers at /api/mcp to handle the MCP Streamable HTTP protocol:
* - POST /api/mcp JSON-RPC messages (initialize, tool calls, etc.)
* - GET /api/mcp SSE stream for server-initiated notifications
* - DELETE /api/mcp session termination
*
* All endpoints require a valid Bearer token in the Authorization header.
*
* @param {import('restana').Service} service - The restana service instance.
*/
export function registerMcpRoutes(service) {
// POST main JSON-RPC endpoint
service.post('/api/mcp', async (req, res) => {
const auth = authenticateRequest(req);
if (!auth) {
res.statusCode = 401;
return res.send({ error: 'Unauthorized. Provide a valid Bearer token.' });
}
const sessionId = req.headers['mcp-session-id'];
const { server, transport } = getOrCreateSession(sessionId, auth);
// Connect server to transport if not already connected
if (!transport.onmessage) {
await server.connect(transport);
}
// Inject authInfo so tools can access the authenticated user
req.auth = { userId: auth.userId };
await transport.handleRequest(req, res, req.body);
});
// GET SSE stream for server-initiated messages
service.get('/api/mcp', async (req, res) => {
const auth = authenticateRequest(req);
if (!auth) {
res.statusCode = 401;
return res.send({ error: 'Unauthorized. Provide a valid Bearer token.' });
}
const sessionId = req.headers['mcp-session-id'];
if (!sessionId || !sessions.has(sessionId)) {
res.statusCode = 400;
return res.send({ error: 'Invalid or missing session. Send an initialize request first.' });
}
const { transport } = sessions.get(sessionId);
await transport.handleRequest(req, res);
});
// DELETE terminate session
service.delete('/api/mcp', async (req, res) => {
const auth = authenticateRequest(req);
if (!auth) {
res.statusCode = 401;
return res.send({ error: 'Unauthorized. Provide a valid Bearer token.' });
}
const sessionId = req.headers['mcp-session-id'];
if (!sessionId || !sessions.has(sessionId)) {
res.statusCode = 404;
return res.send({ error: 'Session not found.' });
}
const { transport } = sessions.get(sessionId);
await transport.close();
sessions.delete(sessionId);
res.statusCode = 200;
res.send({ ok: true });
});
logger.debug('MCP Streamable HTTP endpoint registered at /api/mcp');
}

180
lib/mcp/mcpNormalizer.js Normal file
View File

@@ -0,0 +1,180 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
/**
* MCP Response Normalizer
*
* Transforms raw adapter data into LLM-friendly markdown responses.
* Markdown is significantly better than JSON for LLM consumption because:
* - LLMs are trained extensively on markdown text
* - Markdown tables are ~40-60% more token-efficient than JSON arrays
* - Less syntactic noise (no quotes, brackets, commas around every value)
* - Natively readable and structured
*
* Each response follows a consistent structure:
* 1. Status line (OK/ERROR + tool name)
* 2. Summary (human-readable description)
* 3. Data (markdown table for lists, key-value for single items)
* 4. Pagination info (for list responses)
*/
/**
* Wrap a markdown string as an MCP text content result.
* @param {string} markdown
* @param {boolean} [isError=false]
* @returns {{ content: Array, isError?: boolean }}
*/
function toMcpResponse(markdown, isError = false) {
const result = {
content: [{ type: 'text', text: markdown }],
};
if (isError) result.isError = true;
return result;
}
/**
* Format a unix timestamp (ms) as a human-readable date string.
* @param {number|null|undefined} ts
* @returns {string}
*/
function formatDate(ts) {
if (ts == null) return '';
return new Date(ts)
.toISOString()
.replace('T', ' ')
.replace(/\.\d{3}Z$/, '');
}
/**
* Escape pipe characters in table cell values.
* @param {*} val
* @returns {string}
*/
function cell(val) {
if (val == null) return '';
return String(val).replace(/\|/g, '\\|').replace(/\n/g, ' ');
}
/**
* Normalize a list_jobs response.
* @param {{ totalNumber: number, page: number, result: object[] }} queryResult
* @param {{ page: number, pageSize: number }} params
* @returns {{ content: Array }}
*/
export function normalizeListJobs(queryResult, { page, pageSize }) {
const maxPage = Math.max(1, Math.ceil(queryResult.totalNumber / pageSize));
const hasMore = page < maxPage;
const jobs = queryResult.result;
let md = `**Tool:** list_jobs | **Status:** OK\n\n`;
md += `Found **${queryResult.totalNumber}** job(s). Showing page ${page} of ${maxPage} (${jobs.length} on this page).`;
if (hasMore) md += ` More pages available — use page=${page + 1} to continue.`;
md += '\n\n';
if (jobs.length > 0) {
md += `| ID | Name | Enabled | Active Listings |\n`;
md += `|----|------|---------|----------------|\n`;
for (const j of jobs) {
md += `| ${cell(j.id)} | ${cell(j.name)} | ${j.enabled ? 'yes' : 'no'} | ${j.numberOfFoundListings ?? 0} |\n`;
}
} else {
md += `No jobs found.\n`;
}
md += `\n**Page:** ${page}/${maxPage} | **Has more:** ${hasMore ? 'yes' : 'no'}`;
return toMcpResponse(md);
}
/**
* Normalize a get_job response.
* @param {object} job - The job object from storage.
* @returns {{ content: Array }}
*/
export function normalizeGetJob(job) {
const providers = (job.provider ?? []).map((p) => p.id || p);
let md = `**Tool:** get_job | **Status:** OK\n\n`;
md += `### Job: ${job.name || job.id}\n\n`;
md += `- **ID:** ${job.id}\n`;
md += `- **Name:** ${job.name || ''}\n`;
md += `- **Enabled:** ${job.enabled ? 'yes' : 'no'}\n`;
md += `- **Active Listings:** ${job.numberOfFoundListings ?? 0}\n`;
md += `- **Providers:** ${providers.length > 0 ? providers.join(', ') : ''}\n`;
md += `- **Blacklist:** ${(job.blacklist ?? []).length > 0 ? job.blacklist.join(', ') : ''}\n`;
return toMcpResponse(md);
}
/**
* Normalize a list_listings response.
* @param {{ totalNumber: number, page: number, result: object[] }} queryResult
* @param {{ page: number, pageSize: number }} params
* @returns {{ content: Array }}
*/
export function normalizeListListings(queryResult, { page, pageSize }) {
const maxPage = Math.max(1, Math.ceil(queryResult.totalNumber / pageSize));
const hasMore = page < maxPage;
const listings = queryResult.result;
let md = `**Tool:** list_listings | **Status:** OK\n\n`;
md += `Found **${queryResult.totalNumber}** listing(s). Showing page ${page} of ${maxPage} (${listings.length} on this page).`;
if (hasMore) md += ` More pages available — use page=${page + 1} to continue.`;
md += '\n\n';
if (listings.length > 0) {
md += `| ID | Title | Address | Price | Size | Provider | Active | Created | Job |\n`;
md += `|----|-------|---------|-------|------|----------|--------|---------|-----|\n`;
for (const l of listings) {
md += `| ${cell(l.id)} | ${cell(l.title)} | ${cell(l.address)} | ${cell(l.price)} | ${cell(l.size)} | ${cell(l.provider)} | ${l.is_active ? 'yes' : 'no'} | ${formatDate(l.created_at)} | ${cell(l.job_name)} |\n`;
}
md += `\nUse **get_listing** with an ID for full details (description, link, image).\n`;
} else {
md += `No listings found.\n`;
}
md += `\n**Page:** ${page}/${maxPage} | **Has more:** ${hasMore ? 'yes' : 'no'}`;
return toMcpResponse(md);
}
/**
* Normalize a get_listing response.
* @param {object} listing - The listing object from storage.
* @returns {{ content: Array }}
*/
export function normalizeGetListing(listing) {
let md = `**Tool:** get_listing | **Status:** OK\n\n`;
md += `### Listing: ${listing.title || listing.id}\n\n`;
md += `- **ID:** ${listing.id}\n`;
md += `- **Title:** ${listing.title || ''}\n`;
md += `- **Description:** ${listing.description || ''}\n`;
md += `- **Address:** ${listing.address || ''}\n`;
md += `- **Price:** ${listing.price ?? ''}\n`;
md += `- **Size:** ${listing.size ?? ''}\n`;
md += `- **Provider:** ${listing.provider || ''}\n`;
md += `- **Link:** ${listing.link || ''}\n`;
md += `- **Image:** ${listing.image_url || ''}\n`;
md += `- **Active:** ${listing.is_active ? 'yes' : 'no'}\n`;
md += `- **Created:** ${formatDate(listing.created_at)}\n`;
md += `- **Job:** ${listing.job_name || ''}\n`;
if (listing.latitude != null && listing.longitude != null) {
md += `- **Location:** ${listing.latitude}, ${listing.longitude}\n`;
}
if (listing.distance_to_destination != null) {
md += `- **Distance to destination:** ${listing.distance_to_destination}\n`;
}
return toMcpResponse(md);
}
/**
* Normalize an error response.
* @param {string} message - The error message.
* @param {string} [tool] - Optional tool name for context.
* @returns {{ content: Array, isError: boolean }}
*/
export function normalizeError(message, tool) {
const md = `**Tool:** ${tool ?? 'unknown'} | **Status:** ERROR\n\n${message}`;
return toMcpResponse(md, true);
}

76
lib/mcp/stdio.js Normal file
View File

@@ -0,0 +1,76 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
/**
* Fredy MCP Server stdio transport
*
* Launches the MCP server over stdin/stdout so that local LLM clients
* (e.g. Claude Desktop, llm-cli, mcp-cli) can connect directly.
*
* Usage:
* MCP_TOKEN=fredy_<your-token> node mcp/stdio.js
*
* The MCP_TOKEN environment variable must contain a valid Fredy MCP token.
* Each user has a permanent, non-expiring token shown in the user management list.
*/
import { fileURLToPath } from 'url';
import path from 'path';
import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js';
import SqliteConnection from '../services/storage/SqliteConnection.js';
import { runMigrations } from '../services/storage/migrations/migrate.js';
import { createMcpServer } from './mcpAdapter.js';
import { validateMcpToken } from '../services/storage/userStorage.js';
// Ensure cwd is the project root so that relative DB/config paths resolve correctly
// (LM Studio and other MCP hosts may spawn this process from an arbitrary directory)
const __dirname = path.dirname(fileURLToPath(import.meta.url));
process.chdir(path.resolve(__dirname, '..', '..'));
// Initialize the database (required for standalone usage)
await SqliteConnection.init();
await runMigrations();
const token = process.env.MCP_TOKEN;
if (!token) {
process.stderr.write('Error: MCP_TOKEN environment variable is required.\n');
process.stderr.write('Each user has a permanent MCP token shown in the user management list.\n');
process.exit(1);
}
const auth = validateMcpToken(token);
if (!auth) {
process.stderr.write('Error: Invalid MCP_TOKEN. Token not found or user no longer exists.\n');
process.exit(1);
}
const mcpServer = createMcpServer();
// Wrap the stdio transport to inject authInfo into every message
const transport = new StdioServerTransport();
// Patch: the MCP SDK passes authInfo through the transport's onmessage extra param.
// For stdio we inject the resolved user from the token.
const patchedTransport = new Proxy(transport, {
set(target, prop, value) {
if (prop === 'onmessage') {
target.onmessage = (message, extra) => {
value(message, { ...extra, authInfo: { userId: auth.userId } });
};
return true;
}
target[prop] = value;
return true;
},
get(target, prop) {
return target[prop];
},
});
await mcpServer.connect(patchedTransport);
process.stderr.write('Fredy MCP Server running on stdio\n');

View File

@@ -1,3 +1,8 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import { markdown2Html } from '../../services/markdown.js';
import { getJob } from '../../services/storage/jobStorage.js';
import fetch from 'node-fetch';
@@ -8,7 +13,7 @@ export const send = ({ serviceName, newListings, notificationConfig, jobKey }) =
const jobName = job == null ? jobKey : job.name;
const promises = newListings.map((newListing) => {
const title = `${jobName} at ${serviceName}: ${newListing.title}`;
const message = `Address: ${newListing.address}\nSize: ${newListing.size}\nPrice: ${newListing.price}\Link: ${newListing.link}`;
const message = `Address: ${newListing.address}\nSize: ${newListing.size}\nPrice: ${newListing.price}\nLink: ${newListing.link}`;
return fetch(server, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
@@ -18,7 +23,6 @@ export const send = ({ serviceName, newListings, notificationConfig, jobKey }) =
}),
});
});
return Promise.all(promises);
};
export const config = {

View File

@@ -1,3 +1,8 @@
### Apprise Adapter
Refer to the [instructions](https://github.com/caronc/apprise-api#installation) on how to set up an Apprise instance and how to configure your preferred notification service.
Use [Apprise](https://github.com/caronc/apprise-api#installation) to forward notifications to many different services.
Quick start:
- Set up an Apprise API instance (see the installation guide linked above).
- Configure your preferred notification service(s) within Apprise.
- In Fredy, point the Apprise adapter to your Apprise API endpoint.

View File

@@ -1,3 +1,8 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import { markdown2Html } from '../../services/markdown.js';
export const send = ({ serviceName, newListings, jobKey }) => {

View File

@@ -1,4 +1,3 @@
### Console Adapter
The console adapter prints everything found by Fredy into the console (not sending any notifications to you). This can be useful when you want to check if your search
criteria meet the expectations.
The console adapter prints everything found by Fredy to the console (it does not send notifications). This is useful to verify that your search criteria work as expected before enabling a real notification service.

View File

@@ -0,0 +1,135 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import fetch from 'node-fetch';
import { getJob } from '../../services/storage/jobStorage.js';
import { markdown2Html } from '../../services/markdown.js';
import { normalizeImageUrl } from '../../utils.js';
/**
* Generates an idempotent decimal color code. The input string-based color code is
* generated using the djb2 hash algorithm.
*
* @param {string} str - Input string as color code base
* @returns {number} Generated decimal color code (0 - 16777215)
*/
const generateColorFromString = (str) => {
let hash = 5381; // initial value
const input = String(str);
for (let i = 0; i < input.length; i++) {
// hash * 33 + charCode
hash = (hash << 5) + hash + input.charCodeAt(i);
// Ensure the hash is 32 bit
hash |= 0;
}
let positiveHash = hash >>> 0;
const maxColorValue = 16777215;
const colorDecimal = positiveHash % maxColorValue;
return colorDecimal;
};
/**
* Creates an embed per listing
* (-> see https://birdie0.github.io/discord-webhooks-guide/structure/embeds.html).
*
* @param {string} jobKey - Key of job (used to set embed color)
* @param {object} listing - Object holding listing details
* @returns {object} Discord webhook embed
*/
const buildEmbed = (jobKey, listing) => {
const maxTitleLength = 252; // Max embed title length is 256 characters
let title = String(listing.title ?? 'N/A');
if (title.length > maxTitleLength) {
title = title.substring(0, maxTitleLength) + '...';
}
const fields = [
{
name: 'Price',
value: String(listing.price ?? 'n/a'),
inline: true,
},
{
name: 'Size',
value: listing?.size?.replace(/2m/g, 'm²') ?? 'n/a',
inline: true,
},
{
name: 'Address',
value: String(listing.address ?? 'n/a'),
inline: true,
},
];
const embed = {
title: title,
color: generateColorFromString(jobKey),
url: listing.link,
fields: fields,
};
if (listing.image) {
embed.image = {
url: normalizeImageUrl(listing.image),
};
}
return embed;
};
export const send = ({ serviceName, newListings, notificationConfig, jobKey }) => {
const adapter = notificationConfig.find((adapter) => adapter.id === config.id);
const webhookUrl = adapter?.fields?.webhookUrl;
if (!webhookUrl || newListings.length === 0) return Promise.resolve([]);
const job = getJob(jobKey);
const jobName = job?.name || jobKey;
const embeds = newListings.map((listing) => buildEmbed(jobKey, listing));
const maxEmbedsPerMessage = 10; // Discord only allows up to 10 embeds
const webhookPromises = [];
for (let i = 0; i < embeds.length; i += maxEmbedsPerMessage) {
// Send multiple Discord messages with up to 10 embeds per message
const embedChunk = embeds.slice(i, i + maxEmbedsPerMessage);
const content = i === 0 ? `*${jobName}:* ${serviceName} found **${newListings.length}** new listings.` : '';
const body = JSON.stringify({
content: content,
embeds: embedChunk,
});
const fetchPromise = fetch(webhookUrl, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body,
}).catch((error) => {
console.error(`Error sending Discord webhook for chunk starting at ${i}:`, error);
return Promise.reject(new Error(`Webhook failed: ${error.message}`));
});
webhookPromises.push(fetchPromise);
}
return Promise.allSettled(webhookPromises);
};
export const config = {
id: 'discord_webhook',
name: 'Discord Webhook',
readme: markdown2Html('lib/notification/adapter/discord_webhook.md'),
description: 'Fredy will send new listings to the Discord channel of your choice.',
fields: {
webhookUrl: {
type: 'text',
label: 'Webhook URL',
description: 'The URL of the Discord webhook to send messages to.',
},
},
};

View File

@@ -0,0 +1,8 @@
### Discord Webhook Adapter
Use a Discord channel webhook to receive notifications.
Quick start:
- Create a webhook in your target Discord channel. See the "Intro to Webhooks" guide on the Discord support site: https://support.discord.com/hc/en-us/articles/228383668-Intro-to-Webhooks
- Copy the generated webhook URL.
- In Fredy, configure the Discord adapter with this webhook URL.

View File

@@ -0,0 +1,75 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import { markdown2Html } from '../../services/markdown.js';
const mapListing = (listing) => ({
address: listing.address,
description: listing.description,
id: listing.id,
imageUrl: listing.image,
price: listing.price,
size: listing.size,
title: listing.title,
url: listing.link,
});
export const send = async ({ serviceName, newListings, notificationConfig, jobKey }) => {
const { authToken, endpointUrl, selfSignedCerts } = notificationConfig.find((a) => a.id === config.id).fields;
const listings = newListings.map(mapListing);
const body = {
jobId: jobKey,
timestamp: new Date().toISOString(),
provider: serviceName,
listings,
};
const headers = {
'Content-Type': 'application/json',
};
if (authToken != null) {
headers['Authorization'] = `Bearer ${authToken}`;
}
let fetchOptions = {
method: 'POST',
headers,
timeout: 10000,
body: JSON.stringify(body),
};
if (selfSignedCerts === true) {
fetchOptions.dispatcher = new (await import('undici')).Agent({
connect: { rejectUnauthorized: false },
});
}
return fetch(endpointUrl, fetchOptions);
};
export const config = {
id: 'http',
name: 'HTTP',
readme: markdown2Html('lib/notification/adapter/http.md'),
description: 'Fredy will send a generic HTTP POST request.',
fields: {
endpointUrl: {
description: "Your application's endpoint URL.",
label: 'Endpoint URL',
type: 'text',
},
selfSignedCerts: {
label: 'Self-signed certificates',
type: 'boolean',
},
authToken: {
description: "Your application's auth token, if required by your endpoint.",
label: 'Auth token (optional)',
optional: true,
type: 'text',
},
},
};

View File

@@ -0,0 +1,43 @@
### HTTP Adapter
This is a generic adapter for sending notifications via HTTP requests.
You can leverage this adapter to integrate with various webhooks or APIs that accept HTTP requests. (e.g. Supabase
Functions, a Node.js server, etc.)
HTTP adapter supports a `authToken` field, which can be used to include an authorization token in the request headers.
Your token would be included as a Bearer token in the `Authorization` header, which is a common method for securing API requests.
Request Details:
<details>
Request Method: POST
Headers:
```
Content Type: `application/json`
Authorization: Bearer {your-optional-auth-token}
```
Body:
```json
{
"jobId": "mg1waX4RHmIzL5NDYtYp-",
"provider": "immoscout",
"timestamp": "2024-06-15T12:34:56Z",
"listings": [
{
"address": "Str. 123, Bielefeld, Germany",
"description": "Möbliert: Einziehen & wohlfühlen: Neu möbliert.",
"id": "123456789",
"imageUrl": "https://<target-url>.com/listings/123456789.jpg",
"price": "1.240 €",
"size": "38 m²",
"title": "Schöne 1-Zimmer-Wohnung in Bielefeld",
"url": "https://<target-url>.com/listings/123456789"
}
]
}
```
</details>

View File

@@ -1,43 +1,118 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import mailjet from 'node-mailjet';
import path from 'path';
import fs from 'fs';
import Handlebars from 'handlebars';
import fetch from 'node-fetch';
import { markdown2Html } from '../../services/markdown.js';
import { getDirName } from '../../utils.js';
import { getDirName, normalizeImageUrl } from '../../utils.js';
import logger from '../../services/logger.js';
const __dirname = getDirName();
const template = fs.readFileSync(path.resolve(__dirname + '/notification/emailTemplate/template.hbs'), 'utf8');
const emailTemplate = Handlebars.compile(template);
export const send = ({ serviceName, newListings, notificationConfig, jobKey }) => {
const guessMime = (url) => {
const lower = url.split('?')[0].toLowerCase();
if (lower.endsWith('.png')) return 'image/png';
if (lower.endsWith('.gif')) return 'image/gif';
return 'image/jpeg';
};
const toBase64 = async (url) => {
try {
const res = await fetch(url);
if (!res.ok) throw new Error(`Fetch failed with status ${res.status} for URL: ${url}`);
const ab = await res.arrayBuffer();
return Buffer.from(ab).toString('base64');
} catch (error) {
logger.error(`Error fetching image from ${url}:`, error.message);
throw error;
}
};
const mapListingsWithCid = async (serviceName, jobKey, listings) => {
const out = [];
const attachments = [];
for (let i = 0; i < listings.length; i++) {
const l = listings[i] || {};
const imgUrl = normalizeImageUrl(l.image);
const item = {
title: l.title || '',
link: l.link || '',
address: l.address || '',
size: l.size || '',
price: l.price || '',
serviceName,
jobKey,
hasImage: false,
imageCid: '',
};
if (imgUrl) {
try {
const base64 = await toBase64(imgUrl);
const cid = `listing-${i}`;
attachments.push({
ContentType: guessMime(imgUrl),
Filename: `listing-${i}.${imgUrl.split('.').pop().split('?')[0] || 'jpg'}`,
Base64Content: base64,
ContentID: cid,
});
item.hasImage = true;
item.imageCid = cid;
} catch (error) {
logger.warn(`Skipping image for listing ${i} due to error: ${error.message}`);
}
}
out.push(item);
}
return { listings: out, attachments };
};
export const send = async ({ serviceName, newListings, notificationConfig, jobKey }) => {
const { apiPublicKey, apiPrivateKey, receiver, from } = notificationConfig.find(
(adapter) => adapter.id === config.id,
).fields;
const to = receiver
.trim()
.split(',')
.map((r) => ({
Email: r.trim(),
}));
.map((r) => ({ Email: r.trim() }))
.filter((r) => r.Email.length > 0);
const { listings, attachments } = await mapListingsWithCid(serviceName, jobKey, newListings);
const html = emailTemplate({
serviceName: `Job: (${jobKey}) | Service: ${serviceName}`,
numberOfListings: listings.length,
listings,
});
return mailjet
.apiConnect(apiPublicKey, apiPrivateKey)
.post('send', { version: 'v3.1' })
.request({
Messages: [
{
From: {
Email: from,
Name: 'Fredy',
},
From: { Email: from, Name: 'Fredy' },
To: to,
Subject: `Fredy found ${newListings.length} new listings for ${serviceName}`,
HTMLPart: emailTemplate({
serviceName: `Job: (${jobKey}) | Service: ${serviceName}`,
numberOfListings: newListings.length,
listings: newListings,
}),
Subject: `Fredy found ${listings.length} new listing(s) for ${serviceName}`,
HTMLPart: html,
InlinedAttachments: attachments,
},
],
});
};
export const config = {
id: 'mailjet',
name: 'MailJet',

View File

@@ -1,8 +1,8 @@
### MailJet Adapter
To use [MailJet](https://mailjet.com), you need to create an account. You'll need to decided from which email address you want Fredy to send from.
E.g. if you use yourGmailAccount@gmail.com, you have to add this to MailJet and verify it as well.
The given public/private api keys are needed in order to use MailJet with Fredy. Fredy will use the same template, it is using for SendGrid.
### Mailjet Adapter
If this email should be sent to multiple receiver use a comma separator (some@email.com, someOther@email.com).
To use [Mailjet](https://mailjet.com), create an account and decide which email address Fredy should send from.
For example, if you use yourGmailAccount@gmail.com, add and verify this address in Mailjet.
Provide your public/private API keys in Fredy's configuration. Fredy uses the same email template as for SendGrid.
To send to multiple recipients, separate email addresses with commas (e.g., some@email.com, someOther@email.com).

View File

@@ -1,3 +1,8 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import { markdown2Html } from '../../services/markdown.js';
import { getJob } from '../../services/storage/jobStorage.js';
import fetch from 'node-fetch';
@@ -13,10 +18,10 @@ export const send = ({ serviceName, newListings, notificationConfig, jobKey }) =
return fetch(webhook, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: {
body: JSON.stringify({
channel: channel,
text: message,
},
}),
});
};
export const config = {

View File

@@ -1,5 +1,8 @@
### Mattermost Adapter
For Mattermost, you need to create a incoming webhook. This is pretty easy. Please visit the steps in the [developer docs](https://docs.mattermost.com/developer/webhooks-incoming.html) and follow the instructions.
Receive notifications in Mattermost via an incoming webhook.
As a result, you get the webhook URL for configuration in fredy. In addition, the target channel must be defined.
Quick start:
- Create an incoming webhook following the Mattermost developer docs: https://docs.mattermost.com/developer/webhooks-incoming.html
- Copy the webhook URL.
- In Fredy, configure the Mattermost adapter with this URL and the target channel.

View File

@@ -1,30 +1,62 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import { markdown2Html } from '../../services/markdown.js';
import { getJob } from '../../services/storage/jobStorage.js';
import fetch from 'node-fetch';
import { normalizeImageUrl } from '../../utils.js';
export const send = ({ serviceName, newListings, notificationConfig, jobKey }) => {
const { priority, server, topic } = notificationConfig.find((adapter) => adapter.id === config.id).fields;
const job = getJob(jobKey);
const jobName = job == null ? jobKey : job.name;
const promises = newListings.map((newListing) => {
const message = `Address: ${newListing.address} Size: ${newListing.size.replace(/2m/g, '$m^2$')} Price: ${
newListing.price
}`;
return fetch(server, {
const message = `
Address: ${newListing.address}
Size: ${newListing.size == null ? 'N/A' : newListing.size.replace(/2m/g, '$m^2$')}
Price: ${newListing.price}
Link: ${newListing.link}`;
const sanitizeHeaderValue = (value) =>
String(value ?? '')
.replace(/[\r\n]+/g, ' ')
.replace(/[^\x20-\x7E]/g, ' ')
.trim();
const headers = {
Title: sanitizeHeaderValue(newListing.title),
Priority: sanitizeHeaderValue(priority),
Tags: sanitizeHeaderValue(`${serviceName},${jobName}`),
Click: sanitizeHeaderValue(newListing.link),
};
if (newListing.image && typeof newListing.image === 'string') {
headers.Attach = normalizeImageUrl(newListing.image);
}
return fetch(`${server}/${topic}`, {
method: 'POST',
body: JSON.stringify({
topic: topic,
message: message,
title: newListing.title,
tags: [serviceName, jobName],
priority: parseInt(priority),
click: newListing.link,
}),
});
headers,
body: message,
})
.then((res) => {
if (!res.ok) {
throw new Error(`Ntfy message could not be sent. Status code: ${res.status}`);
}
return res.text();
})
.catch((error) => {
// Ensure we reject with an Error object and prevent unhandled rejections
throw error instanceof Error ? error : new Error(String(error));
});
});
return Promise.all(promises);
};
export const config = {
id: 'ntfy',
name: 'ntfy',

View File

@@ -1,5 +1,8 @@
### ntfy Adapter
For ntfy, you need to create a topic on your preferred ntfy instance. This is pretty easy. Please visit the steps in the [docs](https://docs.ntfy.sh/publish/) and follow the instructions.
Send push notifications using an ntfy topic.
As a result, you get the URL for configuration in fredy. In addition, the priority must be defined.
Quick start:
- Create or choose a topic on your preferred ntfy instance (see docs: https://docs.ntfy.sh/publish/).
- Copy the publish URL for that topic.
- In Fredy, configure the ntfy adapter with the topic URL and set a priority.

View File

@@ -1,28 +1,61 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import { markdown2Html } from '../../services/markdown.js';
import { getJob } from '../../services/storage/jobStorage.js';
import fetch from 'node-fetch';
export const send = ({ serviceName, newListings, notificationConfig, jobKey }) => {
export const send = async ({ serviceName, newListings, notificationConfig, jobKey }) => {
const { token, user, device } = notificationConfig.find((adapter) => adapter.id === config.id).fields;
const job = getJob(jobKey);
const jobName = job == null ? jobKey : job.name;
const promises = newListings.map((newListing) => {
const title = `${jobName} at ${serviceName}: ${newListing.title}`;
const message = `Address: ${newListing.address}\nSize: ${newListing.size}\nPrice: ${newListing.price}\nLink: ${newListing.link}`;
return fetch('https://api.pushover.net/1/messages.json', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
token: token,
user: user,
message: message,
device: device,
title: title,
}),
});
});
return Promise.all(promises);
const results = await Promise.all(
newListings.map(async (newListing) => {
const title = `${jobName} at ${serviceName}: ${newListing.title}`;
const message = `Address: ${newListing.address}\nSize: ${newListing.size}\nPrice: ${newListing.price}\nLink: ${newListing.link}`;
const form = new FormData();
form.append('token', token);
form.append('user', user);
form.append('title', title);
form.append('message', message);
if (device) form.append('device', device);
// Try to attach image if available
if (newListing.image && typeof newListing.image === 'string') {
try {
const imgRes = await fetch(newListing.image);
if (imgRes.ok) {
const ab = await imgRes.arrayBuffer();
form.append('attachment', new Blob([ab]), 'image.jpg');
}
} catch {
// fail silently, just skip the image
}
}
const res = await fetch('https://api.pushover.net/1/messages.json', {
method: 'POST',
body: form,
});
return res.json();
}),
);
// Collect errors
const errors = results
.map((r) => (r.errors && r.errors.length > 0 ? r.errors.join(', ') : null))
.filter((e) => e !== null);
if (errors.length > 0) {
return Promise.reject(errors.join('; '));
}
return results;
};
export const config = {
@@ -34,7 +67,7 @@ export const config = {
token: {
type: 'text',
label: 'API token',
description: 'Your application\'s API token.',
description: "Your application's API token.",
},
user: {
type: 'text',
@@ -44,7 +77,8 @@ export const config = {
device: {
type: 'text',
label: 'Device name',
description: 'The device name to send your notification to. Messages may be addressed to multiple specific devices by joining them with a comma.',
description:
'The device name to send your notification to. Messages may be addressed to multiple specific devices by joining them with a comma.',
},
},
};

View File

@@ -1,5 +1,8 @@
### Pushover Adapter
Refer to the [instructions](https://support.pushover.net/i7-what-is-pushover-and-how-do-i-use-it) to set up your Pushover application.
Use Pushover to receive push notifications on your devices.
After setting up the application, please enter both your newly created User key and API token.
Setup:
- Follow Pushover's getting-started guide: https://support.pushover.net/i7-what-is-pushover-and-how-do-i-use-it
- Create an application and obtain your User Key and API Token.
- In Fredy, configure the Pushover adapter with both values.

View File

@@ -0,0 +1,88 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import { Resend } from 'resend';
import path from 'path';
import fs from 'fs';
import Handlebars from 'handlebars';
import { markdown2Html } from '../../services/markdown.js';
import { getDirName, normalizeImageUrl } from '../../utils.js';
const __dirname = getDirName();
const template = fs.readFileSync(path.resolve(__dirname + '/notification/emailTemplate/template.hbs'), 'utf8');
const emailTemplate = Handlebars.compile(template);
const mapListings = (serviceName, jobKey, listings) =>
listings.map((l) => {
const image = normalizeImageUrl(l.image);
return {
title: l.title || '',
link: l.link || '',
address: l.address || '',
size: l.size || '',
price: l.price || '',
image,
hasImage: Boolean(image),
serviceName,
jobKey,
};
});
export const send = async ({ serviceName, newListings, notificationConfig, jobKey }) => {
const { apiKey, receiver, from } = notificationConfig.find((adapter) => adapter.id === config.id).fields;
const to = receiver
.trim()
.split(',')
.map((r) => r.trim())
.filter(Boolean);
const resend = new Resend(apiKey);
const listings = mapListings(serviceName, jobKey, newListings);
const html = emailTemplate({
serviceName: `Job: (${jobKey}) | Service: ${serviceName}`,
numberOfListings: listings.length,
listings,
});
const { error } = await resend.emails.send({
from,
to,
subject: `Fredy found ${listings.length} new listing(s) for ${serviceName}`,
html,
});
if (!error) {
return Promise.resolve();
} else {
return Promise.reject(error.message);
}
};
export const config = {
id: 'resend',
name: 'Resend',
description: 'Resend is being used to send new listings via mail.',
readme: markdown2Html('lib/notification/adapter/resend.md'),
fields: {
apiKey: {
type: 'text',
label: 'Api Key',
description: 'The Resend API key used to send emails.',
},
receiver: {
type: 'email',
label: 'Receiver Email',
description: 'Comma-separated email addresses Fredy will send notifications to.',
},
from: {
type: 'email',
label: 'Sender Email',
description: 'The verified email address or domain you send from in Resend.',
},
},
};

View File

@@ -0,0 +1,17 @@
### Resend Adapter
Resend is a modern email delivery service that Fredy can use to send notifications.
Setup:
- Create a Resend account: https://resend.com/
- Create an API key and add it to Fredy's configuration.
- Choose the sender address (e.g., you@yourdomain.com). Verify the domain (https://resend.com/domains/) in Resend before using it.
- Optional for local testing: you can use `onboarding@resend.dev`, but Resend may restrict who you can send to when using test domains.
Multiple recipients:
- Separate email addresses with commas (e.g., some@email.com, someOther@email.com).
Notes & Troubleshooting:
- Ensure the `from` address is verified or belongs to a verified domain in Resend.
- If emails don't arrive, check your spam folder and Resend dashboard logs.
- The template displays listing images via their public URLs; make sure images are reachable.

View File

@@ -1,24 +1,58 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import sgMail from '@sendgrid/mail';
import { markdown2Html } from '../../services/markdown.js';
import { normalizeImageUrl } from '../../utils.js';
const mapListings = (serviceName, jobKey, listings) =>
listings.map((l) => {
const image = normalizeImageUrl(l.image);
return {
title: l.title || '',
link: l.link || '',
address: l.address || '',
size: l.size || '',
price: l.price || '',
image,
hasImage: Boolean(image),
// optional plain text snippet
snippet: [l.address, l.price, l.size].filter(Boolean).join(' | '),
serviceName,
jobKey,
};
});
export const send = ({ serviceName, newListings, notificationConfig, jobKey }) => {
const { apiKey, receiver, from, templateId } = notificationConfig.find((adapter) => adapter.id === config.id).fields;
sgMail.setApiKey(apiKey);
const to = receiver
.trim()
.split(',')
.map((r) => r.trim())
.filter(Boolean);
const listings = mapListings(serviceName, jobKey, newListings);
const msg = {
templateId,
to: receiver
.trim()
.split(',')
.map((r) => r.trim()),
to,
from,
subject: `Job ${jobKey} | Service ${serviceName} found ${newListings.length} new listing(s)`,
dynamic_template_data: {
serviceName: `Job: (${jobKey}) | Service: ${serviceName}`,
numberOfListings: newListings.length,
listings: newListings,
listings,
},
};
return sgMail.send(msg);
};
export const config = {
id: 'sendgrid',
name: 'SendGrid',

View File

@@ -1,10 +1,12 @@
### SendGrid Adapter
SendGrid is a free email service (free as in "you cannot send more than 100(Sendgrid) and 200(Mailjet) emails a day"), which is more than enough for Fredy.
To use [SendGrid](https://sendgrid.com/), you need to create an account. You'll need to decided from which email address you want Fredy to send from. E.g. if you use yourGmailAccount@gmail.com, you have to add this to sendgrid and verify it as well.
SendGrid is an email delivery service with a generous free tier, which is more than enough for Fredy.
Lastly you have to create an api-key and feed it into Fredy's config, as well as creating a new dynamic template. For this new template, I recommend copying and pasting the code from the one I have provided under `/lib/notification/emailTemplate/template.hbs`.
Setup:
- Create a SendGrid account: https://sendgrid.com/
- Decide which email address Fredy should send from (e.g., yourGmailAccount@gmail.com), add it to SendGrid, and complete the verification.
- Create an API key and add it to Fredy's configuration.
- Create a Dynamic Template in SendGrid. You can copy the template from `/lib/notification/emailTemplate/template.hbs`.
If this email should be sent to multiple receiver use a comma separator (some@email.com, someOther@email.com).
Sending to multiple recipients:
- Separate email addresses with commas (e.g., some@email.com, someOther@email.com).

View File

@@ -1,43 +1,66 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import Slack from 'slack';
import { markdown2Html } from '../../services/markdown.js';
const msg = Slack.chat.postMessage;
export const send = ({ serviceName, newListings, notificationConfig, jobKey }) => {
const { token, channel } = notificationConfig.find((adapter) => adapter.id === config.id).fields;
return newListings.map((payload) =>
msg({
token,
channel,
text: `*(${serviceName} - ${jobKey})* - ${payload.title}`,
attachments: [
{
fallback: payload.title,
color: '#36a64f',
title: 'Link to Exposé',
title_link: payload.link,
fields: [
{
title: 'Price',
value: payload.price,
short: false,
},
{
title: 'Size',
value: payload.size,
short: false,
},
{
title: 'Address',
value: payload.address,
short: false,
},
],
footer: 'Powered by Fredy',
ts: new Date().getTime() / 1000,
},
import { normalizeImageUrl } from '../../utils.js';
const buildBlocks = (serviceName, jobKey, p) => {
const blocks = [
{
type: 'header',
text: { type: 'plain_text', text: `New Listing from ${serviceName} (${jobKey})`, emoji: false },
},
{
type: 'section',
text: { type: 'mrkdwn', text: `*<${p.link}|${p.title}>*` },
},
{
type: 'section',
fields: [
{ type: 'mrkdwn', text: `*Price*\n${p.price ?? 'n/a'}` },
{ type: 'mrkdwn', text: `*Size*\n${p.size ?? 'n/a'}` },
{ type: 'mrkdwn', text: `*Address*\n${p.address ?? 'n/a'}` },
],
}),
},
];
const img = normalizeImageUrl(p.image);
if (img) {
blocks.push({
type: 'image',
image_url: img,
alt_text: p.title || 'listing image',
});
}
blocks.push({
type: 'context',
elements: [{ type: 'mrkdwn', text: 'Powered by Fredy' }],
});
return blocks;
};
export const send = ({ serviceName, newListings, notificationConfig, jobKey }) => {
const { token, channel } = notificationConfig.find((a) => a.id === config.id).fields;
return Promise.allSettled(
newListings.map((p) =>
Slack.chat.postMessage({
token,
channel,
text: `${serviceName} ${jobKey}: ${p.title}`,
blocks: buildBlocks(serviceName, jobKey, p),
unfurl_links: false,
unfurl_media: false,
}),
),
);
};
export const config = {
id: 'slack',
name: 'Slack',

View File

@@ -1,6 +1,5 @@
### Slack Adapter
### Slack Adapter (Legacy)
*IMPORTANT:*
This legacy adapter is outdated and kept only for backward compatibility. Please use the Slack adapter with webhooks instead.
In order to use [Slack](https://slack.com), you need to create an account. When done, you need to create a new App in your workspace. Give it the permission `chat:write:bot` and `chat:write:user`.
Now you need to create a user token and a channel. Make sure the bot is installed to this channel.

View File

@@ -0,0 +1,84 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import fetch from 'node-fetch';
import { markdown2Html } from '../../services/markdown.js';
import { normalizeImageUrl } from '../../utils.js';
const buildBlocks = (serviceName, jobKey, p) => {
const blocks = [
{
type: 'header',
text: { type: 'plain_text', text: `New Listing from ${serviceName} (${jobKey})`, emoji: false },
},
{
type: 'section',
text: { type: 'mrkdwn', text: `*<${p.link}|${p.title}>*` },
},
{
type: 'section',
fields: [
{ type: 'mrkdwn', text: `*Price*\n${p.price ?? 'n/a'}` },
{ type: 'mrkdwn', text: `*Size*\n${p.size ?? 'n/a'}` },
{ type: 'mrkdwn', text: `*Address*\n${p.address ?? 'n/a'}` },
],
},
];
const img = normalizeImageUrl(p.image);
if (img) {
blocks.push({
type: 'image',
image_url: img,
alt_text: p.title || 'listing image',
});
}
blocks.push({
type: 'context',
elements: [{ type: 'mrkdwn', text: 'Powered by Fredy' }],
});
return blocks;
};
const postJson = (url, body) =>
fetch(url, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body,
});
export const send = ({ serviceName, newListings, notificationConfig, jobKey }) => {
const adapter = notificationConfig.find((a) => a.id === config.id);
const webhookUrl = adapter?.fields?.webhookUrl;
if (!webhookUrl) return Promise.resolve([]);
const promises = newListings.map((p) => {
const body = JSON.stringify({
text: `${serviceName} ${jobKey}: ${p.title}`,
blocks: buildBlocks(serviceName, jobKey, p),
unfurl_links: false,
unfurl_media: false,
});
return postJson(webhookUrl, body);
});
return Promise.allSettled(promises);
};
export const config = {
id: 'slack_with_webhooks',
name: 'Slack with Webhooks',
readme: markdown2Html('lib/notification/adapter/slack_with_webhooks.md'),
description: 'Fredy will send new listings to the slack channel of your choice..',
fields: {
webhookUrl: {
type: 'text',
label: 'Webhook-Url',
description: 'The Url of the Webhook to send messages to.',
},
},
};

View File

@@ -0,0 +1,10 @@
### Slack Adapter (Webhooks)
*IMPORTANT:*
This is the recommended Slack adapter. The old Slack adapter is unmaintained and kept only for backward compatibility.
Setup:
- Create a Slack account and workspace if you don't have one: https://slack.com
- Create a channel where you want to receive notifications.
- Add the Incoming Webhooks integration to that channel and copy the Webhook URL.
- In Fredy, configure the Slack Webhook adapter with this URL.

View File

@@ -0,0 +1,112 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import nodemailer from 'nodemailer';
import path from 'path';
import fs from 'fs';
import Handlebars from 'handlebars';
import { markdown2Html } from '../../services/markdown.js';
import { getDirName, normalizeImageUrl } from '../../utils.js';
const __dirname = getDirName();
const template = fs.readFileSync(path.resolve(__dirname + '/notification/emailTemplate/template.hbs'), 'utf8');
const emailTemplate = Handlebars.compile(template);
const mapListings = (serviceName, jobKey, listings) =>
listings.map((l) => {
const image = normalizeImageUrl(l.image);
return {
title: l.title || '',
link: l.link || '',
address: l.address || '',
size: l.size || '',
price: l.price || '',
image,
hasImage: Boolean(image),
serviceName,
jobKey,
};
});
export const send = async ({ serviceName, newListings, notificationConfig, jobKey }) => {
const { host, port, secure, username, password, receiver, from } = notificationConfig.find(
(adapter) => adapter.id === config.id,
).fields;
const to = receiver
.trim()
.split(',')
.map((r) => r.trim())
.filter(Boolean);
const transporter = nodemailer.createTransport({
host,
port: Number(port),
secure: secure === 'true',
auth: {
user: username,
pass: password,
},
});
const listings = mapListings(serviceName, jobKey, newListings);
const html = emailTemplate({
serviceName: `Job: (${jobKey}) | Service: ${serviceName}`,
numberOfListings: listings.length,
listings,
});
return transporter.sendMail({
from,
to: to.join(','),
subject: `Fredy found ${listings.length} new listing(s) for ${serviceName}`,
html,
});
};
export const config = {
id: 'smtp',
name: 'SMTP',
description: 'Send notifications via any SMTP server using Nodemailer.',
readme: markdown2Html('lib/notification/adapter/smtp.md'),
fields: {
host: {
type: 'text',
label: 'SMTP Host',
description: 'The hostname of the SMTP server (e.g., smtp.gmail.com).',
},
port: {
type: 'text',
label: 'SMTP Port',
description: 'The port of the SMTP server (e.g., 587 for STARTTLS, 465 for SSL).',
},
secure: {
type: 'text',
label: 'Secure (SSL/TLS)',
description: 'Set to "true" for port 465 (SSL). Leave empty or "false" for STARTTLS on port 587.',
},
username: {
type: 'text',
label: 'Username',
description: 'The username for SMTP authentication.',
},
password: {
type: 'text',
label: 'Password',
description: 'The password (or app password) for SMTP authentication.',
},
receiver: {
type: 'text',
label: 'Receiver Email(s)',
description: 'Comma-separated email addresses Fredy will send notifications to.',
},
from: {
type: 'email',
label: 'Sender Email',
description: 'The email address Fredy sends from.',
},
},
};

View File

@@ -0,0 +1,22 @@
### SMTP Adapter
Send notifications through any SMTP server using [Nodemailer](https://nodemailer.com/).
This works with Gmail, Outlook, self-hosted mail servers, or any provider that supports SMTP.
Setup:
- Provide the SMTP host and port of your mail server.
- For **SSL/TLS** (port 465), set Secure to `true`.
- For **STARTTLS** (port 587), leave Secure empty or set it to `false`.
- Enter the username and password for authentication. For Gmail, use an [App Password](https://support.google.com/accounts/answer/185833).
- Set the sender email address (must be allowed by your SMTP server).
Multiple recipients:
- Separate email addresses with commas (e.g., `some@email.com`, `someOther@email.com`).
Common SMTP settings:
- **Gmail** — `smtp.gmail.com`, port 587, secure: false
- **Outlook** — `smtp.office365.com`, port 587, secure: false
- **Yahoo** — `smtp.mail.yahoo.com`, port 465, secure: true

View File

@@ -0,0 +1,61 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import { markdown2Html } from '../../services/markdown.js';
import Database from 'better-sqlite3';
import path from 'path';
import fs from 'fs';
export const send = ({ serviceName, newListings, jobKey, notificationConfig }) => {
const sqliteConfig = notificationConfig.find((adapter) => adapter.id === config.id);
const dbPath = sqliteConfig?.fields?.dbPath || 'db/listings.db';
const dbDir = path.dirname(dbPath);
if (!fs.existsSync(dbDir)) {
fs.mkdirSync(dbDir, { recursive: true });
}
const db = new Database(dbPath);
const fields = [
'serviceName',
'jobKey',
'id',
'size',
'rooms',
'price',
'address',
'title',
'link',
'description',
'image',
];
db.prepare(`CREATE TABLE IF NOT EXISTS listing (${fields.join(' TEXT, ')} TEXT);`).run();
const insert = db.prepare(`INSERT INTO listing (${fields.join(', ')}) VALUES (@${fields.join(', @')})`);
newListings.map((listing) => {
let insertListing = {};
fields.map((field) => {
insertListing[field] = listing[field];
});
insertListing.serviceName = serviceName;
insertListing.jobKey = jobKey;
insert.run(insertListing);
});
return Promise.resolve();
};
export const config = {
id: 'sqlite',
name: 'SQLite',
description: 'This adapter stores listings in a local SQLite 3 database.',
fields: {
dbPath: {
type: 'text',
label: 'Database Path',
description:
'Path to the SQLite database file (e.g., db/listings.db). If not specified, defaults to db/listings.db',
placeholder: 'db/listings.db',
},
},
readme: markdown2Html('lib/notification/adapter/sqlite.md'),
};

View File

@@ -0,0 +1,21 @@
### SQLite Adapter
This adapter stores search results in an SQLite database. By default, the database is located at `db/listings.db`, but you can configure a custom location. The file can be used for analysis later.
The table contains the following columns (all stored as `TEXT`):
```json
[
"serviceName",
"jobKey",
"id",
"size",
"rooms",
"price",
"address",
"title",
"link",
"description",
"image"
]
```

View File

@@ -1,63 +1,202 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import { markdown2Html } from '../../services/markdown.js';
import { getJob } from '../../services/storage/jobStorage.js';
import fetch from 'node-fetch';
const MAX_ENTITIES_PER_CHUNK = 8;
const RATE_LIMIT_INTERVAL = 1010;
import pThrottle from 'p-throttle';
import { normalizeImageUrl } from '../../utils.js';
import logger from '../../services/logger.js';
const RATE_LIMIT_INTERVAL = 1000;
const chatThrottleMap = new Map();
/**
* splitting an array into chunks because Telegram only allows for messages up to
* 4096 chars, thus we have to split messages into chunks
* @param inputArray
* @param perChunk
* Removes stale throttled call entries to keep memory bounded.
*/
const arrayChunks = (inputArray, perChunk) =>
inputArray.reduce((all, one, i) => {
const ch = Math.floor(i / perChunk);
all[ch] = [].concat(all[ch] || [], one);
return all;
}, []);
function shorten(str, len = 30) {
return str.length > len ? str.substring(0, len) + '...' : str;
function cleanupOldThrottles() {
const now = Date.now();
const maxAge = RATE_LIMIT_INTERVAL + 1000;
const toBeDeleted = [];
for (const [chatId, chatThrottle] of chatThrottleMap.entries()) {
if (now - chatThrottle.lastUsedAt > maxAge) toBeDeleted.push(chatId);
}
for (const chatId of toBeDeleted) chatThrottleMap.delete(chatId);
}
export const send = ({ serviceName, newListings, notificationConfig, jobKey }) => {
const { token, chatId } = notificationConfig.find((adapter) => adapter.id === config.id).fields;
/**
* Return a throttled wrapper for a chatId to limit Telegram API calls.
* Uses p-throttle with 1 request per RATE_LIMIT_INTERVAL per chat.
*
* @template {Function} T
* @param {string|number} chatId
* @param {T} call - async function (endpoint: string, body: any) => Promise<Response>
* @returns {T}
*/
function getThrottled(chatId, call) {
cleanupOldThrottles();
const now = Date.now();
const chatThrottle = chatThrottleMap.get(chatId);
if (chatThrottle) {
chatThrottle.lastUsedAt = now;
return chatThrottle.throttled;
}
const throttled = pThrottle({ limit: 1, interval: RATE_LIMIT_INTERVAL })(call);
chatThrottleMap.set(chatId, { lastUsedAt: now, throttled });
return throttled;
}
/**
* Shorten a string to a maximum length with an ellipsis suffix.
* @param {string} str
* @param {number} [len=90]
* @returns {string}
*/
function shorten(str, len = 90) {
if (!str) return '';
return str.length > len ? str.substring(0, len).trim() + '...' : str;
}
/**
* Escape basic HTML entities for Telegram HTML parse mode.
* @param {string} [s='']
* @returns {string}
*/
function escapeHtml(s = '') {
return s.replace(/&/g, '&amp;').replace(/</g, '&lt;').replace(/>/g, '&gt;');
}
/**
* Build a Telegram photo caption (max 1024 characters) using HTML parse mode.
* @param {string} jobName
* @param {string} serviceName
* @param {Object} o - Listing object
* @param {string} [o.title]
* @param {string} [o.address]
* @param {string|number} [o.price]
* @param {string|number} [o.size]
* @param {string} [o.link]
* @returns {string}
*/
function buildCaption(jobName, serviceName, o) {
const title = shorten((o.title || '').replace(/\*/g, ''), 90);
const meta = [o.address, o.price, o.size].filter(Boolean).join(' | ');
return `<i>${escapeHtml(jobName)}</i> (${escapeHtml(serviceName)})\n<a href='${escapeHtml(
o.link || '',
)}'><b>${escapeHtml(title)}</b></a>\n${escapeHtml(meta)}`.slice(0, 1024);
}
/**
* Build a Telegram message text using HTML parse mode.
* @param {string} jobName
* @param {string} serviceName
* @param {Object} o - Listing object
* @returns {string}
*/
function buildText(jobName, serviceName, o) {
const title = shorten((o.title || '').replace(/\*/g, ''), 90);
const meta = [o.address, o.price, o.size].filter(Boolean).join(' | ');
return (
`<i>${escapeHtml(jobName)}</i> (${escapeHtml(serviceName)})\n` +
`<a href='${escapeHtml(o.link || '')}'><b>${escapeHtml(title)}</b></a>\n` +
`${escapeHtml(meta)}`
);
}
/**
* Send new listings to Telegram.
* - Respects per-chat Telegram rate limits using a lightweight throttle cache.
* - Falls back to sendMessage when sendPhoto fails or image is missing.
*
* @param {Object} params
* @param {string} params.serviceName - Name of the crawler/service producing the listings.
* @param {Array<Object>} params.newListings - Array of new listing objects.
* @param {Array<Object>} params.notificationConfig - Notification adapters configuration array.
* @param {string} params.jobKey - Storage job key to resolve the human readable job name.
* @returns {Promise<Array<Response>>} Promise resolving when all send operations complete.
*/
export const send = ({ serviceName, newListings = [], notificationConfig, jobKey }) => {
const adapterCfg = notificationConfig.find((adapter) => adapter.id === config.id);
if (!adapterCfg || !adapterCfg.fields) {
throw new Error(`Telegram adapter configuration missing for job '${jobKey || ''}'`);
}
const { token, chatId, messageThreadId } = adapterCfg.fields;
if (!token || !chatId) {
throw new Error("Telegram 'token' and 'chatId' must be provided in notification config");
}
// Optional Telegram topic/thread support (supergroups)
let message_thread_id;
if (messageThreadId !== undefined && messageThreadId !== null && `${messageThreadId}`.trim() !== '') {
const n = Number(messageThreadId);
if (Number.isInteger(n) && n > 0) {
message_thread_id = n;
} else {
logger.warn(
`Telegram adapter: 'messageThreadId' is invalid ('${messageThreadId}'). It must be a positive integer. Ignoring.`,
);
}
}
const job = getJob(jobKey);
const jobName = job == null ? jobKey : job.name;
//we have to split messages into chunk, because otherwise messages are going to become too big and will fail
const chunks = arrayChunks(newListings, MAX_ENTITIES_PER_CHUNK);
const promises = chunks.map((chunk) => {
let message = `<i>${jobName}</i> (${serviceName}) found <b>${newListings.length}</b> new listings:\n\n`;
message += chunk.map(
(o) =>
`<a href='${o.link}'><b>${shorten(o.title.replace(/\*/g, ''), 45).trim()}</b></a>\n` +
[o.address, o.price, o.size].join(' | ') +
'\n\n',
);
/**
* This is to not break the rate limit. It is to only send 1 message per second
*/
return new Promise((resolve, reject) => {
setTimeout(() => {
fetch(`https://api.telegram.org/bot${token}/sendMessage`, {
method: 'post',
body: JSON.stringify({
chat_id: chatId,
text: message,
parse_mode: 'HTML',
disable_web_page_preview: true,
}),
headers: { 'Content-Type': 'application/json' },
})
.then(() => {
resolve();
})
.catch(() => {
reject();
});
}, RATE_LIMIT_INTERVAL);
const throttledCall = getThrottled(chatId, async function (endpoint, body) {
const res = await fetch(`https://api.telegram.org/bot${token}/${endpoint}`, {
method: 'post',
body: JSON.stringify(body),
headers: { 'Content-Type': 'application/json' },
});
if (!res.ok) {
const errorBody = await res.text();
throw new Error(`API error for '${jobName}'. '${endpoint}' returned ${errorBody}`);
}
return res;
});
if (!Array.isArray(newListings) || newListings.length === 0) return Promise.resolve([]);
const promises = newListings.map(async (o) => {
const img = normalizeImageUrl(o.image);
const textPayload = {
chat_id: chatId,
text: buildText(jobName, serviceName, o),
parse_mode: 'HTML',
disable_web_page_preview: true,
...(message_thread_id ? { message_thread_id } : {}),
};
if (!img) {
return await throttledCall('sendMessage', textPayload).catch(async (e) => {
logger.error(`Error sending message to Telegram: ${e.message}`);
});
}
return await throttledCall('sendPhoto', {
chat_id: chatId,
photo: img,
caption: buildCaption(jobName, serviceName, o),
parse_mode: 'HTML',
...(message_thread_id ? { message_thread_id } : {}),
}).catch(async (e) => {
logger.error(`Error sending photo to Telegram and use a fallback: ${e.message}`);
return await throttledCall('sendMessage', textPayload).catch((e) => {
logger.error(`Error sending message to Telegram: ${e.message}`);
throw e;
});
});
});
return Promise.all(promises);
};
/**
* Telegram notification adapter configuration schema.
* @type {{id:string,name:string,readme:string,description:string,fields:{token:{type:string,label:string,description:string},chatId:{type:string,label:string,description:string},messageThreadId?:{type:string,label:string,description:string}}}}
*/
export const config = {
id: 'telegram',
name: 'Telegram',
@@ -74,5 +213,12 @@ export const config = {
label: 'Chat Id',
description: 'The chat id to send messages to you.',
},
messageThreadId: {
type: 'text',
optional: true,
label: 'Message Thread Id (optional)',
description:
'Optional: The topic/thread id within a supergroup to post into (Telegram message_thread_id). Provide a positive integer.',
},
},
};

View File

@@ -1,13 +1,55 @@
### Telegram Adapter
For Telegram, you need to create a Bot. This is pretty easy. Open [this](https://telegram.me/BotFather) url on your smartphone and follow the instructions.
A telegram bot is not allowed to send messages directly to a user, you as a user need to first contact the bot to get a chatId.
After the user has send a message to your bot the first time, you can gather the chatId like this:
Use this adapter to send notifications to Telegram via a bot. You will need:
- A Telegram Bot token (from BotFather)
- A chat ID (where messages will be sent)
- Optionally: a thread ID if you want to post into a specific forum topic in a group
```
curl -X GET https://api.telegram.org/bot{YOUR_TELEGRAM_TOKEN}/getUpdates
```
A more detailed list of instructions can be found here [https://core.telegram.org/bots#botfather](https://core.telegram.org/bots#botfather)
#### Create a bot
Create a bot with BotFather: open https://telegram.me/BotFather on your phone or in Telegram Desktop and follow the instructions to get your bot token.
#### Getting the chat ID
A Telegram bot cannot message a user first; you must create a conversation (or add the bot to a group/channel) so Telegram assigns a chat the bot can access.
Steps:
1. Start a chat with your bot in Telegram (or add the bot to your group/supergroup/channel) and send any message.
2. Fetch recent updates from the Bot API:
```
curl -X GET "https://api.telegram.org/bot{YOUR_TELEGRAM_TOKEN}/getUpdates"
```
3. In the JSON response, find the message that you just sent and read `message.chat.id`. That value is your `chatId`.
- Private chats: `chat.id` is a positive number
- Groups/supergroups: `chat.id` is a negative number
Keep your bot token secret. If `getUpdates` returns an empty list, send a new message and try again, or make sure your bots privacy settings allow it to see group messages when used in groups.
#### Getting the thread ID (this is optional to be used for forum topics)
If you want messages to appear inside a specific forum topic of a supergroup with Topics enabled, you also need a thread ID. In the Telegram Bot API this is called `message_thread_id`.
When you need it:
- Required only for supergroups with Topics enabled when targeting a topic
- Not used for private chats, basic groups without Topics, or channels
Steps to obtain it:
1. In your supergroup, enable Topics (Group settings → Manage group → Topics → Enable). Now add a new topic.
2. Add your created bot to the topic. (Click on the bot and on "Add to group")
3. Open the desired topic (or create a new one) and send any message inside that topic.
4. Call `getUpdates` again:
```
curl -X GET "https://api.telegram.org/bot{YOUR_TELEGRAM_TOKEN}/getUpdates"
```
4. In the update for the message you sent inside the topic, read `message.message_thread_id`. That number is your `threadId` for this topic.
Example (truncated):
```
{
"message": {
"chat": { "id": -1001234567890, "type": "supergroup" },
"message_thread_id": 42,
"text": "hello from the topic"
}
}
```
Use `chat.id` as `chatId` and `message_thread_id` as `threadId` in your configuration.
More details about bots and BotFather: https://core.telegram.org/bots#botfather

View File

@@ -0,0 +1,123 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
<title>Listings</title>
<style type="text/css">
body { margin:0; padding:0; background:#000000; }
table { border-collapse:collapse; }
img { border:0; outline:none; text-decoration:none; display:block; }
a { text-decoration:none; }
.container { width:100%; max-width:640px; margin:0 auto; }
.card { background:#111111; border:1px solid #222222; border-radius:8px; overflow:hidden; }
.divider { height:2px; line-height:2px; font-size:0; background:#00dc73; }
.h1 { font:700 18px/1.4 Arial, Helvetica, sans-serif; color:#ffffff; margin:0; }
.h2 { font:700 16px/1.4 Arial, Helvetica, sans-serif; color:#ffffff; margin:0; }
.p { font:400 14px/1.6 Arial, Helvetica, sans-serif; color:#d9d9d9; margin:0; }
.meta { font:400 13px/1.5 Arial, Helvetica, sans-serif; color:#bfbfbf; }
.btn { background:#00dc73; color:#0b0b0b; font:700 14px/1 Arial, Helvetica, sans-serif; padding:12px 18px; border-radius:6px; display:inline-block; }
.sp-8 { height:8px; line-height:8px; font-size:0; }
.sp-12 { height:12px; line-height:12px; font-size:0; }
.sp-16 { height:16px; line-height:16px; font-size:0; }
.sp-20 { height:20px; line-height:20px; font-size:0; }
.sp-24 { height:24px; line-height:24px; font-size:0; }
@media screen and (max-width:480px){
.container { width:100% !important; }
.stack { display:block !important; width:100% !important; }
}
</style>
</head>
<body style="background:#000000;">
<table role="presentation" width="100%" bgcolor="#000000">
<tr>
<td align="center">
<table role="presentation" class="container" width="640">
<tr><td class="sp-20"></td></tr>
<tr>
<td align="center">
<h1 class="h1" style="text-align:center;">
Service {{serviceName}} found {{numberOfListings}} new listings
</h1>
</td>
</tr>
<tr><td class="sp-12"></td></tr>
<tr><td class="divider"></td></tr>
<tr><td class="sp-16"></td></tr>
{{#each listings}}
<tr>
<td>
<table role="presentation" class="card" width="100%">
{{#if this.hasImage}}
<tr>
<td>
<a href="{{this.link}}" target="_blank">
<img src="cid:{{this.imageCid}}" alt="{{this.title}}" width="640"
style="width:100%; height:auto; background:#1a1a1a;" />
</a>
</td>
</tr>
{{/if}}
<tr>
<td style="padding:16px 18px 0 18px;">
<a href="{{this.link}}" target="_blank" style="color:#ffffff;">
<h2 class="h2">{{this.title}}</h2>
</a>
</td>
</tr>
<tr><td class="sp-8"></td></tr>
<tr>
<td style="padding:0 18px;">
<table role="presentation" width="100%">
<tr>
<td class="stack" style="vertical-align:top; width:50%; padding-right:8px;">
<p class="meta"><strong>Price</strong><br/>{{#if this.price}}{{this.price}}{{else}}unknown{{/if}}</p>
</td>
<td class="stack" style="vertical-align:top; width:50%; padding-left:8px;">
<p class="meta"><strong>Size</strong><br/>{{#if this.size}}{{this.size}}{{else}}unknown{{/if}}</p>
</td>
</tr>
<tr><td class="sp-8"></td><td class="sp-8"></td></tr>
<tr>
<td colspan="2">
<p class="meta"><strong>Address</strong><br/>{{#if this.address}}{{this.address}}{{else}}unknown{{/if}}</p>
</td>
</tr>
</table>
</td>
</tr>
<tr><td class="sp-16"></td></tr>
<tr>
<td align="left" style="padding:0 18px 18px 18px;">
<a href="{{this.link}}" class="btn" target="_blank">View Listing</a>
</td>
</tr>
</table>
</td>
</tr>
<tr><td class="sp-24"></td></tr>
{{/each}}
<tr><td class="divider"></td></tr>
<tr><td class="sp-16"></td></tr>
<tr>
<td align="center">
<p class="p" style="color:#9f9f9f;">Powered by Fredy</p>
</td>
</tr>
<tr><td class="sp-20"></td></tr>
</table>
</td>
</tr>
</table>
</body>
</html>

View File

@@ -1,237 +1,131 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html data-editor-version="2" class="sg-campaigns" xmlns="http://www.w3.org/1999/xhtml"><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, minimum-scale=1, maximum-scale=1">
<!--[if !mso]><!-->
<meta http-equiv="X-UA-Compatible" content="IE=Edge">
<!--<![endif]-->
<!--[if (gte mso 9)|(IE)]>
<xml>
<o:OfficeDocumentSettings>
<o:AllowPNG/>
<o:PixelsPerInch>96</o:PixelsPerInch>
</o:OfficeDocumentSettings>
</xml>
<![endif]-->
<!--[if (gte mso 9)|(IE)]>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
<title>Fredy found some new listings</title>
<style type="text/css">
body {width: 600px;margin: 0 auto;}
table {border-collapse: collapse;}
table, td {mso-table-lspace: 0pt;mso-table-rspace: 0pt;}
img {-ms-interpolation-mode: bicubic;}
</style>
<![endif]-->
<style type="text/css">
body, p, div {
font-family: arial,helvetica,sans-serif;
font-size: 14px;
}
body {
color: #000000;
}
body a {
color: #42ee99;
text-decoration: none;
}
p { margin: 0; padding: 0; }
table.wrapper {
width:100% !important;
table-layout: fixed;
-webkit-font-smoothing: antialiased;
-webkit-text-size-adjust: 100%;
-moz-text-size-adjust: 100%;
-ms-text-size-adjust: 100%;
}
img.max-width {
max-width: 100% !important;
}
.column.of-2 {
width: 50%;
}
.column.of-3 {
width: 33.333%;
}
.column.of-4 {
width: 25%;
}
@media screen and (max-width:480px) {
.preheader .rightColumnContent,
.footer .rightColumnContent {
text-align: left !important;
}
.preheader .rightColumnContent div,
.preheader .rightColumnContent span,
.footer .rightColumnContent div,
.footer .rightColumnContent span {
text-align: left !important;
}
.preheader .rightColumnContent,
.preheader .leftColumnContent {
font-size: 80% !important;
padding: 5px 0;
}
table.wrapper-mobile {
width: 100% !important;
table-layout: fixed;
}
img.max-width {
height: auto !important;
max-width: 100% !important;
}
a.bulletproof-button {
display: block !important;
width: auto !important;
font-size: 80%;
padding-left: 0 !important;
padding-right: 0 !important;
}
.columns {
width: 100% !important;
}
.column {
display: block !important;
width: 100% !important;
padding-left: 0 !important;
padding-right: 0 !important;
margin-left: 0 !important;
margin-right: 0 !important;
}
body { margin:0; padding:0; background:#000000; }
table { border-collapse:collapse; }
img { border:0; outline:none; text-decoration:none; display:block; }
a { text-decoration:none; }
.container { width:100%; max-width:640px; margin:0 auto; }
.card { background:#111111; border:1px solid #222222; border-radius:8px; overflow:hidden; }
.divider { height:2px; line-height:2px; font-size:0; background:#00dc73; }
.h1 { font:700 18px/1.4 Arial, Helvetica, sans-serif; color:#ffffff; margin:0; }
.h2 { font:700 16px/1.4 Arial, Helvetica, sans-serif; color:#ffffff; margin:0; }
.p { font:400 14px/1.6 Arial, Helvetica, sans-serif; color:#d9d9d9; margin:0; }
.meta { font:400 13px/1.5 Arial, Helvetica, sans-serif; color:#bfbfbf; }
.btn { background:#00dc73; color:#0b0b0b; font:700 14px/1 Arial, Helvetica, sans-serif; padding:12px 18px; border-radius:6px; display:inline-block; }
.sp-8 { height:8px; line-height:8px; font-size:0; }
.sp-12 { height:12px; line-height:12px; font-size:0; }
.sp-16 { height:16px; line-height:16px; font-size:0; }
.sp-20 { height:20px; line-height:20px; font-size:0; }
.sp-24 { height:24px; line-height:24px; font-size:0; }
@media screen and (max-width:480px){
.container { width:100% !important; }
.stack { display:block !important; width:100% !important; }
}
</style>
<!--user entered Head Start-->
<!--End Head user entered-->
</head>
<body>
<center class="wrapper" data-link-color="#42ee99" data-body-style="font-size:14px; font-family:arial,helvetica,sans-serif; color:#000000; background-color:#000000;">
<div class="webkit">
<table cellpadding="0" cellspacing="0" border="0" width="100%" class="wrapper" bgcolor="#000000">
<tbody><tr>
<td valign="top" bgcolor="#000000" width="100%">
<table width="100%" role="content-container" class="outer" align="center" cellpadding="0" cellspacing="0" border="0">
<tbody><tr>
<td width="100%">
<table width="100%" cellpadding="0" cellspacing="0" border="0">
<tbody><tr>
<body style="background:#000000;">
<table role="presentation" width="100%" bgcolor="#000000">
<tr>
<td align="center">
<table role="presentation" class="container" width="640">
<tr><td class="sp-20"></td></tr>
<tr>
<td align="center">
<table role="presentation" width="100%">
<tr>
<td align="center">
<h1 class="h1" style="text-align:center;">
Service {{serviceName}} found {{numberOfListings}} new listings
</h1>
</td>
</tr>
<tr><td class="sp-12"></td></tr>
<tr><td class="divider"></td></tr>
<tr><td class="sp-16"></td></tr>
</table>
</td>
</tr>
{{#each listings}}
<tr>
<td>
<table role="presentation" class="card" width="100%">
{{#if this.hasImage}}
<tr>
<td>
<!--[if mso]>
<center>
<table><tr><td width="600">
<![endif]-->
<table width="100%" cellpadding="0" cellspacing="0" border="0" style="width:100%; max-width:600px;" align="center">
<tbody><tr>
<td role="modules-container" style="padding:0px 0px 0px 0px; color:#000000; text-align:left;" bgcolor="#FFFFFF" width="100%" align="left"><table class="module preheader preheader-hide" role="module" data-type="preheader" border="0" cellpadding="0" cellspacing="0" width="100%" style="display: none !important; mso-hide: all; visibility: hidden; opacity: 0; color: transparent; height: 0; width: 0;">
</table><table class="module" role="module" data-type="spacer" border="0" cellpadding="0" cellspacing="0" width="100%" style="table-layout: fixed;" data-muid="vB9TDziyvx65CC2nx3oyRH">
<tbody><tr>
<td style="padding:0px 0px 20px 0px;" role="module-content" bgcolor="#000000">
</td>
</tr>
</tbody></table><table class="wrapper" role="module" data-type="image" border="0" cellpadding="0" cellspacing="0" width="100%" style="table-layout: fixed;" data-muid="uXsDxMnn1bRMmDcX8NB6rW">
</table><table class="module" role="module" data-type="text" border="0" cellpadding="0" cellspacing="0" width="100%" style="table-layout: fixed;" data-muid="hL6wjQ2qknNd5qDwT1p7Up">
<tbody><tr>
<td style="background-color:#000000; padding:10px 20px 10px 20px; line-height:40px; text-align:justify;" height="100%" valign="top" bgcolor="#000000"><div><h1 style="text-align: center"><span style="color: #ffffff; font-size: 14px; font-family: verdana,geneva,sans-serif"><strong>Service {{serviceName}} found {{numberOfListings}} new listing(s)!</strong></span></h1><div></div></div></td>
</tr>
</tbody></table>
<table border="0" cellpadding="0" cellspacing="0" align="center" width="100%" height="1px" style="line-height:3px; font-size:3px;">
<tbody><tr>
<td style="padding:0px 0px 1px 0px;" bgcolor="#42ee99"></td>
</tr>
</tbody></table>
</td>
</tr>
</tbody>
</table>
<table class="module" role="module" data-type="text" border="0" cellpadding="0" cellspacing="0" width="100%" style="table-layout: fixed;" data-muid="qk51Jjn4bm3rn2Yb31Dxzb">
<tbody>
{{#each listings}}
<tr>
<td style="padding:50px 50px 10px 50px; line-height:22px; text-align:center; color:white" bgcolor="#000000" height="100%" valign="top">
<div>
<span style="font-size: 12px; font-family: verdana,geneva,sans-serif; color: white;"><b>{{this.title}}</b></span>
<br/>
<span style="font-size: 12px; font-family: verdana,geneva,sans-serif; color: white;">Size: {{#if this.size}}{{this.size}}{{else}}unknown{{/if}}</span>
<br/>
<span style="font-size: 12px; font-family: verdana,geneva,sans-serif; color: white;">Price: {{#if this.price}}{{this.price}}{{else}}unknown{{/if}}</span>
<br/>
<span style="font-size: 12px; font-family: verdana,geneva,sans-serif; color: white;">{{#if this.address}}{{this.address}}{{else}}unknown{{/if}}</span>
<br/>
<a href="{{this.link}}" target="_blank" style="color:#00dc73; font-size:13px">{{this.link}}</a>
<br/>
<span style="color: white;">---------------------------</span>
</div>
</td>
</tr>
{{/each}}
</tbody>
</table>
<table class="module" role="module" data-type="spacer" border="0" cellpadding="0" cellspacing="0" width="100%" style="table-layout: fixed;" data-muid="2ga5f7koD5ApvUfnqUK6aT">
<tbody><tr>
<td style="padding:0px 0px 30px 0px;" role="module-content" bgcolor="#000000">
</td>
</tr>
</tbody></table>
<table class="module" role="module" data-type="divider" border="0" cellpadding="0" cellspacing="0" width="100%" style="table-layout: fixed;" data-muid="c3nRrjMndqXf1snYDFPSF9">
<tbody><tr>
<td style="padding:0px 0px 0px 0px;" role="module-content" height="100%" valign="top" bgcolor="#000000">
<table border="0" cellpadding="0" cellspacing="0" align="center" width="100%" height="2px" style="line-height:1px; font-size:2px;">
<tbody><tr>
<td style="padding:0px 0px 2px 0px;" bgcolor="#42ee99"></td>
</tr>
</tbody></table>
</td>
</tr>
</tbody>
</table>
<table class="module" role="module" data-type="spacer" border="0" cellpadding="0" cellspacing="0" width="100%" style="table-layout: fixed;" data-muid="pa9PeYjCEFyByuP5878Sd2">
<tbody><tr>
<td style="padding:0px 0px 30px 0px;" role="module-content" bgcolor="#000000">
</td>
</tr>
</tbody></table>
<table class="module" role="module" data-type="social" align="center" border="0" cellpadding="0" cellspacing="0" width="100%" style="table-layout: fixed;" data-muid="n7FceQWVnLmounEt32B1gj">
<tbody>
<tr>
<td valign="top" style="padding:0px 0px 0px 0px; font-size:6px; line-height:10px; background-color:#000000;" align="center">
<table align="center">
<tbody>
<tr><td style="padding: 0px 5px;">
<a href="https://github.com/orangecoding/fredy" target="_blank" alt="Fredy" title="Powered by Fredy" style="color:#00dc73; font-size:17px">
Powered by Fredy
</a>
</td></tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table></table><table class="module" role="module" data-type="spacer" border="0" cellpadding="0" cellspacing="0" width="100%" style="table-layout: fixed;" data-muid="35xFa9abxGTBYt9yR9BeQ2">
<tbody><tr>
<td style="padding:0px 0px 30px 0px;" role="module-content" bgcolor="#000000">
<a href="{{this.link}}" target="_blank">
<img src="{{this.image}}" alt="{{this.title}}" width="640" style="width:100%;height:auto;background:#1a1a1a;" />
</a>
</td>
</tr>
{{/if}}
<tr>
<td style="padding:16px 18px 0 18px;">
<a href="{{this.link}}" target="_blank" style="color:#ffffff;">
<h2 class="h2">{{this.title}}</h2>
</a>
</td>
</tr>
</tbody></table></td>
</tr>
</tbody></table>
<!--[if mso]>
</td>
</tr>
</table>
</center>
<tr><td class="sp-8"></td></tr>
<tr>
<td style="padding:0 18px;">
<table role="presentation" width="100%">
<tr>
<td class="stack" style="vertical-align:top; width:50%; padding-right:8px;">
<p class="meta"><strong>Price</strong><br/>{{#if this.price}}{{this.price}}{{else}}unknown{{/if}}</p>
</td>
<td class="stack" style="vertical-align:top; width:50%; padding-left:8px;">
<p class="meta"><strong>Size</strong><br/>{{#if this.size}}{{this.size}}{{else}}unknown{{/if}}</p>
</td>
</tr>
<tr><td class="sp-8"></td><td class="sp-8"></td></tr>
<tr>
<td colspan="2">
<p class="meta"><strong>Address</strong><br/>{{#if this.address}}{{this.address}}{{else}}unknown{{/if}}</p>
</td>
</tr>
</table>
</td>
</tr>
<tr><td class="sp-16"></td></tr>
<tr>
<td align="left" style="padding:0 18px 18px 18px;">
<!--[if mso]>
<v:roundrect xmlns:v="urn:schemas-microsoft-com:vml" href="{{this.link}}" arcsize="8%" strokecolor="#00dc73" strokeweight="0" fillcolor="#00dc73" style="height:40px;v-text-anchor:middle;width:180px;">
<w:anchorlock/>
<center style="color:#0b0b0b;font-family:Arial;font-size:14px;font-weight:bold;">
View Listing
</center>
</v:roundrect>
<![endif]-->
</td>
</tr>
</tbody></table>
</td>
</tr>
</tbody></table>
</td>
</tr>
</tbody></table>
</div>
</center>
<!--[if !mso]><!-- -->
<a href="{{this.link}}" class="btn" target="_blank">View Listing</a>
<!--<![endif]-->
</td>
</tr>
</table>
</td>
</tr>
<tr><td class="sp-24"></td></tr>
{{/each}}
</body></html>
<tr><td class="divider"></td></tr>
<tr><td class="sp-16"></td></tr>
<tr>
<td align="center">
<p class="p" style="color:#9f9f9f;">Powered by Fredy</p>
</td>
</tr>
<tr><td class="sp-20"></td></tr>
</table>
</td>
</tr>
</table>
</body>
</html>

View File

@@ -1,3 +1,8 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import fs from 'fs';
const path = './adapter';
@@ -6,7 +11,7 @@ const adapter = await Promise.all(
fs
.readdirSync('./lib/notification/adapter')
.filter((file) => file.endsWith('.js'))
.map(async (integPath) => await import(`${path}/${integPath}`))
.map(async (integPath) => await import(`${path}/${integPath}`)),
);
if (adapter.length === 0) {

Some files were not shown because too many files have changed in this diff Show More