Compare commits

...

27 Commits

Author SHA1 Message Date
orangecoding
7deffc64af next release version 2025-09-18 20:48:49 +02:00
orangecoding
d1dad7fd3b adding new unique index, adding button to start now 2025-09-18 20:48:25 +02:00
Christian Kellner
4f79c5cba2 replacing rematch with zustand (#180)
* replacing rematch with zustand

* upgrading dependencies

* next release version
2025-09-18 20:09:11 +02:00
orangecoding
28e885f6c7 fixing migration checksum 2025-09-18 18:42:19 +02:00
orangecoding
1d99fc95f7 using cron to run demo cleanup every day at midnight 2025-09-18 18:04:49 +02:00
orangecoding
28f0a167e6 fixing docker migration path 2025-09-18 17:28:30 +02:00
Christian Kellner
8d95f052c6 Migrate to SQLite (#174)
* Migrating Fredy from LowDb to SqLite 🎉

* adding new sql migration system for future sql migrations

* adding setting to change  sqlite path for db files

* create migration plan for graceful migration lowdb -> sqlite

* Improving Documentation

* adding test for sqliteconnection

* upgrading dependencies

* making nodejs 22 as min version

* improve scraper

* adding overwrite ability for db migra
2025-09-18 15:38:23 +02:00
orangecoding
18fdbd761a next release version 2025-09-17 09:12:45 +02:00
Iaroslav Postovalov
027e7d70ed Update SQLite adapter: configurable database path (#169) 2025-09-17 09:12:04 +02:00
Christian Kellner
de119c9199 Update logger.js 2025-09-14 15:46:31 +02:00
orangecoding
ce7f0bca9f next release version 2025-09-14 10:40:41 +02:00
orangecoding
ae1c4d936b do not log debug on production 2025-09-14 10:40:18 +02:00
orangecoding
d01a1a94d0 Merge branch 'master' of github.com:orangecoding/fredy 2025-09-14 10:32:52 +02:00
orangecoding
bda4212249 improve logging 2025-09-14 10:32:39 +02:00
Christian Kellner
694809fedf Using white fredy logo on dark background 2025-09-13 22:20:50 +02:00
Christian Kellner
3cd1893b51 Update Jetbrains logo to use the correct one on dark background 2025-09-13 22:16:16 +02:00
orangecoding
21415dcff3 using winston logger 2025-09-13 18:57:56 +02:00
orangecoding
e868cdce86 Merge branch 'master' of github.com:orangecoding/fredy 2025-09-13 17:06:30 +02:00
orangecoding
d66dc2cd93 improve tracking 2025-09-13 17:06:18 +02:00
Christian Kellner
5e0405f1ec Update README.md 2025-09-12 18:47:10 +02:00
orangecoding
251de1e42d next release version 2025-09-12 13:48:05 +02:00
orangecoding
edc91291b6 fixing telegram 2025-09-12 13:45:54 +02:00
orangecoding
ac0ea64c07 remove unnecessary logging 2025-09-12 13:41:08 +02:00
orangecoding
9f7506a1b3 Merge branch 'master' of github.com:orangecoding/fredy 2025-09-12 13:39:15 +02:00
orangecoding
85cea66051 improving tracking. now using internal tracking 2025-09-12 13:38:53 +02:00
Christian Kellner
05c2df917c Adding link to fredy demo 2025-09-12 13:00:43 +02:00
Christian Kellner
4ad2895eec Update docker command 2025-09-10 11:31:49 +02:00
67 changed files with 2171 additions and 914 deletions

View File

@@ -13,7 +13,7 @@ jobs:
- uses: actions/setup-node@v4
with:
node-version: 20
node-version: 22
cache: 'yarn'
- name: Install dependencies

View File

@@ -15,7 +15,7 @@ jobs:
- uses: actions/setup-node@v4
with:
node-version: 20
node-version: 22
cache: 'yarn'
- run: yarn install

3
.gitignore vendored
View File

@@ -1,6 +1,7 @@
node_modules/
ui/public/
db/
db/*.json
db/*.db*
npm-debug.log
.DS_Store
.idea

View File

@@ -1,3 +1,20 @@
<p align="center">
<a href="https://fredy.orange-coding.net/">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://github.com/orangecoding/fredy/blob/master/doc/logo_white.png" width="400">
<source media="(prefers-color-scheme: light)" srcset="https://github.com/orangecoding/fredy/blob/master/doc/logo.png" width="400">
<img alt="Jetbrains Open Source" src="https://github.com/orangecoding/fredy/blob/master/doc/logo.png">
</picture>
</a>
</p>
![Tests](https://github.com/orangecoding/fredy/actions/workflows/test.yml/badge.svg)
[![Docker](https://github.com/orangecoding/fredy/actions/workflows/docker.yml/badge.svg)](https://github.com/orangecoding/fredy/actions/workflows/docker.yml)
![Source](https://github.com/orangecoding/fredy/actions/workflows/check_source.yml/badge.svg)
![Docker Pulls](https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fghcr-badge.elias.eu.org%2Fapi%2Forangecoding%2Ffredy%2Ffredy&query=%24.downloadCount&label=Docker%20Pulls)
# Fredy 🏡 Your Self-Hosted Real Estate Finder for Germany
Finding an apartment or house in Germany can be stressful and
@@ -11,12 +28,7 @@ With a modern architecture, Fredy provides a **clean Web UI**, removes
duplicates across platforms, and stores results so you never see the
same listing twice.
<img src="https://github.com/orangecoding/fredy/blob/master/doc/logo.png" width="400">
![Tests](https://github.com/orangecoding/fredy/actions/workflows/test.yml/badge.svg)
[![Docker](https://github.com/orangecoding/fredy/actions/workflows/docker.yml/badge.svg)](https://github.com/orangecoding/fredy/actions/workflows/docker.yml)
![Source](https://github.com/orangecoding/fredy/actions/workflows/check_source.yml/badge.svg)
![Docker Pulls](https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fghcr-badge.elias.eu.org%2Fapi%2Forangecoding%2Ffredy%2Ffredy&query=%24.downloadCount&label=Docker%20Pulls)
------------------------------------------------------------------------
@@ -41,7 +53,17 @@ I maintain Fredy and other open-source projects in my free time.\
If you find it useful, consider supporting the project 💙
Fredy is proudly backed by the **JetBrains Open Source Support Program**.
[<img src="https://resources.jetbrains.com/storage/products/company/brand/logos/jetbrains.svg" alt="JetBrains" width="120"/>](https://jb.gg/OpenSourceSupport)
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://www.jetbrains.com/company/brand/img/logo_jb_dos_3.svg">
<source media="(prefers-color-scheme: light)" srcset="https://resources.jetbrains.com/storage/products/company/brand/logos/jetbrains.svg">
<img alt="Jetbrains Open Source" src="https://resources.jetbrains.com/storage/products/company/brand/logos/jetbrains.svg">
</picture>
------------------------------------------------------------------------
## 👨‍🏫 Demo
You can try out Fredy here: [Fredy Demo](https://fredy-demo.orange-coding.net/)
------------------------------------------------------------------------
@@ -53,7 +75,11 @@ Fredy is proudly backed by the **JetBrains Open Source Support Program**.
> In order to start Fredy, you must provide a config.json. As a start, use the one in this repo: https://github.com/orangecoding/fredy/blob/master/conf/config.json
``` bash
docker run -d --name fredy -v fredy_conf:/conf -p 9998:9998 ghcr.io/orangecoding/fredy:master
docker run -d --name fredy \
-v fredy_conf:/conf \
-v fredy_db:/db \
-p 9998:9998 \
ghcr.io/orangecoding/fredy:master
```
Logs:
@@ -64,7 +90,7 @@ docker logs fredy -f
### Manual (Node.js)
- Requirement: **Node.js 20 or higher**
- Requirement: **Node.js 22 or higher**
- Install dependencies and start:
``` bash
@@ -128,7 +154,7 @@ Immoscout has implemented advanced bot detection. In order to work around this,
Fredy is completely free (and will always remain free). However, it would be a huge help if youd allow me to collect some analytical data.
Before you freak out, let me explain...
If you agree, Fredy will send a ping to my Mixpanel project each time it runs.
If you agree, Fredy will send a ping once every 6 hours to my internal tracking project (Will be open sourced soon).
The data includes: names of active adapters/providers, OS, architecture, Node version, and language. The information is entirely anonymous and helps me understand which adapters/providers are most frequently used.</p>
**Thanks**🤘

View File

@@ -1 +1 @@
{"interval":"60","port":9998,"workingHours":{"from":"","to":""},"demoMode":false,"analyticsEnabled":null}
{"interval":"60","port":9998,"workingHours":{"from":"","to":""},"demoMode":false,"analyticsEnabled":null,"sqlitepath":"/db"}

0
db/.gitkeep Normal file
View File

105
index.js
View File

@@ -1,62 +1,79 @@
import fs from 'fs';
import path from 'path';
import { config } from './lib/utils.js';
import * as similarityCache from './lib/services/similarity-check/similarityCache.js';
import { setLastJobExecution } from './lib/services/storage/listingsStorage.js';
import * as jobStorage from './lib/services/storage/jobStorage.js';
import FredyRuntime from './lib/FredyRuntime.js';
import { duringWorkingHoursOrNotSet } from './lib/utils.js';
import './lib/api/api.js';
import { track } from './lib/services/tracking/Tracker.js';
import { handleDemoUser } from './lib/services/storage/userStorage.js';
import { runMigrations } from './lib/services/storage/migrations/migrate.js';
import { ensureDemoUserExists, ensureAdminUserExists } from './lib/services/storage/userStorage.js';
import { cleanupDemoAtMidnight } from './lib/services/demoCleanup.js';
//if db folder does not exist, ensure to create it before loading anything else
if (!fs.existsSync('./db')) {
fs.mkdirSync('./db');
import { initTrackerCron } from './lib/services/tracking/Tracker-Cron.js';
import logger from './lib/services/logger.js';
import { bus } from './lib/services/events/event-bus.js';
// Ensure sqlite directory exists before loading anything else (based on config.sqlitepath)
const rawDir = config.sqlitepath || '/db';
const relDir = rawDir.startsWith('/') ? rawDir.slice(1) : rawDir;
const absDir = path.isAbsolute(relDir) ? relDir : path.join(process.cwd(), relDir);
if (!fs.existsSync(absDir)) {
fs.mkdirSync(absDir, { recursive: true });
}
const path = './lib/provider';
const provider = fs.readdirSync(path).filter((file) => file.endsWith('.js'));
// Run DB migrations once at startup and block until finished
await runMigrations();
const providersPath = './lib/provider';
const provider = fs.readdirSync(providersPath).filter((file) => file.endsWith('.js'));
//assuming interval is always in minutes
const INTERVAL = config.interval * 60 * 1000;
/* eslint-disable no-console */
console.log(`Started Fredy successfully. Ui can be accessed via http://localhost:${config.port}`);
// Initialize API only after migrations completed
await import('./lib/api/api.js');
if (config.demoMode) {
console.info('Running in demo mode');
logger.info('Running in demo mode');
cleanupDemoAtMidnight();
}
/* eslint-enable no-console */
logger.info(`Started Fredy successfully. Ui can be accessed via http://localhost:${config.port}`);
const fetchedProvider = await Promise.all(
provider.filter((provider) => provider.endsWith('.js')).map(async (pro) => import(`${path}/${pro}`)),
provider.filter((provider) => provider.endsWith('.js')).map(async (pro) => import(`${providersPath}/${pro}`)),
);
handleDemoUser();
ensureAdminUserExists();
ensureDemoUserExists();
await initTrackerCron();
setInterval(
(function exec() {
const isDuringWorkingHoursOrNotSet = duringWorkingHoursOrNotSet(config, Date.now());
if (!config.demoMode) {
if (isDuringWorkingHoursOrNotSet) {
track();
config.lastRun = Date.now();
jobStorage
.getJobs()
.filter((job) => job.enabled)
.forEach((job) => {
job.provider
.filter((p) => fetchedProvider.find((fp) => fp.metaInformation.id === p.id) != null)
.forEach(async (prov) => {
const pro = fetchedProvider.find((fp) => fp.metaInformation.id === prov.id);
pro.init(prov, job.blacklist);
await new FredyRuntime(pro.config, job.notificationAdapter, prov.id, job.id, similarityCache).execute();
setLastJobExecution(job.id);
});
});
} else {
/* eslint-disable no-console */
console.debug('Working hours set. Skipping as outside of working hours.');
/* eslint-enable no-console */
}
bus.on('jobs:runAll', () => {
logger.debug('Running Fredy Job manually');
execute();
});
const execute = () => {
const isDuringWorkingHoursOrNotSet = duringWorkingHoursOrNotSet(config, Date.now());
if (!config.demoMode) {
if (isDuringWorkingHoursOrNotSet) {
config.lastRun = Date.now();
jobStorage
.getJobs()
.filter((job) => job.enabled)
.forEach((job) => {
job.provider
.filter((p) => fetchedProvider.find((fp) => fp.metaInformation.id === p.id) != null)
.forEach(async (prov) => {
const pro = fetchedProvider.find((fp) => fp.metaInformation.id === prov.id);
pro.init(prov, job.blacklist);
await new FredyRuntime(pro.config, job.notificationAdapter, prov.id, job.id, similarityCache).execute();
});
});
} else {
logger.debug('Working hours set. Skipping as outside of working hours.');
}
return exec;
})(),
INTERVAL,
);
}
};
setInterval(execute, INTERVAL);
//start once at startup
execute();

View File

@@ -1,8 +1,9 @@
import { NoNewListingsWarning } from './errors.js';
import { setKnownListings, getKnownListings } from './services/storage/listingsStorage.js';
import { storeListings, getKnownListingHashesForJobAndProvider } from './services/storage/listingsStorage.js';
import * as notify from './notification/notify.js';
import Extractor from './services/extractor/extractor.js';
import urlModifier from './services/queryStringMutator.js';
import logger from './services/logger.js';
class FredyRuntime {
/**
@@ -59,7 +60,7 @@ class FredyRuntime {
})
.catch((err) => {
reject(err);
console.error(err);
logger.error(err);
});
});
}
@@ -76,7 +77,9 @@ class FredyRuntime {
}
_findNew(listings) {
const newListings = listings.filter((o) => getKnownListings(this._jobKey, this._providerId)[o.id] == null);
const hashes = getKnownListingHashesForJobAndProvider(this._jobKey, this._providerId) || [];
const newListings = listings.filter((o) => !hashes.includes(o.id));
if (newListings.length === 0) {
throw new NoNewListingsWarning();
}
@@ -92,11 +95,7 @@ class FredyRuntime {
}
_save(newListings) {
const currentListings = getKnownListings(this._jobKey, this._providerId) || {};
newListings.forEach((listing) => {
currentListings[listing.id] = Date.now();
});
setKnownListings(this._jobKey, this._providerId, currentListings);
storeListings(this._jobKey, this._providerId, newListings);
return newListings;
}
@@ -104,9 +103,7 @@ class FredyRuntime {
const filteredList = listings.filter((listing) => {
const similar = this._similarityCache.hasSimilarEntries(listing.title, listing.address);
if (similar) {
/* eslint-disable no-console */
console.debug(`Filtering similar entry for title: ${listing.title} and address ${listing.address}`);
/* eslint-enable no-console */
logger.debug(`Filtering similar entry for title: ${listing.title} and address ${listing.address}`);
}
return !similar;
});
@@ -115,7 +112,7 @@ class FredyRuntime {
}
_handleError(err) {
if (err.name !== 'NoNewListingsWarning') console.error(err);
if (err.name !== 'NoNewListingsWarning') logger.error(err);
}
}

View File

@@ -13,6 +13,7 @@ import files from 'serve-static';
import path from 'path';
import { getDirName } from '../utils.js';
import { demoRouter } from './routes/demoRouter.js';
import logger from '../services/logger.js';
const service = restana();
const staticService = files(path.join(getDirName(), '../ui/public'));
const PORT = config.port || 9998;
@@ -34,7 +35,6 @@ service.use('/api/login', loginRouter);
//this route is unsecured intentionally as it is being queried from the login page
service.use('/api/demo', demoRouter);
/* eslint-disable no-console */
service.start(PORT).then(() => {
console.info(`Started API service on port ${PORT}`);
logger.debug(`Started API service on port ${PORT}`);
});

View File

@@ -1,7 +1,8 @@
import restana from 'restana';
import { config, getDirName, readConfigFromStorage, refreshConfig } from '../../utils.js';
import fs from 'fs';
import { handleDemoUser } from '../../services/storage/userStorage.js';
import { ensureDemoUserExists } from '../../services/storage/userStorage.js';
import logger from '../../services/logger.js';
const service = restana();
const generalSettingsRouter = service.newRouter();
generalSettingsRouter.get('/', async (req, res) => {
@@ -18,9 +19,9 @@ generalSettingsRouter.post('/', async (req, res) => {
const currentConfig = await readConfigFromStorage();
fs.writeFileSync(`${getDirName()}/../conf/config.json`, JSON.stringify({ ...currentConfig, ...settings }));
await refreshConfig();
handleDemoUser();
ensureDemoUserExists();
} catch (err) {
console.error(err);
logger.error(err);
res.send(new Error('Error while trying to write settings.'));
return;
}

View File

@@ -3,9 +3,12 @@ import * as jobStorage from '../../services/storage/jobStorage.js';
import * as userStorage from '../../services/storage/userStorage.js';
import { config } from '../../utils.js';
import { isAdmin } from '../security.js';
import { trackDemoJobCreated } from '../../services/tracking/Tracker.js';
import logger from '../../services/logger.js';
import { bus } from '../../services/events/event-bus.js';
const service = restana();
const jobRouter = service.newRouter();
function doesJobBelongsToUser(job, req) {
const userId = req.session.currentUser;
if (userId == null) {
@@ -17,6 +20,7 @@ function doesJobBelongsToUser(job, req) {
}
return user.isAdmin || job.userId === user.id;
}
jobRouter.get('/', async (req, res) => {
const isUserAdmin = isAdmin(req);
//show only the jobs which belongs to the user (or all of the user is an admin)
@@ -30,6 +34,12 @@ jobRouter.get('/processingTimes', async (req, res) => {
};
res.send();
});
jobRouter.post('/startAll', async (req, res) => {
bus.emit('jobs:runAll');
res.send();
});
jobRouter.post('/', async (req, res) => {
const { provider, notificationAdapter, name, blacklist = [], jobId, enabled } = req.body;
try {
@@ -44,13 +54,8 @@ jobRouter.post('/', async (req, res) => {
});
} catch (error) {
res.send(new Error(error));
console.error(error);
logger.error(error);
}
trackDemoJobCreated({
name,
provider,
adapter: notificationAdapter,
});
res.send();
});
jobRouter.delete('', async (req, res) => {
@@ -64,7 +69,7 @@ jobRouter.delete('', async (req, res) => {
}
} catch (error) {
res.send(new Error(error));
console.error(error);
logger.error(error);
}
res.send();
});
@@ -83,7 +88,7 @@ jobRouter.put('/:jobId/status', async (req, res) => {
}
} catch (error) {
res.send(new Error(error));
console.error(error);
logger.error(error);
}
res.send();
});

View File

@@ -3,6 +3,7 @@ import * as userStorage from '../../services/storage/userStorage.js';
import * as hasher from '../../services/security/hash.js';
import { config } from '../../utils.js';
import { trackDemoAccessed } from '../../services/tracking/Tracker.js';
import logger from '../../services/logger.js';
const service = restana();
const loginRouter = service.newRouter();
loginRouter.get('/user', async (req, res) => {
@@ -27,7 +28,7 @@ loginRouter.post('/', async (req, res) => {
}
if (user.password === hasher.hash(password)) {
if (config.demoMode) {
trackDemoAccessed();
await trackDemoAccessed();
}
req.session.currentUser = user.id;
@@ -35,7 +36,7 @@ loginRouter.post('/', async (req, res) => {
res.send(200);
return;
} else {
console.error(`User ${username} tried to login, but password was wrong.`);
logger.error(`User ${username} tried to login, but password was wrong.`);
}
res.send(401);
});

View File

@@ -4,4 +4,6 @@ export const DEFAULT_CONFIG = {
workingHours: { from: '', to: '' },
demoMode: false,
analyticsEnabled: null,
// Default path for sqlite storage directory. Interpreted relative to project root.
sqlitepath: '/db',
};

View File

@@ -5,6 +5,7 @@ import Handlebars from 'handlebars';
import fetch from 'node-fetch';
import { markdown2Html } from '../../services/markdown.js';
import { getDirName, normalizeImageUrl } from '../../utils.js';
import logger from '../../services/logger.js';
const __dirname = getDirName();
const template = fs.readFileSync(path.resolve(__dirname + '/notification/emailTemplate/template.hbs'), 'utf8');
@@ -24,7 +25,7 @@ const toBase64 = async (url) => {
const ab = await res.arrayBuffer();
return Buffer.from(ab).toString('base64');
} catch (error) {
console.error(`Error fetching image from ${url}:`, error.message);
logger.error(`Error fetching image from ${url}:`, error.message);
throw error;
}
};
@@ -62,7 +63,7 @@ const mapListingsWithCid = async (serviceName, jobKey, listings) => {
item.hasImage = true;
item.imageCid = cid;
} catch (error) {
console.warn(`Skipping image for listing ${i} due to error: ${error.message}`);
logger.warn(`Skipping image for listing ${i} due to error: ${error.message}`);
}
}

View File

@@ -1,7 +1,18 @@
import { markdown2Html } from '../../services/markdown.js';
import Database from 'better-sqlite3';
export const send = ({ serviceName, newListings, jobKey }) => {
const db = new Database('db/listings.db');
import path from 'path';
import fs from 'fs';
export const send = ({ serviceName, newListings, jobKey, notificationConfig }) => {
const sqliteConfig = notificationConfig.find((adapter) => adapter.id === config.id);
const dbPath = sqliteConfig?.fields?.dbPath || 'db/listings.db';
const dbDir = path.dirname(dbPath);
if (!fs.existsSync(dbDir)) {
fs.mkdirSync(dbDir, { recursive: true });
}
const db = new Database(dbPath);
const fields = [
'serviceName',
'jobKey',
@@ -30,8 +41,16 @@ export const send = ({ serviceName, newListings, jobKey }) => {
};
export const config = {
id: 'sqlite',
name: 'Sqlite',
description: 'This adapter stores listings in a local sqlite3 database.',
config: {},
name: 'SQLite',
description: 'This adapter stores listings in a local SQLite 3 database.',
fields: {
dbPath: {
type: 'text',
label: 'Database Path',
description:
'Path to the SQLite database file (e.g., db/listings.db). If not specified, defaults to db/listings.db',
placeholder: 'db/listings.db',
},
},
readme: markdown2Html('lib/notification/adapter/sqlite.md'),
};

View File

@@ -1,9 +1,9 @@
### Sqlite Adapter
### SQLite Adapter
This adapter stores search results in a sqlite database located in db/listings.db. This file can be used for further analysis later on.
This adapter stores search results in an SQLite database. By default, the database is located at `db/listings.db`, but you can configure a custom location. This file can be used for further analysis later.
Fields are:
The database table contains the following columns (all stored as `TEXT` type):
```
['serviceName', 'jobKey', 'id', 'size', 'rooms', 'price', 'address', 'title', 'link', 'description']
['serviceName', 'jobKey', 'id', 'size', 'rooms', 'price', 'address', 'title', 'link', 'description', 'image']
```

View File

@@ -63,31 +63,41 @@ export const send = ({ serviceName, newListings, notificationConfig, jobKey }) =
const jobName = job == null ? jobKey : job.name;
const throttledCall = getThrottled(chatId, async function (endpoint, body) {
await fetch(`https://api.telegram.org/bot${token}/${endpoint}`, {
const res = await fetch(`https://api.telegram.org/bot${token}/${endpoint}`, {
method: 'post',
body: JSON.stringify(body),
headers: { 'Content-Type': 'application/json' },
});
return res;
});
const promises = newListings.map(async (o) => {
const img = normalizeImageUrl(o.image);
const textPayload = {
chat_id: chatId,
text: buildText(jobName, serviceName, o),
parse_mode: 'HTML',
disable_web_page_preview: true,
};
if (img) {
return throttledCall('sendPhoto', {
if (!img) {
return throttledCall('sendMessage', textPayload);
}
try {
return await throttledCall('sendPhoto', {
chat_id: chatId,
photo: img,
caption: buildCaption(jobName, serviceName, o),
parse_mode: 'HTML',
});
} catch (e) {
// If we see a timeout due to sending an image, try sending it without
if (e && (e.code === 'ETIMEDOUT' || e.errno === 'ETIMEDOUT')) {
return throttledCall('sendMessage', textPayload);
}
throw e;
}
return throttledCall('sendMessage', {
chat_id: chatId,
text: buildText(jobName, serviceName, o),
parse_mode: 'HTML',
disable_web_page_preview: true,
});
});
return Promise.all(promises);

View File

@@ -7,7 +7,8 @@ function normalize(o) {
const price = normalizePrice(o.price);
const id = buildHash(o.id, price);
const image = baseUrl + o.image;
return Object.assign(o, { id, price, link, image });
const address = o.address == null ? null : o.address.trim().replaceAll('/', ',');
return Object.assign(o, { id, price, link, image, address });
}
/**
@@ -44,6 +45,7 @@ const config = {
size: '.tabelle .tabelle_inhalt_infos .single_data_box | removeNewline | trim',
title: '.inner_object_data .tabelle_inhalt_titel_black | removeNewline | trim',
image: '.inner_object_pic img@src',
address: '.tabelle .tabelle_inhalt_infos .left_information > div:nth-child(2) | removeNewline | trim',
},
normalize: normalize,
filter: applyBlacklist,

View File

@@ -12,10 +12,10 @@ function parseId(shortenedLink) {
function normalize(o) {
const baseUrl = 'https://www.immobilien.de';
const size = o.size || 'N/A m²';
const price = o.price || 'N/A €';
const size = o.size || null;
const price = o.price || null;
const title = o.title || 'No title available';
const address = o.address || 'No address available';
const address = o.address || null;
const shortLink = shortenLink(o.link);
const link = `${baseUrl}/${shortLink}`;
const image = baseUrl + o.image;

View File

@@ -1,19 +1,12 @@
import utils, { buildHash } from '../utils.js';
let appliedBlackList = [];
/**
* Note, Immonet is rly a piece of sh*t. It is using a weird combination of React and some buttons (instead of links),
* so that if somebody clicks the listing, a new page will open with the actual link to the listing. Of course, a scraper
* cannot do this (which is why I always just return the link to the whole list of listings).
* This is not only bad for us, but also bad for ppl with disabilities...
*/
function normalize(o) {
const size = o.size != null ? o.size.replace('Wohnfläche ', '') : 'N/A m²';
const price = o.price.replace('Kaufpreis ', '');
const address = o.address?.split(' • ')?.pop() ?? null;
const title = o.title || 'No title available';
const link = config.url;
const link = o.link != null ? decodeURIComponent(o.link) : config.url;
const id = buildHash(title, price);
return Object.assign(o, { id, address, price, size, title, link });
}
@@ -28,12 +21,13 @@ const config = {
sortByDateParam: 'sortby=19',
waitForSelector: 'div[data-testid="serp-gridcontainer-testid"]',
crawlFields: {
id: 'button@title |trim', // immonet is a piece of sh*t. See comment above
id: 'button@title |trim',
title: 'button@title |trim',
price: 'div[data-testid="cardmfe-price-testid"] | trim',
size: 'div[data-testid="cardmfe-keyfacts-testid"] | trim',
address: 'div[data-testid="cardmfe-description-box-address"] | trim',
image: 'div[data-testid="cardmfe-picture-box-test-id"] img@src',
link: 'button@data-base',
},
normalize: normalize,
filter: applyBlacklist,

View File

@@ -37,6 +37,7 @@
import utils, { buildHash } from '../utils.js';
import { convertWebToMobile } from '../services/immoscout/immoscout-web-translator.js';
import logger from '../services/logger.js';
let appliedBlackList = [];
async function getListings(url) {
@@ -52,7 +53,7 @@ async function getListings(url) {
}),
});
if (!response.ok) {
console.error('Error fetching data from ImmoScout Mobile API:', response.statusText);
logger.error('Error fetching data from ImmoScout Mobile API:', response.statusText);
return [];
}
@@ -68,6 +69,7 @@ async function getListings(url) {
price: price?.value,
size: size?.value,
title: item.title,
description: item.description,
link: `${metaInformation.baseUrl}expose/${item.id}`,
address: item.address?.line,
image,

View File

@@ -1,37 +1,23 @@
import { setInterval } from 'node:timers';
import { removeJobsByUserName } from './storage/jobStorage.js';
import { removeJobsByUserId } from './storage/jobStorage.js';
import { config } from '../utils.js';
import { getUsers } from './storage/userStorage.js';
import logger from './logger.js';
import cron from 'node-cron';
/**
* if we are running in demo environment, we have to cleanup the db files (specifically the jobs table)
*/
export function cleanupDemoAtMidnight() {
const now = new Date();
const millisUntilMidnightUTC =
(24 - now.getUTCHours()) * 60 * 60 * 1000 -
now.getUTCMinutes() * 60 * 1000 -
now.getUTCSeconds() * 1000 -
now.getUTCMilliseconds();
cleanup();
setTimeout(() => {
setInterval(
() => {
cleanup();
},
24 * 60 * 60 * 1000,
);
}, millisUntilMidnightUTC);
cron.schedule('0 0 * * *', cleanup);
}
function cleanup() {
if (config.demoMode) {
const demoUser = getUsers(false).find((user) => user.username === 'demo');
if (demoUser == null) {
console.error('Demo user not found, cannot remove Jobs');
logger.error('Demo user not found, cannot remove Jobs');
return;
}
removeJobsByUserName(demoUser.id);
removeJobsByUserId(demoUser.id);
}
}

View File

@@ -0,0 +1,2 @@
import { EventEmitter } from 'node:events';
export const bus = new EventEmitter();

View File

@@ -1,6 +1,7 @@
import { setDebug } from './utils.js';
import puppeteerExtractor from './puppeteerExtractor.js';
import { loadParser, parse } from './parser/parser.js';
import logger from '../logger.js';
const DEFAULT_OPTIONS = {
debug: false,
@@ -32,7 +33,7 @@ export default class Extractor {
loadParser(this.responseText);
}
} catch (error) {
console.error('Error trying to load page.', error);
logger.error('Error trying to load page.', error);
}
return this;
};

View File

@@ -1,4 +1,5 @@
import * as cheerio from 'cheerio';
import logger from '../../logger.js';
let $ = null;
@@ -8,19 +9,19 @@ export function loadParser(text) {
export function parse(crawlContainer, crawlFields, text, url) {
if (!text) {
console.warn('No content found for ', url);
logger.warn('No content found for ', url);
return null;
}
if (!crawlContainer || !crawlFields) {
console.warn('Cannot parse, selector was empty for url ', url);
logger.warn('Cannot parse, selector was empty for url ', url);
return null;
}
const result = [];
if ($(crawlContainer).length === 0) {
console.warn('No elements in crawl container found for url ', url);
logger.debug('No elements in crawl container found for url ', url);
return null;
}
@@ -58,7 +59,7 @@ export function parse(crawlContainer, crawlFields, text, url) {
parsedObject[key] = value || null;
} catch (error) {
console.error(`Error parsing field '${key}' with selector '${fieldSelector}':`, error);
logger.error(`Error parsing field '${key}' with selector '${fieldSelector}':`, error);
parsedObject[key] = null;
}
}
@@ -66,7 +67,7 @@ export function parse(crawlContainer, crawlFields, text, url) {
if (parsedObject.id != null) {
result.push(parsedObject);
} else {
console.warn('ID not found. Not relaying object.');
logger.debug('ID not found. Not relaying object.');
}
});
@@ -89,7 +90,7 @@ function applyModifiers(value, modifiers) {
value = value.replace(/\n/g, ' ');
break;
default:
console.warn(`Unknown modifier: ${modifier}`);
logger.warn(`Unknown modifier: ${modifier}`);
}
});

View File

@@ -1,6 +1,7 @@
import puppeteer from 'puppeteer-extra';
import StealthPlugin from 'puppeteer-extra-plugin-stealth';
import { debug, DEFAULT_HEADER, botDetected } from './utils.js';
import logger from '../logger.js';
puppeteer.use(StealthPlugin());
@@ -33,13 +34,13 @@ export default async function execute(url, waitForSelector, options) {
const statusCode = response.status();
if (botDetected(pageSource, statusCode)) {
console.warn('We have been detected as a bot :-/ Tried url: => ', url);
logger.warn('We have been detected as a bot :-/ Tried url: => ', url);
return null;
}
return await page.content();
} catch (error) {
console.error('Error executing with puppeteer executor', error);
logger.error('Error executing with puppeteer executor', error);
return null;
} finally {
if (browser != null) {

View File

@@ -1,3 +1,5 @@
import logger from '../logger.js';
let debuggingOn = false;
export const DEFAULT_HEADER = {
@@ -15,9 +17,7 @@ export const setDebug = (options) => {
export const debug = (message) => {
if (debuggingOn) {
/* eslint-disable no-console */
console.debug(message);
/* eslint-enable no-console */
logger.debug(message);
}
};

59
lib/services/logger.js Normal file
View File

@@ -0,0 +1,59 @@
const COLORS = {
debug: '\x1b[36m',
info: '\x1b[32m',
warn: '\x1b[33m',
error: '\x1b[31m',
reset: '\x1b[0m',
};
const env = process.env.NODE_ENV || 'development';
const useColor = process.stdout.isTTY || process.stderr.isTTY;
function ts() {
const d = new Date();
const yyyy = d.getFullYear();
const mm = String(d.getMonth() + 1).padStart(2, '0');
const dd = String(d.getDate()).padStart(2, '0');
const hh = String(d.getHours()).padStart(2, '0');
const mi = String(d.getMinutes()).padStart(2, '0');
const ss = String(d.getSeconds()).padStart(2, '0');
return `${yyyy}-${mm}-${dd} ${hh}:${mi}:${ss}`;
}
function lvl(level) {
const upper = level.toUpperCase();
if (!useColor) return upper;
return `${COLORS[level] || ''}${upper}${COLORS.reset}`;
}
/* eslint-disable no-console */
function log(level, ...args) {
if (level === 'debug' && env !== 'development') {
return; // Skip debug logs in non-development environments
}
const prefix = `[${ts()}] ${lvl(level)}:`;
switch (level) {
case 'debug':
console.debug(prefix, ...args);
break;
case 'info':
console.info(prefix, ...args);
break;
case 'warn':
console.warn(prefix, ...args);
break;
case 'error':
console.error(prefix, ...args);
break;
default:
console.log(prefix, ...args);
}
}
export default {
debug: (...a) => log('debug', ...a),
info: (...a) => log('info', ...a),
warn: (...a) => log('warn', ...a),
error: (...a) => log('error', ...a),
};

View File

@@ -1,8 +0,0 @@
import lodash from 'lodash';
import { LowSync } from 'lowdb';
export default class LowdashAdapter extends LowSync {
constructor(adapter, defaultData = {}) {
super(adapter, defaultData);
this.chain = lodash.chain(this).get('data');
}
}

View File

@@ -0,0 +1,140 @@
import fs from 'fs';
import path from 'path';
import Database from 'better-sqlite3';
import logger from '../../services/logger.js';
import { config } from '../../utils.js';
/**
* SqliteConnection
* A small, high-performance wrapper around better-sqlite3 that provides a
* singleton connection, sensible PRAGMA tuning, and helper methods. This
* module is safe to import and reuse.
*
* Performance notes:
* - journal_mode = WAL: allows concurrent readers with a single writer and
* yields better performance for server apps.
* - synchronous = NORMAL: trades a bit of durability for significant speed
* while still being safe in most environments.
* - cache_size = -64000: ~64MB page cache (negative value sets KB) to improve
* query performance for frequent reads.
* - foreign_keys = ON: ensure referential integrity is enforced.
* - optimize: runs SQLite's auto-analysis and purges internal caches. It is
* cheap; we call it at startup and before process exit. You can also call
* optimize() manually after large schema changes or bulk operations.
*/
class SqliteConnection {
static #db = null;
/**
* Returns a singleton instance of better-sqlite3 Database.
* Respects env var SQLITE_DB_PATH and defaults to db/listings.db.
*/
static getConnection() {
if (this.#db) return this.#db;
// Interpret config.sqlitepath as a directory relative to project root when it starts with '/'
const cfg = typeof config === 'object' && config ? config.sqlitepath : undefined;
const rawDir = cfg && cfg.length > 0 ? cfg : '/db';
const relDir = rawDir.startsWith('/') ? rawDir.slice(1) : rawDir;
const absDir = path.isAbsolute(relDir) ? relDir : path.join(process.cwd(), relDir);
const dbPath = path.join(absDir, 'listings.db');
// Ensure directory exists
const dir = path.dirname(dbPath);
if (!fs.existsSync(dir)) fs.mkdirSync(dir, { recursive: true });
// Open the database synchronously (better-sqlite3 is sync and very fast)
this.#db = new Database(dbPath, { verbose: undefined });
// Apply high-performance PRAGMA's
try {
this.#db.pragma('journal_mode = WAL');
this.#db.pragma('synchronous = NORMAL');
this.#db.pragma('cache_size = -64000');
this.#db.pragma('foreign_keys = ON');
this.#db.pragma('optimize');
} catch (e) {
logger.warn('Failed to apply one or more PRAGMAs:', e.message);
}
// Run optimize on exit to persist analysis and cleanup internal caches.
process.once('beforeExit', () => {
try {
this.#db?.pragma('optimize');
} catch (e) {
logger.debug('PRAGMA optimize on exit failed:', e.message);
}
});
return this.#db;
}
/**
* Execute a write statement (INSERT/UPDATE/DELETE/DDL). Returns better-sqlite3 run info.
*/
static execute(sql, params = {}) {
const db = this.getConnection();
return db.prepare(sql).run(params);
}
/**
* Execute a query and returns all rows.
*/
static query(sql, params = {}) {
const db = this.getConnection();
return db.prepare(sql).all(params);
}
/**
* Check whether a table exists.
*/
static tableExists(tableName) {
const db = this.getConnection();
const row = db.prepare("SELECT 1 FROM sqlite_master WHERE type = 'table' AND name = ?").get(tableName);
return !!row;
}
/**
* Run the given callback inside a transaction. The callback receives the Database instance.
* If the callback throws, the transaction is rolled back and the error re-thrown.
*/
static withTransaction(callback) {
const db = this.getConnection();
const trx = db.transaction((cb) => cb(db));
return trx(callback);
}
/**
* Run SQLite PRAGMA optimize. See https://sqlite.org/pragma.html#pragma_optimize
*
* Explanation: PRAGMA optimize triggers internal housekeeping, such as
* recomputing query planner statistics (similar to ANALYZE) when appropriate
* and purging unused pages from caches. It is inexpensive and can improve
* performance after schema changes or heavy write activity.
*/
static optimize() {
const db = this.getConnection();
try {
db.pragma('optimize');
} catch (e) {
logger.warn('PRAGMA optimize failed:', e.message);
}
}
/**
* Close the database connection. Typically not needed for long-running apps.
*/
static close() {
if (this.#db) {
try {
this.#db.pragma('optimize');
} catch (e) {
logger.debug('PRAGMA optimize before close failed:', e.message);
}
this.#db.close();
this.#db = null;
}
}
}
export default SqliteConnection;

View File

@@ -1,107 +1,144 @@
import { JSONFileSync } from 'lowdb/node';
import { nanoid } from 'nanoid';
import * as listingStorage from './listingsStorage.js';
import { getDirName } from '../../utils.js';
import path from 'path';
import LowdashAdapter from './LowDashAdapter.js';
const file = path.join(getDirName(), '../', 'db/jobs.json');
const adapter = new JSONFileSync(file);
const db = new LowdashAdapter(adapter, { jobs: [] });
db.read();
import SqliteConnection from './SqliteConnection.js';
import logger from '../logger.js';
import { toJson, fromJson } from '../../utils.js';
/**
* Insert or update a job. Preserves original owner (userId) when updating an existing job.
*
* @param {Object} params
* @param {string} [params.jobId] - Existing job id to update; omit to insert a new job.
* @param {string} [params.name] - Job display name.
* @param {Array<any>} [params.blacklist] - Blacklist entries; defaults to empty array.
* @param {boolean} [params.enabled] - Whether the job is enabled; defaults to true.
* @param {Array<any>} params.provider - Provider configuration list.
* @param {Array<any>} params.notificationAdapter - Notification adapter configuration list.
* @param {string} params.userId - Owner user id for inserts; preserved on updates.
* @returns {void}
*/
export const upsertJob = ({ jobId, name, blacklist = [], enabled = true, provider, notificationAdapter, userId }) => {
const currentJob =
jobId == null
? null
: db.chain
.get('jobs')
.find((job) => job.id === jobId)
.value();
const jobs = db.chain
.get('jobs')
.filter((job) => job.id !== jobId)
.value();
jobs.push({
id: jobId || nanoid(),
//make sure to not overwrite the user id in case an admin changes the job
userId: currentJob == null ? userId : currentJob.userId,
enabled,
name,
blacklist,
provider,
notificationAdapter,
});
db.chain.set('jobs', jobs).value();
db.write();
};
export const getJob = (jobId) => {
const job = db.chain
.get('jobs')
.find((job) => job.id === jobId)
.value();
if (job == null) {
return null;
const id = jobId || nanoid();
const existing = SqliteConnection.query(`SELECT id, user_id FROM jobs WHERE id = @id LIMIT 1`, { id })[0];
const ownerId = existing ? existing.user_id : userId;
if (existing) {
SqliteConnection.execute(
`UPDATE jobs
SET enabled = @enabled,
name = @name,
blacklist = @blacklist,
provider = @provider,
notification_adapter = @notification_adapter
WHERE id = @id`,
{
id,
enabled: enabled ? 1 : 0,
name: name ?? null,
blacklist: toJson(blacklist ?? []),
provider: toJson(provider ?? []),
notification_adapter: toJson(notificationAdapter ?? []),
},
);
} else {
SqliteConnection.execute(
`INSERT INTO jobs (id, user_id, enabled, name, blacklist, provider, notification_adapter)
VALUES (@id, @user_id, @enabled, @name, @blacklist, @provider, @notification_adapter)`,
{
id,
user_id: ownerId,
enabled: enabled ? 1 : 0,
name: name ?? null,
blacklist: toJson(blacklist ?? []),
provider: toJson(provider ?? []),
notification_adapter: toJson(notificationAdapter ?? []),
},
);
}
};
/**
* Get a single job by id.
* @param {string} jobId - Job primary key.
* @returns {Job|null} The job or null if not found.
*/
export const getJob = (jobId) => {
const row = SqliteConnection.query(
`SELECT j.id,
j.user_id AS userId,
j.enabled,
j.name,
j.blacklist,
j.provider,
j.notification_adapter AS notificationAdapter,
(SELECT COUNT(1) FROM listings l WHERE l.job_id = j.id) AS numberOfFoundListings
FROM jobs j
WHERE j.id = @id
LIMIT 1`,
{ id: jobId },
)[0];
if (!row) return null;
return {
...job,
numberOfFoundListings: listingStorage.getNumberOfAllKnownListings(job.id).length,
...row,
enabled: !!row.enabled,
blacklist: fromJson(row.blacklist, []),
provider: fromJson(row.provider, []),
notificationAdapter: fromJson(row.notificationAdapter, []),
};
};
/**
* Update job enabled status.
* @param {{jobId: string, status: boolean}} params - Parameters.
* @returns {void}
*/
export const setJobStatus = ({ jobId, status }) => {
db.chain
.get('jobs')
.find((job) => job.id === jobId)
.assign({ enabled: status })
.value();
db.write();
SqliteConnection.execute(`UPDATE jobs SET enabled = @enabled WHERE id = @id`, {
id: jobId,
enabled: status ? 1 : 0,
});
};
/**
* Remove a job by id. Listings are deleted automatically due to FK ON DELETE CASCADE.
* @param {string} jobId - Job id.
* @returns {void}
*/
export const removeJob = (jobId) => {
listingStorage.removeListings(jobId);
db.chain
.get('jobs')
.remove((job) => job.id === jobId)
.value();
db.write();
// listings table has FK ON DELETE CASCADE via job_id
SqliteConnection.execute(`DELETE FROM jobs WHERE id = @id`, { id: jobId });
};
export const removeJobsByUserId = (userId) => {
db.chain
.get('jobs')
.filter((job) => job.userId === userId)
.forEach((job) => listingStorage.removeListings(job.id));
db.chain
.get('jobs')
.remove((job) => job.userId === userId)
.value();
db.write();
};
export const removeJobsByUserName = (userId) => {
let removedDemoJobs = 0;
db.chain
.get('jobs')
.filter((job) => job.userId === userId)
.forEach((job) => {
removedDemoJobs++;
listingStorage.removeListings(job.id);
});
db.chain
.get('jobs')
.remove((job) => job.userId === userId)
.value();
db.write();
if (removedDemoJobs > 0) {
/* eslint-disable no-console */
console.log(`Removed ${removedDemoJobs} demo jobs`);
/* eslint-enable no-console */
// Count jobs to log similar to previous behavior
const count =
SqliteConnection.query(`SELECT COUNT(1) AS c FROM jobs WHERE user_id = @user_id`, { user_id: userId })[0]?.c ?? 0;
SqliteConnection.execute(`DELETE FROM jobs WHERE user_id = @user_id`, { user_id: userId });
if (count > 0) {
logger.info(`Removed ${count} jobs for user ${userId}`);
}
};
/**
* Get all jobs.
* @returns {Job[]} List of jobs ordered by name (NULLs last).
*/
export const getJobs = () => {
return db.chain
.get('jobs')
.map((job) => ({
...job,
numberOfFoundListings: listingStorage.getNumberOfAllKnownListings(job.id),
}))
.value();
const rows = SqliteConnection.query(
`SELECT j.id,
j.user_id AS userId,
j.enabled,
j.name,
j.blacklist,
j.provider,
j.notification_adapter AS notificationAdapter,
(SELECT COUNT(1) FROM listings l WHERE l.job_id = j.id) AS numberOfFoundListings
FROM jobs j
ORDER BY j.name IS NULL, j.name`,
);
return rows.map((row) => ({
...row,
enabled: !!row.enabled,
blacklist: fromJson(row.blacklist, []),
provider: fromJson(row.provider, []),
notificationAdapter: fromJson(row.notificationAdapter, []),
}));
};

View File

@@ -1,52 +1,138 @@
import { JSONFileSync } from 'lowdb/node';
import { getDirName } from '../../utils.js';
import path from 'path';
import LowdashAdapter from './LowDashAdapter.js';
import { nullOrEmpty } from '../../utils.js';
import SqliteConnection from './SqliteConnection.js';
import { nanoid } from 'nanoid';
const file = path.join(getDirName(), '../', 'db/jobListingData.json');
const adapter = new JSONFileSync(file);
const db = new LowdashAdapter(adapter, {});
db.read();
const buildKey = (jobKey, providerId, endpoint) => {
let key = `${jobKey}`;
if (jobKey == null && endpoint == null) {
return key;
}
if (providerId != null) {
key += `.${providerId}`;
}
if (endpoint != null) {
key += `.${endpoint}`;
}
return key;
};
export const getNumberOfAllKnownListings = (jobId) => {
const data = db.chain.get(`${jobId}.providerData`).value() || {};
return Object.values(data)
.map((values) => Object.keys(values).length)
.reduce((accumulator, currentValue) => accumulator + currentValue, 0);
};
/**
* Build analytics data for a given job by grouping all listings by provider and
* mapping each listing hash to its creation timestamp.
*
* SQL shape:
* SELECT json_group_object(provider, json_object(hash, created_at)) AS result
* FROM listings WHERE job_id = @jobId;
*
* The resulting object has the shape:
* {
* providerA: { "<hash1>": <created_at_ms>, "<hash2>": <created_at_ms>, ... },
* providerB: { ... }
* }
*
* @param {string} jobId - ID of the job whose listings should be aggregated.
* @returns {Record<string, Record<string, number>>} Object grouped by provider mapping listing-hash -> created_at epoch ms.
*/
export const getListingProviderDataForAnalytics = (jobId) => {
const key = buildKey(jobId, 'providerData');
return db.chain.get(key).value() || {};
const row = SqliteConnection.query(
`SELECT COALESCE(
json_group_object(provider, json(provider_map)),
json('{}')
) AS result
FROM (SELECT provider,
json_group_object(hash, created_at) AS provider_map
FROM listings
WHERE job_id = @jobId
GROUP BY provider);`,
{ jobId },
);
return row?.length > 0 ? JSON.parse(row[0].result) : {};
};
export const getKnownListings = (jobId, providerId) => {
const providerListingsKey = buildKey(jobId, 'providerData', providerId, 'listings');
return db.chain.get(providerListingsKey).value() || {};
/**
* Return a list of known listing hashes for a given job and provider.
* Useful to de-duplicate before inserting new listings.
*
* @param {string} jobId - The job identifier.
* @param {string} providerId - The provider identifier (e.g., 'immoscout').
* @returns {string[]} Array of listing hashes.
*/
export const getKnownListingHashesForJobAndProvider = (jobId, providerId) => {
return SqliteConnection.query(
`SELECT hash
FROM listings
WHERE job_id = @jobId AND provider = @providerId`,
{ jobId, providerId },
).map((r) => r.hash);
};
export const setKnownListings = (jobId, providerId, listings) => {
const providerListingsKey = buildKey(jobId, 'providerData', providerId, 'listings');
db.chain.set(providerListingsKey, listings).value();
return db.write();
};
export const setLastJobExecution = (jobId) => {
const key = buildKey(jobId, null, 'lastExecution');
db.chain.set(key, Date.now()).value();
return db.write();
};
export const removeListings = (jobId) => {
db.chain.unset(jobId).value();
db.write();
/**
* Persist a batch of scraped listings for a given job and provider.
*
* - Empty or non-array inputs are ignored.
* - Each listing is inserted with ON CONFLICT(hash) DO NOTHING to avoid duplicates.
* - Performs inserts in a single transaction for performance.
*
* Listing input shape (minimal expected):
* {
* id: string, // unique id
* hash: string // stable hash/id of the listing (used as unique hash)
* price?: string, // e.g., "1.234 €" or "1,234€"
* size?: string, // e.g., "70 m²"
* title?: string,
* image?: string, // image URL
* description?: string,
* address?: string, // free-text address possibly containing parentheses
* link?: string
* }
*
* @param {string} jobId - The job identifier.
* @param {string} providerId - The provider identifier.
* @param {Array<Object>} listings - Array of listing objects as described above.
* @returns {void}
*/
export const storeListings = (jobId, providerId, listings) => {
if (!Array.isArray(listings) || listings.length === 0) {
return;
}
SqliteConnection.withTransaction((db) => {
const stmt = db.prepare(
`INSERT INTO listings (id, hash, provider, job_id, price, size, title, image_url, description, address,
link, created_at)
VALUES (@id, @hash, @provider, @job_id, @price, @size, @title, @image_url, @description, @address, @link,
@created_at)
ON CONFLICT(job_id, hash) DO NOTHING`,
);
for (const item of listings) {
const params = {
id: nanoid(),
hash: item.id,
provider: providerId,
job_id: jobId,
price: extractNumber(item.price),
size: extractNumber(item.size),
title: item.title,
image_url: item.image,
description: item.description,
address: removeParentheses(item.address),
link: item.link,
created_at: Date.now(),
};
stmt.run(params);
}
});
/**
* Extract the first number from a string like "1.234 €" or "70 m²".
* Removes dots/commas before parsing. Returns null on invalid input.
* @param {string|undefined|null} str
* @returns {number|null}
*/
function extractNumber(str) {
if (!str) return null;
const match = str.replace(/[.,]/g, '').match(/\d+/);
return match ? +match[0] : null;
}
/**
* Remove any parentheses segments (including surrounding whitespace) from a string.
* Returns null for empty input.
* @param {string|undefined|null} str
* @returns {string|null}
*/
function removeParentheses(str) {
if (nullOrEmpty(str)) {
return null;
}
return str.replace(/\s*\([^)]*\)/g, '');
}
};

View File

@@ -0,0 +1,185 @@
/**
* Migration Runner for better-sqlite3
* I know there are external libs out there, but
* a) most of them are pretty bloated
* b) I wanted to have something that fit's this limited use-case
* c) I was searching for justifications anyway to build a migration system on my own. Don't judge me ;)
*
* Executes all migration files in lib/services/storage/migrations/sql in natural order.
* Each migration runs in its own transaction. If a migration fails, only that
* migration is rolled back and the process stops with a non-zero exit code.
* Already applied migrations are skipped using the schema_migrations table.
*
* Usage:
* CLI: yarn run migratedb
* Programmatic:
* import { runMigrations } from './lib/services/storage/migrations/migrate.js';
* await runMigrations();
*
* Migration file format (example: lib/services/storage/migrations/sql/1.add-users.js):
* export function up(db) {
* db.exec("CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT NOT NULL)");
* }
*
*/
import fs from 'fs';
import path from 'path';
import { pathToFileURL } from 'url';
import crypto from 'crypto';
import SqliteConnection from '../SqliteConnection.js';
import logger from '../../logger.js';
const ROOT = path.resolve('.');
const MIGRATIONS_DIR = path.join(ROOT, 'lib', 'services', 'storage', 'migrations', 'sql');
/**
* Ensures that the given directory exists, creating it recursively if needed.
* @param {string} p - Path to the directory.
*/
function ensureDir(p) {
if (!fs.existsSync(p)) fs.mkdirSync(p, { recursive: true });
}
/**
* Lists all migration files in the migrations directory.
* Migration files must follow the format: <number>.<label>.js
* @returns {Array<{id:number, name:string, label:string, path:string}>}
*/
function listMigrationFiles() {
ensureDir(MIGRATIONS_DIR);
return fs
.readdirSync(MIGRATIONS_DIR)
.filter((f) => /^\d+\..+\.js$/.test(f))
.map((file) => {
const [idStr, ...rest] = file.split('.');
const id = Number.parseInt(idStr, 10);
const label = rest.slice(0, -1).join('.');
const fullPath = path.join(MIGRATIONS_DIR, file);
return { id, name: file, label, path: fullPath };
})
.sort((a, b) => (a.id === b.id ? a.name.localeCompare(b.name) : a.id - b.id));
}
/**
* Calculates the SHA-256 checksum of a file.
* @param {string} filePath - Path to the file.
* @returns {string} Hex-encoded checksum.
*/
function sha256File(filePath) {
const buf = fs.readFileSync(filePath);
return crypto.createHash('sha256').update(buf).digest('hex');
}
/**
* Dynamically imports a migration module and returns its `up` function.
* @param {string} filePath - Path to the migration file.
* @returns {Promise<Function>} Migration function.
* @throws {Error} If the migration file does not export a valid function.
*/
async function loadMigrationModule(filePath) {
const testImporter = globalThis.__TEST_MIGRATE_IMPORT__;
const url = pathToFileURL(filePath);
const mod = testImporter ? await testImporter(filePath, url) : await import(url.href);
const fn = mod.up || mod.default;
if (typeof fn !== 'function') {
throw new Error(`Migration ${filePath} must export function up(db) or default function(db)`);
}
return fn;
}
/**
* Loads all previously executed migrations from the database.
* @returns {Map<string,string>} Map of migration name to checksum.
*/
function loadExecutedMigrations() {
const executed = new Map();
const hasTable = SqliteConnection.tableExists('schema_migrations');
if (!hasTable) return executed;
const rows = SqliteConnection.query('SELECT name, checksum FROM schema_migrations ORDER BY applied_at ASC');
for (const r of rows) executed.set(r.name, r.checksum);
return executed;
}
/**
* Executes all pending migrations.
* Ensures that each migration runs inside its own transaction.
* Already applied migrations are skipped, unless checksum updates are allowed.
* On success, updates the schema_migrations table and runs PRAGMA optimize.
*/
export async function runMigrations() {
ensureDir(path.join(ROOT, 'db'));
ensureDir(MIGRATIONS_DIR);
const files = listMigrationFiles();
if (files.length === 0) {
logger.info('No migration files found under', MIGRATIONS_DIR);
return;
}
SqliteConnection.getConnection();
const executed = loadExecutedMigrations();
let appliedMigrations = 0;
for (const m of files) {
const checksum = sha256File(m.path);
if (executed.has(m.name)) {
const prev = executed.get(m.name);
if (prev !== checksum) {
logger.info(`Mismatch found in migration ${m.name}. Fixing.`);
SqliteConnection.execute('UPDATE schema_migrations SET checksum = @checksum WHERE name = @name', {
checksum,
name: m.name,
});
executed.set(m.name, checksum);
}
continue;
}
appliedMigrations++;
logger.info(`Applying migration: ${m.name}`);
const fn = await loadMigrationModule(m.path);
try {
let duration = 0;
SqliteConnection.withTransaction((db) => {
const t0 = Date.now();
fn(db);
duration = Date.now() - t0;
db.prepare(
"INSERT INTO schema_migrations (name, checksum, applied_at, duration_ms) VALUES (?, ?, datetime('now'), ?)",
).run(m.name, checksum, duration);
});
logger.info(`Migration applied: ${m.name} (${duration} ms)`);
} catch (e) {
logger.error(`Migration failed and was rolled back: ${m.name}`, e);
process.exitCode = 1;
return;
}
}
SqliteConnection.optimize();
if (appliedMigrations > 0) {
logger.info('All migrations completed successfully.');
}
}
/**
* Detects whether the current file is being executed directly via Node.js.
* This allows `node lib/services/storage/migrations/migrate.js` to run migrations directly.
* @returns {boolean} True if the file was run directly.
*/
const isDirectRun = (() => {
try {
const thisFile = import.meta.url;
const invoked = pathToFileURL(process.argv[1] || '').href;
return thisFile === invoked;
} catch {
return false;
}
})();
if (isDirectRun) {
await runMigrations();
}

View File

@@ -0,0 +1,16 @@
// Initial migration: creates schema_migrations table used by the migration runner.
//
export function up(db) {
db.exec(`
CREATE TABLE IF NOT EXISTS schema_migrations (
id INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT NOT NULL UNIQUE,
checksum TEXT NOT NULL,
applied_at TEXT NOT NULL DEFAULT (datetime('now')),
duration_ms INTEGER NOT NULL DEFAULT 0
);
CREATE INDEX IF NOT EXISTS idx_schema_migrations_applied_at
ON schema_migrations(applied_at);
`);
}

View File

@@ -0,0 +1,117 @@
// Migration: Create fredy's base structure (users, jobs and listings) import initial
// data from JSON files if present. (This applies only for jobs and users, for the old jobListingData,
// I cannot migrate the data as the new format is totally different.
import fs from 'fs';
import path from 'path';
import { toJson } from '../../../../utils.js';
export function up(db) {
// 1) Create tables
db.exec(`
CREATE TABLE IF NOT EXISTS users
(
id TEXT PRIMARY KEY,
username TEXT NOT NULL,
password TEXT NOT NULL,
last_login INTEGER,
is_admin INTEGER NOT NULL DEFAULT 0
);
CREATE UNIQUE INDEX IF NOT EXISTS idx_users_username ON users (username);
CREATE TABLE IF NOT EXISTS jobs
(
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL,
enabled INTEGER NOT NULL DEFAULT 1,
name TEXT,
blacklist JSONB NOT NULL DEFAULT '[]',
provider JSONB NOT NULL DEFAULT '[]',
notification_adapter JSONB NOT NULL DEFAULT '[]',
FOREIGN KEY (user_id) REFERENCES users (id) ON DELETE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_jobs_user_id ON jobs (user_id);
CREATE INDEX IF NOT EXISTS idx_jobs_enabled ON jobs (enabled);
CREATE TABLE IF NOT EXISTS listings
(
id TEXT PRIMARY KEY,
created_at INTEGER,
hash TEXT,
provider TEXT,
job_id TEXT,
price INTEGER,
size INTEGER,
title TEXT,
image_url TEXT,
description TEXT,
address TEXT,
link TEXT,
FOREIGN KEY (job_id) REFERENCES jobs (id) ON DELETE CASCADE
);
CREATE UNIQUE INDEX IF NOT EXISTS idx_listings_hash ON listings (hash);
`);
// 2) Optionally import data from JSON files if present for users and jobs
const ROOT = path.resolve('.');
const usersJsonPath = path.join(ROOT, 'db', 'users.json');
const jobsJsonPath = path.join(ROOT, 'db', 'jobs.json');
// Insert users
if (fs.existsSync(usersJsonPath)) {
try {
const raw = fs.readFileSync(usersJsonPath, 'utf8');
const json = JSON.parse(raw);
const arr = Array.isArray(json?.user) ? json.user : [];
if (arr.length > 0) {
const stmt = db.prepare(
`INSERT INTO users (id, username, password, last_login, is_admin)
VALUES (@id, @username, @password, @last_login, @is_admin)`,
);
for (const u of arr) {
stmt.run({
id: u.id,
username: u.username,
password: u.password,
last_login: u.lastLogin ?? null,
is_admin: u.isAdmin ? 1 : 0,
});
}
}
} catch (e) {
// If parsing fails, let it throw to rollback the migration
throw new Error(`Failed to import users from ${usersJsonPath}: ${e.message}`);
}
}
// Insert jobs
if (fs.existsSync(jobsJsonPath)) {
try {
const raw = fs.readFileSync(jobsJsonPath, 'utf8');
const json = JSON.parse(raw);
const arr = Array.isArray(json?.jobs) ? json.jobs : [];
if (arr.length > 0) {
const stmt = db.prepare(
`INSERT INTO jobs (id, user_id, enabled, name, blacklist, provider, notification_adapter)
VALUES (@id, @user_id, @enabled, @name, @blacklist, @provider, @notification_adapter)`,
);
for (const j of arr) {
stmt.run({
id: j.id,
user_id: j.userId,
enabled: j.enabled ? 1 : 0,
name: j.name ?? null,
blacklist: toJson(j.blacklist ?? []),
provider: toJson(j.provider ?? []),
notification_adapter: toJson(j.notificationAdapter ?? []),
});
}
}
} catch (e) {
throw new Error(`Failed to import jobs from ${jobsJsonPath}: ${e.message}`);
}
}
}

View File

@@ -0,0 +1,10 @@
// Migration: there needs to be a unique index on job_id and hash as only
// this makes the listing indeed unique
export function up(db) {
db.exec(`
DROP INDEX IF EXISTS idx_listings_hash;
CREATE UNIQUE INDEX IF NOT EXISTS idx_listings_job_hash
ON listings (job_id, hash);
`);
}

View File

@@ -1,123 +1,176 @@
import { JSONFileSync } from 'lowdb/node';
import { config, getDirName } from '../../utils.js';
import { config } from '../../utils.js';
import * as hasher from '../security/hash.js';
import { nanoid } from 'nanoid';
import * as jobStorage from './jobStorage.js';
import path from 'path';
import LowdashAdapter from './LowDashAdapter.js';
const defaultData = {
user: [
//you probably want to change the default password ;)
{
id: nanoid(),
lastLogin: Date.now(),
username: 'admin',
password: hasher.hash('admin'),
isAdmin: true,
},
{
id: nanoid(),
lastLogin: Date.now(),
username: 'demo',
password: hasher.hash('demo'),
isAdmin: true,
},
],
};
const file = path.join(getDirName(), '../', 'db/users.json');
const adapter = new JSONFileSync(file);
const db = new LowdashAdapter(adapter, defaultData);
db.read();
import SqliteConnection from './SqliteConnection.js';
/**
* Get all users.
*
* Notes:
* - Password hashes are omitted by default to avoid leaking them to callers that dont need them.
* - numberOfJobs is computed via a subquery for each user.
*
* @param {boolean} withPassword - If true, include the hashed password in the returned objects; otherwise set password to null.
* @returns {User[]} Array of users ordered by username.
*/
export const getUsers = (withPassword) => {
const jobs = jobStorage.getJobs();
return db.chain
.get('user')
.value()
.map((user) => ({
//we dont want the password in the frontend, even tho it's hashed
...user,
password: withPassword ? user.password : null,
numberOfJobs: jobs.filter((job) => job.userId === user.id).length,
}));
};
export const getUser = (id) => {
const jobs = jobStorage.getJobs();
const user = db.chain
.get('user')
.find((user) => user.id === id)
.value();
if (user == null) {
return null;
}
return {
...user,
numberOfJobs: jobs.filter((job) => job.userId === user.id).length,
};
};
export const upsertUser = ({ username, password, userId, isAdmin }) => {
const user = db.chain
.get('user')
.filter((u) => u.id !== userId)
.value();
user.push({
id: userId || nanoid(),
username,
lastLogin: user.lastLogin,
password: hasher.hash(password),
isAdmin,
});
db.chain.set('user', user).value();
db.write();
};
export const setLastLoginToNow = ({ userId }) => {
db.chain
.get('user')
.find((u) => u.id === userId)
.assign({ lastLogin: Date.now() })
.value();
db.write();
};
export const removeUser = (userId) => {
const user = db.chain.get('user').value();
db.chain
.set(
'user',
user.filter((u) => u.id !== userId),
)
.value();
db.write();
const rows = SqliteConnection.query(
`SELECT u.id, u.username, u.password, u.last_login AS lastLogin, u.is_admin AS isAdmin,
(SELECT COUNT(1) FROM jobs j WHERE j.user_id = u.id) AS numberOfJobs
FROM users u
ORDER BY u.username`,
);
return rows.map((u) => ({
...u,
password: withPassword ? u.password : null,
isAdmin: !!u.isAdmin,
}));
};
export const handleDemoUser = () => {
if (!config.demoMode) {
const user = db.chain.get('user').value();
db.chain
.set(
'user',
user.filter((u) => u.username !== 'demo'),
)
.value();
db.write();
/**
* Get a single user by id.
*
* @param {string} id - User id (primary key).
* @returns {User|null} The user when found; otherwise null. The password field is included but callers should not expose it.
*/
export const getUser = (id) => {
const rows = SqliteConnection.query(
`SELECT u.id, u.username, u.password, u.last_login AS lastLogin, u.is_admin AS isAdmin,
(SELECT COUNT(1) FROM jobs j WHERE j.user_id = u.id) AS numberOfJobs
FROM users u
WHERE u.id = @id
LIMIT 1`,
{ id },
);
const u = rows[0];
if (!u) return null;
return { ...u, isAdmin: !!u.isAdmin };
};
/**
* Insert a new user or update an existing one.
*
* Behavior:
* - When userId is provided and exists: updates username and isAdmin. Password is only updated when a non-empty password is provided.
* - When userId is missing or does not exist: inserts a new user with a freshly generated id. last_login is initialized to null.
* - Passwords are hashed using the same hashing function used for login comparison.
*
* @param {Object} params
* @param {string} params.username - Username (must be unique in DB).
* @param {string} [params.password] - Plain text password to set; if omitted on update, existing hash is preserved.
* @param {string} [params.userId] - Existing user id to update; if missing, a new id is generated.
* @param {boolean} params.isAdmin - Whether the user should have admin privileges.
* @returns {void}
*/
export const upsertUser = ({ username, password, userId, isAdmin }) => {
const id = userId || nanoid();
// Check if user exists
const exists = SqliteConnection.query(`SELECT 1 FROM users WHERE id = @id LIMIT 1`, { id }).length > 0;
if (exists) {
// Update existing user. Update password only if provided (non-empty string)
if (password && password.length > 0) {
SqliteConnection.execute(
`UPDATE users SET username = @username, password = @password, is_admin = @is_admin WHERE id = @id`,
{ id, username, password: hasher.hash(password), is_admin: isAdmin ? 1 : 0 },
);
} else {
SqliteConnection.execute(`UPDATE users SET username = @username, is_admin = @is_admin WHERE id = @id`, {
id,
username,
is_admin: isAdmin ? 1 : 0,
});
}
} else {
const demoUser = db.chain
.get('user')
.filter((u) => u.username === 'demo')
.value();
if (demoUser == null || demoUser.length === 0) {
db.chain
.get('user')
.value()
.push({
id: nanoid(),
username: 'demo',
password: hasher.hash('demo'),
isAdmin: true,
});
db.write();
SqliteConnection.execute(
`INSERT INTO users (id, username, password, last_login, is_admin)
VALUES (@id, @username, @password, @last_login, @is_admin)`,
{
id,
username,
password: hasher.hash(password || ''),
last_login: null,
is_admin: isAdmin ? 1 : 0,
},
);
}
};
/**
* Update the last_login timestamp to now for the given user.
*
* @param {{userId: string}} params - Parameters.
* @param {string} params.userId - The user's id.
* @returns {void}
*/
export const setLastLoginToNow = ({ userId }) => {
SqliteConnection.execute(`UPDATE users SET last_login = @now WHERE id = @id`, { id: userId, now: Date.now() });
};
/**
* Remove a user by id.
*
* Notes:
* - In the SQLite schema, jobs reference users with ON DELETE CASCADE, so jobs (and their listings via jobs) are removed automatically.
*
* @param {string} userId - The id of the user to remove.
* @returns {void}
*/
export const removeUser = (userId) => {
SqliteConnection.execute(`DELETE FROM users WHERE id = @id`, { id: userId });
};
/**
* Ensure the demo user matches the demo mode setting.
*
* Behavior:
* - When config.demoMode is false: remove the demo user (and its cascading data via FKs).
* - When config.demoMode is true: ensure a 'demo' user exists with password 'demo' and admin rights.
*
* Security: The demo user's password is set to a known value ('demo') and should only be enabled in demoMode.
* @returns {void}
*/
export const ensureDemoUserExists = () => {
if (!config.demoMode) {
// Remove demo user (and cascade delete their jobs/listings)
SqliteConnection.execute(`DELETE FROM users WHERE username = 'demo'`);
return;
}
// Ensure demo user exists when demo mode is on
const existing = SqliteConnection.query(`SELECT id FROM users WHERE username = 'demo' LIMIT 1`);
if (existing.length === 0) {
SqliteConnection.execute(
`INSERT INTO users (id, username, password, last_login, is_admin)
VALUES (@id, 'demo', @password, NULL, 1)`,
{ id: nanoid(), password: hasher.hash('demo') },
);
}
};
/**
* Ensure there is at least one administrator in the system.
*
* Behavior:
* - If there are no users at all, create default 'admin' user with password 'admin'.
* - If users exist but none is admin, promote the first existing user to admin.
*
* Security: On a fresh instance, a default admin/admin is created; change this password immediately.
* @returns {void}
*/
export const ensureAdminUserExists = () => {
const anyUser = SqliteConnection.query(`SELECT id FROM users LIMIT 1`).length > 0;
if (!anyUser) {
SqliteConnection.execute(
`INSERT INTO users (id, username, password, last_login, is_admin)
VALUES (@id, 'admin', @password, @last_login, 1)`,
{ id: nanoid(), password: hasher.hash('admin'), last_login: Date.now() },
);
return;
}
const adminCount = SqliteConnection.query(`SELECT COUNT(1) AS c FROM users WHERE is_admin = 1`)[0]?.c ?? 0;
if (adminCount === 0) {
const firstUser = SqliteConnection.query(`SELECT id FROM users LIMIT 1`)[0];
if (firstUser) {
SqliteConnection.execute(`UPDATE users SET is_admin = 1 WHERE id = @id`, { id: firstUser.id });
}
}
};

View File

@@ -0,0 +1,17 @@
import cron from 'node-cron';
import { config, inDevMode } from '../../utils.js';
import { trackMainEvent } from './Tracker.js';
async function runTask() {
//make sure to only send tracking events if the user gave us the green light and we are not in dev mode
if (config.analyticsEnabled && !inDevMode()) {
await trackMainEvent();
}
}
export async function initTrackerCron() {
//run directly on start
await runTask();
// then every 6 hours
cron.schedule('0 */6 * * *', runTask);
}

View File

@@ -1,65 +1,66 @@
import Mixpanel from 'mixpanel';
import { getJobs } from '../storage/jobStorage.js';
import { getUniqueId } from './uniqueId.js';
import { config, inDevMode } from '../../utils.js';
import os from 'os';
import { readFileSync } from 'fs';
import { packageUp } from 'package-up';
import fetch from 'node-fetch';
import logger from '../logger.js';
const mixpanelTracker = Mixpanel.init('718670ef1c58c0208256c1e408a3d75e');
const distinct_id = getUniqueId() || 'N/A';
const deviceId = getUniqueId() || 'N/A';
const version = await getPackageVersion();
const FREDY_TRACKING_URL = 'https://fredy.orange-coding.net/tracking';
export const track = function () {
//only send tracking information if the user allowed to do so.
if (config.analyticsEnabled && !inDevMode()) {
const activeProvider = new Set();
const activeAdapter = new Set();
export const trackMainEvent = async () => {
try {
if (config.analyticsEnabled && !inDevMode()) {
const activeProvider = new Set();
const activeAdapter = new Set();
const jobs = getJobs();
const jobs = getJobs();
if (jobs != null && jobs.length > 0) {
jobs.forEach((job) => {
job.provider.forEach((provider) => {
activeProvider.add(provider.id);
if (jobs != null && jobs.length > 0) {
jobs.forEach((job) => {
job.provider.forEach((provider) => activeProvider.add(provider.id));
job.notificationAdapter.forEach((adapter) => activeAdapter.add(adapter.id));
});
job.notificationAdapter.forEach((adapter) => {
activeAdapter.add(adapter.id);
});
});
mixpanelTracker.track(
'fredy_tracking',
enrichTrackingObject({
const trackingObj = enrichTrackingObject({
adapter: Array.from(activeAdapter),
provider: Array.from(activeProvider),
}),
);
});
await fetch(`${FREDY_TRACKING_URL}/main`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(trackingObj),
});
}
}
} catch (error) {
logger.warn('Error sending tracking data', error);
}
};
/**
* Note, this will only be used when Fredy runs in demo mode
*/
export function trackDemoJobCreated(jobData) {
export async function trackDemoAccessed() {
if (config.analyticsEnabled && !inDevMode() && config.demoMode) {
mixpanelTracker.track('demoJobCreated', enrichTrackingObject(jobData));
}
}
/**
* Note, this will only be used when Fredy runs in demo mode
*/
export function trackDemoAccessed() {
if (config.analyticsEnabled && !inDevMode() && config.demoMode) {
mixpanelTracker.track('demoAccessed', enrichTrackingObject({}));
try {
await fetch(`${FREDY_TRACKING_URL}/demo/accessed`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
});
} catch (error) {
logger.warn('Error sending tracking data', error);
}
}
}
function enrichTrackingObject(trackingObject) {
const operating_system = os.platform();
const os_version = os.release();
const operatingSystem = os.platform();
const osVersion = os.release();
const arch = process.arch;
const language = process.env.LANG || 'en';
const nodeVersion = process.version || 'N/A';
@@ -67,13 +68,13 @@ function enrichTrackingObject(trackingObject) {
return {
...trackingObject,
isDemo: config.demoMode,
operating_system,
os_version,
operatingSystem,
osVersion,
arch,
nodeVersion,
language,
distinct_id,
fredy_version: version,
deviceId,
version,
};
}
@@ -84,7 +85,7 @@ async function getPackageVersion() {
const json = JSON.parse(packageJson);
return json.version;
} catch (error) {
console.error('Error reading version from package.json', error);
logger.error('Error reading version from package.json', error);
}
return 'N/A';
}

View File

@@ -4,26 +4,79 @@ import { readFile } from 'fs/promises';
import { createHash } from 'crypto';
import { DEFAULT_CONFIG } from './defaultConfig.js';
import fs from 'fs';
import logger from './services/logger.js';
const RE_GT = />/g;
const RE_WEBP = /\/format\/webp/gi;
const RE_EXT = /\.(jpe?g|png|gif)(\?.*)?$/i;
const HTTPS_PREFIX = 'https://';
/**
* Safely stringify a value to JSON for storage.
* - Returns null when the input is null or undefined.
* - Uses JSON.stringify directly otherwise.
*
* @template T
* @param {T} v - Any JSON-serializable value.
* @returns {string|null} JSON string or null.
*/
export const toJson = (v) => (v == null ? null : JSON.stringify(v));
/**
* Safely parse JSON text coming from storage.
* - Returns the provided fallback when input is null/undefined.
* - Returns the fallback when parsing fails.
*
* @template T
* @param {string|null|undefined} txt - JSON text from DB/storage.
* @param {T} fallback - Value to return when txt is null/invalid.
* @returns {T} Parsed value or fallback.
*/
export const fromJson = (txt, fallback) => {
if (txt == null) return fallback;
try {
return JSON.parse(txt);
} catch {
return fallback;
}
};
/**
* Determine if the current process runs in development mode.
* Returns true when NODE_ENV is not 'production'.
* @returns {boolean}
*/
function inDevMode() {
return process.env.NODE_ENV == null || process.env.NODE_ENV !== 'production';
}
/**
* Check if a word contains any of the strings in the given array (case-insensitive, substring match).
* @param {string} word
* @param {string[]} arr
* @returns {boolean}
*/
function isOneOf(word, arr) {
if (!arr || arr.length === 0 || word == null) return false;
const lowerWord = word.toLowerCase();
return arr.some((item) => lowerWord.indexOf(item.toLowerCase()) !== -1);
}
/**
* Check if a value is null or an empty string/array.
* @param {any} val
* @returns {boolean}
*/
function nullOrEmpty(val) {
return val == null || val.length === 0;
}
/**
* Convert a day time string (HH:mm) to epoch milliseconds for the given reference date.
* @param {string} timeString - Format HH:mm
* @param {number} now - Epoch ms used as the date basis
* @returns {number}
*/
function timeStringToMs(timeString, now) {
const d = new Date(now);
const parts = timeString.split(':');
@@ -33,6 +86,13 @@ function timeStringToMs(timeString, now) {
return d.getTime();
}
/**
* Check whether current time is within configured working hours, or no hours are set.
* If working hours are missing or incomplete, returns true.
* @param {{workingHours?: {from?: string, to?: string}}} config
* @param {number} now - Epoch ms
* @returns {boolean}
*/
function duringWorkingHoursOrNotSet(config, now) {
const { workingHours } = config;
if (workingHours == null || nullOrEmpty(workingHours.from) || nullOrEmpty(workingHours.to)) {
@@ -43,10 +103,20 @@ function duringWorkingHoursOrNotSet(config, now) {
return fromDate <= now && toDate >= now;
}
/**
* Return the directory name of the current module (ESM equivalent of __dirname).
* @returns {string}
*/
function getDirName() {
return dirname(fileURLToPath(import.meta.url));
}
/**
* Build a sha256 hash string from the provided inputs (ignores null/empty strings).
* Returns null if there are no valid inputs.
* @param {...(string|null|undefined)} inputs
* @returns {string|null}
*/
function buildHash(...inputs) {
if (inputs == null) {
return null;
@@ -58,37 +128,61 @@ function buildHash(...inputs) {
return createHash('sha256').update(cleaned.join(',')).digest('hex');
}
/**
* The in-memory configuration object. Call refreshConfig() to populate/update.
* @type {any}
*/
let config = {};
/**
* Read config JSON from disk (conf/config.json) and parse it.
* @returns {Promise<any>} Parsed configuration object.
*/
export async function readConfigFromStorage() {
return JSON.parse(await readFile(new URL('../conf/config.json', import.meta.url)));
}
/**
* Refresh the in-memory config, ensuring the file exists and setting backward-compatible defaults.
* Populates defaults for analyticsEnabled, demoMode, sqlitepath when missing.
* @returns {Promise<void>}
*/
export async function refreshConfig() {
checkIfConfigExistsAndWriteIfNot();
try {
config = await readConfigFromStorage();
//backwards compatability...
//backwards compatibility...
config.analyticsEnabled ??= null;
config.demoMode ??= false;
// default sqlitepath when missing in older configs
config.sqlitepath ??= '/db';
} catch (error) {
config = { ...DEFAULT_CONFIG };
/* eslint-disable no-console */
console.info('Error reading config file.', error);
logger.info('Error reading config file.', error);
}
}
/**
* If the config file does not exist, we will create it.
* If the config file does not exist, create it with DEFAULT_CONFIG.
* @returns {void}
*/
const checkIfConfigExistsAndWriteIfNot = () => {
if (!fs.existsSync(`${getDirName()}/../conf/config.json`)) {
console.info('Could not find config file. Will create one with default values now');
logger.info('Could not find config file. Will create one with default values now');
fs.writeFileSync(`${getDirName()}/../conf/config.json`, JSON.stringify({ ...DEFAULT_CONFIG }));
}
};
/**
* Normalize image URLs:
* - Trim, remove stray '>' characters.
* - Convert '/format/webp' segments to '/format/jpg'.
* - Enforce HTTPS and ensure a valid image extension (jpg/png/gif). If URL contains '.jpg' without query, cut trailing parts.
* - Return null for invalid inputs.
* @param {string} url
* @returns {string|null}
*/
const normalizeImageUrl = (url) => {
if (typeof url !== 'string' || url.length === 0) return null;
@@ -118,4 +212,6 @@ export default {
duringWorkingHoursOrNotSet,
getDirName,
config,
toJson,
fromJson,
};

View File

@@ -1,6 +1,6 @@
{
"name": "fredy",
"version": "11.6.0",
"version": "12.1.1",
"description": "[F]ind [R]eal [E]states [d]amn eas[y].",
"scripts": {
"prepare": "husky",
@@ -13,7 +13,9 @@
"format:check": "prettier --check \"**/*.js\"",
"test": "node --import ./test/esmock-loader.mjs ./node_modules/mocha/bin/mocha.js --timeout 60000 test/**/*.test.js",
"lint": "eslint .",
"lint:fix": "yarn lint --fix"
"lint:fix": "yarn lint --fix",
"migratedb": "node lib/services/storage/migrations/migrate.js",
"migratedb:overwrite": "x-var MIGRATION_ALLOW_CHECKSUM_UPDATE=true node lib/services/storage/migrations/migrate.js"
},
"type": "module",
"lint-staged": {
@@ -44,7 +46,7 @@
},
"license": "MIT",
"engines": {
"node": ">=20.0.0",
"node": ">=22.0.0",
"npm": ">=7.0.0"
},
"browserslist": [
@@ -54,44 +56,40 @@
"Firefox ESR"
],
"dependencies": {
"@douyinfe/semi-icons": "^2.86.0",
"@douyinfe/semi-ui": "2.86.0",
"@rematch/core": "2.2.0",
"@rematch/loading": "2.1.2",
"@sendgrid/mail": "8.1.5",
"@visactor/react-vchart": "^2.0.4",
"@visactor/vchart": "^2.0.4",
"@visactor/vchart-semi-theme": "^1.12.2",
"@vitejs/plugin-react": "5.0.2",
"@vitejs/plugin-react": "5.0.3",
"better-sqlite3": "^12.2.0",
"body-parser": "2.2.0",
"cheerio": "^1.1.2",
"cookie-session": "2.1.1",
"handlebars": "4.7.8",
"lodash": "4.17.21",
"lowdb": "7.0.1",
"markdown": "^0.5.0",
"mixpanel": "^0.18.1",
"nanoid": "5.1.5",
"node-cron": "^4.2.1",
"node-fetch": "3.3.2",
"node-mailjet": "6.0.9",
"p-throttle": "^8.0.0",
"package-up": "^5.0.0",
"puppeteer": "^24.19.0",
"puppeteer": "^24.22.0",
"puppeteer-extra": "^3.3.6",
"puppeteer-extra-plugin-stealth": "^2.11.2",
"query-string": "9.3.0",
"query-string": "9.3.1",
"react": "18.3.1",
"react-dom": "18.3.1",
"react-redux": "9.2.0",
"react-router": "7.8.2",
"react-router-dom": "7.8.2",
"redux": "5.0.1",
"redux-thunk": "3.1.0",
"react-router": "7.9.1",
"react-router-dom": "7.9.1",
"restana": "5.1.0",
"serve-static": "2.2.0",
"slack": "11.0.2",
"vite": "7.1.5",
"x-var": "^2.1.0"
"vite": "7.1.6",
"x-var": "^3.0.1",
"zustand": "^5.0.8"
},
"devDependencies": {
"@babel/core": "7.28.4",
@@ -102,14 +100,13 @@
"eslint": "9.35.0",
"eslint-config-prettier": "10.1.8",
"eslint-plugin-react": "7.37.5",
"esmock": "2.7.2",
"esmock": "2.7.3",
"history": "5.3.0",
"husky": "9.1.7",
"less": "4.4.1",
"lint-staged": "16.1.6",
"mocha": "11.7.2",
"nodemon": "^3.1.10",
"prettier": "3.6.2",
"redux-logger": "3.0.6"
"prettier": "3.6.2"
}
}

View File

@@ -0,0 +1,329 @@
import { expect } from 'chai';
import esmock from 'esmock';
// We will fully mock fs, crypto, SqliteConnection, and dynamic import of migration modules
describe('db/migrations/migrate.js - runMigrations', () => {
let calls;
let runMigrations;
let prevExitCode;
beforeEach(async () => {
calls = {
fs: { existsSync: [], mkdirSync: [], readdirSync: [], readFileSync: [] },
sql: {
getConnection: 0,
tableExists: false,
query: [],
execute: [],
withTransaction: [],
optimize: 0,
},
logs: { info: [], warn: [], error: [] },
};
// Mock fs to avoid touching disk
const fsMock = {
existsSync: (p) => {
calls.fs.existsSync.push(p);
return true;
},
mkdirSync: (p, opts) => {
calls.fs.mkdirSync.push({ p, opts });
},
readdirSync: (p) => {
calls.fs.readdirSync.push(p);
return [];
},
readFileSync: (p) => {
calls.fs.readFileSync.push(p);
return Buffer.from('dummy');
},
};
// Mock crypto sha256
const cryptoMock = {
createHash: () => ({ update: () => ({ digest: () => 'sha256sum' }) }),
};
// Mock logger
const loggerMock = {
info: (...a) => calls.logs.info.push(a),
warn: (...a) => calls.logs.warn.push(a),
error: (...a) => calls.logs.error.push(a),
};
// Mock SqliteConnection
const sqlMock = {
getConnection: () => {
calls.sql.getConnection += 1;
return {};
},
tableExists: () => calls.sql.tableExists,
query: (sql) => {
calls.sql.query.push(sql);
return [];
},
execute: (sql, params) => {
calls.sql.execute.push({ sql, params });
return { changes: 1 };
},
withTransaction: (cb) => {
calls.sql.withTransaction.push(true);
const db = {
prepare: (s) => ({ run: (...args) => calls.sql.execute.push({ sql: s, params: args }) }),
};
return cb(db);
},
optimize: () => {
calls.sql.optimize += 1;
},
};
// esmock with dependency replacements
const path = await import('node:path');
const ROOT = path.resolve('.');
const sqlPath = path.join(ROOT, 'lib', 'services', 'storage', 'SqliteConnection.js');
const loggerPath = path.join(ROOT, 'lib', 'services', 'logger.js');
const mod = await esmock(
'../../../db/migrations/migrate.js',
{},
{
fs: fsMock,
crypto: cryptoMock,
[sqlPath]: sqlMock,
[loggerPath]: loggerMock,
},
);
runMigrations = mod.runMigrations;
// remember original exitCode to restore later
prevExitCode = process.exitCode;
});
afterEach(() => {
// restore original process.exitCode
process.exitCode = prevExitCode;
});
it('logs and returns when no migration files are found', async () => {
await runMigrations();
expect(calls.logs.info.some((a) => String(a[0]).includes('No migration files'))).to.equal(true);
expect(calls.sql.getConnection).to.equal(0);
expect(calls.sql.optimize).to.equal(0);
});
it('applies a single new migration inside a transaction and records it', async () => {
// Re-mock with one file and module loader
const fsMock = {
existsSync: () => true,
mkdirSync: () => {},
readdirSync: () => ['1.init.js'],
readFileSync: () => Buffer.from('dummy'),
};
const cryptoMock = { createHash: () => ({ update: () => ({ digest: () => 'abc' }) }) };
const loggerMock = {
info: (...a) => calls.logs.info.push(a),
warn: (...a) => calls.logs.warn.push(a),
error: (...a) => calls.logs.error.push(a),
};
const sqlMock = {
getConnection: () => {
calls.sql.getConnection += 1;
return {};
},
tableExists: () => false, // schema_migrations not present yet
query: () => [],
execute: (sql, params) => {
calls.sql.execute.push({ sql, params });
return { changes: 1 };
},
withTransaction: (cb) => {
calls.sql.withTransaction.push(true);
const db = {
exec: () => {},
prepare: (s) => ({ run: (...args) => calls.sql.execute.push({ sql: s, params: args }) }),
};
return cb(db);
},
optimize: () => {
calls.sql.optimize += 1;
},
};
// The migration module: exports up(db)
const migrationModule = {
up: (db) => {
db.exec && db.exec('CREATE TABLE schema_migrations(name TEXT)');
},
};
// We need to intercept dynamic import by esmock: provide a stub for import(url)
// esmock supports mocking via a virtual module using URL matching, but simpler approach:
// place the file path that migrate.js will compute and make Node import resolve to our stub
// We simulate by mocking url.pathToFileURL is still used, but dynamic import will be handled by esmock when we map the computed path.
const path = await import('node:path');
const ROOT = path.resolve('.');
const sqlPath = path.join(ROOT, 'lib', 'services', 'storage', 'SqliteConnection.js');
const loggerPath = path.join(ROOT, 'lib', 'services', 'logger.js');
// Use global importer hook to bypass dynamic import
globalThis.__TEST_MIGRATE_IMPORT__ = async () => migrationModule;
const mod = await esmock(
'../../../db/migrations/migrate.js',
{},
{
fs: fsMock,
crypto: cryptoMock,
[sqlPath]: sqlMock,
[loggerPath]: loggerMock,
},
);
runMigrations = mod.runMigrations;
await runMigrations();
// Should have started a transaction and inserted into schema_migrations
expect(calls.sql.withTransaction.length).to.equal(1);
const inserted = calls.sql.execute.find((e) => String(e.sql).includes('INSERT INTO schema_migrations'));
expect(!!inserted).to.equal(true);
expect(calls.sql.optimize).to.equal(1);
});
it('skips already executed migration with same checksum', async () => {
const fsMock = {
existsSync: () => true,
mkdirSync: () => {},
readdirSync: () => ['1.init.js'],
readFileSync: () => Buffer.from('dummy'),
};
const cryptoMock = { createHash: () => ({ update: () => ({ digest: () => 'same' }) }) };
const loggerMock = {
info: (...a) => calls.logs.info.push(a),
warn: (...a) => calls.logs.warn.push(a),
error: (...a) => calls.logs.error.push(a),
};
const sqlMock = {
getConnection: () => {
calls.sql.getConnection += 1;
return {};
},
tableExists: () => true,
query: () => [{ name: '1.init.js', checksum: 'same' }],
execute: (sql, params) => {
calls.sql.execute.push({ sql, params });
return { changes: 1 };
},
withTransaction: (cb) => {
calls.sql.withTransaction.push(true);
const db = { prepare: (s) => ({ run: (...args) => calls.sql.execute.push({ sql: s, params: args }) }) };
return cb(db);
},
optimize: () => {
calls.sql.optimize += 1;
},
};
const path = await import('node:path');
const ROOT = path.resolve('.');
const sqlPath = path.join(ROOT, 'lib', 'services', 'storage', 'SqliteConnection.js');
const loggerPath = path.join(ROOT, 'lib', 'services', 'logger.js');
globalThis.__TEST_MIGRATE_IMPORT__ = async () => ({ up: () => {} });
const mod = await esmock(
'../../../db/migrations/migrate.js',
{},
{
fs: fsMock,
crypto: cryptoMock,
[sqlPath]: sqlMock,
[loggerPath]: loggerMock,
},
);
runMigrations = mod.runMigrations;
await runMigrations();
// Should not run transaction because it's skipped
expect(calls.sql.withTransaction.length).to.equal(0);
expect(calls.sql.optimize).to.equal(1);
});
it('aborts with exitCode=1 when a migration throws, without applying insert', async () => {
const fsMock = {
existsSync: () => true,
mkdirSync: () => {},
readdirSync: () => ['1.bad.js'],
readFileSync: () => Buffer.from('dummy'),
};
const cryptoMock = { createHash: () => ({ update: () => ({ digest: () => 'bad' }) }) };
const loggerMock = {
info: (...a) => calls.logs.info.push(a),
warn: (...a) => calls.logs.warn.push(a),
error: (...a) => calls.logs.error.push(a),
};
const sqlMock = {
getConnection: () => {
calls.sql.getConnection += 1;
return {};
},
tableExists: () => false,
query: () => [],
execute: (sql, params) => {
calls.sql.execute.push({ sql, params });
return { changes: 1 };
},
withTransaction: (cb) => {
calls.sql.withTransaction.push(true);
const db = {
exec: () => {},
prepare: (s) => ({ run: (...args) => calls.sql.execute.push({ sql: s, params: args }) }),
};
return cb(db);
},
optimize: () => {
calls.sql.optimize += 1;
},
};
const path = await import('node:path');
const ROOT = path.resolve('.');
globalThis.__TEST_MIGRATE_IMPORT__ = async () => ({
up: () => {
throw new Error('boom');
},
});
const sqlPath = path.join(ROOT, 'lib', 'services', 'storage', 'SqliteConnection.js');
const loggerPath = path.join(ROOT, 'lib', 'services', 'logger.js');
const mod = await esmock(
'../../../lib/services/storage/migrations/migrate.js',
{},
{
fs: fsMock,
crypto: cryptoMock,
[sqlPath]: sqlMock,
[loggerPath]: loggerMock,
},
);
runMigrations = mod.runMigrations;
await runMigrations();
expect(process.exitCode).to.equal(1);
// No insert into schema_migrations should be recorded since transaction failed
const inserted = calls.sql.execute.find((e) => String(e.sql).includes('INSERT INTO schema_migrations'));
expect(inserted).to.equal(undefined);
});
});

View File

@@ -1,8 +1,8 @@
const db = {};
export const setKnownListings = (jobKey, providerId, listings) => {
export const storeListings = (jobKey, providerId, listings) => {
if (!Array.isArray(listings)) throw Error('Not a valid array');
db[providerId] = listings;
};
export const getKnownListings = (jobKey, providerId) => {
export const getKnownListingHashesForJobAndProvider = (jobKey, providerId) => {
return db[providerId] || [];
};

View File

@@ -25,6 +25,7 @@ describe('#einsAImmobilien testsuite()', () => {
expect(notify.size).to.be.a('string');
expect(notify.title).to.be.a('string');
expect(notify.link).to.be.a('string');
expect(notify.address).to.be.a('string');
/** check the values if possible **/
expect(notify.size).to.be.not.empty;
expect(notify.title).to.be.not.empty;

View File

@@ -0,0 +1,142 @@
import { expect } from 'chai';
import esmock from 'esmock';
// We explicitly avoid touching the real filesystem or creating a real DB file.
// better-sqlite3 is fully mocked and operates in-memory via our stubs.
describe('SqliteConnection', () => {
let SqliteConnection;
let calls;
beforeEach(async () => {
calls = {
fs: { existsSync: [], mkdirSync: [] },
db: { pragma: [], prepare: [], transactionWraps: 0, close: 0 },
prepareAll: [],
prepareRun: [],
prepareGet: [],
processOnce: [],
logs: { warn: [], debug: [] },
};
// stub for fs
const fsMock = {
existsSync: (dir) => {
calls.fs.existsSync.push(dir);
// Pretend directory always exists to avoid mkdir
return true;
},
mkdirSync: (dir, opts) => {
calls.fs.mkdirSync.push({ dir, opts });
},
};
// Prepare object returned from db.prepare()
const prepareObj = {
all: (params) => {
calls.prepareAll.push(params);
return [{ x: 1 }];
},
run: (params) => {
calls.prepareRun.push(params);
return { changes: 1 };
},
get: (param) => {
calls.prepareGet.push(param);
// return truthy by default
return { one: 1 };
},
};
// Database mock constructor
const BetterSqlite3Mock = function (filepath, options) {
// expose on instance
this.filepath = filepath;
this.options = options;
this.pragma = (p) => {
calls.db.pragma.push(p);
return undefined;
};
this.prepare = (sql) => {
calls.db.prepare.push(sql);
return prepareObj;
};
this.transaction = (fn) => {
// better-sqlite3 returns a function that executes inside a transaction
return (cb) => {
calls.db.transactionWraps += 1;
return fn(cb);
};
};
this.close = () => {
calls.db.close += 1;
};
};
// esmock the module with our stubs
SqliteConnection = await esmock(
'../../lib/services/storage/SqliteConnection.js',
{},
{
fs: fsMock,
'better-sqlite3': { default: BetterSqlite3Mock },
},
);
});
afterEach(() => {
// ensure we can close between tests
SqliteConnection.close();
});
it('creates singleton connection and applies PRAGMAs without touching disk', () => {
const db1 = SqliteConnection.getConnection();
const db2 = SqliteConnection.getConnection();
expect(db1).to.equal(db2);
// journal_mode, synchronous, cache_size, foreign_keys, optimize
expect(calls.db.pragma).to.deep.equal([
'journal_mode = WAL',
'synchronous = NORMAL',
'cache_size = -64000',
'foreign_keys = ON',
'optimize',
]);
// mkdirSync should not be called because existsSync returned true
expect(calls.fs.mkdirSync).to.have.length(0);
});
it('executes query and execute helpers', () => {
const rows = SqliteConnection.query('SELECT 1', {});
expect(rows).to.be.an('array');
expect(rows[0]).to.deep.equal({ x: 1 });
const info = SqliteConnection.execute('UPDATE x SET y=1 WHERE id=@id', { id: 5 });
expect(info).to.have.property('changes', 1);
});
it('tableExists uses sqlite_master get()', () => {
const exists = SqliteConnection.tableExists('users');
expect(exists).to.equal(true);
});
it('withTransaction wraps callback', () => {
const result = SqliteConnection.withTransaction((db) => {
// ensure we can use the db to prepare
db.prepare('SELECT inside').all({});
return 42;
});
expect(result).to.equal(42);
expect(calls.db.prepare).to.include('SELECT inside');
});
it('optimize() delegates to PRAGMA optimize and close() calls it again then closes', () => {
SqliteConnection.optimize();
// It will use the existing connection and call pragma('optimize')
expect(calls.db.pragma).to.include('optimize');
SqliteConnection.close();
// close increments close counter
expect(calls.db.close).to.equal(1);
});
});

View File

@@ -6,7 +6,7 @@ import GeneralSettings from './views/generalSettings/GeneralSettings';
import JobMutation from './views/jobs/mutation/JobMutation';
import UserMutator from './views/user/mutation/UserMutator';
import JobInsight from './views/jobs/insights/JobInsight.jsx';
import { useDispatch, useSelector } from 'react-redux';
import { useActions, useSelector } from './services/state/store';
import { Routes, Route, Navigate } from 'react-router-dom';
import Logout from './components/logout/Logout';
import Logo from './components/logo/Logo';
@@ -20,20 +20,20 @@ import TrackingModal from './components/tracking/TrackingModal.jsx';
import { Banner } from '@douyinfe/semi-ui';
export default function FredyApp() {
const dispatch = useDispatch();
const actions = useActions();
const [loading, setLoading] = React.useState(true);
const currentUser = useSelector((state) => state.user.currentUser);
const settings = useSelector((state) => state.generalSettings.settings);
useEffect(() => {
async function init() {
await dispatch.user.getCurrentUser();
await actions.user.getCurrentUser();
if (!needsLogin()) {
await dispatch.provider.getProvider();
await dispatch.jobs.getJobs();
await dispatch.jobs.getProcessingTimes();
await dispatch.notificationAdapter.getAdapter();
await dispatch.generalSettings.getGeneralSettings();
await actions.provider.getProvider();
await actions.jobs.getJobs();
await actions.jobs.getProcessingTimes();
await actions.notificationAdapter.getAdapter();
await actions.generalSettings.getGeneralSettings();
}
setLoading(false);
}

View File

@@ -1,8 +1,6 @@
import React from 'react';
import { reduxStore } from './services/rematch/store';
import { HashRouter } from 'react-router-dom';
import { Provider } from 'react-redux';
import { createRoot } from 'react-dom/client';
import en_US from '@douyinfe/semi-ui/lib/es/locale/source/en_US';
import { LocaleProvider } from '@douyinfe/semi-ui';
@@ -18,11 +16,9 @@ initVChartSemiTheme({
});
root.render(
<Provider store={reduxStore}>
<HashRouter>
<LocaleProvider locale={en_US}>
<App />
</LocaleProvider>
</HashRouter>
</Provider>,
<HashRouter>
<LocaleProvider locale={en_US}>
<App />
</LocaleProvider>
</HashRouter>,
);

View File

@@ -43,7 +43,8 @@ export default function TrackingModal() {
</p>
<p>
However, it would be a huge help if youd allow me to collect some analytical data. Wait, before you click
"no", let me explain. If you agree, Fredy will send a ping to my Mixpanel project each time it runs.
"no", let me explain. If you agree, Fredy will send a ping once every 6 hours to my internal tracking project.
(Will be open-sourced soon)
</p>
<p>
The data includes: names of active adapters/providers, OS, architecture, Node version, and language. The

View File

@@ -1,24 +0,0 @@
import { xhrGet } from '../../xhr';
export const demoMode = {
state: {
demoMode: false,
},
reducers: {
setDemoMode: (state, payload) => {
return {
...state,
demoMode: payload.demoMode,
};
},
},
effects: {
async getDemoMode() {
try {
const response = await xhrGet('/api/demo');
this.setDemoMode(response.json);
} catch (Exception) {
console.error('Error while trying to get resource for api/demo. Error:', Exception);
}
},
},
};

View File

@@ -1,25 +0,0 @@
import { xhrGet } from '../../xhr';
export const generalSettings = {
state: {
settings: {},
},
reducers: {
//only admins
setGeneralSettings: (state, payload) => {
return {
...state,
settings: payload,
};
},
},
effects: {
async getGeneralSettings() {
try {
const response = await xhrGet('/api/admin/generalSettings');
this.setGeneralSettings(response.json);
} catch (Exception) {
console.error('Error while trying to get resource for api/admin/generalSettings. Error:', Exception);
}
},
},
};

View File

@@ -1,57 +0,0 @@
import { xhrGet } from '../../xhr';
export const jobs = {
state: {
jobs: [],
insights: {},
processingTimes: {},
},
reducers: {
setJobs: (state, payload) => {
return {
...state,
jobs: Object.freeze(payload),
};
},
setProcessingTimes: (state, payload) => {
return {
...state,
processingTimes: Object.freeze(payload),
};
},
setJobInsights: (state, payload, jobId) => {
return {
...state,
insights: {
...state.insights,
[jobId]: Object.freeze(payload),
},
};
},
},
effects: {
async getJobs() {
try {
const response = await xhrGet('/api/jobs');
this.setJobs(response.json);
} catch (Exception) {
console.error(`Error while trying to get resource for api/jobs. Error:`, Exception);
}
},
async getProcessingTimes() {
try {
const response = await xhrGet('/api/jobs/processingTimes');
this.setProcessingTimes(response.json);
} catch (Exception) {
console.error(`Error while trying to get resource for api/processingTimes. Error:`, Exception);
}
},
async getInsightDataForJob(jobId) {
try {
const response = await xhrGet(`/api/jobs/insights/${jobId}`);
this.setJobInsights(response.json, jobId);
} catch (Exception) {
console.error(`Error while trying to get resource for api/jobs/insights. Error:`, Exception);
}
},
},
};

View File

@@ -1,19 +0,0 @@
import { xhrGet } from '../../xhr';
export const notificationAdapter = {
state: [],
reducers: {
setAdapter: (state, payload) => {
return [...Object.freeze(payload)];
},
},
effects: {
async getAdapter() {
try {
const response = await xhrGet('/api/jobs/notificationAdapter');
this.setAdapter(response.json);
} catch (Exception) {
console.error(`Error while trying to get resource for api/jobs/notificationAdapter. Error:`, Exception);
}
},
},
};

View File

@@ -1,19 +0,0 @@
import { xhrGet } from '../../xhr';
export const provider = {
state: [],
reducers: {
setProvider: (state, payload) => {
return [...Object.freeze(payload)];
},
},
effects: {
async getProvider() {
try {
const response = await xhrGet('/api/jobs/provider');
this.setProvider(response.json);
} catch (Exception) {
console.error(`Error while trying to get resource for api/jobs/provider. Error:`, Exception);
}
},
},
};

View File

@@ -1,40 +0,0 @@
import { xhrGet } from '../../xhr';
export const user = {
state: {
users: [],
currentUser: null,
},
reducers: {
//only admins
setUsers: (state, payload) => {
return {
...state,
users: payload,
};
},
setCurrentUser: (state, payload) => {
return {
...state,
currentUser: Object.freeze(payload),
};
},
},
effects: {
async getUsers() {
try {
const response = await xhrGet('/api/admin/users');
this.setUsers(response.json);
} catch (Exception) {
console.error('Error while trying to get resource for api/admin/users. Error:', Exception);
}
},
async getCurrentUser() {
try {
const response = await xhrGet('/api/login/user');
this.setCurrentUser(response.json);
} catch (Exception) {
console.error('Error while trying to get resource for api/login/user. Error:', Exception);
}
},
},
};

View File

@@ -1,29 +0,0 @@
import { notificationAdapter } from './models/notificationAdapter';
import { generalSettings } from './models/generalSettings';
import createLoadingPlugin from '@rematch/loading';
import { provider } from './models/provider';
import { createLogger } from 'redux-logger';
import { jobs } from './models/jobs';
import { user } from './models/user';
import { demoMode } from './models/demoMode.js';
import { init } from '@rematch/core';
const middleware = [];
if (process.env.NODE_ENV === 'development') {
middleware.push(createLogger({ duration: false, collapsed: (getState, action, logEntry) => !logEntry.error }));
}
const store = init({
name: 'fredy',
models: {
notificationAdapter,
generalSettings,
demoMode,
provider,
jobs,
user,
},
plugins: [createLoadingPlugin({})],
redux: {
middlewares: middleware,
},
});
export const reduxStore = store;

View File

@@ -0,0 +1,171 @@
/**
* Zustand store for Fredy ui state.
*/
import { create } from 'zustand';
import { shallow } from 'zustand/shallow';
import { xhrGet } from '../xhr.js';
const logger = (config) => (set, get, api) =>
config(
(partial, replace) => {
const prev = get();
set(partial, replace);
const next = get();
if (process.env.NODE_ENV !== 'production') {
/* eslint-disable no-console */
console.info('[zustand] state changed:', { prev, next });
/* eslint-enable no-console */
}
},
get,
api,
);
// Create the Zustand store with slices and actions
export const useFredyState = create(
logger(
(set) => {
// Async actions that directly set state (no separate reducer concept)
const effects = {
notificationAdapter: {
async getAdapter() {
try {
const response = await xhrGet('/api/jobs/notificationAdapter');
set(() => ({ notificationAdapter: Object.freeze([...response.json]) }));
} catch (Exception) {
console.error(`Error while trying to get resource for api/jobs/notificationAdapter. Error:`, Exception);
}
},
},
generalSettings: {
async getGeneralSettings() {
try {
const response = await xhrGet('/api/admin/generalSettings');
set((state) => ({ generalSettings: { ...state.generalSettings, settings: response.json } }));
} catch (Exception) {
console.error('Error while trying to get resource for api/admin/generalSettings. Error:', Exception);
}
},
},
provider: {
async getProvider() {
try {
const response = await xhrGet('/api/jobs/provider');
set(() => ({ provider: Object.freeze([...response.json]) }));
} catch (Exception) {
console.error(`Error while trying to get resource for api/jobs/provider. Error:`, Exception);
}
},
},
jobs: {
async getJobs() {
try {
const response = await xhrGet('/api/jobs');
set((state) => ({ jobs: { ...state.jobs, jobs: Object.freeze(response.json) } }));
} catch (Exception) {
console.error(`Error while trying to get resource for api/jobs. Error:`, Exception);
}
},
async getProcessingTimes() {
try {
const response = await xhrGet('/api/jobs/processingTimes');
set((state) => ({ jobs: { ...state.jobs, processingTimes: Object.freeze(response.json) } }));
} catch (Exception) {
console.error(`Error while trying to get resource for api/processingTimes. Error:`, Exception);
}
},
async getInsightDataForJob(jobId) {
try {
const response = await xhrGet(`/api/jobs/insights/${jobId}`);
set((state) => ({
jobs: {
...state.jobs,
insights: { ...state.jobs.insights, [jobId]: Object.freeze(response.json) },
},
}));
} catch (Exception) {
console.error(`Error while trying to get resource for api/jobs/insights. Error:`, Exception);
}
},
},
user: {
async getUsers() {
try {
const response = await xhrGet('/api/admin/users');
set((state) => ({ user: { ...state.user, users: response.json } }));
} catch (Exception) {
console.error('Error while trying to get resource for api/admin/users. Error:', Exception);
}
},
async getCurrentUser() {
try {
const response = await xhrGet('/api/login/user');
set((state) => ({ user: { ...state.user, currentUser: Object.freeze(response.json) } }));
} catch (Exception) {
console.error('Error while trying to get resource for api/login/user. Error:', Exception);
}
},
},
demoMode: {
async getDemoMode() {
try {
const response = await xhrGet('/api/demo');
set((state) => ({
demoMode: { ...state.demoMode, demoMode: response.json.demoMode },
}));
} catch (Exception) {
console.error('Error while trying to get resource for api/demo. Error:', Exception);
}
},
},
};
// Initial state
const initial = {
notificationAdapter: [],
generalSettings: { settings: {} },
demoMode: { demoMode: false },
provider: [],
jobs: { jobs: [], insights: {}, processingTimes: {} },
user: { users: [], currentUser: null },
};
// Expose actions by grouping them per slice
const actions = {
notificationAdapter: { ...effects.notificationAdapter },
generalSettings: { ...effects.generalSettings },
demoMode: { ...effects.demoMode },
provider: { ...effects.provider },
jobs: { ...effects.jobs },
user: { ...effects.user },
};
return {
...initial,
__actions: { actions },
};
},
{ name: 'fredy' },
),
);
/**
* Selector hook, drop-in replacement for react-redux useSelector.
* Pass a selector function and optional equality function. Defaults to shallow comparison.
* @template T
* @param {(state: FredyState) => T} selector
* @param {(a: T, b: T) => boolean} [equalityFn]
* @returns {T}
*/
export function useSelector(selector, equalityFn = shallow) {
return useFredyState(selector, equalityFn);
}
/**
* Actions hook returning grouped async actions per slice.
* Example: const { jobs } = useActions(); await jobs.getJobs();
* @returns {{notificationAdapter: any, generalSettings: any, demoMode: any, provider: any, jobs: any, user: any}}
*/
export function useActions() {
return useFredyState((s) => s.__actions.actions);
}

View File

@@ -1,8 +1,8 @@
import React from 'react';
import { useDispatch, useSelector } from 'react-redux';
import { useActions, useSelector } from '../../services/state/store';
import { Divider, TimePicker, Button, Checkbox } from '@douyinfe/semi-ui';
import { Divider, TimePicker, Button, Checkbox, Input } from '@douyinfe/semi-ui';
import { InputNumber } from '@douyinfe/semi-ui';
import Headline from '../../components/headline/Headline';
import { xhrPost } from '../../services/xhr';
@@ -15,6 +15,7 @@ import {
IconSignal,
IconLineChartStroked,
IconSearch,
IconFolder,
} from '@douyinfe/semi-icons';
import './GeneralSettings.less';
@@ -35,7 +36,7 @@ function formatFromTBackend(time) {
}
const GeneralSettings = function GeneralSettings() {
const dispatch = useDispatch();
const actions = useActions();
const [loading, setLoading] = React.useState(true);
const settings = useSelector((state) => state.generalSettings.settings);
@@ -46,10 +47,11 @@ const GeneralSettings = function GeneralSettings() {
const [workingHourTo, setWorkingHourTo] = React.useState(null);
const [demoMode, setDemoMode] = React.useState(null);
const [analyticsEnabled, setAnalyticsEnabled] = React.useState(null);
const [sqlitePath, setSqlitePath] = React.useState(null);
React.useEffect(() => {
async function init() {
await dispatch.generalSettings.getGeneralSettings();
await actions.generalSettings.getGeneralSettings();
setLoading(false);
}
@@ -64,6 +66,7 @@ const GeneralSettings = function GeneralSettings() {
setWorkingHourTo(settings?.workingHours?.to);
setAnalyticsEnabled(settings?.analyticsEnabled || false);
setDemoMode(settings?.demoMode || false);
setSqlitePath(settings?.sqlitepath);
}
init();
@@ -87,6 +90,10 @@ const GeneralSettings = function GeneralSettings() {
Toast.error('Working hours to and from must be set if either to or from has been set before.');
return;
}
if (nullOrEmpty(sqlitePath)) {
Toast.error('SQLite db path cannot be empty.');
return;
}
try {
await xhrPost('/api/admin/generalSettings', {
interval,
@@ -97,6 +104,7 @@ const GeneralSettings = function GeneralSettings() {
},
demoMode,
analyticsEnabled,
sqlitepath: sqlitePath,
});
} catch (exception) {
console.error(exception);
@@ -146,6 +154,36 @@ const GeneralSettings = function GeneralSettings() {
/>
</SegmentPart>
<Divider margin="1rem" />
<SegmentPart
name="SQLite Database path"
helpText="The directory where Fredy stores its SQLite database files."
Icon={IconFolder}
>
<Banner
fullMode={false}
type="warning"
closeIcon={null}
title={<div style={{ fontWeight: 600, fontSize: '14px', lineHeight: '20px' }}>Warning</div>}
style={{ marginBottom: '1rem' }}
description={
<div>
Changing the path later may result in data loss.
<br />
You <b>must</b> restart Fredy immediately after changing this setting!
</div>
}
/>
<Input
type="text"
placeholder="Select folder"
value={sqlitePath}
onChange={(value) => {
setSqlitePath(value);
}}
/>
</SegmentPart>
<Divider margin="1rem" />
<SegmentPart
name="Working hours"
helpText="During this hours, Fredy will search for new apartments. If nothing is configured, Fredy will search around the clock."

View File

@@ -1,7 +1,7 @@
import React from 'react';
import JobTable from '../../components/table/JobTable';
import { useSelector, useDispatch } from 'react-redux';
import { useSelector, useActions } from '../../services/state/store';
import { xhrDelete, xhrPut } from '../../services/xhr';
import { useNavigate } from 'react-router-dom';
import ProcessingTimes from './ProcessingTimes';
@@ -13,13 +13,13 @@ export default function Jobs() {
const jobs = useSelector((state) => state.jobs.jobs);
const processingTimes = useSelector((state) => state.jobs.processingTimes);
const navigate = useNavigate();
const dispatch = useDispatch();
const actions = useActions();
const onJobRemoval = async (jobId) => {
try {
await xhrDelete('/api/jobs', { jobId });
Toast.success('Job successfully remove');
await dispatch.jobs.getJobs();
await actions.jobs.getJobs();
} catch (error) {
Toast.error(error);
}
@@ -29,7 +29,7 @@ export default function Jobs() {
try {
await xhrPut(`/api/jobs/${jobId}/status`, { status });
Toast.success('Job status successfully changed');
await dispatch.jobs.getJobs();
await actions.jobs.getJobs();
} catch (error) {
Toast.error(error);
}

View File

@@ -1,6 +1,8 @@
import React from 'react';
import { format } from '../../services/time/timeService';
import { Descriptions } from '@douyinfe/semi-ui';
import { Button, Descriptions, Toast } from '@douyinfe/semi-ui';
import { IconPlayCircle } from '@douyinfe/semi-icons';
import { xhrPost } from '../../services/xhr.js';
export default function ProcessingTimes({ processingTimes = {} }) {
if (Object.keys(processingTimes).length === 0) {
@@ -24,6 +26,19 @@ export default function ProcessingTimes({ processingTimes = {} }) {
<Descriptions.Item itemKey="Next run">
{format(processingTimes.lastRun + processingTimes.interval * 60000)}
</Descriptions.Item>
<Descriptions.Item itemKey="Find Listings now">
<Button
size="small"
icon={<IconPlayCircle />}
aria-label="Start now"
onClick={async () => {
await xhrPost('/api/jobs/startAll', null);
Toast.success('Successfully triggered Fredy search.');
}}
>
Search now
</Button>
</Descriptions.Item>
</>
)}
</Descriptions>

View File

@@ -2,20 +2,20 @@ import React from 'react';
import { roundToHour } from '../../../services/time/timeService';
import Headline from '../../../components/headline/Headline';
import { useDispatch, useSelector } from 'react-redux';
import { useActions, useSelector } from '../../../services/state/store';
import { useParams } from 'react-router-dom';
import Linechart from './Linechart';
const JobInsight = function JobInsight() {
const dispatch = useDispatch();
const actions = useActions();
const insights = useSelector((state) => state.jobs.insights);
const jobs = useSelector((state) => state.jobs.jobs);
const params = useParams();
React.useEffect(() => {
dispatch.jobs.getInsightDataForJob(params.jobId);
dispatch.jobs.getJobs();
actions.jobs.getInsightDataForJob(params.jobId);
actions.jobs.getJobs();
}, []);
const getData = () => {

View File

@@ -5,7 +5,7 @@ import NotificationAdapterTable from '../../../components/table/NotificationAdap
import ProviderTable from '../../../components/table/ProviderTable';
import ProviderMutator from './components/provider/ProviderMutator';
import Headline from '../../../components/headline/Headline';
import { useDispatch, useSelector } from 'react-redux';
import { useActions, useSelector } from '../../../services/state/store';
import { xhrPost } from '../../../services/xhr';
import { useNavigate, useParams } from 'react-router-dom';
import { Divider, Input, Switch, Button, TagInput, Toast } from '@douyinfe/semi-ui';
@@ -34,7 +34,7 @@ export default function JobMutator() {
const [notificationAdapterData, setNotificationAdapterData] = useState(defaultNotificationAdapter);
const [enabled, setEnabled] = useState(defaultEnabled);
const navigate = useNavigate();
const dispatch = useDispatch();
const actions = useActions();
const isSavingEnabled = () => {
return Boolean(notificationAdapterData.length && providerData.length && name);
@@ -50,7 +50,7 @@ export default function JobMutator() {
enabled,
jobId: jobToBeEdit?.id || null,
});
await dispatch.jobs.getJobs();
await actions.jobs.getJobs();
Toast.success('Job successfully saved...');
navigate('/jobs');
} catch (Exception) {

View File

@@ -3,7 +3,7 @@ import React, { useState } from 'react';
import { transform } from '../../../../../services/transformer/notificationAdapterTransformer';
import { xhrPost } from '../../../../../services/xhr';
import Help from './NotificationHelpDisplay';
import { useSelector } from 'react-redux';
import { useSelector } from '../../../../../services/state/store';
import { Banner, Button, Form, Modal, Select, Switch } from '@douyinfe/semi-ui';
import './NotificationAdapterMutator.less';

View File

@@ -2,7 +2,7 @@ import React, { useState } from 'react';
import { Banner, Modal, Select, Input } from '@douyinfe/semi-ui';
import { transform } from '../../../../../services/transformer/providerTransformer';
import { useSelector } from 'react-redux';
import { useSelector } from '../../../../../services/state/store';
import { IconLikeHeart } from '@douyinfe/semi-icons';
import './ProviderMutator.less';

View File

@@ -4,14 +4,14 @@ import cityBackground from '../../assets/city_background.jpg';
import Logo from '../../components/logo/Logo';
import { xhrPost } from '../../services/xhr';
import { useNavigate } from 'react-router-dom';
import { useDispatch, useSelector } from 'react-redux';
import { useActions, useSelector } from '../../services/state/store';
import { Input, Button, Banner, Toast } from '@douyinfe/semi-ui';
import './login.less';
import { IconUser, IconLock } from '@douyinfe/semi-icons';
export default function Login() {
const dispatch = useDispatch();
const actions = useActions();
const [username, setUserName] = React.useState('');
const [password, setPassword] = React.useState('');
const [error, setError] = React.useState(null);
@@ -20,7 +20,7 @@ export default function Login() {
useEffect(() => {
async function init() {
await dispatch.demoMode.getDemoMode();
await actions.demoMode.getDemoMode();
}
init();
@@ -46,7 +46,7 @@ export default function Login() {
Toast.success('Login successful!');
await dispatch.user.getCurrentUser();
await actions.user.getCurrentUser();
navigate('/jobs');
};

View File

@@ -2,7 +2,7 @@ import React from 'react';
import { Toast } from '@douyinfe/semi-ui';
import UserTable from '../../components/table/UserTable';
import { useDispatch, useSelector } from 'react-redux';
import { useActions, useSelector } from '../../services/state/store';
import { IconPlus } from '@douyinfe/semi-icons';
import { Button } from '@douyinfe/semi-ui';
import UserRemovalModal from './UserRemovalModal';
@@ -12,7 +12,7 @@ import { useNavigate } from 'react-router-dom';
import './Users.less';
const Users = function Users() {
const dispatch = useDispatch();
const actions = useActions();
const [loading, setLoading] = React.useState(true);
const users = useSelector((state) => state.user.users);
const [userIdToBeRemoved, setUserIdToBeRemoved] = React.useState(null);
@@ -20,7 +20,7 @@ const Users = function Users() {
React.useEffect(() => {
async function init() {
await dispatch.user.getUsers();
await actions.user.getUsers();
setLoading(false);
}
@@ -32,8 +32,8 @@ const Users = function Users() {
await xhrDelete('/api/admin/users', { userId: userIdToBeRemoved });
Toast.success('User successfully remove');
setUserIdToBeRemoved(null);
await dispatch.jobs.getJobs();
await dispatch.user.getUsers();
await actions.jobs.getJobs();
await actions.user.getUsers();
} catch (error) {
Toast.error(error);
setUserIdToBeRemoved(null);

View File

@@ -2,7 +2,7 @@ import React from 'react';
import { xhrGet, xhrPost } from '../../../services/xhr';
import { useNavigate, useParams } from 'react-router-dom';
import { useDispatch } from 'react-redux';
import { useActions } from '../../../services/state/store';
import { Divider, Input, Switch, Button, Toast } from '@douyinfe/semi-ui';
import './UserMutator.less';
import { SegmentPart } from '../../../components/segment/SegmentPart';
@@ -16,7 +16,7 @@ const UserMutator = function UserMutator() {
const [isAdmin, setIsAdmin] = React.useState(false);
const navigate = useNavigate();
const dispatch = useDispatch();
const actions = useActions();
React.useEffect(() => {
async function init() {
@@ -48,7 +48,7 @@ const UserMutator = function UserMutator() {
password2,
isAdmin,
});
await dispatch.user.getUsers();
await actions.user.getUsers();
Toast.success('User successfully saved...');
navigate('/users');
} catch (error) {

252
yarn.lock
View File

@@ -11,14 +11,6 @@
regexparam "^3.0.0"
trouter "^4.0.0"
"@ampproject/remapping@^2.2.0":
version "2.3.0"
resolved "https://registry.yarnpkg.com/@ampproject/remapping/-/remapping-2.3.0.tgz#ed441b6fa600072520ce18b43d2c8cc8caecc7f4"
integrity sha512-30iZtAPgz+LTIYoeivqYo853f02jBYSd5uGnGpkFV0M3xOt9aN73erkgYAmZU43x4VfqcnLxW9Kpg3R5LC4YYw==
dependencies:
"@jridgewell/gen-mapping" "^0.3.5"
"@jridgewell/trace-mapping" "^0.3.24"
"@babel/code-frame@^7.0.0", "@babel/code-frame@^7.27.1":
version "7.27.1"
resolved "https://registry.yarnpkg.com/@babel/code-frame/-/code-frame-7.27.1.tgz#200f715e66d52a23b221a9435534a91cc13ad5be"
@@ -33,7 +25,7 @@
resolved "https://registry.yarnpkg.com/@babel/compat-data/-/compat-data-7.28.0.tgz#9fc6fd58c2a6a15243cd13983224968392070790"
integrity sha512-60X7qkglvrap8mn1lh2ebxXdZYtUcpd7gsmy9kLaBJ4i/WdY8PqTSdxyA8qraikqKQK5C1KRBKXqznrVapyNaw==
"@babel/core@7.28.4":
"@babel/core@7.28.4", "@babel/core@^7.28.4":
version "7.28.4"
resolved "https://registry.yarnpkg.com/@babel/core/-/core-7.28.4.tgz#12a550b8794452df4c8b084f95003bce1742d496"
integrity sha512-2BCOP7TN8M+gVDj7/ht3hsaO/B/n5oDbiAyyvnRlNOs+u1o+JWNYTQrmpuNp1/Wq2gcFrI01JAW+paEKDMx/CA==
@@ -54,27 +46,6 @@
json5 "^2.2.3"
semver "^6.3.1"
"@babel/core@^7.28.3":
version "7.28.3"
resolved "https://registry.yarnpkg.com/@babel/core/-/core-7.28.3.tgz#aceddde69c5d1def69b839d09efa3e3ff59c97cb"
integrity sha512-yDBHV9kQNcr2/sUr9jghVyz9C3Y5G2zUM2H2lo+9mKv4sFgbA8s8Z9t8D1jiTkGoO/NoIfKMyKWr4s6CN23ZwQ==
dependencies:
"@ampproject/remapping" "^2.2.0"
"@babel/code-frame" "^7.27.1"
"@babel/generator" "^7.28.3"
"@babel/helper-compilation-targets" "^7.27.2"
"@babel/helper-module-transforms" "^7.28.3"
"@babel/helpers" "^7.28.3"
"@babel/parser" "^7.28.3"
"@babel/template" "^7.27.2"
"@babel/traverse" "^7.28.3"
"@babel/types" "^7.28.2"
convert-source-map "^2.0.0"
debug "^4.1.0"
gensync "^1.0.0-beta.2"
json5 "^2.2.3"
semver "^6.3.1"
"@babel/eslint-parser@7.28.4":
version "7.28.4"
resolved "https://registry.yarnpkg.com/@babel/eslint-parser/-/eslint-parser-7.28.4.tgz#80dd86e0aeaae9704411a044db60e1ae6477d93f"
@@ -238,14 +209,6 @@
"@babel/traverse" "^7.28.3"
"@babel/types" "^7.28.2"
"@babel/helpers@^7.28.3":
version "7.28.3"
resolved "https://registry.yarnpkg.com/@babel/helpers/-/helpers-7.28.3.tgz#b83156c0a2232c133d1b535dd5d3452119c7e441"
integrity sha512-PTNtvUQihsAsDHMOP5pfobP8C6CM4JWXmP8DrEIt46c3r2bf87Ua1zoqevsMo9g+tWDwgWrFP5EIxuBx5RudAw==
dependencies:
"@babel/template" "^7.27.2"
"@babel/types" "^7.28.2"
"@babel/helpers@^7.28.4":
version "7.28.4"
resolved "https://registry.yarnpkg.com/@babel/helpers/-/helpers-7.28.4.tgz#fe07274742e95bdf7cf1443593eeb8926ab63827"
@@ -1010,7 +973,7 @@
remark-gfm "^4.0.0"
scroll-into-view-if-needed "^2.2.24"
"@douyinfe/semi-icons@2.86.0":
"@douyinfe/semi-icons@2.86.0", "@douyinfe/semi-icons@^2.86.0":
version "2.86.0"
resolved "https://registry.yarnpkg.com/@douyinfe/semi-icons/-/semi-icons-2.86.0.tgz#ee4355c81616ea4325627a3bb607ed9f9b9afac3"
integrity sha512-KEDlYYP1wdOqN28Ck0YcdCx7mSks8SRY4w4KKbXPaROzYNEyT2BRcJxwysMHfxL2IDfsroHrRPJsX9pnrmQqTg==
@@ -1374,12 +1337,12 @@
resolved "https://registry.yarnpkg.com/@pkgjs/parseargs/-/parseargs-0.11.0.tgz#a77ea742fab25775145434eb1d2328cf5013ac33"
integrity sha512-+1VkjdD0QBLPodGrJUeqarH8VAIvQODIbwh9XpP5Syisf7YoQgsJKPNFoqqLQlu+VQ/tVSshMR6loPMn8U+dPg==
"@puppeteer/browsers@2.10.8":
version "2.10.8"
resolved "https://registry.yarnpkg.com/@puppeteer/browsers/-/browsers-2.10.8.tgz#80e983ca0365478b39c4c0f559785345393f8fa2"
integrity sha512-f02QYEnBDE0p8cteNoPYHHjbDuwyfbe4cCIVlNi8/MRicIxFW4w4CfgU0LNgWEID6s06P+hRJ1qjpBLMhPRCiQ==
"@puppeteer/browsers@2.10.10":
version "2.10.10"
resolved "https://registry.yarnpkg.com/@puppeteer/browsers/-/browsers-2.10.10.tgz#f806f92d966918c931fb9c48052eba2db848beaa"
integrity sha512-3ZG500+ZeLql8rE0hjfhkycJjDj0pI/btEh3L9IkWUYcOrgP0xCNRq3HbtbqOPbvDhFaAWD88pDFtlLv8ns8gA==
dependencies:
debug "^4.4.1"
debug "^4.4.3"
extract-zip "^2.0.1"
progress "^2.0.3"
proxy-agent "^6.5.0"
@@ -1387,16 +1350,6 @@
tar-fs "^3.1.0"
yargs "^17.7.2"
"@rematch/core@2.2.0":
version "2.2.0"
resolved "https://registry.yarnpkg.com/@rematch/core/-/core-2.2.0.tgz#c4e6cc9d369d341afe2345842f43c255b7a44e90"
integrity sha512-Sj3nC/2X+bOBZeOf4jdJ00nhCcx9wLbVK9SOs6eFR4Y1qKXqRY0hGigbQgfTpCdjRFlwTHHfN3m41MlNvMhDgw==
"@rematch/loading@2.1.2":
version "2.1.2"
resolved "https://registry.yarnpkg.com/@rematch/loading/-/loading-2.1.2.tgz#1dc680d445cd2d1234489cb69816278d02cf2216"
integrity sha512-3fWUvWkIxP+BEi2LCKYKaUkMFCT0MDcN1xQD19tPNufMry7skqybahqm9/ugs9wIji1n3ObF7yHkrb01E+N3Tw==
"@resvg/resvg-js-android-arm-eabi@2.4.1":
version "2.4.1"
resolved "https://registry.yarnpkg.com/@resvg/resvg-js-android-arm-eabi/-/resvg-js-android-arm-eabi-2.4.1.tgz#49dc9722f95096f8aff70186deae8e148d60dce5"
@@ -1475,10 +1428,10 @@
"@resvg/resvg-js-win32-ia32-msvc" "2.4.1"
"@resvg/resvg-js-win32-x64-msvc" "2.4.1"
"@rolldown/pluginutils@1.0.0-beta.34":
version "1.0.0-beta.34"
resolved "https://registry.yarnpkg.com/@rolldown/pluginutils/-/pluginutils-1.0.0-beta.34.tgz#4421645c676926faa4574940d72fa7ce0ec7d419"
integrity sha512-LyAREkZHP5pMom7c24meKmJCdhf2hEyvam2q0unr3or9ydwDL+DJ8chTF6Av/RFPb3rH8UFBdMzO5MxTZW97oA==
"@rolldown/pluginutils@1.0.0-beta.35":
version "1.0.0-beta.35"
resolved "https://registry.yarnpkg.com/@rolldown/pluginutils/-/pluginutils-1.0.0-beta.35.tgz#1a477e7742b154b67519d40e4fc17485de338e7a"
integrity sha512-slYrCpoxJUqzFDDNlvrOYRazQUNRvWPjXA17dAOISY3rDMxX6k8K4cj2H+hEYMHF81HO3uNd5rHVigAWRM5dSg==
"@rollup/rollup-android-arm-eabi@4.49.0":
version "4.49.0"
@@ -1764,11 +1717,6 @@
resolved "https://registry.yarnpkg.com/@types/unist/-/unist-2.0.11.tgz#11af57b127e32487774841f7a4e54eab166d03c4"
integrity sha512-CmBKiL6NNo/OqgmMn95Fk9Whlp2mtvIv+KNpQKN2F4SjvrEesubTRWGYSg+BnWZOnlCaSTU1sMpsBOzgbYhnsA==
"@types/use-sync-external-store@^0.0.6":
version "0.0.6"
resolved "https://registry.yarnpkg.com/@types/use-sync-external-store/-/use-sync-external-store-0.0.6.tgz#60be8d21baab8c305132eb9cb912ed497852aadc"
integrity sha512-zFDAD+tlpf2r4asuHEj0XH6pY6i0g5NeAHPn+15wk3BV6JA69eERFXC1gyGThDkVa1zCyKr5jox1+2LbV/AMLg==
"@types/yauzl@^2.9.1":
version "2.10.3"
resolved "https://registry.yarnpkg.com/@types/yauzl/-/yauzl-2.10.3.tgz#e9b2808b4f109504a03cda958259876f61017999"
@@ -1947,15 +1895,15 @@
"@turf/invariant" "^6.5.0"
eventemitter3 "^4.0.7"
"@vitejs/plugin-react@5.0.2":
version "5.0.2"
resolved "https://registry.yarnpkg.com/@vitejs/plugin-react/-/plugin-react-5.0.2.tgz#3b5d73fc0e4370a0fafe27154d2c208e2bca8f71"
integrity sha512-tmyFgixPZCx2+e6VO9TNITWcCQl8+Nl/E8YbAyPVv85QCc7/A3JrdfG2A8gIzvVhWuzMOVrFW1aReaNxrI6tbw==
"@vitejs/plugin-react@5.0.3":
version "5.0.3"
resolved "https://registry.yarnpkg.com/@vitejs/plugin-react/-/plugin-react-5.0.3.tgz#182ea45406d89e55b4e35c92a4a8c2c8388726c8"
integrity sha512-PFVHhosKkofGH0Yzrw1BipSedTH68BFF8ZWy1kfUpCtJcouXXY0+racG8sExw7hw0HoX36813ga5o3LTWZ4FUg==
dependencies:
"@babel/core" "^7.28.3"
"@babel/core" "^7.28.4"
"@babel/plugin-transform-react-jsx-self" "^7.27.1"
"@babel/plugin-transform-react-jsx-source" "^7.27.1"
"@rolldown/pluginutils" "1.0.0-beta.34"
"@rolldown/pluginutils" "1.0.0-beta.35"
"@types/babel__core" "^7.20.5"
react-refresh "^0.17.0"
@@ -1979,13 +1927,6 @@ acorn@^8.0.0, acorn@^8.15.0:
resolved "https://registry.yarnpkg.com/acorn/-/acorn-8.15.0.tgz#a360898bc415edaac46c8241f6383975b930b816"
integrity sha512-NZyJarBfL7nWwIq+FDL6Zp/yHEhePMNnnJ0y3qfieCrmNvYct8uvtiV41UvlSe6apAfk0fY1FbWx+NwfmpvtTg==
agent-base@6:
version "6.0.2"
resolved "https://registry.yarnpkg.com/agent-base/-/agent-base-6.0.2.tgz#49fff58577cfee3f37176feab4c22e00f86d7f77"
integrity sha512-RZNwNclF7+MS/8bDg70amg32dyeZGZxiDuQmZxKLAlQjr3jGyLx+4Kkk58UO7D2QdgFIQCovuSuZESne6RG6XQ==
dependencies:
debug "4"
agent-base@^7.1.0, agent-base@^7.1.2:
version "7.1.4"
resolved "https://registry.yarnpkg.com/agent-base/-/agent-base-7.1.4.tgz#e3cd76d4c548ee895d3c3fd8dc1f6c5b9032e7a8"
@@ -2827,6 +2768,13 @@ debug@4, debug@^4, debug@^4.0.0, debug@^4.1.0, debug@^4.1.1, debug@^4.3.1, debug
dependencies:
ms "^2.1.3"
debug@^4.4.3:
version "4.4.3"
resolved "https://registry.yarnpkg.com/debug/-/debug-4.4.3.tgz#c6ae432d9bd9662582fce08709b038c58e9e3d6a"
integrity sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==
dependencies:
ms "^2.1.3"
decamelize@^4.0.0:
version "4.0.0"
resolved "https://registry.yarnpkg.com/decamelize/-/decamelize-4.0.0.tgz#aa472d7bf660eb15f3494efd531cab7f2a709837"
@@ -2851,11 +2799,6 @@ decompress-response@^6.0.0:
dependencies:
mimic-response "^3.1.0"
deep-diff@^0.3.5:
version "0.3.8"
resolved "https://registry.yarnpkg.com/deep-diff/-/deep-diff-0.3.8.tgz#c01de63efb0eec9798801d40c7e0dae25b582c84"
integrity sha512-yVn6RZmHiGnxRKR9sJb3iVV2XTF1Ghh2DiWRZ3dMnGc43yUdWWF/kX6lQyk3+P84iprfWKU/8zFTrlkvtFm1ug==
deep-extend@^0.6.0:
version "0.6.0"
resolved "https://registry.yarnpkg.com/deep-extend/-/deep-extend-0.6.0.tgz#c4fa7c95404a17a9c3e8ca7e1537312b736330ac"
@@ -3379,10 +3322,10 @@ eslint@9.35.0:
natural-compare "^1.4.0"
optionator "^0.9.3"
esmock@2.7.2:
version "2.7.2"
resolved "https://registry.yarnpkg.com/esmock/-/esmock-2.7.2.tgz#af8f0116d1b550809f46d2fc36fc24c88c73faf7"
integrity sha512-/ilhkWbW4FXgQpRbS0LZpKG1AFkiFZkmapP/868Lqa4hSKgKVtMilFXlQrIMssLzyvpeDVg2Q9L3VInnqYoTAg==
esmock@2.7.3:
version "2.7.3"
resolved "https://registry.yarnpkg.com/esmock/-/esmock-2.7.3.tgz#25d8fd57b9608f9430185c501e7dab91fb1247bc"
integrity sha512-/M/YZOjgyLaVoY6K83pwCsGE1AJQnj4S4GyXLYgi/Y79KL8EeW6WU7Rmjc89UO7jv6ec8+j34rKeWOfiLeEu0A==
espree@^10.0.1, espree@^10.4.0:
version "10.4.0"
@@ -4056,14 +3999,6 @@ http-proxy-agent@^7.0.0, http-proxy-agent@^7.0.1:
agent-base "^7.1.0"
debug "^4.3.4"
https-proxy-agent@5.0.0:
version "5.0.0"
resolved "https://registry.yarnpkg.com/https-proxy-agent/-/https-proxy-agent-5.0.0.tgz#e2a90542abb68a762e0a0850f6c9edadfd8506b2"
integrity sha512-EkYm5BcKUGiduxzSt3Eppko+PiNWNEpa4ySk9vTC6wDsQJW9rHSa+UhGNJoRYp7bz6Ht1eaRIa6QaJqO5rCFbA==
dependencies:
agent-base "6"
debug "4"
https-proxy-agent@^7.0.6:
version "7.0.6"
resolved "https://registry.yarnpkg.com/https-proxy-agent/-/https-proxy-agent-7.0.6.tgz#da8dfeac7da130b05c2ba4b59c9b6cd66611a6b9"
@@ -4720,13 +4655,6 @@ lottie-web@^5.12.2:
resolved "https://registry.yarnpkg.com/lottie-web/-/lottie-web-5.13.0.tgz#441d3df217cc8ba302338c3f168e1a3af0f221d3"
integrity sha512-+gfBXl6sxXMPe8tKQm7qzLnUy5DUPJPKIyRHwtpCpyUEYjHYRJC/5gjUvdkuO2c3JllrPtHXH5UJJK8LRYl5yQ==
lowdb@7.0.1:
version "7.0.1"
resolved "https://registry.yarnpkg.com/lowdb/-/lowdb-7.0.1.tgz#7354a684547d76206b1c730b9434604235b125e5"
integrity sha512-neJAj8GwF0e8EpycYIDFqEPcx9Qz4GUho20jWFR7YiFeXzF1YMLdxB36PypcTSPMA+4+LvgyMacYhlr18Zlymw==
dependencies:
steno "^4.0.2"
lru-cache@^10.2.0:
version "10.4.3"
resolved "https://registry.yarnpkg.com/lru-cache/-/lru-cache-10.4.3.tgz#410fc8a17b70e598013df257c2446b7f3383f119"
@@ -5450,13 +5378,6 @@ mixin-object@^2.0.1:
for-in "^0.1.3"
is-extendable "^0.1.1"
mixpanel@^0.18.1:
version "0.18.1"
resolved "https://registry.yarnpkg.com/mixpanel/-/mixpanel-0.18.1.tgz#beefdce6c260165f4e2059c8cdd34c5c557162f7"
integrity sha512-YD1xfn6WP6ZLQ6Pmgh0KgdXhueJEsrodThMTsHzHMH0VbWa9ck8s+ynDtM83OSgt+yQ61W/SQNrH8Y4wIwocGg==
dependencies:
https-proxy-agent "5.0.0"
mkdirp-classic@^0.5.2, mkdirp-classic@^0.5.3:
version "0.5.3"
resolved "https://registry.yarnpkg.com/mkdirp-classic/-/mkdirp-classic-0.5.3.tgz#fa10c9115cc6d8865be221ba47ee9bed78601113"
@@ -5543,6 +5464,11 @@ node-abi@^3.3.0:
dependencies:
semver "^7.3.5"
node-cron@^4.2.1:
version "4.2.1"
resolved "https://registry.yarnpkg.com/node-cron/-/node-cron-4.2.1.tgz#6979be4aee4702f06322d21220df8de252c8e265"
integrity sha512-lgimEHPE/QDgFlywTd8yTR61ptugX3Qer29efeyWw2rv259HtGBNn1vZVmp8lB9uo9wC0t/AT4iGqXxia+CJFg==
node-domexception@^1.0.0:
version "1.0.0"
resolved "https://registry.yarnpkg.com/node-domexception/-/node-domexception-1.0.0.tgz#6888db46a1f71c0b76b3f7555016b63fe64766e5"
@@ -6049,16 +5975,17 @@ punycode@^2.1.0:
resolved "https://registry.yarnpkg.com/punycode/-/punycode-2.3.1.tgz#027422e2faec0b25e1549c3e1bd8309b9133b6e5"
integrity sha512-vYt7UD1U9Wg6138shLtLOvdAu+8DsC/ilFtEVHcH+wydcSpNE20AfSOduf6MkRFahL5FY7X1oU7nKVZFtfq8Fg==
puppeteer-core@24.19.0:
version "24.19.0"
resolved "https://registry.yarnpkg.com/puppeteer-core/-/puppeteer-core-24.19.0.tgz#038f5229b9910f5daf717d5aaff3b63228afbf6c"
integrity sha512-qsEys4OIb2VGC2tNWKAs4U0mnjkIAxueMOOzk2nEFM9g4Y8QuvYkEMtmwsEdvzNGsUFd7DprOQfABmlN7WBOlg==
puppeteer-core@24.22.0:
version "24.22.0"
resolved "https://registry.yarnpkg.com/puppeteer-core/-/puppeteer-core-24.22.0.tgz#4d576b1a2b7699c088d3f0e843c32d81df82c3a6"
integrity sha512-oUeWlIg0pMz8YM5pu0uqakM+cCyYyXkHBxx9di9OUELu9X9+AYrNGGRLK9tNME3WfN3JGGqQIH3b4/E9LGek/w==
dependencies:
"@puppeteer/browsers" "2.10.8"
"@puppeteer/browsers" "2.10.10"
chromium-bidi "8.0.0"
debug "^4.4.1"
debug "^4.4.3"
devtools-protocol "0.0.1495869"
typed-query-selector "^2.12.0"
webdriver-bidi-protocol "0.2.11"
ws "^8.18.3"
puppeteer-extra-plugin-stealth@^2.11.2:
@@ -6108,16 +6035,16 @@ puppeteer-extra@^3.3.6:
debug "^4.1.1"
deepmerge "^4.2.2"
puppeteer@^24.19.0:
version "24.19.0"
resolved "https://registry.yarnpkg.com/puppeteer/-/puppeteer-24.19.0.tgz#86cef2d1cc45066c9f5ed9edabf93b2d3b206eb3"
integrity sha512-gUWgHX36m9K6yUbvNBEA7CXElIL92yXMoAVFrO8OpZkItqrruLVqYA8ikmfgwcw/cNfYgkt0n2+yP9jd9RSETA==
puppeteer@^24.22.0:
version "24.22.0"
resolved "https://registry.yarnpkg.com/puppeteer/-/puppeteer-24.22.0.tgz#9f6905e9c3d5c316c364adb598903a1dfbfe800f"
integrity sha512-QabGIvu7F0hAMiKGHZCIRHMb6UoH0QAJA2OaqxEU2tL5noXPrxUcotg2l3ttOA4p1PFnVIGkr6PXRAWlM2evVQ==
dependencies:
"@puppeteer/browsers" "2.10.8"
"@puppeteer/browsers" "2.10.10"
chromium-bidi "8.0.0"
cosmiconfig "^9.0.0"
devtools-protocol "0.0.1495869"
puppeteer-core "24.19.0"
puppeteer-core "24.22.0"
typed-query-selector "^2.12.0"
qs@^6.14.0:
@@ -6127,10 +6054,10 @@ qs@^6.14.0:
dependencies:
side-channel "^1.1.0"
query-string@9.3.0:
version "9.3.0"
resolved "https://registry.yarnpkg.com/query-string/-/query-string-9.3.0.tgz#f2d60d6b4442cb445f374b5ff749b937b2cccd03"
integrity sha512-IQHOQ9aauHAApwAaUYifpEyLHv6fpVGVkMOnwPzcDScLjbLj8tLsILn6unSW79NafOw1llh8oK7Gd0VwmXBFmA==
query-string@9.3.1:
version "9.3.1"
resolved "https://registry.yarnpkg.com/query-string/-/query-string-9.3.1.tgz#d0c93e6c7fb7c17bdf04aa09e382114580ede270"
integrity sha512-5fBfMOcDi5SA9qj5jZhWAcTtDfKF5WFdd2uD9nVNlbxVv1baq65aALy6qofpNEGELHvisjjasxQp7BlM9gvMzw==
dependencies:
decode-uri-component "^0.4.1"
filter-obj "^5.1.0"
@@ -6194,14 +6121,6 @@ react-is@^18.2.0:
resolved "https://registry.yarnpkg.com/react-is/-/react-is-18.3.1.tgz#e83557dc12eae63a99e003a46388b1dcbb44db7e"
integrity sha512-/LLMVyas0ljjAtoYiPqYiL8VWXzUUdThrmU5+n20DZv+a+ClRoevUzw5JxU+Ieh5/c87ytoTBV9G1FiKfNJdmg==
react-redux@9.2.0:
version "9.2.0"
resolved "https://registry.yarnpkg.com/react-redux/-/react-redux-9.2.0.tgz#96c3ab23fb9a3af2cb4654be4b51c989e32366f5"
integrity sha512-ROY9fvHhwOD9ySfrF0wmvu//bKCQ6AeZZq1nJNtbDC+kk5DuSuNX/n6YWYF/SYy7bSba4D4FSz8DJeKY/S/r+g==
dependencies:
"@types/use-sync-external-store" "^0.0.6"
use-sync-external-store "^1.4.0"
react-refresh@^0.17.0:
version "0.17.0"
resolved "https://registry.yarnpkg.com/react-refresh/-/react-refresh-0.17.0.tgz#b7e579c3657f23d04eccbe4ad2e58a8ed51e7e53"
@@ -6215,17 +6134,17 @@ react-resizable@^3.0.5:
prop-types "15.x"
react-draggable "^4.0.3"
react-router-dom@7.8.2:
version "7.8.2"
resolved "https://registry.yarnpkg.com/react-router-dom/-/react-router-dom-7.8.2.tgz#25a8fc36588189baf3bbb5e360c8ffffbd2beabc"
integrity sha512-Z4VM5mKDipal2jQ385H6UBhiiEDlnJPx6jyWsTYoZQdl5TrjxEV2a9yl3Fi60NBJxYzOTGTTHXPi0pdizvTwow==
react-router-dom@7.9.1:
version "7.9.1"
resolved "https://registry.yarnpkg.com/react-router-dom/-/react-router-dom-7.9.1.tgz#48044923701773da6362f9003ec46f308f293f15"
integrity sha512-U9WBQssBE9B1vmRjo9qTM7YRzfZ3lUxESIZnsf4VjR/lXYz9MHjvOxHzr/aUm4efpktbVOrF09rL/y4VHa8RMw==
dependencies:
react-router "7.8.2"
react-router "7.9.1"
react-router@7.8.2:
version "7.8.2"
resolved "https://registry.yarnpkg.com/react-router/-/react-router-7.8.2.tgz#9d2d4147ca72832c550acc60ed688062d18f70b8"
integrity sha512-7M2fR1JbIZ/jFWqelpvSZx+7vd7UlBTfdZqf6OSdF9g6+sfdqJDAWcak6ervbHph200ePlu+7G8LdoiC3ReyAQ==
react-router@7.9.1:
version "7.9.1"
resolved "https://registry.yarnpkg.com/react-router/-/react-router-7.9.1.tgz#b227410c31f24dd416c939ca5d0f8d5c8a1404d4"
integrity sha512-pfAByjcTpX55mqSDGwGnY9vDCpxqBLASg0BMNAuMmpSGESo/TaOUG6BllhAtAkCGx8Rnohik/XtaqiYUJtgW2g==
dependencies:
cookie "^1.0.1"
set-cookie-parser "^2.6.0"
@@ -6323,23 +6242,6 @@ recma-stringify@^1.0.0:
unified "^11.0.0"
vfile "^6.0.0"
redux-logger@3.0.6:
version "3.0.6"
resolved "https://registry.yarnpkg.com/redux-logger/-/redux-logger-3.0.6.tgz#f7555966f3098f3c88604c449cf0baf5778274bf"
integrity sha512-JoCIok7bg/XpqA1JqCqXFypuqBbQzGQySrhFzewB7ThcnysTO30l4VCst86AuB9T9tuT03MAA56Jw2PNhRSNCg==
dependencies:
deep-diff "^0.3.5"
redux-thunk@3.1.0:
version "3.1.0"
resolved "https://registry.yarnpkg.com/redux-thunk/-/redux-thunk-3.1.0.tgz#94aa6e04977c30e14e892eae84978c1af6058ff3"
integrity sha512-NW2r5T6ksUKXCabzhL9z+h206HQw/NJkcLm1GPImRQ8IzfXwRGqjVhKJGauHirT0DAuyy6hjdnMZaRoAcy0Klw==
redux@5.0.1:
version "5.0.1"
resolved "https://registry.yarnpkg.com/redux/-/redux-5.0.1.tgz#97fa26881ce5746500125585d5642c77b6e9447b"
integrity sha512-M9/ELqF6fy8FwmkpnF0S3YKOqMyoWJ4+CS5Efg2ct3oY9daQvd/Pc71FpGZsVsbl3Cpb+IIcjBDUnnyBdQbq4w==
reflect.getprototypeof@^1.0.6, reflect.getprototypeof@^1.0.9:
version "1.0.10"
resolved "https://registry.yarnpkg.com/reflect.getprototypeof/-/reflect.getprototypeof-1.0.10.tgz#c629219e78a3316d8b604c765ef68996964e7bf9"
@@ -6931,11 +6833,6 @@ statuses@^2.0.1:
resolved "https://registry.yarnpkg.com/statuses/-/statuses-2.0.2.tgz#8f75eecef765b5e1cfcdc080da59409ed424e382"
integrity sha512-DvEy55V3DB7uknRo+4iOGT5fP1slR8wQohVdknigZPMpMstaKJQWhwiYBACJE3Ul2pTnATihhBYnRhZQHGBiRw==
steno@^4.0.2:
version "4.0.2"
resolved "https://registry.yarnpkg.com/steno/-/steno-4.0.2.tgz#9bd9b0ffc226a1f9436f29132c8b8e7199d22c50"
integrity sha512-yhPIQXjrlt1xv7dyPQg2P17URmXbuM5pdGkpiMB3RenprfiBlvK415Lctfe0eshk90oA7/tNq7WEiMK8RSP39A==
stop-iteration-iterator@^1.1.0:
version "1.1.0"
resolved "https://registry.yarnpkg.com/stop-iteration-iterator/-/stop-iteration-iterator-1.1.0.tgz#f481ff70a548f6124d0312c3aa14cbfa7aa542ad"
@@ -7498,11 +7395,6 @@ url-join@^4.0.0:
resolved "https://registry.yarnpkg.com/url-join/-/url-join-4.0.1.tgz#b642e21a2646808ffa178c4c5fda39844e12cde7"
integrity sha512-jk1+QP6ZJqyOiuEI9AEWQfju/nB2Pw466kbA0LEZljHwKeMgd9WrAEgEGxjPDD2+TNbbb37rTyhEfrCXfuKXnA==
use-sync-external-store@^1.4.0:
version "1.5.0"
resolved "https://registry.yarnpkg.com/use-sync-external-store/-/use-sync-external-store-1.5.0.tgz#55122e2a3edd2a6c106174c27485e0fd59bcfca0"
integrity sha512-Rb46I4cGGVBmjamjphe8L/UnvJD+uPPtTkNvX5mZgqdbavhI4EbgIWJiIHXJ8bc/i9EQGPRh4DwEURJ552Do0A==
util-deprecate@^1.0.1:
version "1.0.2"
resolved "https://registry.yarnpkg.com/util-deprecate/-/util-deprecate-1.0.2.tgz#450d4dc9fa70de732762fbd2d4a28981419a0ccf"
@@ -7529,10 +7421,10 @@ vfile@^6.0.0:
"@types/unist" "^3.0.0"
vfile-message "^4.0.0"
vite@7.1.5:
version "7.1.5"
resolved "https://registry.yarnpkg.com/vite/-/vite-7.1.5.tgz#4dbcb48c6313116689be540466fc80faa377be38"
integrity sha512-4cKBO9wR75r0BeIWWWId9XK9Lj6La5X846Zw9dFfzMRw38IlTk2iCcUt6hsyiDRcPidc55ZParFYDXi0nXOeLQ==
vite@7.1.6:
version "7.1.6"
resolved "https://registry.yarnpkg.com/vite/-/vite-7.1.6.tgz#336806d29983135677f498a05efb0fd46c5eef2d"
integrity sha512-SRYIB8t/isTwNn8vMB3MR6E+EQZM/WG1aKmmIUCfDXfVvKfc20ZpamngWHKzAmmu9ppsgxsg4b2I7c90JZudIQ==
dependencies:
esbuild "^0.25.0"
fdir "^6.5.0"
@@ -7548,6 +7440,11 @@ web-streams-polyfill@^3.0.3:
resolved "https://registry.yarnpkg.com/web-streams-polyfill/-/web-streams-polyfill-3.3.3.tgz#2073b91a2fdb1fbfbd401e7de0ac9f8214cecb4b"
integrity sha512-d2JWLCivmZYTSIoge9MsgFCZrt571BikcWGYkjC1khllbTeDlGqZ2D8vD8E/lJa8WGWbb7Plm8/XJYV7IJHZZw==
webdriver-bidi-protocol@0.2.11:
version "0.2.11"
resolved "https://registry.yarnpkg.com/webdriver-bidi-protocol/-/webdriver-bidi-protocol-0.2.11.tgz#dba18d9b0a33aed33fab272dbd6e42411ac753cc"
integrity sha512-Y9E1/oi4XMxcR8AT0ZC4OvYntl34SPgwjmELH+owjBr0korAX4jKgZULBWILGCVGdVCQ0dodTToIETozhG8zvA==
whatwg-encoding@^3.1.1:
version "3.1.1"
resolved "https://registry.yarnpkg.com/whatwg-encoding/-/whatwg-encoding-3.1.1.tgz#d0f4ef769905d426e1688f3e34381a99b60b76e5"
@@ -7681,10 +7578,10 @@ ws@^8.18.3:
resolved "https://registry.yarnpkg.com/ws/-/ws-8.18.3.tgz#b56b88abffde62791c639170400c93dcb0c95472"
integrity sha512-PEIGCY5tSlUt50cqyMXfCzX+oOPqN0vuGqWzbcJ2xvnkzkq46oOpz7dQaTDBdfICb4N14+GARUDw2XV2N4tvzg==
x-var@^2.1.0:
version "2.1.0"
resolved "https://registry.yarnpkg.com/x-var/-/x-var-2.1.0.tgz#9143461ad050b83a8043987ebb263606a1e8274f"
integrity sha512-EResegCrATlvIVNwrSt5wb4ip6XzUkjGp9cfr8nNcmfZB8Swg1NiesfcHBdvCs4Ed45cbWADeHcio0ZebJFYuQ==
x-var@^3.0.1:
version "3.0.1"
resolved "https://registry.yarnpkg.com/x-var/-/x-var-3.0.1.tgz#10a8d118930c143563cef7b7b3fc988f12936bb0"
integrity sha512-+DAw3e9txViMk/aONbLQS10Xg2+N5KBDyyfX7sJaRXkQ8bkpYqgBfrXaW0EvwEfVmFTTZHj0voXMeVlp2VJZ5Q==
dependencies:
dotenv "^16.4.5"
shelljs "^0.8.5"
@@ -7750,6 +7647,11 @@ zod@^3.24.1:
resolved "https://registry.yarnpkg.com/zod/-/zod-3.25.76.tgz#26841c3f6fd22a6a2760e7ccb719179768471e34"
integrity sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ==
zustand@^5.0.8:
version "5.0.8"
resolved "https://registry.yarnpkg.com/zustand/-/zustand-5.0.8.tgz#b998a0c088c7027a20f2709141a91cb07ac57f8a"
integrity sha512-gyPKpIaxY9XcO2vSMrLbiER7QMAMGOQZVRdJ6Zi782jkbzZygq5GI9nG8g+sMgitRtndwaBSl7uiqC49o1SSiw==
zwitch@^2.0.0:
version "2.0.4"
resolved "https://registry.yarnpkg.com/zwitch/-/zwitch-2.0.4.tgz#c827d4b0acb76fc3e685a4c6ec2902d51070e9d7"