Compare commits

..

17 Commits

Author SHA1 Message Date
orangecoding
0ce93acaf6 more demo fixes 2026-05-12 13:12:26 +02:00
orangecoding
cabef973a2 forbid backuo/restore in demo mode 2026-05-12 12:42:25 +02:00
orangecoding
3d0fa87d19 upgrading dependencies 2026-05-12 09:23:52 +02:00
orangecoding
8b012ef2f1 upgrading dependencies / new pois 2026-05-11 09:18:32 +02:00
orangecoding
6816b0aded next release version 2026-05-10 15:43:13 +02:00
Christian Kellner
ac02817d4e Switch browser engine from puppeteer-extra/stealth to CloakBrowser (#307)
* Switch browser engine from puppeteer-extra/stealth to CloakBrowser

- Replace puppeteer, puppeteer-extra, puppeteer-extra-plugin-stealth with
  cloakbrowser + puppeteer-core; CloakBrowser applies 49 source-level C++
  fingerprint patches that cannot be detected at the JS layer.
- Enable humanize:true in launchBrowser() for Bézier mouse curves, natural
  keyboard timing, and realistic scroll physics.
- Remove manual userDataDir management and ARM64 executablePath override;
  CloakBrowser ships its own binary for x86_64 and arm64.
- Proxy is now passed via CloakBrowser's native proxy option instead of
  --proxy-server Chrome flag.
- Dockerfile: add fonts-noto-color-emoji + fonts-freefont-ttf so canvas
  fingerprint hashes match real browsers (required for Kasada/Akamai);
  replace npx puppeteer browsers install with node ensureBinary() call;
  remove TARGETARCH ARG and ARM64 system-Chromium branch.
- Update test mock to reflect simplified browser object (no __fredy_* fields).

https://claude.ai/code/session_01WXzA3orbwE2hdk723c6MgH

* Add --ignore-certificate-errors for CloakBrowser's custom Chromium

CloakBrowser ships its own Chromium binary with an independent CA bundle.
This flag prevents ERR_CERT_AUTHORITY_INVALID failures in environments with
SSL-inspecting proxies or non-standard root CAs (Docker CI, corporate networks).

https://claude.ai/code/session_01WXzA3orbwE2hdk723c6MgH

* Harden CloakBrowser integration and fix kleinanzeigen detail test

- Remove all CDP overrides (applyBotPreventionToPage, applyLanguagePersistence,
  applyPostNavigationHumanSignals) that created detectable inconsistencies on top
  of CloakBrowser's C++ patches; pass locale to CloakBrowser launch instead
- Drop --lang arg (replaced by CloakBrowser locale flag)
- Extend immowelt puppeteerTimeout to 90 s to accommodate React SPA rendering
  latency under CloakBrowser's humanise delays
- Fix kleinanzeigen detail test: serve the offline fixture for the search URL
  so only individual detail pages are fetched live, avoiding rate limiting from
  a second fresh session hitting the same search endpoint

https://claude.ai/code/session_01WXzA3orbwE2hdk723c6MgH

* Fix immowelt bot detection with two-phase navigation and fixture-backed detail test

Immowelt's CDN challenges cold browser sessions before React can render the
listing grid, causing the old waitForSelector approach to silently timeout.

- Add preNavigateUrl option to puppeteerExtractor: visits a warm-up page
  first so the site sees an established session before the search URL
- Add waitForNetworkIdle option: a second idle-wait phase after domcontentloaded
  that catches React's listing API round-trip (which fires long after the
  initial HTML is parsed); errors are swallowed so partial DOM is still used
- Switch immowelt config to waitForSelector=null + networkidle warm-up so
  page.content() is returned after the SPA has loaded its data
- Set immowelt preNavigateUrl to the homepage to warm the session
- In the detail enrichment test, spy on puppeteerExtractor to serve the
  offline fixture for the search URL; only individual listing detail pages
  are fetched live (they are far less aggressively protected)

https://claude.ai/code/session_01WXzA3orbwE2hdk723c6MgH

* Ensure CloakBrowser binary is present before any live test runs

Add a Vitest globalSetup that calls ensureBinary() once in the main process
before workers start. Without this, running yarn test on a fresh checkout
(or after the binary cache is cleared) immediately fails every browser-based
test with "Failed to launch the browser process" before any useful output
appears. The setup is a no-op in offline mode and when the binary is already
cached.

https://claude.ai/code/session_01WXzA3orbwE2hdk723c6MgH

* Ensure CloakBrowser binary at startup for non-Docker installs

Direct runs (yarn start:backend) on a fresh checkout have no binary and
only crash when the first scraping job fires. Calling ensureBinary() at
startup downloads it on first run and is instant when already cached.
In Docker it stays a no-op since the binary is pre-baked during docker build.

https://claude.ai/code/session_01WXzA3orbwE2hdk723c6MgH

* Fix --no-zygote comment: ICU crash was corrupted .4 binary, not fd issue

The "Invalid file descriptor to ICU data received" crash seen in Sparkasse
tests was caused by a partially-extracted CloakBrowser .4 binary that
contained only the chrome executable but was missing icudtl.dat and other
resource files. The ensureBinary() function returned this incomplete
installation because latest_version_linux-x64 pointed to .4.

The --no-zygote flag is kept as a safeguard for container environments
with limited kernel namespaces, but the comment now accurately describes
its purpose rather than attributing it to a non-existent fd inheritance issue.

https://claude.ai/code/session_01WXzA3orbwE2hdk723c6MgH

* Add ensureValidBinary() to detect and auto-heal corrupt CloakBrowser installs

CloakBrowser's ensureBinary() only checks that the chrome executable exists,
not that required resource files (icudtl.dat, resources.pak) are present.
A partial extraction — e.g. an interrupted update — can leave a directory
that passes ensureBinary()'s check but causes Chrome to crash immediately
with "Invalid file descriptor to ICU data received".

ensureValidBinary() wraps ensureBinary() with a completeness check:
- If the required resource files are missing it removes the corrupt directory
  and all latest_version* markers, then calls ensureBinary() again so it
  falls back to (or re-downloads) a complete build.
- It pins the validated path via CLOAKBROWSER_BINARY_PATH so CloakBrowser's
  own internal ensureBinary() call inside launch() always uses the same,
  verified binary.

Used in index.js (app startup) and test/globalSetup.js (before live tests).

https://claude.ai/code/session_01WXzA3orbwE2hdk723c6MgH

* Fix sparkasse detail test: serve search URL from fixture to avoid rate-limiting

The second sparkasse test launched a fresh browser against the live search
endpoint right after the first test already did, leaving the IP in a suspicious
state that caused bot detection or rate-limiting to return empty results.
When getListings() returns nothing, execute() resolves to undefined and
expect(listings).toBeInstanceOf(Array) fails.

Apply the same hybrid fixture approach used by kleinanzeigen and immowelt:
intercept puppeteerExtractor calls whose pathname matches the search URL and
return the offline fixture, while letting individual detail page requests go
live (they are less aggressively rate-limited than the search endpoint).

https://claude.ai/code/session_01WXzA3orbwE2hdk723c6MgH

* Fix sparkasse detail test: shared browser, direct fetchDetails call

Remove the fixture-backed spy — live tests must hit the real server.

Root problem: two cold browser sessions hitting sparkasse in quick succession
triggered bot detection, causing the second search request to return empty
results and execute() to resolve undefined.

Fix:
- One browser launched in beforeAll and reused across both tests, so both
  the search and detail requests come from the same warm session.
- The detail test calls provider.config.fetchDetails() directly on the
  listings returned by the first test instead of re-running the full pipeline.
  This avoids a redundant second scrape of the search page while still
  exercising the live detail endpoint.

https://claude.ai/code/session_01WXzA3orbwE2hdk723c6MgH

* Eliminate fixture spies and double live requests in all provider detail tests

All five provider tests with a 'with provider_details enabled' describe block
were either (a) intercepting the search URL with an offline fixture to avoid
hitting the live server twice, or (b) re-running the full execute() pipeline
with a fresh browser, which triggered rate-limiting / bot detection on the
second cold request.

Pattern applied to all five:
- immowelt, kleinanzeigen, wgGesucht, immobilienDe: launch one browser in
  beforeAll/afterAll, pass it to the first test's Fredy constructor, and call
  provider.config.fetchDetails() directly in the second test using the listings
  and browser already in hand. One warm session, two live endpoints tested.
- immoscout: API-based (no browser), so no browser sharing needed. Second test
  calls provider.config.fetchDetails() directly on liveListings[0] from the
  first test instead of re-querying the search API.

Removed: all readFixture spies, getKnownListingHashesForJobAndProvider mocks,
and the puppeteerExtractorMod imports that were only needed for the spy.

https://claude.ai/code/session_01WXzA3orbwE2hdk723c6MgH

* Fix ensureValidBinary for macOS: platform-aware completeness check

On macOS the CloakBrowser binary lives at:
  ~/.cloakbrowser/chromium-X.Y.Z/Chromium.app/Contents/MacOS/Chromium

path.dirname() gave Contents/MacOS/ — but icudtl.dat and resources.pak
are inside Contents/Frameworks/…, not next to the binary. So the old
code incorrectly flagged every macOS installation as corrupt, deleted only
the MacOS/ subdirectory (not the full versioned dir), then failed again.

Fixes:
- isBinaryComplete: on macOS check for Info.plist and Frameworks/ inside
  Chromium.app/Contents/ instead of looking for Linux resource files next
  to the binary. On Linux/Windows the existing check is unchanged.
- getVersionedDir: resolves the full chromium-X.Y.Z/ directory regardless
  of platform (4 levels up on macOS, 1 on Linux/Windows) so
  removeCorruptInstallation always deletes the entire versioned tree.
- missingDescription: reports the correct missing items per platform.

https://claude.ai/code/session_01WXzA3orbwE2hdk723c6MgH

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-05-10 15:42:31 +02:00
datenwurm
fe0a09fe1c fix: Sort direction toggle not working in listings overview (#306)
Co-authored-by: datenwurm <git@datenwurm.net>
2026-05-08 09:24:20 +02:00
orangecoding
2f00966f27 next release version 2026-05-07 19:12:17 +02:00
orangecoding
921057252d adding immoscout shape search 2026-05-07 19:11:47 +02:00
orangecoding
703c602527 table overview for jobs 2026-05-07 15:59:55 +02:00
orangecoding
0e29c9b9c6 next release version 2026-05-07 12:45:52 +02:00
datenwurm
f60c5859f9 feat: Add grid/table view toggle to listings overview (#305)
* feat: Add delete button to listing detail view

* feat: Add grid/table view toggle to listings overview

---------

Co-authored-by: datenwurm <git@datenwurm.net>
2026-05-07 12:12:49 +02:00
datenwurm
ee54cc495b feat: Add delete button to listing detail view (#304)
Co-authored-by: datenwurm <git@datenwurm.net>
2026-05-07 11:53:51 +02:00
orangecoding
96582ecff4 gmx example 2026-05-02 20:07:03 +02:00
orangecoding
3de82dfa41 fixing error message when passwords do not match / fixing placeholder image 2026-05-02 20:00:11 +02:00
orangecoding
d7ee4f6909 adding gitattributed§ 2026-05-01 20:12:58 +02:00
orangecoding
bf4bae9bf5 upgrading dependencies 2026-05-01 20:12:40 +02:00
50 changed files with 2690 additions and 1842 deletions

1
.gitattributes vendored Normal file
View File

@@ -0,0 +1 @@
test/testFixtures/** linguist-vendored

View File

@@ -1,16 +1,15 @@
FROM node:22-slim
ARG TARGETARCH
# System deps for Chrome for Testing + build tools for native modules (better-sqlite3)
# On ARM64 we also install system Chromium (Chrome for Testing has no ARM64 binary)
# System deps for CloakBrowser + build tools for native modules (better-sqlite3)
# fonts-noto-color-emoji and fonts-freefont-ttf are required so canvas fingerprint
# hashes match real browsers; missing emoji fonts cause bot detection on Kasada/Akamai.
RUN apt-get update && apt-get install -y --no-install-recommends \
curl ca-certificates fonts-liberation libasound2 \
libatk-bridge2.0-0 libatk1.0-0 libcups2 libdbus-1-3 \
libdrm2 libgbm1 libgtk-3-0 libnspr4 libnss3 \
libx11-xcb1 libxcomposite1 libxdamage1 libxrandr2 xdg-utils \
fonts-noto-color-emoji fonts-freefont-ttf \
python3 make g++ \
&& if [ "$TARGETARCH" = "arm64" ]; then apt-get install -y --no-install-recommends chromium; fi \
&& rm -rf /var/lib/apt/lists/* \
&& mkdir -p /db /conf /fredy
@@ -26,8 +25,8 @@ RUN yarn config set network-timeout 600000 \
&& yarn --frozen-lockfile \
&& yarn cache clean
# on arm64 use the system Chromium installed above
RUN if [ "$TARGETARCH" != "arm64" ]; then npx puppeteer browsers install chrome; fi
# Pre-download the CloakBrowser stealth Chromium binary (supports x86_64 and arm64)
RUN node -e "import('cloakbrowser').then(({ensureBinary}) => ensureBinary())"
# Purge build tools now that native modules are compiled
RUN apt-get purge -y python3 make g++ \

View File

@@ -15,6 +15,15 @@ import { initGeocodingCron } from './lib/services/crons/geocoding-cron.js';
import { getSettings } from './lib/services/storage/settingsStorage.js';
import SqliteConnection, { computeDbPath } from './lib/services/storage/SqliteConnection.js';
import { initJobExecutionService } from './lib/services/jobs/jobExecutionService.js';
import { ensureValidBinary } from './lib/services/ensureValidBinary.js';
// Ensure the CloakBrowser stealth Chromium binary is present and complete before
// jobs run. ensureValidBinary() also detects and auto-heals partial extractions
// (e.g. a newer version that was downloaded but only the chrome executable was
// written) so Chrome never crashes with "Invalid file descriptor to ICU data".
logger.info('Checking CloakBrowser binary...');
await ensureValidBinary();
logger.info('CloakBrowser binary ready.');
//in the config, we store the path of the sqlite file, thus we must check if it is available
const isConfigAccessible = await checkIfConfigIsAccessible();

View File

@@ -7,4 +7,8 @@ export const TRACKING_POIS = {
DISTANCE_ADDRESS_ENTERED: 'DISTANCE_ADDRESS_ENTERED',
WELCOME_FINISHED: 'WELCOME_FINISHED',
WELCOME_SKIPPED: 'WELCOME_SKIPPED',
JOBS_TABLE_VIEW: 'JOBS_TABLE_VIEW',
LISTING_TABLE_VIEW: 'LISTING_TABLE_VIEW',
BASE_URL_SETTING: 'BASE_URL_SETTING',
DETECTED_AS_BOT: 'DETECTED_AS_BOT',
};

View File

@@ -76,13 +76,13 @@ fastify.register(async (app) => {
app.register(dashboardPlugin, { prefix: '/api/dashboard' });
app.register(userSettingsPlugin, { prefix: '/api/user/settings' });
app.register(trackingPlugin, { prefix: '/api/tracking' });
app.register(generalSettingsPlugin, { prefix: '/api/admin/generalSettings' });
});
// Admin-only routes
fastify.register(async (app) => {
app.addHook('preHandler', authHook);
app.addHook('preHandler', adminHook);
app.register(generalSettingsPlugin, { prefix: '/api/admin/generalSettings' });
app.register(backupPlugin, { prefix: '/api/admin/backup' });
app.register(userPlugin, { prefix: '/api/admin/users' });
});

View File

@@ -9,6 +9,10 @@ import {
precheckRestore,
restoreFromZip,
} from '../../services/storage/backupRestoreService.js';
import { getSettings } from '../../services/storage/settingsStorage.js';
import { isAdmin } from '../security.js';
const DEMO_MODE_ERROR = 'Backup and restore are not available in demo mode.';
/**
* @param {import('fastify').FastifyInstance} fastify
@@ -21,7 +25,11 @@ export default async function backupPlugin(fastify) {
(req, body, done) => done(null, body),
);
fastify.get('/', async (_request, reply) => {
fastify.get('/', async (request, reply) => {
const settings = await getSettings();
if (settings.demoMode && !isAdmin(request)) {
return reply.code(403).send({ error: DEMO_MODE_ERROR });
}
const zipBuffer = await createBackupZip();
const fileName = await buildBackupFileName();
reply.header('Content-Type', 'application/zip');
@@ -30,6 +38,10 @@ export default async function backupPlugin(fastify) {
});
fastify.post('/restore', async (request, reply) => {
const settings = await getSettings();
if (settings.demoMode && !isAdmin(request)) {
return reply.code(403).send({ error: DEMO_MODE_ERROR });
}
const { dryRun = 'false', force = 'false' } = request.query || {};
const doDryRun = String(dryRun) === 'true';
const doForce = String(force) === 'true';

View File

@@ -9,6 +9,8 @@ import { ensureDemoUserExists } from '../../services/storage/userStorage.js';
import logger from '../../services/logger.js';
import { getSettings, upsertSettings } from '../../services/storage/settingsStorage.js';
import { isAdmin } from '../security.js';
import { trackPoi } from '../../services/tracking/Tracker.js';
import { TRACKING_POIS } from '../../TRACKING_POIS.js';
/**
* @param {import('fastify').FastifyInstance} fastify
@@ -25,16 +27,23 @@ export default async function generalSettingsPlugin(fastify) {
}
const localSettings = await getSettings();
if (localSettings.demoMode && !isAdmin(request)) {
return reply.code(403).send({ error: 'In demo mode, it is not allowed to change these settings.' });
if (!isAdmin(request)) {
const reason = localSettings.demoMode
? 'In demo mode, it is not allowed to change these settings.'
: 'Only admins can change these settings.';
return reply.code(403).send({ error: reason });
}
try {
if (typeof sqlitepath !== 'undefined') {
fs.writeFileSync(`${getDirName()}/../conf/config.json`, JSON.stringify({ sqlitepath }));
}
upsertSettings(appSettings);
ensureDemoUserExists();
if (appSettings.baseUrl != null) {
await trackPoi(TRACKING_POIS.BASE_URL_SETTING);
}
} catch (err) {
logger.error(err);
return reply.code(500).send({ error: 'Error while trying to write settings.' });

View File

@@ -58,7 +58,7 @@ export default async function userPlugin(fastify) {
const { username, password, password2, isAdmin, userId } = request.body;
if (password !== password2) {
return reply.code(400).send({ error: 'Passwords does not match' });
return reply.code(400).send({ error: 'Passwords do not match.' });
}
if (nullOrEmpty(username) || nullOrEmpty(password) || nullOrEmpty(password2)) {
return reply.code(400).send({ error: 'Username and password are mandatory.' });

View File

@@ -109,4 +109,46 @@ export default async function userSettingsPlugin(fastify) {
return reply.code(500).send({ error: error.message });
}
});
fastify.post('/listings-view-mode', async (request, reply) => {
const userId = request.session.currentUser;
const { listings_view_mode } = request.body;
if (listings_view_mode !== 'grid' && listings_view_mode !== 'table') {
return reply.code(400).send({ error: 'listings_view_mode must be "grid" or "table".' });
}
if (listings_view_mode === 'table') {
await trackPoi(TRACKING_POIS.LISTING_TABLE_VIEW);
}
try {
upsertSettings({ listings_view_mode }, userId);
return { success: true };
} catch (error) {
logger.error('Error updating listings view mode setting', error);
return reply.code(500).send({ error: error.message });
}
});
fastify.post('/jobs-view-mode', async (request, reply) => {
const userId = request.session.currentUser;
const { jobs_view_mode } = request.body;
if (jobs_view_mode !== 'grid' && jobs_view_mode !== 'table') {
return reply.code(400).send({ error: 'jobs_view_mode must be "grid" or "table".' });
}
if (jobs_view_mode === 'table') {
await trackPoi(TRACKING_POIS.JOBS_TABLE_VIEW);
}
try {
upsertSettings({ jobs_view_mode }, userId);
return { success: true };
} catch (error) {
logger.error('Error updating jobs view mode setting', error);
return reply.code(500).send({ error: error.message });
}
});
}

View File

@@ -20,3 +20,4 @@ Common SMTP settings:
- **Gmail** - `smtp.gmail.com`, port 587, secure: false
- **Outlook** - `smtp.office365.com`, port 587, secure: false
- **Yahoo** - `smtp.mail.yahoo.com`, port 465, secure: true
- **Gmx** - `mail.gmx.net`, port 587, secure: true

View File

@@ -87,7 +87,19 @@ const config = {
crawlContainer:
'div[data-testid="serp-core-scrollablelistview-testid"]:not(div[data-testid="serp-enlargementlist-testid"] div[data-testid="serp-card-testid"]) div[data-testid="serp-core-classified-card-testid"]',
sortByDateParam: 'order=DateDesc',
waitForSelector: 'div[data-testid="serp-gridcontainer-testid"]',
// waitForSelector is null: extract the full page via page.content() so the
// Cheerio crawler can search anywhere in the rendered document.
// preNavigateUrl visits the homepage first to establish a trusted session
// before hitting the search URL; this prevents CDN-level bot challenges that
// fire on cold sessions. waitForNetworkIdle (phase 2) then catches React's
// listing API round-trip that fires well after domcontentloaded.
waitForSelector: null,
puppeteerOptions: {
puppeteerTimeout: 60_000,
preNavigateUrl: 'https://www.immowelt.de/',
waitForNetworkIdle: true,
waitForNetworkIdleTimeout: 60_000,
},
crawlFields: {
id: 'a@href',
price: 'div[data-testid="cardmfe-price-testid"] | removeNewline | trim',

View File

@@ -0,0 +1,147 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import { ensureBinary } from 'cloakbrowser';
import fs from 'fs';
import path from 'path';
import os from 'os';
/**
* Resource files required on Linux/Windows — they must live next to the chrome binary.
* macOS packages these inside the .app bundle's Frameworks directory so a different
* check is used there (see isBinaryComplete).
*/
const LINUX_WIN_REQUIRED_FILES = ['icudtl.dat', 'resources.pak'];
/**
* Return the top-level versioned installation directory for any platform.
*
* - Linux/Windows: binaryPath is ~/.cloakbrowser/chromium-X.Y.Z/chrome
* → dirname ~/.cloakbrowser/chromium-X.Y.Z/
* - macOS: binaryPath is ~/.cloakbrowser/chromium-X.Y.Z/Chromium.app/Contents/MacOS/Chromium
* → 4 levels up ~/.cloakbrowser/chromium-X.Y.Z/
*
* @param {string} binaryPath
* @returns {string}
*/
function getVersionedDir(binaryPath) {
if (process.platform === 'darwin') {
return path.resolve(path.dirname(binaryPath), '../../..');
}
return path.dirname(binaryPath);
}
/**
* Return true when the binary at binaryPath belongs to a complete installation.
*
* On macOS the binary lives inside an .app bundle:
* Chromium.app/Contents/MacOS/Chromium
* Resource files (icudtl.dat etc.) are deep inside
* Chromium.app/Contents/Frameworks/…
* so checking for them next to the binary is wrong. Instead we verify the two
* structural markers that are only present after a full extraction: Info.plist
* and the Frameworks directory inside Contents/.
*
* On Linux/Windows the binary and all resource files are siblings in the same
* directory.
*
* @param {string} binaryPath
* @returns {boolean}
*/
function isBinaryComplete(binaryPath) {
if (process.platform === 'darwin') {
const contentsDir = path.resolve(path.dirname(binaryPath), '..');
return fs.existsSync(path.join(contentsDir, 'Info.plist')) && fs.existsSync(path.join(contentsDir, 'Frameworks'));
}
const dir = path.dirname(binaryPath);
return LINUX_WIN_REQUIRED_FILES.every((f) => fs.existsSync(path.join(dir, f)));
}
/**
* Return a human-readable description of which required files/dirs are missing.
*
* @param {string} binaryPath
* @returns {string}
*/
function missingDescription(binaryPath) {
if (process.platform === 'darwin') {
const contentsDir = path.resolve(path.dirname(binaryPath), '..');
return ['Info.plist', 'Frameworks'].filter((f) => !fs.existsSync(path.join(contentsDir, f))).join(', ');
}
const dir = path.dirname(binaryPath);
return LINUX_WIN_REQUIRED_FILES.filter((f) => !fs.existsSync(path.join(dir, f))).join(', ');
}
/**
* Remove a corrupt binary installation and all `latest_version*` markers from
* the CloakBrowser cache so the next `ensureBinary()` call falls back to the
* package-bundled version.
*
* Removes the full versioned directory (e.g. chromium-X.Y.Z/) on all platforms,
* not just the subdirectory that contains the binary.
*
* @param {string} binaryPath - Path to the (corrupt) chrome/Chromium binary.
*/
function removeCorruptInstallation(binaryPath) {
const versionedDir = getVersionedDir(binaryPath);
const cacheDir = process.env.CLOAKBROWSER_CACHE_DIR || path.join(os.homedir(), '.cloakbrowser');
fs.rmSync(versionedDir, { recursive: true, force: true });
try {
for (const entry of fs.readdirSync(cacheDir)) {
if (entry.startsWith('latest_version')) {
fs.rmSync(path.join(cacheDir, entry), { force: true });
}
}
} catch {
// Cache dir may not exist if versionedDir was the only entry — ignore.
}
}
/**
* Ensure the CloakBrowser stealth Chromium binary is present **and** complete.
*
* `cloakbrowser`'s own `ensureBinary()` only checks that the chrome/Chromium
* file exists. An incomplete extraction (e.g. interrupted download, disk full)
* can leave a directory that contains the executable but is missing essential
* resource files. Chrome then crashes immediately on launch.
*
* This wrapper validates the path returned by `ensureBinary()`. If the
* installation is incomplete it removes the corrupt directory, clears the
* version marker files, and calls `ensureBinary()` again so it falls back to
* (or re-downloads) a complete build.
*
* The validated path is also pinned via `CLOAKBROWSER_BINARY_PATH` so that
* CloakBrowser's own internal `ensureBinary()` call inside `launch()` always
* picks up the same, verified binary.
*
* @returns {Promise<string>} Absolute path to the validated binary.
* @throws {Error} When even the fallback binary is incomplete.
*/
export async function ensureValidBinary() {
const binaryPath = await ensureBinary();
if (isBinaryComplete(binaryPath)) {
process.env.CLOAKBROWSER_BINARY_PATH = binaryPath;
return binaryPath;
}
console.warn(
`[fredy] CloakBrowser installation at ${getVersionedDir(binaryPath)} is missing: ${missingDescription(binaryPath)}. Removing and retrying.`,
);
removeCorruptInstallation(binaryPath);
const fallbackPath = await ensureBinary();
if (!isBinaryComplete(fallbackPath)) {
throw new Error(
`CloakBrowser binary at ${getVersionedDir(fallbackPath)} is still missing required files after re-download: ${missingDescription(fallbackPath)}`,
);
}
process.env.CLOAKBROWSER_BINARY_PATH = fallbackPath;
return fallbackPath;
}

View File

@@ -3,121 +3,135 @@
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import puppeteer from 'puppeteer-extra';
import StealthPlugin from 'puppeteer-extra-plugin-stealth';
import { launch } from 'cloakbrowser/puppeteer';
import { debug, botDetected } from './utils.js';
import {
getPreLaunchConfig,
applyBotPreventionToPage,
applyLanguagePersistence,
applyPostNavigationHumanSignals,
} from './botPrevention.js';
import { getPreLaunchConfig } from './botPrevention.js';
import logger from '../logger.js';
import fs from 'fs';
import os from 'os';
import path from 'path';
puppeteer.use(StealthPlugin());
import { trackPoi } from '../tracking/Tracker.js';
import { TRACKING_POIS } from '../../TRACKING_POIS.js';
/**
* Launch a CloakBrowser/Puppeteer browser instance with stealth and humanizer enabled.
*
* CloakBrowser applies 49 C++ source-level patches (canvas, WebGL, audio, WebRTC,
* navigator.*, automation signals) that are indistinguishable from a real browser.
* All fingerprinting and human-behaviour simulation is handled natively; no CDP
* overrides (setUserAgent, setExtraHTTPHeaders, evaluateOnNewDocument) are applied
* here because they would create detectable inconsistencies on top of the C++ patches.
*
* @param {string} url - Initial URL (used to derive locale/timezone hints).
* @param {object} [options]
* @param {boolean} [options.puppeteerHeadless]
* @param {number} [options.puppeteerTimeout]
* @param {string} [options.proxyUrl]
* @param {string} [options.timezone]
* @param {string} [options.acceptLanguage]
* @param {object} [options.viewport]
* @returns {Promise<import('puppeteer-core').Browser>}
*/
export async function launchBrowser(url, options) {
const preCfg = getPreLaunchConfig(url, options || {});
const launchArgs = [
// Docker requires --no-sandbox; CloakBrowser handles all stealth args internally.
// --ignore-certificate-errors is needed because CloakBrowser ships its own Chromium
// binary with an independent CA bundle that may not trust proxies or interceptors
// present in the host environment.
const args = [
'--no-sandbox',
'--disable-gpu',
'--disable-setuid-sandbox',
'--disable-dev-shm-usage',
'--disable-crash-reporter',
'--no-first-run',
'--no-default-browser-check',
preCfg.langArg,
'--ignore-certificate-errors',
// Disables the zygote process model. Required in some container environments
// (e.g. limited kernel namespaces) where the zygote cannot acquire the
// locks it needs and exits with "Invalid file descriptor to ICU data received".
'--no-zygote',
preCfg.windowSizeArg,
...preCfg.extraArgs,
];
if (options?.proxyUrl) {
launchArgs.push(`--proxy-server=${options.proxyUrl}`);
}
let userDataDir;
let removeUserDataDir = false;
if (options && options.userDataDir) {
userDataDir = options.userDataDir;
} else {
const prefix = path.join(os.tmpdir(), 'puppeteer-fredy-');
userDataDir = fs.mkdtempSync(prefix);
removeUserDataDir = true;
}
// On ARM64 Docker, Chrome for Testing has no native binary - use system Chromium instead.
const executablePath =
options?.executablePath ||
(process.arch === 'arm64' && process.env.IS_DOCKER === 'true' ? '/usr/bin/chromium' : undefined);
const browser = await puppeteer.launch({
const browser = await launch({
headless: options?.puppeteerHeadless ?? true,
args: launchArgs,
timeout: options?.puppeteerTimeout || 45_000,
userDataDir,
executablePath,
humanize: true,
args,
// locale sets Accept-Language headers and JS navigator.language consistently
locale: preCfg.langForFlag,
...(options?.proxyUrl ? { proxy: options.proxyUrl } : {}),
...(preCfg.timezone ? { timezone: preCfg.timezone } : {}),
});
browser.__fredy_userDataDir = userDataDir;
browser.__fredy_removeUserDataDir = removeUserDataDir;
return browser;
}
/**
* Close a browser instance returned by {@link launchBrowser}.
*
* @param {import('puppeteer-core').Browser | null} browser
*/
export async function closeBrowser(browser) {
if (!browser) return;
const userDataDir = browser.__fredy_userDataDir;
const removeUserDataDir = browser.__fredy_removeUserDataDir;
try {
await browser.close();
} catch {
// ignore
}
if (removeUserDataDir && userDataDir) {
try {
await fs.promises.rm(userDataDir, { recursive: true, force: true });
} catch {
// ignore
}
}
}
/**
* Open a page in a (possibly reused) browser, navigate to `url`, and return the HTML source.
* Returns `null` when a bot-detection page is encountered or on timeout.
*
* @param {string} url
* @param {string | null} waitForSelector
* @param {object} [options]
* @returns {Promise<string | null>}
*/
export default async function execute(url, waitForSelector, options) {
let browser = options?.browser;
let isExternalBrowser = !!browser;
let page;
let result;
try {
debug(`Sending request to ${url} using Puppeteer.`);
debug(`Sending request to ${url} using CloakBrowser.`);
if (!isExternalBrowser) {
browser = await launchBrowser(url, options);
}
page = await browser.newPage();
const preCfg = getPreLaunchConfig(url, options || {});
await applyBotPreventionToPage(page, preCfg);
// Provide languages value before navigation
await applyLanguagePersistence(page, preCfg);
// Optional cookies
if (Array.isArray(options?.cookies) && options.cookies.length > 0) {
await page.setCookie(...options.cookies);
}
// Navigation
// Warm-up navigation: visit a trusted page first so the site sees an
// established session before the actual target URL. Silently ignored on
// failure so it never blocks the main request.
if (options?.preNavigateUrl) {
try {
await page.goto(options.preNavigateUrl, { waitUntil: 'domcontentloaded', timeout: 30_000 });
await new Promise((r) => setTimeout(r, 1500 + Math.random() * 2000));
} catch {
// ignore
}
}
const response = await page.goto(url, {
waitUntil: options?.waitUntil || 'domcontentloaded',
timeout: options?.puppeteerTimeout || 60000,
});
// Optionally wait and add subtle human-like interactions
await applyPostNavigationHumanSignals(page, preCfg);
// Optional second idle wait: useful for React SPAs that trigger API calls
// after domcontentloaded. Times out silently so we use whatever is rendered.
if (options?.waitForNetworkIdle) {
try {
await page.waitForNetworkIdle({ timeout: options?.waitForNetworkIdleTimeout ?? 60_000 });
} catch {
// ignore — we proceed with whatever the DOM contains at this point
}
}
let pageSource;
// if we're extracting data from a SPA, we must wait for the selector
if (waitForSelector != null) {
const selectorTimeout = options?.puppeteerSelectorTimeout ?? options?.puppeteerTimeout ?? 30_000;
await page.waitForSelector(waitForSelector, { timeout: selectorTimeout });
@@ -133,15 +147,18 @@ export default async function execute(url, waitForSelector, options) {
if (botDetected(pageSource, statusCode)) {
logger.warn('We have been detected as a bot :-/ Tried url: => ', url);
await trackPoi(TRACKING_POIS.DETECTED_AS_BOT);
result = null;
} else {
result = pageSource || (await page.content());
}
} catch (error) {
if (error?.name?.includes('Timeout')) {
logger.debug('Error executing with puppeteer executor', error);
logger.debug('Error executing with CloakBrowser executor', error);
} else {
logger.warn('Error executing with puppeteer executor', error);
logger.warn('Error executing with CloakBrowser executor', error);
}
result = null;
} finally {

View File

@@ -168,10 +168,6 @@ export function convertWebToMobile(webUrl) {
}
}
if (segments.includes('shape')) {
throw new Error('Shape is currently not supported using Immoscout');
}
const { query: rawParams } = queryString.parseUrl(webUrl, { arrayFormat: 'comma' });
const webParams = Object.fromEntries(
Object.entries(rawParams).filter(([key]) => key !== 'enteredFrom' && PARAM_NAME_MAP[key]),
@@ -179,18 +175,31 @@ export function convertWebToMobile(webUrl) {
const geocodes = `/${segments.slice(2, segments.length - 1).join('/')}`;
const isRadius = segments.includes('radius');
const isShape = segments.includes('shape');
const mobileParams = {
searchType: isRadius ? 'radius' : 'region',
searchType: isRadius ? 'radius' : isShape ? 'shape' : 'region',
realestatetype: realType,
...(isRadius ? {} : { geocodes }),
...(isRadius || isShape ? {} : { geocodes }),
...additionalParamsFromWebPath,
};
if (isShape && !webParams.shape) {
throw new Error('Shape search URL is missing the required "shape" query parameter');
}
if (isShape && webParams.shape) {
const browserShape = webParams.shape;
const normalized = browserShape.replace(/\.\./g, '==').replace(/\./g, '=');
const polyline = Buffer.from(normalized, 'base64').toString('utf-8');
mobileParams.shape = polyline;
}
if (webParams.geocoordinates) {
mobileParams.geocoordinates = webParams.geocoordinates;
}
for (const [key, val] of Object.entries(webParams)) {
if (key === 'shape') continue;
if (key === 'equipment') {
const items = [].concat(val).flatMap((v) => `${v}`.split(','));
const currentEquipmentParams = mobileParams[PARAM_NAME_MAP[key]];

View File

@@ -1,6 +1,6 @@
{
"name": "fredy",
"version": "21.1.0",
"version": "22.0.5",
"description": "[F]ind [R]eal [E]states [d]amn eas[y].",
"scripts": {
"prepare": "husky",
@@ -62,9 +62,9 @@
"Firefox ESR"
],
"dependencies": {
"@douyinfe/semi-icons": "^2.95.1",
"@douyinfe/semi-ui": "2.95.1",
"@douyinfe/semi-ui-19": "^2.95.1",
"@douyinfe/semi-icons": "^2.97.0",
"@douyinfe/semi-ui": "2.97.0",
"@douyinfe/semi-ui-19": "^2.97.0",
"@fastify/cookie": "^11.0.2",
"@fastify/helmet": "^13.0.2",
"@fastify/session": "^11.1.1",
@@ -75,53 +75,52 @@
"@turf/boolean-point-in-polygon": "^7.3.5",
"@vitejs/plugin-react": "6.0.1",
"adm-zip": "^0.5.17",
"better-sqlite3": "^12.9.0",
"better-sqlite3": "^12.10.0",
"chart.js": "^4.5.1",
"cheerio": "^1.2.0",
"cloakbrowser": "^0.3.28",
"fastify": "^5.8.5",
"handlebars": "4.7.9",
"maplibre-gl": "^5.24.0",
"nanoid": "5.1.9",
"nanoid": "5.1.11",
"node-cron": "^4.2.1",
"node-fetch": "3.3.2",
"node-mailjet": "6.0.11",
"nodemailer": "^8.0.7",
"p-throttle": "^8.1.0",
"package-up": "^5.0.0",
"puppeteer": "^24.42.0",
"puppeteer-extra": "^3.3.6",
"puppeteer-extra-plugin-stealth": "^2.11.2",
"puppeteer-core": "^24.43.1",
"query-string": "9.3.1",
"react": "19.2.5",
"react": "19.2.6",
"react-chartjs-2": "^5.3.1",
"react-dom": "19.2.5",
"react-dom": "19.2.6",
"react-range-slider-input": "^3.3.5",
"react-router": "7.14.2",
"react-router-dom": "7.14.2",
"resend": "^6.12.2",
"semver": "^7.7.4",
"react-router": "7.15.0",
"react-router-dom": "7.15.0",
"resend": "^6.12.3",
"semver": "^7.8.0",
"slack": "11.0.2",
"vite": "8.0.10",
"vite": "8.0.12",
"x-var": "^3.0.1",
"zustand": "^5.0.12"
"zustand": "^5.0.13"
},
"devDependencies": {
"@babel/core": "7.29.0",
"@babel/eslint-parser": "7.28.6",
"@babel/preset-env": "7.29.2",
"@babel/preset-env": "7.29.5",
"@babel/preset-react": "7.28.5",
"@eslint/js": "^10.0.1",
"chalk": "^5.6.2",
"eslint": "10.2.1",
"eslint": "10.3.0",
"eslint-config-prettier": "10.1.8",
"eslint-plugin-react": "7.37.5",
"globals": "^17.5.0",
"globals": "^17.6.0",
"history": "5.3.0",
"husky": "9.1.7",
"less": "4.6.4",
"lint-staged": "16.4.0",
"lint-staged": "17.0.4",
"nodemon": "^3.1.14",
"prettier": "3.8.3",
"vitest": "^4.1.5"
"vitest": "^4.1.6"
}
}

18
test/globalSetup.js Normal file
View File

@@ -0,0 +1,18 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import { ensureValidBinary } from '../lib/services/ensureValidBinary.js';
/**
* Vitest global setup — runs once in the main process before any workers start.
* Downloads and validates the CloakBrowser stealth Chromium binary.
* ensureValidBinary() also removes and re-downloads partial/corrupt installations
* so tests never fail with "Invalid file descriptor to ICU data received".
* Skipped in offline mode because the browser is fully mocked there.
*/
export async function setup() {
if (process.env.TEST_MODE === 'offline') return;
await ensureValidBinary();
}

View File

@@ -6,83 +6,89 @@
import * as similarityCache from '../../lib/services/similarity-check/similarityCache.js';
import { get } from '../mocks/mockNotification.js';
import { providerConfig, mockFredy } from '../utils.js';
import { expect, vi } from 'vitest';
import { expect } from 'vitest';
import * as provider from '../../lib/provider/immobilienDe.js';
import * as mockStore from '../mocks/mockStore.js';
import { launchBrowser, closeBrowser } from '../../lib/services/extractor/puppeteerExtractor.js';
// One browser shared across the whole suite so both requests (search + detail)
// come from the same warm session, avoiding double cold-start bot detection.
const TEST_TIMEOUT = 120_000;
describe('#immobilien.de testsuite()', () => {
provider.init(providerConfig.immobilienDe, [], []);
it('should test immobilien.de provider', async () => {
const mockedJob = {
id: 'test1',
notificationAdapter: null,
spatialFilter: null,
specFilter: null,
};
const Fredy = await mockFredy();
const fredy = new Fredy(provider.config, mockedJob, provider.metaInformation.id, similarityCache, undefined);
const listing = await fredy.execute();
let browser;
let liveListings;
if (listing == null || listing.length === 0) {
throw new Error('Listings is empty!');
}
beforeAll(async () => {
browser = await launchBrowser(providerConfig.immobilienDe.url);
}, TEST_TIMEOUT);
expect(listing).toBeInstanceOf(Array);
const notificationObj = get();
expect(notificationObj).toBeTypeOf('object');
expect(notificationObj.serviceName).toBe('immobilienDe');
notificationObj.payload.forEach((notify) => {
/** check the actual structure **/
expect(notify.id).toBeTypeOf('string');
expect(notify.price).toBeTypeOf('string');
expect(notify.size).toBeTypeOf('string');
expect(notify.title).toBeTypeOf('string');
expect(notify.link).toBeTypeOf('string');
expect(notify.address).toBeTypeOf('string');
/** check the values if possible **/
expect(notify.price).toContain('€');
expect(notify.size).toContain('m²');
expect(notify.title).not.toBe('');
expect(notify.link).toContain('https://www.immobilien.de');
expect(notify.address).not.toBe('');
});
afterAll(async () => {
await closeBrowser(browser);
});
it(
'should test immobilien.de provider',
async () => {
const mockedJob = {
id: 'test1',
notificationAdapter: null,
spatialFilter: null,
specFilter: null,
};
const Fredy = await mockFredy();
const fredy = new Fredy(provider.config, mockedJob, provider.metaInformation.id, similarityCache, browser);
liveListings = await fredy.execute();
if (liveListings == null || liveListings.length === 0) {
throw new Error('Listings is empty!');
}
expect(liveListings).toBeInstanceOf(Array);
const notificationObj = get();
expect(notificationObj).toBeTypeOf('object');
expect(notificationObj.serviceName).toBe('immobilienDe');
notificationObj.payload.forEach((notify) => {
/** check the actual structure **/
expect(notify.id).toBeTypeOf('string');
expect(notify.price).toBeTypeOf('string');
expect(notify.size).toBeTypeOf('string');
expect(notify.title).toBeTypeOf('string');
expect(notify.link).toBeTypeOf('string');
expect(notify.address).toBeTypeOf('string');
/** check the values if possible **/
expect(notify.price).toContain('€');
expect(notify.size).toContain('m²');
expect(notify.title).not.toBe('');
expect(notify.link).toContain('https://www.immobilien.de');
expect(notify.address).not.toBe('');
});
},
TEST_TIMEOUT,
);
describe('with provider_details enabled', () => {
beforeEach(() => {
vi.spyOn(mockStore, 'getUserSettings').mockReturnValue({ provider_details: [provider.metaInformation.id] });
vi.spyOn(mockStore, 'getKnownListingHashesForJobAndProvider').mockReturnValue([]);
});
it(
'should enrich listings with details',
async () => {
if (!liveListings?.length) throw new Error('No listings from first test to enrich');
afterEach(() => {
vi.restoreAllMocks();
});
// Call fetchDetails directly on the first live listing — no need to
// re-scrape the search page. The shared browser keeps the session warm.
const enriched = await provider.config.fetchDetails(liveListings[0], browser);
it('should enrich listings with details', async () => {
const Fredy = await mockFredy();
provider.init(providerConfig.immobilienDe, [], []);
const mockedJob = { id: 'test1', notificationAdapter: null, specFilter: null, spatialFilter: null };
const fredy = new Fredy(
provider.config,
mockedJob,
provider.metaInformation.id,
{ checkAndAddEntry: () => false },
undefined,
);
const listings = await fredy.execute();
if (listings == null) return;
expect(listings).toBeInstanceOf(Array);
listings.forEach((listing) => {
expect(listing.link).toContain('https://www.immobilien.de');
expect(listing.address).toBeTypeOf('string');
expect(listing.address).not.toBe('');
// description may be null if selectors don't match yet - falls back gracefully
if (listing.description != null) {
expect(listing.description).toBeTypeOf('string');
if (enriched == null) return;
expect(enriched.link).toContain('https://www.immobilien.de');
expect(enriched.address).toBeTypeOf('string');
expect(enriched.address).not.toBe('');
// description may be null if selectors don't match yet — falls back gracefully
if (enriched.description != null) {
expect(enriched.description).toBeTypeOf('string');
}
});
});
},
TEST_TIMEOUT,
);
});
});

View File

@@ -3,85 +3,85 @@
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import { expect, vi } from 'vitest';
import { expect } from 'vitest';
import * as similarityCache from '../../lib/services/similarity-check/similarityCache.js';
import { mockFredy, providerConfig } from '../utils.js';
import { get } from '../mocks/mockNotification.js';
import * as provider from '../../lib/provider/immoscout.js';
import * as mockStore from '../mocks/mockStore.js';
// immoscout uses the mobile REST API (fetch-based, no browser). Both tests share
// the same module-level listings so the API is only queried once, avoiding
// duplicate requests that could trigger rate-limiting.
const TEST_TIMEOUT = 120_000;
describe('#immoscout provider testsuite()', () => {
provider.init(providerConfig.immoscout, [], []);
it('should test immoscout provider', async () => {
const Fredy = await mockFredy();
const mockedJob = {
id: '',
notificationAdapter: null,
spatialFilter: null,
specFilter: null,
};
return await new Promise((resolve, reject) => {
const fredy = new Fredy(provider.config, mockedJob, provider.metaInformation.id, similarityCache, undefined);
fredy.execute().then((listings) => {
if (listings == null || listings.length === 0) {
reject('Listings is empty!');
return;
}
let liveListings;
expect(listings).toBeInstanceOf(Array);
const notificationObj = get();
expect(notificationObj).toBeTypeOf('object');
it(
'should test immoscout provider',
async () => {
const Fredy = await mockFredy();
const mockedJob = {
id: '',
notificationAdapter: null,
spatialFilter: null,
specFilter: null,
};
// check if there is at least one valid notification
const hasValidNotification = notificationObj.payload.some((notify) => {
return (
typeof notify.id === 'string' &&
typeof notify.price === 'string' &&
notify.price.includes('€') &&
typeof notify.size === 'string' &&
notify.size.includes('m²') &&
typeof notify.title === 'string' &&
notify.title !== '' &&
typeof notify.link === 'string' &&
notify.link.includes('https://www.immobilienscout24.de/') &&
typeof notify.address === 'string'
);
return await new Promise((resolve, reject) => {
const fredy = new Fredy(provider.config, mockedJob, provider.metaInformation.id, similarityCache, undefined);
fredy.execute().then((listings) => {
if (listings == null || listings.length === 0) {
reject('Listings is empty!');
return;
}
liveListings = listings;
expect(listings).toBeInstanceOf(Array);
const notificationObj = get();
expect(notificationObj).toBeTypeOf('object');
// check if there is at least one valid notification
const hasValidNotification = notificationObj.payload.some((notify) => {
return (
typeof notify.id === 'string' &&
typeof notify.price === 'string' &&
notify.price.includes('€') &&
typeof notify.size === 'string' &&
notify.size.includes('m²') &&
typeof notify.title === 'string' &&
notify.title !== '' &&
typeof notify.link === 'string' &&
notify.link.includes('https://www.immobilienscout24.de/') &&
typeof notify.address === 'string'
);
});
expect(hasValidNotification).toBe(true);
resolve();
});
expect(hasValidNotification).toBe(true);
resolve();
});
});
});
},
TEST_TIMEOUT,
);
describe('with provider_details enabled', () => {
beforeEach(() => {
vi.spyOn(mockStore, 'getUserSettings').mockReturnValue({ provider_details: [provider.metaInformation.id] });
vi.spyOn(mockStore, 'getKnownListingHashesForJobAndProvider').mockReturnValue([]);
});
it(
'should enrich listings with details',
async () => {
if (!liveListings?.length) throw new Error('No listings from first test to enrich');
afterEach(() => {
vi.restoreAllMocks();
});
// Call fetchDetails directly on the first live listing — no need to
// re-query the search API. immoscout uses fetch (no browser).
const enriched = await provider.config.fetchDetails(liveListings[0]);
it('should enrich listings with details', async () => {
const Fredy = await mockFredy();
provider.init(providerConfig.immoscout, [], []);
const mockedJob = { id: '', notificationAdapter: null, specFilter: null, spatialFilter: null };
const fredy = new Fredy(
provider.config,
mockedJob,
provider.metaInformation.id,
{ checkAndAddEntry: () => false },
undefined,
);
const listings = await fredy.execute();
expect(listings).toBeInstanceOf(Array);
listings.forEach((listing) => {
expect(listing.description).toBeTypeOf('string');
expect(listing.description).not.toBe('');
});
});
expect(enriched).toBeTruthy();
expect(enriched.description).toBeTypeOf('string');
expect(enriched.description).not.toBe('');
},
TEST_TIMEOUT,
);
});
});

View File

@@ -6,87 +6,95 @@
import * as similarityCache from '../../lib/services/similarity-check/similarityCache.js';
import { get } from '../mocks/mockNotification.js';
import { mockFredy, providerConfig } from '../utils.js';
import { expect, vi } from 'vitest';
import { expect } from 'vitest';
import * as provider from '../../lib/provider/immowelt.js';
import * as mockStore from '../mocks/mockStore.js';
import { launchBrowser, closeBrowser } from '../../lib/services/extractor/puppeteerExtractor.js';
// One browser shared across the whole suite so both requests (search + detail)
// come from the same warm session. Immowelt's CDN challenges cold sessions
// aggressively; a shared warm browser prevents the second request from being
// blocked as a bot hit.
const TEST_TIMEOUT = 180_000;
describe('#immowelt testsuite()', () => {
it('should test immowelt provider', async () => {
const Fredy = await mockFredy();
const mockedJob = {
id: 'immowelt',
notificationAdapter: null,
spatialFilter: null,
specFilter: null,
};
provider.init(providerConfig.immowelt, [], []);
let browser;
let liveListings;
const fredy = new Fredy(provider.config, mockedJob, provider.metaInformation.id, similarityCache, undefined);
beforeAll(async () => {
browser = await launchBrowser(providerConfig.immowelt.url);
}, TEST_TIMEOUT);
const listing = await fredy.execute();
if (listing == null || listing.length === 0) {
throw new Error('Listings is empty!');
}
expect(listing).toBeInstanceOf(Array);
const notificationObj = get();
expect(notificationObj).toBeTypeOf('object');
expect(notificationObj.serviceName).toBe('immowelt');
notificationObj.payload.forEach((notify) => {
/** check the actual structure **/
expect(notify.id).toBeTypeOf('string');
if (notify.price != null) {
expect(notify.price).toBeTypeOf('string');
expect(notify.price).toContain('€');
}
expect(notify.title).toBeTypeOf('string');
expect(notify.link).toBeTypeOf('string');
expect(notify.address).toBeTypeOf('string');
/** check the values if possible **/
if (notify.size != null && notify.size.trim().toLowerCase() !== 'k.a.') {
expect(notify.size).toBeTypeOf('string');
expect(notify.size).toContain('m²');
}
expect(notify.title).not.toBe('');
expect(notify.link).toContain('https://www.immowelt.de');
expect(notify.address).not.toBe('');
});
afterAll(async () => {
await closeBrowser(browser);
});
it(
'should test immowelt provider',
async () => {
const Fredy = await mockFredy();
const mockedJob = {
id: 'immowelt',
notificationAdapter: null,
spatialFilter: null,
specFilter: null,
};
provider.init(providerConfig.immowelt, [], []);
const fredy = new Fredy(provider.config, mockedJob, provider.metaInformation.id, similarityCache, browser);
liveListings = await fredy.execute();
if (liveListings == null || liveListings.length === 0) {
throw new Error('Listings is empty!');
}
expect(liveListings).toBeInstanceOf(Array);
const notificationObj = get();
expect(notificationObj).toBeTypeOf('object');
expect(notificationObj.serviceName).toBe('immowelt');
notificationObj.payload.forEach((notify) => {
/** check the actual structure **/
expect(notify.id).toBeTypeOf('string');
if (notify.price != null) {
expect(notify.price).toBeTypeOf('string');
expect(notify.price).toContain('€');
}
expect(notify.title).toBeTypeOf('string');
expect(notify.link).toBeTypeOf('string');
expect(notify.address).toBeTypeOf('string');
/** check the values if possible **/
if (notify.size != null && notify.size.trim().toLowerCase() !== 'k.a.') {
expect(notify.size).toBeTypeOf('string');
expect(notify.size).toContain('m²');
}
expect(notify.title).not.toBe('');
expect(notify.link).toContain('https://www.immowelt.de');
expect(notify.address).not.toBe('');
});
},
TEST_TIMEOUT,
);
describe('with provider_details enabled', () => {
beforeEach(() => {
vi.spyOn(mockStore, 'getUserSettings').mockReturnValue({ provider_details: [provider.metaInformation.id] });
vi.spyOn(mockStore, 'getKnownListingHashesForJobAndProvider').mockReturnValue([]);
});
it(
'should enrich listings with details',
async () => {
if (!liveListings?.length) throw new Error('No listings from first test to enrich');
afterEach(() => {
vi.restoreAllMocks();
});
// Call fetchDetails directly on the first live listing — no need to
// re-scrape the search page. The shared browser keeps the session warm.
const enriched = await provider.config.fetchDetails(liveListings[0], browser);
it('should enrich listings with details', async () => {
const Fredy = await mockFredy();
provider.init(providerConfig.immowelt, [], []);
const mockedJob = { id: 'immowelt', notificationAdapter: null, specFilter: null, spatialFilter: null };
const fredy = new Fredy(
provider.config,
mockedJob,
provider.metaInformation.id,
{ checkAndAddEntry: () => false },
undefined,
);
const listings = await fredy.execute();
expect(listings).toBeInstanceOf(Array);
listings.forEach((listing) => {
expect(listing.link).toContain('https://www.immowelt.de');
expect(listing.address).toBeTypeOf('string');
expect(listing.address).not.toBe('');
expect(enriched).toBeTruthy();
expect(enriched.link).toContain('https://www.immowelt.de');
expect(enriched.address).toBeTypeOf('string');
expect(enriched.address).not.toBe('');
// description is enriched from the detail page; falls back gracefully if blocked
if (listing.description != null) {
expect(listing.description).toBeTypeOf('string');
if (enriched.description != null) {
expect(enriched.description).toBeTypeOf('string');
}
});
});
},
TEST_TIMEOUT,
);
});
});

View File

@@ -6,80 +6,88 @@
import * as similarityCache from '../../lib/services/similarity-check/similarityCache.js';
import { get } from '../mocks/mockNotification.js';
import { mockFredy, providerConfig } from '../utils.js';
import { expect, vi } from 'vitest';
import { expect } from 'vitest';
import * as provider from '../../lib/provider/kleinanzeigen.js';
import * as mockStore from '../mocks/mockStore.js';
import { launchBrowser, closeBrowser } from '../../lib/services/extractor/puppeteerExtractor.js';
// One browser shared across the whole suite so both requests (search + detail)
// come from the same warm session. Kleinanzeigen rate-limits cold browser
// sessions; a shared warm browser prevents the second request from being blocked.
const TEST_TIMEOUT = 180_000;
describe('#kleinanzeigen testsuite()', () => {
it('should test kleinanzeigen provider', async () => {
const Fredy = await mockFredy();
const mockedJob = {
id: 'kleinanzeigen',
notificationAdapter: null,
spatialFilter: null,
specFilter: null,
};
provider.init(providerConfig.kleinanzeigen, [], []);
return await new Promise((resolve, reject) => {
const fredy = new Fredy(provider.config, mockedJob, provider.metaInformation.id, similarityCache, undefined);
let browser;
let liveListings;
fredy.execute().then((listing) => {
if (listing == null || listing.length === 0) {
reject('Listings is empty!');
return;
}
beforeAll(async () => {
browser = await launchBrowser(providerConfig.kleinanzeigen.url);
}, TEST_TIMEOUT);
expect(listing).toBeInstanceOf(Array);
const notificationObj = get();
expect(notificationObj).toBeTypeOf('object');
expect(notificationObj.serviceName).toBe('kleinanzeigen');
notificationObj.payload.forEach((notify) => {
/** check the actual structure **/
expect(notify.id).toBeTypeOf('string');
expect(notify.title).toBeTypeOf('string');
expect(notify.link).toBeTypeOf('string');
expect(notify.address).toBeTypeOf('string');
/** check the values if possible **/
expect(notify.title).not.toBe('');
expect(notify.link).toContain('https://www.kleinanzeigen.de');
expect(notify.address).not.toBe('');
});
resolve();
});
});
afterAll(async () => {
await closeBrowser(browser);
});
it(
'should test kleinanzeigen provider',
async () => {
const Fredy = await mockFredy();
const mockedJob = {
id: 'kleinanzeigen',
notificationAdapter: null,
spatialFilter: null,
specFilter: null,
};
provider.init(providerConfig.kleinanzeigen, [], []);
return await new Promise((resolve, reject) => {
const fredy = new Fredy(provider.config, mockedJob, provider.metaInformation.id, similarityCache, browser);
fredy.execute().then((listing) => {
if (listing == null || listing.length === 0) {
reject('Listings is empty!');
return;
}
liveListings = listing;
expect(listing).toBeInstanceOf(Array);
const notificationObj = get();
expect(notificationObj).toBeTypeOf('object');
expect(notificationObj.serviceName).toBe('kleinanzeigen');
notificationObj.payload.forEach((notify) => {
/** check the actual structure **/
expect(notify.id).toBeTypeOf('string');
expect(notify.title).toBeTypeOf('string');
expect(notify.link).toBeTypeOf('string');
expect(notify.address).toBeTypeOf('string');
/** check the values if possible **/
expect(notify.title).not.toBe('');
expect(notify.link).toContain('https://www.kleinanzeigen.de');
expect(notify.address).not.toBe('');
});
resolve();
});
});
},
TEST_TIMEOUT,
);
describe('with provider_details enabled', () => {
beforeEach(() => {
vi.spyOn(mockStore, 'getUserSettings').mockReturnValue({ provider_details: [provider.metaInformation.id] });
vi.spyOn(mockStore, 'getKnownListingHashesForJobAndProvider').mockReturnValue([]);
});
it(
'should enrich listings with details',
async () => {
if (!liveListings?.length) throw new Error('No listings from first test to enrich');
afterEach(() => {
vi.restoreAllMocks();
});
// Call fetchDetails directly on the first live listing — no need to
// re-scrape the search page. The shared browser keeps the session warm.
const enriched = await provider.config.fetchDetails(liveListings[0], browser);
it('should enrich listings with details', async () => {
const Fredy = await mockFredy();
provider.init(providerConfig.kleinanzeigen, [], []);
const mockedJob = { id: 'kleinanzeigen', notificationAdapter: null, specFilter: null, spatialFilter: null };
const fredy = new Fredy(
provider.config,
mockedJob,
provider.metaInformation.id,
{ checkAndAddEntry: () => false },
undefined,
);
const listings = await fredy.execute();
expect(listings).toBeInstanceOf(Array);
listings.forEach((listing) => {
expect(listing.link).toContain('https://www.kleinanzeigen.de');
expect(listing.address).toBeTypeOf('string');
expect(listing.address).not.toBe('');
expect(listing.description).toBeTypeOf('string');
expect(listing.description).not.toBe('');
});
});
expect(enriched).toBeTruthy();
expect(enriched.link).toContain('https://www.kleinanzeigen.de');
expect(enriched.address).toBeTypeOf('string');
expect(enriched.address).not.toBe('');
expect(enriched.description).toBeTypeOf('string');
expect(enriched.description).not.toBe('');
},
TEST_TIMEOUT,
);
});
});

View File

@@ -9,81 +9,97 @@ import { mockFredy, providerConfig } from '../utils.js';
import { expect, vi } from 'vitest';
import * as provider from '../../lib/provider/sparkasse.js';
import * as mockStore from '../mocks/mockStore.js';
import { launchBrowser, closeBrowser } from '../../lib/services/extractor/puppeteerExtractor.js';
// One browser shared across the whole suite so both requests (search + detail)
// come from the same warm session. This prevents the second request from being
// flagged as a cold-start bot hit.
const TEST_TIMEOUT = 120_000;
describe('#sparkasse testsuite()', () => {
it('should test sparkasse provider', async () => {
const Fredy = await mockFredy();
const mockedJob = {
id: 'sparkasse',
notificationAdapter: null,
spatialFilter: null,
specFilter: null,
};
provider.init(providerConfig.sparkasse, []);
let browser;
let liveListings;
const fredy = new Fredy(provider.config, mockedJob, provider.metaInformation.id, similarityCache, undefined);
beforeAll(async () => {
browser = await launchBrowser(providerConfig.sparkasse.url);
}, TEST_TIMEOUT);
const listing = await fredy.execute();
if (listing == null || listing.length === 0) {
throw new Error('Listings is empty!');
}
expect(listing).toBeInstanceOf(Array);
const notificationObj = get();
expect(notificationObj).toBeTypeOf('object');
expect(notificationObj.serviceName).toBe('sparkasse');
notificationObj.payload.forEach((notify) => {
/** check the actual structure **/
expect(notify.id).toBeTypeOf('string');
expect(notify.price).toBeTypeOf('string');
expect(notify.price).toContain('€');
expect(notify.size).toBeTypeOf('string');
expect(notify.size).toContain('m²');
expect(notify.title).toBeTypeOf('string');
expect(notify.link).toBeTypeOf('string');
expect(notify.address).toBeTypeOf('string');
/** check the values if possible **/
expect(notify.size).toBeTypeOf('string');
expect(notify.title).not.toBe('');
expect(notify.address).not.toBe('');
});
afterAll(async () => {
await closeBrowser(browser);
});
it(
'should test sparkasse provider',
async () => {
const Fredy = await mockFredy();
const mockedJob = {
id: 'sparkasse',
notificationAdapter: null,
spatialFilter: null,
specFilter: null,
};
provider.init(providerConfig.sparkasse, []);
const fredy = new Fredy(provider.config, mockedJob, provider.metaInformation.id, similarityCache, browser);
liveListings = await fredy.execute();
if (liveListings == null || liveListings.length === 0) {
throw new Error('Listings is empty!');
}
expect(liveListings).toBeInstanceOf(Array);
const notificationObj = get();
expect(notificationObj).toBeTypeOf('object');
expect(notificationObj.serviceName).toBe('sparkasse');
notificationObj.payload.forEach((notify) => {
/** check the actual structure **/
expect(notify.id).toBeTypeOf('string');
expect(notify.price).toBeTypeOf('string');
expect(notify.price).toContain('€');
expect(notify.size).toBeTypeOf('string');
expect(notify.size).toContain('m²');
expect(notify.title).toBeTypeOf('string');
expect(notify.link).toBeTypeOf('string');
expect(notify.address).toBeTypeOf('string');
/** check the values if possible **/
expect(notify.size).toBeTypeOf('string');
expect(notify.title).not.toBe('');
expect(notify.address).not.toBe('');
});
},
TEST_TIMEOUT,
);
describe('with provider_details enabled', () => {
beforeEach(() => {
vi.spyOn(mockStore, 'getUserSettings').mockReturnValue({ provider_details: [provider.metaInformation.id] });
vi.spyOn(mockStore, 'getKnownListingHashesForJobAndProvider').mockReturnValue([]);
});
afterEach(() => {
vi.restoreAllMocks();
});
it('should enrich listings with details', async () => {
const Fredy = await mockFredy();
provider.init(providerConfig.sparkasse, []);
const mockedJob = { id: 'sparkasse', notificationAdapter: null, specFilter: null, spatialFilter: null };
it(
'should enrich listings with details',
async () => {
if (!liveListings?.length) throw new Error('No listings from first test to enrich');
const fredy = new Fredy(
provider.config,
mockedJob,
provider.metaInformation.id,
{ checkAndAddEntry: () => false },
undefined,
);
const listings = await fredy.execute();
expect(listings).toBeInstanceOf(Array);
listings.forEach((listing) => {
expect(listing.link).toContain('https://immobilien.sparkasse.de');
expect(listing.address).toBeTypeOf('string');
expect(listing.address).not.toBe('');
// description is enriched from the detail page; falls back gracefully if bot-detected
if (listing.description != null) {
expect(listing.description).toBeTypeOf('string');
expect(listing.description).not.toBe('');
// Call fetchDetails directly on the first live listing — no need to
// re-scrape the search page. The shared browser keeps the session warm.
const enriched = await provider.config.fetchDetails(liveListings[0], browser);
expect(enriched).toBeTruthy();
expect(enriched.link).toContain('https://immobilien.sparkasse.de');
expect(enriched.address).toBeTypeOf('string');
expect(enriched.address).not.toBe('');
// description is enriched from the detail page; falls back gracefully if blocked
if (enriched.description != null) {
expect(enriched.description).toBeTypeOf('string');
expect(enriched.description).not.toBe('');
}
});
});
},
TEST_TIMEOUT,
);
});
});

View File

@@ -6,77 +6,85 @@
import * as similarityCache from '../../lib/services/similarity-check/similarityCache.js';
import { get } from '../mocks/mockNotification.js';
import { mockFredy, providerConfig } from '../utils.js';
import { expect, vi } from 'vitest';
import { expect } from 'vitest';
import * as provider from '../../lib/provider/wgGesucht.js';
import * as mockStore from '../mocks/mockStore.js';
import { launchBrowser, closeBrowser } from '../../lib/services/extractor/puppeteerExtractor.js';
// One browser shared across the whole suite so both requests (search + detail)
// come from the same warm session, avoiding double cold-start bot detection.
const TEST_TIMEOUT = 120_000;
describe('#wgGesucht testsuite()', () => {
provider.init(providerConfig.wgGesucht, [], []);
it('should test wgGesucht provider', { timeout: 120000 }, async () => {
const Fredy = await mockFredy();
const mockedJob = {
id: 'wgGesucht',
notificationAdapter: null,
spatialFilter: null,
specFilter: null,
};
return await new Promise((resolve, reject) => {
const fredy = new Fredy(provider.config, mockedJob, provider.metaInformation.id, similarityCache, undefined);
let browser;
let liveListings;
fredy.execute().then((listing) => {
if (listing == null || listing.length === 0) {
reject('Listings is empty!');
return;
}
beforeAll(async () => {
browser = await launchBrowser(providerConfig.wgGesucht.url);
}, TEST_TIMEOUT);
expect(listing).toBeInstanceOf(Array);
const notificationObj = get();
expect(notificationObj.serviceName).toBe('wgGesucht');
notificationObj.payload.forEach((notify) => {
expect(notify).toBeTypeOf('object');
/** check the actual structure **/
expect(notify.id).toBeTypeOf('string');
expect(notify.title).toBeTypeOf('string');
// expect(notify.details).toBeTypeOf('string');
expect(notify.price).toBeTypeOf('string');
expect(notify.price).toContain('€');
expect(notify.link).toBeTypeOf('string');
});
resolve();
});
});
afterAll(async () => {
await closeBrowser(browser);
});
it(
'should test wgGesucht provider',
async () => {
const Fredy = await mockFredy();
const mockedJob = {
id: 'wgGesucht',
notificationAdapter: null,
spatialFilter: null,
specFilter: null,
};
return await new Promise((resolve, reject) => {
const fredy = new Fredy(provider.config, mockedJob, provider.metaInformation.id, similarityCache, browser);
fredy.execute().then((listing) => {
if (listing == null || listing.length === 0) {
reject('Listings is empty!');
return;
}
liveListings = listing;
expect(listing).toBeInstanceOf(Array);
const notificationObj = get();
expect(notificationObj.serviceName).toBe('wgGesucht');
notificationObj.payload.forEach((notify) => {
expect(notify).toBeTypeOf('object');
/** check the actual structure **/
expect(notify.id).toBeTypeOf('string');
expect(notify.title).toBeTypeOf('string');
// expect(notify.details).toBeTypeOf('string');
expect(notify.price).toBeTypeOf('string');
expect(notify.price).toContain('€');
expect(notify.link).toBeTypeOf('string');
});
resolve();
});
});
},
TEST_TIMEOUT,
);
describe('with provider_details enabled', () => {
beforeEach(() => {
vi.spyOn(mockStore, 'getUserSettings').mockReturnValue({ provider_details: [provider.metaInformation.id] });
vi.spyOn(mockStore, 'getKnownListingHashesForJobAndProvider').mockReturnValue([]);
});
it(
'should enrich listings with details',
async () => {
if (!liveListings?.length) throw new Error('No listings from first test to enrich');
afterEach(() => {
vi.restoreAllMocks();
});
// Call fetchDetails directly on the first live listing — no need to
// re-scrape the search page. The shared browser keeps the session warm.
const enriched = await provider.config.fetchDetails(liveListings[0], browser);
it('should enrich listings with details', async () => {
const Fredy = await mockFredy();
provider.init(providerConfig.wgGesucht, [], []);
const mockedJob = { id: 'wgGesucht', notificationAdapter: null, specFilter: null, spatialFilter: null };
const fredy = new Fredy(
provider.config,
mockedJob,
provider.metaInformation.id,
{ checkAndAddEntry: () => false },
undefined,
);
const listings = await fredy.execute();
expect(listings).toBeInstanceOf(Array);
listings.forEach((listing) => {
expect(listing.link).toContain('https://www.wg-gesucht.de');
expect(listing.description).toBeTypeOf('string');
expect(listing.description).not.toBe('');
});
});
expect(enriched).toBeTruthy();
expect(enriched.link).toContain('https://www.wg-gesucht.de');
expect(enriched.description).toBeTypeOf('string');
expect(enriched.description).not.toBe('');
},
TEST_TIMEOUT,
);
});
});

View File

@@ -10,6 +10,17 @@ import { readFile } from 'fs/promises';
export const testData = JSON.parse(await readFile(new URL('./testdata.json', import.meta.url)));
describe('#immoscout-mobile URL conversion', () => {
// Test shape URL conversion
it('should convert a full web URL with shape to mobile URL', () => {
const webUrl =
'https://www.immobilienscout24.de/Suche/shape/haus-kaufen?shape=aW9yfkhfa3htQXJgUGlnYEBmekhte3BAcXNAfWBsQGNyQ2lkUHVvbEB3eX5Ab25WYn5Fa2BLaGRQY29FaGtTfEhme3xBdHBEdHFMamlHbmdRfHhMcmxPeHlWYnpS&price=-600000.0&ground=240.0-&enteredFrom=result_list';
const expectedMobileUrl =
'https://api.mobile.immobilienscout24.de/search/list?ground=240.0-&price=-600000.0&realestatetype=housebuy&searchType=shape&shape=ior~H_kxmAr%60Pig%60%40fzHm%7Bp%40qs%40%7D%60l%40crCidPuol%40wy~%40onVb~Ek%60KhdPcoEhkS%7CHf%7B%7CAtpDtqLjiGngQ%7CxLrlOxyVbzR';
const actualMobileUrl = convertWebToMobile(webUrl);
expect(actualMobileUrl).toBe(expectedMobileUrl);
});
// Test URL conversion
it('should convert a full web URL to mobile URL', () => {
const webUrl =

View File

@@ -18,5 +18,9 @@
"rentHouse": {
"url": "https://www.immobilienscout24.de/Suche/de/nordrhein-westfalen/duesseldorf/haus-mieten?enteredFrom=one_step_search",
"type": "houserent"
},
"buyHouseWithShape": {
"url": "https://www.immobilienscout24.de/Suche/shape/haus-kaufen?shape=aW9yfkhfa3htQXJgUGlnYEBmekhte3BAcXNAfWBsQGNyQ2lkUHVvbEB3eX5Ab25WYn5Fa2BLaGRQY29FaGtTfEhme3xBdHBEdHFMamlHbmdRfHhMcmxPeHlWYnpS&price=-600000.0&ground=240.0-&enteredFrom=result_list",
"type": "housebuy"
}
}

View File

@@ -38,6 +38,7 @@ describe('services/jobs/jobExecutionService', () => {
}));
vi.doMock(utilsPath, () => ({
duringWorkingHoursOrNotSet: () => false,
getPackageVersion: async () => '0.0.0-test',
}));
vi.doMock(loggerPath, () => {
const m = { debug: () => {}, info: () => {}, warn: () => {}, error: () => {} };

View File

@@ -29,7 +29,7 @@ vi.mock('../lib/services/extractor/puppeteerExtractor.js', async (importOriginal
const { readFixture } = await import('./offlineFixtures.js');
return {
default: (url) => readFixture(url),
launchBrowser: async () => ({ close: async () => {}, __fredy_removeUserDataDir: false }),
launchBrowser: async () => ({ close: async () => {}, isConnected: () => true }),
closeBrowser: async () => {},
};
});

BIN
ui/src/assets/news/1.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 835 KiB

BIN
ui/src/assets/news/2.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 367 KiB

View File

@@ -1,10 +1,16 @@
{
"key": "00e6b81777a275f5a140fc9101cb943810db6a69f6eb3927319c5aee0c876542",
"key": "00e6b81777a275f5a140fc9101cb943810db6a69f6eb3927319c5aee0c876221",
"content":
[
{
"title": "Open in...Fredy ;)",
"text": "With the latest version of Fredy, every notification now comes with a link that opens the listing directly inside Fredy. This is also a key step toward an upcoming...milestone :).<br/>To make this work, Fredy needs to know where it lives on the network. We try to guess the public base URL, but lets be honest, you probably know better. Take a quick look at the baseUrl in the system settings and fix it if it looks off."
"title": "Table overview for listings",
"text": "Thanks to https://github.com/datenwurm, we now have a table overview for listings. If you decide to use the table view, the decision will be stored.",
"media": "1.png"
},
{
"title": "Table overview for jobs",
"text": "Based on datenwurm's, work, I created a table overview for jobs. If you decide to use the table view, the decision will be stored.",
"media": "2.png"
}
]
}

View File

@@ -21,6 +21,7 @@ import {
Empty,
Radio,
RadioGroup,
Tooltip,
} from '@douyinfe/semi-ui-19';
import {
IconAlertTriangle,
@@ -35,6 +36,8 @@ import {
IconArrowUp,
IconArrowDown,
IconHome,
IconGridView,
IconList,
} from '@douyinfe/semi-icons';
import { useNavigate } from 'react-router-dom';
import ListingDeletionModal from '../../ListingDeletionModal.jsx';
@@ -42,6 +45,7 @@ import { useActions, useSelector } from '../../../services/state/store.js';
import { xhrDelete, xhrPut, xhrPost } from '../../../services/xhr.js';
import { debounce } from '../../../utils';
import { IllustrationNoResult, IllustrationNoResultDark } from '@douyinfe/semi-illustrations';
import JobsTable from '../../table/JobsTable.jsx';
import './JobGrid.less';
@@ -54,6 +58,9 @@ const JobGrid = () => {
const actions = useActions();
const navigate = useNavigate();
const userSettings = useSelector((state) => state.userSettings.settings);
const viewMode = userSettings?.jobs_view_mode ?? 'grid';
const [page, setPage] = useState(1);
const pageSize = 12;
@@ -234,6 +241,27 @@ const JobGrid = () => {
onClick={() => setSortDir((d) => (d === 'asc' ? 'desc' : 'asc'))}
title={sortDir === 'asc' ? 'Ascending' : 'Descending'}
/>
<div className="jobGrid__topbar__view-toggle">
<Tooltip content="Grid view">
<Button
icon={<IconGridView />}
theme={viewMode === 'grid' ? 'solid' : 'borderless'}
onClick={() => actions.userSettings.setJobsViewMode('grid')}
aria-label="Grid view"
aria-pressed={viewMode === 'grid'}
/>
</Tooltip>
<Tooltip content="Table view">
<Button
icon={<IconList />}
theme={viewMode === 'table' ? 'solid' : 'borderless'}
onClick={() => actions.userSettings.setJobsViewMode('table')}
aria-label="Table view"
aria-pressed={viewMode === 'table'}
/>
</Tooltip>
</div>
</div>
{(jobsData?.result || []).length === 0 && (
@@ -244,136 +272,144 @@ const JobGrid = () => {
/>
)}
<Row gutter={[16, 16]}>
{(jobsData?.result || []).map((job) => (
<Col key={job.id} xs={24} sm={12} md={12} lg={8} xl={8} xxl={6}>
<Card className="jobGrid__card" bodyStyle={{ padding: '16px' }}>
<div className="jobGrid__card__header">
<div className="jobGrid__card__name">
<span className={`jobGrid__card__dot${job.enabled ? ' jobGrid__card__dot--active' : ''}`} />
<Title heading={5} ellipsis={{ showTooltip: true }} className="jobGrid__title">
{job.name}
</Title>
{viewMode === 'grid' ? (
<Row gutter={[16, 16]}>
{(jobsData?.result || []).map((job) => (
<Col key={job.id} xs={24} sm={12} md={12} lg={8} xl={8} xxl={6}>
<Card className="jobGrid__card" bodyStyle={{ padding: '16px' }}>
<div className="jobGrid__card__header">
<div className="jobGrid__card__name">
<span className={`jobGrid__card__dot${job.enabled ? ' jobGrid__card__dot--active' : ''}`} />
<Title heading={5} ellipsis={{ showTooltip: true }} className="jobGrid__title">
{job.name}
</Title>
</div>
<div style={{ display: 'flex', alignItems: 'center', gap: 6, flexShrink: 0 }}>
{job.isOnlyShared && (
<Popover content={getPopoverContent('This job has been shared with you — read only.')}>
<div>
<IconAlertTriangle style={{ color: 'rgba(var(--semi-yellow-7), 1)' }} />
</div>
</Popover>
)}
{job.running && (
<Tag color="green" variant="light" size="small">
RUNNING
</Tag>
)}
</div>
</div>
<div style={{ display: 'flex', alignItems: 'center', gap: 6, flexShrink: 0 }}>
{job.isOnlyShared && (
<Popover
content={getPopoverContent(
'This job has been shared with you by another user, therefor it is read-only.',
)}
>
<div className="jobGrid__card__stats">
<div className="jobGrid__card__stat jobGrid__card__stat--blue">
<span className="jobGrid__card__stat__number">{job.numberOfFoundListings || 0}</span>
<span className="jobGrid__card__stat__label">
<IconHome size="small" /> Listings
</span>
</div>
<div className="jobGrid__card__stat jobGrid__card__stat--orange">
<span className="jobGrid__card__stat__number">{job.provider?.length || 0}</span>
<span className="jobGrid__card__stat__label">
<IconBriefcase size="small" /> Providers
</span>
</div>
<div className="jobGrid__card__stat jobGrid__card__stat--purple">
<span className="jobGrid__card__stat__number">{job.notificationAdapter?.length || 0}</span>
<span className="jobGrid__card__stat__label">
<IconBell size="small" /> Adapters
</span>
</div>
</div>
<Divider margin="12px" />
<div className="jobGrid__card__footer">
<div style={{ display: 'flex', alignItems: 'center', gap: 6 }}>
<Switch
onChange={(checked) => onJobStatusChanged(job.id, checked)}
checked={job.enabled}
disabled={job.isOnlyShared}
size="small"
/>
<Text type="secondary" size="small">
Active
</Text>
</div>
<div className="jobGrid__actions">
<Popover content={getPopoverContent('Run Job')}>
<div>
<IconAlertTriangle style={{ color: 'rgba(var(--semi-yellow-7), 1)' }} />
<Button
type="primary"
style={{ background: '#21aa21b5' }}
size="small"
theme="solid"
icon={<IconPlayCircle />}
disabled={job.isOnlyShared || job.running}
onClick={() => onJobRun(job.id)}
/>
</div>
</Popover>
)}
{job.running && (
<Tag color="green" variant="light" size="small">
RUNNING
</Tag>
)}
<Popover content={getPopoverContent('Edit a Job')}>
<div>
<Button
type="secondary"
size="small"
icon={<IconEdit />}
disabled={job.isOnlyShared}
onClick={() => navigate(`/jobs/edit/${job.id}`)}
/>
</div>
</Popover>
<Popover content={getPopoverContent('Clone Job')}>
<div>
<Button
type="tertiary"
size="small"
icon={<IconCopy />}
disabled={job.isOnlyShared}
onClick={() => navigate('/jobs/new', { state: { cloneFrom: job.id } })}
/>
</div>
</Popover>
<Popover content={getPopoverContent('Delete all found Listings of this Job')}>
<div>
<Button
type="danger"
size="small"
icon={<IconDescend2 />}
disabled={job.isOnlyShared}
onClick={() => onListingRemoval(job.id)}
/>
</div>
</Popover>
<Popover content={getPopoverContent('Delete Job')}>
<div>
<Button
type="danger"
size="small"
icon={<IconDelete />}
disabled={job.isOnlyShared}
onClick={() => onJobRemoval(job.id)}
/>
</div>
</Popover>
</div>
</div>
</div>
<div className="jobGrid__card__stats">
<div className="jobGrid__card__stat jobGrid__card__stat--blue">
<span className="jobGrid__card__stat__number">{job.numberOfFoundListings || 0}</span>
<span className="jobGrid__card__stat__label">
<IconHome size="small" /> Listings
</span>
</div>
<div className="jobGrid__card__stat jobGrid__card__stat--orange">
<span className="jobGrid__card__stat__number">{job.provider.length || 0}</span>
<span className="jobGrid__card__stat__label">
<IconBriefcase size="small" /> Providers
</span>
</div>
<div className="jobGrid__card__stat jobGrid__card__stat--purple">
<span className="jobGrid__card__stat__number">{job.notificationAdapter.length || 0}</span>
<span className="jobGrid__card__stat__label">
<IconBell size="small" /> Adapters
</span>
</div>
</div>
<Divider margin="12px" />
<div className="jobGrid__card__footer">
<div style={{ display: 'flex', alignItems: 'center', gap: 6 }}>
<Switch
onChange={(checked) => onJobStatusChanged(job.id, checked)}
checked={job.enabled}
disabled={job.isOnlyShared}
size="small"
/>
<Text type="secondary" size="small">
Active
</Text>
</div>
<div className="jobGrid__actions">
<Popover content={getPopoverContent('Run Job')}>
<div>
<Button
type="primary"
style={{ background: '#21aa21b5' }}
size="small"
theme="solid"
icon={<IconPlayCircle />}
disabled={job.isOnlyShared || job.running}
onClick={() => onJobRun(job.id)}
/>
</div>
</Popover>
<Popover content={getPopoverContent('Edit a Job')}>
<div>
<Button
type="secondary"
size="small"
icon={<IconEdit />}
disabled={job.isOnlyShared}
onClick={() => navigate(`/jobs/edit/${job.id}`)}
/>
</div>
</Popover>
<Popover content={getPopoverContent('Clone Job')}>
<div>
<Button
type="tertiary"
size="small"
icon={<IconCopy />}
disabled={job.isOnlyShared}
onClick={() => navigate('/jobs/new', { state: { cloneFrom: job.id } })}
/>
</div>
</Popover>
<Popover content={getPopoverContent('Delete all found Listings of this Job')}>
<div>
<Button
type="danger"
size="small"
icon={<IconDescend2 />}
disabled={job.isOnlyShared}
onClick={() => onListingRemoval(job.id)}
/>
</div>
</Popover>
<Popover content={getPopoverContent('Delete Job')}>
<div>
<Button
type="danger"
size="small"
icon={<IconDelete />}
disabled={job.isOnlyShared}
onClick={() => onJobRemoval(job.id)}
/>
</div>
</Popover>
</div>
</div>
</Card>
</Col>
))}
</Row>
</Card>
</Col>
))}
</Row>
) : (
<JobsTable
jobs={jobsData?.result || []}
onRun={onJobRun}
onEdit={(id) => navigate(`/jobs/edit/${id}`)}
onClone={(id) => navigate('/jobs/new', { state: { cloneFrom: id } })}
onDeleteListings={onListingRemoval}
onDeleteJob={onJobRemoval}
onStatusChange={onJobStatusChanged}
/>
)}
{(jobsData?.result || []).length > 0 && jobsData?.totalNumber > 12 && (
<div className="jobGrid__pagination">
<Pagination

View File

@@ -17,6 +17,12 @@
flex: 1;
min-width: 160px;
}
&__view-toggle {
display: flex;
gap: 4px;
flex-shrink: 0;
}
}
&__card {

View File

@@ -3,14 +3,7 @@
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import { useState, useEffect, useMemo } from 'react';
import {
useSearchParamState,
parseNumber,
parseString,
parseNullableBoolean,
} from '../../../hooks/useSearchParamState.js';
import { Button, Pagination, Toast, Input, Select, Empty, Radio, RadioGroup, Tooltip } from '@douyinfe/semi-ui-19';
import { Button, Tooltip } from '@douyinfe/semi-ui-19';
import {
IconBriefcase,
IconCart,
@@ -19,323 +12,117 @@ import {
IconMapPin,
IconStar,
IconStarStroked,
IconSearch,
IconEyeOpened,
IconArrowUp,
IconArrowDown,
} from '@douyinfe/semi-icons';
import { useNavigate, useSearchParams } from 'react-router-dom';
import ListingDeletionModal from '../../ListingDeletionModal.jsx';
import no_image from '../../../assets/no_image.png';
import * as timeService from '../../../services/time/timeService.js';
import { xhrDelete, xhrPost } from '../../../services/xhr.js';
import { useActions, useSelector } from '../../../services/state/store.js';
import { debounce } from '../../../utils';
import './ListingsGrid.less';
import { IllustrationNoResult, IllustrationNoResultDark } from '@douyinfe/semi-illustrations';
const ListingsGrid = () => {
const listingsData = useSelector((state) => state.listingsData);
const providers = useSelector((state) => state.provider);
const jobs = useSelector((state) => state.jobsData.jobs);
const actions = useActions();
const navigate = useNavigate();
const sp = useSearchParams();
const [page, setPage] = useSearchParamState(sp, 'page', 1, parseNumber);
const pageSize = 40;
const [sortField, setSortField] = useSearchParamState(sp, 'sort', 'created_at', parseString);
const [sortDir, setSortDir] = useSearchParamState(sp, 'dir', 'desc', parseString);
const [freeTextFilter, setFreeTextFilter] = useSearchParamState(sp, 'q', null, parseString);
const [watchListFilter, setWatchListFilter] = useSearchParamState(sp, 'watch', null, parseNullableBoolean);
const [jobNameFilter, setJobNameFilter] = useSearchParamState(sp, 'job', null, parseString);
const [activityFilter, setActivityFilter] = useSearchParamState(sp, 'active', null, parseNullableBoolean);
const [providerFilter, setProviderFilter] = useSearchParamState(sp, 'provider', null, parseString);
const [deleteModalVisible, setDeleteModalVisible] = useState(false);
const [listingToDelete, setListingToDelete] = useState(null);
const loadData = () => {
actions.listingsData.getListingsData({
page,
pageSize,
sortfield: sortField,
sortdir: sortDir,
freeTextFilter,
filter: { watchListFilter, jobNameFilter, activityFilter, providerFilter },
});
};
useEffect(() => {
loadData();
}, [page, sortField, sortDir, freeTextFilter, providerFilter, activityFilter, jobNameFilter, watchListFilter]);
const handleFilterChange = useMemo(
() =>
debounce((value) => {
setFreeTextFilter(value || null);
setPage(1);
}, 500),
[],
);
useEffect(() => {
return () => {
// cleanup debounced handler to avoid memory leaks
handleFilterChange.cancel && handleFilterChange.cancel();
};
}, [handleFilterChange]);
const handleWatch = async (e, item) => {
e.preventDefault();
e.stopPropagation();
try {
await xhrPost('/api/listings/watch', { listingId: item.id });
Toast.success(item.isWatched === 1 ? 'Listing removed from Watchlist' : 'Listing added to Watchlist');
loadData();
} catch (e) {
console.error(e);
Toast.error('Failed to operate Watchlist');
}
};
const handlePageChange = (_page) => {
setPage(_page);
};
const confirmDeletion = async (hardDelete) => {
try {
await xhrDelete('/api/listings/', { ids: [listingToDelete], hardDelete });
Toast.success('Listing successfully removed');
loadData();
} catch (error) {
Toast.error(error.message || 'Error deleting listing');
} finally {
setDeleteModalVisible(false);
setListingToDelete(null);
}
};
return (
<div className="listingsGrid">
<div className="listingsGrid__topbar">
<Input
className="listingsGrid__topbar__search"
prefix={<IconSearch />}
showClear
placeholder="Search"
defaultValue={freeTextFilter ?? ''}
onChange={handleFilterChange}
/>
<RadioGroup
type="button"
buttonSize="middle"
value={activityFilter === null ? 'all' : String(activityFilter)}
onChange={(e) => {
const v = e.target.value;
setActivityFilter(v === 'all' ? null : v === 'true');
setPage(1);
}}
>
<Radio value="all">All</Radio>
<Radio value="true">Active</Radio>
<Radio value="false">Inactive</Radio>
</RadioGroup>
<RadioGroup
type="button"
buttonSize="middle"
value={watchListFilter === null ? 'all' : String(watchListFilter)}
onChange={(e) => {
const v = e.target.value;
setWatchListFilter(v === 'all' ? null : v === 'true');
setPage(1);
}}
>
<Radio value="all">All</Radio>
<Radio value="true">Watched</Radio>
<Radio value="false">Unwatched</Radio>
</RadioGroup>
<Select
placeholder="Provider"
showClear
onChange={(val) => {
setProviderFilter(val);
setPage(1);
}}
value={providerFilter}
style={{ width: 130 }}
>
{providers?.map((p) => (
<Select.Option key={p.id} value={p.id}>
{p.name}
</Select.Option>
))}
</Select>
<Select
placeholder="Job"
showClear
onChange={(val) => {
setJobNameFilter(val);
setPage(1);
}}
value={jobNameFilter}
style={{ width: 130 }}
>
{jobs?.map((j) => (
<Select.Option key={j.id} value={j.id}>
{j.name}
</Select.Option>
))}
</Select>
<Select prefix="Sort by" style={{ width: 185 }} value={sortField} onChange={(val) => setSortField(val)}>
<Select.Option value="job_name">Job Name</Select.Option>
<Select.Option value="created_at">Listing Date</Select.Option>
<Select.Option value="price">Price</Select.Option>
<Select.Option value="provider">Provider</Select.Option>
</Select>
<Button
icon={sortDir === 'asc' ? <IconArrowUp /> : <IconArrowDown />}
onClick={() => setSortDir((d) => (d === 'asc' ? 'desc' : 'asc'))}
title={sortDir === 'asc' ? 'Ascending' : 'Descending'}
/>
</div>
{(listingsData?.result || []).length === 0 && (
<Empty
image={<IllustrationNoResult />}
darkModeImage={<IllustrationNoResultDark />}
description="No listings available yet..."
/>
)}
<div className="listingsGrid__grid">
{(listingsData?.result || []).map((item) => (
<div
key={item.id}
className="listingsGrid__card"
style={{ cursor: 'pointer' }}
role="button"
tabIndex={0}
onClick={() => navigate(`/listings/listing/${item.id}`)}
onKeyDown={(e) => {
if (e.key === 'Enter' || e.key === ' ') navigate(`/listings/listing/${item.id}`);
}}
>
<div className="listingsGrid__card__image-wrapper">
<img
src={item.image_url || no_image}
alt={item.title}
onError={(e) => {
e.target.src = no_image;
}}
/>
{!item.is_active && (
<div className="listingsGrid__card__inactive-watermark">
<span>Inactive</span>
</div>
)}
<button
type="button"
className="listingsGrid__card__star"
onClick={(e) => handleWatch(e, item)}
aria-label={item.isWatched === 1 ? 'Remove from watchlist' : 'Add to watchlist'}
>
{item.isWatched === 1 ? <IconStar /> : <IconStarStroked />}
</button>
</div>
<div className="listingsGrid__card__body">
<div className="listingsGrid__card__title" title={item.title}>
{item.title}
</div>
{item.price && (
<div className="listingsGrid__card__price">
<IconCart size="small" />
{item.price}
</div>
)}
{item.address && (
<div className="listingsGrid__card__meta">
<IconMapPin />
{item.address}
</div>
)}
<div className="listingsGrid__card__meta">
<IconBriefcase />
{item.provider}
</div>
<div className="listingsGrid__card__provider">{timeService.format(item.created_at, false)}</div>
</div>
<div className="listingsGrid__card__actions" onClick={(e) => e.stopPropagation()}>
<Tooltip content="Original Listing">
<Button
size="small"
icon={<IconLink />}
style={{ color: '#60a5fa' }}
theme="borderless"
onClick={(e) => {
e.stopPropagation();
window.open(item.link, '_blank');
}}
/>
</Tooltip>
<Tooltip content="View in Fredy">
<Button
size="small"
icon={<IconEyeOpened />}
style={{ color: '#34d399' }}
theme="borderless"
onClick={(e) => {
e.stopPropagation();
navigate(`/listings/listing/${item.id}`);
}}
/>
</Tooltip>
<Tooltip content="Remove">
<Button
size="small"
icon={<IconDelete />}
style={{ color: '#fb7185' }}
theme="borderless"
onClick={(e) => {
e.stopPropagation();
setListingToDelete(item.id);
setDeleteModalVisible(true);
}}
/>
</Tooltip>
</div>
</div>
))}
</div>
{(listingsData?.result || []).length > 0 && (
<div className="listingsGrid__pagination">
<Pagination
currentPage={page}
pageSize={pageSize}
total={listingsData?.totalNumber || 0}
onPageChange={handlePageChange}
showSizeChanger={false}
/>
</div>
)}
<ListingDeletionModal
visible={deleteModalVisible}
onConfirm={confirmDeletion}
onCancel={() => {
setDeleteModalVisible(false);
setListingToDelete(null);
/**
* @param {{ listings: object[], onWatch: Function, onNavigate: Function, onDelete: Function }} props
*/
const ListingsGrid = ({ listings, onWatch, onNavigate, onDelete }) => (
<div className="listingsGrid__grid">
{listings.map((item) => (
<div
key={item.id}
className="listingsGrid__card"
style={{ cursor: 'pointer' }}
role="button"
tabIndex={0}
onClick={() => onNavigate(item.id)}
onKeyDown={(e) => {
if (e.key === 'Enter' || e.key === ' ') onNavigate(item.id);
}}
/>
</div>
);
};
>
<div className="listingsGrid__card__image-wrapper">
<img
src={item.image_url || no_image}
alt={item.title}
onError={(e) => {
e.target.src = no_image;
}}
/>
{!item.is_active && (
<div className="listingsGrid__card__inactive-watermark">
<span>Inactive</span>
</div>
)}
<button
type="button"
className="listingsGrid__card__star"
onClick={(e) => onWatch(e, item)}
aria-label={item.isWatched === 1 ? 'Remove from watchlist' : 'Add to watchlist'}
>
{item.isWatched === 1 ? <IconStar /> : <IconStarStroked />}
</button>
</div>
<div className="listingsGrid__card__body">
<div className="listingsGrid__card__title" title={item.title}>
{item.title}
</div>
{item.price && (
<div className="listingsGrid__card__price">
<IconCart size="small" />
{item.price}
</div>
)}
{item.address && (
<div className="listingsGrid__card__meta">
<IconMapPin />
{item.address}
</div>
)}
<div className="listingsGrid__card__meta">
<IconBriefcase />
{item.provider}
</div>
<div className="listingsGrid__card__provider">{timeService.format(item.created_at, false)}</div>
</div>
<div className="listingsGrid__card__actions" onClick={(e) => e.stopPropagation()}>
<Tooltip content="Original Listing">
<Button
size="small"
icon={<IconLink />}
style={{ color: '#60a5fa' }}
theme="borderless"
onClick={(e) => {
e.stopPropagation();
window.open(item.link, '_blank');
}}
/>
</Tooltip>
<Tooltip content="View in Fredy">
<Button
size="small"
icon={<IconEyeOpened />}
style={{ color: '#34d399' }}
theme="borderless"
onClick={(e) => {
e.stopPropagation();
onNavigate(item.id);
}}
/>
</Tooltip>
<Tooltip content="Remove">
<Button
size="small"
icon={<IconDelete />}
style={{ color: '#fb7185' }}
theme="borderless"
onClick={(e) => {
e.stopPropagation();
onDelete(item.id);
}}
/>
</Tooltip>
</div>
</div>
))}
</div>
);
export default ListingsGrid;

View File

@@ -1,181 +1,143 @@
@import '../../../tokens.less';
.listingsGrid {
&__topbar {
.listingsGrid__grid {
display: grid;
grid-template-columns: repeat(auto-fill, minmax(240px, 1fr));
gap: 12px;
}
.listingsGrid__card {
background: @color-elevated !important;
border: 1px solid @color-border !important;
border-radius: @radius-card !important;
overflow: hidden;
transition: transform @transition-card, box-shadow @transition-card;
display: flex;
flex-direction: column;
&:hover {
transform: translateY(-2px);
box-shadow: 0 6px 24px -4px rgba(0,0,0,0.6);
}
&__image-wrapper {
position: relative;
height: 160px;
overflow: hidden;
flex-shrink: 0;
img {
width: 100%;
height: 100%;
object-fit: cover;
display: block;
}
}
&__inactive-watermark {
position: absolute;
inset: 0;
display: flex;
align-items: center;
gap: @space-3;
margin-bottom: @space-4;
flex-wrap: wrap;
justify-content: center;
background: rgba(0,0,0,0.35);
&__search {
min-width: 200px;
flex: 1;
}
@media (max-width: 768px) {
.listingsGrid__topbar__search {
width: 100%;
flex: unset;
}
.semi-radio-group {
flex: 1;
}
.semi-select {
flex: 1;
min-width: 100px;
width: auto !important;
}
span {
font-size: 18px;
font-weight: 800;
color: rgba(251,113,133,0.9);
text-transform: uppercase;
letter-spacing: 0.15em;
transform: rotate(-30deg);
border: 2px solid rgba(251,113,133,0.5);
padding: 4px 12px;
border-radius: @radius-chip;
backdrop-filter: blur(2px);
}
}
&__grid {
display: grid;
grid-template-columns: repeat(auto-fill, minmax(240px, 1fr));
gap: 12px;
}
&__card {
background: @color-elevated !important;
border: 1px solid @color-border !important;
border-radius: @radius-card !important;
overflow: hidden;
transition: transform @transition-card, box-shadow @transition-card;
&__star {
position: absolute;
top: 8px;
right: 8px;
background: rgba(0,0,0,0.5);
border: none;
border-radius: 50%;
width: 28px;
height: 28px;
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
cursor: pointer;
transition: background @transition-fast;
padding: 0;
&:hover {
transform: translateY(-2px);
box-shadow: 0 6px 24px -4px rgba(0,0,0,0.6);
background: rgba(0,0,0,0.75);
}
&__image-wrapper {
position: relative;
height: 160px;
overflow: hidden;
flex-shrink: 0;
img {
width: 100%;
height: 100%;
object-fit: cover;
display: block;
}
svg {
color: @color-accent;
font-size: 14px;
}
}
&__inactive-watermark {
position: absolute;
inset: 0;
display: flex;
align-items: center;
justify-content: center;
background: rgba(0,0,0,0.35);
&__body {
padding: 12px;
display: flex;
flex-direction: column;
gap: 4px;
flex: 1;
}
span {
font-size: 18px;
font-weight: 800;
color: rgba(251,113,133,0.9);
text-transform: uppercase;
letter-spacing: 0.15em;
transform: rotate(-30deg);
border: 2px solid rgba(251,113,133,0.5);
padding: 4px 12px;
border-radius: @radius-chip;
backdrop-filter: blur(2px);
}
}
&__title {
font-weight: 700;
font-size: @text-sm;
color: @color-text;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
&__star {
position: absolute;
top: 8px;
right: 8px;
background: rgba(0,0,0,0.5);
border: none;
border-radius: 50%;
width: 28px;
height: 28px;
display: flex;
align-items: center;
justify-content: center;
cursor: pointer;
transition: background @transition-fast;
padding: 0;
&__price {
font-size: @text-base;
font-weight: 600;
color: @color-success;
display: flex;
align-items: center;
gap: 4px;
}
&:hover {
background: rgba(0,0,0,0.75);
}
&__meta {
font-size: @text-xs;
color: @color-muted;
display: flex;
align-items: center;
gap: 4px;
svg {
color: @color-accent;
font-size: 14px;
}
}
&__body {
padding: 12px;
display: flex;
flex-direction: column;
gap: 4px;
flex: 1;
}
&__title {
font-weight: 700;
font-size: @text-sm;
color: @color-text;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
&__price {
font-size: @text-base;
font-weight: 600;
color: @color-success;
display: flex;
align-items: center;
gap: 4px;
}
&__meta {
font-size: @text-xs;
color: @color-muted;
display: flex;
align-items: center;
gap: 4px;
.semi-icon {
font-size: 11px;
color: @color-faint;
}
}
&__provider {
font-size: @text-xs;
.semi-icon {
font-size: 11px;
color: @color-faint;
}
}
&__actions {
display: flex;
justify-content: space-around;
padding: 8px 12px;
border-top: 1px solid @color-border;
gap: 4px;
margin-top: auto;
&__provider {
font-size: @text-xs;
color: @color-faint;
}
button {
flex: 1;
border: none !important;
border-radius: @radius-chip !important;
}
&__actions {
display: flex;
justify-content: space-around;
padding: 8px 12px;
border-top: 1px solid @color-border;
gap: 4px;
margin-top: auto;
button {
flex: 1;
border: none !important;
border-radius: @radius-chip !important;
}
}
&__pagination {
margin-top: @space-4;
display: flex;
justify-content: center;
}
}

View File

@@ -0,0 +1,264 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import { useState, useEffect, useMemo } from 'react';
import {
useSearchParamState,
parseNumber,
parseString,
parseNullableBoolean,
} from '../../hooks/useSearchParamState.js';
import { Button, Pagination, Toast, Input, Select, Empty, Radio, RadioGroup, Tooltip } from '@douyinfe/semi-ui-19';
import { IconSearch, IconArrowUp, IconArrowDown, IconGridView, IconList } from '@douyinfe/semi-icons';
import { useNavigate, useSearchParams } from 'react-router-dom';
import ListingDeletionModal from '../ListingDeletionModal.jsx';
import { xhrDelete, xhrPost } from '../../services/xhr.js';
import { useActions, useSelector } from '../../services/state/store.js';
import { debounce } from '../../utils';
import ListingsGrid from '../grid/listings/ListingsGrid.jsx';
import ListingsTable from '../table/ListingsTable.jsx';
import { IllustrationNoResult, IllustrationNoResultDark } from '@douyinfe/semi-illustrations';
import './ListingsOverview.less';
const ListingsOverview = () => {
const listingsData = useSelector((state) => state.listingsData);
const providers = useSelector((state) => state.provider);
const jobs = useSelector((state) => state.jobsData.jobs);
const userSettings = useSelector((state) => state.userSettings.settings);
const actions = useActions();
const navigate = useNavigate();
const sp = useSearchParams();
const viewMode = userSettings?.listings_view_mode ?? 'grid';
const [page, setPage] = useSearchParamState(sp, 'page', 1, parseNumber);
const pageSize = 40;
const [sortField, setSortField] = useSearchParamState(sp, 'sort', 'created_at', parseString);
const [sortDir, setSortDir] = useSearchParamState(sp, 'dir', 'desc', parseString);
const [freeTextFilter, setFreeTextFilter] = useSearchParamState(sp, 'q', null, parseString);
const [watchListFilter, setWatchListFilter] = useSearchParamState(sp, 'watch', null, parseNullableBoolean);
const [jobNameFilter, setJobNameFilter] = useSearchParamState(sp, 'job', null, parseString);
const [activityFilter, setActivityFilter] = useSearchParamState(sp, 'active', null, parseNullableBoolean);
const [providerFilter, setProviderFilter] = useSearchParamState(sp, 'provider', null, parseString);
const [deleteModalVisible, setDeleteModalVisible] = useState(false);
const [listingToDelete, setListingToDelete] = useState(null);
const loadData = () => {
actions.listingsData.getListingsData({
page,
pageSize,
sortfield: sortField,
sortdir: sortDir,
freeTextFilter,
filter: { watchListFilter, jobNameFilter, activityFilter, providerFilter },
});
};
useEffect(() => {
loadData();
}, [page, sortField, sortDir, freeTextFilter, providerFilter, activityFilter, jobNameFilter, watchListFilter]);
const handleFilterChange = useMemo(
() =>
debounce((value) => {
setFreeTextFilter(value || null);
setPage(1);
}, 500),
[],
);
useEffect(() => {
return () => {
handleFilterChange.cancel && handleFilterChange.cancel();
};
}, [handleFilterChange]);
const handleWatch = async (e, item) => {
e.preventDefault();
e.stopPropagation();
try {
await xhrPost('/api/listings/watch', { listingId: item.id });
Toast.success(item.isWatched === 1 ? 'Listing removed from Watchlist' : 'Listing added to Watchlist');
loadData();
} catch (e) {
console.error(e);
Toast.error('Failed to operate Watchlist');
}
};
const handleDelete = (id) => {
setListingToDelete(id);
setDeleteModalVisible(true);
};
const handleNavigate = (id) => navigate(`/listings/listing/${id}`);
const confirmDeletion = async (hardDelete) => {
try {
await xhrDelete('/api/listings/', { ids: [listingToDelete], hardDelete });
Toast.success('Listing successfully removed');
loadData();
} catch (error) {
Toast.error(error.message || 'Error deleting listing');
} finally {
setDeleteModalVisible(false);
setListingToDelete(null);
}
};
const listings = listingsData?.result || [];
return (
<div className="listingsOverview">
<div className="listingsOverview__topbar">
<Input
className="listingsOverview__topbar__search"
prefix={<IconSearch />}
showClear
placeholder="Search"
defaultValue={freeTextFilter ?? ''}
onChange={handleFilterChange}
/>
<RadioGroup
type="button"
buttonSize="middle"
value={activityFilter === null ? 'all' : String(activityFilter)}
onChange={(e) => {
const v = e.target.value;
setActivityFilter(v === 'all' ? null : v === 'true');
setPage(1);
}}
>
<Radio value="all">All</Radio>
<Radio value="true">Active</Radio>
<Radio value="false">Inactive</Radio>
</RadioGroup>
<RadioGroup
type="button"
buttonSize="middle"
value={watchListFilter === null ? 'all' : String(watchListFilter)}
onChange={(e) => {
const v = e.target.value;
setWatchListFilter(v === 'all' ? null : v === 'true');
setPage(1);
}}
>
<Radio value="all">All</Radio>
<Radio value="true">Watched</Radio>
<Radio value="false">Unwatched</Radio>
</RadioGroup>
<Select
placeholder="Provider"
showClear
onChange={(val) => {
setProviderFilter(val);
setPage(1);
}}
value={providerFilter}
style={{ width: 130 }}
>
{providers?.map((p) => (
<Select.Option key={p.id} value={p.id}>
{p.name}
</Select.Option>
))}
</Select>
<Select
placeholder="Job"
showClear
onChange={(val) => {
setJobNameFilter(val);
setPage(1);
}}
value={jobNameFilter}
style={{ width: 130 }}
>
{jobs?.map((j) => (
<Select.Option key={j.id} value={j.id}>
{j.name}
</Select.Option>
))}
</Select>
<Select prefix="Sort by" style={{ width: 185 }} value={sortField} onChange={(val) => setSortField(val)}>
<Select.Option value="job_name">Job Name</Select.Option>
<Select.Option value="created_at">Listing Date</Select.Option>
<Select.Option value="price">Price</Select.Option>
<Select.Option value="provider">Provider</Select.Option>
</Select>
<Button
icon={sortDir === 'asc' ? <IconArrowUp /> : <IconArrowDown />}
onClick={() => setSortDir(sortDir === 'asc' ? 'desc' : 'asc')}
title={sortDir === 'asc' ? 'Ascending' : 'Descending'}
/>
<div className="listingsOverview__topbar__view-toggle">
<Tooltip content="Grid view">
<Button
icon={<IconGridView />}
theme={viewMode === 'grid' ? 'solid' : 'borderless'}
onClick={() => actions.userSettings.setListingsViewMode('grid')}
aria-label="Grid view"
aria-pressed={viewMode === 'grid'}
/>
</Tooltip>
<Tooltip content="Table view">
<Button
icon={<IconList />}
theme={viewMode === 'table' ? 'solid' : 'borderless'}
onClick={() => actions.userSettings.setListingsViewMode('table')}
aria-label="Table view"
aria-pressed={viewMode === 'table'}
/>
</Tooltip>
</div>
</div>
{listings.length === 0 && (
<Empty
image={<IllustrationNoResult />}
darkModeImage={<IllustrationNoResultDark />}
description="No listings available yet..."
/>
)}
{viewMode === 'grid' ? (
<ListingsGrid listings={listings} onWatch={handleWatch} onNavigate={handleNavigate} onDelete={handleDelete} />
) : (
<ListingsTable listings={listings} onWatch={handleWatch} onNavigate={handleNavigate} onDelete={handleDelete} />
)}
{listings.length > 0 && (
<div className="listingsOverview__pagination">
<Pagination
currentPage={page}
pageSize={pageSize}
total={listingsData?.totalNumber || 0}
onPageChange={setPage}
showSizeChanger={false}
/>
</div>
)}
<ListingDeletionModal
visible={deleteModalVisible}
onConfirm={confirmDeletion}
onCancel={() => {
setDeleteModalVisible(false);
setListingToDelete(null);
}}
/>
</div>
);
};
export default ListingsOverview;

View File

@@ -0,0 +1,45 @@
@import '../../tokens.less';
.listingsOverview {
&__topbar {
display: flex;
align-items: center;
gap: @space-3;
margin-bottom: @space-4;
flex-wrap: wrap;
&__search {
min-width: 200px;
flex: 1;
}
&__view-toggle {
display: flex;
gap: 2px;
flex-shrink: 0;
}
@media (max-width: 768px) {
.listingsOverview__topbar__search {
width: 100%;
flex: unset;
}
.semi-radio-group {
flex: 1;
}
.semi-select {
flex: 1;
min-width: 100px;
width: auto !important;
}
}
}
&__pagination {
margin-top: @space-4;
display: flex;
justify-content: center;
}
}

View File

@@ -0,0 +1,128 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import { Button, Tag, Tooltip, Switch } from '@douyinfe/semi-ui-19';
import {
IconAlertTriangle,
IconBell,
IconBriefcase,
IconCopy,
IconDelete,
IconDescend2,
IconEdit,
IconHome,
IconPlayCircle,
} from '@douyinfe/semi-icons';
import './JobsTable.less';
/**
* @param {{ jobs: object[], onRun: Function, onEdit: Function, onClone: Function, onDeleteListings: Function, onDeleteJob: Function, onStatusChange: Function }} props
*/
const JobsTable = ({ jobs, onRun, onEdit, onClone, onDeleteListings, onDeleteJob, onStatusChange }) => (
<div className="jobsTable">
{jobs.map((job) => (
<div key={job.id} className={`jobsTable__row${!job.enabled ? ' jobsTable__row--inactive' : ''}`}>
<div className="jobsTable__row__dot">
<span
className={`jobsTable__row__dot__indicator${job.enabled ? ' jobsTable__row__dot__indicator--active' : ''}`}
/>
</div>
<div className="jobsTable__row__name" title={job.name}>
{job.name}
</div>
<div className="jobsTable__row__stat jobsTable__row__stat--blue">
<IconHome size="small" />
{job.numberOfFoundListings || 0}
</div>
<div className="jobsTable__row__stat jobsTable__row__stat--orange">
<IconBriefcase size="small" />
{job.provider?.length || 0}
</div>
<div className="jobsTable__row__stat jobsTable__row__stat--purple">
<IconBell size="small" />
{job.notificationAdapter?.length || 0}
</div>
<div className="jobsTable__row__badges">
<Switch
size="small"
checked={job.enabled}
disabled={job.isOnlyShared}
onChange={(checked) => onStatusChange(job.id, checked)}
/>
{job.running && (
<Tag color="green" variant="light" size="small">
RUNNING
</Tag>
)}
{job.isOnlyShared && (
<Tooltip content="Shared with you — read only">
<span style={{ display: 'flex', alignItems: 'center' }}>
<IconAlertTriangle style={{ color: 'rgba(var(--semi-yellow-7), 1)' }} />
</span>
</Tooltip>
)}
</div>
<div className="jobsTable__row__actions">
<Tooltip content="Run Job">
<Button
type="primary"
style={{ background: '#21aa21b5' }}
size="small"
theme="solid"
icon={<IconPlayCircle />}
disabled={job.isOnlyShared || job.running}
onClick={() => onRun(job.id)}
/>
</Tooltip>
<Tooltip content="Edit Job">
<Button
type="secondary"
size="small"
icon={<IconEdit />}
disabled={job.isOnlyShared}
onClick={() => onEdit(job.id)}
/>
</Tooltip>
<Tooltip content="Clone Job">
<Button
type="tertiary"
size="small"
icon={<IconCopy />}
disabled={job.isOnlyShared}
onClick={() => onClone(job.id)}
/>
</Tooltip>
<Tooltip content="Delete all found Listings">
<Button
type="danger"
size="small"
icon={<IconDescend2 />}
disabled={job.isOnlyShared}
onClick={() => onDeleteListings(job.id)}
/>
</Tooltip>
<Tooltip content="Delete Job">
<Button
type="danger"
size="small"
icon={<IconDelete />}
disabled={job.isOnlyShared}
onClick={() => onDeleteJob(job.id)}
/>
</Tooltip>
</div>
</div>
))}
</div>
);
export default JobsTable;

View File

@@ -0,0 +1,105 @@
@import '../../tokens.less';
.jobsTable {
display: flex;
flex-direction: column;
gap: 4px;
&__row {
display: grid;
grid-template-columns: 24px 1fr 80px 80px 80px auto auto;
align-items: center;
gap: @space-3;
padding: 8px 12px;
background: @color-elevated;
border: 1px solid @color-border;
border-radius: @radius-chip;
transition: background @transition-fast;
&:hover {
background: #252525;
}
&--inactive {
opacity: 0.6;
}
&__dot {
display: flex;
align-items: center;
justify-content: center;
&__indicator {
width: 8px;
height: 8px;
border-radius: 50%;
flex-shrink: 0;
background-color: rgba(251, 113, 133, 0.7);
&--active {
background-color: rgba(52, 211, 153, 0.8);
}
}
}
&__name {
font-weight: 600;
font-size: @text-sm;
color: @color-text;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
&__stat {
font-size: @text-sm;
font-weight: 600;
display: flex;
align-items: center;
gap: 4px;
white-space: nowrap;
&--blue {
color: @color-blue-text;
}
&--orange {
color: @color-orange-text;
}
&--purple {
color: @color-purple-text;
}
}
&__badges {
display: flex;
align-items: center;
gap: 4px;
min-width: 0;
}
&__actions {
display: flex;
align-items: center;
gap: 2px;
}
@media (max-width: 900px) {
grid-template-columns: 24px 1fr 80px auto auto;
.jobsTable__row__stat--orange,
.jobsTable__row__stat--purple {
display: none;
}
}
@media (max-width: 560px) {
grid-template-columns: 24px 1fr auto auto;
.jobsTable__row__stat--blue {
display: none;
}
}
}
}

View File

@@ -0,0 +1,132 @@
/*
* Copyright (c) 2026 by Christian Kellner.
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import { Button, Tooltip } from '@douyinfe/semi-ui-19';
import {
IconBriefcase,
IconCart,
IconDelete,
IconLink,
IconMapPin,
IconStar,
IconStarStroked,
IconEyeOpened,
} from '@douyinfe/semi-icons';
import no_image from '../../assets/no_image.png';
import * as timeService from '../../services/time/timeService.js';
import './ListingsTable.less';
/**
* @param {{ listings: object[], onWatch: Function, onNavigate: Function, onDelete: Function }} props
*/
const ListingsTable = ({ listings, onWatch, onNavigate, onDelete }) => (
<div className="listingsTable">
{listings.map((item) => (
<div
key={item.id}
className={`listingsTable__row${!item.is_active ? ' listingsTable__row--inactive' : ''}`}
role="button"
tabIndex={0}
onClick={() => onNavigate(item.id)}
onKeyDown={(e) => {
if (e.key === 'Enter' || e.key === ' ') onNavigate(item.id);
}}
>
<div className="listingsTable__row__thumb">
<img
src={item.image_url || no_image}
alt={item.title}
onError={(e) => {
e.target.src = no_image;
}}
/>
</div>
<div className="listingsTable__row__title" title={item.title}>
{item.title}
</div>
<div className="listingsTable__row__price">
{item.price ? (
<>
<IconCart size="small" />
{item.price}
</>
) : (
<span className="listingsTable__row__empty"></span>
)}
</div>
<div className="listingsTable__row__address">
{item.address ? (
<>
<IconMapPin size="small" />
{item.address}
</>
) : (
<span className="listingsTable__row__empty"></span>
)}
</div>
<div className="listingsTable__row__meta">
<IconBriefcase size="small" />
{item.provider}
</div>
<div className="listingsTable__row__date">{timeService.format(item.created_at, false)}</div>
<div className="listingsTable__row__actions" onClick={(e) => e.stopPropagation()}>
<button
type="button"
className="listingsTable__row__star"
onClick={(e) => onWatch(e, item)}
aria-label={item.isWatched === 1 ? 'Remove from watchlist' : 'Add to watchlist'}
>
{item.isWatched === 1 ? <IconStar /> : <IconStarStroked />}
</button>
<Tooltip content="Original Listing">
<Button
size="small"
icon={<IconLink />}
style={{ color: '#60a5fa' }}
theme="borderless"
onClick={(e) => {
e.stopPropagation();
window.open(item.link, '_blank');
}}
/>
</Tooltip>
<Tooltip content="View in Fredy">
<Button
size="small"
icon={<IconEyeOpened />}
style={{ color: '#34d399' }}
theme="borderless"
onClick={(e) => {
e.stopPropagation();
onNavigate(item.id);
}}
/>
</Tooltip>
<Tooltip content="Remove">
<Button
size="small"
icon={<IconDelete />}
style={{ color: '#fb7185' }}
theme="borderless"
onClick={(e) => {
e.stopPropagation();
onDelete(item.id);
}}
/>
</Tooltip>
</div>
</div>
))}
</div>
);
export default ListingsTable;

View File

@@ -0,0 +1,142 @@
@import '../../tokens.less';
.listingsTable {
display: flex;
flex-direction: column;
gap: 4px;
&__row {
display: grid;
grid-template-columns: 56px 1fr 140px 200px 120px 110px auto;
align-items: center;
gap: @space-3;
padding: 8px 12px;
background: @color-elevated;
border: 1px solid @color-border;
border-radius: @radius-chip;
cursor: pointer;
transition: background @transition-fast;
&:hover {
background: #252525;
}
&--inactive {
opacity: 0.6;
}
&__thumb {
width: 56px;
height: 40px;
flex-shrink: 0;
border-radius: @radius-chip;
overflow: hidden;
img {
width: 100%;
height: 100%;
object-fit: cover;
display: block;
}
}
&__title {
font-weight: 600;
font-size: @text-sm;
color: @color-text;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
&__price {
font-size: @text-sm;
font-weight: 600;
color: @color-success;
display: flex;
align-items: center;
gap: 4px;
white-space: nowrap;
}
&__address {
font-size: @text-xs;
color: @color-muted;
display: flex;
align-items: center;
gap: 4px;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
&__meta {
font-size: @text-xs;
color: @color-muted;
display: flex;
align-items: center;
gap: 4px;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
&__date {
font-size: @text-xs;
color: @color-faint;
white-space: nowrap;
}
&__actions {
display: flex;
align-items: center;
gap: 2px;
}
&__star {
width: 28px;
height: 28px;
background: transparent;
border: none;
border-radius: 50%;
display: flex;
align-items: center;
justify-content: center;
cursor: pointer;
padding: 0;
transition: background @transition-fast;
flex-shrink: 0;
&:hover {
background: rgba(0, 0, 0, 0.1);
}
svg {
color: @color-accent;
font-size: 14px;
}
}
&__empty {
color: @color-faint;
}
@media (max-width: 900px) {
grid-template-columns: 56px 1fr 120px auto;
.listingsTable__row__address,
.listingsTable__row__meta,
.listingsTable__row__date {
display: none;
}
}
@media (max-width: 560px) {
grid-template-columns: 56px 1fr auto;
.listingsTable__row__price {
display: none;
}
}
}
}

View File

@@ -47,7 +47,7 @@ export default function UserTable({ user = [], onUserRemoval, onUserEdit } = {})
{
title: 'Last login',
dataIndex: 'lastLogin',
render: (value) => format(value),
render: (value) => (value == null ? '---' : format(value)),
},
{
title: 'Jobs',

View File

@@ -321,6 +321,34 @@ export const useFredyState = create(
throw Exception;
}
},
async setListingsViewMode(listings_view_mode) {
try {
await xhrPost('/api/user/settings/listings-view-mode', { listings_view_mode });
set((state) => ({
userSettings: {
...state.userSettings,
settings: { ...state.userSettings.settings, listings_view_mode },
},
}));
} catch (Exception) {
console.error('Error while trying to update listings view mode setting. Error:', Exception);
throw Exception;
}
},
async setJobsViewMode(jobs_view_mode) {
try {
await xhrPost('/api/user/settings/jobs-view-mode', { jobs_view_mode });
set((state) => ({
userSettings: {
...state.userSettings,
settings: { ...state.userSettings.settings, jobs_view_mode },
},
}));
} catch (Exception) {
console.error('Error while trying to update jobs view mode setting. Error:', Exception);
throw Exception;
}
},
},
};

View File

@@ -13,5 +13,3 @@ export function format(ts, showSeconds = true) {
...(showSeconds ? { second: 'numeric' } : {}),
}).format(ts);
}
export const roundToHour = (ts) => Math.ceil(ts / (1000 * 60 * 60)) * (1000 * 60 * 60);

View File

@@ -54,6 +54,7 @@ const GeneralSettings = function GeneralSettings() {
const [loading, setLoading] = React.useState(true);
const settings = useSelector((state) => state.generalSettings.settings);
const currentUser = useSelector((state) => state.user.currentUser);
const [interval, setInterval] = React.useState('');
const [port, setPort] = React.useState('');
@@ -467,12 +468,26 @@ const GeneralSettings = function GeneralSettings() {
itemKey="backup"
>
<div className="generalSettings__tab-content">
{demoMode && !currentUser?.isAdmin && (
<Banner
fullMode={false}
type="warning"
closeIcon={null}
style={{ marginBottom: '12px' }}
description="Backup and restore are not available in demo mode."
/>
)}
<SegmentPart
name="Backup & Restore"
helpText="Download a zipped backup of your database or restore from a backup zip."
>
<div style={{ display: 'flex', gap: '0.5rem', alignItems: 'center', flexWrap: 'wrap' }}>
<Button theme="solid" icon={<IconSave />} onClick={handleDownloadBackup}>
<Button
theme="solid"
icon={<IconSave />}
onClick={handleDownloadBackup}
disabled={demoMode && !currentUser?.isAdmin}
>
Download Backup
</Button>
<input
@@ -482,7 +497,12 @@ const GeneralSettings = function GeneralSettings() {
style={{ display: 'none' }}
onChange={handleSelectRestoreFile}
/>
<Button onClick={handleOpenFilePicker} theme="light" icon={<IconFolder />}>
<Button
onClick={handleOpenFilePicker}
theme="light"
icon={<IconFolder />}
disabled={demoMode && !currentUser?.isAdmin}
>
Restore from Zip
</Button>
</div>

View File

@@ -141,21 +141,6 @@ export default function ProviderMutator({
</p>
</>
)}
<Banner
fullMode={false}
type="warning"
closeIcon={null}
title={<div style={{ fontWeight: 600, fontSize: '14px', lineHeight: '20px' }}>Warning</div>}
style={{ marginBottom: '1rem' }}
description={
<div>
<p>
Currently, our Immoscout implementation does not support drawing shapes on a map. Use a radius instead.
</p>
</div>
}
/>
<Select
filter
placeholder="Select a provider"

View File

@@ -31,6 +31,7 @@ import {
IconLink,
IconStar,
IconStarStroked,
IconDelete,
IconExpand,
IconGridView,
} from '@douyinfe/semi-icons';
@@ -39,7 +40,8 @@ import 'maplibre-gl/dist/maplibre-gl.css';
import no_image from '../../assets/no_image.png';
import * as timeService from '../../services/time/timeService.js';
import { distanceMeters, getBoundsFromCoords } from './mapUtils.js';
import { xhrPost } from '../../services/xhr.js';
import { xhrPost, xhrDelete } from '../../services/xhr.js';
import ListingDeletionModal from '../../components/ListingDeletionModal.jsx';
import Headline from '../../components/headline/Headline.jsx';
import './ListingDetail.less';
@@ -59,6 +61,7 @@ export default function ListingDetail() {
const mapContainer = useRef(null);
const map = useRef(null);
const [loading, setLoading] = useState(true);
const [deleteModalVisible, setDeleteModalVisible] = useState(false);
useEffect(() => {
async function fetchListing() {
@@ -239,6 +242,18 @@ export default function ListingDetail() {
};
}, [listing, loading, homeAddress]);
const confirmDeletion = async (hardDelete) => {
try {
await xhrDelete('/api/listings/', { ids: [listing.id], hardDelete });
Toast.success('Listing successfully removed');
navigate('/listings');
} catch (e) {
Toast.error(e.message || 'Error deleting listing');
} finally {
setDeleteModalVisible(false);
}
};
const handleWatch = async () => {
try {
await xhrPost('/api/listings/watch', { listingId: listing.id });
@@ -330,21 +345,25 @@ export default function ListingDetail() {
<IconLink style={{ marginRight: 6 }} />
Open listing
</a>
<Button
icon={<IconDelete />}
onClick={() => setDeleteModalVisible(true)}
theme="light"
type="danger"
>
Delete
</Button>
</Space>
</div>
<Row>
<Col span={24} lg={12}>
<div className="listing-detail__image-container">
<div
className={`listing-detail__image-container${!listing.image_url ? ' listing-detail__image-container--placeholder' : ''}`}
>
<Image
src={listing.image_url ?? no_image}
fallback={
<img
src={no_image}
alt="No image available"
style={{ width: '100%', height: '100%', objectFit: 'cover' }}
/>
}
fallback={<img src={no_image} alt="No image available" />}
style={{ width: '100%', height: '100%' }}
preview={!!listing.image_url}
/>
@@ -401,6 +420,12 @@ export default function ListingDetail() {
<div ref={mapContainer} className="listing-detail__map-container" />
)}
</div>
<ListingDeletionModal
visible={deleteModalVisible}
onConfirm={confirmDeletion}
onCancel={() => setDeleteModalVisible(false)}
/>
</div>
);
}

View File

@@ -69,6 +69,13 @@
object-fit: cover !important;
display: block !important;
}
&--placeholder {
img,
.semi-image-img {
object-fit: contain !important;
}
}
}
&__address-link {

View File

@@ -3,14 +3,14 @@
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
*/
import ListingsGrid from '../../components/grid/listings/ListingsGrid.jsx';
import ListingsOverview from '../../components/listings/ListingsOverview.jsx';
import Headline from '../../components/headline/Headline.jsx';
export default function Listings() {
return (
<>
<Headline text="Listings" />
<ListingsGrid />
<ListingsOverview />
</>
);
}

View File

@@ -59,7 +59,7 @@ const UserMutator = function UserMutator() {
navigate('/users');
} catch (error) {
console.error(error);
Toast.error(error.json.message);
Toast.error(error.json.error);
}
};

View File

@@ -10,6 +10,7 @@ export default defineConfig({
globals: true,
environment: 'node',
include: ['test/**/*.test.js'],
globalSetup: ['./test/globalSetup.js'],
testTimeout: 60000,
reporters: ['verbose'],
},

1268
yarn.lock

File diff suppressed because it is too large Load Diff