mirror of
https://github.com/orangecoding/fredy.git
synced 2026-06-16 12:31:07 +00:00
Compare commits
25 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
33161de087 | ||
|
|
acab23207e | ||
|
|
2896d531e4 | ||
|
|
0cbfa25062 | ||
|
|
bcd3042026 | ||
|
|
0ce93acaf6 | ||
|
|
cabef973a2 | ||
|
|
3d0fa87d19 | ||
|
|
8b012ef2f1 | ||
|
|
6816b0aded | ||
|
|
ac02817d4e | ||
|
|
fe0a09fe1c | ||
|
|
2f00966f27 | ||
|
|
921057252d | ||
|
|
703c602527 | ||
|
|
0e29c9b9c6 | ||
|
|
f60c5859f9 | ||
|
|
ee54cc495b | ||
|
|
96582ecff4 | ||
|
|
3de82dfa41 | ||
|
|
d7ee4f6909 | ||
|
|
bf4bae9bf5 | ||
|
|
3d10dc6042 | ||
|
|
fef6d06a9d | ||
|
|
951b69a67f |
1
.gitattributes
vendored
Normal file
1
.gitattributes
vendored
Normal file
@@ -0,0 +1 @@
|
||||
test/testFixtures/** linguist-vendored
|
||||
26
CLAUDE.md
26
CLAUDE.md
@@ -8,7 +8,7 @@ Fredy is a self-hosted real estate finder for Germany. It scrapes German real es
|
||||
|
||||
- Node.js >= 22, ESM-only (`"type": "module"`)
|
||||
- Default port: 9998, default login: admin / admin
|
||||
- SQLite via `better-sqlite3` (synchronous — all DB ops are sync; only network I/O is async)
|
||||
- SQLite via `better-sqlite3` (synchronous - all DB ops are sync; only network I/O is async)
|
||||
|
||||
## Commands
|
||||
|
||||
@@ -66,13 +66,13 @@ scheduler (every N minutes) or manual trigger via POST /api/jobs/:id/run
|
||||
|
||||
### Plugin systems
|
||||
|
||||
**Providers** (`lib/provider/*.js`) — each module exports:
|
||||
- `metaInformation` — `{ id, name, baseUrl }`
|
||||
- `config` — `ProviderConfig` with `requiredFieldNames`, `crawlContainer`, `crawlFields`, `sortByDateParam`, `normalize()`, `filter()`, optional `getListings()`, `fetchDetails()`, `activeTester()`
|
||||
- `init(sourceConfig, blacklist)` — called before each job run; providers are **stateful modules** holding mutable `url` and `appliedBlackList` at module scope
|
||||
**Providers** (`lib/provider/*.js`) - each module exports:
|
||||
- `metaInformation` - `{ id, name, baseUrl }`
|
||||
- `config` - `ProviderConfig` with `requiredFieldNames`, `crawlContainer`, `crawlFields`, `sortByDateParam`, `normalize()`, `filter()`, optional `getListings()`, `fetchDetails()`, `activeTester()`
|
||||
- `init(sourceConfig, blacklist)` - called before each job run; providers are **stateful modules** holding mutable `url` and `appliedBlackList` at module scope
|
||||
|
||||
**Notification adapters** (`lib/notification/adapter/*.js`) — each exports:
|
||||
- `config` — `{ id, name, description, fields }` (drives the UI form)
|
||||
**Notification adapters** (`lib/notification/adapter/*.js`) - each exports:
|
||||
- `config` - `{ id, name, description, fields }` (drives the UI form)
|
||||
- `send({ serviceName, newListings, notificationConfig, jobKey, baseUrl })`
|
||||
- Loaded dynamically at startup via `fs.readdirSync`
|
||||
|
||||
@@ -98,18 +98,18 @@ scheduler (every N minutes) or manual trigger via POST /api/jobs/:id/run
|
||||
### MCP server
|
||||
|
||||
Two transports:
|
||||
1. **stdio** (`lib/mcp/stdio.js`) — for Claude Desktop/LM Studio; opens its own DB connection (main process need not be running)
|
||||
2. **HTTP** (`/api/mcp`) — authenticated via Bearer token (`mcp_token` column in `users` table)
|
||||
1. **stdio** (`lib/mcp/stdio.js`) - for Claude Desktop/LM Studio; opens its own DB connection (main process need not be running)
|
||||
2. **HTTP** (`/api/mcp`) - authenticated via Bearer token (`mcp_token` column in `users` table)
|
||||
|
||||
Tools: `list_jobs`, `get_job`, `list_listings`, `get_listing`, `get_current_date_time`. Responses are Markdown via `lib/mcp/mcpNormalizer.js`.
|
||||
|
||||
## Key Conventions
|
||||
|
||||
- **ESM only** — `import`/`export` everywhere, no CommonJS
|
||||
- **JSDoc typedefs** (no TypeScript) in `lib/types/` — `listing.js`, `job.js`, `filter.js`, `providerConfig.js`
|
||||
- **Copyright header** required on all `.js` files — enforced by `lint-staged` pre-commit hook via `copyright.js`
|
||||
- **ESM only** - `import`/`export` everywhere, no CommonJS
|
||||
- **JSDoc typedefs** (no TypeScript) in `lib/types/` - `listing.js`, `job.js`, `filter.js`, `providerConfig.js`
|
||||
- **Copyright header** required on all `.js` files - enforced by `lint-staged` pre-commit hook via `copyright.js`
|
||||
- **`NoNewListingsWarning`** (`lib/errors.js`) is used as control flow to short-circuit the pipeline (not an error)
|
||||
- **Test fixtures** in `test/testFixtures/` — HTML/JSON snapshots per provider; `TEST_MODE=offline` mocks `puppeteerExtractor` and global `fetch` via `test/offlineFixtures.js`
|
||||
- **Test fixtures** in `test/testFixtures/` - HTML/JSON snapshots per provider; `TEST_MODE=offline` mocks `puppeteerExtractor` and global `fetch` via `test/offlineFixtures.js`
|
||||
- **`conf/config.json`** is the only runtime config file; created with defaults if missing
|
||||
|
||||
## Coding
|
||||
|
||||
13
Dockerfile
13
Dockerfile
@@ -1,16 +1,15 @@
|
||||
FROM node:22-slim
|
||||
|
||||
ARG TARGETARCH
|
||||
|
||||
# System deps for Chrome for Testing + build tools for native modules (better-sqlite3)
|
||||
# On ARM64 we also install system Chromium (Chrome for Testing has no ARM64 binary)
|
||||
# System deps for CloakBrowser + build tools for native modules (better-sqlite3)
|
||||
# fonts-noto-color-emoji and fonts-freefont-ttf are required so canvas fingerprint
|
||||
# hashes match real browsers; missing emoji fonts cause bot detection on Kasada/Akamai.
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
curl ca-certificates fonts-liberation libasound2 \
|
||||
libatk-bridge2.0-0 libatk1.0-0 libcups2 libdbus-1-3 \
|
||||
libdrm2 libgbm1 libgtk-3-0 libnspr4 libnss3 \
|
||||
libx11-xcb1 libxcomposite1 libxdamage1 libxrandr2 xdg-utils \
|
||||
fonts-noto-color-emoji fonts-freefont-ttf \
|
||||
python3 make g++ \
|
||||
&& if [ "$TARGETARCH" = "arm64" ]; then apt-get install -y --no-install-recommends chromium; fi \
|
||||
&& rm -rf /var/lib/apt/lists/* \
|
||||
&& mkdir -p /db /conf /fredy
|
||||
|
||||
@@ -26,8 +25,8 @@ RUN yarn config set network-timeout 600000 \
|
||||
&& yarn --frozen-lockfile \
|
||||
&& yarn cache clean
|
||||
|
||||
# on arm64 use the system Chromium installed above
|
||||
RUN if [ "$TARGETARCH" != "arm64" ]; then npx puppeteer browsers install chrome; fi
|
||||
# Pre-download the CloakBrowser stealth Chromium binary (supports x86_64 and arm64)
|
||||
RUN node -e "import('cloakbrowser').then(({ensureBinary}) => ensureBinary())"
|
||||
|
||||
# Purge build tools now that native modules are compiled
|
||||
RUN apt-get purge -y python3 make g++ \
|
||||
|
||||
@@ -43,13 +43,13 @@ for i in $(seq 1 30); do
|
||||
done
|
||||
|
||||
# Verify the DB is readable/writable via the API.
|
||||
# /api/demo is unauthenticated and reads the settings table — if SQLite is broken this returns an error.
|
||||
# /api/demo is unauthenticated and reads the settings table - if SQLite is broken this returns an error.
|
||||
echo "Testing DB via API (/api/demo)..."
|
||||
DEMO_RESPONSE=$(docker exec fredy curl -sf http://localhost:9998/api/demo 2>&1)
|
||||
if echo "$DEMO_RESPONSE" | grep -q "demoMode"; then
|
||||
echo "DB is readable (got demoMode from /api/demo)"
|
||||
else
|
||||
echo "DB check failed — unexpected response from /api/demo: $DEMO_RESPONSE"
|
||||
echo "DB check failed - unexpected response from /api/demo: $DEMO_RESPONSE"
|
||||
docker logs fredy
|
||||
exit 1
|
||||
fi
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
9
index.js
9
index.js
@@ -15,6 +15,15 @@ import { initGeocodingCron } from './lib/services/crons/geocoding-cron.js';
|
||||
import { getSettings } from './lib/services/storage/settingsStorage.js';
|
||||
import SqliteConnection, { computeDbPath } from './lib/services/storage/SqliteConnection.js';
|
||||
import { initJobExecutionService } from './lib/services/jobs/jobExecutionService.js';
|
||||
import { ensureValidBinary } from './lib/services/ensureValidBinary.js';
|
||||
|
||||
// Ensure the CloakBrowser stealth Chromium binary is present and complete before
|
||||
// jobs run. ensureValidBinary() also detects and auto-heals partial extractions
|
||||
// (e.g. a newer version that was downloaded but only the chrome executable was
|
||||
// written) so Chrome never crashes with "Invalid file descriptor to ICU data".
|
||||
logger.info('Checking CloakBrowser binary...');
|
||||
await ensureValidBinary();
|
||||
logger.info('CloakBrowser binary ready.');
|
||||
|
||||
//in the config, we store the path of the sqlite file, thus we must check if it is available
|
||||
const isConfigAccessible = await checkIfConfigIsAccessible();
|
||||
|
||||
@@ -227,7 +227,7 @@ class FredyPipelineExecutioner {
|
||||
const extractor = new Extractor({ ...this._providerConfig.puppeteerOptions, browser: this._browser });
|
||||
return new Promise((resolve, reject) => {
|
||||
extractor
|
||||
.execute(url, this._providerConfig.waitForSelector)
|
||||
.execute(url, this._providerConfig.waitForSelector, this._providerId)
|
||||
.then(() => {
|
||||
const listings = extractor.parseResponseText(
|
||||
this._providerConfig.crawlContainer,
|
||||
|
||||
@@ -7,4 +7,8 @@ export const TRACKING_POIS = {
|
||||
DISTANCE_ADDRESS_ENTERED: 'DISTANCE_ADDRESS_ENTERED',
|
||||
WELCOME_FINISHED: 'WELCOME_FINISHED',
|
||||
WELCOME_SKIPPED: 'WELCOME_SKIPPED',
|
||||
JOBS_TABLE_VIEW: 'JOBS_TABLE_VIEW',
|
||||
LISTING_TABLE_VIEW: 'LISTING_TABLE_VIEW',
|
||||
BASE_URL_SETTING: 'BASE_URL_SETTING',
|
||||
DETECTED_AS_BOT: 'DETECTED_AS_BOT',
|
||||
};
|
||||
|
||||
142
lib/api/api.js
142
lib/api/api.js
@@ -3,64 +3,100 @@
|
||||
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
|
||||
*/
|
||||
|
||||
import { notificationAdapterRouter } from './routes/notificationAdapterRouter.js';
|
||||
import { authInterceptor, cookieSession, adminInterceptor } from './security.js';
|
||||
import { generalSettingsRouter } from './routes/generalSettingsRoute.js';
|
||||
import { providerRouter } from './routes/providerRouter.js';
|
||||
import { versionRouter } from './routes/versionRouter.js';
|
||||
import { loginRouter } from './routes/loginRoute.js';
|
||||
import { userRouter } from './routes/userRoute.js';
|
||||
import { userSettingsRouter } from './routes/userSettingsRoute.js';
|
||||
import { jobRouter } from './routes/jobRouter.js';
|
||||
import bodyParser from 'body-parser';
|
||||
import restana from 'restana';
|
||||
import files from 'serve-static';
|
||||
import Fastify from 'fastify';
|
||||
import fastifyHelmet from '@fastify/helmet';
|
||||
import fastifyCookie from '@fastify/cookie';
|
||||
import fastifySession from '@fastify/session';
|
||||
import fastifyStatic from '@fastify/static';
|
||||
import path from 'path';
|
||||
import { getDirName } from '../utils.js';
|
||||
import { demoRouter } from './routes/demoRouter.js';
|
||||
import logger from '../services/logger.js';
|
||||
import { listingsRouter } from './routes/listingsRouter.js';
|
||||
import { getSettings, getOrCreateSessionSecret } from '../services/storage/settingsStorage.js';
|
||||
import { dashboardRouter } from './routes/dashboardRouter.js';
|
||||
import { backupRouter } from './routes/backupRouter.js';
|
||||
import { trackingRouter } from './routes/trackingRoute.js';
|
||||
import logger from '../services/logger.js';
|
||||
import { authHook, adminHook } from './security.js';
|
||||
|
||||
import loginPlugin from './routes/loginRoute.js';
|
||||
import demoPlugin from './routes/demoRouter.js';
|
||||
import jobPlugin from './routes/jobRouter.js';
|
||||
import versionPlugin from './routes/versionRouter.js';
|
||||
import listingsPlugin from './routes/listingsRouter.js';
|
||||
import dashboardPlugin from './routes/dashboardRouter.js';
|
||||
import userSettingsPlugin from './routes/userSettingsRoute.js';
|
||||
import trackingPlugin from './routes/trackingRoute.js';
|
||||
import generalSettingsPlugin from './routes/generalSettingsRoute.js';
|
||||
import backupPlugin from './routes/backupRouter.js';
|
||||
import userPlugin from './routes/userRoute.js';
|
||||
import notificationAdapterPlugin from './routes/notificationAdapterRouter.js';
|
||||
import providerPlugin from './routes/providerRouter.js';
|
||||
import { registerMcpRoutes } from '../mcp/mcpHttpRoute.js';
|
||||
const service = restana();
|
||||
const staticService = files(path.join(getDirName(), '../ui/public'));
|
||||
|
||||
const PORT = (await getSettings()).port || 9998;
|
||||
const sessionSecret = await getOrCreateSessionSecret();
|
||||
const SESSION_MAX_AGE = 2 * 60 * 60 * 1000;
|
||||
|
||||
service.use(bodyParser.json());
|
||||
service.use(cookieSession(sessionSecret));
|
||||
service.use(staticService);
|
||||
service.use('/api/admin', authInterceptor());
|
||||
service.use('/api/jobs', authInterceptor());
|
||||
service.use('/api/version', authInterceptor());
|
||||
service.use('/api/listings', authInterceptor());
|
||||
service.use('/api/dashboard', authInterceptor());
|
||||
service.use('/api/user/settings', authInterceptor());
|
||||
service.use('/api/tracking', authInterceptor());
|
||||
|
||||
// /admin can only be accessed when user is having admin permissions
|
||||
service.use('/api/admin', adminInterceptor());
|
||||
service.use('/api/jobs/notificationAdapter', notificationAdapterRouter);
|
||||
service.use('/api/admin/generalSettings', generalSettingsRouter);
|
||||
service.use('/api/admin/backup', backupRouter);
|
||||
service.use('/api/jobs/provider', providerRouter);
|
||||
service.use('/api/admin/users', userRouter);
|
||||
service.use('/api/user/settings', userSettingsRouter);
|
||||
service.use('/api/version', versionRouter);
|
||||
service.use('/api/jobs', jobRouter);
|
||||
service.use('/api/login', loginRouter);
|
||||
service.use('/api/listings', listingsRouter);
|
||||
service.use('/api/dashboard', dashboardRouter);
|
||||
service.use('/api/tracking', trackingRouter);
|
||||
//this route is unsecured intentionally as it is being queried from the login page
|
||||
service.use('/api/demo', demoRouter);
|
||||
|
||||
// MCP Streamable HTTP endpoint (secured via Bearer token, not cookie-session)
|
||||
registerMcpRoutes(service);
|
||||
|
||||
service.start(PORT).then(() => {
|
||||
logger.debug(`Started API service on port ${PORT}`);
|
||||
const fastify = Fastify({
|
||||
logger: false,
|
||||
bodyLimit: 50 * 1024 * 1024, // 50 MB for backup uploads
|
||||
});
|
||||
|
||||
// Security headers (CSP disabled to avoid breaking the SPA)
|
||||
await fastify.register(fastifyHelmet, { contentSecurityPolicy: false });
|
||||
|
||||
// Cookie + session (in-memory store, signed cookie)
|
||||
await fastify.register(fastifyCookie);
|
||||
await fastify.register(fastifySession, {
|
||||
secret: sessionSecret,
|
||||
cookieName: 'fredy-admin-session',
|
||||
cookie: {
|
||||
maxAge: SESSION_MAX_AGE,
|
||||
httpOnly: true,
|
||||
secure: false,
|
||||
sameSite: 'lax',
|
||||
},
|
||||
saveUninitialized: false,
|
||||
});
|
||||
|
||||
// Serve the React SPA from ui/public/
|
||||
await fastify.register(fastifyStatic, {
|
||||
root: path.join(getDirName(), '../ui/public'),
|
||||
wildcard: false,
|
||||
});
|
||||
|
||||
// Public routes - no auth required
|
||||
fastify.register(loginPlugin, { prefix: '/api/login' });
|
||||
fastify.register(demoPlugin, { prefix: '/api/demo' });
|
||||
|
||||
// User-authenticated routes
|
||||
fastify.register(async (app) => {
|
||||
app.addHook('preHandler', authHook);
|
||||
app.register(jobPlugin, { prefix: '/api/jobs' });
|
||||
app.register(notificationAdapterPlugin, { prefix: '/api/jobs/notificationAdapter' });
|
||||
app.register(providerPlugin, { prefix: '/api/jobs/provider' });
|
||||
app.register(versionPlugin, { prefix: '/api/version' });
|
||||
app.register(listingsPlugin, { prefix: '/api/listings' });
|
||||
app.register(dashboardPlugin, { prefix: '/api/dashboard' });
|
||||
app.register(userSettingsPlugin, { prefix: '/api/user/settings' });
|
||||
app.register(trackingPlugin, { prefix: '/api/tracking' });
|
||||
app.register(generalSettingsPlugin, { prefix: '/api/admin/generalSettings' });
|
||||
});
|
||||
|
||||
// Admin-only routes
|
||||
fastify.register(async (app) => {
|
||||
app.addHook('preHandler', authHook);
|
||||
app.addHook('preHandler', adminHook);
|
||||
app.register(backupPlugin, { prefix: '/api/admin/backup' });
|
||||
app.register(userPlugin, { prefix: '/api/admin/users' });
|
||||
});
|
||||
|
||||
// MCP Streamable HTTP (Bearer token auth - no session)
|
||||
registerMcpRoutes(fastify);
|
||||
|
||||
// SPA fallback - serve index.html for all non-API GET requests
|
||||
fastify.setNotFoundHandler((request, reply) => {
|
||||
if (!request.url.startsWith('/api/')) {
|
||||
return reply.sendFile('index.html');
|
||||
}
|
||||
return reply.code(404).send({ error: 'Not found' });
|
||||
});
|
||||
|
||||
await fastify.listen({ port: PORT, host: '0.0.0.0' });
|
||||
logger.debug(`Started API service on port ${PORT}`);
|
||||
|
||||
@@ -3,73 +3,61 @@
|
||||
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
|
||||
*/
|
||||
|
||||
import restana from 'restana';
|
||||
import {
|
||||
buildBackupFileName,
|
||||
createBackupZip,
|
||||
precheckRestore,
|
||||
restoreFromZip,
|
||||
} from '../../services/storage/backupRestoreService.js';
|
||||
import { getSettings } from '../../services/storage/settingsStorage.js';
|
||||
import { isAdmin } from '../security.js';
|
||||
|
||||
const DEMO_MODE_ERROR = 'Backup and restore are not available in demo mode.';
|
||||
|
||||
/**
|
||||
* Backup & Restore Admin Router
|
||||
*
|
||||
* Endpoints:
|
||||
* - GET /api/admin/backup
|
||||
* Returns the current database as a zip download. Content-Type: application/zip
|
||||
* - POST /api/admin/backup/restore?dryRun=true
|
||||
* Accepts a zip file (raw body). Returns a compatibility report, does not restore.
|
||||
* - POST /api/admin/backup/restore?force=true|false
|
||||
* Accepts a zip file (raw body). Restores the database; when incompatible and force=false, returns 400.
|
||||
* @param {import('fastify').FastifyInstance} fastify
|
||||
*/
|
||||
const service = restana();
|
||||
const backupRouter = service.newRouter();
|
||||
export default async function backupPlugin(fastify) {
|
||||
// Parse raw binary uploads as Buffer
|
||||
fastify.addContentTypeParser(
|
||||
['application/zip', 'application/octet-stream'],
|
||||
{ parseAs: 'buffer' },
|
||||
(req, body, done) => done(null, body),
|
||||
);
|
||||
|
||||
backupRouter.get('/', async (req, res) => {
|
||||
const zipBuffer = await createBackupZip();
|
||||
const fileName = await buildBackupFileName();
|
||||
res.setHeader('Content-Type', 'application/zip');
|
||||
res.setHeader('Content-Disposition', `attachment; filename="${fileName}"`);
|
||||
res.send(zipBuffer);
|
||||
});
|
||||
fastify.get('/', async (request, reply) => {
|
||||
const settings = await getSettings();
|
||||
if (settings.demoMode && !isAdmin(request)) {
|
||||
return reply.code(403).send({ error: DEMO_MODE_ERROR });
|
||||
}
|
||||
const zipBuffer = await createBackupZip();
|
||||
const fileName = await buildBackupFileName();
|
||||
reply.header('Content-Type', 'application/zip');
|
||||
reply.header('Content-Disposition', `attachment; filename="${fileName}"`);
|
||||
return reply.send(zipBuffer);
|
||||
});
|
||||
|
||||
/**
|
||||
* Read the full request body as a Buffer. Used for raw zip uploads.
|
||||
* @param {import('http').IncomingMessage} req
|
||||
* @returns {Promise<Buffer>}
|
||||
*/
|
||||
function readBody(req) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const chunks = [];
|
||||
req.on('data', (c) => chunks.push(c));
|
||||
req.on('end', () => resolve(Buffer.concat(chunks)));
|
||||
req.on('error', (e) => reject(e));
|
||||
fastify.post('/restore', async (request, reply) => {
|
||||
const settings = await getSettings();
|
||||
if (settings.demoMode && !isAdmin(request)) {
|
||||
return reply.code(403).send({ error: DEMO_MODE_ERROR });
|
||||
}
|
||||
const { dryRun = 'false', force = 'false' } = request.query || {};
|
||||
const doDryRun = String(dryRun) === 'true';
|
||||
const doForce = String(force) === 'true';
|
||||
const body = request.body; // Buffer from addContentTypeParser
|
||||
|
||||
if (doDryRun) {
|
||||
return precheckRestore(body);
|
||||
}
|
||||
|
||||
try {
|
||||
return restoreFromZip(body, { force: doForce });
|
||||
} catch (e) {
|
||||
return reply.code(400).send({
|
||||
message: e?.message || 'Restore failed',
|
||||
details: e?.payload || null,
|
||||
});
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// Upload endpoint. Accepts raw zip (Content-Type: application/zip or application/octet-stream)
|
||||
// Query parameters:
|
||||
// - dryRun=true => only validate and return compatibility info
|
||||
// - force=true => proceed even if incompatible
|
||||
backupRouter.post('/restore', async (req, res) => {
|
||||
const { dryRun = 'false', force = 'false' } = req.query || {};
|
||||
const doDryRun = String(dryRun) === 'true';
|
||||
const doForce = String(force) === 'true';
|
||||
const body = await readBody(req);
|
||||
|
||||
if (doDryRun) {
|
||||
res.body = await precheckRestore(body);
|
||||
return res.send();
|
||||
}
|
||||
|
||||
try {
|
||||
res.body = await restoreFromZip(body, { force: doForce });
|
||||
return res.send();
|
||||
} catch (e) {
|
||||
res.statusCode = 400;
|
||||
res.body = { message: e?.message || 'Restore failed', details: e?.payload || null };
|
||||
return res.send();
|
||||
}
|
||||
});
|
||||
|
||||
export { backupRouter };
|
||||
|
||||
@@ -3,23 +3,14 @@
|
||||
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
|
||||
*/
|
||||
|
||||
import restana from 'restana';
|
||||
import * as jobStorage from '../../services/storage/jobStorage.js';
|
||||
import * as userStorage from '../../services/storage/userStorage.js';
|
||||
import { getListingsKpisForJobIds, getProviderDistributionForJobIds } from '../../services/storage/listingsStorage.js';
|
||||
import { getSettings } from '../../services/storage/settingsStorage.js';
|
||||
import { isAdmin } from '../security.js';
|
||||
|
||||
const service = restana();
|
||||
export const dashboardRouter = service.newRouter();
|
||||
|
||||
function isAdmin(req) {
|
||||
const user = req.session?.currentUser ? userStorage.getUser(req.session.currentUser) : null;
|
||||
return !!user?.isAdmin;
|
||||
}
|
||||
|
||||
function getAccessibleJobs(req) {
|
||||
const currentUser = req.session.currentUser;
|
||||
const admin = isAdmin(req);
|
||||
function getAccessibleJobs(request) {
|
||||
const currentUser = request.session.currentUser;
|
||||
const admin = isAdmin(request);
|
||||
return jobStorage
|
||||
.getJobs()
|
||||
.filter((job) => admin || job.userId === currentUser || job.shared_with_user.includes(currentUser));
|
||||
@@ -29,43 +20,45 @@ function cap(val) {
|
||||
return String(val).charAt(0).toUpperCase() + String(val).slice(1);
|
||||
}
|
||||
|
||||
dashboardRouter.get('/', async (req, res) => {
|
||||
const jobs = getAccessibleJobs(req);
|
||||
const settings = await getSettings();
|
||||
/**
|
||||
* @param {import('fastify').FastifyInstance} fastify
|
||||
*/
|
||||
export default async function dashboardPlugin(fastify) {
|
||||
fastify.get('/', async (request) => {
|
||||
const jobs = getAccessibleJobs(request);
|
||||
const settings = await getSettings();
|
||||
|
||||
// KPIs
|
||||
const totalJobs = jobs.length;
|
||||
const totalListings = jobs.reduce((sum, j) => sum + (j.numberOfFoundListings || 0), 0);
|
||||
const jobIds = jobs.map((j) => j.id);
|
||||
const { numberOfActiveListings, medianPriceOfListings } = getListingsKpisForJobIds(jobIds);
|
||||
// Build Pie data in a simple shape the frontend can consume directly
|
||||
// Shape: { labels: string[], values: number[] } with values as percentages
|
||||
const providerPieRaw = getProviderDistributionForJobIds(jobIds);
|
||||
const providerPie = Array.isArray(providerPieRaw)
|
||||
? {
|
||||
labels: providerPieRaw.map((p) => cap(p.type)),
|
||||
values: providerPieRaw.map((p) => Number(p.value) || 0),
|
||||
}
|
||||
: providerPieRaw && typeof providerPieRaw === 'object'
|
||||
const totalJobs = jobs.length;
|
||||
const totalListings = jobs.reduce((sum, j) => sum + (j.numberOfFoundListings || 0), 0);
|
||||
const jobIds = jobs.map((j) => j.id);
|
||||
const { numberOfActiveListings, medianPriceOfListings } = getListingsKpisForJobIds(jobIds);
|
||||
|
||||
const providerPieRaw = getProviderDistributionForJobIds(jobIds);
|
||||
const providerPie = Array.isArray(providerPieRaw)
|
||||
? {
|
||||
labels: Array.isArray(providerPieRaw.labels) ? providerPieRaw.labels : [],
|
||||
values: Array.isArray(providerPieRaw.values) ? providerPieRaw.values : [],
|
||||
labels: providerPieRaw.map((p) => cap(p.type)),
|
||||
values: providerPieRaw.map((p) => Number(p.value) || 0),
|
||||
}
|
||||
: { labels: [], values: [] };
|
||||
: providerPieRaw && typeof providerPieRaw === 'object'
|
||||
? {
|
||||
labels: Array.isArray(providerPieRaw.labels) ? providerPieRaw.labels : [],
|
||||
values: Array.isArray(providerPieRaw.values) ? providerPieRaw.values : [],
|
||||
}
|
||||
: { labels: [], values: [] };
|
||||
|
||||
res.body = {
|
||||
general: {
|
||||
interval: settings.interval,
|
||||
lastRun: settings.lastRun || null,
|
||||
nextRun: settings.lastRun == null ? 0 : settings.lastRun + settings.interval * 60000,
|
||||
},
|
||||
kpis: {
|
||||
totalJobs,
|
||||
totalListings,
|
||||
numberOfActiveListings,
|
||||
medianPriceOfListings,
|
||||
},
|
||||
pie: providerPie,
|
||||
};
|
||||
res.send();
|
||||
});
|
||||
return {
|
||||
general: {
|
||||
interval: settings.interval,
|
||||
lastRun: settings.lastRun || null,
|
||||
nextRun: settings.lastRun == null ? 0 : settings.lastRun + settings.interval * 60000,
|
||||
},
|
||||
kpis: {
|
||||
totalJobs,
|
||||
totalListings,
|
||||
numberOfActiveListings,
|
||||
medianPriceOfListings,
|
||||
},
|
||||
pie: providerPie,
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
@@ -3,15 +3,14 @@
|
||||
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
|
||||
*/
|
||||
|
||||
import restana from 'restana';
|
||||
import { getSettings } from '../../services/storage/settingsStorage.js';
|
||||
const service = restana();
|
||||
const demoRouter = service.newRouter();
|
||||
|
||||
demoRouter.get('/', async (req, res) => {
|
||||
const settings = await getSettings();
|
||||
res.body = Object.assign({}, { demoMode: settings.demoMode });
|
||||
res.send();
|
||||
});
|
||||
|
||||
export { demoRouter };
|
||||
/**
|
||||
* @param {import('fastify').FastifyInstance} fastify
|
||||
*/
|
||||
export default async function demoPlugin(fastify) {
|
||||
fastify.get('/', async () => {
|
||||
const settings = await getSettings();
|
||||
return { demoMode: settings.demoMode };
|
||||
});
|
||||
}
|
||||
|
||||
@@ -3,43 +3,51 @@
|
||||
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
|
||||
*/
|
||||
|
||||
import restana from 'restana';
|
||||
import { getDirName } from '../../utils.js';
|
||||
import fs from 'fs';
|
||||
import { ensureDemoUserExists } from '../../services/storage/userStorage.js';
|
||||
import logger from '../../services/logger.js';
|
||||
import { getSettings, upsertSettings } from '../../services/storage/settingsStorage.js';
|
||||
import { isAdmin } from '../security.js';
|
||||
const service = restana();
|
||||
const generalSettingsRouter = service.newRouter();
|
||||
import { trackPoi } from '../../services/tracking/Tracker.js';
|
||||
import { TRACKING_POIS } from '../../TRACKING_POIS.js';
|
||||
|
||||
generalSettingsRouter.get('/', async (req, res) => {
|
||||
res.body = Object.assign({}, await getSettings());
|
||||
res.send();
|
||||
});
|
||||
generalSettingsRouter.post('/', async (req, res) => {
|
||||
const { sqlitepath, ...appSettings } = req.body || {};
|
||||
if (typeof appSettings.baseUrl === 'string') {
|
||||
appSettings.baseUrl = appSettings.baseUrl.trim().replace(/\/$/, '');
|
||||
}
|
||||
const localSettings = await getSettings();
|
||||
/**
|
||||
* @param {import('fastify').FastifyInstance} fastify
|
||||
*/
|
||||
export default async function generalSettingsPlugin(fastify) {
|
||||
fastify.get('/', async () => {
|
||||
return Object.assign({}, await getSettings());
|
||||
});
|
||||
|
||||
if (localSettings.demoMode && !isAdmin(req)) {
|
||||
res.send(new Error('In demo mode, it is not allowed to change these settings.'));
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
if (typeof sqlitepath !== 'undefined') {
|
||||
fs.writeFileSync(`${getDirName()}/../conf/config.json`, JSON.stringify({ sqlitepath }));
|
||||
fastify.post('/', async (request, reply) => {
|
||||
const { sqlitepath, ...appSettings } = request.body || {};
|
||||
if (typeof appSettings.baseUrl === 'string') {
|
||||
appSettings.baseUrl = appSettings.baseUrl.trim().replace(/\/$/, '');
|
||||
}
|
||||
upsertSettings(appSettings);
|
||||
ensureDemoUserExists();
|
||||
} catch (err) {
|
||||
logger.error(err);
|
||||
res.send(new Error('Error while trying to write settings.'));
|
||||
return;
|
||||
}
|
||||
res.send();
|
||||
});
|
||||
export { generalSettingsRouter };
|
||||
const localSettings = await getSettings();
|
||||
|
||||
if (!isAdmin(request)) {
|
||||
const reason = localSettings.demoMode
|
||||
? 'In demo mode, it is not allowed to change these settings.'
|
||||
: 'Only admins can change these settings.';
|
||||
return reply.code(403).send({ error: reason });
|
||||
}
|
||||
|
||||
try {
|
||||
if (typeof sqlitepath !== 'undefined') {
|
||||
fs.writeFileSync(`${getDirName()}/../conf/config.json`, JSON.stringify({ sqlitepath }));
|
||||
}
|
||||
|
||||
upsertSettings(appSettings);
|
||||
ensureDemoUserExists();
|
||||
if (appSettings.baseUrl != null) {
|
||||
await trackPoi(TRACKING_POIS.BASE_URL_SETTING);
|
||||
}
|
||||
} catch (err) {
|
||||
logger.error(err);
|
||||
return reply.code(500).send({ error: 'Error while trying to write settings.' });
|
||||
}
|
||||
return reply.send();
|
||||
});
|
||||
}
|
||||
|
||||
@@ -3,7 +3,6 @@
|
||||
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
|
||||
*/
|
||||
|
||||
import restana from 'restana';
|
||||
import * as jobStorage from '../../services/storage/jobStorage.js';
|
||||
import * as userStorage from '../../services/storage/userStorage.js';
|
||||
import { isAdmin } from '../security.js';
|
||||
@@ -13,257 +12,234 @@ import { isRunning as isJobRunning } from '../../services/jobs/run-state.js';
|
||||
import { addClient as addSseClient, removeClient } from '../../services/sse/sse-broker.js';
|
||||
import { getSettings } from '../../services/storage/settingsStorage.js';
|
||||
|
||||
const service = restana();
|
||||
const jobRouter = service.newRouter();
|
||||
|
||||
const DEMO_JOB_NAME = 'Demo-Job';
|
||||
|
||||
function doesJobBelongsToUser(job, req) {
|
||||
const userId = req.session.currentUser;
|
||||
if (userId == null) {
|
||||
return false;
|
||||
}
|
||||
function doesJobBelongsToUser(job, request) {
|
||||
const userId = request.session.currentUser;
|
||||
if (userId == null) return false;
|
||||
const user = userStorage.getUser(userId);
|
||||
if (user == null) {
|
||||
return false;
|
||||
}
|
||||
if (user == null) return false;
|
||||
return user.isAdmin || job.userId === user.id;
|
||||
}
|
||||
|
||||
jobRouter.get('/', async (req, res) => {
|
||||
const isUserAdmin = isAdmin(req);
|
||||
//show only the jobs which belongs to the user (or all of the user is an admin)
|
||||
res.body = jobStorage
|
||||
.getJobs()
|
||||
.filter(
|
||||
(job) =>
|
||||
isUserAdmin || job.userId === req.session.currentUser || job.shared_with_user.includes(req.session.currentUser),
|
||||
)
|
||||
.map((job) => {
|
||||
return {
|
||||
/**
|
||||
* @param {import('fastify').FastifyInstance} fastify
|
||||
*/
|
||||
export default async function jobPlugin(fastify) {
|
||||
fastify.get('/', async (request) => {
|
||||
const isUserAdmin = isAdmin(request);
|
||||
return jobStorage
|
||||
.getJobs()
|
||||
.filter(
|
||||
(job) =>
|
||||
isUserAdmin ||
|
||||
job.userId === request.session.currentUser ||
|
||||
job.shared_with_user.includes(request.session.currentUser),
|
||||
)
|
||||
.map((job) => ({
|
||||
...job,
|
||||
running: isJobRunning(job.id),
|
||||
isOnlyShared:
|
||||
!isUserAdmin &&
|
||||
job.userId !== req.session.currentUser &&
|
||||
job.shared_with_user.includes(req.session.currentUser),
|
||||
};
|
||||
});
|
||||
|
||||
res.send();
|
||||
});
|
||||
|
||||
jobRouter.get('/data', async (req, res) => {
|
||||
const { page, pageSize = 50, activityFilter, sortfield = null, sortdir = 'asc', freeTextFilter } = req.query || {};
|
||||
|
||||
// normalize booleans
|
||||
const toBool = (v) => {
|
||||
if (v === true || v === 'true' || v === 1 || v === '1') return true;
|
||||
if (v === false || v === 'false' || v === 0 || v === '0') return false;
|
||||
return null;
|
||||
};
|
||||
const normalizedActivity = toBool(activityFilter);
|
||||
|
||||
const queryResult = jobStorage.queryJobs({
|
||||
page: page ? parseInt(page, 10) : 1,
|
||||
pageSize: pageSize ? parseInt(pageSize, 10) : 50,
|
||||
freeTextFilter: freeTextFilter || null,
|
||||
activityFilter: normalizedActivity,
|
||||
sortField: sortfield || null,
|
||||
sortDir: sortdir === 'desc' ? 'desc' : 'asc',
|
||||
userId: req.session.currentUser,
|
||||
isAdmin: isAdmin(req),
|
||||
job.userId !== request.session.currentUser &&
|
||||
job.shared_with_user.includes(request.session.currentUser),
|
||||
}));
|
||||
});
|
||||
|
||||
const isUserAdmin = isAdmin(req);
|
||||
fastify.get('/data', async (request) => {
|
||||
const {
|
||||
page,
|
||||
pageSize = 50,
|
||||
activityFilter,
|
||||
sortfield = null,
|
||||
sortdir = 'asc',
|
||||
freeTextFilter,
|
||||
} = request.query || {};
|
||||
|
||||
// Map result to include runtime status
|
||||
queryResult.result = queryResult.result.map((job) => {
|
||||
return {
|
||||
const toBool = (v) => {
|
||||
if (v === true || v === 'true' || v === 1 || v === '1') return true;
|
||||
if (v === false || v === 'false' || v === 0 || v === '0') return false;
|
||||
return null;
|
||||
};
|
||||
const normalizedActivity = toBool(activityFilter);
|
||||
|
||||
const queryResult = jobStorage.queryJobs({
|
||||
page: page ? parseInt(page, 10) : 1,
|
||||
pageSize: pageSize ? parseInt(pageSize, 10) : 50,
|
||||
freeTextFilter: freeTextFilter || null,
|
||||
activityFilter: normalizedActivity,
|
||||
sortField: sortfield || null,
|
||||
sortDir: sortdir === 'desc' ? 'desc' : 'asc',
|
||||
userId: request.session.currentUser,
|
||||
isAdmin: isAdmin(request),
|
||||
});
|
||||
|
||||
const isUserAdmin = isAdmin(request);
|
||||
queryResult.result = queryResult.result.map((job) => ({
|
||||
...job,
|
||||
running: isJobRunning(job.id),
|
||||
isOnlyShared:
|
||||
!isUserAdmin &&
|
||||
job.userId !== req.session.currentUser &&
|
||||
job.shared_with_user.includes(req.session.currentUser),
|
||||
};
|
||||
job.userId !== request.session.currentUser &&
|
||||
job.shared_with_user.includes(request.session.currentUser),
|
||||
}));
|
||||
|
||||
return queryResult;
|
||||
});
|
||||
|
||||
res.body = queryResult;
|
||||
res.send();
|
||||
});
|
||||
// Server-Sent Events for real-time job status updates
|
||||
fastify.get('/events', async (request, reply) => {
|
||||
const userId = request.session?.currentUser;
|
||||
if (userId == null) {
|
||||
return reply.code(401).send({ message: 'Unauthorized' });
|
||||
}
|
||||
|
||||
reply.hijack();
|
||||
const raw = reply.raw;
|
||||
raw.setHeader('Content-Type', 'text/event-stream');
|
||||
raw.setHeader('Cache-Control', 'no-cache');
|
||||
raw.setHeader('Connection', 'keep-alive');
|
||||
|
||||
// Server-Sent Events for job status updates
|
||||
jobRouter.get('/events', async (req, res) => {
|
||||
const userId = req.session.currentUser;
|
||||
if (userId == null) {
|
||||
res.send({ message: 'Unauthorized' }, 401);
|
||||
return;
|
||||
}
|
||||
// SSE headers
|
||||
res.setHeader('Content-Type', 'text/event-stream');
|
||||
res.setHeader('Cache-Control', 'no-cache');
|
||||
res.setHeader('Connection', 'keep-alive');
|
||||
try {
|
||||
// Initial comment to establish stream
|
||||
res.write(': connected\n\n');
|
||||
addSseClient(userId, res);
|
||||
// Cleanup on close/aborted
|
||||
const onClose = () => removeClient(userId, res);
|
||||
// restana exposes original req/res; use both close and finish
|
||||
req.on('close', onClose);
|
||||
req.on('aborted', onClose);
|
||||
res.on('close', onClose);
|
||||
} catch (e) {
|
||||
logger.error('Error establishing SSE connection', e);
|
||||
try {
|
||||
res.end();
|
||||
} catch {
|
||||
//noop
|
||||
raw.write(': connected\n\n');
|
||||
addSseClient(userId, raw);
|
||||
const onClose = () => removeClient(userId, raw);
|
||||
request.raw.on('close', onClose);
|
||||
} catch (e) {
|
||||
logger.error('Error establishing SSE connection', e);
|
||||
try {
|
||||
raw.end();
|
||||
} catch {
|
||||
/* noop */
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
jobRouter.post('/startAll', async (req, res) => {
|
||||
try {
|
||||
const userId = req.session.currentUser;
|
||||
// Emit only the userId; handler will decide based on admin/ownership
|
||||
bus.emit('jobs:runAll', { userId });
|
||||
res.send({ message: 'Run all accepted' }, 202);
|
||||
} catch (err) {
|
||||
logger.error('Failed to trigger startAll', err);
|
||||
res.send({ message: 'Unexpected error' }, 500);
|
||||
}
|
||||
});
|
||||
|
||||
// Trigger a single job run
|
||||
jobRouter.post('/:jobId/run', async (req, res) => {
|
||||
const { jobId } = req.params;
|
||||
try {
|
||||
const job = jobStorage.getJob(jobId);
|
||||
if (!job) {
|
||||
res.send({ message: 'Job not found' }, 404);
|
||||
return;
|
||||
fastify.post('/startAll', async (request, reply) => {
|
||||
try {
|
||||
const userId = request.session.currentUser;
|
||||
bus.emit('jobs:runAll', { userId });
|
||||
return reply.code(202).send({ message: 'Run all accepted' });
|
||||
} catch (err) {
|
||||
logger.error('Failed to trigger startAll', err);
|
||||
return reply.code(500).send({ message: 'Unexpected error' });
|
||||
}
|
||||
if (!doesJobBelongsToUser(job, req)) {
|
||||
res.send({ message: 'You are trying to run a job that is not associated to your user' }, 403);
|
||||
return;
|
||||
}
|
||||
if (isJobRunning(jobId)) {
|
||||
res.send({ message: 'Job is already running' }, 409);
|
||||
return;
|
||||
}
|
||||
// fire and forget; actual execution handled by index.js listener
|
||||
bus.emit('jobs:runOne', { jobId });
|
||||
res.send({ message: 'Job run accepted' }, 202);
|
||||
} catch (error) {
|
||||
logger.error(error);
|
||||
res.send({ message: 'Unexpected error triggering job' }, 500);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
jobRouter.post('/', async (req, res) => {
|
||||
const {
|
||||
provider,
|
||||
notificationAdapter,
|
||||
name,
|
||||
blacklist = [],
|
||||
jobId,
|
||||
enabled,
|
||||
shareWithUsers = [],
|
||||
spatialFilter = null,
|
||||
specFilter = null,
|
||||
} = req.body;
|
||||
const settings = await getSettings();
|
||||
try {
|
||||
let jobFromDb = jobStorage.getJob(jobId);
|
||||
|
||||
if (jobFromDb && !doesJobBelongsToUser(jobFromDb, req)) {
|
||||
res.send(new Error('You are trying to change a job that is not associated to your user.'));
|
||||
return;
|
||||
fastify.post('/:jobId/run', async (request, reply) => {
|
||||
const { jobId } = request.params;
|
||||
try {
|
||||
const job = jobStorage.getJob(jobId);
|
||||
if (!job) {
|
||||
return reply.code(404).send({ message: 'Job not found' });
|
||||
}
|
||||
if (!doesJobBelongsToUser(job, request)) {
|
||||
return reply.code(403).send({ message: 'You are trying to run a job that is not associated to your user' });
|
||||
}
|
||||
if (isJobRunning(jobId)) {
|
||||
return reply.code(409).send({ message: 'Job is already running' });
|
||||
}
|
||||
bus.emit('jobs:runOne', { jobId });
|
||||
return reply.code(202).send({ message: 'Job run accepted' });
|
||||
} catch (error) {
|
||||
logger.error(error);
|
||||
return reply.code(500).send({ message: 'Unexpected error triggering job' });
|
||||
}
|
||||
});
|
||||
|
||||
if (settings.demoMode && !isAdmin(req) && jobFromDb && jobFromDb.name === DEMO_JOB_NAME) {
|
||||
res.send(new Error('Sorry, but you cannot change the Status of our Demo Job ;)'));
|
||||
return;
|
||||
}
|
||||
|
||||
jobStorage.upsertJob({
|
||||
userId: req.session.currentUser,
|
||||
jobId,
|
||||
enabled,
|
||||
name,
|
||||
blacklist,
|
||||
fastify.post('/', async (request, reply) => {
|
||||
const {
|
||||
provider,
|
||||
notificationAdapter,
|
||||
shareWithUsers,
|
||||
spatialFilter,
|
||||
specFilter,
|
||||
});
|
||||
} catch (error) {
|
||||
res.send(new Error(error));
|
||||
logger.error(error);
|
||||
}
|
||||
res.send();
|
||||
});
|
||||
name,
|
||||
blacklist = [],
|
||||
jobId,
|
||||
enabled,
|
||||
shareWithUsers = [],
|
||||
spatialFilter = null,
|
||||
specFilter = null,
|
||||
} = request.body;
|
||||
const settings = await getSettings();
|
||||
try {
|
||||
const jobFromDb = jobStorage.getJob(jobId);
|
||||
|
||||
jobRouter.delete('', async (req, res) => {
|
||||
const { jobId } = req.body;
|
||||
const settings = await getSettings();
|
||||
try {
|
||||
const job = jobStorage.getJob(jobId);
|
||||
if (settings.demoMode && !isAdmin(req) && job.name === DEMO_JOB_NAME) {
|
||||
res.send(new Error('Sorry, but you cannot remove the Demo Job ;)'));
|
||||
return;
|
||||
}
|
||||
if (jobFromDb && !doesJobBelongsToUser(jobFromDb, request)) {
|
||||
return reply.code(403).send({ error: 'You are trying to change a job that is not associated to your user.' });
|
||||
}
|
||||
|
||||
if (!doesJobBelongsToUser(job, req)) {
|
||||
res.send(new Error('You are trying to remove a job that is not associated to your user'));
|
||||
} else {
|
||||
jobStorage.removeJob(jobId);
|
||||
}
|
||||
} catch (error) {
|
||||
res.send(new Error(error));
|
||||
logger.error(error);
|
||||
}
|
||||
res.send();
|
||||
});
|
||||
jobRouter.put('/:jobId/status', async (req, res) => {
|
||||
const { status } = req.body;
|
||||
const { jobId } = req.params;
|
||||
const settings = await getSettings();
|
||||
try {
|
||||
const job = jobStorage.getJob(jobId);
|
||||
if (settings.demoMode && !isAdmin(request) && jobFromDb && jobFromDb.name === DEMO_JOB_NAME) {
|
||||
return reply.code(403).send({ error: 'Sorry, but you cannot change the Status of our Demo Job ;)' });
|
||||
}
|
||||
|
||||
if (settings.demoMode && !isAdmin(req) && job.name === DEMO_JOB_NAME) {
|
||||
res.send(new Error('Sorry, but you cannot change the Status of our Demo Job ;)'));
|
||||
return;
|
||||
}
|
||||
|
||||
if (!doesJobBelongsToUser(job, req)) {
|
||||
res.send(new Error('You are trying change a job that is not associated to your user'));
|
||||
} else {
|
||||
jobStorage.setJobStatus({
|
||||
jobStorage.upsertJob({
|
||||
userId: request.session.currentUser,
|
||||
jobId,
|
||||
status,
|
||||
enabled,
|
||||
name,
|
||||
blacklist,
|
||||
provider,
|
||||
notificationAdapter,
|
||||
shareWithUsers,
|
||||
spatialFilter,
|
||||
specFilter,
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error(error);
|
||||
return reply.code(500).send({ error: error.message });
|
||||
}
|
||||
} catch (error) {
|
||||
res.send(new Error(error));
|
||||
logger.error(error);
|
||||
}
|
||||
res.send();
|
||||
});
|
||||
return reply.send();
|
||||
});
|
||||
|
||||
jobRouter.get('/shareableUserList', async (req, res) => {
|
||||
const currentUser = req.session.currentUser;
|
||||
const users = userStorage.getUsers(false);
|
||||
res.body = users
|
||||
.filter((user) => !user.isAdmin && user.id !== currentUser)
|
||||
.map((user) => ({
|
||||
id: user.id,
|
||||
name: user.username,
|
||||
}));
|
||||
res.send();
|
||||
});
|
||||
export { jobRouter };
|
||||
fastify.delete('/', async (request, reply) => {
|
||||
const { jobId } = request.body;
|
||||
const settings = await getSettings();
|
||||
try {
|
||||
const job = jobStorage.getJob(jobId);
|
||||
if (settings.demoMode && !isAdmin(request) && job.name === DEMO_JOB_NAME) {
|
||||
return reply.code(403).send({ error: 'Sorry, but you cannot remove the Demo Job ;)' });
|
||||
}
|
||||
|
||||
if (!doesJobBelongsToUser(job, request)) {
|
||||
return reply.code(403).send({ error: 'You are trying to remove a job that is not associated to your user' });
|
||||
}
|
||||
jobStorage.removeJob(jobId);
|
||||
} catch (error) {
|
||||
logger.error(error);
|
||||
return reply.code(500).send({ error: error.message });
|
||||
}
|
||||
return reply.send();
|
||||
});
|
||||
|
||||
fastify.put('/:jobId/status', async (request, reply) => {
|
||||
const { status } = request.body;
|
||||
const { jobId } = request.params;
|
||||
const settings = await getSettings();
|
||||
try {
|
||||
const job = jobStorage.getJob(jobId);
|
||||
|
||||
if (settings.demoMode && !isAdmin(request) && job.name === DEMO_JOB_NAME) {
|
||||
return reply.code(403).send({ error: 'Sorry, but you cannot change the Status of our Demo Job ;)' });
|
||||
}
|
||||
|
||||
if (!doesJobBelongsToUser(job, request)) {
|
||||
return reply.code(403).send({ error: 'You are trying change a job that is not associated to your user' });
|
||||
}
|
||||
jobStorage.setJobStatus({ jobId, status });
|
||||
} catch (error) {
|
||||
logger.error(error);
|
||||
return reply.code(500).send({ error: error.message });
|
||||
}
|
||||
return reply.send();
|
||||
});
|
||||
|
||||
fastify.get('/shareableUserList', async (request) => {
|
||||
const currentUser = request.session.currentUser;
|
||||
const users = userStorage.getUsers(false);
|
||||
return users
|
||||
.filter((user) => !user.isAdmin && user.id !== currentUser)
|
||||
.map((user) => ({
|
||||
id: user.id,
|
||||
name: user.username,
|
||||
}));
|
||||
});
|
||||
}
|
||||
|
||||
@@ -3,7 +3,6 @@
|
||||
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
|
||||
*/
|
||||
|
||||
import restana from 'restana';
|
||||
import * as listingStorage from '../../services/storage/listingsStorage.js';
|
||||
import * as watchListStorage from '../../services/storage/watchListStorage.js';
|
||||
import { isAdmin as isAdminFn } from '../security.js';
|
||||
@@ -12,128 +11,114 @@ import { nullOrEmpty } from '../../utils.js';
|
||||
import { getJobs } from '../../services/storage/jobStorage.js';
|
||||
import { getSettings } from '../../services/storage/settingsStorage.js';
|
||||
|
||||
const service = restana();
|
||||
/**
|
||||
* @param {import('fastify').FastifyInstance} fastify
|
||||
*/
|
||||
export default async function listingsPlugin(fastify) {
|
||||
fastify.get('/table', async (request) => {
|
||||
const {
|
||||
page,
|
||||
pageSize = 50,
|
||||
activityFilter,
|
||||
jobNameFilter,
|
||||
providerFilter,
|
||||
watchListFilter,
|
||||
sortfield = null,
|
||||
sortdir = 'asc',
|
||||
freeTextFilter,
|
||||
} = request.query || {};
|
||||
|
||||
const listingsRouter = service.newRouter();
|
||||
const toBool = (v) => {
|
||||
if (v === true || v === 'true' || v === 1 || v === '1') return true;
|
||||
if (v === false || v === 'false' || v === 0 || v === '0') return false;
|
||||
return null;
|
||||
};
|
||||
const normalizedActivity = toBool(activityFilter);
|
||||
const normalizedWatch = toBool(watchListFilter);
|
||||
|
||||
listingsRouter.get('/table', async (req, res) => {
|
||||
const {
|
||||
page,
|
||||
pageSize = 50,
|
||||
activityFilter,
|
||||
jobNameFilter,
|
||||
providerFilter,
|
||||
watchListFilter,
|
||||
sortfield = null,
|
||||
sortdir = 'asc',
|
||||
freeTextFilter,
|
||||
} = req.query || {};
|
||||
let jobFilter = null;
|
||||
let jobIdFilter = null;
|
||||
const jobs = getJobs();
|
||||
if (!nullOrEmpty(jobNameFilter)) {
|
||||
const job = jobs.find((j) => j.id === jobNameFilter);
|
||||
jobFilter = job != null ? job.name : null;
|
||||
jobIdFilter = job != null ? job.id : null;
|
||||
}
|
||||
|
||||
// normalize booleans (accept true, 'true', 1, '1' for true; false, 'false', 0, '0' for false)
|
||||
const toBool = (v) => {
|
||||
if (v === true || v === 'true' || v === 1 || v === '1') return true;
|
||||
if (v === false || v === 'false' || v === 0 || v === '0') return false;
|
||||
return null;
|
||||
};
|
||||
const normalizedActivity = toBool(activityFilter);
|
||||
const normalizedWatch = toBool(watchListFilter);
|
||||
|
||||
let jobFilter = null;
|
||||
let jobIdFilter = null;
|
||||
const jobs = getJobs();
|
||||
if (!nullOrEmpty(jobNameFilter)) {
|
||||
const job = jobs.find((j) => j.id === jobNameFilter);
|
||||
jobFilter = job != null ? job.name : null;
|
||||
jobIdFilter = job != null ? job.id : null;
|
||||
}
|
||||
|
||||
res.body = listingStorage.queryListings({
|
||||
page: page ? parseInt(page, 10) : 1,
|
||||
pageSize: pageSize ? parseInt(pageSize, 10) : 50,
|
||||
freeTextFilter: freeTextFilter || null,
|
||||
activityFilter: normalizedActivity,
|
||||
jobNameFilter: jobFilter,
|
||||
jobIdFilter: jobIdFilter,
|
||||
providerFilter,
|
||||
watchListFilter: normalizedWatch,
|
||||
sortField: sortfield || null,
|
||||
sortDir: sortdir === 'desc' ? 'desc' : 'asc',
|
||||
userId: req.session.currentUser,
|
||||
isAdmin: isAdminFn(req),
|
||||
return listingStorage.queryListings({
|
||||
page: page ? parseInt(page, 10) : 1,
|
||||
pageSize: pageSize ? parseInt(pageSize, 10) : 50,
|
||||
freeTextFilter: freeTextFilter || null,
|
||||
activityFilter: normalizedActivity,
|
||||
jobNameFilter: jobFilter,
|
||||
jobIdFilter: jobIdFilter,
|
||||
providerFilter,
|
||||
watchListFilter: normalizedWatch,
|
||||
sortField: sortfield || null,
|
||||
sortDir: sortdir === 'desc' ? 'desc' : 'asc',
|
||||
userId: request.session.currentUser,
|
||||
isAdmin: isAdminFn(request),
|
||||
});
|
||||
});
|
||||
res.send();
|
||||
});
|
||||
|
||||
listingsRouter.get('/map', async (req, res) => {
|
||||
const { jobId } = req.query || {};
|
||||
|
||||
res.body = listingStorage.getListingsForMap({
|
||||
jobId: nullOrEmpty(jobId) ? null : jobId,
|
||||
userId: req.session.currentUser,
|
||||
isAdmin: isAdminFn(req),
|
||||
fastify.get('/map', async (request) => {
|
||||
const { jobId } = request.query || {};
|
||||
return listingStorage.getListingsForMap({
|
||||
jobId: nullOrEmpty(jobId) ? null : jobId,
|
||||
userId: request.session.currentUser,
|
||||
isAdmin: isAdminFn(request),
|
||||
});
|
||||
});
|
||||
res.send();
|
||||
});
|
||||
|
||||
listingsRouter.get('/:listingId', async (req, res) => {
|
||||
const { listingId } = req.params;
|
||||
const listing = listingStorage.getListingById(listingId, req.session.currentUser, isAdminFn(req));
|
||||
if (!listing) {
|
||||
res.statusCode = 404;
|
||||
res.body = { message: 'Listing not found' };
|
||||
return res.send();
|
||||
}
|
||||
res.body = listing;
|
||||
res.send();
|
||||
});
|
||||
|
||||
// Toggle watch state for the current user on a listing
|
||||
listingsRouter.post('/watch', async (req, res) => {
|
||||
try {
|
||||
const { listingId } = req.body || {};
|
||||
const userId = req.session?.currentUser;
|
||||
if (!listingId || !userId) {
|
||||
res.statusCode = 400;
|
||||
res.body = { message: 'listingId or user not provided' };
|
||||
return res.send();
|
||||
fastify.get('/:listingId', async (request, reply) => {
|
||||
const { listingId } = request.params;
|
||||
const listing = listingStorage.getListingById(listingId, request.session.currentUser, isAdminFn(request));
|
||||
if (!listing) {
|
||||
return reply.code(404).send({ message: 'Listing not found' });
|
||||
}
|
||||
watchListStorage.toggleWatch(listingId, userId);
|
||||
} catch (error) {
|
||||
logger.error(error);
|
||||
res.statusCode = 500;
|
||||
res.body = { message: 'Failed to toggle watch' };
|
||||
}
|
||||
res.send();
|
||||
});
|
||||
return listing;
|
||||
});
|
||||
|
||||
listingsRouter.delete('/job', async (req, res) => {
|
||||
const { jobId, hardDelete = false } = req.body;
|
||||
const settings = await getSettings();
|
||||
try {
|
||||
if (settings.demoMode && !isAdminFn(req)) {
|
||||
res.send(new Error('Sorry, but you cannot remove listings in demo mode ;)'));
|
||||
return;
|
||||
fastify.post('/watch', async (request, reply) => {
|
||||
try {
|
||||
const { listingId } = request.body || {};
|
||||
const userId = request.session?.currentUser;
|
||||
if (!listingId || !userId) {
|
||||
return reply.code(400).send({ message: 'listingId or user not provided' });
|
||||
}
|
||||
watchListStorage.toggleWatch(listingId, userId);
|
||||
} catch (error) {
|
||||
logger.error(error);
|
||||
return reply.code(500).send({ message: 'Failed to toggle watch' });
|
||||
}
|
||||
return reply.send();
|
||||
});
|
||||
|
||||
listingStorage.deleteListingsByJobId(jobId, hardDelete);
|
||||
} catch (error) {
|
||||
res.send(new Error(error));
|
||||
logger.error(error);
|
||||
}
|
||||
res.send();
|
||||
});
|
||||
|
||||
listingsRouter.delete('/', async (req, res) => {
|
||||
const { ids, hardDelete = false } = req.body;
|
||||
try {
|
||||
if (Array.isArray(ids) && ids.length > 0) {
|
||||
listingStorage.deleteListingsById(ids, hardDelete);
|
||||
fastify.delete('/job', async (request, reply) => {
|
||||
const { jobId, hardDelete = false } = request.body;
|
||||
const settings = await getSettings();
|
||||
try {
|
||||
if (settings.demoMode && !isAdminFn(request)) {
|
||||
return reply.code(403).send({ error: 'Sorry, but you cannot remove listings in demo mode ;)' });
|
||||
}
|
||||
listingStorage.deleteListingsByJobId(jobId, hardDelete);
|
||||
} catch (error) {
|
||||
logger.error(error);
|
||||
return reply.code(500).send({ error: error.message });
|
||||
}
|
||||
} catch (error) {
|
||||
res.send(new Error(error));
|
||||
logger.error(error);
|
||||
}
|
||||
res.send();
|
||||
});
|
||||
return reply.send();
|
||||
});
|
||||
|
||||
export { listingsRouter };
|
||||
fastify.delete('/', async (request, reply) => {
|
||||
const { ids, hardDelete = false } = request.body;
|
||||
try {
|
||||
if (Array.isArray(ids) && ids.length > 0) {
|
||||
listingStorage.deleteListingsById(ids, hardDelete);
|
||||
}
|
||||
} catch (error) {
|
||||
logger.error(error);
|
||||
return reply.code(500).send({ error: error.message });
|
||||
}
|
||||
return reply.send();
|
||||
});
|
||||
}
|
||||
|
||||
@@ -3,7 +3,6 @@
|
||||
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
|
||||
*/
|
||||
|
||||
import restana from 'restana';
|
||||
import * as userStorage from '../../services/storage/userStorage.js';
|
||||
import * as hasher from '../../services/security/hash.js';
|
||||
import { trackDemoAccessed } from '../../services/tracking/Tracker.js';
|
||||
@@ -11,12 +10,12 @@ import logger from '../../services/logger.js';
|
||||
import { getSettings } from '../../services/storage/settingsStorage.js';
|
||||
|
||||
const MAX_LOGIN_ATTEMPTS = 10;
|
||||
const LOGIN_WINDOW_MS = 15 * 60 * 1000; // 15 minutes
|
||||
const loginAttempts = new Map(); // ip -> { count, firstAttempt }
|
||||
const LOGIN_WINDOW_MS = 15 * 60 * 1000;
|
||||
const loginAttempts = new Map();
|
||||
|
||||
function getClientIp(req) {
|
||||
const forwarded = req.headers['x-forwarded-for'];
|
||||
return (forwarded ? forwarded.split(',')[0] : req.socket?.remoteAddress) || 'unknown';
|
||||
function getClientIp(request) {
|
||||
const forwarded = request.headers['x-forwarded-for'];
|
||||
return (forwarded ? forwarded.split(',')[0] : request.socket?.remoteAddress) || 'unknown';
|
||||
}
|
||||
|
||||
function isRateLimited(ip) {
|
||||
@@ -30,53 +29,51 @@ function isRateLimited(ip) {
|
||||
return record.count > MAX_LOGIN_ATTEMPTS;
|
||||
}
|
||||
|
||||
const service = restana();
|
||||
const loginRouter = service.newRouter();
|
||||
loginRouter.get('/user', async (req, res) => {
|
||||
const currentUserId = req.session.currentUser;
|
||||
const currentUser = currentUserId == null ? null : userStorage.getUser(currentUserId);
|
||||
if (currentUser == null) {
|
||||
res.body = {};
|
||||
} else {
|
||||
res.body = {
|
||||
/**
|
||||
* @param {import('fastify').FastifyInstance} fastify
|
||||
*/
|
||||
export default async function loginPlugin(fastify) {
|
||||
fastify.get('/user', async (request) => {
|
||||
const currentUserId = request.session?.currentUser;
|
||||
const currentUser = currentUserId == null ? null : userStorage.getUser(currentUserId);
|
||||
if (currentUser == null) {
|
||||
return {};
|
||||
}
|
||||
return {
|
||||
userId: currentUser.id,
|
||||
isAdmin: currentUser.isAdmin,
|
||||
};
|
||||
}
|
||||
res.send();
|
||||
});
|
||||
loginRouter.post('/', async (req, res) => {
|
||||
const ip = getClientIp(req);
|
||||
if (isRateLimited(ip)) {
|
||||
logger.error(`Login rate limit exceeded for IP ${ip}`);
|
||||
res.send(429);
|
||||
return;
|
||||
}
|
||||
const settings = await getSettings();
|
||||
const { username, password } = req.body;
|
||||
const user = userStorage.getUsers(true).find((user) => user.username === username);
|
||||
if (user == null) {
|
||||
res.send(401);
|
||||
return;
|
||||
}
|
||||
if (user.password === hasher.hash(password)) {
|
||||
if (settings.demoMode) {
|
||||
await trackDemoAccessed();
|
||||
}
|
||||
});
|
||||
|
||||
req.session.currentUser = user.id;
|
||||
req.session.createdAt = Date.now();
|
||||
loginAttempts.delete(ip);
|
||||
userStorage.setLastLoginToNow({ userId: user.id });
|
||||
res.send(200);
|
||||
return;
|
||||
} else {
|
||||
logger.error(`User ${username} tried to login, but password was wrong.`);
|
||||
}
|
||||
res.send(401);
|
||||
});
|
||||
loginRouter.post('/logout', async (req, res) => {
|
||||
req.session = null;
|
||||
res.send(200);
|
||||
});
|
||||
export { loginRouter };
|
||||
fastify.post('/', async (request, reply) => {
|
||||
const ip = getClientIp(request);
|
||||
if (isRateLimited(ip)) {
|
||||
logger.error(`Login rate limit exceeded for IP ${ip}`);
|
||||
return reply.code(429).send();
|
||||
}
|
||||
const settings = await getSettings();
|
||||
const { username, password } = request.body;
|
||||
const user = userStorage.getUsers(true).find((u) => u.username === username);
|
||||
if (user == null) {
|
||||
return reply.code(401).send();
|
||||
}
|
||||
if (user.password === hasher.hash(password)) {
|
||||
if (settings.demoMode) {
|
||||
await trackDemoAccessed();
|
||||
}
|
||||
request.session.currentUser = user.id;
|
||||
request.session.createdAt = Date.now();
|
||||
loginAttempts.delete(ip);
|
||||
userStorage.setLastLoginToNow({ userId: user.id });
|
||||
return reply.code(200).send();
|
||||
} else {
|
||||
logger.error(`User ${username} tried to login, but password was wrong.`);
|
||||
}
|
||||
return reply.code(401).send();
|
||||
});
|
||||
|
||||
fastify.post('/logout', async (request, reply) => {
|
||||
await request.session.destroy();
|
||||
return reply.code(200).send();
|
||||
});
|
||||
}
|
||||
|
||||
@@ -4,62 +4,64 @@
|
||||
*/
|
||||
|
||||
import fs from 'fs';
|
||||
import restana from 'restana';
|
||||
import logger from '../../services/logger.js';
|
||||
|
||||
const service = restana();
|
||||
const notificationAdapterRouter = service.newRouter();
|
||||
const notificationAdapterList = fs.readdirSync('./lib//notification/adapter').filter((file) => file.endsWith('.js'));
|
||||
const notificationAdapter = await Promise.all(
|
||||
notificationAdapterList.map(async (pro) => {
|
||||
return await import(`../../notification/adapter/${pro}`);
|
||||
}),
|
||||
);
|
||||
notificationAdapterRouter.post('/try', async (req, res) => {
|
||||
const { id, fields } = req.body;
|
||||
const adapter = notificationAdapter.find((adapter) => adapter.config.id === id);
|
||||
if (adapter == null) {
|
||||
res.send(404);
|
||||
}
|
||||
const notificationConfig = [];
|
||||
const notificationObject = {};
|
||||
Object.keys(fields).forEach((key) => {
|
||||
notificationObject[key] = fields[key].value;
|
||||
|
||||
/**
|
||||
* @param {import('fastify').FastifyInstance} fastify
|
||||
*/
|
||||
export default async function notificationAdapterPlugin(fastify) {
|
||||
fastify.get('/', async () => {
|
||||
return notificationAdapter.map((adapter) => adapter.config);
|
||||
});
|
||||
notificationConfig.push({
|
||||
fields: { ...notificationObject },
|
||||
enabled: true,
|
||||
id,
|
||||
});
|
||||
try {
|
||||
await adapter.send({
|
||||
serviceName: 'TestCall',
|
||||
newListings: [
|
||||
{
|
||||
address: 'Heidestrasse 17, 51147 Köln',
|
||||
description: exampleDescription,
|
||||
id: '1',
|
||||
imageUrl: 'https://placehold.co/600x400/png',
|
||||
price: '1.000 €',
|
||||
size: '76 m²',
|
||||
title: 'Stilvolle gepflegte 3-Raum-Wohnung mit gehobener Innenausstattung',
|
||||
url: 'https://www.orange-coding.net',
|
||||
},
|
||||
],
|
||||
notificationConfig,
|
||||
jobKey: 'TestJob',
|
||||
|
||||
fastify.post('/try', async (request, reply) => {
|
||||
const { id, fields } = request.body;
|
||||
const adapter = notificationAdapter.find((adapter) => adapter.config.id === id);
|
||||
if (adapter == null) {
|
||||
return reply.code(404).send();
|
||||
}
|
||||
const notificationConfig = [];
|
||||
const notificationObject = {};
|
||||
Object.keys(fields).forEach((key) => {
|
||||
notificationObject[key] = fields[key].value;
|
||||
});
|
||||
res.send();
|
||||
} catch (Exception) {
|
||||
logger.error('Error during notification adapter test:', Exception);
|
||||
res.send(new Error(Exception));
|
||||
}
|
||||
});
|
||||
notificationAdapterRouter.get('/', async (req, res) => {
|
||||
res.body = notificationAdapter.map((adapter) => adapter.config);
|
||||
res.send();
|
||||
});
|
||||
export { notificationAdapterRouter };
|
||||
notificationConfig.push({
|
||||
fields: { ...notificationObject },
|
||||
enabled: true,
|
||||
id,
|
||||
});
|
||||
try {
|
||||
await adapter.send({
|
||||
serviceName: 'TestCall',
|
||||
newListings: [
|
||||
{
|
||||
address: 'Heidestrasse 17, 51147 Köln',
|
||||
description: exampleDescription,
|
||||
id: '1',
|
||||
imageUrl: 'https://placehold.co/600x400/png',
|
||||
price: '1.000 €',
|
||||
size: '76 m²',
|
||||
title: 'Stilvolle gepflegte 3-Raum-Wohnung mit gehobener Innenausstattung',
|
||||
url: 'https://www.orange-coding.net',
|
||||
},
|
||||
],
|
||||
notificationConfig,
|
||||
jobKey: 'TestJob',
|
||||
});
|
||||
return reply.send();
|
||||
} catch (Exception) {
|
||||
logger.error('Error during notification adapter test:', Exception);
|
||||
return reply.code(500).send({ error: String(Exception) });
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
const exampleDescription = `
|
||||
Wohnungstyp: Etagenwohnung
|
||||
@@ -94,7 +96,7 @@ Die Wohnung ist ideal für Paare oder kleine Familien geeignet.
|
||||
Ausstattung:
|
||||
- neuer hochwertiger Bodenbelag (Holzoptik) in allen Räumen außer Bad/Küche
|
||||
- sonniger Balkon (Süd)
|
||||
- Tiefgaragenstellplatz
|
||||
- Tiefgaragenstellplatz
|
||||
- Kellerabteil
|
||||
- gepflegtes Mehrfamilienhaus
|
||||
|
||||
@@ -104,7 +106,7 @@ Vermietung direkt vom Eigentümer - provisionsfrei!
|
||||
|
||||
Lage:
|
||||
• Park: 1 Minute zu Fuß
|
||||
• S-Bahn Station: 2 Minuten zu Fuß
|
||||
• S-Bahn Station: 2 Minuten zu Fuß
|
||||
• Supermärkte, Restaurants, täglicher Bedarf in der Nähe
|
||||
• Gute Anbindung Richtung Großstadt und Flughafen
|
||||
`;
|
||||
|
||||
@@ -4,17 +4,15 @@
|
||||
*/
|
||||
|
||||
import fs from 'fs';
|
||||
import restana from 'restana';
|
||||
const service = restana();
|
||||
const providerRouter = service.newRouter();
|
||||
|
||||
const providerList = fs.readdirSync('./lib/provider').filter((file) => file.endsWith('.js'));
|
||||
const provider = await Promise.all(
|
||||
providerList.map(async (pro) => {
|
||||
return await import(`../../provider/${pro}`);
|
||||
}),
|
||||
);
|
||||
providerRouter.get('/', async (req, res) => {
|
||||
res.body = provider.map((p) => p.metaInformation);
|
||||
res.send();
|
||||
});
|
||||
export { providerRouter };
|
||||
const providers = await Promise.all(providerList.map(async (pro) => import(`../../provider/${pro}`)));
|
||||
|
||||
/**
|
||||
* @param {import('fastify').FastifyInstance} fastify
|
||||
*/
|
||||
export default async function providerPlugin(fastify) {
|
||||
fastify.get('/', async () => {
|
||||
return providers.map((p) => p.metaInformation);
|
||||
});
|
||||
}
|
||||
|
||||
@@ -3,35 +3,29 @@
|
||||
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
|
||||
*/
|
||||
|
||||
import restana from 'restana';
|
||||
import { trackPoi } from '../../services/tracking/Tracker.js';
|
||||
import { TRACKING_POIS } from '../../TRACKING_POIS.js';
|
||||
import logger from '../../services/logger.js';
|
||||
|
||||
const service = restana();
|
||||
const trackingRouter = service.newRouter();
|
||||
/**
|
||||
* @param {import('fastify').FastifyInstance} fastify
|
||||
*/
|
||||
export default async function trackingPlugin(fastify) {
|
||||
fastify.get('/trackingPois', async () => {
|
||||
return TRACKING_POIS;
|
||||
});
|
||||
|
||||
trackingRouter.get('/trackingPois', async (req, res) => {
|
||||
res.body = TRACKING_POIS;
|
||||
res.send();
|
||||
});
|
||||
|
||||
trackingRouter.post('/poi', async (req, res) => {
|
||||
const { poi } = req.body;
|
||||
if (!poi) {
|
||||
res.statusCode = 400;
|
||||
res.send({ error: 'Feature name is required' });
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
await trackPoi(poi);
|
||||
res.send({ success: true });
|
||||
} catch (error) {
|
||||
logger.error('Error tracking feature', error);
|
||||
res.statusCode = 500;
|
||||
res.send({ error: error.message });
|
||||
}
|
||||
});
|
||||
|
||||
export { trackingRouter };
|
||||
fastify.post('/poi', async (request, reply) => {
|
||||
const { poi } = request.body;
|
||||
if (!poi) {
|
||||
return reply.code(400).send({ error: 'Feature name is required' });
|
||||
}
|
||||
try {
|
||||
await trackPoi(poi);
|
||||
return { success: true };
|
||||
} catch (error) {
|
||||
logger.error('Error tracking feature', error);
|
||||
return reply.code(500).send({ error: error.message });
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
@@ -3,82 +3,73 @@
|
||||
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
|
||||
*/
|
||||
|
||||
import restana from 'restana';
|
||||
import * as userStorage from '../../services/storage/userStorage.js';
|
||||
import * as jobStorage from '../../services/storage/jobStorage.js';
|
||||
import { getSettings } from '../../services/storage/settingsStorage.js';
|
||||
import { isAdmin as isAdminUser } from '../security.js';
|
||||
const service = restana();
|
||||
const userRouter = service.newRouter();
|
||||
|
||||
function checkIfAnyAdminAfterRemovingUser(userIdToBeRemoved, allUser) {
|
||||
return allUser.filter((user) => user.id !== userIdToBeRemoved && user.isAdmin).length > 0;
|
||||
}
|
||||
function checkIfUserToBeRemovedIsLoggedIn(userIdToBeRemoved, req) {
|
||||
return req.session.currentUser === userIdToBeRemoved;
|
||||
|
||||
function checkIfUserToBeRemovedIsLoggedIn(userIdToBeRemoved, request) {
|
||||
return request.session.currentUser === userIdToBeRemoved;
|
||||
}
|
||||
|
||||
const nullOrEmpty = (str) => str == null || str.length === 0;
|
||||
|
||||
userRouter.get('/', async (req, res) => {
|
||||
res.body = userStorage.getUsers(false);
|
||||
res.send();
|
||||
});
|
||||
|
||||
userRouter.get('/:userId', async (req, res) => {
|
||||
const { userId } = req.params;
|
||||
res.body = userStorage.getUser(userId);
|
||||
res.send();
|
||||
});
|
||||
userRouter.delete('/', async (req, res) => {
|
||||
const settings = await getSettings();
|
||||
if (settings.demoMode && !isAdminUser(req)) {
|
||||
res.send(new Error('In demo mode, it is not allowed to remove user.'));
|
||||
return;
|
||||
}
|
||||
|
||||
const { userId } = req.body;
|
||||
const allUser = userStorage.getUsers(false);
|
||||
if (!checkIfAnyAdminAfterRemovingUser(userId, allUser)) {
|
||||
res.send(new Error('You are trying to remove the last admin user. This is prohibited.'));
|
||||
return;
|
||||
}
|
||||
if (checkIfUserToBeRemovedIsLoggedIn(userId, req)) {
|
||||
res.send(new Error('You are trying to remove yourself. This is prohibited.'));
|
||||
return;
|
||||
}
|
||||
//TODO: Remove also analytics
|
||||
jobStorage.removeJobsByUserId(userId);
|
||||
userStorage.removeUser(userId);
|
||||
res.send();
|
||||
});
|
||||
userRouter.post('/', async (req, res) => {
|
||||
const settings = await getSettings();
|
||||
if (settings.demoMode && !isAdminUser(req)) {
|
||||
res.send(new Error('In demo mode, it is not allowed to change or add user.'));
|
||||
return;
|
||||
}
|
||||
|
||||
const { username, password, password2, isAdmin, userId } = req.body;
|
||||
if (password !== password2) {
|
||||
res.send(new Error('Passwords does not match'));
|
||||
return;
|
||||
}
|
||||
if (nullOrEmpty(username) || nullOrEmpty(password) || nullOrEmpty(password2)) {
|
||||
res.send(new Error('Username and password are mandatory.'));
|
||||
return;
|
||||
}
|
||||
const allUser = userStorage.getUsers(false);
|
||||
if (!isAdmin && !checkIfAnyAdminAfterRemovingUser(userId, allUser)) {
|
||||
res.send(
|
||||
new Error('You cannot change the admin flag for this user as otherwise, there is no other user in the system'),
|
||||
);
|
||||
return;
|
||||
}
|
||||
userStorage.upsertUser({
|
||||
userId,
|
||||
username,
|
||||
password,
|
||||
isAdmin,
|
||||
/**
|
||||
* @param {import('fastify').FastifyInstance} fastify
|
||||
*/
|
||||
export default async function userPlugin(fastify) {
|
||||
fastify.get('/', async () => {
|
||||
return userStorage.getUsers(false);
|
||||
});
|
||||
res.send();
|
||||
});
|
||||
export { userRouter };
|
||||
|
||||
fastify.get('/:userId', async (request) => {
|
||||
const { userId } = request.params;
|
||||
return userStorage.getUser(userId);
|
||||
});
|
||||
|
||||
fastify.delete('/', async (request, reply) => {
|
||||
const settings = await getSettings();
|
||||
if (settings.demoMode && !isAdminUser(request)) {
|
||||
return reply.code(403).send({ error: 'In demo mode, it is not allowed to remove user.' });
|
||||
}
|
||||
|
||||
const { userId } = request.body;
|
||||
const allUser = userStorage.getUsers(false);
|
||||
if (!checkIfAnyAdminAfterRemovingUser(userId, allUser)) {
|
||||
return reply.code(400).send({ error: 'You are trying to remove the last admin user. This is prohibited.' });
|
||||
}
|
||||
if (checkIfUserToBeRemovedIsLoggedIn(userId, request)) {
|
||||
return reply.code(400).send({ error: 'You are trying to remove yourself. This is prohibited.' });
|
||||
}
|
||||
jobStorage.removeJobsByUserId(userId);
|
||||
userStorage.removeUser(userId);
|
||||
return reply.send();
|
||||
});
|
||||
|
||||
fastify.post('/', async (request, reply) => {
|
||||
const settings = await getSettings();
|
||||
if (settings.demoMode && !isAdminUser(request)) {
|
||||
return reply.code(403).send({ error: 'In demo mode, it is not allowed to change or add user.' });
|
||||
}
|
||||
|
||||
const { username, password, password2, isAdmin, userId } = request.body;
|
||||
if (password !== password2) {
|
||||
return reply.code(400).send({ error: 'Passwords do not match.' });
|
||||
}
|
||||
if (nullOrEmpty(username) || nullOrEmpty(password) || nullOrEmpty(password2)) {
|
||||
return reply.code(400).send({ error: 'Username and password are mandatory.' });
|
||||
}
|
||||
const allUser = userStorage.getUsers(false);
|
||||
if (!isAdmin && !checkIfAnyAdminAfterRemovingUser(userId, allUser)) {
|
||||
return reply.code(400).send({
|
||||
error: 'You cannot change the admin flag for this user as otherwise, there is no other user in the system',
|
||||
});
|
||||
}
|
||||
userStorage.upsertUser({ userId, username, password, isAdmin });
|
||||
return reply.send();
|
||||
});
|
||||
}
|
||||
|
||||
@@ -3,7 +3,6 @@
|
||||
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
|
||||
*/
|
||||
|
||||
import restana from 'restana';
|
||||
import SqliteConnection from '../../services/storage/SqliteConnection.js';
|
||||
import { getSettings, upsertSettings } from '../../services/storage/settingsStorage.js';
|
||||
import { isAdmin } from '../security.js';
|
||||
@@ -16,113 +15,140 @@ import { TRACKING_POIS } from '../../TRACKING_POIS.js';
|
||||
import logger from '../../services/logger.js';
|
||||
import { runGeoCordTask } from '../../services/crons/geocoding-cron.js';
|
||||
|
||||
const service = restana();
|
||||
const userSettingsRouter = service.newRouter();
|
||||
|
||||
userSettingsRouter.get('/', async (req, res) => {
|
||||
const userId = req.session.currentUser;
|
||||
const rows = SqliteConnection.query('SELECT name, value FROM settings WHERE user_id = @userId', { userId });
|
||||
const settings = {};
|
||||
for (const r of rows) {
|
||||
settings[r.name] = fromJson(r.value, null);
|
||||
}
|
||||
res.body = settings;
|
||||
res.send();
|
||||
});
|
||||
|
||||
userSettingsRouter.get('/autocomplete', async (req, res) => {
|
||||
const { q } = req.query;
|
||||
try {
|
||||
const results = await autocompleteAddress(q);
|
||||
res.body = results;
|
||||
res.send();
|
||||
} catch (error) {
|
||||
res.statusCode = 500;
|
||||
res.send({ error: error.message });
|
||||
}
|
||||
});
|
||||
|
||||
userSettingsRouter.post('/home-address', async (req, res) => {
|
||||
const userId = req.session.currentUser;
|
||||
const { home_address } = req.body;
|
||||
const settings = await getSettings();
|
||||
|
||||
if (settings.demoMode && !isAdmin(req)) {
|
||||
res.send(new Error('In demo mode, it is not allowed to change the home address.'));
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
if (home_address) {
|
||||
await trackPoi(TRACKING_POIS.DISTANCE_ADDRESS_ENTERED);
|
||||
const coords = await geocodeAddress(home_address);
|
||||
if (coords && coords.lat !== -1) {
|
||||
upsertSettings({ home_address: { address: home_address, coords } }, userId);
|
||||
resetGeocoordinatesAndDistanceForUser(userId);
|
||||
//we do NOT wait for this to finish, as we don't want to block the response
|
||||
runGeoCordTask();
|
||||
res.send({ success: true, coords });
|
||||
} else {
|
||||
res.statusCode = 400;
|
||||
res.send({ error: 'Could not geocode address' });
|
||||
}
|
||||
} else {
|
||||
upsertSettings({ home_address: null }, userId);
|
||||
res.send({ success: true });
|
||||
/**
|
||||
* @param {import('fastify').FastifyInstance} fastify
|
||||
*/
|
||||
export default async function userSettingsPlugin(fastify) {
|
||||
fastify.get('/', async (request) => {
|
||||
const userId = request.session.currentUser;
|
||||
const rows = SqliteConnection.query('SELECT name, value FROM settings WHERE user_id = @userId', { userId });
|
||||
const settings = {};
|
||||
for (const r of rows) {
|
||||
settings[r.name] = fromJson(r.value, null);
|
||||
}
|
||||
} catch (error) {
|
||||
logger.error('Error updating home address settings', error);
|
||||
res.statusCode = 500;
|
||||
res.send({ error: error.message });
|
||||
}
|
||||
});
|
||||
return settings;
|
||||
});
|
||||
|
||||
userSettingsRouter.post('/news-hash', async (req, res) => {
|
||||
const userId = req.session.currentUser;
|
||||
const { news_hash } = req.body;
|
||||
fastify.get('/autocomplete', async (request, reply) => {
|
||||
const { q } = request.query;
|
||||
try {
|
||||
const results = await autocompleteAddress(q);
|
||||
return results;
|
||||
} catch (error) {
|
||||
return reply.code(500).send({ error: error.message });
|
||||
}
|
||||
});
|
||||
|
||||
const globalSettings = await getSettings();
|
||||
if (globalSettings.demoMode && !isAdmin(req)) {
|
||||
res.statusCode = 403;
|
||||
res.send({ error: 'In demo mode, it is not allowed to change settings.' });
|
||||
return;
|
||||
}
|
||||
fastify.post('/home-address', async (request, reply) => {
|
||||
const userId = request.session.currentUser;
|
||||
const { home_address } = request.body;
|
||||
const settings = await getSettings();
|
||||
|
||||
try {
|
||||
upsertSettings({ news_hash }, userId);
|
||||
res.send({ success: true });
|
||||
} catch (error) {
|
||||
logger.error('Error updating news hash', error);
|
||||
res.statusCode = 500;
|
||||
res.send({ error: error.message });
|
||||
}
|
||||
});
|
||||
if (settings.demoMode && !isAdmin(request)) {
|
||||
return reply.code(403).send({ error: 'In demo mode, it is not allowed to change the home address.' });
|
||||
}
|
||||
|
||||
userSettingsRouter.post('/provider-details', async (req, res) => {
|
||||
const userId = req.session.currentUser;
|
||||
const { provider_details } = req.body;
|
||||
try {
|
||||
if (home_address) {
|
||||
await trackPoi(TRACKING_POIS.DISTANCE_ADDRESS_ENTERED);
|
||||
const coords = await geocodeAddress(home_address);
|
||||
if (coords && coords.lat !== -1) {
|
||||
upsertSettings({ home_address: { address: home_address, coords } }, userId);
|
||||
resetGeocoordinatesAndDistanceForUser(userId);
|
||||
runGeoCordTask();
|
||||
return { success: true, coords };
|
||||
} else {
|
||||
return reply.code(400).send({ error: 'Could not geocode address' });
|
||||
}
|
||||
} else {
|
||||
upsertSettings({ home_address: null }, userId);
|
||||
return { success: true };
|
||||
}
|
||||
} catch (error) {
|
||||
logger.error('Error updating home address settings', error);
|
||||
return reply.code(500).send({ error: error.message });
|
||||
}
|
||||
});
|
||||
|
||||
const globalSettings = await getSettings();
|
||||
if (globalSettings.demoMode && !isAdmin(req)) {
|
||||
res.statusCode = 403;
|
||||
res.send({ error: 'In demo mode, it is not allowed to change settings.' });
|
||||
return;
|
||||
}
|
||||
fastify.post('/news-hash', async (request, reply) => {
|
||||
const userId = request.session.currentUser;
|
||||
const { news_hash } = request.body;
|
||||
|
||||
if (!Array.isArray(provider_details)) {
|
||||
res.statusCode = 400;
|
||||
res.send({ error: 'provider_details must be an array of provider ids.' });
|
||||
return;
|
||||
}
|
||||
const globalSettings = await getSettings();
|
||||
if (globalSettings.demoMode && !isAdmin(request)) {
|
||||
return reply.code(403).send({ error: 'In demo mode, it is not allowed to change settings.' });
|
||||
}
|
||||
|
||||
try {
|
||||
upsertSettings({ provider_details }, userId);
|
||||
res.send({ success: true });
|
||||
} catch (error) {
|
||||
logger.error('Error updating provider details setting', error);
|
||||
res.statusCode = 500;
|
||||
res.send({ error: error.message });
|
||||
}
|
||||
});
|
||||
try {
|
||||
upsertSettings({ news_hash }, userId);
|
||||
return { success: true };
|
||||
} catch (error) {
|
||||
logger.error('Error updating news hash', error);
|
||||
return reply.code(500).send({ error: error.message });
|
||||
}
|
||||
});
|
||||
|
||||
export { userSettingsRouter };
|
||||
fastify.post('/provider-details', async (request, reply) => {
|
||||
const userId = request.session.currentUser;
|
||||
const { provider_details } = request.body;
|
||||
|
||||
const globalSettings = await getSettings();
|
||||
if (globalSettings.demoMode && !isAdmin(request)) {
|
||||
return reply.code(403).send({ error: 'In demo mode, it is not allowed to change settings.' });
|
||||
}
|
||||
|
||||
if (!Array.isArray(provider_details)) {
|
||||
return reply.code(400).send({ error: 'provider_details must be an array of provider ids.' });
|
||||
}
|
||||
|
||||
try {
|
||||
upsertSettings({ provider_details }, userId);
|
||||
return { success: true };
|
||||
} catch (error) {
|
||||
logger.error('Error updating provider details setting', error);
|
||||
return reply.code(500).send({ error: error.message });
|
||||
}
|
||||
});
|
||||
|
||||
fastify.post('/listings-view-mode', async (request, reply) => {
|
||||
const userId = request.session.currentUser;
|
||||
const { listings_view_mode } = request.body;
|
||||
|
||||
if (listings_view_mode !== 'grid' && listings_view_mode !== 'table') {
|
||||
return reply.code(400).send({ error: 'listings_view_mode must be "grid" or "table".' });
|
||||
}
|
||||
|
||||
if (listings_view_mode === 'table') {
|
||||
await trackPoi(TRACKING_POIS.LISTING_TABLE_VIEW);
|
||||
}
|
||||
|
||||
try {
|
||||
upsertSettings({ listings_view_mode }, userId);
|
||||
return { success: true };
|
||||
} catch (error) {
|
||||
logger.error('Error updating listings view mode setting', error);
|
||||
return reply.code(500).send({ error: error.message });
|
||||
}
|
||||
});
|
||||
|
||||
fastify.post('/jobs-view-mode', async (request, reply) => {
|
||||
const userId = request.session.currentUser;
|
||||
const { jobs_view_mode } = request.body;
|
||||
|
||||
if (jobs_view_mode !== 'grid' && jobs_view_mode !== 'table') {
|
||||
return reply.code(400).send({ error: 'jobs_view_mode must be "grid" or "table".' });
|
||||
}
|
||||
|
||||
if (jobs_view_mode === 'table') {
|
||||
await trackPoi(TRACKING_POIS.JOBS_TABLE_VIEW);
|
||||
}
|
||||
|
||||
try {
|
||||
upsertSettings({ jobs_view_mode }, userId);
|
||||
return { success: true };
|
||||
} catch (error) {
|
||||
logger.error('Error updating jobs view mode setting', error);
|
||||
return reply.code(500).send({ error: error.message });
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
@@ -3,27 +3,10 @@
|
||||
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
|
||||
*/
|
||||
|
||||
import restana from 'restana';
|
||||
import fetch from 'node-fetch';
|
||||
import { getPackageVersion } from '../../utils.js';
|
||||
import semver from 'semver';
|
||||
|
||||
const service = restana();
|
||||
const versionRouter = service.newRouter();
|
||||
|
||||
versionRouter.get('/', async (req, res) => {
|
||||
const versionPayload = await getCurrentVersionFromGithub();
|
||||
const localFredyVersion = await getPackageVersion();
|
||||
res.body =
|
||||
versionPayload == null
|
||||
? {
|
||||
newVersion: false,
|
||||
localFredyVersion,
|
||||
}
|
||||
: versionPayload;
|
||||
res.send();
|
||||
});
|
||||
|
||||
async function getCurrentVersionFromGithub() {
|
||||
const raw = await fetch('https://api.github.com/repos/orangecoding/fredy/releases/latest');
|
||||
const data = await raw.json();
|
||||
@@ -40,4 +23,13 @@ async function getCurrentVersionFromGithub() {
|
||||
};
|
||||
}
|
||||
|
||||
export { versionRouter };
|
||||
/**
|
||||
* @param {import('fastify').FastifyInstance} fastify
|
||||
*/
|
||||
export default async function versionPlugin(fastify) {
|
||||
fastify.get('/', async () => {
|
||||
const versionPayload = await getCurrentVersionFromGithub();
|
||||
const localFredyVersion = await getPackageVersion();
|
||||
return versionPayload ?? { newVersion: false, localFredyVersion };
|
||||
});
|
||||
}
|
||||
|
||||
@@ -4,53 +4,50 @@
|
||||
*/
|
||||
|
||||
import * as userStorage from '../services/storage/userStorage.js';
|
||||
import cookieSession from 'cookie-session';
|
||||
|
||||
const SESSION_MAX_AGE = 2 * 60 * 60 * 1000; // 2 hours
|
||||
const unauthorized = (res) => {
|
||||
return res.send(401);
|
||||
};
|
||||
const isUnauthorized = (req) => {
|
||||
if (req.session.currentUser == null) return true;
|
||||
if (Date.now() - req.session.createdAt > SESSION_MAX_AGE) {
|
||||
req.session = null;
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns true when the request has no valid, non-expired session.
|
||||
* @param {import('fastify').FastifyRequest} request
|
||||
* @returns {boolean}
|
||||
*/
|
||||
export function isUnauthorized(request) {
|
||||
if (!request.session?.currentUser) return true;
|
||||
if (Date.now() - (request.session.createdAt || 0) > SESSION_MAX_AGE) return true;
|
||||
return false;
|
||||
};
|
||||
const isAdmin = (req) => {
|
||||
if (!isUnauthorized(req)) {
|
||||
const user = userStorage.getUser(req.session.currentUser);
|
||||
return user != null && user.isAdmin;
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns true when the session belongs to an admin user.
|
||||
* @param {import('fastify').FastifyRequest} request
|
||||
* @returns {boolean}
|
||||
*/
|
||||
export function isAdmin(request) {
|
||||
if (isUnauthorized(request)) return false;
|
||||
const user = userStorage.getUser(request.session.currentUser);
|
||||
return user != null && user.isAdmin;
|
||||
}
|
||||
|
||||
/**
|
||||
* Fastify preHandler hook - rejects unauthenticated requests with 401.
|
||||
* @param {import('fastify').FastifyRequest} request
|
||||
* @param {import('fastify').FastifyReply} reply
|
||||
*/
|
||||
export async function authHook(request, reply) {
|
||||
if (isUnauthorized(request)) {
|
||||
reply.code(401).send();
|
||||
}
|
||||
return false;
|
||||
};
|
||||
const authInterceptor = () => {
|
||||
return (req, res, next) => {
|
||||
if (isUnauthorized(req)) {
|
||||
return unauthorized(res);
|
||||
} else {
|
||||
next();
|
||||
}
|
||||
};
|
||||
};
|
||||
const adminInterceptor = () => {
|
||||
return (req, res, next) => {
|
||||
if (!isAdmin(req)) {
|
||||
return unauthorized(res);
|
||||
} else {
|
||||
next();
|
||||
}
|
||||
};
|
||||
};
|
||||
const cookieSession$0 = (secret) => {
|
||||
return cookieSession({
|
||||
name: 'fredy-admin-session',
|
||||
keys: [secret],
|
||||
maxAge: SESSION_MAX_AGE,
|
||||
});
|
||||
};
|
||||
export { cookieSession$0 as cookieSession };
|
||||
export { adminInterceptor };
|
||||
export { authInterceptor };
|
||||
export { isUnauthorized };
|
||||
export { isAdmin };
|
||||
}
|
||||
|
||||
/**
|
||||
* Fastify preHandler hook - rejects non-admin requests with 401.
|
||||
* Apply after authHook.
|
||||
* @param {import('fastify').FastifyRequest} request
|
||||
* @param {import('fastify').FastifyReply} reply
|
||||
*/
|
||||
export async function adminHook(request, reply) {
|
||||
if (!isAdmin(request)) {
|
||||
reply.code(401).send();
|
||||
}
|
||||
}
|
||||
|
||||
@@ -133,7 +133,7 @@ The LLM will automatically call the appropriate Fredy MCP tools and present the
|
||||
#### Setup
|
||||
|
||||
1. Open **Claude Desktop**
|
||||
2. Go to **Settings → Developer → Edit Config** — this opens the `claude_desktop_config.json` file
|
||||
2. Go to **Settings → Developer → Edit Config** - this opens the `claude_desktop_config.json` file
|
||||
3. Add the `fredy` server to the `mcpServers` object:
|
||||
|
||||
```json
|
||||
@@ -158,7 +158,7 @@ The LLM will automatically call the appropriate Fredy MCP tools and present the
|
||||
> - nvm: `/Users/<you>/.nvm/versions/node/<version>/bin/node`
|
||||
|
||||
4. Save the file and **restart Claude Desktop**
|
||||
5. You should see a hammer icon (🔨) in the chat input — click it to confirm the Fredy tools are listed
|
||||
5. You should see a hammer icon (🔨) in the chat input - click it to confirm the Fredy tools are listed
|
||||
|
||||
#### Usage
|
||||
|
||||
@@ -170,7 +170,7 @@ Once connected, simply ask Claude about your real estate data:
|
||||
|
||||
Claude will automatically call the appropriate Fredy MCP tools.
|
||||
|
||||
> **Note:** Fredy's main web process does not need to be running — the stdio transport opens its own database connection directly. But the SQLite database file must exist and migrations must have been applied.
|
||||
> **Note:** Fredy's main web process does not need to be running - the stdio transport opens its own database connection directly. But the SQLite database file must exist and migrations must have been applied.
|
||||
|
||||
---
|
||||
|
||||
@@ -252,7 +252,7 @@ Example list response:
|
||||
```
|
||||
**Tool:** list_listings | **Status:** OK
|
||||
|
||||
Found **85** listing(s). Showing page 1 of 2 (50 on this page). More pages available — use page=2 to continue.
|
||||
Found **85** listing(s). Showing page 1 of 2 (50 on this page). More pages available - use page=2 to continue.
|
||||
|
||||
| ID | Title | Address | Price | Size | Provider | Active | Created | Job |
|
||||
|----|-------|---------|-------|------|----------|--------|---------|-----|
|
||||
|
||||
@@ -49,7 +49,7 @@ export function createMcpServer() {
|
||||
'list_listings to search listings (supports time filters like createdAfter/createdBefore), ' +
|
||||
'and get_listing for full details of a single listing. ' +
|
||||
'Responses are formatted as markdown with a summary, data (tables for lists, key-value for details), and pagination info. ' +
|
||||
'Always present results to the user as soon as you have them — do NOT call the tool again unless you need additional pages or different data.',
|
||||
'Always present results to the user as soon as you have them - do NOT call the tool again unless you need additional pages or different data.',
|
||||
},
|
||||
);
|
||||
|
||||
|
||||
@@ -3,10 +3,6 @@
|
||||
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
|
||||
*/
|
||||
|
||||
/*
|
||||
* Copyright (c) 2026 by Christian Kellner.
|
||||
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
|
||||
*/
|
||||
import { StreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/streamableHttp.js';
|
||||
import { createMcpServer } from './mcpAdapter.js';
|
||||
import { authenticateRequest } from './mcpAuthentication.js';
|
||||
@@ -15,16 +11,13 @@ import crypto from 'crypto';
|
||||
|
||||
/**
|
||||
* Active transports keyed by session id.
|
||||
* Each session gets its own McpServer + StreamableHTTPServerTransport pair.
|
||||
* @type {Map<string, { server: McpServer, transport: StreamableHTTPServerTransport }>}
|
||||
*/
|
||||
const sessions = new Map();
|
||||
|
||||
/**
|
||||
* Get or create a session for the given session id with authentication.
|
||||
* @param {string|undefined} sessionId
|
||||
* @param {{ userId: string }} auth
|
||||
* @returns {{ server: McpServer, transport: StreamableHTTPServerTransport }}
|
||||
*/
|
||||
function getOrCreateSession(sessionId, auth) {
|
||||
if (sessionId && sessions.has(sessionId)) {
|
||||
@@ -54,77 +47,67 @@ function getOrCreateSession(sessionId, auth) {
|
||||
}
|
||||
|
||||
/**
|
||||
* Register MCP Streamable HTTP routes on a restana service.
|
||||
* Register MCP Streamable HTTP routes on a fastify instance.
|
||||
*
|
||||
* Mounts handlers at /api/mcp to handle the MCP Streamable HTTP protocol:
|
||||
* - POST /api/mcp – JSON-RPC messages (initialize, tool calls, etc.)
|
||||
* - GET /api/mcp – SSE stream for server-initiated notifications
|
||||
* - DELETE /api/mcp – session termination
|
||||
* POST /api/mcp – JSON-RPC messages
|
||||
* GET /api/mcp – SSE stream for server-initiated notifications
|
||||
* DELETE /api/mcp – session termination
|
||||
*
|
||||
* All endpoints require a valid Bearer token in the Authorization header.
|
||||
*
|
||||
* @param {import('restana').Service} service - The restana service instance.
|
||||
* @param {import('fastify').FastifyInstance} fastify
|
||||
*/
|
||||
export function registerMcpRoutes(service) {
|
||||
// POST – main JSON-RPC endpoint
|
||||
service.post('/api/mcp', async (req, res) => {
|
||||
const auth = authenticateRequest(req);
|
||||
export function registerMcpRoutes(fastify) {
|
||||
fastify.post('/api/mcp', async (request, reply) => {
|
||||
const auth = authenticateRequest(request.raw);
|
||||
if (!auth) {
|
||||
res.statusCode = 401;
|
||||
return res.send({ error: 'Unauthorized. Provide a valid Bearer token.' });
|
||||
return reply.code(401).send({ error: 'Unauthorized. Provide a valid Bearer token.' });
|
||||
}
|
||||
|
||||
const sessionId = req.headers['mcp-session-id'];
|
||||
const sessionId = request.raw.headers['mcp-session-id'];
|
||||
const { server, transport } = getOrCreateSession(sessionId, auth);
|
||||
|
||||
// Connect server to transport if not already connected
|
||||
if (!transport.onmessage) {
|
||||
await server.connect(transport);
|
||||
}
|
||||
|
||||
// Inject authInfo so tools can access the authenticated user
|
||||
req.auth = { userId: auth.userId };
|
||||
request.raw.auth = { userId: auth.userId };
|
||||
|
||||
await transport.handleRequest(req, res, req.body);
|
||||
reply.hijack();
|
||||
await transport.handleRequest(request.raw, reply.raw, request.body);
|
||||
});
|
||||
|
||||
// GET – SSE stream for server-initiated messages
|
||||
service.get('/api/mcp', async (req, res) => {
|
||||
const auth = authenticateRequest(req);
|
||||
fastify.get('/api/mcp', async (request, reply) => {
|
||||
const auth = authenticateRequest(request.raw);
|
||||
if (!auth) {
|
||||
res.statusCode = 401;
|
||||
return res.send({ error: 'Unauthorized. Provide a valid Bearer token.' });
|
||||
return reply.code(401).send({ error: 'Unauthorized. Provide a valid Bearer token.' });
|
||||
}
|
||||
|
||||
const sessionId = req.headers['mcp-session-id'];
|
||||
const sessionId = request.raw.headers['mcp-session-id'];
|
||||
if (!sessionId || !sessions.has(sessionId)) {
|
||||
res.statusCode = 400;
|
||||
return res.send({ error: 'Invalid or missing session. Send an initialize request first.' });
|
||||
return reply.code(400).send({ error: 'Invalid or missing session. Send an initialize request first.' });
|
||||
}
|
||||
|
||||
const { transport } = sessions.get(sessionId);
|
||||
await transport.handleRequest(req, res);
|
||||
reply.hijack();
|
||||
await transport.handleRequest(request.raw, reply.raw);
|
||||
});
|
||||
|
||||
// DELETE – terminate session
|
||||
service.delete('/api/mcp', async (req, res) => {
|
||||
const auth = authenticateRequest(req);
|
||||
fastify.delete('/api/mcp', async (request, reply) => {
|
||||
const auth = authenticateRequest(request.raw);
|
||||
if (!auth) {
|
||||
res.statusCode = 401;
|
||||
return res.send({ error: 'Unauthorized. Provide a valid Bearer token.' });
|
||||
return reply.code(401).send({ error: 'Unauthorized. Provide a valid Bearer token.' });
|
||||
}
|
||||
|
||||
const sessionId = req.headers['mcp-session-id'];
|
||||
const sessionId = request.raw.headers['mcp-session-id'];
|
||||
if (!sessionId || !sessions.has(sessionId)) {
|
||||
res.statusCode = 404;
|
||||
return res.send({ error: 'Session not found.' });
|
||||
return reply.code(404).send({ error: 'Session not found.' });
|
||||
}
|
||||
|
||||
const { transport } = sessions.get(sessionId);
|
||||
await transport.close();
|
||||
sessions.delete(sessionId);
|
||||
res.statusCode = 200;
|
||||
res.send({ ok: true });
|
||||
return { ok: true };
|
||||
});
|
||||
|
||||
logger.debug('MCP Streamable HTTP endpoint registered at /api/mcp');
|
||||
|
||||
@@ -70,7 +70,7 @@ export function normalizeListJobs(queryResult, { page, pageSize }) {
|
||||
|
||||
let md = `**Tool:** list_jobs | **Status:** OK\n\n`;
|
||||
md += `Found **${queryResult.totalNumber}** job(s). Showing page ${page} of ${maxPage} (${jobs.length} on this page).`;
|
||||
if (hasMore) md += ` More pages available — use page=${page + 1} to continue.`;
|
||||
if (hasMore) md += ` More pages available - use page=${page + 1} to continue.`;
|
||||
md += '\n\n';
|
||||
|
||||
if (jobs.length > 0) {
|
||||
@@ -120,7 +120,7 @@ export function normalizeListListings(queryResult, { page, pageSize }) {
|
||||
|
||||
let md = `**Tool:** list_listings | **Status:** OK\n\n`;
|
||||
md += `Found **${queryResult.totalNumber}** listing(s). Showing page ${page} of ${maxPage} (${listings.length} on this page).`;
|
||||
if (hasMore) md += ` More pages available — use page=${page + 1} to continue.`;
|
||||
if (hasMore) md += ` More pages available - use page=${page + 1} to continue.`;
|
||||
md += '\n\n';
|
||||
|
||||
if (listings.length > 0) {
|
||||
|
||||
@@ -17,6 +17,7 @@ Multiple recipients:
|
||||
|
||||
Common SMTP settings:
|
||||
|
||||
- **Gmail** — `smtp.gmail.com`, port 587, secure: false
|
||||
- **Outlook** — `smtp.office365.com`, port 587, secure: false
|
||||
- **Yahoo** — `smtp.mail.yahoo.com`, port 465, secure: true
|
||||
- **Gmail** - `smtp.gmail.com`, port 587, secure: false
|
||||
- **Outlook** - `smtp.office365.com`, port 587, secure: false
|
||||
- **Yahoo** - `smtp.mail.yahoo.com`, port 465, secure: true
|
||||
- **Gmx** - `mail.gmx.net`, port 587, secure: true
|
||||
|
||||
@@ -26,7 +26,7 @@ function parseId(shortenedLink) {
|
||||
|
||||
async function fetchDetails(listing, browser) {
|
||||
try {
|
||||
const html = await puppeteerExtractor(listing.link, null, { browser });
|
||||
const html = await puppeteerExtractor(listing.link, null, { browser, name: 'immobilienDe_details' });
|
||||
if (!html) return listing;
|
||||
|
||||
const $ = cheerio.load(html);
|
||||
|
||||
@@ -16,7 +16,7 @@ let appliedBlackList = [];
|
||||
|
||||
async function fetchDetails(listing, browser) {
|
||||
try {
|
||||
const html = await puppeteerExtractor(listing.link, null, { browser });
|
||||
const html = await puppeteerExtractor(listing.link, null, { browser, name: 'immowelt_details' });
|
||||
if (!html) return listing;
|
||||
|
||||
const $ = cheerio.load(html);
|
||||
@@ -87,7 +87,19 @@ const config = {
|
||||
crawlContainer:
|
||||
'div[data-testid="serp-core-scrollablelistview-testid"]:not(div[data-testid="serp-enlargementlist-testid"] div[data-testid="serp-card-testid"]) div[data-testid="serp-core-classified-card-testid"]',
|
||||
sortByDateParam: 'order=DateDesc',
|
||||
waitForSelector: 'div[data-testid="serp-gridcontainer-testid"]',
|
||||
// waitForSelector is null: extract the full page via page.content() so the
|
||||
// Cheerio crawler can search anywhere in the rendered document.
|
||||
// preNavigateUrl visits the homepage first to establish a trusted session
|
||||
// before hitting the search URL; this prevents CDN-level bot challenges that
|
||||
// fire on cold sessions. waitForNetworkIdle (phase 2) then catches React's
|
||||
// listing API round-trip that fires well after domcontentloaded.
|
||||
waitForSelector: null,
|
||||
puppeteerOptions: {
|
||||
puppeteerTimeout: 60_000,
|
||||
preNavigateUrl: 'https://www.immowelt.de/',
|
||||
waitForNetworkIdle: true,
|
||||
waitForNetworkIdleTimeout: 60_000,
|
||||
},
|
||||
crawlFields: {
|
||||
id: 'a@href',
|
||||
price: 'div[data-testid="cardmfe-price-testid"] | removeNewline | trim',
|
||||
|
||||
@@ -128,7 +128,7 @@ async function enrichListingFromDetails(listing, browser) {
|
||||
if (!absoluteLink) return listing;
|
||||
|
||||
try {
|
||||
const html = await puppeteerExtractor(absoluteLink, null, { browser });
|
||||
const html = await puppeteerExtractor(absoluteLink, null, { browser, name: 'kleinanzeigen_details' });
|
||||
if (!html) return { ...listing, link: absoluteLink };
|
||||
|
||||
const { detailAddress, detailDescription } = extractDetailFromHtml(html);
|
||||
@@ -196,8 +196,8 @@ const config = {
|
||||
id: '.aditem@data-adid',
|
||||
price: '.aditem-main--middle--price-shipping--price | removeNewline | trim',
|
||||
tags: '.aditem-main--middle--tags | removeNewline | trim',
|
||||
title: '.aditem-main .text-module-begin a | removeNewline | trim',
|
||||
link: '.aditem-main .text-module-begin a@href | removeNewline | trim',
|
||||
title: '.aditem-main .text-module-begin | removeNewline | trim',
|
||||
link: '.aditem@data-href',
|
||||
description: '.aditem-main .aditem-main--middle--description | removeNewline | trim',
|
||||
address: '.aditem-main--top--left | trim | removeNewline',
|
||||
image: 'img@src',
|
||||
|
||||
@@ -16,7 +16,7 @@ let appliedBlackList = [];
|
||||
|
||||
async function fetchDetails(listing, browser) {
|
||||
try {
|
||||
const html = await puppeteerExtractor(listing.link, 'body', { browser });
|
||||
const html = await puppeteerExtractor(listing.link, 'body', { browser, name: 'sparkasse_details' });
|
||||
|
||||
const $ = cheerio.load(html);
|
||||
const nextDataRaw = $('#__NEXT_DATA__').text;
|
||||
|
||||
@@ -16,7 +16,7 @@ let appliedBlackList = [];
|
||||
|
||||
async function fetchDetails(listing, browser) {
|
||||
try {
|
||||
const html = await puppeteerExtractor(listing.link, null, { browser });
|
||||
const html = await puppeteerExtractor(listing.link, null, { browser, name: 'wgGesucht_details' });
|
||||
if (!html) return listing;
|
||||
|
||||
const $ = cheerio.load(html);
|
||||
|
||||
147
lib/services/ensureValidBinary.js
Normal file
147
lib/services/ensureValidBinary.js
Normal file
@@ -0,0 +1,147 @@
|
||||
/*
|
||||
* Copyright (c) 2026 by Christian Kellner.
|
||||
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
|
||||
*/
|
||||
|
||||
import { ensureBinary } from 'cloakbrowser';
|
||||
import fs from 'fs';
|
||||
import path from 'path';
|
||||
import os from 'os';
|
||||
|
||||
/**
|
||||
* Resource files required on Linux/Windows — they must live next to the chrome binary.
|
||||
* macOS packages these inside the .app bundle's Frameworks directory so a different
|
||||
* check is used there (see isBinaryComplete).
|
||||
*/
|
||||
const LINUX_WIN_REQUIRED_FILES = ['icudtl.dat', 'resources.pak'];
|
||||
|
||||
/**
|
||||
* Return the top-level versioned installation directory for any platform.
|
||||
*
|
||||
* - Linux/Windows: binaryPath is ~/.cloakbrowser/chromium-X.Y.Z/chrome
|
||||
* → dirname ~/.cloakbrowser/chromium-X.Y.Z/
|
||||
* - macOS: binaryPath is ~/.cloakbrowser/chromium-X.Y.Z/Chromium.app/Contents/MacOS/Chromium
|
||||
* → 4 levels up ~/.cloakbrowser/chromium-X.Y.Z/
|
||||
*
|
||||
* @param {string} binaryPath
|
||||
* @returns {string}
|
||||
*/
|
||||
function getVersionedDir(binaryPath) {
|
||||
if (process.platform === 'darwin') {
|
||||
return path.resolve(path.dirname(binaryPath), '../../..');
|
||||
}
|
||||
return path.dirname(binaryPath);
|
||||
}
|
||||
|
||||
/**
|
||||
* Return true when the binary at binaryPath belongs to a complete installation.
|
||||
*
|
||||
* On macOS the binary lives inside an .app bundle:
|
||||
* Chromium.app/Contents/MacOS/Chromium
|
||||
* Resource files (icudtl.dat etc.) are deep inside
|
||||
* Chromium.app/Contents/Frameworks/…
|
||||
* so checking for them next to the binary is wrong. Instead we verify the two
|
||||
* structural markers that are only present after a full extraction: Info.plist
|
||||
* and the Frameworks directory inside Contents/.
|
||||
*
|
||||
* On Linux/Windows the binary and all resource files are siblings in the same
|
||||
* directory.
|
||||
*
|
||||
* @param {string} binaryPath
|
||||
* @returns {boolean}
|
||||
*/
|
||||
function isBinaryComplete(binaryPath) {
|
||||
if (process.platform === 'darwin') {
|
||||
const contentsDir = path.resolve(path.dirname(binaryPath), '..');
|
||||
return fs.existsSync(path.join(contentsDir, 'Info.plist')) && fs.existsSync(path.join(contentsDir, 'Frameworks'));
|
||||
}
|
||||
const dir = path.dirname(binaryPath);
|
||||
return LINUX_WIN_REQUIRED_FILES.every((f) => fs.existsSync(path.join(dir, f)));
|
||||
}
|
||||
|
||||
/**
|
||||
* Return a human-readable description of which required files/dirs are missing.
|
||||
*
|
||||
* @param {string} binaryPath
|
||||
* @returns {string}
|
||||
*/
|
||||
function missingDescription(binaryPath) {
|
||||
if (process.platform === 'darwin') {
|
||||
const contentsDir = path.resolve(path.dirname(binaryPath), '..');
|
||||
return ['Info.plist', 'Frameworks'].filter((f) => !fs.existsSync(path.join(contentsDir, f))).join(', ');
|
||||
}
|
||||
const dir = path.dirname(binaryPath);
|
||||
return LINUX_WIN_REQUIRED_FILES.filter((f) => !fs.existsSync(path.join(dir, f))).join(', ');
|
||||
}
|
||||
|
||||
/**
|
||||
* Remove a corrupt binary installation and all `latest_version*` markers from
|
||||
* the CloakBrowser cache so the next `ensureBinary()` call falls back to the
|
||||
* package-bundled version.
|
||||
*
|
||||
* Removes the full versioned directory (e.g. chromium-X.Y.Z/) on all platforms,
|
||||
* not just the subdirectory that contains the binary.
|
||||
*
|
||||
* @param {string} binaryPath - Path to the (corrupt) chrome/Chromium binary.
|
||||
*/
|
||||
function removeCorruptInstallation(binaryPath) {
|
||||
const versionedDir = getVersionedDir(binaryPath);
|
||||
const cacheDir = process.env.CLOAKBROWSER_CACHE_DIR || path.join(os.homedir(), '.cloakbrowser');
|
||||
|
||||
fs.rmSync(versionedDir, { recursive: true, force: true });
|
||||
|
||||
try {
|
||||
for (const entry of fs.readdirSync(cacheDir)) {
|
||||
if (entry.startsWith('latest_version')) {
|
||||
fs.rmSync(path.join(cacheDir, entry), { force: true });
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
// Cache dir may not exist if versionedDir was the only entry — ignore.
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Ensure the CloakBrowser stealth Chromium binary is present **and** complete.
|
||||
*
|
||||
* `cloakbrowser`'s own `ensureBinary()` only checks that the chrome/Chromium
|
||||
* file exists. An incomplete extraction (e.g. interrupted download, disk full)
|
||||
* can leave a directory that contains the executable but is missing essential
|
||||
* resource files. Chrome then crashes immediately on launch.
|
||||
*
|
||||
* This wrapper validates the path returned by `ensureBinary()`. If the
|
||||
* installation is incomplete it removes the corrupt directory, clears the
|
||||
* version marker files, and calls `ensureBinary()` again so it falls back to
|
||||
* (or re-downloads) a complete build.
|
||||
*
|
||||
* The validated path is also pinned via `CLOAKBROWSER_BINARY_PATH` so that
|
||||
* CloakBrowser's own internal `ensureBinary()` call inside `launch()` always
|
||||
* picks up the same, verified binary.
|
||||
*
|
||||
* @returns {Promise<string>} Absolute path to the validated binary.
|
||||
* @throws {Error} When even the fallback binary is incomplete.
|
||||
*/
|
||||
export async function ensureValidBinary() {
|
||||
const binaryPath = await ensureBinary();
|
||||
|
||||
if (isBinaryComplete(binaryPath)) {
|
||||
process.env.CLOAKBROWSER_BINARY_PATH = binaryPath;
|
||||
return binaryPath;
|
||||
}
|
||||
|
||||
console.warn(
|
||||
`[fredy] CloakBrowser installation at ${getVersionedDir(binaryPath)} is missing: ${missingDescription(binaryPath)}. Removing and retrying.`,
|
||||
);
|
||||
|
||||
removeCorruptInstallation(binaryPath);
|
||||
|
||||
const fallbackPath = await ensureBinary();
|
||||
if (!isBinaryComplete(fallbackPath)) {
|
||||
throw new Error(
|
||||
`CloakBrowser binary at ${getVersionedDir(fallbackPath)} is still missing required files after re-download: ${missingDescription(fallbackPath)}`,
|
||||
);
|
||||
}
|
||||
|
||||
process.env.CLOAKBROWSER_BINARY_PATH = fallbackPath;
|
||||
return fallbackPath;
|
||||
}
|
||||
@@ -94,7 +94,7 @@ export async function applyBotPreventionToPage(page, cfg) {
|
||||
// webdriver
|
||||
Object.defineProperty(navigator, 'webdriver', { get: () => undefined });
|
||||
|
||||
// chrome runtime — expose loadTimes, csi and app like real Chrome
|
||||
// chrome runtime - expose loadTimes, csi and app like real Chrome
|
||||
// @ts-ignore
|
||||
window.chrome = {
|
||||
runtime: {},
|
||||
@@ -129,7 +129,7 @@ export async function applyBotPreventionToPage(page, cfg) {
|
||||
get: () => (window.localStorage.getItem('__LANGS__') || 'de-DE,de').split(','),
|
||||
});
|
||||
|
||||
// plugins — mimic real Chrome's built-in PDF plugins
|
||||
// plugins - mimic real Chrome's built-in PDF plugins
|
||||
const makePlugin = (name, filename, description, mimeType, mimeTypeSuffix) => {
|
||||
const mimeObj = { type: mimeType, suffixes: mimeTypeSuffix, description, enabledPlugin: null };
|
||||
const plugin = { name, filename, description, length: 1, 0: mimeObj };
|
||||
@@ -274,14 +274,14 @@ export async function applyBotPreventionToPage(page, cfg) {
|
||||
//noop
|
||||
}
|
||||
|
||||
// document.hasFocus — headless returns false; real active tabs return true
|
||||
// document.hasFocus - headless returns false; real active tabs return true
|
||||
try {
|
||||
document.hasFocus = () => true;
|
||||
} catch {
|
||||
//noop
|
||||
}
|
||||
|
||||
// screen color depth — normalise in case headless reports 0
|
||||
// screen color depth - normalise in case headless reports 0
|
||||
try {
|
||||
Object.defineProperty(screen, 'colorDepth', { get: () => 24 });
|
||||
Object.defineProperty(screen, 'pixelDepth', { get: () => 24 });
|
||||
|
||||
@@ -29,11 +29,12 @@ export default class Extractor {
|
||||
* your response will never contain what you are really looking for
|
||||
* @param url
|
||||
* @param waitForSelector
|
||||
* @param jobKey
|
||||
*/
|
||||
execute = async (url, waitForSelector = null) => {
|
||||
execute = async (url, waitForSelector = null, jobKey = null) => {
|
||||
this.responseText = null;
|
||||
try {
|
||||
this.responseText = await puppeteerExtractor(url, waitForSelector, this.options);
|
||||
this.responseText = await puppeteerExtractor(url, waitForSelector, { ...this.options, name: jobKey });
|
||||
if (this.responseText != null) {
|
||||
loadParser(this.responseText);
|
||||
}
|
||||
|
||||
@@ -3,121 +3,135 @@
|
||||
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
|
||||
*/
|
||||
|
||||
import puppeteer from 'puppeteer-extra';
|
||||
import StealthPlugin from 'puppeteer-extra-plugin-stealth';
|
||||
import { launch } from 'cloakbrowser/puppeteer';
|
||||
import { debug, botDetected } from './utils.js';
|
||||
import {
|
||||
getPreLaunchConfig,
|
||||
applyBotPreventionToPage,
|
||||
applyLanguagePersistence,
|
||||
applyPostNavigationHumanSignals,
|
||||
} from './botPrevention.js';
|
||||
import { getPreLaunchConfig } from './botPrevention.js';
|
||||
import logger from '../logger.js';
|
||||
import fs from 'fs';
|
||||
import os from 'os';
|
||||
import path from 'path';
|
||||
|
||||
puppeteer.use(StealthPlugin());
|
||||
import { trackPoi } from '../tracking/Tracker.js';
|
||||
import { TRACKING_POIS } from '../../TRACKING_POIS.js';
|
||||
|
||||
/**
|
||||
* Launch a CloakBrowser/Puppeteer browser instance with stealth and humanizer enabled.
|
||||
*
|
||||
* CloakBrowser applies 49 C++ source-level patches (canvas, WebGL, audio, WebRTC,
|
||||
* navigator.*, automation signals) that are indistinguishable from a real browser.
|
||||
* All fingerprinting and human-behaviour simulation is handled natively; no CDP
|
||||
* overrides (setUserAgent, setExtraHTTPHeaders, evaluateOnNewDocument) are applied
|
||||
* here because they would create detectable inconsistencies on top of the C++ patches.
|
||||
*
|
||||
* @param {string} url - Initial URL (used to derive locale/timezone hints).
|
||||
* @param {object} [options]
|
||||
* @param {boolean} [options.puppeteerHeadless]
|
||||
* @param {number} [options.puppeteerTimeout]
|
||||
* @param {string} [options.proxyUrl]
|
||||
* @param {string} [options.timezone]
|
||||
* @param {string} [options.acceptLanguage]
|
||||
* @param {object} [options.viewport]
|
||||
* @returns {Promise<import('puppeteer-core').Browser>}
|
||||
*/
|
||||
export async function launchBrowser(url, options) {
|
||||
const preCfg = getPreLaunchConfig(url, options || {});
|
||||
const launchArgs = [
|
||||
|
||||
// Docker requires --no-sandbox; CloakBrowser handles all stealth args internally.
|
||||
// --ignore-certificate-errors is needed because CloakBrowser ships its own Chromium
|
||||
// binary with an independent CA bundle that may not trust proxies or interceptors
|
||||
// present in the host environment.
|
||||
const args = [
|
||||
'--no-sandbox',
|
||||
'--disable-gpu',
|
||||
'--disable-setuid-sandbox',
|
||||
'--disable-dev-shm-usage',
|
||||
'--disable-crash-reporter',
|
||||
'--no-first-run',
|
||||
'--no-default-browser-check',
|
||||
preCfg.langArg,
|
||||
'--ignore-certificate-errors',
|
||||
// Disables the zygote process model. Required in some container environments
|
||||
// (e.g. limited kernel namespaces) where the zygote cannot acquire the
|
||||
// locks it needs and exits with "Invalid file descriptor to ICU data received".
|
||||
'--no-zygote',
|
||||
preCfg.windowSizeArg,
|
||||
...preCfg.extraArgs,
|
||||
];
|
||||
if (options?.proxyUrl) {
|
||||
launchArgs.push(`--proxy-server=${options.proxyUrl}`);
|
||||
}
|
||||
|
||||
let userDataDir;
|
||||
let removeUserDataDir = false;
|
||||
if (options && options.userDataDir) {
|
||||
userDataDir = options.userDataDir;
|
||||
} else {
|
||||
const prefix = path.join(os.tmpdir(), 'puppeteer-fredy-');
|
||||
userDataDir = fs.mkdtempSync(prefix);
|
||||
removeUserDataDir = true;
|
||||
}
|
||||
|
||||
// On ARM64 Docker, Chrome for Testing has no native binary — use system Chromium instead.
|
||||
const executablePath =
|
||||
options?.executablePath ||
|
||||
(process.arch === 'arm64' && process.env.IS_DOCKER === 'true' ? '/usr/bin/chromium' : undefined);
|
||||
|
||||
const browser = await puppeteer.launch({
|
||||
const browser = await launch({
|
||||
headless: options?.puppeteerHeadless ?? true,
|
||||
args: launchArgs,
|
||||
timeout: options?.puppeteerTimeout || 45_000,
|
||||
userDataDir,
|
||||
executablePath,
|
||||
humanize: true,
|
||||
args,
|
||||
// locale sets Accept-Language headers and JS navigator.language consistently
|
||||
locale: preCfg.langForFlag,
|
||||
...(options?.proxyUrl ? { proxy: options.proxyUrl } : {}),
|
||||
...(preCfg.timezone ? { timezone: preCfg.timezone } : {}),
|
||||
});
|
||||
|
||||
browser.__fredy_userDataDir = userDataDir;
|
||||
browser.__fredy_removeUserDataDir = removeUserDataDir;
|
||||
|
||||
return browser;
|
||||
}
|
||||
|
||||
/**
|
||||
* Close a browser instance returned by {@link launchBrowser}.
|
||||
*
|
||||
* @param {import('puppeteer-core').Browser | null} browser
|
||||
*/
|
||||
export async function closeBrowser(browser) {
|
||||
if (!browser) return;
|
||||
const userDataDir = browser.__fredy_userDataDir;
|
||||
const removeUserDataDir = browser.__fredy_removeUserDataDir;
|
||||
try {
|
||||
await browser.close();
|
||||
} catch {
|
||||
// ignore
|
||||
}
|
||||
if (removeUserDataDir && userDataDir) {
|
||||
try {
|
||||
await fs.promises.rm(userDataDir, { recursive: true, force: true });
|
||||
} catch {
|
||||
// ignore
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Open a page in a (possibly reused) browser, navigate to `url`, and return the HTML source.
|
||||
* Returns `null` when a bot-detection page is encountered or on timeout.
|
||||
*
|
||||
* @param {string} url
|
||||
* @param {string | null} waitForSelector
|
||||
* @param {object} [options]
|
||||
* @returns {Promise<string | null>}
|
||||
*/
|
||||
export default async function execute(url, waitForSelector, options) {
|
||||
let browser = options?.browser;
|
||||
let isExternalBrowser = !!browser;
|
||||
let page;
|
||||
let result;
|
||||
try {
|
||||
debug(`Sending request to ${url} using Puppeteer.`);
|
||||
debug(`Sending request to ${url} using CloakBrowser.`);
|
||||
|
||||
if (!isExternalBrowser) {
|
||||
browser = await launchBrowser(url, options);
|
||||
}
|
||||
|
||||
page = await browser.newPage();
|
||||
const preCfg = getPreLaunchConfig(url, options || {});
|
||||
await applyBotPreventionToPage(page, preCfg);
|
||||
// Provide languages value before navigation
|
||||
await applyLanguagePersistence(page, preCfg);
|
||||
|
||||
// Optional cookies
|
||||
if (Array.isArray(options?.cookies) && options.cookies.length > 0) {
|
||||
await page.setCookie(...options.cookies);
|
||||
}
|
||||
|
||||
// Navigation
|
||||
// Warm-up navigation: visit a trusted page first so the site sees an
|
||||
// established session before the actual target URL. Silently ignored on
|
||||
// failure so it never blocks the main request.
|
||||
if (options?.preNavigateUrl) {
|
||||
try {
|
||||
await page.goto(options.preNavigateUrl, { waitUntil: 'domcontentloaded', timeout: 30_000 });
|
||||
await new Promise((r) => setTimeout(r, 1500 + Math.random() * 2000));
|
||||
} catch {
|
||||
// ignore
|
||||
}
|
||||
}
|
||||
|
||||
const response = await page.goto(url, {
|
||||
waitUntil: options?.waitUntil || 'domcontentloaded',
|
||||
timeout: options?.puppeteerTimeout || 60000,
|
||||
});
|
||||
|
||||
// Optionally wait and add subtle human-like interactions
|
||||
await applyPostNavigationHumanSignals(page, preCfg);
|
||||
// Optional second idle wait: useful for React SPAs that trigger API calls
|
||||
// after domcontentloaded. Times out silently so we use whatever is rendered.
|
||||
if (options?.waitForNetworkIdle) {
|
||||
try {
|
||||
await page.waitForNetworkIdle({ timeout: options?.waitForNetworkIdleTimeout ?? 60_000 });
|
||||
} catch {
|
||||
// ignore — we proceed with whatever the DOM contains at this point
|
||||
}
|
||||
}
|
||||
|
||||
let pageSource;
|
||||
// if we're extracting data from a SPA, we must wait for the selector
|
||||
if (waitForSelector != null) {
|
||||
const selectorTimeout = options?.puppeteerSelectorTimeout ?? options?.puppeteerTimeout ?? 30_000;
|
||||
await page.waitForSelector(waitForSelector, { timeout: selectorTimeout });
|
||||
@@ -133,15 +147,22 @@ export default async function execute(url, waitForSelector, options) {
|
||||
|
||||
if (botDetected(pageSource, statusCode)) {
|
||||
logger.warn('We have been detected as a bot :-/ Tried url: => ', url);
|
||||
|
||||
if (options != null && options.name != null) {
|
||||
await trackPoi(TRACKING_POIS.DETECTED_AS_BOT + '_' + options.name);
|
||||
} else {
|
||||
await trackPoi(TRACKING_POIS.DETECTED_AS_BOT);
|
||||
}
|
||||
|
||||
result = null;
|
||||
} else {
|
||||
result = pageSource || (await page.content());
|
||||
}
|
||||
} catch (error) {
|
||||
if (error?.name?.includes('Timeout')) {
|
||||
logger.debug('Error executing with puppeteer executor', error);
|
||||
logger.debug('Error executing with CloakBrowser executor', error);
|
||||
} else {
|
||||
logger.warn('Error executing with puppeteer executor', error);
|
||||
logger.warn('Error executing with CloakBrowser executor', error);
|
||||
}
|
||||
result = null;
|
||||
} finally {
|
||||
|
||||
@@ -168,10 +168,6 @@ export function convertWebToMobile(webUrl) {
|
||||
}
|
||||
}
|
||||
|
||||
if (segments.includes('shape')) {
|
||||
throw new Error('Shape is currently not supported using Immoscout');
|
||||
}
|
||||
|
||||
const { query: rawParams } = queryString.parseUrl(webUrl, { arrayFormat: 'comma' });
|
||||
const webParams = Object.fromEntries(
|
||||
Object.entries(rawParams).filter(([key]) => key !== 'enteredFrom' && PARAM_NAME_MAP[key]),
|
||||
@@ -179,18 +175,31 @@ export function convertWebToMobile(webUrl) {
|
||||
|
||||
const geocodes = `/${segments.slice(2, segments.length - 1).join('/')}`;
|
||||
const isRadius = segments.includes('radius');
|
||||
const isShape = segments.includes('shape');
|
||||
const mobileParams = {
|
||||
searchType: isRadius ? 'radius' : 'region',
|
||||
searchType: isRadius ? 'radius' : isShape ? 'shape' : 'region',
|
||||
realestatetype: realType,
|
||||
...(isRadius ? {} : { geocodes }),
|
||||
...(isRadius || isShape ? {} : { geocodes }),
|
||||
...additionalParamsFromWebPath,
|
||||
};
|
||||
|
||||
if (isShape && !webParams.shape) {
|
||||
throw new Error('Shape search URL is missing the required "shape" query parameter');
|
||||
}
|
||||
|
||||
if (isShape && webParams.shape) {
|
||||
const browserShape = webParams.shape;
|
||||
const normalized = browserShape.replace(/\.\./g, '==').replace(/\./g, '=');
|
||||
const polyline = Buffer.from(normalized, 'base64').toString('utf-8');
|
||||
mobileParams.shape = polyline;
|
||||
}
|
||||
|
||||
if (webParams.geocoordinates) {
|
||||
mobileParams.geocoordinates = webParams.geocoordinates;
|
||||
}
|
||||
|
||||
for (const [key, val] of Object.entries(webParams)) {
|
||||
if (key === 'shape') continue;
|
||||
if (key === 'equipment') {
|
||||
const items = [].concat(val).flatMap((v) => `${v}`.split(','));
|
||||
const currentEquipmentParams = mobileParams[PARAM_NAME_MAP[key]];
|
||||
|
||||
56
package.json
56
package.json
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "fredy",
|
||||
"version": "21.0.1",
|
||||
"version": "22.0.8",
|
||||
"description": "[F]ind [R]eal [E]states [d]amn eas[y].",
|
||||
"scripts": {
|
||||
"prepare": "husky",
|
||||
@@ -62,65 +62,65 @@
|
||||
"Firefox ESR"
|
||||
],
|
||||
"dependencies": {
|
||||
"@douyinfe/semi-icons": "^2.95.1",
|
||||
"@douyinfe/semi-ui": "2.95.1",
|
||||
"@douyinfe/semi-ui-19": "^2.95.1",
|
||||
"@douyinfe/semi-icons": "^2.97.0",
|
||||
"@douyinfe/semi-ui": "2.97.0",
|
||||
"@douyinfe/semi-ui-19": "^2.97.0",
|
||||
"@fastify/cookie": "^11.0.2",
|
||||
"@fastify/helmet": "^13.0.2",
|
||||
"@fastify/session": "^11.1.1",
|
||||
"@fastify/static": "^9.1.3",
|
||||
"@mapbox/mapbox-gl-draw": "^1.5.1",
|
||||
"@modelcontextprotocol/sdk": "^1.29.0",
|
||||
"@sendgrid/mail": "8.1.6",
|
||||
"@turf/boolean-point-in-polygon": "^7.3.5",
|
||||
"@vitejs/plugin-react": "6.0.1",
|
||||
"@vitejs/plugin-react": "6.0.2",
|
||||
"adm-zip": "^0.5.17",
|
||||
"better-sqlite3": "^12.9.0",
|
||||
"body-parser": "2.2.2",
|
||||
"better-sqlite3": "^12.10.0",
|
||||
"chart.js": "^4.5.1",
|
||||
"cheerio": "^1.2.0",
|
||||
"cookie-session": "2.1.1",
|
||||
"cloakbrowser": "^0.3.28",
|
||||
"fastify": "^5.8.5",
|
||||
"handlebars": "4.7.9",
|
||||
"maplibre-gl": "^5.24.0",
|
||||
"nanoid": "5.1.9",
|
||||
"nanoid": "5.1.11",
|
||||
"node-cron": "^4.2.1",
|
||||
"node-fetch": "3.3.2",
|
||||
"node-mailjet": "6.0.11",
|
||||
"nodemailer": "^8.0.6",
|
||||
"nodemailer": "^8.0.7",
|
||||
"p-throttle": "^8.1.0",
|
||||
"package-up": "^5.0.0",
|
||||
"puppeteer": "^24.42.0",
|
||||
"puppeteer-extra": "^3.3.6",
|
||||
"puppeteer-extra-plugin-stealth": "^2.11.2",
|
||||
"puppeteer-core": "^25.0.4",
|
||||
"query-string": "9.3.1",
|
||||
"react": "19.2.5",
|
||||
"react": "19.2.6",
|
||||
"react-chartjs-2": "^5.3.1",
|
||||
"react-dom": "19.2.5",
|
||||
"react-dom": "19.2.6",
|
||||
"react-range-slider-input": "^3.3.5",
|
||||
"react-router": "7.14.2",
|
||||
"react-router-dom": "7.14.2",
|
||||
"resend": "^6.12.2",
|
||||
"restana": "6.0.0",
|
||||
"semver": "^7.7.4",
|
||||
"serve-static": "2.2.1",
|
||||
"react-router": "7.15.1",
|
||||
"react-router-dom": "7.15.1",
|
||||
"resend": "^6.12.3",
|
||||
"semver": "^7.8.0",
|
||||
"slack": "11.0.2",
|
||||
"vite": "8.0.10",
|
||||
"vite": "8.0.13",
|
||||
"x-var": "^3.0.1",
|
||||
"zustand": "^5.0.12"
|
||||
"zustand": "^5.0.13"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@babel/core": "7.29.0",
|
||||
"@babel/eslint-parser": "7.28.6",
|
||||
"@babel/preset-env": "7.29.2",
|
||||
"@babel/preset-env": "7.29.5",
|
||||
"@babel/preset-react": "7.28.5",
|
||||
"@eslint/js": "^10.0.1",
|
||||
"chalk": "^5.6.2",
|
||||
"eslint": "10.2.1",
|
||||
"eslint": "10.4.0",
|
||||
"eslint-config-prettier": "10.1.8",
|
||||
"eslint-plugin-react": "7.37.5",
|
||||
"globals": "^17.5.0",
|
||||
"globals": "^17.6.0",
|
||||
"history": "5.3.0",
|
||||
"husky": "9.1.7",
|
||||
"less": "4.6.4",
|
||||
"lint-staged": "16.4.0",
|
||||
"lint-staged": "17.0.5",
|
||||
"nodemon": "^3.1.14",
|
||||
"prettier": "3.8.3",
|
||||
"vitest": "^4.1.5"
|
||||
"vitest": "^4.1.6"
|
||||
}
|
||||
}
|
||||
|
||||
18
test/globalSetup.js
Normal file
18
test/globalSetup.js
Normal file
@@ -0,0 +1,18 @@
|
||||
/*
|
||||
* Copyright (c) 2026 by Christian Kellner.
|
||||
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
|
||||
*/
|
||||
|
||||
import { ensureValidBinary } from '../lib/services/ensureValidBinary.js';
|
||||
|
||||
/**
|
||||
* Vitest global setup — runs once in the main process before any workers start.
|
||||
* Downloads and validates the CloakBrowser stealth Chromium binary.
|
||||
* ensureValidBinary() also removes and re-downloads partial/corrupt installations
|
||||
* so tests never fail with "Invalid file descriptor to ICU data received".
|
||||
* Skipped in offline mode because the browser is fully mocked there.
|
||||
*/
|
||||
export async function setup() {
|
||||
if (process.env.TEST_MODE === 'offline') return;
|
||||
await ensureValidBinary();
|
||||
}
|
||||
@@ -6,83 +6,89 @@
|
||||
import * as similarityCache from '../../lib/services/similarity-check/similarityCache.js';
|
||||
import { get } from '../mocks/mockNotification.js';
|
||||
import { providerConfig, mockFredy } from '../utils.js';
|
||||
import { expect, vi } from 'vitest';
|
||||
import { expect } from 'vitest';
|
||||
import * as provider from '../../lib/provider/immobilienDe.js';
|
||||
import * as mockStore from '../mocks/mockStore.js';
|
||||
import { launchBrowser, closeBrowser } from '../../lib/services/extractor/puppeteerExtractor.js';
|
||||
|
||||
// One browser shared across the whole suite so both requests (search + detail)
|
||||
// come from the same warm session, avoiding double cold-start bot detection.
|
||||
const TEST_TIMEOUT = 120_000;
|
||||
|
||||
describe('#immobilien.de testsuite()', () => {
|
||||
provider.init(providerConfig.immobilienDe, [], []);
|
||||
it('should test immobilien.de provider', async () => {
|
||||
const mockedJob = {
|
||||
id: 'test1',
|
||||
notificationAdapter: null,
|
||||
spatialFilter: null,
|
||||
specFilter: null,
|
||||
};
|
||||
|
||||
const Fredy = await mockFredy();
|
||||
const fredy = new Fredy(provider.config, mockedJob, provider.metaInformation.id, similarityCache, undefined);
|
||||
const listing = await fredy.execute();
|
||||
let browser;
|
||||
let liveListings;
|
||||
|
||||
if (listing == null || listing.length === 0) {
|
||||
throw new Error('Listings is empty!');
|
||||
}
|
||||
beforeAll(async () => {
|
||||
browser = await launchBrowser(providerConfig.immobilienDe.url);
|
||||
}, TEST_TIMEOUT);
|
||||
|
||||
expect(listing).toBeInstanceOf(Array);
|
||||
const notificationObj = get();
|
||||
expect(notificationObj).toBeTypeOf('object');
|
||||
expect(notificationObj.serviceName).toBe('immobilienDe');
|
||||
notificationObj.payload.forEach((notify) => {
|
||||
/** check the actual structure **/
|
||||
expect(notify.id).toBeTypeOf('string');
|
||||
expect(notify.price).toBeTypeOf('string');
|
||||
expect(notify.size).toBeTypeOf('string');
|
||||
expect(notify.title).toBeTypeOf('string');
|
||||
expect(notify.link).toBeTypeOf('string');
|
||||
expect(notify.address).toBeTypeOf('string');
|
||||
/** check the values if possible **/
|
||||
expect(notify.price).toContain('€');
|
||||
expect(notify.size).toContain('m²');
|
||||
expect(notify.title).not.toBe('');
|
||||
expect(notify.link).toContain('https://www.immobilien.de');
|
||||
expect(notify.address).not.toBe('');
|
||||
});
|
||||
afterAll(async () => {
|
||||
await closeBrowser(browser);
|
||||
});
|
||||
|
||||
it(
|
||||
'should test immobilien.de provider',
|
||||
async () => {
|
||||
const mockedJob = {
|
||||
id: 'test1',
|
||||
notificationAdapter: null,
|
||||
spatialFilter: null,
|
||||
specFilter: null,
|
||||
};
|
||||
|
||||
const Fredy = await mockFredy();
|
||||
const fredy = new Fredy(provider.config, mockedJob, provider.metaInformation.id, similarityCache, browser);
|
||||
liveListings = await fredy.execute();
|
||||
|
||||
if (liveListings == null || liveListings.length === 0) {
|
||||
throw new Error('Listings is empty!');
|
||||
}
|
||||
|
||||
expect(liveListings).toBeInstanceOf(Array);
|
||||
const notificationObj = get();
|
||||
expect(notificationObj).toBeTypeOf('object');
|
||||
expect(notificationObj.serviceName).toBe('immobilienDe');
|
||||
notificationObj.payload.forEach((notify) => {
|
||||
/** check the actual structure **/
|
||||
expect(notify.id).toBeTypeOf('string');
|
||||
expect(notify.price).toBeTypeOf('string');
|
||||
expect(notify.size).toBeTypeOf('string');
|
||||
expect(notify.title).toBeTypeOf('string');
|
||||
expect(notify.link).toBeTypeOf('string');
|
||||
expect(notify.address).toBeTypeOf('string');
|
||||
/** check the values if possible **/
|
||||
expect(notify.price).toContain('€');
|
||||
expect(notify.size).toContain('m²');
|
||||
expect(notify.title).not.toBe('');
|
||||
expect(notify.link).toContain('https://www.immobilien.de');
|
||||
expect(notify.address).not.toBe('');
|
||||
});
|
||||
},
|
||||
TEST_TIMEOUT,
|
||||
);
|
||||
|
||||
describe('with provider_details enabled', () => {
|
||||
beforeEach(() => {
|
||||
vi.spyOn(mockStore, 'getUserSettings').mockReturnValue({ provider_details: [provider.metaInformation.id] });
|
||||
vi.spyOn(mockStore, 'getKnownListingHashesForJobAndProvider').mockReturnValue([]);
|
||||
});
|
||||
it(
|
||||
'should enrich listings with details',
|
||||
async () => {
|
||||
if (!liveListings?.length) throw new Error('No listings from first test to enrich');
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
// Call fetchDetails directly on the first live listing — no need to
|
||||
// re-scrape the search page. The shared browser keeps the session warm.
|
||||
const enriched = await provider.config.fetchDetails(liveListings[0], browser);
|
||||
|
||||
it('should enrich listings with details', async () => {
|
||||
const Fredy = await mockFredy();
|
||||
provider.init(providerConfig.immobilienDe, [], []);
|
||||
const mockedJob = { id: 'test1', notificationAdapter: null, specFilter: null, spatialFilter: null };
|
||||
|
||||
const fredy = new Fredy(
|
||||
provider.config,
|
||||
mockedJob,
|
||||
provider.metaInformation.id,
|
||||
{ checkAndAddEntry: () => false },
|
||||
undefined,
|
||||
);
|
||||
const listings = await fredy.execute();
|
||||
if (listings == null) return;
|
||||
expect(listings).toBeInstanceOf(Array);
|
||||
listings.forEach((listing) => {
|
||||
expect(listing.link).toContain('https://www.immobilien.de');
|
||||
expect(listing.address).toBeTypeOf('string');
|
||||
expect(listing.address).not.toBe('');
|
||||
if (enriched == null) return;
|
||||
expect(enriched.link).toContain('https://www.immobilien.de');
|
||||
expect(enriched.address).toBeTypeOf('string');
|
||||
expect(enriched.address).not.toBe('');
|
||||
// description may be null if selectors don't match yet — falls back gracefully
|
||||
if (listing.description != null) {
|
||||
expect(listing.description).toBeTypeOf('string');
|
||||
if (enriched.description != null) {
|
||||
expect(enriched.description).toBeTypeOf('string');
|
||||
}
|
||||
});
|
||||
});
|
||||
},
|
||||
TEST_TIMEOUT,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -3,85 +3,85 @@
|
||||
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
|
||||
*/
|
||||
|
||||
import { expect, vi } from 'vitest';
|
||||
import { expect } from 'vitest';
|
||||
import * as similarityCache from '../../lib/services/similarity-check/similarityCache.js';
|
||||
import { mockFredy, providerConfig } from '../utils.js';
|
||||
import { get } from '../mocks/mockNotification.js';
|
||||
import * as provider from '../../lib/provider/immoscout.js';
|
||||
import * as mockStore from '../mocks/mockStore.js';
|
||||
|
||||
// immoscout uses the mobile REST API (fetch-based, no browser). Both tests share
|
||||
// the same module-level listings so the API is only queried once, avoiding
|
||||
// duplicate requests that could trigger rate-limiting.
|
||||
const TEST_TIMEOUT = 120_000;
|
||||
|
||||
describe('#immoscout provider testsuite()', () => {
|
||||
provider.init(providerConfig.immoscout, [], []);
|
||||
it('should test immoscout provider', async () => {
|
||||
const Fredy = await mockFredy();
|
||||
const mockedJob = {
|
||||
id: '',
|
||||
notificationAdapter: null,
|
||||
spatialFilter: null,
|
||||
specFilter: null,
|
||||
};
|
||||
|
||||
return await new Promise((resolve, reject) => {
|
||||
const fredy = new Fredy(provider.config, mockedJob, provider.metaInformation.id, similarityCache, undefined);
|
||||
fredy.execute().then((listings) => {
|
||||
if (listings == null || listings.length === 0) {
|
||||
reject('Listings is empty!');
|
||||
return;
|
||||
}
|
||||
let liveListings;
|
||||
|
||||
expect(listings).toBeInstanceOf(Array);
|
||||
const notificationObj = get();
|
||||
expect(notificationObj).toBeTypeOf('object');
|
||||
it(
|
||||
'should test immoscout provider',
|
||||
async () => {
|
||||
const Fredy = await mockFredy();
|
||||
const mockedJob = {
|
||||
id: '',
|
||||
notificationAdapter: null,
|
||||
spatialFilter: null,
|
||||
specFilter: null,
|
||||
};
|
||||
|
||||
// check if there is at least one valid notification
|
||||
const hasValidNotification = notificationObj.payload.some((notify) => {
|
||||
return (
|
||||
typeof notify.id === 'string' &&
|
||||
typeof notify.price === 'string' &&
|
||||
notify.price.includes('€') &&
|
||||
typeof notify.size === 'string' &&
|
||||
notify.size.includes('m²') &&
|
||||
typeof notify.title === 'string' &&
|
||||
notify.title !== '' &&
|
||||
typeof notify.link === 'string' &&
|
||||
notify.link.includes('https://www.immobilienscout24.de/') &&
|
||||
typeof notify.address === 'string'
|
||||
);
|
||||
return await new Promise((resolve, reject) => {
|
||||
const fredy = new Fredy(provider.config, mockedJob, provider.metaInformation.id, similarityCache, undefined);
|
||||
fredy.execute().then((listings) => {
|
||||
if (listings == null || listings.length === 0) {
|
||||
reject('Listings is empty!');
|
||||
return;
|
||||
}
|
||||
|
||||
liveListings = listings;
|
||||
expect(listings).toBeInstanceOf(Array);
|
||||
const notificationObj = get();
|
||||
expect(notificationObj).toBeTypeOf('object');
|
||||
|
||||
// check if there is at least one valid notification
|
||||
const hasValidNotification = notificationObj.payload.some((notify) => {
|
||||
return (
|
||||
typeof notify.id === 'string' &&
|
||||
typeof notify.price === 'string' &&
|
||||
notify.price.includes('€') &&
|
||||
typeof notify.size === 'string' &&
|
||||
notify.size.includes('m²') &&
|
||||
typeof notify.title === 'string' &&
|
||||
notify.title !== '' &&
|
||||
typeof notify.link === 'string' &&
|
||||
notify.link.includes('https://www.immobilienscout24.de/') &&
|
||||
typeof notify.address === 'string'
|
||||
);
|
||||
});
|
||||
|
||||
expect(hasValidNotification).toBe(true);
|
||||
resolve();
|
||||
});
|
||||
|
||||
expect(hasValidNotification).toBe(true);
|
||||
resolve();
|
||||
});
|
||||
});
|
||||
});
|
||||
},
|
||||
TEST_TIMEOUT,
|
||||
);
|
||||
|
||||
describe('with provider_details enabled', () => {
|
||||
beforeEach(() => {
|
||||
vi.spyOn(mockStore, 'getUserSettings').mockReturnValue({ provider_details: [provider.metaInformation.id] });
|
||||
vi.spyOn(mockStore, 'getKnownListingHashesForJobAndProvider').mockReturnValue([]);
|
||||
});
|
||||
it(
|
||||
'should enrich listings with details',
|
||||
async () => {
|
||||
if (!liveListings?.length) throw new Error('No listings from first test to enrich');
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
// Call fetchDetails directly on the first live listing — no need to
|
||||
// re-query the search API. immoscout uses fetch (no browser).
|
||||
const enriched = await provider.config.fetchDetails(liveListings[0]);
|
||||
|
||||
it('should enrich listings with details', async () => {
|
||||
const Fredy = await mockFredy();
|
||||
provider.init(providerConfig.immoscout, [], []);
|
||||
const mockedJob = { id: '', notificationAdapter: null, specFilter: null, spatialFilter: null };
|
||||
const fredy = new Fredy(
|
||||
provider.config,
|
||||
mockedJob,
|
||||
provider.metaInformation.id,
|
||||
{ checkAndAddEntry: () => false },
|
||||
undefined,
|
||||
);
|
||||
const listings = await fredy.execute();
|
||||
expect(listings).toBeInstanceOf(Array);
|
||||
listings.forEach((listing) => {
|
||||
expect(listing.description).toBeTypeOf('string');
|
||||
expect(listing.description).not.toBe('');
|
||||
});
|
||||
});
|
||||
expect(enriched).toBeTruthy();
|
||||
expect(enriched.description).toBeTypeOf('string');
|
||||
expect(enriched.description).not.toBe('');
|
||||
},
|
||||
TEST_TIMEOUT,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -6,87 +6,95 @@
|
||||
import * as similarityCache from '../../lib/services/similarity-check/similarityCache.js';
|
||||
import { get } from '../mocks/mockNotification.js';
|
||||
import { mockFredy, providerConfig } from '../utils.js';
|
||||
import { expect, vi } from 'vitest';
|
||||
import { expect } from 'vitest';
|
||||
import * as provider from '../../lib/provider/immowelt.js';
|
||||
import * as mockStore from '../mocks/mockStore.js';
|
||||
import { launchBrowser, closeBrowser } from '../../lib/services/extractor/puppeteerExtractor.js';
|
||||
|
||||
// One browser shared across the whole suite so both requests (search + detail)
|
||||
// come from the same warm session. Immowelt's CDN challenges cold sessions
|
||||
// aggressively; a shared warm browser prevents the second request from being
|
||||
// blocked as a bot hit.
|
||||
const TEST_TIMEOUT = 180_000;
|
||||
|
||||
describe('#immowelt testsuite()', () => {
|
||||
it('should test immowelt provider', async () => {
|
||||
const Fredy = await mockFredy();
|
||||
const mockedJob = {
|
||||
id: 'immowelt',
|
||||
notificationAdapter: null,
|
||||
spatialFilter: null,
|
||||
specFilter: null,
|
||||
};
|
||||
provider.init(providerConfig.immowelt, [], []);
|
||||
let browser;
|
||||
let liveListings;
|
||||
|
||||
const fredy = new Fredy(provider.config, mockedJob, provider.metaInformation.id, similarityCache, undefined);
|
||||
beforeAll(async () => {
|
||||
browser = await launchBrowser(providerConfig.immowelt.url);
|
||||
}, TEST_TIMEOUT);
|
||||
|
||||
const listing = await fredy.execute();
|
||||
|
||||
if (listing == null || listing.length === 0) {
|
||||
throw new Error('Listings is empty!');
|
||||
}
|
||||
|
||||
expect(listing).toBeInstanceOf(Array);
|
||||
const notificationObj = get();
|
||||
expect(notificationObj).toBeTypeOf('object');
|
||||
expect(notificationObj.serviceName).toBe('immowelt');
|
||||
notificationObj.payload.forEach((notify) => {
|
||||
/** check the actual structure **/
|
||||
expect(notify.id).toBeTypeOf('string');
|
||||
if (notify.price != null) {
|
||||
expect(notify.price).toBeTypeOf('string');
|
||||
expect(notify.price).toContain('€');
|
||||
}
|
||||
expect(notify.title).toBeTypeOf('string');
|
||||
expect(notify.link).toBeTypeOf('string');
|
||||
expect(notify.address).toBeTypeOf('string');
|
||||
/** check the values if possible **/
|
||||
if (notify.size != null && notify.size.trim().toLowerCase() !== 'k.a.') {
|
||||
expect(notify.size).toBeTypeOf('string');
|
||||
expect(notify.size).toContain('m²');
|
||||
}
|
||||
expect(notify.title).not.toBe('');
|
||||
expect(notify.link).toContain('https://www.immowelt.de');
|
||||
expect(notify.address).not.toBe('');
|
||||
});
|
||||
afterAll(async () => {
|
||||
await closeBrowser(browser);
|
||||
});
|
||||
|
||||
it(
|
||||
'should test immowelt provider',
|
||||
async () => {
|
||||
const Fredy = await mockFredy();
|
||||
const mockedJob = {
|
||||
id: 'immowelt',
|
||||
notificationAdapter: null,
|
||||
spatialFilter: null,
|
||||
specFilter: null,
|
||||
};
|
||||
provider.init(providerConfig.immowelt, [], []);
|
||||
|
||||
const fredy = new Fredy(provider.config, mockedJob, provider.metaInformation.id, similarityCache, browser);
|
||||
|
||||
liveListings = await fredy.execute();
|
||||
|
||||
if (liveListings == null || liveListings.length === 0) {
|
||||
throw new Error('Listings is empty!');
|
||||
}
|
||||
|
||||
expect(liveListings).toBeInstanceOf(Array);
|
||||
const notificationObj = get();
|
||||
expect(notificationObj).toBeTypeOf('object');
|
||||
expect(notificationObj.serviceName).toBe('immowelt');
|
||||
notificationObj.payload.forEach((notify) => {
|
||||
/** check the actual structure **/
|
||||
expect(notify.id).toBeTypeOf('string');
|
||||
if (notify.price != null) {
|
||||
expect(notify.price).toBeTypeOf('string');
|
||||
expect(notify.price).toContain('€');
|
||||
}
|
||||
expect(notify.title).toBeTypeOf('string');
|
||||
expect(notify.link).toBeTypeOf('string');
|
||||
expect(notify.address).toBeTypeOf('string');
|
||||
/** check the values if possible **/
|
||||
if (notify.size != null && notify.size.trim().toLowerCase() !== 'k.a.') {
|
||||
expect(notify.size).toBeTypeOf('string');
|
||||
expect(notify.size).toContain('m²');
|
||||
}
|
||||
expect(notify.title).not.toBe('');
|
||||
expect(notify.link).toContain('https://www.immowelt.de');
|
||||
expect(notify.address).not.toBe('');
|
||||
});
|
||||
},
|
||||
TEST_TIMEOUT,
|
||||
);
|
||||
|
||||
describe('with provider_details enabled', () => {
|
||||
beforeEach(() => {
|
||||
vi.spyOn(mockStore, 'getUserSettings').mockReturnValue({ provider_details: [provider.metaInformation.id] });
|
||||
vi.spyOn(mockStore, 'getKnownListingHashesForJobAndProvider').mockReturnValue([]);
|
||||
});
|
||||
it(
|
||||
'should enrich listings with details',
|
||||
async () => {
|
||||
if (!liveListings?.length) throw new Error('No listings from first test to enrich');
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
// Call fetchDetails directly on the first live listing — no need to
|
||||
// re-scrape the search page. The shared browser keeps the session warm.
|
||||
const enriched = await provider.config.fetchDetails(liveListings[0], browser);
|
||||
|
||||
it('should enrich listings with details', async () => {
|
||||
const Fredy = await mockFredy();
|
||||
provider.init(providerConfig.immowelt, [], []);
|
||||
const mockedJob = { id: 'immowelt', notificationAdapter: null, specFilter: null, spatialFilter: null };
|
||||
|
||||
const fredy = new Fredy(
|
||||
provider.config,
|
||||
mockedJob,
|
||||
provider.metaInformation.id,
|
||||
{ checkAndAddEntry: () => false },
|
||||
undefined,
|
||||
);
|
||||
const listings = await fredy.execute();
|
||||
expect(listings).toBeInstanceOf(Array);
|
||||
listings.forEach((listing) => {
|
||||
expect(listing.link).toContain('https://www.immowelt.de');
|
||||
expect(listing.address).toBeTypeOf('string');
|
||||
expect(listing.address).not.toBe('');
|
||||
expect(enriched).toBeTruthy();
|
||||
expect(enriched.link).toContain('https://www.immowelt.de');
|
||||
expect(enriched.address).toBeTypeOf('string');
|
||||
expect(enriched.address).not.toBe('');
|
||||
// description is enriched from the detail page; falls back gracefully if blocked
|
||||
if (listing.description != null) {
|
||||
expect(listing.description).toBeTypeOf('string');
|
||||
if (enriched.description != null) {
|
||||
expect(enriched.description).toBeTypeOf('string');
|
||||
}
|
||||
});
|
||||
});
|
||||
},
|
||||
TEST_TIMEOUT,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -6,80 +6,88 @@
|
||||
import * as similarityCache from '../../lib/services/similarity-check/similarityCache.js';
|
||||
import { get } from '../mocks/mockNotification.js';
|
||||
import { mockFredy, providerConfig } from '../utils.js';
|
||||
import { expect, vi } from 'vitest';
|
||||
import { expect } from 'vitest';
|
||||
import * as provider from '../../lib/provider/kleinanzeigen.js';
|
||||
import * as mockStore from '../mocks/mockStore.js';
|
||||
import { launchBrowser, closeBrowser } from '../../lib/services/extractor/puppeteerExtractor.js';
|
||||
|
||||
// One browser shared across the whole suite so both requests (search + detail)
|
||||
// come from the same warm session. Kleinanzeigen rate-limits cold browser
|
||||
// sessions; a shared warm browser prevents the second request from being blocked.
|
||||
const TEST_TIMEOUT = 180_000;
|
||||
|
||||
describe('#kleinanzeigen testsuite()', () => {
|
||||
it('should test kleinanzeigen provider', async () => {
|
||||
const Fredy = await mockFredy();
|
||||
const mockedJob = {
|
||||
id: 'kleinanzeigen',
|
||||
notificationAdapter: null,
|
||||
spatialFilter: null,
|
||||
specFilter: null,
|
||||
};
|
||||
provider.init(providerConfig.kleinanzeigen, [], []);
|
||||
return await new Promise((resolve, reject) => {
|
||||
const fredy = new Fredy(provider.config, mockedJob, provider.metaInformation.id, similarityCache, undefined);
|
||||
let browser;
|
||||
let liveListings;
|
||||
|
||||
fredy.execute().then((listing) => {
|
||||
if (listing == null || listing.length === 0) {
|
||||
reject('Listings is empty!');
|
||||
return;
|
||||
}
|
||||
beforeAll(async () => {
|
||||
browser = await launchBrowser(providerConfig.kleinanzeigen.url);
|
||||
}, TEST_TIMEOUT);
|
||||
|
||||
expect(listing).toBeInstanceOf(Array);
|
||||
const notificationObj = get();
|
||||
expect(notificationObj).toBeTypeOf('object');
|
||||
expect(notificationObj.serviceName).toBe('kleinanzeigen');
|
||||
notificationObj.payload.forEach((notify) => {
|
||||
/** check the actual structure **/
|
||||
expect(notify.id).toBeTypeOf('string');
|
||||
expect(notify.title).toBeTypeOf('string');
|
||||
expect(notify.link).toBeTypeOf('string');
|
||||
expect(notify.address).toBeTypeOf('string');
|
||||
/** check the values if possible **/
|
||||
expect(notify.title).not.toBe('');
|
||||
expect(notify.link).toContain('https://www.kleinanzeigen.de');
|
||||
expect(notify.address).not.toBe('');
|
||||
});
|
||||
resolve();
|
||||
});
|
||||
});
|
||||
afterAll(async () => {
|
||||
await closeBrowser(browser);
|
||||
});
|
||||
|
||||
it(
|
||||
'should test kleinanzeigen provider',
|
||||
async () => {
|
||||
const Fredy = await mockFredy();
|
||||
const mockedJob = {
|
||||
id: 'kleinanzeigen',
|
||||
notificationAdapter: null,
|
||||
spatialFilter: null,
|
||||
specFilter: null,
|
||||
};
|
||||
provider.init(providerConfig.kleinanzeigen, [], []);
|
||||
return await new Promise((resolve, reject) => {
|
||||
const fredy = new Fredy(provider.config, mockedJob, provider.metaInformation.id, similarityCache, browser);
|
||||
|
||||
fredy.execute().then((listing) => {
|
||||
if (listing == null || listing.length === 0) {
|
||||
reject('Listings is empty!');
|
||||
return;
|
||||
}
|
||||
|
||||
liveListings = listing;
|
||||
expect(listing).toBeInstanceOf(Array);
|
||||
const notificationObj = get();
|
||||
expect(notificationObj).toBeTypeOf('object');
|
||||
expect(notificationObj.serviceName).toBe('kleinanzeigen');
|
||||
notificationObj.payload.forEach((notify) => {
|
||||
/** check the actual structure **/
|
||||
expect(notify.id).toBeTypeOf('string');
|
||||
expect(notify.title).toBeTypeOf('string');
|
||||
expect(notify.link).toBeTypeOf('string');
|
||||
expect(notify.address).toBeTypeOf('string');
|
||||
/** check the values if possible **/
|
||||
expect(notify.title).not.toBe('');
|
||||
expect(notify.link).toContain('https://www.kleinanzeigen.de');
|
||||
expect(notify.address).not.toBe('');
|
||||
});
|
||||
resolve();
|
||||
});
|
||||
});
|
||||
},
|
||||
TEST_TIMEOUT,
|
||||
);
|
||||
|
||||
describe('with provider_details enabled', () => {
|
||||
beforeEach(() => {
|
||||
vi.spyOn(mockStore, 'getUserSettings').mockReturnValue({ provider_details: [provider.metaInformation.id] });
|
||||
vi.spyOn(mockStore, 'getKnownListingHashesForJobAndProvider').mockReturnValue([]);
|
||||
});
|
||||
it(
|
||||
'should enrich listings with details',
|
||||
async () => {
|
||||
if (!liveListings?.length) throw new Error('No listings from first test to enrich');
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
// Call fetchDetails directly on the first live listing — no need to
|
||||
// re-scrape the search page. The shared browser keeps the session warm.
|
||||
const enriched = await provider.config.fetchDetails(liveListings[0], browser);
|
||||
|
||||
it('should enrich listings with details', async () => {
|
||||
const Fredy = await mockFredy();
|
||||
provider.init(providerConfig.kleinanzeigen, [], []);
|
||||
const mockedJob = { id: 'kleinanzeigen', notificationAdapter: null, specFilter: null, spatialFilter: null };
|
||||
|
||||
const fredy = new Fredy(
|
||||
provider.config,
|
||||
mockedJob,
|
||||
provider.metaInformation.id,
|
||||
{ checkAndAddEntry: () => false },
|
||||
undefined,
|
||||
);
|
||||
const listings = await fredy.execute();
|
||||
expect(listings).toBeInstanceOf(Array);
|
||||
listings.forEach((listing) => {
|
||||
expect(listing.link).toContain('https://www.kleinanzeigen.de');
|
||||
expect(listing.address).toBeTypeOf('string');
|
||||
expect(listing.address).not.toBe('');
|
||||
expect(listing.description).toBeTypeOf('string');
|
||||
expect(listing.description).not.toBe('');
|
||||
});
|
||||
});
|
||||
expect(enriched).toBeTruthy();
|
||||
expect(enriched.link).toContain('https://www.kleinanzeigen.de');
|
||||
expect(enriched.address).toBeTypeOf('string');
|
||||
expect(enriched.address).not.toBe('');
|
||||
expect(enriched.description).toBeTypeOf('string');
|
||||
expect(enriched.description).not.toBe('');
|
||||
},
|
||||
TEST_TIMEOUT,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -9,81 +9,97 @@ import { mockFredy, providerConfig } from '../utils.js';
|
||||
import { expect, vi } from 'vitest';
|
||||
import * as provider from '../../lib/provider/sparkasse.js';
|
||||
import * as mockStore from '../mocks/mockStore.js';
|
||||
import { launchBrowser, closeBrowser } from '../../lib/services/extractor/puppeteerExtractor.js';
|
||||
|
||||
// One browser shared across the whole suite so both requests (search + detail)
|
||||
// come from the same warm session. This prevents the second request from being
|
||||
// flagged as a cold-start bot hit.
|
||||
const TEST_TIMEOUT = 120_000;
|
||||
|
||||
describe('#sparkasse testsuite()', () => {
|
||||
it('should test sparkasse provider', async () => {
|
||||
const Fredy = await mockFredy();
|
||||
const mockedJob = {
|
||||
id: 'sparkasse',
|
||||
notificationAdapter: null,
|
||||
spatialFilter: null,
|
||||
specFilter: null,
|
||||
};
|
||||
provider.init(providerConfig.sparkasse, []);
|
||||
let browser;
|
||||
let liveListings;
|
||||
|
||||
const fredy = new Fredy(provider.config, mockedJob, provider.metaInformation.id, similarityCache, undefined);
|
||||
beforeAll(async () => {
|
||||
browser = await launchBrowser(providerConfig.sparkasse.url);
|
||||
}, TEST_TIMEOUT);
|
||||
|
||||
const listing = await fredy.execute();
|
||||
|
||||
if (listing == null || listing.length === 0) {
|
||||
throw new Error('Listings is empty!');
|
||||
}
|
||||
|
||||
expect(listing).toBeInstanceOf(Array);
|
||||
const notificationObj = get();
|
||||
expect(notificationObj).toBeTypeOf('object');
|
||||
expect(notificationObj.serviceName).toBe('sparkasse');
|
||||
notificationObj.payload.forEach((notify) => {
|
||||
/** check the actual structure **/
|
||||
expect(notify.id).toBeTypeOf('string');
|
||||
expect(notify.price).toBeTypeOf('string');
|
||||
expect(notify.price).toContain('€');
|
||||
expect(notify.size).toBeTypeOf('string');
|
||||
expect(notify.size).toContain('m²');
|
||||
expect(notify.title).toBeTypeOf('string');
|
||||
expect(notify.link).toBeTypeOf('string');
|
||||
expect(notify.address).toBeTypeOf('string');
|
||||
/** check the values if possible **/
|
||||
expect(notify.size).toBeTypeOf('string');
|
||||
expect(notify.title).not.toBe('');
|
||||
expect(notify.address).not.toBe('');
|
||||
});
|
||||
afterAll(async () => {
|
||||
await closeBrowser(browser);
|
||||
});
|
||||
|
||||
it(
|
||||
'should test sparkasse provider',
|
||||
async () => {
|
||||
const Fredy = await mockFredy();
|
||||
const mockedJob = {
|
||||
id: 'sparkasse',
|
||||
notificationAdapter: null,
|
||||
spatialFilter: null,
|
||||
specFilter: null,
|
||||
};
|
||||
provider.init(providerConfig.sparkasse, []);
|
||||
|
||||
const fredy = new Fredy(provider.config, mockedJob, provider.metaInformation.id, similarityCache, browser);
|
||||
|
||||
liveListings = await fredy.execute();
|
||||
|
||||
if (liveListings == null || liveListings.length === 0) {
|
||||
throw new Error('Listings is empty!');
|
||||
}
|
||||
|
||||
expect(liveListings).toBeInstanceOf(Array);
|
||||
const notificationObj = get();
|
||||
expect(notificationObj).toBeTypeOf('object');
|
||||
expect(notificationObj.serviceName).toBe('sparkasse');
|
||||
notificationObj.payload.forEach((notify) => {
|
||||
/** check the actual structure **/
|
||||
expect(notify.id).toBeTypeOf('string');
|
||||
expect(notify.price).toBeTypeOf('string');
|
||||
expect(notify.price).toContain('€');
|
||||
expect(notify.size).toBeTypeOf('string');
|
||||
expect(notify.size).toContain('m²');
|
||||
expect(notify.title).toBeTypeOf('string');
|
||||
expect(notify.link).toBeTypeOf('string');
|
||||
expect(notify.address).toBeTypeOf('string');
|
||||
/** check the values if possible **/
|
||||
expect(notify.size).toBeTypeOf('string');
|
||||
expect(notify.title).not.toBe('');
|
||||
expect(notify.address).not.toBe('');
|
||||
});
|
||||
},
|
||||
TEST_TIMEOUT,
|
||||
);
|
||||
|
||||
describe('with provider_details enabled', () => {
|
||||
beforeEach(() => {
|
||||
vi.spyOn(mockStore, 'getUserSettings').mockReturnValue({ provider_details: [provider.metaInformation.id] });
|
||||
vi.spyOn(mockStore, 'getKnownListingHashesForJobAndProvider').mockReturnValue([]);
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
it('should enrich listings with details', async () => {
|
||||
const Fredy = await mockFredy();
|
||||
provider.init(providerConfig.sparkasse, []);
|
||||
const mockedJob = { id: 'sparkasse', notificationAdapter: null, specFilter: null, spatialFilter: null };
|
||||
it(
|
||||
'should enrich listings with details',
|
||||
async () => {
|
||||
if (!liveListings?.length) throw new Error('No listings from first test to enrich');
|
||||
|
||||
const fredy = new Fredy(
|
||||
provider.config,
|
||||
mockedJob,
|
||||
provider.metaInformation.id,
|
||||
{ checkAndAddEntry: () => false },
|
||||
undefined,
|
||||
);
|
||||
const listings = await fredy.execute();
|
||||
expect(listings).toBeInstanceOf(Array);
|
||||
listings.forEach((listing) => {
|
||||
expect(listing.link).toContain('https://immobilien.sparkasse.de');
|
||||
expect(listing.address).toBeTypeOf('string');
|
||||
expect(listing.address).not.toBe('');
|
||||
// description is enriched from the detail page; falls back gracefully if bot-detected
|
||||
if (listing.description != null) {
|
||||
expect(listing.description).toBeTypeOf('string');
|
||||
expect(listing.description).not.toBe('');
|
||||
// Call fetchDetails directly on the first live listing — no need to
|
||||
// re-scrape the search page. The shared browser keeps the session warm.
|
||||
const enriched = await provider.config.fetchDetails(liveListings[0], browser);
|
||||
|
||||
expect(enriched).toBeTruthy();
|
||||
expect(enriched.link).toContain('https://immobilien.sparkasse.de');
|
||||
expect(enriched.address).toBeTypeOf('string');
|
||||
expect(enriched.address).not.toBe('');
|
||||
// description is enriched from the detail page; falls back gracefully if blocked
|
||||
if (enriched.description != null) {
|
||||
expect(enriched.description).toBeTypeOf('string');
|
||||
expect(enriched.description).not.toBe('');
|
||||
}
|
||||
});
|
||||
});
|
||||
},
|
||||
TEST_TIMEOUT,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -6,77 +6,85 @@
|
||||
import * as similarityCache from '../../lib/services/similarity-check/similarityCache.js';
|
||||
import { get } from '../mocks/mockNotification.js';
|
||||
import { mockFredy, providerConfig } from '../utils.js';
|
||||
import { expect, vi } from 'vitest';
|
||||
import { expect } from 'vitest';
|
||||
import * as provider from '../../lib/provider/wgGesucht.js';
|
||||
import * as mockStore from '../mocks/mockStore.js';
|
||||
import { launchBrowser, closeBrowser } from '../../lib/services/extractor/puppeteerExtractor.js';
|
||||
|
||||
// One browser shared across the whole suite so both requests (search + detail)
|
||||
// come from the same warm session, avoiding double cold-start bot detection.
|
||||
const TEST_TIMEOUT = 120_000;
|
||||
|
||||
describe('#wgGesucht testsuite()', () => {
|
||||
provider.init(providerConfig.wgGesucht, [], []);
|
||||
it('should test wgGesucht provider', { timeout: 120000 }, async () => {
|
||||
const Fredy = await mockFredy();
|
||||
const mockedJob = {
|
||||
id: 'wgGesucht',
|
||||
notificationAdapter: null,
|
||||
spatialFilter: null,
|
||||
specFilter: null,
|
||||
};
|
||||
|
||||
return await new Promise((resolve, reject) => {
|
||||
const fredy = new Fredy(provider.config, mockedJob, provider.metaInformation.id, similarityCache, undefined);
|
||||
let browser;
|
||||
let liveListings;
|
||||
|
||||
fredy.execute().then((listing) => {
|
||||
if (listing == null || listing.length === 0) {
|
||||
reject('Listings is empty!');
|
||||
return;
|
||||
}
|
||||
beforeAll(async () => {
|
||||
browser = await launchBrowser(providerConfig.wgGesucht.url);
|
||||
}, TEST_TIMEOUT);
|
||||
|
||||
expect(listing).toBeInstanceOf(Array);
|
||||
const notificationObj = get();
|
||||
expect(notificationObj.serviceName).toBe('wgGesucht');
|
||||
notificationObj.payload.forEach((notify) => {
|
||||
expect(notify).toBeTypeOf('object');
|
||||
/** check the actual structure **/
|
||||
expect(notify.id).toBeTypeOf('string');
|
||||
expect(notify.title).toBeTypeOf('string');
|
||||
// expect(notify.details).toBeTypeOf('string');
|
||||
expect(notify.price).toBeTypeOf('string');
|
||||
expect(notify.price).toContain('€');
|
||||
expect(notify.link).toBeTypeOf('string');
|
||||
});
|
||||
resolve();
|
||||
});
|
||||
});
|
||||
afterAll(async () => {
|
||||
await closeBrowser(browser);
|
||||
});
|
||||
|
||||
it(
|
||||
'should test wgGesucht provider',
|
||||
async () => {
|
||||
const Fredy = await mockFredy();
|
||||
const mockedJob = {
|
||||
id: 'wgGesucht',
|
||||
notificationAdapter: null,
|
||||
spatialFilter: null,
|
||||
specFilter: null,
|
||||
};
|
||||
|
||||
return await new Promise((resolve, reject) => {
|
||||
const fredy = new Fredy(provider.config, mockedJob, provider.metaInformation.id, similarityCache, browser);
|
||||
|
||||
fredy.execute().then((listing) => {
|
||||
if (listing == null || listing.length === 0) {
|
||||
reject('Listings is empty!');
|
||||
return;
|
||||
}
|
||||
|
||||
liveListings = listing;
|
||||
expect(listing).toBeInstanceOf(Array);
|
||||
const notificationObj = get();
|
||||
expect(notificationObj.serviceName).toBe('wgGesucht');
|
||||
notificationObj.payload.forEach((notify) => {
|
||||
expect(notify).toBeTypeOf('object');
|
||||
/** check the actual structure **/
|
||||
expect(notify.id).toBeTypeOf('string');
|
||||
expect(notify.title).toBeTypeOf('string');
|
||||
// expect(notify.details).toBeTypeOf('string');
|
||||
expect(notify.price).toBeTypeOf('string');
|
||||
expect(notify.price).toContain('€');
|
||||
expect(notify.link).toBeTypeOf('string');
|
||||
});
|
||||
resolve();
|
||||
});
|
||||
});
|
||||
},
|
||||
TEST_TIMEOUT,
|
||||
);
|
||||
|
||||
describe('with provider_details enabled', () => {
|
||||
beforeEach(() => {
|
||||
vi.spyOn(mockStore, 'getUserSettings').mockReturnValue({ provider_details: [provider.metaInformation.id] });
|
||||
vi.spyOn(mockStore, 'getKnownListingHashesForJobAndProvider').mockReturnValue([]);
|
||||
});
|
||||
it(
|
||||
'should enrich listings with details',
|
||||
async () => {
|
||||
if (!liveListings?.length) throw new Error('No listings from first test to enrich');
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
// Call fetchDetails directly on the first live listing — no need to
|
||||
// re-scrape the search page. The shared browser keeps the session warm.
|
||||
const enriched = await provider.config.fetchDetails(liveListings[0], browser);
|
||||
|
||||
it('should enrich listings with details', async () => {
|
||||
const Fredy = await mockFredy();
|
||||
provider.init(providerConfig.wgGesucht, [], []);
|
||||
const mockedJob = { id: 'wgGesucht', notificationAdapter: null, specFilter: null, spatialFilter: null };
|
||||
|
||||
const fredy = new Fredy(
|
||||
provider.config,
|
||||
mockedJob,
|
||||
provider.metaInformation.id,
|
||||
{ checkAndAddEntry: () => false },
|
||||
undefined,
|
||||
);
|
||||
const listings = await fredy.execute();
|
||||
expect(listings).toBeInstanceOf(Array);
|
||||
listings.forEach((listing) => {
|
||||
expect(listing.link).toContain('https://www.wg-gesucht.de');
|
||||
expect(listing.description).toBeTypeOf('string');
|
||||
expect(listing.description).not.toBe('');
|
||||
});
|
||||
});
|
||||
expect(enriched).toBeTruthy();
|
||||
expect(enriched.link).toContain('https://www.wg-gesucht.de');
|
||||
expect(enriched.description).toBeTypeOf('string');
|
||||
expect(enriched.description).not.toBe('');
|
||||
},
|
||||
TEST_TIMEOUT,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -10,6 +10,17 @@ import { readFile } from 'fs/promises';
|
||||
export const testData = JSON.parse(await readFile(new URL('./testdata.json', import.meta.url)));
|
||||
|
||||
describe('#immoscout-mobile URL conversion', () => {
|
||||
// Test shape URL conversion
|
||||
it('should convert a full web URL with shape to mobile URL', () => {
|
||||
const webUrl =
|
||||
'https://www.immobilienscout24.de/Suche/shape/haus-kaufen?shape=aW9yfkhfa3htQXJgUGlnYEBmekhte3BAcXNAfWBsQGNyQ2lkUHVvbEB3eX5Ab25WYn5Fa2BLaGRQY29FaGtTfEhme3xBdHBEdHFMamlHbmdRfHhMcmxPeHlWYnpS&price=-600000.0&ground=240.0-&enteredFrom=result_list';
|
||||
const expectedMobileUrl =
|
||||
'https://api.mobile.immobilienscout24.de/search/list?ground=240.0-&price=-600000.0&realestatetype=housebuy&searchType=shape&shape=ior~H_kxmAr%60Pig%60%40fzHm%7Bp%40qs%40%7D%60l%40crCidPuol%40wy~%40onVb~Ek%60KhdPcoEhkS%7CHf%7B%7CAtpDtqLjiGngQ%7CxLrlOxyVbzR';
|
||||
|
||||
const actualMobileUrl = convertWebToMobile(webUrl);
|
||||
expect(actualMobileUrl).toBe(expectedMobileUrl);
|
||||
});
|
||||
|
||||
// Test URL conversion
|
||||
it('should convert a full web URL to mobile URL', () => {
|
||||
const webUrl =
|
||||
|
||||
@@ -18,5 +18,9 @@
|
||||
"rentHouse": {
|
||||
"url": "https://www.immobilienscout24.de/Suche/de/nordrhein-westfalen/duesseldorf/haus-mieten?enteredFrom=one_step_search",
|
||||
"type": "houserent"
|
||||
},
|
||||
"buyHouseWithShape": {
|
||||
"url": "https://www.immobilienscout24.de/Suche/shape/haus-kaufen?shape=aW9yfkhfa3htQXJgUGlnYEBmekhte3BAcXNAfWBsQGNyQ2lkUHVvbEB3eX5Ab25WYn5Fa2BLaGRQY29FaGtTfEhme3xBdHBEdHFMamlHbmdRfHhMcmxPeHlWYnpS&price=-600000.0&ground=240.0-&enteredFrom=result_list",
|
||||
"type": "housebuy"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -38,6 +38,7 @@ describe('services/jobs/jobExecutionService', () => {
|
||||
}));
|
||||
vi.doMock(utilsPath, () => ({
|
||||
duringWorkingHoursOrNotSet: () => false,
|
||||
getPackageVersion: async () => '0.0.0-test',
|
||||
}));
|
||||
vi.doMock(loggerPath, () => {
|
||||
const m = { debug: () => {}, info: () => {}, warn: () => {}, error: () => {} };
|
||||
|
||||
10
test/testFixtures/immobilienDe.html
vendored
10
test/testFixtures/immobilienDe.html
vendored
@@ -713,7 +713,7 @@
|
||||
|
||||
</div><!-- /srb-filters -->
|
||||
|
||||
<!-- Search button — CMS handles via onclick="{{submit()}}"; srbDoSearch() is
|
||||
<!-- Search button - CMS handles via onclick="{{submit()}}"; srbDoSearch() is
|
||||
a standalone fallback that skips empty params (used outside CMS context) -->
|
||||
<button type="button" class="srb-search-btn" id="srbSearchBtn">
|
||||
<svg width="17" height="17" viewBox="0 0 24 24" fill="none" aria-hidden="true">
|
||||
@@ -740,7 +740,7 @@
|
||||
</div><!-- /srb-wrap -->
|
||||
|
||||
<!-- ═══════════════════════════════════════════════════════════
|
||||
MODAL — WEITERE FILTER
|
||||
MODAL - WEITERE FILTER
|
||||
═══════════════════════════════════════════════════════════ -->
|
||||
<!-- Screen-reader live region: announces filter changes and modal state -->
|
||||
<div id="srbLiveRegion" role="status" aria-live="polite" aria-atomic="true" style="position:absolute;width:1px;height:1px;overflow:hidden;clip:rect(0,0,0,0);white-space:nowrap"></div>
|
||||
@@ -768,7 +768,7 @@
|
||||
<circle cx="12" cy="10" r="3" stroke="currentColor" stroke-width="2"></circle>
|
||||
</svg>
|
||||
<input type="text" class="srb-modal-location-input" id="srbModalLocation" placeholder="Ort, Postleitzahl, Stadt" autocomplete="off" b-event="keyup" role="combobox" aria-autocomplete="list" aria-controls="srbModalLocationList" aria-expanded="false">
|
||||
<!-- Mirror of the desktop grbWoHidden — Piglet updates this via b-model
|
||||
<!-- Mirror of the desktop grbWoHidden - Piglet updates this via b-model
|
||||
when select(entry) fires in the modal controller's own scope. -->
|
||||
<input type="hidden" id="srbModalWoHidden" value="district:2434,2695,2621,2700,2967,2734,2909,2955,2392,2746,2767,2982,2904,2612,2892,2587,2871,2975,2591,2887,2569,2640,2735">
|
||||
<div b-redrawable="autocomplete" class="srb-autocomplete-wrapper">
|
||||
@@ -899,7 +899,7 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- ── Additional criteria — CMS server-side rendered per typ+objektart ── -->
|
||||
<!-- ── Additional criteria - CMS server-side rendered per typ+objektart ── -->
|
||||
<div class="srb-modal-criteria-groups">
|
||||
<div class="srb-modal-section srb-modal-section--animated srb-modal-section--criteria" data-valid-searches="["kaufen_grundstueck","kaufen_haus","kaufen_rendite","kaufen_wohnung","mieten_grundstueck","mieten_haus","mieten_waz","mieten_wohnung"]">
|
||||
<div class="srb-section-body">
|
||||
@@ -4393,7 +4393,7 @@
|
||||
void g.offsetHeight;
|
||||
g.classList.remove('lr-card__gallery--no-transition');
|
||||
|
||||
// counter total (CMS doesn't evaluate expressions in attrs — read from hidden span)
|
||||
// counter total (CMS doesn't evaluate expressions in attrs - read from hidden span)
|
||||
var totalEl = g.querySelector('.lr-card__gallery-total');
|
||||
var counter = g.querySelector('.lr-card__gallery-counter');
|
||||
if (counter && totalEl) counter.dataset.total = totalEl.textContent.trim();
|
||||
|
||||
2
test/testFixtures/ohneMakler.html
vendored
2
test/testFixtures/ohneMakler.html
vendored
@@ -1523,7 +1523,7 @@
|
||||
|
||||
|
||||
<div class="hidden print:flex print:items-center print:justify-between mb-2">
|
||||
<img src="/static/img/brand/logo.b187f1e54302.svg" class="w-32" alt="ohne-makler — Immobilien selbst vermarkten">
|
||||
<img src="/static/img/brand/logo.b187f1e54302.svg" class="w-32" alt="ohne-makler - Immobilien selbst vermarkten">
|
||||
<br>
|
||||
Diese Seite wurde ausgedruckt von:<br>
|
||||
https://www.ohne-makler.net/immobilien/wohnung-kaufen/nordrhein-westfalen/dusseldorf/
|
||||
|
||||
@@ -29,7 +29,7 @@ vi.mock('../lib/services/extractor/puppeteerExtractor.js', async (importOriginal
|
||||
const { readFixture } = await import('./offlineFixtures.js');
|
||||
return {
|
||||
default: (url) => readFixture(url),
|
||||
launchBrowser: async () => ({ close: async () => {}, __fredy_removeUserDataDir: false }),
|
||||
launchBrowser: async () => ({ close: async () => {}, isConnected: () => true }),
|
||||
closeBrowser: async () => {},
|
||||
};
|
||||
});
|
||||
|
||||
@@ -95,7 +95,10 @@ async function downloadHtmlProvider(name, providerConfig, launchBrowser, closeBr
|
||||
|
||||
const browser = await launchBrowser(providerConfig.url, {});
|
||||
try {
|
||||
const html = await puppeteerExtractor(providerConfig.url, providerConfig.waitForSelector, { browser });
|
||||
const html = await puppeteerExtractor(providerConfig.url, providerConfig.waitForSelector, {
|
||||
browser,
|
||||
name: 'dowload_fixtures',
|
||||
});
|
||||
|
||||
if (!html) {
|
||||
console.warn(` Failed to download ${name}`);
|
||||
|
||||
@@ -12,7 +12,7 @@ import './Index.less';
|
||||
|
||||
// Semi UI uses react-dom (not react-dom/client) internally for imperative renders
|
||||
// like Toast, Notification, etc. In React 19, createRoot was removed from react-dom
|
||||
// and lives only in react-dom/client — inject it so Toast can create its own root.
|
||||
// and lives only in react-dom/client - inject it so Toast can create its own root.
|
||||
semiGlobal.config.createRoot = createRoot;
|
||||
|
||||
const container = document.getElementById('fredy');
|
||||
|
||||
@@ -71,7 +71,7 @@ body {
|
||||
vertical-align: middle;
|
||||
}
|
||||
|
||||
// Suppress focus outlines — Semi uses --semi-color-primary (our red) for all rings
|
||||
// Suppress focus outlines - Semi uses --semi-color-primary (our red) for all rings
|
||||
button:focus,
|
||||
button:focus-visible,
|
||||
.semi-button:focus,
|
||||
|
||||
BIN
ui/src/assets/news/1.png
Normal file
BIN
ui/src/assets/news/1.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 835 KiB |
BIN
ui/src/assets/news/2.png
Normal file
BIN
ui/src/assets/news/2.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 367 KiB |
@@ -1,10 +1,16 @@
|
||||
{
|
||||
"key": "00e6b81777a275f5a140fc9101cb943810db6a69f6eb3927319c5aee0c876542",
|
||||
"key": "00e6b81777a275f5a140fc9101cb943810db6a69f6eb3927319c5aee0c876221",
|
||||
"content":
|
||||
[
|
||||
{
|
||||
"title": "Open in...Fredy ;)",
|
||||
"text": "With the latest version of Fredy, every notification now comes with a link that opens the listing directly inside Fredy. This is also a key step toward an upcoming...milestone :).<br/>To make this work, Fredy needs to know where it lives on the network. We try to guess the public base URL, but let’s be honest, you probably know better. Take a quick look at the baseUrl in the system settings and fix it if it looks off."
|
||||
"title": "Table overview for listings",
|
||||
"text": "Thanks to https://github.com/datenwurm, we now have a table overview for listings. If you decide to use the table view, the decision will be stored.",
|
||||
"media": "1.png"
|
||||
},
|
||||
{
|
||||
"title": "Table overview for jobs",
|
||||
"text": "Based on datenwurm's, work, I created a table overview for jobs. If you decide to use the table view, the decision will be stored.",
|
||||
"media": "2.png"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 242 KiB |
BIN
ui/src/assets/no_image.png
Normal file
BIN
ui/src/assets/no_image.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 1.6 MiB |
@@ -21,6 +21,7 @@ import {
|
||||
Empty,
|
||||
Radio,
|
||||
RadioGroup,
|
||||
Tooltip,
|
||||
} from '@douyinfe/semi-ui-19';
|
||||
import {
|
||||
IconAlertTriangle,
|
||||
@@ -35,6 +36,8 @@ import {
|
||||
IconArrowUp,
|
||||
IconArrowDown,
|
||||
IconHome,
|
||||
IconGridView,
|
||||
IconList,
|
||||
} from '@douyinfe/semi-icons';
|
||||
import { useNavigate } from 'react-router-dom';
|
||||
import ListingDeletionModal from '../../ListingDeletionModal.jsx';
|
||||
@@ -42,6 +45,7 @@ import { useActions, useSelector } from '../../../services/state/store.js';
|
||||
import { xhrDelete, xhrPut, xhrPost } from '../../../services/xhr.js';
|
||||
import { debounce } from '../../../utils';
|
||||
import { IllustrationNoResult, IllustrationNoResultDark } from '@douyinfe/semi-illustrations';
|
||||
import JobsTable from '../../table/JobsTable.jsx';
|
||||
|
||||
import './JobGrid.less';
|
||||
|
||||
@@ -54,6 +58,9 @@ const JobGrid = () => {
|
||||
const actions = useActions();
|
||||
const navigate = useNavigate();
|
||||
|
||||
const userSettings = useSelector((state) => state.userSettings.settings);
|
||||
const viewMode = userSettings?.jobs_view_mode ?? 'grid';
|
||||
|
||||
const [page, setPage] = useState(1);
|
||||
const pageSize = 12;
|
||||
|
||||
@@ -167,7 +174,7 @@ const JobGrid = () => {
|
||||
Toast.success('Job status successfully changed');
|
||||
loadData();
|
||||
} catch (error) {
|
||||
Toast.error(error);
|
||||
Toast.error(error.error);
|
||||
}
|
||||
};
|
||||
|
||||
@@ -234,6 +241,27 @@ const JobGrid = () => {
|
||||
onClick={() => setSortDir((d) => (d === 'asc' ? 'desc' : 'asc'))}
|
||||
title={sortDir === 'asc' ? 'Ascending' : 'Descending'}
|
||||
/>
|
||||
|
||||
<div className="jobGrid__topbar__view-toggle">
|
||||
<Tooltip content="Grid view">
|
||||
<Button
|
||||
icon={<IconGridView />}
|
||||
theme={viewMode === 'grid' ? 'solid' : 'borderless'}
|
||||
onClick={() => actions.userSettings.setJobsViewMode('grid')}
|
||||
aria-label="Grid view"
|
||||
aria-pressed={viewMode === 'grid'}
|
||||
/>
|
||||
</Tooltip>
|
||||
<Tooltip content="Table view">
|
||||
<Button
|
||||
icon={<IconList />}
|
||||
theme={viewMode === 'table' ? 'solid' : 'borderless'}
|
||||
onClick={() => actions.userSettings.setJobsViewMode('table')}
|
||||
aria-label="Table view"
|
||||
aria-pressed={viewMode === 'table'}
|
||||
/>
|
||||
</Tooltip>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{(jobsData?.result || []).length === 0 && (
|
||||
@@ -244,136 +272,144 @@ const JobGrid = () => {
|
||||
/>
|
||||
)}
|
||||
|
||||
<Row gutter={[16, 16]}>
|
||||
{(jobsData?.result || []).map((job) => (
|
||||
<Col key={job.id} xs={24} sm={12} md={12} lg={8} xl={8} xxl={6}>
|
||||
<Card className="jobGrid__card" bodyStyle={{ padding: '16px' }}>
|
||||
<div className="jobGrid__card__header">
|
||||
<div className="jobGrid__card__name">
|
||||
<span className={`jobGrid__card__dot${job.enabled ? ' jobGrid__card__dot--active' : ''}`} />
|
||||
<Title heading={5} ellipsis={{ showTooltip: true }} className="jobGrid__title">
|
||||
{job.name}
|
||||
</Title>
|
||||
{viewMode === 'grid' ? (
|
||||
<Row gutter={[16, 16]}>
|
||||
{(jobsData?.result || []).map((job) => (
|
||||
<Col key={job.id} xs={24} sm={12} md={12} lg={8} xl={8} xxl={6}>
|
||||
<Card className="jobGrid__card" bodyStyle={{ padding: '16px' }}>
|
||||
<div className="jobGrid__card__header">
|
||||
<div className="jobGrid__card__name">
|
||||
<span className={`jobGrid__card__dot${job.enabled ? ' jobGrid__card__dot--active' : ''}`} />
|
||||
<Title heading={5} ellipsis={{ showTooltip: true }} className="jobGrid__title">
|
||||
{job.name}
|
||||
</Title>
|
||||
</div>
|
||||
<div style={{ display: 'flex', alignItems: 'center', gap: 6, flexShrink: 0 }}>
|
||||
{job.isOnlyShared && (
|
||||
<Popover content={getPopoverContent('This job has been shared with you — read only.')}>
|
||||
<div>
|
||||
<IconAlertTriangle style={{ color: 'rgba(var(--semi-yellow-7), 1)' }} />
|
||||
</div>
|
||||
</Popover>
|
||||
)}
|
||||
{job.running && (
|
||||
<Tag color="green" variant="light" size="small">
|
||||
RUNNING
|
||||
</Tag>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
<div style={{ display: 'flex', alignItems: 'center', gap: 6, flexShrink: 0 }}>
|
||||
{job.isOnlyShared && (
|
||||
<Popover
|
||||
content={getPopoverContent(
|
||||
'This job has been shared with you by another user, therefor it is read-only.',
|
||||
)}
|
||||
>
|
||||
|
||||
<div className="jobGrid__card__stats">
|
||||
<div className="jobGrid__card__stat jobGrid__card__stat--blue">
|
||||
<span className="jobGrid__card__stat__number">{job.numberOfFoundListings || 0}</span>
|
||||
<span className="jobGrid__card__stat__label">
|
||||
<IconHome size="small" /> Listings
|
||||
</span>
|
||||
</div>
|
||||
<div className="jobGrid__card__stat jobGrid__card__stat--orange">
|
||||
<span className="jobGrid__card__stat__number">{job.provider?.length || 0}</span>
|
||||
<span className="jobGrid__card__stat__label">
|
||||
<IconBriefcase size="small" /> Providers
|
||||
</span>
|
||||
</div>
|
||||
<div className="jobGrid__card__stat jobGrid__card__stat--purple">
|
||||
<span className="jobGrid__card__stat__number">{job.notificationAdapter?.length || 0}</span>
|
||||
<span className="jobGrid__card__stat__label">
|
||||
<IconBell size="small" /> Adapters
|
||||
</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<Divider margin="12px" />
|
||||
|
||||
<div className="jobGrid__card__footer">
|
||||
<div style={{ display: 'flex', alignItems: 'center', gap: 6 }}>
|
||||
<Switch
|
||||
onChange={(checked) => onJobStatusChanged(job.id, checked)}
|
||||
checked={job.enabled}
|
||||
disabled={job.isOnlyShared}
|
||||
size="small"
|
||||
/>
|
||||
<Text type="secondary" size="small">
|
||||
Active
|
||||
</Text>
|
||||
</div>
|
||||
<div className="jobGrid__actions">
|
||||
<Popover content={getPopoverContent('Run Job')}>
|
||||
<div>
|
||||
<IconAlertTriangle style={{ color: 'rgba(var(--semi-yellow-7), 1)' }} />
|
||||
<Button
|
||||
type="primary"
|
||||
style={{ background: '#21aa21b5' }}
|
||||
size="small"
|
||||
theme="solid"
|
||||
icon={<IconPlayCircle />}
|
||||
disabled={job.isOnlyShared || job.running}
|
||||
onClick={() => onJobRun(job.id)}
|
||||
/>
|
||||
</div>
|
||||
</Popover>
|
||||
)}
|
||||
{job.running && (
|
||||
<Tag color="green" variant="light" size="small">
|
||||
RUNNING
|
||||
</Tag>
|
||||
)}
|
||||
<Popover content={getPopoverContent('Edit a Job')}>
|
||||
<div>
|
||||
<Button
|
||||
type="secondary"
|
||||
size="small"
|
||||
icon={<IconEdit />}
|
||||
disabled={job.isOnlyShared}
|
||||
onClick={() => navigate(`/jobs/edit/${job.id}`)}
|
||||
/>
|
||||
</div>
|
||||
</Popover>
|
||||
<Popover content={getPopoverContent('Clone Job')}>
|
||||
<div>
|
||||
<Button
|
||||
type="tertiary"
|
||||
size="small"
|
||||
icon={<IconCopy />}
|
||||
disabled={job.isOnlyShared}
|
||||
onClick={() => navigate('/jobs/new', { state: { cloneFrom: job.id } })}
|
||||
/>
|
||||
</div>
|
||||
</Popover>
|
||||
<Popover content={getPopoverContent('Delete all found Listings of this Job')}>
|
||||
<div>
|
||||
<Button
|
||||
type="danger"
|
||||
size="small"
|
||||
icon={<IconDescend2 />}
|
||||
disabled={job.isOnlyShared}
|
||||
onClick={() => onListingRemoval(job.id)}
|
||||
/>
|
||||
</div>
|
||||
</Popover>
|
||||
<Popover content={getPopoverContent('Delete Job')}>
|
||||
<div>
|
||||
<Button
|
||||
type="danger"
|
||||
size="small"
|
||||
icon={<IconDelete />}
|
||||
disabled={job.isOnlyShared}
|
||||
onClick={() => onJobRemoval(job.id)}
|
||||
/>
|
||||
</div>
|
||||
</Popover>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="jobGrid__card__stats">
|
||||
<div className="jobGrid__card__stat jobGrid__card__stat--blue">
|
||||
<span className="jobGrid__card__stat__number">{job.numberOfFoundListings || 0}</span>
|
||||
<span className="jobGrid__card__stat__label">
|
||||
<IconHome size="small" /> Listings
|
||||
</span>
|
||||
</div>
|
||||
<div className="jobGrid__card__stat jobGrid__card__stat--orange">
|
||||
<span className="jobGrid__card__stat__number">{job.provider.length || 0}</span>
|
||||
<span className="jobGrid__card__stat__label">
|
||||
<IconBriefcase size="small" /> Providers
|
||||
</span>
|
||||
</div>
|
||||
<div className="jobGrid__card__stat jobGrid__card__stat--purple">
|
||||
<span className="jobGrid__card__stat__number">{job.notificationAdapter.length || 0}</span>
|
||||
<span className="jobGrid__card__stat__label">
|
||||
<IconBell size="small" /> Adapters
|
||||
</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<Divider margin="12px" />
|
||||
|
||||
<div className="jobGrid__card__footer">
|
||||
<div style={{ display: 'flex', alignItems: 'center', gap: 6 }}>
|
||||
<Switch
|
||||
onChange={(checked) => onJobStatusChanged(job.id, checked)}
|
||||
checked={job.enabled}
|
||||
disabled={job.isOnlyShared}
|
||||
size="small"
|
||||
/>
|
||||
<Text type="secondary" size="small">
|
||||
Active
|
||||
</Text>
|
||||
</div>
|
||||
<div className="jobGrid__actions">
|
||||
<Popover content={getPopoverContent('Run Job')}>
|
||||
<div>
|
||||
<Button
|
||||
type="primary"
|
||||
style={{ background: '#21aa21b5' }}
|
||||
size="small"
|
||||
theme="solid"
|
||||
icon={<IconPlayCircle />}
|
||||
disabled={job.isOnlyShared || job.running}
|
||||
onClick={() => onJobRun(job.id)}
|
||||
/>
|
||||
</div>
|
||||
</Popover>
|
||||
<Popover content={getPopoverContent('Edit a Job')}>
|
||||
<div>
|
||||
<Button
|
||||
type="secondary"
|
||||
size="small"
|
||||
icon={<IconEdit />}
|
||||
disabled={job.isOnlyShared}
|
||||
onClick={() => navigate(`/jobs/edit/${job.id}`)}
|
||||
/>
|
||||
</div>
|
||||
</Popover>
|
||||
<Popover content={getPopoverContent('Clone Job')}>
|
||||
<div>
|
||||
<Button
|
||||
type="tertiary"
|
||||
size="small"
|
||||
icon={<IconCopy />}
|
||||
disabled={job.isOnlyShared}
|
||||
onClick={() => navigate('/jobs/new', { state: { cloneFrom: job.id } })}
|
||||
/>
|
||||
</div>
|
||||
</Popover>
|
||||
<Popover content={getPopoverContent('Delete all found Listings of this Job')}>
|
||||
<div>
|
||||
<Button
|
||||
type="danger"
|
||||
size="small"
|
||||
icon={<IconDescend2 />}
|
||||
disabled={job.isOnlyShared}
|
||||
onClick={() => onListingRemoval(job.id)}
|
||||
/>
|
||||
</div>
|
||||
</Popover>
|
||||
<Popover content={getPopoverContent('Delete Job')}>
|
||||
<div>
|
||||
<Button
|
||||
type="danger"
|
||||
size="small"
|
||||
icon={<IconDelete />}
|
||||
disabled={job.isOnlyShared}
|
||||
onClick={() => onJobRemoval(job.id)}
|
||||
/>
|
||||
</div>
|
||||
</Popover>
|
||||
</div>
|
||||
</div>
|
||||
</Card>
|
||||
</Col>
|
||||
))}
|
||||
</Row>
|
||||
</Card>
|
||||
</Col>
|
||||
))}
|
||||
</Row>
|
||||
) : (
|
||||
<JobsTable
|
||||
jobs={jobsData?.result || []}
|
||||
onRun={onJobRun}
|
||||
onEdit={(id) => navigate(`/jobs/edit/${id}`)}
|
||||
onClone={(id) => navigate('/jobs/new', { state: { cloneFrom: id } })}
|
||||
onDeleteListings={onListingRemoval}
|
||||
onDeleteJob={onJobRemoval}
|
||||
onStatusChange={onJobStatusChanged}
|
||||
/>
|
||||
)}
|
||||
{(jobsData?.result || []).length > 0 && jobsData?.totalNumber > 12 && (
|
||||
<div className="jobGrid__pagination">
|
||||
<Pagination
|
||||
|
||||
@@ -17,6 +17,12 @@
|
||||
flex: 1;
|
||||
min-width: 160px;
|
||||
}
|
||||
|
||||
&__view-toggle {
|
||||
display: flex;
|
||||
gap: 4px;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
}
|
||||
|
||||
&__card {
|
||||
|
||||
@@ -3,14 +3,7 @@
|
||||
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
|
||||
*/
|
||||
|
||||
import { useState, useEffect, useMemo } from 'react';
|
||||
import {
|
||||
useSearchParamState,
|
||||
parseNumber,
|
||||
parseString,
|
||||
parseNullableBoolean,
|
||||
} from '../../../hooks/useSearchParamState.js';
|
||||
import { Button, Pagination, Toast, Input, Select, Empty, Radio, RadioGroup, Tooltip } from '@douyinfe/semi-ui-19';
|
||||
import { Button, Tooltip } from '@douyinfe/semi-ui-19';
|
||||
import {
|
||||
IconBriefcase,
|
||||
IconCart,
|
||||
@@ -19,323 +12,117 @@ import {
|
||||
IconMapPin,
|
||||
IconStar,
|
||||
IconStarStroked,
|
||||
IconSearch,
|
||||
IconEyeOpened,
|
||||
IconArrowUp,
|
||||
IconArrowDown,
|
||||
} from '@douyinfe/semi-icons';
|
||||
import { useNavigate, useSearchParams } from 'react-router-dom';
|
||||
import ListingDeletionModal from '../../ListingDeletionModal.jsx';
|
||||
import no_image from '../../../assets/no_image.jpg';
|
||||
import no_image from '../../../assets/no_image.png';
|
||||
import * as timeService from '../../../services/time/timeService.js';
|
||||
import { xhrDelete, xhrPost } from '../../../services/xhr.js';
|
||||
import { useActions, useSelector } from '../../../services/state/store.js';
|
||||
import { debounce } from '../../../utils';
|
||||
|
||||
import './ListingsGrid.less';
|
||||
import { IllustrationNoResult, IllustrationNoResultDark } from '@douyinfe/semi-illustrations';
|
||||
|
||||
const ListingsGrid = () => {
|
||||
const listingsData = useSelector((state) => state.listingsData);
|
||||
const providers = useSelector((state) => state.provider);
|
||||
const jobs = useSelector((state) => state.jobsData.jobs);
|
||||
const actions = useActions();
|
||||
const navigate = useNavigate();
|
||||
const sp = useSearchParams();
|
||||
|
||||
const [page, setPage] = useSearchParamState(sp, 'page', 1, parseNumber);
|
||||
const pageSize = 40;
|
||||
|
||||
const [sortField, setSortField] = useSearchParamState(sp, 'sort', 'created_at', parseString);
|
||||
const [sortDir, setSortDir] = useSearchParamState(sp, 'dir', 'desc', parseString);
|
||||
const [freeTextFilter, setFreeTextFilter] = useSearchParamState(sp, 'q', null, parseString);
|
||||
const [watchListFilter, setWatchListFilter] = useSearchParamState(sp, 'watch', null, parseNullableBoolean);
|
||||
const [jobNameFilter, setJobNameFilter] = useSearchParamState(sp, 'job', null, parseString);
|
||||
const [activityFilter, setActivityFilter] = useSearchParamState(sp, 'active', null, parseNullableBoolean);
|
||||
const [providerFilter, setProviderFilter] = useSearchParamState(sp, 'provider', null, parseString);
|
||||
const [deleteModalVisible, setDeleteModalVisible] = useState(false);
|
||||
const [listingToDelete, setListingToDelete] = useState(null);
|
||||
|
||||
const loadData = () => {
|
||||
actions.listingsData.getListingsData({
|
||||
page,
|
||||
pageSize,
|
||||
sortfield: sortField,
|
||||
sortdir: sortDir,
|
||||
freeTextFilter,
|
||||
filter: { watchListFilter, jobNameFilter, activityFilter, providerFilter },
|
||||
});
|
||||
};
|
||||
|
||||
useEffect(() => {
|
||||
loadData();
|
||||
}, [page, sortField, sortDir, freeTextFilter, providerFilter, activityFilter, jobNameFilter, watchListFilter]);
|
||||
|
||||
const handleFilterChange = useMemo(
|
||||
() =>
|
||||
debounce((value) => {
|
||||
setFreeTextFilter(value || null);
|
||||
setPage(1);
|
||||
}, 500),
|
||||
[],
|
||||
);
|
||||
|
||||
useEffect(() => {
|
||||
return () => {
|
||||
// cleanup debounced handler to avoid memory leaks
|
||||
handleFilterChange.cancel && handleFilterChange.cancel();
|
||||
};
|
||||
}, [handleFilterChange]);
|
||||
|
||||
const handleWatch = async (e, item) => {
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
try {
|
||||
await xhrPost('/api/listings/watch', { listingId: item.id });
|
||||
Toast.success(item.isWatched === 1 ? 'Listing removed from Watchlist' : 'Listing added to Watchlist');
|
||||
loadData();
|
||||
} catch (e) {
|
||||
console.error(e);
|
||||
Toast.error('Failed to operate Watchlist');
|
||||
}
|
||||
};
|
||||
|
||||
const handlePageChange = (_page) => {
|
||||
setPage(_page);
|
||||
};
|
||||
|
||||
const confirmDeletion = async (hardDelete) => {
|
||||
try {
|
||||
await xhrDelete('/api/listings/', { ids: [listingToDelete], hardDelete });
|
||||
Toast.success('Listing successfully removed');
|
||||
loadData();
|
||||
} catch (error) {
|
||||
Toast.error(error.message || 'Error deleting listing');
|
||||
} finally {
|
||||
setDeleteModalVisible(false);
|
||||
setListingToDelete(null);
|
||||
}
|
||||
};
|
||||
|
||||
return (
|
||||
<div className="listingsGrid">
|
||||
<div className="listingsGrid__topbar">
|
||||
<Input
|
||||
className="listingsGrid__topbar__search"
|
||||
prefix={<IconSearch />}
|
||||
showClear
|
||||
placeholder="Search"
|
||||
defaultValue={freeTextFilter ?? ''}
|
||||
onChange={handleFilterChange}
|
||||
/>
|
||||
|
||||
<RadioGroup
|
||||
type="button"
|
||||
buttonSize="middle"
|
||||
value={activityFilter === null ? 'all' : String(activityFilter)}
|
||||
onChange={(e) => {
|
||||
const v = e.target.value;
|
||||
setActivityFilter(v === 'all' ? null : v === 'true');
|
||||
setPage(1);
|
||||
}}
|
||||
>
|
||||
<Radio value="all">All</Radio>
|
||||
<Radio value="true">Active</Radio>
|
||||
<Radio value="false">Inactive</Radio>
|
||||
</RadioGroup>
|
||||
|
||||
<RadioGroup
|
||||
type="button"
|
||||
buttonSize="middle"
|
||||
value={watchListFilter === null ? 'all' : String(watchListFilter)}
|
||||
onChange={(e) => {
|
||||
const v = e.target.value;
|
||||
setWatchListFilter(v === 'all' ? null : v === 'true');
|
||||
setPage(1);
|
||||
}}
|
||||
>
|
||||
<Radio value="all">All</Radio>
|
||||
<Radio value="true">Watched</Radio>
|
||||
<Radio value="false">Unwatched</Radio>
|
||||
</RadioGroup>
|
||||
|
||||
<Select
|
||||
placeholder="Provider"
|
||||
showClear
|
||||
onChange={(val) => {
|
||||
setProviderFilter(val);
|
||||
setPage(1);
|
||||
}}
|
||||
value={providerFilter}
|
||||
style={{ width: 130 }}
|
||||
>
|
||||
{providers?.map((p) => (
|
||||
<Select.Option key={p.id} value={p.id}>
|
||||
{p.name}
|
||||
</Select.Option>
|
||||
))}
|
||||
</Select>
|
||||
|
||||
<Select
|
||||
placeholder="Job"
|
||||
showClear
|
||||
onChange={(val) => {
|
||||
setJobNameFilter(val);
|
||||
setPage(1);
|
||||
}}
|
||||
value={jobNameFilter}
|
||||
style={{ width: 130 }}
|
||||
>
|
||||
{jobs?.map((j) => (
|
||||
<Select.Option key={j.id} value={j.id}>
|
||||
{j.name}
|
||||
</Select.Option>
|
||||
))}
|
||||
</Select>
|
||||
|
||||
<Select prefix="Sort by" style={{ width: 185 }} value={sortField} onChange={(val) => setSortField(val)}>
|
||||
<Select.Option value="job_name">Job Name</Select.Option>
|
||||
<Select.Option value="created_at">Listing Date</Select.Option>
|
||||
<Select.Option value="price">Price</Select.Option>
|
||||
<Select.Option value="provider">Provider</Select.Option>
|
||||
</Select>
|
||||
|
||||
<Button
|
||||
icon={sortDir === 'asc' ? <IconArrowUp /> : <IconArrowDown />}
|
||||
onClick={() => setSortDir((d) => (d === 'asc' ? 'desc' : 'asc'))}
|
||||
title={sortDir === 'asc' ? 'Ascending' : 'Descending'}
|
||||
/>
|
||||
</div>
|
||||
|
||||
{(listingsData?.result || []).length === 0 && (
|
||||
<Empty
|
||||
image={<IllustrationNoResult />}
|
||||
darkModeImage={<IllustrationNoResultDark />}
|
||||
description="No listings available yet..."
|
||||
/>
|
||||
)}
|
||||
<div className="listingsGrid__grid">
|
||||
{(listingsData?.result || []).map((item) => (
|
||||
<div
|
||||
key={item.id}
|
||||
className="listingsGrid__card"
|
||||
style={{ cursor: 'pointer' }}
|
||||
role="button"
|
||||
tabIndex={0}
|
||||
onClick={() => navigate(`/listings/listing/${item.id}`)}
|
||||
onKeyDown={(e) => {
|
||||
if (e.key === 'Enter' || e.key === ' ') navigate(`/listings/listing/${item.id}`);
|
||||
}}
|
||||
>
|
||||
<div className="listingsGrid__card__image-wrapper">
|
||||
<img
|
||||
src={item.image_url || no_image}
|
||||
alt={item.title}
|
||||
onError={(e) => {
|
||||
e.target.src = no_image;
|
||||
}}
|
||||
/>
|
||||
{!item.is_active && (
|
||||
<div className="listingsGrid__card__inactive-watermark">
|
||||
<span>Inactive</span>
|
||||
</div>
|
||||
)}
|
||||
<button
|
||||
type="button"
|
||||
className="listingsGrid__card__star"
|
||||
onClick={(e) => handleWatch(e, item)}
|
||||
aria-label={item.isWatched === 1 ? 'Remove from watchlist' : 'Add to watchlist'}
|
||||
>
|
||||
{item.isWatched === 1 ? <IconStar /> : <IconStarStroked />}
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<div className="listingsGrid__card__body">
|
||||
<div className="listingsGrid__card__title" title={item.title}>
|
||||
{item.title}
|
||||
</div>
|
||||
{item.price && (
|
||||
<div className="listingsGrid__card__price">
|
||||
<IconCart size="small" />
|
||||
{item.price}
|
||||
</div>
|
||||
)}
|
||||
{item.address && (
|
||||
<div className="listingsGrid__card__meta">
|
||||
<IconMapPin />
|
||||
{item.address}
|
||||
</div>
|
||||
)}
|
||||
<div className="listingsGrid__card__meta">
|
||||
<IconBriefcase />
|
||||
{item.provider}
|
||||
</div>
|
||||
<div className="listingsGrid__card__provider">{timeService.format(item.created_at, false)}</div>
|
||||
</div>
|
||||
|
||||
<div className="listingsGrid__card__actions" onClick={(e) => e.stopPropagation()}>
|
||||
<Tooltip content="Original Listing">
|
||||
<Button
|
||||
size="small"
|
||||
icon={<IconLink />}
|
||||
style={{ color: '#60a5fa' }}
|
||||
theme="borderless"
|
||||
onClick={(e) => {
|
||||
e.stopPropagation();
|
||||
window.open(item.link, '_blank');
|
||||
}}
|
||||
/>
|
||||
</Tooltip>
|
||||
<Tooltip content="View in Fredy">
|
||||
<Button
|
||||
size="small"
|
||||
icon={<IconEyeOpened />}
|
||||
style={{ color: '#34d399' }}
|
||||
theme="borderless"
|
||||
onClick={(e) => {
|
||||
e.stopPropagation();
|
||||
navigate(`/listings/listing/${item.id}`);
|
||||
}}
|
||||
/>
|
||||
</Tooltip>
|
||||
<Tooltip content="Remove">
|
||||
<Button
|
||||
size="small"
|
||||
icon={<IconDelete />}
|
||||
style={{ color: '#fb7185' }}
|
||||
theme="borderless"
|
||||
onClick={(e) => {
|
||||
e.stopPropagation();
|
||||
setListingToDelete(item.id);
|
||||
setDeleteModalVisible(true);
|
||||
}}
|
||||
/>
|
||||
</Tooltip>
|
||||
</div>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
{(listingsData?.result || []).length > 0 && (
|
||||
<div className="listingsGrid__pagination">
|
||||
<Pagination
|
||||
currentPage={page}
|
||||
pageSize={pageSize}
|
||||
total={listingsData?.totalNumber || 0}
|
||||
onPageChange={handlePageChange}
|
||||
showSizeChanger={false}
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
<ListingDeletionModal
|
||||
visible={deleteModalVisible}
|
||||
onConfirm={confirmDeletion}
|
||||
onCancel={() => {
|
||||
setDeleteModalVisible(false);
|
||||
setListingToDelete(null);
|
||||
/**
|
||||
* @param {{ listings: object[], onWatch: Function, onNavigate: Function, onDelete: Function }} props
|
||||
*/
|
||||
const ListingsGrid = ({ listings, onWatch, onNavigate, onDelete }) => (
|
||||
<div className="listingsGrid__grid">
|
||||
{listings.map((item) => (
|
||||
<div
|
||||
key={item.id}
|
||||
className="listingsGrid__card"
|
||||
style={{ cursor: 'pointer' }}
|
||||
role="button"
|
||||
tabIndex={0}
|
||||
onClick={() => onNavigate(item.id)}
|
||||
onKeyDown={(e) => {
|
||||
if (e.key === 'Enter' || e.key === ' ') onNavigate(item.id);
|
||||
}}
|
||||
/>
|
||||
</div>
|
||||
);
|
||||
};
|
||||
>
|
||||
<div className="listingsGrid__card__image-wrapper">
|
||||
<img
|
||||
src={item.image_url || no_image}
|
||||
alt={item.title}
|
||||
onError={(e) => {
|
||||
e.target.src = no_image;
|
||||
}}
|
||||
/>
|
||||
{!item.is_active && (
|
||||
<div className="listingsGrid__card__inactive-watermark">
|
||||
<span>Inactive</span>
|
||||
</div>
|
||||
)}
|
||||
<button
|
||||
type="button"
|
||||
className="listingsGrid__card__star"
|
||||
onClick={(e) => onWatch(e, item)}
|
||||
aria-label={item.isWatched === 1 ? 'Remove from watchlist' : 'Add to watchlist'}
|
||||
>
|
||||
{item.isWatched === 1 ? <IconStar /> : <IconStarStroked />}
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<div className="listingsGrid__card__body">
|
||||
<div className="listingsGrid__card__title" title={item.title}>
|
||||
{item.title}
|
||||
</div>
|
||||
{item.price && (
|
||||
<div className="listingsGrid__card__price">
|
||||
<IconCart size="small" />
|
||||
{item.price}
|
||||
</div>
|
||||
)}
|
||||
{item.address && (
|
||||
<div className="listingsGrid__card__meta">
|
||||
<IconMapPin />
|
||||
{item.address}
|
||||
</div>
|
||||
)}
|
||||
<div className="listingsGrid__card__meta">
|
||||
<IconBriefcase />
|
||||
{item.provider}
|
||||
</div>
|
||||
<div className="listingsGrid__card__provider">{timeService.format(item.created_at, false)}</div>
|
||||
</div>
|
||||
|
||||
<div className="listingsGrid__card__actions" onClick={(e) => e.stopPropagation()}>
|
||||
<Tooltip content="Original Listing">
|
||||
<Button
|
||||
size="small"
|
||||
icon={<IconLink />}
|
||||
style={{ color: '#60a5fa' }}
|
||||
theme="borderless"
|
||||
onClick={(e) => {
|
||||
e.stopPropagation();
|
||||
window.open(item.link, '_blank');
|
||||
}}
|
||||
/>
|
||||
</Tooltip>
|
||||
<Tooltip content="View in Fredy">
|
||||
<Button
|
||||
size="small"
|
||||
icon={<IconEyeOpened />}
|
||||
style={{ color: '#34d399' }}
|
||||
theme="borderless"
|
||||
onClick={(e) => {
|
||||
e.stopPropagation();
|
||||
onNavigate(item.id);
|
||||
}}
|
||||
/>
|
||||
</Tooltip>
|
||||
<Tooltip content="Remove">
|
||||
<Button
|
||||
size="small"
|
||||
icon={<IconDelete />}
|
||||
style={{ color: '#fb7185' }}
|
||||
theme="borderless"
|
||||
onClick={(e) => {
|
||||
e.stopPropagation();
|
||||
onDelete(item.id);
|
||||
}}
|
||||
/>
|
||||
</Tooltip>
|
||||
</div>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
);
|
||||
|
||||
export default ListingsGrid;
|
||||
|
||||
@@ -1,181 +1,143 @@
|
||||
@import '../../../tokens.less';
|
||||
|
||||
.listingsGrid {
|
||||
&__topbar {
|
||||
.listingsGrid__grid {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fill, minmax(240px, 1fr));
|
||||
gap: 12px;
|
||||
}
|
||||
|
||||
.listingsGrid__card {
|
||||
background: @color-elevated !important;
|
||||
border: 1px solid @color-border !important;
|
||||
border-radius: @radius-card !important;
|
||||
overflow: hidden;
|
||||
transition: transform @transition-card, box-shadow @transition-card;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
|
||||
&:hover {
|
||||
transform: translateY(-2px);
|
||||
box-shadow: 0 6px 24px -4px rgba(0,0,0,0.6);
|
||||
}
|
||||
|
||||
&__image-wrapper {
|
||||
position: relative;
|
||||
height: 160px;
|
||||
overflow: hidden;
|
||||
flex-shrink: 0;
|
||||
|
||||
img {
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
object-fit: cover;
|
||||
display: block;
|
||||
}
|
||||
}
|
||||
|
||||
&__inactive-watermark {
|
||||
position: absolute;
|
||||
inset: 0;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: @space-3;
|
||||
margin-bottom: @space-4;
|
||||
flex-wrap: wrap;
|
||||
justify-content: center;
|
||||
background: rgba(0,0,0,0.35);
|
||||
|
||||
&__search {
|
||||
min-width: 200px;
|
||||
flex: 1;
|
||||
}
|
||||
|
||||
@media (max-width: 768px) {
|
||||
.listingsGrid__topbar__search {
|
||||
width: 100%;
|
||||
flex: unset;
|
||||
}
|
||||
|
||||
.semi-radio-group {
|
||||
flex: 1;
|
||||
}
|
||||
|
||||
.semi-select {
|
||||
flex: 1;
|
||||
min-width: 100px;
|
||||
width: auto !important;
|
||||
}
|
||||
span {
|
||||
font-size: 18px;
|
||||
font-weight: 800;
|
||||
color: rgba(251,113,133,0.9);
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.15em;
|
||||
transform: rotate(-30deg);
|
||||
border: 2px solid rgba(251,113,133,0.5);
|
||||
padding: 4px 12px;
|
||||
border-radius: @radius-chip;
|
||||
backdrop-filter: blur(2px);
|
||||
}
|
||||
}
|
||||
|
||||
&__grid {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fill, minmax(240px, 1fr));
|
||||
gap: 12px;
|
||||
}
|
||||
|
||||
&__card {
|
||||
background: @color-elevated !important;
|
||||
border: 1px solid @color-border !important;
|
||||
border-radius: @radius-card !important;
|
||||
overflow: hidden;
|
||||
transition: transform @transition-card, box-shadow @transition-card;
|
||||
&__star {
|
||||
position: absolute;
|
||||
top: 8px;
|
||||
right: 8px;
|
||||
background: rgba(0,0,0,0.5);
|
||||
border: none;
|
||||
border-radius: 50%;
|
||||
width: 28px;
|
||||
height: 28px;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
cursor: pointer;
|
||||
transition: background @transition-fast;
|
||||
padding: 0;
|
||||
|
||||
&:hover {
|
||||
transform: translateY(-2px);
|
||||
box-shadow: 0 6px 24px -4px rgba(0,0,0,0.6);
|
||||
background: rgba(0,0,0,0.75);
|
||||
}
|
||||
|
||||
&__image-wrapper {
|
||||
position: relative;
|
||||
height: 160px;
|
||||
overflow: hidden;
|
||||
flex-shrink: 0;
|
||||
|
||||
img {
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
object-fit: cover;
|
||||
display: block;
|
||||
}
|
||||
svg {
|
||||
color: @color-accent;
|
||||
font-size: 14px;
|
||||
}
|
||||
}
|
||||
|
||||
&__inactive-watermark {
|
||||
position: absolute;
|
||||
inset: 0;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
background: rgba(0,0,0,0.35);
|
||||
&__body {
|
||||
padding: 12px;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 4px;
|
||||
flex: 1;
|
||||
}
|
||||
|
||||
span {
|
||||
font-size: 18px;
|
||||
font-weight: 800;
|
||||
color: rgba(251,113,133,0.9);
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.15em;
|
||||
transform: rotate(-30deg);
|
||||
border: 2px solid rgba(251,113,133,0.5);
|
||||
padding: 4px 12px;
|
||||
border-radius: @radius-chip;
|
||||
backdrop-filter: blur(2px);
|
||||
}
|
||||
}
|
||||
&__title {
|
||||
font-weight: 700;
|
||||
font-size: @text-sm;
|
||||
color: @color-text;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
&__star {
|
||||
position: absolute;
|
||||
top: 8px;
|
||||
right: 8px;
|
||||
background: rgba(0,0,0,0.5);
|
||||
border: none;
|
||||
border-radius: 50%;
|
||||
width: 28px;
|
||||
height: 28px;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
cursor: pointer;
|
||||
transition: background @transition-fast;
|
||||
padding: 0;
|
||||
&__price {
|
||||
font-size: @text-base;
|
||||
font-weight: 600;
|
||||
color: @color-success;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 4px;
|
||||
}
|
||||
|
||||
&:hover {
|
||||
background: rgba(0,0,0,0.75);
|
||||
}
|
||||
&__meta {
|
||||
font-size: @text-xs;
|
||||
color: @color-muted;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 4px;
|
||||
|
||||
svg {
|
||||
color: @color-accent;
|
||||
font-size: 14px;
|
||||
}
|
||||
}
|
||||
|
||||
&__body {
|
||||
padding: 12px;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 4px;
|
||||
flex: 1;
|
||||
}
|
||||
|
||||
&__title {
|
||||
font-weight: 700;
|
||||
font-size: @text-sm;
|
||||
color: @color-text;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
&__price {
|
||||
font-size: @text-base;
|
||||
font-weight: 600;
|
||||
color: @color-success;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 4px;
|
||||
}
|
||||
|
||||
&__meta {
|
||||
font-size: @text-xs;
|
||||
color: @color-muted;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 4px;
|
||||
|
||||
.semi-icon {
|
||||
font-size: 11px;
|
||||
color: @color-faint;
|
||||
}
|
||||
}
|
||||
|
||||
&__provider {
|
||||
font-size: @text-xs;
|
||||
.semi-icon {
|
||||
font-size: 11px;
|
||||
color: @color-faint;
|
||||
}
|
||||
}
|
||||
|
||||
&__actions {
|
||||
display: flex;
|
||||
justify-content: space-around;
|
||||
padding: 8px 12px;
|
||||
border-top: 1px solid @color-border;
|
||||
gap: 4px;
|
||||
margin-top: auto;
|
||||
&__provider {
|
||||
font-size: @text-xs;
|
||||
color: @color-faint;
|
||||
}
|
||||
|
||||
button {
|
||||
flex: 1;
|
||||
border: none !important;
|
||||
border-radius: @radius-chip !important;
|
||||
}
|
||||
&__actions {
|
||||
display: flex;
|
||||
justify-content: space-around;
|
||||
padding: 8px 12px;
|
||||
border-top: 1px solid @color-border;
|
||||
gap: 4px;
|
||||
margin-top: auto;
|
||||
|
||||
button {
|
||||
flex: 1;
|
||||
border: none !important;
|
||||
border-radius: @radius-chip !important;
|
||||
}
|
||||
}
|
||||
|
||||
&__pagination {
|
||||
margin-top: @space-4;
|
||||
display: flex;
|
||||
justify-content: center;
|
||||
}
|
||||
}
|
||||
|
||||
264
ui/src/components/listings/ListingsOverview.jsx
Normal file
264
ui/src/components/listings/ListingsOverview.jsx
Normal file
@@ -0,0 +1,264 @@
|
||||
/*
|
||||
* Copyright (c) 2026 by Christian Kellner.
|
||||
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
|
||||
*/
|
||||
|
||||
import { useState, useEffect, useMemo } from 'react';
|
||||
import {
|
||||
useSearchParamState,
|
||||
parseNumber,
|
||||
parseString,
|
||||
parseNullableBoolean,
|
||||
} from '../../hooks/useSearchParamState.js';
|
||||
import { Button, Pagination, Toast, Input, Select, Empty, Radio, RadioGroup, Tooltip } from '@douyinfe/semi-ui-19';
|
||||
import { IconSearch, IconArrowUp, IconArrowDown, IconGridView, IconList } from '@douyinfe/semi-icons';
|
||||
import { useNavigate, useSearchParams } from 'react-router-dom';
|
||||
import ListingDeletionModal from '../ListingDeletionModal.jsx';
|
||||
import { xhrDelete, xhrPost } from '../../services/xhr.js';
|
||||
import { useActions, useSelector } from '../../services/state/store.js';
|
||||
import { debounce } from '../../utils';
|
||||
import ListingsGrid from '../grid/listings/ListingsGrid.jsx';
|
||||
import ListingsTable from '../table/ListingsTable.jsx';
|
||||
import { IllustrationNoResult, IllustrationNoResultDark } from '@douyinfe/semi-illustrations';
|
||||
|
||||
import './ListingsOverview.less';
|
||||
|
||||
const ListingsOverview = () => {
|
||||
const listingsData = useSelector((state) => state.listingsData);
|
||||
const providers = useSelector((state) => state.provider);
|
||||
const jobs = useSelector((state) => state.jobsData.jobs);
|
||||
const userSettings = useSelector((state) => state.userSettings.settings);
|
||||
const actions = useActions();
|
||||
const navigate = useNavigate();
|
||||
const sp = useSearchParams();
|
||||
|
||||
const viewMode = userSettings?.listings_view_mode ?? 'grid';
|
||||
|
||||
const [page, setPage] = useSearchParamState(sp, 'page', 1, parseNumber);
|
||||
const pageSize = 40;
|
||||
|
||||
const [sortField, setSortField] = useSearchParamState(sp, 'sort', 'created_at', parseString);
|
||||
const [sortDir, setSortDir] = useSearchParamState(sp, 'dir', 'desc', parseString);
|
||||
const [freeTextFilter, setFreeTextFilter] = useSearchParamState(sp, 'q', null, parseString);
|
||||
const [watchListFilter, setWatchListFilter] = useSearchParamState(sp, 'watch', null, parseNullableBoolean);
|
||||
const [jobNameFilter, setJobNameFilter] = useSearchParamState(sp, 'job', null, parseString);
|
||||
const [activityFilter, setActivityFilter] = useSearchParamState(sp, 'active', null, parseNullableBoolean);
|
||||
const [providerFilter, setProviderFilter] = useSearchParamState(sp, 'provider', null, parseString);
|
||||
const [deleteModalVisible, setDeleteModalVisible] = useState(false);
|
||||
const [listingToDelete, setListingToDelete] = useState(null);
|
||||
|
||||
const loadData = () => {
|
||||
actions.listingsData.getListingsData({
|
||||
page,
|
||||
pageSize,
|
||||
sortfield: sortField,
|
||||
sortdir: sortDir,
|
||||
freeTextFilter,
|
||||
filter: { watchListFilter, jobNameFilter, activityFilter, providerFilter },
|
||||
});
|
||||
};
|
||||
|
||||
useEffect(() => {
|
||||
loadData();
|
||||
}, [page, sortField, sortDir, freeTextFilter, providerFilter, activityFilter, jobNameFilter, watchListFilter]);
|
||||
|
||||
const handleFilterChange = useMemo(
|
||||
() =>
|
||||
debounce((value) => {
|
||||
setFreeTextFilter(value || null);
|
||||
setPage(1);
|
||||
}, 500),
|
||||
[],
|
||||
);
|
||||
|
||||
useEffect(() => {
|
||||
return () => {
|
||||
handleFilterChange.cancel && handleFilterChange.cancel();
|
||||
};
|
||||
}, [handleFilterChange]);
|
||||
|
||||
const handleWatch = async (e, item) => {
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
try {
|
||||
await xhrPost('/api/listings/watch', { listingId: item.id });
|
||||
Toast.success(item.isWatched === 1 ? 'Listing removed from Watchlist' : 'Listing added to Watchlist');
|
||||
loadData();
|
||||
} catch (e) {
|
||||
console.error(e);
|
||||
Toast.error('Failed to operate Watchlist');
|
||||
}
|
||||
};
|
||||
|
||||
const handleDelete = (id) => {
|
||||
setListingToDelete(id);
|
||||
setDeleteModalVisible(true);
|
||||
};
|
||||
|
||||
const handleNavigate = (id) => navigate(`/listings/listing/${id}`);
|
||||
|
||||
const confirmDeletion = async (hardDelete) => {
|
||||
try {
|
||||
await xhrDelete('/api/listings/', { ids: [listingToDelete], hardDelete });
|
||||
Toast.success('Listing successfully removed');
|
||||
loadData();
|
||||
} catch (error) {
|
||||
Toast.error(error.message || 'Error deleting listing');
|
||||
} finally {
|
||||
setDeleteModalVisible(false);
|
||||
setListingToDelete(null);
|
||||
}
|
||||
};
|
||||
|
||||
const listings = listingsData?.result || [];
|
||||
|
||||
return (
|
||||
<div className="listingsOverview">
|
||||
<div className="listingsOverview__topbar">
|
||||
<Input
|
||||
className="listingsOverview__topbar__search"
|
||||
prefix={<IconSearch />}
|
||||
showClear
|
||||
placeholder="Search"
|
||||
defaultValue={freeTextFilter ?? ''}
|
||||
onChange={handleFilterChange}
|
||||
/>
|
||||
|
||||
<RadioGroup
|
||||
type="button"
|
||||
buttonSize="middle"
|
||||
value={activityFilter === null ? 'all' : String(activityFilter)}
|
||||
onChange={(e) => {
|
||||
const v = e.target.value;
|
||||
setActivityFilter(v === 'all' ? null : v === 'true');
|
||||
setPage(1);
|
||||
}}
|
||||
>
|
||||
<Radio value="all">All</Radio>
|
||||
<Radio value="true">Active</Radio>
|
||||
<Radio value="false">Inactive</Radio>
|
||||
</RadioGroup>
|
||||
|
||||
<RadioGroup
|
||||
type="button"
|
||||
buttonSize="middle"
|
||||
value={watchListFilter === null ? 'all' : String(watchListFilter)}
|
||||
onChange={(e) => {
|
||||
const v = e.target.value;
|
||||
setWatchListFilter(v === 'all' ? null : v === 'true');
|
||||
setPage(1);
|
||||
}}
|
||||
>
|
||||
<Radio value="all">All</Radio>
|
||||
<Radio value="true">Watched</Radio>
|
||||
<Radio value="false">Unwatched</Radio>
|
||||
</RadioGroup>
|
||||
|
||||
<Select
|
||||
placeholder="Provider"
|
||||
showClear
|
||||
onChange={(val) => {
|
||||
setProviderFilter(val);
|
||||
setPage(1);
|
||||
}}
|
||||
value={providerFilter}
|
||||
style={{ width: 130 }}
|
||||
>
|
||||
{providers?.map((p) => (
|
||||
<Select.Option key={p.id} value={p.id}>
|
||||
{p.name}
|
||||
</Select.Option>
|
||||
))}
|
||||
</Select>
|
||||
|
||||
<Select
|
||||
placeholder="Job"
|
||||
showClear
|
||||
onChange={(val) => {
|
||||
setJobNameFilter(val);
|
||||
setPage(1);
|
||||
}}
|
||||
value={jobNameFilter}
|
||||
style={{ width: 130 }}
|
||||
>
|
||||
{jobs?.map((j) => (
|
||||
<Select.Option key={j.id} value={j.id}>
|
||||
{j.name}
|
||||
</Select.Option>
|
||||
))}
|
||||
</Select>
|
||||
|
||||
<Select prefix="Sort by" style={{ width: 185 }} value={sortField} onChange={(val) => setSortField(val)}>
|
||||
<Select.Option value="job_name">Job Name</Select.Option>
|
||||
<Select.Option value="created_at">Listing Date</Select.Option>
|
||||
<Select.Option value="price">Price</Select.Option>
|
||||
<Select.Option value="provider">Provider</Select.Option>
|
||||
</Select>
|
||||
|
||||
<Button
|
||||
icon={sortDir === 'asc' ? <IconArrowUp /> : <IconArrowDown />}
|
||||
onClick={() => setSortDir(sortDir === 'asc' ? 'desc' : 'asc')}
|
||||
title={sortDir === 'asc' ? 'Ascending' : 'Descending'}
|
||||
/>
|
||||
|
||||
<div className="listingsOverview__topbar__view-toggle">
|
||||
<Tooltip content="Grid view">
|
||||
<Button
|
||||
icon={<IconGridView />}
|
||||
theme={viewMode === 'grid' ? 'solid' : 'borderless'}
|
||||
onClick={() => actions.userSettings.setListingsViewMode('grid')}
|
||||
aria-label="Grid view"
|
||||
aria-pressed={viewMode === 'grid'}
|
||||
/>
|
||||
</Tooltip>
|
||||
<Tooltip content="Table view">
|
||||
<Button
|
||||
icon={<IconList />}
|
||||
theme={viewMode === 'table' ? 'solid' : 'borderless'}
|
||||
onClick={() => actions.userSettings.setListingsViewMode('table')}
|
||||
aria-label="Table view"
|
||||
aria-pressed={viewMode === 'table'}
|
||||
/>
|
||||
</Tooltip>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{listings.length === 0 && (
|
||||
<Empty
|
||||
image={<IllustrationNoResult />}
|
||||
darkModeImage={<IllustrationNoResultDark />}
|
||||
description="No listings available yet..."
|
||||
/>
|
||||
)}
|
||||
|
||||
{viewMode === 'grid' ? (
|
||||
<ListingsGrid listings={listings} onWatch={handleWatch} onNavigate={handleNavigate} onDelete={handleDelete} />
|
||||
) : (
|
||||
<ListingsTable listings={listings} onWatch={handleWatch} onNavigate={handleNavigate} onDelete={handleDelete} />
|
||||
)}
|
||||
|
||||
{listings.length > 0 && (
|
||||
<div className="listingsOverview__pagination">
|
||||
<Pagination
|
||||
currentPage={page}
|
||||
pageSize={pageSize}
|
||||
total={listingsData?.totalNumber || 0}
|
||||
onPageChange={setPage}
|
||||
showSizeChanger={false}
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
|
||||
<ListingDeletionModal
|
||||
visible={deleteModalVisible}
|
||||
onConfirm={confirmDeletion}
|
||||
onCancel={() => {
|
||||
setDeleteModalVisible(false);
|
||||
setListingToDelete(null);
|
||||
}}
|
||||
/>
|
||||
</div>
|
||||
);
|
||||
};
|
||||
|
||||
export default ListingsOverview;
|
||||
45
ui/src/components/listings/ListingsOverview.less
Normal file
45
ui/src/components/listings/ListingsOverview.less
Normal file
@@ -0,0 +1,45 @@
|
||||
@import '../../tokens.less';
|
||||
|
||||
.listingsOverview {
|
||||
&__topbar {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: @space-3;
|
||||
margin-bottom: @space-4;
|
||||
flex-wrap: wrap;
|
||||
|
||||
&__search {
|
||||
min-width: 200px;
|
||||
flex: 1;
|
||||
}
|
||||
|
||||
&__view-toggle {
|
||||
display: flex;
|
||||
gap: 2px;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
@media (max-width: 768px) {
|
||||
.listingsOverview__topbar__search {
|
||||
width: 100%;
|
||||
flex: unset;
|
||||
}
|
||||
|
||||
.semi-radio-group {
|
||||
flex: 1;
|
||||
}
|
||||
|
||||
.semi-select {
|
||||
flex: 1;
|
||||
min-width: 100px;
|
||||
width: auto !important;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
&__pagination {
|
||||
margin-top: @space-4;
|
||||
display: flex;
|
||||
justify-content: center;
|
||||
}
|
||||
}
|
||||
@@ -136,9 +136,9 @@
|
||||
color: @color-muted !important;
|
||||
}
|
||||
|
||||
// Collapsed state — icons perfectly centered
|
||||
// Collapsed state - icons perfectly centered
|
||||
.semi-navigation-collapsed {
|
||||
// Text span is display:block and takes up flex space — must be removed so justify-content:center works
|
||||
// Text span is display:block and takes up flex space - must be removed so justify-content:center works
|
||||
.semi-navigation-item-text {
|
||||
display: none !important;
|
||||
}
|
||||
@@ -165,7 +165,7 @@
|
||||
min-width: 0 !important;
|
||||
}
|
||||
|
||||
// Semi adds margin-right to icons for text spacing — remove it when collapsed
|
||||
// Semi adds margin-right to icons for text spacing - remove it when collapsed
|
||||
.semi-navigation-item-icon,
|
||||
.semi-navigation-sub-title-icon {
|
||||
display: flex !important;
|
||||
@@ -179,13 +179,13 @@
|
||||
}
|
||||
}
|
||||
|
||||
// Semi Nav.Footer — full width, no extra padding (our BEM class controls it)
|
||||
// Semi Nav.Footer - full width, no extra padding (our BEM class controls it)
|
||||
.semi-navigation-footer {
|
||||
width: 100% !important;
|
||||
box-sizing: border-box !important;
|
||||
}
|
||||
|
||||
// Collapsed submenu popup — actual class used by Semi UI is .semi-navigation-popover
|
||||
// Collapsed submenu popup - actual class used by Semi UI is .semi-navigation-popover
|
||||
.semi-navigation-popover {
|
||||
background: @color-elevated !important;
|
||||
border: 1px solid @color-border !important;
|
||||
|
||||
@@ -32,7 +32,7 @@
|
||||
padding: 16px 20px !important;
|
||||
}
|
||||
|
||||
// Semi input focus — subtle, not accent
|
||||
// Semi input focus - subtle, not accent
|
||||
.semi-input-wrapper:focus-within,
|
||||
.semi-select:focus-within {
|
||||
border-color: @color-border-bright !important;
|
||||
|
||||
128
ui/src/components/table/JobsTable.jsx
Normal file
128
ui/src/components/table/JobsTable.jsx
Normal file
@@ -0,0 +1,128 @@
|
||||
/*
|
||||
* Copyright (c) 2026 by Christian Kellner.
|
||||
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
|
||||
*/
|
||||
|
||||
import { Button, Tag, Tooltip, Switch } from '@douyinfe/semi-ui-19';
|
||||
import {
|
||||
IconAlertTriangle,
|
||||
IconBell,
|
||||
IconBriefcase,
|
||||
IconCopy,
|
||||
IconDelete,
|
||||
IconDescend2,
|
||||
IconEdit,
|
||||
IconHome,
|
||||
IconPlayCircle,
|
||||
} from '@douyinfe/semi-icons';
|
||||
|
||||
import './JobsTable.less';
|
||||
|
||||
/**
|
||||
* @param {{ jobs: object[], onRun: Function, onEdit: Function, onClone: Function, onDeleteListings: Function, onDeleteJob: Function, onStatusChange: Function }} props
|
||||
*/
|
||||
const JobsTable = ({ jobs, onRun, onEdit, onClone, onDeleteListings, onDeleteJob, onStatusChange }) => (
|
||||
<div className="jobsTable">
|
||||
{jobs.map((job) => (
|
||||
<div key={job.id} className={`jobsTable__row${!job.enabled ? ' jobsTable__row--inactive' : ''}`}>
|
||||
<div className="jobsTable__row__dot">
|
||||
<span
|
||||
className={`jobsTable__row__dot__indicator${job.enabled ? ' jobsTable__row__dot__indicator--active' : ''}`}
|
||||
/>
|
||||
</div>
|
||||
|
||||
<div className="jobsTable__row__name" title={job.name}>
|
||||
{job.name}
|
||||
</div>
|
||||
|
||||
<div className="jobsTable__row__stat jobsTable__row__stat--blue">
|
||||
<IconHome size="small" />
|
||||
{job.numberOfFoundListings || 0}
|
||||
</div>
|
||||
|
||||
<div className="jobsTable__row__stat jobsTable__row__stat--orange">
|
||||
<IconBriefcase size="small" />
|
||||
{job.provider?.length || 0}
|
||||
</div>
|
||||
|
||||
<div className="jobsTable__row__stat jobsTable__row__stat--purple">
|
||||
<IconBell size="small" />
|
||||
{job.notificationAdapter?.length || 0}
|
||||
</div>
|
||||
|
||||
<div className="jobsTable__row__badges">
|
||||
<Switch
|
||||
size="small"
|
||||
checked={job.enabled}
|
||||
disabled={job.isOnlyShared}
|
||||
onChange={(checked) => onStatusChange(job.id, checked)}
|
||||
/>
|
||||
{job.running && (
|
||||
<Tag color="green" variant="light" size="small">
|
||||
RUNNING
|
||||
</Tag>
|
||||
)}
|
||||
{job.isOnlyShared && (
|
||||
<Tooltip content="Shared with you — read only">
|
||||
<span style={{ display: 'flex', alignItems: 'center' }}>
|
||||
<IconAlertTriangle style={{ color: 'rgba(var(--semi-yellow-7), 1)' }} />
|
||||
</span>
|
||||
</Tooltip>
|
||||
)}
|
||||
</div>
|
||||
|
||||
<div className="jobsTable__row__actions">
|
||||
<Tooltip content="Run Job">
|
||||
<Button
|
||||
type="primary"
|
||||
style={{ background: '#21aa21b5' }}
|
||||
size="small"
|
||||
theme="solid"
|
||||
icon={<IconPlayCircle />}
|
||||
disabled={job.isOnlyShared || job.running}
|
||||
onClick={() => onRun(job.id)}
|
||||
/>
|
||||
</Tooltip>
|
||||
<Tooltip content="Edit Job">
|
||||
<Button
|
||||
type="secondary"
|
||||
size="small"
|
||||
icon={<IconEdit />}
|
||||
disabled={job.isOnlyShared}
|
||||
onClick={() => onEdit(job.id)}
|
||||
/>
|
||||
</Tooltip>
|
||||
<Tooltip content="Clone Job">
|
||||
<Button
|
||||
type="tertiary"
|
||||
size="small"
|
||||
icon={<IconCopy />}
|
||||
disabled={job.isOnlyShared}
|
||||
onClick={() => onClone(job.id)}
|
||||
/>
|
||||
</Tooltip>
|
||||
<Tooltip content="Delete all found Listings">
|
||||
<Button
|
||||
type="danger"
|
||||
size="small"
|
||||
icon={<IconDescend2 />}
|
||||
disabled={job.isOnlyShared}
|
||||
onClick={() => onDeleteListings(job.id)}
|
||||
/>
|
||||
</Tooltip>
|
||||
<Tooltip content="Delete Job">
|
||||
<Button
|
||||
type="danger"
|
||||
size="small"
|
||||
icon={<IconDelete />}
|
||||
disabled={job.isOnlyShared}
|
||||
onClick={() => onDeleteJob(job.id)}
|
||||
/>
|
||||
</Tooltip>
|
||||
</div>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
);
|
||||
|
||||
export default JobsTable;
|
||||
105
ui/src/components/table/JobsTable.less
Normal file
105
ui/src/components/table/JobsTable.less
Normal file
@@ -0,0 +1,105 @@
|
||||
@import '../../tokens.less';
|
||||
|
||||
.jobsTable {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 4px;
|
||||
|
||||
&__row {
|
||||
display: grid;
|
||||
grid-template-columns: 24px 1fr 80px 80px 80px auto auto;
|
||||
align-items: center;
|
||||
gap: @space-3;
|
||||
padding: 8px 12px;
|
||||
background: @color-elevated;
|
||||
border: 1px solid @color-border;
|
||||
border-radius: @radius-chip;
|
||||
transition: background @transition-fast;
|
||||
|
||||
&:hover {
|
||||
background: #252525;
|
||||
}
|
||||
|
||||
&--inactive {
|
||||
opacity: 0.6;
|
||||
}
|
||||
|
||||
&__dot {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
|
||||
&__indicator {
|
||||
width: 8px;
|
||||
height: 8px;
|
||||
border-radius: 50%;
|
||||
flex-shrink: 0;
|
||||
background-color: rgba(251, 113, 133, 0.7);
|
||||
|
||||
&--active {
|
||||
background-color: rgba(52, 211, 153, 0.8);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
&__name {
|
||||
font-weight: 600;
|
||||
font-size: @text-sm;
|
||||
color: @color-text;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
&__stat {
|
||||
font-size: @text-sm;
|
||||
font-weight: 600;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 4px;
|
||||
white-space: nowrap;
|
||||
|
||||
&--blue {
|
||||
color: @color-blue-text;
|
||||
}
|
||||
|
||||
&--orange {
|
||||
color: @color-orange-text;
|
||||
}
|
||||
|
||||
&--purple {
|
||||
color: @color-purple-text;
|
||||
}
|
||||
}
|
||||
|
||||
&__badges {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 4px;
|
||||
min-width: 0;
|
||||
}
|
||||
|
||||
&__actions {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 2px;
|
||||
}
|
||||
|
||||
@media (max-width: 900px) {
|
||||
grid-template-columns: 24px 1fr 80px auto auto;
|
||||
|
||||
.jobsTable__row__stat--orange,
|
||||
.jobsTable__row__stat--purple {
|
||||
display: none;
|
||||
}
|
||||
}
|
||||
|
||||
@media (max-width: 560px) {
|
||||
grid-template-columns: 24px 1fr auto auto;
|
||||
|
||||
.jobsTable__row__stat--blue {
|
||||
display: none;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
132
ui/src/components/table/ListingsTable.jsx
Normal file
132
ui/src/components/table/ListingsTable.jsx
Normal file
@@ -0,0 +1,132 @@
|
||||
/*
|
||||
* Copyright (c) 2026 by Christian Kellner.
|
||||
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
|
||||
*/
|
||||
|
||||
import { Button, Tooltip } from '@douyinfe/semi-ui-19';
|
||||
import {
|
||||
IconBriefcase,
|
||||
IconCart,
|
||||
IconDelete,
|
||||
IconLink,
|
||||
IconMapPin,
|
||||
IconStar,
|
||||
IconStarStroked,
|
||||
IconEyeOpened,
|
||||
} from '@douyinfe/semi-icons';
|
||||
import no_image from '../../assets/no_image.png';
|
||||
import * as timeService from '../../services/time/timeService.js';
|
||||
|
||||
import './ListingsTable.less';
|
||||
|
||||
/**
|
||||
* @param {{ listings: object[], onWatch: Function, onNavigate: Function, onDelete: Function }} props
|
||||
*/
|
||||
const ListingsTable = ({ listings, onWatch, onNavigate, onDelete }) => (
|
||||
<div className="listingsTable">
|
||||
{listings.map((item) => (
|
||||
<div
|
||||
key={item.id}
|
||||
className={`listingsTable__row${!item.is_active ? ' listingsTable__row--inactive' : ''}`}
|
||||
role="button"
|
||||
tabIndex={0}
|
||||
onClick={() => onNavigate(item.id)}
|
||||
onKeyDown={(e) => {
|
||||
if (e.key === 'Enter' || e.key === ' ') onNavigate(item.id);
|
||||
}}
|
||||
>
|
||||
<div className="listingsTable__row__thumb">
|
||||
<img
|
||||
src={item.image_url || no_image}
|
||||
alt={item.title}
|
||||
onError={(e) => {
|
||||
e.target.src = no_image;
|
||||
}}
|
||||
/>
|
||||
</div>
|
||||
|
||||
<div className="listingsTable__row__title" title={item.title}>
|
||||
{item.title}
|
||||
</div>
|
||||
|
||||
<div className="listingsTable__row__price">
|
||||
{item.price ? (
|
||||
<>
|
||||
<IconCart size="small" />
|
||||
{item.price}
|
||||
</>
|
||||
) : (
|
||||
<span className="listingsTable__row__empty">—</span>
|
||||
)}
|
||||
</div>
|
||||
|
||||
<div className="listingsTable__row__address">
|
||||
{item.address ? (
|
||||
<>
|
||||
<IconMapPin size="small" />
|
||||
{item.address}
|
||||
</>
|
||||
) : (
|
||||
<span className="listingsTable__row__empty">—</span>
|
||||
)}
|
||||
</div>
|
||||
|
||||
<div className="listingsTable__row__meta">
|
||||
<IconBriefcase size="small" />
|
||||
{item.provider}
|
||||
</div>
|
||||
|
||||
<div className="listingsTable__row__date">{timeService.format(item.created_at, false)}</div>
|
||||
|
||||
<div className="listingsTable__row__actions" onClick={(e) => e.stopPropagation()}>
|
||||
<button
|
||||
type="button"
|
||||
className="listingsTable__row__star"
|
||||
onClick={(e) => onWatch(e, item)}
|
||||
aria-label={item.isWatched === 1 ? 'Remove from watchlist' : 'Add to watchlist'}
|
||||
>
|
||||
{item.isWatched === 1 ? <IconStar /> : <IconStarStroked />}
|
||||
</button>
|
||||
<Tooltip content="Original Listing">
|
||||
<Button
|
||||
size="small"
|
||||
icon={<IconLink />}
|
||||
style={{ color: '#60a5fa' }}
|
||||
theme="borderless"
|
||||
onClick={(e) => {
|
||||
e.stopPropagation();
|
||||
window.open(item.link, '_blank');
|
||||
}}
|
||||
/>
|
||||
</Tooltip>
|
||||
<Tooltip content="View in Fredy">
|
||||
<Button
|
||||
size="small"
|
||||
icon={<IconEyeOpened />}
|
||||
style={{ color: '#34d399' }}
|
||||
theme="borderless"
|
||||
onClick={(e) => {
|
||||
e.stopPropagation();
|
||||
onNavigate(item.id);
|
||||
}}
|
||||
/>
|
||||
</Tooltip>
|
||||
<Tooltip content="Remove">
|
||||
<Button
|
||||
size="small"
|
||||
icon={<IconDelete />}
|
||||
style={{ color: '#fb7185' }}
|
||||
theme="borderless"
|
||||
onClick={(e) => {
|
||||
e.stopPropagation();
|
||||
onDelete(item.id);
|
||||
}}
|
||||
/>
|
||||
</Tooltip>
|
||||
</div>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
);
|
||||
|
||||
export default ListingsTable;
|
||||
142
ui/src/components/table/ListingsTable.less
Normal file
142
ui/src/components/table/ListingsTable.less
Normal file
@@ -0,0 +1,142 @@
|
||||
@import '../../tokens.less';
|
||||
|
||||
.listingsTable {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 4px;
|
||||
|
||||
&__row {
|
||||
display: grid;
|
||||
grid-template-columns: 56px 1fr 140px 200px 120px 110px auto;
|
||||
align-items: center;
|
||||
gap: @space-3;
|
||||
padding: 8px 12px;
|
||||
background: @color-elevated;
|
||||
border: 1px solid @color-border;
|
||||
border-radius: @radius-chip;
|
||||
cursor: pointer;
|
||||
transition: background @transition-fast;
|
||||
|
||||
&:hover {
|
||||
background: #252525;
|
||||
}
|
||||
|
||||
&--inactive {
|
||||
opacity: 0.6;
|
||||
}
|
||||
|
||||
&__thumb {
|
||||
width: 56px;
|
||||
height: 40px;
|
||||
flex-shrink: 0;
|
||||
border-radius: @radius-chip;
|
||||
overflow: hidden;
|
||||
|
||||
img {
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
object-fit: cover;
|
||||
display: block;
|
||||
}
|
||||
}
|
||||
|
||||
&__title {
|
||||
font-weight: 600;
|
||||
font-size: @text-sm;
|
||||
color: @color-text;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
&__price {
|
||||
font-size: @text-sm;
|
||||
font-weight: 600;
|
||||
color: @color-success;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 4px;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
&__address {
|
||||
font-size: @text-xs;
|
||||
color: @color-muted;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 4px;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
&__meta {
|
||||
font-size: @text-xs;
|
||||
color: @color-muted;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 4px;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
&__date {
|
||||
font-size: @text-xs;
|
||||
color: @color-faint;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
&__actions {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 2px;
|
||||
}
|
||||
|
||||
&__star {
|
||||
width: 28px;
|
||||
height: 28px;
|
||||
background: transparent;
|
||||
border: none;
|
||||
border-radius: 50%;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
cursor: pointer;
|
||||
padding: 0;
|
||||
transition: background @transition-fast;
|
||||
flex-shrink: 0;
|
||||
|
||||
&:hover {
|
||||
background: rgba(0, 0, 0, 0.1);
|
||||
}
|
||||
|
||||
svg {
|
||||
color: @color-accent;
|
||||
font-size: 14px;
|
||||
}
|
||||
}
|
||||
|
||||
&__empty {
|
||||
color: @color-faint;
|
||||
}
|
||||
|
||||
@media (max-width: 900px) {
|
||||
grid-template-columns: 56px 1fr 120px auto;
|
||||
|
||||
.listingsTable__row__address,
|
||||
.listingsTable__row__meta,
|
||||
.listingsTable__row__date {
|
||||
display: none;
|
||||
}
|
||||
}
|
||||
|
||||
@media (max-width: 560px) {
|
||||
grid-template-columns: 56px 1fr auto;
|
||||
|
||||
.listingsTable__row__price {
|
||||
display: none;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -47,7 +47,7 @@ export default function UserTable({ user = [], onUserRemoval, onUserEdit } = {})
|
||||
{
|
||||
title: 'Last login',
|
||||
dataIndex: 'lastLogin',
|
||||
render: (value) => format(value),
|
||||
render: (value) => (value == null ? '---' : format(value)),
|
||||
},
|
||||
{
|
||||
title: 'Jobs',
|
||||
|
||||
@@ -321,6 +321,34 @@ export const useFredyState = create(
|
||||
throw Exception;
|
||||
}
|
||||
},
|
||||
async setListingsViewMode(listings_view_mode) {
|
||||
try {
|
||||
await xhrPost('/api/user/settings/listings-view-mode', { listings_view_mode });
|
||||
set((state) => ({
|
||||
userSettings: {
|
||||
...state.userSettings,
|
||||
settings: { ...state.userSettings.settings, listings_view_mode },
|
||||
},
|
||||
}));
|
||||
} catch (Exception) {
|
||||
console.error('Error while trying to update listings view mode setting. Error:', Exception);
|
||||
throw Exception;
|
||||
}
|
||||
},
|
||||
async setJobsViewMode(jobs_view_mode) {
|
||||
try {
|
||||
await xhrPost('/api/user/settings/jobs-view-mode', { jobs_view_mode });
|
||||
set((state) => ({
|
||||
userSettings: {
|
||||
...state.userSettings,
|
||||
settings: { ...state.userSettings.settings, jobs_view_mode },
|
||||
},
|
||||
}));
|
||||
} catch (Exception) {
|
||||
console.error('Error while trying to update jobs view mode setting. Error:', Exception);
|
||||
throw Exception;
|
||||
}
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
|
||||
@@ -13,5 +13,3 @@ export function format(ts, showSeconds = true) {
|
||||
...(showSeconds ? { second: 'numeric' } : {}),
|
||||
}).format(ts);
|
||||
}
|
||||
|
||||
export const roundToHour = (ts) => Math.ceil(ts / (1000 * 60 * 60)) * (1000 * 60 * 60);
|
||||
|
||||
@@ -54,6 +54,7 @@ const GeneralSettings = function GeneralSettings() {
|
||||
const [loading, setLoading] = React.useState(true);
|
||||
|
||||
const settings = useSelector((state) => state.generalSettings.settings);
|
||||
const currentUser = useSelector((state) => state.user.currentUser);
|
||||
|
||||
const [interval, setInterval] = React.useState('');
|
||||
const [port, setPort] = React.useState('');
|
||||
@@ -299,7 +300,7 @@ const GeneralSettings = function GeneralSettings() {
|
||||
|
||||
<SegmentPart
|
||||
name="Analytics"
|
||||
helpText="Anonymous usage data to help improve Fredy — provider names, adapter names, OS, Node version, and architecture."
|
||||
helpText="Anonymous usage data to help improve Fredy - provider names, adapter names, OS, Node version, and architecture."
|
||||
>
|
||||
<Checkbox checked={analyticsEnabled} onChange={(e) => setAnalyticsEnabled(e.target.checked)}>
|
||||
Enable analytics
|
||||
@@ -467,12 +468,26 @@ const GeneralSettings = function GeneralSettings() {
|
||||
itemKey="backup"
|
||||
>
|
||||
<div className="generalSettings__tab-content">
|
||||
{demoMode && !currentUser?.isAdmin && (
|
||||
<Banner
|
||||
fullMode={false}
|
||||
type="warning"
|
||||
closeIcon={null}
|
||||
style={{ marginBottom: '12px' }}
|
||||
description="Backup and restore are not available in demo mode."
|
||||
/>
|
||||
)}
|
||||
<SegmentPart
|
||||
name="Backup & Restore"
|
||||
helpText="Download a zipped backup of your database or restore from a backup zip."
|
||||
>
|
||||
<div style={{ display: 'flex', gap: '0.5rem', alignItems: 'center', flexWrap: 'wrap' }}>
|
||||
<Button theme="solid" icon={<IconSave />} onClick={handleDownloadBackup}>
|
||||
<Button
|
||||
theme="solid"
|
||||
icon={<IconSave />}
|
||||
onClick={handleDownloadBackup}
|
||||
disabled={demoMode && !currentUser?.isAdmin}
|
||||
>
|
||||
Download Backup
|
||||
</Button>
|
||||
<input
|
||||
@@ -482,7 +497,12 @@ const GeneralSettings = function GeneralSettings() {
|
||||
style={{ display: 'none' }}
|
||||
onChange={handleSelectRestoreFile}
|
||||
/>
|
||||
<Button onClick={handleOpenFilePicker} theme="light" icon={<IconFolder />}>
|
||||
<Button
|
||||
onClick={handleOpenFilePicker}
|
||||
theme="light"
|
||||
icon={<IconFolder />}
|
||||
disabled={demoMode && !currentUser?.isAdmin}
|
||||
>
|
||||
Restore from Zip
|
||||
</Button>
|
||||
</div>
|
||||
|
||||
@@ -43,7 +43,7 @@
|
||||
}
|
||||
}
|
||||
|
||||
// TimePicker fix — scoped so it doesn't pollute modal headers
|
||||
// TimePicker fix - scoped so it doesn't pollute modal headers
|
||||
.semi-timepicker .semi-input-wrapper,
|
||||
.semi-timepicker .semi-input-inset-label-wrapper {
|
||||
background: @color-elevated !important;
|
||||
|
||||
@@ -141,21 +141,6 @@ export default function ProviderMutator({
|
||||
</p>
|
||||
</>
|
||||
)}
|
||||
<Banner
|
||||
fullMode={false}
|
||||
type="warning"
|
||||
closeIcon={null}
|
||||
title={<div style={{ fontWeight: 600, fontSize: '14px', lineHeight: '20px' }}>Warning</div>}
|
||||
style={{ marginBottom: '1rem' }}
|
||||
description={
|
||||
<div>
|
||||
<p>
|
||||
Currently, our Immoscout implementation does not support drawing shapes on a map. Use a radius instead.
|
||||
</p>
|
||||
</div>
|
||||
}
|
||||
/>
|
||||
|
||||
<Select
|
||||
filter
|
||||
placeholder="Select a provider"
|
||||
|
||||
@@ -31,15 +31,17 @@ import {
|
||||
IconLink,
|
||||
IconStar,
|
||||
IconStarStroked,
|
||||
IconDelete,
|
||||
IconExpand,
|
||||
IconGridView,
|
||||
} from '@douyinfe/semi-icons';
|
||||
import maplibregl from 'maplibre-gl';
|
||||
import 'maplibre-gl/dist/maplibre-gl.css';
|
||||
import no_image from '../../assets/no_image.jpg';
|
||||
import no_image from '../../assets/no_image.png';
|
||||
import * as timeService from '../../services/time/timeService.js';
|
||||
import { distanceMeters, getBoundsFromCoords } from './mapUtils.js';
|
||||
import { xhrPost } from '../../services/xhr.js';
|
||||
import { xhrPost, xhrDelete } from '../../services/xhr.js';
|
||||
import ListingDeletionModal from '../../components/ListingDeletionModal.jsx';
|
||||
|
||||
import Headline from '../../components/headline/Headline.jsx';
|
||||
import './ListingDetail.less';
|
||||
@@ -59,6 +61,7 @@ export default function ListingDetail() {
|
||||
const mapContainer = useRef(null);
|
||||
const map = useRef(null);
|
||||
const [loading, setLoading] = useState(true);
|
||||
const [deleteModalVisible, setDeleteModalVisible] = useState(false);
|
||||
|
||||
useEffect(() => {
|
||||
async function fetchListing() {
|
||||
@@ -239,6 +242,18 @@ export default function ListingDetail() {
|
||||
};
|
||||
}, [listing, loading, homeAddress]);
|
||||
|
||||
const confirmDeletion = async (hardDelete) => {
|
||||
try {
|
||||
await xhrDelete('/api/listings/', { ids: [listing.id], hardDelete });
|
||||
Toast.success('Listing successfully removed');
|
||||
navigate('/listings');
|
||||
} catch (e) {
|
||||
Toast.error(e.message || 'Error deleting listing');
|
||||
} finally {
|
||||
setDeleteModalVisible(false);
|
||||
}
|
||||
};
|
||||
|
||||
const handleWatch = async () => {
|
||||
try {
|
||||
await xhrPost('/api/listings/watch', { listingId: listing.id });
|
||||
@@ -330,17 +345,27 @@ export default function ListingDetail() {
|
||||
<IconLink style={{ marginRight: 6 }} />
|
||||
Open listing
|
||||
</a>
|
||||
<Button
|
||||
icon={<IconDelete />}
|
||||
onClick={() => setDeleteModalVisible(true)}
|
||||
theme="light"
|
||||
type="danger"
|
||||
>
|
||||
Delete
|
||||
</Button>
|
||||
</Space>
|
||||
</div>
|
||||
|
||||
<Row>
|
||||
<Col span={24} lg={12}>
|
||||
<div className="listing-detail__image-container">
|
||||
<div
|
||||
className={`listing-detail__image-container${!listing.image_url ? ' listing-detail__image-container--placeholder' : ''}`}
|
||||
>
|
||||
<Image
|
||||
src={listing.image_url}
|
||||
fallback={no_image}
|
||||
src={listing.image_url ?? no_image}
|
||||
fallback={<img src={no_image} alt="No image available" />}
|
||||
style={{ width: '100%', height: '100%' }}
|
||||
preview={true}
|
||||
preview={!!listing.image_url}
|
||||
/>
|
||||
</div>
|
||||
</Col>
|
||||
@@ -395,6 +420,12 @@ export default function ListingDetail() {
|
||||
<div ref={mapContainer} className="listing-detail__map-container" />
|
||||
)}
|
||||
</div>
|
||||
|
||||
<ListingDeletionModal
|
||||
visible={deleteModalVisible}
|
||||
onConfirm={confirmDeletion}
|
||||
onCancel={() => setDeleteModalVisible(false)}
|
||||
/>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
@@ -69,6 +69,13 @@
|
||||
object-fit: cover !important;
|
||||
display: block !important;
|
||||
}
|
||||
|
||||
&--placeholder {
|
||||
img,
|
||||
.semi-image-img {
|
||||
object-fit: contain !important;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
&__address-link {
|
||||
|
||||
@@ -3,14 +3,14 @@
|
||||
* Licensed under Apache-2.0 with Commons Clause and Attribution/Naming Clause
|
||||
*/
|
||||
|
||||
import ListingsGrid from '../../components/grid/listings/ListingsGrid.jsx';
|
||||
import ListingsOverview from '../../components/listings/ListingsOverview.jsx';
|
||||
import Headline from '../../components/headline/Headline.jsx';
|
||||
|
||||
export default function Listings() {
|
||||
return (
|
||||
<>
|
||||
<Headline text="Listings" />
|
||||
<ListingsGrid />
|
||||
<ListingsOverview />
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
||||
@@ -13,7 +13,7 @@ import { distanceMeters, generateCircleCoords, getBoundsFromCenter, getBoundsFro
|
||||
import { Banner, Select, Switch, Toast, Typography } from '@douyinfe/semi-ui-19';
|
||||
import { IconDelete, IconEyeOpened, IconLink } from '@douyinfe/semi-icons';
|
||||
|
||||
import no_image from '../../assets/no_image.jpg';
|
||||
import no_image from '../../assets/no_image.png';
|
||||
import _RangeSlider from 'react-range-slider-input';
|
||||
import 'react-range-slider-input/dist/style.css';
|
||||
import './Map.less';
|
||||
|
||||
@@ -37,7 +37,7 @@ const Users = function Users() {
|
||||
await actions.jobsData.getJobs();
|
||||
await actions.user.getUsers();
|
||||
} catch (error) {
|
||||
Toast.error(error);
|
||||
Toast.error(error.error);
|
||||
setUserIdToBeRemoved(null);
|
||||
}
|
||||
};
|
||||
|
||||
@@ -59,7 +59,7 @@ const UserMutator = function UserMutator() {
|
||||
navigate('/users');
|
||||
} catch (error) {
|
||||
console.error(error);
|
||||
Toast.error(error.json.message);
|
||||
Toast.error(error.json.error);
|
||||
}
|
||||
};
|
||||
|
||||
|
||||
@@ -10,6 +10,7 @@ export default defineConfig({
|
||||
globals: true,
|
||||
environment: 'node',
|
||||
include: ['test/**/*.test.js'],
|
||||
globalSetup: ['./test/globalSetup.js'],
|
||||
testTimeout: 60000,
|
||||
reporters: ['verbose'],
|
||||
},
|
||||
|
||||
Reference in New Issue
Block a user