mirror of
https://github.com/devkitshq/notifkit.git
synced 2026-10-19 18:24:52 +02:00
coo1estguy
ceeecf3937
- Upgrade zod to ^4.6.5 along with compatible dependency updates across ai, dotenv, drizzle-orm, ioredis, pino, postgres, and dev tools. - Adjust tests/workers-ai.test.ts mock UUIDs to adhere to RFC 4122 v4 validation in Zod 4. - Update tests/http.test.ts validation message expectations for Zod 4. - Replace generic server container names with role-based definitions (api-server-1, api-server-2, enricher-worker, engine-worker, delivery-worker). - Add serverNodes option in distributed profiling harness to route and health-check designated worker and API nodes. - Add multi-run statistical metrics (mean, median, min, max, stdDev) reporting to money-scaling benchmark runner.
Notifkit Profiling & Benchmarking Suite
This folder provides a realistic, container-isolated profiling and benchmarking environment for Notifkit.
🏗️ Architecture & Resource Constraints
The environment simulates a standard small production node (e.g. $15/mo cloud VM):
- Server: 1.0 vCPU, 1 GB RAM (
notifkitstandalone orchestrator) - PostgreSQL 15: 0.5 vCPU, 512 MB RAM
- Redis 7: 0.5 vCPU, 256 MB RAM
🚀 Quick Start
1. Install Dependencies
cd profiling
npm install
2. Run the Load Test
npm run test:load
The load test will:
- Boot the isolated 3-container stack (
postgres,redis,server) viatestcontainers. - Run database schema migrations and seed project/template data.
- Fire 50 concurrent virtual users generating notifications against
/v1/notify. - Stream and buffer container logs until 100% of messages are delivered to the transport.
- Output detailed throughput, p95, and p99 percentiles for both API ingestion and background delivery.
- Automatically clean up and shut down the containers.
3. Run the CPU Scaling Benchmark
Evaluate server performance scaling from 0.5 vCPU up to 4.0 vCPUs:
npm run test:scaling
CLI Flags & Customization:
# Rapid test (5s duration per tier)
npm run test:scaling -- --quick
# Specify custom CPU tiers
npm run test:scaling -- --cpus=0.5,1.0,2.0,4.0
# Customize duration and concurrency
npm run test:scaling -- --duration=15 --concurrency=100
The scaling benchmark produces an aggregated comparative table with:
- Ingestion Throughput (
req/sec) - Active Delivery Throughput (
msgs/sec) - Wall Drain Throughput (
msgs/sec) - API & Delivery Latencies (
p50,p95,p99) - Speedup vs Baseline & Diminishing Returns Analysis
➕ Adding New Profiling Tests
To add a new benchmark (e.g. batch delivery, quiet hours queuing, multi-channel fallback):
- Create a new test file under
tests/(e.g.tests/batch.test.ts). - Add a corresponding script to
package.json:"test:batch": "tsx tests/batch.test.ts" - Run
npm run test:batch.