Commit Graph
51 Commits
Author SHA1 Message Date
coo1estguy e4162c47f6 fix: SIGTERM went to the npx wrapper, not the server process, so the shutdown check wouldn't have been reliable even with a working script. 2026-09-29 23:33:21 +05:30
coo1estguy 0ed1924b8d fix: scan for stale pending messages on worker startup and increase check frequency 2026-09-29 23:11:51 +05:30
coo1estguy 3291dc64de feat: add integration test harness, workflow cancellation, and provider send gating 2026-09-29 17:41:06 +05:30
coo1estguy 7b4d7f8efd fix: restore scheduled queues from postgres and recover consumer groups after redis data loss 2026-09-29 14:33:41 +05:30
coo1estguy 35a34df2fe refactor: extract worker bootstrap runtime and remove unused code 2026-09-29 13:10:15 +05:30
coo1estguy 880fe32b03 perf: rebuild profiling harness and reduce per-message pipeline cost
- Rebuild profiling suite with compose CLI, in-network k6 load generator, and production-shaped tiers
- Add fused pipeline mode for in-memory single-user stage hand-offs
- Route directly by contact ID and defer template rendering to delivery
- Add Redis connection sharing, auto-pipelining, stream read coalescing, and chunked provider throttling
- Switch hot message_logs and delivery_outbox inserts to unnest bulk SQL queries
- Drop redundant indexes on message_logs
- Optimize worker batching, flush concurrency, and dataloader windowing
2026-09-29 00:35:01 +05:30
coo1estguy 516ea88bb1 fix(ci): resolve TypeScript typecheck OOM and add workspace typecheck step
- Scope root tsconfig to exclude workspace packages and restore declaration resolution

- Align MCP package zod dependency to ^4.6.5 and update record schemas to explicit key/value syntax

- Upgrade ai dependency to ^7.0.118 to restore bundled dist/index.d.ts declarations

- Fix rate limiter inFlightLease promise initialization in TS strict mode

- Add typecheck:workspaces script and wire into CI workflow
2026-09-28 13:11:07 +05:30
coo1estguy 8af3f36833 refactor(profiling): simplify profiling suite and document scaling bottlenecks 2026-09-27 18:00:37 +05:30
coo1estguy ad89e9c66d fix(api): fix project ID typing in auth cache and undefined url reference 2026-09-27 16:16:08 +05:30
coo1estguy c82d424b60 perf(queue): add event loop write coalescing for stream publish
- Buffer concurrent publish and publishBatch calls into a single pipeline flush per event loop tick
- Apply MAXLEN ~ trimming only to the last command in each batch to reduce scan overhead
- Throttle periodic stream length monitoring to once every 5 seconds
2026-09-27 15:47:28 +05:30
coo1estguy d33edac03d perf(api): avoid sha256 calculation on auth cache hits
- Key in-memory authCache by raw token instead of tokenHash

- Compute SHA-256 only on cache misses when querying database

- Store structured auth record to avoid string splitting and parseInt

- Maintain tokenHash-to-token reverse map for targeted cache invalidations
2026-09-26 23:30:27 +05:30
coo1estguy 39da6c23d7 perf(api/notify): drop per-request log from info to debug
- Demote logger.info to logger.debug in notify handler

- Eliminate worker-thread pipe overhead from pino-pretty on the hot path
2026-09-26 23:22:42 +05:30
coo1estguy 7680570bee perf(api): stream utf-8 strings in readJsonBody to avoid buffer copy
- Accumulate UTF-8 strings directly instead of Buffer.concat and toString copies

- Add fallback handling for mock streams and non-string chunks
2026-09-26 23:21:31 +05:30
coo1estguy 77cd28c4ad perf(api/router): index routes by method and segment count
- Key route lookup table by METHOD:segmentCount to avoid scanning all routes

- Drop comparisons on /v1/notify from 35 routes to 4 candidates
2026-09-26 23:20:49 +05:30
coo1estguy a1670bfeec perf(api): wire in-memory rate limiter and use lazy url parsing on ingress
- Replace per-request synchronous Redis check with ApiRateLimiter in request pipeline

- Replace eager new URL() allocation with indexOf query pathname slice and lazy getters

- Invalidate in-memory rate limiter leases on API key broadcast
2026-09-26 23:20:15 +05:30
coo1estguy ca95c8b917 perf(rate-limiter): add sliding-window token leasing to cut redis roundtrips
- Add LUA_LEASE_API_RATE_LIMIT script and register leaseApiRateLimit custom command

- Implement ApiRateLimiter with in-memory token leasing and 5s lease target

- Coalesce concurrent requests on depleted lease to avoid thundering-herd refills

- Prefill tokens immediately on minute bucket transition

- Add unit tests covering memory caching, pre-fetching, concurrency, and fail-open
2026-09-26 23:16:19 +05:30
coo1estguy 9e6a9631db refactor: remove fast-path bypasses and restore engine decision stage 2026-09-25 23:29:07 +05:30
coo1estguy 4642308fe8 fix(delivery): enforce two-phase idempotency leases and stream retry tracking
- Add two-phase idempotency with acquireLease and markProcessed using a single-key SET NX fast path
- Return { ack: false } on lock collisions to leave messages pending in the stream instead of acknowledging them
- Bound provider dispatch timeout within the lock lease window to prevent duplicate delivery under slow responses
- Replace per-message Redis MULTI/INCR retry keys with Redis Stream PEL delivery counts
- Handle LockHeldError in BaseWorker to skip message acknowledgement when another worker holds the lock
2026-09-25 22:46:06 +05:30
coo1estguy 905485c0c7 perf: replace dynamic imports with static top-level imports
Eliminate dynamic imports on execution hot paths and module startup:
- Import drizzle-orm sql statically in delivery worker
- Import node:crypto and transport registry statically in API handlers
- Import provider SDKs statically in Twilio and FCM packages
- Replace root barrel imports in config and db with direct submodules to prevent circular evaluation
- Update workflow tests to preserve redis mock exports via importOriginal
2026-09-25 14:43:05 +05:30
coo1estguy 3be502943f perf(redis): pre-compile Lua scripts via defineCommand across services
- Register custom Lua scripts during RedisClient initialization via defineCommand
- Pre-compile checkApiRateLimit, throttleProvider, throttleUser, releaseLock, renewLock, and schedulerPoll
- Augment ioredis Redis and ChainableCommander type interfaces for custom commands
- Maintain eval fallbacks across callers for compatibility with test mocks
- Add test suite in tests/redis.test.ts verifying registration, execution, and fallbacks
2026-09-25 14:20:46 +05:30
coo1estguy 6c4315d9e7 perf(api): replace sorted sets with sliding-window counter in rate limiting
- Replace O(N) ZSET with O(1) sliding-window counter across two buckets
- Define LUA_SLIDING_WINDOW_COUNTER at module level to avoid per-request string allocations
- Eliminate crypto randomBytes generation on hot path
- Use Redis hash tags for cluster-safe multi-key evaluation
- Add tests for window weighting, TTL, and limit rejection
2026-09-25 13:52:01 +05:30
coo1estguy d439f051a6 fix(metrics): compute true stream backlog via consumer group lag and set retention to 100k
- Cap Redis stream retention at 100,000 messages (down from 10,000,000) in
  src/config/index.ts and .env.example to prevent unbounded memory growth.
- In src/services/api/handlers.ts, update getSystemMetrics to resolve stream
  queue depths using consumer group lag and in-flight pending entries (via
  XINFO GROUPS) instead of raw XLEN. This prevents acknowledged and delivered
  notifications from persisting as ghost backlog in the dashboard.
- Retain XLEN measurement for DEAD_LETTER queue where messages persist until
  replayed or deleted.
- Adjust dashboard capacity scaling to 100,000 items in AnalyticsPage.tsx and
  update backpressure warning threshold to 80,000 in DashboardHeader.tsx.
- Ensure 0-depth queues render empty (0% width) in the backlog gauge meter.
- Update queue test expectations in tests/queue.test.ts and add unit tests
  verifying consumer group lag resolution in tests/api-admin.test.ts.
2026-09-25 12:57:02 +05:30
coo1estguy 7624d5d3c2 perf(pipeline): collapse enricher-engine streams and add direct critical fast-path
- Collapse Enricher and Decision Engine into direct outbound dispatch: Enricher evaluates quiet hours, topic opt-outs, and renders templates in-memory before publishing directly to STREAMS.OUTBOUND_*, eliminating intermediate STREAMS.ENRICHED_* Redis stream hops
- Add transactional direct fast-path in /v1/notify: Critical priority notifications with inline user targets bypass inbound and decision queues, rendering templates from cache and dispatching directly to STREAMS.OUTBOUND_CRITICAL
- Reduce first-delivery latency from 6.2s to 0.38s and increase wall drain throughput by up to 58% on budget infrastructure
- Add unit and integration tests covering Enricher direct outbound dispatch and API transactional fast-path routing
2026-09-24 22:58:13 +05:30
coo1estguy c327a28933 perf(pipeline): batch stream ACKs, consolidate outbox writes, and remove blocking Redis round-trips
- Implement stream-keyed ACK buffering in StreamConsumer with periodic and capacity-triggered flushes
- Consolidate delivery outbox persistence to a single post-dispatch write, cutting PostgreSQL outbox writes by 50%
- Make retry key deletion and idempotency TTL extension non-blocking on message success
- Add host-unique consumer identifiers across distributed delivery, engine, and enricher workers
- Add unit tests verifying buffered stream ACK flushing behavior
2026-09-24 22:31:42 +05:30
coo1estguy 57e6197080 perf: optimize delivery pipeline batching, timers, and caching 2026-09-24 16:28:51 +05:30
coo1estguy fed2b27211 chore(release): bump version to 0.1.10 2026-09-19 23:18:20 +05:30
coo1estguy 46a08915e7 fix(security): reject DLQ replay/delete when message projectId is absent for project-scoped callers 2026-09-19 15:30:32 +05:30
coo1estguy 020c71bd9f fix(security): require UNSUBSCRIBE_SECRET in production to prevent silent opt-out drops 2026-09-19 15:27:34 +05:30
coo1estguy b305d0ea03 fix(security): restrict CORS to configured origin for session-auth routes 2026-09-19 15:25:46 +05:30
coo1estguy ace80fe5a1 fix(api): remove redundant decodeURIComponent in deleteSuppression 2026-09-19 15:00:59 +05:30
coo1estguy 1a6ba84b3a fix(api): handle malformed URI parameters gracefully in router 2026-09-19 14:58:53 +05:30
coo1estguy d876be851f fix(api): scope getUserDetails message logs to user contacts 2026-09-19 14:57:24 +05:30
coo1estguy d0ca566d6c fix(api): prevent path prefix traversal in admin static handler 2026-09-19 14:54:28 +05:30
coo1estguy 1625cdc8f8 fix(api): protect metrics endpoint and prevent proxy header spoofing 2026-09-19 14:27:48 +05:30
coo1estguy 8c907fa6bc fix(api): isolate tenant metrics and scope global queue depths to admin 2026-09-19 14:23:50 +05:30
coo1estguy 151db527e1 fix(auth): add login rate limiting and eliminate timing side-channel 2026-09-19 14:18:35 +05:30
coo1estguy 6f64922151 fix(api): enforce project isolation on dead letter queue endpoints 2026-09-19 14:12:09 +05:30
coo1estguy afbbaae72e refactor(dashboard): migrate to vite spa and standard shadcn ui
- migrate dashboard from Next.js to Vite SPA with React Router
- install standard shadcn/ui components using shadcn CLI
- use standard shadcn theme with default CSS variables
- handle optional project scoping in system metrics and health handlers
- update admin-static service to serve Vite dist and proxy dev server
2026-09-15 23:07:59 +05:30
coo1estguy 231fb19349 feat(server): serve admin dashboard static export at /admin [skip ci] 2026-09-14 23:10:54 +05:30
coo1estguy f4a60f8cf6 feat(auth): implement scrypt password hashing and session management [skip ci] 2026-09-14 23:09:51 +05:30
coo1estguy b5b5db0136 feat(db): add admin_users schema and repository with create-admin script [skip ci] 2026-09-14 23:07:39 +05:30
coo1estguy 35190585d7 feat(client): support dynamic contacts array, ergonomic addUser signature, and strong SDK types 2026-09-13 00:02:46 +05:30
coo1estguy d293685ca6 fix(config): trim ADMIN_API_KEY to prevent trailing whitespace auth failures 2026-09-08 15:45:36 +05:30
coo1estguy 5fca3ee5d2 feat: promote slack to a dedicated native notification channel 2026-09-07 17:02:39 +05:30
coo1estguy d5ff54dcee Merge branch 'channel/meta/whatsapp' into main 2026-09-07 16:53:28 +05:30
Suhaanth V V 1993b2e99c feat: add new providers 2026-09-02 15:09:30 +05:30
coo1estguy 8438cdb3a2 fix(ci): fix cross-platform test spawning, redis disconnect handling, and ci step guards
- Fix worker spawning in tests/chaos/crash.test.ts to support Linux runners

- Gracefully handle closed connections during RedisClient.disconnect()

- Detach SIGINT/SIGTERM process listeners on NotifkitServer.stop()

- Only run coverage badge generation if the test step was not skipped
2026-09-01 14:17:38 +05:30
kishanaik5 e0f376faba added webhooks for whatsapp channels 2026-09-01 14:05:49 +05:30
kishanaik5 03c30441ec added whatsapp channel support irrespective of sms body 2026-08-29 14:01:30 +05:30
coo1estguy e565cca5a4 feat(mcp): add template preview/render, multichannel campaign, workflow search, and user filtering 2026-08-20 15:41:08 +05:30