feat: split honeypot and dashboard into separate services on dedicated ports

- Honeypot listens on port 5010
- Dashboard listens on port 5123 at URL /
- Add app_dashboard.py as standalone FastAPI app
- Add start.sh to launch both uvicorn processes
- Fix dashboard_path computation to return "" at root (avoid double-slash URLs)
- Update Dockerfile, docker-compose.yaml, config.yaml, config.py

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Dockerfile

@@ -15,16 +15,18 @@ RUN pip install --no-cache-dir --upgrade pip && \
 COPY src/ /app/src/
 COPY wordlists.json /app/
 COPY entrypoint.sh /app/
+COPY start.sh /app/
 COPY config.yaml /app/
 
 RUN useradd -m -u 1000 krawl && \
     mkdir -p /app/logs /app/data /app/exports && \
     chown -R krawl:krawl /app && \
-    chmod +x /app/entrypoint.sh
+    chmod +x /app/entrypoint.sh /app/start.sh
 
-EXPOSE 5000
+EXPOSE 5010
+EXPOSE 5123
 
 ENV PYTHONUNBUFFERED=1
 
 ENTRYPOINT ["/app/entrypoint.sh"]
-CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "5000", "--app-dir", "src"]
+CMD ["/app/start.sh"]

README.md

@@ -39,7 +39,7 @@
 - [Demo](#demo)
 - [What is Krawl?](#what-is-krawl)
 - [Krawl Dashboard](#krawl-dashboard)
-- [Installation](#-installation)
+- [Quickstart](#quickstart)
   - [Docker Run](#docker-run)
   - [Docker Compose](#docker-compose)
   - [Kubernetes](#kubernetes)
@@ -51,7 +51,7 @@
 - [IP Reputation](#ip-reputation)
 - [Forward Server Header](#forward-server-header)
 - [Additional Documentation](#additional-documentation)
-- [Contributing](#-contributing)
+- [Contributing](#contributing)
 
 ## Demo
 Tip: crawl the `robots.txt` paths for additional fun
@@ -88,24 +88,29 @@ You can easily expose Krawl alongside your other services to shield them from we
 
 Krawl provides a comprehensive dashboard, accessible at a **random secret path** generated at startup or at a **custom path** configured via `KRAWL_DASHBOARD_SECRET_PATH`. This keeps the dashboard hidden from attackers scanning your honeypot.
 
-The dashboard is organized in three main tabs:
+The dashboard is organized in five tabs:
 
-- **Overview** — High-level view of attack activity: an interactive map of IP origins, recent suspicious requests, and top IPs, User-Agents, and paths.
+- **Overview**: high-level view of attack activity: an interactive map of IP origins, recent suspicious requests, and top IPs, User-Agents, and paths.
 
 ![geoip](img/geoip_dashboard.png)
 
-- **Attacks** — Detailed breakdown of captured credentials, honeypot triggers, and detected attack types (SQLi, XSS, path traversal, etc.) with charts and tables.
+- **Attacks**: detailed breakdown of captured credentials, honeypot triggers, and detected attack types (SQLi, XSS, path traversal, etc.) with charts and tables.
 
 ![attack_types](img/attack_types.png)
 
-- **IP Insight** — In-depth forensic view of a selected IP: geolocation, ISP/ASN info, reputation flags, behavioral timeline, attack type distribution, and full access history.
+- **IP Insight**: in-depth forensic view of a selected IP: geolocation, ISP/ASN info, reputation flags, behavioral timeline, attack type distribution, and full access history.
 
 ![ipinsight](img/ip_insight_dashboard.png)
 
+Additionally, after authenticating with the dashboard password, two protected tabs become available:
+
+- **Tracked IPs**: maintain a watchlist of IP addresses you want to monitor over time.
+- **IP Banlist**: manage IP bans, view detected attackers, and export the banlist in raw or IPTables format.
+
 For more details, see the [Dashboard documentation](docs/dashboard.md).
 
 
-## 🚀 Installation
+## Quickstart
 
 ### Docker Run
 
@@ -308,7 +313,7 @@ location / {
 | [Wordlist](docs/wordlist.md) | Customize fake usernames, passwords, and directory listings |
 | [Dashboard](docs/dashboard.md) | Access and explore the real-time monitoring dashboard |
 
-## 🤝 Contributing
+## Contributing
 
 Contributions welcome! Please:
 1. Fork the repository

config.yaml

@@ -1,7 +1,7 @@
 # Krawl Honeypot Configuration
 
 server:
-  port: 5000
+  port: 5010
   delay: 100  # Response delay in milliseconds
 
   # manually set the server header, if null a random one will be used.
@@ -20,10 +20,9 @@ canary:
   token_tries: 10
 
 dashboard:
-  # if set to "null" this will Auto-generates random path if not set
+  port: 5123
-  # can be set to "/dashboard" or similar <-- note this MUST include a forward slash
+  # Set to empty string "" to serve dashboard at root "/" on its own dedicated port
-  # secret_path: super-secret-dashboard-path
+  secret_path: ""
-  secret_path: null
 
   # Password for accessing protected dashboard panels.
   # If null, a random password will be generated and printed in the logs.

docker-compose.yaml

@@ -1,5 +1,4 @@
 ---
-# THIS IS FOR DEVELOPMENT PURPOSES
 services:
   krawl:
     build:
@@ -7,12 +6,15 @@ services:
       dockerfile: Dockerfile
     container_name: krawl-server
     ports:
-      - "5000:5000"
+      - "5010:5010"   # Honeypot (crawler trap)
+      - "5123:5123"   # Dashboard (public UI)
     environment:
       - CONFIG_LOCATION=config.yaml
       # Uncomment to set a custom dashboard password (auto-generated if not set)
       # - KRAWL_DASHBOARD_PASSWORD=your-secret-password
-      # set this to change timezone, alternatively mount /etc/timezone or /etc/localtime based on the time system management of the host environment
+      # Override ports if needed
+      # - HONEYPOT_PORT=5010
+      # - DASHBOARD_PORT=5123
       # - TZ=${TZ}
     volumes:
       - ./wordlists.json:/app/wordlists.json:ro

docs/dashboard.md

@@ -2,20 +2,182 @@
 
 Access the dashboard at `http://<server-ip>:<port>/<dashboard-path>`
 
-The dashboard shows:
+The Krawl dashboard is a single-page application with **5 tabs**: Overview, Attacks, IP Insight, Tracked IPs, and IP Banlist. The last two tabs are only visible after authenticating with the dashboard password.
-- Total and unique accesses
-- Suspicious activity and attack detection
-- Top IPs, paths, user-agents and GeoIP localization
-- Real-time monitoring
 
-The attackers' access to the honeypot endpoint and related suspicious activities (such as failed login attempts) are logged.
+---
 
-Krawl also implements a scoring system designed to distinguish between malicious and legitimate behavior on the website.
+## Overview
 
-![dashboard-1](../img/dashboard-1.png)
+The default landing page provides a high-level summary of all traffic and suspicious activity detected by Krawl.
 
-The top IP Addresses is shown along with top paths and User Agents
+### Stats Cards
 
-![dashboard-2](../img/dashboard-2.png)
+Seven metric cards are displayed at the top:
 
-![dashboard-3](../img/dashboard-3.png)
+- **Total Accesses** — total number of requests received
+- **Unique IPs** — distinct IP addresses observed
+- **Unique Paths** — distinct request paths
+- **Suspicious Accesses** — requests flagged as suspicious
+- **Honeypot Caught** — requests that hit honeypot endpoints
+- **Credentials Captured** — login attempts captured by the honeypot
+- **Unique Attackers** — distinct IPs classified as attackers
+
+### Search
+
+A real-time search bar lets you search across attacks, IPs, patterns, and locations. Results are loaded dynamically as you type.
+
+### IP Origins Map
+
+An interactive world map (powered by Leaflet) displays the geolocation of top IP addresses. You can filter by category:
+
+- Attackers
+- Bad Crawlers
+- Good Crawlers
+- Regular Users
+- Unknown
+
+The number of displayed IPs is configurable (top 10, 100, 1,000, or all).
+
+![Overview — Stats and Map](../img/geoip_dashboard.png)
+
+### Recent Suspicious Activity
+
+A table showing the last 10 suspicious requests with IP address, path, user-agent, and timestamp. Each entry provides actions to view the raw HTTP request or inspect the IP in detail.
+
+### Top IP Addresses
+
+A paginated, sortable table ranking IPs by access count. Each IP shows its category badge and can be clicked to expand inline details or open the IP Insight tab.
+
+### Top Paths
+
+A paginated table of the most accessed HTTP paths and their request counts.
+
+### Top User-Agents
+
+A paginated table of the most common user-agent strings with their frequency.
+
+![Overview — Tables](../img/overview_tables_dashboard.png)
+
+---
+
+## Attacks
+
+The Attacks tab focuses on detected malicious activity, attack patterns, and captured credentials.
+
+### Attackers by Total Requests
+
+A paginated table listing all detected attackers ranked by total requests. Columns include IP, total requests, first seen, last seen, and location. Sortable by multiple fields.
+
+![Attacks — Attackers and Credentials](../img/top_attackers_dashboard.png)
+
+### Captured Credentials
+
+A table of usernames and passwords captured from honeypot login forms, with timestamps. Useful for analyzing common credential stuffing patterns.
+
+### Honeypot Triggers by IP
+
+Shows which IPs accessed honeypot endpoints and how many times, sorted by trigger count.
+
+### Detected Attack Types
+
+A detailed table of individual attack detections showing IP, path, attack type classifications, user-agent, and timestamp. Each entry can be expanded to view the raw HTTP request.
+
+### Most Recurring Attack Types
+
+A Chart.js visualization showing the frequency distribution of detected attack categories (e.g., SQL injection, path traversal, XSS).
+
+### Most Recurring Attack Patterns
+
+A paginated table of specific attack patterns and their occurrence counts across all traffic.
+
+![Attacks — Attack Types and Patterns](../img/attack_types_dashboard.png)
+
+---
+
+## IP Insight
+
+The IP Insight tab provides a deep-dive view for a single IP address. It is activated by clicking "Inspect IP" from any table in the dashboard.
+
+### IP Information Card
+
+Displays comprehensive details about the selected IP:
+
+- **Activity** — total requests, first seen, last seen, last analysis timestamp
+- **Geo & Network** — location, region, timezone, ISP, ASN, reverse DNS
+- **Category** — classification badge (Attacker, Good Crawler, Bad Crawler, Regular User, Unknown)
+
+### Ban & Track Actions
+
+When authenticated, admin actions are available:
+
+- **Ban/Unban** — immediately add or remove the IP from the banlist
+- **Track/Untrack** — add the IP to your watchlist for ongoing monitoring
+
+### Blocklist Memberships
+
+Shows which threat intelligence blocklists the IP appears on, providing external reputation context.
+
+### Access Logs
+
+A filtered view of all requests made by this specific IP, with full request details.
+
+![IP Insight — Detail View](../img/ip_insight_dashboard.png)
+
+---
+
+## Tracked IPs
+
+> Requires authentication with the dashboard password.
+
+The Tracked IPs tab lets you maintain a watchlist of IP addresses you want to monitor over time.
+
+### Track New IP
+
+A form to add any IP address to your tracking list for ongoing observation.
+
+### Currently Tracked IPs
+
+A paginated table of all manually tracked IPs, with the option to untrack each one.
+
+![Tracked IPs](../img/tracked_ips_dashboard.png)
+
+---
+
+## IP Banlist
+
+> Requires authentication with the dashboard password.
+
+The IP Banlist tab provides tools for managing IP bans. Bans are exported every 5 minutes.
+
+### Force Ban IP
+
+A form to immediately ban any IP address by entering it manually.
+
+### Detected Attackers
+
+A paginated list of all IPs detected as attackers, with quick-ban actions for each entry.
+
+![IP Banlist — Detected](../img/banlist_attackers_dashboard.png)
+
+### Active Ban Overrides
+
+A table of currently active manual ban overrides, with options to unban or reset the override status for each IP.
+
+![IP Banlist — Overrides](../img/banlist_overrides_dashboard.png)
+
+### Export Banlist
+
+A dropdown menu to download the current banlist in two formats:
+
+- **Raw IPs List** — plain text, one IP per line
+- **IPTables Rules** — ready-to-use firewall rules
+
+---
+
+## Authentication
+
+The dashboard uses session-based authentication with secure HTTP-only cookies. Protected features (Tracked IPs, IP Banlist, ban/track actions) require entering the dashboard password. The login includes brute-force protection with IP-based rate limiting and exponential backoff.
+
+Click the lock icon in the top-right corner of the navigation bar to authenticate or log out.
+
+![Authentication Modal](../img/auth_prompt.png)

helm/Chart.yaml

@@ -2,8 +2,8 @@ apiVersion: v2
 name: krawl-chart
 description: A Helm chart for Krawl honeypot server
 type: application
-version: 1.1.7
+version: 1.2.0
-appVersion: 1.1.7
+appVersion: 1.2.0
 keywords:
   - honeypot
   - security

helm/README.md

@@ -14,7 +14,7 @@ A Helm chart for deploying the Krawl honeypot application on Kubernetes.
 
 ```bash
 helm install krawl oci://ghcr.io/blessedrebus/krawl-chart \
-  --version 1.1.3 \
+  --version 1.2.0 \
   --namespace krawl-system \
   --create-namespace \
   -f values.yaml  # optional
@@ -170,7 +170,7 @@ kubectl get secret krawl-server -n krawl-system \
 You can override individual values with `--set` without a values file:
 
 ```bash
-helm install krawl oci://ghcr.io/blessedrebus/krawl-chart --version 1.1.3 \
+helm install krawl oci://ghcr.io/blessedrebus/krawl-chart --version 1.2.0 \
   --set ingress.hosts[0].host=honeypot.example.com \
   --set config.canary.token_url=https://canarytokens.com/your-token
 ```
@@ -178,7 +178,7 @@ helm install krawl oci://ghcr.io/blessedrebus/krawl-chart --version 1.1.3 \
 ## Upgrading
 
 ```bash
-helm upgrade krawl oci://ghcr.io/blessedrebus/krawl-chart --version 1.1.3 -f values.yaml
+helm upgrade krawl oci://ghcr.io/blessedrebus/krawl-chart --version 1.2.0 -f values.yaml
 ```
 
 ## Uninstalling

helm/values.yaml

@@ -3,7 +3,7 @@ replicaCount: 1
 image:
   repository: ghcr.io/blessedrebus/krawl
   pullPolicy: Always
-  tag: "1.1.3"
+  tag: "1.2.0"
 
 imagePullSecrets: []
 nameOverride: "krawl"

src/app.py

@@ -10,7 +10,6 @@ import os
 from contextlib import asynccontextmanager
 
 from fastapi import FastAPI, Request, Response
-from fastapi.staticfiles import StaticFiles
 
 from config import get_config
 from tracker import AccessTracker, set_tracker
@@ -72,21 +71,7 @@ async def lifespan(app: FastAPI):
     tasks_master = get_tasksmaster()
     tasks_master.run_scheduled_tasks()
 
-    password_line = ""
+    app_logger.info(f"Starting honeypot deception server on port {config.port}...")
-    if config.dashboard_password_generated:
-        password_line = (
-            f"\n\nDASHBOARD PASSWORD (auto-generated)\n{config.dashboard_password}"
-        )
-
-    banner = f"""
-
-============================================================
-DASHBOARD AVAILABLE AT
-{config.dashboard_secret_path}{password_line}
-============================================================
-    """
-    app_logger.info(banner)
-    app_logger.info(f"Starting deception server on port {config.port}...")
     if config.canary_token_url:
         app_logger.info(
             f"Canary token will appear after {config.canary_token_tries} tries"
@@ -155,29 +140,10 @@ def create_app() -> FastAPI:
             access_logger.info(f"[{method}] {client_ip} - {path} - {status}")
         return response
 
-    # Mount static files for the dashboard
-    config = get_config()
-    secret = config.dashboard_secret_path.lstrip("/")
-    static_dir = os.path.join(os.path.dirname(__file__), "templates", "static")
-    application.mount(
-        f"/{secret}/static",
-        StaticFiles(directory=static_dir),
-        name="dashboard-static",
-    )
-
     # Import and include routers
     from routes.honeypot import router as honeypot_router
-    from routes.api import router as api_router
-    from routes.dashboard import router as dashboard_router
-    from routes.htmx import router as htmx_router
 
-    # Dashboard/API/HTMX routes (prefixed with secret path, before honeypot catch-all)
+    # Honeypot routes (catch-all)
-    dashboard_prefix = f"/{secret}"
-    application.include_router(dashboard_router, prefix=dashboard_prefix)
-    application.include_router(api_router, prefix=dashboard_prefix)
-    application.include_router(htmx_router, prefix=dashboard_prefix)
-
-    # Honeypot routes (catch-all must be last)
     application.include_router(honeypot_router)
 
     return application

src/app_dashboard.py

@@ -0,0 +1,94 @@
+#!/usr/bin/env python3
+
+"""
+FastAPI application for the Krawl dashboard.
+Runs on a dedicated port (default 5123) and serves the dashboard at /.
+"""
+
+import os
+from contextlib import asynccontextmanager
+
+from fastapi import FastAPI, Request, Response
+from fastapi.staticfiles import StaticFiles
+
+from config import get_config
+from database import initialize_database
+from tasks_master import get_tasksmaster
+from logger import initialize_logging, get_app_logger
+
+
+@asynccontextmanager
+async def lifespan(app: FastAPI):
+    """Dashboard application startup and shutdown lifecycle."""
+    config = get_config()
+    # Force dashboard path to empty (root) for this dedicated app
+    config.dashboard_secret_path = ""
+
+    initialize_logging(log_level=config.log_level)
+    app_logger = get_app_logger()
+
+    try:
+        app_logger.info(f"Initializing database at: {config.database_path}")
+        initialize_database(config.database_path)
+        app_logger.info("Database ready")
+    except Exception as e:
+        app_logger.warning(
+            f"Database initialization failed: {e}. Continuing with in-memory only."
+        )
+
+    app.state.config = config
+
+    # Start scheduled tasks (handles cache warming, backups, etc.)
+    tasks_master = get_tasksmaster()
+    tasks_master.run_scheduled_tasks()
+
+    password_line = ""
+    if config.dashboard_password_generated:
+        password_line = (
+            f"\n\nDASHBOARD PASSWORD (auto-generated)\n{config.dashboard_password}"
+        )
+
+    banner = f"""
+
+============================================================
+DASHBOARD AVAILABLE AT http://0.0.0.0:{config.dashboard_port}/{password_line}
+============================================================
+    """
+    app_logger.info(banner)
+    app_logger.info(f"Starting dashboard server on port {config.dashboard_port}...")
+
+    yield
+
+    app_logger.info("Dashboard server shutting down...")
+
+
+def create_dashboard_app() -> FastAPI:
+    """Create and configure the dashboard FastAPI application."""
+    application = FastAPI(
+        docs_url=None,
+        redoc_url=None,
+        openapi_url=None,
+        lifespan=lifespan,
+    )
+
+    # Mount static files at /static
+    static_dir = os.path.join(os.path.dirname(__file__), "templates", "static")
+    application.mount(
+        "/static",
+        StaticFiles(directory=static_dir),
+        name="dashboard-static",
+    )
+
+    from routes.api import router as api_router
+    from routes.dashboard import router as dashboard_router
+    from routes.htmx import router as htmx_router
+
+    # All dashboard routes at root prefix
+    application.include_router(dashboard_router, prefix="")
+    application.include_router(api_router, prefix="")
+    application.include_router(htmx_router, prefix="")
+
+    return application
+
+
+app = create_dashboard_app()

src/config.py

@@ -18,7 +18,8 @@ import yaml
 class Config:
     """Configuration class for the deception server"""
 
-    port: int = 5000
+    port: int = 5010
+    dashboard_port: int = 5123
     delay: int = 100  # milliseconds
     server_header: str = ""
     links_length_range: Tuple[int, int] = (5, 15)
@@ -173,6 +174,8 @@ class Config:
         dashboard_path = dashboard.get("secret_path")
         if dashboard_path is None:
             dashboard_path = f"/{os.urandom(16).hex()}"
+        elif dashboard_path == "":
+            dashboard_path = ""
         else:
             # ensure the dashboard path starts with a /
             if dashboard_path[:1] != "/":
@@ -186,7 +189,8 @@ class Config:
             dashboard_password_generated = True
 
         return cls(
-            port=server.get("port", 5000),
+            port=server.get("port", 5010),
+            dashboard_port=dashboard.get("port", 5123),
             delay=server.get("delay", 100),
             server_header=server.get("server_header", ""),
             links_length_range=(

src/routes/api.py

@@ -254,7 +254,13 @@ async def all_ips(
     page_size = min(max(1, page_size), 10000)
 
     # Serve from cache on default map request (top 100 IPs)
-    if page == 1 and page_size == 100 and sort_by == "total_requests" and sort_order == "desc" and is_warm():
+    if (
+        page == 1
+        and page_size == 100
+        and sort_by == "total_requests"
+        and sort_order == "desc"
+        and is_warm()
+    ):
         cached = get_cached("map_ips")
         if cached:
             return JSONResponse(content=cached, headers=_no_cache_headers())

src/routes/dashboard.py

@@ -19,7 +19,8 @@ router = APIRouter()
 @router.get("/")
 async def dashboard_page(request: Request):
     config = request.app.state.config
-    dashboard_path = "/" + config.dashboard_secret_path.lstrip("/")
+    _path = config.dashboard_secret_path.strip("/")
+    dashboard_path = f"/{_path}" if _path else ""
 
     # Serve from pre-computed cache when available, fall back to live queries
     if is_warm():
@@ -50,7 +51,8 @@ async def ip_page(ip_address: str, request: Request):
     try:
         stats = db.get_ip_stats_by_ip(ip_address)
         config = request.app.state.config
-        dashboard_path = "/" + config.dashboard_secret_path.lstrip("/")
+        _path = config.dashboard_secret_path.strip("/")
+        dashboard_path = f"/{_path}" if _path else ""
 
         if stats:
             # Transform fields for template compatibility

src/routes/htmx.py

@@ -17,7 +17,8 @@ router = APIRouter()
 
 def _dashboard_path(request: Request) -> str:
     config = request.app.state.config
-    return "/" + config.dashboard_secret_path.lstrip("/")
+    path = config.dashboard_secret_path.strip("/")
+    return f"/{path}" if path else ""
 
 
 # ── Honeypot Triggers ────────────────────────────────────────────────
@@ -60,7 +61,11 @@ async def htmx_top_ips(
     sort_order: str = Query("desc"),
 ):
     # Serve from cache on default first-page request
-    cached = get_cached("top_ips") if (page == 1 and sort_by == "count" and sort_order == "desc" and is_warm()) else None
+    cached = (
+        get_cached("top_ips")
+        if (page == 1 and sort_by == "count" and sort_order == "desc" and is_warm())
+        else None
+    )
     if cached:
         result = cached
     else:
@@ -93,7 +98,11 @@ async def htmx_top_paths(
     sort_by: str = Query("count"),
     sort_order: str = Query("desc"),
 ):
-    cached = get_cached("top_paths") if (page == 1 and sort_by == "count" and sort_order == "desc" and is_warm()) else None
+    cached = (
+        get_cached("top_paths")
+        if (page == 1 and sort_by == "count" and sort_order == "desc" and is_warm())
+        else None
+    )
     if cached:
         result = cached
     else:
@@ -126,7 +135,11 @@ async def htmx_top_ua(
     sort_by: str = Query("count"),
     sort_order: str = Query("desc"),
 ):
-    cached = get_cached("top_ua") if (page == 1 and sort_by == "count" and sort_order == "desc" and is_warm()) else None
+    cached = (
+        get_cached("top_ua")
+        if (page == 1 and sort_by == "count" and sort_order == "desc" and is_warm())
+        else None
+    )
     if cached:
         result = cached
     else:

start.sh

@@ -0,0 +1,17 @@
+#!/bin/sh
+set -e
+
+HONEYPOT_PORT=${HONEYPOT_PORT:-5010}
+DASHBOARD_PORT=${DASHBOARD_PORT:-5123}
+
+echo "Starting Krawl honeypot on port $HONEYPOT_PORT..."
+uvicorn app:app --host 0.0.0.0 --port "$HONEYPOT_PORT" --app-dir src &
+HONEYPOT_PID=$!
+
+echo "Starting Krawl dashboard on port $DASHBOARD_PORT..."
+uvicorn app_dashboard:app --host 0.0.0.0 --port "$DASHBOARD_PORT" --app-dir src &
+DASHBOARD_PID=$!
+
+# Wait for either process to exit; if one dies, kill the other
+wait -n 2>/dev/null || wait
+kill $HONEYPOT_PID $DASHBOARD_PID 2>/dev/null || true