Merge pull request #128 from BlessedRebuS/dev

Feat/release 1.2
This commit is contained in:
Patrick Di Fazio
2026-03-10 14:09:46 +01:00
committed by GitHub
52 changed files with 2170 additions and 85 deletions

View File

@@ -39,11 +39,11 @@
- [Demo](#demo)
- [What is Krawl?](#what-is-krawl)
- [Krawl Dashboard](#krawl-dashboard)
- [Installation](#-installation)
- [Quickstart](#quickstart)
- [Docker Run](#docker-run)
- [Docker Compose](#docker-compose)
- [Kubernetes](#kubernetes)
- [Local (Python)](#local-python)
- [Uvicorn (Python)](#uvicorn-python)
- [Configuration](#configuration)
- [config.yaml](#configuration-via-configyaml)
- [Environment Variables](#configuration-via-enviromental-variables)
@@ -51,7 +51,7 @@
- [IP Reputation](#ip-reputation)
- [Forward Server Header](#forward-server-header)
- [Additional Documentation](#additional-documentation)
- [Contributing](#-contributing)
- [Contributing](#contributing)
## Demo
Tip: crawl the `robots.txt` paths for additional fun
@@ -88,24 +88,29 @@ You can easily expose Krawl alongside your other services to shield them from we
Krawl provides a comprehensive dashboard, accessible at a **random secret path** generated at startup or at a **custom path** configured via `KRAWL_DASHBOARD_SECRET_PATH`. This keeps the dashboard hidden from attackers scanning your honeypot.
The dashboard is organized in three main tabs:
The dashboard is organized in five tabs:
- **Overview** — High-level view of attack activity: an interactive map of IP origins, recent suspicious requests, and top IPs, User-Agents, and paths.
- **Overview**: high-level view of attack activity: an interactive map of IP origins, recent suspicious requests, and top IPs, User-Agents, and paths.
![geoip](img/geoip_dashboard.png)
- **Attacks** — Detailed breakdown of captured credentials, honeypot triggers, and detected attack types (SQLi, XSS, path traversal, etc.) with charts and tables.
- **Attacks**: detailed breakdown of captured credentials, honeypot triggers, and detected attack types (SQLi, XSS, path traversal, etc.) with charts and tables.
![attack_types](img/attack_types.png)
- **IP Insight** — In-depth forensic view of a selected IP: geolocation, ISP/ASN info, reputation flags, behavioral timeline, attack type distribution, and full access history.
- **IP Insight**: in-depth forensic view of a selected IP: geolocation, ISP/ASN info, reputation flags, behavioral timeline, attack type distribution, and full access history.
![ipinsight](img/ip_insight_dashboard.png)
Additionally, after authenticating with the dashboard password, two protected tabs become available:
- **Tracked IPs**: maintain a watchlist of IP addresses you want to monitor over time.
- **IP Banlist**: manage IP bans, view detected attackers, and export the banlist in raw or IPTables format.
For more details, see the [Dashboard documentation](docs/dashboard.md).
## 🚀 Installation
## Quickstart
### Docker Run
@@ -117,6 +122,7 @@ docker run -d \
-e KRAWL_PORT=5000 \
-e KRAWL_DELAY=100 \
-e KRAWL_DASHBOARD_SECRET_PATH="/my-secret-dashboard" \
-e KRAWL_DASHBOARD_PASSWORD="my-secret-password" \
-v krawl-data:/app/data \
--name krawl \
ghcr.io/blessedrebus/krawl:latest
@@ -138,6 +144,8 @@ services:
environment:
- CONFIG_LOCATION=config.yaml
- TZ=Europe/Rome
# - KRAWL_DASHBOARD_SECRET_PATH="/my-secret-dashboard"
# - KRAWL_DASHBOARD_PASSWORD=my-secret-password
volumes:
- ./config.yaml:/app/config.yaml:ro
# bind mount for firewall exporters
@@ -164,7 +172,7 @@ docker-compose down
### Kubernetes
**Krawl is also available natively on Kubernetes**. Installation can be done either [via manifest](kubernetes/README.md) or [using the helm chart](helm/README.md).
### Python + Uvicorn
### Uvicorn (Python)
Run Krawl directly with Python (suggested version 13) and uvicorn for local development or testing:
@@ -197,6 +205,7 @@ You can use the [config.yaml](config.yaml) file for advanced configurations, suc
| `KRAWL_CANARY_TOKEN_URL` | External canary token URL | None |
| `KRAWL_CANARY_TOKEN_TRIES` | Requests before showing canary token | `10` |
| `KRAWL_DASHBOARD_SECRET_PATH` | Custom dashboard path | Auto-generated |
| `KRAWL_DASHBOARD_PASSWORD` | Password for protected dashboard panels | Auto-generated |
| `KRAWL_PROBABILITY_ERROR_CODES` | Error response probability (0-100%) | `0` |
| `KRAWL_DATABASE_PATH` | Database file location | `data/krawl.db` |
| `KRAWL_EXPORTS_PATH` | Path where firewalls rule sets are exported | `exports` |
@@ -228,8 +237,9 @@ export KRAWL_LINKS_PER_PAGE_RANGE="5,25"
export KRAWL_HTTP_RISKY_METHODS_THRESHOLD="0.2"
export KRAWL_VIOLATED_ROBOTS_THRESHOLD="0.15"
# Set custom dashboard path
# Set custom dashboard path and password
export KRAWL_DASHBOARD_SECRET_PATH="/my-secret-dashboard"
export KRAWL_DASHBOARD_PASSWORD="my-secret-password"
```
Example of a Docker run with env variables:
@@ -239,6 +249,7 @@ docker run -d \
-p 5000:5000 \
-e KRAWL_PORT=5000 \
-e KRAWL_DELAY=100 \
-e KRAWL_DASHBOARD_PASSWORD="my-secret-password" \
-e KRAWL_CANARY_TOKEN_URL="http://your-canary-token-url" \
--name krawl \
ghcr.io/blessedrebus/krawl:latest
@@ -302,7 +313,7 @@ location / {
| [Wordlist](docs/wordlist.md) | Customize fake usernames, passwords, and directory listings |
| [Dashboard](docs/dashboard.md) | Access and explore the real-time monitoring dashboard |
## 🤝 Contributing
## Contributing
Contributions welcome! Please:
1. Fork the repository

View File

@@ -25,6 +25,11 @@ dashboard:
# secret_path: super-secret-dashboard-path
secret_path: null
# Password for accessing protected dashboard panels.
# If null, a random password will be generated and printed in the logs.
# Can also be set via KRAWL_DASHBOARD_PASSWORD env var.
password: null
backups:
path: "backups"
cron: "*/30 * * * *"

View File

@@ -10,6 +10,8 @@ services:
- "5000:5000"
environment:
- CONFIG_LOCATION=config.yaml
# Uncomment to set a custom dashboard password (auto-generated if not set)
# - KRAWL_DASHBOARD_PASSWORD=your-secret-password
# set this to change timezone, alternatively mount /etc/timezone or /etc/localtime based on the time system management of the host environment
# - TZ=${TZ}
volumes:

View File

@@ -2,20 +2,182 @@
Access the dashboard at `http://<server-ip>:<port>/<dashboard-path>`
The dashboard shows:
- Total and unique accesses
- Suspicious activity and attack detection
- Top IPs, paths, user-agents and GeoIP localization
- Real-time monitoring
The Krawl dashboard is a single-page application with **5 tabs**: Overview, Attacks, IP Insight, Tracked IPs, and IP Banlist. The last two tabs are only visible after authenticating with the dashboard password.
The attackers' access to the honeypot endpoint and related suspicious activities (such as failed login attempts) are logged.
---
Krawl also implements a scoring system designed to distinguish between malicious and legitimate behavior on the website.
## Overview
![dashboard-1](../img/dashboard-1.png)
The default landing page provides a high-level summary of all traffic and suspicious activity detected by Krawl.
The top IP Addresses is shown along with top paths and User Agents
### Stats Cards
![dashboard-2](../img/dashboard-2.png)
Seven metric cards are displayed at the top:
![dashboard-3](../img/dashboard-3.png)
- **Total Accesses** — total number of requests received
- **Unique IPs** — distinct IP addresses observed
- **Unique Paths** — distinct request paths
- **Suspicious Accesses** — requests flagged as suspicious
- **Honeypot Caught** — requests that hit honeypot endpoints
- **Credentials Captured** — login attempts captured by the honeypot
- **Unique Attackers** — distinct IPs classified as attackers
### Search
A real-time search bar lets you search across attacks, IPs, patterns, and locations. Results are loaded dynamically as you type.
### IP Origins Map
An interactive world map (powered by Leaflet) displays the geolocation of top IP addresses. You can filter by category:
- Attackers
- Bad Crawlers
- Good Crawlers
- Regular Users
- Unknown
The number of displayed IPs is configurable (top 10, 100, 1,000, or all).
![Overview — Stats and Map](../img/geoip_dashboard.png)
### Recent Suspicious Activity
A table showing the last 10 suspicious requests with IP address, path, user-agent, and timestamp. Each entry provides actions to view the raw HTTP request or inspect the IP in detail.
### Top IP Addresses
A paginated, sortable table ranking IPs by access count. Each IP shows its category badge and can be clicked to expand inline details or open the IP Insight tab.
### Top Paths
A paginated table of the most accessed HTTP paths and their request counts.
### Top User-Agents
A paginated table of the most common user-agent strings with their frequency.
![Overview — Tables](../img/overview_tables_dashboard.png)
---
## Attacks
The Attacks tab focuses on detected malicious activity, attack patterns, and captured credentials.
### Attackers by Total Requests
A paginated table listing all detected attackers ranked by total requests. Columns include IP, total requests, first seen, last seen, and location. Sortable by multiple fields.
![Attacks — Attackers and Credentials](../img/top_attackers_dashboard.png)
### Captured Credentials
A table of usernames and passwords captured from honeypot login forms, with timestamps. Useful for analyzing common credential stuffing patterns.
### Honeypot Triggers by IP
Shows which IPs accessed honeypot endpoints and how many times, sorted by trigger count.
### Detected Attack Types
A detailed table of individual attack detections showing IP, path, attack type classifications, user-agent, and timestamp. Each entry can be expanded to view the raw HTTP request.
### Most Recurring Attack Types
A Chart.js visualization showing the frequency distribution of detected attack categories (e.g., SQL injection, path traversal, XSS).
### Most Recurring Attack Patterns
A paginated table of specific attack patterns and their occurrence counts across all traffic.
![Attacks — Attack Types and Patterns](../img/attack_types_dashboard.png)
---
## IP Insight
The IP Insight tab provides a deep-dive view for a single IP address. It is activated by clicking "Inspect IP" from any table in the dashboard.
### IP Information Card
Displays comprehensive details about the selected IP:
- **Activity** — total requests, first seen, last seen, last analysis timestamp
- **Geo & Network** — location, region, timezone, ISP, ASN, reverse DNS
- **Category** — classification badge (Attacker, Good Crawler, Bad Crawler, Regular User, Unknown)
### Ban & Track Actions
When authenticated, admin actions are available:
- **Ban/Unban** — immediately add or remove the IP from the banlist
- **Track/Untrack** — add the IP to your watchlist for ongoing monitoring
### Blocklist Memberships
Shows which threat intelligence blocklists the IP appears on, providing external reputation context.
### Access Logs
A filtered view of all requests made by this specific IP, with full request details.
![IP Insight — Detail View](../img/ip_insight_dashboard.png)
---
## Tracked IPs
> Requires authentication with the dashboard password.
The Tracked IPs tab lets you maintain a watchlist of IP addresses you want to monitor over time.
### Track New IP
A form to add any IP address to your tracking list for ongoing observation.
### Currently Tracked IPs
A paginated table of all manually tracked IPs, with the option to untrack each one.
![Tracked IPs](../img/tracked_ips_dashboard.png)
---
## IP Banlist
> Requires authentication with the dashboard password.
The IP Banlist tab provides tools for managing IP bans. Bans are exported every 5 minutes.
### Force Ban IP
A form to immediately ban any IP address by entering it manually.
### Detected Attackers
A paginated list of all IPs detected as attackers, with quick-ban actions for each entry.
![IP Banlist — Detected](../img/banlist_attackers_dashboard.png)
### Active Ban Overrides
A table of currently active manual ban overrides, with options to unban or reset the override status for each IP.
![IP Banlist — Overrides](../img/banlist_overrides_dashboard.png)
### Export Banlist
A dropdown menu to download the current banlist in two formats:
- **Raw IPs List** — plain text, one IP per line
- **IPTables Rules** — ready-to-use firewall rules
---
## Authentication
The dashboard uses session-based authentication with secure HTTP-only cookies. Protected features (Tracked IPs, IP Banlist, ban/track actions) require entering the dashboard password. The login includes brute-force protection with IP-based rate limiting and exponential backoff.
Click the lock icon in the top-right corner of the navigation bar to authenticate or log out.
![Authentication Modal](../img/auth_prompt.png)

View File

@@ -2,8 +2,8 @@ apiVersion: v2
name: krawl-chart
description: A Helm chart for Krawl honeypot server
type: application
version: 1.1.3
appVersion: 1.1.3
version: 1.2.0
appVersion: 1.2.0
keywords:
- honeypot
- security

View File

@@ -14,7 +14,7 @@ A Helm chart for deploying the Krawl honeypot application on Kubernetes.
```bash
helm install krawl oci://ghcr.io/blessedrebus/krawl-chart \
--version 1.1.3 \
--version 1.2.0 \
--namespace krawl-system \
--create-namespace \
-f values.yaml # optional
@@ -96,6 +96,7 @@ The following table lists the main configuration parameters of the Krawl chart a
| Parameter | Description | Default |
|-----------|-------------|---------|
| `config.dashboard.secret_path` | Secret dashboard path (auto-generated if null) | `null` |
| `dashboardPassword` | Password for protected panels (injected via Secret as `KRAWL_DASHBOARD_PASSWORD` env, auto-generated if empty) | `""` |
### API Configuration
@@ -169,7 +170,7 @@ kubectl get secret krawl-server -n krawl-system \
You can override individual values with `--set` without a values file:
```bash
helm install krawl oci://ghcr.io/blessedrebus/krawl-chart --version 1.1.3 \
helm install krawl oci://ghcr.io/blessedrebus/krawl-chart --version 1.2.0 \
--set ingress.hosts[0].host=honeypot.example.com \
--set config.canary.token_url=https://canarytokens.com/your-token
```
@@ -177,7 +178,7 @@ helm install krawl oci://ghcr.io/blessedrebus/krawl-chart --version 1.1.3 \
## Upgrading
```bash
helm upgrade krawl oci://ghcr.io/blessedrebus/krawl-chart --version 1.1.3 -f values.yaml
helm upgrade krawl oci://ghcr.io/blessedrebus/krawl-chart --version 1.2.0 -f values.yaml
```
## Uninstalling

View File

@@ -47,6 +47,13 @@ spec:
- name: TZ
value: {{ .Values.timezone | quote }}
{{- end }}
{{- if .Values.dashboardPassword }}
- name: KRAWL_DASHBOARD_PASSWORD
valueFrom:
secretKeyRef:
name: {{ include "krawl.fullname" . }}-dashboard
key: dashboard-password
{{- end }}
volumeMounts:
- name: config
mountPath: /app/config.yaml

View File

@@ -0,0 +1,11 @@
{{- if .Values.dashboardPassword }}
apiVersion: v1
kind: Secret
metadata:
name: {{ include "krawl.fullname" . }}-dashboard
labels:
{{- include "krawl.labels" . | nindent 4 }}
type: Opaque
stringData:
dashboard-password: {{ .Values.dashboardPassword | quote }}
{{- end }}

View File

@@ -3,7 +3,7 @@ replicaCount: 1
image:
repository: ghcr.io/blessedrebus/krawl
pullPolicy: Always
tag: "1.1.3"
tag: "1.2.0"
imagePullSecrets: []
nameOverride: "krawl"
@@ -60,6 +60,10 @@ tolerations: []
affinity: {}
# Dashboard password for protected panels
# If empty, a random password will be auto-generated and printed in the logs
dashboardPassword: ""
# Application configuration (config.yaml structure)
config:
server:

Binary file not shown.

After

Width:  |  Height:  |  Size: 283 KiB

BIN
img/auth_prompt.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 70 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 173 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 77 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 133 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 76 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 206 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 72 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 343 KiB

After

Width:  |  Height:  |  Size: 808 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 290 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 312 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 131 KiB

View File

@@ -29,6 +29,10 @@ kubectl get secret krawl-server -n krawl-system \
-o jsonpath='{.data.dashboard-path}' | base64 -d && echo
```
### Setting Dashboard Password
To set a custom password for protected dashboard panels, create the `secret.yaml` manifest (see `kubernetes/manifests/secret.yaml`) and uncomment the `KRAWL_DASHBOARD_PASSWORD` env var in the deployment. If not set, a random password is auto-generated and printed in the pod logs.
### From Source (Python 3.11+)
Clone the repository:

View File

@@ -33,6 +33,12 @@ spec:
env:
- name: CONFIG_LOCATION
value: "config.yaml"
# Uncomment to use dashboard password from secret
# - name: KRAWL_DASHBOARD_PASSWORD
# valueFrom:
# secretKeyRef:
# name: krawl-dashboard
# key: dashboard-password
volumeMounts:
- name: config
mountPath: /app/config.yaml

View File

@@ -0,0 +1,15 @@
# Source: krawl-chart/templates/secret.yaml
# Uncomment and set your dashboard password below.
# If not created, the password will be auto-generated and printed in the pod logs.
#
# apiVersion: v1
# kind: Secret
# metadata:
# name: krawl-dashboard
# namespace: krawl-system
# labels:
# app.kubernetes.io/name: krawl
# app.kubernetes.io/instance: krawl
# type: Opaque
# stringData:
# dashboard-password: "your-secret-password"

View File

@@ -16,7 +16,7 @@ from config import get_config
from tracker import AccessTracker, set_tracker
from database import initialize_database
from tasks_master import get_tasksmaster
from logger import initialize_logging, get_app_logger
from logger import initialize_logging, get_app_logger, get_access_logger
from generators import random_server_header
@@ -72,11 +72,17 @@ async def lifespan(app: FastAPI):
tasks_master = get_tasksmaster()
tasks_master.run_scheduled_tasks()
password_line = ""
if config.dashboard_password_generated:
password_line = (
f"\n\nDASHBOARD PASSWORD (auto-generated)\n{config.dashboard_password}"
)
banner = f"""
============================================================
DASHBOARD AVAILABLE AT
{config.dashboard_secret_path}
{config.dashboard_secret_path}{password_line}
============================================================
"""
app_logger.info(banner)
@@ -115,11 +121,40 @@ def create_app() -> FastAPI:
application.add_middleware(DeceptionMiddleware)
# Banned IP check middleware (outermost — runs first on request)
# Banned IP check middleware
from middleware.ban_check import BanCheckMiddleware
application.add_middleware(BanCheckMiddleware)
# Access log middleware (outermost — logs every request with real client IP)
@application.middleware("http")
async def access_log_middleware(request: Request, call_next):
from dependencies import get_client_ip
response: Response = await call_next(request)
# Banned requests are already logged by BanCheckMiddleware
if getattr(request.state, "banned", False):
return response
client_ip = get_client_ip(request)
path = request.url.path
method = request.method
status = response.status_code
access_logger = get_access_logger()
user_agent = request.headers.get("User-Agent", "")
tracker = request.app.state.tracker
suspicious = tracker.is_suspicious_user_agent(user_agent)
if suspicious:
access_logger.warning(
f"[SUSPICIOUS] [{method}] {client_ip} - {path} - {status} - {user_agent[:50]}"
)
else:
access_logger.info(f"[{method}] {client_ip} - {path} - {status}")
return response
# Mount static files for the dashboard
config = get_config()
secret = config.dashboard_secret_path.lstrip("/")

View File

@@ -28,6 +28,8 @@ class Config:
canary_token_url: Optional[str] = None
canary_token_tries: int = 10
dashboard_secret_path: str = None
dashboard_password: Optional[str] = None
dashboard_password_generated: bool = False
probability_error_codes: int = 0 # Percentage (0-100)
# Crawl limiting settings - for legitimate vs malicious crawlers
@@ -176,6 +178,13 @@ class Config:
if dashboard_path[:1] != "/":
dashboard_path = f"/{dashboard_path}"
# Handle dashboard_password - auto-generate if null/not set
dashboard_password = dashboard.get("password")
dashboard_password_generated = False
if dashboard_password is None:
dashboard_password = os.urandom(25).hex()
dashboard_password_generated = True
return cls(
port=server.get("port", 5000),
delay=server.get("delay", 100),
@@ -196,6 +205,8 @@ class Config:
canary_token_url=canary.get("token_url"),
canary_token_tries=canary.get("token_tries", 10),
dashboard_secret_path=dashboard_path,
dashboard_password=dashboard_password,
dashboard_password_generated=dashboard_password_generated,
probability_error_codes=behavior.get("probability_error_codes", 0),
exports_path=exports.get("path", "exports"),
backups_path=backups.get("path", "backups"),
@@ -247,6 +258,9 @@ def override_config_from_env(config: Config = None):
try:
field_type = config.__dataclass_fields__[field].type
env_value = os.environ[env_var]
# If password is overridden, it's no longer auto-generated
if field == "dashboard_password":
config.dashboard_password_generated = False
if field_type == int:
setattr(config, field, int(env_value))
elif field_type == float:

32
src/dashboard_cache.py Normal file
View File

@@ -0,0 +1,32 @@
"""
In-memory cache for dashboard Overview data.
A background task periodically refreshes this cache so the dashboard
serves pre-computed data instantly instead of hitting SQLite cold.
Memory footprint is fixed — each key is overwritten on every refresh.
"""
import threading
from typing import Any, Dict, Optional
_lock = threading.Lock()
_cache: Dict[str, Any] = {}
def get_cached(key: str) -> Optional[Any]:
"""Get a value from the dashboard cache."""
with _lock:
return _cache.get(key)
def set_cached(key: str, value: Any) -> None:
"""Set a value in the dashboard cache."""
with _lock:
_cache[key] = value
def is_warm() -> bool:
"""Check if the cache has been populated at least once."""
with _lock:
return "stats" in _cache

View File

@@ -34,6 +34,7 @@ from models import (
AttackDetection,
IpStats,
CategoryHistory,
TrackedIp,
)
from sanitizer import (
sanitize_ip,
@@ -2231,6 +2232,208 @@ class DatabaseManager:
finally:
self.close_session()
# ── Ban Override Management ──────────────────────────────────────────
def set_ban_override(self, ip: str, override: Optional[bool]) -> bool:
"""
Set ban override for an IP.
override=True: force into banlist
override=False: force remove from banlist
override=None: reset to automatic (category-based)
Returns True if the IP exists and was updated.
"""
session = self.session
sanitized_ip = sanitize_ip(ip)
ip_stats = session.query(IpStats).filter(IpStats.ip == sanitized_ip).first()
if not ip_stats:
return False
ip_stats.ban_override = override
try:
session.commit()
return True
except Exception as e:
session.rollback()
applogger.error(f"Error setting ban override for {sanitized_ip}: {e}")
return False
def force_ban_ip(self, ip: str) -> bool:
"""
Force-ban an IP that may not exist in ip_stats yet.
Creates a minimal entry if needed.
"""
session = self.session
sanitized_ip = sanitize_ip(ip)
ip_stats = session.query(IpStats).filter(IpStats.ip == sanitized_ip).first()
if not ip_stats:
ip_stats = IpStats(
ip=sanitized_ip,
total_requests=0,
first_seen=datetime.now(),
last_seen=datetime.now(),
)
session.add(ip_stats)
ip_stats.ban_override = True
try:
session.commit()
return True
except Exception as e:
session.rollback()
applogger.error(f"Error force-banning {sanitized_ip}: {e}")
return False
def get_ban_overrides_paginated(
self,
page: int = 1,
page_size: int = 25,
) -> Dict[str, Any]:
"""Get all IPs with a non-null ban_override, paginated."""
session = self.session
try:
base_query = session.query(IpStats).filter(IpStats.ban_override.isnot(None))
total = base_query.count()
total_pages = max(1, (total + page_size - 1) // page_size)
results = (
base_query.order_by(IpStats.last_seen.desc())
.offset((page - 1) * page_size)
.limit(page_size)
.all()
)
overrides = []
for r in results:
overrides.append(
{
"ip": r.ip,
"ban_override": r.ban_override,
"category": r.category,
"total_requests": r.total_requests,
"country_code": r.country_code,
"city": r.city,
"last_seen": r.last_seen.isoformat() if r.last_seen else None,
}
)
return {
"overrides": overrides,
"pagination": {
"page": page,
"page_size": page_size,
"total": total,
"total_pages": total_pages,
},
}
finally:
self.close_session()
# ── IP Tracking ──────────────────────────────────────────────────
def track_ip(self, ip: str) -> bool:
"""Add an IP to the tracked list with a snapshot of its current stats."""
session = self.session
sanitized_ip = sanitize_ip(ip)
existing = session.query(TrackedIp).filter(TrackedIp.ip == sanitized_ip).first()
if existing:
return True # already tracked
# Snapshot essential data from ip_stats
stats = session.query(IpStats).filter(IpStats.ip == sanitized_ip).first()
tracked = TrackedIp(
ip=sanitized_ip,
tracked_since=datetime.now(),
category=stats.category if stats else None,
total_requests=stats.total_requests if stats else 0,
country_code=stats.country_code if stats else None,
city=stats.city if stats else None,
last_seen=stats.last_seen if stats else None,
)
session.add(tracked)
try:
session.commit()
return True
except Exception as e:
session.rollback()
applogger.error(f"Error tracking IP {sanitized_ip}: {e}")
return False
def untrack_ip(self, ip: str) -> bool:
"""Remove an IP from the tracked list."""
session = self.session
sanitized_ip = sanitize_ip(ip)
tracked = session.query(TrackedIp).filter(TrackedIp.ip == sanitized_ip).first()
if not tracked:
return False
session.delete(tracked)
try:
session.commit()
return True
except Exception as e:
session.rollback()
applogger.error(f"Error untracking IP {sanitized_ip}: {e}")
return False
def is_ip_tracked(self, ip: str) -> bool:
"""Check if an IP is currently tracked."""
session = self.session
sanitized_ip = sanitize_ip(ip)
try:
return (
session.query(TrackedIp).filter(TrackedIp.ip == sanitized_ip).first()
is not None
)
finally:
self.close_session()
def get_tracked_ips_paginated(
self,
page: int = 1,
page_size: int = 25,
) -> Dict[str, Any]:
"""Get all tracked IPs, paginated. Reads only from tracked_ips table."""
session = self.session
try:
total = session.query(TrackedIp).count()
total_pages = max(1, (total + page_size - 1) // page_size)
tracked_rows = (
session.query(TrackedIp)
.order_by(TrackedIp.tracked_since.desc())
.offset((page - 1) * page_size)
.limit(page_size)
.all()
)
items = []
for t in tracked_rows:
items.append(
{
"ip": t.ip,
"tracked_since": (
t.tracked_since.isoformat() if t.tracked_since else None
),
"category": t.category,
"total_requests": t.total_requests or 0,
"country_code": t.country_code,
"city": t.city,
"last_seen": t.last_seen.isoformat() if t.last_seen else None,
}
)
return {
"tracked_ips": items,
"pagination": {
"page": page,
"page_size": page_size,
"total": total,
"total_pages": total_pages,
},
}
finally:
self.close_session()
# Module-level singleton instance
_db_manager = DatabaseManager()

View File

@@ -112,6 +112,10 @@ class LoggerManager:
credential_file_handler.setFormatter(credential_format)
self._credential_logger.addHandler(credential_file_handler)
# Disable uvicorn's default access log to avoid duplicate entries
# with the wrong (proxy) IP. Our custom access logger handles this.
logging.getLogger("uvicorn.access").setLevel(logging.WARNING)
self._initialized = True
@property

View File

@@ -2,6 +2,7 @@
"""
Middleware for checking if client IP is banned.
Resets the connection for banned IPs instead of sending a response.
"""
from starlette.middleware.base import BaseHTTPMiddleware
@@ -23,6 +24,15 @@ class BanCheckMiddleware(BaseHTTPMiddleware):
tracker = request.app.state.tracker
if tracker.is_banned_ip(client_ip):
from logger import get_access_logger
get_access_logger().info(
f"[BANNED] [{request.method}] {client_ip} - {request.url.path}"
)
request.state.banned = True
transport = request.scope.get("transport")
if transport:
transport.close()
return Response(status_code=500)
response = await call_next(request)

View File

@@ -84,6 +84,14 @@ def _migrate_performance_indexes(cursor) -> List[str]:
return added
def _migrate_ban_override_column(cursor) -> bool:
"""Add ban_override column to ip_stats if missing."""
if _column_exists(cursor, "ip_stats", "ban_override"):
return False
cursor.execute("ALTER TABLE ip_stats ADD COLUMN ban_override BOOLEAN DEFAULT NULL")
return True
def run_migrations(database_path: str) -> None:
"""
Check the database schema and apply any pending migrations.
@@ -110,6 +118,9 @@ def run_migrations(database_path: str) -> None:
for col in ban_cols:
applied.append(f"add {col} column to ip_stats")
if _migrate_ban_override_column(cursor):
applied.append("add ban_override column to ip_stats")
idx_added = _migrate_performance_indexes(cursor)
for idx in idx_added:
applied.append(f"add index {idx}")

View File

@@ -210,6 +210,11 @@ class IpStats(Base):
total_violations: Mapped[int] = mapped_column(Integer, default=0, nullable=True)
ban_multiplier: Mapped[int] = mapped_column(Integer, default=1, nullable=True)
# Admin ban override: True = force ban, False = force unban, None = automatic
ban_override: Mapped[Optional[bool]] = mapped_column(
Boolean, nullable=True, default=None
)
def __repr__(self) -> str:
return f"<IpStats(ip='{self.ip}', total_requests={self.total_requests})>"
@@ -239,6 +244,32 @@ class CategoryHistory(Base):
return f"<CategoryHistory(ip='{self.ip}', {self.old_category} -> {self.new_category})>"
class TrackedIp(Base):
"""
Manually tracked IP addresses for monitoring.
Stores a snapshot of essential ip_stats data at tracking time
so the tracked IPs panel never needs to query the large ip_stats table.
"""
__tablename__ = "tracked_ips"
ip: Mapped[str] = mapped_column(String(MAX_IP_LENGTH), primary_key=True)
tracked_since: Mapped[datetime] = mapped_column(
DateTime, nullable=False, default=datetime.utcnow
)
# Snapshot from ip_stats at tracking time
category: Mapped[Optional[str]] = mapped_column(String(50), nullable=True)
total_requests: Mapped[int] = mapped_column(Integer, default=0, nullable=True)
country_code: Mapped[Optional[str]] = mapped_column(String(2), nullable=True)
city: Mapped[Optional[str]] = mapped_column(String(MAX_CITY_LENGTH), nullable=True)
last_seen: Mapped[Optional[datetime]] = mapped_column(DateTime, nullable=True)
def __repr__(self) -> str:
return f"<TrackedIp(ip='{self.ip}')>"
# class IpLog(Base):
# """
# Records all IPs that have accessed the honeypot, along with aggregated stats and inferred user category.

View File

@@ -6,13 +6,28 @@ Migrated from handler.py dashboard API endpoints.
All endpoints are prefixed with the secret dashboard path.
"""
import hashlib
import hmac
import os
import secrets
import time
from fastapi import APIRouter, Request, Response, Query
from fastapi import APIRouter, Request, Response, Query, Cookie
from fastapi.responses import JSONResponse, PlainTextResponse
from pydantic import BaseModel
from dependencies import get_db
from dependencies import get_db, get_client_ip
from logger import get_app_logger
from dashboard_cache import get_cached, is_warm
# Server-side session token store (valid tokens for authenticated sessions)
_auth_tokens: set = set()
# Bruteforce protection: tracks failed attempts per IP
# { ip: { "attempts": int, "locked_until": float } }
_auth_attempts: dict = {}
_AUTH_MAX_ATTEMPTS = 5
_AUTH_BASE_LOCKOUT = 30 # seconds, doubles on each lockout
router = APIRouter()
@@ -26,6 +41,171 @@ def _no_cache_headers() -> dict:
}
class AuthRequest(BaseModel):
fingerprint: str
def verify_auth(request: Request) -> bool:
"""Check if the request has a valid auth session cookie."""
token = request.cookies.get("krawl_auth")
return token is not None and token in _auth_tokens
@router.post("/api/auth")
async def authenticate(request: Request, body: AuthRequest):
ip = get_client_ip(request)
# Check if IP is currently locked out
record = _auth_attempts.get(ip)
if record and record["locked_until"] > time.time():
remaining = int(record["locked_until"] - time.time())
return JSONResponse(
content={
"authenticated": False,
"error": f"Too many attempts. Try again in {remaining}s",
"locked": True,
"retry_after": remaining,
},
status_code=429,
)
config = request.app.state.config
expected = hashlib.sha256(config.dashboard_password.encode()).hexdigest()
if hmac.compare_digest(body.fingerprint, expected):
# Success — clear failed attempts
_auth_attempts.pop(ip, None)
get_app_logger().info(f"[AUTH] Successful login from {ip}")
token = secrets.token_hex(32)
_auth_tokens.add(token)
response = JSONResponse(content={"authenticated": True})
response.set_cookie(
key="krawl_auth",
value=token,
httponly=True,
samesite="strict",
)
return response
# Failed attempt — track and possibly lock out
get_app_logger().warning(f"[AUTH] Failed login attempt from {ip}")
if not record:
record = {"attempts": 0, "locked_until": 0, "lockouts": 0}
_auth_attempts[ip] = record
record["attempts"] += 1
if record["attempts"] >= _AUTH_MAX_ATTEMPTS:
lockout = _AUTH_BASE_LOCKOUT * (2 ** record["lockouts"])
record["locked_until"] = time.time() + lockout
record["lockouts"] += 1
record["attempts"] = 0
get_app_logger().warning(
f"Auth bruteforce: IP {ip} locked out for {lockout}s "
f"(lockout #{record['lockouts']})"
)
return JSONResponse(
content={
"authenticated": False,
"error": f"Too many attempts. Locked for {lockout}s",
"locked": True,
"retry_after": lockout,
},
status_code=429,
)
remaining_attempts = _AUTH_MAX_ATTEMPTS - record["attempts"]
return JSONResponse(
content={
"authenticated": False,
"error": f"Invalid password. {remaining_attempts} attempt{'s' if remaining_attempts != 1 else ''} remaining",
},
status_code=401,
)
@router.post("/api/auth/logout")
async def logout(request: Request):
token = request.cookies.get("krawl_auth")
if token and token in _auth_tokens:
_auth_tokens.discard(token)
response = JSONResponse(content={"authenticated": False})
response.delete_cookie(key="krawl_auth")
return response
@router.get("/api/auth/check")
async def auth_check(request: Request):
"""Check if the current session is authenticated."""
if verify_auth(request):
return JSONResponse(content={"authenticated": True})
return JSONResponse(content={"authenticated": False}, status_code=401)
# ── Protected Ban Management API ─────────────────────────────────────
class BanOverrideRequest(BaseModel):
ip: str
action: str # "ban", "unban", or "reset"
@router.post("/api/ban-override")
async def ban_override(request: Request, body: BanOverrideRequest):
if not verify_auth(request):
return JSONResponse(content={"error": "Unauthorized"}, status_code=401)
db = get_db()
action_map = {"ban": True, "unban": False, "reset": None}
if body.action not in action_map:
return JSONResponse(
content={"error": "Invalid action. Use: ban, unban, reset"},
status_code=400,
)
if body.action == "ban":
success = db.force_ban_ip(body.ip)
else:
success = db.set_ban_override(body.ip, action_map[body.action])
if success:
get_app_logger().info(f"Ban override: {body.action} on IP {body.ip}")
return JSONResponse(
content={"success": True, "ip": body.ip, "action": body.action}
)
return JSONResponse(content={"error": "IP not found"}, status_code=404)
# ── Protected IP Tracking API ────────────────────────────────────────
class TrackIpRequest(BaseModel):
ip: str
action: str # "track" or "untrack"
@router.post("/api/track-ip")
async def track_ip(request: Request, body: TrackIpRequest):
if not verify_auth(request):
return JSONResponse(content={"error": "Unauthorized"}, status_code=401)
db = get_db()
if body.action == "track":
success = db.track_ip(body.ip)
elif body.action == "untrack":
success = db.untrack_ip(body.ip)
else:
return JSONResponse(
content={"error": "Invalid action. Use: track, untrack"},
status_code=400,
)
if success:
get_app_logger().info(f"IP tracking: {body.action} on IP {body.ip}")
return JSONResponse(
content={"success": True, "ip": body.ip, "action": body.action}
)
return JSONResponse(content={"error": "IP not found"}, status_code=404)
@router.get("/api/all-ip-stats")
async def all_ip_stats(request: Request):
db = get_db()
@@ -70,10 +250,22 @@ async def all_ips(
sort_by: str = Query("total_requests"),
sort_order: str = Query("desc"),
):
db = get_db()
page = max(1, page)
page_size = min(max(1, page_size), 10000)
# Serve from cache on default map request (top 100 IPs)
if (
page == 1
and page_size == 100
and sort_by == "total_requests"
and sort_order == "desc"
and is_warm()
):
cached = get_cached("map_ips")
if cached:
return JSONResponse(content=cached, headers=_no_cache_headers())
db = get_db()
try:
result = db.get_all_ips_paginated(
page=page, page_size=page_size, sort_by=sort_by, sort_order=sort_order

View File

@@ -10,6 +10,7 @@ from fastapi.responses import JSONResponse
from logger import get_app_logger
from dependencies import get_db, get_templates
from dashboard_cache import get_cached, is_warm
router = APIRouter()
@@ -17,17 +18,19 @@ router = APIRouter()
@router.get("")
@router.get("/")
async def dashboard_page(request: Request):
db = get_db()
config = request.app.state.config
dashboard_path = "/" + config.dashboard_secret_path.lstrip("/")
# Get initial data for server-rendered sections
stats = db.get_dashboard_counts()
suspicious = db.get_recent_suspicious(limit=10)
# Get credential count for the stats card
cred_result = db.get_credentials_paginated(page=1, page_size=1)
stats["credential_count"] = cred_result["pagination"]["total"]
# Serve from pre-computed cache when available, fall back to live queries
if is_warm():
stats = get_cached("stats")
suspicious = get_cached("suspicious")
else:
db = get_db()
stats = db.get_dashboard_counts()
suspicious = db.get_recent_suspicious(limit=10)
cred_result = db.get_credentials_paginated(page=1, page_size=1)
stats["credential_count"] = cred_result["pagination"]["total"]
templates = get_templates()
return templates.TemplateResponse(

View File

@@ -394,13 +394,6 @@ async def trap_page(request: Request, path: str):
is_suspicious = tracker.is_suspicious_user_agent(user_agent)
if is_suspicious:
access_logger.warning(
f"[SUSPICIOUS] {client_ip} - {user_agent[:50]} - {full_path}"
)
else:
access_logger.info(f"[REQUEST] {client_ip} - {full_path}")
# Record access unless the router dependency already handled it
# (attack pattern or honeypot path → already recorded by _track_honeypot_request)
if not tracker.detect_attack_type(full_path) and not tracker.is_honeypot_path(

View File

@@ -6,8 +6,11 @@ Server-rendered HTML partials for table pagination, sorting, IP details, and sea
"""
from fastapi import APIRouter, Request, Response, Query
from fastapi.responses import HTMLResponse
from dependencies import get_db, get_templates
from routes.api import verify_auth
from dashboard_cache import get_cached, is_warm
router = APIRouter()
@@ -56,10 +59,19 @@ async def htmx_top_ips(
sort_by: str = Query("count"),
sort_order: str = Query("desc"),
):
db = get_db()
result = db.get_top_ips_paginated(
page=max(1, page), page_size=8, sort_by=sort_by, sort_order=sort_order
# Serve from cache on default first-page request
cached = (
get_cached("top_ips")
if (page == 1 and sort_by == "count" and sort_order == "desc" and is_warm())
else None
)
if cached:
result = cached
else:
db = get_db()
result = db.get_top_ips_paginated(
page=max(1, page), page_size=8, sort_by=sort_by, sort_order=sort_order
)
templates = get_templates()
return templates.TemplateResponse(
@@ -85,10 +97,18 @@ async def htmx_top_paths(
sort_by: str = Query("count"),
sort_order: str = Query("desc"),
):
db = get_db()
result = db.get_top_paths_paginated(
page=max(1, page), page_size=5, sort_by=sort_by, sort_order=sort_order
cached = (
get_cached("top_paths")
if (page == 1 and sort_by == "count" and sort_order == "desc" and is_warm())
else None
)
if cached:
result = cached
else:
db = get_db()
result = db.get_top_paths_paginated(
page=max(1, page), page_size=5, sort_by=sort_by, sort_order=sort_order
)
templates = get_templates()
return templates.TemplateResponse(
@@ -114,10 +134,18 @@ async def htmx_top_ua(
sort_by: str = Query("count"),
sort_order: str = Query("desc"),
):
db = get_db()
result = db.get_top_user_agents_paginated(
page=max(1, page), page_size=5, sort_by=sort_by, sort_order=sort_order
cached = (
get_cached("top_ua")
if (page == 1 and sort_by == "count" and sort_order == "desc" and is_warm())
else None
)
if cached:
result = cached
else:
db = get_db()
result = db.get_top_user_agents_paginated(
page=max(1, page), page_size=5, sort_by=sort_by, sort_order=sort_order
)
templates = get_templates()
return templates.TemplateResponse(
@@ -341,6 +369,8 @@ async def htmx_ip_insight(ip_address: str, request: Request):
stats["blocklist_memberships"] = list(list_on.keys()) if list_on else []
stats["reverse_dns"] = stats.get("reverse")
is_tracked = db.is_ip_tracked(ip_address)
templates = get_templates()
return templates.TemplateResponse(
"dashboard/partials/ip_insight.html",
@@ -349,6 +379,7 @@ async def htmx_ip_insight(ip_address: str, request: Request):
"dashboard_path": _dashboard_path(request),
"stats": stats,
"ip_address": ip_address,
"is_tracked": is_tracked,
},
)
@@ -369,6 +400,8 @@ async def htmx_ip_detail(ip_address: str, request: Request):
stats["blocklist_memberships"] = list(list_on.keys()) if list_on else []
stats["reverse_dns"] = stats.get("reverse")
is_tracked = db.is_ip_tracked(ip_address)
templates = get_templates()
return templates.TemplateResponse(
"dashboard/partials/ip_detail.html",
@@ -376,6 +409,7 @@ async def htmx_ip_detail(ip_address: str, request: Request):
"request": request,
"dashboard_path": _dashboard_path(request),
"stats": stats,
"is_tracked": is_tracked,
},
)
@@ -408,3 +442,129 @@ async def htmx_search(
"pagination": result["pagination"],
},
)
# ── Protected Banlist Panel ───────────────────────────────────────────
@router.get("/htmx/banlist")
async def htmx_banlist(request: Request):
if not verify_auth(request):
return HTMLResponse(
'<div class="table-container" style="text-align:center;padding:80px 20px;">'
'<h1 style="color:#f0883e;font-size:48px;margin:20px 0 10px;">Nice try bozo</h1>'
"<br>"
'<img src="https://media0.giphy.com/media/v1.Y2lkPTZjMDliOTUyaHQ3dHRuN2wyOW1kZndjaHdkY2dhYzJ6d2gzMDJkNm53ZnNrdnNlZCZlcD12MV9naWZzX3NlYXJjaCZjdD1n/mOY97EXNisstZqJht9/200w.gif" alt="Diddy">'
"</div>",
status_code=200,
)
templates = get_templates()
return templates.TemplateResponse(
"dashboard/partials/banlist_panel.html",
{
"request": request,
"dashboard_path": _dashboard_path(request),
},
)
# ── Ban Management HTMX Endpoints ───────────────────────────────────
@router.get("/htmx/ban/attackers")
async def htmx_ban_attackers(
request: Request,
page: int = Query(1),
page_size: int = Query(25),
):
if not verify_auth(request):
return HTMLResponse(
"<p style='color:#f85149;'>Unauthorized</p>", status_code=200
)
db = get_db()
result = db.get_attackers_paginated(page=max(1, page), page_size=page_size)
templates = get_templates()
return templates.TemplateResponse(
"dashboard/partials/ban_attackers_table.html",
{
"request": request,
"dashboard_path": _dashboard_path(request),
"items": result["attackers"],
"pagination": result["pagination"],
},
)
# ── Protected Tracked IPs Panel ──────────────────────────────────────
@router.get("/htmx/tracked-ips")
async def htmx_tracked_ips(request: Request):
if not verify_auth(request):
return HTMLResponse(
'<div class="table-container" style="text-align:center;padding:80px 20px;">'
'<h1 style="color:#f0883e;font-size:48px;margin:20px 0 10px;">Nice try bozo</h1>'
"<br>"
'<img src="https://media0.giphy.com/media/v1.Y2lkPTZjMDliOTUyaHQ3dHRuN2wyOW1kZndjaHdkY2dhYzJ6d2gzMDJkNm53ZnNrdnNlZCZlcD12MV9naWZzX3NlYXJjaCZjdD1n/mOY97EXNisstZqJht9/200w.gif" alt="Diddy">'
"</div>",
status_code=200,
)
templates = get_templates()
return templates.TemplateResponse(
"dashboard/partials/tracked_ips_panel.html",
{
"request": request,
"dashboard_path": _dashboard_path(request),
},
)
@router.get("/htmx/tracked-ips/list")
async def htmx_tracked_ips_list(
request: Request,
page: int = Query(1),
page_size: int = Query(25),
):
if not verify_auth(request):
return HTMLResponse(
"<p style='color:#f85149;'>Unauthorized</p>", status_code=200
)
db = get_db()
result = db.get_tracked_ips_paginated(page=max(1, page), page_size=page_size)
templates = get_templates()
return templates.TemplateResponse(
"dashboard/partials/tracked_ips_table.html",
{
"request": request,
"dashboard_path": _dashboard_path(request),
"items": result["tracked_ips"],
"pagination": result["pagination"],
},
)
@router.get("/htmx/ban/overrides")
async def htmx_ban_overrides(
request: Request,
page: int = Query(1),
page_size: int = Query(25),
):
if not verify_auth(request):
return HTMLResponse(
"<p style='color:#f85149;'>Unauthorized</p>", status_code=200
)
db = get_db()
result = db.get_ban_overrides_paginated(page=max(1, page), page_size=page_size)
templates = get_templates()
return templates.TemplateResponse(
"dashboard/partials/ban_overrides_table.html",
{
"request": request,
"dashboard_path": _dashboard_path(request),
"items": result["overrides"],
"pagination": result["pagination"],
},
)

View File

@@ -0,0 +1,68 @@
# tasks/dashboard_warmup.py
"""
Pre-computes all Overview tab data and stores it in the in-memory cache.
This keeps SQLite page buffers warm and lets the dashboard respond instantly.
"""
from logger import get_app_logger
from database import get_database
from dashboard_cache import set_cached
app_logger = get_app_logger()
# ----------------------
# TASK CONFIG
# ----------------------
TASK_CONFIG = {
"name": "dashboard-warmup",
"cron": "*/1 * * * *",
"enabled": True,
"run_when_loaded": True,
}
# ----------------------
# TASK LOGIC
# ----------------------
def main():
"""
Refresh the in-memory dashboard cache with current Overview data.
TasksMaster will call this function based on the cron schedule.
"""
task_name = TASK_CONFIG.get("name")
app_logger.info(f"[Background Task] {task_name} starting...")
try:
db = get_database()
# --- Server-rendered data (stats cards + suspicious table) ---
stats = db.get_dashboard_counts()
cred_result = db.get_credentials_paginated(page=1, page_size=1)
stats["credential_count"] = cred_result["pagination"]["total"]
suspicious = db.get_recent_suspicious(limit=10)
# --- HTMX Overview tables (first page, default sort) ---
top_ips = db.get_top_ips_paginated(page=1, page_size=8)
top_ua = db.get_top_user_agents_paginated(page=1, page_size=5)
top_paths = db.get_top_paths_paginated(page=1, page_size=5)
# --- Map data (default: top 100 IPs by total_requests) ---
map_ips = db.get_all_ips_paginated(
page=1, page_size=100, sort_by="total_requests", sort_order="desc"
)
# Store everything in the cache (overwrites previous values)
set_cached("stats", stats)
set_cached("suspicious", suspicious)
set_cached("top_ips", top_ips)
set_cached("top_ua", top_ua)
set_cached("top_paths", top_paths)
set_cached("map_ips", map_ips)
app_logger.info(f"[Background Task] {task_name} cache refreshed successfully.")
except Exception as e:
app_logger.error(f"[Background Task] {task_name} failed: {e}")

View File

@@ -45,9 +45,25 @@ def main():
session = db.session
# Query attacker IPs from IpStats (same as dashboard "Attackers by Total Requests")
attackers = (
# Also include IPs with ban_override=True (force-banned by admin)
# Exclude IPs with ban_override=False (force-unbanned by admin)
from sqlalchemy import or_, and_
banned_ips = (
session.query(IpStats)
.filter(IpStats.category == "attacker")
.filter(
or_(
# Automatic: attacker category without explicit unban
and_(
IpStats.category == "attacker",
or_(
IpStats.ban_override.is_(None), IpStats.ban_override == True
),
),
# Manual: force-banned by admin regardless of category
IpStats.ban_override == True,
)
)
.order_by(IpStats.total_requests.desc())
.all()
)
@@ -56,9 +72,7 @@ def main():
server_ip = config.get_server_ip()
public_ips = [
attacker.ip
for attacker in attackers
if is_valid_public_ip(attacker.ip, server_ip)
entry.ip for entry in banned_ips if is_valid_public_ip(entry.ip, server_ip)
]
# Ensure exports directory exists
@@ -81,7 +95,7 @@ def main():
app_logger.info(
f"[Background Task] {task_name} exported {len(public_ips)} in {fwname} public IPs"
f"(filtered {len(attackers) - len(public_ips)} local/private IPs) to {output_file}"
f"(filtered {len(banned_ips) - len(public_ips)} local/private IPs) to {output_file}"
)
except Exception as e:

View File

@@ -129,7 +129,7 @@
</head>
<body>
<div class="container">
<h1>Admin Panel</h1>
<h1>Krawl Login</h1>
<p class="subtitle">Please log in to continue</p>
<form action="/admin/login" method="post">

View File

@@ -8,6 +8,7 @@
<link rel="stylesheet" href="{{ dashboard_path }}/static/vendor/css/leaflet.min.css" />
<link rel="stylesheet" href="{{ dashboard_path }}/static/vendor/css/MarkerCluster.css" />
<link rel="stylesheet" href="{{ dashboard_path }}/static/vendor/css/MarkerCluster.Default.css" />
<link rel="stylesheet" href="{{ dashboard_path }}/static/vendor/css/material-symbols.css" />
<link rel="stylesheet" href="{{ dashboard_path }}/static/css/dashboard.css" />
<script src="{{ dashboard_path }}/static/vendor/js/leaflet.min.js" defer></script>
<script src="{{ dashboard_path }}/static/vendor/js/leaflet.markercluster.js" defer></script>

View File

@@ -59,6 +59,20 @@
<a class="tab-button" :class="{ active: tab === 'ip-insight', disabled: !insightIp }" @click.prevent="insightIp && switchToIpInsight()" href="#ip-insight">
IP Insight<span x-show="insightIp" x-text="' (' + insightIp + ')'"></span>
</a>
<a class="tab-button tab-right" :class="{ active: tab === 'tracked-ips' }" x-show="authenticated" x-cloak @click.prevent="switchToTrackedIps()" href="#tracked-ips">Tracked IPs</a>
<a class="tab-button" :class="{ active: tab === 'banlist' }" x-show="authenticated" x-cloak @click.prevent="switchToBanlist()" href="#banlist">IP Banlist</a>
{# Lock icon (not authenticated) #}
<a class="tab-button tab-lock-btn" :class="{ 'tab-right': !authenticated }" @click.prevent="promptAuth()" x-show="!authenticated" href="#" title="Unlock protected panels">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 16 16" width="16" height="16" fill="currentColor">
<path d="M4 4a4 4 0 0 1 8 0v2h.25c.966 0 1.75.784 1.75 1.75v5.5A1.75 1.75 0 0 1 12.25 15h-8.5A1.75 1.75 0 0 1 2 13.25v-5.5C2 6.784 2.784 6 3.75 6H4Zm8.25 3.5h-8.5a.25.25 0 0 0-.25.25v5.5c0 .138.112.25.25.25h8.5a.25.25 0 0 0 .25-.25v-5.5a.25.25 0 0 0-.25-.25ZM10.5 6V4a2.5 2.5 0 1 0-5 0v2Z"/>
</svg>
</a>
{# Logout icon (authenticated) #}
<a class="tab-button tab-lock-btn" @click.prevent="logout()" x-show="authenticated" x-cloak href="#" title="Logout">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 16 16" width="16" height="16" fill="currentColor">
<path d="M2 2.75C2 1.784 2.784 1 3.75 1h2.5a.75.75 0 0 1 0 1.5h-2.5a.25.25 0 0 0-.25.25v10.5c0 .138.112.25.25.25h2.5a.75.75 0 0 1 0 1.5h-2.5A1.75 1.75 0 0 1 2 13.25Zm10.44 4.5-1.97-1.97a.749.749 0 0 1 .326-1.275.749.749 0 0 1 .734.215l3.25 3.25a.75.75 0 0 1 0 1.06l-3.25 3.25a.749.749 0 0 1-1.275-.326.749.749 0 0 1 .215-.734l1.97-1.97H6.75a.75.75 0 0 1 0-1.5Z"/>
</svg>
</a>
</div>
{# ==================== OVERVIEW TAB ==================== #}
@@ -184,8 +198,21 @@
</div>
</div>
{# ==================== TRACKED IPS TAB (protected, loaded via HTMX with server-side auth) ==================== #}
<div x-show="tab === 'tracked-ips'" x-cloak>
<div id="tracked-ips-htmx-container"></div>
</div>
{# ==================== IP BANLIST TAB (protected, loaded via HTMX with server-side auth) ==================== #}
<div x-show="tab === 'banlist'" x-cloak>
<div id="banlist-htmx-container"></div>
</div>
{# Raw request modal - Alpine.js #}
{% include "dashboard/partials/raw_request_modal.html" %}
{# Auth modal - Alpine.js #}
{% include "dashboard/partials/auth_modal.html" %}
</div>
{% endblock %}

View File

@@ -3,14 +3,41 @@
{# Page header #}
<div class="ip-page-header">
<h1>
<span class="ip-address-title">{{ ip_address }}</span>
{% if stats.category %}
<span class="category-badge category-{{ stats.category | lower | replace('_', '-') }}">
{{ stats.category | replace('_', ' ') | title }}
<div class="ip-page-header-row">
<h1>
<span class="ip-address-title">{{ ip_address }}</span>
{% if stats.category %}
<span class="category-badge category-{{ stats.category | lower | replace('_', '-') }}">
{{ stats.category | replace('_', ' ') | title }}
</span>
{% endif %}
</h1>
{# Ban/Unban + Track/Untrack actions — visible only when authenticated #}
<span class="ip-ban-actions" style="display: none; gap: 4px;">
{% if stats.category and stats.category | lower == 'attacker' %}
<button class="ban-icon-btn ban-icon-unban" onclick="ipBanAction('{{ ip_address | e }}', 'unban')" title="Unban">
<span class="material-symbols-outlined">health_and_safety</span>
<span class="ban-icon-tooltip">Unban</span>
</button>
{% else %}
<button class="ban-icon-btn ban-icon-ban" onclick="ipBanAction('{{ ip_address | e }}', 'ban')" title="Add to banlist">
<span class="material-symbols-outlined">gavel</span>
<span class="ban-icon-tooltip">Add to banlist</span>
</button>
{% endif %}
{% if is_tracked %}
<button class="ban-icon-btn track-icon-untrack" onclick="ipTrackAction('{{ ip_address | e }}', 'untrack')" title="Untrack IP">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24" fill="currentColor"><path d="M12 7c2.76 0 5 2.24 5 5 0 .65-.13 1.26-.36 1.83l2.92 2.92c1.51-1.26 2.7-2.89 3.43-4.75-1.73-4.39-6-7.5-11-7.5-1.4 0-2.74.25-3.98.7l2.16 2.16C10.74 7.13 11.35 7 12 7zM2 4.27l2.28 2.28.46.46C3.08 8.3 1.78 10.02 1 12c1.73 4.39 6 7.5 11 7.5 1.55 0 3.03-.3 4.38-.84l.42.42L19.73 22 21 20.73 3.27 3 2 4.27zM7.53 9.8l1.55 1.55c-.05.21-.08.43-.08.65 0 1.66 1.34 3 3 3 .22 0 .44-.03.65-.08l1.55 1.55c-.67.33-1.41.53-2.2.53-2.76 0-5-2.24-5-5 0-.79.2-1.53.53-2.2zm4.31-.78l3.15 3.15.02-.16c0-1.66-1.34-3-3-3l-.17.01z"/></svg>
<span class="ban-icon-tooltip">Untrack</span>
</button>
{% else %}
<button class="ban-icon-btn track-icon-track" onclick="ipTrackAction('{{ ip_address | e }}', 'track')" title="Track IP">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24" fill="currentColor"><path d="M12 4.5C7 4.5 2.73 7.61 1 12c1.73 4.39 6 7.5 11 7.5s9.27-3.11 11-7.5c-1.73-4.39-6-7.5-11-7.5zM12 17c-2.76 0-5-2.24-5-5s2.24-5 5-5 5 2.24 5 5-2.24 5-5 5zm0-8c-1.66 0-3 1.34-3 3s1.34 3 3 3 3-1.34 3-3-1.34-3-3-3z"/></svg>
<span class="ban-icon-tooltip">Track</span>
</button>
{% endif %}
</span>
{% endif %}
</h1>
</div>
{% if stats.city or stats.country %}
<p class="ip-location-subtitle">
{{ stats.city | default('') }}{% if stats.city and stats.country %}, {% endif %}{{ stats.country | default(stats.country_code | default('')) }}

View File

@@ -0,0 +1,39 @@
{# Authentication modal - Alpine.js controlled #}
<div class="auth-modal"
x-show="authModal.show"
x-cloak
@click.self="closeAuthModal()"
@keydown.escape.window="authModal.show && closeAuthModal()"
>
<div class="auth-modal-content">
<div class="auth-modal-header">
<div class="auth-modal-title">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 16 16" width="20" height="20" fill="currentColor">
<path d="M4 4a4 4 0 0 1 8 0v2h.25c.966 0 1.75.784 1.75 1.75v5.5A1.75 1.75 0 0 1 12.25 15h-8.5A1.75 1.75 0 0 1 2 13.25v-5.5C2 6.784 2.784 6 3.75 6H4Zm8.25 3.5h-8.5a.25.25 0 0 0-.25.25v5.5c0 .138.112.25.25.25h8.5a.25.25 0 0 0 .25-.25v-5.5a.25.25 0 0 0-.25-.25ZM10.5 6V4a2.5 2.5 0 1 0-5 0v2Z"/>
</svg>
<h3>Authentication Required</h3>
</div>
<span class="auth-modal-close" @click="closeAuthModal()">&times;</span>
</div>
<form class="auth-modal-body" @submit.prevent="submitAuth()">
<p class="auth-modal-description">Enter the dashboard password to access protected panels.</p>
<div class="auth-modal-input-group">
<input type="password"
class="auth-modal-input"
:class="{ 'auth-modal-input-error': authModal.error }"
x-model="authModal.password"
x-ref="authPasswordInput"
placeholder="Password"
autocomplete="off" />
<p class="auth-modal-error" x-show="authModal.error" x-text="authModal.error" x-cloak></p>
</div>
<div class="auth-modal-footer">
<button type="button" class="auth-modal-btn auth-modal-btn-cancel" @click="closeAuthModal()">Cancel</button>
<button type="submit" class="auth-modal-btn auth-modal-btn-submit" :disabled="authModal.loading">
<span x-show="!authModal.loading">Unlock</span>
<span x-show="authModal.loading" x-cloak>Verifying...</span>
</button>
</div>
</form>
</div>
</div>

View File

@@ -0,0 +1,49 @@
{# HTMX fragment: Attackers with unban action #}
<div style="display: flex; justify-content: space-between; align-items: center; margin-bottom: 10px;">
<span class="pagination-info">Page {{ pagination.page }}/{{ pagination.total_pages }} &mdash; {{ pagination.total }} attackers</span>
<div style="display: flex; gap: 8px;">
<button class="pagination-btn"
hx-get="{{ dashboard_path }}/htmx/ban/attackers?page={{ pagination.page - 1 }}"
hx-target="closest .htmx-container"
hx-swap="innerHTML"
{% if pagination.page <= 1 %}disabled{% endif %}>Prev</button>
<button class="pagination-btn"
hx-get="{{ dashboard_path }}/htmx/ban/attackers?page={{ pagination.page + 1 }}"
hx-target="closest .htmx-container"
hx-swap="innerHTML"
{% if pagination.page >= pagination.total_pages %}disabled{% endif %}>Next</button>
</div>
</div>
<table class="ip-stats-table">
<thead>
<tr>
<th>#</th>
<th>IP Address</th>
<th>Total Requests</th>
<th>Category</th>
<th>Location</th>
<th>Last Seen</th>
<th style="width: 40px;"></th>
</tr>
</thead>
<tbody>
{% for ip in items %}
<tr>
<td class="rank">{{ loop.index + (pagination.page - 1) * pagination.page_size }}</td>
<td>{{ ip.ip | e }}</td>
<td>{{ ip.total_requests }}</td>
<td><span class="category-badge category-{{ ip.category | default('unknown') | replace('_', '-') }}">{{ ip.category | default('unknown') | replace('_', ' ') | title }}</span></td>
<td>{{ ip.city | default('') | e }}{% if ip.city and ip.country_code %}, {% endif %}{{ ip.country_code | default('N/A') | e }}</td>
<td>{{ ip.last_seen | format_ts }}</td>
<td>
<button class="ban-icon-btn ban-icon-unban" onclick="ipBanAction('{{ ip.ip | e }}', 'unban')" title="Unban">
<span class="material-symbols-outlined">health_and_safety</span>
<span class="ban-icon-tooltip">Unban</span>
</button>
</td>
</tr>
{% else %}
<tr><td colspan="7" class="empty-state">No attackers found</td></tr>
{% endfor %}
</tbody>
</table>

View File

@@ -0,0 +1,57 @@
{# HTMX fragment: Active ban overrides #}
<div style="display: flex; justify-content: space-between; align-items: center; margin-bottom: 10px;">
<span class="pagination-info">Page {{ pagination.page }}/{{ pagination.total_pages }} &mdash; {{ pagination.total }} overrides</span>
<div style="display: flex; gap: 8px;">
<button class="pagination-btn"
hx-get="{{ dashboard_path }}/htmx/ban/overrides?page={{ pagination.page - 1 }}"
hx-target="closest .htmx-container"
hx-swap="innerHTML"
{% if pagination.page <= 1 %}disabled{% endif %}>Prev</button>
<button class="pagination-btn"
hx-get="{{ dashboard_path }}/htmx/ban/overrides?page={{ pagination.page + 1 }}"
hx-target="closest .htmx-container"
hx-swap="innerHTML"
{% if pagination.page >= pagination.total_pages %}disabled{% endif %}>Next</button>
</div>
</div>
<table class="ip-stats-table">
<thead>
<tr>
<th>#</th>
<th>IP Address</th>
<th>Override</th>
<th>Category</th>
<th>Total Requests</th>
<th>Location</th>
<th>Last Seen</th>
<th style="width: 40px;"></th>
</tr>
</thead>
<tbody>
{% for ip in items %}
<tr>
<td class="rank">{{ loop.index + (pagination.page - 1) * pagination.page_size }}</td>
<td>{{ ip.ip | e }}</td>
<td>
{% if ip.ban_override == true %}
<span class="ban-override-badge ban-override-banned">Force Banned</span>
{% elif ip.ban_override == false %}
<span class="ban-override-badge ban-override-unbanned">Force Unbanned</span>
{% endif %}
</td>
<td><span class="category-badge category-{{ ip.category | default('unknown') | replace('_', '-') }}">{{ ip.category | default('unknown') | replace('_', ' ') | title }}</span></td>
<td>{{ ip.total_requests }}</td>
<td>{{ ip.city | default('') | e }}{% if ip.city and ip.country_code %}, {% endif %}{{ ip.country_code | default('N/A') | e }}</td>
<td>{{ ip.last_seen | format_ts }}</td>
<td>
<button class="ban-icon-btn ban-icon-reset" onclick="ipBanAction('{{ ip.ip | e }}', 'reset')" title="Reset to automatic">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 16 16" width="16" height="16" fill="currentColor"><path d="M1.705 8.005a.75.75 0 0 1 .834.656 5.5 5.5 0 0 0 9.592 2.97l-1.204-1.204a.25.25 0 0 1 .177-.427h3.646a.25.25 0 0 1 .25.25v3.646a.25.25 0 0 1-.427.177l-1.38-1.38A7.002 7.002 0 0 1 1.05 8.84a.75.75 0 0 1 .656-.834ZM8 2.5a5.487 5.487 0 0 0-4.131 1.869l1.204 1.204A.25.25 0 0 1 4.896 6H1.25A.25.25 0 0 1 1 5.75V2.104a.25.25 0 0 1 .427-.177l1.38 1.38A7.002 7.002 0 0 1 14.95 7.16a.75.75 0 0 1-1.49.178A5.5 5.5 0 0 0 8 2.5Z"/></svg>
<span class="ban-icon-tooltip">Reset</span>
</button>
</td>
</tr>
{% else %}
<tr><td colspan="8" class="empty-state">No active overrides</td></tr>
{% endfor %}
</tbody>
</table>

View File

@@ -0,0 +1,101 @@
{# Ban management panel #}
<div x-data="banManagement()" x-init="init()">
{# Force ban IP form #}
<div class="table-container" style="margin-bottom: 20px;">
<h2>IP Banlist</h2>
<p style="color: #8b949e; font-size: 14px; margin-bottom: 16px;">
Force-ban a new IP or manage existing ban overrides. Changes take effect on the next banlist export cycle (every 5 minutes).
</p>
<form @submit.prevent="forceBan()" style="display: flex; gap: 10px; align-items: flex-end; flex-wrap: wrap;">
<div style="flex: 1; min-width: 200px;">
<label style="display: block; color: #8b949e; font-size: 12px; margin-bottom: 4px;">IP Address</label>
<input type="text"
x-model="newBanIp"
placeholder="e.g. 192.168.1.100"
class="auth-modal-input"
style="width: 100%;" />
</div>
<button type="submit" class="ban-form-btn" :disabled="!newBanIp || banLoading">
<span class="material-symbols-outlined">gavel</span>
<span x-text="banLoading ? 'Banning...' : 'Force Ban IP'"></span>
</button>
</form>
<p x-show="banMessage" x-text="banMessage" :style="{ color: banSuccess ? '#3fb950' : '#f85149' }" style="margin-top: 10px; font-size: 13px;" x-cloak></p>
</div>
{# Attackers list with unban option #}
<div class="table-container" style="margin-bottom: 20px;">
<h2>Detected Attackers</h2>
<div class="htmx-container"
hx-get="{{ dashboard_path }}/htmx/ban/attackers?page=1"
hx-trigger="load"
hx-swap="innerHTML">
<div class="htmx-indicator">Loading...</div>
</div>
</div>
{# Active overrides #}
<div class="table-container">
<h2>Active Ban Overrides</h2>
<div id="overrides-container"
class="htmx-container"
hx-get="{{ dashboard_path }}/htmx/ban/overrides?page=1"
hx-trigger="load"
hx-swap="innerHTML">
<div class="htmx-indicator">Loading...</div>
</div>
</div>
</div>
<script>
document.addEventListener('alpine:init', () => {
Alpine.data('banManagement', () => ({
newBanIp: '',
banLoading: false,
banMessage: '',
banSuccess: false,
init() {},
async forceBan() {
if (!this.newBanIp) return;
this.banLoading = true;
this.banMessage = '';
try {
const resp = await fetch(`${window.__DASHBOARD_PATH__}/api/ban-override`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
credentials: 'same-origin',
body: JSON.stringify({ ip: this.newBanIp, action: 'ban' }),
});
const data = await resp.json();
if (resp.ok) {
this.banSuccess = true;
this.banMessage = `IP ${this.newBanIp} added to banlist`;
this.newBanIp = '';
this.refreshOverrides();
} else {
this.banSuccess = false;
this.banMessage = data.error || 'Failed to ban IP';
}
} catch {
this.banSuccess = false;
this.banMessage = 'Request failed';
}
this.banLoading = false;
},
refreshOverrides() {
const container = document.getElementById('overrides-container');
if (container && typeof htmx !== 'undefined') {
htmx.ajax('GET', `${window.__DASHBOARD_PATH__}/htmx/ban/overrides?page=1`, {
target: '#overrides-container',
swap: 'innerHTML'
});
}
},
}));
});
</script>

View File

@@ -0,0 +1,89 @@
{# IP Tracking management panel #}
<div x-data="trackManagement()" x-init="init()">
{# Track IP form #}
<div class="table-container" style="margin-bottom: 20px;">
<h2>Tracked IPs</h2>
<p style="color: #8b949e; font-size: 14px; margin-bottom: 16px;">
Track an IP address to monitor its activity. You can also track IPs from the IP Insight page.
</p>
<form @submit.prevent="trackIp()" style="display: flex; gap: 10px; align-items: flex-end; flex-wrap: wrap;">
<div style="flex: 1; min-width: 200px;">
<label style="display: block; color: #8b949e; font-size: 12px; margin-bottom: 4px;">IP Address</label>
<input type="text"
x-model="newTrackIp"
placeholder="e.g. 192.168.1.100"
class="auth-modal-input"
style="width: 100%;" />
</div>
<button type="submit" class="track-form-btn" :disabled="!newTrackIp || trackLoading">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="18" height="18" fill="currentColor"><path d="M12 4.5C7 4.5 2.73 7.61 1 12c1.73 4.39 6 7.5 11 7.5s9.27-3.11 11-7.5c-1.73-4.39-6-7.5-11-7.5zM12 17c-2.76 0-5-2.24-5-5s2.24-5 5-5 5 2.24 5 5-2.24 5-5 5zm0-8c-1.66 0-3 1.34-3 3s1.34 3 3 3 3-1.34 3-3-1.34-3-3-3z"/></svg>
<span x-text="trackLoading ? 'Tracking...' : 'Track IP'"></span>
</button>
</form>
<p x-show="trackMessage" x-text="trackMessage" :style="{ color: trackSuccess ? '#3fb950' : '#f85149' }" style="margin-top: 10px; font-size: 13px;" x-cloak></p>
</div>
{# Tracked IPs list #}
<div class="table-container">
<h2>Currently Tracked</h2>
<div id="tracked-ips-container"
class="htmx-container"
hx-get="{{ dashboard_path }}/htmx/tracked-ips/list?page=1"
hx-trigger="load"
hx-swap="innerHTML">
<div class="htmx-indicator">Loading...</div>
</div>
</div>
</div>
<script>
document.addEventListener('alpine:init', () => {
Alpine.data('trackManagement', () => ({
newTrackIp: '',
trackLoading: false,
trackMessage: '',
trackSuccess: false,
init() {},
async trackIp() {
if (!this.newTrackIp) return;
this.trackLoading = true;
this.trackMessage = '';
try {
const resp = await fetch(`${window.__DASHBOARD_PATH__}/api/track-ip`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
credentials: 'same-origin',
body: JSON.stringify({ ip: this.newTrackIp, action: 'track' }),
});
const data = await resp.json();
if (resp.ok) {
this.trackSuccess = true;
this.trackMessage = `IP ${this.newTrackIp} is now being tracked`;
this.newTrackIp = '';
this.refreshList();
} else {
this.trackSuccess = false;
this.trackMessage = data.error || 'Failed to track IP';
}
} catch {
this.trackSuccess = false;
this.trackMessage = 'Request failed';
}
this.trackLoading = false;
},
refreshList() {
const container = document.getElementById('tracked-ips-container');
if (container && typeof htmx !== 'undefined') {
htmx.ajax('GET', `${window.__DASHBOARD_PATH__}/htmx/tracked-ips/list?page=1`, {
target: '#tracked-ips-container',
swap: 'innerHTML'
});
}
},
}));
});
</script>

View File

@@ -0,0 +1,55 @@
{# HTMX fragment: Tracked IPs list #}
<div style="display: flex; justify-content: space-between; align-items: center; margin-bottom: 10px;">
<span class="pagination-info">Page {{ pagination.page }}/{{ pagination.total_pages }} &mdash; {{ pagination.total }} tracked</span>
<div style="display: flex; gap: 8px;">
<button class="pagination-btn"
hx-get="{{ dashboard_path }}/htmx/tracked-ips/list?page={{ pagination.page - 1 }}"
hx-target="closest .htmx-container"
hx-swap="innerHTML"
{% if pagination.page <= 1 %}disabled{% endif %}>Prev</button>
<button class="pagination-btn"
hx-get="{{ dashboard_path }}/htmx/tracked-ips/list?page={{ pagination.page + 1 }}"
hx-target="closest .htmx-container"
hx-swap="innerHTML"
{% if pagination.page >= pagination.total_pages %}disabled{% endif %}>Next</button>
</div>
</div>
<table class="ip-stats-table">
<thead>
<tr>
<th>#</th>
<th>IP Address</th>
<th>Category</th>
<th>Total Requests</th>
<th>Location</th>
<th>Last Seen</th>
<th>Tracked Since</th>
<th style="width: 80px;"></th>
</tr>
</thead>
<tbody>
{% for ip in items %}
<tr>
<td class="rank">{{ loop.index + (pagination.page - 1) * pagination.page_size }}</td>
<td>{{ ip.ip | e }}</td>
<td><span class="category-badge category-{{ ip.category | default('unknown') | replace('_', '-') }}">{{ ip.category | default('unknown') | replace('_', ' ') | title }}</span></td>
<td>{{ ip.total_requests }}</td>
<td>{{ ip.city | default('') | e }}{% if ip.city and ip.country_code %}, {% endif %}{{ ip.country_code | default('N/A') | e }}</td>
<td>{{ ip.last_seen | format_ts }}</td>
<td>{{ ip.tracked_since | format_ts }}</td>
<td style="display: flex; gap: 4px;">
<button class="ban-icon-btn track-icon-inspect" onclick="openIpInsight('{{ ip.ip | e }}')" title="Inspect">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 16 16" width="16" height="16" fill="currentColor"><path d="M10.68 11.74a6 6 0 0 1-7.922-8.982 6 6 0 0 1 8.982 7.922l3.04 3.04a.749.749 0 0 1-.326 1.275.749.749 0 0 1-.734-.215ZM11.5 7a4.499 4.499 0 1 0-8.997 0A4.499 4.499 0 0 0 11.5 7Z"/></svg>
<span class="ban-icon-tooltip">Inspect</span>
</button>
<button class="ban-icon-btn track-icon-untrack" onclick="ipTrackAction('{{ ip.ip | e }}', 'untrack')" title="Untrack">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="18" height="18" fill="currentColor"><path d="M12 7c2.76 0 5 2.24 5 5 0 .65-.13 1.26-.36 1.83l2.92 2.92c1.51-1.26 2.7-2.89 3.43-4.75-1.73-4.39-6-7.5-11-7.5-1.4 0-2.74.25-3.98.7l2.16 2.16C10.74 7.13 11.35 7 12 7zM2 4.27l2.28 2.28.46.46C3.08 8.3 1.78 10.02 1 12c1.73 4.39 6 7.5 11 7.5 1.55 0 3.03-.3 4.38-.84l.42.42L19.73 22 21 20.73 3.27 3 2 4.27zM7.53 9.8l1.55 1.55c-.05.21-.08.43-.08.65 0 1.66 1.34 3 3 3 .22 0 .44-.03.65-.08l1.55 1.55c-.67.33-1.41.53-2.2.53-2.76 0-5-2.24-5-5 0-.79.2-1.53.53-2.2zm4.31-.78l3.15 3.15.02-.16c0-1.66-1.34-3-3-3l-.17.01z"/></svg>
<span class="ban-icon-tooltip">Untrack</span>
</button>
</td>
</tr>
{% else %}
<tr><td colspan="8" class="empty-state">No tracked IPs</td></tr>
{% endfor %}
</tbody>
</table>

View File

@@ -454,6 +454,12 @@ tbody {
.ip-page-header {
margin-bottom: 20px;
}
.ip-page-header-row {
display: flex;
align-items: center;
justify-content: space-between;
gap: 12px;
}
.ip-page-header h1 {
display: flex;
align-items: center;
@@ -710,9 +716,8 @@ tbody {
min-width: 0;
}
.ip-attack-chart-wrapper {
flex: 1;
position: relative;
min-height: 180px;
height: 220px;
}
/* Radar chart */
@@ -854,6 +859,23 @@ tbody {
color: #484f58;
background: transparent;
}
.tab-right {
margin-left: auto;
}
.tab-lock-btn {
display: flex;
align-items: center;
padding: 12px 16px;
color: #8b949e;
}
.tab-lock-btn:hover {
color: #f0883e;
background: #1c2128;
}
.tab-lock-btn svg {
width: 16px;
height: 16px;
}
.tab-content {
display: none;
}
@@ -1151,6 +1173,344 @@ tbody {
opacity: 1;
}
/* Auth Modal */
.auth-modal {
position: fixed;
z-index: 1000;
left: 0;
top: 0;
width: 100%;
height: 100%;
background-color: rgba(0, 0, 0, 0.7);
display: flex;
align-items: center;
justify-content: center;
backdrop-filter: blur(4px);
}
.auth-modal-content {
background-color: #161b22;
border: 1px solid #30363d;
border-radius: 12px;
width: 400px;
max-width: 90vw;
box-shadow: 0 16px 48px rgba(0, 0, 0, 0.6);
animation: authModalIn 0.2s ease-out;
}
@keyframes authModalIn {
from { opacity: 0; transform: scale(0.95) translateY(-10px); }
to { opacity: 1; transform: scale(1) translateY(0); }
}
.auth-modal-header {
padding: 20px 24px 16px;
display: flex;
justify-content: space-between;
align-items: center;
border-bottom: 1px solid #30363d;
}
.auth-modal-title {
display: flex;
align-items: center;
gap: 10px;
color: #58a6ff;
}
.auth-modal-title h3 {
margin: 0;
font-size: 16px;
font-weight: 600;
color: #e6edf3;
}
.auth-modal-close {
color: #8b949e;
font-size: 24px;
font-weight: bold;
cursor: pointer;
line-height: 20px;
transition: color 0.2s;
}
.auth-modal-close:hover {
color: #c9d1d9;
}
.auth-modal-body {
padding: 24px;
}
.auth-modal-description {
margin: 0 0 20px;
color: #8b949e;
font-size: 14px;
line-height: 1.5;
}
.auth-modal-input-group {
margin-bottom: 20px;
}
.auth-modal-input {
width: 100%;
padding: 10px 14px;
background: #0d1117;
border: 1px solid #30363d;
border-radius: 6px;
color: #e6edf3;
font-size: 14px;
outline: none;
transition: border-color 0.2s, box-shadow 0.2s;
box-sizing: border-box;
}
.auth-modal-input:focus {
border-color: #58a6ff;
box-shadow: 0 0 0 3px rgba(88, 166, 255, 0.15);
}
.auth-modal-input-error {
border-color: #f85149;
}
.auth-modal-input-error:focus {
border-color: #f85149;
box-shadow: 0 0 0 3px rgba(248, 81, 73, 0.15);
}
.auth-modal-error {
margin: 8px 0 0;
color: #f85149;
font-size: 13px;
}
.auth-modal-footer {
display: flex;
justify-content: flex-end;
gap: 10px;
}
.auth-modal-btn {
padding: 8px 20px;
border-radius: 6px;
font-size: 14px;
font-weight: 500;
cursor: pointer;
border: 1px solid transparent;
transition: all 0.2s;
}
.auth-modal-btn-cancel {
background: transparent;
border-color: #30363d;
color: #8b949e;
}
.auth-modal-btn-cancel:hover {
background: #21262d;
color: #c9d1d9;
}
.auth-modal-btn-submit {
background: #238636;
color: #fff;
border-color: rgba(240, 246, 252, 0.1);
}
.auth-modal-btn-submit:hover {
background: #2ea043;
}
.auth-modal-btn-submit:disabled {
opacity: 0.6;
cursor: not-allowed;
}
/* Ban Management */
.ban-form-btn {
display: inline-flex;
align-items: center;
gap: 6px;
padding: 10px 18px;
background: rgba(248, 81, 73, 0.15);
color: #f85149;
border: 1px solid rgba(248, 81, 73, 0.3);
border-radius: 6px;
font-size: 14px;
font-weight: 500;
cursor: pointer;
transition: all 0.2s;
white-space: nowrap;
}
.ban-form-btn:hover:not(:disabled) {
background: rgba(248, 81, 73, 0.3);
border-color: rgba(248, 81, 73, 0.5);
}
.ban-form-btn:disabled {
opacity: 0.4;
cursor: not-allowed;
}
.ban-form-btn .material-symbols-outlined {
font-size: 18px;
}
.ban-icon-btn {
position: relative;
display: inline-flex;
align-items: center;
justify-content: center;
padding: 6px;
background: none;
border: none;
border-radius: 4px;
cursor: pointer;
transition: color 0.2s, background 0.2s;
}
.ban-icon-btn svg {
width: 24px;
height: 24px;
fill: currentColor;
}
.ban-icon-btn .material-symbols-outlined {
font-size: 26px;
}
.ban-icon-unban {
color: #3fb950;
}
.ban-icon-unban:hover {
background: rgba(63, 185, 80, 0.15);
}
.ban-icon-reset {
color: #8b949e;
}
.ban-icon-reset:hover {
color: #c9d1d9;
background: rgba(139, 148, 158, 0.15);
}
.ban-icon-ban {
color: #f85149;
}
.ban-icon-ban:hover {
background: rgba(248, 81, 73, 0.15);
}
.ban-icon-tooltip {
position: absolute;
bottom: calc(100% + 6px);
left: 50%;
transform: translateX(-50%);
padding: 4px 8px;
background: #1c2128;
color: #e6edf3;
border: 1px solid #30363d;
border-radius: 4px;
font-size: 11px;
white-space: nowrap;
opacity: 0;
pointer-events: none;
transition: opacity 0.15s;
}
.ban-icon-btn:hover .ban-icon-tooltip {
opacity: 1;
}
/* IP Tracking buttons */
.track-form-btn {
display: inline-flex;
align-items: center;
gap: 6px;
padding: 10px 18px;
background: rgba(88, 166, 255, 0.15);
color: #58a6ff;
border: 1px solid rgba(88, 166, 255, 0.3);
border-radius: 6px;
font-size: 14px;
font-weight: 500;
cursor: pointer;
transition: all 0.2s;
white-space: nowrap;
}
.track-form-btn:hover:not(:disabled) {
background: rgba(88, 166, 255, 0.3);
border-color: rgba(88, 166, 255, 0.5);
}
.track-form-btn:disabled {
opacity: 0.4;
cursor: not-allowed;
}
.track-form-btn .material-symbols-outlined {
font-size: 18px;
}
.track-icon-track {
color: #58a6ff;
}
.track-icon-track:hover {
background: rgba(88, 166, 255, 0.15);
}
.track-icon-untrack {
color: #8b949e;
}
.track-icon-untrack:hover {
color: #c9d1d9;
background: rgba(139, 148, 158, 0.15);
}
.track-icon-inspect {
color: #d2a8ff;
}
.track-icon-inspect:hover {
background: rgba(210, 168, 255, 0.15);
}
/* Custom confirm/alert modal */
.krawl-modal-overlay {
position: fixed;
z-index: 1100;
left: 0;
top: 0;
width: 100%;
height: 100%;
background-color: rgba(0, 0, 0, 0.7);
display: flex;
align-items: center;
justify-content: center;
backdrop-filter: blur(4px);
}
.krawl-modal-box {
background-color: #161b22;
border: 1px solid #30363d;
border-radius: 12px;
width: 380px;
max-width: 90vw;
box-shadow: 0 16px 48px rgba(0, 0, 0, 0.6);
animation: authModalIn 0.2s ease-out;
overflow: hidden;
}
.krawl-modal-icon {
display: flex;
align-items: center;
justify-content: center;
padding: 24px 24px 0;
}
.krawl-modal-icon .material-symbols-outlined {
font-size: 40px;
}
.krawl-modal-icon.krawl-modal-icon-warn .material-symbols-outlined {
color: #d29922;
}
.krawl-modal-icon.krawl-modal-icon-success .material-symbols-outlined {
color: #3fb950;
}
.krawl-modal-icon.krawl-modal-icon-error .material-symbols-outlined {
color: #f85149;
}
.krawl-modal-message {
padding: 16px 24px 8px;
text-align: center;
color: #e6edf3;
font-size: 15px;
line-height: 1.5;
}
.krawl-modal-actions {
display: flex;
justify-content: center;
gap: 10px;
padding: 16px 24px 24px;
}
.ban-override-badge {
padding: 2px 8px;
border-radius: 12px;
font-size: 11px;
font-weight: 600;
text-transform: uppercase;
letter-spacing: 0.5px;
}
.ban-override-banned {
background: rgba(248, 81, 73, 0.15);
color: #f85149;
}
.ban-override-unbanned {
background: rgba(63, 185, 80, 0.15);
color: #3fb950;
}
/* Attack Types Cell Styling */
.attack-types-cell {
max-width: 280px;

View File

@@ -20,7 +20,21 @@ document.addEventListener('alpine:init', () => {
// IP Insight state
insightIp: null,
init() {
// Auth state (UI only — actual security enforced server-side via cookie)
authenticated: false,
authModal: { show: false, password: '', error: '', loading: false },
async init() {
// Check if already authenticated (cookie-based)
try {
const resp = await fetch(`${this.dashboardPath}/api/auth/check`, { credentials: 'same-origin' });
if (resp.ok) this.authenticated = true;
} catch {}
// Sync ban action button visibility with auth state
this.$watch('authenticated', (val) => updateBanActionVisibility(val));
updateBanActionVisibility(this.authenticated);
// Handle hash-based tab routing
const hash = window.location.hash.slice(1);
if (hash === 'ip-stats' || hash === 'attacks') {
@@ -32,8 +46,11 @@ document.addEventListener('alpine:init', () => {
const h = window.location.hash.slice(1);
if (h === 'ip-stats' || h === 'attacks') {
this.switchToAttacks();
} else if (h === 'banlist') {
if (this.authenticated) this.switchToBanlist();
} else if (h === 'tracked-ips') {
if (this.authenticated) this.switchToTrackedIps();
} else if (h !== 'ip-insight') {
// Don't switch away from ip-insight via hash if already there
if (this.tab !== 'ip-insight') {
this.switchToOverview();
}
@@ -61,6 +78,108 @@ document.addEventListener('alpine:init', () => {
window.location.hash = '#overview';
},
switchToBanlist() {
if (!this.authenticated) return;
this.tab = 'banlist';
window.location.hash = '#banlist';
this.$nextTick(() => {
const container = document.getElementById('banlist-htmx-container');
if (container && typeof htmx !== 'undefined') {
htmx.ajax('GET', `${this.dashboardPath}/htmx/banlist`, {
target: '#banlist-htmx-container',
swap: 'innerHTML'
});
}
});
},
switchToTrackedIps() {
if (!this.authenticated) return;
this.tab = 'tracked-ips';
window.location.hash = '#tracked-ips';
this.$nextTick(() => {
const container = document.getElementById('tracked-ips-htmx-container');
if (container && typeof htmx !== 'undefined') {
htmx.ajax('GET', `${this.dashboardPath}/htmx/tracked-ips`, {
target: '#tracked-ips-htmx-container',
swap: 'innerHTML'
});
}
});
},
async logout() {
try {
await fetch(`${this.dashboardPath}/api/auth/logout`, {
method: 'POST',
credentials: 'same-origin',
});
} catch {}
this.authenticated = false;
if (this.tab === 'banlist' || this.tab === 'tracked-ips') this.switchToOverview();
},
promptAuth() {
this.authModal = { show: true, password: '', error: '', loading: false };
this.$nextTick(() => {
if (this.$refs.authPasswordInput) this.$refs.authPasswordInput.focus();
});
},
closeAuthModal() {
this.authModal.show = false;
this.authModal.password = '';
this.authModal.error = '';
this.authModal.loading = false;
},
async submitAuth() {
const password = this.authModal.password;
if (!password) {
this.authModal.error = 'Please enter a password';
return;
}
this.authModal.error = '';
this.authModal.loading = true;
try {
const msgBuf = new TextEncoder().encode(password);
const hashBuf = await crypto.subtle.digest('SHA-256', msgBuf);
const fingerprint = Array.from(new Uint8Array(hashBuf))
.map(b => b.toString(16).padStart(2, '0')).join('');
const resp = await fetch(`${this.dashboardPath}/api/auth`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
credentials: 'same-origin',
body: JSON.stringify({ fingerprint }),
});
if (resp.ok) {
this.authenticated = true;
this.closeAuthModal();
this.switchToBanlist();
} else {
const data = await resp.json().catch(() => ({}));
this.authModal.error = data.error || 'Invalid password';
this.authModal.password = '';
this.authModal.loading = false;
if (data.locked && data.retry_after) {
let remaining = data.retry_after;
const interval = setInterval(() => {
remaining--;
if (remaining <= 0) {
clearInterval(interval);
this.authModal.error = '';
} else {
this.authModal.error = `Too many attempts. Try again in ${remaining}s`;
}
}, 1000);
}
}
} catch {
this.authModal.error = 'Authentication failed';
this.authModal.loading = false;
}
},
switchToIpInsight() {
// Only allow switching if an IP is selected
if (!this.insightIp) return;
@@ -150,19 +269,160 @@ document.addEventListener('alpine:init', () => {
}));
});
// Helper to access Alpine.js component data
function getAlpineData(selector) {
const container = document.querySelector(selector);
if (!container) return null;
return Alpine.$data ? Alpine.$data(container) : (container._x_dataStack && container._x_dataStack[0]);
}
// Global function for opening IP Insight (used by map popups)
window.openIpInsight = function(ip) {
// Find the Alpine component and call openIpInsight
const container = document.querySelector('[x-data="dashboardApp()"]');
if (container) {
// Try Alpine 3.x API first, then fall back to older API
const data = Alpine.$data ? Alpine.$data(container) : (container._x_dataStack && container._x_dataStack[0]);
if (data && typeof data.openIpInsight === 'function') {
data.openIpInsight(ip);
}
const data = getAlpineData('[x-data="dashboardApp()"]');
if (data && typeof data.openIpInsight === 'function') {
data.openIpInsight(ip);
}
};
// Escape HTML to prevent XSS when inserting into innerHTML
function escapeHtml(str) {
const div = document.createElement('div');
div.textContent = str;
return div.innerHTML;
}
// Custom modal system (replaces native confirm/alert)
window.krawlModal = {
_create(icon, iconClass, message, buttons) {
return new Promise(resolve => {
const overlay = document.createElement('div');
overlay.className = 'krawl-modal-overlay';
overlay.innerHTML = `
<div class="krawl-modal-box">
<div class="krawl-modal-icon ${iconClass}">
<span class="material-symbols-outlined">${icon}</span>
</div>
<div class="krawl-modal-message">${message}</div>
<div class="krawl-modal-actions" id="krawl-modal-actions"></div>
</div>`;
const actions = overlay.querySelector('#krawl-modal-actions');
buttons.forEach(btn => {
const el = document.createElement('button');
el.className = `auth-modal-btn ${btn.cls}`;
el.textContent = btn.label;
el.onclick = () => { overlay.remove(); resolve(btn.value); };
actions.appendChild(el);
});
overlay.addEventListener('click', e => {
if (e.target === overlay) { overlay.remove(); resolve(false); }
});
document.body.appendChild(overlay);
});
},
confirm(message) {
return this._create('warning', 'krawl-modal-icon-warn', message, [
{ label: 'Cancel', cls: 'auth-modal-btn-cancel', value: false },
{ label: 'Confirm', cls: 'auth-modal-btn-submit', value: true },
]);
},
success(message) {
return this._create('check_circle', 'krawl-modal-icon-success', message, [
{ label: 'OK', cls: 'auth-modal-btn-submit', value: true },
]);
},
error(message) {
return this._create('error', 'krawl-modal-icon-error', message, [
{ label: 'OK', cls: 'auth-modal-btn-cancel', value: true },
]);
},
};
// Global ban action for IP insight page (auth-gated)
window.ipBanAction = async function(ip, action) {
// Check if authenticated
const data = getAlpineData('[x-data="dashboardApp()"]');
if (!data || !data.authenticated) {
if (data && typeof data.promptAuth === 'function') data.promptAuth();
return;
}
const safeIp = escapeHtml(ip);
const safeAction = escapeHtml(action);
const confirmed = await krawlModal.confirm(`Are you sure you want to ${safeAction} IP <strong>${safeIp}</strong>?`);
if (!confirmed) return;
try {
const resp = await fetch(`${window.__DASHBOARD_PATH__}/api/ban-override`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
credentials: 'same-origin',
body: JSON.stringify({ ip, action }),
});
const result = await resp.json().catch(() => ({}));
if (resp.ok) {
krawlModal.success(escapeHtml(result.message || `${action} successful for ${ip}`));
const overrides = document.getElementById('overrides-container');
if (overrides) {
htmx.ajax('GET', `${window.__DASHBOARD_PATH__}/htmx/ban/overrides?page=1`, {
target: '#overrides-container',
swap: 'innerHTML'
});
}
} else {
krawlModal.error(escapeHtml(result.error || `Failed to ${action} IP ${ip}`));
}
} catch {
krawlModal.error('Request failed');
}
};
// Global track action for IP insight page (auth-gated)
window.ipTrackAction = async function(ip, action) {
const data = getAlpineData('[x-data="dashboardApp()"]');
if (!data || !data.authenticated) {
if (data && typeof data.promptAuth === 'function') data.promptAuth();
return;
}
const safeIp = escapeHtml(ip);
const label = action === 'track' ? 'track' : 'untrack';
const confirmed = await krawlModal.confirm(`Are you sure you want to ${label} IP <strong>${safeIp}</strong>?`);
if (!confirmed) return;
try {
const resp = await fetch(`${window.__DASHBOARD_PATH__}/api/track-ip`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
credentials: 'same-origin',
body: JSON.stringify({ ip, action }),
});
const result = await resp.json().catch(() => ({}));
if (resp.ok) {
krawlModal.success(escapeHtml(result.message || `${label} successful for ${ip}`));
// Refresh tracked IPs list if visible
const container = document.getElementById('tracked-ips-container');
if (container && typeof htmx !== 'undefined') {
htmx.ajax('GET', `${window.__DASHBOARD_PATH__}/htmx/tracked-ips/list?page=1`, {
target: '#tracked-ips-container',
swap: 'innerHTML'
});
}
} else {
krawlModal.error(escapeHtml(result.error || `Failed to ${label} IP ${ip}`));
}
} catch {
krawlModal.error('Request failed');
}
};
// Show/hide ban action buttons based on auth state
function updateBanActionVisibility(authenticated) {
document.querySelectorAll('.ip-ban-actions').forEach(el => {
el.style.display = authenticated ? 'inline-flex' : 'none';
});
}
// Update visibility after HTMX swaps in new content
document.addEventListener('htmx:afterSwap', () => {
const data = getAlpineData('[x-data="dashboardApp()"]');
if (data) updateBanActionVisibility(data.authenticated);
});
// Utility function for formatting timestamps (used by map popups)
function formatTimestamp(isoTimestamp) {
if (!isoTimestamp) return 'N/A';

View File

@@ -0,0 +1,22 @@
@font-face {
font-family: 'Material Symbols Outlined';
font-style: normal;
font-weight: 400;
src: url(fonts/material-symbols-outlined.woff2) format('woff2');
}
.material-symbols-outlined {
font-family: 'Material Symbols Outlined';
font-weight: normal;
font-style: normal;
font-size: 24px;
line-height: 1;
letter-spacing: normal;
text-transform: none;
display: inline-block;
white-space: nowrap;
word-wrap: normal;
direction: ltr;
-webkit-font-feature-settings: 'liga';
-webkit-font-smoothing: antialiased;
}