Commit Graph

59 Commits

Author SHA1 Message Date
Lorenzo Venerandi
130e81ad64 Feat/dashboard improvements (#55)
* fixed external ip resoultion

* added dashboard logic division, filtering capabilities, geoip map, attacker stats

* refactor: replace print statements with applogger for error logging in DatabaseManager

* feat: add click listeners for IP cells in dashboard tables to fetch and display stats

---------

Co-authored-by: BlessedRebuS <patrick.difa@gmail.com>
2026-01-25 22:50:27 +01:00
Patrick Di Fazio
c7fe588bc4 fixed external ip resoultion (#54) 2026-01-25 01:19:30 +01:00
BlessedRebuS
14d616fae3 added ip logging memory improvements, added local ip and public ip exlusion 2026-01-24 23:28:10 +01:00
Lorenzo Venerandi
4addf41a5b Add logging for environment variable overrides in config 2026-01-23 22:26:14 +01:00
Lorenzo Venerandi
4450d3a4e3 Linted code iwht black tool 2026-01-23 22:00:21 +01:00
leonardobambini
4e4c370b72 added site depth limit mechanism (#48)
* added site depth limit mechanism

* modified max pages limit and ban duration seconds

---------

Co-authored-by: Leonardo Bambini <lbambini@Leonardos-MacBook-Air.local>
Co-authored-by: BlessedRebuS <patrick.difa@gmail.com>
2026-01-23 21:33:32 +01:00
Lorenzo Venerandi
223883a781 Configuration override from environment variable (#47)
* Add environment variable override for config fields

Introduces functions to override configuration fields from environment variables, allowing dynamic configuration without modifying YAML files. The environment variable names are generated from field names, and type conversion is handled for int, float, and tuple fields.

* update chart version to 0.1.4

* Update README.md to enhance environment variable configuration details and improve overall clarity
2026-01-23 17:34:23 +01:00
Patrick Di Fazio
ac936978da Merge branch 'dev' into fix/dashboard 2026-01-17 23:09:14 +01:00
BlessedRebuS
59d99484e9 fixed dashboard alignment 2026-01-17 22:43:42 +01:00
Patrick Di Fazio
2dd35234c0 fixed dashboard alignment 2026-01-17 22:41:19 +01:00
carnivuth
9d9a718aec feat:removed manual timezone management, delegate timezone configuration to execution environment
removed code that manages timezone setup from config file, krawl now
obeys to the environment configuration
2026-01-17 18:06:09 +01:00
Phillip Tarrant
541b5d0f1b Optimize scheduled tasks to reduce unnecessary processing
- Add conditional execution to export-malicious-ips task: only runs
    when honeypot was accessed in last 5 minutes
  - Add since_minutes parameter to get_access_logs() for time filtering
  - Optimize analyze-ips task to only process IPs with activity in the
    last minute, fetching full history per-IP instead of all logs
  - Exclude RFC1918 private addresses and non-routable IPs from IP
    reputation enrichment (10.x, 172.16-31.x, 192.168.x, 127.x, 169.254.x)
2026-01-15 13:30:35 -06:00
Phillip Tarrant
f7b9ee54e3 Fix SQLite "database is locked" errors in fetch_ip_rep task
- Add missing session.commit() and cleanup to update_ip_rep_infos() **bugfix**
  - Enable SQLite WAL mode and 30s busy timeout for better concurrency **race condition prevention**
  - Add get_unenriched_ips() method to only fetch IPs needing enrichment **don't enrich what's already done*
  - Rewrite fetch_ip_rep task to process only unenriched IPs (limit 50) **API kindness**
  - Change task frequency from every 1 minute to every 5 minutes **API kindness**
  - Add request timeout (10s) and proper error handling **can update to longer if needed**
2026-01-10 14:59:15 -06:00
Patrick Di Fazio
c2dbcf588c added iprep to the dashboard, fixed bugs 2026-01-10 20:00:33 +01:00
Leonardo Bambini
77196952d1 made ip analysis and ip rep info fetch a scheduled task 2026-01-10 14:53:31 +01:00
Patrick Di Fazio
431a42aa0a removed exports 2026-01-09 20:39:25 +01:00
Patrick Di Fazio
be7ba1f820 added download button 2026-01-09 20:37:20 +01:00
Phillip Tarrant
0b276b1a47 Merge branch 'dev' into feat/background-tasks 2026-01-09 09:39:56 -06:00
Patrick Di Fazio
b61461d028 fixed categorization visualization, fixed date in the dashboard, fixed attack regex detection 2026-01-08 19:20:22 +01:00
Leonardo Bambini
4f42b946f3 added ip rep fetch + bug fix 2026-01-07 22:56:01 +01:00
Phillip Tarrant
edb288a271 Fixed some print statements to leverage logging, pulled in most recent dev edits, added exports to gitignore 2026-01-07 12:33:43 -06:00
Phillip Tarrant
5fe1984365 Merge branch 'dev' into feat/background-tasks 2026-01-07 11:51:27 -06:00
Patrick Di Fazio
7690841029 added categorization visualization and itmeline 2026-01-07 18:24:43 +01:00
Patrick Di Fazio
02aed9e65a added drop down menu and scoring graph to the dashboard 2026-01-06 18:50:36 +01:00
Phillip Tarrant
7d9f0616b7 Add background task to export suspicious IPs to text file
- Implement export-malicious-ips task that queries distinct IPs flagged
    as is_suspicious from database and writes to exports/malicious_ips.txt
  - Add exports volume mount to docker-compose.yaml for host persistence
  - Update entrypoint.sh to fix ownership of exports directory for krawl user
  - Update Dockerfile to create /app/exports directory during build

Other tasks can be added by creating them in the tasks dir using the same setup as this task.
All tasks *MUST* include a TASK_CONFIG dict and a main method in the file to work correctly.
2026-01-05 11:54:02 -06:00
Patrick Di Fazio
190d74e1a7 modified krawl template for single page visualization 2026-01-05 17:27:27 +01:00
Patrick Di Fazio
4478c60956 added krawl homepage to templates 2026-01-05 17:07:10 +01:00
Patrick Di Fazio
bd8c326918 tuned weights 2026-01-05 16:54:43 +01:00
Leonardo Bambini
bf02fdb351 modified default analyzer values 2026-01-05 10:01:51 +01:00
Leonardo Bambini
ff98a77e1a parametrized into config.yaml + bug fix 2026-01-04 22:20:10 +01:00
Leonardo Bambini
48f38cb28e added scoring system + db model modifications 2026-01-04 19:12:23 +01:00
Phillip Tarrant
07efa5bd40 Merge branch 'feat/config-yaml' into chore/fix-merge-conflicts 2026-01-03 14:45:32 -06:00
Phillip Tarrant
7bd9d5055c fixing merge issues with main 2026-01-03 14:40:31 -06:00
Phillip Tarrant
4c490e30cb fixing dashboard to ensure starts with forward slash, put back the server_header option to allow pinning 2026-01-03 13:56:16 -06:00
Patrick Di Fazio
4a1d1cf7be added random SQL errors, random server errors, XSS baits 2026-01-03 17:16:37 +01:00
Patrick Di Fazio
5f8bb73546 added random SQL errors, random server errors, XSS baits 2026-01-03 17:14:58 +01:00
Phillip Tarrant
d458eb471d Migrate configuration from environment variables to YAML file
- Add YAML-based configuration loaded from config.yaml (CONFIG_LOCATION env var)
  - Add PyYAML dependency and install requirements in Dockerfile
  - Replace Config.from_env() with get_config() singleton pattern
  - Remove server_header from config (now randomized from wordlists only)
  - Update docker-compose.yaml to mount config.yaml read-only
  - Update Helm chart: restructure values.yaml, generate config.yaml in ConfigMap
  - Update Kubernetes manifests: ConfigMap now contains config.yaml, deployments mount it
  - Remove Helm secret.yaml (dashboard path now auto-generated in config.yaml)
2026-01-02 13:39:54 -06:00
Patrick Di Fazio
0b1e9537d2 Added demo 2025-12-30 00:59:15 +01:00
Patrick Di Fazio
852a15976f Added demo 2025-12-30 00:59:15 +01:00
Patrick Di Fazio
828f04261f Added POST log and dashboard for used credentials 2025-12-30 00:59:15 +01:00
Phillip Tarrant
1486dfc913 Add configurable HTTP Server header for deception
Add SERVER_HEADER environment variable to customize the HTTP Server
  response header, defaulting to Apache/2.2.22 (Ubuntu). This allows the
  honeypot to masquerade as different web servers to attract attackers.

  - Add server_header field to Config dataclass
  - Override version_string() in Handler to return configured header
  - Update documentation and all deployment configs
2025-12-30 00:59:15 +01:00
Patrick Di Fazio
354f8bf895 Fix indentation for server_header in config.py 2025-12-30 00:24:36 +01:00
Patrick Di Fazio
12dcf5ba39 Merge branch 'dev' into feat/randomized-server-header 2025-12-30 00:14:14 +01:00
Patrick Di Fazio
8a645c22e6 Merge branch 'dev' into feat/randomized-server-header 2025-12-30 00:02:44 +01:00
Patrick Di Fazio
06ffa2c480 Added wordlists and server header logic to helm 2025-12-29 23:57:37 +01:00
Leonardo Bambini
c2c43ac985 Added randomized server header and changed behavior of SERVER_HEADER env var 2025-12-29 18:51:37 +01:00
Phillip Tarrant
a4baedffd9 updated dashboard to pull data from db. This closes issue #10 2025-12-28 13:56:23 -06:00
Phillip Tarrant
6487cb493d Merge branch 'dev' into feat/sqlite3-storage 2025-12-28 11:08:10 -06:00
Phillip Tarrant
f1c142c53d feat: add SQLite persistent storage for request logging
- Add SQLAlchemy-based database layer for persistent storage
  - Create models for access_logs, credential_attempts, attack_detections, ip_stats
  - Include fields for future GeoIP and reputation enrichment
  - Implement sanitization utilities to protect against malicious payloads
  - Fix XSS vulnerability in dashboard template (HTML escape all user data)
  - Add DATABASE_PATH and DATABASE_RETENTION_DAYS config options
  - Dual storage: in-memory for dashboard performance + SQLite for persistence

  New files:
  - src/models.py - SQLAlchemy ORM models
  - src/database.py - DatabaseManager singleton
  - src/sanitizer.py - Input sanitization and HTML escaping
  - requirements.txt - SQLAlchemy dependency

  Security protections:
  - Parameterized queries via SQLAlchemy ORM
  - Field length limits to prevent storage exhaustion
  - Null byte and control character stripping
  - HTML escaping on dashboard output
2025-12-28 10:43:32 -06:00
Patrick Di Fazio
6556e17f91 Added timezone env variable handling 2025-12-28 17:07:18 +01:00