Commit Graph

54 Commits

Author SHA1 Message Date
Lorenzo Venerandi
5f1fedab3d feat: add sort_order parameter to access log retrieval for flexible timestamp sorting 2026-03-04 13:55:10 +01:00
Lorenzo Venerandi
a385381dbd feat: add log_id to suspicious activities for detailed request viewing 2026-03-04 12:37:42 +01:00
Lorenzo Venerandi
911628fe61 fix: correct logic for flagging stale IPs for reevaluation based on last_seen timestamp 2026-03-03 19:45:36 +01:00
Lorenzo Venerandi
36df289a5e feat: enhance database retention logic to use configurable retention period and delete stale IPs 2026-03-03 19:03:40 +01:00
Lorenzo Venerandi
f3ec4f8420 fix: update stale IP reevaluation logic to flag IPs last seen over 30 days ago 2026-03-03 18:58:06 +01:00
Lorenzo Venerandi
991dca6a9d code linted 2026-03-01 18:02:22 +01:00
Lorenzo Venerandi
ed4fe0dcfb feat: Add IP filtering to attack types pagination and detail views 2026-03-01 18:01:19 +01:00
Lorenzo Venerandi
fbc757f0a6 feat: Enhance logging configuration to support dynamic log levels 2026-03-01 17:36:29 +01:00
Lorenzo Venerandi
8fc2d47e96 feat: Add detailed IP information view and refactor IP insight template
- Introduced a new partial template `_ip_detail.html` for displaying comprehensive IP details, including activity, geo & network information, reputation, and access history.
- Updated `ip_insight.html` to include the new `_ip_detail.html` partial, streamlining the code and enhancing maintainability.
- Enhanced CSS styles for improved layout and responsiveness, including adjustments to the radar chart size and the introduction of a two-column grid layout for IP details.
- Refactored JavaScript for loading attack types charts to support multiple instances and improved error handling.
2026-03-01 17:00:10 +01:00
Lorenzo Venerandi
95ab55c428 lint code 2026-03-01 15:57:54 +01:00
Lorenzo Venerandi
7401783847 refactor: optimize database queries by utilizing IpStats for performance improvements 2026-03-01 15:57:40 +01:00
Lorenzo Venerandi
b8f0cc25d0 style: format code for better readability in DatabaseManager class 2026-02-28 19:42:32 +01:00
Patrick Di Fazio
82d4757ad1 Merge branch 'feat/dashboard-single-ip-page' into feat/add-search-bar 2026-02-28 18:47:36 +01:00
BlessedRebuS
62bb091926 added search bar feature, refactored the dashboard 2026-02-28 18:43:09 +01:00
Lorenzo Venerandi
368265dea4 feat: update suspicious activity display to show last 10 requests and improve database filtering 2026-02-28 18:12:40 +01:00
Lorenzo Venerandi
3d8178ff0e feat: enhance dashboard with IP category display and improved data tables 2026-02-28 18:04:26 +01:00
Lorenzo Venerandi
d9ae55c0aa feat: add IP insight feature with detailed view and actions
- Updated various tables to include "Actions" column with inspect buttons for IP insights.
- Created a new IP insight template for displaying detailed information about an IP address.
- Implemented JavaScript functions to handle opening the IP insight view and loading data via HTMX.
- Enhanced map markers to include inspect buttons for quick access to IP insights.
- Added styles for the new IP insight page and buttons to maintain UI consistency.
2026-02-28 17:43:50 +01:00
carnivuth
f7416518fe added first version of single ip page breakdiwn 2026-02-26 17:10:14 +01:00
carnivuth
75722051d6 added first version of single ip page breakdiwn 2026-02-26 17:10:14 +01:00
Lorenzo Venerandi
b559fd6a84 feat: add task to flag stale IPs for reevaluation 2026-02-22 18:41:50 +01:00
Lorenzo Venerandi
71adbe2605 feat: refine IP reevaluation logic to include unanalyzed addresses 2026-02-22 17:57:06 +01:00
Lorenzo Venerandi
b500b22041 code linting 2026-02-22 16:27:59 +01:00
Lorenzo Venerandi
8ce8b6b40a feat: implement IP ban and rate-limiting logic in database with migration scripts 2026-02-22 16:23:52 +01:00
Lorenzo Venerandi
db848e7ecb feat: add need_reevaluation column to IpStats and update related logic 2026-02-22 16:05:39 +01:00
Lorenzo Venerandi
2f82d3a3bd feat: implement database migration runner and initialize migrations on startup 2026-02-22 16:05:39 +01:00
Lorenzo Venerandi
152682de2c code linted 2026-02-18 00:02:44 +01:00
Lorenzo Venerandi
cccc9cde35 refactor: optimize access log query to count unique entries and improve pagination 2026-02-17 18:13:25 +01:00
Lorenzo Venerandi
b94cd38b61 refactor: optimize database access and implement retention task 2026-02-17 18:04:09 +01:00
Lorenzo Venerandi
5d38ea45a8 starting full refactor with FastAPI routes + HTMX and AlpineJS on client side 2026-02-17 13:09:01 +01:00
carnivuth
396b9b1710 linted code 2026-02-15 15:10:41 +01:00
Patrick Di Fazio
771174c6a9 added raw request handling, enanched attack detection for GET and POSTS, templatized suspicioius activity to fetch from wordlists.json, aligned helm to load new wordlist config, added migration scripts from 1.0.0 to new krawl versions, removed old and unused functions, added test scripts 2026-02-08 16:02:18 +01:00
BlessedRebuS
01547dbf8f modified doc 2026-02-02 23:01:25 +01:00
BlessedRebuS
863fac251d modified dashboard, added ip-api data fetch 2026-02-01 22:43:12 +01:00
Lorenzo Venerandi
e93bcb959a Doc/updated documentation (#60)
* added documentation, updated repo pointer in the dashboard, added dashboard link highlighting and mionor fixes

* added doc

* added logo to dashboard

* Fixed dashboard attack chart

* Enhance fake data generation with varied request counts for better visualization

* Add automatic migrations and support for latitude/longitude in IP stats

* Update Helm chart version to 0.2.2 and add timezone configuration option

---------

Co-authored-by: BlessedRebuS <patrick.difa@gmail.com>
2026-01-29 11:55:06 +01:00
Lorenzo Venerandi
39d9d62247 Feat/attack map improvement (#58)
* Enhance geolocation functionality and improve unenriched IP retrieval logic

* Refactor test_insert_fake_ips.py to enhance geolocation data handling and improve IP data structure

* Refactor code for improved readability and consistency in database and geolocation utilities
2026-01-27 17:32:38 +01:00
Lorenzo Venerandi
5aca684df9 Feat/attack map improvement (#57)
* feat: enhance IP reputation management with city data and geolocation integration

* feat: enhance dashboard with city coordinates and improved marker handling

* feat: update chart version to 0.2.1 in Chart.yaml, README.md, and values.yaml

* feat: update logo format and size in README.md

* feat: improve location display logic in dashboard for attackers and IPs
2026-01-27 16:56:34 +01:00
Lorenzo Venerandi
8c76f6c847 Feat/deployment update (#56)
* feat: update analyzer thresholds and add crawl configuration options

* feat: update Helm chart version and add README for installation instructions

* feat: update installation instructions in README and add Docker support

* feat: update deployment manifests and configuration for improved service handling and analyzer settings

* feat: add API endpoint for paginated IP retrieval and enhance dashboard visualization with category filters

* feat: update configuration for Krawl service to use external config file

* feat: refactor code for improved readability and consistency across multiple files

* feat: remove Flake8, Pylint, and test steps from PR checks workflow
2026-01-26 12:36:22 +01:00
Lorenzo Venerandi
130e81ad64 Feat/dashboard improvements (#55)
* fixed external ip resoultion

* added dashboard logic division, filtering capabilities, geoip map, attacker stats

* refactor: replace print statements with applogger for error logging in DatabaseManager

* feat: add click listeners for IP cells in dashboard tables to fetch and display stats

---------

Co-authored-by: BlessedRebuS <patrick.difa@gmail.com>
2026-01-25 22:50:27 +01:00
BlessedRebuS
14d616fae3 added ip logging memory improvements, added local ip and public ip exlusion 2026-01-24 23:28:10 +01:00
Lorenzo Venerandi
4450d3a4e3 Linted code iwht black tool 2026-01-23 22:00:21 +01:00
carnivuth
9d9a718aec feat:removed manual timezone management, delegate timezone configuration to execution environment
removed code that manages timezone setup from config file, krawl now
obeys to the environment configuration
2026-01-17 18:06:09 +01:00
Phillip Tarrant
541b5d0f1b Optimize scheduled tasks to reduce unnecessary processing
- Add conditional execution to export-malicious-ips task: only runs
    when honeypot was accessed in last 5 minutes
  - Add since_minutes parameter to get_access_logs() for time filtering
  - Optimize analyze-ips task to only process IPs with activity in the
    last minute, fetching full history per-IP instead of all logs
  - Exclude RFC1918 private addresses and non-routable IPs from IP
    reputation enrichment (10.x, 172.16-31.x, 192.168.x, 127.x, 169.254.x)
2026-01-15 13:30:35 -06:00
Phillip Tarrant
f7b9ee54e3 Fix SQLite "database is locked" errors in fetch_ip_rep task
- Add missing session.commit() and cleanup to update_ip_rep_infos() **bugfix**
  - Enable SQLite WAL mode and 30s busy timeout for better concurrency **race condition prevention**
  - Add get_unenriched_ips() method to only fetch IPs needing enrichment **don't enrich what's already done*
  - Rewrite fetch_ip_rep task to process only unenriched IPs (limit 50) **API kindness**
  - Change task frequency from every 1 minute to every 5 minutes **API kindness**
  - Add request timeout (10s) and proper error handling **can update to longer if needed**
2026-01-10 14:59:15 -06:00
Patrick Di Fazio
c2dbcf588c added iprep to the dashboard, fixed bugs 2026-01-10 20:00:33 +01:00
Leonardo Bambini
77196952d1 made ip analysis and ip rep info fetch a scheduled task 2026-01-10 14:53:31 +01:00
Phillip Tarrant
0b276b1a47 Merge branch 'dev' into feat/background-tasks 2026-01-09 09:39:56 -06:00
Patrick Di Fazio
b61461d028 fixed categorization visualization, fixed date in the dashboard, fixed attack regex detection 2026-01-08 19:20:22 +01:00
Leonardo Bambini
4f42b946f3 added ip rep fetch + bug fix 2026-01-07 22:56:01 +01:00
Phillip Tarrant
edb288a271 Fixed some print statements to leverage logging, pulled in most recent dev edits, added exports to gitignore 2026-01-07 12:33:43 -06:00
Patrick Di Fazio
7690841029 added categorization visualization and itmeline 2026-01-07 18:24:43 +01:00