Commit Graph

20 Commits

Author SHA1 Message Date
carnivuth
789d77c784 linted code 2026-02-05 17:57:38 +01:00
carnivuth
7621932602 added task dump krawl data and adjusted configuration files 2026-02-05 17:26:06 +01:00
BlessedRebuS
01547dbf8f modified doc 2026-02-02 23:01:25 +01:00
Patrick Di Fazio
3cdb3a360b Merge branch 'dev' into feat/blocklist-api 2026-02-02 22:38:44 +01:00
carnivuth
35c8a90013 addd exports path configuration variable and default values, restored old filename 2026-02-02 14:19:40 +01:00
BlessedRebuS
863fac251d modified dashboard, added ip-api data fetch 2026-02-01 22:43:12 +01:00
carnivuth
09b986f1b0 changed workflow from live computation to scheduled job to support previous structure 2026-01-30 16:13:45 +01:00
Lorenzo Venerandi
d3caa99ecc Fix/export malicious ip file creation (#62)
* Fix: update EXPORTS_DIR path to ensure correct directory structure

* Fix: remove unused imports and honeypot access check from export task
2026-01-29 13:54:07 +01:00
Lorenzo Venerandi
a778262701 Fix: update EXPORTS_DIR path to ensure correct directory structure (#61) 2026-01-29 12:52:48 +01:00
Lorenzo Venerandi
e93bcb959a Doc/updated documentation (#60)
* added documentation, updated repo pointer in the dashboard, added dashboard link highlighting and mionor fixes

* added doc

* added logo to dashboard

* Fixed dashboard attack chart

* Enhance fake data generation with varied request counts for better visualization

* Add automatic migrations and support for latitude/longitude in IP stats

* Update Helm chart version to 0.2.2 and add timezone configuration option

---------

Co-authored-by: BlessedRebuS <patrick.difa@gmail.com>
2026-01-29 11:55:06 +01:00
Lorenzo Venerandi
39d9d62247 Feat/attack map improvement (#58)
* Enhance geolocation functionality and improve unenriched IP retrieval logic

* Refactor test_insert_fake_ips.py to enhance geolocation data handling and improve IP data structure

* Refactor code for improved readability and consistency in database and geolocation utilities
2026-01-27 17:32:38 +01:00
Lorenzo Venerandi
5aca684df9 Feat/attack map improvement (#57)
* feat: enhance IP reputation management with city data and geolocation integration

* feat: enhance dashboard with city coordinates and improved marker handling

* feat: update chart version to 0.2.1 in Chart.yaml, README.md, and values.yaml

* feat: update logo format and size in README.md

* feat: improve location display logic in dashboard for attackers and IPs
2026-01-27 16:56:34 +01:00
Lorenzo Venerandi
8c76f6c847 Feat/deployment update (#56)
* feat: update analyzer thresholds and add crawl configuration options

* feat: update Helm chart version and add README for installation instructions

* feat: update installation instructions in README and add Docker support

* feat: update deployment manifests and configuration for improved service handling and analyzer settings

* feat: add API endpoint for paginated IP retrieval and enhance dashboard visualization with category filters

* feat: update configuration for Krawl service to use external config file

* feat: refactor code for improved readability and consistency across multiple files

* feat: remove Flake8, Pylint, and test steps from PR checks workflow
2026-01-26 12:36:22 +01:00
BlessedRebuS
14d616fae3 added ip logging memory improvements, added local ip and public ip exlusion 2026-01-24 23:28:10 +01:00
Lorenzo Venerandi
4450d3a4e3 Linted code iwht black tool 2026-01-23 22:00:21 +01:00
carnivuth
9d9a718aec feat:removed manual timezone management, delegate timezone configuration to execution environment
removed code that manages timezone setup from config file, krawl now
obeys to the environment configuration
2026-01-17 18:06:09 +01:00
Phillip Tarrant
541b5d0f1b Optimize scheduled tasks to reduce unnecessary processing
- Add conditional execution to export-malicious-ips task: only runs
    when honeypot was accessed in last 5 minutes
  - Add since_minutes parameter to get_access_logs() for time filtering
  - Optimize analyze-ips task to only process IPs with activity in the
    last minute, fetching full history per-IP instead of all logs
  - Exclude RFC1918 private addresses and non-routable IPs from IP
    reputation enrichment (10.x, 172.16-31.x, 192.168.x, 127.x, 169.254.x)
2026-01-15 13:30:35 -06:00
Phillip Tarrant
f7b9ee54e3 Fix SQLite "database is locked" errors in fetch_ip_rep task
- Add missing session.commit() and cleanup to update_ip_rep_infos() **bugfix**
  - Enable SQLite WAL mode and 30s busy timeout for better concurrency **race condition prevention**
  - Add get_unenriched_ips() method to only fetch IPs needing enrichment **don't enrich what's already done*
  - Rewrite fetch_ip_rep task to process only unenriched IPs (limit 50) **API kindness**
  - Change task frequency from every 1 minute to every 5 minutes **API kindness**
  - Add request timeout (10s) and proper error handling **can update to longer if needed**
2026-01-10 14:59:15 -06:00
Leonardo Bambini
77196952d1 made ip analysis and ip rep info fetch a scheduled task 2026-01-10 14:53:31 +01:00
Phillip Tarrant
7d9f0616b7 Add background task to export suspicious IPs to text file
- Implement export-malicious-ips task that queries distinct IPs flagged
    as is_suspicious from database and writes to exports/malicious_ips.txt
  - Add exports volume mount to docker-compose.yaml for host persistence
  - Update entrypoint.sh to fix ownership of exports directory for krawl user
  - Update Dockerfile to create /app/exports directory during build

Other tasks can be added by creating them in the tasks dir using the same setup as this task.
All tasks *MUST* include a TASK_CONFIG dict and a main method in the file to work correctly.
2026-01-05 11:54:02 -06:00