Malin/krawl.es
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
103 Commits 1 Branch 0 Tags
261a7b26b918d01fa6c02257dd809b82d55650e8
Commit Graph

103 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Lorenzo Venerandi
261a7b26b9 feat: add GitHub Actions workflows for PR checks and security scans 2026-01-22 11:10:04 +01:00
Lorenzo Venerandi
143b301bcb feat: add Kubernetes validation workflow for pull requests 2026-01-22 10:57:28 +01:00
Lorenzo Venerandi
2ff6bb34b2 feat: add GitHub Actions workflow for packaging and pushing Helm charts 2026-01-22 10:51:11 +01:00
Lorenzo Venerandi
adbbe4d4ea feat: add GitHub Actions workflow for building and pushing Docker images 2026-01-22 10:39:09 +01:00
Patrick Di Fazio
fa8403c5a9 Merge pull request #42 from BlessedRebuS/feat/task-optimization 
Optimize scheduled tasks to reduce unnecessary processing
 2026-01-17 23:06:30 +01:00
BlessedRebuS
6f07ab8409 Removed old Dockerfile, added volume name 2026-01-17 23:05:47 +01:00
carnivuth
00b222b754 added development docker compose file 2026-01-17 18:12:41 +01:00
carnivuth
8deabe8ce0 added direnv file 2026-01-17 18:06:27 +01:00
carnivuth
9d9a718aec feat:removed manual timezone management, delegate timezone configuration to execution environment 
removed code that manages timezone setup from config file, krawl now
obeys to the environment configuration
 2026-01-17 18:06:09 +01:00
Phillip Tarrant
541b5d0f1b Optimize scheduled tasks to reduce unnecessary processing 
- Add conditional execution to export-malicious-ips task: only runs
    when honeypot was accessed in last 5 minutes
  - Add since_minutes parameter to get_access_logs() for time filtering
  - Optimize analyze-ips task to only process IPs with activity in the
    last minute, fetching full history per-IP instead of all logs
  - Exclude RFC1918 private addresses and non-routable IPs from IP
    reputation enrichment (10.x, 172.16-31.x, 192.168.x, 127.x, 169.254.x)
 2026-01-15 13:30:35 -06:00
Patrick Di Fazio
554bd486da Merge pull request #39 from luismco/logo 
Adds SVG logo
 2026-01-14 13:51:22 +01:00
Luís Oliveira
f5e403a5f4 Rename krawl.svg to krawl-svg.svg 2026-01-14 12:19:30 +00:00
Luís Oliveira
d39d33c337 Adds SVG logo 2026-01-14 12:19:01 +00:00
Patrick Di Fazio
323aa0d15d Merge pull request #36 from BlessedRebuS/bug/db-locked-fix 
Bug/db locked fix
 2026-01-10 22:31:03 +01:00
Phillip Tarrant
72a213387e fixing requirements file to versioned pull for request 2026-01-10 15:17:16 -06:00
Phillip Tarrant
66d970ef92 Merge pull request #35 from BlessedRebuS/fix/requirements 
added requests
 2026-01-10 15:15:22 -06:00
Phillip Tarrant
0e0639ba8b adding requests to requirements 2026-01-10 15:14:17 -06:00
Phillip Tarrant
f7b9ee54e3 Fix SQLite "database is locked" errors in fetch_ip_rep task 
- Add missing session.commit() and cleanup to update_ip_rep_infos() **bugfix**
  - Enable SQLite WAL mode and 30s busy timeout for better concurrency **race condition prevention**
  - Add get_unenriched_ips() method to only fetch IPs needing enrichment **don't enrich what's already done*
  - Rewrite fetch_ip_rep task to process only unenriched IPs (limit 50) **API kindness**
  - Change task frequency from every 1 minute to every 5 minutes **API kindness**
  - Add request timeout (10s) and proper error handling **can update to longer if needed**
 2026-01-10 14:59:15 -06:00
Patrick Di Fazio
5d25fb9d6b added requests 2026-01-10 21:37:16 +01:00
Phillip Tarrant
edaafb0263 Merge pull request #34 from BlessedRebuS/feat/scoring-alogorithm 
Feat/scoring alogorithm
 2026-01-10 13:30:18 -06:00
Patrick Di Fazio
c65dca850c restored port 2026-01-10 20:06:33 +01:00
Patrick Di Fazio
c2dbcf588c added iprep to the dashboard, fixed bugs 2026-01-10 20:00:33 +01:00
Leonardo Bambini
77196952d1 made ip analysis and ip rep info fetch a scheduled task 2026-01-10 14:53:31 +01:00
Patrick Di Fazio
5a57c0774f Merge pull request #30 from BlessedRebuS/feat/background-tasks 
Feat/background tasks
 2026-01-09 20:39:38 +01:00
Patrick Di Fazio
431a42aa0a removed exports 2026-01-09 20:39:25 +01:00
Patrick Di Fazio
be7ba1f820 added download button 2026-01-09 20:37:20 +01:00
Phillip Tarrant
0b276b1a47 Merge branch 'dev' into feat/background-tasks 2026-01-09 09:39:56 -06:00
Phillip Tarrant
e1c49f1805 Merge pull request #31 from BlessedRebuS/fix/scoring-algorithm 
fixed categorization visualization, fixed date in the dashboard, fixe…
 2026-01-08 13:41:54 -06:00
Patrick Di Fazio
b61461d028 fixed categorization visualization, fixed date in the dashboard, fixed attack regex detection 2026-01-08 19:20:22 +01:00
Leonardo Bambini
4f42b946f3 added ip rep fetch + bug fix 2026-01-07 22:56:01 +01:00
Phillip Tarrant
edb288a271 Fixed some print statements to leverage logging, pulled in most recent dev edits, added exports to gitignore 2026-01-07 12:33:43 -06:00
Phillip Tarrant
5fe1984365 Merge branch 'dev' into feat/background-tasks 2026-01-07 11:51:27 -06:00
Patrick Di Fazio
e8a79ef970 Merge pull request #29 from BlessedRebuS/feat/scoring-alogorithm 
Feat/scoring alogorithm
 2026-01-07 18:50:44 +01:00
Phillip Tarrant
b6c3710ca3 Merge branch 'dev' into feat/scoring-alogorithm 2026-01-07 11:43:58 -06:00
Patrick Di Fazio
7690841029 added categorization visualization and itmeline 2026-01-07 18:24:43 +01:00
Patrick Di Fazio
02aed9e65a added drop down menu and scoring graph to the dashboard 2026-01-06 18:50:36 +01:00
Phillip Tarrant
7d9f0616b7 Add background task to export suspicious IPs to text file 
- Implement export-malicious-ips task that queries distinct IPs flagged
    as is_suspicious from database and writes to exports/malicious_ips.txt
  - Add exports volume mount to docker-compose.yaml for host persistence
  - Update entrypoint.sh to fix ownership of exports directory for krawl user
  - Update Dockerfile to create /app/exports directory during build

Other tasks can be added by creating them in the tasks dir using the same setup as this task.
All tasks *MUST* include a TASK_CONFIG dict and a main method in the file to work correctly.
 2026-01-05 11:54:02 -06:00
Patrick Di Fazio
190d74e1a7 modified krawl template for single page visualization 2026-01-05 17:27:27 +01:00
Patrick Di Fazio
4478c60956 added krawl homepage to templates 2026-01-05 17:07:10 +01:00
Patrick Di Fazio
bd8c326918 tuned weights 2026-01-05 16:54:43 +01:00
Leonardo Bambini
bf02fdb351 modified default analyzer values 2026-01-05 10:01:51 +01:00
Leonardo Bambini
ff98a77e1a parametrized into config.yaml + bug fix 2026-01-04 22:20:10 +01:00
Leonardo Bambini
48f38cb28e added scoring system + db model modifications 2026-01-04 19:12:23 +01:00
Patrick Di Fazio
604eba6c82 Merge pull request #27 from BlessedRebuS/chore/fix-merge-conflicts 
Sync Main and Dev with All Feature Branches
 2026-01-04 17:10:57 +01:00
Phillip Tarrant
07efa5bd40 Merge branch 'feat/config-yaml' into chore/fix-merge-conflicts 2026-01-03 14:45:32 -06:00
Phillip Tarrant
cde279960a adding feat/sql 2026-01-03 14:42:14 -06:00
Phillip Tarrant
7bd9d5055c fixing merge issues with main 2026-01-03 14:40:31 -06:00
Phillip Tarrant
4d1fd5491d Merge pull request #26 from BlessedRebuS/feat/add-deception-features 
Feat/add deception features
 2026-01-03 14:01:02 -06:00
Phillip Tarrant
4c490e30cb fixing dashboard to ensure starts with forward slash, put back the server_header option to allow pinning 2026-01-03 13:56:16 -06:00
Patrick Di Fazio
4a1d1cf7be added random SQL errors, random server errors, XSS baits 2026-01-03 17:16:37 +01:00