diff --git a/README.md b/README.md index 83c6aa4..2ee2132 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,14 @@

🕷️ Krawl

+

+ + +

+
+

A modern, customizable zero-dependencies honeypot server designed to detect and track malicious activity through deceptive web pages, fake credentials, and canary tokens.

@@ -28,19 +37,22 @@

- Overview • + What is Krawl? • Quick Start • - Configuration • - Dashboard • - Deception Techniques • + Honeypot Pages • + Dashboard • Contributing

-![asd](img/deception-page.png) +
## What is Krawl? -Krawl is a simple cloud native deception server that creates fake web applications with low hanging fruit and juicy fake random information. +**Krawl** is a cloud‑native deception server designed to detect, delay, and analyze malicious web crawlers and automated scanners. + +It creates realistic fake web applications filled with low‑hanging fruit such as admin panels, configuration files, and exposed fake credentials to attract and identify suspicious activity. + +By wasting attacker resources, Krawl helps clearly distinguish malicious behavior from legitimate crawlers. It features: @@ -48,61 +60,65 @@ It features: - **Fake Login Pages**: WordPress, phpMyAdmin, admin panels - **Honeypot Paths**: Advertised in robots.txt to catch scanners - **Fake Credentials**: Realistic-looking usernames, passwords, API keys -- **Canary Token Integration**: External alert triggering +- **[Canary Token](#customizing-the-canary-token) Integration**: External alert triggering - **Real-time Dashboard**: Monitor suspicious activity - **Customizable Wordlists**: Easy JSON-based configuration - **Random Error Injection**: Mimic real server behavior +![asd](img/deception-page.png) + ## 🚀 Quick Start ## Helm Chart Install with default values ```bash -helm install krawl ./helm \ +helm install krawl oci://ghcr.io/blessedrebus/krawl-chart \ --namespace krawl-system \ --create-namespace ``` -Install with custom values +Install with custom [canary token](#customizing-the-canary-token) ```bash -helm install krawl ./helm \ - --namespace krawl-system \ - --create-namespace \ - --values values.yaml -``` - -Install with custom canary token - -```bash -helm install krawl ./helm \ +helm install krawl oci://ghcr.io/blessedrebus/krawl-chart \ --namespace krawl-system \ --create-namespace \ --set config.canaryTokenUrl="http://your-canary-token-url" ``` -Uninstall with +To access the deception server + ```bash -helm uninstall krawl --namespace krawl-system +kubectl get svc krawl -n krawl-system +``` + +Once the EXTERNAL-IP is assigned, access your deception server at: + +``` +http://:5000 ``` ## Kubernetes / Kustomize -Apply all manifests +Apply all manifests with ```bash -kubectl apply -k manifests/ +kubectl apply -f https://raw.githubusercontent.com/BlessedRebuS/Krawl/refs/heads/main/manifests/krawl-all-in-one-deploy.yaml ``` -Retrieve dashboard path + +Retrieve dashboard path with ```bash kubectl get secret krawl-server -n krawl-system -o jsonpath='{.data.dashboard-path}' | base64 -d ``` -Uninstall with + +Or clone the repo and apply the `manifest` folder with + ```bash -kubectl delete -k manifests/ +kubectl apply -k manifests ``` ## Docker +Run Krawl as a docker container with ```bash docker run -d \ @@ -113,11 +129,18 @@ docker run -d \ ``` ## Docker Compose +Run Krawl with docker-compose in the project folder with ```bash docker-compose up -d ``` +Stop it with + +```bash +docker-compose down +``` + ## Python 3.11+ Clone the repository @@ -137,7 +160,7 @@ Visit To access the dashboard -`http://localhost:5000/dashboard-secret-path` +`http://localhost:5000/` ## Configuration via Environment Variables @@ -184,7 +207,9 @@ Disallow: /db_backup.sql ## Honeypot pages Requests to common admin endpoints (`/admin/`, `/wp-admin/`, `/phpMyAdmin/`) return a fake login page. Any login attempt triggers a 1-second delay to simulate real processing and is fully logged in the dashboard (credentials, IP, headers, timing). -![admin-page](img/admin-page.png) +
+ +
Requests to paths like `/backup/`, `/config/`, `/database/`, `/private/`, or `/uploads/` return a fake directory listing populated with “interesting” files, each assigned a random file size to look realistic. @@ -208,9 +233,19 @@ The pages `/credentials.txt` and `/passwords.txt` show fake users and random sec +## Customizing the Canary Token +To create a custom canary token, visit https://canarytokens.org + +and generate a “Web bug” canary token. + +This optional token is triggered when a crawler fully traverses the webpage until it reaches 0. At that point, a URL is returned. When this URL is requested, it sends an alert to the user via email, including the visitor’s IP address and user agent. + + +To enable this feature, set the canary token URL [using the environment variable](#configuration-via-environment-variables) `CANARY_TOKEN_URL`. + ## Wordlists Customization -Edit `wordlists.json` to customize fake data: +Edit `wordlists.json` to customize fake data for your use case ```json { @@ -271,43 +306,7 @@ kubectl get secret krawl -n krawl-system \ -o jsonpath='{.data.dashboard-path}' | base64 -d && echo ``` -## Deception Techniques - -### 1. Robots.txt Honeypots -Advertises forbidden paths that legitimate crawlers avoid but scanners investigate: -- `/admin/`, `/backup/`, `/config/` -- `/credentials.txt`, `/.env`, `/passwords.txt` - -### 2. Fake Services -Mimics real applications: -- WordPress (`/wp-admin`, `/wp-login.php`) -- phpMyAdmin (`/phpmyadmin`) -- Admin panels (`/admin`, `/login`) - -### 3. Credential Traps -Generates realistic but fake: -- Usernames and passwords -- API keys and tokens -- Database connection strings -- AWS credentials - -### 4. Spider Traps -Infinite random links to waste automated scanner time - -### 5. Error Simulation -Random HTTP errors to appear more realistic - - -### Custom Canary Token - -Generate a canary token at [canarytokens.org](https://canarytokens.org) and configure: - -```bash -export CANARY_TOKEN_URL="http://canarytokens.com/..." -python3 src/server.py -``` - -## Contributing +## 🤝 Contributing Contributions welcome! Please: 1. Fork the repository @@ -318,7 +317,7 @@ Contributions welcome! Please:
-## Disclaimer +## ⚠️ Disclaimer **This is a deception/honeypot system.** Deploy in isolated environments and monitor carefully for security events. diff --git a/img/krawl-logo.jpg b/img/krawl-logo.jpg new file mode 100644 index 0000000..ea15908 Binary files /dev/null and b/img/krawl-logo.jpg differ diff --git a/kubernetes/krawl-all-in-one-deploy.yaml b/kubernetes/krawl-all-in-one-deploy.yaml new file mode 100644 index 0000000..0362220 --- /dev/null +++ b/kubernetes/krawl-all-in-one-deploy.yaml @@ -0,0 +1,372 @@ +--- +apiVersion: v1 +kind: Namespace +metadata: + name: krawl-system +--- +apiVersion: v1 +kind: ConfigMap +metadata: + name: krawl-config + namespace: krawl-system +data: + PORT: "5000" + DELAY: "100" + LINKS_MIN_LENGTH: "5" + LINKS_MAX_LENGTH: "15" + LINKS_MIN_PER_PAGE: "10" + LINKS_MAX_PER_PAGE: "15" + MAX_COUNTER: "10" + CANARY_TOKEN_TRIES: "10" + PROBABILITY_ERROR_CODES: "0" +# CANARY_TOKEN_URL: set-your-canary-token-url-here +--- +apiVersion: v1 +kind: ConfigMap +metadata: + name: krawl-wordlists + namespace: krawl-system +data: + wordlists.json: | + { + "usernames": { + "prefixes": [ + "admin", + "user", + "developer", + "root", + "system", + "db", + "api", + "service", + "deploy", + "test", + "prod", + "backup", + "monitor", + "jenkins", + "webapp" + ], + "suffixes": [ + "", + "_prod", + "_dev", + "_test", + "123", + "2024", + "_backup", + "_admin", + "01", + "02", + "_user", + "_service", + "_api" + ] + }, + "passwords": { + "prefixes": [ + "P@ssw0rd", + "Passw0rd", + "Admin", + "Secret", + "Welcome", + "System", + "Database", + "Secure", + "Master", + "Root" + ], + "simple": [ + "test", + "demo", + "temp", + "change", + "password", + "admin", + "letmein", + "welcome", + "default", + "sample" + ] + }, + "emails": { + "domains": [ + "example.com", + "company.com", + "localhost.com", + "test.com", + "domain.com", + "corporate.com", + "internal.net", + "enterprise.com", + "business.org" + ] + }, + "api_keys": { + "prefixes": [ + "sk_live_", + "sk_test_", + "api_", + "key_", + "token_", + "access_", + "secret_", + "prod_", + "" + ] + }, + "databases": { + "names": [ + "production", + "prod_db", + "main_db", + "app_database", + "users_db", + "customer_data", + "analytics", + "staging_db", + "dev_database", + "wordpress", + "ecommerce", + "crm_db", + "inventory" + ], + "hosts": [ + "localhost", + "db.internal", + "mysql.local", + "postgres.internal", + "127.0.0.1", + "db-server-01", + "database.prod", + "sql.company.com" + ] + }, + "applications": { + "names": [ + "WebApp", + "API Gateway", + "Dashboard", + "Admin Panel", + "CMS", + "Portal", + "Manager", + "Console", + "Control Panel", + "Backend" + ] + }, + "users": { + "roles": [ + "Administrator", + "Developer", + "Manager", + "User", + "Guest", + "Moderator", + "Editor", + "Viewer", + "Analyst", + "Support" + ] + }, + "directory_listing": { + "files": [ + "admin.txt", + "test.exe", + "backup.sql", + "database.sql", + "db_backup.sql", + "dump.sql", + "config.php", + "credentials.txt", + "passwords.txt", + "users.csv", + ".env", + "id_rsa", + "id_rsa.pub", + "private_key.pem", + "api_keys.json", + "secrets.yaml", + "admin_notes.txt", + "settings.ini", + "database.yml", + "wp-config.php", + ".htaccess", + "server.key", + "cert.pem", + "shadow.bak", + "passwd.old" + ], + "directories": [ + "uploads/", + "backups/", + "logs/", + "temp/", + "cache/", + "private/", + "config/", + "admin/", + "database/", + "backup/", + "old/", + "archive/", + ".git/", + "keys/", + "credentials/" + ] + }, + "error_codes": [ + 400, + 401, + 403, + 404, + 500, + 502, + 503 + ] + } +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: krawl-server + namespace: krawl-system + labels: + app: krawl-server +spec: + replicas: 1 + selector: + matchLabels: + app: krawl-server + template: + metadata: + labels: + app: krawl-server + spec: + containers: + - name: krawl + image: ghcr.io/blessedrebus/krawl:latest + imagePullPolicy: Always + ports: + - containerPort: 5000 + name: http + protocol: TCP + envFrom: + - configMapRef: + name: krawl-config + volumeMounts: + - name: wordlists + mountPath: /app/wordlists.json + subPath: wordlists.json + readOnly: true + resources: + requests: + memory: "64Mi" + cpu: "100m" + limits: + memory: "256Mi" + cpu: "500m" + volumes: + - name: wordlists + configMap: + name: krawl-wordlists +--- +apiVersion: v1 +kind: Service +metadata: + name: krawl-server + namespace: krawl-system + labels: + app: krawl-server +spec: + type: LoadBalancer + ports: + - port: 5000 + targetPort: 5000 + protocol: TCP + name: http + selector: + app: krawl-server +--- +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + name: krawl-ingress + namespace: krawl-system + annotations: + nginx.ingress.kubernetes.io/rewrite-target: / +spec: + ingressClassName: nginx + rules: + - host: krawl.example.com # Change to your domain + http: + paths: + - path: / + pathType: Prefix + backend: + service: + name: krawl-server + port: + number: 5000 + # tls: + # - hosts: + # - krawl.example.com + # secretName: krawl-tls +--- +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: krawl-network-policy + namespace: krawl-system +spec: + podSelector: + matchLabels: + app: krawl-server + policyTypes: + - Ingress + - Egress + ingress: + - from: + - podSelector: {} + - namespaceSelector: {} + - ipBlock: + cidr: 0.0.0.0/0 + ports: + - protocol: TCP + port: 5000 + egress: + - to: + - namespaceSelector: {} + - ipBlock: + cidr: 0.0.0.0/0 + ports: + - protocol: TCP + - protocol: UDP +--- +# Optional: HorizontalPodAutoscaler for auto-scaling +apiVersion: autoscaling/v2 +kind: HorizontalPodAutoscaler +metadata: + name: krawl-hpa + namespace: krawl-system +spec: + scaleTargetRef: + apiVersion: apps/v1 + kind: Deployment + name: krawl-server + minReplicas: 1 + maxReplicas: 5 + metrics: + - type: Resource + resource: + name: cpu + target: + type: Utilization + averageUtilization: 70 + - type: Resource + resource: + name: memory + target: + type: Utilization + averageUtilization: 80 diff --git a/manifests/configmap.yaml b/kubernetes/manifests/configmap.yaml similarity index 100% rename from manifests/configmap.yaml rename to kubernetes/manifests/configmap.yaml diff --git a/manifests/deployment.yaml b/kubernetes/manifests/deployment.yaml similarity index 100% rename from manifests/deployment.yaml rename to kubernetes/manifests/deployment.yaml diff --git a/manifests/hpa.yaml b/kubernetes/manifests/hpa.yaml similarity index 100% rename from manifests/hpa.yaml rename to kubernetes/manifests/hpa.yaml diff --git a/manifests/ingress.yaml b/kubernetes/manifests/ingress.yaml similarity index 100% rename from manifests/ingress.yaml rename to kubernetes/manifests/ingress.yaml diff --git a/manifests/kustomization.yaml b/kubernetes/manifests/kustomization.yaml similarity index 100% rename from manifests/kustomization.yaml rename to kubernetes/manifests/kustomization.yaml diff --git a/manifests/namespace.yaml b/kubernetes/manifests/namespace.yaml similarity index 100% rename from manifests/namespace.yaml rename to kubernetes/manifests/namespace.yaml diff --git a/manifests/network-policy.yaml b/kubernetes/manifests/network-policy.yaml similarity index 100% rename from manifests/network-policy.yaml rename to kubernetes/manifests/network-policy.yaml diff --git a/manifests/service.yaml b/kubernetes/manifests/service.yaml similarity index 100% rename from manifests/service.yaml rename to kubernetes/manifests/service.yaml diff --git a/manifests/wordlists-configmap.yaml b/kubernetes/manifests/wordlists-configmap.yaml similarity index 100% rename from manifests/wordlists-configmap.yaml rename to kubernetes/manifests/wordlists-configmap.yaml