diff --git a/.github/workflows/update_patterns.yml b/.github/workflows/update_patterns.yml
index 59d9a29..52349a2 100644
--- a/.github/workflows/update_patterns.yml
+++ b/.github/workflows/update_patterns.yml
@@ -54,13 +54,8 @@ jobs:
- name: 🔄 Convert OWASP to HAProxy WAF
run: python json2haproxy.py
- - name: 🔄 Generate Bad Bot Blockers (Placeholder - Provide badbots.py)
- run: |
- # Placeholder: Replace this with your actual badbots.py script.
- # Assuming badbots.py generates files in waf_patterns/
- # Example (if badbots.py creates nginx/bots.conf):
- # python badbots.py
- echo "Placeholder for badbots.py execution"
+ - name: 🔄 Generate Bad Bot Blockers
+ run: python badbots.py
- name: 🚀 Commit and Push Changes (if any)
run: |
@@ -98,6 +93,7 @@ jobs:
- name: 🚀 Create GitHub Release (if previous steps succeeded)
+ id: create_release
if: success() # Only create release if previous steps were successful
uses: actions/create-release@v1
env:
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index c2e3007..bb885c5 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -1 +1,109 @@
-You can send a PR any time.
+# Contributing to Patterns
+
+Thank you for your interest in contributing to the Patterns project! We appreciate your help in making this project better.
+
+## How to Contribute
+
+### Reporting Issues
+
+If you find a bug or have a suggestion for improvement:
+
+1. Check if the issue already exists in the [Issues](https://github.com/fabriziosalmi/patterns/issues) section.
+2. If not, create a new issue with a clear title and description.
+3. Include relevant details such as:
+ - Steps to reproduce (for bugs)
+ - Expected vs. actual behavior
+ - Your environment (OS, Python version, web server type)
+
+### Submitting Pull Requests
+
+We welcome pull requests! Here's how to submit one:
+
+1. **Fork the Repository**
+ ```bash
+ git clone https://github.com/YOUR_USERNAME/patterns.git
+ cd patterns
+ ```
+
+2. **Create a Feature Branch**
+
+ Use descriptive branch names following this convention:
+ - `feature/description` - For new features
+ - `fix/description` - For bug fixes
+ - `docs/description` - For documentation changes
+ - `refactor/description` - For code refactoring
+
+ Example:
+ ```bash
+ git checkout -b feature/add-caddy-support
+ ```
+
+3. **Make Your Changes**
+ - Write clear, concise commit messages
+ - Follow the existing code style and conventions
+ - Add comments where necessary
+ - Update documentation if you're changing functionality
+
+4. **Test Your Changes**
+
+ Before submitting, ensure your code works correctly:
+ ```bash
+ # Install dependencies
+ pip install -r requirements.txt
+
+ # Test the OWASP scraper
+ python owasp2json.py
+
+ # Test the converters
+ python json2nginx.py
+ python json2apache.py
+ python json2traefik.py
+ python json2haproxy.py
+
+ # Test bad bot generation
+ python badbots.py
+ ```
+
+ For web server specific testing, check the respective workflow files in `.github/workflows/`.
+
+5. **Commit and Push**
+ ```bash
+ git add .
+ git commit -m "feat: add support for Caddy web server"
+ git push origin feature/add-caddy-support
+ ```
+
+6. **Open a Pull Request**
+ - Go to the original repository on GitHub
+ - Click "New Pull Request"
+ - Select your branch
+ - Provide a clear title and description of your changes
+ - Reference any related issues
+
+## Code Style Guidelines
+
+- Use Python 3.11 or higher
+- Follow PEP 8 style guidelines
+- Use meaningful variable and function names
+- Add docstrings to functions and classes
+- Keep functions focused and modular
+- Handle errors gracefully with try-except blocks
+
+## Adding Support for New Web Servers
+
+If you want to add support for a new web server:
+
+1. Create a new converter script: `json2WEBSERVER.py`
+2. Create output directory: `waf_patterns/WEBSERVER/`
+3. Add README.md with integration instructions
+4. Update the main README.md to include the new web server
+5. Update the GitHub Actions workflow to include the new converter
+6. Add example configurations
+
+## Questions?
+
+If you have questions about contributing, feel free to:
+- Open an issue for discussion
+- Contact the maintainers
+
+Thank you for contributing!
diff --git a/README.md b/README.md
index ee6756b..b64e2d2 100644
--- a/README.md
+++ b/README.md
@@ -36,14 +36,15 @@ patterns/
│ ├── apache/ # Apache WAF configs (ModSecurity)
│ ├── traefik/ # Traefik WAF configs
│ └── haproxy/ # HAProxy WAF configs
-│── import_apache_waf.py
-│── import_haproxy_waf.py
-│── import_nginx_waf.py
-│── import_traefik_waf.py
-├── owasp.py # 🕵️ OWASP scraper (fetch CRS rules)
-├── owasp2nginx.py # 🔄 Convert OWASP JSON to Nginx WAF configs
-├── owasp2apache.py # 🔄 Convert OWASP JSON to Apache ModSecurity configs
-├── owasp2haproxy.py # 🔄 Convert OWASP JSON to HAProxy WAF configs
+├── import_apache_waf.py # 📥 Import Apache WAF configurations
+├── import_haproxy_waf.py # 📥 Import HAProxy WAF configurations
+├── import_nginx_waf.py # 📥 Import Nginx WAF configurations
+├── import_traefik_waf.py # 📥 Import Traefik WAF configurations
+├── owasp2json.py # 🕵️ OWASP scraper (fetch CRS rules)
+├── json2nginx.py # 🔄 Convert OWASP JSON to Nginx WAF configs
+├── json2apache.py # 🔄 Convert OWASP JSON to Apache ModSecurity configs
+├── json2traefik.py # 🔄 Convert OWASP JSON to Traefik WAF configs
+├── json2haproxy.py # 🔄 Convert OWASP JSON to HAProxy WAF configs
├── badbots.py # 🤖 Generate WAF configs to block bad bots
├── requirements.txt # 📄 Required dependencies
└── .github/workflows/ # 🤖 GitHub Actions for automation
@@ -54,14 +55,14 @@ patterns/
## 🛠️ How It Works
### 🔹 1. Scraping OWASP Rules
-- **`owasp.py`** scrapes the latest OWASP CRS patterns from GitHub.
+- **`owasp2json.py`** scrapes the latest OWASP CRS patterns from GitHub.
- Extracts **SQLi, XSS, RCE, LFI** patterns from OWASP CRS `.conf` files.
### 🔹 2. Generating WAF Configs for Each Platform
-- **`owasp2nginx.py`** – Generates **Nginx WAF** configurations.
-- **`owasp2apache.py`** – Outputs **Apache ModSecurity** rules.
-- **`owasp2traefik.py`** – Creates **Traefik WAF** rules.
-- **`owasp2haproxy.py`** – Builds **HAProxy ACL** files.
+- **`json2nginx.py`** – Generates **Nginx WAF** configurations.
+- **`json2apache.py`** – Outputs **Apache ModSecurity** rules.
+- **`json2traefik.py`** – Creates **Traefik WAF** rules.
+- **`json2haproxy.py`** – Builds **HAProxy ACL** files.
### 🔹 3. Bad Bot/User-Agent Detection
- **`badbots.py`** fetches public bot lists and generates bot-blocking configs.
@@ -94,11 +95,11 @@ pip install -r requirements.txt
**3. Run Manually (Optional):**
```bash
-python owasp.py
-python owasp2nginx.py
-python owasp2apache.py
-python owasp2haproxy.py
-python owasp2traefik.py
+python owasp2json.py
+python json2nginx.py
+python json2apache.py
+python json2haproxy.py
+python json2traefik.py
python badbots.py
```
@@ -169,9 +170,9 @@ See the [LICENSE](LICENSE) file for details.
---
-## Others projects
+## Other Projects
-If You like my projects, you may also like these ones:
+If you like this project, you may also like these:
- [caddy-waf](https://github.com/fabriziosalmi/caddy-waf) Caddy WAF (Regex Rules, IP and DNS filtering, Rate Limiting, GeoIP, Tor, Anomaly Detection)
- [blacklists](https://github.com/fabriziosalmi/blacklists) Hourly updated domains blacklist 🚫
diff --git a/waf_patterns/apache/README.md b/waf_patterns/apache/README.md
index 8b13789..6b7ee8d 100644
--- a/waf_patterns/apache/README.md
+++ b/waf_patterns/apache/README.md
@@ -1 +1,117 @@
+# Apache ModSecurity WAF Configuration
+This directory contains Apache ModSecurity WAF configuration files generated from OWASP CRS rules.
+You can include these files in your existing Apache configuration to enhance security.
+
+## Prerequisites
+
+- Apache HTTP Server (2.4 or higher)
+- ModSecurity module installed and enabled
+- Core Rule Set (CRS) base configuration
+
+## Installation
+
+### Ubuntu/Debian
+```bash
+sudo apt-get update
+sudo apt-get install libapache2-mod-security2
+sudo a2enmod security2
+sudo systemctl restart apache2
+```
+
+### CentOS/RHEL
+```bash
+sudo yum install mod_security
+sudo systemctl restart httpd
+```
+
+## Usage
+
+1. Copy the generated configuration files to your Apache configuration directory:
+ ```bash
+ sudo cp waf_patterns/apache/*.conf /etc/apache2/modsecurity.d/
+ # or for CentOS/RHEL:
+ # sudo cp waf_patterns/apache/*.conf /etc/httpd/modsecurity.d/
+ ```
+
+2. Include the configuration files in your Apache configuration.
+
+ Edit `/etc/apache2/mods-enabled/security2.conf` (Ubuntu/Debian) or `/etc/httpd/conf.d/mod_security.conf` (CentOS/RHEL):
+ ```apache
+
+ Include /etc/apache2/modsecurity.d/*.conf
+
+ ```
+
+3. Test the configuration:
+ ```bash
+ # Ubuntu/Debian
+ sudo apache2ctl configtest
+
+ # CentOS/RHEL
+ sudo httpd -t
+ ```
+
+4. Reload Apache to apply the changes:
+ ```bash
+ # Ubuntu/Debian
+ sudo systemctl reload apache2
+
+ # CentOS/RHEL
+ sudo systemctl reload httpd
+ ```
+
+## Configuration Details
+
+The generated rules include:
+- **SQL Injection (SQLi)** detection patterns
+- **Cross-Site Scripting (XSS)** prevention rules
+- **Remote Code Execution (RCE)** blocking
+- **Local File Inclusion (LFI)** protection
+- **Bad Bot/User-Agent** blocking
+
+## Customization
+
+You can adjust the severity and actions for each rule by modifying the configuration files.
+Common actions include:
+- `deny` - Block the request
+- `log` - Log the event
+- `status:403` - Return HTTP 403 Forbidden
+
+## Troubleshooting
+
+### Check ModSecurity is loaded
+```bash
+# Ubuntu/Debian
+apache2ctl -M | grep security
+
+# CentOS/RHEL
+httpd -M | grep security
+```
+
+### View ModSecurity logs
+```bash
+# Ubuntu/Debian
+sudo tail -f /var/log/apache2/modsec_audit.log
+
+# CentOS/RHEL
+sudo tail -f /var/log/httpd/modsec_audit.log
+```
+
+### Test with a sample attack
+```bash
+curl "http://yourserver.com/?id=1' OR '1'='1"
+# Should return 403 Forbidden if WAF is working
+```
+
+## Notes
+
+- Rules are updated daily via GitHub Actions
+- Blocked requests return a `403 Forbidden` response by default
+- Review the ModSecurity documentation for advanced configuration options
+
+## Resources
+
+- [ModSecurity Documentation](https://github.com/SpiderLabs/ModSecurity)
+- [OWASP CRS](https://coreruleset.org/)
+- [Apache ModSecurity Module](https://modsecurity.org/)
diff --git a/waf_patterns/haproxy/README.md b/waf_patterns/haproxy/README.md
index 8b13789..f96e622 100644
--- a/waf_patterns/haproxy/README.md
+++ b/waf_patterns/haproxy/README.md
@@ -1 +1,183 @@
+# HAProxy WAF Configuration
+This directory contains HAProxy WAF configuration files generated from OWASP CRS rules.
+You can include these ACL (Access Control List) files in your HAProxy configuration to enhance security.
+
+## Prerequisites
+
+- HAProxy 2.0 or higher
+- Basic understanding of HAProxy ACLs and rules
+
+## Configuration Files
+
+The generated files include:
+- ACL files with pattern matching rules
+- Request filtering configurations
+- Bad bot/User-Agent blocking lists
+
+## Usage
+
+1. Copy the generated ACL files to your HAProxy configuration directory:
+ ```bash
+ sudo cp waf_patterns/haproxy/*.acl /etc/haproxy/
+ ```
+
+2. Include the ACL files in your HAProxy configuration.
+
+ Edit `/etc/haproxy/haproxy.cfg`:
+ ```haproxy
+ frontend http-in
+ bind *:80
+
+ # Load WAF ACL files
+ acl is_sql_injection path_reg -i -f /etc/haproxy/sqli_patterns.acl
+ acl is_xss_attack path_reg -i -f /etc/haproxy/xss_patterns.acl
+ acl is_bad_bot hdr_reg(User-Agent) -i -f /etc/haproxy/bad_bots.acl
+
+ # Block malicious requests
+ http-request deny if is_sql_injection
+ http-request deny if is_xss_attack
+ http-request deny if is_bad_bot
+
+ # Default backend
+ default_backend web_servers
+
+ backend web_servers
+ balance roundrobin
+ server web1 10.0.0.1:80 check
+ server web2 10.0.0.2:80 check
+ ```
+
+3. Test the configuration:
+ ```bash
+ sudo haproxy -c -f /etc/haproxy/haproxy.cfg
+ ```
+
+4. Reload HAProxy to apply the changes:
+ ```bash
+ sudo systemctl reload haproxy
+ # or
+ sudo service haproxy reload
+ ```
+
+## Advanced Configuration
+
+### Logging Blocked Requests
+
+Add logging for better visibility:
+
+```haproxy
+frontend http-in
+ bind *:80
+
+ # ... ACL definitions ...
+
+ # Log blocked requests
+ http-request capture req.hdr(User-Agent) len 200
+ http-request deny deny_status 403 if is_sql_injection
+ log-format "%ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %CC %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r"
+```
+
+### Custom Error Pages
+
+Return custom error pages for blocked requests:
+
+```haproxy
+frontend http-in
+ bind *:80
+
+ # ... ACL definitions ...
+
+ # Return custom error page
+ http-request deny deny_status 403 if is_sql_injection
+ errorfile 403 /etc/haproxy/errors/403.http
+```
+
+### Rate Limiting
+
+Combine with rate limiting for additional protection:
+
+```haproxy
+frontend http-in
+ bind *:80
+
+ # Track request rate
+ stick-table type ip size 100k expire 30s store http_req_rate(10s)
+ http-request track-sc0 src
+
+ # Deny if rate limit exceeded
+ http-request deny deny_status 429 if { sc_http_req_rate(0) gt 100 }
+
+ # ... WAF ACLs ...
+```
+
+## Testing
+
+### Test SQL Injection Protection
+```bash
+curl "http://yourserver.com/?id=1' OR '1'='1"
+# Should return 403 Forbidden
+```
+
+### Test XSS Protection
+```bash
+curl "http://yourserver.com/?q="
+# Should return 403 Forbidden
+```
+
+### Test Bad Bot Blocking
+```bash
+curl -H "User-Agent: AhrefsBot" http://yourserver.com
+# Should return 403 Forbidden
+```
+
+## Monitoring
+
+### Check HAProxy Stats
+```bash
+# Enable stats in haproxy.cfg
+listen stats
+ bind *:8404
+ stats enable
+ stats uri /stats
+ stats refresh 10s
+```
+
+Visit `http://yourserver:8404/stats` to view statistics.
+
+### View Logs
+```bash
+sudo tail -f /var/log/haproxy.log
+```
+
+## Performance Considerations
+
+- ACL pattern matching is highly efficient in HAProxy
+- Use regular expressions sparingly for better performance
+- Consider using stick tables for rate limiting
+- Monitor CPU and memory usage under load
+- Test thoroughly before deploying to production
+
+## Configuration Details
+
+The ACL files protect against:
+- **SQL Injection (SQLi)** - Common SQL injection patterns
+- **Cross-Site Scripting (XSS)** - JavaScript injection attempts
+- **Remote Code Execution (RCE)** - Command injection patterns
+- **Local File Inclusion (LFI)** - Path traversal attempts
+- **Bad Bots** - Known malicious crawlers and scrapers
+
+## Notes
+
+- Rules are updated daily via GitHub Actions
+- Blocked requests return `403 Forbidden` by default
+- ACLs are case-insensitive (`-i` flag)
+- Regular expressions are used for pattern matching (`-f` for file-based ACLs)
+- Compatible with HAProxy 2.0 and higher
+
+## Resources
+
+- [HAProxy Documentation](https://www.haproxy.org/#docs)
+- [HAProxy ACL Guide](https://www.haproxy.com/documentation/hapee/latest/onepage/#7)
+- [OWASP CRS](https://coreruleset.org/)
+- [HAProxy Configuration Manual](http://cbonte.github.io/haproxy-dconv/)
diff --git a/waf_patterns/traefik/README.md b/waf_patterns/traefik/README.md
index 8b13789..4781bc1 100644
--- a/waf_patterns/traefik/README.md
+++ b/waf_patterns/traefik/README.md
@@ -1 +1,136 @@
+# Traefik WAF Configuration
+This directory contains Traefik WAF configuration files generated from OWASP CRS rules.
+You can use these middleware configurations to enhance security in your Traefik setup.
+
+## Prerequisites
+
+- Traefik v2.x or higher
+- Basic understanding of Traefik middleware
+
+## Configuration Files
+
+The generated configuration includes:
+- Middleware definitions for request filtering
+- Regular expression patterns for attack detection
+- Bad bot/User-Agent blocking rules
+
+## Usage
+
+### Option 1: File Provider (Recommended)
+
+1. Copy the generated configuration files to your Traefik configuration directory:
+ ```bash
+ cp waf_patterns/traefik/*.toml /etc/traefik/dynamic/
+ # or to your custom config directory
+ ```
+
+2. Configure Traefik to load dynamic configuration from files.
+
+ In your `traefik.yml` or `traefik.toml`:
+ ```yaml
+ providers:
+ file:
+ directory: "/etc/traefik/dynamic"
+ watch: true
+ ```
+
+3. Apply the middleware to your routes by referencing it in your service configuration:
+ ```yaml
+ http:
+ routers:
+ my-router:
+ rule: "Host(`example.com`)"
+ service: my-service
+ middlewares:
+ - waf-middleware
+ ```
+
+### Option 2: Docker Labels
+
+If you're using Docker, you can apply the middleware via labels:
+
+```yaml
+services:
+ my-service:
+ image: my-app:latest
+ labels:
+ - "traefik.enable=true"
+ - "traefik.http.routers.my-router.rule=Host(`example.com`)"
+ - "traefik.http.routers.my-router.middlewares=waf-middleware@file"
+```
+
+### Option 3: Kubernetes IngressRoute
+
+For Kubernetes deployments:
+
+```yaml
+apiVersion: traefik.containo.us/v1alpha1
+kind: Middleware
+metadata:
+ name: waf-middleware
+spec:
+ plugin:
+ # Reference your WAF plugin configuration here
+```
+
+## Configuration Details
+
+The middleware includes protection against:
+- **SQL Injection (SQLi)** attacks
+- **Cross-Site Scripting (XSS)** attempts
+- **Remote Code Execution (RCE)** patterns
+- **Local File Inclusion (LFI)** attempts
+- **Malicious bots and crawlers**
+
+## Testing
+
+Test the WAF is working by sending a malicious request:
+
+```bash
+curl -H "User-Agent: AhrefsBot" http://yourserver.com
+# Should be blocked if bot protection is working
+
+curl "http://yourserver.com/?id=1' OR '1'='1"
+# Should be blocked if SQLi protection is working
+```
+
+## Monitoring
+
+Monitor blocked requests in Traefik logs:
+
+```bash
+# Docker
+docker logs traefik 2>&1 | grep -i "blocked\|forbidden"
+
+# Standard installation
+tail -f /var/log/traefik/access.log | grep -i "403"
+```
+
+## Customization
+
+You can customize the middleware behavior by:
+1. Editing the generated `.toml` files
+2. Adjusting regex patterns for your specific needs
+3. Modifying response codes and error pages
+4. Adding custom headers for blocked requests
+
+## Performance Considerations
+
+- Regular expression matching can impact performance under high load
+- Consider using caching middleware in combination with WAF
+- Monitor CPU usage and adjust rules if needed
+- Use Traefik's built-in rate limiting for additional protection
+
+## Notes
+
+- Rules are updated daily via GitHub Actions
+- Blocked requests typically return `403 Forbidden` or `400 Bad Request`
+- Middleware is applied at the router level
+- Compatible with other Traefik middlewares (chain them as needed)
+
+## Resources
+
+- [Traefik Documentation](https://doc.traefik.io/traefik/)
+- [Traefik Middleware](https://doc.traefik.io/traefik/middlewares/overview/)
+- [OWASP CRS](https://coreruleset.org/)