docs: Fix script names, improve CONTRIBUTING, add WAF READMEs, fix workflow

Co-authored-by: fabriziosalmi <1569108+fabriziosalmi@users.noreply.github.com>

.github/workflows/update_patterns.yml

@@ -54,13 +54,8 @@ jobs:
     - name: 🔄 Convert OWASP to HAProxy WAF
       run: python json2haproxy.py
 
-    - name: 🔄 Generate Bad Bot Blockers (Placeholder - Provide badbots.py)
+    - name: 🔄 Generate Bad Bot Blockers
-      run: |
+      run: python badbots.py
-        # Placeholder:  Replace this with your actual badbots.py script.
-        # Assuming badbots.py generates files in waf_patterns/
-        # Example (if badbots.py creates nginx/bots.conf):
-        # python badbots.py
-        echo "Placeholder for badbots.py execution"
 
     - name: 🚀 Commit and Push Changes (if any)
       run: |
@@ -98,6 +93,7 @@ jobs:
 
 
     - name: 🚀 Create GitHub Release (if previous steps succeeded)
+      id: create_release
       if: success()  # Only create release if previous steps were successful
       uses: actions/create-release@v1
       env:

CONTRIBUTING.md

@@ -1 +1,109 @@
-You can send a PR any time.
+# Contributing to Patterns
+
+Thank you for your interest in contributing to the Patterns project! We appreciate your help in making this project better.
+
+## How to Contribute
+
+### Reporting Issues
+
+If you find a bug or have a suggestion for improvement:
+
+1. Check if the issue already exists in the [Issues](https://github.com/fabriziosalmi/patterns/issues) section.
+2. If not, create a new issue with a clear title and description.
+3. Include relevant details such as:
+   - Steps to reproduce (for bugs)
+   - Expected vs. actual behavior
+   - Your environment (OS, Python version, web server type)
+
+### Submitting Pull Requests
+
+We welcome pull requests! Here's how to submit one:
+
+1. **Fork the Repository**
+   ```bash
+   git clone https://github.com/YOUR_USERNAME/patterns.git
+   cd patterns
+   ```
+
+2. **Create a Feature Branch**
+   
+   Use descriptive branch names following this convention:
+   - `feature/description` - For new features
+   - `fix/description` - For bug fixes
+   - `docs/description` - For documentation changes
+   - `refactor/description` - For code refactoring
+   
+   Example:
+   ```bash
+   git checkout -b feature/add-caddy-support
+   ```
+
+3. **Make Your Changes**
+   - Write clear, concise commit messages
+   - Follow the existing code style and conventions
+   - Add comments where necessary
+   - Update documentation if you're changing functionality
+
+4. **Test Your Changes**
+   
+   Before submitting, ensure your code works correctly:
+   ```bash
+   # Install dependencies
+   pip install -r requirements.txt
+   
+   # Test the OWASP scraper
+   python owasp2json.py
+   
+   # Test the converters
+   python json2nginx.py
+   python json2apache.py
+   python json2traefik.py
+   python json2haproxy.py
+   
+   # Test bad bot generation
+   python badbots.py
+   ```
+   
+   For web server specific testing, check the respective workflow files in `.github/workflows/`.
+
+5. **Commit and Push**
+   ```bash
+   git add .
+   git commit -m "feat: add support for Caddy web server"
+   git push origin feature/add-caddy-support
+   ```
+
+6. **Open a Pull Request**
+   - Go to the original repository on GitHub
+   - Click "New Pull Request"
+   - Select your branch
+   - Provide a clear title and description of your changes
+   - Reference any related issues
+
+## Code Style Guidelines
+
+- Use Python 3.11 or higher
+- Follow PEP 8 style guidelines
+- Use meaningful variable and function names
+- Add docstrings to functions and classes
+- Keep functions focused and modular
+- Handle errors gracefully with try-except blocks
+
+## Adding Support for New Web Servers
+
+If you want to add support for a new web server:
+
+1. Create a new converter script: `json2WEBSERVER.py`
+2. Create output directory: `waf_patterns/WEBSERVER/`
+3. Add README.md with integration instructions
+4. Update the main README.md to include the new web server
+5. Update the GitHub Actions workflow to include the new converter
+6. Add example configurations
+
+## Questions?
+
+If you have questions about contributing, feel free to:
+- Open an issue for discussion
+- Contact the maintainers
+
+Thank you for contributing!

README.md

@@ -36,14 +36,15 @@ patterns/
 │   ├── apache/             # Apache WAF configs (ModSecurity)
 │   ├── traefik/            # Traefik WAF configs
 │   └── haproxy/            # HAProxy WAF configs
-│── import_apache_waf.py
+├── import_apache_waf.py    # 📥 Import Apache WAF configurations
-│── import_haproxy_waf.py
+├── import_haproxy_waf.py   # 📥 Import HAProxy WAF configurations
-│── import_nginx_waf.py
+├── import_nginx_waf.py     # 📥 Import Nginx WAF configurations
-│── import_traefik_waf.py
+├── import_traefik_waf.py   # 📥 Import Traefik WAF configurations
-├── owasp.py                # 🕵️ OWASP scraper (fetch CRS rules)
+├── owasp2json.py           # 🕵️ OWASP scraper (fetch CRS rules)
-├── owasp2nginx.py          # 🔄 Convert OWASP JSON to Nginx WAF configs
+├── json2nginx.py           # 🔄 Convert OWASP JSON to Nginx WAF configs
-├── owasp2apache.py         # 🔄 Convert OWASP JSON to Apache ModSecurity configs
+├── json2apache.py          # 🔄 Convert OWASP JSON to Apache ModSecurity configs
-├── owasp2haproxy.py        # 🔄 Convert OWASP JSON to HAProxy WAF configs
+├── json2traefik.py         # 🔄 Convert OWASP JSON to Traefik WAF configs
+├── json2haproxy.py         # 🔄 Convert OWASP JSON to HAProxy WAF configs
 ├── badbots.py              # 🤖 Generate WAF configs to block bad bots
 ├── requirements.txt        # 📄 Required dependencies
 └── .github/workflows/      # 🤖 GitHub Actions for automation
@@ -54,14 +55,14 @@ patterns/
 
 ## 🛠️ How It Works  
 ### 🔹 1. Scraping OWASP Rules  
-- **`owasp.py`** scrapes the latest OWASP CRS patterns from GitHub.  
+- **`owasp2json.py`** scrapes the latest OWASP CRS patterns from GitHub.  
 - Extracts **SQLi, XSS, RCE, LFI** patterns from OWASP CRS `.conf` files.  
 
 ### 🔹 2. Generating WAF Configs for Each Platform  
-- **`owasp2nginx.py`** – Generates **Nginx WAF** configurations.  
+- **`json2nginx.py`** – Generates **Nginx WAF** configurations.  
-- **`owasp2apache.py`** – Outputs **Apache ModSecurity** rules.  
+- **`json2apache.py`** – Outputs **Apache ModSecurity** rules.  
-- **`owasp2traefik.py`** – Creates **Traefik WAF** rules.  
+- **`json2traefik.py`** – Creates **Traefik WAF** rules.  
-- **`owasp2haproxy.py`** – Builds **HAProxy ACL** files.  
+- **`json2haproxy.py`** – Builds **HAProxy ACL** files.  
 
 ### 🔹 3. Bad Bot/User-Agent Detection  
 - **`badbots.py`** fetches public bot lists and generates bot-blocking configs.  
@@ -94,11 +95,11 @@ pip install -r requirements.txt
 
 **3. Run Manually (Optional):**  
 ```bash
-python owasp.py
+python owasp2json.py
-python owasp2nginx.py
+python json2nginx.py
-python owasp2apache.py
+python json2apache.py
-python owasp2haproxy.py
+python json2haproxy.py
-python owasp2traefik.py
+python json2traefik.py
 python badbots.py
 ```
 
@@ -169,9 +170,9 @@ See the [LICENSE](LICENSE) file for details.
 
 ---
 
-## Others projects
+## Other Projects
 
-If You like my projects, you may also like these ones:
+If you like this project, you may also like these:
 
 - [caddy-waf](https://github.com/fabriziosalmi/caddy-waf) Caddy WAF (Regex Rules, IP and DNS filtering, Rate Limiting, GeoIP, Tor, Anomaly Detection) 
 - [blacklists](https://github.com/fabriziosalmi/blacklists) Hourly updated domains blacklist 🚫 

waf_patterns/apache/README.md

@@ -1 +1,117 @@
+# Apache ModSecurity WAF Configuration
 
+This directory contains Apache ModSecurity WAF configuration files generated from OWASP CRS rules.
+You can include these files in your existing Apache configuration to enhance security.
+
+## Prerequisites
+
+- Apache HTTP Server (2.4 or higher)
+- ModSecurity module installed and enabled
+- Core Rule Set (CRS) base configuration
+
+## Installation
+
+### Ubuntu/Debian
+```bash
+sudo apt-get update
+sudo apt-get install libapache2-mod-security2
+sudo a2enmod security2
+sudo systemctl restart apache2
+```
+
+### CentOS/RHEL
+```bash
+sudo yum install mod_security
+sudo systemctl restart httpd
+```
+
+## Usage
+
+1. Copy the generated configuration files to your Apache configuration directory:
+   ```bash
+   sudo cp waf_patterns/apache/*.conf /etc/apache2/modsecurity.d/
+   # or for CentOS/RHEL:
+   # sudo cp waf_patterns/apache/*.conf /etc/httpd/modsecurity.d/
+   ```
+
+2. Include the configuration files in your Apache configuration.
+   
+   Edit `/etc/apache2/mods-enabled/security2.conf` (Ubuntu/Debian) or `/etc/httpd/conf.d/mod_security.conf` (CentOS/RHEL):
+   ```apache
+   <IfModule security2_module>
+       Include /etc/apache2/modsecurity.d/*.conf
+   </IfModule>
+   ```
+
+3. Test the configuration:
+   ```bash
+   # Ubuntu/Debian
+   sudo apache2ctl configtest
+   
+   # CentOS/RHEL
+   sudo httpd -t
+   ```
+
+4. Reload Apache to apply the changes:
+   ```bash
+   # Ubuntu/Debian
+   sudo systemctl reload apache2
+   
+   # CentOS/RHEL
+   sudo systemctl reload httpd
+   ```
+
+## Configuration Details
+
+The generated rules include:
+- **SQL Injection (SQLi)** detection patterns
+- **Cross-Site Scripting (XSS)** prevention rules
+- **Remote Code Execution (RCE)** blocking
+- **Local File Inclusion (LFI)** protection
+- **Bad Bot/User-Agent** blocking
+
+## Customization
+
+You can adjust the severity and actions for each rule by modifying the configuration files.
+Common actions include:
+- `deny` - Block the request
+- `log` - Log the event
+- `status:403` - Return HTTP 403 Forbidden
+
+## Troubleshooting
+
+### Check ModSecurity is loaded
+```bash
+# Ubuntu/Debian
+apache2ctl -M | grep security
+
+# CentOS/RHEL
+httpd -M | grep security
+```
+
+### View ModSecurity logs
+```bash
+# Ubuntu/Debian
+sudo tail -f /var/log/apache2/modsec_audit.log
+
+# CentOS/RHEL
+sudo tail -f /var/log/httpd/modsec_audit.log
+```
+
+### Test with a sample attack
+```bash
+curl "http://yourserver.com/?id=1' OR '1'='1"
+# Should return 403 Forbidden if WAF is working
+```
+
+## Notes
+
+- Rules are updated daily via GitHub Actions
+- Blocked requests return a `403 Forbidden` response by default
+- Review the ModSecurity documentation for advanced configuration options
+
+## Resources
+
+- [ModSecurity Documentation](https://github.com/SpiderLabs/ModSecurity)
+- [OWASP CRS](https://coreruleset.org/)
+- [Apache ModSecurity Module](https://modsecurity.org/)

waf_patterns/haproxy/README.md

@@ -1 +1,183 @@
+# HAProxy WAF Configuration
 
+This directory contains HAProxy WAF configuration files generated from OWASP CRS rules.
+You can include these ACL (Access Control List) files in your HAProxy configuration to enhance security.
+
+## Prerequisites
+
+- HAProxy 2.0 or higher
+- Basic understanding of HAProxy ACLs and rules
+
+## Configuration Files
+
+The generated files include:
+- ACL files with pattern matching rules
+- Request filtering configurations
+- Bad bot/User-Agent blocking lists
+
+## Usage
+
+1. Copy the generated ACL files to your HAProxy configuration directory:
+   ```bash
+   sudo cp waf_patterns/haproxy/*.acl /etc/haproxy/
+   ```
+
+2. Include the ACL files in your HAProxy configuration.
+   
+   Edit `/etc/haproxy/haproxy.cfg`:
+   ```haproxy
+   frontend http-in
+       bind *:80
+       
+       # Load WAF ACL files
+       acl is_sql_injection path_reg -i -f /etc/haproxy/sqli_patterns.acl
+       acl is_xss_attack path_reg -i -f /etc/haproxy/xss_patterns.acl
+       acl is_bad_bot hdr_reg(User-Agent) -i -f /etc/haproxy/bad_bots.acl
+       
+       # Block malicious requests
+       http-request deny if is_sql_injection
+       http-request deny if is_xss_attack
+       http-request deny if is_bad_bot
+       
+       # Default backend
+       default_backend web_servers
+   
+   backend web_servers
+       balance roundrobin
+       server web1 10.0.0.1:80 check
+       server web2 10.0.0.2:80 check
+   ```
+
+3. Test the configuration:
+   ```bash
+   sudo haproxy -c -f /etc/haproxy/haproxy.cfg
+   ```
+
+4. Reload HAProxy to apply the changes:
+   ```bash
+   sudo systemctl reload haproxy
+   # or
+   sudo service haproxy reload
+   ```
+
+## Advanced Configuration
+
+### Logging Blocked Requests
+
+Add logging for better visibility:
+
+```haproxy
+frontend http-in
+    bind *:80
+    
+    # ... ACL definitions ...
+    
+    # Log blocked requests
+    http-request capture req.hdr(User-Agent) len 200
+    http-request deny deny_status 403 if is_sql_injection
+    log-format "%ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %CC %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r"
+```
+
+### Custom Error Pages
+
+Return custom error pages for blocked requests:
+
+```haproxy
+frontend http-in
+    bind *:80
+    
+    # ... ACL definitions ...
+    
+    # Return custom error page
+    http-request deny deny_status 403 if is_sql_injection
+    errorfile 403 /etc/haproxy/errors/403.http
+```
+
+### Rate Limiting
+
+Combine with rate limiting for additional protection:
+
+```haproxy
+frontend http-in
+    bind *:80
+    
+    # Track request rate
+    stick-table type ip size 100k expire 30s store http_req_rate(10s)
+    http-request track-sc0 src
+    
+    # Deny if rate limit exceeded
+    http-request deny deny_status 429 if { sc_http_req_rate(0) gt 100 }
+    
+    # ... WAF ACLs ...
+```
+
+## Testing
+
+### Test SQL Injection Protection
+```bash
+curl "http://yourserver.com/?id=1' OR '1'='1"
+# Should return 403 Forbidden
+```
+
+### Test XSS Protection
+```bash
+curl "http://yourserver.com/?q=<script>alert('xss')</script>"
+# Should return 403 Forbidden
+```
+
+### Test Bad Bot Blocking
+```bash
+curl -H "User-Agent: AhrefsBot" http://yourserver.com
+# Should return 403 Forbidden
+```
+
+## Monitoring
+
+### Check HAProxy Stats
+```bash
+# Enable stats in haproxy.cfg
+listen stats
+    bind *:8404
+    stats enable
+    stats uri /stats
+    stats refresh 10s
+```
+
+Visit `http://yourserver:8404/stats` to view statistics.
+
+### View Logs
+```bash
+sudo tail -f /var/log/haproxy.log
+```
+
+## Performance Considerations
+
+- ACL pattern matching is highly efficient in HAProxy
+- Use regular expressions sparingly for better performance
+- Consider using stick tables for rate limiting
+- Monitor CPU and memory usage under load
+- Test thoroughly before deploying to production
+
+## Configuration Details
+
+The ACL files protect against:
+- **SQL Injection (SQLi)** - Common SQL injection patterns
+- **Cross-Site Scripting (XSS)** - JavaScript injection attempts
+- **Remote Code Execution (RCE)** - Command injection patterns
+- **Local File Inclusion (LFI)** - Path traversal attempts
+- **Bad Bots** - Known malicious crawlers and scrapers
+
+## Notes
+
+- Rules are updated daily via GitHub Actions
+- Blocked requests return `403 Forbidden` by default
+- ACLs are case-insensitive (`-i` flag)
+- Regular expressions are used for pattern matching (`-f` for file-based ACLs)
+- Compatible with HAProxy 2.0 and higher
+
+## Resources
+
+- [HAProxy Documentation](https://www.haproxy.org/#docs)
+- [HAProxy ACL Guide](https://www.haproxy.com/documentation/hapee/latest/onepage/#7)
+- [OWASP CRS](https://coreruleset.org/)
+- [HAProxy Configuration Manual](http://cbonte.github.io/haproxy-dconv/)

waf_patterns/traefik/README.md

@@ -1 +1,136 @@
+# Traefik WAF Configuration
 
+This directory contains Traefik WAF configuration files generated from OWASP CRS rules.
+You can use these middleware configurations to enhance security in your Traefik setup.
+
+## Prerequisites
+
+- Traefik v2.x or higher
+- Basic understanding of Traefik middleware
+
+## Configuration Files
+
+The generated configuration includes:
+- Middleware definitions for request filtering
+- Regular expression patterns for attack detection
+- Bad bot/User-Agent blocking rules
+
+## Usage
+
+### Option 1: File Provider (Recommended)
+
+1. Copy the generated configuration files to your Traefik configuration directory:
+   ```bash
+   cp waf_patterns/traefik/*.toml /etc/traefik/dynamic/
+   # or to your custom config directory
+   ```
+
+2. Configure Traefik to load dynamic configuration from files.
+   
+   In your `traefik.yml` or `traefik.toml`:
+   ```yaml
+   providers:
+     file:
+       directory: "/etc/traefik/dynamic"
+       watch: true
+   ```
+
+3. Apply the middleware to your routes by referencing it in your service configuration:
+   ```yaml
+   http:
+     routers:
+       my-router:
+         rule: "Host(`example.com`)"
+         service: my-service
+         middlewares:
+           - waf-middleware
+   ```
+
+### Option 2: Docker Labels
+
+If you're using Docker, you can apply the middleware via labels:
+
+```yaml
+services:
+  my-service:
+    image: my-app:latest
+    labels:
+      - "traefik.enable=true"
+      - "traefik.http.routers.my-router.rule=Host(`example.com`)"
+      - "traefik.http.routers.my-router.middlewares=waf-middleware@file"
+```
+
+### Option 3: Kubernetes IngressRoute
+
+For Kubernetes deployments:
+
+```yaml
+apiVersion: traefik.containo.us/v1alpha1
+kind: Middleware
+metadata:
+  name: waf-middleware
+spec:
+  plugin:
+    # Reference your WAF plugin configuration here
+```
+
+## Configuration Details
+
+The middleware includes protection against:
+- **SQL Injection (SQLi)** attacks
+- **Cross-Site Scripting (XSS)** attempts
+- **Remote Code Execution (RCE)** patterns
+- **Local File Inclusion (LFI)** attempts
+- **Malicious bots and crawlers**
+
+## Testing
+
+Test the WAF is working by sending a malicious request:
+
+```bash
+curl -H "User-Agent: AhrefsBot" http://yourserver.com
+# Should be blocked if bot protection is working
+
+curl "http://yourserver.com/?id=1' OR '1'='1"
+# Should be blocked if SQLi protection is working
+```
+
+## Monitoring
+
+Monitor blocked requests in Traefik logs:
+
+```bash
+# Docker
+docker logs traefik 2>&1 | grep -i "blocked\|forbidden"
+
+# Standard installation
+tail -f /var/log/traefik/access.log | grep -i "403"
+```
+
+## Customization
+
+You can customize the middleware behavior by:
+1. Editing the generated `.toml` files
+2. Adjusting regex patterns for your specific needs
+3. Modifying response codes and error pages
+4. Adding custom headers for blocked requests
+
+## Performance Considerations
+
+- Regular expression matching can impact performance under high load
+- Consider using caching middleware in combination with WAF
+- Monitor CPU usage and adjust rules if needed
+- Use Traefik's built-in rate limiting for additional protection
+
+## Notes
+
+- Rules are updated daily via GitHub Actions
+- Blocked requests typically return `403 Forbidden` or `400 Bad Request`
+- Middleware is applied at the router level
+- Compatible with other Traefik middlewares (chain them as needed)
+
+## Resources
+
+- [Traefik Documentation](https://doc.traefik.io/traefik/)
+- [Traefik Middleware](https://doc.traefik.io/traefik/middlewares/overview/)
+- [OWASP CRS](https://coreruleset.org/)