diff --git a/.github/workflows/update_patterns.yml b/.github/workflows/update_patterns.yml index 59d9a29..52349a2 100644 --- a/.github/workflows/update_patterns.yml +++ b/.github/workflows/update_patterns.yml @@ -54,13 +54,8 @@ jobs: - name: 🔄 Convert OWASP to HAProxy WAF run: python json2haproxy.py - - name: 🔄 Generate Bad Bot Blockers (Placeholder - Provide badbots.py) - run: | - # Placeholder: Replace this with your actual badbots.py script. - # Assuming badbots.py generates files in waf_patterns/ - # Example (if badbots.py creates nginx/bots.conf): - # python badbots.py - echo "Placeholder for badbots.py execution" + - name: 🔄 Generate Bad Bot Blockers + run: python badbots.py - name: 🚀 Commit and Push Changes (if any) run: | @@ -98,6 +93,7 @@ jobs: - name: 🚀 Create GitHub Release (if previous steps succeeded) + id: create_release if: success() # Only create release if previous steps were successful uses: actions/create-release@v1 env: diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index c2e3007..bb885c5 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1 +1,109 @@ -You can send a PR any time. +# Contributing to Patterns + +Thank you for your interest in contributing to the Patterns project! We appreciate your help in making this project better. + +## How to Contribute + +### Reporting Issues + +If you find a bug or have a suggestion for improvement: + +1. Check if the issue already exists in the [Issues](https://github.com/fabriziosalmi/patterns/issues) section. +2. If not, create a new issue with a clear title and description. +3. Include relevant details such as: + - Steps to reproduce (for bugs) + - Expected vs. actual behavior + - Your environment (OS, Python version, web server type) + +### Submitting Pull Requests + +We welcome pull requests! Here's how to submit one: + +1. **Fork the Repository** + ```bash + git clone https://github.com/YOUR_USERNAME/patterns.git + cd patterns + ``` + +2. **Create a Feature Branch** + + Use descriptive branch names following this convention: + - `feature/description` - For new features + - `fix/description` - For bug fixes + - `docs/description` - For documentation changes + - `refactor/description` - For code refactoring + + Example: + ```bash + git checkout -b feature/add-caddy-support + ``` + +3. **Make Your Changes** + - Write clear, concise commit messages + - Follow the existing code style and conventions + - Add comments where necessary + - Update documentation if you're changing functionality + +4. **Test Your Changes** + + Before submitting, ensure your code works correctly: + ```bash + # Install dependencies + pip install -r requirements.txt + + # Test the OWASP scraper + python owasp2json.py + + # Test the converters + python json2nginx.py + python json2apache.py + python json2traefik.py + python json2haproxy.py + + # Test bad bot generation + python badbots.py + ``` + + For web server specific testing, check the respective workflow files in `.github/workflows/`. + +5. **Commit and Push** + ```bash + git add . + git commit -m "feat: add support for Caddy web server" + git push origin feature/add-caddy-support + ``` + +6. **Open a Pull Request** + - Go to the original repository on GitHub + - Click "New Pull Request" + - Select your branch + - Provide a clear title and description of your changes + - Reference any related issues + +## Code Style Guidelines + +- Use Python 3.11 or higher +- Follow PEP 8 style guidelines +- Use meaningful variable and function names +- Add docstrings to functions and classes +- Keep functions focused and modular +- Handle errors gracefully with try-except blocks + +## Adding Support for New Web Servers + +If you want to add support for a new web server: + +1. Create a new converter script: `json2WEBSERVER.py` +2. Create output directory: `waf_patterns/WEBSERVER/` +3. Add README.md with integration instructions +4. Update the main README.md to include the new web server +5. Update the GitHub Actions workflow to include the new converter +6. Add example configurations + +## Questions? + +If you have questions about contributing, feel free to: +- Open an issue for discussion +- Contact the maintainers + +Thank you for contributing! diff --git a/README.md b/README.md index ee6756b..b64e2d2 100644 --- a/README.md +++ b/README.md @@ -36,14 +36,15 @@ patterns/ │ ├── apache/ # Apache WAF configs (ModSecurity) │ ├── traefik/ # Traefik WAF configs │ └── haproxy/ # HAProxy WAF configs -│── import_apache_waf.py -│── import_haproxy_waf.py -│── import_nginx_waf.py -│── import_traefik_waf.py -├── owasp.py # 🕵️ OWASP scraper (fetch CRS rules) -├── owasp2nginx.py # 🔄 Convert OWASP JSON to Nginx WAF configs -├── owasp2apache.py # 🔄 Convert OWASP JSON to Apache ModSecurity configs -├── owasp2haproxy.py # 🔄 Convert OWASP JSON to HAProxy WAF configs +├── import_apache_waf.py # 📥 Import Apache WAF configurations +├── import_haproxy_waf.py # 📥 Import HAProxy WAF configurations +├── import_nginx_waf.py # 📥 Import Nginx WAF configurations +├── import_traefik_waf.py # 📥 Import Traefik WAF configurations +├── owasp2json.py # 🕵️ OWASP scraper (fetch CRS rules) +├── json2nginx.py # 🔄 Convert OWASP JSON to Nginx WAF configs +├── json2apache.py # 🔄 Convert OWASP JSON to Apache ModSecurity configs +├── json2traefik.py # 🔄 Convert OWASP JSON to Traefik WAF configs +├── json2haproxy.py # 🔄 Convert OWASP JSON to HAProxy WAF configs ├── badbots.py # 🤖 Generate WAF configs to block bad bots ├── requirements.txt # 📄 Required dependencies └── .github/workflows/ # 🤖 GitHub Actions for automation @@ -54,14 +55,14 @@ patterns/ ## 🛠️ How It Works ### 🔹 1. Scraping OWASP Rules -- **`owasp.py`** scrapes the latest OWASP CRS patterns from GitHub. +- **`owasp2json.py`** scrapes the latest OWASP CRS patterns from GitHub. - Extracts **SQLi, XSS, RCE, LFI** patterns from OWASP CRS `.conf` files. ### 🔹 2. Generating WAF Configs for Each Platform -- **`owasp2nginx.py`** – Generates **Nginx WAF** configurations. -- **`owasp2apache.py`** – Outputs **Apache ModSecurity** rules. -- **`owasp2traefik.py`** – Creates **Traefik WAF** rules. -- **`owasp2haproxy.py`** – Builds **HAProxy ACL** files. +- **`json2nginx.py`** – Generates **Nginx WAF** configurations. +- **`json2apache.py`** – Outputs **Apache ModSecurity** rules. +- **`json2traefik.py`** – Creates **Traefik WAF** rules. +- **`json2haproxy.py`** – Builds **HAProxy ACL** files. ### 🔹 3. Bad Bot/User-Agent Detection - **`badbots.py`** fetches public bot lists and generates bot-blocking configs. @@ -94,11 +95,11 @@ pip install -r requirements.txt **3. Run Manually (Optional):** ```bash -python owasp.py -python owasp2nginx.py -python owasp2apache.py -python owasp2haproxy.py -python owasp2traefik.py +python owasp2json.py +python json2nginx.py +python json2apache.py +python json2haproxy.py +python json2traefik.py python badbots.py ``` @@ -169,9 +170,9 @@ See the [LICENSE](LICENSE) file for details. --- -## Others projects +## Other Projects -If You like my projects, you may also like these ones: +If you like this project, you may also like these: - [caddy-waf](https://github.com/fabriziosalmi/caddy-waf) Caddy WAF (Regex Rules, IP and DNS filtering, Rate Limiting, GeoIP, Tor, Anomaly Detection) - [blacklists](https://github.com/fabriziosalmi/blacklists) Hourly updated domains blacklist 🚫 diff --git a/waf_patterns/apache/README.md b/waf_patterns/apache/README.md index 8b13789..6b7ee8d 100644 --- a/waf_patterns/apache/README.md +++ b/waf_patterns/apache/README.md @@ -1 +1,117 @@ +# Apache ModSecurity WAF Configuration +This directory contains Apache ModSecurity WAF configuration files generated from OWASP CRS rules. +You can include these files in your existing Apache configuration to enhance security. + +## Prerequisites + +- Apache HTTP Server (2.4 or higher) +- ModSecurity module installed and enabled +- Core Rule Set (CRS) base configuration + +## Installation + +### Ubuntu/Debian +```bash +sudo apt-get update +sudo apt-get install libapache2-mod-security2 +sudo a2enmod security2 +sudo systemctl restart apache2 +``` + +### CentOS/RHEL +```bash +sudo yum install mod_security +sudo systemctl restart httpd +``` + +## Usage + +1. Copy the generated configuration files to your Apache configuration directory: + ```bash + sudo cp waf_patterns/apache/*.conf /etc/apache2/modsecurity.d/ + # or for CentOS/RHEL: + # sudo cp waf_patterns/apache/*.conf /etc/httpd/modsecurity.d/ + ``` + +2. Include the configuration files in your Apache configuration. + + Edit `/etc/apache2/mods-enabled/security2.conf` (Ubuntu/Debian) or `/etc/httpd/conf.d/mod_security.conf` (CentOS/RHEL): + ```apache + + Include /etc/apache2/modsecurity.d/*.conf + + ``` + +3. Test the configuration: + ```bash + # Ubuntu/Debian + sudo apache2ctl configtest + + # CentOS/RHEL + sudo httpd -t + ``` + +4. Reload Apache to apply the changes: + ```bash + # Ubuntu/Debian + sudo systemctl reload apache2 + + # CentOS/RHEL + sudo systemctl reload httpd + ``` + +## Configuration Details + +The generated rules include: +- **SQL Injection (SQLi)** detection patterns +- **Cross-Site Scripting (XSS)** prevention rules +- **Remote Code Execution (RCE)** blocking +- **Local File Inclusion (LFI)** protection +- **Bad Bot/User-Agent** blocking + +## Customization + +You can adjust the severity and actions for each rule by modifying the configuration files. +Common actions include: +- `deny` - Block the request +- `log` - Log the event +- `status:403` - Return HTTP 403 Forbidden + +## Troubleshooting + +### Check ModSecurity is loaded +```bash +# Ubuntu/Debian +apache2ctl -M | grep security + +# CentOS/RHEL +httpd -M | grep security +``` + +### View ModSecurity logs +```bash +# Ubuntu/Debian +sudo tail -f /var/log/apache2/modsec_audit.log + +# CentOS/RHEL +sudo tail -f /var/log/httpd/modsec_audit.log +``` + +### Test with a sample attack +```bash +curl "http://yourserver.com/?id=1' OR '1'='1" +# Should return 403 Forbidden if WAF is working +``` + +## Notes + +- Rules are updated daily via GitHub Actions +- Blocked requests return a `403 Forbidden` response by default +- Review the ModSecurity documentation for advanced configuration options + +## Resources + +- [ModSecurity Documentation](https://github.com/SpiderLabs/ModSecurity) +- [OWASP CRS](https://coreruleset.org/) +- [Apache ModSecurity Module](https://modsecurity.org/) diff --git a/waf_patterns/haproxy/README.md b/waf_patterns/haproxy/README.md index 8b13789..f96e622 100644 --- a/waf_patterns/haproxy/README.md +++ b/waf_patterns/haproxy/README.md @@ -1 +1,183 @@ +# HAProxy WAF Configuration +This directory contains HAProxy WAF configuration files generated from OWASP CRS rules. +You can include these ACL (Access Control List) files in your HAProxy configuration to enhance security. + +## Prerequisites + +- HAProxy 2.0 or higher +- Basic understanding of HAProxy ACLs and rules + +## Configuration Files + +The generated files include: +- ACL files with pattern matching rules +- Request filtering configurations +- Bad bot/User-Agent blocking lists + +## Usage + +1. Copy the generated ACL files to your HAProxy configuration directory: + ```bash + sudo cp waf_patterns/haproxy/*.acl /etc/haproxy/ + ``` + +2. Include the ACL files in your HAProxy configuration. + + Edit `/etc/haproxy/haproxy.cfg`: + ```haproxy + frontend http-in + bind *:80 + + # Load WAF ACL files + acl is_sql_injection path_reg -i -f /etc/haproxy/sqli_patterns.acl + acl is_xss_attack path_reg -i -f /etc/haproxy/xss_patterns.acl + acl is_bad_bot hdr_reg(User-Agent) -i -f /etc/haproxy/bad_bots.acl + + # Block malicious requests + http-request deny if is_sql_injection + http-request deny if is_xss_attack + http-request deny if is_bad_bot + + # Default backend + default_backend web_servers + + backend web_servers + balance roundrobin + server web1 10.0.0.1:80 check + server web2 10.0.0.2:80 check + ``` + +3. Test the configuration: + ```bash + sudo haproxy -c -f /etc/haproxy/haproxy.cfg + ``` + +4. Reload HAProxy to apply the changes: + ```bash + sudo systemctl reload haproxy + # or + sudo service haproxy reload + ``` + +## Advanced Configuration + +### Logging Blocked Requests + +Add logging for better visibility: + +```haproxy +frontend http-in + bind *:80 + + # ... ACL definitions ... + + # Log blocked requests + http-request capture req.hdr(User-Agent) len 200 + http-request deny deny_status 403 if is_sql_injection + log-format "%ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %CC %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r" +``` + +### Custom Error Pages + +Return custom error pages for blocked requests: + +```haproxy +frontend http-in + bind *:80 + + # ... ACL definitions ... + + # Return custom error page + http-request deny deny_status 403 if is_sql_injection + errorfile 403 /etc/haproxy/errors/403.http +``` + +### Rate Limiting + +Combine with rate limiting for additional protection: + +```haproxy +frontend http-in + bind *:80 + + # Track request rate + stick-table type ip size 100k expire 30s store http_req_rate(10s) + http-request track-sc0 src + + # Deny if rate limit exceeded + http-request deny deny_status 429 if { sc_http_req_rate(0) gt 100 } + + # ... WAF ACLs ... +``` + +## Testing + +### Test SQL Injection Protection +```bash +curl "http://yourserver.com/?id=1' OR '1'='1" +# Should return 403 Forbidden +``` + +### Test XSS Protection +```bash +curl "http://yourserver.com/?q=" +# Should return 403 Forbidden +``` + +### Test Bad Bot Blocking +```bash +curl -H "User-Agent: AhrefsBot" http://yourserver.com +# Should return 403 Forbidden +``` + +## Monitoring + +### Check HAProxy Stats +```bash +# Enable stats in haproxy.cfg +listen stats + bind *:8404 + stats enable + stats uri /stats + stats refresh 10s +``` + +Visit `http://yourserver:8404/stats` to view statistics. + +### View Logs +```bash +sudo tail -f /var/log/haproxy.log +``` + +## Performance Considerations + +- ACL pattern matching is highly efficient in HAProxy +- Use regular expressions sparingly for better performance +- Consider using stick tables for rate limiting +- Monitor CPU and memory usage under load +- Test thoroughly before deploying to production + +## Configuration Details + +The ACL files protect against: +- **SQL Injection (SQLi)** - Common SQL injection patterns +- **Cross-Site Scripting (XSS)** - JavaScript injection attempts +- **Remote Code Execution (RCE)** - Command injection patterns +- **Local File Inclusion (LFI)** - Path traversal attempts +- **Bad Bots** - Known malicious crawlers and scrapers + +## Notes + +- Rules are updated daily via GitHub Actions +- Blocked requests return `403 Forbidden` by default +- ACLs are case-insensitive (`-i` flag) +- Regular expressions are used for pattern matching (`-f` for file-based ACLs) +- Compatible with HAProxy 2.0 and higher + +## Resources + +- [HAProxy Documentation](https://www.haproxy.org/#docs) +- [HAProxy ACL Guide](https://www.haproxy.com/documentation/hapee/latest/onepage/#7) +- [OWASP CRS](https://coreruleset.org/) +- [HAProxy Configuration Manual](http://cbonte.github.io/haproxy-dconv/) diff --git a/waf_patterns/traefik/README.md b/waf_patterns/traefik/README.md index 8b13789..4781bc1 100644 --- a/waf_patterns/traefik/README.md +++ b/waf_patterns/traefik/README.md @@ -1 +1,136 @@ +# Traefik WAF Configuration +This directory contains Traefik WAF configuration files generated from OWASP CRS rules. +You can use these middleware configurations to enhance security in your Traefik setup. + +## Prerequisites + +- Traefik v2.x or higher +- Basic understanding of Traefik middleware + +## Configuration Files + +The generated configuration includes: +- Middleware definitions for request filtering +- Regular expression patterns for attack detection +- Bad bot/User-Agent blocking rules + +## Usage + +### Option 1: File Provider (Recommended) + +1. Copy the generated configuration files to your Traefik configuration directory: + ```bash + cp waf_patterns/traefik/*.toml /etc/traefik/dynamic/ + # or to your custom config directory + ``` + +2. Configure Traefik to load dynamic configuration from files. + + In your `traefik.yml` or `traefik.toml`: + ```yaml + providers: + file: + directory: "/etc/traefik/dynamic" + watch: true + ``` + +3. Apply the middleware to your routes by referencing it in your service configuration: + ```yaml + http: + routers: + my-router: + rule: "Host(`example.com`)" + service: my-service + middlewares: + - waf-middleware + ``` + +### Option 2: Docker Labels + +If you're using Docker, you can apply the middleware via labels: + +```yaml +services: + my-service: + image: my-app:latest + labels: + - "traefik.enable=true" + - "traefik.http.routers.my-router.rule=Host(`example.com`)" + - "traefik.http.routers.my-router.middlewares=waf-middleware@file" +``` + +### Option 3: Kubernetes IngressRoute + +For Kubernetes deployments: + +```yaml +apiVersion: traefik.containo.us/v1alpha1 +kind: Middleware +metadata: + name: waf-middleware +spec: + plugin: + # Reference your WAF plugin configuration here +``` + +## Configuration Details + +The middleware includes protection against: +- **SQL Injection (SQLi)** attacks +- **Cross-Site Scripting (XSS)** attempts +- **Remote Code Execution (RCE)** patterns +- **Local File Inclusion (LFI)** attempts +- **Malicious bots and crawlers** + +## Testing + +Test the WAF is working by sending a malicious request: + +```bash +curl -H "User-Agent: AhrefsBot" http://yourserver.com +# Should be blocked if bot protection is working + +curl "http://yourserver.com/?id=1' OR '1'='1" +# Should be blocked if SQLi protection is working +``` + +## Monitoring + +Monitor blocked requests in Traefik logs: + +```bash +# Docker +docker logs traefik 2>&1 | grep -i "blocked\|forbidden" + +# Standard installation +tail -f /var/log/traefik/access.log | grep -i "403" +``` + +## Customization + +You can customize the middleware behavior by: +1. Editing the generated `.toml` files +2. Adjusting regex patterns for your specific needs +3. Modifying response codes and error pages +4. Adding custom headers for blocked requests + +## Performance Considerations + +- Regular expression matching can impact performance under high load +- Consider using caching middleware in combination with WAF +- Monitor CPU usage and adjust rules if needed +- Use Traefik's built-in rate limiting for additional protection + +## Notes + +- Rules are updated daily via GitHub Actions +- Blocked requests typically return `403 Forbidden` or `400 Bad Request` +- Middleware is applied at the router level +- Compatible with other Traefik middlewares (chain them as needed) + +## Resources + +- [Traefik Documentation](https://doc.traefik.io/traefik/) +- [Traefik Middleware](https://doc.traefik.io/traefik/middlewares/overview/) +- [OWASP CRS](https://coreruleset.org/)