diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md index dd84ea7..666c40f 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -12,27 +12,25 @@ A clear and concise description of what the bug is. **To Reproduce** Steps to reproduce the behavior: -1. Go to '...' -2. Click on '....' -3. Scroll down to '....' -4. See error +1. Run command '...' +2. With configuration '...' +3. See error **Expected behavior** A clear and concise description of what you expected to happen. -**Screenshots** -If applicable, add screenshots to help explain your problem. +**Error messages/logs** +If applicable, add error messages or log output to help explain your problem. -**Desktop (please complete the following information):** - - OS: [e.g. iOS] - - Browser [e.g. chrome, safari] - - Version [e.g. 22] +``` +Paste error messages or logs here +``` -**Smartphone (please complete the following information):** - - Device: [e.g. iPhone6] - - OS: [e.g. iOS8.1] - - Browser [e.g. stock browser, safari] - - Version [e.g. 22] +**Environment (please complete the following information):** + - OS: [e.g. Ubuntu 22.04, macOS 13, Windows 11] + - Python Version: [e.g. 3.11.0] + - Web Server: [e.g. Nginx 1.22, Apache 2.4, Traefik 2.9, HAProxy 2.6] + - Installation Method: [e.g. built from source, downloaded pre-generated configs] **Additional context** Add any other context about the problem here. diff --git a/.github/workflows/update_patterns.yml b/.github/workflows/update_patterns.yml index 59d9a29..52349a2 100644 --- a/.github/workflows/update_patterns.yml +++ b/.github/workflows/update_patterns.yml @@ -54,13 +54,8 @@ jobs: - name: 🔄 Convert OWASP to HAProxy WAF run: python json2haproxy.py - - name: 🔄 Generate Bad Bot Blockers (Placeholder - Provide badbots.py) - run: | - # Placeholder: Replace this with your actual badbots.py script. - # Assuming badbots.py generates files in waf_patterns/ - # Example (if badbots.py creates nginx/bots.conf): - # python badbots.py - echo "Placeholder for badbots.py execution" + - name: 🔄 Generate Bad Bot Blockers + run: python badbots.py - name: 🚀 Commit and Push Changes (if any) run: | @@ -98,6 +93,7 @@ jobs: - name: 🚀 Create GitHub Release (if previous steps succeeded) + id: create_release if: success() # Only create release if previous steps were successful uses: actions/create-release@v1 env: diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index c2e3007..bb885c5 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1 +1,109 @@ -You can send a PR any time. +# Contributing to Patterns + +Thank you for your interest in contributing to the Patterns project! We appreciate your help in making this project better. + +## How to Contribute + +### Reporting Issues + +If you find a bug or have a suggestion for improvement: + +1. Check if the issue already exists in the [Issues](https://github.com/fabriziosalmi/patterns/issues) section. +2. If not, create a new issue with a clear title and description. +3. Include relevant details such as: + - Steps to reproduce (for bugs) + - Expected vs. actual behavior + - Your environment (OS, Python version, web server type) + +### Submitting Pull Requests + +We welcome pull requests! Here's how to submit one: + +1. **Fork the Repository** + ```bash + git clone https://github.com/YOUR_USERNAME/patterns.git + cd patterns + ``` + +2. **Create a Feature Branch** + + Use descriptive branch names following this convention: + - `feature/description` - For new features + - `fix/description` - For bug fixes + - `docs/description` - For documentation changes + - `refactor/description` - For code refactoring + + Example: + ```bash + git checkout -b feature/add-caddy-support + ``` + +3. **Make Your Changes** + - Write clear, concise commit messages + - Follow the existing code style and conventions + - Add comments where necessary + - Update documentation if you're changing functionality + +4. **Test Your Changes** + + Before submitting, ensure your code works correctly: + ```bash + # Install dependencies + pip install -r requirements.txt + + # Test the OWASP scraper + python owasp2json.py + + # Test the converters + python json2nginx.py + python json2apache.py + python json2traefik.py + python json2haproxy.py + + # Test bad bot generation + python badbots.py + ``` + + For web server specific testing, check the respective workflow files in `.github/workflows/`. + +5. **Commit and Push** + ```bash + git add . + git commit -m "feat: add support for Caddy web server" + git push origin feature/add-caddy-support + ``` + +6. **Open a Pull Request** + - Go to the original repository on GitHub + - Click "New Pull Request" + - Select your branch + - Provide a clear title and description of your changes + - Reference any related issues + +## Code Style Guidelines + +- Use Python 3.11 or higher +- Follow PEP 8 style guidelines +- Use meaningful variable and function names +- Add docstrings to functions and classes +- Keep functions focused and modular +- Handle errors gracefully with try-except blocks + +## Adding Support for New Web Servers + +If you want to add support for a new web server: + +1. Create a new converter script: `json2WEBSERVER.py` +2. Create output directory: `waf_patterns/WEBSERVER/` +3. Add README.md with integration instructions +4. Update the main README.md to include the new web server +5. Update the GitHub Actions workflow to include the new converter +6. Add example configurations + +## Questions? + +If you have questions about contributing, feel free to: +- Open an issue for discussion +- Contact the maintainers + +Thank you for contributing! diff --git a/README.md b/README.md index ee6756b..c3bb320 100644 --- a/README.md +++ b/README.md @@ -36,14 +36,15 @@ patterns/ │ ├── apache/ # Apache WAF configs (ModSecurity) │ ├── traefik/ # Traefik WAF configs │ └── haproxy/ # HAProxy WAF configs -│── import_apache_waf.py -│── import_haproxy_waf.py -│── import_nginx_waf.py -│── import_traefik_waf.py -├── owasp.py # 🕵️ OWASP scraper (fetch CRS rules) -├── owasp2nginx.py # 🔄 Convert OWASP JSON to Nginx WAF configs -├── owasp2apache.py # 🔄 Convert OWASP JSON to Apache ModSecurity configs -├── owasp2haproxy.py # 🔄 Convert OWASP JSON to HAProxy WAF configs +├── import_apache_waf.py # 📥 Import Apache WAF configurations +├── import_haproxy_waf.py # 📥 Import HAProxy WAF configurations +├── import_nginx_waf.py # 📥 Import Nginx WAF configurations +├── import_traefik_waf.py # 📥 Import Traefik WAF configurations +├── owasp2json.py # 🕵️ OWASP scraper (fetch CRS rules) +├── json2nginx.py # 🔄 Convert OWASP JSON to Nginx WAF configs +├── json2apache.py # 🔄 Convert OWASP JSON to Apache ModSecurity configs +├── json2traefik.py # 🔄 Convert OWASP JSON to Traefik WAF configs +├── json2haproxy.py # 🔄 Convert OWASP JSON to HAProxy WAF configs ├── badbots.py # 🤖 Generate WAF configs to block bad bots ├── requirements.txt # 📄 Required dependencies └── .github/workflows/ # 🤖 GitHub Actions for automation @@ -54,14 +55,14 @@ patterns/ ## 🛠️ How It Works ### 🔹 1. Scraping OWASP Rules -- **`owasp.py`** scrapes the latest OWASP CRS patterns from GitHub. +- **`owasp2json.py`** scrapes the latest OWASP CRS patterns from GitHub. - Extracts **SQLi, XSS, RCE, LFI** patterns from OWASP CRS `.conf` files. ### 🔹 2. Generating WAF Configs for Each Platform -- **`owasp2nginx.py`** – Generates **Nginx WAF** configurations. -- **`owasp2apache.py`** – Outputs **Apache ModSecurity** rules. -- **`owasp2traefik.py`** – Creates **Traefik WAF** rules. -- **`owasp2haproxy.py`** – Builds **HAProxy ACL** files. +- **`json2nginx.py`** – Generates **Nginx WAF** configurations. +- **`json2apache.py`** – Outputs **Apache ModSecurity** rules. +- **`json2traefik.py`** – Creates **Traefik WAF** rules. +- **`json2haproxy.py`** – Builds **HAProxy ACL** files. ### 🔹 3. Bad Bot/User-Agent Detection - **`badbots.py`** fetches public bot lists and generates bot-blocking configs. @@ -71,6 +72,13 @@ patterns/ ## ⚙️ Installation +### Prerequisites + +Before installing, ensure you have the following: +- **Python 3.11 or higher** (the project uses Python 3.11 as specified in the GitHub Actions workflow) +- **pip** (Python package installer) +- **git** (for cloning the repository) + ### Option 1: Download Pre-Generated Configurations You can download the latest pre-generated WAF configurations directly from the [GitHub Releases](https://github.com/fabriziosalmi/patterns/releases) page. @@ -94,11 +102,11 @@ pip install -r requirements.txt **3. Run Manually (Optional):** ```bash -python owasp.py -python owasp2nginx.py -python owasp2apache.py -python owasp2haproxy.py -python owasp2traefik.py +python owasp2json.py +python json2nginx.py +python json2apache.py +python json2haproxy.py +python json2traefik.py python badbots.py ``` @@ -169,9 +177,9 @@ See the [LICENSE](LICENSE) file for details. --- -## Others projects +## Other Projects -If You like my projects, you may also like these ones: +If you like this project, you may also like these: - [caddy-waf](https://github.com/fabriziosalmi/caddy-waf) Caddy WAF (Regex Rules, IP and DNS filtering, Rate Limiting, GeoIP, Tor, Anomaly Detection) - [blacklists](https://github.com/fabriziosalmi/blacklists) Hourly updated domains blacklist 🚫 diff --git a/SECURITY.md b/SECURITY.md index ad1f95d..4019ffd 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -2,12 +2,71 @@ ## Supported Versions +We actively support the current version of this project. The WAF patterns are updated daily via automated GitHub Actions. + | Version | Supported | | ------- | ------------------ | -| current | :white_check_mark: | +| current (main branch) | :white_check_mark: | +| latest release | :white_check_mark: | ## Reporting a Vulnerability -Please open an issue to report a vulnerability. +We take security vulnerabilities seriously. If you discover a security issue, please follow these steps: + +### For Non-Critical Issues +For general security concerns or minor issues: +1. Open an issue in the [Issues](https://github.com/fabriziosalmi/patterns/issues) section +2. Use the label "security" if available +3. Provide a clear description of the issue + +### For Critical Vulnerabilities +For critical security vulnerabilities (e.g., in the WAF patterns themselves): +1. **DO NOT** open a public issue +2. Email the maintainer directly at: fabrizio.salmi@gmail.com +3. Include: + - Description of the vulnerability + - Steps to reproduce + - Potential impact + - Suggested fix (if available) + +### What to Include +When reporting a vulnerability, please include: +- Type of vulnerability (e.g., regex bypass, pattern detection issue) +- Affected web server(s) (Nginx, Apache, Traefik, HAProxy) +- Attack pattern that bypasses detection +- Suggested regex or pattern improvement +- Any proof-of-concept code (if applicable) + +### Response Time +- We aim to acknowledge vulnerability reports within **48 hours** +- Critical vulnerabilities will be addressed in the next daily update +- Less critical issues will be prioritized based on severity + +### After Reporting +Once you report a vulnerability: +1. We will acknowledge receipt +2. We will investigate and validate the issue +3. We will work on a fix and test it +4. We will deploy the fix in the next update +5. We will credit you in the release notes (unless you prefer to remain anonymous) + +## Security Best Practices + +When using the WAF patterns from this project: +- Always test new rules in a staging environment first +- Monitor your logs for false positives +- Keep your web server and WAF software up to date +- Review the OWASP CRS documentation for additional hardening +- Consider layering multiple security controls (WAF + rate limiting + IPS, etc.) + +## Scope + +This security policy covers: +- WAF pattern generation logic +- Regex patterns for attack detection +- GitHub Actions workflow security +- Dependencies listed in requirements.txt + +Thank you for helping keep this project secure! diff --git a/waf_patterns/apache/README.md b/waf_patterns/apache/README.md index 8b13789..6b7ee8d 100644 --- a/waf_patterns/apache/README.md +++ b/waf_patterns/apache/README.md @@ -1 +1,117 @@ +# Apache ModSecurity WAF Configuration +This directory contains Apache ModSecurity WAF configuration files generated from OWASP CRS rules. +You can include these files in your existing Apache configuration to enhance security. + +## Prerequisites + +- Apache HTTP Server (2.4 or higher) +- ModSecurity module installed and enabled +- Core Rule Set (CRS) base configuration + +## Installation + +### Ubuntu/Debian +```bash +sudo apt-get update +sudo apt-get install libapache2-mod-security2 +sudo a2enmod security2 +sudo systemctl restart apache2 +``` + +### CentOS/RHEL +```bash +sudo yum install mod_security +sudo systemctl restart httpd +``` + +## Usage + +1. Copy the generated configuration files to your Apache configuration directory: + ```bash + sudo cp waf_patterns/apache/*.conf /etc/apache2/modsecurity.d/ + # or for CentOS/RHEL: + # sudo cp waf_patterns/apache/*.conf /etc/httpd/modsecurity.d/ + ``` + +2. Include the configuration files in your Apache configuration. + + Edit `/etc/apache2/mods-enabled/security2.conf` (Ubuntu/Debian) or `/etc/httpd/conf.d/mod_security.conf` (CentOS/RHEL): + ```apache + + Include /etc/apache2/modsecurity.d/*.conf + + ``` + +3. Test the configuration: + ```bash + # Ubuntu/Debian + sudo apache2ctl configtest + + # CentOS/RHEL + sudo httpd -t + ``` + +4. Reload Apache to apply the changes: + ```bash + # Ubuntu/Debian + sudo systemctl reload apache2 + + # CentOS/RHEL + sudo systemctl reload httpd + ``` + +## Configuration Details + +The generated rules include: +- **SQL Injection (SQLi)** detection patterns +- **Cross-Site Scripting (XSS)** prevention rules +- **Remote Code Execution (RCE)** blocking +- **Local File Inclusion (LFI)** protection +- **Bad Bot/User-Agent** blocking + +## Customization + +You can adjust the severity and actions for each rule by modifying the configuration files. +Common actions include: +- `deny` - Block the request +- `log` - Log the event +- `status:403` - Return HTTP 403 Forbidden + +## Troubleshooting + +### Check ModSecurity is loaded +```bash +# Ubuntu/Debian +apache2ctl -M | grep security + +# CentOS/RHEL +httpd -M | grep security +``` + +### View ModSecurity logs +```bash +# Ubuntu/Debian +sudo tail -f /var/log/apache2/modsec_audit.log + +# CentOS/RHEL +sudo tail -f /var/log/httpd/modsec_audit.log +``` + +### Test with a sample attack +```bash +curl "http://yourserver.com/?id=1' OR '1'='1" +# Should return 403 Forbidden if WAF is working +``` + +## Notes + +- Rules are updated daily via GitHub Actions +- Blocked requests return a `403 Forbidden` response by default +- Review the ModSecurity documentation for advanced configuration options + +## Resources + +- [ModSecurity Documentation](https://github.com/SpiderLabs/ModSecurity) +- [OWASP CRS](https://coreruleset.org/) +- [Apache ModSecurity Module](https://modsecurity.org/) diff --git a/waf_patterns/haproxy/README.md b/waf_patterns/haproxy/README.md index 8b13789..f96e622 100644 --- a/waf_patterns/haproxy/README.md +++ b/waf_patterns/haproxy/README.md @@ -1 +1,183 @@ +# HAProxy WAF Configuration +This directory contains HAProxy WAF configuration files generated from OWASP CRS rules. +You can include these ACL (Access Control List) files in your HAProxy configuration to enhance security. + +## Prerequisites + +- HAProxy 2.0 or higher +- Basic understanding of HAProxy ACLs and rules + +## Configuration Files + +The generated files include: +- ACL files with pattern matching rules +- Request filtering configurations +- Bad bot/User-Agent blocking lists + +## Usage + +1. Copy the generated ACL files to your HAProxy configuration directory: + ```bash + sudo cp waf_patterns/haproxy/*.acl /etc/haproxy/ + ``` + +2. Include the ACL files in your HAProxy configuration. + + Edit `/etc/haproxy/haproxy.cfg`: + ```haproxy + frontend http-in + bind *:80 + + # Load WAF ACL files + acl is_sql_injection path_reg -i -f /etc/haproxy/sqli_patterns.acl + acl is_xss_attack path_reg -i -f /etc/haproxy/xss_patterns.acl + acl is_bad_bot hdr_reg(User-Agent) -i -f /etc/haproxy/bad_bots.acl + + # Block malicious requests + http-request deny if is_sql_injection + http-request deny if is_xss_attack + http-request deny if is_bad_bot + + # Default backend + default_backend web_servers + + backend web_servers + balance roundrobin + server web1 10.0.0.1:80 check + server web2 10.0.0.2:80 check + ``` + +3. Test the configuration: + ```bash + sudo haproxy -c -f /etc/haproxy/haproxy.cfg + ``` + +4. Reload HAProxy to apply the changes: + ```bash + sudo systemctl reload haproxy + # or + sudo service haproxy reload + ``` + +## Advanced Configuration + +### Logging Blocked Requests + +Add logging for better visibility: + +```haproxy +frontend http-in + bind *:80 + + # ... ACL definitions ... + + # Log blocked requests + http-request capture req.hdr(User-Agent) len 200 + http-request deny deny_status 403 if is_sql_injection + log-format "%ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %CC %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r" +``` + +### Custom Error Pages + +Return custom error pages for blocked requests: + +```haproxy +frontend http-in + bind *:80 + + # ... ACL definitions ... + + # Return custom error page + http-request deny deny_status 403 if is_sql_injection + errorfile 403 /etc/haproxy/errors/403.http +``` + +### Rate Limiting + +Combine with rate limiting for additional protection: + +```haproxy +frontend http-in + bind *:80 + + # Track request rate + stick-table type ip size 100k expire 30s store http_req_rate(10s) + http-request track-sc0 src + + # Deny if rate limit exceeded + http-request deny deny_status 429 if { sc_http_req_rate(0) gt 100 } + + # ... WAF ACLs ... +``` + +## Testing + +### Test SQL Injection Protection +```bash +curl "http://yourserver.com/?id=1' OR '1'='1" +# Should return 403 Forbidden +``` + +### Test XSS Protection +```bash +curl "http://yourserver.com/?q=" +# Should return 403 Forbidden +``` + +### Test Bad Bot Blocking +```bash +curl -H "User-Agent: AhrefsBot" http://yourserver.com +# Should return 403 Forbidden +``` + +## Monitoring + +### Check HAProxy Stats +```bash +# Enable stats in haproxy.cfg +listen stats + bind *:8404 + stats enable + stats uri /stats + stats refresh 10s +``` + +Visit `http://yourserver:8404/stats` to view statistics. + +### View Logs +```bash +sudo tail -f /var/log/haproxy.log +``` + +## Performance Considerations + +- ACL pattern matching is highly efficient in HAProxy +- Use regular expressions sparingly for better performance +- Consider using stick tables for rate limiting +- Monitor CPU and memory usage under load +- Test thoroughly before deploying to production + +## Configuration Details + +The ACL files protect against: +- **SQL Injection (SQLi)** - Common SQL injection patterns +- **Cross-Site Scripting (XSS)** - JavaScript injection attempts +- **Remote Code Execution (RCE)** - Command injection patterns +- **Local File Inclusion (LFI)** - Path traversal attempts +- **Bad Bots** - Known malicious crawlers and scrapers + +## Notes + +- Rules are updated daily via GitHub Actions +- Blocked requests return `403 Forbidden` by default +- ACLs are case-insensitive (`-i` flag) +- Regular expressions are used for pattern matching (`-f` for file-based ACLs) +- Compatible with HAProxy 2.0 and higher + +## Resources + +- [HAProxy Documentation](https://www.haproxy.org/#docs) +- [HAProxy ACL Guide](https://www.haproxy.com/documentation/hapee/latest/onepage/#7) +- [OWASP CRS](https://coreruleset.org/) +- [HAProxy Configuration Manual](http://cbonte.github.io/haproxy-dconv/) diff --git a/waf_patterns/traefik/README.md b/waf_patterns/traefik/README.md index 8b13789..4781bc1 100644 --- a/waf_patterns/traefik/README.md +++ b/waf_patterns/traefik/README.md @@ -1 +1,136 @@ +# Traefik WAF Configuration +This directory contains Traefik WAF configuration files generated from OWASP CRS rules. +You can use these middleware configurations to enhance security in your Traefik setup. + +## Prerequisites + +- Traefik v2.x or higher +- Basic understanding of Traefik middleware + +## Configuration Files + +The generated configuration includes: +- Middleware definitions for request filtering +- Regular expression patterns for attack detection +- Bad bot/User-Agent blocking rules + +## Usage + +### Option 1: File Provider (Recommended) + +1. Copy the generated configuration files to your Traefik configuration directory: + ```bash + cp waf_patterns/traefik/*.toml /etc/traefik/dynamic/ + # or to your custom config directory + ``` + +2. Configure Traefik to load dynamic configuration from files. + + In your `traefik.yml` or `traefik.toml`: + ```yaml + providers: + file: + directory: "/etc/traefik/dynamic" + watch: true + ``` + +3. Apply the middleware to your routes by referencing it in your service configuration: + ```yaml + http: + routers: + my-router: + rule: "Host(`example.com`)" + service: my-service + middlewares: + - waf-middleware + ``` + +### Option 2: Docker Labels + +If you're using Docker, you can apply the middleware via labels: + +```yaml +services: + my-service: + image: my-app:latest + labels: + - "traefik.enable=true" + - "traefik.http.routers.my-router.rule=Host(`example.com`)" + - "traefik.http.routers.my-router.middlewares=waf-middleware@file" +``` + +### Option 3: Kubernetes IngressRoute + +For Kubernetes deployments: + +```yaml +apiVersion: traefik.containo.us/v1alpha1 +kind: Middleware +metadata: + name: waf-middleware +spec: + plugin: + # Reference your WAF plugin configuration here +``` + +## Configuration Details + +The middleware includes protection against: +- **SQL Injection (SQLi)** attacks +- **Cross-Site Scripting (XSS)** attempts +- **Remote Code Execution (RCE)** patterns +- **Local File Inclusion (LFI)** attempts +- **Malicious bots and crawlers** + +## Testing + +Test the WAF is working by sending a malicious request: + +```bash +curl -H "User-Agent: AhrefsBot" http://yourserver.com +# Should be blocked if bot protection is working + +curl "http://yourserver.com/?id=1' OR '1'='1" +# Should be blocked if SQLi protection is working +``` + +## Monitoring + +Monitor blocked requests in Traefik logs: + +```bash +# Docker +docker logs traefik 2>&1 | grep -i "blocked\|forbidden" + +# Standard installation +tail -f /var/log/traefik/access.log | grep -i "403" +``` + +## Customization + +You can customize the middleware behavior by: +1. Editing the generated `.toml` files +2. Adjusting regex patterns for your specific needs +3. Modifying response codes and error pages +4. Adding custom headers for blocked requests + +## Performance Considerations + +- Regular expression matching can impact performance under high load +- Consider using caching middleware in combination with WAF +- Monitor CPU usage and adjust rules if needed +- Use Traefik's built-in rate limiting for additional protection + +## Notes + +- Rules are updated daily via GitHub Actions +- Blocked requests typically return `403 Forbidden` or `400 Bad Request` +- Middleware is applied at the router level +- Compatible with other Traefik middlewares (chain them as needed) + +## Resources + +- [Traefik Documentation](https://doc.traefik.io/traefik/) +- [Traefik Middleware](https://doc.traefik.io/traefik/middlewares/overview/) +- [OWASP CRS](https://coreruleset.org/)