diff --git a/agents/security-auditor.md b/agents/security-auditor.md index 07bc30b..14c3c67 100644 --- a/agents/security-auditor.md +++ b/agents/security-auditor.md @@ -43,6 +43,16 @@ You are an experienced Security Engineer conducting a security review. Your role - Are webhook payloads verified (signature validation)? - Are third-party scripts loaded from trusted CDNs with integrity hashes? - Are OAuth flows using PKCE and state parameters? +- Are server-side fetches of user-supplied URLs allowlisted (SSRF)? + +### 6. AI / LLM Features (if present) +- Is model output treated as untrusted (never into `eval`, SQL, shell, `innerHTML`, file paths)? +- Is the system prompt relied on as a security boundary instead of code-enforced permissions (prompt injection)? +- Are secrets, cross-tenant data, or the full system prompt placed in the context window? +- Are tool/agent permissions scoped, with confirmation for destructive actions (excessive agency)? +- Are token, rate, and recursion limits set (unbounded consumption)? + +Map findings to the OWASP Top 10 for LLM Applications where relevant. ## Severity Classification @@ -90,9 +100,10 @@ You are an experienced Security Engineer conducting a security review. Your role 2. Every finding must include a specific, actionable recommendation 3. Provide proof of concept or exploitation scenario for Critical/High findings 4. Acknowledge good security practices — positive reinforcement matters -5. Check the OWASP Top 10 as a minimum baseline -6. Review dependencies for known CVEs +5. Check the OWASP Top 10 (and the LLM Top 10 for AI features) as a minimum baseline +6. Review dependencies for known CVEs and supply-chain risk (typosquats, postinstall scripts) 7. Never suggest disabling security controls as a "fix" +8. Start from trust boundaries — where untrusted data enters — and reason about each with STRIDE before enumerating findings ## Composition diff --git a/references/security-checklist.md b/references/security-checklist.md index 4eec509..553c388 100644 --- a/references/security-checklist.md +++ b/references/security-checklist.md @@ -4,6 +4,7 @@ Quick reference for web application security. Use alongside the `security-and-ha ## Table of Contents +- [Threat Modeling (Start Here)](#threat-modeling-start-here) - [Pre-Commit Checks](#pre-commit-checks) - [Authentication](#authentication) - [Authorization](#authorization) @@ -12,8 +13,19 @@ Quick reference for web application security. Use alongside the `security-and-ha - [CORS Configuration](#cors-configuration) - [Data Protection](#data-protection) - [Dependency Security](#dependency-security) +- [AI / LLM Security](#ai--llm-security) - [Error Handling](#error-handling) - [OWASP Top 10 Quick Reference](#owasp-top-10-quick-reference) +- [OWASP Top 10 for LLMs Quick Reference](#owasp-top-10-for-llms-quick-reference) + +## Threat Modeling (Start Here) + +Before reaching for controls, spend five minutes thinking like an attacker: + +- [ ] Trust boundaries mapped (requests, uploads, webhooks, third-party APIs, LLM output) +- [ ] Assets named (credentials, PII, payment data, admin actions, money movement) +- [ ] STRIDE run per boundary (Spoofing, Tampering, Repudiation, Info disclosure, DoS, Elevation) +- [ ] Abuse cases written next to use cases ("how would I misuse this?") ## Pre-Commit Checks @@ -50,6 +62,7 @@ Quick reference for web application security. Use alongside the `security-and-ha - [ ] SQL queries parameterized (no string concatenation) - [ ] HTML output encoded (use framework auto-escaping) - [ ] URLs validated before redirect (prevent open redirect) +- [ ] Server-side URL fetches allowlisted; private/reserved IPs blocked (prevent SSRF) ## Security Headers @@ -102,6 +115,21 @@ npm audit --audit-level=critical npx npm-check-updates ``` +**Supply-chain hygiene** (`npm audit` won't catch malicious packages): +- [ ] Lockfile committed; CI installs with `npm ci` (not `npm install`) +- [ ] New dependencies reviewed (maintenance, downloads, `postinstall` scripts) +- [ ] No typosquats (`cross-env` vs `crossenv`, `react-dom` vs `reactdom`) + +## AI / LLM Security + +For any feature that calls an LLM (chatbots, summarizers, agents, RAG): + +- [ ] Model output treated as untrusted — never into `eval`/SQL/shell/`innerHTML`/file paths +- [ ] Prompt injection assumed; permissions enforced in code, not in the system prompt +- [ ] Secrets, cross-tenant data, and full system prompts kept out of the context window +- [ ] Tool/agent permissions scoped; destructive or irreversible actions require confirmation +- [ ] Token, rate, and recursion/loop limits set (bound consumption) + ## Error Handling ```typescript @@ -132,3 +160,20 @@ res.status(500).json({ | 8 | Data Integrity Failures | Verify updates/dependencies, signed artifacts | | 9 | Logging Failures | Log security events, don't log secrets | | 10 | SSRF | Validate/allowlist URLs, restrict outbound requests | + +## OWASP Top 10 for LLMs Quick Reference + +For apps with LLM features. See the [OWASP GenAI Security Project](https://genai.owasp.org/llm-top-10/). + +| ID | Risk | Prevention | +|---|---|---| +| LLM01 | Prompt Injection | Don't trust the system prompt as a boundary; enforce permissions in code | +| LLM02 | Sensitive Information Disclosure | Keep secrets/PII out of prompts; filter outputs | +| LLM03 | Supply Chain | Vet models, datasets, and plugins like any dependency | +| LLM04 | Data and Model Poisoning | Use trusted model sources, verify integrity; vet fine-tuning and RAG data | +| LLM05 | Improper Output Handling | Treat model output as untrusted; validate, parameterize, encode | +| LLM06 | Excessive Agency | Scope tool permissions; confirm destructive actions | +| LLM07 | System Prompt Leakage | Assume the system prompt can leak; put no secrets in it | +| LLM08 | Vector and Embedding Weaknesses | Partition RAG embeddings per tenant; validate documents before indexing | +| LLM09 | Misinformation | Ground answers with citations; validate critical claims; keep a human in the loop | +| LLM10 | Unbounded Consumption | Cap tokens, request rate, and loop/recursion depth | diff --git a/skills/security-and-hardening/SKILL.md b/skills/security-and-hardening/SKILL.md index 5b36a7b..2c641ac 100644 --- a/skills/security-and-hardening/SKILL.md +++ b/skills/security-and-hardening/SKILL.md @@ -18,6 +18,27 @@ Security-first development practices for web applications. Treat every external - Adding file uploads, webhooks, or callbacks - Handling payment or PII data +## Process: Threat Model First + +Controls bolted on without a threat model are guesses. Before hardening, spend five minutes thinking like an attacker: + +1. **Map the trust boundaries.** Where does untrusted data cross into your system? HTTP requests, form fields, file uploads, webhooks, third-party APIs, message queues, and **LLM output**. Every boundary is attack surface. +2. **Name the assets.** What's worth stealing or breaking? Credentials, PII, payment data, admin actions, money movement. +3. **Run STRIDE over each boundary** — a quick lens, not a ceremony: + +| Threat | Ask | Typical mitigation | +|---|---|---| +| **S**poofing | Can someone impersonate a user/service? | Authentication, signature verification | +| **T**ampering | Can data be altered in transit or at rest? | Integrity checks, parameterized queries, HTTPS | +| **R**epudiation | Can an action be denied later? | Audit logging of security events | +| **I**nformation disclosure | Can data leak? | Encryption, field allowlists, generic errors | +| **D**enial of service | Can it be overwhelmed? | Rate limiting, input size caps, timeouts | +| **E**levation of privilege | Can a user gain rights they shouldn't? | Authorization checks, least privilege | + +4. **Write abuse cases next to use cases.** For each feature, ask "how would I misuse this?" — then make that your first test. + +If you can't name the trust boundaries for a feature, you're not ready to secure it. This is OWASP **A04: Insecure Design** — most breaches begin in design, not code. + ## The Three-Tier Boundary System ### Always Do (No Exceptions) @@ -51,9 +72,11 @@ Security-first development practices for web applications. Treat every external - **Never store sessions in client-accessible storage** (localStorage for auth tokens) - **Never expose stack traces** or internal error details to users -## OWASP Top 10 Prevention +## OWASP Top 10 Prevention Patterns -### 1. Injection (SQL, NoSQL, OS Command) +These are prevention patterns, not a ranking. For the 2021 ordering, see the quick-reference table in `references/security-checklist.md`. + +### Injection (SQL, NoSQL, OS Command) ```typescript // BAD: SQL injection via string concatenation @@ -66,7 +89,7 @@ const user = await db.query('SELECT * FROM users WHERE id = $1', [userId]); const user = await prisma.user.findUnique({ where: { id: userId } }); ``` -### 2. Broken Authentication +### Broken Authentication ```typescript // Password hashing @@ -90,7 +113,7 @@ app.use(session({ })); ``` -### 3. Cross-Site Scripting (XSS) +### Cross-Site Scripting (XSS) ```typescript // BAD: Rendering user input as HTML @@ -104,7 +127,7 @@ import DOMPurify from 'dompurify'; const clean = DOMPurify.sanitize(userInput); ``` -### 4. Broken Access Control +### Broken Access Control ```typescript // Always check authorization, not just authentication @@ -124,7 +147,7 @@ app.patch('/api/tasks/:id', authenticate, async (req, res) => { }); ``` -### 5. Security Misconfiguration +### Security Misconfiguration ```typescript // Security headers (use helmet for Express) @@ -149,7 +172,7 @@ app.use(cors({ })); ``` -### 6. Sensitive Data Exposure +### Sensitive Data Exposure ```typescript // Never return sensitive fields in API responses @@ -163,6 +186,39 @@ const API_KEY = process.env.STRIPE_API_KEY; if (!API_KEY) throw new Error('STRIPE_API_KEY not configured'); ``` +### Server-Side Request Forgery (SSRF) + +Any time the server fetches a URL the user influenced — webhooks, "import from URL", image proxies, link previews — an attacker can aim it at internal services (cloud metadata, `localhost`, private IPs). + +```typescript +// BAD: fetch whatever the user gives you +await fetch(req.body.webhookUrl); + +// GOOD: allowlist scheme + host, reject if ANY resolved IP is private, forbid redirects +import { lookup } from 'node:dns/promises'; +import ipaddr from 'ipaddr.js'; + +const ALLOWED_HOSTS = new Set(['hooks.example.com']); + +async function assertSafeUrl(raw: string): Promise { + const url = new URL(raw); + if (url.protocol !== 'https:') throw new Error('https only'); + if (!ALLOWED_HOSTS.has(url.hostname)) throw new Error('host not allowed'); + // Resolve ALL records; a single private/reserved address fails the check. + const addrs = await lookup(url.hostname, { all: true }); + if (addrs.some((a) => ipaddr.parse(a.address).range() !== 'unicast')) { + throw new Error('private/reserved IP'); + } + return url; +} + +await fetch(await assertSafeUrl(req.body.webhookUrl), { redirect: 'error' }); +``` + +The `range() !== 'unicast'` check covers loopback, link-local `169.254.169.254` (cloud metadata, the #1 SSRF target), private, and unique-local ranges across IPv4 and IPv6. + +**Caveat — this still has a TOCTOU gap.** `fetch` resolves DNS again after the check, so an attacker using a short-TTL record can rebind to an internal IP between validation and connection. For high-risk surfaces, resolve once and connect to the pinned IP, or put a filtering agent in front (`request-filtering-agent` / `ssrf-req-filter`). + ## Input Validation Patterns ### Schema Validation at Boundaries @@ -240,6 +296,15 @@ npm audit reports a vulnerability When you defer a fix, document the reason and set a review date. +### Supply-Chain Hygiene + +`npm audit` catches known CVEs; it won't catch a malicious or typosquatted package. Also: + +- **Commit the lockfile** and install with `npm ci` (not `npm install`) in CI — reproducible builds, no silent version drift. +- **Review new dependencies before adding them** — maintenance, download counts, and whether they truly earn their place. Every dependency is attack surface (OWASP **A06: Vulnerable Components**, **LLM03: Supply Chain**). +- **Be wary of `postinstall` scripts** in unfamiliar packages — they run arbitrary code at install time. +- **Watch for typosquats** — `cross-env` vs `crossenv`, `react-dom` vs `reactdom`. + ## Rate Limiting ```typescript @@ -282,6 +347,36 @@ app.use('/api/auth/', rateLimit({ git diff --cached | grep -i "password\|secret\|api_key\|token" ``` +**If a secret is ever committed, rotate it.** Deleting the line or rewriting history is not enough — assume it's compromised the moment it reaches a remote. Revoke and reissue the key first, then purge it from history. + +## Securing AI / LLM Features + +If your app calls an LLM — chatbots, summarizers, agents, RAG — it inherits a new attack surface. Map it to the [OWASP Top 10 for LLM Applications (2025)](https://genai.owasp.org/llm-top-10/): + +- **Treat all model output as untrusted input (LLM05: Improper Output Handling).** Never pass LLM output straight into `eval`, SQL, a shell, `innerHTML`, or a file path. Validate and encode it exactly as you would raw user input. +- **Assume prompts can be hijacked (LLM01: Prompt Injection).** Untrusted text in the context window — a user message, a fetched web page, a PDF — can carry instructions. The system prompt is not a security boundary; enforce permissions in code, not in the prompt. +- **Keep secrets and other users' data out of prompts (LLM02 / LLM07).** Anything in the context can be echoed back. Don't put API keys, cross-tenant data, or the full system prompt where the model can repeat it. +- **Constrain tool and agent permissions (LLM06: Excessive Agency).** Scope tools to the minimum, require confirmation for destructive or irreversible actions, and validate every tool argument. +- **Bound consumption (LLM10: Unbounded Consumption).** Cap tokens, request rate, and loop/recursion depth so a crafted input can't run up cost or hang the system. +- **Isolate retrieval data (LLM08: Vector and Embedding Weaknesses).** In RAG, treat the vector store as a trust boundary: partition embeddings per tenant so one user can't retrieve another's data, and validate documents before indexing so poisoned content can't steer answers. + +```typescript +// BAD: trusting model output as a command or as markup +const sql = await llm.generate(`Write SQL for: ${userQuestion}`); +await db.query(sql); // arbitrary query execution +container.innerHTML = await llm.reply(userMessage); // stored XSS, via the model + +// GOOD: model output is data — parse defensively, then validate, then encode +let intent; +try { + intent = CommandSchema.parse(JSON.parse(await llm.replyJson(userMessage))); +} catch { + throw new ValidationError('unexpected model output'); // JSON.parse or schema failed +} +await runAllowlistedAction(intent.action, intent.params); +container.textContent = await llm.reply(userMessage); +``` + ## Security Review Checklist ```markdown @@ -300,6 +395,7 @@ git diff --cached | grep -i "password\|secret\|api_key\|token" - [ ] All user input validated at the boundary - [ ] SQL queries are parameterized - [ ] HTML output is encoded/escaped +- [ ] Server-side URL fetches are allowlisted (no SSRF to internal services) ### Data - [ ] No secrets in code or version control @@ -311,6 +407,15 @@ git diff --cached | grep -i "password\|secret\|api_key\|token" - [ ] CORS restricted to known origins - [ ] Dependencies audited for vulnerabilities - [ ] Error messages don't expose internals + +### Supply Chain +- [ ] Lockfile committed; CI installs with `npm ci` +- [ ] New dependencies reviewed (maintenance, downloads, postinstall scripts) + +### AI / LLM (if used) +- [ ] Model output treated as untrusted (no eval/SQL/innerHTML/shell) +- [ ] Secrets and other users' data kept out of prompts +- [ ] Tool/agent permissions scoped; destructive actions require confirmation ``` ## See Also @@ -325,6 +430,8 @@ For detailed security checklists and pre-commit verification steps, see `referen | "No one would try to exploit this" | Automated scanners will find it. Security by obscurity is not security. | | "The framework handles security" | Frameworks provide tools, not guarantees. You still need to use them correctly. | | "It's just a prototype" | Prototypes become production. Security habits from day one. | +| "Threat modeling is overkill here" | Five minutes of "how would I attack this?" prevents the design flaws no control can patch later. | +| "It's just LLM output, it's only text" | That "text" can be a SQL statement, a script tag, or a shell command. Treat it like any untrusted input. | ## Red Flags @@ -335,6 +442,9 @@ For detailed security checklists and pre-commit verification steps, see `referen - No rate limiting on authentication endpoints - Stack traces or internal errors exposed to users - Dependencies with known critical vulnerabilities +- Server fetches user-supplied URLs without an allowlist (SSRF) +- LLM/model output passed into a query, the DOM, a shell, or `eval` +- Secrets, PII, or the full system prompt placed inside an LLM context window ## Verification @@ -347,3 +457,5 @@ After implementing security-relevant code: - [ ] Security headers present in response (check with browser DevTools) - [ ] Error responses don't expose internal details - [ ] Rate limiting active on auth endpoints +- [ ] Server-side URL fetches validated against an allowlist (no SSRF) +- [ ] LLM/model output validated and encoded before use (if AI features present)