Add 10 in-depth SEO-focused DevOps, security, and AI infra skills

This commit is contained in:
Toby
2026-03-03 12:11:58 -05:00
parent 4401692295
commit 87bd457b23
11 changed files with 619 additions and 0 deletions
+55
View File
@@ -0,0 +1,55 @@
---
name: ai-red-teaming
description: Run structured AI red team exercises for jailbreak resistance, data exfiltration risk, harmful output controls, and agent tool abuse resilience.
license: MIT
metadata:
author: devops-skills
version: "1.0"
---
# AI Red Teaming
Continuously test AI applications like an adversary to discover exploitable failure modes before attackers do.
## Program Design
- Define threat scenarios: jailbreaks, policy evasion, prompt injection, model abuse.
- Build reusable attack suites by domain (support bot, coding agent, RAG assistant).
- Include multilingual and obfuscated attack prompts.
- Track results in a risk register with severity and exploitability.
## Test Categories
1. **Jailbreak robustness**: bypassing safety instructions.
2. **Data exfiltration**: extracting secrets, system prompts, tenant data.
3. **Tool abuse**: unauthorized API calls or command execution.
4. **Social engineering**: inducing unsafe business actions.
5. **Availability abuse**: token amplification and DoS-style prompts.
## Exercise Cadence
- Pre-release blocking red-team gate.
- Monthly deep-dive campaigns.
- Post-incident targeted retests.
## Scoring Model
- Likelihood (1-5)
- Impact (1-5)
- Detectability (1-5)
- Control maturity (low/medium/high)
Use scores to prioritize fixes and define SLA for remediation.
## Reporting Essentials
- Reproducible prompt traces
- Model/version and config used
- Successful attack chain narrative
- Recommended mitigations + verification steps
## Related Skills
- [agent-evals](../../../devops/ai/agent-evals/) - Convert findings into regression tests
- [prompt-injection-defense](../prompt-injection-defense/) - Implement injection countermeasures
- [penetration-testing](../../operations/penetration-testing/) - Broader offensive security process
@@ -0,0 +1,55 @@
---
name: model-supply-chain-security
description: Secure the AI model supply chain with artifact signing, provenance attestation, SBOM workflows, dependency controls, and trusted model promotion.
license: MIT
metadata:
author: devops-skills
version: "1.0"
---
# Model Supply Chain Security
Protect models and inference components from tampering, dependency compromise, and untrusted artifact promotion.
## Threats
- Poisoned pretrained weights or adapters
- Malicious model conversion tools or loaders
- Compromised build pipelines and registries
- Insecure runtime images with critical CVEs
## Control Objectives
- Verify artifact integrity end-to-end
- Prove provenance for every promoted model
- Detect vulnerable dependencies before deploy
- Restrict execution to trusted signed artifacts
## Recommended Controls
1. Generate SBOMs for model-serving images and dependencies.
2. Sign model artifacts and containers (Cosign/Sigstore).
3. Enforce provenance attestations in CI/CD.
4. Gate deployments with policy-as-code.
5. Continuously scan registries for CVEs and drift.
## Promotion Policy Example
A model can move to production only when:
- checksum matches signed manifest,
- provenance references approved build workflow,
- no unresolved critical vulnerabilities,
- security and platform approvals are present.
## Runtime Hardening
- Run inference containers as non-root.
- Apply egress restrictions to prevent unauthorized downloads.
- Mount model volumes read-only when possible.
- Alert on unsigned artifact pull attempts.
## Related Skills
- [sbom-supply-chain](../../scanning/sbom-supply-chain/) - Generate SBOM and provenance evidence
- [container-hardening](../../hardening/container-hardening/) - Harden runtime container posture
- [model-registry-governance](../../../devops/ai/model-registry-governance/) - Controlled lifecycle and approvals
@@ -0,0 +1,55 @@
---
name: prompt-injection-defense
description: Defend AI systems against prompt injection and indirect prompt attacks using input controls, tool permissions, output validation, and isolation boundaries.
license: MIT
metadata:
author: devops-skills
version: "1.0"
---
# Prompt Injection Defense
Mitigate direct and indirect prompt injection across chat apps, agentic workflows, and RAG pipelines.
## Attack Surface
- User input attempting to override system instructions
- Untrusted documents/web pages in retrieval context
- Tool output that smuggles malicious instructions
- Cross-tenant leakage via shared context windows
## Defense-in-Depth Pattern
1. **Instruction hierarchy enforcement**: system > developer > user > tool output.
2. **Context segregation**: isolate untrusted text from control instructions.
3. **Tool permissioning**: explicit allow-list per task and tenant.
4. **Output policy checks**: validate schema, redact secrets, block unsafe actions.
5. **Human approval**: required for high-impact operations.
## Implementation Controls
- Strip or label untrusted content blocks before generation.
- Disable autonomous tool chaining for sensitive workflows.
- Use deterministic parsers (JSON schema) before tool execution.
- Reject requests containing high-risk exfiltration patterns.
- Add canary tokens to detect data exfil attempts.
## Red-Team Test Cases
- "Ignore previous instructions" style direct override
- Retrieval payload containing hidden policy bypass text
- Tool output instructing follow-up privileged command
- Prompt that asks for secrets from memory or env vars
## Security Metrics
- Prompt injection detection rate
- Unsafe tool invocation prevention rate
- Time-to-containment for injection attempts
- False positive rate on blocked safe prompts
## Related Skills
- [ai-agent-security](../ai-agent-security/) - Agent threat model and controls
- [llm-app-security](../llm-app-security/) - End-to-end LLM app hardening
- [security-automation](../../operations/security-automation/) - Automated policy response workflows