mirror of
https://github.com/BagelHole/DevOps-Security-Agent-Skills.git
synced 2026-08-22 12:49:53 +02:00
Add 10 in-depth SEO-focused DevOps, security, and AI infra skills
This commit is contained in:
@@ -0,0 +1,55 @@
|
||||
---
|
||||
name: ai-red-teaming
|
||||
description: Run structured AI red team exercises for jailbreak resistance, data exfiltration risk, harmful output controls, and agent tool abuse resilience.
|
||||
license: MIT
|
||||
metadata:
|
||||
author: devops-skills
|
||||
version: "1.0"
|
||||
---
|
||||
|
||||
# AI Red Teaming
|
||||
|
||||
Continuously test AI applications like an adversary to discover exploitable failure modes before attackers do.
|
||||
|
||||
## Program Design
|
||||
|
||||
- Define threat scenarios: jailbreaks, policy evasion, prompt injection, model abuse.
|
||||
- Build reusable attack suites by domain (support bot, coding agent, RAG assistant).
|
||||
- Include multilingual and obfuscated attack prompts.
|
||||
- Track results in a risk register with severity and exploitability.
|
||||
|
||||
## Test Categories
|
||||
|
||||
1. **Jailbreak robustness**: bypassing safety instructions.
|
||||
2. **Data exfiltration**: extracting secrets, system prompts, tenant data.
|
||||
3. **Tool abuse**: unauthorized API calls or command execution.
|
||||
4. **Social engineering**: inducing unsafe business actions.
|
||||
5. **Availability abuse**: token amplification and DoS-style prompts.
|
||||
|
||||
## Exercise Cadence
|
||||
|
||||
- Pre-release blocking red-team gate.
|
||||
- Monthly deep-dive campaigns.
|
||||
- Post-incident targeted retests.
|
||||
|
||||
## Scoring Model
|
||||
|
||||
- Likelihood (1-5)
|
||||
- Impact (1-5)
|
||||
- Detectability (1-5)
|
||||
- Control maturity (low/medium/high)
|
||||
|
||||
Use scores to prioritize fixes and define SLA for remediation.
|
||||
|
||||
## Reporting Essentials
|
||||
|
||||
- Reproducible prompt traces
|
||||
- Model/version and config used
|
||||
- Successful attack chain narrative
|
||||
- Recommended mitigations + verification steps
|
||||
|
||||
## Related Skills
|
||||
|
||||
- [agent-evals](../../../devops/ai/agent-evals/) - Convert findings into regression tests
|
||||
- [prompt-injection-defense](../prompt-injection-defense/) - Implement injection countermeasures
|
||||
- [penetration-testing](../../operations/penetration-testing/) - Broader offensive security process
|
||||
@@ -0,0 +1,55 @@
|
||||
---
|
||||
name: model-supply-chain-security
|
||||
description: Secure the AI model supply chain with artifact signing, provenance attestation, SBOM workflows, dependency controls, and trusted model promotion.
|
||||
license: MIT
|
||||
metadata:
|
||||
author: devops-skills
|
||||
version: "1.0"
|
||||
---
|
||||
|
||||
# Model Supply Chain Security
|
||||
|
||||
Protect models and inference components from tampering, dependency compromise, and untrusted artifact promotion.
|
||||
|
||||
## Threats
|
||||
|
||||
- Poisoned pretrained weights or adapters
|
||||
- Malicious model conversion tools or loaders
|
||||
- Compromised build pipelines and registries
|
||||
- Insecure runtime images with critical CVEs
|
||||
|
||||
## Control Objectives
|
||||
|
||||
- Verify artifact integrity end-to-end
|
||||
- Prove provenance for every promoted model
|
||||
- Detect vulnerable dependencies before deploy
|
||||
- Restrict execution to trusted signed artifacts
|
||||
|
||||
## Recommended Controls
|
||||
|
||||
1. Generate SBOMs for model-serving images and dependencies.
|
||||
2. Sign model artifacts and containers (Cosign/Sigstore).
|
||||
3. Enforce provenance attestations in CI/CD.
|
||||
4. Gate deployments with policy-as-code.
|
||||
5. Continuously scan registries for CVEs and drift.
|
||||
|
||||
## Promotion Policy Example
|
||||
|
||||
A model can move to production only when:
|
||||
- checksum matches signed manifest,
|
||||
- provenance references approved build workflow,
|
||||
- no unresolved critical vulnerabilities,
|
||||
- security and platform approvals are present.
|
||||
|
||||
## Runtime Hardening
|
||||
|
||||
- Run inference containers as non-root.
|
||||
- Apply egress restrictions to prevent unauthorized downloads.
|
||||
- Mount model volumes read-only when possible.
|
||||
- Alert on unsigned artifact pull attempts.
|
||||
|
||||
## Related Skills
|
||||
|
||||
- [sbom-supply-chain](../../scanning/sbom-supply-chain/) - Generate SBOM and provenance evidence
|
||||
- [container-hardening](../../hardening/container-hardening/) - Harden runtime container posture
|
||||
- [model-registry-governance](../../../devops/ai/model-registry-governance/) - Controlled lifecycle and approvals
|
||||
@@ -0,0 +1,55 @@
|
||||
---
|
||||
name: prompt-injection-defense
|
||||
description: Defend AI systems against prompt injection and indirect prompt attacks using input controls, tool permissions, output validation, and isolation boundaries.
|
||||
license: MIT
|
||||
metadata:
|
||||
author: devops-skills
|
||||
version: "1.0"
|
||||
---
|
||||
|
||||
# Prompt Injection Defense
|
||||
|
||||
Mitigate direct and indirect prompt injection across chat apps, agentic workflows, and RAG pipelines.
|
||||
|
||||
## Attack Surface
|
||||
|
||||
- User input attempting to override system instructions
|
||||
- Untrusted documents/web pages in retrieval context
|
||||
- Tool output that smuggles malicious instructions
|
||||
- Cross-tenant leakage via shared context windows
|
||||
|
||||
## Defense-in-Depth Pattern
|
||||
|
||||
1. **Instruction hierarchy enforcement**: system > developer > user > tool output.
|
||||
2. **Context segregation**: isolate untrusted text from control instructions.
|
||||
3. **Tool permissioning**: explicit allow-list per task and tenant.
|
||||
4. **Output policy checks**: validate schema, redact secrets, block unsafe actions.
|
||||
5. **Human approval**: required for high-impact operations.
|
||||
|
||||
## Implementation Controls
|
||||
|
||||
- Strip or label untrusted content blocks before generation.
|
||||
- Disable autonomous tool chaining for sensitive workflows.
|
||||
- Use deterministic parsers (JSON schema) before tool execution.
|
||||
- Reject requests containing high-risk exfiltration patterns.
|
||||
- Add canary tokens to detect data exfil attempts.
|
||||
|
||||
## Red-Team Test Cases
|
||||
|
||||
- "Ignore previous instructions" style direct override
|
||||
- Retrieval payload containing hidden policy bypass text
|
||||
- Tool output instructing follow-up privileged command
|
||||
- Prompt that asks for secrets from memory or env vars
|
||||
|
||||
## Security Metrics
|
||||
|
||||
- Prompt injection detection rate
|
||||
- Unsafe tool invocation prevention rate
|
||||
- Time-to-containment for injection attempts
|
||||
- False positive rate on blocked safe prompts
|
||||
|
||||
## Related Skills
|
||||
|
||||
- [ai-agent-security](../ai-agent-security/) - Agent threat model and controls
|
||||
- [llm-app-security](../llm-app-security/) - End-to-end LLM app hardening
|
||||
- [security-automation](../../operations/security-automation/) - Automated policy response workflows
|
||||
Reference in New Issue
Block a user