diff --git a/README.md b/README.md index 2b590e6..64dfb55 100644 --- a/README.md +++ b/README.md @@ -216,6 +216,10 @@ No agent? No problem. Browse the skills, copy the scripts, use the configs. It's | [llm-cost-optimization](devops/ai/llm-cost-optimization/) | Cut LLM API costs with caching, batching, model routing, and self-hosting | | [llm-caching](devops/ai/llm-caching/) | Exact and semantic caching layers to reduce API calls by 30–70% | | [ai-pipeline-orchestration](devops/ai/ai-pipeline-orchestration/) | Orchestrate RAG ingestion, training, and batch inference with Prefect/Airflow | +| [llmops-platform-engineering](devops/ai/llmops-platform-engineering/) | Build enterprise LLMOps platforms with evaluation gates, promotions, rollback, and governance | +| [model-registry-governance](devops/ai/model-registry-governance/) | Define model metadata, approvals, lifecycle policy, and auditable promotion controls | +| [rag-observability-evals](devops/ai/rag-observability-evals/) | Measure retrieval quality, groundedness, hallucination risk, and RAG regressions continuously | +| [ai-sre-incident-response](devops/ai/ai-sre-incident-response/) | AI-specific SRE playbooks for model outages, quality regressions, safety incidents, and spend spikes | ### Release Management | Skill | Description | @@ -281,6 +285,9 @@ No agent? No problem. Browse the skills, copy the scripts, use the configs. It's | [ai-agent-security](security/ai/ai-agent-security/) | Defend agents against injection, tool abuse, and exfiltration | | [llm-app-security](security/ai/llm-app-security/) | Harden LLM app inputs, outputs, and tenant isolation | | [ai-security-hardening](security/ai/ai-security-hardening/) | Harden LLM deployments against prompt injection, model theft, and data exfiltration | +| [prompt-injection-defense](security/ai/prompt-injection-defense/) | Defend against direct/indirect prompt injection with isolation, tool controls, and output validation | +| [ai-red-teaming](security/ai/ai-red-teaming/) | Run adversarial AI red team programs for jailbreaks, exfiltration, and tool abuse resilience | +| [model-supply-chain-security](security/ai/model-supply-chain-security/) | Protect model artifacts with signing, provenance, SBOM workflows, and trusted promotion policies | @@ -350,6 +357,7 @@ No agent? No problem. Browse the skills, copy the scripts, use the configs. It's | [reverse-proxy](infrastructure/networking/reverse-proxy/) | nginx, Traefik | | [service-mesh](infrastructure/networking/service-mesh/) | Istio, Linkerd | | [llm-gateway](infrastructure/networking/llm-gateway/) | Unified LLM API gateway with routing, rate limiting, virtual keys, and semantic caching | +| [ai-inference-service-mesh](infrastructure/networking/ai-inference-service-mesh/) | Service mesh patterns for mTLS, canary inference routing, and resilient AI east-west traffic | ### Databases | Skill | Description | @@ -387,6 +395,8 @@ No agent? No problem. Browse the skills, copy the scripts, use the configs. It's | [llm-inference-scaling](infrastructure/local-ai/llm-inference-scaling/) | Auto-scale LLM inference clusters on Kubernetes with KEDA and GPU-aware scheduling | | [rag-infrastructure](infrastructure/local-ai/rag-infrastructure/) | Production RAG with vector stores, hybrid search, embedding pipelines, and reranking | | [llm-fine-tuning](infrastructure/local-ai/llm-fine-tuning/) | QLoRA and full fine-tuning with Axolotl, DeepSpeed, and DPO alignment on GPU clusters | +| [gpu-kubernetes-operations](infrastructure/local-ai/gpu-kubernetes-operations/) | Run GPU Kubernetes clusters with MIG, autoscaling, node health checks, and AI cost controls | +| [multi-tenant-llm-hosting](infrastructure/local-ai/multi-tenant-llm-hosting/) | Secure multi-tenant LLM hosting with quotas, isolation boundaries, and per-tenant billing controls | ### IT Operations | Skill | Description | diff --git a/devops/ai/ai-sre-incident-response/SKILL.md b/devops/ai/ai-sre-incident-response/SKILL.md new file mode 100644 index 0000000..843cd2c --- /dev/null +++ b/devops/ai/ai-sre-incident-response/SKILL.md @@ -0,0 +1,66 @@ +--- +name: ai-sre-incident-response +description: Build AI-focused SRE incident response practices for LLM outages, degraded quality, runaway cost events, and safety regressions. +license: MIT +metadata: + author: devops-skills + version: "1.0" +--- + +# AI SRE Incident Response + +Apply SRE rigor to AI systems where incidents include quality regressions, unsafe outputs, and budget explosions. + +## AI Incident Classes + +- **Availability incident**: model/provider unavailable, timeout storm. +- **Quality incident**: answer accuracy or tool success drops below SLO. +- **Safety incident**: harmful or policy-violating outputs increase. +- **Cost incident**: unexpected token or provider spend spike. + +## Severity Framework (Example) + +- **SEV1**: user-facing outage, critical compliance risk, or active data leak. +- **SEV2**: major degradation affecting key flows. +- **SEV3**: limited impact or internal-only issue. + +## Golden Signals for AI Services + +- Request success rate +- Latency (queue + generation + tool execution) +- Hallucination/groundedness proxy metrics +- Cost per minute and per tenant +- Guardrail violation rate + +## Response Playbooks + +### Model Outage +1. Freeze deployments. +2. Shift traffic to fallback model/provider. +3. Enforce stricter rate limits. +4. Communicate ETA and mitigation. + +### Quality Regression +1. Roll back prompt/model version. +2. Disable risky optimization flags. +3. Increase sampling for trace review. +4. Re-run latest eval baseline. + +### Cost Spike +1. Identify top tenants/routes/models. +2. Enable cache + cheaper fallback path. +3. Apply temporary token caps. +4. Open postmortem with prevention actions. + +## Postmortem Requirements + +- Timeline with detector and responder timestamps +- Blast radius by tenant and feature +- Missed signals and alert tuning actions +- Concrete hardening tasks with owners and due dates + +## Related Skills + +- [incident-response](../../../security/operations/incident-response/) - Standard incident process and evidence +- [alerting-oncall](../../observability/alerting-oncall/) - Paging and escalation policy +- [llm-cost-optimization](../llm-cost-optimization/) - Spend controls and efficiency patterns diff --git a/devops/ai/llmops-platform-engineering/SKILL.md b/devops/ai/llmops-platform-engineering/SKILL.md new file mode 100644 index 0000000..8a44736 --- /dev/null +++ b/devops/ai/llmops-platform-engineering/SKILL.md @@ -0,0 +1,79 @@ +--- +name: llmops-platform-engineering +description: Build production LLMOps platforms with CI/CD, model promotion workflows, evaluation gates, rollback, and governance across cloud and self-hosted inference. +license: MIT +metadata: + author: devops-skills + version: "1.0" +--- + +# LLMOps Platform Engineering + +Design and operate an internal LLM platform that supports rapid experimentation without compromising reliability, cost, or compliance. + +## Outcomes + +- Standardized path from experiment to production +- Safe model rollout with quality and safety gates +- Repeatable infra modules for inference, vector DB, and observability +- Clear ownership model across platform, app, and security teams + +## Reference Architecture + +1. **Control Plane**: model registry, prompt/version catalog, policy checks, eval pipeline. +2. **Data Plane**: inference gateway, vector database, cache, feature store. +3. **Ops Plane**: telemetry, alerting, SLO dashboards, cost analytics. +4. **Security Plane**: IAM boundaries, secret rotation, content filters, audit logs. + +## Golden Delivery Workflow + +1. Train/fine-tune or onboard provider model. +2. Register artifact and metadata (license, intended use, constraints). +3. Run automated eval suite (quality + safety + latency + cost). +4. Deploy canary behind gateway with strict traffic policy. +5. Promote after SLO and business KPI thresholds pass. +6. Keep rollback target hot for fast reversion. + +## CI/CD Design for AI Services + +- Build immutable containers with pinned dependencies and model hashes. +- Use environment promotion: `dev -> stage -> prod`. +- Fail deployment if: + - regression evals drop below baseline, + - safety tests exceed risk threshold, + - p95 latency exceeds SLO budget. +- Store deployment evidence for audits (commit SHA, eval report, approver). + +## Operational SLOs + +- Availability: `99.9%` for synchronous inference endpoints. +- Latency: p95 under product-specific target (for example, `<1200ms`). +- Cost: per-request and per-tenant budget ceilings. +- Quality: task success rate and groundedness thresholds. + +## Platform Guardrails + +- Enforce tenant quotas and model allow-lists. +- Require structured output contracts for automation paths. +- Default to low-risk model settings for critical workflows. +- Disable unconstrained tool execution in production. + +## Tooling Stack (Example) + +- **Orchestration**: Argo Workflows / GitHub Actions / Airflow. +- **Model Registry**: MLflow / custom metadata DB. +- **Gateway**: LiteLLM / Envoy-based API gateway. +- **Observability**: OpenTelemetry + Prometheus + Grafana + Langfuse. +- **Policy**: OPA/Rego for deployment and runtime checks. + +## Incident Readiness + +- Runbooks for model outage, provider timeout spikes, and cost surges. +- Chaos drills for provider failover and vector DB degradation. +- Pre-approved rollback path with one-command execution. + +## Related Skills + +- [ai-pipeline-orchestration](../ai-pipeline-orchestration/) - Orchestrate ingestion and inference workflows +- [agent-evals](../agent-evals/) - Build evaluation gates for releases +- [llm-gateway](../../../infrastructure/networking/llm-gateway/) - Route and control LLM traffic diff --git a/devops/ai/model-registry-governance/SKILL.md b/devops/ai/model-registry-governance/SKILL.md new file mode 100644 index 0000000..db7adab --- /dev/null +++ b/devops/ai/model-registry-governance/SKILL.md @@ -0,0 +1,68 @@ +--- +name: model-registry-governance +description: Establish model registry standards, governance controls, metadata schemas, approvals, and lifecycle policies for enterprise AI deployments. +license: MIT +metadata: + author: devops-skills + version: "1.0" +--- + +# Model Registry Governance + +Create a trustworthy system of record for model artifacts, prompts, adapters, and evaluation evidence. + +## Core Principles + +- **Traceability**: every production model maps to source code, data snapshot, and evaluation results. +- **Reproducibility**: builds are deterministic with pinned dependencies. +- **Policy-driven promotion**: no manual bypass for critical safety checks. +- **Lifecycle hygiene**: stale, vulnerable, or unowned models are retired automatically. + +## Required Metadata Schema + +Track at minimum: + +- Model name, semantic version, checksum, and storage URI +- Base model lineage and fine-tune method +- Training/eval datasets and time windows +- License, allowed use cases, prohibited use cases +- Security risk rating and mitigation controls +- Owner, backup owner, and escalation contact + +## Approval Workflow + +1. Registration request created from CI. +2. Security checks (artifact scan, dependency scan, provenance). +3. Evaluation package uploaded (quality, toxicity, jailbreak, bias, latency, cost). +4. Required approvals: platform + product + security (as policy dictates). +5. Promotion to stage/prod based on signed decision record. + +## Lifecycle States + +- `draft`: internal experimentation. +- `candidate`: passed baseline tests. +- `approved`: authorized for production rollout. +- `deprecated`: replacement announced, new usage blocked. +- `retired`: no serving allowed, archived for audit. + +## Governance Policies + +- Reject artifacts without SBOM/provenance. +- Block promotion if known critical CVEs remain unresolved. +- Require refreshed evals after prompt/template changes. +- Expire approvals after a configurable period (for example 90 days). + +## Audit Readiness + +Maintain immutable records of: + +- Who approved and when +- Which policy checks executed +- Which exceptions were granted +- What model/version served each customer request window + +## Related Skills + +- [sbom-supply-chain](../../../security/scanning/sbom-supply-chain/) - Provenance and signing +- [policy-as-code](../../../compliance/governance/policy-as-code/) - Enforce governance with policy engines +- [llm-fine-tuning](../../../infrastructure/local-ai/llm-fine-tuning/) - Version adapters and training outputs diff --git a/devops/ai/rag-observability-evals/SKILL.md b/devops/ai/rag-observability-evals/SKILL.md new file mode 100644 index 0000000..2464ff9 --- /dev/null +++ b/devops/ai/rag-observability-evals/SKILL.md @@ -0,0 +1,64 @@ +--- +name: rag-observability-evals +description: Monitor and evaluate RAG systems with retrieval quality metrics, groundedness checks, hallucination detection, and continuous regression testing. +license: MIT +metadata: + author: devops-skills + version: "1.0" +--- + +# RAG Observability and Evaluations + +Run retrieval-augmented generation like a measurable production system, not a black box. + +## What to Measure + +### Retrieval Quality +- Recall@k and MRR for top-k chunks +- Citation coverage and source freshness +- Embedding drift and index staleness + +### Generation Quality +- Groundedness score (answer supported by retrieved context) +- Hallucination rate by route/use case +- Instruction adherence and format validity + +### Reliability and Cost +- p50/p95 latency split by retrieval vs generation +- Token usage per stage +- Cache hit rate and cost per successful answer + +## Evaluation Pipeline + +1. Curate a benchmark set with gold answers and source docs. +2. Run nightly offline evals for every retriever/model configuration. +3. Execute online shadow evals on sampled production traffic. +4. Gate releases on minimum quality + safety + latency thresholds. + +## Alerting Strategy + +Page on: +- sharp decline in groundedness, +- spike in unanswered or fallback responses, +- index freshness SLA breach, +- cost-per-answer anomaly. + +## Practical Guardrails + +- Force citations for high-risk domains. +- Return abstain/fallback when confidence is below threshold. +- Re-rank retrieved chunks before final generation. +- Use query rewriting only with strict regression tests. + +## Incident Triage Checklist + +- Did embedding model change? +- Did chunking/indexing logic change? +- Did source corpus ingestion fail? +- Did gateway route to unintended model tier? + +## Related Skills + +- [rag-infrastructure](../../../infrastructure/local-ai/rag-infrastructure/) - Deploy robust RAG backends +- [agent-observability](../agent-observability/) - Instrument requests, traces, and costs +- [agent-evals](../agent-evals/) - Build repeatable eval suites diff --git a/infrastructure/local-ai/gpu-kubernetes-operations/SKILL.md b/infrastructure/local-ai/gpu-kubernetes-operations/SKILL.md new file mode 100644 index 0000000..e60073c --- /dev/null +++ b/infrastructure/local-ai/gpu-kubernetes-operations/SKILL.md @@ -0,0 +1,58 @@ +--- +name: gpu-kubernetes-operations +description: Operate GPU-backed Kubernetes clusters for AI inference and training with scheduling, autoscaling, node health, MIG partitioning, and cost controls. +license: MIT +metadata: + author: devops-skills + version: "1.0" +--- + +# GPU Kubernetes Operations + +Run resilient and cost-efficient GPU clusters for production AI workloads. + +## Key Capabilities + +- NVIDIA device plugin and GPU operator lifecycle +- MIG partitioning for multi-workload efficiency +- GPU-aware autoscaling (KEDA/cluster autoscaler) +- Node health checks and proactive remediation + +## Cluster Baseline + +- Dedicated GPU node pools with taints and tolerations +- Runtime class and driver/toolkit compatibility checks +- Local SSD or high-throughput network storage for model weights +- DCGM metrics exported to Prometheus + +## Scheduling Patterns + +- Use node affinity by GPU type (A10/L4/A100/H100). +- Separate latency-critical inference from batch training. +- Pin model replicas with anti-affinity for availability. +- Reserve headroom for failover and rolling updates. + +## Autoscaling Strategy + +- Scale on queue depth + GPU utilization, not CPU alone. +- Warm spare replicas for large model cold-start mitigation. +- Cap burst scaling to avoid quota exhaustion. + +## Reliability Checks + +- ECC error and Xid monitoring +- GPU memory pressure alerts +- Driver mismatch detection during upgrades +- Pod preemption impact analysis + +## Cost Optimization + +- Prefer MIG slices for smaller inference services. +- Schedule batch jobs in off-peak windows. +- Route low-priority traffic to cheaper model tiers. + +## Related Skills + +- [llm-inference-scaling](../llm-inference-scaling/) - Autoscale inference workloads +- [model-serving-kubernetes](../../../devops/orchestration/model-serving-kubernetes/) - Production model serving patterns +- [gpu-server-management](../../servers/gpu-server-management/) - Host-level GPU management fundamentals diff --git a/infrastructure/local-ai/multi-tenant-llm-hosting/SKILL.md b/infrastructure/local-ai/multi-tenant-llm-hosting/SKILL.md new file mode 100644 index 0000000..1859b0f --- /dev/null +++ b/infrastructure/local-ai/multi-tenant-llm-hosting/SKILL.md @@ -0,0 +1,55 @@ +--- +name: multi-tenant-llm-hosting +description: Design secure, multi-tenant LLM hosting platforms with tenant isolation, quotas, billing attribution, noisy-neighbor protection, and per-tenant policy controls. +license: MIT +metadata: + author: devops-skills + version: "1.0" +--- + +# Multi-Tenant LLM Hosting + +Host many teams/customers on shared inference infrastructure without sacrificing security, performance, or cost governance. + +## Isolation Model + +- Strong tenant identity on every request +- Per-tenant API keys and scoped model access +- Namespace or workload isolation for high-risk tenants +- Strict data retention and log partitioning controls + +## Noisy-Neighbor Controls + +- Per-tenant RPM/TPM limits +- Concurrency caps and queue isolation +- Fair scheduling with weighted priority classes +- Backpressure and graceful degradation policies + +## Billing and Chargeback + +Track per-tenant: +- prompt/completion/cached tokens, +- model type and route, +- latency and success rate, +- cost with markup or internal transfer pricing. + +## Security Baseline + +- Encrypt data in transit and at rest. +- Disallow cross-tenant cache leakage. +- Restrict debug data access by role. +- Audit all privileged administrative actions. + +## Operational Runbook + +1. Onboard tenant with policy template. +2. Issue virtual key and quota profile. +3. Validate observability and billing tags. +4. Run tenant-specific load/safety tests. +5. Enable production traffic with canary limits. + +## Related Skills + +- [llm-gateway](../../networking/llm-gateway/) - Key management and traffic routing +- [llm-cost-optimization](../../../devops/ai/llm-cost-optimization/) - Cost controls and optimization tactics +- [zero-trust](../../../security/network/zero-trust/) - Identity-centric network and access patterns diff --git a/infrastructure/networking/ai-inference-service-mesh/SKILL.md b/infrastructure/networking/ai-inference-service-mesh/SKILL.md new file mode 100644 index 0000000..272b5c4 --- /dev/null +++ b/infrastructure/networking/ai-inference-service-mesh/SKILL.md @@ -0,0 +1,54 @@ +--- +name: ai-inference-service-mesh +description: Use service mesh patterns for AI inference traffic management, mTLS, canary releases, policy enforcement, and cross-cluster resilience. +license: MIT +metadata: + author: devops-skills + version: "1.0" +--- + +# AI Inference Service Mesh + +Apply Istio/Linkerd mesh controls to secure and optimize east-west AI traffic across inference microservices. + +## Why Mesh for AI + +- Enforce mTLS between gateway, retriever, reranker, and model services +- Apply fine-grained traffic policies without app code changes +- Run progressive delivery for model-serving backends +- Observe latency hops for retrieval + generation chains + +## Core Patterns + +### Security +- mTLS strict mode cluster-wide +- AuthorizationPolicy per service account +- Egress policies for approved model endpoints only + +### Traffic Management +- Canary by header or percentage for new model versions +- Retry budgets tuned for long-running streaming requests +- Circuit breakers to protect overloaded inference backends + +### Resilience +- Outlier detection on failing pods +- Locality-aware routing in multi-zone clusters +- Failover to secondary cluster/provider + +## Observability + +- Capture distributed traces across the full AI request path +- Emit service-level and route-level p95/p99 latency +- Segment metrics by model and tenant labels + +## Pitfalls to Avoid + +- Aggressive timeouts that break streaming responses +- Blanket retries that amplify expensive generation calls +- Missing identity boundaries between tenant-facing and internal services + +## Related Skills + +- [service-mesh](../service-mesh/) - Foundational mesh concepts +- [llm-gateway](../llm-gateway/) - North-south API gateway controls +- [opentelemetry](../../../devops/observability/opentelemetry/) - End-to-end tracing and metrics diff --git a/security/ai/ai-red-teaming/SKILL.md b/security/ai/ai-red-teaming/SKILL.md new file mode 100644 index 0000000..c52ae79 --- /dev/null +++ b/security/ai/ai-red-teaming/SKILL.md @@ -0,0 +1,55 @@ +--- +name: ai-red-teaming +description: Run structured AI red team exercises for jailbreak resistance, data exfiltration risk, harmful output controls, and agent tool abuse resilience. +license: MIT +metadata: + author: devops-skills + version: "1.0" +--- + +# AI Red Teaming + +Continuously test AI applications like an adversary to discover exploitable failure modes before attackers do. + +## Program Design + +- Define threat scenarios: jailbreaks, policy evasion, prompt injection, model abuse. +- Build reusable attack suites by domain (support bot, coding agent, RAG assistant). +- Include multilingual and obfuscated attack prompts. +- Track results in a risk register with severity and exploitability. + +## Test Categories + +1. **Jailbreak robustness**: bypassing safety instructions. +2. **Data exfiltration**: extracting secrets, system prompts, tenant data. +3. **Tool abuse**: unauthorized API calls or command execution. +4. **Social engineering**: inducing unsafe business actions. +5. **Availability abuse**: token amplification and DoS-style prompts. + +## Exercise Cadence + +- Pre-release blocking red-team gate. +- Monthly deep-dive campaigns. +- Post-incident targeted retests. + +## Scoring Model + +- Likelihood (1-5) +- Impact (1-5) +- Detectability (1-5) +- Control maturity (low/medium/high) + +Use scores to prioritize fixes and define SLA for remediation. + +## Reporting Essentials + +- Reproducible prompt traces +- Model/version and config used +- Successful attack chain narrative +- Recommended mitigations + verification steps + +## Related Skills + +- [agent-evals](../../../devops/ai/agent-evals/) - Convert findings into regression tests +- [prompt-injection-defense](../prompt-injection-defense/) - Implement injection countermeasures +- [penetration-testing](../../operations/penetration-testing/) - Broader offensive security process diff --git a/security/ai/model-supply-chain-security/SKILL.md b/security/ai/model-supply-chain-security/SKILL.md new file mode 100644 index 0000000..266f576 --- /dev/null +++ b/security/ai/model-supply-chain-security/SKILL.md @@ -0,0 +1,55 @@ +--- +name: model-supply-chain-security +description: Secure the AI model supply chain with artifact signing, provenance attestation, SBOM workflows, dependency controls, and trusted model promotion. +license: MIT +metadata: + author: devops-skills + version: "1.0" +--- + +# Model Supply Chain Security + +Protect models and inference components from tampering, dependency compromise, and untrusted artifact promotion. + +## Threats + +- Poisoned pretrained weights or adapters +- Malicious model conversion tools or loaders +- Compromised build pipelines and registries +- Insecure runtime images with critical CVEs + +## Control Objectives + +- Verify artifact integrity end-to-end +- Prove provenance for every promoted model +- Detect vulnerable dependencies before deploy +- Restrict execution to trusted signed artifacts + +## Recommended Controls + +1. Generate SBOMs for model-serving images and dependencies. +2. Sign model artifacts and containers (Cosign/Sigstore). +3. Enforce provenance attestations in CI/CD. +4. Gate deployments with policy-as-code. +5. Continuously scan registries for CVEs and drift. + +## Promotion Policy Example + +A model can move to production only when: +- checksum matches signed manifest, +- provenance references approved build workflow, +- no unresolved critical vulnerabilities, +- security and platform approvals are present. + +## Runtime Hardening + +- Run inference containers as non-root. +- Apply egress restrictions to prevent unauthorized downloads. +- Mount model volumes read-only when possible. +- Alert on unsigned artifact pull attempts. + +## Related Skills + +- [sbom-supply-chain](../../scanning/sbom-supply-chain/) - Generate SBOM and provenance evidence +- [container-hardening](../../hardening/container-hardening/) - Harden runtime container posture +- [model-registry-governance](../../../devops/ai/model-registry-governance/) - Controlled lifecycle and approvals diff --git a/security/ai/prompt-injection-defense/SKILL.md b/security/ai/prompt-injection-defense/SKILL.md new file mode 100644 index 0000000..a6aa22e --- /dev/null +++ b/security/ai/prompt-injection-defense/SKILL.md @@ -0,0 +1,55 @@ +--- +name: prompt-injection-defense +description: Defend AI systems against prompt injection and indirect prompt attacks using input controls, tool permissions, output validation, and isolation boundaries. +license: MIT +metadata: + author: devops-skills + version: "1.0" +--- + +# Prompt Injection Defense + +Mitigate direct and indirect prompt injection across chat apps, agentic workflows, and RAG pipelines. + +## Attack Surface + +- User input attempting to override system instructions +- Untrusted documents/web pages in retrieval context +- Tool output that smuggles malicious instructions +- Cross-tenant leakage via shared context windows + +## Defense-in-Depth Pattern + +1. **Instruction hierarchy enforcement**: system > developer > user > tool output. +2. **Context segregation**: isolate untrusted text from control instructions. +3. **Tool permissioning**: explicit allow-list per task and tenant. +4. **Output policy checks**: validate schema, redact secrets, block unsafe actions. +5. **Human approval**: required for high-impact operations. + +## Implementation Controls + +- Strip or label untrusted content blocks before generation. +- Disable autonomous tool chaining for sensitive workflows. +- Use deterministic parsers (JSON schema) before tool execution. +- Reject requests containing high-risk exfiltration patterns. +- Add canary tokens to detect data exfil attempts. + +## Red-Team Test Cases + +- "Ignore previous instructions" style direct override +- Retrieval payload containing hidden policy bypass text +- Tool output instructing follow-up privileged command +- Prompt that asks for secrets from memory or env vars + +## Security Metrics + +- Prompt injection detection rate +- Unsafe tool invocation prevention rate +- Time-to-containment for injection attempts +- False positive rate on blocked safe prompts + +## Related Skills + +- [ai-agent-security](../ai-agent-security/) - Agent threat model and controls +- [llm-app-security](../llm-app-security/) - End-to-end LLM app hardening +- [security-automation](../../operations/security-automation/) - Automated policy response workflows