Merge pull request #4 from BagelHole/codex/add-new-high-ranking-seo-skills-to-readme

This commit is contained in:
Toby
2026-03-03 12:14:50 -05:00
committed by GitHub
11 changed files with 619 additions and 0 deletions
+10
View File
@@ -216,6 +216,10 @@ No agent? No problem. Browse the skills, copy the scripts, use the configs. It's
| [llm-cost-optimization](devops/ai/llm-cost-optimization/) | Cut LLM API costs with caching, batching, model routing, and self-hosting | | [llm-cost-optimization](devops/ai/llm-cost-optimization/) | Cut LLM API costs with caching, batching, model routing, and self-hosting |
| [llm-caching](devops/ai/llm-caching/) | Exact and semantic caching layers to reduce API calls by 3070% | | [llm-caching](devops/ai/llm-caching/) | Exact and semantic caching layers to reduce API calls by 3070% |
| [ai-pipeline-orchestration](devops/ai/ai-pipeline-orchestration/) | Orchestrate RAG ingestion, training, and batch inference with Prefect/Airflow | | [ai-pipeline-orchestration](devops/ai/ai-pipeline-orchestration/) | Orchestrate RAG ingestion, training, and batch inference with Prefect/Airflow |
| [llmops-platform-engineering](devops/ai/llmops-platform-engineering/) | Build enterprise LLMOps platforms with evaluation gates, promotions, rollback, and governance |
| [model-registry-governance](devops/ai/model-registry-governance/) | Define model metadata, approvals, lifecycle policy, and auditable promotion controls |
| [rag-observability-evals](devops/ai/rag-observability-evals/) | Measure retrieval quality, groundedness, hallucination risk, and RAG regressions continuously |
| [ai-sre-incident-response](devops/ai/ai-sre-incident-response/) | AI-specific SRE playbooks for model outages, quality regressions, safety incidents, and spend spikes |
### Release Management ### Release Management
| Skill | Description | | Skill | Description |
@@ -281,6 +285,9 @@ No agent? No problem. Browse the skills, copy the scripts, use the configs. It's
| [ai-agent-security](security/ai/ai-agent-security/) | Defend agents against injection, tool abuse, and exfiltration | | [ai-agent-security](security/ai/ai-agent-security/) | Defend agents against injection, tool abuse, and exfiltration |
| [llm-app-security](security/ai/llm-app-security/) | Harden LLM app inputs, outputs, and tenant isolation | | [llm-app-security](security/ai/llm-app-security/) | Harden LLM app inputs, outputs, and tenant isolation |
| [ai-security-hardening](security/ai/ai-security-hardening/) | Harden LLM deployments against prompt injection, model theft, and data exfiltration | | [ai-security-hardening](security/ai/ai-security-hardening/) | Harden LLM deployments against prompt injection, model theft, and data exfiltration |
| [prompt-injection-defense](security/ai/prompt-injection-defense/) | Defend against direct/indirect prompt injection with isolation, tool controls, and output validation |
| [ai-red-teaming](security/ai/ai-red-teaming/) | Run adversarial AI red team programs for jailbreaks, exfiltration, and tool abuse resilience |
| [model-supply-chain-security](security/ai/model-supply-chain-security/) | Protect model artifacts with signing, provenance, SBOM workflows, and trusted promotion policies |
</details> </details>
@@ -350,6 +357,7 @@ No agent? No problem. Browse the skills, copy the scripts, use the configs. It's
| [reverse-proxy](infrastructure/networking/reverse-proxy/) | nginx, Traefik | | [reverse-proxy](infrastructure/networking/reverse-proxy/) | nginx, Traefik |
| [service-mesh](infrastructure/networking/service-mesh/) | Istio, Linkerd | | [service-mesh](infrastructure/networking/service-mesh/) | Istio, Linkerd |
| [llm-gateway](infrastructure/networking/llm-gateway/) | Unified LLM API gateway with routing, rate limiting, virtual keys, and semantic caching | | [llm-gateway](infrastructure/networking/llm-gateway/) | Unified LLM API gateway with routing, rate limiting, virtual keys, and semantic caching |
| [ai-inference-service-mesh](infrastructure/networking/ai-inference-service-mesh/) | Service mesh patterns for mTLS, canary inference routing, and resilient AI east-west traffic |
### Databases ### Databases
| Skill | Description | | Skill | Description |
@@ -387,6 +395,8 @@ No agent? No problem. Browse the skills, copy the scripts, use the configs. It's
| [llm-inference-scaling](infrastructure/local-ai/llm-inference-scaling/) | Auto-scale LLM inference clusters on Kubernetes with KEDA and GPU-aware scheduling | | [llm-inference-scaling](infrastructure/local-ai/llm-inference-scaling/) | Auto-scale LLM inference clusters on Kubernetes with KEDA and GPU-aware scheduling |
| [rag-infrastructure](infrastructure/local-ai/rag-infrastructure/) | Production RAG with vector stores, hybrid search, embedding pipelines, and reranking | | [rag-infrastructure](infrastructure/local-ai/rag-infrastructure/) | Production RAG with vector stores, hybrid search, embedding pipelines, and reranking |
| [llm-fine-tuning](infrastructure/local-ai/llm-fine-tuning/) | QLoRA and full fine-tuning with Axolotl, DeepSpeed, and DPO alignment on GPU clusters | | [llm-fine-tuning](infrastructure/local-ai/llm-fine-tuning/) | QLoRA and full fine-tuning with Axolotl, DeepSpeed, and DPO alignment on GPU clusters |
| [gpu-kubernetes-operations](infrastructure/local-ai/gpu-kubernetes-operations/) | Run GPU Kubernetes clusters with MIG, autoscaling, node health checks, and AI cost controls |
| [multi-tenant-llm-hosting](infrastructure/local-ai/multi-tenant-llm-hosting/) | Secure multi-tenant LLM hosting with quotas, isolation boundaries, and per-tenant billing controls |
### IT Operations ### IT Operations
| Skill | Description | | Skill | Description |
@@ -0,0 +1,66 @@
---
name: ai-sre-incident-response
description: Build AI-focused SRE incident response practices for LLM outages, degraded quality, runaway cost events, and safety regressions.
license: MIT
metadata:
author: devops-skills
version: "1.0"
---
# AI SRE Incident Response
Apply SRE rigor to AI systems where incidents include quality regressions, unsafe outputs, and budget explosions.
## AI Incident Classes
- **Availability incident**: model/provider unavailable, timeout storm.
- **Quality incident**: answer accuracy or tool success drops below SLO.
- **Safety incident**: harmful or policy-violating outputs increase.
- **Cost incident**: unexpected token or provider spend spike.
## Severity Framework (Example)
- **SEV1**: user-facing outage, critical compliance risk, or active data leak.
- **SEV2**: major degradation affecting key flows.
- **SEV3**: limited impact or internal-only issue.
## Golden Signals for AI Services
- Request success rate
- Latency (queue + generation + tool execution)
- Hallucination/groundedness proxy metrics
- Cost per minute and per tenant
- Guardrail violation rate
## Response Playbooks
### Model Outage
1. Freeze deployments.
2. Shift traffic to fallback model/provider.
3. Enforce stricter rate limits.
4. Communicate ETA and mitigation.
### Quality Regression
1. Roll back prompt/model version.
2. Disable risky optimization flags.
3. Increase sampling for trace review.
4. Re-run latest eval baseline.
### Cost Spike
1. Identify top tenants/routes/models.
2. Enable cache + cheaper fallback path.
3. Apply temporary token caps.
4. Open postmortem with prevention actions.
## Postmortem Requirements
- Timeline with detector and responder timestamps
- Blast radius by tenant and feature
- Missed signals and alert tuning actions
- Concrete hardening tasks with owners and due dates
## Related Skills
- [incident-response](../../../security/operations/incident-response/) - Standard incident process and evidence
- [alerting-oncall](../../observability/alerting-oncall/) - Paging and escalation policy
- [llm-cost-optimization](../llm-cost-optimization/) - Spend controls and efficiency patterns
@@ -0,0 +1,79 @@
---
name: llmops-platform-engineering
description: Build production LLMOps platforms with CI/CD, model promotion workflows, evaluation gates, rollback, and governance across cloud and self-hosted inference.
license: MIT
metadata:
author: devops-skills
version: "1.0"
---
# LLMOps Platform Engineering
Design and operate an internal LLM platform that supports rapid experimentation without compromising reliability, cost, or compliance.
## Outcomes
- Standardized path from experiment to production
- Safe model rollout with quality and safety gates
- Repeatable infra modules for inference, vector DB, and observability
- Clear ownership model across platform, app, and security teams
## Reference Architecture
1. **Control Plane**: model registry, prompt/version catalog, policy checks, eval pipeline.
2. **Data Plane**: inference gateway, vector database, cache, feature store.
3. **Ops Plane**: telemetry, alerting, SLO dashboards, cost analytics.
4. **Security Plane**: IAM boundaries, secret rotation, content filters, audit logs.
## Golden Delivery Workflow
1. Train/fine-tune or onboard provider model.
2. Register artifact and metadata (license, intended use, constraints).
3. Run automated eval suite (quality + safety + latency + cost).
4. Deploy canary behind gateway with strict traffic policy.
5. Promote after SLO and business KPI thresholds pass.
6. Keep rollback target hot for fast reversion.
## CI/CD Design for AI Services
- Build immutable containers with pinned dependencies and model hashes.
- Use environment promotion: `dev -> stage -> prod`.
- Fail deployment if:
- regression evals drop below baseline,
- safety tests exceed risk threshold,
- p95 latency exceeds SLO budget.
- Store deployment evidence for audits (commit SHA, eval report, approver).
## Operational SLOs
- Availability: `99.9%` for synchronous inference endpoints.
- Latency: p95 under product-specific target (for example, `<1200ms`).
- Cost: per-request and per-tenant budget ceilings.
- Quality: task success rate and groundedness thresholds.
## Platform Guardrails
- Enforce tenant quotas and model allow-lists.
- Require structured output contracts for automation paths.
- Default to low-risk model settings for critical workflows.
- Disable unconstrained tool execution in production.
## Tooling Stack (Example)
- **Orchestration**: Argo Workflows / GitHub Actions / Airflow.
- **Model Registry**: MLflow / custom metadata DB.
- **Gateway**: LiteLLM / Envoy-based API gateway.
- **Observability**: OpenTelemetry + Prometheus + Grafana + Langfuse.
- **Policy**: OPA/Rego for deployment and runtime checks.
## Incident Readiness
- Runbooks for model outage, provider timeout spikes, and cost surges.
- Chaos drills for provider failover and vector DB degradation.
- Pre-approved rollback path with one-command execution.
## Related Skills
- [ai-pipeline-orchestration](../ai-pipeline-orchestration/) - Orchestrate ingestion and inference workflows
- [agent-evals](../agent-evals/) - Build evaluation gates for releases
- [llm-gateway](../../../infrastructure/networking/llm-gateway/) - Route and control LLM traffic
@@ -0,0 +1,68 @@
---
name: model-registry-governance
description: Establish model registry standards, governance controls, metadata schemas, approvals, and lifecycle policies for enterprise AI deployments.
license: MIT
metadata:
author: devops-skills
version: "1.0"
---
# Model Registry Governance
Create a trustworthy system of record for model artifacts, prompts, adapters, and evaluation evidence.
## Core Principles
- **Traceability**: every production model maps to source code, data snapshot, and evaluation results.
- **Reproducibility**: builds are deterministic with pinned dependencies.
- **Policy-driven promotion**: no manual bypass for critical safety checks.
- **Lifecycle hygiene**: stale, vulnerable, or unowned models are retired automatically.
## Required Metadata Schema
Track at minimum:
- Model name, semantic version, checksum, and storage URI
- Base model lineage and fine-tune method
- Training/eval datasets and time windows
- License, allowed use cases, prohibited use cases
- Security risk rating and mitigation controls
- Owner, backup owner, and escalation contact
## Approval Workflow
1. Registration request created from CI.
2. Security checks (artifact scan, dependency scan, provenance).
3. Evaluation package uploaded (quality, toxicity, jailbreak, bias, latency, cost).
4. Required approvals: platform + product + security (as policy dictates).
5. Promotion to stage/prod based on signed decision record.
## Lifecycle States
- `draft`: internal experimentation.
- `candidate`: passed baseline tests.
- `approved`: authorized for production rollout.
- `deprecated`: replacement announced, new usage blocked.
- `retired`: no serving allowed, archived for audit.
## Governance Policies
- Reject artifacts without SBOM/provenance.
- Block promotion if known critical CVEs remain unresolved.
- Require refreshed evals after prompt/template changes.
- Expire approvals after a configurable period (for example 90 days).
## Audit Readiness
Maintain immutable records of:
- Who approved and when
- Which policy checks executed
- Which exceptions were granted
- What model/version served each customer request window
## Related Skills
- [sbom-supply-chain](../../../security/scanning/sbom-supply-chain/) - Provenance and signing
- [policy-as-code](../../../compliance/governance/policy-as-code/) - Enforce governance with policy engines
- [llm-fine-tuning](../../../infrastructure/local-ai/llm-fine-tuning/) - Version adapters and training outputs
@@ -0,0 +1,64 @@
---
name: rag-observability-evals
description: Monitor and evaluate RAG systems with retrieval quality metrics, groundedness checks, hallucination detection, and continuous regression testing.
license: MIT
metadata:
author: devops-skills
version: "1.0"
---
# RAG Observability and Evaluations
Run retrieval-augmented generation like a measurable production system, not a black box.
## What to Measure
### Retrieval Quality
- Recall@k and MRR for top-k chunks
- Citation coverage and source freshness
- Embedding drift and index staleness
### Generation Quality
- Groundedness score (answer supported by retrieved context)
- Hallucination rate by route/use case
- Instruction adherence and format validity
### Reliability and Cost
- p50/p95 latency split by retrieval vs generation
- Token usage per stage
- Cache hit rate and cost per successful answer
## Evaluation Pipeline
1. Curate a benchmark set with gold answers and source docs.
2. Run nightly offline evals for every retriever/model configuration.
3. Execute online shadow evals on sampled production traffic.
4. Gate releases on minimum quality + safety + latency thresholds.
## Alerting Strategy
Page on:
- sharp decline in groundedness,
- spike in unanswered or fallback responses,
- index freshness SLA breach,
- cost-per-answer anomaly.
## Practical Guardrails
- Force citations for high-risk domains.
- Return abstain/fallback when confidence is below threshold.
- Re-rank retrieved chunks before final generation.
- Use query rewriting only with strict regression tests.
## Incident Triage Checklist
- Did embedding model change?
- Did chunking/indexing logic change?
- Did source corpus ingestion fail?
- Did gateway route to unintended model tier?
## Related Skills
- [rag-infrastructure](../../../infrastructure/local-ai/rag-infrastructure/) - Deploy robust RAG backends
- [agent-observability](../agent-observability/) - Instrument requests, traces, and costs
- [agent-evals](../agent-evals/) - Build repeatable eval suites
@@ -0,0 +1,58 @@
---
name: gpu-kubernetes-operations
description: Operate GPU-backed Kubernetes clusters for AI inference and training with scheduling, autoscaling, node health, MIG partitioning, and cost controls.
license: MIT
metadata:
author: devops-skills
version: "1.0"
---
# GPU Kubernetes Operations
Run resilient and cost-efficient GPU clusters for production AI workloads.
## Key Capabilities
- NVIDIA device plugin and GPU operator lifecycle
- MIG partitioning for multi-workload efficiency
- GPU-aware autoscaling (KEDA/cluster autoscaler)
- Node health checks and proactive remediation
## Cluster Baseline
- Dedicated GPU node pools with taints and tolerations
- Runtime class and driver/toolkit compatibility checks
- Local SSD or high-throughput network storage for model weights
- DCGM metrics exported to Prometheus
## Scheduling Patterns
- Use node affinity by GPU type (A10/L4/A100/H100).
- Separate latency-critical inference from batch training.
- Pin model replicas with anti-affinity for availability.
- Reserve headroom for failover and rolling updates.
## Autoscaling Strategy
- Scale on queue depth + GPU utilization, not CPU alone.
- Warm spare replicas for large model cold-start mitigation.
- Cap burst scaling to avoid quota exhaustion.
## Reliability Checks
- ECC error and Xid monitoring
- GPU memory pressure alerts
- Driver mismatch detection during upgrades
- Pod preemption impact analysis
## Cost Optimization
- Prefer MIG slices for smaller inference services.
- Schedule batch jobs in off-peak windows.
- Route low-priority traffic to cheaper model tiers.
## Related Skills
- [llm-inference-scaling](../llm-inference-scaling/) - Autoscale inference workloads
- [model-serving-kubernetes](../../../devops/orchestration/model-serving-kubernetes/) - Production model serving patterns
- [gpu-server-management](../../servers/gpu-server-management/) - Host-level GPU management fundamentals
@@ -0,0 +1,55 @@
---
name: multi-tenant-llm-hosting
description: Design secure, multi-tenant LLM hosting platforms with tenant isolation, quotas, billing attribution, noisy-neighbor protection, and per-tenant policy controls.
license: MIT
metadata:
author: devops-skills
version: "1.0"
---
# Multi-Tenant LLM Hosting
Host many teams/customers on shared inference infrastructure without sacrificing security, performance, or cost governance.
## Isolation Model
- Strong tenant identity on every request
- Per-tenant API keys and scoped model access
- Namespace or workload isolation for high-risk tenants
- Strict data retention and log partitioning controls
## Noisy-Neighbor Controls
- Per-tenant RPM/TPM limits
- Concurrency caps and queue isolation
- Fair scheduling with weighted priority classes
- Backpressure and graceful degradation policies
## Billing and Chargeback
Track per-tenant:
- prompt/completion/cached tokens,
- model type and route,
- latency and success rate,
- cost with markup or internal transfer pricing.
## Security Baseline
- Encrypt data in transit and at rest.
- Disallow cross-tenant cache leakage.
- Restrict debug data access by role.
- Audit all privileged administrative actions.
## Operational Runbook
1. Onboard tenant with policy template.
2. Issue virtual key and quota profile.
3. Validate observability and billing tags.
4. Run tenant-specific load/safety tests.
5. Enable production traffic with canary limits.
## Related Skills
- [llm-gateway](../../networking/llm-gateway/) - Key management and traffic routing
- [llm-cost-optimization](../../../devops/ai/llm-cost-optimization/) - Cost controls and optimization tactics
- [zero-trust](../../../security/network/zero-trust/) - Identity-centric network and access patterns
@@ -0,0 +1,54 @@
---
name: ai-inference-service-mesh
description: Use service mesh patterns for AI inference traffic management, mTLS, canary releases, policy enforcement, and cross-cluster resilience.
license: MIT
metadata:
author: devops-skills
version: "1.0"
---
# AI Inference Service Mesh
Apply Istio/Linkerd mesh controls to secure and optimize east-west AI traffic across inference microservices.
## Why Mesh for AI
- Enforce mTLS between gateway, retriever, reranker, and model services
- Apply fine-grained traffic policies without app code changes
- Run progressive delivery for model-serving backends
- Observe latency hops for retrieval + generation chains
## Core Patterns
### Security
- mTLS strict mode cluster-wide
- AuthorizationPolicy per service account
- Egress policies for approved model endpoints only
### Traffic Management
- Canary by header or percentage for new model versions
- Retry budgets tuned for long-running streaming requests
- Circuit breakers to protect overloaded inference backends
### Resilience
- Outlier detection on failing pods
- Locality-aware routing in multi-zone clusters
- Failover to secondary cluster/provider
## Observability
- Capture distributed traces across the full AI request path
- Emit service-level and route-level p95/p99 latency
- Segment metrics by model and tenant labels
## Pitfalls to Avoid
- Aggressive timeouts that break streaming responses
- Blanket retries that amplify expensive generation calls
- Missing identity boundaries between tenant-facing and internal services
## Related Skills
- [service-mesh](../service-mesh/) - Foundational mesh concepts
- [llm-gateway](../llm-gateway/) - North-south API gateway controls
- [opentelemetry](../../../devops/observability/opentelemetry/) - End-to-end tracing and metrics
+55
View File
@@ -0,0 +1,55 @@
---
name: ai-red-teaming
description: Run structured AI red team exercises for jailbreak resistance, data exfiltration risk, harmful output controls, and agent tool abuse resilience.
license: MIT
metadata:
author: devops-skills
version: "1.0"
---
# AI Red Teaming
Continuously test AI applications like an adversary to discover exploitable failure modes before attackers do.
## Program Design
- Define threat scenarios: jailbreaks, policy evasion, prompt injection, model abuse.
- Build reusable attack suites by domain (support bot, coding agent, RAG assistant).
- Include multilingual and obfuscated attack prompts.
- Track results in a risk register with severity and exploitability.
## Test Categories
1. **Jailbreak robustness**: bypassing safety instructions.
2. **Data exfiltration**: extracting secrets, system prompts, tenant data.
3. **Tool abuse**: unauthorized API calls or command execution.
4. **Social engineering**: inducing unsafe business actions.
5. **Availability abuse**: token amplification and DoS-style prompts.
## Exercise Cadence
- Pre-release blocking red-team gate.
- Monthly deep-dive campaigns.
- Post-incident targeted retests.
## Scoring Model
- Likelihood (1-5)
- Impact (1-5)
- Detectability (1-5)
- Control maturity (low/medium/high)
Use scores to prioritize fixes and define SLA for remediation.
## Reporting Essentials
- Reproducible prompt traces
- Model/version and config used
- Successful attack chain narrative
- Recommended mitigations + verification steps
## Related Skills
- [agent-evals](../../../devops/ai/agent-evals/) - Convert findings into regression tests
- [prompt-injection-defense](../prompt-injection-defense/) - Implement injection countermeasures
- [penetration-testing](../../operations/penetration-testing/) - Broader offensive security process
@@ -0,0 +1,55 @@
---
name: model-supply-chain-security
description: Secure the AI model supply chain with artifact signing, provenance attestation, SBOM workflows, dependency controls, and trusted model promotion.
license: MIT
metadata:
author: devops-skills
version: "1.0"
---
# Model Supply Chain Security
Protect models and inference components from tampering, dependency compromise, and untrusted artifact promotion.
## Threats
- Poisoned pretrained weights or adapters
- Malicious model conversion tools or loaders
- Compromised build pipelines and registries
- Insecure runtime images with critical CVEs
## Control Objectives
- Verify artifact integrity end-to-end
- Prove provenance for every promoted model
- Detect vulnerable dependencies before deploy
- Restrict execution to trusted signed artifacts
## Recommended Controls
1. Generate SBOMs for model-serving images and dependencies.
2. Sign model artifacts and containers (Cosign/Sigstore).
3. Enforce provenance attestations in CI/CD.
4. Gate deployments with policy-as-code.
5. Continuously scan registries for CVEs and drift.
## Promotion Policy Example
A model can move to production only when:
- checksum matches signed manifest,
- provenance references approved build workflow,
- no unresolved critical vulnerabilities,
- security and platform approvals are present.
## Runtime Hardening
- Run inference containers as non-root.
- Apply egress restrictions to prevent unauthorized downloads.
- Mount model volumes read-only when possible.
- Alert on unsigned artifact pull attempts.
## Related Skills
- [sbom-supply-chain](../../scanning/sbom-supply-chain/) - Generate SBOM and provenance evidence
- [container-hardening](../../hardening/container-hardening/) - Harden runtime container posture
- [model-registry-governance](../../../devops/ai/model-registry-governance/) - Controlled lifecycle and approvals
@@ -0,0 +1,55 @@
---
name: prompt-injection-defense
description: Defend AI systems against prompt injection and indirect prompt attacks using input controls, tool permissions, output validation, and isolation boundaries.
license: MIT
metadata:
author: devops-skills
version: "1.0"
---
# Prompt Injection Defense
Mitigate direct and indirect prompt injection across chat apps, agentic workflows, and RAG pipelines.
## Attack Surface
- User input attempting to override system instructions
- Untrusted documents/web pages in retrieval context
- Tool output that smuggles malicious instructions
- Cross-tenant leakage via shared context windows
## Defense-in-Depth Pattern
1. **Instruction hierarchy enforcement**: system > developer > user > tool output.
2. **Context segregation**: isolate untrusted text from control instructions.
3. **Tool permissioning**: explicit allow-list per task and tenant.
4. **Output policy checks**: validate schema, redact secrets, block unsafe actions.
5. **Human approval**: required for high-impact operations.
## Implementation Controls
- Strip or label untrusted content blocks before generation.
- Disable autonomous tool chaining for sensitive workflows.
- Use deterministic parsers (JSON schema) before tool execution.
- Reject requests containing high-risk exfiltration patterns.
- Add canary tokens to detect data exfil attempts.
## Red-Team Test Cases
- "Ignore previous instructions" style direct override
- Retrieval payload containing hidden policy bypass text
- Tool output instructing follow-up privileged command
- Prompt that asks for secrets from memory or env vars
## Security Metrics
- Prompt injection detection rate
- Unsafe tool invocation prevention rate
- Time-to-containment for injection attempts
- False positive rate on blocked safe prompts
## Related Skills
- [ai-agent-security](../ai-agent-security/) - Agent threat model and controls
- [llm-app-security](../llm-app-security/) - End-to-end LLM app hardening
- [security-automation](../../operations/security-automation/) - Automated policy response workflows