mirror of
https://github.com/BagelHole/DevOps-Security-Agent-Skills.git
synced 2026-08-22 12:49:53 +02:00
Merge pull request #4 from BagelHole/codex/add-new-high-ranking-seo-skills-to-readme
This commit is contained in:
@@ -216,6 +216,10 @@ No agent? No problem. Browse the skills, copy the scripts, use the configs. It's
|
|||||||
| [llm-cost-optimization](devops/ai/llm-cost-optimization/) | Cut LLM API costs with caching, batching, model routing, and self-hosting |
|
| [llm-cost-optimization](devops/ai/llm-cost-optimization/) | Cut LLM API costs with caching, batching, model routing, and self-hosting |
|
||||||
| [llm-caching](devops/ai/llm-caching/) | Exact and semantic caching layers to reduce API calls by 30–70% |
|
| [llm-caching](devops/ai/llm-caching/) | Exact and semantic caching layers to reduce API calls by 30–70% |
|
||||||
| [ai-pipeline-orchestration](devops/ai/ai-pipeline-orchestration/) | Orchestrate RAG ingestion, training, and batch inference with Prefect/Airflow |
|
| [ai-pipeline-orchestration](devops/ai/ai-pipeline-orchestration/) | Orchestrate RAG ingestion, training, and batch inference with Prefect/Airflow |
|
||||||
|
| [llmops-platform-engineering](devops/ai/llmops-platform-engineering/) | Build enterprise LLMOps platforms with evaluation gates, promotions, rollback, and governance |
|
||||||
|
| [model-registry-governance](devops/ai/model-registry-governance/) | Define model metadata, approvals, lifecycle policy, and auditable promotion controls |
|
||||||
|
| [rag-observability-evals](devops/ai/rag-observability-evals/) | Measure retrieval quality, groundedness, hallucination risk, and RAG regressions continuously |
|
||||||
|
| [ai-sre-incident-response](devops/ai/ai-sre-incident-response/) | AI-specific SRE playbooks for model outages, quality regressions, safety incidents, and spend spikes |
|
||||||
|
|
||||||
### Release Management
|
### Release Management
|
||||||
| Skill | Description |
|
| Skill | Description |
|
||||||
@@ -281,6 +285,9 @@ No agent? No problem. Browse the skills, copy the scripts, use the configs. It's
|
|||||||
| [ai-agent-security](security/ai/ai-agent-security/) | Defend agents against injection, tool abuse, and exfiltration |
|
| [ai-agent-security](security/ai/ai-agent-security/) | Defend agents against injection, tool abuse, and exfiltration |
|
||||||
| [llm-app-security](security/ai/llm-app-security/) | Harden LLM app inputs, outputs, and tenant isolation |
|
| [llm-app-security](security/ai/llm-app-security/) | Harden LLM app inputs, outputs, and tenant isolation |
|
||||||
| [ai-security-hardening](security/ai/ai-security-hardening/) | Harden LLM deployments against prompt injection, model theft, and data exfiltration |
|
| [ai-security-hardening](security/ai/ai-security-hardening/) | Harden LLM deployments against prompt injection, model theft, and data exfiltration |
|
||||||
|
| [prompt-injection-defense](security/ai/prompt-injection-defense/) | Defend against direct/indirect prompt injection with isolation, tool controls, and output validation |
|
||||||
|
| [ai-red-teaming](security/ai/ai-red-teaming/) | Run adversarial AI red team programs for jailbreaks, exfiltration, and tool abuse resilience |
|
||||||
|
| [model-supply-chain-security](security/ai/model-supply-chain-security/) | Protect model artifacts with signing, provenance, SBOM workflows, and trusted promotion policies |
|
||||||
|
|
||||||
</details>
|
</details>
|
||||||
|
|
||||||
@@ -350,6 +357,7 @@ No agent? No problem. Browse the skills, copy the scripts, use the configs. It's
|
|||||||
| [reverse-proxy](infrastructure/networking/reverse-proxy/) | nginx, Traefik |
|
| [reverse-proxy](infrastructure/networking/reverse-proxy/) | nginx, Traefik |
|
||||||
| [service-mesh](infrastructure/networking/service-mesh/) | Istio, Linkerd |
|
| [service-mesh](infrastructure/networking/service-mesh/) | Istio, Linkerd |
|
||||||
| [llm-gateway](infrastructure/networking/llm-gateway/) | Unified LLM API gateway with routing, rate limiting, virtual keys, and semantic caching |
|
| [llm-gateway](infrastructure/networking/llm-gateway/) | Unified LLM API gateway with routing, rate limiting, virtual keys, and semantic caching |
|
||||||
|
| [ai-inference-service-mesh](infrastructure/networking/ai-inference-service-mesh/) | Service mesh patterns for mTLS, canary inference routing, and resilient AI east-west traffic |
|
||||||
|
|
||||||
### Databases
|
### Databases
|
||||||
| Skill | Description |
|
| Skill | Description |
|
||||||
@@ -387,6 +395,8 @@ No agent? No problem. Browse the skills, copy the scripts, use the configs. It's
|
|||||||
| [llm-inference-scaling](infrastructure/local-ai/llm-inference-scaling/) | Auto-scale LLM inference clusters on Kubernetes with KEDA and GPU-aware scheduling |
|
| [llm-inference-scaling](infrastructure/local-ai/llm-inference-scaling/) | Auto-scale LLM inference clusters on Kubernetes with KEDA and GPU-aware scheduling |
|
||||||
| [rag-infrastructure](infrastructure/local-ai/rag-infrastructure/) | Production RAG with vector stores, hybrid search, embedding pipelines, and reranking |
|
| [rag-infrastructure](infrastructure/local-ai/rag-infrastructure/) | Production RAG with vector stores, hybrid search, embedding pipelines, and reranking |
|
||||||
| [llm-fine-tuning](infrastructure/local-ai/llm-fine-tuning/) | QLoRA and full fine-tuning with Axolotl, DeepSpeed, and DPO alignment on GPU clusters |
|
| [llm-fine-tuning](infrastructure/local-ai/llm-fine-tuning/) | QLoRA and full fine-tuning with Axolotl, DeepSpeed, and DPO alignment on GPU clusters |
|
||||||
|
| [gpu-kubernetes-operations](infrastructure/local-ai/gpu-kubernetes-operations/) | Run GPU Kubernetes clusters with MIG, autoscaling, node health checks, and AI cost controls |
|
||||||
|
| [multi-tenant-llm-hosting](infrastructure/local-ai/multi-tenant-llm-hosting/) | Secure multi-tenant LLM hosting with quotas, isolation boundaries, and per-tenant billing controls |
|
||||||
|
|
||||||
### IT Operations
|
### IT Operations
|
||||||
| Skill | Description |
|
| Skill | Description |
|
||||||
|
|||||||
@@ -0,0 +1,66 @@
|
|||||||
|
---
|
||||||
|
name: ai-sre-incident-response
|
||||||
|
description: Build AI-focused SRE incident response practices for LLM outages, degraded quality, runaway cost events, and safety regressions.
|
||||||
|
license: MIT
|
||||||
|
metadata:
|
||||||
|
author: devops-skills
|
||||||
|
version: "1.0"
|
||||||
|
---
|
||||||
|
|
||||||
|
# AI SRE Incident Response
|
||||||
|
|
||||||
|
Apply SRE rigor to AI systems where incidents include quality regressions, unsafe outputs, and budget explosions.
|
||||||
|
|
||||||
|
## AI Incident Classes
|
||||||
|
|
||||||
|
- **Availability incident**: model/provider unavailable, timeout storm.
|
||||||
|
- **Quality incident**: answer accuracy or tool success drops below SLO.
|
||||||
|
- **Safety incident**: harmful or policy-violating outputs increase.
|
||||||
|
- **Cost incident**: unexpected token or provider spend spike.
|
||||||
|
|
||||||
|
## Severity Framework (Example)
|
||||||
|
|
||||||
|
- **SEV1**: user-facing outage, critical compliance risk, or active data leak.
|
||||||
|
- **SEV2**: major degradation affecting key flows.
|
||||||
|
- **SEV3**: limited impact or internal-only issue.
|
||||||
|
|
||||||
|
## Golden Signals for AI Services
|
||||||
|
|
||||||
|
- Request success rate
|
||||||
|
- Latency (queue + generation + tool execution)
|
||||||
|
- Hallucination/groundedness proxy metrics
|
||||||
|
- Cost per minute and per tenant
|
||||||
|
- Guardrail violation rate
|
||||||
|
|
||||||
|
## Response Playbooks
|
||||||
|
|
||||||
|
### Model Outage
|
||||||
|
1. Freeze deployments.
|
||||||
|
2. Shift traffic to fallback model/provider.
|
||||||
|
3. Enforce stricter rate limits.
|
||||||
|
4. Communicate ETA and mitigation.
|
||||||
|
|
||||||
|
### Quality Regression
|
||||||
|
1. Roll back prompt/model version.
|
||||||
|
2. Disable risky optimization flags.
|
||||||
|
3. Increase sampling for trace review.
|
||||||
|
4. Re-run latest eval baseline.
|
||||||
|
|
||||||
|
### Cost Spike
|
||||||
|
1. Identify top tenants/routes/models.
|
||||||
|
2. Enable cache + cheaper fallback path.
|
||||||
|
3. Apply temporary token caps.
|
||||||
|
4. Open postmortem with prevention actions.
|
||||||
|
|
||||||
|
## Postmortem Requirements
|
||||||
|
|
||||||
|
- Timeline with detector and responder timestamps
|
||||||
|
- Blast radius by tenant and feature
|
||||||
|
- Missed signals and alert tuning actions
|
||||||
|
- Concrete hardening tasks with owners and due dates
|
||||||
|
|
||||||
|
## Related Skills
|
||||||
|
|
||||||
|
- [incident-response](../../../security/operations/incident-response/) - Standard incident process and evidence
|
||||||
|
- [alerting-oncall](../../observability/alerting-oncall/) - Paging and escalation policy
|
||||||
|
- [llm-cost-optimization](../llm-cost-optimization/) - Spend controls and efficiency patterns
|
||||||
@@ -0,0 +1,79 @@
|
|||||||
|
---
|
||||||
|
name: llmops-platform-engineering
|
||||||
|
description: Build production LLMOps platforms with CI/CD, model promotion workflows, evaluation gates, rollback, and governance across cloud and self-hosted inference.
|
||||||
|
license: MIT
|
||||||
|
metadata:
|
||||||
|
author: devops-skills
|
||||||
|
version: "1.0"
|
||||||
|
---
|
||||||
|
|
||||||
|
# LLMOps Platform Engineering
|
||||||
|
|
||||||
|
Design and operate an internal LLM platform that supports rapid experimentation without compromising reliability, cost, or compliance.
|
||||||
|
|
||||||
|
## Outcomes
|
||||||
|
|
||||||
|
- Standardized path from experiment to production
|
||||||
|
- Safe model rollout with quality and safety gates
|
||||||
|
- Repeatable infra modules for inference, vector DB, and observability
|
||||||
|
- Clear ownership model across platform, app, and security teams
|
||||||
|
|
||||||
|
## Reference Architecture
|
||||||
|
|
||||||
|
1. **Control Plane**: model registry, prompt/version catalog, policy checks, eval pipeline.
|
||||||
|
2. **Data Plane**: inference gateway, vector database, cache, feature store.
|
||||||
|
3. **Ops Plane**: telemetry, alerting, SLO dashboards, cost analytics.
|
||||||
|
4. **Security Plane**: IAM boundaries, secret rotation, content filters, audit logs.
|
||||||
|
|
||||||
|
## Golden Delivery Workflow
|
||||||
|
|
||||||
|
1. Train/fine-tune or onboard provider model.
|
||||||
|
2. Register artifact and metadata (license, intended use, constraints).
|
||||||
|
3. Run automated eval suite (quality + safety + latency + cost).
|
||||||
|
4. Deploy canary behind gateway with strict traffic policy.
|
||||||
|
5. Promote after SLO and business KPI thresholds pass.
|
||||||
|
6. Keep rollback target hot for fast reversion.
|
||||||
|
|
||||||
|
## CI/CD Design for AI Services
|
||||||
|
|
||||||
|
- Build immutable containers with pinned dependencies and model hashes.
|
||||||
|
- Use environment promotion: `dev -> stage -> prod`.
|
||||||
|
- Fail deployment if:
|
||||||
|
- regression evals drop below baseline,
|
||||||
|
- safety tests exceed risk threshold,
|
||||||
|
- p95 latency exceeds SLO budget.
|
||||||
|
- Store deployment evidence for audits (commit SHA, eval report, approver).
|
||||||
|
|
||||||
|
## Operational SLOs
|
||||||
|
|
||||||
|
- Availability: `99.9%` for synchronous inference endpoints.
|
||||||
|
- Latency: p95 under product-specific target (for example, `<1200ms`).
|
||||||
|
- Cost: per-request and per-tenant budget ceilings.
|
||||||
|
- Quality: task success rate and groundedness thresholds.
|
||||||
|
|
||||||
|
## Platform Guardrails
|
||||||
|
|
||||||
|
- Enforce tenant quotas and model allow-lists.
|
||||||
|
- Require structured output contracts for automation paths.
|
||||||
|
- Default to low-risk model settings for critical workflows.
|
||||||
|
- Disable unconstrained tool execution in production.
|
||||||
|
|
||||||
|
## Tooling Stack (Example)
|
||||||
|
|
||||||
|
- **Orchestration**: Argo Workflows / GitHub Actions / Airflow.
|
||||||
|
- **Model Registry**: MLflow / custom metadata DB.
|
||||||
|
- **Gateway**: LiteLLM / Envoy-based API gateway.
|
||||||
|
- **Observability**: OpenTelemetry + Prometheus + Grafana + Langfuse.
|
||||||
|
- **Policy**: OPA/Rego for deployment and runtime checks.
|
||||||
|
|
||||||
|
## Incident Readiness
|
||||||
|
|
||||||
|
- Runbooks for model outage, provider timeout spikes, and cost surges.
|
||||||
|
- Chaos drills for provider failover and vector DB degradation.
|
||||||
|
- Pre-approved rollback path with one-command execution.
|
||||||
|
|
||||||
|
## Related Skills
|
||||||
|
|
||||||
|
- [ai-pipeline-orchestration](../ai-pipeline-orchestration/) - Orchestrate ingestion and inference workflows
|
||||||
|
- [agent-evals](../agent-evals/) - Build evaluation gates for releases
|
||||||
|
- [llm-gateway](../../../infrastructure/networking/llm-gateway/) - Route and control LLM traffic
|
||||||
@@ -0,0 +1,68 @@
|
|||||||
|
---
|
||||||
|
name: model-registry-governance
|
||||||
|
description: Establish model registry standards, governance controls, metadata schemas, approvals, and lifecycle policies for enterprise AI deployments.
|
||||||
|
license: MIT
|
||||||
|
metadata:
|
||||||
|
author: devops-skills
|
||||||
|
version: "1.0"
|
||||||
|
---
|
||||||
|
|
||||||
|
# Model Registry Governance
|
||||||
|
|
||||||
|
Create a trustworthy system of record for model artifacts, prompts, adapters, and evaluation evidence.
|
||||||
|
|
||||||
|
## Core Principles
|
||||||
|
|
||||||
|
- **Traceability**: every production model maps to source code, data snapshot, and evaluation results.
|
||||||
|
- **Reproducibility**: builds are deterministic with pinned dependencies.
|
||||||
|
- **Policy-driven promotion**: no manual bypass for critical safety checks.
|
||||||
|
- **Lifecycle hygiene**: stale, vulnerable, or unowned models are retired automatically.
|
||||||
|
|
||||||
|
## Required Metadata Schema
|
||||||
|
|
||||||
|
Track at minimum:
|
||||||
|
|
||||||
|
- Model name, semantic version, checksum, and storage URI
|
||||||
|
- Base model lineage and fine-tune method
|
||||||
|
- Training/eval datasets and time windows
|
||||||
|
- License, allowed use cases, prohibited use cases
|
||||||
|
- Security risk rating and mitigation controls
|
||||||
|
- Owner, backup owner, and escalation contact
|
||||||
|
|
||||||
|
## Approval Workflow
|
||||||
|
|
||||||
|
1. Registration request created from CI.
|
||||||
|
2. Security checks (artifact scan, dependency scan, provenance).
|
||||||
|
3. Evaluation package uploaded (quality, toxicity, jailbreak, bias, latency, cost).
|
||||||
|
4. Required approvals: platform + product + security (as policy dictates).
|
||||||
|
5. Promotion to stage/prod based on signed decision record.
|
||||||
|
|
||||||
|
## Lifecycle States
|
||||||
|
|
||||||
|
- `draft`: internal experimentation.
|
||||||
|
- `candidate`: passed baseline tests.
|
||||||
|
- `approved`: authorized for production rollout.
|
||||||
|
- `deprecated`: replacement announced, new usage blocked.
|
||||||
|
- `retired`: no serving allowed, archived for audit.
|
||||||
|
|
||||||
|
## Governance Policies
|
||||||
|
|
||||||
|
- Reject artifacts without SBOM/provenance.
|
||||||
|
- Block promotion if known critical CVEs remain unresolved.
|
||||||
|
- Require refreshed evals after prompt/template changes.
|
||||||
|
- Expire approvals after a configurable period (for example 90 days).
|
||||||
|
|
||||||
|
## Audit Readiness
|
||||||
|
|
||||||
|
Maintain immutable records of:
|
||||||
|
|
||||||
|
- Who approved and when
|
||||||
|
- Which policy checks executed
|
||||||
|
- Which exceptions were granted
|
||||||
|
- What model/version served each customer request window
|
||||||
|
|
||||||
|
## Related Skills
|
||||||
|
|
||||||
|
- [sbom-supply-chain](../../../security/scanning/sbom-supply-chain/) - Provenance and signing
|
||||||
|
- [policy-as-code](../../../compliance/governance/policy-as-code/) - Enforce governance with policy engines
|
||||||
|
- [llm-fine-tuning](../../../infrastructure/local-ai/llm-fine-tuning/) - Version adapters and training outputs
|
||||||
@@ -0,0 +1,64 @@
|
|||||||
|
---
|
||||||
|
name: rag-observability-evals
|
||||||
|
description: Monitor and evaluate RAG systems with retrieval quality metrics, groundedness checks, hallucination detection, and continuous regression testing.
|
||||||
|
license: MIT
|
||||||
|
metadata:
|
||||||
|
author: devops-skills
|
||||||
|
version: "1.0"
|
||||||
|
---
|
||||||
|
|
||||||
|
# RAG Observability and Evaluations
|
||||||
|
|
||||||
|
Run retrieval-augmented generation like a measurable production system, not a black box.
|
||||||
|
|
||||||
|
## What to Measure
|
||||||
|
|
||||||
|
### Retrieval Quality
|
||||||
|
- Recall@k and MRR for top-k chunks
|
||||||
|
- Citation coverage and source freshness
|
||||||
|
- Embedding drift and index staleness
|
||||||
|
|
||||||
|
### Generation Quality
|
||||||
|
- Groundedness score (answer supported by retrieved context)
|
||||||
|
- Hallucination rate by route/use case
|
||||||
|
- Instruction adherence and format validity
|
||||||
|
|
||||||
|
### Reliability and Cost
|
||||||
|
- p50/p95 latency split by retrieval vs generation
|
||||||
|
- Token usage per stage
|
||||||
|
- Cache hit rate and cost per successful answer
|
||||||
|
|
||||||
|
## Evaluation Pipeline
|
||||||
|
|
||||||
|
1. Curate a benchmark set with gold answers and source docs.
|
||||||
|
2. Run nightly offline evals for every retriever/model configuration.
|
||||||
|
3. Execute online shadow evals on sampled production traffic.
|
||||||
|
4. Gate releases on minimum quality + safety + latency thresholds.
|
||||||
|
|
||||||
|
## Alerting Strategy
|
||||||
|
|
||||||
|
Page on:
|
||||||
|
- sharp decline in groundedness,
|
||||||
|
- spike in unanswered or fallback responses,
|
||||||
|
- index freshness SLA breach,
|
||||||
|
- cost-per-answer anomaly.
|
||||||
|
|
||||||
|
## Practical Guardrails
|
||||||
|
|
||||||
|
- Force citations for high-risk domains.
|
||||||
|
- Return abstain/fallback when confidence is below threshold.
|
||||||
|
- Re-rank retrieved chunks before final generation.
|
||||||
|
- Use query rewriting only with strict regression tests.
|
||||||
|
|
||||||
|
## Incident Triage Checklist
|
||||||
|
|
||||||
|
- Did embedding model change?
|
||||||
|
- Did chunking/indexing logic change?
|
||||||
|
- Did source corpus ingestion fail?
|
||||||
|
- Did gateway route to unintended model tier?
|
||||||
|
|
||||||
|
## Related Skills
|
||||||
|
|
||||||
|
- [rag-infrastructure](../../../infrastructure/local-ai/rag-infrastructure/) - Deploy robust RAG backends
|
||||||
|
- [agent-observability](../agent-observability/) - Instrument requests, traces, and costs
|
||||||
|
- [agent-evals](../agent-evals/) - Build repeatable eval suites
|
||||||
@@ -0,0 +1,58 @@
|
|||||||
|
---
|
||||||
|
name: gpu-kubernetes-operations
|
||||||
|
description: Operate GPU-backed Kubernetes clusters for AI inference and training with scheduling, autoscaling, node health, MIG partitioning, and cost controls.
|
||||||
|
license: MIT
|
||||||
|
metadata:
|
||||||
|
author: devops-skills
|
||||||
|
version: "1.0"
|
||||||
|
---
|
||||||
|
|
||||||
|
# GPU Kubernetes Operations
|
||||||
|
|
||||||
|
Run resilient and cost-efficient GPU clusters for production AI workloads.
|
||||||
|
|
||||||
|
## Key Capabilities
|
||||||
|
|
||||||
|
- NVIDIA device plugin and GPU operator lifecycle
|
||||||
|
- MIG partitioning for multi-workload efficiency
|
||||||
|
- GPU-aware autoscaling (KEDA/cluster autoscaler)
|
||||||
|
- Node health checks and proactive remediation
|
||||||
|
|
||||||
|
## Cluster Baseline
|
||||||
|
|
||||||
|
- Dedicated GPU node pools with taints and tolerations
|
||||||
|
- Runtime class and driver/toolkit compatibility checks
|
||||||
|
- Local SSD or high-throughput network storage for model weights
|
||||||
|
- DCGM metrics exported to Prometheus
|
||||||
|
|
||||||
|
## Scheduling Patterns
|
||||||
|
|
||||||
|
- Use node affinity by GPU type (A10/L4/A100/H100).
|
||||||
|
- Separate latency-critical inference from batch training.
|
||||||
|
- Pin model replicas with anti-affinity for availability.
|
||||||
|
- Reserve headroom for failover and rolling updates.
|
||||||
|
|
||||||
|
## Autoscaling Strategy
|
||||||
|
|
||||||
|
- Scale on queue depth + GPU utilization, not CPU alone.
|
||||||
|
- Warm spare replicas for large model cold-start mitigation.
|
||||||
|
- Cap burst scaling to avoid quota exhaustion.
|
||||||
|
|
||||||
|
## Reliability Checks
|
||||||
|
|
||||||
|
- ECC error and Xid monitoring
|
||||||
|
- GPU memory pressure alerts
|
||||||
|
- Driver mismatch detection during upgrades
|
||||||
|
- Pod preemption impact analysis
|
||||||
|
|
||||||
|
## Cost Optimization
|
||||||
|
|
||||||
|
- Prefer MIG slices for smaller inference services.
|
||||||
|
- Schedule batch jobs in off-peak windows.
|
||||||
|
- Route low-priority traffic to cheaper model tiers.
|
||||||
|
|
||||||
|
## Related Skills
|
||||||
|
|
||||||
|
- [llm-inference-scaling](../llm-inference-scaling/) - Autoscale inference workloads
|
||||||
|
- [model-serving-kubernetes](../../../devops/orchestration/model-serving-kubernetes/) - Production model serving patterns
|
||||||
|
- [gpu-server-management](../../servers/gpu-server-management/) - Host-level GPU management fundamentals
|
||||||
@@ -0,0 +1,55 @@
|
|||||||
|
---
|
||||||
|
name: multi-tenant-llm-hosting
|
||||||
|
description: Design secure, multi-tenant LLM hosting platforms with tenant isolation, quotas, billing attribution, noisy-neighbor protection, and per-tenant policy controls.
|
||||||
|
license: MIT
|
||||||
|
metadata:
|
||||||
|
author: devops-skills
|
||||||
|
version: "1.0"
|
||||||
|
---
|
||||||
|
|
||||||
|
# Multi-Tenant LLM Hosting
|
||||||
|
|
||||||
|
Host many teams/customers on shared inference infrastructure without sacrificing security, performance, or cost governance.
|
||||||
|
|
||||||
|
## Isolation Model
|
||||||
|
|
||||||
|
- Strong tenant identity on every request
|
||||||
|
- Per-tenant API keys and scoped model access
|
||||||
|
- Namespace or workload isolation for high-risk tenants
|
||||||
|
- Strict data retention and log partitioning controls
|
||||||
|
|
||||||
|
## Noisy-Neighbor Controls
|
||||||
|
|
||||||
|
- Per-tenant RPM/TPM limits
|
||||||
|
- Concurrency caps and queue isolation
|
||||||
|
- Fair scheduling with weighted priority classes
|
||||||
|
- Backpressure and graceful degradation policies
|
||||||
|
|
||||||
|
## Billing and Chargeback
|
||||||
|
|
||||||
|
Track per-tenant:
|
||||||
|
- prompt/completion/cached tokens,
|
||||||
|
- model type and route,
|
||||||
|
- latency and success rate,
|
||||||
|
- cost with markup or internal transfer pricing.
|
||||||
|
|
||||||
|
## Security Baseline
|
||||||
|
|
||||||
|
- Encrypt data in transit and at rest.
|
||||||
|
- Disallow cross-tenant cache leakage.
|
||||||
|
- Restrict debug data access by role.
|
||||||
|
- Audit all privileged administrative actions.
|
||||||
|
|
||||||
|
## Operational Runbook
|
||||||
|
|
||||||
|
1. Onboard tenant with policy template.
|
||||||
|
2. Issue virtual key and quota profile.
|
||||||
|
3. Validate observability and billing tags.
|
||||||
|
4. Run tenant-specific load/safety tests.
|
||||||
|
5. Enable production traffic with canary limits.
|
||||||
|
|
||||||
|
## Related Skills
|
||||||
|
|
||||||
|
- [llm-gateway](../../networking/llm-gateway/) - Key management and traffic routing
|
||||||
|
- [llm-cost-optimization](../../../devops/ai/llm-cost-optimization/) - Cost controls and optimization tactics
|
||||||
|
- [zero-trust](../../../security/network/zero-trust/) - Identity-centric network and access patterns
|
||||||
@@ -0,0 +1,54 @@
|
|||||||
|
---
|
||||||
|
name: ai-inference-service-mesh
|
||||||
|
description: Use service mesh patterns for AI inference traffic management, mTLS, canary releases, policy enforcement, and cross-cluster resilience.
|
||||||
|
license: MIT
|
||||||
|
metadata:
|
||||||
|
author: devops-skills
|
||||||
|
version: "1.0"
|
||||||
|
---
|
||||||
|
|
||||||
|
# AI Inference Service Mesh
|
||||||
|
|
||||||
|
Apply Istio/Linkerd mesh controls to secure and optimize east-west AI traffic across inference microservices.
|
||||||
|
|
||||||
|
## Why Mesh for AI
|
||||||
|
|
||||||
|
- Enforce mTLS between gateway, retriever, reranker, and model services
|
||||||
|
- Apply fine-grained traffic policies without app code changes
|
||||||
|
- Run progressive delivery for model-serving backends
|
||||||
|
- Observe latency hops for retrieval + generation chains
|
||||||
|
|
||||||
|
## Core Patterns
|
||||||
|
|
||||||
|
### Security
|
||||||
|
- mTLS strict mode cluster-wide
|
||||||
|
- AuthorizationPolicy per service account
|
||||||
|
- Egress policies for approved model endpoints only
|
||||||
|
|
||||||
|
### Traffic Management
|
||||||
|
- Canary by header or percentage for new model versions
|
||||||
|
- Retry budgets tuned for long-running streaming requests
|
||||||
|
- Circuit breakers to protect overloaded inference backends
|
||||||
|
|
||||||
|
### Resilience
|
||||||
|
- Outlier detection on failing pods
|
||||||
|
- Locality-aware routing in multi-zone clusters
|
||||||
|
- Failover to secondary cluster/provider
|
||||||
|
|
||||||
|
## Observability
|
||||||
|
|
||||||
|
- Capture distributed traces across the full AI request path
|
||||||
|
- Emit service-level and route-level p95/p99 latency
|
||||||
|
- Segment metrics by model and tenant labels
|
||||||
|
|
||||||
|
## Pitfalls to Avoid
|
||||||
|
|
||||||
|
- Aggressive timeouts that break streaming responses
|
||||||
|
- Blanket retries that amplify expensive generation calls
|
||||||
|
- Missing identity boundaries between tenant-facing and internal services
|
||||||
|
|
||||||
|
## Related Skills
|
||||||
|
|
||||||
|
- [service-mesh](../service-mesh/) - Foundational mesh concepts
|
||||||
|
- [llm-gateway](../llm-gateway/) - North-south API gateway controls
|
||||||
|
- [opentelemetry](../../../devops/observability/opentelemetry/) - End-to-end tracing and metrics
|
||||||
@@ -0,0 +1,55 @@
|
|||||||
|
---
|
||||||
|
name: ai-red-teaming
|
||||||
|
description: Run structured AI red team exercises for jailbreak resistance, data exfiltration risk, harmful output controls, and agent tool abuse resilience.
|
||||||
|
license: MIT
|
||||||
|
metadata:
|
||||||
|
author: devops-skills
|
||||||
|
version: "1.0"
|
||||||
|
---
|
||||||
|
|
||||||
|
# AI Red Teaming
|
||||||
|
|
||||||
|
Continuously test AI applications like an adversary to discover exploitable failure modes before attackers do.
|
||||||
|
|
||||||
|
## Program Design
|
||||||
|
|
||||||
|
- Define threat scenarios: jailbreaks, policy evasion, prompt injection, model abuse.
|
||||||
|
- Build reusable attack suites by domain (support bot, coding agent, RAG assistant).
|
||||||
|
- Include multilingual and obfuscated attack prompts.
|
||||||
|
- Track results in a risk register with severity and exploitability.
|
||||||
|
|
||||||
|
## Test Categories
|
||||||
|
|
||||||
|
1. **Jailbreak robustness**: bypassing safety instructions.
|
||||||
|
2. **Data exfiltration**: extracting secrets, system prompts, tenant data.
|
||||||
|
3. **Tool abuse**: unauthorized API calls or command execution.
|
||||||
|
4. **Social engineering**: inducing unsafe business actions.
|
||||||
|
5. **Availability abuse**: token amplification and DoS-style prompts.
|
||||||
|
|
||||||
|
## Exercise Cadence
|
||||||
|
|
||||||
|
- Pre-release blocking red-team gate.
|
||||||
|
- Monthly deep-dive campaigns.
|
||||||
|
- Post-incident targeted retests.
|
||||||
|
|
||||||
|
## Scoring Model
|
||||||
|
|
||||||
|
- Likelihood (1-5)
|
||||||
|
- Impact (1-5)
|
||||||
|
- Detectability (1-5)
|
||||||
|
- Control maturity (low/medium/high)
|
||||||
|
|
||||||
|
Use scores to prioritize fixes and define SLA for remediation.
|
||||||
|
|
||||||
|
## Reporting Essentials
|
||||||
|
|
||||||
|
- Reproducible prompt traces
|
||||||
|
- Model/version and config used
|
||||||
|
- Successful attack chain narrative
|
||||||
|
- Recommended mitigations + verification steps
|
||||||
|
|
||||||
|
## Related Skills
|
||||||
|
|
||||||
|
- [agent-evals](../../../devops/ai/agent-evals/) - Convert findings into regression tests
|
||||||
|
- [prompt-injection-defense](../prompt-injection-defense/) - Implement injection countermeasures
|
||||||
|
- [penetration-testing](../../operations/penetration-testing/) - Broader offensive security process
|
||||||
@@ -0,0 +1,55 @@
|
|||||||
|
---
|
||||||
|
name: model-supply-chain-security
|
||||||
|
description: Secure the AI model supply chain with artifact signing, provenance attestation, SBOM workflows, dependency controls, and trusted model promotion.
|
||||||
|
license: MIT
|
||||||
|
metadata:
|
||||||
|
author: devops-skills
|
||||||
|
version: "1.0"
|
||||||
|
---
|
||||||
|
|
||||||
|
# Model Supply Chain Security
|
||||||
|
|
||||||
|
Protect models and inference components from tampering, dependency compromise, and untrusted artifact promotion.
|
||||||
|
|
||||||
|
## Threats
|
||||||
|
|
||||||
|
- Poisoned pretrained weights or adapters
|
||||||
|
- Malicious model conversion tools or loaders
|
||||||
|
- Compromised build pipelines and registries
|
||||||
|
- Insecure runtime images with critical CVEs
|
||||||
|
|
||||||
|
## Control Objectives
|
||||||
|
|
||||||
|
- Verify artifact integrity end-to-end
|
||||||
|
- Prove provenance for every promoted model
|
||||||
|
- Detect vulnerable dependencies before deploy
|
||||||
|
- Restrict execution to trusted signed artifacts
|
||||||
|
|
||||||
|
## Recommended Controls
|
||||||
|
|
||||||
|
1. Generate SBOMs for model-serving images and dependencies.
|
||||||
|
2. Sign model artifacts and containers (Cosign/Sigstore).
|
||||||
|
3. Enforce provenance attestations in CI/CD.
|
||||||
|
4. Gate deployments with policy-as-code.
|
||||||
|
5. Continuously scan registries for CVEs and drift.
|
||||||
|
|
||||||
|
## Promotion Policy Example
|
||||||
|
|
||||||
|
A model can move to production only when:
|
||||||
|
- checksum matches signed manifest,
|
||||||
|
- provenance references approved build workflow,
|
||||||
|
- no unresolved critical vulnerabilities,
|
||||||
|
- security and platform approvals are present.
|
||||||
|
|
||||||
|
## Runtime Hardening
|
||||||
|
|
||||||
|
- Run inference containers as non-root.
|
||||||
|
- Apply egress restrictions to prevent unauthorized downloads.
|
||||||
|
- Mount model volumes read-only when possible.
|
||||||
|
- Alert on unsigned artifact pull attempts.
|
||||||
|
|
||||||
|
## Related Skills
|
||||||
|
|
||||||
|
- [sbom-supply-chain](../../scanning/sbom-supply-chain/) - Generate SBOM and provenance evidence
|
||||||
|
- [container-hardening](../../hardening/container-hardening/) - Harden runtime container posture
|
||||||
|
- [model-registry-governance](../../../devops/ai/model-registry-governance/) - Controlled lifecycle and approvals
|
||||||
@@ -0,0 +1,55 @@
|
|||||||
|
---
|
||||||
|
name: prompt-injection-defense
|
||||||
|
description: Defend AI systems against prompt injection and indirect prompt attacks using input controls, tool permissions, output validation, and isolation boundaries.
|
||||||
|
license: MIT
|
||||||
|
metadata:
|
||||||
|
author: devops-skills
|
||||||
|
version: "1.0"
|
||||||
|
---
|
||||||
|
|
||||||
|
# Prompt Injection Defense
|
||||||
|
|
||||||
|
Mitigate direct and indirect prompt injection across chat apps, agentic workflows, and RAG pipelines.
|
||||||
|
|
||||||
|
## Attack Surface
|
||||||
|
|
||||||
|
- User input attempting to override system instructions
|
||||||
|
- Untrusted documents/web pages in retrieval context
|
||||||
|
- Tool output that smuggles malicious instructions
|
||||||
|
- Cross-tenant leakage via shared context windows
|
||||||
|
|
||||||
|
## Defense-in-Depth Pattern
|
||||||
|
|
||||||
|
1. **Instruction hierarchy enforcement**: system > developer > user > tool output.
|
||||||
|
2. **Context segregation**: isolate untrusted text from control instructions.
|
||||||
|
3. **Tool permissioning**: explicit allow-list per task and tenant.
|
||||||
|
4. **Output policy checks**: validate schema, redact secrets, block unsafe actions.
|
||||||
|
5. **Human approval**: required for high-impact operations.
|
||||||
|
|
||||||
|
## Implementation Controls
|
||||||
|
|
||||||
|
- Strip or label untrusted content blocks before generation.
|
||||||
|
- Disable autonomous tool chaining for sensitive workflows.
|
||||||
|
- Use deterministic parsers (JSON schema) before tool execution.
|
||||||
|
- Reject requests containing high-risk exfiltration patterns.
|
||||||
|
- Add canary tokens to detect data exfil attempts.
|
||||||
|
|
||||||
|
## Red-Team Test Cases
|
||||||
|
|
||||||
|
- "Ignore previous instructions" style direct override
|
||||||
|
- Retrieval payload containing hidden policy bypass text
|
||||||
|
- Tool output instructing follow-up privileged command
|
||||||
|
- Prompt that asks for secrets from memory or env vars
|
||||||
|
|
||||||
|
## Security Metrics
|
||||||
|
|
||||||
|
- Prompt injection detection rate
|
||||||
|
- Unsafe tool invocation prevention rate
|
||||||
|
- Time-to-containment for injection attempts
|
||||||
|
- False positive rate on blocked safe prompts
|
||||||
|
|
||||||
|
## Related Skills
|
||||||
|
|
||||||
|
- [ai-agent-security](../ai-agent-security/) - Agent threat model and controls
|
||||||
|
- [llm-app-security](../llm-app-security/) - End-to-end LLM app hardening
|
||||||
|
- [security-automation](../../operations/security-automation/) - Automated policy response workflows
|
||||||
Reference in New Issue
Block a user