Add 10 in-depth SEO-focused DevOps, security, and AI infra skills

This commit is contained in:
Toby
2026-03-03 12:11:58 -05:00
parent 4401692295
commit 87bd457b23
11 changed files with 619 additions and 0 deletions
@@ -0,0 +1,58 @@
---
name: gpu-kubernetes-operations
description: Operate GPU-backed Kubernetes clusters for AI inference and training with scheduling, autoscaling, node health, MIG partitioning, and cost controls.
license: MIT
metadata:
author: devops-skills
version: "1.0"
---
# GPU Kubernetes Operations
Run resilient and cost-efficient GPU clusters for production AI workloads.
## Key Capabilities
- NVIDIA device plugin and GPU operator lifecycle
- MIG partitioning for multi-workload efficiency
- GPU-aware autoscaling (KEDA/cluster autoscaler)
- Node health checks and proactive remediation
## Cluster Baseline
- Dedicated GPU node pools with taints and tolerations
- Runtime class and driver/toolkit compatibility checks
- Local SSD or high-throughput network storage for model weights
- DCGM metrics exported to Prometheus
## Scheduling Patterns
- Use node affinity by GPU type (A10/L4/A100/H100).
- Separate latency-critical inference from batch training.
- Pin model replicas with anti-affinity for availability.
- Reserve headroom for failover and rolling updates.
## Autoscaling Strategy
- Scale on queue depth + GPU utilization, not CPU alone.
- Warm spare replicas for large model cold-start mitigation.
- Cap burst scaling to avoid quota exhaustion.
## Reliability Checks
- ECC error and Xid monitoring
- GPU memory pressure alerts
- Driver mismatch detection during upgrades
- Pod preemption impact analysis
## Cost Optimization
- Prefer MIG slices for smaller inference services.
- Schedule batch jobs in off-peak windows.
- Route low-priority traffic to cheaper model tiers.
## Related Skills
- [llm-inference-scaling](../llm-inference-scaling/) - Autoscale inference workloads
- [model-serving-kubernetes](../../../devops/orchestration/model-serving-kubernetes/) - Production model serving patterns
- [gpu-server-management](../../servers/gpu-server-management/) - Host-level GPU management fundamentals
@@ -0,0 +1,55 @@
---
name: multi-tenant-llm-hosting
description: Design secure, multi-tenant LLM hosting platforms with tenant isolation, quotas, billing attribution, noisy-neighbor protection, and per-tenant policy controls.
license: MIT
metadata:
author: devops-skills
version: "1.0"
---
# Multi-Tenant LLM Hosting
Host many teams/customers on shared inference infrastructure without sacrificing security, performance, or cost governance.
## Isolation Model
- Strong tenant identity on every request
- Per-tenant API keys and scoped model access
- Namespace or workload isolation for high-risk tenants
- Strict data retention and log partitioning controls
## Noisy-Neighbor Controls
- Per-tenant RPM/TPM limits
- Concurrency caps and queue isolation
- Fair scheduling with weighted priority classes
- Backpressure and graceful degradation policies
## Billing and Chargeback
Track per-tenant:
- prompt/completion/cached tokens,
- model type and route,
- latency and success rate,
- cost with markup or internal transfer pricing.
## Security Baseline
- Encrypt data in transit and at rest.
- Disallow cross-tenant cache leakage.
- Restrict debug data access by role.
- Audit all privileged administrative actions.
## Operational Runbook
1. Onboard tenant with policy template.
2. Issue virtual key and quota profile.
3. Validate observability and billing tags.
4. Run tenant-specific load/safety tests.
5. Enable production traffic with canary limits.
## Related Skills
- [llm-gateway](../../networking/llm-gateway/) - Key management and traffic routing
- [llm-cost-optimization](../../../devops/ai/llm-cost-optimization/) - Cost controls and optimization tactics
- [zero-trust](../../../security/network/zero-trust/) - Identity-centric network and access patterns