mirror of
https://github.com/BagelHole/DevOps-Security-Agent-Skills.git
synced 2026-08-22 12:49:53 +02:00
Add 10 in-depth SEO-focused DevOps, security, and AI infra skills
This commit is contained in:
@@ -0,0 +1,55 @@
|
||||
---
|
||||
name: multi-tenant-llm-hosting
|
||||
description: Design secure, multi-tenant LLM hosting platforms with tenant isolation, quotas, billing attribution, noisy-neighbor protection, and per-tenant policy controls.
|
||||
license: MIT
|
||||
metadata:
|
||||
author: devops-skills
|
||||
version: "1.0"
|
||||
---
|
||||
|
||||
# Multi-Tenant LLM Hosting
|
||||
|
||||
Host many teams/customers on shared inference infrastructure without sacrificing security, performance, or cost governance.
|
||||
|
||||
## Isolation Model
|
||||
|
||||
- Strong tenant identity on every request
|
||||
- Per-tenant API keys and scoped model access
|
||||
- Namespace or workload isolation for high-risk tenants
|
||||
- Strict data retention and log partitioning controls
|
||||
|
||||
## Noisy-Neighbor Controls
|
||||
|
||||
- Per-tenant RPM/TPM limits
|
||||
- Concurrency caps and queue isolation
|
||||
- Fair scheduling with weighted priority classes
|
||||
- Backpressure and graceful degradation policies
|
||||
|
||||
## Billing and Chargeback
|
||||
|
||||
Track per-tenant:
|
||||
- prompt/completion/cached tokens,
|
||||
- model type and route,
|
||||
- latency and success rate,
|
||||
- cost with markup or internal transfer pricing.
|
||||
|
||||
## Security Baseline
|
||||
|
||||
- Encrypt data in transit and at rest.
|
||||
- Disallow cross-tenant cache leakage.
|
||||
- Restrict debug data access by role.
|
||||
- Audit all privileged administrative actions.
|
||||
|
||||
## Operational Runbook
|
||||
|
||||
1. Onboard tenant with policy template.
|
||||
2. Issue virtual key and quota profile.
|
||||
3. Validate observability and billing tags.
|
||||
4. Run tenant-specific load/safety tests.
|
||||
5. Enable production traffic with canary limits.
|
||||
|
||||
## Related Skills
|
||||
|
||||
- [llm-gateway](../../networking/llm-gateway/) - Key management and traffic routing
|
||||
- [llm-cost-optimization](../../../devops/ai/llm-cost-optimization/) - Cost controls and optimization tactics
|
||||
- [zero-trust](../../../security/network/zero-trust/) - Identity-centric network and access patterns
|
||||
Reference in New Issue
Block a user