Merge pull request #1 from BagelHole/codex/add-new-agent-skills-and-update-readme

2026-08-22 12:49:53 +02:00 · 2026-02-21 09:57:15 -05:00
parent 2639af6531 b3be11e88e
commit cdbce82fd1
17 changed files with 672 additions and 0 deletions
@@ -53,6 +53,7 @@ This repo is a **comprehensive knowledge base** designed to be loaded into AI ag
 | 🔧 **DevOps** | CI/CD, containers, K8s, observability, release management |
 | 🔒 **Security** | Scanning, secrets, hardening, network security, incident response |
 | ☁️ **Infrastructure** | AWS, Azure, GCP, servers, networking, databases, storage |
+| 🤖 **AI & Platforms** | Agent infrastructure, local LLM ops, and modern app platforms |
 | 📋 **Compliance** | SOC2, HIPAA, GDPR, PCI-DSS, governance, auditing |

 ---
@@ -205,6 +206,12 @@ No agent? No problem. Browse the skills, copy the scripts, use the configs. It's
 | [new-relic](devops/observability/new-relic/) | New Relic observability |
 | [alerting-oncall](devops/observability/alerting-oncall/) | Alert rules and on-call rotations |

+### AI Engineering
+| Skill | Description |
+|-------|-------------|
+| [agent-observability](devops/ai/agent-observability/) | Tracing, latency, token, and cost telemetry for agents |
+| [agent-evals](devops/ai/agent-evals/) | Automated regression and safety eval suites for agents |
+
 ### Release Management
 | Skill | Description |
 |-------|-------------|
@@ -262,6 +269,12 @@ No agent? No problem. Browse the skills, copy the scripts, use the configs. It's
 | [penetration-testing](security/operations/penetration-testing/) | Basic pentesting |
 | [security-automation](security/operations/security-automation/) | Security workflow automation |

+### AI Security
+| Skill | Description |
+|-------|-------------|
+| [ai-agent-security](security/ai/ai-agent-security/) | Defend agents against injection, tool abuse, and exfiltration |
+| [llm-app-security](security/ai/llm-app-security/) | Harden LLM app inputs, outputs, and tenant isolation |
+
 </details>

 <details>
@@ -280,6 +293,14 @@ No agent? No problem. Browse the skills, copy the scripts, use the configs. It's
 | [aws-vpc](infrastructure/cloud-aws/aws-vpc/) | Networking |
 | [aws-iam](infrastructure/cloud-aws/aws-iam/) | Identity and access |

+### Cloudflare
+| Skill | Description |
+|-------|-------------|
+| [cloudflare-workers](infrastructure/cloudflare/cloudflare-workers/) | Edge functions and APIs with Wrangler |
+| [cloudflare-pages](infrastructure/cloudflare/cloudflare-pages/) | Static/full-stack deployments with previews |
+| [cloudflare-r2](infrastructure/cloudflare/cloudflare-r2/) | S3-compatible object storage without egress fees |
+| [cloudflare-zero-trust](infrastructure/cloudflare/cloudflare-zero-trust/) | Access policies and private app protection |
+
 ### Azure
 | Skill | Description |
 |-------|-------------|
@@ -325,6 +346,7 @@ No agent? No problem. Browse the skills, copy the scripts, use the configs. It's
 |-------|-------------|
 | [postgresql](infrastructure/databases/postgresql/) | PostgreSQL admin |
 | [mysql](infrastructure/databases/mysql/) | MySQL/MariaDB |
+| [planetscale](infrastructure/databases/planetscale/) | Branch-based MySQL schema deployments |
 | [mongodb](infrastructure/databases/mongodb/) | MongoDB clusters |
 | [redis](infrastructure/databases/redis/) | Redis caching |
 | [database-backups](infrastructure/databases/database-backups/) | Backup strategies |
@@ -337,6 +359,25 @@ No agent? No problem. Browse the skills, copy the scripts, use the configs. It's
 | [nfs-storage](infrastructure/storage/nfs-storage/) | NFS servers |
 | [backup-recovery](infrastructure/storage/backup-recovery/) | Backup with restic |

+### Platforms
+| Skill | Description |
+|-------|-------------|
+| [vercel-deployments](infrastructure/platforms/vercel-deployments/) | Preview and production web app deployments |
+| [convex-backend](infrastructure/platforms/convex-backend/) | Realtime managed backend with typed functions |
+| [firebase-app-platform](infrastructure/platforms/firebase-app-platform/) | Firebase auth, data, functions, and hosting |
+
+### Local AI Infrastructure
+| Skill | Description |
+|-------|-------------|
+| [ollama-stack](infrastructure/local-ai/ollama-stack/) | Private local inference stack with Ollama |
+| [mac-mini-llm-lab](infrastructure/local-ai/mac-mini-llm-lab/) | Mac mini setup for always-on local LLM serving |
+| [openclaw-local-mac-mini](infrastructure/local-ai/openclaw-local-mac-mini/) | OpenClaw setup for local development and Mac mini hosting |
+
+### IT Operations
+| Skill | Description |
+|-------|-------------|
+| [startup-it-troubleshooting](infrastructure/it/startup-it-troubleshooting/) | Practical IT troubleshooting for small teams |
+
 </details>

 <details>
@@ -0,0 +1,39 @@
+---
+name: agent-evals
+description: Build automated evaluation suites for AI agents using golden datasets, rubrics, and regression gates.
+license: MIT
+metadata:
+  author: devops-skills
+  version: "1.0"
+---
+
+# Agent Evals
+
+Create repeatable checks so agent behavior improves safely over time.
+
+## Evaluation Layers
+
+- Unit evals: prompt-level correctness
+- Tool evals: API/tool call decision quality
+- End-to-end evals: realistic multi-step tasks
+- Safety evals: prompt injection and data leak resistance
+
+## CI/CD Integration
+
+```bash
+# Example eval pipeline steps
+make evals-smoke
+make evals-regression
+make evals-safety
+```
+
+## Best Practices
+
+- Version datasets with expected outputs.
+- Track pass rates and score drift over time.
+- Block deploys on critical safety regressions.
+
+## Related Skills
+
+- [github-actions](../../ci-cd/github-actions/) - Eval automation in CI
+- [ai-agent-security](../../../security/ai/ai-agent-security/) - Security-focused eval cases
@@ -0,0 +1,38 @@
+---
+name: agent-observability
+description: Instrument AI agents with tracing, token metrics, latency, and cost visibility. Use for reliability and debugging.
+license: MIT
+metadata:
+  author: devops-skills
+  version: "1.0"
+---
+
+# Agent Observability
+
+Monitor AI agent behavior with logs, traces, metrics, and cost telemetry.
+
+## Track Core Signals
+
+- Request latency (p50/p95/p99)
+- Token usage (prompt/completion/cached)
+- Tool call success and failure rates
+- Cost per task and per customer
+- Hallucination and retry frequency
+
+## Implementation Pattern
+
+1. Add trace IDs to every user request.
+2. Capture each LLM call and tool call as child spans.
+3. Emit structured logs with model, temperature, and response status.
+4. Create SLOs for success rate and median response time.
+
+## Best Practices
+
+- Redact PII before exporting traces.
+- Keep a replayable request envelope for incident review.
+- Alert on abnormal token spikes and tool error bursts.
+
+## Related Skills
+
+- [alerting-oncall](../../observability/alerting-oncall/) - Alert workflows
+- [agent-evals](../agent-evals/) - Quality verification
@@ -0,0 +1,39 @@
+---
+name: cloudflare-pages
+description: Deploy static sites and full-stack apps on Cloudflare Pages with previews, functions, and custom domains.
+license: MIT
+metadata:
+  author: devops-skills
+  version: "1.0"
+---
+
+# Cloudflare Pages
+
+Deploy frontend projects with preview builds and edge functions.
+
+## Connect Project
+
+1. Create a Pages project in Cloudflare dashboard.
+2. Link your GitHub repository.
+3. Set build command and output directory.
+4. Configure environment variables per environment.
+
+## Wrangler-Based Deploy
+
+```bash
+npm install -D wrangler
+npx wrangler pages project create my-site
+npx wrangler pages deploy dist --project-name=my-site
+```
+
+## Best Practices
+
+- Require previews for pull requests.
+- Separate production and preview secrets.
+- Enable Web Analytics for performance visibility.
+- Add Cloudflare WAF rules for abuse protection.
+
+## Related Skills
+
+- [cloudflare-workers](../cloudflare-workers/) - Edge backend logic
+- [vercel-deployments](../../platforms/vercel-deployments/) - Alternative frontend hosting
@@ -0,0 +1,42 @@
+---
+name: cloudflare-r2
+description: Manage Cloudflare R2 buckets, lifecycle, and signed URLs. Use for low-egress object storage and media delivery.
+license: MIT
+metadata:
+  author: devops-skills
+  version: "1.0"
+---
+
+# Cloudflare R2
+
+Use S3-compatible object storage without egress fees.
+
+## Setup
+
+```bash
+# Create bucket
+npx wrangler r2 bucket create app-assets
+
+# List buckets
+npx wrangler r2 bucket list
+
+# Upload object
+npx wrangler r2 object put app-assets/logo.png --file ./logo.png
+```
+
+## S3-Compatible Access
+
+- Generate R2 API tokens with least privilege.
+- Use endpoint format: `https://<accountid>.r2.cloudflarestorage.com`.
+- Configure lifecycle rules for archive/delete.
+
+## Best Practices
+
+- Use short-lived signed URLs for private content.
+- Store user uploads in tenant-specific prefixes.
+- Enable object versioning for recovery-critical buckets.
+
+## Related Skills
+
+- [cloudflare-workers](../cloudflare-workers/) - Signed URL generation
+- [object-storage](../../storage/object-storage/) - Storage patterns
@@ -0,0 +1,46 @@
+---
+name: cloudflare-workers
+description: Build and deploy edge functions with Cloudflare Workers and Wrangler. Use for APIs, cron jobs, and edge middleware.
+license: MIT
+metadata:
+  author: devops-skills
+  version: "1.0"
+---
+
+# Cloudflare Workers
+
+Deploy JavaScript/TypeScript functions globally at the edge.
+
+## Quick Start
+
+```bash
+npm create cloudflare@latest my-worker
+cd my-worker
+npx wrangler login
+npx wrangler deploy
+```
+
+## Common Commands
+
+```bash
+# Local dev
+npx wrangler dev
+
+# Set secret
+npx wrangler secret put API_TOKEN
+
+# Tail logs
+npx wrangler tail
+```
+
+## Best Practices
+
+- Keep workers stateless and fast.
+- Use KV, D1, or R2 for persistence.
+- Add rate limits for public APIs.
+- Version Wrangler config in git.
+
+## Related Skills
+
+- [cloudflare-pages](../cloudflare-pages/) - Frontend deployments
+- [cloudflare-r2](../cloudflare-r2/) - Object storage at the edge
@@ -0,0 +1,39 @@
+---
+name: cloudflare-zero-trust
+description: Protect internal apps with Cloudflare Access, device posture, and Zero Trust policies.
+license: MIT
+metadata:
+  author: devops-skills
+  version: "1.0"
+---
+
+# Cloudflare Zero Trust
+
+Secure access to internal services without exposing public VPN endpoints.
+
+## Core Workflow
+
+1. Register application in Cloudflare Access.
+2. Integrate identity provider (Google Workspace, Okta, Entra ID).
+3. Define access policies by group, email domain, and device posture.
+4. Add logging and alerts for blocked requests.
+
+## Tunnel Setup
+
+```bash
+cloudflared tunnel login
+cloudflared tunnel create internal-app
+cloudflared tunnel route dns internal-app app.example.com
+cloudflared tunnel run internal-app
+```
+
+## Best Practices
+
+- Enforce MFA and managed-device posture checks.
+- Use service tokens for CI/CD automation.
+- Review app policies quarterly.
+
+## Related Skills
+
+- [zero-trust](../../../security/network/zero-trust/) - Zero trust architecture fundamentals
+- [dns-management](../../networking/dns-management/) - DNS routing concepts
@@ -0,0 +1,31 @@
+---
+name: planetscale
+description: Operate MySQL-compatible databases on PlanetScale with branching workflows, safe migrations, and production rollouts.
+license: MIT
+metadata:
+  author: devops-skills
+  version: "1.0"
+---
+
+# PlanetScale
+
+Use PlanetScale for serverless MySQL with non-blocking schema change workflows.
+
+## Branching Workflow
+
+1. Create a database branch for schema work.
+2. Apply migrations to the branch.
+3. Open a deploy request and run checks.
+4. Merge to production during low-risk windows.
+
+## Operational Best Practices
+
+- Keep schema changes backward compatible first.
+- Use connection pooling for serverless apps.
+- Monitor query insights for slow statements.
+- Define rollback strategy for every deploy request.
+
+## Related Skills
+
+- [mysql](../mysql/) - MySQL tuning fundamentals
+- [database-backups](../database-backups/) - Recovery planning
@@ -0,0 +1,39 @@
+---
+name: startup-it-troubleshooting
+description: Practical IT troubleshooting playbooks for small teams without dedicated IT staff.
+license: MIT
+metadata:
+  author: devops-skills
+  version: "1.0"
+---
+
+# Startup IT Troubleshooting
+
+Run lightweight IT operations for startups and small teams.
+
+## Priority Triage Order
+
+1. Company-wide outages (internet, SSO, email)
+2. Executive or customer-facing blockers
+3. Team-wide performance degradations
+4. Individual workstation issues
+
+## Common Fix Playbooks
+
+- Identity and access lockouts
+- VPN and Wi-Fi reliability issues
+- Laptop disk and memory pressure
+- Endpoint patching and update failures
+- Printer and conferencing room failures
+
+## Process Best Practices
+
+- Keep an internal runbook and known-issues log.
+- Standardize onboarding/offboarding checklists.
+- Track asset ownership and warranty windows.
+- Escalate recurring incidents into root-cause fixes.
+
+## Related Skills
+
+- [incident-management](../../../compliance/continuity/incident-management/) - Structured incident handling
+- [runbook-creation](../../../compliance/continuity/runbook-creation/) - Documentation standards
@@ -0,0 +1,37 @@
+---
+name: mac-mini-llm-lab
+description: Configure a Mac mini as a reliable local LLM server with remote access, observability, and power-safe operation.
+license: MIT
+metadata:
+  author: devops-skills
+  version: "1.0"
+---
+
+# Mac mini LLM Lab
+
+Turn a Mac mini into a low-noise, always-on local AI appliance.
+
+## System Setup
+
+1. Update macOS and Xcode command line tools.
+2. Install Homebrew and core packages (`tmux`, `htop`, `ollama`).
+3. Enable automatic login and restart-after-power-failure.
+4. Configure Tailscale or WireGuard for remote access.
+
+## Reliability Checklist
+
+- Keep device on wired Ethernet.
+- Use UPS for power protection.
+- Schedule weekly reboot window.
+- Add launchd service for Ollama auto-start.
+
+## Security Checklist
+
+- Disable unnecessary sharing services.
+- Enforce FileVault and strong local admin password.
+- Restrict SSH to key-based auth only.
+
+## Related Skills
+
+- [ollama-stack](../ollama-stack/) - Local inference software stack
+- [ssh-configuration](../../servers/ssh-configuration/) - Secure remote shell access
@@ -0,0 +1,38 @@
+---
+name: ollama-stack
+description: Run local LLM workloads with Ollama, Open WebUI, and GPU-aware tuning for private development environments.
+license: MIT
+metadata:
+  author: devops-skills
+  version: "1.0"
+---
+
+# Ollama Stack
+
+Deploy a local LLM stack for offline and privacy-first workflows.
+
+## Minimal Setup
+
+```bash
+curl -fsSL https://ollama.com/install.sh | sh
+ollama serve
+ollama pull llama3.1:8b
+ollama run llama3.1:8b
+```
+
+## Docker Compose Pattern
+
+- Ollama container with persistent model volume
+- Open WebUI for chat interface
+- Optional LiteLLM proxy for unified API routing
+
+## Best Practices
+
+- Pin model versions for reproducibility.
+- Monitor VRAM, RAM, and swap utilization.
+- Restrict network exposure to trusted subnets.
+
+## Related Skills
+
+- [mac-mini-llm-lab](../mac-mini-llm-lab/) - Apple Silicon optimization
+- [docker-compose](../../../devops/containers/docker-compose/) - Service orchestration
@@ -0,0 +1,74 @@
+---
+name: openclaw-local-mac-mini
+description: Set up OpenClaw locally and run it reliably on a Mac mini for private, always-on local agent workflows.
+license: MIT
+metadata:
+  author: devops-skills
+  version: "1.0"
+---
+
+# OpenClaw Local + Mac mini Setup
+
+Use this skill when you want to run [OpenClaw](https://github.com/openclaw/openclaw) on a developer laptop or promote it to a stable Mac mini host.
+
+## Local Setup (any modern dev machine)
+
+1. Clone and enter repository.
+2. Follow upstream prerequisites from OpenClaw README (runtime, package manager, model/provider requirements).
+3. Create a local environment file from the example and configure keys/endpoints.
+4. Install dependencies and run the development command.
+5. Validate startup by loading the local UI/API health endpoint.
+
+```bash
+git clone https://github.com/openclaw/openclaw.git
+cd openclaw
+# Follow upstream bootstrap steps in repo docs
+# cp .env.example .env
+# <install deps>
+# <run dev server>
+```
+
+## Mac mini Production-ish Setup
+
+### Host baseline
+
+- Keep macOS updated and enable automatic security updates.
+- Use wired Ethernet and a UPS for stability.
+- Enable FileVault and lock down local admin access.
+- Configure Tailscale or WireGuard for secure remote admin.
+
+### Service operation
+
+- Run OpenClaw in a dedicated user account.
+- Store secrets in macOS Keychain or a managed secret store (avoid plain-text files in shared folders).
+- Use `tmux` for manual operation or `launchd` for auto-start on reboot.
+- Keep logs rotated and monitor disk usage.
+
+### launchd pattern (example)
+
+Create `/Library/LaunchDaemons/com.openclaw.service.plist` to run startup command from the OpenClaw directory, then:
+
+```bash
+sudo launchctl load -w /Library/LaunchDaemons/com.openclaw.service.plist
+sudo launchctl list | rg openclaw
+```
+
+## Validation Checklist
+
+- App starts after reboot without manual intervention.
+- Health check succeeds from local network.
+- Secrets are not committed and not world-readable.
+- Access to admin interfaces is restricted to trusted users/devices.
+
+## Troubleshooting Quick Hits
+
+- Slow responses: verify model backend availability and local RAM/CPU pressure.
+- Boot failures: inspect launchd logs and working directory paths.
+- Auth errors: re-check provider keys, scopes, and endpoint URLs.
+- Random crashes: pin dependency versions and restart with clean environment.
+
+## Related Skills
+
+- [ollama-stack](../ollama-stack/) - Local model serving patterns
+- [mac-mini-llm-lab](../mac-mini-llm-lab/) - Mac mini reliability and security baseline
+- [startup-it-troubleshooting](../../it/startup-it-troubleshooting/) - Small-team operational triage
@@ -0,0 +1,32 @@
+---
+name: convex-backend
+description: Build reactive backends with Convex functions, schema validation, auth integration, and deployment workflows.
+license: MIT
+metadata:
+  author: devops-skills
+  version: "1.0"
+---
+
+# Convex Backend
+
+Use Convex to build type-safe backend logic with realtime data sync.
+
+## Quick Start
+
+```bash
+npm install convex
+npx convex dev
+npx convex deploy
+```
+
+## Implementation Tips
+
+- Define schema and validation before writing functions.
+- Keep mutations idempotent where possible.
+- Use auth identity checks in every privileged query/mutation.
+- Add indexing early for high-read collections.
+
+## Related Skills
+
+- [firebase-app-platform](../firebase-app-platform/) - Alternative managed backend
+- [agent-observability](../../../devops/ai/agent-observability/) - Instrument AI-driven backend flows
@@ -0,0 +1,33 @@
+---
+name: firebase-app-platform
+description: Build and operate apps on Firebase using Auth, Firestore, Cloud Functions, and Hosting.
+license: MIT
+metadata:
+  author: devops-skills
+  version: "1.0"
+---
+
+# Firebase App Platform
+
+Ship mobile and web backends with Firebase managed services.
+
+## Core Setup
+
+```bash
+npm install -g firebase-tools
+firebase login
+firebase init
+firebase deploy
+```
+
+## Security and Scale
+
+- Write strict Firestore security rules first.
+- Separate environments by Firebase project.
+- Enable budget alerts and quota monitoring.
+- Move privileged logic into Cloud Functions.
+
+## Related Skills
+
+- [gcp-cloud-functions](../../cloud-gcp/gcp-cloud-functions/) - Function runtime patterns
+- [vercel-deployments](../vercel-deployments/) - Frontend deployment option
@@ -0,0 +1,34 @@
+---
+name: vercel-deployments
+description: Deploy frontend and full-stack apps on Vercel with previews, edge functions, and environment promotion.
+license: MIT
+metadata:
+  author: devops-skills
+  version: "1.0"
+---
+
+# Vercel Deployments
+
+Ship web apps quickly with preview environments and managed edge infrastructure.
+
+## Core Workflow
+
+```bash
+npm i -g vercel
+vercel login
+vercel link
+vercel
+vercel --prod
+```
+
+## Production Guardrails
+
+- Require preview checks before merge.
+- Separate preview and production environment variables.
+- Use branch protection with required deployment status.
+- Monitor function duration and cold start behavior.
+
+## Related Skills
+
+- [github-actions](../../../devops/ci-cd/github-actions/) - Automated deployment gates
+- [cloudflare-pages](../../cloudflare/cloudflare-pages/) - Alternative edge hosting
@@ -0,0 +1,38 @@
+---
+name: ai-agent-security
+description: Secure AI agents against prompt injection, tool abuse, and data exfiltration with defense-in-depth controls.
+license: MIT
+metadata:
+  author: devops-skills
+  version: "1.0"
+---
+
+# AI Agent Security
+
+Protect agentic systems from adversarial input and unsafe tool execution.
+
+## Threats to Model
+
+- Prompt injection through untrusted content
+- Excessive permissions on tools and APIs
+- Data exfiltration via model responses
+- Cross-tenant context leakage
+
+## Security Controls
+
+1. Isolate tool execution with strict allowlists.
+2. Add policy checks before sensitive actions.
+3. Limit token scope and credential lifetimes.
+4. Apply output filtering for sensitive data.
+5. Log every privileged tool invocation.
+
+## Incident Readiness
+
+- Keep immutable audit trails for prompts and tool calls.
+- Build kill switches for high-risk tools.
+- Run regular red-team scenarios.
+
+## Related Skills
+
+- [llm-app-security](../llm-app-security/) - Application-layer LLM defenses
+- [threat-modeling](../operations/threat-modeling/) - Structured risk analysis
@@ -0,0 +1,32 @@
+---
+name: llm-app-security
+description: Secure LLM-powered applications with input validation, output controls, tenant isolation, and abuse prevention.
+license: MIT
+metadata:
+  author: devops-skills
+  version: "1.0"
+---
+
+# LLM Application Security
+
+Harden chatbots and AI features embedded in web and mobile products.
+
+## Baseline Security Checklist
+
+- Validate and classify all user-provided context.
+- Separate system prompts from user content strictly.
+- Add moderation for toxic, harmful, and policy-violating outputs.
+- Enforce tenant boundaries in retrieval and memory layers.
+- Rate-limit high-cost endpoints.
+
+## Secure RAG Pattern
+
+1. Ingest content with malware and secret scanning.
+2. Tag documents by tenant and access policy.
+3. Filter retrieval candidates by user authorization.
+4. Add provenance metadata in final responses.
+
+## Related Skills
+
+- [ai-agent-security](../ai-agent-security/) - Agent-specific controls
+- [sast-scanning](../scanning/sast-scanning/) - Secure coding checks