Add OpenClaw local and Mac mini setup skill

This commit is contained in:
Toby
2026-02-21 09:54:14 -05:00
parent 2639af6531
commit b3be11e88e
17 changed files with 672 additions and 0 deletions
+39
View File
@@ -0,0 +1,39 @@
---
name: agent-evals
description: Build automated evaluation suites for AI agents using golden datasets, rubrics, and regression gates.
license: MIT
metadata:
author: devops-skills
version: "1.0"
---
# Agent Evals
Create repeatable checks so agent behavior improves safely over time.
## Evaluation Layers
- Unit evals: prompt-level correctness
- Tool evals: API/tool call decision quality
- End-to-end evals: realistic multi-step tasks
- Safety evals: prompt injection and data leak resistance
## CI/CD Integration
```bash
# Example eval pipeline steps
make evals-smoke
make evals-regression
make evals-safety
```
## Best Practices
- Version datasets with expected outputs.
- Track pass rates and score drift over time.
- Block deploys on critical safety regressions.
## Related Skills
- [github-actions](../../ci-cd/github-actions/) - Eval automation in CI
- [ai-agent-security](../../../security/ai/ai-agent-security/) - Security-focused eval cases
+38
View File
@@ -0,0 +1,38 @@
---
name: agent-observability
description: Instrument AI agents with tracing, token metrics, latency, and cost visibility. Use for reliability and debugging.
license: MIT
metadata:
author: devops-skills
version: "1.0"
---
# Agent Observability
Monitor AI agent behavior with logs, traces, metrics, and cost telemetry.
## Track Core Signals
- Request latency (p50/p95/p99)
- Token usage (prompt/completion/cached)
- Tool call success and failure rates
- Cost per task and per customer
- Hallucination and retry frequency
## Implementation Pattern
1. Add trace IDs to every user request.
2. Capture each LLM call and tool call as child spans.
3. Emit structured logs with model, temperature, and response status.
4. Create SLOs for success rate and median response time.
## Best Practices
- Redact PII before exporting traces.
- Keep a replayable request envelope for incident review.
- Alert on abnormal token spikes and tool error bursts.
## Related Skills
- [alerting-oncall](../../observability/alerting-oncall/) - Alert workflows
- [agent-evals](../agent-evals/) - Quality verification