External/DevOps-Security-Agent-Skills
1
0
Fork 0
mirror of https://github.com/BagelHole/DevOps-Security-Agent-Skills.git synced 2026-08-22 12:49:53 +02:00
Code Issues Packages Projects Releases Wiki Activity
Files
f7d2562069d47c4a90124ca68b8d20f699e8d846
DevOps-Security-Agent-Skills/security/ai/ai-coding-agent-guardrails/SKILL.md
T
Toby ba9e489584 V2
2026-03-24 18:02:50 -04:00

1139 lines
32 KiB
Markdown
Raw Blame History

---
name: ai-coding-agent-guardrails
description: Secure AI coding agents (Claude Code, Cursor, Codex, Copilot) with permission boundaries, secret protection, code review gates, and safe sandbox configurations for team environments.
license: MIT
metadata:
author: devops-skills
version: "1.0"
---
# AI Coding Agent Guardrails
Secure the use of AI coding agents across engineering teams. This skill covers permission boundaries, secret protection, sandbox isolation, code review gates, and audit trails for Claude Code, Cursor, Copilot, and Codex.
---
## When to Use
Apply these guardrails when:
- Onboarding AI coding agents into an engineering team for the first time
- Developers are using Claude Code, Cursor, Copilot, or Codex to generate production code
- Agents have access to repositories containing secrets, infrastructure configs, or sensitive business logic
- Your compliance framework (SOC 2, ISO 27001, FedRAMP) requires controls around automated code generation
- Autonomous or semi-autonomous agents are creating pull requests without direct human typing
- You need to enforce consistent security policies across multiple agents and team members
Signs you need tighter guardrails:
- Agents have committed secrets or credentials to version control
- Agent-generated code has introduced vulnerabilities caught late in the pipeline
- No clear audit trail distinguishes human-written from AI-generated code
- Developers are bypassing code review for "simple" agent changes
- Agents are executing arbitrary shell commands in production-connected environments
---
## Permission Boundaries
### CLAUDE.md Configuration
Create a `CLAUDE.md` at the repository root to restrict Claude Code behavior:
```markdown
# CLAUDE.md
## Restrictions
- NEVER read or output contents of .env, .env.*, secrets.yaml, or any file matching *.pem, *.key
- NEVER execute `rm -rf`, `DROP TABLE`, `kubectl delete`, or `terraform destroy` commands
- NEVER push directly to main or master branches
- NEVER modify files in the infrastructure/, terraform/, or .github/workflows/ directories without explicit user approval
- NEVER install new dependencies without listing them first for review
- NEVER access or display API keys, tokens, passwords, or connection strings
## Allowed Operations
- Read and modify application source code in src/, lib/, and tests/
- Run test suites with `npm test`, `pytest`, `go test`
- Run linters with `eslint`, `ruff`, `golangci-lint`
- Create new branches with prefix `ai/` or `agent/`
- Create and modify files in docs/ directory
## Code Standards
- All new functions must include docstrings or JSDoc comments
- All new code must have corresponding unit tests
- Follow existing code style and patterns in the repository
- Maximum file length: 500 lines. Suggest splitting if exceeded.
```
### Command Allowlists
For agents that execute shell commands, define an explicit allowlist:
```yaml
# .agent-permissions.yaml
agent_permissions:
allowed_commands:
- "npm test"
- "npm run lint"
- "npm run build"
- "pytest"
- "ruff check"
- "go test ./..."
- "git status"
- "git diff"
- "git log"
- "git checkout -b"
- "git add"
- "git commit"
- "ls"
- "cat"
- "head"
- "tail"
blocked_commands:
- "rm -rf"
- "curl"
- "wget"
- "ssh"
- "scp"
- "kubectl"
- "terraform"
- "aws"
- "gcloud"
- "az"
- "docker push"
- "npm publish"
blocked_paths:
- ".env*"
- "**/*.pem"
- "**/*.key"
- "**/secrets/**"
- "infrastructure/**"
- ".github/workflows/**"
allowed_paths:
- "src/**"
- "lib/**"
- "tests/**"
- "docs/**"
- "package.json"
- "pyproject.toml"
```
### File System Access Controls
Use filesystem permissions to enforce boundaries at the OS level:
```bash
#!/bin/bash
# setup-agent-workspace.sh
# Create a restricted workspace for agent execution
AGENT_USER="ai-agent"
REPO_DIR="/workspace/repo"
# Create agent user with limited permissions
useradd --system --shell /bin/bash --no-create-home "$AGENT_USER"
# Set ownership: developers own everything, agent gets read on most
chown -R root:developers "$REPO_DIR"
chmod -R 750 "$REPO_DIR"
# Grant agent write access only to safe directories
setfacl -R -m u:${AGENT_USER}:rwx "${REPO_DIR}/src"
setfacl -R -m u:${AGENT_USER}:rwx "${REPO_DIR}/tests"
setfacl -R -m u:${AGENT_USER}:rwx "${REPO_DIR}/docs"
# Deny agent access to sensitive files
setfacl -m u:${AGENT_USER}:--- "${REPO_DIR}/.env"
setfacl -R -m u:${AGENT_USER}:--- "${REPO_DIR}/infrastructure"
setfacl -R -m u:${AGENT_USER}:--- "${REPO_DIR}/.github/workflows"
echo "Agent workspace permissions configured."
```
---
## Secret Protection
### Pre-commit Hooks with git-secrets
```bash
#!/bin/bash
# install-secret-scanning.sh
# Install git-secrets
git clone https://github.com/awslabs/git-secrets.git /tmp/git-secrets
cd /tmp/git-secrets && make install
# Initialize in repository
cd /path/to/repo
git secrets --install
# Register common secret patterns
git secrets --register-aws
# Add custom patterns for common credential formats
git secrets --add '-----BEGIN (RSA |EC |DSA )?PRIVATE KEY-----'
git secrets --add 'AKIA[0-9A-Z]{16}'
git secrets --add 'ghp_[a-zA-Z0-9]{36}'
git secrets --add 'sk-[a-zA-Z0-9]{48}'
git secrets --add 'xox[baprs]-[0-9a-zA-Z-]{10,}'
git secrets --add 'password\s*[:=]\s*["\x27][^\s]{8,}'
git secrets --add 'api[_-]?key\s*[:=]\s*["\x27][^\s]{8,}'
# Add allowed patterns (false positive exclusions)
git secrets --add --allowed 'EXAMPLE_KEY'
git secrets --add --allowed 'your-api-key-here'
```
### Agent Output Scanning
Scan agent-generated output before it reaches version control:
```python
#!/usr/bin/env python3
"""scan_agent_output.py - Scan AI agent output for leaked secrets."""
import re
import sys
from pathlib import Path
SECRET_PATTERNS = [
(r'AKIA[0-9A-Z]{16}', 'AWS Access Key'),
(r'(?i)aws_secret_access_key\s*[:=]\s*\S+', 'AWS Secret Key'),
(r'ghp_[a-zA-Z0-9]{36}', 'GitHub Personal Access Token'),
(r'gho_[a-zA-Z0-9]{36}', 'GitHub OAuth Token'),
(r'sk-[a-zA-Z0-9]{48,}', 'OpenAI/Anthropic API Key'),
(r'xox[baprs]-[0-9a-zA-Z\-]{10,}', 'Slack Token'),
(r'-----BEGIN (RSA |EC |DSA )?PRIVATE KEY-----', 'Private Key'),
(r'(?i)(password|passwd|pwd)\s*[:=]\s*["\x27][^\s]{4,}', 'Hardcoded Password'),
(r'(?i)(api[_-]?key|apikey)\s*[:=]\s*["\x27][^\s]{8,}', 'API Key'),
(r'eyJ[a-zA-Z0-9_-]{10,}\.[a-zA-Z0-9_-]{10,}', 'JWT Token'),
(r'(?i)database_url\s*[:=]\s*\S+', 'Database Connection String'),
]
def scan_file(filepath: str) -> list[dict]:
findings = []
content = Path(filepath).read_text(errors="ignore")
for line_num, line in enumerate(content.splitlines(), 1):
for pattern, label in SECRET_PATTERNS:
if re.search(pattern, line):
findings.append({
"file": filepath,
"line": line_num,
"type": label,
"content": line.strip()[:120],
})
return findings
def main():
files = sys.argv[1:]
if not files:
print("Usage: scan_agent_output.py <file1> [file2] ...")
sys.exit(1)
all_findings = []
for f in files:
all_findings.extend(scan_file(f))
if all_findings:
print(f"BLOCKED: {len(all_findings)} potential secret(s) detected:\n")
for finding in all_findings:
print(f" [{finding['type']}] {finding['file']}:{finding['line']}")
print(f" {finding['content']}\n")
sys.exit(1)
print("OK: No secrets detected in agent output.")
sys.exit(0)
if __name__ == "__main__":
main()
```
### Git Pre-commit Hook Integration
```bash
#!/bin/bash
# .git/hooks/pre-commit
# Block commits containing secrets from AI agents
STAGED_FILES=$(git diff --cached --name-only --diff-filter=ACM)
if [ -z "$STAGED_FILES" ]; then
exit 0
fi
echo "Scanning staged files for secrets..."
# Run git-secrets
git secrets --pre_commit_hook -- "$@"
GIT_SECRETS_EXIT=$?
# Run custom scanner on staged files
python3 .tools/scan_agent_output.py $STAGED_FILES
SCANNER_EXIT=$?
if [ $GIT_SECRETS_EXIT -ne 0 ] || [ $SCANNER_EXIT -ne 0 ]; then
echo ""
echo "COMMIT BLOCKED: Secrets detected in staged files."
echo "If this is a false positive, use: git commit --no-verify"
exit 1
fi
```
---
## Sandbox Configuration
### Docker Sandbox for Agent Execution
```dockerfile
# Dockerfile.agent-sandbox
FROM ubuntu:24.04
RUN apt-get update && apt-get install -y \
git \
nodejs \
npm \
python3 \
python3-pip \
&& rm -rf /var/lib/apt/lists/*
# Create non-root agent user
RUN useradd -m -s /bin/bash agent && \
mkdir -p /workspace && \
chown agent:agent /workspace
# Drop capabilities
USER agent
WORKDIR /workspace
# No network by default - override at runtime if needed
# No access to Docker socket
# No access to host filesystem beyond mounted volume
```
```bash
#!/bin/bash
# run-agent-sandbox.sh
# Launch an AI coding agent inside a locked-down container
REPO_DIR="$(pwd)"
CONTAINER_NAME="agent-sandbox-$$"
docker run \
--name "$CONTAINER_NAME" \
--rm \
--network none \
--read-only \
--tmpfs /tmp:size=512m \
--tmpfs /home/agent:size=256m \
--memory 4g \
--cpus 2 \
--pids-limit 256 \
--security-opt no-new-privileges:true \
--security-opt seccomp=seccomp-agent.json \
--cap-drop ALL \
--cap-add DAC_OVERRIDE \
-v "${REPO_DIR}/src:/workspace/src" \
-v "${REPO_DIR}/tests:/workspace/tests:rw" \
-v "${REPO_DIR}/docs:/workspace/docs:rw" \
-v "${REPO_DIR}/package.json:/workspace/package.json:ro" \
-e "NO_COLOR=1" \
agent-sandbox:latest \
"$@"
```
### Seccomp Profile for Agent Containers
```json
{
"defaultAction": "SCMP_ACT_ERRNO",
"comment": "seccomp-agent.json - Restrictive profile for AI coding agents",
"syscalls": [
{
"names": [
"read", "write", "open", "close", "stat", "fstat", "lstat",
"poll", "lseek", "mmap", "mprotect", "munmap", "brk",
"access", "pipe", "select", "sched_yield", "mremap",
"dup", "dup2", "nanosleep", "getpid", "getuid", "getgid",
"geteuid", "getegid", "getppid", "getpgrp", "setsid",
"getgroups", "uname", "fcntl", "flock", "fsync",
"getcwd", "chdir", "readlink", "chmod", "mkdir",
"rmdir", "unlink", "rename", "symlink", "readlinkat",
"openat", "mkdirat", "newfstatat", "unlinkat", "renameat",
"faccessat", "pselect6", "ppoll", "set_robust_list",
"get_robust_list", "epoll_create1", "epoll_ctl", "epoll_wait",
"eventfd2", "pipe2", "dup3", "pread64", "pwrite64",
"futex", "clock_gettime", "clock_getres", "exit_group",
"wait4", "clone", "execve", "arch_prctl", "set_tid_address",
"exit", "getdents64", "rt_sigaction", "rt_sigprocmask",
"rt_sigreturn", "ioctl", "writev", "madvise", "getrandom"
],
"action": "SCMP_ACT_ALLOW"
},
{
"names": [
"socket", "connect", "bind", "listen", "accept",
"sendto", "recvfrom", "sendmsg", "recvmsg"
],
"action": "SCMP_ACT_ERRNO",
"comment": "Block all network syscalls"
},
{
"names": ["ptrace", "process_vm_readv", "process_vm_writev"],
"action": "SCMP_ACT_ERRNO",
"comment": "Block debugging and process inspection"
}
]
}
```
---
## Code Review Gates
### GitHub Actions Workflow for Agent PRs
```yaml
# .github/workflows/agent-pr-review.yaml
name: Agent PR Security Review
on:
pull_request:
types: [opened, synchronize]
jobs:
detect-agent-pr:
runs-on: ubuntu-latest
outputs:
is_agent: ${{ steps.check.outputs.is_agent }}
steps:
- name: Check if PR is from an AI agent
id: check
run: |
BRANCH="${{ github.head_ref }}"
AUTHOR="${{ github.event.pull_request.user.login }}"
BODY="${{ github.event.pull_request.body }}"
IS_AGENT="false"
if [[ "$BRANCH" == ai/* ]] || [[ "$BRANCH" == agent/* ]]; then
IS_AGENT="true"
fi
if echo "$BODY" | grep -qi "generated with.*claude\|generated by.*copilot\|generated by.*cursor\|generated by.*codex"; then
IS_AGENT="true"
fi
if [[ "$AUTHOR" == *"bot"* ]] || [[ "$AUTHOR" == *"agent"* ]]; then
IS_AGENT="true"
fi
echo "is_agent=$IS_AGENT" >> "$GITHUB_OUTPUT"
security-scan:
needs: detect-agent-pr
if: needs.detect-agent-pr.outputs.is_agent == 'true'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Run Semgrep security scan
uses: semgrep/semgrep-action@v1
with:
config: >-
p/default
p/owasp-top-ten
p/command-injection
p/sql-injection
p/xss
- name: Scan for secrets with Trufflehog
uses: trufflesecurity/trufflehog@main
with:
extra_args: --only-verified
- name: Check for dependency changes
run: |
CHANGED_FILES=$(git diff --name-only origin/main...HEAD)
DEP_FILES="package.json package-lock.json requirements.txt Pipfile.lock go.sum Cargo.lock"
for dep_file in $DEP_FILES; do
if echo "$CHANGED_FILES" | grep -q "$dep_file"; then
echo "::warning::Agent modified dependency file: $dep_file"
echo "DEPENDENCY_CHANGED=true" >> "$GITHUB_ENV"
fi
done
- name: Require extra review for dependency changes
if: env.DEPENDENCY_CHANGED == 'true'
run: |
gh pr edit "${{ github.event.pull_request.number }}" \
--add-label "agent-dependency-change" \
--add-label "requires-security-review"
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
enforce-review:
needs: detect-agent-pr
if: needs.detect-agent-pr.outputs.is_agent == 'true'
runs-on: ubuntu-latest
steps:
- name: Label as agent-generated
run: |
gh pr edit "${{ github.event.pull_request.number }}" \
--add-label "ai-generated"
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Enforce minimum reviewers
run: |
echo "Agent-generated PR detected."
echo "This PR requires at least 2 human approvals before merge."
```
### Branch Protection for Agent Branches
```bash
#!/bin/bash
# configure-branch-protection.sh
# Set up branch protection rules for agent-generated PRs via GitHub API
OWNER="your-org"
REPO="your-repo"
gh api repos/${OWNER}/${REPO}/rulesets \
--method POST \
--input - <<'EOF'
{
"name": "Agent Branch Protection",
"target": "branch",
"enforcement": "active",
"conditions": {
"ref_name": {
"include": ["refs/heads/ai/*", "refs/heads/agent/*"],
"exclude": []
}
},
"rules": [
{
"type": "pull_request",
"parameters": {
"required_approving_review_count": 2,
"dismiss_stale_reviews_on_push": true,
"require_code_owner_review": true,
"require_last_push_approval": true
}
},
{
"type": "required_status_checks",
"parameters": {
"required_status_checks": [
{ "context": "security-scan" },
{ "context": "test-suite" },
{ "context": "secret-detection" }
],
"strict_required_status_checks_policy": true
}
}
]
}
EOF
```
---
## Repository Configuration
### Cursor Rules (.cursorrules)
```text
# .cursorrules
You are working in a production codebase. Follow these rules strictly:
## Security Rules
- Never hardcode secrets, API keys, tokens, or passwords in source code.
- Never read or display the contents of .env files or secret configuration files.
- Never disable SSL verification, CSRF protection, or authentication middleware.
- Never use eval(), exec(), or similar dynamic code execution functions.
- Never introduce SQL string concatenation; always use parameterized queries.
## File Restrictions
- Do not modify any files in: infrastructure/, terraform/, .github/workflows/, deploy/
- Do not create or modify Dockerfiles without explicit approval.
- Do not modify CI/CD configuration files.
## Code Quality
- Every new function must have a corresponding unit test.
- All error handling must be explicit; never silently swallow exceptions.
- Follow existing patterns for logging, error handling, and API responses.
- Maximum function length: 50 lines. Refactor if exceeded.
## Git Behavior
- Create branches with the prefix: ai/
- Write descriptive commit messages referencing the task or issue.
- Never force push or rebase shared branches.
```
### GitHub Copilot Configuration
```yaml
# .github/copilot-config.yml
#
# Note: Copilot content exclusion is configured at the org/repo level
# via GitHub settings. This file documents intended exclusions and
# can be referenced by org admins when configuring the settings at
# github.com > Org Settings > Copilot > Content Exclusions.
content_exclusions:
paths:
- "**/.env*"
- "**/secrets/**"
- "**/*.pem"
- "**/*.key"
- "**/infrastructure/**"
- "**/terraform/**"
- "**/.aws/**"
- "**/credentials*"
# Copilot content exclusion via organization settings (recommended):
# 1. Go to Organization Settings > Copilot > Content exclusion
# 2. Add repository paths:
# - ".env*"
# - "secrets/**"
# - "infrastructure/**"
# - "*.pem"
# - "*.key"
```
### OpenAI Codex Configuration
```markdown
<!-- codex.md or AGENTS.md for Codex CLI -->
## Rules
- Do not modify files outside of src/ and tests/ directories.
- Do not install packages or modify dependency files without listing changes first.
- Run `npm test` after every code change to verify nothing is broken.
- Never access network resources or make HTTP requests during code generation.
- All generated code must include error handling.
- Prefix all branch names with `agent/codex/`.
```
---
## Network Controls
### Firewall Rules for Agent Environments
```bash
#!/bin/bash
# agent-network-controls.sh
# Restrict network access for agent execution environments
# Create a dedicated chain for agent traffic
iptables -N AGENT_CHAIN
# Allow DNS resolution
iptables -A AGENT_CHAIN -p udp --dport 53 -j ACCEPT
iptables -A AGENT_CHAIN -p tcp --dport 53 -j ACCEPT
# Allow package registries
iptables -A AGENT_CHAIN -d registry.npmjs.org -p tcp --dport 443 -j ACCEPT
iptables -A AGENT_CHAIN -d pypi.org -p tcp --dport 443 -j ACCEPT
iptables -A AGENT_CHAIN -d files.pythonhosted.org -p tcp --dport 443 -j ACCEPT
iptables -A AGENT_CHAIN -d proxy.golang.org -p tcp --dport 443 -j ACCEPT
# Allow GitHub for git operations
iptables -A AGENT_CHAIN -d github.com -p tcp --dport 443 -j ACCEPT
iptables -A AGENT_CHAIN -d github.com -p tcp --dport 22 -j ACCEPT
# Block everything else
iptables -A AGENT_CHAIN -j DROP
# Apply to agent user
iptables -A OUTPUT -m owner --uid-owner ai-agent -j AGENT_CHAIN
```
### Squid Proxy for Agent Traffic
```conf
# /etc/squid/squid-agent.conf
# Transparent proxy for AI agent network requests
acl agent_user proxy_auth ai-agent
acl allowed_domains dstdomain .npmjs.org .pypi.org .github.com .golang.org
# Allow only specific domains
http_access allow agent_user allowed_domains
http_access deny agent_user
# Log all agent requests for auditing
access_log /var/log/squid/agent-access.log squid
log_mime_hdrs on
# Request size limits
request_body_max_size 10 MB
reply_body_max_size 50 MB
http_port 3128
```
### Docker Compose with Network Isolation
```yaml
# docker-compose.agent.yaml
version: "3.8"
services:
agent-sandbox:
build:
context: .
dockerfile: Dockerfile.agent-sandbox
networks:
- agent-restricted
volumes:
- ./src:/workspace/src
- ./tests:/workspace/tests:rw
mem_limit: 4g
cpus: 2
pids_limit: 256
security_opt:
- no-new-privileges:true
read_only: true
tmpfs:
- /tmp:size=512m
agent-proxy:
image: ubuntu/squid:latest
networks:
- agent-restricted
- external
volumes:
- ./squid-agent.conf:/etc/squid/squid.conf:ro
ports:
- "3128:3128"
networks:
agent-restricted:
internal: true # No external access from this network
external:
driver: bridge
```
---
## Audit Trail
### Git Trailers for AI-Generated Code
```bash
#!/bin/bash
# git-ai-commit.sh
# Wrapper for committing agent-generated code with proper attribution
AGENT_NAME="${AI_AGENT_NAME:-unknown-agent}"
AGENT_VERSION="${AI_AGENT_VERSION:-unknown}"
TASK_ID="${AI_TASK_ID:-none}"
git commit -m "$(cat <<EOF
$1
AI-Generated-By: ${AGENT_NAME} ${AGENT_VERSION}
AI-Task-ID: ${TASK_ID}
AI-Reviewed-By: pending
Co-Authored-By: ${AGENT_NAME} <noreply@${AGENT_NAME}.ai>
EOF
)"
```
### Agent Action Logger
```python
#!/usr/bin/env python3
"""agent_audit_logger.py - Log all AI agent actions for compliance."""
import json
import logging
import os
import time
from datetime import datetime, timezone
from pathlib import Path
LOG_DIR = Path(os.environ.get("AGENT_LOG_DIR", "/var/log/ai-agents"))
LOG_DIR.mkdir(parents=True, exist_ok=True)
logger = logging.getLogger("agent_audit")
handler = logging.FileHandler(LOG_DIR / "agent-actions.jsonl")
handler.setFormatter(logging.Formatter("%(message)s"))
logger.addHandler(handler)
logger.setLevel(logging.INFO)
def log_action(
agent: str,
action: str,
target: str,
details: dict | None = None,
user: str = "system",
) -> None:
entry = {
"timestamp": datetime.now(timezone.utc).isoformat(),
"agent": agent,
"user": user,
"action": action,
"target": target,
"details": details or {},
"session_id": os.environ.get("AGENT_SESSION_ID", "unknown"),
}
logger.info(json.dumps(entry))
def log_file_read(agent: str, filepath: str) -> None:
log_action(agent, "file_read", filepath)
def log_file_write(agent: str, filepath: str, lines_changed: int) -> None:
log_action(agent, "file_write", filepath, {"lines_changed": lines_changed})
def log_command(agent: str, command: str, exit_code: int) -> None:
log_action(agent, "command_exec", command, {"exit_code": exit_code})
def log_pr_created(agent: str, pr_url: str, files_changed: list[str]) -> None:
log_action(agent, "pr_created", pr_url, {"files_changed": files_changed})
# Usage example:
# log_file_write("claude-code", "src/api/handler.py", 42)
# log_command("claude-code", "npm test", 0)
# log_pr_created("claude-code", "https://github.com/org/repo/pull/99", ["src/main.py"])
```
### Querying the Audit Log
```bash
#!/bin/bash
# query-agent-audit.sh
# Query agent audit logs for compliance reporting
LOG_FILE="/var/log/ai-agents/agent-actions.jsonl"
echo "=== Agent Activity Summary ==="
echo ""
echo "Actions by agent (last 24h):"
jq -r 'select(.timestamp > (now - 86400 | todate)) | .agent' "$LOG_FILE" \
| sort | uniq -c | sort -rn
echo ""
echo "File writes by agent:"
jq -r 'select(.action == "file_write") | "\(.agent) -> \(.target)"' "$LOG_FILE" \
| sort | uniq -c | sort -rn
echo ""
echo "Commands executed:"
jq -r 'select(.action == "command_exec") | "\(.agent): \(.target) (exit: \(.details.exit_code))"' "$LOG_FILE" \
| tail -20
echo ""
echo "PRs created by agents:"
jq -r 'select(.action == "pr_created") | "\(.agent): \(.target)"' "$LOG_FILE"
```
---
## Team Policies
### Agent Usage Policy Template
```yaml
# .github/agent-policy.yaml
# Team policy for AI coding agent usage
policy:
version: "1.0"
last_updated: "2025-06-01"
general:
- All developers may use AI coding agents for code generation
- Agent-generated code has the same quality and security bar as human code
- Developers are responsible for all code they submit, regardless of origin
required_review:
standard_code:
min_reviewers: 1
agent_generated: 2
infrastructure_changes:
min_reviewers: 2
agent_generated: "blocked" # Agents may not modify infra
security_sensitive:
min_reviewers: 2
requires: ["security-team-member"]
agent_generated: 2
additional_requires: ["security-team-lead"]
escalation:
- type: "dependency_addition"
action: "require security review"
- type: "auth_or_crypto_changes"
action: "require security team approval"
- type: "ci_cd_changes"
action: "blocked for agents"
- type: "database_migration"
action: "require DBA review"
- type: "api_contract_change"
action: "require API owner approval"
allowed_use_cases:
- "Writing unit and integration tests"
- "Implementing well-specified features with clear requirements"
- "Refactoring code with existing test coverage"
- "Writing documentation and code comments"
- "Fixing linter warnings and code style issues"
- "Generating boilerplate code from templates"
prohibited_use_cases:
- "Modifying authentication or authorization logic"
- "Writing or changing cryptographic implementations"
- "Modifying CI/CD pipelines or deployment configs"
- "Changing infrastructure-as-code without human authorship"
- "Accessing production databases or systems"
- "Modifying security controls or audit logging"
```
### CODEOWNERS for Agent Oversight
```text
# .github/CODEOWNERS
# Require specific reviewers for agent-sensitive areas
# All agent-generated branches require security team review
# (enforced via branch protection rules for ai/* and agent/* branches)
# Infrastructure is off-limits to agents and requires platform team
/infrastructure/ @platform-team
/terraform/ @platform-team
/.github/workflows/ @platform-team @security-team
# Security-sensitive code requires security team
/src/auth/ @security-team
/src/crypto/ @security-team
/src/middleware/auth* @security-team
# Dependency files require security review
package.json @security-team @tech-leads
package-lock.json @security-team
requirements.txt @security-team @tech-leads
go.sum @security-team
```
---
## Testing Agent Output
### Mandatory Test Coverage for Agent Code
```yaml
# .github/workflows/agent-test-gate.yaml
name: Agent Code Test Gate
on:
pull_request:
types: [opened, synchronize]
jobs:
test-coverage:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Detect changed files
id: changes
run: |
FILES=$(git diff --name-only origin/main...HEAD -- '*.py' '*.js' '*.ts' '*.go')
echo "changed_files=$FILES" >> "$GITHUB_OUTPUT"
- name: Run tests with coverage
run: |
# Python
if ls tests/*.py &>/dev/null; then
pip install pytest pytest-cov
pytest --cov=src --cov-report=json --cov-fail-under=80
fi
# Node.js
if [ -f package.json ]; then
npm ci
npm test -- --coverage --coverageThreshold='{"global":{"branches":80,"functions":80,"lines":80}}'
fi
- name: Verify new code has tests
run: |
NEW_FILES=$(git diff --name-only --diff-filter=A origin/main...HEAD -- 'src/**')
for file in $NEW_FILES; do
base=$(basename "$file" | sed 's/\.[^.]*$//')
if ! find tests/ -name "*${base}*" | grep -q .; then
echo "::error::New file $file has no corresponding test file"
exit 1
fi
done
security-scan:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run Bandit (Python)
if: hashFiles('**/*.py') != ''
run: |
pip install bandit
bandit -r src/ -f json -o bandit-report.json || true
ISSUES=$(jq '.results | length' bandit-report.json)
if [ "$ISSUES" -gt 0 ]; then
echo "::warning::Bandit found $ISSUES security issue(s) in agent-generated code"
jq -r '.results[] | " \(.severity): \(.issue_text) in \(.filename):\(.line_number)"' bandit-report.json
fi
- name: Run ESLint security plugin (JavaScript/TypeScript)
if: hashFiles('**/*.js') != '' || hashFiles('**/*.ts') != ''
run: |
npm ci
npx eslint --no-eslintrc \
--plugin security \
--rule '{"security/detect-eval-with-expression": "error"}' \
--rule '{"security/detect-non-literal-fs-filename": "warn"}' \
--rule '{"security/detect-possible-timing-attacks": "error"}' \
--rule '{"security/detect-no-csrf-before-method-override": "error"}' \
src/ || true
mutation-testing:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run mutation testing on agent-changed files
run: |
CHANGED=$(git diff --name-only origin/main...HEAD -- 'src/**/*.py')
if [ -n "$CHANGED" ]; then
pip install mutmut
for file in $CHANGED; do
echo "Mutation testing: $file"
mutmut run --paths-to-mutate="$file" --no-progress || true
done
mutmut results
fi
```
### Pre-merge Validation Script
```bash
#!/bin/bash
# validate-agent-pr.sh
# Run all validation checks before merging an agent-generated PR
set -euo pipefail
PR_BRANCH="${1:?Usage: validate-agent-pr.sh <branch-name>}"
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m'
PASS=0
FAIL=0
WARN=0
check() {
local name="$1"
shift
if "$@" >/dev/null 2>&1; then
echo -e "${GREEN}PASS${NC}: $name"
((PASS++))
else
echo -e "${RED}FAIL${NC}: $name"
((FAIL++))
fi
}
warn_check() {
local name="$1"
shift
if "$@" >/dev/null 2>&1; then
echo -e "${GREEN}PASS${NC}: $name"
((PASS++))
else
echo -e "${YELLOW}WARN${NC}: $name"
((WARN++))
fi
}
echo "Validating agent PR: $PR_BRANCH"
echo "=============================="
# Check that branch follows naming convention
check "Branch naming convention" [[ "$PR_BRANCH" == ai/* || "$PR_BRANCH" == agent/* ]]
# Check for secrets in diff
check "No secrets in diff" git secrets --scan
# Check that tests pass
check "Test suite passes" npm test
# Check test coverage threshold
warn_check "Coverage above 80%" npm test -- --coverage --coverageThreshold='{"global":{"lines":80}}'
# Check for forbidden file modifications
FORBIDDEN_CHANGES=$(git diff --name-only origin/main..."$PR_BRANCH" -- \
'infrastructure/' 'terraform/' '.github/workflows/' '.env*' '*.pem' '*.key')
check "No forbidden file changes" [ -z "$FORBIDDEN_CHANGES" ]
# Check for new dependencies
DEP_CHANGES=$(git diff --name-only origin/main..."$PR_BRANCH" -- \
'package.json' 'requirements.txt' 'go.mod' 'Cargo.toml')
warn_check "No dependency changes" [ -z "$DEP_CHANGES" ]
# Check commit messages have AI trailers
MISSING_TRAILERS=$(git log origin/main.."$PR_BRANCH" --format='%B' \
| grep -cL "AI-Generated-By:" || true)
warn_check "All commits have AI attribution trailers" [ "$MISSING_TRAILERS" -eq 0 ]
echo ""
echo "=============================="
echo -e "Results: ${GREEN}${PASS} passed${NC}, ${RED}${FAIL} failed${NC}, ${YELLOW}${WARN} warnings${NC}"
if [ "$FAIL" -gt 0 ]; then
echo -e "${RED}PR validation FAILED. Address issues before merging.${NC}"
exit 1
fi
echo -e "${GREEN}PR validation passed.${NC}"
```
---
## Quick Reference
| Control | Tool | Purpose |
|---|---|---|
| Permission boundaries | CLAUDE.md, .cursorrules, codex.md | Restrict agent behavior per-repo |
| Secret scanning | git-secrets, pre-commit hooks | Block credential leaks |
| Sandbox isolation | Docker, seccomp, network=none | Contain agent execution |
| Code review gates | GitHub Actions, branch protection | Enforce human review |
| Network controls | iptables, Squid proxy | Limit agent internet access |
| Audit trail | Git trailers, JSONL logger | Track AI-generated code |
| Test requirements | Coverage gates, mutation testing | Validate agent output quality |
| Team policies | agent-policy.yaml, CODEOWNERS | Govern agent usage org-wide |
Reference in New Issue View Git Blame Copy Permalink